Compare commits

..
65 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 2e8d8461cc feat(settings): the two new retrieval bars get their controls (#3927)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 48s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m27s
CI & Build / Build & push image (push) Successful in 36s
#3852 and #3853 each added a threshold and neither added its input, so two
of the eleven retrieval settings were reachable only through the database.
The other nine have had UI all along.

That matters because the issue said otherwise. #3927 claimed none of the ten
appeared in the frontend, on the strength of a grep across `web/src` — a
directory this repo does not have. An empty result from a path that cannot
match was read as "no UI anywhere", and a rule-25 argument was written on top
of it. The issue is corrected rather than quietly rewritten: it is #3720's
defect, absence read as non-existence, committed while filing issues about
the product doing the same thing. A grep that returns nothing and a grep that
cannot match produce the same output, and only a positive control tells them
apart.

So this is small, which is the honest size:

- `kb_toolrule_threshold` — the command arm's bar. The hint says why it sits
  BELOW the write-path one rather than leaving that looking like a mistake: a
  shell command is short, so it scores lower for the same relevance, and at a
  shared bar this arm spoke on 2% of calls against the write path's 37%.
- `kb_promptrule_threshold` — the prompt boundary, a third query shape again,
  and the only moment that reaches a rule about how to ANSWER.

Both follow the five-site pattern the existing controls use: ref, clamp on
save, write-back, payload key, load. Separate keys, because the finding of
#3853 is that one number cannot serve arms whose queries differ in shape.

Also corrects copy that went stale this morning. The standing-rule hint still
said the arm surfaces "only rules marked conditional, since always-on ones
are already loaded" — describing a tier milestone 394 removed, on the surface
whose whole job is telling the operator what the bar does.

The guard is the part worth keeping. A form's initial value is a CLAIM about
the server's default, and nothing connected the two: retune the Python
constant and the input keeps rendering the old number, which the operator
reads as the bar in force. It pins the relationship across all five
thresholds, never the values, so retuning stays free as long as both move —
and it is falsified against a drifted form value.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 20:30:06 -04:00
bvandeusenandClaude Opus 5 fe2f88cdb6 fix(telemetry): migration 0099 matched inside words and mangled rows (#3925)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / integration (push) Successful in 49s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / Python tests (push) Successful in 1m26s
CI & Build / Build & push image (push) Successful in 24s
Found post-deploy, by reading the telemetry it had just rewritten.

The token it was written for IS gone — `TOK=[redacted:token]` where a
credential used to be. But `<task-notification>` came back as
`<ta[redacted:token]>`, across rows, because `sk-` matched INSIDE the word:
`sk-` + `notification` is a vendor prefix followed by twelve word characters.

TWO PORTING MISTAKES, COMPOUNDED. The live scrubber's pattern begins with
`\b`; the migration's inlined copy had no boundary at all, dropped when I
ported it to SQL. And `\b` would not have saved it either — in Postgres ARE
`\b` is a BACKSPACE, not a word boundary. `\m` (start of word) is the
spelling that means what Python's `\b` means. Two things that look
interchangeable, are not, and fail in the same direction.

The live scrubber was never affected, and the evidence says so cleanly: rows
written after the deploy carry `<task-notification>` intact, while
migration-rewritten ones are mangled. Only the frozen copy was wrong.

THE DAMAGE HERE IS PERMANENT. The UPDATE overwrote the only copy of that
text, so those rows cannot be restored. What this fixes is every OTHER
install: 0099 has run exactly once, on one instance, and shipping a known
evidence-destroying migration in the chain for everyone else would be the
worse half of the mistake. The docstring records what it cost rather than
tidying it away.

The guard pins the property no reader can eyeball — `\m` present, `\b`
absent, in both patterns — and is falsified against the shape that shipped.

This is the third time this scrubber has eaten evidence it should not have
(`--author=`, then `task-notification`), and the pattern is consistent: the
redaction half is easy to verify and the SURVIVAL half only fails on inputs
I did not think to include. The evidence set is where the work is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 20:11:43 -04:00
bvandeusenandClaude Opus 5 32db56c0df fix(394): the co_surfaces test could no longer fail, so it was repaired not relaxed
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 50s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m25s
CI & Build / Build & push image (push) Successful in 13s
Last run left one failure, and it was the useful kind: the partner ARRIVED
(the scoping fix in 9c5ab1d worked) but carried no `via`, because it came
through the ordinary query rather than being dragged in by the edge.

The assertion was about to be read as "the edge is broken". It was not. The
edge was never exercised: since 394 an untagged rule in a subscribed rulebook
applies on its own, so the partner was already applicable and there was
nothing left for `co_surfaces` to do. The test had quietly stopped testing
anything — passing the first assertion for a reason unrelated to the
mechanism it names.

Rule 167's case exactly, so the fix is to restore its ability to fail rather
than to soften the assertion. The partner is now TAGGED to an area this
project does not work in, which puts it out of reach of everything except the
edge, and the test asserts that unreachability before drawing the relation.
If the edge ever stops dragging partners in, this fails again — which it
could not have done a commit ago.

The neighbouring suppression test keeps an UNTAGGED partner, deliberately and
now explicitly: that one is about a suppression outranking an edge, so its
partner should be reachable by every route, not none.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 16:38:14 -04:00
bvandeusenandClaude Opus 5 9c5ab1d6ad fix(394): subscription is the scope — areas narrow only where an author asked
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Failing after 40s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / Python tests (push) Successful in 1m24s
CI & Build / Build & push image (push) Successful in 39s
I got this wrong in 0e10f6b and the integration suite caught it.

get_applicable_rules filtered `always_on OR area-reachable`. Removing the
tier, I kept only the reachable arm and argued that a project with no
canonical-tagged Systems should get no bulk rules and reach them by
retrieval instead.

Two things wrong with that. The query is ALREADY scoped to rulebooks the
project SUBSCRIBED to, so the project had opted in and was then handed a
subset of what it asked for — subscription is not bulk delivery, it is the
opt-in. And milestone 394 is explicit that subscription-derived rules are not
the always-on tier and are not what it removes; I narrowed something the
milestone said to leave alone.

The failure that surfaced it is a good one: a co_surfaces partner never
arrived, because the rule it travels with had been filtered out before the
edge could drag it in. A behaviour two steps from the change.

What ships instead is narrower than "drop the clause" and wider than what I
had: every rule in a subscribed rulebook applies, EXCEPT that a rule tagged
to specific areas applies only to a project working in one of them. An
untagged rule was never narrowed by anyone, so it is general to its rulebook
by construction; a tagged one is an author saying "this is about CI" and
meaning it. D7's deterministic narrowing is kept where it was asked for and
not invented where it was not.

Also: three tests covering the SessionStart preload, the always-on tool and
the two marker paths — all surfaces that no longer exist — and the backup's
declared-section list, which still named the section its table took with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 16:33:16 -04:00
bvandeusenandClaude Opus 5 8820551058 fix(394): unbreak collection, the TS typecheck and the plugin version
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / Python lint (push) Successful in 2s
CI & Build / integration (push) Failing after 39s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / Python tests (push) Failing after 56s
CI & Build / Build & push image (push) Skipped
Five validators failed on 0e10f6b; these are the ones the logs named.

COLLECTION died first and hid everything else: test_inception.py still
imported project_rulebook_exclusions, so pytest aborted before running a
single test in either the unit or the integration lane. The migrations
therefore never ran, which means 0099 and 0100 are still unverified — this
push is what puts them in front of real Postgres.

The obsolete table test went with the import, and its module docstring now
says why rather than just describing one fewer thing.

TypeScript: RulebookDetailPane's currentRulebook computed existed only to
feed the always-on toggle, so removing the toggle left it unread; the vue
'computed' import went with it.

Plugin version minted — plugin content changed and the manifest gates the
executing cache (#2209), so the hook check fails until it moves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 16:26:56 -04:00
bvandeusenandClaude Opus 5 4eebe271ed fix(rules): restore system_ids and clear the imports the deletions orphaned (#394)
Three defects from the sweep, all caught by ruff.

system_ids was REMOVED FROM create_rule AND create_project_rule — a real API
regression, not a lint nit. The parameter shared a signature line with tier,
so deleting the tier deleted it too, and the tools lost the ability to tag a
new rule to an area. Areas are what let a rule reach a project after this
milestone, so the one parameter that decides reach went missing from the two
tools that create reachable rules.

The other two are imports left holding nothing: services/rulebooks.py's
module-level datetime and its IntegrityError were used only by functions this
milestone deleted, and plugin_context lost four (select, async_session,
RulebookTopic, rulebooks_svc) with the preload and _topic_titles.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 16:24:38 -04:00
bvandeusenandClaude Opus 5 0e10f6bb8a feat(rules)!: retire the always-on tier — every rule arrives by retrieval (#394)
CI & Build / Python lint (push) Failing after 3s
CI & Build / Plugin hooks (push) Failing after 12s
CI & Build / integration (push) Failing after 27s
CI & Build / TypeScript typecheck (push) Failing after 35s
CI & Build / Python tests (push) Failing after 37s
CI & Build / Build & push image (push) Skipped
Milestone 394, steps 5-8. Operator: "remove the always on rule functionality
as the goal was to not have it at all since it didn't seem to work as
expected."

Unconditional preload had three failures the retrieval arms do not. It could
not be MEASURED — a resident rule is in the context whether or not it
mattered, so nothing distinguished "this governed the act" from "this was
scenery", and it was the one surface structurally exempt from the scoreboard
judging every other. It was SUMMARISED AWAY by compaction while the session
went on believing it held the rules. And it CROWDED OUT the few rules that
applied with the thirty that did not.

WHAT GOES

Schema (0100): rules.tier + ck_rules_tier, rule_versions.tier,
rulebooks.always_on, and project_rulebook_exclusions — a table recording a
project's opt-out of something that no longer binds it unasked.

Tools: list_always_on_rules, exclude_always_on_rulebook,
include_always_on_rulebook. Service: the same three plus rules_etag_for,
_valid_tier and the whole etag family. The SessionStart preload and the
write-path staleness arm go with them: nothing is resident, so nothing can
have drifted since a session loaded it.

THREE CALLS WORTH REVIEWING

enter_project got NARROWER, not wider. Its filter was `always_on OR
area-tagged`; dropping the tier arm leaves the deterministic half, so a
project with no canonical-tagged Systems gets no bulk rules and reaches them
by retrieval instead. Dropping the whole clause would have made that payload
bigger than the preload this milestone deletes.

Backups import tolerantly. A pre-394 archive carries tier, always_on and the
retired inception choice; none is read, and the exclusion key is DROPPED
rather than remapped, because restoring it would write data that
validate_inception now rejects as unknown.

The migration is irreversible in the way that matters and says so: downgrade
recreates the columns at their defaults and cannot restore which rules were
always-on. A value invented to fill a hole is not a measurement.

THE INSTRUCTION SURFACES SAY THE HARDER THING

Deleting "call list_always_on_rules()" is easy; replacing it is not, because
the new model asks a session to trust something it cannot see. All three
surfaces now say a session holds nothing, that rules arrive when work matches
them, and — the half that got dangerous — that "no rule arrived" means
"nothing matched", never "there is no rule". Under residency an empty session
was rare and suspicious; it is now the ordinary state of most turns, so
reading it as permission is wrong on nearly every turn rather than
occasionally. That is #3720's defect at session scale.

test_instruction_surfaces_agree is repointed rather than retired: its two
halves collapsed into one instruction, and it gains a guard that every
surface states what absence means. _INSTRUCTIONS is back at 1999/2000 —
the inception clause paid for the longer HOW line.

UI (rule 27, and the opportunity step 8 named)

The tier selector is gone, and what replaces it is the point: `when_to_apply`
is now the field that decides whether a rule is ever seen, so the editor
marks it required, warns while it is empty, and both rule lists badge a
trigger-less rule "never surfaces". A rule without one is not quiet, it is
unreachable.

TESTS

Two files deleted outright — test_rules_etag.py and test_inception_rules.py
tested subsystems that no longer exist. Elsewhere obsolete cases were removed
and the rest repointed. One deserves naming: the wiring test asserted the act
arms pass no `tier`, which had become an assertion that could not fail. It is
repointed onto `kind`, which does still exist and where the same claim is
live — a preference must reach a write exactly as a rule does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 16:22:17 -04:00
bvandeusen 18e3cf9f2f fix(telemetry): scrub existing rows, and stop --author= being eaten (#3925) 2026-09-11 15:43:31 -04:00
bvandeusenandClaude Opus 5 bfa2d419f9 fix(telemetry): a logged query never carries a credential (#3925)
pre_tool_rule retrieves against the RAW COMMAND TEXT and write_path_rule
against the code being written, so whatever was on the command line or in
the buffer is what record_retrieval stored in retrieval_logs.query. A
command that exported a token stored the token.

Storing it was not the worst of it. near_miss_samples is the readout the
threshold docs tell you to open before moving a bar, so the value came back
OUT into an agent's context on the next tuning pass — which is exactly how
this was found, mid-way through #3853's threshold spike.

SCRUBBED ON WRITE, at _build_payload — the single seam every source reaches
the column through. A read-side filter would leave the secret in the table
where a backup or a debug query still reaches it, and a per-caller scrub
would be three places for one to be forgotten by whoever adds the fourth
arm.

REDACTED VISIBLY. `[redacted:<kind>]` rather than a silent deletion: a
reader who cannot tell a scrubbed query from a short one is being lied to
by the readout itself.

DELIBERATELY CONSERVATIVE — vendor-prefixed credentials, values assigned to
secret-NAMED variables, auth headers, PEM blocks. Things that are secrets by
construction. Entropy heuristics and long-opaque-string detection start
eating real queries, and a query is evidence: missing an exotic secret costs
one redaction nobody made, while eating a query costs the ability to tune
the bar at all.

The guard pins BOTH directions, and the second half is the one that matters.
A scrubber that eats evidence fails silently — it keeps looking like it
works while turning the one instrument for tuning a threshold into
unreadable stubs, which is the #2663 shape in a new place. So nine REAL
queries from this install's near-miss samples must survive byte for byte. If
a future pattern touches one, the pattern is too greedy.

Verified against the real shapes before commit: six credential formats
redacted (fabricated values), nine real queries unchanged, and the payload
seam confirmed to store "export API_TOKEN=[redacted:assigned] && git push".

This does NOT scrub rows already written. Purging those is separate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 15:38:40 -04:00
bvandeusen c149ef31a3 wip(394): steps 6+7 — backend path and instruction surfaces 2026-09-11 15:15:33 -04:00
bvandeusenandClaude Opus 5 690ca0306e feat(rules): the command arm gets its own bar, measured (#3853)
CI & Build / TypeScript typecheck (push) Successful in 1m2s
CI & Build / Python tests (push) Successful in 1m34s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Successful in 35s
One threshold served both act arms. The telemetry says they are not the
same problem:

  write_path_rule   2,325 calls, speaks on 37%, near-miss p50 0.6989
  pre_tool_rule    11,768 calls, speaks on  2%, near-miss p50 0.6794

The second is not quiet, it is mute — 11,530 of 11,768 calls said nothing,
with near-miss p90 at 0.7097 against a 0.72 bar. Refused mass piled one
hundredth under the line is what a bar set too high leaves behind, and the
note arms are the control: auto_inject refuses at p90 0.5463, write_path at
0.6738, both far below theirs.

The cause is query shape, not corpus. A write-path query is a code payload,
long and rich — the case 0.72 was calibrated on. A pre-tool query is a shell
command, often under a dozen words: less text, less signal, lower scores for
the same relevance.

MEASURED. Eight replayed queries against the post-#3855 corpus, consequential
acts against innocuous ones:

  0.7571  git push origin dev              consequential
  0.7245  cd ...; git fetch; git add -A    consequential
  0.7193  git pull --rebase origin dev     consequential
  0.6850  docker compose up -d             consequential
  ------------------------------------- 0.68
  0.6735  wc -l src/*.py && date           innocuous
  0.6544  grep -rn useState src/           innocuous
  0.6099  sed -n '120,160p' package.json   innocuous
  0.6056  ls -la && cat README.md          innocuous

At 0.72 three of four consequential acts retrieved nothing, including
`git pull --rebase origin dev`, where rules 153, 1 and 2 all ranked correctly
between 0.7126 and 0.7193 and were all refused.

The separation is 0.0115 wide. That is a direction, not a settled number, and
the comment says so — near_miss_samples on a few days of post-#3855 traffic
is what settles it.

This also corrects an assumption the old comment stated: it argued 0.68 sat
"below where this corpus's noise sits", inferring a higher floor from the
corpus being homogeneous. Measured, the command arm's noise ceiling is 0.6735,
so 0.68 clears it barely rather than sitting under it.

Lowering is safer now than it would have been. Until #3851 this arm had one
slot, so the bar was the only noise control; the band now filters downstream,
so the bar's job shrank and the bar can.

write_path_rule is unchanged — healthy at 0.72 on its own evidence.

Guards: the two bars parse independently, garbage falls back to its OWN
default rather than to the sibling's (which would silently re-merge them),
the command default stays below the write-path default as a direction check,
and each arm both SEARCHES and REPORTS at its own bar. That last one is a
failure the single-bar code could not have had: retrieval_logs.threshold is
what near-miss analysis is read against, so an arm searching at one number
and logging another misreports the refusal and invites moving the bar that
was already right.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 14:36:18 -04:00
bvandeusenandClaude Opus 5 40189147d2 fix(rules): a shortened rule line must not decide what it says about holding (#3851)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m24s
CI & Build / integration (push) Successful in 40s
CI & Build / Build & push image (push) Successful in 26s
CI run 6485 was red. Six failures, three causes, and only one of them was a
stale test.

THE REAL DEFECT. The compact branch dropped the `seen` TAIL along with the
trigger, so a rule the session had already been told rendered exactly like
one it had not. #3750's whole argument is that those are different claims —
a repeat is rendered precisely because the session may no longer HOLD what
it was told — and the tail is the entire difference a reader can act on.
test_a_rule_the_session_already_holds_is_referenced_not_re_offered caught it
within one commit, which is that guard working as intended.

Fixed by keeping the tail and dropping only the trigger, which is both the
cheaper and the safer cut: a trigger runs 300-400 characters after #3855, a
tail about 100. Re-measured on the real renderer — top-full-plus-references
is ~299 tokens against ~568 for five full lines, so about 2x the old single
line rather than the 1.4x claimed before, for four more rules and no lost
information. The comments carrying the old figure are corrected rather than
left to read as a decision nobody made.

THE FIXTURE THAT STRADDLED THE BAND. `_THREE_HITS` spanned 0.81-0.74 against
a 0.05 band, so the act arms dropped its lowest hit and four cases of
test_both_recorders_report_the_same_rules_for_one_call failed reporting a
count mismatch — under a message blaming the exclusion filter. A guard
pointing confidently at the wrong subsystem costs more than no guard,
because it is believed. Scores retightened to 0.81/0.80/0.79 and the
precondition is now asserted by a named test, so a future band change is
told where the problem is instead of through four confusing failures.

THE STALE CONSTANT GUARD. test_the_rule_arm_asks_for_one_rule_not_two pinned
RULEHINT_LIMIT == 1 — a real decision, correctly guarded, for a world with a
resident set. Rewritten to pin what replaced it, as relationships rather
than values (rule 115): the arm can return several, and rules are narrowed
HARDER than the notes menu because they measured flatter, not sharper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 14:07:32 -04:00
bvandeusenandClaude Opus 5 10343a6019 feat(rules): an act surfaces a banded SET of rules, quieter after the first (#3851)
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 53s
CI & Build / Build & push image (push) Skipped
CI & Build / integration (push) Successful in 45s
RULEHINT_LIMIT was 1. That was correct while retrieval SUPPLEMENTED a
33-rule resident set — one salient rule beside everything already loaded.
Milestone 394 removes residency, and then this arm is the whole delivery:
`git push origin dev` is governed by rules 1, 2, 9 and 140 simultaneously,
and each alone permits the mistake the others catch.

A cap plus a band, not a bigger cap. The old argument's real content is that
a fixed k invents lines — it fills slots whether or not anything deserves
them. A band keeps only what scored close to the top, so one clearly
relevant rule still shows one and four competing rules show four. The corpus
decides; the cap is a ceiling on the worst case, not the usual answer.

MEASURED, AND IT CORRECTED THE PREDICTION. The expectation was that rules
would rank sharply, since rule_document() shapes them like snippets and note
2485 measured snippets separating their top hit by 0.153 against 0.010-0.023
for every other kind. Three probes against real act queries say otherwise:

  `git push origin dev`      top 0.757, gap 0.022
  `docker compose up -d`     top 0.685, gap 0.016
  a bare-owner-filter query  top 0.656, gap 0.020

Dev-log territory, not snippet territory — rules arrive as a packed block,
so shaping alone did not buy separation. The band is therefore narrow: at
0.10 (the notes menu's value) every one of the top eight on the push probe
falls inside, including a CI-registry rule and another project's branch
policy. 0.05 admits about three ranks.

COST, MEASURED RATHER THAN ASSUMED. The old comment claimed a line costs
~40 tokens. It is ~143 once the trigger is rendered, and #3855 roughly
tripled trigger lengths, so five full lines are ~646 tokens before EVERY
Bash call. Hence rank decides volume: the top hit keeps the full rendering,
later hits are cited (~198 tokens total, 1.4x the old single line, for four
more rules). The old paragraph's instinct — a fourth voice at full volume is
where a reader stops reading — is answered by making later lines quieter
rather than by refusing to have them.

Band before dedup, deliberately. The band is a statement about scores;
letting the ledger reorder it would make "you were told this already" change
what counts as relevant. Same axis independence the renderer already keeps
between `kind` and `seen`, and `rule_ids` stays fresh-only (#3752) so
#3668's identity between logged results and surfacing rows survives.
`suppressed` now covers both causes and says so.

Both act arms take the same band: their score distributions are the same
shape, and only the query differs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 14:00:36 -04:00
bvandeusenandClaude Opus 5 8e06cdf749 feat(rules): the trigger contract is shown as a worked contrast, and pinned (#3855)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / integration (push) Successful in 51s
CI & Build / TypeScript typecheck (push) Successful in 56s
CI & Build / Python tests (push) Successful in 1m29s
CI & Build / Build & push image (push) Successful in 25s
Follows 8c9f947, which taught the trigger shape on the two update_*
surfaces but left the softer regression unguarded: guidance kept and
abstracted back to "name the moment in session vocabulary" — advice about
being concrete that is not itself concrete, which is the shape that was
already on file while the corpus filled with categories.

Two attempts to detect that in free prose were written and discarded:

- Counting quoted multi-word phrases anywhere in a docstring measured
  ambient quotation rather than demonstrated triggers. It PASSED the
  abstracted version by scoring unrelated prose, and text with an odd
  number of quote characters produced matches spanning the gap BETWEEN two
  unrelated phrases.
- Scoping that count to a window after each trigger mention then FAILED
  create_preference in its CORRECT state, its examples sitting further from
  the first mention than any defensible window reaches.

Both were proxies inferring demonstration from prose. Where a property
cannot be measured, changing the shape of the thing is cheaper than a
cleverer measurement — so all five trigger-writing surfaces now carry a
two-line labelled contrast:

  RETRIEVES: "the migration failed with a check violation on a column we
    just extended"
  COLLAPSES: "when working on migrations"

Unambiguous to parse, free in its wording, and a better teaching form than
the sentences it replaces: the labels name the mechanism, so they do work
for the reader rather than only for the test.

The guard now pins both halves — the field is documented, and the contrast
is present, complete and non-identical. Falsified against three regressions
before committing: the paragraph stripped, the examples abstracted away,
and one half of the pair removed. All three fail; the current tree passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 11:48:14 -04:00
bvandeusenandClaude Opus 5 8c9f947f09 feat(rules): the update surfaces teach the trigger shape, not just the create ones (#3855)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 47s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m26s
CI & Build / Build & push image (push) Successful in 24s
A trigger is two-thirds of a rule's embedded document, so one naming a
CATEGORY rather than a moment collapses the record toward its title and it
never arrives. #3835 measured that across 113 rules; #3855 hit it again on
the eight preferences, where six named a category and two did not.

The split was not carelessness, it was an uneven contract. create_rule has
carried the full argument since c61925b (2026-08-27) and the two
preferences authored that day got good triggers; the six written weeks
earlier got categories. The guidance worked wherever it existed — and it
existed on three of five write surfaces. Both update_* tools were silent,
and the update path is where every RETROFITTED trigger is written, which is
most of them: a trigger that already reads fine as English is the one
nobody rewrites.

So:

- update_rule gains the retrofit case, which is a different trap from the
  create case. There the field is empty and the instruction is "write one".
  Here one exists, reads perfectly well, and the honest-looking verdict is
  that it is fine.
- update_preference gains it too, plus why the field is load-bearing there
  specifically: preferences get a reserved slot filled by a kind-filtered
  query at limit=1, so the corpus ranks against ITSELF and the trigger is
  nearly all that separates one from the next.
- create_preference and create_project_rule now SHOW a moment instead of
  describing one. Advice about being concrete that is not itself concrete
  is the shape that was already on file while the corpus filled up.

The guard pins one property: a tool taking when_to_apply mentions it. That
is exactly what update_preference failed. The surface list is derived from
register() rather than hand-kept, so a write tool added later is in scope
the day it lands.

Two stronger predicates were written for the softer regression — guidance
kept but abstracted — and both were discarded after falsification: counting
quoted phrases measured ambient quotation and passed the broken version,
and scoping that count to a window failed create_preference while correct.
Rule 167 settles it; the discarded attempts are recorded in the test
docstring so the next author does not repeat them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 11:39:00 -04:00
bvandeusenandClaude Opus 5 d5f96563fd feat(rules): a slot a preference cannot lose (#3894)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 1m1s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Successful in 1m44s
CI & Build / Build & push image (push) Successful in 35s
Milestone 399 step 4. A rule and a preference are not equally served by one
ranking, because their losses are not equal:

  - a RULE crowded out at the prompt boundary still fires at an act arm. A
    push reaches pre_tool_rule, a write reaches write_path_rule. The prompt
    hit is a preview of a second chance.
  - a PREFERENCE about how to answer has no second chance. The response IS
    the act, so crowded out there it is never delivered at all.

A straight ranking therefore favours the record whose loss is recoverable
over the one whose loss is total, and does it INVISIBLY: the rule that won is
a legitimate hit, the telemetry reads healthy, and the only symptom is a
preference that quietly never arrives. reuse_slot exists for the same shape
one corpus over (#2463).

`semantic_search_rules` gains a `kind` filter, so the slot's query can only
answer with what the slot is for. Verifying afterwards would be weaker — an
unfiltered search that happened to return a rule would spend the slot on it,
and that line would be indistinguishable from one that earned its place.

THE SLOT BUYS POSITION, NOT A LOWER BAR, matching reuse_slot. A weak
preference cannot buy it, so silence stays the default. The task asked for a
separate threshold; I did not add one, and the reason is that the worry
behind it — reading a miss rate as a fact about preferences — is answered by
`preference_slot` being its own logged source, where best_available_id names
which preference was refused. A knob added on a guess is a way to
misconfigure the surface; a bar moved on evidence is an argument. The
evidence arrives on its own now.

IT EXTENDS, IT NEVER DISPLACES — and here it parts from reuse_slot, which
evicts its menu's weakest hit. A displaced hit sits in prompt_rule's
retrieval_logs row while never being surfaced, so that source's two tables
stop agreeing and #3668's identity breaks for a reason nothing in the data
explains. Milestone #379 is what losing that identity costs: five steps
planned against two counters disagreeing, not a write path dropping rows. One
extra line in a rare case is the cheaper price.

It also runs BEFORE the bail-out. An empty general result is not proof no
preference qualifies: that search overfetches by distance then collapses, so
a preference ranked below the window is invisible to it while a kind-filtered
query finds it at once. Bailing first would make the slot dead in exactly the
corpus it exists for.

One existing assertion repinned from a bare call_count to a per-source
filter: the slot logs its own query on the same call, and a count would pin
the number of arms rather than the property — going red the next time one is
added, which is rule 167's false alarm about the thing it protects.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 08:44:24 -04:00
bvandeusenandClaude Opus 5 44e0b0541f feat(rules): rules retrieve against the operator's message (#3852)
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m1s
CI & Build / Python tests (push) Successful in 1m35s
CI & Build / Build & push image (push) Successful in 38s
The third rule arm, and the one the other two cannot reach. `write_path_rule`
is keyed on code, `pre_tool_rule` on a command — both things the session is
about to DO. A rule that governs what to SAY has no such trigger: extract
intent from loose phrasing, raise a conflict before acting, hand off an
action with its reason, end a finding with an offer all bind on a RESPONSE,
and no tool call precedes one.

The operator's message is the only query that exists before a response is
composed. That hook searched notes alone, so no rule had ever been retrieved
against a thing the operator actually said — and residency was the only
surface those rules had, which is what milestone 394 removes.

A SEPARATE FUNCTION, not a branch in build_autoinject_hint, because of its
early returns. That arm bails when auto-inject is disabled, when the query is
blank, when nothing clears the note bar — every one a statement about NOTES.
Folded in, an operator who turned the awareness menu off would silently lose
their rules, a coupling with no symptom since both look like a quiet hook.
Two functions, two sets of gates, composed in the route. Guarded as "the rule
arm never asks the notes arm's config", which is the structural fact.

Joins _ARMS rather than getting its own test file. #3497's history is that
the pre-tool arm inherited a defect from its sibling by being MODELLED on it
instead of sharing with it, and a third arm modelled on two is two chances to
repeat that. Repeat rendering, fresh-only counting, log-before-bailout, the
kind register and the two-recorders identity are properties of every arm or
of none.

The bar is INHERITED and says so. 0.72 was tuned against code and commands;
prose is a different query shape against the same documents, and triggers are
written in the vocabulary of the moment — which for most rules is act
vocabulary. Starting at the only number with evidence behind it and logging
every call from the first deploy is what makes it settleable; guessing lower
would put an unmeasured bar in front of a corpus that binds.

k=3, anchored on this hook's own budget rather than the act arms'.
RULEHINT_LIMIT is 1 because that arm fires before every Bash call; this one
fires once per turn, beside a notes menu already spending three slots. And a
prompt genuinely contains more than one act — "merge to main and then start
on X" is two — where a command is one thing.

`prompt_rule` added to RANKED_SOURCES: a ranker picked it, and a ranked
source missing from that tuple is silently counted as bulk delivery and drops
out of the pull-through denominator.

The hook reads and writes the SHARED rule ledger under scribe-priorart, not a
private one — one session keeps one list, aged (#3751), so a rule named here
is not re-announced before the next Bash call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 07:56:19 -04:00
bvandeusenandClaude Opus 5 8406871085 fix(plugin): the instruction surfaces still said every rule binds (#3849)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Successful in 1m27s
CI & Build / Build & push image (push) Successful in 41s
Step 3 shipped a line a session can receive — "Preference that may apply
here …" — into surfaces that told it, in the most authoritative voice it
has, that anything arriving in that shape is binding. That is the confusion
milestone 399 exists to prevent, arriving through the one channel a session
has least reason to doubt.

Silent in both directions, which is why it could not wait for step 6. A
session treating a preference as a rule refuses to proceed over something
the operator merely preferred; and it loses the whole reason preferences
exist, which is that they are brought up to date rather than obeyed.

Three surfaces, each to its own budget:

- SKILL.md gets the full account: kind decides force, the injected line names
  which in its opening words, and a preference is the one record a session
  keeps current itself (update_preference, with what taught the change).
- scribe_static_context.md gets six lines — enough to tell the kinds apart
  and to say a preference is yours to update.
- _INSTRUCTIONS gets four words. It is a MAP at 1978 of its 2000-char budget
  (#2562), and the detail belongs in the surfaces above and in the tool
  docstrings, which is what that budget exists to force.

Guarded so it cannot drift back: a surface that claims rules bind must name
the kind that does not. Pinned on the CLAIM rather than the word "bind",
because a bare substring also matches bind_repo, list_repo_bindings and
server.py's DNS-rebinding comment — a guard that would one day fail a skill
about repo binding is rule 167's named failure, raising a false alarm about
the very thing it protects. Falsified against all three surfaces losing the
mention.

Plugin version minted: the cache refreshes only on a version bump (#2209),
so a skill edit without one reaches no installed plugin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-10 23:20:04 -04:00
bvandeusenandClaude Opus 5 26e0dff706 feat(rules): a preference does not speak in a rule's voice (#3849 step 3)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m27s
CI & Build / integration (push) Successful in 1m31s
CI & Build / Build & push image (push) Successful in 41s
Two independent axes on one hint line. `kind` decides the head, `seen`
decides the tail, and neither reads the other — which is what let a second
kind arrive without reopening #3750's repeat question. Whether a record is
already on the exclusion ledger has nothing to do with how much force it
carries, so the seen branch is shared verbatim.

The noun carries the whole visual difference, deliberately. A reader skimming
an injected block gets one word to place the register, so the word that moves
is the one naming force: "Standing rule" / "Preference". Everything
structural after it is identical, so the kinds read as one set rather than
two formats.

Force is asserted in exactly one other place, and that moves too. A rule's
line says to read it BEFORE DECIDING IT DOES NOT APPLY, because dismissing a
rule unread is how the thing it prevents happens. A preference makes no such
claim: it says where to find HOW THIS HAS BEEN DONE BEFORE, and following it
buys consistency rather than correctness.

Guarded on both places at once. Pinning the noun alone would pass a line
reading "Preference … before deciding it does not apply" — label swapped,
instruction kept — which is worse than not distinguishing them, because it
looks handled.

And a guard on the independence claim itself, exercising all four
combinations: the way this breaks silently is a seen branch that grows a kind
test, leaving one combination rendered by nobody's intention.

Noted, not fixed: plugin/skills/using-scribe/SKILL.md still says "Standing
rules are binding" with no room for a kind that does not. That surface is
step 6's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-10 22:04:14 -04:00
bvandeusenandClaude Opus 5 89d16d89a9 feat(rules): a preference updates without asking, and says what taught it (#3849 step 2)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 1m7s
CI & Build / integration (push) Successful in 1m8s
CI & Build / Python tests (push) Successful in 1m37s
CI & Build / Build & push image (push) Successful in 33s
The write path, and the step where a preference stops being a relabelled
rule. `create_preference` / `update_preference` on the MCP surface, plus
`kind` on update_rule and both HTTP doors.

SEPARATE TOOLS, NOT A `kind=` ARGUMENT. create_rule's docstring IS the
approval gate (#3557): propose, offer three answers, wait. That is right for
a rule — the person it binds should have agreed. A preference inverts it, and
one reached through create_rule would be read through that prose, so the
caller would hesitate over exactly the act this kind exists to make routine.
Two doors, two contracts, one table. Reads stay shared: a preference IS a
rule row, and "what governs this" wants both.

Two required fields, each buying something:

- `when_to_apply`, because the trigger is two-thirds of the embedded
  document. Without one the record is written, stored, and silently never
  delivered — indistinguishable from one nobody wrote.
- `arose_from_id`, the price of the ungated write. A corpus that drifts with
  no record of what taught each change cannot be audited, and the operator's
  veto over drift is worth exactly as much as their ability to read why it
  happened.

The near-duplicate gate is what lets this corpus be written freely and stay
small: the second preference about a thing updates the first. It is
title-scoped and kind-blind, so it also catches a preference restating a rule
that already binds.

The asymmetry is guarded as two PRESENCE facts — the rule door still asks,
the preference door still says write it — never as an absence. An absence
check passes against a docstring that was deleted or rewritten into something
else, which is snippet #3352's warning and would read as coverage here while
proving nothing.

`_plain_detail` moved to tests/helpers on its second copy, per that module's
own reason for existing (#2825).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-10 21:59:23 -04:00
bvandeusenandClaude Opus 5 4aae4973f7 fix(tests): the backup stand-ins predate kind (#3849 step 1)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 45s
CI & Build / TypeScript typecheck (push) Successful in 56s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / Build & push image (push) Successful in 48s
Two unit tests build a rule with SimpleNamespace rather than the model, so
adding a column broke them — the fixture has no `kind` for `_rule_rows` to
read. Fixture-only; the export itself was already right, which the existing
column-coverage guard confirmed by passing.

Adds the guard that coverage check cannot make. `_stand_in` walks
`__table__.columns` and proves the KEY is emitted; it cannot prove the VALUE
survives. A preference exported as a rule is a silent failure — the restored
rule reads fine and simply binds when it was only ever a preference — and a
fixture carrying the default would pass against a `_rule_rows` that dropped
the field and let the importer's `or "rule"` refill it. So the new test
asserts on `preference`, the one value that cannot be reconstructed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-10 21:31:16 -04:00
bvandeusenandClaude Opus 5 c63172272d feat(rules): a preference is a rule that does not bind (#3849 step 1)
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Failing after 1m11s
CI & Build / Build & push image (push) Skipped
Adds `kind` to rules — `rule` binds, `preference` is how the operator wants
work done. One column, because the two differ in exactly one dimension and
everything else a preference needs already lives on `rules`: a trigger
column, a trigger-dominated embedding document, ownership-scoped search,
three retrieval arms with telemetry, typed relations, and versioning.

Defaults to `rule`, so nothing changes force on upgrade — 0088's argument
for `tier`, unchanged.

`rule_versions` gets the column too, and that half is not bookkeeping.
`record_if_changed` decides whether an edit deserves a snapshot by comparing
the fields a version carries, so a field absent from SNAPSHOT_FIELDS is a
field whose change records no history at all. Without it, turning a rule
into a preference — the moment something stops binding, and the single most
consequential edit either kind can undergo — would leave the history silent.

Backup carries it through all four seams. A missed one would have restored
every preference as a rule, quietly.

Guarded on real Postgres in three halves: a preference writes, a typo is
refused (without which every other assertion would pass against a table
whose CHECK had been dropped), and a row written with no kind reads back as
`rule` — the migration's whole safety claim, asserted rather than assumed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-10 21:14:29 -04:00
bvandeusenandClaude Opus 5 c1aa1d8e92 feat(plugin): rule exclusions age, so salience decays without a context event (#3751)
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 38s
CI & Build / integration (push) Successful in 45s
CI & Build / Python tests (push) Successful in 1m9s
CI & Build / Build & push image (push) Successful in 27s
#3749 clears the ledger when an EVENT destroys context — a compaction, a
/clear. This is the case with no event at all: a long session where a
rule was named two hundred turns ago and has simply fallen out of
attention. It is #3702's argument at the tier level — present in context
and salient at the moment are different properties — applied to time
instead of to tier.

FORMAT: `id<TAB>epoch`, per entry.

Not a whole-file mtime: that is one line of shell and wrong in exactly
the session that needs it, since a single recent write keeps every stale
id alive, and the ids that go stale first come from the rules that fire
most.

Not a turn counter, though it would be the truer model — an idle session
does not forget. A hook has no turn number without keeping its own, which
is a second piece of session state to write, read, clear on compaction
and get wrong. Wall time costs a `date` call. The failure it accepts is a
session left idle over lunch treating its rules as forgotten, worth one
extra full line per rule and nothing else.

TTL 2700s (45 minutes), reasoned rather than picked (rule 32). About one
working stretch on a single task: long enough that a rule does not
re-announce itself while you are still doing the thing it governs, short
enough that a multi-hour session gets a refresh rather than one 9am
mention. It leans short because since #3750 being wrong on the short side
is the cheaper error — an expired entry costs one full line instead of
one short one, and the exclusion re-arms the moment it is spent. There is
no data on this yet; #3807's near-miss listing is what should revise it.

BOTH READERS THROUGH ONE HELPER, in scribe_defs.sh. The two hooks share
one ledger so a rule named by one arm is not re-offered by the other; a
format only one of them understood would break that on the first read.
The flat `tr '\n' ','` read would now send `156<TAB>1789002860` as an
exclude id — verified, which is why this is not a per-hook edit.

THE LAST ENTRY FOR AN ID WINS. The file is append-only, so a rule that
ages out, is surfaced fresh and is appended again has two lines. Reading
the first leaves it permanently expired, and it then re-announces itself
on every call for the rest of the session — the mechanism meant to
quieten things becoming the loudest thing in the hint.

A BARE ID IS LIVE. That is the pre-#3751 format, and every session in
flight when this ships has a ledger full of them. Reading unknown as
EXPIRED would make all of those sessions re-announce every rule they had
already been told, at once — the exact noise this prevents, delivered by
the feature on the day it ships. Unknown means "not measured", never
"old", the same discipline the nullable retrieval_logs columns use.

GUARDS (tests/test_rule_ledger_ageing.py, real shell, no credentials)

- old ages out AND recent survives, in ONE assertion (rule 167): either
  half alone passes against a broken helper — "old is gone" passes
  against one returning nothing, "recent survives" passes against the
  flat read this replaces, i.e. against the defect itself.
- the ping-pong case, which is the one that costs the most to get wrong.
- a bare id is live, including beside a stale stamped one.
- missing/empty ledger excludes nothing.
- an id repeated in the ledger appears once, with no empty list element.
- structural: neither hook reads the rule ledger flat again — pinned on
  the flat-read SHAPE, so a rename of the helper is not a failure and a
  hook that ages correctly some other way is not either.

The TTL's VALUE is deliberately not asserted. The tests read the constant
out of the shell and assert the property around it, so a later tuning
change stays a tuning change instead of a red build.

Dropped a boundary test (`ttl` vs `ttl + 1`) before committing: racy by
construction, since a ledger written at T is read at T+n and the two
cases swap. A one-second distinction on a 45-minute window is also not
observable behaviour, so it pinned a flake rather than a property.

Plugin version minted to 2026.09.10.0221 — this is entirely hook-side,
so without the bump the cache would never pick it up and the merge would
ship nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-09 22:21:32 -04:00
bvandeusenandClaude Opus 5 c4908f093f feat(rules): a repeat is referenced, not withheld (#3750, #3752)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / integration (push) Successful in 40s
CI & Build / Python tests (push) Successful in 1m9s
CI & Build / Build & push image (push) Successful in 37s
Both arms used to drop a hit already on the session's exclusion ledger
and emit nothing. That is correct only while the session still HOLDS
what it was told, and a compaction breaks exactly that: the earlier
injection is summarized away while the id stays on the ledger, leaving
the rule absent from context AND unreachable for the rest of the
session. #3749 closed the compaction half by clearing the ledger; this
closes the ordinary half, where a session simply stops holding a line
it read an hour ago.

Only ONE CLAUSE of the existing line is false on a repeat — the claim
that the rule is not in the session's loaded set — so only that clause
changes. The fresh line is byte-identical to what it was.

Both tails now come from one `_rule_hint_line`. The arms phrase their
heads differently on purpose; everything after must not differ, and
#3497's history is that the pre-tool arm inherited a defect by being
modelled on its sibling rather than sharing with it.

THE BUDGET DECISION, recorded at RULEHINT_LIMIT. A repeat competes for
the single slot on rank alone: nothing is fetched behind it, and it
never rides alongside as a second line. Promoting a fresh rule past a
better-ranked repeat would reinstate the withholding one rank deeper,
and a second line is the one thing the limit exists to forbid. The
consequence is deliberate — a rule that keeps ranking first for a
recurring situation keeps being referenced, and its decay belongs to
exclusion ageing (#3751), not to a first-place rule being demoted for
having won before.

THE TELEMETRY, decided before shipping rather than after a number moved
(#3752): nothing changes. A reference is a RENDERING decision, not a
retrieval outcome. `results` stays `fresh`, `suppressed` stays
len(hits) - len(fresh), and `record_rule_surfaced` still counts only
what the arm freshly chose. This matters more than it reads: the naive
implementation drops the `fresh` filter and takes suppressed_count to
zero everywhere — and #3739's near-miss fix identifies repeat-caused
zeros by `suppressed_count > 0`, so the contamination corrected on
2026-09-08 would return by a different route, in the same field, with
the fix still in the code and no longer working. A test asserts the
counters as unmoved, because "nothing changed" is only worth something
if it is checkable.

Also corrects two comments that outlived #3702 — both arms still
claimed CONDITIONAL ONLY while the module-level note above
RULEHINT_LIMIT said the opposite, in the exact code this change edits.

GUARDS

- an already-held hit produces a rule line at all (the regression),
  asserted on `get_rule(<id>)` rather than a truthy context: the
  write-path arm fills its context from four other sources, so
  truthiness passes under the OLD behaviour and pins nothing there.
- the two tails are distinguishable, each excludes the other, and
  neither injects the rule statement — the budget claim, both branches.
- the counters are unmoved, per #3752.
- test_a_rule_the_session_already_holds_is_not_re_offered asserted the
  old contract (`"161" not in context`). Repinned rather than deleted:
  the telemetry half of what it protected still holds.

Repinned the #3497 log-placement guard on structure (rule 167). It
read `body.index("if not fresh:")` — a local variable NAME, not the
property. This change renames that guard to `if not hits:`, so the
old assertion would have raised ValueError and reported #3497 as
back while the arm was entirely correct. Now walks the AST for the
first early return after the search and asserts the call row is
written before it. Falsified both ways: it fails on the #3497
mutation, and it refuses to pass when no early return exists at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-09 21:48:50 -04:00
bvandeusenandClaude Opus 5 d5ac8408f6 feat(telemetry): record WHAT the bar turned away, not only how close it came (#3807)
CI & Build / Python lint (push) Successful in 8s
CI & Build / Plugin hooks (push) Successful in 16s
CI & Build / integration (push) Successful in 40s
CI & Build / TypeScript typecheck (push) Successful in 43s
CI & Build / Python tests (push) Successful in 1m14s
CI & Build / Build & push image (push) Successful in 2m59s
#3670 added `best_available_score` so a threshold could be judged from its
rejections. It records how CLOSE the bar came to firing and not WHAT it
refused, and that is the half a decision actually needs.

Live, pre_tool_rule sits at a ~0.72 bar with a near-miss p90 of 0.7071 —
about 117 declines a day within 0.013 of firing. Dropping to 0.707 would
take that arm from 22 hits a day to roughly 139: six-fold, on a surface
that runs before every Bash call. The percentile says the mass is there.
Nothing said whether it was worth showing.

NEITHER OBVIOUS INSTRUMENT ANSWERS IT. Pull-through cannot: the injected
rule line already carries title and trigger, so a session can comply
without ever calling get_rule, and rule pull-through understates
usefulness by construction. Reading the rejected records can — and
`result_ids` holds only what was RETURNED, so on a zero-result call the
near-missed record had no name at all.

So the id, from the SAME ranked candidate as the score. Both searches
unpack `best` once and read both fields off it, because splitting that
into two expressions is exactly how a later edit pairs a score with its
neighbour's id — and a score attached to the wrong record is worse than no
id, since it invites judging the wrong one and concluding the bar is fine.

write_path withholds the id on the same condition it withholds the score
(#3739): a surviving id beside a null score names a record without saying
what it scored, the pair disagreeing in the other direction.

THE READ PATH IS A LISTING, NOT A STATISTIC — an id cannot be percentiled,
and a reader tuning a bar needs to go and read the records. Opt-in via
`near_miss_samples` (0-20, default 0) so the ordinary readout keeps its
size, and deliberately NOT a window function: this module's one production
outage was a grouped query Postgres rejected, swallowed by the broad
except, every counter reading zero while the mocked tests passed (#2663).
One flat ordered query, overfetched, bucketed in Python — the shape that
lesson prescribes.

Migration 0097, nullable and unbackfilled. Not a foreign key: the table
spans record types and `source` says which, exactly as result_ids works.

The integration guard pins the listing as PER SOURCE. A global LIMIT would
let a noisy source eat the whole quota and leave the surface being tuned
showing nothing — which reads as "nothing was close", the misreading this
milestone has spent itself correcting.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-09 21:24:32 -04:00
bvandeusenandClaude Opus 5 623464323e fix(telemetry): a search that never ran is not a decline (#3765)
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / Python lint (push) Successful in 4s
CI & Build / TypeScript typecheck (push) Successful in 46s
CI & Build / integration (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m21s
CI & Build / Build & push image (push) Successful in 36s
`best_available_score` was added by #3670 so a bar could be judged from
what it rejected, and it arrived null on four unrelated causes: the corpus
offered nothing, the query was empty, the embedder was down, or the
DATABASE QUERY FAILED. Only the first is a measurement. The fourth is the
#2663 shape — a swallowed failure rendering as a clean zero — inside the
field added to fix an instance of the #2663 shape.

Found while trying to explain why reuse_slot returned nothing on 45 of 45
calls, and auto_inject on 153 of 161. That investigation is still open;
what it established first is that the readout could not answer it.

THE FIX IS NOT A NEW COLUMN. A call that never searched writes no row, so
every remaining null means one thing: searched, and nothing came close.
That is the convention the pre-tool arm already follows for a blank
command — "a row here would report a call that never happened and drag the
clear-rate down with phantom declines" — extended from the case a caller
can see in advance to the ones only the search knows about.

Both searches stamp `report["searched"]` FALSE before anything can return
and True only where a real result set exists, so every early return leaves
it false. It has to be the first thing done to the dict: a return added
above that line would leave the key absent.

ABSENT IS A THIRD STATE AND IT DEFAULTS TO TRUE. A caller that passes no
report cannot know, and the safe reading there is the old behaviour. Only
a real search can report False, so absent means "nobody asked" and never
"it failed" — which is also why 66 existing mocked searches across twelve
test files keep working unchanged rather than being rewritten to simulate
a flag they do not care about.

A FAILURE IS NOT MADE INVISIBLE. semantic_search_notes already logs a
WARNING on a query failure, which is where a broken search belongs: a
counter cannot say "I am broken" without a reader already trusting it.

Three tests, and the middle one is what makes them discriminate — a
blanket `return` passes the first and fails the second, because a call
that searched and came back empty is the only evidence a threshold is too
high (#3497).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-09 12:47:46 -04:00
bvandeusenandClaude Opus 5 277f5df515 fix(telemetry): write_path reported records it withheld itself as near misses (#3739)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 24s
Found by verifying the previous fix on live data — the check that fix was
meant to make possible.

    write_path   near_misses.max  0.822    p90 0.7521
                 top_score.min    0.6857   so the bar is at or below this

A "rejection" that outscored every acceptance, and not one outlier: the p90
is above the bar too.

#3739's fix keyed on `suppressed_count`, and I justified its NULL branch as
"null means the caller passed its exclusions INTO the search, so the score
is already post-exclusion". That holds for auto_inject and reuse_slot, both
of which log the RAW search output and do their Python filtering after. It
does not hold for write_path, the one note arm that filters TWICE:
`exclude_ids` takes `seen - pulled_seen` into the search, but the
pulled-and-seen ids stay in the query on purpose — the arm's query doubles
as the resemblance test — and are dropped afterwards in Python. So the row
carries a POST-filter count beside a PRE-filter score.

The suppression column cannot rescue it the way it does for the rule arms.
This arm's count would be PARTIAL — covering the drops made here and not
the ones `exclude_ids` made inside the search — and a partial number under
a name that reads as complete is the substitution this milestone exists to
stop.

So it reports null whenever its own filter removed anything: not measured
on this call, because the bar was not the only thing that turned something
away. Calls that withheld nothing keep reporting, which is most of them.

Both directions are asserted. Without the second test, setting the field to
null unconditionally would pass the first while deleting the measurement
#3670 was built for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-09 00:34:11 -04:00
bvandeusenandClaude Opus 5 ab14f783e1 chore(plugin): mint 2026.09.09.0408 — the hook change has to reach the cache (#3749)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 1m8s
CI & Build / Build & push image (push) Successful in 16s
The manifest gate caught this, which is what it is for:

    FAIL  plugin content changed but the version is still 2026.09.04.0140.

An install has two halves and only one self-updates. The marketplace clone
pulls on its own; the cache that actually EXECUTES refreshes only when this
string changes. So a hook edit shipped without a bump reaches the repo and
stops there — and the obvious debugging move, inspecting the clone, shows
the fix present while the broken copy keeps running. That is #2209, #1040
and #2220, and the only detector was the operator saying "I don't think it
updated".

Minted with scripts/mint_plugin_version.py rather than hand-edited: the
plugin ships straight from the repo with no build step, so there is no
moment at which CI could stamp a value, and the script is the path the
mint guard pins.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-09 00:08:55 -04:00
bvandeusenandClaude Opus 5 1cfbf43ccd fix(plugin): the rule ledger clears when the context it describes is destroyed (#3749)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Failing after 8s
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 21s
The prior-art and tool-rule hooks record every rule id they have named in
<state>/<sid>.rules.ids and hand it back as exclude_rule_ids, so a rule is
surfaced once per session and then goes quiet. That is correct while the
session still HOLDS what it was told.

A compaction breaks it in the worst available way: it summarizes the
earlier injections out of context and does not touch the filesystem. The
rule ends up absent from context AND still excluded — unreachable for the
rest of the session. The compaction banner this hook already prints tells
the model to re-pull its ALWAYS-ON rules, but a rule an arm surfaced is
conditional and is not in that set, so it has no other way back. The rules
most likely to be in that state are the ones that fire most often.

The stale ledger is genuinely found again rather than orphaned: the etag
marker further down this same hook is rewritten on `compact` and keyed by
session_id, which is only meaningful if the id survives a compaction.

CLEARED ON THE SOURCES THAT DESTROY CONTEXT, AND ONLY THOSE. `compact` and
`clear` destroy it while the file survives. `resume` does not — the
context came back intact, so clearing there would re-surface every rule
after a restore that lost nothing, which is the same defect from the other
side. `startup` is a no-op against a new session id. `fork` keeps it, and
the answer holds whichever way forks are keyed: a fork carries the
conversation, so an inherited id means an accurate ledger and a new id
means an empty file.

Only the RULE ledger. The same directory holds .ids / .sync.ids /
.derive.ids for the note arms; whether a surfaced note should return after
a compaction is a different question with a different answer, and a `rm`
glob would have decided it silently.

The guard pins the DISCRIMINATION, not the deletion: the whole source
table is asserted in one statement, so a blanket delete (all False) and a
no-op (all True) both fail, and neither can be made to pass by editing one
case. A second test pins the scope against that glob, and a third proves
an event with no session id clears nothing rather than falling back to a
wildcard.

Runs with no SCRIBE_URL/SCRIBE_TOKEN on purpose — the clear is local,
keyless and networkless, and must still happen against an unreachable
instance. That is also why it sits above the config read rather than
inside the dynamic tier.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-09 00:04:33 -04:00
bvandeusenandClaude Opus 5 a165483b92 fix(telemetry): a repeat is not a rejection, and near_misses counted it as one (#3739)
CI & Build / Build & push image (push) Successful in 31s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / integration (push) Successful in 45s
CI & Build / Python tests (push) Successful in 1m27s
Caught on the first live read after deploying #3670. The readout
contradicted itself:

    pre_tool_rule   top_score.min    0.7204   the lowest score ever RETURNED
                    near_misses.max  0.7457   "rejected", but scored higher

`best_available_score` is measured pre-threshold, which is right, but for
the rule arms it is also PRE-EXCLUSION, which is not. The note arms pass
`exclude_ids` into semantic_search_notes so their score is already
post-exclusion and clean; `semantic_search_rules` takes no such parameter,
so the rule arms filter in Python after the search and a rule that cleared
the bar and was dropped as a repeat still reported its score on a
zero-result row.

That is #3497's distinction — a ranker decline versus a reader already
ahead of it — reintroduced one level up, inside the field built to replace
a tautology.

The population now also requires `suppressed_count IS NULL OR = 0`. The
NULL arm is principled rather than permissive: null means the caller
filtered INSIDE the search, which is exactly the case where the reported
score cannot be contaminated.

Deliberately conservative — a call carrying both a repeat and a lower
genuine miss is dropped whole, losing that point. It undercounts; it
cannot corrupt, which is the right way round for a number read against a
bar.

It also makes `near_misses.max < threshold` true BY CONSTRUCTION rather
than by fixture: an above-bar candidate nobody excluded would have been
returned, so its call is not in the population at all.

THE TEST DID NOT CATCH THIS, and that is the part worth keeping. The
assertion `nm["max"] < 0.72` was already there, with exactly the right
intent. It passed because the fixture contained no suppressed call — the
guard held because the breaking shape was absent, not because the code was
right. Rule 167's stated failure mode, in a test written while citing rule
167. The fixture now builds that shape: a 0.9 hit dropped as a repeat,
which lands in the population and drags `max` above the threshold unless
the predicate excludes it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 16:42:51 -04:00
bvandeusenandClaude Opus 5 e7c1af32a0 fix(telemetry): the bar can only be judged from what it rejected (#3670)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 28s
`cleared_threshold` was documented as the number to read first. It was a
tautology. The search applies the threshold before returning, so every
returned result cleared it by construction and a call with no results has
no top_score to compare — the condition was true exactly when
`result_count > 0`. It was `calls - zero_result_calls` under a name that
promised a second opinion, and `zero + cleared == calls` held on all
nineteen source/window readings ever taken, today's live seven included.

The reading procedure built on it asked the reader to compare a number
with itself, and a threshold change was unobservable through it: raise the
bar and both numbers move together, so the field could never show a bar
set too high.

REPLACED, NOT JUST REMOVED. The question the table exists to answer is
whether the bar is in the right place, and that is only answerable from
the calls that returned NOTHING: how close did the best rejected candidate
come? A 0.72 bar turning away a stream of 0.71s is set too high by a hair;
the same bar turning away 0.30s is working. Both render as a zero-result
call today and nothing separates them, because the losing score is
discarded inside the search.

So both searches now rank WITHOUT the bar and apply it in Python. The
qualifying set is provably identical — rows arrive ordered by distance, so
every above-bar row sorts ahead of every below-bar one, and an over-fetch
that returned N above-bar rows returns the same N plus some losers. What
changes is that the losers are visible instead of dropped in the query.
`report` carries the score out without changing what a search RETURNS:
eight of eleven call sites want hits and nothing else.

New column (migration 0096), nullable and unbackfilled. A row written
before this genuinely does not know, and a 0.0 would read as "the corpus
held nothing remotely relevant" — a claim invented out of a caller's
silence, which is the substitution this whole milestone corrects.

The new aggregate is a percentile_cont WITHIN GROUP over a CASE, one step
from the shape that produced #2663, where a rejected query was swallowed
by the broad except and every counter read zero. It carries an integration
guard for that reason: only real Postgres can say it parses, and the
symptom of failure is silence.

Also adds a guard that no int field in a bucket equals
`calls - zero_result_calls`. That identity is what `cleared_threshold`
satisfied for its whole life, and it survived because it had its own name
and nobody added the two numbers beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 13:50:05 -04:00
bvandeusenandClaude Opus 5 277aea58e4 test(telemetry): pin the identity that falsified this milestone (#3668)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / TypeScript typecheck (push) Successful in 10s
CI & Build / integration (push) Successful in 27s
CI & Build / Build & push image (push) Successful in 23s
Reverts the deliberate break from 5e19a1b. `plugin_context.py` is now
byte-identical to before it; the only change against that baseline is the
guard itself.

FALSIFIED, not argued (rule 167). CI run 6046 with `results=hits` in both
arms failed exactly the predicted four cases —

    [one-already-held-write_path]   FAILED
    [one-already-held-pre_tool]     FAILED
    [all-already-held-write_path]   FAILED
    [all-already-held-pre_tool]     FAILED
    [nothing-held-*]                passed

— and the nothing-held cases passing is the point, not a gap: with no
exclusions both recorders see the same list however wrongly they are
wired, so that case can never discriminate and a guard built only from it
would read as coverage while catching nothing.

WHAT IS PINNED. Both arms build one `fresh` list and hand it to two
recorders in one function, so the call log and the surfacing log cannot
disagree about what a single call showed. Ids, not counts: equal counts
drawn from different lists is a real way for this to break, and a count
comparison would call it agreement.

Three hits, where production returns at most one. The identity holds at
any limit because both recorders read the same list, and stating it that
way survives RULEHINT_LIMIT moving again — it has moved once already
(2 → 1, 2385100), and that move is half of why the original
reconstruction misread its own numbers.

NOT DONE, deliberately: the readout-level self-check the task also
proposed. `cleared_threshold` counts CALLS that beat the bar while
`surfaced` counts RULES, so that identity holds only while the limit is
1 — it would fire on a healthy system the moment the limit rises. The
arm-level form has no such coupling, which is why the guard lives here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 13:36:03 -04:00
bvandeusenandClaude Opus 5 5e19a1b028 test(telemetry): FALSIFICATION — prove the identity guard can fail (#3668)
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 34s
CI & Build / Python tests (push) Failing after 45s
CI & Build / Build & push image (push) Skipped
Deliberately broken, reverted in the next commit. Rule 167 requires every
guard be falsified against the regression it names before it is trusted,
and rule 10 puts CI as the only place that can run it — so the failure has
to be made to happen here rather than argued for.

The guard: both rule arms feed one `fresh` list to two recorders, so the
call log and the surfacing log cannot disagree about what one call showed.

The regression: `results=fresh` becomes `results=hits` in both arms, so
the call log counts what the ranker found while the surfacing log counts
what was shown. That is not a hypothetical shape. It is exactly the
divergence that would make a correct system report a lost write when the
two tables are later compared in aggregate — the reading that scoped this
milestone at five steps against a defect that did not exist.

Expected red: the one-already-held and all-already-held cases on both
arms. The nothing-held case must still PASS — with no exclusions both
recorders see the same list however wrongly they are wired, which is why
it could never have been the discriminating case.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 13:31:49 -04:00
bvandeusenandClaude Opus 5 7a2aff7bc1 fix(telemetry): a surface that stopped recording is not one that never ran (#3720)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 34s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 24s
`out["sources"]` was built only from the windowed aggregate, so a source
with rows in `retrieval_logs` but none inside the window got no bucket at
all. Absent is exactly how a source that never existed renders, so a
surface that WAS recording and went silent became unreadable — #2663 one
level up, the failure that looks like the correct answer.

Two queries at different scopes, and only one shaped the output.
`_complete_from` reads all-time and knows every source the table has ever
held; the windowed loop dropped whatever it did not return.

Every such source now gets a zero bucket. Zero is a real measurement here
rather than a manufactured one: the all-time query proves the source was
recording, and it made no calls across a window it fully covers. No
`covers_window` special case is needed either — a source whose first row
fell after `since` would have that row IN the window and already hold a
bucket, so anything reaching this branch began before it.

The counts are 0 and everything else is null. A sampled distribution is
not the same claim as a call count, and rendering p50 as 0.0 for a source
nobody sampled would assert a measurement — #3311's mistake, in the
readout built to prevent it.

Found while fixing #3712's fixture, which failed with KeyError for this
exact reason.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 11:07:21 -04:00
bvandeusenandClaude Opus 5 0808e8259a test(telemetry): the old surface needs a row in the window to have a bucket at all (#3712)
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 15s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 11s
The per-source grain test gave `auto_inject` a single row 90 days back and
`pre_tool_rule` one 2 days back, then asserted on both buckets. Only the
young arm got a bucket: `out["sources"]` is built from the WINDOWED query,
so a source with no rows inside the window is absent entirely, and the
assertion died on KeyError before it could test anything.

`complete_from` and the bucket come from different queries — all-time for
the first, windowed for the second — and the fixture only satisfied one of
them. Gave `auto_inject` a second row inside the window, which is also the
shape being described: an old surface that is STILL recording. The 90-day
row still sets its `complete_from`.

Still discriminating: auto_inject reads True and pre_tool_rule False, and a
per-table `_complete_from` would make both 90 days and fail the second
assertion — the regression this test is for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 10:39:42 -04:00
bvandeusenandClaude Opus 5 950c93c5d4 fix(telemetry): the coverage helpers were defined inside the function they serve (#3712)
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Failing after 28s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 32s
`_complete_from` and `_coverage` landed between `retrieval_summary`'s
docstring and its body. Python does not object to that the way it looks
like it should: blank lines do not close a block, so the whole remaining
body — indented four spaces, sitting after `_coverage`'s `return` —
became unreachable code INSIDE `_coverage`, and `retrieval_summary`
became a function that is nothing but a docstring.

The error surfaced three ways at once, none of which named the cause:
fourteen F821s for `days` and `user_id` (real: those are
`retrieval_summary`'s parameters, and the body no longer lived there), a
SyntaxError on `async with` (real: `_coverage` is sync), and every test
module that imports this file failing to collect.

Moved both helpers above `retrieval_summary`, beside `_bucket` and
`_round`, where the file's other helpers already are. No behaviour
change — this is the code that was meant to be there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 10:34:57 -04:00
bvandeusenandClaude Opus 5 21a5831479 feat(telemetry): every counter says when it started being recorded (#3712)
CI & Build / Python lint (push) Failing after 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Failing after 17s
CI & Build / Python tests (push) Failing after 26s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Build & push image (push) Skipped
A window that opens before a counter existed reports that counter as though
it had been measured throughout. The reader cannot tell "zero because
nothing happened" from "zero because nobody was counting yet", and — worse
— cannot tell a partial count from a complete one. That middle case yields
a plausible FRACTION rather than an obvious zero, which is what makes it
dangerous.

It is not hypothetical. A 7-day window opened while the ranked rule
surfacing recorders were four days old produced an apparent 64% write loss,
which survived a code review, four ruled-out alternative causes and a
five-step milestone before an identity check falsified it in one read.

Every counter block now carries `complete_from` and `covers_window`.

THE GRAIN IS THE SOURCE. retrieval_logs accumulates for months, so a
per-table earliest row says months for every source it holds — including an
arm added days ago whose counter means something else entirely. The old
source would vouch for the young one, which is the exact reading this
prevents.

A SECTION TAKES ITS LATEST CONTRIBUTOR, NOT ITS EARLIEST. A figure summing
several sources is complete only once every one of them was being written,
so "*" is a max. Using min would reproduce the original error in miniature.

`covers_window` is null, never false, when nothing was ever recorded: "no
measurement" is not "partial measurement" — the null convention #3497
established for `suppression`, one level up.

Also corrects a stale claim in the tool docstring: it still taught readers
that write_path_rule "has never once declined to fire" (#3311). That was
the arm writing its retrieval_logs row only on calls that found something;
#3497 fixed it, and the arm declines the large majority of its calls.

_complete_from takes the caller's session rather than opening its own,
departing from the services canon (#2860) because it runs inside an
existing block; to be recorded against the ledger once it ingests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 10:29:02 -04:00
bvandeusenandClaude Opus 5 dd1e6e2645 feat(rules): the rule arms stop filtering the corpus to one tier (#3702)
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 1m3s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / Build & push image (push) Successful in 1m29s
Both arms passed tier="conditional", on the reasoning that an always-on
rule is already in the session so re-surfacing it is pure noise. That
conflates two different things:

  PRESENT IN CONTEXT    the rule was delivered at session start
  SALIENT AT THE MOMENT the rule is in front of the reader when the action
                        it governs is about to be taken

A rule handed over in a list at turn zero is present while a session writes
a config value three hundred turns later. It is not surfaced. So the filter
did not skip a redundant hint — it made a whole class of rules permanently
ineligible for the only mechanism that puts a rule in front of an agent AT
the moment, and the more important a rule is, the likelier it sat in that
class.

Underneath, the filter was doing the THRESHOLD's job. Whether a rule belongs
in a hint is a relevance question and a similarity bar is the control for
relevance. A categorical exclusion standing in for a relevance judgment
cannot be tuned, cannot be measured, and cannot be wrong in a way anybody
notices.

MEASURED, NOT SETTLED. The old comment's fear is real: a hint that fires on
every write and says obvious things teaches the reader to skip the block. It
had simply never been checked, and retrieval_logs already records the scores
to check it with. Rules clearing often and high means the fear was justified
and the BAR is the work; rules clearing rarely in a thin band means relevance
was always sufficient.

Only eligibility moved. The bar stays at 0.72 and k stays at 1, so the
resulting distribution has one cause — and k=1 bounds the blast radius: a
wider pool can change which rule surfaces and how often, never how long a
single hint gets.

The threshold rationale's first premise ("the eligible corpus is TINY —
tier=conditional only") is updated rather than deleted: a larger pool makes
clearing the bar mean MORE, so that argument weakened, and the bar was left
alone anyway rather than move two variables at once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 00:24:02 -04:00
bvandeusenandClaude Opus 5 c3ecdf0972 feat(rules): the rule gate becomes a practice with a question, not a prohibition (#3557)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 25s
The first cut opened "NOT YOURS TO CALL UNPROMPTED", and that is the wrong
instrument. A caller reading a prohibition stops NOTICING rule-shaped things
rather than noticing them and asking — which trades a small failure for a
larger one. The wanted behaviour is more proposals, not fewer.

So both docstrings now describe the practice: propose readily, state the
four things, and close with a question the operator answers in one word —
approve it as written / let's talk about it / no. Named options where the
interface has them, three written-out options where it does not.

"Approve it as written" is what makes element 1 load-bearing: they approved
TEXT, so that text is stored verbatim. "Let's talk about it" is framed as
the expected answer rather than a setback. "No" routes the observation to
create_note, which records without binding.

The argument for asking is also better than consent. The operator's yes is
the one moment the rule is certainly in front of them: afterwards a
conditional rule is not read aloud at session start, and a project rule is
absent from an unfiltered list_rules(). The proposal IS the review.

Guard gains the answers-offered-back element and drops the wording that
forbade; its header records why the framing changed, so the prohibition does
not get reintroduced as a tidy-up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-04 20:11:42 -04:00
bvandeusenandClaude Opus 5 1a34363059 feat(rules): the rule-creation tools ask for approval, and ask what would enforce it (#3557)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 32s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 41s
Every gate on create_rule and create_project_rule was about SHAPE — rule vs
process vs snippet, one-thing-you-could-violate, general-enough, not-a-dupe.
All of them improve a rule someone has already decided to write. None asked
the prior question: has the person this will bind agreed to be bound by it?

Both docstrings now open with the gate, and with the four things a proposal
carries: what it would require in the words it would carry, its intent, why
now, and how it would be enforced.

The fourth is the one that decides it. "A test, a CI check, a hook, a schema
constraint... or nothing" is a question that sometimes dissolves the rule:
what a test can assert should BE that test, and a rule is what is left when
nothing mechanical can hold the thing. A rulebook grows by default and
shrinks only on purpose.

create_project_rule needs the gate more, not less, and says so: a project
rule is absent from an unfiltered list_rules(), and a conditional one is
absent from session start too, so one written there can bind for months
without ever having been in front of the person it binds.

test_rule_creation_asks_first pins structure, never wording — each element
matches a family of synonyms, and the gate must precede the Args: block,
because a caller who has decided to make the call reads the parameters and
not the prose under them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-04 20:02:05 -04:00
bvandeusenandClaude Opus 5 30d87e461a feat(rules): the instruction surfaces say to RETRIEVE a rule, not only to receive one (#3523)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / TypeScript typecheck (push) Successful in 4m18s
CI & Build / Build & push image (push) Successful in 25s
Every instruction surface told a session to load the always-on rules and
stopped there. None said the loaded set is partial, so an empty one read as
"no rule applies" when it only ever meant "none was pushed" — different
claims, and only one of them has been checked.

That is #2198's asymmetry one level in. The earlier defect was trusting the
SessionStart push over the explicit pull; this is trusting the resident TIER
as if it were the whole rulebook.

It is also why the always-on tier was the only one that worked, on any install
rather than this one (rule 115): a rule nothing retrieves must be resident to
bind at all, and a resident rule costs tokens in every session forever — so a
rulebook that only delivers cannot grow past what one session holds.
Retrieval lifts that ceiling, and it fires only if something asks. A
tool-choice reflex asks least of all (#3476, #161).

The same obligation now lands on all three session-start surfaces, because
rule 119 makes them the specification jointly and a surface stating it
differently IS the product behaving differently (#2497). Pinned by
test_every_session_start_surface_states_the_conditional_retrieval, mirroring
the pull test beside it.

THE BUDGET TRADE. _INSTRUCTIONS sat at 1978 against a 2000 test budget, and
its own comment says an addition there is a trade, never an append. Bought the
new clause by trading out "Processes are saved procedures (follow verbatim)"
and "Deletes are trash-recoverable" — both already in DISPLACED_TOPICS and
already stated on a delivered surface, and both per-tool guidance, which by
this block's doctrine belongs in the tool docstring. Now 1976. Recorded in the
comment above the block so it is not silently reversed.

Plugin version minted: shipped plugin content moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-03 21:41:53 -04:00
bvandeusenandClaude Opus 5 8be555d6dd feat(telemetry): tell a ranker decline from a repeat before the observation window opens (#3497)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 31s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Successful in 28s
Making the rule arms log every call exposed a second ambiguity in the same
row. `result_count == 0` is two unrelated events wearing one number:

  - the ranker found nothing above the bar — the only evidence a threshold is
    set too high; and
  - the ranker found only what this session had already been shown — which
    says nothing whatever about the bar.

A long session excludes its way into the second, so the arm reads worse the
longer it runs correctly. Rows written now carry the ambiguity permanently,
which is why this lands before any watch period rather than after.

`retrieval_logs.suppressed_count` (0095, nullable) holds what the caller
dropped as already-shown. Both rule arms report it; they filter in Python and
always know. The note arms pass exclusions INTO semantic_search_notes and
never see what was dropped, so they store NULL.

THE NULL IS LOAD-BEARING. It means "not measured here", and the readout
renders it as `suppression: null` rather than a zeroed dict. Defaulting to 0
would let an unmeasured surface read as a perfectly clean one — the same
substitution of an artifact for a measurement that #3311 made. No backfill,
for the same reason: existing rows genuinely do not know.

`retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`,
`calls_with_suppression` and `zero_because_already_shown`; subtract the last
from `zero_result_calls` for the true ranker declines. The MCP tool docstring
says to read the two together and warns against reading the null as a zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-03 21:12:38 -04:00
bvandeusenandClaude Opus 5 48804c437d fix(tests): the write-path telemetry test asserted the defect, not the split (#3497)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 28s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 1m8s
CI & Build / Build & push image (push) Successful in 28s
`assert_called_once` held only because the rule arm skipped its retrieval_logs
row when it found nothing. With the arm logging every call, the test now
asserts what it was always about — exactly one `write_path` row, no
`auto_inject`, and the rule arm keeping its own separate source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-03 06:59:52 -04:00
bvandeusenandClaude Opus 5 154a5de13e fix(telemetry): both rule arms logged only their hits, so the clear-rate could only read 100% (#3497)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Failing after 48s
CI & Build / Build & push image (push) Skipped
`write_path_rule` reported `zero_result_calls: 0` and `cleared_threshold:
133/133` — a perfect record no other surface comes near (`write_path` 421
zeroes of 613, `reuse_slot` 124/199, `auto_inject` 114/326). #3311 read that
as a measurement and milestone 333 was scoped on it.

It was an artifact. Both arms called `record_retrieval` inside a guard on
having results — the write-path arm behind `if fresh:`, the pre-tool arm
below `if not fresh: return out` — so a call that found nothing wrote no row.
The statistic was a fact about the shape of the code, true at any threshold
whatsoever.

The call log moves out of the guard in both arms. The surfacing log stays in
it: nothing was shown, so no surfacing occurred. `results=fresh` is kept
deliberately — the note arms pass exclusions into `semantic_search_notes`, so
what they log is already post-exclusion, and logging `hits` here would make
this row mean something other than every other row in the same readout.

The defect bites hardest on the pre-tool arm, which fires on every Bash call:
with no rows at all, a ranker that declined is indistinguishable from a hook
that never fired — the silent failure the arm exists to stop.

Tests cover both arms behaviourally (found nothing; found only what the
session already held; searched nothing at all, which must stay silent) plus a
structural guard, because this was one level of indentation and it appeared
independently in two places.

#3311 and the `rule_usage` docstring corrected rather than quietly rewritten.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-03 06:57:13 -04:00
bvandeusenandClaude Opus 5 2ee24b9d2b feat(rules): rules before tools — a PreToolUse arm keyed on the action (#3476)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 32s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python tests (push) Successful in 1m9s
CI & Build / Build & push image (push) Successful in 25s
The only just-in-time rule surface was registered on `Write|Edit` and queried
with `code or path`, so a rule could be retrieved at the moment of a code
write and nowhere else. Every rule about which tool to reach for — don't curl
the forge, don't stand up a stack, don't run the suite locally, don't branch —
was unreachable exactly when it mattered, and residency in the always-on
preload was the only surface it had. That is the pressure that grew the
resident set to 31 against #3089's ceiling of ~23; it was never a judgment
anybody made.

A reflex generates no query, so an instruction to check the rules cannot catch
one. A mechanical trigger can: the tool call IS the query, and a reflex has to
become a tool call before it can do anything.

`build_tool_rule_hint` is deliberately tool-agnostic — a name and a string —
so widening the matcher later is a hooks.json edit with no server change. The
hook starts on Bash, which is where the action reflexes live.

The two pre-tool arms share ONE session ledger of already-named rules
(`<state>/<sid>.rules.ids`). Two ledgers would mean a rule named by one arm
gets re-offered by the other, and the hint that fires most often is exactly
the one that must not repeat itself. A test asserts both scripts build the
same path, and another checks the shell hook and the Python route agree on
every query-arg name (rule 33) — a rename there fails silently, looking like
a surface that never finds anything rather than a broken one.

Deliberately silent on outage, unlike the prior-art hook: a write is
occasional, a Bash call is not, and an outage line before every command is
what gets a channel muted.

`tier="conditional"` matches the write arm and is the transition point — an
always-on rule is already resident, so re-tier one and it starts arriving here
instead of in every session's preamble. `pre_tool_rule` joins RANKED_SOURCES:
this arm chose what it showed, so a pull can settle whether the choice landed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-02 23:31:22 -04:00
bvandeusenandClaude Opus 5 8b9b3a1d9b feat(telemetry): the preload emits, and the always-on set stops being unfalsifiable (#3473)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 27s
The ranked rule arm became measurable in M333. The preload did not — and
that is the surface whose value is actually in question. `list_always_on_rules`,
the SessionStart block and every `rules_payload` caller handed rules over
wholesale and emitted nothing, so the resident set's token cost was certain
and its usefulness could not be tested even in principle.

Bulk deliveries now record as AMBIENT, beside the ranked count and never
inside pull-through. Folding them in would mean growing the always-on set
depressed the arm's measured precision and trimming it flattered the arm,
neither for any reason to do with the arm.

`RANKED_SOURCES` inverts the note twin's `AMBIENT_SOURCES` deliberately: there
is one ranked rule source and this change adds seven bulk ones, so naming the
rare half makes a forgotten surface default to ambient — under-counting it —
rather than padding the denominator with surfacings nobody chose.

Two lookalike call sites are deliberately left silent, with a test to keep
them that way: the write-path etag arm and `rules_etag_for` read the rules to
build or compare a MARKER and show nobody anything.

No migration — `event` and `source` are plain Text with no CHECK (rule 36
does not apply). Snippet #2858 updated to the new `rules_payload` contract.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-02 23:06:40 -04:00
bvandeusenandClaude Opus 5 6627cfc2f0 feat(rules): a usage badge on the rule list, and the badge becomes canon instead of a second copy (#3319)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 32s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 33s
Milestone 333 step 5, and rule 27 — the counter had a tuning point from step
4 and no operator-facing one until now.

The task said to reuse the snippet badge's classes rather than mint a parallel
set, citing the eight duplicated CSS families the ledger already carries
(#3207). `.usage-tag` lived in SnippetListView's SCOPED block, so "reuse" was
not available: copying it into the rule pane would have been the ninth family,
and importing it is not a thing a scoped block permits. So it was promoted
rather than copied.

Three pieces, each of which existed once and now exists once:

- `components.css` gains `.usage-tag` / `.usage-dead`, geometry and colour
  only, with the scoped original deleted rather than left behind.
- `UsageBadge.vue` holds the logic the two lists would otherwise duplicate —
  the >=3 dead-weight threshold, the empty-string-renders-nothing rule, the
  tooltip.
- `types/usage.ts` holds `RecordUsage`, one client type over two tables.
  `SnippetUsage` becomes an alias, so no existing consumer changes.

THE ADVICE IS A PROP, and that is the substance rather than the plumbing. The
counts read identically for every kind; the remedy does not. A snippet offered
and never opened should probably be rewritten or deleted — one action. A rule
in the same position has TWO possible causes and the operator has to pick:
its trigger may fire on the wrong work, in which case `when_to_apply` wants
rewording, or it may genuinely not be wanted. Baking "delete it" into the
component would give the wrong nudge half the time on the surface where being
wrong is most expensive, since a deleted rule stops binding behaviour.

The route zero-fills every row through `usage_for_rules`, one aggregate per
page — per-row would be N+1 by construction. That matters more here than for
snippets: every rule on every existing install predates `rule_usage_events`,
so the zero-filled shape IS the common case for a while, and a route that
attached the key only where it found events would leave the badge reading
undefined on almost every row.

`usage_for_rules` had no test at all — step 1 covered the write path and the
zero shape and left the aggregate uncovered, which only became load-bearing
when a list started rendering it. It now has an integration test over real
Postgres, including that a rule with no events comes back zero-filled rather
than absent.

Recorded as snippet #3460, per the design system's own instruction that the
component layer lives as snippets rather than as prose.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 18:37:55 -04:00
bvandeusenandClaude Opus 5 238510080e feat(retrieval): the standing-rule arm gets its own bar, and asks for one rule not two (#3318)
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Successful in 35s
Milestone 333 step 4 — the split #2223 made one surface down, now made for the
third corpus. The arm inherited WRITEPATH_DEFAULT_THRESHOLD = 0.68, a number
measured against code-vs-note-PROSE and never re-derived for code-vs-RULE-TEXT.

THE DEFAULT IS ARGUED STRUCTURALLY, NOT READ OFF A HISTOGRAM (rule 115). Two
facts hold on any install, including one with six rules and no telemetry:

- The eligible corpus is tiny — conditional rules only, a handful to a few
  dozen against thousands of notes. A top-k over forty candidates always
  returns something, so "the best match cleared the bar" stops meaning "a good
  match exists". A bar calibrated for best-of-thousands is cleared by
  best-of-forty as arithmetic, not relevance.
- Rules are short imperative technical English, far more homogeneous than note
  prose. #2223 put the code-vs-prose floor at 0.55-0.63 and set 0.68 above it;
  a more homogeneous corpus has a HIGHER floor, so 0.68 is not merely
  inherited, it sits below where this corpus's noise lives.

0.72 errs deliberately toward silence on an asymmetry that is also structural:
this hint fires on EVERY write. A missed rule is recoverable — it is still in
Scribe and the agent can search it. A hint that cries wolf is not: it teaches
the reader to skip the whole block, and the true positives go with it. The
arm's own comment already said "noise on a hint that fires on every write is
how a hint gets ignored".

Pinned as an INEQUALITY, not a value: test_the_rule_bar_defaults_above_the_code_bar
asserts RULEHINT > WRITEPATH, so tuning the number stays free while inverting
the relationship — which would silently reinstate #3311 — does not.

RULEHINT_LIMIT = 1, and deliberately not a knob. With a corpus this small, k=2
means the second line is almost always the second-best noise wearing the same
confident framing as the first; halving k halves that regardless of the bar.
It stays a constant because it is a decision about how loud one hint may be,
not a per-install tuning question — and a knob nobody turns only adds a way to
misconfigure the surface.

Reachable from Settings, no restart (rule 25), with copy that says which way to
move it and points at retrieval_telemetry's rule pull-through — which step 3
made readable — to tell "arriving unread" from "never arrived".

Every config stand-in in the suite gained the key, not just the one that
noticed. The arm reads `rule_threshold` while BUILDING its search arguments, so
a missing key raises inside its fail-open except and turns the arm into a
silent no-op — indistinguishable from it running and finding nothing. That is
the same vacuous-pass shape that bit step 2, one layer down (rule 33).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 18:05:00 -04:00
bvandeusenandClaude Opus 5 8901c904a9 feat(telemetry): retrieval_telemetry reports rule pull-through where it reported nothing (#3317)
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / integration (push) Successful in 32s
CI & Build / Build & push image (push) Successful in 29s
Milestone 333 step 3, the read half. Steps 1 and 2 built the table and filled
it; until now nothing read it, and `usage` — sourced entirely from
note_usage_events — described notes only while `sources` happily listed a
write_path_rule row above it. A reader takes the aggregate as covering
everything named above it. It did not.

A SEPARATE `rule_usage` BLOCK, not folded into `usage`. Two reasons, and the
second is the one that bites: the corpora differ by orders of magnitude, so a
blended ratio would be the note ratio with noise on it and the rule arm would
stay invisible inside it; and `usage` is what existing callers already read and
compare across windows, so silently changing what it counts would move a number
nobody was told had changed meaning. There is a test asserting rule events stay
out of the note block.

No `ambient` key, unlike the twin. Nothing surfaces a rule un-ranked —
list_always_on_rules and enter_project hand rules over wholesale but emit no
event — so there is no ambient class to subtract. The absence is a fact about
the data, not an oversight, and it returns when a bulk loader starts emitting.

Guarded separately, like `by_source`. This table did not exist a commit ago,
and an instance running upgraded code against un-migrated schema would
otherwise take down two readouts that work perfectly in order to report a third
that cannot. On failure the FLAG is added and the SHAPE is kept — a caller must
not have to choose between crashing on a missing key and quietly rendering
zeros it has no right to.

`pull_through` is None rather than 0.0 on an empty window, matching the note
block. A ratio of zero asserts "rules were shown and none opened"; with an
empty numerator and denominator that is a claim the data does not support, and
it is the reading that would make a brand-new install look like a broken one.

Also fixed, from #3311: the rule arm never timed its search, so it was the one
source in the readout reporting a null p90_duration_ms — a gap that reads as
"this surface is somehow not measurable" rather than "nobody passed the
number".

Both docstrings updated in the same change. The tool's is the agent-facing
contract (rule 119) and it explicitly said rule surfacings were absent and had
"no usage counter at all". Leaving that would have had a reader conclude the
arm has zero pull-through rather than a separate one.

Tests are integration for the reason the block above them is: real GROUP BYs
and count(distinct) against a table a commit old, in a module whose one
production outage was a SQL shape the database rejected inside a broad except.
A mock would agree with whatever the code does, including nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 17:25:11 -04:00
bvandeusenandClaude Opus 5 70761b16d9 test(telemetry): the rule-arm fixture never reached the arm — it returned at the guard (#3316)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / integration (push) Successful in 28s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 24s
The product code was right; the test was wrong, and wrong in a way that made
two assertions fail and two others pass vacuously.

`build_write_path_hint` returns early when a write matched nothing at all — no
staleness, no synced record, no prior-art menu, no shape signal. The rule arm
sits deliberately on the FAR side of that guard, because it runs a semantic
search and hoisting it would mean an embedding query on every write in the
session. My fixture stubbed every other arm to empty, so it hit the early
return and the rule arm never ran: `record_rule_surfaced` was called zero
times, and "the recorder was not called" is also what two of the four tests
were asserting for their own reasons.

The fixture now supplies one prior-art hit — 0.72 against a 0.6 threshold, so
it clears the band and the top_k slice — with a comment saying the hit is the
arm's precondition rather than scenery.

And the gate got its own test, because the fixture now depends on it: a write
matching nothing must NOT reach the arm. Without that, a future change to the
guard would make every assertion in this file pass without exercising
anything. #3311 is explicit that the gate stays until the arm's precision is
fixed, so the test says to go read that issue rather than update the
assertion.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 17:17:59 -04:00
bvandeusenandClaude Opus 5 8f7f447fda feat(telemetry): the rule arm records what it showed, and get_rule records the read (#3316)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 24s
CI & Build / integration (push) Successful in 37s
CI & Build / Python tests (push) Failing after 53s
CI & Build / Build & push image (push) Skipped
Milestone 333 step 2. Step 1 built the table; a counter nobody calls reads zero
and looks exactly like a surface nobody uses, which is #2663's shape.

SURFACED — the standing-rule arm in build_write_path_hint, beside the
record_retrieval it already made. Two tables, and the split is not arbitrary:
retrieval_logs is one row per CALL keyed on the score distribution a threshold
is tuned from; rule_usage_events is one row per RULE per event, the grain "was
this hint ever acted on" needs and the grain a JSONB result_ids array cannot be
indexed at.

The comment there said rule ids had nowhere to go — that note_usage_events
remaps ids on restore, so a rule id would return attached to whatever note took
that number. Still true of the NOTE table, and precisely why step 1 built its
own. Rewritten to say the gap is closed rather than leaving a stale rationale
that would have someone re-derive the same dead end.

Records `fresh`, i.e. AFTER exclude_rule_ids. A rule the session already holds
was considered and not shown; counting it would inflate the denominator with
claims the agent never saw, and the ratio would then fall for a reason that has
nothing to do with whether hints land.

PULLED — two doors, both after their access check so a refused read is not a
pull. mcp_get_rule is the one that matters: the arm's own message ends "Read it
with get_rule(N)", so that call is the exact action a landed hint produces.
rest_rule carries the other prefix, and the prefix is load-bearing — "is this
rule dead weight?" is served by any pull, "did that injected hint land?" by
agent pulls only.

NOT a pull: rule_history. It loads the rule for its title and its own output
says "The current wording is on the rule itself — get_rule(N)", so counting it
would credit a read of the history as a read of the rule and double-count
anyone who then follows that pointer. list_always_on_rules and enter_project
are likewise bulk resident loads, not somebody choosing to open one record.

tests/test_rule_usage_wiring.py is cross-cutting on purpose: the surfaced end
is in plugin_context, the pull end in two other modules, and "both ends meet"
is a property no module-shaped file asserts. It covers the exclusion boundary,
that a failing recorder cannot break the write, that a refused read records
nothing, and two completeness guards — every door records, and the bulk loaders
still do not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 17:15:03 -04:00
bvandeusenandClaude Opus 5 111eef7e30 fix(telemetry): the user-scoped rule_usage export read _rule_ids before it existed (#3315)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 29s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 25s
Ruff F821, twice, on the same two lines. The query was placed next to its
note-usage counterpart — which reads `note_ids`, defined much earlier — while
`_rule_ids` is not built until forty lines further down, beside the rules
themselves. Moved to sit directly after the `rule_versions` query, which is the
other consumer of that variable and the block whose scoping argument this one
restates.

Worth noting what did NOT catch this. The integration round-trip passed on the
same commit: it drives `restore_full_backup` against a hand-built payload, so
it exercises the import side and the full export, and never calls
`export_user_backup` at all. A per-user export of any account owning a rule
would have raised NameError at runtime. The lint lane found it because a
static check does not need the path to be reachable by a test.

The comment moved with it and got sharper, since the hazard is that the
plausible column is the wrong one: `user_id` on a usage row is whoever the arm
fired FOR, not who owns the rule, so scoping a per-user export by it would
carry this user's surfacings of someone else's rule and drop the ones fired for
someone else on theirs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 16:56:54 -04:00
bvandeusenandClaude Opus 5 8826be7a91 feat(telemetry): rule_usage_events — the table, the service, and a restore that maps rule ids through the rule map (#3315)
CI & Build / Python lint (push) Failing after 3s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 30s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Skipped
Milestone 333 step 1. The write-path standing-rule arm is the only retrieval
surface in Scribe whose usefulness cannot be observed — and, not
coincidentally, the only one that has never declined to fire. 296 calls, zero
zero-result, 100% clearing its threshold, while every other surface declines
most of the time (#3311, and re-measured in note #3430). `retrieval_logs`
gives it scores; scores say what the ranker thought, never whether the hint
landed.

WHY A SIBLING TABLE AND NOT A COLUMN ON note_usage_events. The row carries no
note-specific field and the readout is the same shape, which is the strongest
case for sharing that note #3163 admits. What decides against it is identity at
RESTORE: the note importer maps note_id through note_id_map, so a rule id
parked in that column comes back attached to whatever note holds that number in
the target database. Not dropped — reattached. The restore reports success, the
counters are populated, and every one is about the wrong record, with no other
field to disagree with. rule_versions made the same call for the same reason;
this is the third rule-side sibling and it reads like the first two.

FK-free on rule_id and user_id, matching note_usage_events / retrieval_logs /
app_logs, and deliberately unlike rule_versions. A version belongs to a rule's
history and dies with it; telemetry outlives what it describes. Deleting a rule
must not erase the evidence that it was surfaced forty times and opened never,
because that evidence is the case for having deleted it.

The service uses `background.spawn` rather than a third copy of the
strong-reference dance — that module's own docstring says new callers should,
and a fourth copy is how one of them drifts. The AppLog canary #2663 demands is
kept, and since `rule_usage` needed exactly `note_usage`'s semantics, that
canary moved into `background.report_telemetry_failure` and note_usage now
calls it. `retrieval_telemetry` deliberately keeps its own: its canary is a
different shape (one process-wide flag, no AppLog row), so repointing it would
change behaviour rather than consolidate it.

No ambient bucket, and that is a decision. The note twin splits ranked from
ambient surfacings because enter_project and the skill sync deliver records
without choosing them (#2477). Rules have the same problem waiting —
list_always_on_rules loads them wholesale — but nothing emits here yet, so an
empty AMBIENT_SOURCES would be machinery pretending to a distinction the data
does not contain. `source` stays granular, so the split stays a readout-level
change needing no migration.

Backup carries it (v14). The round-trip test seeds a NOTE alongside the rule so
the target database has a note id to collide with — without that decoy, a
restore running rule ids through the wrong map would merely drop them and the
test would pass by absence, rather than failing on the populated-and-wrong
result that is the actual hazard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 16:55:22 -04:00
bvandeusenandClaude Opus 5 e029a7db64 fix(frontend): every request carries a deadline, and expiry arrives as an error callers already handle (#3412)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Python tests (push) Successful in 1m10s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 39s
CI & Build / integration (push) Successful in 34s
CI & Build / Build & push image (push) Successful in 34s
Rule 156, across the whole client. `apiGet`, `apiPost`, `apiPut`, `apiPatch`
and `apiDelete` each called bare `fetch`, whose default is to wait as long as
the browser will — not a long timeout but the absence of one. The only
AbortController in the frontend belonged to the SSE stream and was for
cancellation. So every request in the app could hang forever, and there is no
state a surface can render for "pending forever" that is not a lie: the
spinner that never resolves looks exactly like work still in progress.

Found while building the version readout (#3329), which had to tell "the fetch
failed" apart from "still loading" and could not.

ONE REQUEST PATH. The five verbs were near-identical bodies; they now delegate
to a single `request()` that owns the deadline, so a sixth verb cannot be added
without one. 30s by default — long enough to clear a cold embedding call and a
list view under pool contention (#2384), so tripping it means something is
wrong rather than merely busy. Overridable per call via `timeoutMs`.

EXPIRY IS AN ApiError, which is the half of rule 156 that is easy to skip. A
raw `DOMException: TimeoutError` reaches `apiErrorMessage(e, fallback)` as an
object with no `body`, so all ~330 existing catch sites would have printed
their generic fallback and the timeout would have been invisible in exactly
the situation it exists to expose. Rethrown as `ApiError` with a 408 — a status
no Scribe route returns, so it unambiguously means the client gave up — every
one of those call sites now reports it correctly, untouched.

Only TimeoutError is converted. A deliberate cancellation aborts with
AbortError and passes through: a caller that cancelled its own request does not
want that surfaced as a server failure. Pinned by a test, because collapsing
the two is the obvious "simplification".

STREAMS RELOCATE THE DEADLINE RATHER THAN ESCAPING IT. A wall-clock timeout
would kill a long-lived SSE connection mid-flight, but two different waits are
involved and only one of them is the stream: the CONNECT can fail to answer and
now carries a 15s deadline, cleared the moment headers arrive; the BODY stays
unbounded on purpose, since its failure mode is going quiet, which a timeout
cannot distinguish from being idle — that is what reconnection and
Last-Event-ID are for. Reading the connect as exempt because "the stream is
long-lived" leaves an unreachable server looking like a quiet one.

BULK TRANSFERS get their own value, not the default. Backup, notes export and
admin restore walk the whole store and 30s would cut them off mid-work; they
carry 10 minutes. Bounded, not unbounded — rule 156 asks for a deadline, not a
short one, and no ceiling at all is what leaves a restore that died
server-side spinning forever.

Four source-inspection guards in the unit lane (no frontend test runner): no
bare fetch anywhere; the default is actually applied — pinning the specific
regression, since #3329's opt-in shape would pass every other check while
leaving 330 callers unbounded; expiry converts to ApiError; and cancellation
does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 14:59:41 -04:00
bvandeusenandClaude Opus 5 9bb59b73ba feat(frontend): the app says what it is running, and says so honestly when it cannot find out (#3329)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / TypeScript typecheck (push) Successful in 10s
CI & Build / integration (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 38s
#3127 checklist 12, plus rule 27 — a capability with no surface the operator
can touch is not shipped.

The step was planned on the premise that nothing read `/api/version`. Two
things did, and the state was worse than nothing:

- `App.vue` fetched it, wrote `version` into a ref initialised to the literal
  `"dev"`, and swallowed the error. An instance that could not answer rendered
  EXACTLY what a healthy local build renders. That is checklist 12's named
  failure — a blank standing in for `unknown` — in the one readout whose whole
  job is to say what is running, and it would have made #3298's debugging
  session no cheaper.
- `SettingsView.vue` fetched the same endpoint again on every mount and wrote
  the result into a local ref no template ever read. A duplicate request whose
  answer was discarded.

So this is not "add a readout"; it is "make the existing one honest, and give
it the three fields nobody could see."

The readout — Settings → Config, first section, beside the other "what is this
instance doing" facts. Three states kept apart, because collapsing any two of
them is the defect:

  not asked yet (tab unopened)   nothing
  answered                       the values, each ABSENT field as "unknown"
  the fetch itself failed        its own message, with a retry

`version` and `channel` prominent, `commit` in full with a copy button so it
can be pasted into a `:sha` lookup (rule 145 — the registry's identity and the
artifact's own must be checkable against each other), `build` kept because its
ABSENCE is the diagnostic part: no ordering key means this build is not in any
update order, which is what a local or hand-built image looks like.

Absence, not falsiness. The payload omits what it does not know rather than
sending `""` or `0` (see `build_version_payload`), so the renderer uses `??`
throughout — `build` is a number and `0` is a legitimate ordering key, which
`||` would report as unknown. `tests/test_version_readout.py` pins that
operator specifically, along with the "no plausible default" property, because
`||` is the form a person reaches for by habit.

Rule 156 — the fetch carries a deadline. This readout is consulted when an
instance is misbehaving, which is exactly when it may never answer; without one
the surface sits on "still loading" forever, which is the same blank arrived at
from the other direction. `apiGet` gains an OPT-IN `timeoutMs` rather than a
default, so no existing call site's behaviour moves. Every other call in the
client still has no deadline — reported separately, not fixed here.

No frontend test runner exists, so verification is the typecheck lane plus four
source-inspection guards in the unit lane, each pinning one property.

Also folded in: `plugin/README.md` now leads with the mint script and offers
`make` second, since `make` is not installed on every workstation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 11:23:43 -04:00
bvandeusenandClaude Opus 5 f5a3643da8 refactor(plugin): retire what the hand-bump scheme left behind — the README that taught it, the floor test, the stale rationale (#3328)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 16s
#3127 checklist 19. The step's own deletion list turned out to be largely
spent: `check_version_bump()` came out with #3327, and the machinery the step
expected to delete alongside it is load-bearing for its replacement.

`manifest_version(ref=…)`, `--base`, `--no-version` and the `origin/main`
resolve path all STAY. Derivation makes the value right; it does not make the
comparison unnecessary. `check_version_is_minted` still has to ask "did the
version move when the shipped content did?", and that is a base-branch
question no matter who chose the number. The step was planned before #3327
landed, when the assumption was that these died with the guard.

What was actually still standing, all of it teaching or asserting the retired
scheme:

- `plugin/README.md` told the reader to "set a `version` bump per release."
  A shipped file, instructing the exact act the mint replaced — this is how a
  deleted control gets re-added by someone following the docs. Now says not to
  hand-edit the field, names `make mint-plugin`, and says what a forgotten
  mint costs. (`make` is not installed on every workstation, so the direct
  script invocation is given too.)
- `test_plugin_version_bumped_with_the_hook` asserted `version >= (0,1,31)`
  as a tuple of ints. Under a minted value it passes vacuously — every date
  clears a floor of 0.1.31 — and `int("0415")` silently eats the padding the
  format exists to keep. Superseded by
  `test_the_shipped_manifest_carries_a_minted_version`, which asserts the
  canonical shape instead of an ordering the comparator does not perform.
  Removed whole (rule 22).
- The module preamble still ended on "a written rule that depends on being
  remembered is not a control; this is" — true of the bump guard, and read as
  a stronger claim than the mint can support. Replaced with what the change
  did and did not remove: choosing a number is gone, running the mint is not,
  and the difference is that forgetting is now loud rather than silent.
- An orphaned `# --- the version bump ---` section header with nothing under
  it, and a test docstring still naming `check_version_bump`.

`--no-version` keeps its one legitimate case — on `main` the version is
measured against itself — and now says so in both the usage block and its
`--help`, so it does not read as an escape hatch. `check_session_context_
reports_its_version` stays untouched: a different check with a different job,
and the only thing that makes step 6 readable from a transcript (#2220).

Version minted 2026.09.02.0415 -> 2026.09.02.0438 for the README change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
2026-09-02 00:38:50 -04:00
bvandeusenandClaude Opus 5 64cb719a12 fix(plugin): mint() rendered whatever offset it was handed, not UTC (#3327)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 30s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 1m10s
CI & Build / Build & push image (push) Successful in 15s
Run 5175 red on the Python tests lane. The failing assertion was
test_the_mint_is_UTC_not_local, and it was right: `strftime` renders the
offset the datetime carries, so mint() only produced UTC because its DEFAULT
argument happens to be datetime.now(timezone.utc). Hand it an aware datetime
in any other zone and it formats that zone's wall clock -- 22:52Z and its
+09:00 twin, the same instant, minted as 2026.09.01.2252 and 2026.09.02.0752.

The docstring already claimed "UTC, always", so this was a contract the code
did not hold rather than a test asking for something new. Two people minting
the same instant would disagree, and the string IS the artifact's identity.

Now converts explicitly. A naive datetime is read as UTC rather than as the
machine's zone: that is this function's stated contract, and guessing the
host's offset is how the same bug returns by another route.

Two things found while walking the rest of the module by hand:

- test_a_failed_diff_FAILS_rather_than_passing_quietly stubbed EVERY git call
  to fail, so it tripped the base-branch guard first and passed while proving
  nothing about the diff arm. rev-parse now succeeds and only the diff fails,
  and the assertion names the diff message instead of the substring both
  messages happen to share.
- the base-branch failure still said "version-bump check", a name that went
  away with check_version_bump.

The mint script is in the version-relevant set, so fixing it is itself a
version-relevant change and forced a fresh mint -- 2026.09.02.0415. That is
the asymmetry in #3127 section 3 working as intended rather than a quirk: a
format change that did not re-mint would leave the manifest reporting a value
the current deriver can no longer produce.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN4zBVFWhBST9YqjCfQmPb
2026-09-02 00:15:45 -04:00
bvandeusenandClaude Opus 5 f1896bfe9d feat(plugin): mint the version, and make CI the control that it moved (#3327)
CI & Build / Python tests (push) Failing after 3s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / integration (push) Successful in 32s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Build & push image (push) Skipped
Milestone 334 step 3. 0.1.48 was the last of 48 numbers a person typed by
hand; forgetting to type the 49th is #2209, #1040 and #2220, three separate
times a shipped fix reached the repo and stopped there.

WHY A SCRIPT AND NOT A BUILD STEP. plugin/ is not in the image -- installs
fetch it from this repo via marketplace.json, so the push IS the release and
there is no moment at which CI could stamp a version in. Every other artifact
in the family derives during a build (#3127 section 2). This one has no build
to derive during, so the value is minted before the commit and CI's job is to
prove it moved when it had to.

MINT TIME, a fourth clock section 2 does not name. It prescribes commit time
so two lanes building one source report one string; the plugin has one lane
and no build, so that reason does not reach it. What is given up is
reproducibility-from-history -- you cannot recompute the value, only verify it
moved. That is acceptable ONLY because #3325 read the installer's code and
found the refresh test is `P.version === H`, plain equality, with zero
ordering comparisons anywhere. Where a comparator orders, an unreproducible
version would be unverifiable too.

Two artifacts in one repo now derive from different clocks on purpose, one
directory apart. "Let's make these consistent" is the obvious tidy-up and
breaks whichever loses, so the divergence is pinned in tests rather than only
explained in a comment -- including an AST assertion that the mint script
never imports subprocess, since a mint that can read history is a commit-time
deriver wearing the wrong name.

check_version_bump becomes check_version_is_minted. It gains the shape gate
and a future-value gate, and it keeps deliberately NOT failing when the
version moved without content changing: a needless re-mint costs one cache
refresh, and failing the lane over a harmless act is how a check earns a
--no-version in somebody's muscle memory and stops running at all. The
implication that matters is one-directional.

The mint script joins the version-relevant set, which is step 2's DERIVERS
table finally being read by something. Section 3's asymmetry is why it is not
optional: change the format string, change nothing else, and a diff over the
shipped paths alone says "no content change" while the manifest keeps a value
in the old format forever. Its own introduction demonstrates this -- adding
the deriver is itself the version-relevant change that forced this mint.

fetch-depth: 0 was NOT added, against this step's own brief. The plugin job
carries a comment refusing it, backed by an observed act_runner failure (any
`with:` block made checkout fail to extract, run 3027), and the reasoning
holds: the check diffs two trees and the workflow already fetches main at
depth 1. Checklist 6 is about jobs that derive; this one checks.

Verified live before pushing: the session-context marker reports
v2026.09.01.2252 keylessly, and both failure arms were probed by hand rather
than assumed. The shape gate fires first on a reverted 0.1.48, so the stale
arm is covered by unit test rather than by that probe.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN4zBVFWhBST9YqjCfQmPb
2026-09-01 18:54:55 -04:00
bvandeusenandClaude Opus 5 ea972ac3f7 refactor(plugin): one definition of what ships, and the exclusion that makes the version check mean something (#3326)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 36s
CI & Build / integration (push) Successful in 35s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 31s
Milestone 334 step 2. The set of files that reach a plugin install lived in
two hand-kept copies -- SHIPPED in check_plugin.py and the workflow's paths:
filter -- with a comment asking a human to keep them in step. That is the
shape #3127 section 3 warns about, and both copies had drifted.

The load-bearing change is the exclusion. The version check reads "did
shipped content change against the base?", and plugin.json lives INSIDE
plugin/ -- so bumping the version is itself a change to the set, which then
reads as the change that justifies the bump. Every bump passed, no bump could
ever fail, and the check proved nothing while looking green.

manifest_differs_beyond_version compares parsed objects with `version`
dropped from both sides. One field, never the whole file: plugin.json also
carries description, mcpServers and userConfig, all of which reach an install,
and excluding the file wholesale would let a userConfig-only edit compute an
unchanged version and never refresh -- #2209 again with a narrower trigger.
Unreadable input answers "changed", because a spurious bump costs one cache
refresh while a missed one is the fix reaching the repo and stopping there.

shipped_content_changed returns None, not False, when the diff fails. #2663 is
why: a read that failed inside a broad except reported the same zero as an
empty window, and every counter read zero for weeks.

Two dead trigger paths removed, both found by writing the guard rather than by
review. fable-mcp/** outlived its directory by three months (deleted in
91bafb6, 2026-05-27) and assets/** named a path that never existed at all. A
paths: entry matching nothing never fires, so neither ever failed anything.
Their two orphaned bump scripts go with them -- a third manual-bump mechanism,
wired into no settings file.

DERIVERS is section 3's (deriver -> artifacts whose identity it decides) table.
The membership test is "can changing this file change what the artifact says
about itself?", not "is it copied in" -- a deriver is never in the COPY list.
A checker is not a deriver, which is why check_plugin.py is absent from it;
step 3's mint script adds its own row.

The workflow's paths: filter is YAML and cannot import Python, so "one
definition" is held by drift tests rather than an import. Said plainly in the
test module, because it is the honest shape rather than the ideal one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN4zBVFWhBST9YqjCfQmPb
2026-09-01 18:28:26 -04:00
bvandeusen 0d4b155699 feat(telemetry): pull-through per surface, not just per corpus (#3311)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 30s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 25s
The readout already grouped usage by source — `group_by(event, source)` —
and the loop directly below it threw the source away, collapsing every
surface into one corpus-wide ratio. So the question a threshold is
actually tuned against, "is THIS surface worth its noise", could not be
asked of any surface, while the data to answer it sat in the table.

`usage.by_source` reports notes_surfaced / notes_pulled / pull_through
per surface. The grain is the note, not the call: a pull records the
door it came through, not the surface that led there, so grouping the
pulled rows by source would answer a different question. Joining
surfaced rows to pulled rows on note_id answers this one without the
session identity #2085 declined to invent — at the cost of being an
upper bound per surface, which the docstring says where it is read.

Ambient surfaces report counts and a null ratio: nothing chose those
records, so "surfaced often, opened never" is not a judgment about them.
A surface that genuinely produced nothing reports 0.0, which must not
look like the null.

The join is guarded separately from the two reads above it. #2663 was a
novel SQL shape the database rejected inside a broad except; this is the
novel shape here, and it must not take down two readouts that work.

Tests are integration for that same reason — a mock passes on a query
Postgres refuses. They pin the distinct-first property (three surfacings
of one note are one note), the ambient null, and the LIKE escape, since
an unescaped `mcp_%` also matches `mcpXget_note` and nothing else in the
payload would show the difference.
2026-08-31 15:52:17 -04:00
bvandeusen 05da26eb24 ci(integration): the run: shell is dash, not busybox (#3237)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 27s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 1m9s
CI & Build / Build & push image (push) Successful in 19s
The runner-facts step answered rule 81's check on its first run, and the
answer is the one the check itself warned about: `/bin/sh` resolves to
`/usr/bin/dash`, because ci-python is Debian-based. The constraint the
rule exists for is unchanged — dash has no /dev/tcp, no arrays, no
`[[ ]]` — but the shell has never been busybox, and this comment was
repeating the wrong name at the one place a reader would trust it.

Rule 81's own statement still says busybox; correcting it is a rulebook
edit and goes through propose -> approve -> apply.
2026-08-31 08:24:14 -04:00
bvandeusen 70d84fbfd7 ci(integration): print the runner facts that rules 79 and 81 assert (#3237)
CI & Build / TypeScript typecheck (push) Failing after 2s
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 30s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Skipped
Three conditional rules state facts about this act_runner — services are
not reachable by hostname (79), the service container's name is derived
from the job's truncated display name (80), and `run:` steps execute
under a shell without bash features (81). None had ever been verified,
because each check reads "add a step to a live CI job and read the log"
and nobody wants to arrange a throwaway run to do it.

So the step is not throwaway. Two lines on every integration run turn the
next sweep of these rules into a log read. Rule 80 needs nothing new: the
container listing the suite step already prints for the name filter is
its evidence, and run 5055's log already answers it.

Every command is guarded with a fallback. This observes the lane; it must
not be able to break it.
2026-08-31 08:20:11 -04:00
bvandeusen 9d8104f7a5 fix(embeddings): key_share alone is FOR NO KEY UPDATE, not FOR KEY SHARE (#3262)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 10s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / integration (push) Successful in 24s
CI & Build / Build & push image (push) Successful in 24s
SQLAlchemy spells Postgres's four row locks as a read/key_share pair, so
`with_for_update(key_share=True)` renders FOR NO KEY UPDATE — an
exclusive lock that two refreshes of the same record would fight over,
and that an ordinary concurrent edit would block. The claim needs
`read=True` as well to be the FOR KEY SHARE the docstring describes.

Caught by the unit test that compiles the statement, which is the whole
reason it asserts on the rendered lock mode rather than on behaviour
that looks identical either way.
2026-08-31 08:13:29 -04:00
bvandeusen 7827b4ce63 fix(embeddings): the index refresh loses the race it used to deadlock (#3262)
CI & Build / integration (push) Successful in 38s
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Failing after 55s
CI & Build / Build & push image (push) Skipped
An embedding refresh replaces a record's vectors as delete-then-insert,
which takes the chunk rows first and the parent row second (via the
insert's foreign key). A cascading delete of the parent takes exactly
those two locks in the other order. Postgres calls the cycle a deadlock
and kills one side: sometimes the detached embedder, silently, and
sometimes the user's delete, as a 500 on an operation that should have
worked.

Both upserts now claim the parent row with FOR KEY SHARE NOWAIT before
touching any chunk row. That removes the cycle instead of narrowing it —
either the embedder is first and the delete queues behind it, or the
delete already holds the row and the embedder loses at once, which is
the side designed to lose. FOR KEY SHARE is the lock the insert would
take anyway, so an ordinary edit is unaffected.

The note twin, recorded as unverified on the issue, has the same shape
and the same fix; a trash purge is the hard delete that reaches it.

Unit tests pin the ORDER and the lock mode by compiling the statement;
the integration pair holds a real delete open in one transaction and
proves the embedder returns having written nothing, with a deadline so
a regression fails instead of hanging.
2026-08-31 08:09:09 -04:00
bvandeusenandClaude Opus 5 69ce7afc45 fix(ci): /api/version reported the channel where the build belongs (rule 149)
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / Build & push image (push) Successful in 28s
CI & Build / Python lint (push) Successful in 5s
CI & Build / integration (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m3s
CI set BUILD_VERSION to the CHANNEL — literally "dev", "main", or the tag —
so a running instance answered "which build are you?" with the name of a
branch: {"version":"main"}. The cost was concrete rather than theoretical.
During #3244's live acceptance a deploy was behaving as though it held older
code, and the one endpoint whose job is to settle that could not.

Rule 149's three values, now three fields:

  version  the NAME, YYYY.MM.DD.HHMM from COMMIT time — "is this the same
           code?", so two lanes carrying one commit report one string
  build    the ORDERING KEY, minutes since 2020-01-01 from BUILD time —
           "may this be installed over that?", and the only value anything
           may compare
  channel  its own field. Never a suffix, never a segment of the name

Plus `commit`, so the artifact's claim about itself can be checked against
the :<sha> it was published under (rule 145) — which is exactly the question
that could not be answered tonight.

THE TWO CLOCKS ARE DELIBERATE and look like an inconsistency. The name comes
from the commit so two lanes building one source agree; the key comes from
the build so it cannot go backwards when an older commit is rebuilt. A test
pins both derivations against being "tidied" into one.

ABSENT RATHER THAN EMPTY when unknown. A local build has no ordering key and
no channel; emitting "" or a placeholder would let it claim a position in an
update order it is not part of. A malformed key is dropped rather than passed
through — a reader that cannot order is correct, one that orders on garbage
is not. The key is an int, because a string ordering key is how a comparison
silently becomes lexicographic ("9" > "10").

The payload builder is extracted from the route so it can be tested as a
dict rather than through app startup and a request context.

Tests pin the SHAPE the lanes emit, not the values, including the midnight
leading-zero case rule 149 names specifically — and assert CI never stamps a
branch name as the version again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 00:45:22 -04:00
116 changed files with 11955 additions and 1951 deletions
+61 -9
View File
@@ -46,8 +46,6 @@ on:
- "alembic/**"
- "alembic.ini"
- "Dockerfile"
- "assets/**"
- "fable-mcp/**"
# The plugin ships straight from this repo — installs fetch it via
# .claude-plugin/marketplace.json, NOT from the image. So a push here is
# the release, with no build step in between. Omitting these paths meant
@@ -279,6 +277,21 @@ jobs:
env:
UV_PROJECT_ENVIRONMENT: /opt/venv
run: uv sync --locked --extra dev
# Standing answers to the checks carried by rules 81 and 79 — two facts
# about THIS runner that conditional rules assert as fact, and that
# otherwise need a throwaway job to confirm (#3237). Printing them on
# every integration run makes the next rulebook sweep a log read.
# Rule 80's evidence is the container listing the next step already
# prints. Every command is guarded: a diagnostic that can break the lane
# it observes is worse than no diagnostic.
- name: Runner facts (rules 79 and 81)
run: |
echo "--- rule 81: which shell runs a run: step ---"
readlink -f /bin/sh || echo "/bin/sh: not a symlink"
ps -p $$ -o comm= || true
echo "--- rule 79: is a service reachable by its hostname yet? ---"
getent hosts postgres \
|| echo "no — 'postgres' does not resolve; the bridge-IP lookup is still required"
- name: Integration suite (resolve service IP, migrate, test)
run: |
set -eux
@@ -289,8 +302,9 @@ jobs:
PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
test -n "$PG_IP"
export DATABASE_URL="postgresql+asyncpg://scribe:ci_integration@${PG_IP}:5432/scribe_test"
# Wait for Postgres to accept connections (busybox sh — the runner
# default — has no bash /dev/tcp, so use Python).
# Wait for Postgres to accept connections. The run: shell is dash
# (/bin/sh -> /usr/bin/dash on this Debian-based image, confirmed by
# the step above) — no bash /dev/tcp, so use Python.
/opt/venv/bin/python - "$PG_IP" <<'PY'
import socket, sys, time
for _ in range(30):
@@ -327,6 +341,14 @@ jobs:
packages: write
steps:
- uses: actions/checkout@v6
with:
# Rule 149 asks for this on any job deriving the version NAME. The
# name here comes from HEAD's commit TIME, which a depth-1 clone
# already has — but the rule states it unconditionally because the
# failure it guards is silent (a too-low value, every lane green),
# and a later change to how the name is derived would inherit the
# landmine rather than the guard.
fetch-depth: 0
- name: Generate image tags and version
id: tags
@@ -339,7 +361,27 @@ jobs:
# the runner log on commit 2a374d9.
run: |
TAGS="${{ env.IMAGE }}:${{ github.sha }}"
BUILD_VERSION="dev"
# THREE VALUES, NEVER FOLDED TOGETHER (rule 149). Until 2026-08-31
# BUILD_VERSION was the CHANNEL — "dev" / "main" / the tag — so the
# image self-reported {"version":"main"}, a channel name where a
# build identifier belongs. That cost a debugging session: with the
# deploy misbehaving, nothing on the running instance could say
# which commit was serving it.
# 1. ORDERING KEY — BUILD time, monotonic by construction. Minutes
# since 2020-01-01. Never a commit count (not monotonic across
# branches) and never commit time (goes DOWN when an older
# commit is rebuilt).
BUILD_KEY=$(( ( $(date -u +%s) - 1577836800 ) / 60 ))
# 2. NAME — COMMIT time, so the same source reports the same string
# on every lane and the channel is the only thing that differs.
COMMIT_TS=$(git log --format=%ct -1 HEAD)
BUILD_NAME=$(date -u -d "@$COMMIT_TS" +%Y.%m.%d.%H%M)
# 3. CHANNEL — its own value. Never a suffix, never a segment.
CHANNEL="dev"
case "${{ github.ref }}" in
refs/heads/dev)
TAGS="$TAGS,${{ env.IMAGE }}:dev"
@@ -348,15 +390,17 @@ jobs:
# main IS the production line: publish :latest (plus the :<sha>
# set above). No separate :main tag.
TAGS="$TAGS,${{ env.IMAGE }}:latest"
BUILD_VERSION="main"
CHANNEL="stable"
;;
refs/tags/*)
TAGS="$TAGS,${{ env.IMAGE }}:latest,${{ env.IMAGE }}:${{ github.ref_name }}"
BUILD_VERSION="${{ github.ref_name }}"
CHANNEL="stable"
;;
esac
echo "value=$TAGS" >> $GITHUB_OUTPUT
echo "build_version=$BUILD_VERSION" >> $GITHUB_OUTPUT
echo "build_name=$BUILD_NAME" >> $GITHUB_OUTPUT
echo "build_key=$BUILD_KEY" >> $GITHUB_OUTPUT
echo "channel=$CHANNEL" >> $GITHUB_OUTPUT
- name: Free disk space
# Self-hosted runner housekeeping. Two-step cleanup:
@@ -386,7 +430,15 @@ jobs:
push: true
provenance: false
tags: ${{ steps.tags.outputs.value }}
build-args: BUILD_VERSION=${{ steps.tags.outputs.build_version }}
# All three, plus the commit — rule 145: the registry's identity for
# a build (:<sha>) and the artifact's identity for itself must
# agree, and they can only be checked against each other if the
# artifact says which commit it is.
build-args: |
BUILD_VERSION=${{ steps.tags.outputs.build_name }}
BUILD_KEY=${{ steps.tags.outputs.build_key }}
BUILD_CHANNEL=${{ steps.tags.outputs.channel }}
BUILD_COMMIT=${{ github.sha }}
# Registry-backed layer cache. Pull from :cache to prime
# BuildKit, push updated layers back to :cache so the next
# build starts warm even if the runner's local cache was
+21 -2
View File
@@ -41,10 +41,29 @@ COPY alembic/ alembic/
# Ensure Python finds the source tree (where static files live) before site-packages
ENV PYTHONPATH=/app/src
# Version is injected at build time via --build-arg BUILD_VERSION=YY.MM.DD.N
# Falls back to "dev" for local / untagged builds
# THREE VALUES, NEVER FOLDED TOGETHER (rule 149), plus the commit.
#
# BUILD_VERSION is the NAME (YYYY.MM.DD.HHMM, from COMMIT time) — the same
# string on every lane for the same source, so it answers "is this the same
# code?" rather than "which lane built it?".
# BUILD_KEY is the ORDERING KEY (minutes since 2020-01-01, from BUILD time) —
# the only value anything may compare to decide what is newer.
# BUILD_CHANNEL is its own field. Never a suffix, never a segment of the name.
# BUILD_COMMIT lets the artifact's self-report be checked against the :<sha>
# it was published under (rule 145).
#
# Each defaults to empty rather than to a placeholder, EXCEPT the name: a
# local build genuinely has no ordering key or channel, and the endpoint says
# so by omitting them. Inventing values would make a local image claim a
# position in an update order it is not part of.
ARG BUILD_VERSION=dev
ARG BUILD_KEY=
ARG BUILD_CHANNEL=
ARG BUILD_COMMIT=
ENV APP_VERSION=$BUILD_VERSION
ENV APP_BUILD_KEY=$BUILD_KEY
ENV APP_CHANNEL=$BUILD_CHANNEL
ENV APP_COMMIT=$BUILD_COMMIT
EXPOSE 5000
CMD ["sh", "-c", "alembic upgrade head && hypercorn 'scribe.app:create_app()' --bind 0.0.0.0:5000 --keep-alive 600"]
+10 -1
View File
@@ -1,4 +1,4 @@
.PHONY: build up down logs health migrate lint typecheck test fmt
.PHONY: build up down logs health migrate lint typecheck test fmt mint-plugin
# --- Docker ---
@@ -36,3 +36,12 @@ test:
# Run all checks in one shot (mirrors what CI does)
check: lint typecheck test
# --- Plugin ---
# Run this after changing anything under plugin/ or .claude-plugin/, BEFORE
# committing. The plugin ships straight from git with no build step, so its
# version is minted here rather than stamped by CI; the lane fails if you
# forget, but this is what makes remembering cheap.
mint-plugin:
python3 scripts/mint_plugin_version.py
@@ -0,0 +1,86 @@
"""add rule_usage_events — was a surfaced rule ever read? (milestone 333 step 1)
Revision ID: 0094
Revises: 0093
Create Date: 2026-09-02
The sibling `note_usage_events` has had since 0071, and the third rule-side
table to arrive after `rule_embeddings` and `rule_versions` — each one added
because the rule side kept inheriting machinery built for notes and getting
the weaker version of it.
WHAT IT MEASURES. The write-path standing-rule arm is the only retrieval
surface in Scribe whose usefulness cannot be observed, and — not coincidentally
— the only one that has never declined to fire. Over 30 days it took 296 calls,
returned something on every one, and cleared its threshold 100% of the time,
while every other surface declines most of the time (#3311). That is either a
perfectly tuned surface or a bar it cannot fail to clear, and `retrieval_logs`
cannot tell them apart: it records what the ranker scored, never whether the
hint was any use.
WHY NOT A rule_id COLUMN ON note_usage_events. The row shares no note-specific
fields and the aggregate readout is the same shape, which is the strongest case
for sharing that note #3163 admits. What decides against it is identity at
RESTORE: `note_usage_events`'s importer maps `note_id` through `note_id_map`
and drops what does not resolve. A rule id parked in that column would come
back from a backup silently reattached to whatever note took that number —
telemetry not merely lost but wrong, and wrong in a way nothing downstream
could detect. `rule_versions` made the same call for the same reason.
FK-free on `rule_id` and `user_id`, matching note_usage_events, retrieval_logs
and app_logs — and deliberately unlike `rule_versions`, which does carry FKs.
The difference is what the row is for: a version belongs to a rule's history
and dies with it; telemetry outlives the row it describes. Deleting a rule must
not erase the evidence that it was surfaced forty times and opened never, since
that evidence is exactly the case for having deleted it.
No CHECK on `event`, matching the note twin. Rule 36 governs adding a value to
a column that is already gated; it does not require gating one that never was,
and a two-member enum whose members are written by two functions in one module
is not where that discipline earns its cost.
Downgrade drops the table outright. The data is purely observational — nothing
reads it for correctness, so losing it costs history and no behaviour.
"""
from alembic import op
import sqlalchemy as sa
revision = "0094"
down_revision = "0093"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"rule_usage_events",
# BigInteger throughout where the note twin uses Integer: rules.id is
# BigInteger, so rule_id must be, and a high-churn append-only table is
# a poor place to discover an id ceiling.
sa.Column("id", sa.BigInteger(), primary_key=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("now()"),
),
sa.Column("user_id", sa.BigInteger(), nullable=True),
sa.Column("rule_id", sa.BigInteger(), nullable=False),
sa.Column("event", sa.Text(), nullable=False),
sa.Column("source", sa.Text(), nullable=False),
)
# Every readout is "these rule ids, split by event", so the composite is the
# one that actually gets used; the others serve pruning and per-user views.
op.create_index(
"ix_rule_usage_rule_event", "rule_usage_events", ["rule_id", "event"]
)
op.create_index("ix_rule_usage_created_at", "rule_usage_events", ["created_at"])
op.create_index("ix_rule_usage_user_id", "rule_usage_events", ["user_id"])
def downgrade() -> None:
op.drop_index("ix_rule_usage_user_id", table_name="rule_usage_events")
op.drop_index("ix_rule_usage_created_at", table_name="rule_usage_events")
op.drop_index("ix_rule_usage_rule_event", table_name="rule_usage_events")
op.drop_table("rule_usage_events")
@@ -0,0 +1,52 @@
"""add retrieval_logs.suppressed_count — tell a ranker decline from a repeat (#3497)
Revision ID: 0095
Revises: 0094
Create Date: 2026-09-03
`result_count == 0` has always meant "this surface said nothing", which is the
right number for "was the hint any use" and the wrong one for tuning a
threshold. It folds together two unrelated events:
- the ranker found nothing above the bar — the ONLY evidence a threshold is
set too high; and
- the ranker found something the session had already been shown — a decline
that says nothing whatever about the bar.
The rule arms filter in Python after the search, so they can count the second
kind exactly. The note arms pass `exclude_ids` INTO semantic_search_notes, so
the dropped rows never come back and there is nothing to count.
NULLABLE, AND THE NULL IS THE POINT. A surface that does not measure
suppression stores NULL, not 0, and the readout renders it as "not measured"
rather than "none". Defaulting to 0 would make an unmeasured surface look like
a perfectly clean one — the exact substitution of an artifact for a
measurement that #3311 made and that #3497 exists to correct. Doing it again,
in the migration that fixes it, would be its own small joke.
No backfill for the same reason: existing rows genuinely do not know, and
saying so is the honest state. `retrieval_logs` is not restored from backup,
so no importer changes.
Downgrade drops the column. Purely observational — nothing reads it for
correctness.
"""
from alembic import op
import sqlalchemy as sa
revision = "0095"
down_revision = "0094"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"retrieval_logs",
sa.Column("suppressed_count", sa.Integer(), nullable=True),
)
def downgrade() -> None:
op.drop_column("retrieval_logs", "suppressed_count")
@@ -0,0 +1,62 @@
"""add retrieval_logs.best_available_score — the score the bar rejected (#3670)
Revision ID: 0096
Revises: 0095
Create Date: 2026-09-08
`cleared_threshold` was documented as the number to read FIRST — "a surface
that clears its bar on nearly every call is either well-tuned or too loose,
and p10 says which". It was never a measurement. The search applies the
threshold before returning, so every returned result cleared the bar by
construction and a call with no results has no `top_score` to compare:
the condition is true exactly when `result_count > 0`.
`zero_result_calls + cleared_threshold == calls` held on all nineteen
source/window readings ever taken. It was `calls - zero_result_calls`
wearing a name that promised a second opinion, and a reading procedure was
built on top of it that asked the reader to compare a number against itself.
THE MISSING NUMBER, and the reason this is a column rather than a deletion.
The question the table exists to answer is "is the bar in the right place",
and that question is only answerable from the calls that returned NOTHING:
how close did the best rejected candidate come? A bar at 0.72 turning away
a stream of 0.71s is set too high by a hair. A bar turning away 0.30s is
doing its job. Those two are indistinguishable today — both render as a
zero-result call — and no arrangement of the existing columns separates
them, because the losing score is discarded inside the search.
So the searches now rank without the bar and apply it in Python, which
costs nothing (the rows were already ordered by distance, and the qualifying
set is provably identical — above-threshold rows sort first), and the best
score seen becomes observable.
NULLABLE, AND UNBACKFILLED, for the reason 0095 spells out: a row written
before this shipped genuinely does not know what its best rejected candidate
scored, and saying so is the honest state. A 0.0 default would read as "the
corpus had nothing remotely relevant" — an artifact standing in for a
measurement, which is the whole defect this milestone corrects.
`retrieval_logs` is not restored from backup, so no importer changes.
Downgrade drops the column. Purely observational — nothing reads it for
correctness.
"""
from alembic import op
import sqlalchemy as sa
revision = "0096"
down_revision = "0095"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"retrieval_logs",
sa.Column("best_available_score", sa.Float(), nullable=True),
)
def downgrade() -> None:
op.drop_column("retrieval_logs", "best_available_score")
@@ -0,0 +1,60 @@
"""add retrieval_logs.best_available_id — WHICH record the bar refused (#3807)
Revision ID: 0097
Revises: 0096
Create Date: 2026-09-09
0096 added `best_available_score` so a threshold could be judged from what it
rejected. It records how CLOSE the bar came to firing and not WHAT it turned
away, and that turns out to be the half a decision actually needs.
Live, `pre_tool_rule` shows a bar of ~0.72 with a near-miss p90 of 0.7071 —
about 117 declines a day sitting within 0.013 of firing. Dropping the bar to
0.707 would take that arm from 22 hits a day to roughly 139, a six-fold change
on a surface that runs before every Bash call. The percentile says the mass is
there. Nothing says whether it is worth showing.
AND THE TWO OBVIOUS INSTRUMENTS DO NOT ANSWER IT. Pull-through cannot: the
injected rule line already carries title and trigger, so a session can comply
without ever calling `get_rule`, and rule pull-through therefore understates
usefulness by construction. Reading the rejected records can — and `result_ids`
holds only what was RETURNED, so on a zero-result call it is empty and the
near-missed record has no name.
So: the id, beside the score, from the SAME ranked candidate. The two must
never be able to describe different records — a score paired with its
neighbour's id would be worse than no id at all, because it invites a reader to
judge the wrong record and conclude the bar is fine.
NULLABLE AND UNBACKFILLED, for the reason 0095 and 0096 both spell out: a row
written before this genuinely does not know, and inventing a value would put an
artifact where a measurement belongs. Null here means "not measured", never
"nothing was close".
NOT A FOREIGN KEY, deliberately. `retrieval_logs` spans record types — the
rules arms store rule ids, the note arms store note ids — and `source` is what
says which table an id belongs to, exactly as `result_ids` has always worked.
A constraint would have to point at one table and would be wrong for the other.
Downgrade drops the column. Purely observational — nothing reads it for
correctness.
"""
from alembic import op
import sqlalchemy as sa
revision = "0097"
down_revision = "0096"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"retrieval_logs",
sa.Column("best_available_id", sa.Integer(), nullable=True),
)
def downgrade() -> None:
op.drop_column("retrieval_logs", "best_available_id")
+90
View File
@@ -0,0 +1,90 @@
"""rules gain a `kind` — a preference is a rule that does not bind (#3849)
Revision ID: 0098
Revises: 0097
Create Date: 2026-09-10
A rule says what must be followed. There was no way to record the other
thing the operator kept writing rules for: **how they want work done**.
Several rules in a mature rulebook are not really rules — pace this kind of
debugging, hand off an action with its reason, end a finding with an offer.
Ignoring one of those does not break anything or cross a boundary; it costs
consistency. They were written as rules because a rule was the only record
that is global, keyed to a situation, and delivered when that situation
arrives.
So: `kind`. `rule` binds. `preference` describes how this person wants it
done, and — the part that makes it its own kind rather than a softer label —
**it is expected to change as the work teaches it.** The agent updates a
preference in the ordinary course of working, where a rule waits for its
author.
WHY A COLUMN AND NOT A TABLE. Everything a preference needs already exists on
`rules` and nowhere else: `when_to_apply` as a real column, a
trigger-dominated embedding document, ownership-scoped search that is
deliberately not filtered to one project, three retrieval arms with per-arm
telemetry, typed relations, and versioning. The two differ in exactly one
dimension — force — and one dimension is a field.
The drift machinery is the decisive part. `rule_versions` already snapshots
every write, and `rule_relations.overrides` already models "this supersedes
that for its scope". A separate table would have rebuilt both, and moving
existing rows into it would have changed their ids — silently invalidating
every record in the corpus that cites a rule by number.
DEFAULTS TO `rule`, so this migration changes NOTHING about what binds. An
install upgrades and every existing row keeps the force it had. That is the
same reasoning 0088 used for `tier`, and it is the reason both are safe to
apply without reading the data first.
The CHECK is created with the column (rule 36: there is no prior constraint,
so the pair is created together — a value added to it LATER does DROP + ADD
in one migration).
`rule_versions` GETS THE COLUMN TOO, and that half is not bookkeeping.
`record_if_changed` decides whether an edit is worth a snapshot by comparing
the fields a version carries; a field a version does not carry is a field
whose change records NO HISTORY AT ALL. Without this, turning a rule into a
preference — the single most consequential edit either kind can undergo,
because it is the moment something stops binding — would leave the history
silent. Nullable and no CHECK there, matching `tier`: a version is a record
of what was, and a constraint on it would refuse to store a kind later
dropped from the live whitelist.
Downgrade drops all three. Nothing reads `kind` for correctness; a rule that
was a preference simply becomes a rule again, which is the safe direction.
"""
import sqlalchemy as sa
from alembic import op
revision = "0098"
down_revision = "0097"
branch_labels = None
depends_on = None
# Kept in one place so upgrade and the CHECK agree by construction — 0088's
# idiom, for the same reason.
_KINDS = ("rule", "preference")
def _in_list(column: str, values: tuple[str, ...]) -> str:
return f"{column} IN (" + ", ".join(f"'{v}'" for v in values) + ")"
def upgrade() -> None:
op.add_column(
"rules",
sa.Column("kind", sa.Text(), nullable=False, server_default="rule"),
)
op.create_check_constraint("ck_rules_kind", "rules", _in_list("kind", _KINDS))
# Nullable, no CHECK — see the module docstring. A version written before
# this migration genuinely does not know, and NULL there means "not
# recorded", never "was a rule".
op.add_column("rule_versions", sa.Column("kind", sa.Text(), nullable=True))
def downgrade() -> None:
op.drop_column("rule_versions", "kind")
op.drop_constraint("ck_rules_kind", "rules", type_="check")
op.drop_column("rules", "kind")
@@ -0,0 +1,124 @@
"""scrub credential-shaped spans out of retrieval_logs.query
Revision ID: 0099
Revises: 0098
Create Date: 2026-09-11
`pre_tool_rule` retrieves against the RAW COMMAND TEXT and `write_path_rule`
against the code being written, so whatever was on the command line or in the
buffer is what `record_retrieval` wrote into `retrieval_logs.query`. A command
that exported a token stored the token (#3925).
`services/retrieval_telemetry.scrub_secrets` closes that going forward — the
value never reaches the column. It cannot reach BACKWARDS, and this does: it
rewrites the rows already written.
REDACTED IN PLACE, NOT DELETED. The rest of the row — score, threshold,
result count, duration, the near-miss record id — is legitimate evidence, and
it is what a threshold is tuned from. Deleting the row would throw that away
to remove a secret that lives in one column, so the column is what gets
rewritten. Rows with no credential in them are not touched at all.
THE PATTERNS ARE INLINED RATHER THAN IMPORTED, deliberately, against the DRY
instinct. A migration is a frozen record of a change that already happened on
every install that ran it; importing the live patterns would mean this
migration quietly does something different next year than it did when it ran,
and two installs at the same revision would no longer be in the same state.
The Python twin in `services/retrieval_telemetry.py` is free to grow — this is
what ran here, once. The one thing that must not drift is coverage, and the
guard for that is `test_retrieval_query_scrubbing.py`, which tests the live
function rather than this copy.
POSIX regex, not Python's. Postgres ARE supports the non-greedy `*?` the PEM
pattern needs, and `\\s`/`\\S`, so the shapes port directly. The `'gi'` flags
are global + case-insensitive, matching `re.sub` with `(?i)`.
NO BARE `auth` IN THE ASSIGNED PATTERN. It matches `--author=`, so a commit
naming an address would have had the address redacted — evidence eaten for a
word that only looks credential-shaped. `AUTH_TOKEN` is still caught, by
`token`.
Downgrade is a no-op, and honestly so: the original text is gone and a
migration cannot invent it back. Saying that plainly is better than a
downgrade that appears to restore something and does not.
"""
from alembic import op
revision = "0099"
down_revision = "0098"
branch_labels = None
depends_on = None
# Vendor-prefixed credentials — the prefix IS the tell, so no entropy guessing.
#
# `\m` IS LOAD-BEARING AND IS NOT `\b`. It anchors the prefix to the START OF
# A WORD, which the Python twin spells `\b`. Two separate mistakes were made
# porting this and they compounded:
#
# 1. The boundary was dropped entirely, so `sk-` matched inside any word
# containing it. `<task-notification>` — a string that appears in
# thousands of these rows — became `<ta[redacted:token]>`, because
# `sk-` + `notification` is a prefix followed by twelve word characters.
# 2. Writing `\b` would not have fixed it. In Postgres ARE `\b` is a
# BACKSPACE character, not a word boundary; `\m` (start of word) and
# `\y` (either edge) are the spellings that mean what Python's `\b`
# means.
#
# Both were live for one run of this migration, on one install, and the cost
# is recorded rather than papered over: the mangled rows cannot be restored,
# because the original text is what the UPDATE overwrote. The live scrubber in
# services/retrieval_telemetry.py was never affected — its `\b` is Python's
# and behaves correctly, which is why rows written after the deploy are intact
# and only migration-rewritten ones were damaged.
_TOKEN = (
r"\m(fmcp_|flt_|ghp_|gho_|ghs_|ghu_|github_pat_|glpat-|xox[abprs]-"
r"|sk-[A-Za-z0-9]*-?|AKIA|ASIA)[A-Za-z0-9_\-]{12,}"
)
# A value handed to a secret-NAMED variable, in shell, env, YAML, JSON or a
# query string. The NAME identifies it, so the value can be anything.
_ASSIGNED = (
r"\m([A-Za-z0-9_]*(token|secret|password|passwd|api[_-]?key|access[_-]?key)"
r"[A-Za-z0-9_]*)(\s*[:=]\s*[\"']?)([^\s\"'&]{8,})"
)
_AUTH_HEADER = r"(authorization\s*:\s*(bearer|basic|token)\s+)(\S+)"
_PEM = (
r"-----BEGIN [A-Z ]*PRIVATE KEY-----(.|\n)*?-----END [A-Z ]*PRIVATE KEY-----"
)
def _lit(pattern: str) -> str:
"""A regex as a SQL string literal.
A single quote inside a single-quoted SQL literal has to be DOUBLED, and
the assigned-value pattern contains two of them (it allows an optional
quote around the value). Left unescaped they close the literal early and
the migration dies on a syntax error — which is the whole reason this
helper exists rather than the patterns being pasted in inline.
"""
return pattern.replace("'", "''")
_SCRUB_SQL = f"""
UPDATE retrieval_logs
SET query = regexp_replace(
regexp_replace(
regexp_replace(
regexp_replace(query, '{_lit(_TOKEN)}', '[redacted:token]', 'gi'),
'{_lit(_ASSIGNED)}', '\\1\\3[redacted:assigned]', 'gi'),
'{_lit(_AUTH_HEADER)}', '\\1[redacted:auth-header]', 'gi'),
'{_lit(_PEM)}', '[redacted:private-key]', 'gi')
WHERE query IS NOT NULL
AND (query ~* '{_lit(_TOKEN)}'
OR query ~* '{_lit(_ASSIGNED)}'
OR query ~* '{_lit(_AUTH_HEADER)}'
OR query ~* '{_lit(_PEM)}')
"""
def upgrade() -> None:
op.execute(_SCRUB_SQL)
def downgrade() -> None:
"""Deliberately empty — the original text no longer exists to restore."""
@@ -0,0 +1,96 @@
"""drop the always-on tier: rules.tier, rulebooks.always_on, the exclusions table
Revision ID: 0100
Revises: 0099
Create Date: 2026-09-11
Milestone 394. Every rule now reaches a session by retrieval — because
something it is about to do made the rule relevant — and the machinery that
delivered rules unconditionally goes with it.
WHAT GOES, AND WHERE IT CAME FROM
- ``rules.tier`` and its ``ck_rules_tier`` CHECK (migration 0088). Dropping
the column takes the constraint with it. Rule 36 is about ADDING a value to
a live whitelist, which needs DROP + ADD in the same migration; it does not
speak to removing the column outright, and saying so here is cheaper than
the next reader wondering whether it was forgotten.
- ``rule_versions.tier`` (migration 0098). A version records what a rule
SAID; with no tier on a rule there is nothing for a snapshot to carry.
- ``rulebooks.always_on`` (migration 0058). A rulebook reaches a project by
subscription now, and by nothing else.
- ``project_rulebook_exclusions`` (migration 0085). It recorded a project's
opt-out of an always-on rulebook. Opting out of something that no longer
binds you is not a state that can exist — declining a rulebook is
expressed by not subscribing to it.
IRREVERSIBLE, AND THE DOWNGRADE SAYS SO RATHER THAN PRETENDING
The downgrade recreates the columns and the table with their DEFAULTS. It
cannot restore WHICH rules were always-on, which rulebooks bound every project,
or which projects had opted out — that information is in what this drops.
That distinction is the one this repo keeps insisting on: a value invented to
fill a hole is not a measurement. So a downgraded database is structurally able
to run the old code and is NOT the database the old code was running against —
every rule comes back at the ``always_on`` default, which for the tier column
happens to mean "binding", the safe direction to be wrong in.
Anyone who needs the real prior state restores a backup taken before this ran.
"""
import sqlalchemy as sa
from alembic import op
revision = "0100"
down_revision = "0099"
branch_labels = None
depends_on = None
_TIERS = ("always_on", "conditional")
def _in_list(column: str, values: tuple[str, ...]) -> str:
return f"{column} IN (" + ", ".join(f"'{v}'" for v in values) + ")"
def upgrade() -> None:
op.drop_table("project_rulebook_exclusions")
op.drop_column("rulebooks", "always_on")
op.drop_column("rule_versions", "tier")
# The CHECK goes with the column it constrains; naming it here would be a
# second drop of the same object.
op.drop_column("rules", "tier")
def downgrade() -> None:
"""Structure only. See the module docstring — the values are gone."""
op.add_column(
"rules",
sa.Column("tier", sa.Text(), nullable=False, server_default="always_on"),
)
op.create_check_constraint("ck_rules_tier", "rules", _in_list("tier", _TIERS))
op.add_column("rule_versions", sa.Column("tier", sa.Text(), nullable=True))
op.add_column(
"rulebooks",
sa.Column(
"always_on", sa.Boolean(), nullable=False,
server_default=sa.text("false"),
),
)
op.create_table(
"project_rulebook_exclusions",
sa.Column(
"project_id", sa.BigInteger(),
sa.ForeignKey("projects.id", ondelete="CASCADE"),
primary_key=True, nullable=False,
),
sa.Column(
"rulebook_id", sa.BigInteger(),
sa.ForeignKey("rulebooks.id", ondelete="CASCADE"),
primary_key=True, nullable=False,
),
sa.Column(
"created_at", sa.DateTime(timezone=True),
server_default=sa.text("now()"), nullable=True,
),
)
+1 -1
View File
@@ -89,7 +89,7 @@ table here. The tools are grouped by family:
| Projects / Milestones | `enter_project`, `get_project`, `create_milestone`, … | Containers and outcomes |
| Search / Recall | `search`, `get_recent`, `list_tags`, `retrieval_telemetry` | Semantic + structured recall, and the readout its thresholds are tuned from |
| Systems | `create_system`, `list_systems`, `list_system_records` | Reusable per-project subsystems/areas |
| Rulebooks | `list_always_on_rules`, `list_rules`, `create_rule`, `create_project_rule`, `subscribe_project_to_rulebook`, … | Engineering/workflow rules |
| Rulebooks | `list_rules`, `create_rule`, `create_project_rule`, `subscribe_project_to_rulebook`, … | Engineering/workflow rules |
| Processes | `list_processes`, `get_process`, `create_process` | Saved prompts/workflows |
| Trash | `list_trash`, `restore`, `purge_trash` | Recoverable deletes |
| Admin | `get_app_logs` (write/admin key) | Diagnostics |
+2 -3
View File
@@ -77,7 +77,7 @@ endpoint at `/mcp`, not these REST routes.
|--------|------|-------------|
| GET / POST | `/api/projects` | List (owned + shared) / create |
| GET / PATCH / DELETE | `/api/projects/:id` | Read (with `milestone_summary`, `inception`) / update / delete |
| POST | `/api/projects/:id/inception` | Record what the project inherits `{choices: {exclude_always_on_rulebooks, subscribe_rulebooks, design_system_id, seed_systems}}` (owner-only; `POST /api/projects` accepts the same under `inception`) |
| POST | `/api/projects/:id/inception` | Record what the project inherits `{choices: {subscribe_rulebooks, design_system_id, seed_systems}}` (owner-only; `POST /api/projects` accepts the same under `inception`) |
| GET | `/api/projects/:id/inception/defaults` | What binds if nobody decides — the inception card's payload |
| GET | `/api/projects/:id/notes` | Notes + tasks in this project |
| GET / POST | `/api/projects/:id/milestones` | List / create milestones |
@@ -120,7 +120,6 @@ endpoint at `/mcp`, not these REST routes.
| POST | `/api/projects/:id/rules` | Create a project-scoped rule |
| POST / DELETE | `/api/projects/:id/suppressions/rules/:rid` | Suppress / unsuppress a rule |
| POST / DELETE | `/api/projects/:id/suppressions/topics/:tid` | Suppress / unsuppress a topic |
| POST / DELETE | `/api/projects/:id/exclusions/rulebooks/:rid` | Exclude / include an always-on rulebook for this project (inception) |
## Sharing
@@ -206,6 +205,6 @@ endpoint at `/mcp`, not these REST routes.
Claude clients connect to the built-in MCP server at `POST /mcp` (streamable HTTP,
Bearer auth with an `fmcp_` key), served by `src/scribe/mcp/`. It is not a REST
surface — it exposes the same data as typed tools (`create_note`, `create_task`,
`start_planning`, `search`, `enter_project`, `list_always_on_rules`, …) with
`start_planning`, `search`, `enter_project`, …) with
server-level usage guidance delivered in the MCP `instructions` block. See
[API Keys & MCP](api-keys-and-mcp.md).
+5 -3
View File
@@ -60,8 +60,10 @@ Scribe stores the operator's engineering and workflow **rules** so Claude follow
across sessions.
- **Rulebooks → topics → rules** — Rules are grouped by topic inside a rulebook.
- **Always-on rules** — A rulebook can be flagged always-on; its rules load at the
start of every session through the plugin's push channel.
- **Rules arrive by retrieval** — Nothing is preloaded. A rule reaches a
session when what the agent is about to do matches its trigger: a command,
a file being written, or the operator's own message. `when_to_apply` is
therefore the field that decides whether a rule is ever seen.
- **Per-project scope** — A project subscribes to rulebooks, and can add
project-scoped rules or suppress individual inherited rules/topics.
@@ -94,7 +96,7 @@ The whole store is reachable by Claude through a built-in **MCP endpoint at `/mc
(Bearer-auth with an API key). The **Scribe Claude Code plugin** (shipped in this
repo) wires it up:
- a `SessionStart` hook that injects the operator's always-on rules + active-project
- a `SessionStart` hook that injects active-project
context so Scribe surfaces without being asked (fail-open if Scribe is unreachable);
- universal process-skills — writing-plans, systematic-debugging, verification,
brainstorming — that route their output into Scribe;
+19 -6
View File
@@ -7,12 +7,20 @@ import { useTheme } from "@/composables/useTheme";
import { useShortcuts } from "@/composables/useShortcuts";
import { useAuthStore } from "@/stores/auth";
import { useSettingsStore } from "@/stores/settings";
import { apiGet, apiPut } from "@/api/client";
import { apiPut } from "@/api/client";
import { fetchVersion } from "@/api/version";
useTheme();
const router = useRouter();
const appVersion = ref("dev");
// THREE states, not two (#3127 checklist 12). `null` is "not answered yet" and
// renders nothing; a string renders; `appVersionFailed` renders its own thing.
// This used to default to the literal "dev" and swallow the error, which meant
// an instance that could not answer was indistinguishable from a local build
// that genuinely reports "dev" — a blank standing in for `unknown`, in the one
// readout whose whole job is to say what is running.
const appVersion = ref<string | null>(null);
const appVersionFailed = ref(false);
const authStore = useAuthStore();
const settingsStore = useSettingsStore();
const { showShortcuts, toggleShortcuts, closeShortcuts } = useShortcuts();
@@ -119,10 +127,12 @@ onMounted(async () => {
startAppServices();
}
try {
const data = await apiGet<{ version: string }>("/api/version");
appVersion.value = data.version;
appVersion.value = (await fetchVersion()).version;
} catch {
// silent — version display is non-critical
// Not silent any more: the footer says it could not find out, rather than
// showing a version it never received. The full readout (version, channel,
// commit, build) lives in Settings → Config.
appVersionFailed.value = true;
}
});
@@ -151,7 +161,10 @@ onUnmounted(() => {
<div id="main-content" class="app-content">
<router-view />
</div>
<footer class="app-footer">v{{ appVersion }}</footer>
<footer class="app-footer">
<span v-if="appVersion">v{{ appVersion }}</span>
<span v-else-if="appVersionFailed">version unknown</span>
</footer>
</div>
<!-- Keyboard shortcuts overlay -->
+123 -28
View File
@@ -52,41 +52,121 @@ export function apiErrorMessage(e: unknown, fallback: string): string {
return fallback;
}
export async function apiGet<T>(path: string): Promise<T> {
const res = await fetch(path);
return handleResponse<T>(res, path);
/**
* How long an ordinary JSON call may wait before it is declared failed.
*
* Rule 156: a wait with no deadline is a bug. `fetch`'s own default is to wait
* as long as the browser will, which is not a deadline — it is the absence of
* one, and it renders as a spinner that never resolves. There is no state a
* surface can show for "pending forever" that is not a lie.
*
* 30s is chosen to be longer than anything healthy: it has to clear a cold
* embedding call and a list view under connection-pool contention (#2384 had
* /api/projects fanning 25 concurrent sessions at a 15-connection pool), so
* tripping it means something is genuinely wrong rather than merely busy. Slow
* BY DESIGN is a different case and passes its own value — see the callers in
* SettingsView that do.
*/
const DEFAULT_TIMEOUT_MS = 30_000;
/** HTTP 408. Not a status any Scribe route returns, so it unambiguously means
* "the client gave up" rather than anything the server said. */
const CLIENT_TIMEOUT_STATUS = 408;
/**
* How long a STREAM may take to answer with its headers.
*
* Streams are the one case a wall-clock deadline would break: a long-lived SSE
* connection is *supposed* to stay open, and `AbortSignal.timeout` would kill
* it mid-flight along with the body. But that does not exempt them from rule
* 156 — it relocates the deadline. Two different waits are involved:
*
* connect — the server answering with headers. CAN fail to answer, so it
* carries this deadline, cleared the moment headers arrive.
* stream — the body, open indefinitely on purpose. Its failure mode is
* going quiet, which a timeout cannot tell from being idle; that
* is what reconnection and Last-Event-ID are for, not this.
*
* Reading the connect as exempt because "the stream is long-lived" is the easy
* mistake here, and it leaves an unreachable server looking like a quiet one.
*/
const STREAM_CONNECT_TIMEOUT_MS = 15_000;
/**
* A signal that aborts if headers do not arrive in time, plus the `settle` to
* call once they do. After `settle()` the returned signal never fires, so the
* stream body runs unbounded — which is the intent.
*/
function connectDeadline(base: AbortSignal): { signal: AbortSignal; settle: () => void } {
const gate = new AbortController();
const timer = setTimeout(
() => gate.abort(new DOMException("stream did not connect in time", "TimeoutError")),
STREAM_CONNECT_TIMEOUT_MS,
);
return {
signal: AbortSignal.any([base, gate.signal]),
settle: () => clearTimeout(timer),
};
}
export async function apiPost<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
export interface RequestOpts {
/** Override the deadline. Pass one when the call is slow BY DESIGN. */
timeoutMs?: number;
}
/**
* The one place a request is actually made — every verb below goes through
* here, so the deadline cannot be forgotten by adding a sixth.
*
* EXPIRY SURFACES AS AN `ApiError`, which is rule 156's second half: the
* failure has to arrive in the shape the caller already handles. A bare
* `DOMException: TimeoutError` would reach `apiErrorMessage(e, fallback)` as
* an object with no `body`, so every catch site in the app would report its
* generic fallback and the timeout would be invisible in the very situation it
* exists to expose. Rethrowing as `ApiError` means ~330 existing call sites
* report it correctly without being touched.
*
* Only a TIMEOUT is converted. A deliberate cancellation aborts with
* `AbortError` and is left alone — a caller that cancelled its own request
* does not want it reported as a server failure.
*/
async function request<T>(path: string, init: RequestInit, opts?: RequestOpts): Promise<T> {
const timeoutMs = opts?.timeoutMs ?? DEFAULT_TIMEOUT_MS;
let res: Response;
try {
res = await fetch(path, { ...init, signal: AbortSignal.timeout(timeoutMs) });
} catch (e) {
if (e instanceof DOMException && e.name === "TimeoutError") {
throw new ApiError(CLIENT_TIMEOUT_STATUS, {
error: `The server did not answer within ${Math.round(timeoutMs / 1000)}s.`,
});
}
throw e;
}
return handleResponse<T>(res, path);
}
export async function apiPut<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
return handleResponse<T>(res, path);
/** JSON body headers — the three write verbs sent an identical literal each. */
const JSON_HEADERS = { "Content-Type": "application/json" };
export function apiGet<T>(path: string, opts?: RequestOpts): Promise<T> {
return request<T>(path, {}, opts);
}
export async function apiPatch<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
return handleResponse<T>(res, path);
export function apiPost<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
return request<T>(path, { method: "POST", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
}
export async function apiDelete(path: string): Promise<void> {
const res = await fetch(path, { method: "DELETE" });
return handleResponse<void>(res, path);
export function apiPut<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
return request<T>(path, { method: "PUT", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
}
export function apiPatch<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
return request<T>(path, { method: "PATCH", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
}
export function apiDelete(path: string, opts?: RequestOpts): Promise<void> {
return request<void>(path, { method: "DELETE" }, opts);
}
// ---------------------------------------------------------------------------
@@ -221,7 +301,14 @@ export function apiSSEStream(
}
const done = (async () => {
const res = await fetch(path, { headers, signal: combinedSignal });
// Bounded connect, unbounded stream — see STREAM_CONNECT_TIMEOUT_MS.
const connect = connectDeadline(combinedSignal);
let res: Response;
try {
res = await fetch(path, { headers, signal: connect.signal });
} finally {
connect.settle();
}
if (!res.ok) {
let body: Record<string, unknown> = {};
try {
@@ -318,11 +405,19 @@ export async function apiStreamPost(
body: unknown,
onChunk: (data: Record<string, unknown>) => void
): Promise<void> {
const res = await fetch(path, {
// Bounded connect, unbounded stream — see STREAM_CONNECT_TIMEOUT_MS.
const connect = connectDeadline(new AbortController().signal);
let res: Response;
try {
res = await fetch(path, {
method: "POST",
headers: { "Content-Type": "application/json" },
headers: JSON_HEADERS,
body: JSON.stringify(body),
signal: connect.signal,
});
} finally {
connect.settle();
}
if (!res.ok) {
let errBody: Record<string, unknown> = {};
try {
+2 -5
View File
@@ -2,7 +2,6 @@
import { apiGet, apiPost } from "@/api/client";
export interface InceptionChoices {
exclude_always_on_rulebooks: number[];
subscribe_rulebooks: number[];
design_system_id: number | null;
seed_systems: boolean;
@@ -16,9 +15,7 @@ export interface InceptionRecord {
}
export interface InceptionDefaults {
always_on_rulebooks: { id: number; title: string }[];
other_rulebooks: { id: number; title: string }[];
excluded_always_on: { id: number; title: string }[];
rulebooks: { id: number; title: string }[];
subscribed_rulebooks: { id: number; title: string }[];
design_system_id: number | null;
design_systems: { id: number; title: string }[];
@@ -32,7 +29,7 @@ export interface InceptionDecision {
}
export const emptyChoices = (): InceptionChoices => ({
exclude_always_on_rulebooks: [], subscribe_rulebooks: [], design_system_id: null, seed_systems: false,
subscribe_rulebooks: [], design_system_id: null, seed_systems: false,
});
export const fetchInceptionDefaults = (projectId: number) =>
+10 -26
View File
@@ -1,7 +1,8 @@
import type { RecordUsage } from "@/types/usage";
import { apiGet, apiPost, apiPatch, apiDelete } from "@/api/client";
/** How a rule reaches a session (milestone 307). */
export type RuleTier = "always_on" | "conditional";
/**
* A typed edge between two rules. Each kind exists because its absence forced
@@ -24,7 +25,6 @@ export interface Rulebook {
owner_user_id: number;
title: string;
description: string;
always_on: boolean;
created_at: string | null;
updated_at: string | null;
}
@@ -47,12 +47,6 @@ export interface Rule {
statement: string;
/** WHEN this rule fires — the trigger, not the instruction. */
when_to_apply: string;
/**
* always_on preloads into every session; conditional is reachable and
* surfaced when its trigger fires. A rule with no tier set behaves as
* always_on, which is how every rule behaved before this existed.
*/
tier: RuleTier;
why: string;
how_to_apply: string;
/**
@@ -85,7 +79,6 @@ export interface RuleHeader {
title: string;
statement: string;
topic_id: number | null;
tier: RuleTier;
/** A date (YYYY-MM-DD), not a timestamp. */
updated_at: string | null;
when_to_apply?: string;
@@ -96,6 +89,13 @@ export interface RuleHeader {
* A date (YYYY-MM-DD), or the literal "never".
*/
last_verified?: string;
/**
* Surfaced-vs-opened counts from `rule_usage_events` (milestone 333).
* Zero-filled by the list route, so a rule predating the table reads as
* "never surfaced" rather than as a missing field — which for a while is
* every rule on every install.
*/
usage?: RecordUsage;
}
export interface ApplicableRules {
@@ -125,7 +125,6 @@ export interface ApplicableRules {
truncated: boolean;
subscribed_rulebooks: { id: number; title: string }[];
/** Always-on rulebooks this project opted out of at inception (milestone 297). */
excluded_always_on: { id: number; title: string }[];
}
// ── Rulebooks ───────────────────────────────────────────────────────
@@ -143,7 +142,7 @@ export async function createRulebook(data: { title: string; description?: string
return apiPost("/api/rulebooks", data);
}
export async function updateRulebook(id: number, data: Partial<{ title: string; description: string; always_on: boolean }>): Promise<Rulebook> {
export async function updateRulebook(id: number, data: Partial<{ title: string; description: string }>): Promise<Rulebook> {
return apiPatch(`/api/rulebooks/${id}`, data);
}
@@ -198,7 +197,6 @@ export interface RuleWrite {
title: string;
statement: string;
when_to_apply: string;
tier: RuleTier;
why: string;
how_to_apply: string;
order_index: number;
@@ -249,7 +247,6 @@ export interface RuleVersion {
why?: string;
how_to_apply?: string;
when_to_apply?: string;
tier?: string;
verify_with?: string;
expires_when?: string;
}
@@ -314,16 +311,6 @@ export async function unsuppressTopicForProject(projectId: number, topicId: numb
return apiDelete(`/api/projects/${projectId}/suppressions/topics/${topicId}`);
}
// ── Always-on exclusions (milestone 297) ────────────────────────────────────
export async function excludeAlwaysOnRulebook(projectId: number, rulebookId: number): Promise<void> {
await apiPost(`/api/projects/${projectId}/exclusions/rulebooks/${rulebookId}`, {});
}
export async function includeAlwaysOnRulebook(projectId: number, rulebookId: number): Promise<void> {
await apiDelete(`/api/projects/${projectId}/exclusions/rulebooks/${rulebookId}`);
}
/**
* One row of the staleness sweep. Unlike RuleHeader this carries the CHECK
@@ -334,7 +321,6 @@ export interface RuleVerificationRow {
id: number;
title: string;
statement: string;
tier: RuleTier;
topic_id: number | null;
project_id: number | null;
when_to_apply: string;
@@ -357,12 +343,10 @@ export interface RuleVerificationRow {
*/
export async function listRulesDueForVerification(opts: {
olderThanDays?: number;
tier?: RuleTier;
neverOnly?: boolean;
} = {}): Promise<{ rules: RuleVerificationRow[]; total: number }> {
const q = new URLSearchParams();
if (opts.olderThanDays) q.set("older_than_days", String(opts.olderThanDays));
if (opts.tier) q.set("tier", opts.tier);
if (opts.neverOnly) q.set("never_only", "true");
const qs = q.toString();
return apiGet(`/api/rules-due-for-verification${qs ? `?${qs}` : ""}`);
+7 -9
View File
@@ -1,3 +1,5 @@
import type { RecordUsage } from "@/types/usage";
import { apiGet, apiPost, apiPatch, apiDelete } from "@/api/client";
/** One canonical location of a reusable thing. A snippet that unifies several
@@ -50,15 +52,11 @@ export interface Snippet {
owner?: string | null;
}
/** How often a record was put in front of an agent versus actually opened.
* A high `surfaced_count` with `pull_count: 0` is dead weight — it occupies a
* slot in every future auto-inject menu while never being used. */
export interface SnippetUsage {
surfaced_count: number;
pull_count: number;
last_surfaced_at: string | null;
last_pulled_at: string | null;
}
/** Kept as a name because every consumer here says "snippet usage" — but it IS
* the shared shape, since rules answer the same question off their own table
* (milestone 333). The reasoning lives on `RecordUsage`; duplicating the four
* fields here is how the two drift. */
export type SnippetUsage = RecordUsage;
/** Result of the last drift check — does the recorded location and code still
* match source? The check runs agent-side (Scribe has no checkout); this is the
+44
View File
@@ -0,0 +1,44 @@
import { apiGet } from "./client";
/**
* What `/api/version` answers — the client's half of `build_version_payload`
* (`src/scribe/routes/api.py`), which is where the reasoning for the shape is
* written down.
*
* EVERY FIELD BUT `version` IS OPTIONAL, and an absent one means "this build
* does not know", not "empty". A local build has no ordering key and no
* channel, and the server says so by omitting the keys rather than sending
* `""` — emitting a placeholder would let it claim a position in an update
* order it is not part of.
*
* So a renderer must read ABSENCE, never falsiness. `build` is a number and
* `0` is a legitimate ordering key, so `v.build || "unknown"` would report a
* real value as unknown; `v.build ?? "unknown"` is the correct form.
*/
export interface VersionPayload {
/** The NAME — `YYYY.MM.DD.HHMM` from commit time. Answers "is this the same code?" */
version: string;
/** The ORDERING KEY — minutes since 2020-01-01, from build time. Absent on a local build. */
build?: number;
/** `dev` / `main` / a tag. Its own field, never folded into the name. */
channel?: string;
/** The commit the artifact was published under, so its claim can be checked against the registry. */
commit?: string;
}
/**
* SHORTER than the client's 30s default, deliberately.
*
* This readout answers "what is running?" during an incident, which is exactly
* when the server may be the thing that is unwell — and it is one static field
* off a route that does no work, so a healthy instance answers it immediately.
* Waiting the full default before saying so would leave a person staring at
* "still loading" for half a minute in the moment they are trying to find out
* whether the instance is alive at all. Eight seconds clears a slow-but-alive
* instance and tells them something quickly when it is not.
*/
const VERSION_TIMEOUT_MS = 8000;
export function fetchVersion(): Promise<VersionPayload> {
return apiGet<VersionPayload>("/api/version", { timeoutMs: VERSION_TIMEOUT_MS });
}
+26
View File
@@ -351,3 +351,29 @@
.required { color: var(--fs-error); }
.field-hint { margin: 0.3rem 0 0; font-size: 0.8rem; color: var(--fs-text-tertiary); }
/* --- usage badge ----------------------------------------------------------
"surfaced N×, opened M×" on a list row, for any record kind the retrieval
surfaces can choose: snippets and notes from note_usage_events, rules from
rule_usage_events. Promoted here from SnippetListView's scoped block when
the rule list needed the same chip (milestone 333 step 5) — a second scoped
copy is how the ninth duplicated CSS family starts (#3207).
Geometry and colour only. A view keeps its own spacing as a scoped
remainder, the way it does for every other recipe in this file. */
.usage-tag {
font-size: 0.7rem;
padding: 0.1rem 0.4rem;
border-radius: 4px;
white-space: nowrap;
font-variant-numeric: tabular-nums;
background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent);
color: var(--fs-text-tertiary-fg);
}
/* Dead weight is a nudge, not an error — it warns in the warning colour rather
than the danger one, because the record isn't broken, just unearned. */
.usage-tag.usage-dead {
background: color-mix(in srgb, var(--fs-warning) 18%, transparent);
color: var(--fs-warning-fg);
}
+9
View File
@@ -20,6 +20,15 @@
milestone (tier, then verification) and were byte-identical; a third would
have drifted. The pane's italic serif title is inherited by anything inside
it, so the chip resets family and style explicitly. */
/* A rule with no trigger cannot be retrieved, and since milestone 394
retrieval is the only delivery — so this marks a rule that will never
reach a session. Warning rather than error: the rule is not broken, it is
unreachable, and the fix is one field away. */
.rule-chip-inert {
color: var(--fs-warning-fg);
background: color-mix(in srgb, var(--fs-warning) 12%, var(--fs-surface-raised));
}
.rule-chip {
margin-left: 0.4rem;
font-family: var(--fs-font-body);
+6 -22
View File
@@ -28,7 +28,6 @@ const emit = defineEmits<{
}>();
const local = ref<InceptionChoices>(props.choices ? { ...props.choices } : emptyChoices());
const alwaysOn = ref<{ id: number; title: string }[]>([]);
const others = ref<{ id: number; title: string }[]>([]);
const designSystems = ref<{ id: number; title: string }[]>([]);
const systemsCount = ref(0);
@@ -47,21 +46,18 @@ async function load() {
try {
if (props.mode === "decide" && props.projectId) {
const d: InceptionDefaults = await fetchInceptionDefaults(props.projectId);
alwaysOn.value = d.always_on_rulebooks;
others.value = d.other_rulebooks;
others.value = d.rulebooks;
designSystems.value = d.design_systems;
systemsCount.value = d.systems;
// Start from what stands today so "record" without changes is a true inherit-all.
local.value = {
exclude_always_on_rulebooks: d.excluded_always_on.map((r) => r.id),
subscribe_rulebooks: d.subscribed_rulebooks.map((r) => r.id),
design_system_id: d.design_system_id,
seed_systems: false,
};
} else {
const [rulebooks, ds] = await Promise.all([listRulebooks(), fetchDesignSystems()]);
alwaysOn.value = rulebooks.filter((r) => r.always_on).map((r) => ({ id: r.id, title: r.title }));
others.value = rulebooks.filter((r) => !r.always_on).map((r) => ({ id: r.id, title: r.title }));
others.value = rulebooks.map((r) => ({ id: r.id, title: r.title }));
designSystems.value = ds.design_systems.map((d) => ({ id: d.id, title: d.title }));
}
} catch (e: unknown) {
@@ -71,13 +67,6 @@ async function load() {
}
}
function inherits(id: number): boolean {
return !local.value.exclude_always_on_rulebooks.includes(id);
}
function toggleInherit(id: number) {
const list = local.value.exclude_always_on_rulebooks;
local.value.exclude_always_on_rulebooks = list.includes(id) ? list.filter((x) => x !== id) : [...list, id];
}
function subscribed(id: number): boolean {
return local.value.subscribe_rulebooks.includes(id);
}
@@ -87,7 +76,7 @@ function toggleSubscribe(id: number) {
}
const nothingToDecide = computed(
() => !alwaysOn.value.length && !others.value.length && !designSystems.value.length,
() => !others.value.length && !designSystems.value.length,
);
async function record() {
@@ -118,16 +107,11 @@ onMounted(load);
<p v-if="loading" class="inception-muted">Loading…</p>
<p v-else-if="error" class="error-msg">{{ error }}</p>
<template v-else>
<div v-if="alwaysOn.length" class="inception-group">
<h4>Always-on rulebooks</h4>
<p class="inception-muted">Checked = inherits. Uncheck to exclude a rulebook for this project only.</p>
<label v-for="rb in alwaysOn" :key="rb.id" class="inception-choice">
<input type="checkbox" :checked="inherits(rb.id)" @change="toggleInherit(rb.id)" />
<span>{{ rb.title }}</span>
</label>
</div>
<div v-if="others.length" class="inception-group">
<h4>Subscribe to rulebooks</h4>
<p class="inception-muted">
A rulebook binds this project only if it is subscribed — nothing is inherited automatically.
</p>
<label v-for="rb in others" :key="rb.id" class="inception-choice">
<input type="checkbox" :checked="subscribed(rb.id)" @change="toggleSubscribe(rb.id)" />
<span>{{ rb.title }}</span>
+62
View File
@@ -0,0 +1,62 @@
<script setup lang="ts">
/**
* "N/M used" on a list row — surfaced vs opened, for any record kind.
*
* Extracted from SnippetListView when the rule list needed the same chip
* (milestone 333 step 5). The counts read identically for both; what differs
* is the ADVICE, which is why that is a prop. A snippet surfaced repeatedly
* and never opened should probably go; a rule in the same position may simply
* have a `when_to_apply` that fires on the wrong thing, and telling an
* operator to delete it would be the wrong nudge half the time.
*/
import type { RecordUsage } from "@/types/usage";
const props = defineProps<{
usage?: RecordUsage | null;
/** What to suggest when this record looks like dead weight. Appended to the
* tooltip; kind-specific, because the remedies are. */
deadWeightAdvice: string;
/** What the record is called in the tooltip's own sentence. */
noun?: string;
}>();
/** Offered repeatedly and never opened. Three rather than one because one or
* two surfacings is noise — the record may simply not have come up in a
* relevant context yet. */
const isDeadWeight = () =>
!!props.usage && props.usage.pull_count === 0 && props.usage.surfaced_count >= 3;
/** "" renders nothing. A record nobody has surfaced yet gets no badge at all:
* "0/0" would read as a verdict when it is an absence of evidence — and on a
* freshly-migrated install that is every row. */
const label = () => {
const u = props.usage;
if (!u || u.surfaced_count === 0) return "";
return `${u.pull_count}/${u.surfaced_count} used`;
};
const title = () => {
const u = props.usage;
if (!u) return "";
const last = u.last_pulled_at
? `Last opened ${new Date(u.last_pulled_at).toLocaleDateString()}.`
: "Never opened.";
const verdict = isDeadWeight() ? ` ${props.deadWeightAdvice}` : "";
return (
`Surfaced to an agent ${u.surfaced_count}×, opened in full ` +
`${u.pull_count}×. ${last}${verdict}`
);
};
</script>
<template>
<span
v-if="label()"
class="usage-tag"
:class="{ 'usage-dead': isDeadWeight() }"
:title="title()"
>{{ label() }}</span>
</template>
<!-- The look lives in components.css (canon). Nothing scoped here on purpose:
a view that needs different spacing keeps that as its own remainder. -->
@@ -13,7 +13,6 @@ import {
unsuppressRuleForProject,
suppressTopicForProject,
unsuppressTopicForProject,
includeAlwaysOnRulebook,
} from "@/api/rulebooks";
import type { ApplicableRules, Rulebook } from "@/api/rulebooks";
@@ -32,7 +31,7 @@ const ruleDetails = ref<Record<number, {
const showProjectRuleForm = ref(false);
const newProjectRule = ref({
title: "", statement: "", why: "", how_to_apply: "",
when_to_apply: "", tier: "always_on" as "always_on" | "conditional",
when_to_apply: "",
});
async function load() {
@@ -49,11 +48,6 @@ async function subscribe(rulebookId: number) {
await load();
}
async function includeBack(rulebookId: number) {
await includeAlwaysOnRulebook(props.projectId, rulebookId);
await load();
}
async function unsubscribe(rulebookId: number) {
if (!confirm("Unsubscribe from this rulebook for this project?")) return;
await unsubscribeProject(props.projectId, rulebookId);
@@ -134,11 +128,10 @@ async function submitProjectRule() {
why: newProjectRule.value.why.trim() || undefined,
how_to_apply: newProjectRule.value.how_to_apply.trim() || undefined,
when_to_apply: newProjectRule.value.when_to_apply.trim() || undefined,
tier: newProjectRule.value.tier,
});
newProjectRule.value = {
title: "", statement: "", why: "", how_to_apply: "",
when_to_apply: "", tier: "always_on",
when_to_apply: "",
};
showProjectRuleForm.value = false;
await load();
@@ -210,17 +203,6 @@ watch(() => props.projectId, load);
</div>
</section>
<section v-if="applicable.excluded_always_on?.length" class="excluded">
<h3>Excluded always-on rulebooks</h3>
<p class="excluded-note">Opted out at inception these do not bind this project.</p>
<div class="chips">
<span v-for="rb in applicable.excluded_always_on" :key="rb.id" class="chip chip-excluded">
<a @click="openInRulesView(rb.id)">{{ rb.title }}</a>
<button class="chip-remove" @click="includeBack(rb.id)" aria-label="Include again" title="Include again"></button>
</span>
</div>
</section>
<section class="project-rules">
<div class="section-head">
<h3>Project rules</h3>
@@ -246,22 +228,13 @@ watch(() => props.projectId, load);
></textarea>
<textarea
v-model="newProjectRule.when_to_apply"
placeholder="When to apply — the trigger, not the instruction"
placeholder="When to apply — the moment, in the words a session actually produces"
rows="2"
></textarea>
<div class="tier-row">
<label>
<input v-model="newProjectRule.tier" type="radio" value="always_on" />
Always on
</label>
<label>
<input v-model="newProjectRule.tier" type="radio" value="conditional" />
Conditional
</label>
<span class="tier-hint">
Conditional if you had to name a system, an artifact or a moment to state the trigger.
</span>
</div>
<p v-if="!newProjectRule.when_to_apply.trim()" class="trigger-hint">
Without a trigger the rule will never reach a session nothing is
preloaded, so a rule arrives only when work matches what it names.
</p>
<textarea
v-model="newProjectRule.why"
placeholder="Why (optional) — the rationale"
@@ -405,14 +378,9 @@ watch(() => props.projectId, load);
</template>
<style scoped>
.tier-row { display: flex; align-items: center; gap: 0.75rem; flex-wrap: wrap; font-size: 0.85rem; }
.tier-row label { display: inline-flex; align-items: center; gap: 0.3rem; }
.tier-row input { accent-color: var(--fs-accent); }
.tier-hint { flex: 1; min-width: 12rem; font-size: 0.75rem; color: var(--fs-text-tertiary); }
.trigger-hint { flex: 1; min-width: 12rem; font-size: 0.75rem; color: var(--fs-text-tertiary); }
.excluded-note { margin: 0 0 0.5rem; color: var(--fs-text-tertiary); font-size: 0.85rem; }
.chip-excluded { opacity: 0.8; text-decoration: line-through; }
.chip-excluded .chip-remove { text-decoration: none; }
.rules-tab { padding: 1rem; }
h3 {
font-size: 0.9em; opacity: 0.7; text-transform: uppercase; letter-spacing: 0.05em;
@@ -2,7 +2,6 @@
import { computed, ref, watch, onMounted } from "vue";
import { useRulebooksStore } from "@/stores/rulebooks";
import { useCanonicalSystemsStore } from "@/stores/canonicalSystems";
import type { RuleTier } from "@/api/rulebooks";
import RuleHistoryPanel from "@/components/rules/RuleHistoryPanel.vue";
const props = defineProps<{ ruleId: number | null; topicId: number | null }>();
@@ -13,7 +12,6 @@ const canon = useCanonicalSystemsStore();
const title = ref("");
const statement = ref("");
const whenToApply = ref("");
const tier = ref<RuleTier>("always_on");
const systemIds = ref<number[]>([]);
const why = ref("");
const howToApply = ref("");
@@ -70,7 +68,6 @@ async function load() {
title.value = r.title;
statement.value = r.statement;
whenToApply.value = r.when_to_apply || "";
tier.value = r.tier || "always_on";
systemIds.value = (r.systems ?? []).map((sys) => sys.id);
why.value = r.why || "";
howToApply.value = r.how_to_apply || "";
@@ -81,7 +78,6 @@ async function load() {
title.value = "";
statement.value = "";
whenToApply.value = "";
tier.value = "always_on";
systemIds.value = [];
why.value = "";
howToApply.value = "";
@@ -100,7 +96,6 @@ async function save() {
title: title.value,
statement: statement.value,
when_to_apply: whenToApply.value,
tier: tier.value,
// Always sent, so clearing the last area actually clears it — the server
// reads a list as "these ARE the areas now".
system_ids: systemIds.value,
@@ -147,38 +142,19 @@ watch(() => props.ruleId, load);
Statement <span class="required">*</span>
<textarea v-model="statement" rows="3" placeholder="The actionable instruction (1-2 sentences)." />
</label>
<label>
When to apply
<label :class="{ 'trigger-missing': !whenToApply.trim() }">
When to apply <span class="required">*</span>
<textarea
v-model="whenToApply"
rows="2"
placeholder="The trigger, not the instruction — “before any git push”, “when a release is being cut”."
rows="3"
placeholder="The moment, in the words a session actually produces — “about to run git push with an earlier CI run unread”, not “when pacing actions”."
/>
</label>
<fieldset class="tier">
<legend>How it reaches a session</legend>
<label class="tier-opt">
<input v-model="tier" type="radio" value="always_on" />
<span>
<strong>Always on</strong>
loaded into every session.
</span>
</label>
<label class="tier-opt">
<input v-model="tier" type="radio" value="conditional" />
<span>
<strong>Conditional</strong>
arrives when its trigger fires.
</span>
</label>
<p class="tier-test">
The test: can you name the trigger <em>without</em> naming a system, an artifact type
or a moment? If the honest answer is whenever you are working, it is always on.
Conditional costs nothing when it is irrelevant, which is what lets it be as long as
it needs to be.
<p v-if="!whenToApply.trim()" class="trigger-warning">
<strong>Without this, the rule will never reach a session.</strong>
Nothing is preloaded: a rule arrives when what someone is doing matches
its trigger, so an empty trigger leaves the rule findable by nobody.
</p>
</fieldset>
<fieldset v-if="canon.catalog.length" class="areas">
<legend>Areas this rule is about</legend>
@@ -190,14 +166,14 @@ watch(() => props.ruleId, load);
/>
<span>{{ entry.name }}</span>
</label>
<p class="tier-test">
<p class="field-note">
What lets this rule reach a project working in that area.
</p>
</fieldset>
<fieldset class="check">
<legend>Can this rule go stale?</legend>
<p class="tier-test intro">
<p class="field-note intro">
Most rules are <em>decisions</em> they have no truth value and change only when you
change them. Leave this empty for those. Fill it in when the rule asserts a
<em>fact</em> about something outside your control, because those go false quietly.
@@ -226,7 +202,7 @@ watch(() => props.ruleId, load);
<button type="button" :disabled="verifying" @click="verify(false)">No longer true</button>
</span>
</div>
<p v-if="savedCheck" class="tier-test">
<p v-if="savedCheck" class="field-note">
Record this after actually running the check, never on the strength of the rule
sounding plausible. No longer true deliberately stores nothing the rule is wrong,
not in a state worth recording, so it stays at the top of the sweep until you fix or
@@ -243,7 +219,7 @@ watch(() => props.ruleId, load);
<span v-if="rel.note" class="relation-note">{{ rel.note }}</span>
</li>
</ul>
<p class="tier-test">
<p class="field-note">
Rules that <em>fail together</em> are linked, never merged a merged rule cannot be
cited, surfaced or suppressed a clause at a time.
</p>
@@ -303,9 +279,16 @@ input, textarea {
}
fieldset { border: 1px solid var(--fs-border-color); border-radius: var(--fs-radius-md); padding: 0.75rem; margin-bottom: 1rem; }
legend { padding: 0 0.35rem; font-size: 0.8rem; color: var(--fs-text-tertiary); }
.tier-opt, .area-opt { display: flex; align-items: flex-start; gap: 0.5rem; margin-bottom: 0.4rem; font-size: 0.88rem; }
.tier-opt input, .area-opt input { width: auto; margin-top: 0.2rem; accent-color: var(--fs-accent); }
.tier-test { margin: 0.5rem 0 0; font-size: 0.78rem; color: var(--fs-text-tertiary); line-height: 1.45; }
.area-opt { display: flex; align-items: flex-start; gap: 0.5rem; margin-bottom: 0.4rem; font-size: 0.88rem; }
.area-opt input { width: auto; margin-top: 0.2rem; accent-color: var(--fs-accent); }
.trigger-missing textarea { border-color: var(--fs-warning); }
/* --fs-warning-fg, not --fs-warning: the token set draws the distinction
between the warning HUE and warning text, and this is text. */
.trigger-warning {
margin: -0.35rem 0 0.6rem; font-size: 0.78rem; line-height: 1.45;
color: var(--fs-warning-fg);
}
.field-note { margin: 0.5rem 0 0; font-size: 0.78rem; color: var(--fs-text-tertiary); line-height: 1.45; }
.relations h3 { margin: 0 0 0.5rem; font-size: 0.85rem; color: var(--fs-text-secondary); }
.relations ul { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 0.35rem; }
@@ -40,14 +40,13 @@ const loadingDetail = ref(false);
// Labels rather than column names: a reader is deciding whether to open a
// row, and "How to apply" reads where "how_to_apply" has to be decoded.
type TextField =
| "title" | "statement" | "when_to_apply" | "tier"
| "title" | "statement" | "when_to_apply"
| "why" | "how_to_apply" | "verify_with" | "expires_when";
const FIELDS: Array<[TextField, string]> = [
["title", "Title"],
["statement", "Statement"],
["when_to_apply", "When to apply"],
["tier", "Tier"],
["why", "Why"],
["how_to_apply", "How to apply"],
["verify_with", "Check"],
+20 -3
View File
@@ -1,5 +1,16 @@
<script setup lang="ts">
import type { RuleHeader } from "@/api/rulebooks";
import UsageBadge from "@/components/UsageBadge.vue";
/** The dead-weight nudge for a RULE — two remedies, not one, which is the
* whole reason this advice is per-kind. A snippet nobody opens should
* probably go. A rule nobody opens may be perfectly good and simply firing on
* the wrong thing, so "delete it" would be the wrong nudge half the time and
* the operator has to be the one who picks. */
const RULE_DEAD_WEIGHT =
"Kept arriving without being read. Either its trigger fires on the wrong " +
"work — reword “when to apply” so it says when — or it is not wanted here. " +
"Until one or the other, it takes a slot in every write it matches.";
defineProps<{ topicId: number; rules: RuleHeader[] }>();
const emit = defineEmits<{
@@ -15,9 +26,14 @@ const emit = defineEmits<{
<li v-for="r in rules" :key="r.id" @click="emit('open-rule', r.id)">
<div class="title">
{{ r.title }}
<!-- Only conditional is marked: always-on is the default and
badging every row would say nothing. -->
<span v-if="r.tier === 'conditional'" class="rule-chip" title="Arrives when its trigger fires, rather than in every session">conditional</span>
<!-- Marked only when something is WRONG: every rule arrives by
retrieval now, so "conditional" stopped distinguishing anything.
A missing trigger does it means nothing can retrieve this. -->
<span
v-if="!r.when_to_apply"
class="rule-chip rule-chip-inert"
title="No trigger, so nothing can retrieve it — this rule will never reach a session"
>never surfaces</span>
<!-- Present only on a rule carrying a check, so the chip's very
presence says "this one asserts a fact that can go false". -->
<span
@@ -28,6 +44,7 @@ const emit = defineEmits<{
? 'Asserts a fact nobody has confirmed yet'
: `Check last passed ${r.last_verified}`"
>{{ r.last_verified === "never" ? "unverified" : `checked ${r.last_verified}` }}</span>
<UsageBadge :usage="r.usage" :dead-weight-advice="RULE_DEAD_WEIGHT" />
</div>
<div class="statement">{{ r.statement }}</div>
<div v-if="r.when_to_apply || r.updated_at" class="meta">
@@ -10,19 +10,16 @@
*/
import { onMounted, ref } from "vue";
import { useRulebooksStore } from "@/stores/rulebooks";
import type { RuleTier } from "@/api/rulebooks";
const emit = defineEmits<{ "open-rule": [id: number] }>();
const store = useRulebooksStore();
const neverOnly = ref(false);
const tier = ref<RuleTier | "">("");
const busyId = ref<number | null>(null);
function reload() {
return store.fetchRulesDue({
neverOnly: neverOnly.value || undefined,
tier: tier.value || undefined,
});
}
@@ -53,14 +50,6 @@ onMounted(reload);
<input v-model="neverOnly" type="checkbox" @change="reload" />
<span>Never checked only</span>
</label>
<label class="filter">
<span>Tier</span>
<select v-model="tier" @change="reload">
<option value="">any</option>
<option value="always_on">always on</option>
<option value="conditional">conditional</option>
</select>
</label>
</div>
<p v-if="store.loading" class="state">Loading</p>
@@ -68,14 +57,18 @@ onMounted(reload);
<!-- An empty sweep is GOOD NEWS, and must not read like a broken page. -->
<p v-else-if="!store.rulesDue.length" class="state empty">
Nothing to check.
{{ neverOnly || tier ? "No rule matches these filters." : "No rule carries a check yet add one to a rule that asserts a fact." }}
{{ neverOnly ? "No rule matches these filters." : "No rule carries a check yet add one to a rule that asserts a fact." }}
</p>
<ol v-else class="rows">
<li v-for="r in store.rulesDue" :key="r.id" class="row">
<div class="row-head">
<button class="row-title" @click="emit('open-rule', r.id)">{{ r.title }}</button>
<span v-if="r.tier === 'always_on'" class="rule-chip" title="Loaded into every session — a wrong one is wrong everywhere at once">always on</span>
<span
v-if="!r.when_to_apply"
class="rule-chip rule-chip-inert"
title="No trigger, so nothing can retrieve it — this rule will never reach a session"
>never surfaces</span>
<span class="age" :class="{ unchecked: r.days_since_verified === null }">
{{ r.days_since_verified === null
? "never checked"
@@ -1,5 +1,5 @@
<script setup lang="ts">
import { ref, computed, onMounted, watch } from "vue";
import { ref, onMounted, watch } from "vue";
import { useRulebooksStore } from "@/stores/rulebooks";
import { apiGet } from "@/api/client";
import {
@@ -18,9 +18,6 @@ const store = useRulebooksStore();
const isCreating = ref(false);
const newTitle = ref("");
const currentRulebook = computed(() =>
store.rulebooks.find((rb) => rb.id === props.rulebookId),
);
interface ProjectLite { id: number; title: string }
const projects = ref<ProjectLite[]>([]);
@@ -74,14 +71,6 @@ watch(() => props.rulebookId, () => {/* re-render of isSubscribed from existing
<section class="pane">
<header>
<h2>Topics</h2>
<label v-if="currentRulebook" class="always-on-toggle" title="When on, rules from this rulebook load at session start regardless of project context">
<input
type="checkbox"
:checked="currentRulebook.always_on"
@change="store.toggleAlwaysOn(currentRulebook.id)"
/>
<span>Always on</span>
</label>
</header>
<ul>
<li
@@ -124,12 +113,6 @@ watch(() => props.rulebookId, () => {/* re-render of isSubscribed from existing
<style src="@/assets/rules-shared.css" />
<style scoped>
header { display: flex; align-items: center; justify-content: space-between; gap: 1rem; }
.always-on-toggle {
display: flex; align-items: center; gap: 0.4rem;
font-size: 0.85rem; opacity: 0.85; cursor: pointer;
user-select: none;
}
.always-on-toggle input { cursor: pointer; }
ul { list-style: none; padding: 0; margin: 1rem 0; }
li { padding: 0.5rem; cursor: pointer; border-radius: 6px; }
li.active { background: var(--fs-accent-soft); }
@@ -31,7 +31,6 @@ async function submitNew() {
@click="emit('select', rb.id)"
>
<span class="title">{{ rb.title }}</span>
<span v-if="rb.always_on" class="always-on-badge" title="Loaded at session start">always on</span>
</li>
</ul>
<!-- Not a rulebook, and deliberately below them: a cross-cutting view over
@@ -65,16 +64,6 @@ ul { list-style: none; padding: 0; margin: 1rem 0; }
li { padding: 0.5rem; cursor: pointer; border-radius: 6px; display: flex; align-items: center; gap: 0.5rem; }
li.active { background: var(--fs-accent-soft); }
li:hover { background: var(--fs-surface-hover); }
.always-on-badge {
font-size: 0.7rem;
text-transform: uppercase;
letter-spacing: 0.05em;
padding: 0.1rem 0.4rem;
border-radius: 3px;
background: var(--fs-accent);
color: var(--fs-text-on-action);
margin-left: auto;
}
.sweep-entry {
display: block; width: 100%; text-align: left;
margin-top: var(--fs-space-3);
+4 -11
View File
@@ -13,7 +13,7 @@ export const useRulebooksStore = defineStore("rulebooks", () => {
// Kept so a verify re-reads the sweep with the SAME filters the operator is
// looking at — re-fetching unfiltered would silently widen the list under
// them at the moment they acted on it.
const lastSweepOpts = ref<{ olderThanDays?: number; tier?: api.RuleTier; neverOnly?: boolean }>({});
const lastSweepOpts = ref<{ olderThanDays?: number; neverOnly?: boolean }>({});
const loading = ref(false);
async function fetchRulebooks() {
@@ -57,19 +57,13 @@ export const useRulebooksStore = defineStore("rulebooks", () => {
return rb;
}
async function updateRulebook(id: number, data: Partial<Pick<Rulebook, "title" | "description" | "always_on">>) {
async function updateRulebook(id: number, data: Partial<Pick<Rulebook, "title" | "description">>) {
const rb = await api.updateRulebook(id, data);
const idx = rulebooks.value.findIndex((r) => r.id === id);
if (idx >= 0) rulebooks.value[idx] = rb;
return rb;
}
async function toggleAlwaysOn(id: number) {
const current = rulebooks.value.find((r) => r.id === id);
if (!current) return;
return updateRulebook(id, { always_on: !current.always_on });
}
async function deleteRulebook(id: number) {
await api.deleteRulebook(id);
rulebooks.value = rulebooks.value.filter((r) => r.id !== id);
@@ -112,7 +106,6 @@ export const useRulebooksStore = defineStore("rulebooks", () => {
title: rule.title,
statement: rule.statement,
topic_id: rule.topic_id,
tier: rule.tier,
updated_at: rule.updated_at,
when_to_apply: rule.when_to_apply || undefined,
arose_from_id: rule.arose_from_id ?? undefined,
@@ -162,7 +155,7 @@ export const useRulebooksStore = defineStore("rulebooks", () => {
/** The staleness sweep: rules asserting a fact, oldest verification first. */
async function fetchRulesDue(opts: {
olderThanDays?: number; tier?: api.RuleTier; neverOnly?: boolean;
olderThanDays?: number; neverOnly?: boolean;
} = {}) {
loading.value = true;
lastSweepOpts.value = opts;
@@ -206,7 +199,7 @@ export const useRulebooksStore = defineStore("rulebooks", () => {
return {
rulebooks, topicsByRulebook, rulesByTopic, currentRule, rulesDue, lastSweepOpts, loading,
fetchRulebooks, fetchTopics, fetchRules, fetchRule,
createRulebook, updateRulebook, toggleAlwaysOn, deleteRulebook,
createRulebook, updateRulebook, deleteRulebook,
createTopic, updateTopic, deleteTopic,
createRule, updateRule, deleteRule, relateRules, unrelateRules,
fetchRulesDue, verifyRule,
+23
View File
@@ -0,0 +1,23 @@
/**
* How often a record was put in front of an agent, and how often one then
* opened it in full.
*
* One shape for every record kind the retrieval surfaces can choose. Snippets
* and notes are counted in `note_usage_events`; rules in `rule_usage_events`,
* which is a separate table because a note id and a rule id are different
* namespaces resolved through different maps at restore (milestone 333). The
* TABLES are separate for that reason; the READOUT is the same question, so
* the client type is one.
*
* A high `surfaced_count` with `pull_count: 0` is dead weight — it occupies a
* slot in every future menu while never being used. What to DO about that
* differs by kind, which is why the advice is a prop on the badge rather than
* a property of this type: a snippet nobody opens should probably be deleted,
* while a rule nobody opens may just be mis-triggered.
*/
export interface RecordUsage {
surfaced_count: number;
pull_count: number;
last_surfaced_at: string | null;
last_pulled_at: string | null;
}
-3
View File
@@ -716,9 +716,6 @@ async function confirmDelete() {
/>
<p v-else-if="project.inception" class="inception-line">
Inheritance decided {{ fmtDate(project.inception.decided_at) }} via {{ project.inception.via }}
<template v-if="project.inception.choices.exclude_always_on_rulebooks.length">
· excludes {{ project.inception.choices.exclude_always_on_rulebooks.length }} always-on rulebook(s)
</template>
<template v-if="project.inception.choices.subscribe_rulebooks.length">
· subscribes {{ project.inception.choices.subscribe_rulebooks.length }}
</template>
+229 -7
View File
@@ -9,6 +9,7 @@ import type { User } from "@/types/auth";
import PaginationBar from "@/components/PaginationBar.vue";
import TagInput from "@/components/TagInput.vue";
import { fmtDate, fmtLogStamp } from "@/utils/dateFormat";
import { fetchVersion, type VersionPayload } from "@/api/version";
const store = useSettingsStore();
const authStore = useAuthStore();
@@ -85,6 +86,16 @@ const kbWritePathEnabled = ref(true);
// code embeddings sit on a much higher similarity floor than prose, so 0.55 let
// unrelated code through (#2223). Shares top-k, not the threshold.
const kbWritePathThreshold = ref("0.68");
const kbRuleHintThreshold = ref("0.72");
// The two ACT arms no longer share a bar (#3853). A write-path query is a code
// payload; a pre-tool query is a shell command, often under a dozen words —
// less text, less signal, lower scores for the same relevance. Measured at one
// shared 0.72 the two behaved like different subsystems: the write-path arm
// spoke on 37% of its calls, the command arm on 2% of 11,768.
const kbToolRuleThreshold = ref("0.68");
// And the prompt boundary is a third query shape again — the operator's own
// prose rather than anything a tool produced (#3852).
const kbPromptRuleThreshold = ref("0.72");
// Near-duplicate report floors, one per record kind (services/dedup.py).
// Snippets are single-chunk, so their floor sits below the 0.90 write-time
// gate and catches what it lets through. Notes/tasks are scored at chunk
@@ -147,12 +158,23 @@ async function saveKbInject() {
// Same `|| default` reasoning: falling back to 0 would surface every
// snippet in the corpus on every edit, which is the failure this knob fixes.
const wpT = Math.min(1, Math.max(0, Number(kbWritePathThreshold.value) || 0.68));
// Same `|| default` reasoning again, and it bites harder here: a rule hint
// fires on every write, so a fallback of 0 would attach a standing rule to
// every edit in the session.
const rhT = Math.min(1, Math.max(0, Number(kbRuleHintThreshold.value) || 0.72));
// Same `|| default` guard, and the same reason: this arm fires before every
// Bash call, so a fallback of 0 would put a rule in front of every command.
const trT = Math.min(1, Math.max(0, Number(kbToolRuleThreshold.value) || 0.68));
const prT = Math.min(1, Math.max(0, Number(kbPromptRuleThreshold.value) || 0.72));
kbInjectThreshold.value = String(t);
kbInjectTopK.value = String(k);
kbDupThresholdSnippet.value = String(dupSnip);
kbDupThresholdNote.value = String(dupNote);
kbDupThresholdTask.value = String(dupTask);
kbWritePathThreshold.value = String(wpT);
kbRuleHintThreshold.value = String(rhT);
kbToolRuleThreshold.value = String(trT);
kbPromptRuleThreshold.value = String(prT);
savingKbInject.value = true;
kbInjectSaved.value = false;
try {
@@ -165,6 +187,15 @@ async function saveKbInject() {
// measurements that split them.
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
kb_writepath_threshold: String(wpT),
// A THIRD corpus with a third bar — see RULEHINT_DEFAULT_THRESHOLD
// in services/plugin_context.py for why rules cannot share the
// code threshold any more than code could share the prose one.
kb_rulehint_threshold: String(rhT),
// A FOURTH and FIFTH bar, and they are separate keys on purpose: the
// whole finding of #3853 is that one number cannot serve arms whose
// queries are different shapes. Moving one must not move the others.
kb_toolrule_threshold: String(trT),
kb_promptrule_threshold: String(prT),
kb_duplicate_threshold_snippet: String(dupSnip),
kb_duplicate_threshold_note: String(dupNote),
kb_duplicate_threshold_task: String(dupTask),
@@ -187,7 +218,47 @@ const changingPassword = ref(false);
const invalidatingSessions = ref(false);
const exporting = ref(false);
const restoring = ref(false);
const appVersion = ref('dev');
// Backup, export and restore walk the whole store, so they are slow BY DESIGN
// and the client's ordinary 30s default would cut them off mid-work. They are
// still bounded: rule 156 asks for a deadline, not a short one, and "no ceiling
// at all" is what leaves a restore that died server-side spinning forever.
const BULK_TRANSFER_TIMEOUT_MS = 10 * 60 * 1000;
function bulkDeadline(): AbortSignal {
return AbortSignal.timeout(BULK_TRANSFER_TIMEOUT_MS);
}
// ── What's running (#3127 checklist 12) ─────────────────────────────────
// Three states kept apart, because collapsing any two of them is the defect
// this readout exists to remove: `null` + no error = not asked yet (the Config
// tab has not been opened); a payload = answered, with each ABSENT field shown
// as "unknown"; `versionError` = the fetch itself failed, which is its own
// thing and must never render as a blank or as a plausible-looking value.
const versionInfo = ref<VersionPayload | null>(null);
const versionLoading = ref(false);
const versionError = ref("");
const commitCopied = ref(false);
async function loadVersionPanel() {
if (versionLoading.value) return;
versionLoading.value = true;
versionError.value = "";
try {
versionInfo.value = await fetchVersion();
} catch (e) {
versionInfo.value = null;
versionError.value = apiErrorMessage(e, "Could not reach the instance to ask what it is running.");
} finally {
versionLoading.value = false;
}
}
async function copyCommit() {
if (!versionInfo.value?.commit) return;
await copyToClipboard(versionInfo.value.commit);
commitCopied.value = true;
setTimeout(() => { commitCopied.value = false; }, 2000);
}
const restoreFileInput = ref<HTMLInputElement | null>(null);
// Migrate stored "admin" → "config"; unknown tabs fall back to "general"
@@ -201,6 +272,7 @@ function _loadTabContent(tab: string) {
else if (tab === "logs") loadLogsPanel();
else if (tab === "groups") loadGroupsPanel();
else if (tab === "areas") canonStore.fetchCatalog(true);
else if (tab === "config" && !versionInfo.value) loadVersionPanel();
}
if (tab === "apikeys") { fetchApiKeys(); }
}
@@ -554,10 +626,6 @@ function toggleProfileWorkDay(day: string) {
function emptyTagsFetch(): Promise<string[]> { return Promise.resolve([]) }
onMounted(async () => {
try {
const v = await apiGet<{ version: string }>('/api/version')
appVersion.value = v.version
} catch { /* non-critical */ }
await store.fetchSettings();
newEmail.value = authStore.user?.email ?? "";
@@ -573,6 +641,15 @@ onMounted(async () => {
kbInjectTopK.value = allSettings.kb_autoinject_top_k;
}
kbWritePathEnabled.value = allSettings.kb_writepath_enabled !== "false";
if (allSettings.kb_rulehint_threshold !== undefined) {
kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
}
if (allSettings.kb_toolrule_threshold !== undefined) {
kbToolRuleThreshold.value = allSettings.kb_toolrule_threshold;
}
if (allSettings.kb_promptrule_threshold !== undefined) {
kbPromptRuleThreshold.value = allSettings.kb_promptrule_threshold;
}
if (allSettings.kb_writepath_threshold !== undefined) {
kbWritePathThreshold.value = allSettings.kb_writepath_threshold;
}
@@ -727,7 +804,7 @@ async function exportData(scope: "user" | "full") {
exporting.value = true;
try {
const url = scope === "full" ? "/api/admin/backup" : "/api/admin/backup?scope=user";
const res = await fetch(url);
const res = await fetch(url, { signal: bulkDeadline() });
if (!res.ok) {
const body = await res.json().catch(() => ({ error: `Error ${res.status}` }));
throw new Error((body as Record<string, string>).error || `Error ${res.status}`);
@@ -752,7 +829,7 @@ const exportingNotes = ref(false);
async function exportNotes(format: "markdown" | "json") {
exportingNotes.value = true;
try {
const res = await fetch(`/api/export?format=${format}`);
const res = await fetch(`/api/export?format=${format}`, { signal: bulkDeadline() });
if (!res.ok) throw new Error(`Error ${res.status}`);
const blob = await res.blob();
const ext = format === "json" ? "json" : "zip";
@@ -981,6 +1058,7 @@ async function handleRestoreFile(event: Event) {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(data),
signal: bulkDeadline(),
});
if (!res.ok) {
const body = await res.json().catch(() => ({ error: `Error ${res.status}` }));
@@ -1417,6 +1495,69 @@ async function deleteUser(userId: number) {
location, not by resemblance.
</p>
</div>
<div class="field">
<label for="kb-rulehint-threshold">Standing-rule confidence threshold (01)</label>
<input
id="kb-rulehint-threshold"
v-model="kbRuleHintThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The same hint can mention a standing rule whose trigger resembles the
code being written. <em>Every</em> rule is eligible nothing is
preloaded any more, so this is the only way a rule reaches a write.
Stricter than the threshold above, because there are far fewer rules
than snippets: with a small set something always ranks first, so the
bar has to carry more of the judgement. Raise it if rules keep
arriving unread; lower it if a rule you needed never showed up.
</p>
</div>
<div class="field">
<label for="kb-toolrule-threshold">Command confidence threshold (01)</label>
<input
id="kb-toolrule-threshold"
v-model="kbToolRuleThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a rule surfacing before a <em>command</em> runs, where the
query is the command text rather than code. Lower than the one above
on purpose: a shell command is short, so it scores lower for the same
relevance at a shared bar this arm spoke on 2% of calls against the
write path's 37%. Raise it if commands attract rules that do not
apply; lower it if a <code>git push</code> arrives with nothing.
</p>
</div>
<div class="field">
<label for="kb-promptrule-threshold">Prompt confidence threshold (01)</label>
<input
id="kb-promptrule-threshold"
v-model="kbPromptRuleThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a rule or preference surfacing against <em>what you just
said</em>, before anything is done. It is the only moment that reaches
a rule about how to answer rather than how to act, so it runs once a
turn rather than once a tool call. A third query shape — your prose,
not a command or a file — which is why it carries its own number.
</p>
</div>
<!-- A design system belongs to a PROJECT, and the picker for it lives on
the project. There was a setting here that designated the system
this install's own interface was built from; it only ever described
@@ -2109,6 +2250,48 @@ async function deleteUser(userId: number) {
<!-- ── Admin ── -->
<div v-if="authStore.isAdmin" v-show="activeTab === 'config'" class="settings-grid">
<section class="settings-section full-width">
<h2>What's running</h2>
<p class="section-desc">
The build serving this page. Paste the commit into a <code>:sha</code> image
lookup to check the registry and the app agree about what was published.
</p>
<div v-if="versionLoading" class="state-msg">Reading the ledger&hellip;</div>
<div v-else-if="versionError" class="error-msg">
{{ versionError }}
<button class="btn-ghost btn-compact version-retry" @click="loadVersionPanel">Try again</button>
</div>
<dl v-else-if="versionInfo" class="version-grid">
<dt>Version</dt>
<dd class="version-value">{{ versionInfo.version }}</dd>
<dt>Channel</dt>
<dd :class="versionInfo.channel === undefined ? 'version-unknown' : 'version-value'">
{{ versionInfo.channel ?? "unknown" }}
</dd>
<dt>Commit</dt>
<dd v-if="versionInfo.commit" class="version-value version-commit">
<span class="version-sha">{{ versionInfo.commit }}</span>
<button class="btn-ghost btn-compact" @click="copyCommit">
{{ commitCopied ? "Copied" : "Copy" }}
</button>
</dd>
<dd v-else class="version-unknown">unknown</dd>
<dt>Build</dt>
<!-- The ordering key, kept because its ABSENCE is the diagnostic one:
no key means this build is not part of any update order, which is
what a local or hand-built image looks like. `??` not `||` 0 is
a legitimate key. -->
<dd :class="versionInfo.build === undefined ? 'version-unknown' : 'version-value'">
{{ versionInfo.build ?? "unknown" }}
</dd>
</dl>
<div v-else class="empty-msg">Nothing asked yet.</div>
</section>
<section class="settings-section full-width">
<h2>Application URL</h2>
<p class="section-desc">
@@ -2768,6 +2951,45 @@ async function deleteUser(userId: number) {
letter-spacing: 0.07em;
color: var(--fs-text-tertiary);
}
/* What's running — a definition list of instance facts. Spacing/geometry only;
colour and type come from the tokens. */
.version-grid {
display: grid;
grid-template-columns: max-content 1fr;
gap: 0.4rem 1rem;
margin: 0;
align-items: baseline;
}
.version-grid dt {
font-size: 0.8rem;
color: var(--fs-text-secondary);
}
.version-grid dd {
margin: 0;
font-size: 0.875rem;
font-family: var(--fs-font-mono);
color: var(--fs-text-primary);
}
/* An absent field reads as absent — never as a blank, and never styled to look
like a value it does not have (#3127 checklist 12). */
.version-grid dd.version-unknown {
font-family: inherit;
font-style: italic;
color: var(--fs-text-tertiary);
}
.version-commit {
display: flex;
align-items: center;
gap: 0.5rem;
flex-wrap: wrap;
}
.version-sha {
overflow-wrap: anywhere;
}
.version-retry {
margin-left: 0.5rem;
}
.section-desc {
margin: 0 0 1rem;
font-size: 0.875rem;
+9 -58
View File
@@ -9,6 +9,7 @@ import {
type SnippetListItem,
} from "@/api/snippets";
import { useToastStore } from "@/stores/toast";
import UsageBadge from "@/components/UsageBadge.vue";
const router = useRouter();
const toast = useToastStore();
@@ -198,23 +199,6 @@ function languageOf(tags: string[]): string {
return tags.find((t) => t && t !== "snippet") ?? "";
}
/** A snippet that has been offered repeatedly and never opened. The threshold
* is 3 rather than 1 because one or two surfacings is noise — the record may
* simply not have come up in a relevant context yet. */
function isDeadWeight(s: SnippetListItem): boolean {
const u = s.usage;
return !!u && u.pull_count === 0 && u.surfaced_count >= 3;
}
/** Short badge text, or "" to render nothing. A record nobody has surfaced yet
* gets no badge at all: "0 / 0" would read as a verdict when it's an absence
* of evidence. */
function usageBadge(s: SnippetListItem): string {
const u = s.usage;
if (!u || u.surfaced_count === 0) return "";
return `${u.pull_count}/${u.surfaced_count} used`;
}
/** Short label for the drift verdict, or "" when there's nothing to say.
* An expired verdict is reported as "unchecked" whatever it used to say —
* it was about code that is no longer in the record. */
@@ -254,22 +238,13 @@ function driftTitle(s: SnippetListItem): string {
return v.detail ? `${when}: ${what}. ${v.detail}` : `${when}: ${what}.`;
}
function usageTitle(s: SnippetListItem): string {
const u = s.usage;
if (!u) return "";
const last = u.last_pulled_at
? `Last opened ${new Date(u.last_pulled_at).toLocaleDateString()}.`
: "Never opened.";
const verdict = isDeadWeight(s)
? " Offered repeatedly without ever being opened — consider rewriting its" +
" “when to reach for it” so it says when, or deleting it. It takes a slot" +
" in every future auto-inject menu."
: "";
return (
`Surfaced to an agent ${u.surfaced_count}×, opened in full ` +
`${u.pull_count}×. ${last}${verdict}`
);
}
/** The dead-weight nudge for a SNIPPET, passed to the shared badge. Kept here
* rather than inside the component because the remedy is kind-specific — a
* rule in the same position gets different advice (milestone 333 step 5). */
const SNIPPET_DEAD_WEIGHT =
"Offered repeatedly without ever being opened — consider rewriting its " +
"“when to reach for it” so it says when, or deleting it. It takes a slot " +
"in every future auto-inject menu.";
</script>
<template>
@@ -456,14 +431,7 @@ function usageTitle(s: SnippetListItem): string {
<span v-if="driftBadge(s)" class="drift-tag" :title="driftTitle(s)">
{{ driftBadge(s) }}
</span>
<span
v-if="usageBadge(s)"
class="usage-tag"
:class="{ 'usage-dead': isDeadWeight(s) }"
:title="usageTitle(s)"
>
{{ usageBadge(s) }}
</span>
<UsageBadge :usage="s.usage" :dead-weight-advice="SNIPPET_DEAD_WEIGHT" />
<span v-if="s.shared" class="shared-tag" :title="`Shared by ${s.owner ?? 'another user'} — a suggestion, not your own record`">
by {{ s.owner ?? "another user" }}
</span>
@@ -757,23 +725,6 @@ function usageTitle(s: SnippetListItem): string {
color: var(--fs-error-fg);
}
.usage-tag {
font-size: 0.7rem;
padding: 0.1rem 0.4rem;
border-radius: 4px;
white-space: nowrap;
font-variant-numeric: tabular-nums;
background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent);
color: var(--fs-text-tertiary-fg);
}
/* Dead weight is a nudge, not an error — it warns in the warning colour rather
than the danger one, because the record isn't broken, just unearned. */
.usage-tag.usage-dead {
background: color-mix(in srgb, var(--fs-warning) 18%, transparent);
color: var(--fs-warning-fg);
}
/* Header + select-mode */
.header-actions {
display: flex;
+2 -2
View File
@@ -1,7 +1,7 @@
{
"name": "scribe",
"description": "Scribe system-of-record for Claude Code: MCP tools over your notes/tasks/projects/rules, a session-start push channel that surfaces your always-on rules + active-project context, process-skills (writing-plans, systematic-debugging, verification, brainstorming, reusing-code), and your saved Scribe Processes auto-surfaced as skills (/scribe:sync). Replaces superpowers + file-memory with one app-backed plugin.",
"version": "0.1.48",
"description": "Scribe system-of-record for Claude Code: MCP tools over your notes/tasks/projects/rules, a session-start push channel that surfaces your active-project context, process-skills (writing-plans, systematic-debugging, verification, brainstorming, reusing-code), and your saved Scribe Processes auto-surfaced as skills (/scribe:sync). Replaces superpowers + file-memory with one app-backed plugin.",
"version": "2026.09.11.2026",
"author": {
"name": "Bryan Van Deusen"
},
+8 -3
View File
@@ -5,7 +5,7 @@ instance into a first-class Claude Code extension:
- **MCP tools** over your notes, tasks, projects, milestones, systems, and
rulebook (the `scribe` server).
- **Session-start push channel** — a `SessionStart` hook injects your always-on
- **Session-start push channel** — a `SessionStart` hook injects your
rules + active-project context so Scribe surfaces *without being asked*.
- **Prior-art recall on writes** — a `PreToolUse` hook on Write/Edit checks the
file about to be written against your recorded snippets (what's kept at that
@@ -78,8 +78,13 @@ On install you'll be asked for:
## Notes
- Set a `version` bump in `.claude-plugin/plugin.json` per release so clients
pick up changes.
- **Do not hand-edit `version` in `.claude-plugin/plugin.json`.** It is minted
from the clock — run `python3 scripts/mint_plugin_version.py` (or `make
mint-plugin`, where `make` is installed) after changing anything under
`plugin/`, and commit the result. The installer decides whether to refresh the cache it
executes from by comparing that string, so content that ships without a new
version reaches the repo and stops there (#2209). CI fails the lane if you
forget.
- The session-start, auto-inject and prior-art hooks need only a **read**-scoped
key; the MCP tools need **write** scope to create/update. Every hook is a GET
for that reason — a read key cannot POST.
+9
View File
@@ -33,6 +33,15 @@
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_prior_art.sh\""
}
]
},
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_tool_rules.sh\""
}
]
}
],
"PostToolUse": [
+4 -9
View File
@@ -175,16 +175,11 @@ while IFS= read -r rel_path; do
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
fi
# The rules marker the SessionStart hook stored, handed back so the server
# can say whether those rules moved since (milestone 323). Nothing stored
# means nothing sent, which the server reads as silence rather than as a
# mismatch — an install that never reached /api/plugin/context must not
# start claiming its rules changed.
# The rules marker is gone with the resident set it aged (milestone 394).
# A session no longer holds a fixed set of rules from turn zero, so there
# is nothing that can have drifted since it loaded them — each rule is
# retrieved at the moment it applies.
etag_q=""
if [ -f "$state_dir/${safe_sid}.rules_etag" ]; then
held=$(jq -sRr '@uri' < "$state_dir/${safe_sid}.rules_etag" 2>/dev/null) || held=""
[ -n "$held" ] && etag_q="&rules_etag=${held}"
fi
if [ -n "$path_enc" ]; then
# 8s, not the pre-write hook's 5: this hook runs AFTER the tool, so it
# gates nothing the session is waiting on, and the first prior-art call
+36
View File
@@ -7,6 +7,18 @@
# only — never bodies; the agent calls get_note(id) to pull anything it judges
# relevant. Most turns inject nothing.
#
# TWO ARMS SINCE #3852, on one request. Rules and preferences are retrieved
# against the same prompt and returned in the same payload, ahead of the notes
# menu. That arm exists because the two act arms are keyed on a file write or
# a command, so a rule governing what to SAY — extract intent from loose
# phrasing, raise a conflict before acting, end a finding with an offer — had
# no moment to fire at. The operator's message is the only query that exists
# before a response is composed.
#
# The two arms are gated separately server-side: turning the notes menu off
# leaves rules arriving, because they are different claims with different
# costs of being missed.
#
# Best-effort enrichment ONLY: unlike the SessionStart channel there is no
# static floor here. If the instance is unconfigured/unreachable, or anything
# fails, the hook stays SILENT and exits 0 — it must never block a prompt.
@@ -65,16 +77,32 @@ fi
# Per-session dedup: ids already injected this session are skipped.
state_dir="${TMPDIR:-/tmp}/scribe-autoinject"
mkdir -p "$state_dir" 2>/dev/null || true
# RULES DEDUP IN A DIFFERENT DIRECTORY, and it has to be this one. The rule
# ledger is SHARED by every arm that can name a rule — the two PreToolUse
# hooks already keep it under scribe-priorart — so that one session keeps ONE
# list and a rule named here is not re-announced before the next Bash call.
# A private copy here would make each arm's "already seen" mean something
# different, which is the state #3749/#3750 exist to keep coherent. The
# directory name is the prior-art hook's history, not a scope claim.
rule_state_dir="${TMPDIR:-/tmp}/scribe-priorart"
mkdir -p "$rule_state_dir" 2>/dev/null || true
idfile=""
rulefile=""
exclude_q=""
if [ -n "$session_id" ]; then
# session_id is an opaque token from Claude Code; keep only filename-safe chars.
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
idfile="$state_dir/${safe_sid}.ids"
rulefile="$rule_state_dir/${safe_sid}.rules.ids"
if [ -f "$idfile" ]; then
seen=$(tr '\n' ',' < "$idfile" 2>/dev/null | sed 's/,$//')
[ -n "$seen" ] && exclude_q="&exclude_ids=${seen}"
fi
# AGED, not read flat: an exclusion that never expires means a rule surfaced
# once in a long session is silenced for the rest of it, even as the session
# stops holding what it was told. scribe_rules_live carries the reasoning.
rule_seen=$(scribe_rules_live "$rulefile")
[ -n "$rule_seen" ] && exclude_q="${exclude_q}&exclude_rule_ids=${rule_seen}"
fi
body=$(curl -fsS --max-time 5 \
@@ -89,6 +117,14 @@ context=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null) || exit 0
if [ -n "$idfile" ]; then
printf '%s' "$body" | jq -r '.note_ids[]? // empty' 2>/dev/null >> "$idfile" || true
fi
# Rules onto the SHARED ledger, stamped so they can age out. Only FRESH ids
# come back in rule_ids (#3752) — a rule rendered as a repeat is already on
# the ledger, and re-appending it would keep pushing its stamp forward so it
# never aged at all.
if [ -n "$rulefile" ]; then
printf '%s' "$body" | jq -r '.rule_ids[]? // empty' 2>/dev/null \
| scribe_rules_append "$rulefile"
fi
jq -n --arg c "$context" \
'{hookSpecificOutput: {hookEventName: "UserPromptSubmit", additionalContext: $c}}'
+103
View File
@@ -16,6 +16,9 @@
# scribe_reached STATE SID the server answered: the next outage speaks again
# scribe_config sets `url` + `token` from the env, returns 0
# only if BOTH are usable (#2278)
# scribe_rules_live FILE live rule ids from the exclusion ledger,
# comma-joined; entries age out (#3751)
# scribe_rules_append FILE stdin ids -> the ledger, timestamped
#
# Sourced, not executed: `. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"`.
@@ -175,3 +178,103 @@ scribe_unreached() {
scribe_reached() {
rm -f "$1/$2.unreached" 2>/dev/null || true
}
# ---------------------------------------------------------------------------
# THE RULE EXCLUSION LEDGER, and how entries in it AGE (#3751).
#
# Two hooks write this file and two read it, which is the whole reason the
# parsing lives here: `scribe_prior_art.sh` and `scribe_tool_rules.sh` share
# one ledger so a rule named by one arm is not re-offered by the other, and a
# format only one of them understood would break that on the first read.
#
# WHAT PROBLEM AGEING SOLVES. #3749 clears the ledger when an EVENT destroys
# context — a compaction or a /clear. This is the case with no event at all: a
# long session where the rule was named two hundred turns ago and has simply
# fallen out of attention. It is #3702's argument at the tier level (present in
# context and salient at the moment are different properties) applied to time
# instead of to tier.
#
# PER-ENTRY TIMESTAMPS, NOT A FILE MTIME. Clearing the whole ledger when the
# file is old is one line of shell and wrong in exactly the session that needs
# it: a single recent write keeps every stale id alive, and the ids that go
# stale first are the ones from the rules that fire most.
#
# WALL TIME, NOT A TURN COUNT, and the trade is real rather than dismissed. A
# turn count is a truer model of salience — an idle session does not forget —
# but a hook has no turn number without keeping its own counter, which is a
# second piece of session state to write, read, clear on compaction and get
# wrong. Wall time is available from `date` and costs nothing. The failure mode
# it accepts is a session left idle over lunch treating its rules as forgotten,
# which produces one extra full line per rule and no other harm.
_SCRIBE_RULE_TTL=2700
# 45 MINUTES, and the reasoning rather than the number (rule 32).
#
# There is no data on this yet, so it is a judgement made to be revised — the
# telemetry that would settle it is the one #3807 just built, and a reading of
# how often an aged-out rule gets PULLED after it returns is what should move
# this.
#
# Too short and the exclusion stops existing and the repetition it prevents
# comes back. Too long and it never fires at all in a session short enough to
# matter. 45 minutes is about one working stretch on a single task: long enough
# that a rule does not re-announce itself while you are still doing the thing
# it governs, short enough that a multi-hour session gets a genuine refresh
# rather than one 9am mention.
#
# Being wrong on the short side is now the cheaper error, which is why this
# leans short. Since #3750 an excluded rule is REFERENCED rather than withheld,
# so the ledger is no longer the only thing standing between a session and a
# rule it has forgotten — an expired entry costs one full line instead of one
# short one, and the exclusion re-arms the moment it is spent.
# Live ids from a ledger, comma-joined for `exclude_rule_ids`. Empty output for
# a missing, empty or fully-aged file — the callers already treat "" as "send
# no exclusions".
#
# THE LAST ENTRY FOR AN ID WINS, and this is what stops a rule ping-ponging.
# The file is append-only, so a rule that ages out, gets surfaced fresh and is
# appended again has TWO lines. Reading the first would leave it permanently
# expired and it would re-announce itself on every single call from then on —
# the loudest possible failure, from the mechanism meant to quieten things.
# Appends are chronological, so the last line for an id is its most recent.
#
# A BARE ID — no tab, no timestamp — IS LIVE. That is the pre-#3751 format, and
# a session in flight when this ships has a ledger full of them. Treating
# unknown as expired would make every one of those sessions re-announce every
# rule it had already been told, all at once, which is precisely the noise this
# exists to prevent. Unknown means "not measured" and never "old" — the same
# null discipline the retrieval_logs columns use. Those entries simply never
# age, which is bounded: the session ends.
scribe_rules_live() {
local f="$1" now
[ -n "$f" ] && [ -f "$f" ] || return 0
now=$(date +%s 2>/dev/null) || now=0
awk -F'\t' -v now="$now" -v ttl="$_SCRIBE_RULE_TTL" '
{
id = $1
gsub(/[^0-9]/, "", id)
if (id == "") next
if (!(id in seen)) { seen[id] = 1; seq[++n] = id }
stamp[id] = ($2 ~ /^[0-9]+$/) ? $2 : ""
}
END {
out = ""
for (i = 1; i <= n; i++) {
id = seq[i]
if (stamp[id] != "" && now > 0 && (now - stamp[id]) > ttl) continue
out = out (out == "" ? "" : ",") id
}
print out
}
' "$f" 2>/dev/null || true
}
# Append surfaced ids, stamped. Reads ids on stdin, one per line — the shape
# `jq -r '(.rule_ids // [])[]?'` already produces at both call sites.
scribe_rules_append() {
local f="$1" now
[ -n "$f" ] || return 0
now=$(date +%s 2>/dev/null) || now=0
awk -v ts="$now" 'NF { print $1 "\t" ts }' >> "$f" 2>/dev/null || true
}
+6 -4
View File
@@ -204,10 +204,11 @@ if [ -n "$session_id" ]; then
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
fi
if [ -f "$rulefile" ]; then
rule_seen=$(tr '\n' ',' < "$rulefile" 2>/dev/null | sed 's/,$//')
# Ageing, not a flat read (#3751), and the ONLY ledger here that ages: the
# note channels above are a different question with a different answer, and
# this arm's sibling hook reads the same rule file through the same helper.
rule_seen=$(scribe_rules_live "$rulefile")
[ -n "$rule_seen" ] && rule_exclude_q="&exclude_rule_ids=${rule_seen}"
fi
fi
# Not `|| exit 0`: an unreachable instance must not discard a local finding
@@ -239,7 +240,8 @@ if [ -n "$body" ]; then
printf '%s' "$body" | jq -r '(.sync_note_ids // [])[]?' 2>/dev/null >> "$syncfile" || true
fi
if [ -n "$rulefile" ]; then
printf '%s' "$body" | jq -r '(.rule_ids // [])[]?' 2>/dev/null >> "$rulefile" || true
printf '%s' "$body" | jq -r '(.rule_ids // [])[]?' 2>/dev/null \
| scribe_rules_append "$rulefile"
fi
if [ -n "$derivefile" ]; then
printf '%s' "$body" | jq -r '(.derive_keys // [])[]?' 2>/dev/null >> "$derivefile" || true
+56 -27
View File
@@ -8,7 +8,7 @@
# does not depend on the key or the network.
#
# Tier 2 (DYNAMIC, best-effort enrichment): curls the operator's Scribe instance
# for always-on rules + active-project context and appends it. Config comes from
# for active-project context and appends it. Config comes from
# the plugin's userConfig, exported to hooks as:
# CLAUDE_PLUGIN_OPTION_API_ENDPOINT base URL, no trailing slash
# CLAUDE_PLUGIN_OPTION_API_TOKEN fmcp_ API key (sensitive)
@@ -55,6 +55,53 @@ here=$(CDPATH= cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) || exit 0
event=$(cat 2>/dev/null || true)
source=$(printf '%s' "$event" | jq -r '.source // empty' 2>/dev/null) || source=""
# --- The rule ledger outlives the context it describes (#3749) ---
#
# scribe_prior_art.sh and scribe_tool_rules.sh record every rule id they have
# named in <state>/<sid>.rules.ids and hand it back as exclude_rule_ids, so a
# rule is named once per session and then goes quiet. That is right while the
# session still HOLDS what it was told, and wrong the moment it does not.
#
# A compaction summarizes the earlier injections away and does not touch the
# filesystem, so the rule ends up absent from context AND still excluded —
# unreachable for the rest of the session. The banner below tells the model to
# re-pull its ALWAYS-ON rules, but a rule an arm surfaced is conditional and is
# not in that set, so it has no other way back. The rules most likely to be in
# this state are the ones that fire most often, which is to say the ones that
# apply most.
#
# The session id survives a compaction — the etag marker further down is
# rewritten on `compact` and keyed by session_id, which is only meaningful if
# the id is stable — so the stale ledger is genuinely found again, not orphaned.
#
# CLEARED ON THE SOURCES THAT DESTROY CONTEXT, AND ONLY THOSE:
#
# compact CLEAR — summarized away; the file survived.
# clear CLEAR — context wiped.
# startup nothing to do: a new session id means a new, empty file.
# resume KEEP. The context was genuinely restored, so the ledger still
# describes what the session holds. Clearing here would re-surface
# every rule after a restore that lost nothing — the mirror error.
# fork KEEP, and the answer is the same whichever way forks are keyed: a
# fork carries the conversation, so if it inherits the id the ledger
# is accurate, and if it gets a new one the file is empty anyway.
#
# ONLY the rules ledger. The same directory holds .ids / .sync.ids /
# .derive.ids for the note arms. Whether a surfaced NOTE should return after a
# compaction is a different question with a different answer, and leaving those
# alone is a decision rather than an oversight.
case "$source" in
compact|clear)
sid=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || sid=""
if [ -n "$sid" ]; then
safe_sid=$(printf '%s' "$sid" | tr -c 'A-Za-z0-9._-' '_')
# Best-effort, like every other filesystem touch in these hooks: a ledger
# that cannot be removed costs a repeated exclusion, never a session.
rm -f "${TMPDIR:-/tmp}/scribe-priorart/${safe_sid}.rules.ids" 2>/dev/null || true
fi
;;
esac
out=""
# Append $1 to $out, separated by a horizontal rule when $out already has content.
append() { if [ -n "$out" ]; then out="${out}"$'\n\n---\n\n'"$1"; else out="$1"; fi; }
@@ -109,31 +156,13 @@ if [ -n "$url" ] && [ -n "$token" ] && command -v curl >/dev/null 2>&1; then
-H "Authorization: Bearer ${token}" \
"${url%/}/api/plugin/context${q}" 2>/dev/null) || body=""
[ -n "$body" ] && dyn=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null)
# Stash the rules marker for the write-path hook (milestone 323). THIS is
# where it has to be captured: the model receives one from
# list_always_on_rules too, but a hook cannot see an MCP tool's result. Stored
# under the same state dir the prior-art hook already uses, keyed by session,
# so "changed since" means since THIS session loaded its rules.
#
# Written on `compact` as well as `startup`, and that is correct rather than
# convenient: a compact tells the session to re-pull its rules, so the marker
# should describe the set it is about to hold. It is also why this cannot
# cover the compaction case — see the table in services/plugin_context.py.
if [ -n "$body" ]; then
etag=$(printf '%s' "$body" | jq -r '.rules_etag // empty' 2>/dev/null) || etag=""
if [ -n "$etag" ]; then
sid=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || sid=""
safe_sid=$(printf '%s' "${sid:-nosession}" | tr -c 'A-Za-z0-9._-' '_')
etag_dir="${TMPDIR:-/tmp}/scribe-priorart"
# Best-effort throughout: a marker that cannot be stored costs a hint,
# never the session.
mkdir -p "$etag_dir" 2>/dev/null \
&& printf '%s' "$etag" > "$etag_dir/${safe_sid}.rules_etag" 2>/dev/null || true
fi
fi
[ -z "$dyn" ] && status="> ⚠️ Scribe: live rules/project context could not be loaded this session (instance unreachable or request failed). The standing guidance above still applies — pull rules with \`list_always_on_rules()\` and project context with \`enter_project()\` as needed."
# The rules marker is gone with the resident set it described
# (milestone 394). Nothing is preloaded, so there is no set whose
# drift a later write could be told about — a rule is retrieved at
# the moment it applies, which cannot be stale.
[ -z "$dyn" ] && status="> ⚠️ Scribe: live project context could not be loaded this session (instance unreachable or request failed). The standing guidance above still applies — ask for rules with \`search(content_type=\"rule\")\` and project context with \`enter_project()\` as needed."
elif [ -n "$url" ] && [ -z "$token" ]; then
status="> ⚠️ Scribe: live context disabled this session — the API key is not configured (Scribe base URL is). Set it with \`/plugin\` → Scribe → configure, or export SCRIBE_TOKEN. Tools still work; pull rules with \`list_always_on_rules()\` and project context with \`enter_project()\`."
status="> ⚠️ Scribe: live context disabled this session — the API key is not configured (Scribe base URL is). Set it with \`/plugin\` → Scribe → configure, or export SCRIBE_TOKEN. Tools still work; ask for rules with \`search(content_type=\"rule\")\` and project context with \`enter_project()\`."
elif [ -z "$url" ] && [ -z "$token" ]; then
# NEITHER value arrived. Previously this case stayed silent as "an unconfigured
# install", which made issue #2198 invisible for weeks: a *casing* bug here
@@ -142,7 +171,7 @@ elif [ -z "$url" ] && [ -z "$token" ]; then
# silently disabled auto-inject and the write-path trigger too. It is not a
# benign state — the plugin prompts for both values at enable time, so if
# neither reached the hook, something is wrong. Say so.
status="> ⚠️ Scribe: live context disabled this session — neither the Scribe base URL nor the API key reached this hook. Configure the plugin (\`/plugin\` → Scribe), or export SCRIBE_URL + SCRIBE_TOKEN. Note this also disables prompt auto-inject and the write-path prior-art trigger. Tools still work; pull rules with \`list_always_on_rules()\` and project context with \`enter_project()\`."
status="> ⚠️ Scribe: live context disabled this session — neither the Scribe base URL nor the API key reached this hook. Configure the plugin (\`/plugin\` → Scribe), or export SCRIBE_URL + SCRIBE_TOKEN. Note this also disables prompt auto-inject and the write-path prior-art trigger. Tools still work; ask for rules with \`search(content_type=\"rule\")\` and project context with \`enter_project()\`."
fi
[ -n "$dyn" ] && append "$dyn"
@@ -150,7 +179,7 @@ fi
# Compaction re-grounding: lead with a reload banner when this fire is a compact.
if [ "$source" = "compact" ]; then
prepend "> ⟳ This session was just COMPACTED — earlier turns are now a summary, so in-flight detail may be lost. Before continuing, reload your bearings from Scribe: re-pull the operator's binding rules with \`list_always_on_rules()\` (a compaction can summarize them out of context, leaving only generic harness defaults in their place), re-run \`enter_project()\` for the active project, check its open tasks and recent notes, and reconcile what you're mid-way through against what Scribe records. Don't trust half-remembered state — Scribe is the record."
prepend "> ⟳ This session was just COMPACTED — earlier turns are now a summary, so in-flight detail may be lost. Any rules that had been retrieved went into that summary with everything else, so treat yourself as holding none: before the next consequential act, ask again with \`search(content_type=\"rule\")\` rather than trusting a half-remembered one. Re-run \`enter_project()\` for the active project, check its open tasks and recent notes, and reconcile what you are mid-way through against what Scribe records. Scribe is the record."
fi
# Nothing at all to inject → stay silent.
+23 -5
View File
@@ -6,7 +6,8 @@ of record (notes, tasks, projects, milestones, rules) reachable through the
for the operator's work, and as your own working memory across sessions.
**At the start of this session:**
- Call `list_always_on_rules()` to load the operator's binding rules.
- You hold none of the operator's rules, and there is no call that loads them
all. Rules arrive when something you are about to do matches one.
- If the working repo maps to a Scribe project (check `list_repo_bindings`),
call `enter_project(<id>)` to load that project's rules, open tasks, and
recent notes in one shot.
@@ -17,10 +18,27 @@ for the operator's work, and as your own working memory across sessions.
operator's Scribe rules decide what to do — NOT generic conventions baked
into the harness or your defaults (e.g. "branch before committing," "open a
feature branch per task," "push to a fork"). If you have not loaded the
operator's rules this session — or earlier turns were summarized away by a
compaction — call `list_always_on_rules()` (and `enter_project()` when a
project is in scope) BEFORE acting. When a loaded rule and a default habit
disagree, the rule wins; if no rule speaks to it, ask rather than assume.
no rule has arrived for the act in front of you, `search(content_type=
"rule")` BEFORE acting rather than falling back on a default habit. When a
retrieved rule and a default habit disagree, the rule wins; if no rule
speaks to it, ask rather than assume.
- **Rules bind; preferences do not.** A record's `kind` says which. A **rule**
must be followed — ignoring it breaks something or crosses a boundary. A
**preference** is how the operator wants work done: worth following for
consistency, not a defect to miss. Injected lines name the kind in their
opening words. A preference is also yours to keep current when they correct
you (`update_preference`); a rule waits for them.
- **Silence is not absence.** Nothing is preloaded: every rule is RETRIEVED,
when what you are doing resembles what the rule is about. Most turns
retrieve none, and a rule you were never handed binds exactly as hard as one
you were. So before a consequential act, `search` for a rule about it
(`content_type="rule"`) rather than concluding from an empty session that
nothing applies. "I was not told" is not the same as "there is no rule," and
only one of those is checkable.
This bites hardest on which TOOL to reach for — curling an API that has an
MCP client, standing up a local stack, running a suite CI owns. Those feel
like mechanics rather than decisions, so they raise no doubt and generate no
query; the moment you are most confident is the moment to look.
- **Recall before acting** — before you answer anything about the operator's
work or start a task, `search` Scribe first; assume a related note, task, or
decision already exists. Concretely, reach for recall whenever a request
+117
View File
@@ -0,0 +1,117 @@
#!/usr/bin/env bash
# Scribe — PreToolUse rule arm for ACTIONS (#3476).
#
# The sibling of scribe_prior_art.sh. That hook is registered on Write|Edit and
# asks "what is recorded about the file being written". This one asks "does a
# standing rule speak to the command about to be run" — the question nothing
# could ask before, and the reason every rule about which tool to reach for had
# to live in the preload instead, back when there was one.
#
# WHY A HOOK AND NOT AN INSTRUCTION. A reflex generates no query (note #3089):
# you reach for `curl` confidently, with no moment of doubt, so a surface that
# waits to be asked never fires. Here nothing is asked — the tool call IS the
# query, and the reflex has to become a tool call before it can do anything.
#
# SILENT ON OUTAGE, deliberately, unlike the prior-art hook. A write is
# occasional; a Bash call is not, and an "instance did not answer" line before
# every command is the noise that gets a channel muted. scribe_prior_art.sh
# still speaks for both when the instance is down.
#
# Env:
# SCRIBE_URL / SCRIBE_TOKEN override for the settings.json dogfooding path.
command -v jq >/dev/null 2>&1 || exit 0
command -v curl >/dev/null 2>&1 || exit 0
# PreToolUse delivers { session_id, cwd, tool_name, tool_input: {...}, ... }
event=$(cat 2>/dev/null || true)
tool_name=$(printf '%s' "$event" | jq -r '.tool_name // empty' 2>/dev/null) || exit 0
session_id=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || session_id=""
event_cwd=$(printf '%s' "$event" | jq -r '.cwd // empty' 2>/dev/null) || event_cwd=""
[ -n "$tool_name" ] || exit 0
# The action, as text. `.command` is Bash's field; the fallbacks let the matcher
# in hooks.json widen to other tools without this script changing — which is the
# whole reason the server side takes a name and a string rather than a schema.
command_text=$(printf '%s' "$event" | jq -r '
.tool_input.command //
.tool_input.url //
.tool_input.prompt //
empty' 2>/dev/null) || command_text=""
[ -n "$command_text" ] || exit 0
# shellcheck source=plugin/hooks/scribe_defs.sh
. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
# scribe_config, not a hand-rolled pair of parameter expansions: it also treats
# an UNEXPANDED `${...}` placeholder as unset, which would otherwise be sent as
# a garbage Bearer token and 401 on every call (#2198's class).
scribe_config || exit 0
# Bounded before encoding: a heredoc or a pasted script can be enormous, and
# the verb and its target — the part a rule is about — sit at the front. The
# server bounds it again; this keeps a huge payload off the wire in the first
# place. `head -c`, never `cut -c`: cut truncates each LINE and caps nothing.
command_text=$(printf '%s' "$command_text" | head -c 2000)
# -sRr, never -rR: jq -R without -s reads LINE BY LINE, so a multi-line command
# would encode per line and join with raw newlines — an invalid URL.
cmd_enc=$(printf '%s' "$command_text" | jq -sRr '@uri' 2>/dev/null) || exit 0
tool_enc=$(printf '%s' "$tool_name" | jq -sRr '@uri' 2>/dev/null) || exit 0
repo_q=""
lookup_dir=${event_cwd:-${CLAUDE_PROJECT_DIR:-$PWD}}
repo_remote=$(git -C "$lookup_dir" remote get-url origin 2>/dev/null || true)
if [ -n "$repo_remote" ]; then
repo_enc=$(printf '%s' "$repo_remote" | jq -sRr '@uri' 2>/dev/null) || repo_enc=""
[ -n "$repo_enc" ] && repo_q="&repo=${repo_enc}"
fi
# THE SHARED SESSION LEDGER, and the thing most worth getting right here.
#
# scribe_prior_art.sh keeps the rules it has already named in
# <state>/<sid>.rules.ids and passes them as exclude_rule_ids. This hook reads
# and appends to that SAME file rather than keeping its own: two ledgers would
# mean a rule named by one arm gets re-offered by the other, and the hint that
# fires most often is exactly the one that must not repeat itself.
#
# The directory keeps the prior-art name on purpose — renaming it would orphan
# every live session's state for a cosmetic gain.
state_dir="${TMPDIR:-/tmp}/scribe-priorart"
mkdir -p "$state_dir" 2>/dev/null || true
rulefile=""
rule_exclude_q=""
if [ -n "$session_id" ]; then
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
rulefile="$state_dir/${safe_sid}.rules.ids"
# Ageing, not a flat read (#3751): an id named two hours ago is not one the
# session is still holding. scribe_rules_live carries the reasoning.
rule_seen=$(scribe_rules_live "$rulefile")
[ -n "$rule_seen" ] && rule_exclude_q="&exclude_rule_ids=${rule_seen}"
fi
# `|| exit 0` here, unlike the prior-art hook: there is no local arm whose
# finding would be discarded, and an outage line before every command is worse
# than silence. See the header.
body=$(curl -fsS --max-time 5 \
-H "Authorization: Bearer ${token}" \
"${url%/}/api/plugin/tool-rules?tool=${tool_enc}&command=${cmd_enc}${repo_q}${rule_exclude_q}" 2>/dev/null) || exit 0
context=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null) || exit 0
[ -n "$context" ] || exit 0
# Remember what was named so it is not repeated this session.
if [ -n "$rulefile" ]; then
printf '%s' "$body" | jq -r '(.rule_ids // [])[]?' 2>/dev/null \
| scribe_rules_append "$rulefile"
fi
jq -cn --arg ctx "$context" '{
hookSpecificOutput: {
hookEventName: "PreToolUse",
additionalContext: $ctx
}
}' 2>/dev/null || true
exit 0
+76 -26
View File
@@ -1,6 +1,6 @@
---
name: using-scribe
description: Use at the START of every session, and before answering anything about the operator's work or starting any task — establishes the Scribe-first reflex. FIRST ACTION of a session: call list_always_on_rules() (and enter_project when a repo/project is in scope) to load the operator's binding rules. Then recall before acting, update over duplicate, plan in Scribe not in files.
description: Use at the START of every session, and before answering anything about the operator's work or starting any task — establishes the Scribe-first reflex. You hold none of the operator's rules: they arrive by retrieval when your work matches one, and search(content_type="rule") is how you ask before a consequential act. Call enter_project when a repo/project is in scope. Then recall before acting, update over duplicate, plan in Scribe not in files.
---
# Using Scribe
@@ -13,12 +13,19 @@ asked for.
## Do this first (every session)
**Pull the standing rules yourself — do not wait for them to be handed to you.**
At the start of a session, before substantive work, call
`list_always_on_rules()` to load the operator's always-on rules. If the working
repo maps to a Scribe project (you're in a known repo, or `list_repo_bindings`
shows a binding), call `enter_project(id)` instead/as-well — it returns the
project plus its applicable rules, open tasks, and recent notes in one shot.
**You are not holding the operator's rules, and no call loads them all.**
There is no standing set to pull. A rule reaches you when what you are about to
do matches it — a command, code you are writing, or what the operator just
asked for — and on most turns none will. That is the surface working.
**So the reflex is to ASK, not to load.** Before a consequential act — anything
hard to reverse or outward-facing — `search(content_type="rule")` for the thing
you are about to do. An empty session is not evidence of an empty rulebook.
If the working repo maps to a Scribe project (you're in a known repo, or
`list_repo_bindings` shows a binding), call `enter_project(id)` — it returns the
project plus the rules bound to the areas it works in, open tasks, and recent
notes in one shot.
Do this actively. A SessionStart hook *may* also inject a rule index, but treat
that as a bonus, not a precondition: it can be absent (e.g. when the instance is
@@ -56,10 +63,50 @@ Two constraints on *how* that's achieved:
re-deriving it or opening a duplicate. When a project is in scope, pass its
`project_id` so results stay scoped.
2. **Standing rules are binding.** Load them via `list_always_on_rules()` at
session start (see "Do this first"); treat every one as binding. Pull a
rule's full statement with `get_rule(id)` when it's about to bite. When a
project is in scope, `enter_project(id)` also returns its applicable rules.
2. **Rules are binding, and silence does not mean there are none.** Nothing
is preloaded, so "no rule arrived" means "nothing matched" — never "no rule
exists". Ask with `search(content_type="rule")` before a consequential act,
and pull a record's full statement with `get_rule(id)` when it is about to
bite. When a project is in scope, `enter_project(id)` also returns the rules
bound to its areas.
**`kind` says how much force a record carries, and it is never something to
infer.** A **rule** must be followed: ignoring it breaks something or
crosses a boundary. A **preference** records how the operator wants work
done, and ignoring it costs consistency rather than correctness. Both are
worth following and both arrive the same way; only one is a mistake to
miss. An injected line names which in its opening words — *"Standing rule
that may apply…"* against *"Preference that may apply…"* — and every
payload carries `kind` outright.
**A preference is the one record you keep current yourself.** When the
operator corrects you, or the preference on file no longer matches how they
actually want something done, `update_preference` — that is expected, not a
liberty, and it wants the task or note that taught the change. Say in the
same turn that you did it, so they can disagree while it is in front of
them. A rule waits for the operator instead: `create_rule` proposes and
asks. If what you learned is that something MUST be done a certain way,
that is a rule to propose, not a preference to harden in place.
Rules come in two tiers. **Always-on** rules are delivered — they arrive
whether or not you ask. **Conditional** rules are RETRIEVED, and one binds
just as hard for never having been handed to you. So before a consequential
act, `search(content_type="rule")` on what you are about to do. An empty
loaded set is not evidence that no rule applies; it is only evidence that
none was pushed, and those are different claims.
The tier split exists because delivery does not scale: every resident rule
costs tokens in every session forever, so a rulebook that grows past a few
dozen either stops growing or stops fitting. Retrieval is what lets the
rulebook keep growing — but retrieval only fires if something asks.
**Ask hardest where you feel most certain.** Rules about which TOOL to reach
for — use the forge's MCP client rather than curling its API, don't stand up
a local stack, don't run the suite CI owns — govern moves that feel like
mechanics rather than decisions. A reflex raises no doubt, so it generates
no query, so the rule that would have stopped it is never retrieved. That is
the failure this instruction exists to prevent, and confidence is its only
warning sign.
3. **Update over duplicate.** When recording, prefer updating an existing
note/rule/task over creating a new one. Search first; revise what's there.
@@ -183,13 +230,12 @@ bound — confine the session to it:
## Starting a project: decide what it inherits
A project's inheritance is a **decision, not a default**. Before
`create_project`, ask the operator the four inception questions and pass the
`create_project`, ask the operator the three inception questions and pass the
answers — never create a project bare by default:
- which **always-on rulebooks** it should NOT inherit (`list_rulebooks` shows
which are always_on; default: inherit them all) →
`exclude_always_on_rulebooks=[...]`
- which other rulebooks to **subscribe**`subscribe_rulebooks=[...]`
- which rulebooks to **subscribe** (`list_rulebooks` shows them; default: none
— a rulebook binds a project only when it opts in) →
`subscribe_rulebooks=[...]`
- which **design system** its UI is built from (`list_design_systems`; or
none) → `design_system_id=<id | -1>`
- whether to **seed the standard starter Systems** so records can be tagged
@@ -207,19 +253,23 @@ inception is the moment they are decided together, and the record of why.
When codifying a rule, pick its home by **who it should bind** — and keep
shared homes general:
- **Always-on rulebook** (`create_rule` in an `always_on` rulebook) — universal
norms that bind *every* project. Cross-project standards only.
- **Subscribed rulebook** (`create_rule` + `subscribe_project_to_rulebook`) — a
reusable, *themed* module of general rules that binds only projects that opt
in (e.g. a review checklist → every service). Themed, but project-agnostic.
- **Rulebook** (`create_rule` + `subscribe_project_to_rulebook`) — a reusable,
*themed* module of general rules that binds the projects which opt in (e.g. a
review checklist → every service). Themed, but project-agnostic.
- **Project rule** (`create_project_rule`) — anything specific to one project
(its files, paths, quirks).
Both rulebook tiers are shared, so their rules stay general; they differ in
**reach** (all vs opt-in), not generality. Names one project's specifics →
project rule; a standard a category shares → subscribed rulebook; a universal
norm → always-on rulebook. Never put project-specific detail in a shared
rulebook — it leaks to every other project that gets it.
There used to be a third home — an `always_on` rulebook that bound every
project automatically. It is gone: subscription is the only reach a rulebook
has. Names one project's specifics → project rule; anything a category of
projects shares → rulebook. Never put project-specific detail in a rulebook —
it leaks to every other project that subscribes.
**Whichever home it gets, a rule needs `when_to_apply`.** It is the only thing
that decides whether the rule is ever seen: nothing is preloaded, so a rule
with no trigger is not a quiet rule, it is an unreachable one. Write the moment
in the words a session actually produces — the command, the error, the
half-formed ask — not the category it belongs to.
**First ask whether it's a rule at all.** A rule is prose you have to remember
and apply; Scribe's other entities are structure a tool can resolve and check.
-17
View File
@@ -1,17 +0,0 @@
#!/usr/bin/env bash
# Bump the patch segment of fable-mcp/pyproject.toml version and stage the file.
# Usage: called automatically by the Claude Code pre-commit hook, or manually.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
FILE="$REPO_ROOT/fable-mcp/pyproject.toml"
current=$(grep '^version = ' "$FILE" | sed 's/version = "\(.*\)"/\1/')
major=$(echo "$current" | cut -d. -f1)
minor=$(echo "$current" | cut -d. -f2)
patch=$(echo "$current" | cut -d. -f3)
new_version="$major.$minor.$((patch + 1))"
sed -i "s/^version = \"$current\"/version = \"$new_version\"/" "$FILE"
git -C "$REPO_ROOT" add "$FILE"
echo "fable-mcp: $current$new_version"
+277 -44
View File
@@ -13,9 +13,20 @@ separate defects have reached a live install through that path:
install, because `plugin.json`'s version wasn't bumped and the installer
compares versions to decide whether to refresh its cache.
The rule for the second one was already written down and was still missed. A
written rule that depends on being remembered during a long session is not a
control; this is.
Both were fixed. The second was fixed TWICE — once by bumping the number, and
then properly, by removing the class it came from: `plugin.json`'s version is
no longer a value anybody chooses. `scripts/mint_plugin_version.py` derives it
from the clock (`make mint-plugin`), and `check_version_is_minted` below fails
the lane when shipped content moved and the version did not.
State exactly what that did and did not remove, because a rationale that
overstates its own control is how the control gets trusted past its limit, and
because the paragraph this replaces was itself read that way. Gone: having to
remember which NUMBER to write, and the whole question of whether a chosen
number was the right one. Not gone: the mint still has to be RUN, and
forgetting to run it is still possible. What changed is that forgetting is now
LOUD — a red lane on the batch that forgot, instead of a silent no-op found
weeks later when somebody says "I don't think it updated" (#2220).
shellcheck and jq are NOT in `ci-python` (verified against CI-runner's Dockerfile
and scripts/install-common.sh, not from memory — rule #37). CI installs both
@@ -32,7 +43,14 @@ whole file exists to prevent.
Usage:
python3 scripts/check_plugin.py # all checks
python3 scripts/check_plugin.py --no-version # skip the bump check
python3 scripts/check_plugin.py --no-version # on `main` only — see below
`--no-version` exists for ONE case. The version is measured against
`origin/main`, so on `main` itself the comparison is against itself and answers
nothing; the syntax, pattern and marker checks are the only ones that mean
anything there. It is NOT a way past a red lane — see
`check_version_is_minted`, whose whole design is shaped by keeping this flag
out of anyone's muscle memory.
"""
from __future__ import annotations
@@ -43,16 +61,90 @@ import re
import shutil
import subprocess
import sys
from datetime import datetime, timedelta, timezone
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
# The shape contract is ONE definition, shared with the script that mints it —
# a checker carrying its own copy of the format would drift from the minter
# and pass values the minter can no longer produce. Explicit path insert
# because this file runs both as `python3 scripts/check_plugin.py` (which puts
# `scripts/` on the path, not the root) and as an import from the test suite.
sys.path.insert(0, str(ROOT))
from scripts.mint_plugin_version import VERSION_RE # noqa: E402
PLUGIN_DIR = ROOT / "plugin"
HOOKS_DIR = PLUGIN_DIR / "hooks"
MANIFEST = PLUGIN_DIR / ".claude-plugin" / "plugin.json"
# Paths whose contents reach an install. Keep in step with the workflow's
# `paths:` filter — a path that ships but isn't checked here is the gap again.
SHIPPED = ("plugin", ".claude-plugin")
# ── What ships, and what decides what it says about itself ─────────────────
#
# ONE definition (#3127 §3, milestone 334 step 2). It has TWO consumers that
# need different granularities, and conflating them is the bug:
#
# the workflow's `paths:` trigger whole paths should CI run at all?
# the version check paths MINUS should the version
# the manifest have moved?
# `version`
#
# The second one is why this is not just a tuple of paths. `plugin.json` lives
# INSIDE `plugin/`, so a version bump is itself a change to the shipped set —
# and a check that reads the set naively then treats the bump as its own
# justification. Any bump passes, no bump fails, and it has proved nothing.
# `shipped_content_changed` below is the exclusion-aware reader.
#
# The exclusion is that ONE FIELD, never the whole file: `plugin.json` also
# carries description, mcpServers and userConfig, all of which reach an
# install and all of which matter. Excluding the file wholesale would mean a
# userConfig-only edit computes an unchanged version and never refreshes —
# #2209 again with a narrower trigger.
SHIPPED_PATHS = ("plugin", ".claude-plugin")
# Files that decide what a published artifact SAYS ABOUT ITSELF — kept as a
# table so the next artifact is a one-line addition rather than a third
# bespoke guard (#3127 §3). The membership test is NOT "is this copied into
# the artifact?" but "can changing this file change the published bytes, or
# what the artifact says about itself?" — FC learned that twice in four days
# (#3156, #3202), and a deriver is never in the COPY list.
#
# Note what is absent: a CHECKER does not belong here. Whatever validates a
# version decides whether the lane goes red, not what any artifact reports,
# so `check_plugin.py` itself is not a deriver, while the script that mints
# the plugin version is.
DERIVERS: dict[str, tuple[str, ...]] = {
# The "Generate image tags and version" step computes the server image's
# name, ordering key and channel (#3298).
".forgejo/workflows/ci.yml": ("server-image",),
# Decides the plugin's version FORMAT, so it decides what every future
# manifest says about itself (milestone 334 step 3).
"scripts/mint_plugin_version.py": ("plugin",),
}
def version_relevant_paths() -> tuple[str, ...]:
"""Everything a change to which must produce a NEW plugin version.
Wider than `SHIPPED_PATHS`, and #3127 §3's asymmetry is why it has to be:
A change to how the VERSION is computed is compared against nothing at
all. Left out, the published artifact goes on reporting the OLD value
indefinitely.
Concretely — change the mint script's format string, change nothing else,
and a diff over the shipped paths alone reports "no content change, the
version need not move". The manifest then keeps a value in the old format
forever and nothing ever says so. The mint script reaches no install and
belongs here anyway; that is #3156's exact shape.
A CHECKER is deliberately not here. Whatever validates the version decides
whether the lane goes red, not what any artifact reports — so this file is
absent from its own set, and that is not an oversight.
"""
return SHIPPED_PATHS + tuple(
path for path, artifacts in DERIVERS.items() if "plugin" in artifacts
)
failures: list[str] = []
@@ -143,8 +235,6 @@ def check_patterns() -> None:
ok(f"{rel}: no known-bad patterns")
# --- the version bump ------------------------------------------------------
# --- shellcheck ------------------------------------------------------------
def check_shellcheck() -> None:
@@ -215,6 +305,16 @@ SMOKE_EVENTS: dict[str, str] = {
"tool_input": {"file_path": "src/x.py",
"new_string": f"def {_ABSENT_SYM}():\n pass\n"}}
),
# The pre-tool rule arm (#3476). A real Bash call, and one whose whole
# point is that it looks harmless: reaching for curl against the forge API
# is the reflex the arm exists to catch. With no instance it must stay
# SILENT — it is deliberately not an OUTAGE_SPEAKER, because a Bash call is
# not occasional and an outage line before every command gets the channel
# muted.
"scribe_tool_rules.sh": json.dumps(
{"session_id": "smoke", "cwd": ".", "tool_name": "Bash",
"tool_input": {"command": "curl -s https://example.invalid/api/v1/runs"}}
),
"scribe_sync_processes.sh": json.dumps({"source": "startup"}),
"scribe_session_context.sh": json.dumps({"source": "startup"}),
# The after-write hook (#2901) diffs the working tree; on CI's clean
@@ -391,80 +491,213 @@ def _git(*args: str) -> tuple[int, str]:
return proc.returncode, (proc.stdout or proc.stderr).strip()
def manifest_version(ref: str | None = None) -> str | None:
"""The manifest version at `ref`, or in the working tree when ref is None."""
def manifest_text(ref: str | None = None) -> str | None:
"""The manifest's RAW TEXT at `ref`, or in the working tree when ref is None.
Split out from `manifest_version` because the exclusion below needs every
field except one, not the one field.
"""
if ref is None:
try:
return json.loads(MANIFEST.read_text()).get("version")
except Exception:
return MANIFEST.read_text()
except OSError:
return None
rel = MANIFEST.relative_to(ROOT).as_posix()
code, out = _git("show", f"{ref}:{rel}")
if code != 0:
return out if code == 0 else None
def manifest_version(ref: str | None = None) -> str | None:
"""The manifest version at `ref`, or in the working tree when ref is None."""
text = manifest_text(ref)
if text is None:
return None
try:
return json.loads(out).get("version")
return json.loads(text).get("version")
except Exception:
return None
def check_version_bump(base: str = "origin/main") -> None:
"""If shipped plugin content differs from `base`, the version must too.
# Distinct from None, which is a legitimate "this manifest does not exist".
_UNREADABLE = object()
Stated against the BASE BRANCH rather than the last commit on purpose. A
per-commit rule would demand a bump from every commit in a batch; what
actually matters is that whatever reaches an install carries a version the
installer can tell apart from the one already cached. One bump per batch,
which is also how a human would do it.
def manifest_differs_beyond_version(a: str | None, b: str | None) -> bool:
"""Do two `plugin.json` texts differ in anything OTHER than `version`?
THE exclusion, and it is kept pure — no git, no filesystem — because this
is the half worth testing hard and it needs no repository to exercise.
Compares PARSED objects rather than text, so reformatting, key reordering
and whitespace do not read as content changes. `version` is dropped from
both sides; everything else counts, which is what keeps a userConfig-only
or mcpServers-only edit demanding a new version.
Unreadable input answers True. The conservative direction is "demand a new
version": a spurious bump costs one cache refresh, while a missed one is
#2209 — the fix reaches the repo and stops there.
"""
def without_version(text: str | None):
if text is None:
return None
try:
data = json.loads(text)
except Exception:
return _UNREADABLE
if not isinstance(data, dict):
return _UNREADABLE
return {k: v for k, v in data.items() if k != "version"}
left, right = without_version(a), without_version(b)
if left is _UNREADABLE or right is _UNREADABLE:
return True
return left != right
def shipped_content_changed(base: str) -> tuple[bool | None, list[str]]:
"""Has anything that REACHES AN INSTALL changed against `base`?
Returns `(changed, paths)`. `changed` is **None** when the question could
not be answered — a caller must never read that as "no", which is the
distinction #2663 cost weeks of zeroed telemetry to learn.
The manifest is special-cased, not excluded: if it is the ONLY thing that
moved and the only difference is `version`, nothing that reaches an
install has changed. Any other manifest field, or any other file, counts.
Reads `version_relevant_paths`, which is the shipped set PLUS the files
that decide the version — see there for why the deriver has to be in it.
"""
code, out = _git("diff", "--name-only", base, "--", *version_relevant_paths())
if code != 0:
return None, []
paths = [p for p in out.splitlines() if p.strip()]
if not paths:
return False, []
rel_manifest = MANIFEST.relative_to(ROOT).as_posix()
if paths == [rel_manifest]:
return manifest_differs_beyond_version(
manifest_text(), manifest_text(base)
), paths
return True, paths
def check_version_is_minted(base: str = "origin/main") -> None:
"""THE control (#3127 checklist 4), replacing "somebody remembers".
The checklist asks, of any hand-set component: *say what happens the
release somebody forgets it.* This is the answer — the lane goes red,
deterministically, because CI can compute whether the value should have
moved. Its predecessor could only ask "did the number move at all", which
any bump satisfied and which therefore proved nothing.
Four verdicts:
content changed, version did not FAIL — this is #2209, exactly
version not in canonical shape FAIL — see below
version implausibly in the future FAIL — a bad clock or a hand-edit
version moved, content did not pass, and say so
THE LAST ROW IS NOT A FAILURE, DELIBERATELY. A needless re-mint costs one
cache refresh and nothing else. Failing the lane over a harmless act is how
a check earns a `--no-version` in somebody's muscle memory and stops
running at all — which is the failure mode this whole file exists to
prevent. The implication that matters is one-directional: content changed
IMPLIES version moved.
A malformed version is worth failing on even though the installer would
accept it. `K4` returns the manifest string verbatim, and `H == "unknown"`
sets `forceOverwrite`, so a broken value either sorts as a normal string
or reinstalls the plugin every single session (#3325). Neither is loud.
Stated against the BASE BRANCH rather than the last commit, as its
predecessor was: a per-commit rule would demand a fresh mint from every
commit in a batch, when what matters is that whatever reaches an install
differs from what is cached. One mint per batch, which is also how a person
would do it.
"""
code, _ = _git("rev-parse", "--verify", base)
if code != 0:
# Do NOT pass silently — a check that quietly no-ops is how this class
# of bug survives in the first place.
fail(
f"cannot resolve {base}, so the version-bump check could not run. "
f"cannot resolve {base}, so the minted-version check could not run. "
f"Fetch it first — `git fetch --depth=1 origin main:refs/remotes/"
f"origin/main` is enough, since this diffs two trees and needs no "
f"common ancestor — or pass --no-version deliberately."
)
return
code, changed = _git("diff", "--name-only", base, "--", *SHIPPED)
if code != 0:
fail(f"git diff against {base} failed: {changed}")
return
if not changed.strip():
ok(f"no shipped plugin changes against {base} — version bump not required")
return
here, there = manifest_version(), manifest_version(base)
here = manifest_version()
if here is None:
fail(f"could not read a version from {MANIFEST.relative_to(ROOT)}")
return
if not VERSION_RE.match(here):
fail(
f"the manifest version is {here!r}, which is not YYYY.MM.DD.HHMM.\n"
f" One shape for every version in the family (#3127 checklist "
f"10), zero-padded so the midnight case renders 2026.01.05.0000.\n"
f" Run `make mint-plugin`."
)
return
minted = datetime.strptime(here, "%Y.%m.%d.%H%M").replace(tzinfo=timezone.utc)
# A day of slack: the mint happens on a workstation and the lane runs
# later, so a *small* skew is ordinary. A value further out than that is
# a wrong clock or a typed year, and it makes the version lie about when
# it was minted.
if minted > datetime.now(timezone.utc) + timedelta(days=1):
fail(
f"the manifest version {here} is in the future. Either the clock "
f"that minted it is wrong, or it was typed by hand."
)
return
changed, paths = shipped_content_changed(base)
if changed is None:
fail(f"git diff against {base} failed, so the version check could not run")
return
there = manifest_version(base)
if there is None:
ok(f"no manifest on {base} — treating as a new plugin (version {here})")
return
if here == there:
files = "\n ".join(changed.splitlines())
if changed and here == there:
files = "\n ".join(paths)
fail(
f"plugin content changed but the manifest version is still {here}.\n"
f" The installer compares versions to decide whether to refresh "
f"its cache, so an unchanged version means these edits reach the repo "
f"and stop there — the marketplace clone updates, the cache that "
f"actually executes does not (issue #2209).\n"
f" Bump `version` in {MANIFEST.relative_to(ROOT)}.\n"
f"plugin content changed but the version is still {here}.\n"
f" The installer decides whether to refresh its cache by "
f"comparing this string, so an unchanged version means these edits "
f"reach the repo and stop there — the marketplace clone updates, the "
f"cache that actually executes does not (#2209, #1040, #2220).\n"
f" Run `make mint-plugin`.\n"
f" Changed:\n {files}"
)
elif changed:
ok(f"plugin content changed and the version was minted {there} -> {here}")
elif here != there:
# Not a failure — see the docstring. Named rather than silent, because
# the uninteresting cause (minted twice) and the interesting one (the
# version-relevant set is too narrow to see what actually changed)
# produce the same line, and only a person can tell them apart.
ok(
f"the version moved {there} -> {here} with no version-relevant "
f"change — harmless, unless something DID change that the set "
f"cannot see"
)
else:
ok(f"plugin content changed and version moved {there} -> {here}")
ok(f"nothing version-relevant changed against {base} — no mint required")
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--no-version", action="store_true",
help="skip the manifest version-bump check")
help="skip the minted-version check; for `main`, where "
"it would be measured against itself")
parser.add_argument("--base", default="origin/main",
help="branch the version bump is measured against")
help="branch the version is measured against")
args = parser.parse_args()
if not HOOKS_DIR.is_dir():
@@ -478,7 +711,7 @@ def main() -> int:
check_local_prior_art_needs_no_instance()
check_session_context_reports_its_version()
if not args.no_version:
check_version_bump(args.base)
check_version_is_minted(args.base)
print()
if failures:
+144
View File
@@ -0,0 +1,144 @@
#!/usr/bin/env python3
"""Mint the plugin's version — `YYYY.MM.DD.HHMM`, UTC, zero-padded.
Run this whenever you change something under `plugin/` or `.claude-plugin/`,
before you commit:
make mint-plugin # or: python3 scripts/mint_plugin_version.py
WHY A SCRIPT AND NOT A BUILD STEP. `plugin/` is not in the Docker image.
Installs fetch it straight from this git repo via `.claude-plugin/
marketplace.json`, so **a push IS the release** — there is no build between
you committing and a user fetching, and therefore no moment at which CI could
stamp a version in. Every other artifact in the family derives its version
during a build (note #3127 §2). This one has no build to derive during.
WHICH CLOCK, AND WHY IT DIFFERS FROM THE SERVER IMAGE — the divergence is
deliberate, and it lives one directory away from its opposite, so it is
exactly what a later "let's make these consistent" change would collapse:
server image name from COMMIT time, ordering key from BUILD time
(two lanes building one source must report one string;
a rebuild of an older commit must not go backwards)
plugin one value, from MINT time
§2's reason for commit time is that two lanes build one source. The plugin has
one lane and no build, so that reason does not reach it and paying its cost
buys nothing. What is given up is reproducibility-from-history: you cannot
recompute this value later, only verify that it moved when it had to.
That trade is acceptable ONLY because of what #3325 established by reading the
installer's code: the refresh test is `P.version === H`, plain string
equality, with no ordering comparison anywhere. Where a comparator ORDERS, an
unreproducible version is dangerous — nothing can check it is right. Where it
only tests equality, "did it change when it should have" is the entire
specification, and `check_plugin.py` checks that completely.
The manifest is rewritten with a surgical replacement of the `version` line
rather than `json.dump`, because its formatting and key order are not this
script's to decide and a whole-file reformat would make every mint an
unreadable diff.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from datetime import datetime, timezone
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
MANIFEST = ROOT / "plugin" / ".claude-plugin" / "plugin.json"
# Four dot-separated numeric fields, zero-padded, and nothing else — one shape
# for every human-readable version in the family (#3127 checklist 10). The
# padding is load-bearing for the midnight case the checklist names by hand:
# 2026.01.05.0000, which an unpadded `%-H%M` would render as `0` and silently
# shorten. Harmless while nothing orders these, wrong the moment anything does.
VERSION_RE = re.compile(r"^\d{4}\.\d{2}\.\d{2}\.\d{4}$")
VERSION_FORMAT = "%Y.%m.%d.%H%M"
# The `version` line, captured so its surroundings survive byte-for-byte.
VERSION_LINE_RE = re.compile(r'^(\s*"version"\s*:\s*")([^"]*)(".*)$', re.M)
def mint(now: datetime | None = None) -> str:
"""The version for this moment. UTC, always.
The conversion is not decoration: `strftime` renders whatever offset the
datetime carries, so without it two people minting the same instant in
different zones produce different strings — and the string IS the
artifact's identity. A naive datetime is read as UTC rather than as the
machine's zone, because that is this function's stated contract and
guessing the host's offset is how the bug comes back by another route.
"""
moment = now or datetime.now(timezone.utc)
if moment.tzinfo is None:
moment = moment.replace(tzinfo=timezone.utc)
return moment.astimezone(timezone.utc).strftime(VERSION_FORMAT)
def rewrite(text: str, version: str) -> str:
"""`text` with its `version` value replaced, and everything else untouched.
Raises rather than falling back to a JSON round-trip: a manifest this
cannot match is one whose shape changed, and quietly reformatting the file
to cope would be a much larger edit than the caller asked for.
"""
# Counted BEFORE substituting, not via subn's return: a capped `subn`
# reports the replacements it made, so a manifest with two `version` lines
# would look like a clean single match while the second one — the real one,
# perhaps — kept its old value.
matches = VERSION_LINE_RE.findall(text)
if len(matches) != 1:
raise ValueError(
f"expected exactly one `version` line in the manifest, found {len(matches)}"
)
return VERSION_LINE_RE.sub(
lambda m: f"{m.group(1)}{version}{m.group(3)}", text, count=1
)
def main() -> int:
parser = argparse.ArgumentParser(description="Mint the plugin's version.")
parser.add_argument(
"--check", action="store_true",
help="print the version that WOULD be minted and change nothing",
)
args = parser.parse_args()
version = mint()
if args.check:
print(version)
return 0
try:
text = MANIFEST.read_text()
except OSError as exc:
print(f"cannot read {MANIFEST.relative_to(ROOT)}: {exc}", file=sys.stderr)
return 1
try:
previous = json.loads(text).get("version")
except Exception:
previous = None
if previous == version:
# Same minute. Not an error — the value is already correct for now, and
# failing here would turn "I ran it twice" into a problem to solve.
print(f"plugin version already {version} (same minute) — unchanged")
return 0
try:
MANIFEST.write_text(rewrite(text, version))
except ValueError as exc:
print(f"{MANIFEST.relative_to(ROOT)}: {exc}", file=sys.stderr)
return 1
print(f"plugin version {previous} -> {version}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
-39
View File
@@ -1,39 +0,0 @@
#!/usr/bin/env bash
# Claude Code PreToolUse hook for Bash.
# Reads the tool input JSON from stdin; if the command is a git commit
# and fable-mcp files (other than pyproject.toml) are staged, bumps
# the fable-mcp patch version before the commit proceeds.
#
# Exits 0 always so it never blocks the commit.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
input=$(cat)
command=$(echo "$input" | python3 -c "
import sys, json
data = json.load(sys.stdin)
# Claude Code sends {tool_input: {command: ...}}
ti = data.get('tool_input', data)
print(ti.get('command', ''))
" 2>/dev/null || echo "")
# Only act on git commit commands
if ! echo "$command" | grep -qE "git commit"; then
exit 0
fi
cd "$REPO_ROOT"
# Check if fable-mcp files other than pyproject.toml are staged
fable_staged=$(git diff --cached --name-only 2>/dev/null \
| grep "^fable-mcp/" \
| grep -v "^fable-mcp/pyproject.toml$" \
|| true)
if [ -n "$fable_staged" ]; then
bash "$REPO_ROOT/scripts/bump_fable_mcp_version.sh"
fi
exit 0
+34 -8
View File
@@ -38,13 +38,38 @@ from quart import Quart
# them) was DECLINED a line, deliberately, by the operator — not overlooked.
# The reasoning, so it is not re-litigated blind: this is a map, and its own
# closing line says each tool's description carries the full contract. The
# sweep is a curation act, not a session-start reflex like enter_project or
# list_always_on_rules. Spending the last of the budget on it would leave the
# sweep is a curation act, not a session-start reflex like enter_project.
# Spending the last of the budget on it would leave the
# map unable to grow for something more central later.
#
# The accepted cost: an agent that never opens create_note's docstring never
# learns the field exists. Guidance lives in the create_note / update_note
# docstrings and the using-scribe skill instead.
#
# Milestone 333 step 3 (2026-09-04) bought the HOW bullet's second clause —
# search(content_type="rule") before a consequential act — by TRADING OUT
# "Processes are saved procedures (follow verbatim)" and "Deletes are
# trash-recoverable". Recorded so the trade is not silently reversed:
# - Both were already in test_instruction_surfaces_agree's DISPLACED_TOPICS
# and already stated on a delivered surface, so nothing fell off: the
# process reflex is in every scribe-proc-* skill listing (each says the
# process governs and is followed verbatim), and trash recovery is in the
# delete_*/list_trash/restore docstrings, which is where per-tool guidance
# belongs by this block's own doctrine.
# - What it bought is not per-tool guidance and has nowhere else to live at
# session-start altitude. Rules were retrievable only by RESIDENCY: the
# always-on preload put them in front of the agent, and nothing told a
# session to go looking for one it had not been handed. That preload is
# gone (milestone 394), which makes this line LOAD-BEARING rather than
# supplementary: retrieval is now the only delivery, and retrieval fires
# only if something asks. A session that waits to be handed a rule is
# handed nothing. A tool-choice reflex asks least of all (#3476, #161).
# - It also has to carry what absence MEANS. "No rule arrived" is now the
# ordinary state rather than the exceptional one, and reading it as
# "there is no rule" is the #3720 defect at session scale. Rule 119 makes
# these surfaces the
# specification, so the same sentence lands on all three session-start
# surfaces, and test_instruction_surfaces_agree pins it.
_INSTRUCTIONS = """
Scribe is the operator's self-hosted second brain and system of record — and
yours: recall from it before acting, record as you go. Keep no parallel copy
@@ -52,8 +77,8 @@ in local files (CLAUDE.md, auto-memory); Scribe holds the single copy.
Hierarchy: Project -> Milestone -> Task/Note. The map, by purpose:
- ORIENT: enter_project(id) at session start — rules, open tasks, recent
notes, Systems, design system. `inception` key: ask what the project
inherits, decide_project_inception (create_project takes the same).
notes, Systems, design system. `inception`: ask what the project
inherits, then decide_project_inception.
- DO: create_task. Fixed a problem? kind="issue" (symptom -> root cause ->
fix), never a work-log line on an unrelated task. Log with add_task_log;
keep status honest — in_progress on start, done on finish.
@@ -63,13 +88,14 @@ Hierarchy: Project -> Milestone -> Task/Note. The map, by purpose:
active project_id to stay in scope.
- WHERE work happens: Systems. Tag records with system_ids as you write;
create_system when the area is unmodelled.
- HOW: rules are binding — list_always_on_rules() at session start.
- HOW: rules bind; preferences guide. Nothing preloads — a rule arrives
when your work matches it. Before a consequential act,
search(content_type="rule"); silence means nothing matched, not none.
- UI: the project's design system is binding — resolve_design_system /
get_design_system_stylesheet before hand-writing a value.
- REUSE: search snippets before writing a helper; record what you build with
create_snippet; classify shapes against canon (classify_shapes) — a
consumer map is rows, never prose. Processes are saved procedures (follow
verbatim). Deletes are trash-recoverable.
consumer map is rows, never prose.
A task is a note with status (*_note vs *_task tools).
Creates are duplicate-gated: a near-match BLOCKS and returns the existing
@@ -104,7 +130,7 @@ _READ_ONLY_TOOLS = frozenset({
"get_task", "get_milestone", "get_recent", "enter_project",
"list_milestones", "list_notes", "list_projects", "list_rulebooks",
"list_rules", "list_tags", "list_tasks", "list_topics", "list_trash",
"list_always_on_rules", "search",
"search",
"get_system", "list_systems", "list_system_records",
# The global area catalog and its mapping REPORT — propose writes nothing;
# map_system_to_canonical is the separate, explicitly-called write.
+1 -1
View File
@@ -57,7 +57,7 @@ async def get_milestone(milestone_id: int) -> dict:
return {
"milestone": out,
"steps": [t.to_dict() for t in steps],
**rulebooks_svc.rules_payload(applicable),
**rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_milestone"),
}
+11 -13
View File
@@ -207,7 +207,7 @@ async def enter_project(project_id: int) -> dict:
],
"design_system": design_system,
"milestone_summary": milestone_summary,
**rulebooks_svc.rules_payload(applicable),
**rulebooks_svc.rules_payload(applicable, user_id=uid, source="enter_project"),
"open_tasks": [
{
"id": t.id, "title": t.title, "status": t.status,
@@ -251,22 +251,21 @@ async def get_project(project_id: int) -> dict:
applicable = await rulebooks_svc.get_applicable_rules(
project_id=project_id, user_id=uid,
)
data.update(rulebooks_svc.rules_payload(applicable))
data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_project"))
return data
def _inception_choices(
exclude_always_on_rulebooks, subscribe_rulebooks, design_system_id, seed_systems,
subscribe_rulebooks, design_system_id, seed_systems,
) -> dict | None:
"""The tool args → an inception choices object, or None when no inception
arg was given at all (a bare create stays undecided and enter_project
asks). design_system_id: 0 = not stated, -1 = explicitly none, n = that
system."""
if (exclude_always_on_rulebooks is None and subscribe_rulebooks is None
if (subscribe_rulebooks is None
and not design_system_id and seed_systems is None):
return None
return {
"exclude_always_on_rulebooks": list(exclude_always_on_rulebooks or []),
"subscribe_rulebooks": list(subscribe_rulebooks or []),
"design_system_id": None if design_system_id in (0, -1) else design_system_id,
"seed_systems": bool(seed_systems),
@@ -279,7 +278,6 @@ async def create_project(
goal: str = "",
status: str = "active",
color: str = "",
exclude_always_on_rulebooks: list[int] | None = None,
subscribe_rulebooks: list[int] | None = None,
design_system_id: int = 0,
seed_systems: bool | None = None,
@@ -299,9 +297,10 @@ async def create_project(
goal: The desired outcome or definition of done for the project.
status: one of active (default), paused, completed, archived.
color: Optional hex colour for the project card (e.g. "#6366f1").
exclude_always_on_rulebooks: always-on rulebook ids this project does
subscribe_rulebooks: rulebook ids this project opts into. Since
milestone 394 subscription is the only way a rulebook binds a
project, so there is no automatic tier left to decline. Was
NOT inherit ([] = inherit them all). list_rulebooks shows which are
always_on.
subscribe_rulebooks: rulebook ids to subscribe (the non-always-on ones).
design_system_id: the design system this project's UI is built from
(list_design_systems); -1 = explicitly none; 0 = not stated.
@@ -319,7 +318,7 @@ async def create_project(
)
data = project.to_dict()
choices = _inception_choices(
exclude_always_on_rulebooks, subscribe_rulebooks, design_system_id, seed_systems,
subscribe_rulebooks, design_system_id, seed_systems,
)
if choices is not None:
decided = await inception_svc.decide(uid, project.id, choices=choices, via="mcp")
@@ -336,7 +335,6 @@ async def create_project(
async def decide_project_inception(
project_id: int,
exclude_always_on_rulebooks: list[int] | None = None,
subscribe_rulebooks: list[int] | None = None,
design_system_id: int = 0,
seed_systems: bool | None = None,
@@ -345,11 +343,11 @@ async def decide_project_inception(
or re-decide later (milestone 297).
Owner-only. Applies the effects through the ordinary tools' paths —
exclude_always_on_rulebook, subscribe_project_to_rulebook,
subscribe_project_to_rulebook,
set_project_design_system, the standard Systems seed — and writes the
decision on the project last, so get_project/enter_project can say why
the project has the rules, design and Systems it has. Re-deciding is
additive for exclusions/subscriptions (use include_always_on_rulebook /
additive for subscriptions (use
unsubscribe_project_from_rulebook to undo one), replaces the design
system, and never re-seeds Systems a project already has.
@@ -359,7 +357,7 @@ async def decide_project_inception(
"""
uid = current_user_id()
choices = _inception_choices(
exclude_always_on_rulebooks, subscribe_rulebooks, design_system_id, seed_systems,
subscribe_rulebooks, design_system_id, seed_systems,
) or {}
decided = await inception_svc.decide(uid, project_id, choices=choices, via="mcp")
return {"project_id": project_id, **decided}
+341 -137
View File
@@ -18,6 +18,7 @@ from scribe.mcp._context import current_user_id
from scribe.services import dedup as dedup_svc
from scribe.services import rulebooks as rulebooks_svc
from scribe.services import trash as trash_svc
from scribe.services.rule_usage import record_rule_pulled
# ── Rulebook CRUD ───────────────────────────────────────────────────────
@@ -47,16 +48,13 @@ async def get_rulebook(rulebook_id: int) -> dict:
async def create_rulebook(title: str, description: str = "") -> dict:
"""Create a new rulebook (a shared, reusable module of general rules).
Two ways a rulebook reaches projects, set by its always_on flag (toggle via
update_rulebook):
- always_on = true -> binds EVERY one of your projects automatically.
Use for universal cross-project norms that apply across every
project, not just one.
- always_on = false -> binds only projects that subscribe
(subscribe_project_to_rulebook). Use for a THEMED body of rules a
category of projects shares (e.g. a design system that visual apps
opt into).
Either way a rulebook is SHARED, so its rules must stay general — agnostic
A rulebook reaches a project ONE way: the project subscribes to it
(subscribe_project_to_rulebook). There was a second until milestone 394 —
an `always_on` flag that bound every project automatically — and it is
gone with the tier it belonged to. Opt-in is now the whole model, so a
rulebook binds what asked for it and nothing else.
A rulebook is SHARED, so its rules must stay general — agnostic
to any single project. Project-specific rules go in create_project_rule.
Args:
@@ -72,7 +70,6 @@ async def create_rulebook(title: str, description: str = "") -> dict:
async def update_rulebook(
rulebook_id: int, title: str = "", description: str = "",
always_on: bool | None = None,
) -> dict:
"""Update an existing rulebook. Only non-empty fields are changed.
@@ -80,9 +77,6 @@ async def update_rulebook(
rulebook_id: Rulebook to update.
title: New title. Empty string leaves unchanged.
description: New description. Empty string leaves unchanged.
always_on: When True, rules in this rulebook are loaded at session
start by list_always_on_rules regardless of project context.
Pass None to leave unchanged.
"""
uid = current_user_id()
fields: dict = {}
@@ -90,8 +84,6 @@ async def update_rulebook(
fields["title"] = title
if description:
fields["description"] = description
if always_on is not None:
fields["always_on"] = always_on
rb = await rulebooks_svc.update_rulebook(rulebook_id, uid, **fields)
if rb is None:
raise ValueError(f"rulebook {rulebook_id} not found")
@@ -233,49 +225,6 @@ async def list_rules(
return {"rules": [_rule_summary(r) for r in rows], "total": len(rows)}
async def list_always_on_rules(project_id: int = 0) -> dict:
"""Return all rules from rulebooks flagged always_on for the current user.
Call this at session start. Treat the returned rules as binding for the
session — they apply regardless of which project (if any) is in scope.
Returns the ALWAYS-ON tier only (milestone 307). A `conditional` rule is
still binding when it applies; it just is not resident — it reaches a
session through enter_project (when the project works in an area the rule
is tagged to) or through search(content_type="rule"). Nothing here is a
behaviour change until rules are actually re-tiered: `tier` defaults to
always_on, so an existing rulebook returns exactly what it always did.
Pair with get_project(id).applicable_rules when working on a specific
project to also load that project's subscription-derived rules.
A rule carrying `last_verified` asserts a FACT about something outside the
operator's control — a runner's shell, a tool's existence, a setting
somewhere. It is still binding; the field says how long ago anyone
confirmed it, and "never" means nobody has. Follow the rule, and if you
are already standing where the check could be made, make it: get_rule
gives you its `verify_with`. Most rules have no such field, which means
they are decisions and there is nothing to check.
Args:
project_id: 0 (default) = the user-wide set. Inside a project, pass
its id: an always-on rulebook the project EXCLUDED at inception
(see enter_project's `excluded_always_on`) is left out — the
project decided not to inherit it.
"""
uid = current_user_id()
rules = await rulebooks_svc.list_always_on_rules(uid, project_id=project_id)
return {
"rules": [_rule_summary(r) for r in rules],
"total": len(rules),
# A marker for the set you are now holding. It is not for you to read:
# the write-path hook carries it back and is told if these rules have
# moved since. Deliberately NOT on rules_payload's applicable_rules —
# that is a DIFFERENT set (subscription-derived), and one key name
# over two sets is how a comparison starts reporting phantom changes.
"rules_etag": rulebooks_svc.rules_etag(rules),
}
async def get_rule(rule_id: int) -> dict:
"""Fetch a rule by id — full statement + why + how_to_apply.
@@ -288,20 +237,79 @@ async def get_rule(rule_id: int) -> dict:
rule = await rulebooks_svc.get_rule(rule_id, uid)
if rule is None:
raise ValueError(f"rule {rule_id} not found")
# THE pull that matters. The write-path rule arm's own message ends "Read
# it with get_rule(N)", so this is the exact action the hint asks for and
# the only evidence that one landed. Recorded after the access check, so a
# refused read is not counted as a pull.
record_rule_pulled(user_id=uid, rule_id=int(rule.id), source="mcp_get_rule")
return await rulebooks_svc.rule_detail(uid, rule)
async def create_rule(
topic_id: int, title: str, statement: str, when_to_apply: str = "",
why: str = "", how_to_apply: str = "", order_index: int = 0,
tier: str = "always_on", system_ids: list[int] | None = None,
arose_from_id: int = 0, verify_with: str = "", expires_when: str = "",
force: bool = False,
system_ids: list[int] | None = None, force: bool = False,
) -> dict:
"""Create a new rule in a rulebook (a SHARED rule — keep it general).
PROPOSE RULES READILY, AND WRITE ONE WHEN THE OPERATOR SAYS YES. Noticing
that something has hardened into a standing instruction is valuable work,
and a session that notices it and says nothing has thrown the observation
away. So raise it whenever you see one. The single step that belongs
between noticing and writing is the operator's yes: a rule binds every
future session, and they are the person it binds.
Their yes is also the only moment the rule is reliably IN FRONT of them.
After the write it may not be again for months — a conditional rule is not
read aloud at session start, and a project-scoped one does not appear in
an unfiltered list_rules() at all. So the proposal is the review.
When the operator asks for a rule in so many words, that IS the yes —
write it and move on. The loop below is for the rule you thought of.
A PROPOSAL CARRIES FOUR THINGS, and the fourth is the one that decides it:
1. WHAT it would require — the statement, in the words it would carry,
not a gloss of them. The operator is agreeing to text.
2. INTENT — what it changes about how work gets done, and what goes
wrong today without it. "Be careful about X" is not an intent; the
behaviour that would differ tomorrow is.
3. WHY NOW — the incident, observation or decision behind it. Pass that
record as arose_from_id, and say it in the conversation too: the
field is for the reader six months out, the sentence is for the
person deciding.
4. HOW IT WOULD BE ENFORCED — a test, a CI check, a hook, a schema
constraint, a duplicate gate, a review step... or nothing, in which
case say so plainly: "nothing — this is prose a session has to
remember." Answer this one honestly and it will sometimes dissolve
the rule, which is the point rather than a side effect. What a test
can assert should BE that test; a rule is what remains when nothing
mechanical can hold the thing. A rulebook grows by default and
shrinks only on purpose, so a question that prevents a rule is worth
more than any question that improves one's wording.
THEN CLOSE WITH A QUESTION THEY CAN ANSWER IN ONE WORD. Offer three
answers, and make the middle one the easy one:
* "Approve it AS WRITTEN" — you create it with the statement exactly as
shown. This is what makes element 1 load-bearing: they approved TEXT,
so that text is what gets stored, verbatim.
* "LET'S TALK ABOUT IT" — the wording, the scope, whether it
wants to be a rule at all. Most good rules arrive this way, so treat
this answer as the expected one rather than a setback.
* "NO" — let it go. If the observation is still worth keeping, it is a
note (create_note): recorded, findable, and binding on nobody.
Where the interface offers structured choices, ask it that way — a
question with named options is answered in a click, while the same
question inside a paragraph is answered by scrolling past. Where it does
not, write the three options out as three options. Either way ask once
and let the answer stand; re-raising a declined proposal argues a rule
into existence, which is the thing this whole loop exists to prevent.
A rulebook rule is shared by every project that gets the rulebook: an
always_on rulebook binds ALL your projects; a subscribed rulebook binds the
A subscribed rulebook binds the
projects that opt in. So a rulebook rule must read as a general standard —
never pin it to one project's files, paths, or quirks. For a rule that
applies to a single project only, use create_project_rule instead (no
@@ -338,7 +346,7 @@ async def create_rule(
instruction. State the moment or the material: "before any git
push", "when adding a value to a CHECK-gated column", "when a
release is being cut". Write it even though the parameter is
optional: it decides the tier below, it is how the rule is found
optional: it is how the rule is found
when it matters, and a rule nobody can place is a rule nobody
applies.
This field is also the rule's RETRIEVAL SURFACE — it and the
@@ -350,12 +358,14 @@ async def create_rule(
brought it back as the top hit. Where a rule prevents a specific
failure, put that failure's vocabulary here — the error text,
the wrong behaviour, the dead end.
tier: "always_on" (default) or "conditional".
The test: can you name the trigger WITHOUT naming a system, an
artifact type or a moment? If the honest answer is "whenever you
are working", it is always_on. If you had to name something, it is
conditional — and conditional costs nothing when it is irrelevant,
which is what lets it be as long as it needs to be.
The two spellings, side by side:
RETRIEVES: "the migration failed with a check violation on a
column we just extended"
COLLAPSES: "when working on migrations"
The second names a CATEGORY. No session ever produces a
category — it produces the command, the error, the half-formed
ask — so a trigger written that way leaves the embedded
document to be carried by the title alone.
system_ids: Ids from list_canonical_systems — the global AREAS this
rule is about. This is what lets a rule reach a project that is
working in that area, so a CI rule surfaces on a CI change.
@@ -394,7 +404,7 @@ async def create_rule(
rule = await rulebooks_svc.create_rule(
topic_id=topic_id, user_id=uid,
title=title, statement=statement, when_to_apply=when_to_apply,
tier=tier, arose_from_id=arose_from_id,
arose_from_id=arose_from_id,
why=why, how_to_apply=how_to_apply, order_index=order_index,
verify_with=verify_with, expires_when=expires_when,
)
@@ -404,9 +414,8 @@ async def create_rule(
async def create_project_rule(
project_id: int, statement: str, title: str = "", when_to_apply: str = "",
why: str = "", how_to_apply: str = "", order_index: int = 0,
tier: str = "always_on", system_ids: list[int] | None = None,
arose_from_id: int = 0, verify_with: str = "", expires_when: str = "",
force: bool = False,
system_ids: list[int] | None = None, force: bool = False,
) -> dict:
"""Create a rule scoped to a single project (no rulebook needed).
@@ -418,6 +427,16 @@ async def create_project_rule(
the rule is returned in get_project's applicable_rules (under
project_rules) and in list_rules(project_id=...).
PROPOSE, THEN WRITE ON A YES — create_rule's opening carries the whole
loop: the four things a proposal states (what it would require, its
intent, why now, and how it would be enforced) and the one-word question
that closes it (approve as written / talk about it / no). All of it
applies here unchanged. Reach for that loop MORE readily on this surface,
not less: a project rule stays out of an unfiltered list_rules(), and a
conditional one stays out of session start too, so the operator's yes is
the one moment this rule is certain to have been seen by the person it
binds.
Check first whether a rule is the right shape at all — create_rule's
opening asks that question and it applies identically here. A visual
standard is a design system; a procedure is a process (create_process);
@@ -438,25 +457,12 @@ async def create_project_rule(
characters of statement.
when_to_apply: WHEN this rule fires — the trigger, not the
instruction, and the rule's retrieval surface: name the SYMPTOM,
the words someone would type while stuck. See create_rule for the
full argument. It informs the tier below rather than deciding it,
since a project rule's tier turns on area-scope, not on whether
the trigger can be named.
tier: "always_on" (default) or "conditional". The SAME two values as
create_rule, judged against a different cost — do not import that
tool's test wholesale. There, always_on means every session in
every project, so the bar is high: the trigger must be nameless
("whenever you are working"). Here the rule is already scoped to
one project by construction, so always_on costs only that
project's sessions and the bar is correspondingly lower. A
project rule that names something specific is still ordinarily
always_on — being specific is what project rules are FOR.
Reach for conditional when the rule is about one AREA of a large
project — a CI quirk, a migration gotcha, one subsystem's
convention — so it arrives with that area instead of resident in
every session. The failure to avoid is local: forty always-on
rules on one project reproduces, inside that project, exactly the
preload bloat that made every rule compete for the same budget.
the words someone would type while stuck. Show the moment rather
than classifying it:
RETRIEVES: "the CI job passed locally and fails on the runner
with a permission error"
COLLAPSES: "when touching CI config"
See create_rule for the full argument.
system_ids: Ids from list_canonical_systems — the global AREAS this
rule is about. Worth setting even on a project rule: it is what
lets a conditional one surface when the project is working in
@@ -490,7 +496,7 @@ async def create_project_rule(
rule = await rulebooks_svc.create_project_rule(
project_id=project_id, user_id=uid,
title=derived_title, statement=statement, when_to_apply=when_to_apply,
tier=tier, arose_from_id=arose_from_id,
arose_from_id=arose_from_id,
why=why, how_to_apply=how_to_apply, order_index=order_index,
verify_with=verify_with, expires_when=expires_when,
)
@@ -500,15 +506,48 @@ async def create_project_rule(
async def update_rule(
rule_id: int, title: str = "", statement: str = "", when_to_apply: str = "",
why: str = "", how_to_apply: str = "", order_index: int = -1,
tier: str = "", system_ids: list[int] | None = None, arose_from_id: int = 0,
verify_with: str = "", expires_when: str = "",
system_ids: list[int] | None = None, arose_from_id: int = 0,
verify_with: str = "", expires_when: str = "", kind: str = "",
clear_fields: list[str] | None = None,
) -> dict:
"""Update a rule. Empty strings / order_index=-1 leave fields unchanged.
Adding `when_to_apply` and a `tier` to an existing rule is the ordinary way
a rule stops being preloaded into every session and starts arriving when it
is relevant. `system_ids` REPLACES the rule's areas (pass [] to clear).
`kind` here is how a rule BECOMES a preference, and it is a real change of
force rather than a relabelling — so make it deliberately and say so. The
rule keeps its id, its history and its typed edges, which is why this is a
field rather than a new record: everything that cites it by number stays
correct. Ordinary edits to an existing preference belong in
update_preference, which asks for what taught the change.
`when_to_apply` IS HOW A RULE ARRIVES AT ALL. Nothing is preloaded since
milestone 394, so a rule with no trigger is not a quiet rule — it is one
no session will ever be shown. `system_ids` REPLACES the rule's areas
(pass [] to clear), and they decide which PROJECTS a rule binds by area.
RETROFITTING A TRIGGER HAS ITS OWN TRAP, and it is not the one create_rule
warns about. There the field is empty and the instruction is "write one".
Here a trigger usually already EXISTS and reads perfectly well as English —
"during hard debugging", "when reading any request from the operator",
"before starting an action while a previous one is still settling" — so the
honest-looking verdict is that it is fine. It is not. Those three named a
CATEGORY rather than a moment, and a category is not a thing any session
ever types. `rule_document()` puts this field in twice, as the title's
other half and again above the body, so it dominates the vector: a trigger
describing the abstraction collapses the record toward its title and the
rule never arrives. Measured in #3835 across 113 rules, and again in #3855
on six preferences written before this was understood.
So when you touch a rule with an old trigger, re-read it against the query
that would have to match it — the command about to run, the code being
written, the operator's actual message — and rewrite it in that vocabulary
if it does not. Prefer the words someone produces while the rule applies,
including the rationalisation they would be drafting to talk themselves out
of it — that rationalisation is often the only text in existence at the
moment the rule should fire:
RETRIEVES: "catching yourself drafting 'this is small enough to not
count' about a rule you have already read"
COLLAPSES: "when the next action would conflict with a standing rule"
See create_rule for the full argument and the measurement behind it.
TO EMPTY A FIELD, NAME IT: clear_fields=["verify_with"]. Passing "" cannot
do it — "" means "leave this alone" here, which is what lets you update
@@ -537,8 +576,8 @@ async def update_rule(
fields["statement"] = statement
if when_to_apply:
fields["when_to_apply"] = when_to_apply
if tier:
fields["tier"] = tier
if kind:
fields["kind"] = kind
if arose_from_id:
fields["arose_from_id"] = arose_from_id
if why:
@@ -559,6 +598,204 @@ async def update_rule(
return await rulebooks_svc.rule_detail(uid, rule, system_ids)
# ── Preferences ─────────────────────────────────────────────────────────
#
# Separate tools rather than a `kind=` argument on create_rule, and the reason
# is the docstring rather than the data. create_rule's docstring IS the
# approval gate: it tells its caller to propose, offer three answers, and
# wait. A preference reached through that door would be read through that
# prose, and the caller would hesitate over exactly the act this kind exists
# to make routine. Two doors, two contracts, one table.
#
# Reads stay shared — get_rule and list_rules return preferences as they are,
# because a preference IS a rule row and a reader asking "what governs this"
# wants both. Only the WRITE contracts differ.
async def create_preference(
topic_id: int, title: str, statement: str, when_to_apply: str,
arose_from_id: int, why: str = "", how_to_apply: str = "",
order_index: int = 0, force: bool = False,
) -> dict:
"""Record how the operator wants work done. No approval loop — write it.
A PREFERENCE IS NOT A RULE, and the axis is force rather than importance:
* a RULE is what must be FOLLOWED — ignoring it breaks something or
crosses a boundary. It is the operator's decision, so create_rule
proposes and waits for them.
* a PREFERENCE is how they want it DONE — ignoring it costs consistency,
not correctness. Noticing one and recording it is ordinary work.
If the answer to "what happens if someone doesn't do this" is "something
breaks", you are holding a rule: propose it with create_rule instead.
WHY IT IS WORTH RECORDING AT ALL. A preference stated in one session dies
with that session, and the next one re-derives it or asks again. The point
is consistency: the tenth time you do something it goes the way the ninth
did, without the operator having to say so a tenth time.
`when_to_apply` IS REQUIRED, and not as ceremony. A rule's trigger is
two-thirds of its embedded document, so a preference without one is a
record that will never surface at the moment it applies — written,
findable by nobody, and silently useless. Name the moment in the words a
session would actually be producing then: the command it is about to run,
the code it is writing, the thing the operator just asked for.
Show the moment rather than classifying it:
RETRIEVES: "the operator pasted a stack trace and said it is still
broken"
COLLAPSES: "during hard debugging"
The second is a category, and no session ever produces a category — it
produces the command, the error text, the half-formed ask. A trigger
naming the abstraction collapses the record toward its title and it
never arrives. Both of those describe the same preference; only one of
them can be found at the moment it applies.
`arose_from_id` IS REQUIRED for the same kind of reason. A preference is
expected to change as the work teaches it, and a corpus that drifts with
no record of what taught each change is one nobody can audit. Point it at
the task or note where this became clear.
WHAT A PREFERENCE NEVER DOES: change what gets RECORDED. It shapes how
work is done — pacing, phrasing, which tool to reach for, how much to
check first. A dev-log, an issue and a snippet read the same whoever
produced them, because the record has to outlive the person and their
preferences.
A near-duplicate BLOCKS and returns the existing id. That is the whole
reason this corpus can stay small while being written freely: the second
preference about a thing UPDATES the first rather than sitting beside it,
and two preferences that quietly disagree are worse than none — retrieval
surfaces whichever scores higher and nobody learns the other exists. The
gate is title-based within the topic and does not care about kind, so it
also catches a preference restating a rule that already binds.
Args:
topic_id: The rulebook topic to file it under. A preference is
user-scoped: it follows the operator across every project, which
is what separates it from a project rule.
title: What the preference is about. Half the embedded document —
worth as much care as the statement.
statement: How the operator wants it done, in their terms.
when_to_apply: The moment it applies. Required; see above.
arose_from_id: The task or note that taught this. Required; see above.
force: Bypass the near-duplicate gate. For a genuinely distinct
preference, not for one that is "mostly" different — a mostly
different preference is an update.
"""
uid = current_user_id()
if not when_to_apply.strip():
raise ValueError(
"when_to_apply is required: a preference with no trigger never "
"surfaces at the moment it applies. Name that moment in the words "
"a session would be producing then."
)
if not arose_from_id:
raise ValueError(
"arose_from_id is required: preferences change as the work teaches "
"them, and a change with no record of what taught it cannot be "
"audited. Pass the task or note where this became clear."
)
if not force:
dup = await dedup_svc.find_duplicate_rule(title, topic_id=topic_id)
if dup is not None:
return dedup_svc.duplicate_response(dup, "rule")
rule = await rulebooks_svc.create_rule(
topic_id=topic_id, user_id=uid,
title=title, statement=statement, when_to_apply=when_to_apply,
kind="preference", arose_from_id=arose_from_id,
why=why, how_to_apply=how_to_apply, order_index=order_index,
)
return await rulebooks_svc.rule_detail(uid, rule, None)
async def update_preference(
rule_id: int, arose_from_id: int, statement: str = "",
when_to_apply: str = "", title: str = "", why: str = "",
how_to_apply: str = "", order_index: int = -1,
system_ids: list[int] | None = None, clear_fields: list[str] | None = None,
) -> dict:
"""Bring a preference up to date. Doing this mid-work is expected.
THIS IS THE TOOL THAT MAKES A PREFERENCE DIFFERENT FROM A RULE. A rule
waits for its author; a preference is kept current by whoever is working.
When the operator corrects you, or you notice the preference on file no
longer matches how they actually want this done, edit it — that is the
feature, not a liberty being taken. A preference nothing ever updates has
become a rule nobody enforces.
So: no proposal, no three answers, no waiting. Update it and say in the
conversation that you did, so the operator can disagree while it is still
in front of them.
`arose_from_id` IS REQUIRED, and it is the price of the ungated write.
Every edit here is versioned, and the operator can read what changed and
put it back — but a diff with no reason attached leaves them deciding
whether to trust a change they cannot account for. Point at the task or
note that taught it.
WHEN NOT TO EDIT. If what you learned is that something MUST be done a
certain way — that skipping it breaks something or crosses a boundary —
that is a rule, and rules are the operator's call: propose it with
create_rule rather than hardening a preference in place. Softening in the
other direction is equally an edit worth flagging out loud.
EDITING `when_to_apply` IS THE HIGHEST-LEVERAGE EDIT HERE, and the easiest
to skip, because a preference's trigger is load-bearing in a way a rule's
is not. Preferences get a RESERVED slot at the prompt boundary, filled by a
kind-filtered query at limit=1 — so the corpus does not merely rank against
rules, it ranks against ITSELF, and the trigger is almost all of what
separates one preference from the next. Six preferences whose triggers all
named a category ("during hard debugging", "when reading any request from
the operator") made that slot pick close to arbitrarily on every prompt.
So whenever you are here for any reason, read the trigger against the
operator's message that should have summoned it. If it describes a
situation rather than quoting the moment, rewrite it in the words they
actually type — and in the words YOU would be producing while about to get
this wrong:
RETRIEVES: "the operator said 'clean this up' or 'make it work like',
naming an outcome rather than a change"
COLLAPSES: "when reading any request from the operator"
update_rule carries the full argument.
Empty strings leave fields unchanged; clear_fields empties them by name,
exactly as update_rule does.
Args:
rule_id: The preference to update.
arose_from_id: What taught this change. Required; see above.
when_to_apply: The moment it applies, in session vocabulary. See above.
"""
uid = current_user_id()
if not arose_from_id:
raise ValueError(
"arose_from_id is required: this edit is the record of how the "
"operator's preference changed, and a change with no reason "
"attached cannot be judged. Pass the task or note that taught it."
)
fields: dict = {"arose_from_id": arose_from_id}
if title:
fields["title"] = title
if statement:
fields["statement"] = statement
if when_to_apply:
fields["when_to_apply"] = when_to_apply
if why:
fields["why"] = why
if how_to_apply:
fields["how_to_apply"] = how_to_apply
if order_index >= 0:
fields["order_index"] = order_index
rule = await rulebooks_svc.update_rule(
rule_id, uid, clear=clear_fields or (), **fields,
)
if rule is None:
raise ValueError(f"rule {rule_id} not found")
return await rulebooks_svc.rule_detail(uid, rule, system_ids)
async def rule_history(rule_id: int, version_id: int = 0) -> dict:
"""What a rule USED TO SAY, newest change first.
@@ -653,7 +890,7 @@ async def subscribe_project_to_rulebook(
) -> dict:
"""Subscribe a project to a rulebook — its rules then bind that project.
Subscription is the opt-in path for a non-always_on rulebook: a reusable,
Subscription is the ONLY path for a rulebook (milestone 394): a reusable,
themed module of GENERAL rules shared across the projects that subscribe.
Subscribe a project because it fits the rulebook's theme (e.g. a visual app
-> the design-system rulebook), not to host rules about this one project —
@@ -679,34 +916,6 @@ async def unsubscribe_project_from_rulebook(
# ── Suppressions — project-level mute of rulebook rules / topics ────────
async def exclude_always_on_rulebook(project_id: int, rulebook_id: int) -> dict:
"""Opt a project OUT of a whole always-on rulebook (milestone 297).
Always-on rulebooks bind every project implicitly; an inception decision
can say "not this one, not here". The exclusion is total for that project
— list_always_on_rules(project_id), enter_project/get_project rules and
the session-start context all leave it out and name it under
`excluded_always_on`. Owner-only; the rulebook must be always_on (a
subscribed rulebook is left with unsubscribe_project_from_rulebook).
Idempotent; include_always_on_rulebook reverses it. Normally reached via
decide_project_inception, not by hand.
"""
uid = current_user_id()
await rulebooks_svc.exclude_always_on_rulebook_for_project(
project_id=project_id, rulebook_id=rulebook_id, user_id=uid,
)
return {"project_id": project_id, "rulebook_id": rulebook_id, "excluded": True}
async def include_always_on_rulebook(project_id: int, rulebook_id: int) -> dict:
"""Reverse exclude_always_on_rulebook: the always-on rulebook binds this
project again. Idempotent."""
uid = current_user_id()
await rulebooks_svc.include_always_on_rulebook_for_project(
project_id=project_id, rulebook_id=rulebook_id, user_id=uid,
)
return {"project_id": project_id, "rulebook_id": rulebook_id, "excluded": False}
async def suppress_rule_for_project(
project_id: int, rule_id: int,
@@ -762,8 +971,6 @@ async def unsuppress_topic_for_project(
return {"project_id": project_id, "topic_id": topic_id, "suppressed": False}
async def relate_rules(
from_rule_id: int, to_rule_id: int, kind: str, note: str = "",
) -> dict:
@@ -811,7 +1018,7 @@ async def unrelate_rules(relation_id: int) -> dict:
# ── The staleness sweep (milestone 312) ────────────────────────────────
async def rules_due_for_verification(
older_than_days: int = 0, tier: str = "", never_only: bool = False,
older_than_days: int = 0, never_only: bool = False,
) -> dict:
"""Which standing rules assert a FACT that nobody has confirmed lately.
@@ -838,9 +1045,6 @@ async def rules_due_for_verification(
Args:
older_than_days: only rules last verified longer ago than this.
Never-checked rules always qualify. 0 = no age filter.
tier: "always_on" or "conditional" to narrow. An always-on constraint
that has gone false is the expensive kind — it is preloaded into
every session, so a wrong one is wrong everywhere at once.
never_only: only rules nobody has ever verified.
NOT filterable by project, deliberately: a project reaches rules through
@@ -850,7 +1054,7 @@ async def rules_due_for_verification(
"""
uid = current_user_id()
rules = await rulebooks_svc.rules_due_for_verification(
uid, older_than_days=older_than_days, tier=tier, never_only=never_only,
uid, older_than_days=older_than_days, never_only=never_only,
)
return {
"rules": [rulebooks_svc.verification_row(r) for r in rules],
@@ -903,13 +1107,13 @@ def register(mcp) -> None:
for fn in (
list_rulebooks, get_rulebook, create_rulebook, update_rulebook, delete_rulebook,
list_topics, create_topic, update_topic, delete_topic,
list_rules, list_always_on_rules, get_rule,
list_rules, get_rule,
create_rule, create_project_rule, update_rule, delete_rule,
create_preference, update_preference,
relate_rules, unrelate_rules,
subscribe_project_to_rulebook, unsubscribe_project_from_rulebook,
suppress_rule_for_project, unsuppress_rule_for_project,
suppress_topic_for_project, unsuppress_topic_for_project,
exclude_always_on_rulebook, include_always_on_rulebook,
rules_due_for_verification, mark_rule_verified,
rule_history,
):
+166 -10
View File
@@ -40,7 +40,6 @@ async def _search_rules(uid: int, q: str, limit: int) -> dict:
"title": rule.title,
"statement": rule.statement,
"when_to_apply": rule.when_to_apply or "",
"tier": rule.tier,
"why": rule.why or "",
"how_to_apply": rule.how_to_apply or "",
"verify_with": rule.verify_with or "",
@@ -112,6 +111,7 @@ async def search(
return await _search_rules(uid, q, limit)
is_task = {"note": False, "task": True}.get(content_type) # None => any
t0 = time.perf_counter()
report: dict = {}
raw = await semantic_search_notes(
uid, q, limit=limit, is_task=is_task,
project_id=project_id or None,
@@ -119,12 +119,16 @@ async def search(
# An explicit search reaches everything the operator may read, including
# records shared with them one-to-one.
scope="read",
report=report,
)
record_retrieval(
user_id=uid, source="mcp_search", query=q,
threshold=DEFAULT_SIMILARITY_THRESHOLD, limit=limit,
project_id=project_id or None, is_task=is_task, results=raw,
duration_ms=(time.perf_counter() - t0) * 1000.0,
best_available=report.get("best_available_score"),
best_available_id=report.get("best_available_id"),
searched=bool(report.get("searched", True)),
)
owners = await owner_names_for(
{int(note.user_id) for _s, note in raw if note.user_id != uid}
@@ -149,7 +153,9 @@ async def search(
}
async def retrieval_telemetry(days: int = 30) -> dict:
async def retrieval_telemetry(
days: int = 30, near_miss_samples: int = 0,
) -> dict:
"""What the retrieval telemetry says about YOUR surfaces, over a window.
The read half of the loop the ranker's thresholds are meant to be tuned
@@ -158,17 +164,82 @@ async def retrieval_telemetry(days: int = 30) -> dict:
hand-probing the live instance, which is how the last such decision had to
be made.
Two readouts, from the two tables built for them:
Three readouts, from the three tables built for them:
`sources` — per retrieval surface (`auto_inject`, `write_path`,
`mcp_search`, …), from `retrieval_logs`: `calls`, `zero_result_calls`,
`cleared_threshold` (how often the best hit beat the threshold in force for
that call), the `top_score` spread (p10/p50/p90/min/max), `avg_result_count`
and `p90_duration_ms`. THE number to read first is `cleared_threshold`
against `calls`, with the spread beside it: a surface that clears its bar
on nearly every call is either well-tuned or too loose, and p10 says which.
`near_misses`, the `top_score` spread (p10/p50/p90/min/max),
`avg_result_count` and `p90_duration_ms`.
`usage` — from `note_usage_events`, at the per-note grain
THE NUMBER TO READ FIRST IS `near_misses.p90`, AGAINST THE THRESHOLD IN
FORCE FOR THAT SURFACE. It is measured on the calls the BAR turned away —
zero-result calls, minus the ones whose zero was a repeat the reader had
already been shown — using the best score the ranker reached before the bar
rejected it. So it is the one figure here that says something the bar
cannot make true by construction, and `max` is always below the threshold:
an above-bar candidate nobody excluded would have been returned. A bar at 0.72 turning away a stream of 0.71s is set too
high by a hair and the surface is losing hits it should have had. The same
bar turning away 0.30s is working, and the corpus simply had nothing. Both
render as a zero-result call, and nothing else in this readout tells them
apart.
`near_miss_samples` (0-20, default 0) TURNS THE PERCENTILES INTO RECORDS
YOU CAN READ. Each source then carries `near_miss_records`: its highest
scoring declines, each with the `record_id` the bar refused and the `query`
that asked. Reach for it whenever you are about to move a threshold.
THE PERCENTILE CANNOT SETTLE A BAR ON ITS OWN, and this is the whole reason
the parameter exists. `near_misses.p90` says mass is sitting just under the
line; it says nothing about whether that mass is RELEVANT, and those are
different questions. Lowering a bar to where the mass is, without reading
what is there, is choosing a firing rate rather than a quality. Pull-through
cannot referee it either — the injected rule line already carries title and
trigger, so a session can comply without ever calling `get_rule`, which
makes rule pull-through understate usefulness by construction. Reading the
rejected records is the method that actually answers it.
Off by default because it is a LISTING, not a statistic: it is for the
moment you are making a decision, not for every readout.
`near_misses` is `null` when no declining call in the window measured it —
rows written before #3670 shipped cannot know. That is "not measured", not
"nothing came close"; a 0.0 there would be a claim about the corpus
invented out of a caller's silence.
A NULL HERE NOW MEANS ONE THING, which it did not at first. A semantic
search returns nothing three ways WITHOUT having run — an empty query, an
unavailable embedder, and a failed database query — and each used to write
a row indistinguishable from a ranker that declined (#3765). Those calls no
longer write a row at all, on the same reasoning that already keeps a blank
command out of the log: a row there reports a call that never happened and
drags the clear rate down with phantom declines. So a null is "searched,
and nothing came close", and a broken search shows up as a WARNING in the
application log rather than as a quiet zero in here.
THERE IS NO `cleared_threshold` ANY MORE, and if you remember one, that
memory is of a tautology (#3670). The search applies the bar before
returning, so every returned result cleared it by construction and a call
with no results has no score to compare: the field was true exactly when
`result_count > 0`, i.e. it was `calls - zero_result_calls` under a name
that promised a second opinion. `zero_result_calls + cleared_threshold ==
calls` held on all nineteen readings ever taken. The reading procedure
built on it — "clears its bar on nearly every call" — asked you to compare
a number with itself.
CHECK `suppression` BEFORE CONCLUDING ANYTHING FROM `zero_result_calls`. A
zero-result call is two different events wearing one number: the ranker
found nothing above the bar, or it found only what this session had already
been shown. Just the first is evidence about the bar. `suppression` splits
them where the surface can tell — `zero_because_already_shown` comes off
`zero_result_calls` to leave the true ranker declines.
`suppression` is `null` when NO row in the window reported it, and that is
"not measured here", NOT "none suppressed". Surfaces that pass their
exclusions into the search never see what was dropped, so they cannot say.
Do not read a null as a zero: reading an artifact as a measurement is how
this surface got mis-scoped once already (#3311, #3497).
`usage` — NOTES ONLY, from `note_usage_events`, at the per-note grain
`retrieval_logs` cannot be indexed at: `surfaced` (ranked surfacings — a
scored surface CHOSE the record), `ambient` (the rest), `pulled` split into
`pulled_by_agent` / `pulled_by_human`, the distinct-note counts, and
@@ -182,6 +253,84 @@ async def retrieval_telemetry(days: int = 30) -> dict:
tuned against — only by a pull the agent made. Aggregating across the
mcp_/rest_ prefix would silently answer the wrong one.
`usage["by_source"]` — THE number to tune a threshold against, because the
top-level `pull_through` is a corpus average and averages the surfaces
together. Per surface: `notes_surfaced`, `notes_pulled`, `pull_through`,
and `ambient: true` on surfaces whose surfacings were not scored choices
(their ratio is null — "surfaced often, opened never" is not a judgment
about a record nothing chose). Read it as: of the distinct notes THIS
surface put in front of the agent, how many did the agent then open?
It is an UPPER BOUND per surface: a pull records the door it came
through, not the surface that led there, so a note surfaced by two surfaces
and opened once counts for both — attribution would need the session
identity #2085 declined to invent. `by_source_failed: true` means that one
query failed while the rest of the readout stood.
`rule_usage` — the same question for RULES, from `rule_usage_events`:
`surfaced` and `ambient`, `pulled` split into `pulled_by_agent` /
`pulled_by_human`, the distinct-rule counts, and `pull_through` on the same
definition (agent pulls over RANKED surfacings).
A SEPARATE BLOCK, not folded into `usage`, and reading it as one number
with that is the mistake to avoid. The corpora differ by orders of
magnitude — a few dozen eligible rules against thousands of notes — so a
blended ratio would be the note ratio with noise on it and would hide the
rule arm entirely.
`surfaced` VS `ambient` IS THE READING THAT MATTERS HERE. `surfaced` counts
rules a ranker chose — today only the write-path arm — and those are claims
a pull can settle. `ambient` counts BULK DELIVERIES: the SessionStart
preload and the `rules_payload` surfaces
(`enter_project`, `get_project`, `get_milestone`, `start_planning`,
`get_task`), which hand over the whole applicable set at once with nobody
choosing anything. A large `ambient` says the resident set is big and
arrives often — never that it is useful, and never that it is read.
`pull_through` therefore divides by `surfaced` alone. Fold the preload in
and growing the always-on set would depress the arm's measured precision
while trimming it would flatter it, for reasons having nothing to do with
the arm. To judge the PRELOAD instead, compare `ambient` against pulls of
those same rules over time: a resident set surfaced thousands of times and
opened never is the dead-weight signal, one tier up.
Read it against `sources["write_path_rule"]`. That arm was once believed
never to decline — the reading that scoped #3311 — but it was the arm's
`retrieval_logs` row being written only on calls that FOUND something, so
the zeros were missing rather than absent (#3497). Measured since, it
declines the large majority of its calls like any other surface.
EVERY COUNTER BLOCK CARRIES ITS OWN COVERAGE — `complete_from` and
`covers_window`. `complete_from` is when the number became trustworthy:
for one source, its first recorded row; for a section that sums several,
the LATEST of theirs, because a total is complete only once every
contributor was being written. `covers_window: false` means the window
reaches back further than the recording does, so the count is a fraction
of the period it appears to describe.
READ IT BEFORE COMPARING TWO NUMBERS, and especially before comparing
across a deploy. A counter added last week, read over a 30-day window,
reports a real count against an imagined denominator — and the result is
a plausible fraction rather than an obvious zero, which is what makes it
dangerous. That reading cost milestone #379 five steps aimed at a defect
that did not exist.
`covers_window` is null, never false, when nothing was ever recorded:
"no measurement" is not "partial measurement", the same distinction
`suppression`'s null carries a few paragraphs up.
A SOURCE SHOWING `calls: 0` WAS RECORDING AND MADE NO CALLS. `sources`
lists every source the table has ever held, not only those active in the
window, so a surface that stopped firing stays visible rather than
disappearing — being absent is reserved for a source that has never
recorded at all. Its score fields are null, not zero: the calls are a
real observation, the distribution is not one.
`rule_usage_failed: true` means that read failed while the rest of the
readout stood. The counts are still present so a caller can render, but
they are zeros meaning "could not find out", not "nothing happened" — do
not report a pull-through from a block carrying that flag.
Scoped to your own telemetry — a retrieval log records what your agent
asked for, query text included, and is not a shared record kind.
@@ -191,8 +340,15 @@ async def retrieval_telemetry(days: int = 30) -> dict:
Args:
days: window size, default 30. Clamped to at least 1.
near_miss_samples: 0-20, default 0. How many of each source's highest
scoring DECLINES to list by record, with the query that asked.
Pass it when you are about to move a threshold; leave it off
otherwise. See the near-miss section above for why a percentile
alone cannot settle a bar.
"""
return await retrieval_summary(current_user_id(), days=days)
return await retrieval_summary(
current_user_id(), days=days, near_miss_samples=near_miss_samples,
)
def register(mcp) -> None:
+1 -1
View File
@@ -103,7 +103,7 @@ async def get_task(task_id: int) -> dict:
applicable = await rulebooks_svc.get_applicable_rules(
project_id=note.project_id, user_id=uid,
)
data.update(rulebooks_svc.rules_payload(applicable))
data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_task"))
data.update(await access_svc.describe_provenance(uid, note))
# Same reasoning as get_note's record_pulled, and this is the tool where it
# matters MOST: auto-inject ranks kind-blind over a corpus that is
+1
View File
@@ -28,6 +28,7 @@ from scribe.models.invitation import InvitationToken # noqa: E402, F401
from scribe.models.embedding import NoteEmbedding, RuleEmbedding # noqa: E402, F401
from scribe.models.retrieval_log import RetrievalLog # noqa: E402, F401
from scribe.models.note_usage import NoteUsageEvent # noqa: E402, F401
from scribe.models.rule_usage import RuleUsageEvent # noqa: E402, F401
from scribe.models.project import Project # noqa: E402, F401
from scribe.models.milestone import Milestone # noqa: E402, F401
from scribe.models.task_log import TaskLog # noqa: E402, F401
+1 -1
View File
@@ -39,7 +39,7 @@ class Project(Base, TimestampMixin, SoftDeleteMixin):
)
# The inception record (milestone 297): what this project was decided to
# inherit, when, and through which door — {decided_at, decided_by, via,
# choices: {exclude_always_on_rulebooks, subscribe_rulebooks,
# choices: {subscribe_rulebooks,
# design_system_id, seed_systems}}. NULL means nobody has decided yet,
# and enter_project asks; the effects themselves live in the subscription
# / exclusion tables, design_system_id and the project's Systems — this is
+28
View File
@@ -42,8 +42,35 @@ class RetrievalLog(Base):
# False=notes, NULL=any.
is_task: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
result_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
# How many scored hits this call DROPPED because the session had already
# been shown them. NULLABLE, and the null is load-bearing: it means "this
# surface does not report suppression", which must not read as "nothing was
# suppressed". `result_count == 0` alone conflates two different events —
# the ranker found nothing above threshold, and the ranker found something
# the reader already had — and only the first says a threshold is too high.
# Reading a zero as a ranker decline is how #3311 mis-scoped a milestone;
# an unmeasured value that renders as 0 is the same mistake with a nicer
# face, so surfaces that filter INSIDE the search leave this null.
suppressed_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
top_score: Mapped[float | None] = mapped_column(Float, nullable=True)
min_score: Mapped[float | None] = mapped_column(Float, nullable=True)
# The best score the ranker COULD have offered, before the threshold — as
# against `top_score`, which is the best it DID offer. They are equal on
# any call that returned something, and only this one exists on a call
# that returned nothing, which is the only place a bar can be judged from
# (#3670). Null means the caller did not measure it, never "nothing was
# close": a 0.0 there would read as a corpus with no relevant records at
# all, which is an artifact standing in for a measurement.
best_available_score: Mapped[float | None] = mapped_column(Float, nullable=True)
# WHICH record scored that, so a reader can judge what the bar refused
# rather than only how close it came (#3807). Written from the same ranked
# candidate as the score above — the two describing different records would
# be worse than no id at all, because it invites judging the wrong one.
#
# Not a foreign key on purpose: this table spans record types (the rule arms
# store rule ids, the note arms store note ids) and `source` is what says
# which, exactly as `result_ids` has always worked.
best_available_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
# [{"id": int, "score": float, "rank": int}, ...], highest-first.
result_ids: Mapped[list] = mapped_column(JSONB, nullable=False, default=list)
duration_ms: Mapped[float | None] = mapped_column(Float, nullable=True)
@@ -67,6 +94,7 @@ class RetrievalLog(Base):
"project_id": self.project_id,
"is_task": self.is_task,
"result_count": self.result_count,
"suppressed_count": self.suppressed_count,
"top_score": self.top_score,
"min_score": self.min_score,
"result_ids": self.result_ids,
+95
View File
@@ -0,0 +1,95 @@
from sqlalchemy import BigInteger, Index, Text
from sqlalchemy.orm import Mapped, mapped_column
from scribe.models import Base
from scribe.models.base import CreatedAtMixin, iso
SURFACED = "surfaced"
PULLED = "pulled"
class RuleUsageEvent(Base, CreatedAtMixin):
"""One row per time a rule was SURFACED to the agent, or PULLED in full.
The sibling `note_usage_events` has had since 2026-07, third in the line
after `rule_embeddings` and `rule_versions` — and, like those, it exists
because the rule side kept inheriting machinery built for notes and
quietly getting the weaker version of it.
WHY RULES NEED THEIR OWN AND CANNOT SHARE THE NOTE TABLE. Not squeamishness
about a polymorphic column — the row shares no note-specific fields and the
aggregate readout is the same shape, which is the strongest case for
sharing that note #3163 admits. What decides it is IDENTITY AT RESTORE. A
note id and a rule id are different namespaces resolved through different
maps, and `note_usage_events`'s importer maps `note_id` through
`note_id_map` and drops what does not resolve. A rule id parked in that
column would come back from a backup silently reattached to whatever note
happened to take that number — telemetry that is not merely lost but wrong,
and wrong in a way nothing downstream could detect.
WHAT THIS MEASURES, AND WHY IT DID NOT EXIST. The write-path standing-rule
arm is the only retrieval surface in Scribe whose usefulness cannot be
observed — and, not coincidentally, the only one that has never declined to
fire (#3311: 296 calls, zero zero-result, 100% clearing its threshold).
`retrieval_logs` gives it scores; scores say what the ranker thought, never
whether the hint landed. Without a pull counter no install can tune the arm
from evidence, only from the shape of a histogram.
Deliberately FK-FREE on `rule_id` and `user_id`, matching `note_usage_events`,
`retrieval_logs` and `app_logs` — and diverging from `rule_versions`, which
does carry FKs. The difference is what the row is FOR: a version is part of
a rule's history and dies with it, while telemetry outlives the row it
describes. Deleting a rule must not erase the evidence that it was surfaced
forty times and opened never, because that evidence is precisely the case
for having deleted it.
Cells left deliberately empty (note #3163's step 3): no share ACL — rules
have none of their own; no soft delete — nothing restores a telemetry row,
and the table is append-only; no embedding — an event is not a document.
"""
__tablename__ = "rule_usage_events"
# BigInteger throughout, where the note twin uses Integer. `rule_id` has to
# be, since `rules.id` is BigInteger — and once one column is, matching the
# rest costs nothing and keeps the row uniform. A high-churn append-only
# telemetry table is a poor place to discover an id ceiling.
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
user_id: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
rule_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
# 'surfaced' | 'pulled'
event: Mapped[str] = mapped_column(Text, nullable=False)
# Which surface produced it. A CONVENTION, not a fixed vocabulary, and the
# note twin's comment explains why this one deliberately does not enumerate
# its members: the previous such list went stale, naming a source nothing
# wrote while omitting ones that existed, and a half-true enumeration reads
# as authoritative in exactly the way that misleads (#2476).
# `grep -rn record_rule_pulled\|record_rule_surfaced src/` is the
# authoritative list, and unlike a comment it cannot drift.
#
# The mcp_/rest_ prefix split is load-bearing here for the same reason it is
# for notes, and more so: "is this rule dead weight?" is served by any pull,
# but "did that injected hint land?" — the question this arm exists to
# answer — is served by AGENT pulls only. Never aggregate across the prefix
# without saying why.
source: Mapped[str] = mapped_column(Text, nullable=False)
__table_args__ = (
# Every readout is "these rule ids, split by event" — a covering
# composite beats separate single-column indexes for it.
Index("ix_rule_usage_rule_event", "rule_id", "event"),
Index("ix_rule_usage_created_at", "created_at"),
Index("ix_rule_usage_user_id", "user_id"),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"created_at": iso(self.created_at),
"user_id": self.user_id,
"rule_id": self.rule_id,
"event": self.event,
"source": self.source,
}
+7 -2
View File
@@ -58,7 +58,12 @@ class RuleVersion(Base, CreatedAtMixin):
why: Mapped[str | None] = mapped_column(Text, nullable=True)
how_to_apply: Mapped[str | None] = mapped_column(Text, nullable=True)
when_to_apply: Mapped[str | None] = mapped_column(Text, nullable=True)
tier: Mapped[str | None] = mapped_column(Text, nullable=True)
# Carried so that a change of FORCE leaves a trace. `record_if_changed`
# snapshots only the fields a version holds, so a kind omitted here would
# make "this stopped binding" the one edit with no history behind it.
# NULL means a version older than migration 0098 — not recorded, never
# "was a rule".
kind: Mapped[str | None] = mapped_column(Text, nullable=True)
verify_with: Mapped[str | None] = mapped_column(Text, nullable=True)
expires_when: Mapped[str | None] = mapped_column(Text, nullable=True)
@@ -84,7 +89,7 @@ class RuleVersion(Base, CreatedAtMixin):
"why": self.why or "",
"how_to_apply": self.how_to_apply or "",
"when_to_apply": self.when_to_apply or "",
"tier": self.tier or "",
"kind": self.kind or "",
"verify_with": self.verify_with or "",
"expires_when": self.expires_when or "",
})
+37 -25
View File
@@ -19,9 +19,6 @@ class Rulebook(Base, TimestampMixin, SoftDeleteMixin):
)
title: Mapped[str] = mapped_column(Text)
description: Mapped[str | None] = mapped_column(Text, nullable=True)
always_on: Mapped[bool] = mapped_column(
Boolean, default=False, nullable=False, server_default="false"
)
def to_dict(self) -> dict:
return {
@@ -29,7 +26,6 @@ class Rulebook(Base, TimestampMixin, SoftDeleteMixin):
"owner_user_id": self.owner_user_id,
"title": self.title,
"description": self.description or "",
"always_on": self.always_on,
"created_at": iso(self.created_at),
"updated_at": iso(self.updated_at),
}
@@ -96,15 +92,32 @@ class Rule(Base, TimestampMixin, SoftDeleteMixin):
# WHEN this rule applies — the trigger, not the instruction. Required of
# new rules at the service layer and nullable here, because rules written
# before migration 0088 have none and a migration cannot invent one.
# It carries three jobs at once (note 3026): it is the tier test made
# concrete, the readable form of the canon tag, and the half of the
# document that makes a rule findable by meaning.
# It carries three jobs at once (note 3026): it is the readable form of
# the canon tag, the half of the document that makes a rule findable by
# meaning, and — since milestone 394 removed the always-on tier — the ONLY
# thing that decides whether a rule ever reaches a session at all. A rule
# with no trigger is not a quiet rule, it is an unreachable one.
when_to_apply: Mapped[str | None] = mapped_column(Text, nullable=True)
# always_on = preloaded into every session, as every rule is today.
# conditional = reachable, and surfaced when its trigger fires. The
# default preserves existing behaviour exactly: nothing stops binding
# because of an upgrade. CHECK ck_rules_tier (migration 0088, rule 36).
tier: Mapped[str] = mapped_column(Text, default="always_on", server_default="always_on")
# WHAT KIND of instruction this is. `tier` used to sit beside this and
# carry delivery; milestone 394 removed it, so kind is now the only axis
# on a rule and delivery belongs entirely to retrieval.
# `rule` must be FOLLOWED: ignoring it breaks something or crosses a
# boundary. `preference` is how this person wants work DONE: ignoring it
# costs consistency, not correctness.
#
# The second half is what makes it a kind rather than a softer label — a
# preference is expected to CHANGE as the work teaches it, and the agent
# updates it in the ordinary course of working, where a rule waits for its
# author. So one column decides two behaviours: whether create's approval
# gate fires, and which voice the injected line speaks in.
#
# Lives here and not in its own table because a preference needs exactly
# what a rule has and a note does not — a trigger column, a
# trigger-dominated document, ownership-scoped search, the retrieval arms,
# relations, and `rule_versions`, which is where its drift is recorded.
# Defaults to `rule` so nothing changes force on upgrade.
# CHECK ck_rules_kind (migration 0098, rule 36).
kind: Mapped[str] = mapped_column(Text, default="rule", server_default="rule")
why: Mapped[str | None] = mapped_column(Text, nullable=True)
how_to_apply: Mapped[str | None] = mapped_column(Text, nullable=True)
# The three fields that tell a CONSTRAINT apart from a NORM (milestone
@@ -143,7 +156,13 @@ class Rule(Base, TimestampMixin, SoftDeleteMixin):
"title": self.title,
"statement": self.statement,
"when_to_apply": self.when_to_apply or "",
"tier": self.tier,
# Unconditional, unlike the `if present` keys below. A reader
# deciding how much force a record carries must never infer it
# from an ABSENT key: "no kind field" and "kind is rule" would be
# the same payload, and that equivalence is the defect shape this
# codebase keeps re-encountering. Twenty bytes buys an answer that
# cannot be misread.
"kind": self.kind or "rule",
"why": self.why or "",
"how_to_apply": self.how_to_apply or "",
"verify_with": self.verify_with or "",
@@ -237,18 +256,11 @@ project_rule_suppressions = Table(
Column("created_at", DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)),
)
# A project's opt-out of a whole ALWAYS-ON rulebook (milestone 297): the
# sibling of the two suppression tables below, one level up. Always-on
# rulebooks bind every project implicitly; an inception decision can exclude
# specific ones for this project, and get_applicable_rules /
# list_always_on_rules(project_id) skip them. FKs CASCADE like the others.
project_rulebook_exclusions = Table(
"project_rulebook_exclusions",
Base.metadata,
Column("project_id", BigInteger, ForeignKey("projects.id", ondelete="CASCADE"), primary_key=True),
Column("rulebook_id", BigInteger, ForeignKey("rulebooks.id", ondelete="CASCADE"), primary_key=True),
Column("created_at", DateTime(timezone=True), default=lambda: datetime.now(timezone.utc)),
)
# `project_rulebook_exclusions` lived here until milestone 394. It recorded a
# project's opt-out of a whole always-on rulebook — which only made sense
# while a rulebook could bind a project WITHOUT being asked. Subscription is
# now the only reach a rulebook has, so declining one is expressed by not
# subscribing, and there is nothing left to opt out of.
project_topic_suppressions = Table(
"project_topic_suppressions",
+56 -1
View File
@@ -10,6 +10,61 @@ async def health():
return jsonify({"status": "ok"})
def build_version_payload() -> dict:
"""What build is this, separated into the values that answer different
questions (rule 149).
UNTIL 2026-08-31 THIS RETURNED THE CHANNEL. `BUILD_VERSION` in CI was
literally "dev" / "main" / the tag, so a running instance reported
`{"version": "main"}` — a channel name sitting where a build identifier
belongs. The cost was concrete: with a deploy misbehaving, nothing on the
instance could say which commit was serving it, and the one endpoint whose
job that is answered with the name of a branch.
The three values, and why they are three:
- `version` — the NAME, `YYYY.MM.DD.HHMM` from COMMIT time. Answers "is
this the same code?", so two channels carrying one commit report the
same string.
- `build` — the ORDERING KEY, minutes since 2020-01-01 from BUILD time.
Answers "may this be installed over that?". The ONLY value anything may
compare; it is monotonic by construction, which neither a commit count
(branches diverge) nor a commit time (rebuilds go backwards) is.
- `channel` — its own field, never folded into the name.
Plus `commit`, so the artifact's claim about itself can be checked against
the `:<sha>` it was published under (rule 145).
ABSENT RATHER THAN EMPTY when unknown. A local build has no ordering key
and no channel, and saying so is honest; emitting `""` or a placeholder
would let it claim a position in an update order it is not part of. A
reader must treat a missing `build` as "cannot be ordered", not as zero.
"""
payload: dict = {"version": os.environ.get("APP_VERSION", "dev")}
# Reported verbatim, never validated against an enum — a build claiming
# something unexpected is better shown than dropped (rule 149).
for key, env in (("channel", "APP_CHANNEL"), ("commit", "APP_COMMIT")):
value = (os.environ.get(env) or "").strip()
if value:
payload[key] = value
raw_key = (os.environ.get("APP_BUILD_KEY") or "").strip()
if raw_key:
try:
# An INTEGER, not a string. A string ordering key is how a
# comparison silently becomes lexicographic — "9" > "10" — which
# is the same class of fault as folding the channel in: it reads
# fine and orders wrong.
payload["build"] = int(raw_key)
except ValueError:
# A malformed key is omitted rather than passed through: a reader
# that cannot order is correct, one that orders on garbage is not.
pass
return payload
@api.route("/version")
async def version():
return jsonify({"version": os.environ.get("APP_VERSION", "dev")})
return jsonify(build_version_payload())
+64 -7
View File
@@ -91,13 +91,77 @@ async def autoinject_retrieve():
project_id (opt) — explicit project scope override (ad-hoc/testing).
exclude_ids (opt) — comma-separated note ids already injected this
session; skipped so each note injects at most once.
exclude_rule_ids — comma-separated rule ids already surfaced this
(opt) session. SHARED with /prior-art and /tool-rules on
purpose: one session keeps ONE rule ledger, so a
rule named by any arm is not re-announced by
another. Ages out (#3751), so salience decays.
TWO ARMS, TWO SETS OF GATES. Rules ride the same hook and the same query
but nothing else: the notes menu can be disabled, thresholded and top-k'd
by the operator without touching whether a rule reaches them. Composed
here rather than inside either builder so neither one's early return can
silently suppress the other.
Rules come FIRST in the payload. A rule or preference governing the answer
is more consequential than a menu of things that might be worth reading,
and a reader who stops after the first block should have stopped after
the right one.
"""
q = (request.args.get("q") or "").strip()
project_id, _repo, _unbound = await _project_scope()
exclude_ids = _int_list(request.args.get("exclude_ids"))
exclude_rule_ids = _int_list(request.args.get("exclude_rule_ids"))
rules = await plugin_ctx_svc.build_prompt_rule_hint(
g.user.id, q, project_id=project_id, exclude_rule_ids=exclude_rule_ids
)
result = await plugin_ctx_svc.build_autoinject_hint(
g.user.id, q, project_id=project_id, exclude_ids=exclude_ids
)
blocks = [b for b in (rules["context"], result["context"]) if b]
result["context"] = "\n\n".join(blocks)
result["rule_ids"] = rules["rule_ids"]
return jsonify(result)
@plugin_bp.get("/tool-rules")
@login_required
async def pre_tool_rules():
"""Standing rules for the plugin's PreToolUse hook on ACTIONS (#3476).
Answers "does a recorded rule speak to the command about to be run?" — the
sibling of /prior-art, which can only answer that question about a code
write. Rules about which tool to reach for (don't curl the forge, don't
stand up a stack, don't run the suite locally) had no retrieval surface at
all before this, which is why they all had to live in the resident preload.
Titles + trigger only, never the statement: the hint says a rule may apply
and hands over `get_rule(id)`. One rule at most (RULEHINT_LIMIT), and empty
most of the time.
Query:
tool (str) — the tool about to run, e.g. `Bash`. Used in
the hint's wording, not in the search: a
rule is about the action, not the harness.
command (str) — the command about to run; the semantic query.
Absent or blank → empty, no search.
repo (optional) — working repo remote, resolved to the bound
project exactly as /retrieve and /prior-art.
exclude_rule_ids (opt) — comma-separated rule ids already surfaced
this session. SHARED with /prior-art's
ledger on purpose: one session keeps one
list, so a rule named by either arm is not
re-offered by the other.
"""
tool = (request.args.get("tool") or "tool").strip()
command = request.args.get("command") or ""
project_id, _repo, _unbound = await _project_scope()
exclude_rule_ids = _int_list(request.args.get("exclude_rule_ids"))
result = await plugin_ctx_svc.build_tool_rule_hint(
g.user.id, tool, command,
project_id=project_id, exclude_rule_ids=exclude_rule_ids,
)
return jsonify(result)
@@ -138,11 +202,6 @@ async def write_path_prior_art():
or `canon:<snippet_id>`) already named this
session by the ledger arm (#2900); its own
channel, like the two above.
rules_etag (opt) — the marker the session was given when it loaded
its always-on rules (milestone 323). Sent back
so the server can say whether those rules have
MOVED since. Absent means the hook has nothing
stored, which is silence, not a mismatch.
shapes (opt) — comma-separated `kind:name` definitions the hook
found in (or enclosing) the payload, kind being
css|sym. The shape ledger's write-path feed
@@ -162,7 +221,6 @@ async def write_path_prior_art():
p.strip() for p in (request.args.get("exclude_derive") or "").split(",") if p.strip()
]
exclude_rule_ids = _int_list(request.args.get("exclude_rule_ids"))
rules_etag = (request.args.get("rules_etag") or "").strip()
shapes = _parse_shapes(request.args.get("shapes") or "")
api_key = getattr(g, "api_key", None)
may_stamp = api_key is None or getattr(api_key, "scope", "") == "write"
@@ -174,7 +232,6 @@ async def write_path_prior_art():
repo_key=repo_bindings_svc.normalize_repo_key(repo) if repo else "",
exclude_derive=exclude_derive,
exclude_rule_ids=exclude_rule_ids,
rules_etag=rules_etag,
)
return jsonify(result)
+1 -1
View File
@@ -99,7 +99,7 @@ async def create_project_route():
@login_required
async def decide_inception_route(project_id: int):
"""Record (or re-record) what a project inherits — milestone 297.
Body: the choices object {exclude_always_on_rulebooks, subscribe_rulebooks,
Body: the choices object {subscribe_rulebooks,
design_system_id, seed_systems}; owner-only."""
uid = get_current_user_id()
data = await request.get_json() or {}
+34 -34
View File
@@ -10,6 +10,9 @@ from quart import Blueprint, jsonify, request
from scribe.auth import get_current_user_id, login_required
import scribe.services.rulebooks as rulebooks_svc
from scribe.services.trash import delete as trash_delete
from scribe.services.rule_usage import (
empty_rule_usage, record_rule_pulled, usage_for_rules,
)
rulebooks_bp = Blueprint("rulebooks", __name__, url_prefix="/api")
@@ -52,7 +55,7 @@ async def get_rulebook(rulebook_id: int):
@login_required
async def update_rulebook(rulebook_id: int):
data = await request.get_json() or {}
fields = {k: v for k, v in data.items() if k in ("title", "description", "always_on")}
fields = {k: v for k, v in data.items() if k in ("title", "description")}
rb = await rulebooks_svc.update_rulebook(rulebook_id, get_current_user_id(), **fields)
if rb is None:
return jsonify({"error": "rulebook not found"}), 404
@@ -136,13 +139,24 @@ async def list_rules():
except ValueError:
return jsonify({"error": "rulebook_id, topic_id, project_id must be integers"}), 400
uid = get_current_user_id()
rows = await rulebooks_svc.list_rules(
user_id=get_current_user_id(),
user_id=uid,
rulebook_id=rulebook_id,
topic_id=topic_id,
project_id=project_id,
)
return jsonify({"rules": [r.to_dict() for r in rows]})
items = [r.to_dict() for r in rows]
# One aggregate for the whole page — a per-row lookup here would be N+1 by
# construction, the same reason the snippet list does it this way. Every
# row gets the key, zero-filled, so the UI renders "never surfaced" rather
# than having to treat a missing field as a state. That matters more here
# than for snippets: every rule on every install predates this table, so
# for a while the zero-filled shape IS the common case.
usage = await usage_for_rules([int(it["id"]) for it in items])
for it in items:
it["usage"] = usage.get(int(it["id"]), empty_rule_usage())
return jsonify({"rules": items})
@rulebooks_bp.post("/rulebook-topics/<int:topic_id>/rules")
@@ -163,7 +177,11 @@ async def create_rule(topic_id: int):
how_to_apply=data.get("how_to_apply", ""),
order_index=data.get("order_index", 0),
when_to_apply=data.get("when_to_apply", ""),
tier=data.get("tier", "always_on"),
# The human door carries `kind` too, and without the MCP door's
# required provenance: an operator editing their own preference
# owes nobody an explanation. That requirement is about auditing
# what the AGENT changed, not what they did themselves.
kind=data.get("kind", "rule"),
arose_from_id=data.get("arose_from_id", 0) or 0,
verify_with=data.get("verify_with", ""),
expires_when=data.get("expires_when", ""),
@@ -182,6 +200,11 @@ async def get_rule(rule_id: int):
rule = await rulebooks_svc.get_rule(rule_id, uid)
if rule is None:
return jsonify({"error": "rule not found"}), 404
# `rest_` rather than `mcp_`, and the prefix is load-bearing: "is this rule
# dead weight?" is served by any pull, but "did that injected hint land?"
# — the question this arm exists to answer — is served by AGENT pulls only.
# A person clicking through the rule list says nothing about the hint.
record_rule_pulled(user_id=uid, rule_id=int(rule.id), source="rest_rule")
return jsonify(await rulebooks_svc.rule_detail(uid, rule))
@@ -193,7 +216,7 @@ async def update_rule(rule_id: int):
fields = {
k: v for k, v in data.items()
if k in ("title", "statement", "why", "how_to_apply", "order_index",
"when_to_apply", "tier", "arose_from_id",
"when_to_apply", "kind", "arose_from_id",
"verify_with", "expires_when")
}
# No clear_fields here: a form sends "" for an emptied input, and the
@@ -371,32 +394,6 @@ async def unsuppress_project_topic(project_id: int, topic_id: int):
return "", 204
@rulebooks_bp.post("/projects/<int:project_id>/exclusions/rulebooks/<int:rulebook_id>")
@login_required
async def exclude_project_rulebook(project_id: int, rulebook_id: int):
"""Opt the project out of a whole always-on rulebook (milestone 297)."""
try:
await rulebooks_svc.exclude_always_on_rulebook_for_project(
project_id=project_id, rulebook_id=rulebook_id, user_id=get_current_user_id(),
)
except ValueError as exc:
msg = str(exc)
return jsonify({"error": msg}), (400 if "not always-on" in msg else 404)
return "", 204
@rulebooks_bp.delete("/projects/<int:project_id>/exclusions/rulebooks/<int:rulebook_id>")
@login_required
async def include_project_rulebook(project_id: int, rulebook_id: int):
try:
await rulebooks_svc.include_always_on_rulebook_for_project(
project_id=project_id, rulebook_id=rulebook_id, user_id=get_current_user_id(),
)
except ValueError as exc:
return jsonify({"error": str(exc)}), 404
return "", 204
@rulebooks_bp.post("/projects/<int:project_id>/rules")
@login_required
async def create_project_rule(project_id: int):
@@ -416,7 +413,11 @@ async def create_project_rule(project_id: int):
how_to_apply=data.get("how_to_apply", ""),
order_index=data.get("order_index", 0),
when_to_apply=data.get("when_to_apply", ""),
tier=data.get("tier", "always_on"),
# The human door carries `kind` too, and without the MCP door's
# required provenance: an operator editing their own preference
# owes nobody an explanation. That requirement is about auditing
# what the AGENT changed, not what they did themselves.
kind=data.get("kind", "rule"),
arose_from_id=data.get("arose_from_id", 0) or 0,
verify_with=data.get("verify_with", ""),
expires_when=data.get("expires_when", ""),
@@ -435,7 +436,7 @@ async def create_project_rule(project_id: int):
async def rules_due_for_verification():
"""Rules that carry a check, oldest verification first, never-checked top.
Query params: older_than_days, tier, never_only. A rule with no
Query params: older_than_days, never_only. A rule with no
`verify_with` never appears — it is a decision, not a fact.
"""
uid = get_current_user_id()
@@ -448,7 +449,6 @@ async def rules_due_for_verification():
rules = await rulebooks_svc.rules_due_for_verification(
uid,
older_than_days=older,
tier=args.get("tier", ""),
never_only=args.get("never_only", "").lower() in ("1", "true", "yes"),
)
except ValueError as exc:
+5
View File
@@ -44,17 +44,22 @@ async def search_route():
project_id = request.args.get("project_id", type=int)
t0 = time.perf_counter()
report: dict = {}
results = await semantic_search_notes(
uid, q, limit=limit, is_task=is_task, threshold=_REST_SEARCH_THRESHOLD,
project_id=project_id, system_id=system_id,
# The user typed this, so it reaches everything they may read.
scope="read",
report=report,
)
record_retrieval(
user_id=uid, source="rest_search", query=q,
threshold=_REST_SEARCH_THRESHOLD, limit=limit,
project_id=project_id, is_task=is_task, results=results,
duration_ms=(time.perf_counter() - t0) * 1000.0,
best_available=report.get("best_available_score"),
best_available_id=report.get("best_available_id"),
searched=bool(report.get("searched", True)),
)
owners = await owner_names_for(
{int(note.user_id) for _s, note in results if note.user_id != uid}
+46 -3
View File
@@ -6,20 +6,63 @@ write that never errors and never lands (the #2663 GC footgun). This module is
the one place that gets the pattern right: strong references in ``_pending``,
discarded on completion, with failures logged at WARNING instead of vanishing.
``note_usage`` and ``retrieval_telemetry`` predate this module and carry their
own copies with bespoke canary semantics; new fire-and-forget callers use this
instead of writing a fourth copy.
``retrieval_telemetry`` predates this module and keeps its own copy, because
its canary is a genuinely different shape — one process-wide flag and no
AppLog row. ``note_usage`` and ``rule_usage`` share ``report_telemetry_failure``
below. New fire-and-forget callers use ``spawn`` rather than writing another
copy of the strong-reference dance.
"""
from __future__ import annotations
import asyncio
import logging
import traceback
from collections.abc import Coroutine
logger = logging.getLogger(__name__)
_pending: set[asyncio.Task] = set()
# Sites that have already dropped their once-per-process AppLog row, keyed
# "<subsystem>:<site>". A readout can run on every list render — without this,
# a broken table turns the error log into a firehose that buries the finding it
# exists to surface.
_reported: set[str] = set()
async def report_telemetry_failure(subsystem: str, site: str) -> None:
"""Make a swallowed telemetry failure visible. Call from an except block.
WARNING to the process log every time; one AppLog error row per process per
(subsystem, site) so the admin UI shows the outage without host access.
THIS IS NOT DECORATION. #2663 is the record of a telemetry subsystem running
at zero for weeks — every counter reading empty, indistinguishable from
"nobody uses this" — because every failure went to ``logger.debug``. A
subsystem whose failures are all invisible cannot report its own death.
The AppLog write is itself guarded: when the database is down it fails too,
and that is fine. The WARNING already said so, and a canary must never take
down the surface it watches.
"""
logger.warning("%s telemetry %s failed", subsystem, site, exc_info=True)
key = f"{subsystem}:{site}"
if key in _reported:
return
_reported.add(key)
try:
from scribe.services.logging import log_error
await log_error(
endpoint=subsystem,
error_type=f"{subsystem}_{site}_failed",
error_message=f"{subsystem} telemetry {site} is failing; "
"usage counters will read zero until this is fixed",
traceback=traceback.format_exc(),
)
except Exception:
logger.debug("%s canary write failed", subsystem, exc_info=True)
def spawn(coro: Coroutine, *, site: str) -> None:
"""Schedule ``coro`` fire-and-forget; ``site`` names it in failure logs.
+90 -38
View File
@@ -12,6 +12,7 @@ from scribe.models.note_version import NoteVersion
from scribe.models.rule_version import RuleVersion
from scribe.models.design_system import DesignSystem, DesignToken
from scribe.models.note_usage import NoteUsageEvent
from scribe.models.rule_usage import RuleUsageEvent
from scribe.models.canonical_system import CanonicalSystem
from scribe.models.rulebook import RuleRelation, rule_systems as rule_systems_t
from scribe.models.code_shape import CodeShape, CodeShapeEvent, CodeShapeUse
@@ -22,7 +23,6 @@ from scribe.models.rulebook import (
Rulebook,
RulebookTopic,
project_rule_suppressions,
project_rulebook_exclusions,
project_rulebook_subscriptions,
project_topic_suppressions,
)
@@ -62,8 +62,12 @@ logger = logging.getLogger(__name__)
# _COLUMN_EXCLUSIONS and its guard landed with it, so the next such column
# fails the build instead.
# v13 (2026-08) added rule_versions — a rule's edit history (milestone 323).
# v14 (2026-09) added rule_usage_events — the rule twin of note_usage_events
# (milestone 333). Carrying it is the WHOLE REASON the table is separate: the
# note importer maps note_id through note_id_map, so a rule id parked there
# would restore attached to whatever note took that number.
# Bump when the serialized schema changes.
BACKUP_VERSION = 13
BACKUP_VERSION = 14
# Every table this backup carries, by its REAL name. Paired with _NOT_INCLUDED
# below, these two lists must together account for the entire schema — which is
@@ -77,7 +81,7 @@ _BACKED_UP = [
"users", "projects", "milestones", "notes", "task_logs", "note_drafts",
"note_versions", "settings", "rulebooks", "rulebook_topics", "rules",
"project_rulebook_subscriptions", "project_rule_suppressions",
"project_topic_suppressions", "project_rulebook_exclusions",
"project_topic_suppressions",
# v5 (2026-08): the five-year gap this list was written to stop.
"systems", "record_systems", "design_systems", "design_tokens",
"note_usage_events", "repo_bindings", "note_supersessions",
@@ -92,6 +96,11 @@ _BACKED_UP = [
# v13 (2026-08): a rule's edit history (milestone 323). note_versions has
# always travelled; its sibling has no excuse not to.
"rule_versions",
# v14 (2026-09): rule usage telemetry (milestone 333). Same argument
# note_usage_events makes for itself — pull-through is only ever
# accumulated, so a restore that dropped it would silently reset the
# measurement to zero while everything still looked fine.
"rule_usage_events",
]
# Tables intentionally NOT in the backup, surfaced in the payload so the gap is
@@ -178,6 +187,8 @@ _COLUMN_EXCLUSIONS: dict[str, set[str]] = {
"note_supersessions": {"id", "created_at"},
"rule_relations": {"id", "created_at"},
"note_usage_events": {"id"},
# Same as the note twin: the surrogate key is re-issued on insert.
"rule_usage_events": {"id"},
"design_systems": {"deleted_at", "deleted_batch_id", "created_at", "updated_at"},
"design_tokens": {"deleted_at", "deleted_batch_id", "created_at", "updated_at"},
"repo_bindings": {"id", "created_at", "updated_at"},
@@ -235,8 +246,6 @@ def _topic_suppression_rows(rows) -> list[dict]:
return [{"project_id": r.project_id, "topic_id": r.topic_id} for r in rows]
def _rulebook_exclusion_rows(rows) -> list[dict]:
return [{"project_id": r.project_id, "rulebook_id": r.rulebook_id} for r in rows]
# The v5 sections. Pure row-builders like the join-table helpers above, for the
@@ -321,6 +330,17 @@ def _usage_event_rows(rows) -> list[dict]:
]
def _rule_usage_event_rows(rows) -> list[dict]:
return [
{
"user_id": r.user_id, "rule_id": r.rule_id, "event": r.event,
"source": r.source,
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _code_shape_rows(rows) -> list[dict]:
return [r.to_dict() for r in rows]
@@ -490,7 +510,7 @@ def _rule_version_rows(rows) -> list[dict]:
"id": rv.id, "rule_id": rv.rule_id, "user_id": rv.user_id,
"title": rv.title, "statement": rv.statement, "why": rv.why,
"how_to_apply": rv.how_to_apply, "when_to_apply": rv.when_to_apply,
"tier": rv.tier, "verify_with": rv.verify_with,
"kind": rv.kind, "verify_with": rv.verify_with,
"expires_when": rv.expires_when,
"created_at": rv.created_at.isoformat(),
}
@@ -506,7 +526,7 @@ def _rulebook_rows(rows) -> list[dict]:
return [
{
"id": rb.id, "owner_user_id": rb.owner_user_id, "title": rb.title,
"description": rb.description, "always_on": rb.always_on,
"description": rb.description,
"created_at": rb.created_at.isoformat(),
"updated_at": rb.updated_at.isoformat(),
}
@@ -552,7 +572,7 @@ def _rule_rows(rows) -> list[dict]:
"id": r.id, "topic_id": r.topic_id, "project_id": r.project_id,
"title": r.title, "statement": r.statement, "why": r.why,
"how_to_apply": r.how_to_apply, "order_index": r.order_index,
"when_to_apply": r.when_to_apply, "tier": r.tier,
"when_to_apply": r.when_to_apply, "kind": r.kind,
"verify_with": r.verify_with, "expires_when": r.expires_when,
"verified_at": r.verified_at.isoformat() if r.verified_at else None,
"arose_from_id": r.arose_from_id,
@@ -606,6 +626,9 @@ async def export_full_backup() -> dict:
)).scalars().all()
design_tokens = (await session.execute(select(DesignToken))).scalars().all()
usage_events = (await session.execute(select(NoteUsageEvent))).scalars().all()
rule_usage_events = (
await session.execute(select(RuleUsageEvent))
).scalars().all()
repo_bindings = (await session.execute(select(RepoBinding))).scalars().all()
code_shapes = (await session.execute(select(CodeShape))).scalars().all()
code_shape_events = (await session.execute(
@@ -626,9 +649,6 @@ async def export_full_backup() -> dict:
topic_suppressions = (await session.execute(
select(project_topic_suppressions)
)).all()
rulebook_exclusions = (await session.execute(
select(project_rulebook_exclusions)
)).all()
return {
"version": BACKUP_VERSION,
@@ -654,7 +674,6 @@ async def export_full_backup() -> dict:
"rulebook_subscriptions": _subscription_rows(subscriptions),
"rule_suppressions": _rule_suppression_rows(rule_suppressions),
"topic_suppressions": _topic_suppression_rows(topic_suppressions),
"rulebook_exclusions": _rulebook_exclusion_rows(rulebook_exclusions),
"canonical_systems": _canonical_system_rows(canonical_systems),
"rule_systems": _rule_system_rows(rule_system_rows),
"rule_relations": _rule_relation_rows(rule_relations),
@@ -665,6 +684,7 @@ async def export_full_backup() -> dict:
"design_systems": _design_system_rows(design_systems),
"design_tokens": _design_token_rows(design_tokens),
"note_usage_events": _usage_event_rows(usage_events),
"rule_usage_events": _rule_usage_event_rows(rule_usage_events),
"repo_bindings": _repo_binding_rows(repo_bindings),
"note_supersessions": _note_supersession_rows(supersessions),
"code_shapes": _code_shape_rows(code_shapes),
@@ -791,6 +811,14 @@ async def export_user_backup(user_id: int) -> dict:
select(RuleVersion).where(RuleVersion.rule_id.in_(_rule_ids))
.order_by(RuleVersion.rule_id, RuleVersion.id)
)).scalars().all() if _rule_ids else []
# Scoped through the RULE for the same reason the versions above are,
# and it is worth restating because the column that looks right is
# wrong: `user_id` here is whoever the arm fired FOR, not who owns the
# rule. Filtering on it would carry this user's surfacings of someone
# ELSE's rule and drop the ones fired for someone else on theirs.
rule_usage_events = (await session.execute(
select(RuleUsageEvent).where(RuleUsageEvent.rule_id.in_(_rule_ids))
)).scalars().all() if _rule_ids else []
rule_relations = (await session.execute(
select(RuleRelation).where(
RuleRelation.from_rule_id.in_(_rule_ids),
@@ -813,13 +841,8 @@ async def export_user_backup(user_id: int) -> dict:
project_topic_suppressions.c.project_id.in_(project_ids)
)
)).all()
rulebook_exclusions = (await session.execute(
select(project_rulebook_exclusions).where(
project_rulebook_exclusions.c.project_id.in_(project_ids)
)
)).all()
else:
subscriptions = rule_suppressions = topic_suppressions = rulebook_exclusions = []
subscriptions = rule_suppressions = topic_suppressions = []
return {
"version": BACKUP_VERSION,
@@ -847,7 +870,6 @@ async def export_user_backup(user_id: int) -> dict:
"rulebook_subscriptions": _subscription_rows(subscriptions),
"rule_suppressions": _rule_suppression_rows(rule_suppressions),
"topic_suppressions": _topic_suppression_rows(topic_suppressions),
"rulebook_exclusions": _rulebook_exclusion_rows(rulebook_exclusions),
"canonical_systems": _canonical_system_rows(canonical_systems),
"rule_systems": _rule_system_rows(rule_system_rows),
"rule_relations": _rule_relation_rows(rule_relations),
@@ -858,6 +880,7 @@ async def export_user_backup(user_id: int) -> dict:
"design_systems": _design_system_rows(design_systems),
"design_tokens": _design_token_rows(design_tokens),
"note_usage_events": _usage_event_rows(usage_events),
"rule_usage_events": _rule_usage_event_rows(rule_usage_events),
"repo_bindings": _repo_binding_rows(repo_bindings),
"note_supersessions": _note_supersession_rows(supersessions),
"code_shapes": _code_shape_rows(code_shapes),
@@ -992,9 +1015,10 @@ async def _restore_v2(data: dict) -> dict:
"task_logs": 0, "note_drafts": 0, "note_versions": 0,
"settings": 0, "rulebooks": 0, "rulebook_topics": 0, "rules": 0,
"rulebook_subscriptions": 0, "rule_suppressions": 0,
"topic_suppressions": 0, "rulebook_exclusions": 0,
"topic_suppressions": 0,
"systems": 0, "record_systems": 0, "design_systems": 0,
"design_tokens": 0, "note_usage_events": 0, "repo_bindings": 0,
"design_tokens": 0, "note_usage_events": 0, "rule_usage_events": 0,
"repo_bindings": 0,
"note_supersessions": 0, "code_shapes": 0, "code_shape_events": 0,
"code_shape_uses": 0, "canonical_systems": 0,
"rule_systems": 0, "rule_relations": 0, "rule_versions": 0,
@@ -1201,7 +1225,6 @@ async def _restore_v2(data: dict) -> dict:
owner_user_id=mapped_uid,
title=rb_data.get("title", ""),
description=rb_data.get("description", ""),
always_on=rb_data.get("always_on", False),
created_at=_dt(rb_data.get("created_at")),
updated_at=_dt(rb_data.get("updated_at")),
)
@@ -1242,10 +1265,19 @@ async def _restore_v2(data: dict) -> dict:
why=r_data.get("why") or None,
how_to_apply=r_data.get("how_to_apply") or None,
when_to_apply=r_data.get("when_to_apply") or None,
# A file written before migration 0088 has no tier. always_on
# A file written before milestone 394 carries `tier` and
# `always_on`; neither is read. Dropping a field the schema
# no longer has is the tolerant direction — an archive
# records what WAS, and refusing it because it remembers a
# deleted column would make every pre-394 backup
# unrestorable. Previously: always_on
# is the pre-0088 behaviour, so an old backup restores rules
# that bind exactly as they did when it was taken.
tier=r_data.get("tier") or "always_on",
# Same shape, same reason: a file written before 0098 has no
# kind, and every rule in it was a rule. Defaulting the other
# way would restore an old backup with things that had always
# bound quietly no longer binding.
kind=r_data.get("kind") or "rule",
verify_with=r_data.get("verify_with") or None,
expires_when=r_data.get("expires_when") or None,
# Restored as-is, NOT reset to null. `verified_at` records
@@ -1302,15 +1334,11 @@ async def _restore_v2(data: dict) -> dict:
stats["topic_suppressions"] += 1
# 14b. Always-on rulebook exclusions (v10, milestone 297)
for exc in data.get("rulebook_exclusions", []):
mapped_pid = project_id_map.get(exc.get("project_id", 0))
mapped_rbid = rulebook_id_map.get(exc.get("rulebook_id", 0))
if mapped_pid is None or mapped_rbid is None:
continue
await session.execute(project_rulebook_exclusions.insert().values(
project_id=mapped_pid, rulebook_id=mapped_rbid,
))
stats["rulebook_exclusions"] += 1
# `rulebook_exclusions` was a v10 section and is READ BY NOBODY since
# milestone 394 removed the table. An archive carrying it still
# imports — the key is simply not looked at — because refusing a
# backup for remembering something we deleted would make every v10-v13
# archive unrestorable.
# --- v5 sections. Every one is data.get()-guarded, so a v2/v3/v4
# payload restores without them rather than failing on an absent key.
@@ -1385,7 +1413,10 @@ async def _restore_v2(data: dict) -> dict:
why=rv.get("why"),
how_to_apply=rv.get("how_to_apply"),
when_to_apply=rv.get("when_to_apply"),
tier=rv.get("tier"),
# NOT defaulted, unlike the rule above. A version records what
# was; absent means nobody wrote it down, and inventing "rule"
# here would put an artifact where a measurement belongs.
kind=rv.get("kind"),
verify_with=rv.get("verify_with"),
expires_when=rv.get("expires_when"),
created_at=_dt(rv.get("created_at")),
@@ -1496,6 +1527,25 @@ async def _restore_v2(data: dict) -> dict:
))
stats["note_usage_events"] += 1
# The rule twin — and the reason it is a separate table at all.
# Resolved through rule_id_map, NOT note_id_map. A rule id run through
# the note map would either drop (best case) or land on whatever note
# took that number, producing telemetry that is wrong rather than
# missing and that nothing downstream could detect (milestone 333).
# Must come after the rules themselves; rule_id_map is populated there.
for ev in data.get("rule_usage_events", []):
mapped_rid = rule_id_map.get(ev.get("rule_id", 0))
if mapped_rid is None:
continue
session.add(RuleUsageEvent(
user_id=user_id_map.get(ev.get("user_id") or 0),
rule_id=mapped_rid,
event=ev.get("event", ""),
source=ev.get("source", ""),
created_at=_dt(ev.get("created_at")),
))
stats["rule_usage_events"] += 1
# 20. Repo bindings — small, but losing them means every bound repo
# quietly stops loading its project at session start.
for rb_data in data.get("repo_bindings", []):
@@ -1613,10 +1663,12 @@ async def _restore_v2(data: dict) -> dict:
inception = p_data.get("inception")
if isinstance(inception, dict):
choices = dict(inception.get("choices") or {})
choices["exclude_always_on_rulebooks"] = [
rulebook_id_map[i] for i in choices.get("exclude_always_on_rulebooks") or []
if i in rulebook_id_map
]
# DROPPED, not remapped (milestone 394). A pre-394 archive
# carries the retired exclusion choice; restoring it would put
# a key back that `validate_inception` now rejects as unknown,
# so the next edit to that project would fail on data this
# importer wrote.
choices.pop("exclude_always_on_rulebooks", None)
choices["subscribe_rulebooks"] = [
rulebook_id_map[i] for i in choices.get("subscribe_rulebooks") or []
if i in rulebook_id_map
+159 -15
View File
@@ -344,6 +344,52 @@ def chunk_document(title: str | None, body: str | None) -> list[str]:
return chunks
async def _claim_parent_row(session, id_column, row_id: int, label: str) -> bool:
"""Lock the record a vector belongs to BEFORE rewriting that vector (#3262).
An embedding write and a cascading delete of the same record take the same
two row locks in OPPOSITE orders. The embedder deletes the old chunk rows
and then, on INSERT, needs the foreign key's lock on the parent; a delete
of the parent — or of the rulebook, topic or project above it — locks the
parent first and cascades down into the chunk rows. That is a cycle, and
Postgres breaks it by killing one side at random: sometimes the embedding
write, which is swallowed and invisible, and sometimes the operator's
delete, which surfaces as a 500 on an operation that should have worked.
Claiming the parent first REMOVES the cycle rather than narrowing it.
Either the embedder arrives first and the delete waits its turn behind it,
or the delete already holds the row and NOWAIT makes the embedder lose at
once. The embedder is the side that should lose: a skipped refresh costs a
stale vector until the next write or the startup backfill, and the other
outcome costs a person their request.
FOR KEY SHARE, not FOR UPDATE — it is precisely the lock the INSERT's
foreign key would take anyway, so it conflicts with a delete of the parent
and with nothing else. An ordinary edit of the same record, or a second
refresh racing this one, is unaffected.
Returns False when the row is locked or already gone; the caller skips.
"""
try:
held = (await session.execute(
select(id_column)
.where(id_column == row_id)
# BOTH flags: SQLAlchemy spells the four Postgres row locks as a
# read/key_share pair, and key_share alone is FOR NO KEY UPDATE —
# which would make two refreshes of one record fight each other.
.with_for_update(read=True, key_share=True, nowait=True)
)).scalar_one_or_none()
except Exception:
# LockNotAvailable: this record is being deleted right now. Not an
# error — the delete wins by design.
logger.debug("Skipping embedding for %s %d — row is being deleted", label, row_id)
return False
if held is None:
logger.debug("Skipping embedding for %s %d — row is gone", label, row_id)
return False
return True
async def upsert_note_embedding(
note_id: int, user_id: int, title: str | None, body: str | None
) -> None:
@@ -380,6 +426,8 @@ async def upsert_note_embedding(
try:
async with async_session() as session:
if not await _claim_parent_row(session, Note.id, note_id, "note"):
return
await session.execute(
delete(NoteEmbedding).where(NoteEmbedding.note_id == note_id)
)
@@ -400,6 +448,23 @@ async def upsert_note_embedding(
logger.warning("Failed to persist embedding for note %d", note_id, exc_info=True)
# Both searches rank WITHOUT the threshold and apply it in Python, so the best
# rejected score stays observable (#3670). The qualifying set is provably
# unchanged: rows arrive ordered by distance ascending, so every above-bar row
# sorts ahead of every below-bar one, and an over-fetch that used to return N
# above-bar rows returns the same N plus some losers. What changes is only that
# the losers are now visible instead of discarded inside the query.
#
# That visibility is the entire point. A bar can only be judged from the calls
# it TURNED AWAY — a 0.72 bar rejecting a stream of 0.71s is set too high by a
# hair, one rejecting 0.30s is working — and those two are indistinguishable
# from any arrangement of the columns that survive the filter.
#
# `report` is how the score gets out without changing what a search RETURNS.
# Eight of the eleven call sites want hits and nothing else; the three that
# write telemetry pass a dict and read `best_available_score` back out of it.
async def semantic_search_notes(
user_id: int,
query: str,
@@ -414,12 +479,27 @@ async def semantic_search_notes(
scope: str = "own",
demote_superseded: bool = True,
system_id: int | None = None,
report: dict | None = None,
) -> list[tuple[float, Note]]:
"""Return up to *limit* (score, note) pairs most relevant to *query*.
Scores are cosine similarities in [-1, 1]; only notes at or above
*threshold* are returned, sorted highest-first.
Pass `report` (an empty dict) to learn what the threshold turned away:
the function sets `report["best_available_score"]` to the highest score
anything reached, or None when the corpus offered nothing at all. It is
the only figure that survives a call returning nothing, and therefore the
only one a bar can be judged from (#3670).
It also sets `report["searched"]`: False before anything can return, True
only where a real result set exists. So an empty query, an unavailable
embedder and a failed database query all leave it FALSE, and a caller can
tell a search that found nothing from one that never ran. A caller logging
telemetry must check it — recording a failed search as a zero-result call
reports a decline the ranker never made (#3765). ABSENT means no search
touched the dict at all, which is a stand-in in a test, not a real call.
`note_type` narrows to a record kind, or several (e.g. "snippet", or
("snippet", "note")), for callers that want prior art rather than everything
embedded.
@@ -454,6 +534,13 @@ async def semantic_search_notes(
Returns an empty list if the embedder is unavailable or on any error.
"""
# Stamped FALSE before anything can return, flipped True only where a real
# result set exists (#3765). Every early return below leaves it false, so a
# caller can tell a search that found nothing from one that never ran. It
# has to be the first thing done to `report`: a return added above this
# line would leave the key ABSENT, which reads as "no caller asked".
if report is not None:
report["searched"] = False
if not query or not query.strip():
return []
try:
@@ -465,7 +552,6 @@ async def semantic_search_notes(
# Distance ceiling equivalent to the similarity floor. Clamp to the valid
# cosine-distance range [0, 2] so a threshold of, say, -1 doesn't produce a
# nonsensical ceiling.
max_distance = min(2.0, max(0.0, 1.0 - threshold))
distance = NoteEmbedding.embedding.cosine_distance(query_vec)
try:
@@ -540,11 +626,10 @@ async def semantic_search_notes(
fetch = limit * _CHUNK_OVERFETCH * (
_SUPERSESSION_OVERFETCH if demote_superseded else 1
)
stmt = (
stmt.where(distance <= max_distance)
.order_by(distance.asc())
.limit(fetch)
)
# NO threshold predicate — see the note above this function. The
# bar is applied after the collapse, where the rejected scores can
# still be seen.
stmt = stmt.order_by(distance.asc()).limit(fetch)
rows = list((await session.execute(stmt)).all())
except Exception:
logger.warning("Failed to query note embeddings", exc_info=True)
@@ -563,6 +648,23 @@ async def semantic_search_notes(
continue
seen.add(int(note.id))
scored.append((1.0 - float(dist), note))
# The best score anything reached, bar or no bar. Recorded BEFORE the
# filter because a call that returns nothing is exactly when it matters.
if report is not None:
# `searched` is what stops a null score meaning four things (#3765).
# Every early return above — empty query, embedder down, and the broad
# `except` around the query itself — leaves this key ABSENT, so a
# caller can tell "I looked and there was nothing" from "I never
# looked" and from "the query failed". Set here, at the one point past
# which a real result set exists.
report["searched"] = True
# ONE unpack, so the score and the id cannot describe different records
# (#3807). Splitting these into two expressions is how a later edit
# pairs a score with its neighbour's id.
best = scored[0] if scored else None
report["best_available_score"] = best[0] if best else None
report["best_available_id"] = int(best[1].id) if best else None
scored = [pair for pair in scored if pair[0] >= threshold]
if not demote_superseded:
return scored[:limit]
return await _apply_supersession_penalty(scored, limit)
@@ -666,6 +768,8 @@ async def upsert_rule_embedding(
replacement is atomic per rule so a concurrent read sees the old chunk set
or the new one, never a mixture.
"""
from scribe.models.rulebook import Rule # runtime import: see TYPE_CHECKING above
doc_title, doc_body = rule_document(title, statement, when_to_apply)
chunks = chunk_document(doc_title, doc_body)
try:
@@ -688,6 +792,8 @@ async def upsert_rule_embedding(
try:
async with async_session() as session:
if not await _claim_parent_row(session, Rule.id, rule_id, "rule"):
return
await session.execute(
delete(RuleEmbedding).where(RuleEmbedding.rule_id == rule_id)
)
@@ -711,10 +817,25 @@ async def semantic_search_rules(
query: str,
limit: int = 5,
threshold: float = _SIMILARITY_THRESHOLD,
tier: str | None = None,
kind: str | None = None,
report: dict | None = None,
) -> list[tuple[float, "Rule"]]:
"""Return up to *limit* (score, rule) pairs most relevant to *query*.
Pass `report` (an empty dict) to learn what the threshold turned away:
the function sets `report["best_available_score"]` to the highest score
anything reached, or None when the corpus offered nothing at all. It is
the only figure that survives a call returning nothing, and therefore the
only one a bar can be judged from (#3670).
It also sets `report["searched"]`: False before anything can return, True
only where a real result set exists. So an empty query, an unavailable
embedder and a failed database query all leave it FALSE, and a caller can
tell a search that found nothing from one that never ran. A caller logging
telemetry must check it — recording a failed search as a zero-result call
reports a decline the ranker never made (#3765). ABSENT means no search
touched the dict at all, which is a stand-in in a test, not a real call.
Scoped by OWNERSHIP — a rule is the caller's if they own its rulebook or
its project. Deliberately not filtered to what currently BINDS a given
project: this answers "is there a rule about this", which a person asking
@@ -722,10 +843,23 @@ async def semantic_search_rules(
is the surfacing question, and it has its own machinery
(get_applicable_rules) rather than a second, subtly different copy here.
`tier` narrows to one tier. The write-path hint passes "conditional",
because an always-on rule is ALREADY in the session — surfacing it again as
a suggestion is pure noise, and noise on a hint that fires on every write
is how a hint gets ignored.
THERE IS NO TIER TO NARROW BY ANY MORE (milestone 394). This carried a
`tier` parameter, and the arms deliberately passed nothing: filtering on it
made a whole class of rules permanently ineligible for the one mechanism
that surfaces a rule AT the moment it applies. The tier is now gone
entirely, so every rule is eligible for every arm and relevance is the
threshold's job alone — see the block above RULEHINT_LIMIT in
services/plugin_context.py for what those scores are read against.
`kind` narrows to `rule` or `preference`, and NONE is likewise the ordinary
case: a caller asking "what governs this" wants both, because the reader
needs to know what binds AND how the operator wants it done. The one place
it is passed is a RESERVED SLOT — a query that may only return a
preference, so the slot cannot be spent on something else. That is the
same reason `note_type` exists on the sibling search, and the same failure
it prevents: a slot silently filled by the wrong kind is worse than no
slot, because the line is indistinguishable from one that earned its place
on score.
Collapses to best-chunk-per-rule like the note search, so a long rule split
across chunks competes once rather than crowding the results with itself.
@@ -735,6 +869,9 @@ async def semantic_search_rules(
from scribe.models.project import Project
from scribe.models.rulebook import Rule, Rulebook, RulebookTopic
# See the sibling search: stamped before anything can return (#3765).
if report is not None:
report["searched"] = False
if not query or not query.strip():
return []
try:
@@ -743,7 +880,6 @@ async def semantic_search_rules(
logger.debug("Rule search skipped — embedder unavailable")
return []
max_distance = min(2.0, max(0.0, 1.0 - threshold))
distance = RuleEmbedding.embedding.cosine_distance(query_vec)
try:
@@ -757,13 +893,14 @@ async def semantic_search_rules(
.outerjoin(Project, Rule.project_id == Project.id)
.where(
Rule.deleted_at.is_(None),
distance <= max_distance,
# No threshold predicate — see the note above
# semantic_search_notes. Applied below, after the collapse.
# topic_id XOR project_id, so exactly one arm can match.
or_(
Rulebook.owner_user_id == user_id,
Project.user_id == user_id,
),
*( [Rule.tier == tier] if tier else [] ),
*( [Rule.kind == kind] if kind else [] ),
)
# Overfetch so collapsing chunks to their best row still fills
# the page — the same reason the note search overfetches.
@@ -780,7 +917,14 @@ async def semantic_search_rules(
if rule.id not in best or score > best[rule.id][0]:
best[rule.id] = (score, rule)
ranked = sorted(best.values(), key=lambda pair: pair[0], reverse=True)
return ranked[:limit]
if report is not None:
# See the sibling search: absent means the search never ran (#3765).
report["searched"] = True
# One unpack — see the sibling search (#3807).
best = ranked[0] if ranked else None
report["best_available_score"] = best[0] if best else None
report["best_available_id"] = int(best[1].id) if best else None
return [pair for pair in ranked if pair[0] >= threshold][:limit]
async def backfill_rule_embeddings() -> None:
+32 -45
View File
@@ -7,7 +7,6 @@ A project's inheritance is a decision, not a default. The record lives on
"decided_at": "<iso>", "decided_by": <user id> | null,
"via": "mcp" | "ui" | "legacy",
"choices": {
"exclude_always_on_rulebooks": [rulebook ids],
"subscribe_rulebooks": [rulebook ids],
"design_system_id": <id> | null,
"seed_systems": bool
@@ -18,9 +17,14 @@ NULL = undecided → enter_project asks. ``legacy`` is the migration's stamp on
projects that existed before the step did (inherit-all / no design system /
no seed), so the ask fires only for projects created after this shipped.
``exclude_always_on_rulebooks`` was a fourth choice until milestone 394. It
let a project decline to inherit an always-on rulebook, and with no always-on
tier there is nothing to decline — a rulebook now reaches a project by
subscription, which is opt-IN, so declining is expressed by not subscribing.
The shape and its validator are pure; ``decide`` composes the existing
services — always-on exclusions, subscriptions, set_project_design_system,
the standard Systems seed — checks every target BEFORE touching anything,
services — subscriptions, set_project_design_system, the standard Systems
seed — checks every target BEFORE touching anything,
applies the effects (each idempotent), and writes the record LAST, so a
half-applied decision is re-runnable rather than recorded as done.
``current_defaults`` is what the enter_project ask shows: what binds today
@@ -37,7 +41,7 @@ from scribe.models.project import Project
from scribe.models.rulebook import Rulebook
INCEPTION_VIAS = ("mcp", "ui", "legacy")
CHOICE_KEYS = ("exclude_always_on_rulebooks", "subscribe_rulebooks", "design_system_id", "seed_systems")
CHOICE_KEYS = ("subscribe_rulebooks", "design_system_id", "seed_systems")
def _is_id_list(value) -> bool:
@@ -60,15 +64,9 @@ def validate_inception(choices) -> str | None:
unknown = sorted(set(choices) - set(CHOICE_KEYS))
if unknown:
return f"unknown inception choice(s): {', '.join(unknown)} (one of: {', '.join(CHOICE_KEYS)})"
excl = choices.get("exclude_always_on_rulebooks") or []
subs = choices.get("subscribe_rulebooks") or []
if not _is_id_list(excl):
return "exclude_always_on_rulebooks must be a list of rulebook ids"
if not _is_id_list(subs):
return "subscribe_rulebooks must be a list of rulebook ids"
both = sorted(set(excl) & set(subs))
if both:
return f"rulebook(s) {both} cannot be both excluded and subscribed"
ds = choices.get("design_system_id")
if ds is not None and (isinstance(ds, bool) or not isinstance(ds, int) or ds <= 0):
return "design_system_id must be a positive id or null"
@@ -79,11 +77,10 @@ def validate_inception(choices) -> str | None:
def normalize_choices(choices: dict | None) -> dict:
"""The four keys, always present, in canonical form — what gets stored
"""The three keys, always present, in canonical form — what gets stored
and what the UI/agent reads back. Call after validate_inception."""
choices = choices or {}
return {
"exclude_always_on_rulebooks": sorted(set(choices.get("exclude_always_on_rulebooks") or [])),
"subscribe_rulebooks": sorted(set(choices.get("subscribe_rulebooks") or [])),
"design_system_id": choices.get("design_system_id"),
"seed_systems": bool(choices.get("seed_systems", False)),
@@ -98,8 +95,7 @@ def is_decided(project) -> bool:
async def current_defaults(user_id: int, project_id: int) -> dict:
"""What the project inherits if nobody decides — the ask's payload.
{always_on_rulebooks: [{id,title}], other_rulebooks: [{id,title}],
excluded_always_on: [...], subscribed_rulebooks: [...],
{rulebooks: [{id,title}], subscribed_rulebooks: [...],
design_system_id, design_systems: [{id,title}], systems: <count>}.
Instance-agnostic: an install with no rulebooks / design systems shows
empty lists, and the ask says so rather than inventing a default.
@@ -115,7 +111,7 @@ async def current_defaults(user_id: int, project_id: int) -> dict:
async with async_session() as session:
rows = (
await session.execute(
select(Rulebook.id, Rulebook.title, Rulebook.always_on)
select(Rulebook.id, Rulebook.title)
.where(Rulebook.owner_user_id == user_id, Rulebook.deleted_at.is_(None))
.order_by(Rulebook.title)
)
@@ -124,9 +120,11 @@ async def current_defaults(user_id: int, project_id: int) -> dict:
designs = await design_systems_svc.list_design_systems(user_id)
systems = await systems_svc.list_systems(user_id, project_id, include_archived=True)
return {
"always_on_rulebooks": [{"id": i, "title": t} for i, t, on in rows if on],
"other_rulebooks": [{"id": i, "title": t} for i, t, on in rows if not on],
"excluded_always_on": applicable.get("excluded_always_on", []),
# ONE list since milestone 394. This was split into always-on and
# "other" because the first bound the project whether it asked or not;
# with the tier gone every rulebook is opt-in, so the split named a
# difference that no longer exists.
"rulebooks": [{"id": i, "title": t} for i, t in rows],
"subscribed_rulebooks": applicable.get("subscribed_rulebooks", []),
"design_system_id": project.design_system_id,
"design_systems": [{"id": d.id, "title": d.title} for d in designs],
@@ -139,28 +137,22 @@ async def _check_targets(user_id: int, choices: dict) -> None:
effect lands — a decision applies whole or errors whole."""
from scribe.services import access
wanted = set(choices["exclude_always_on_rulebooks"]) | set(choices["subscribe_rulebooks"])
wanted = set(choices["subscribe_rulebooks"])
if wanted:
async with async_session() as session:
rows = (
await session.execute(
select(Rulebook.id, Rulebook.always_on).where(
select(Rulebook.id).where(
Rulebook.id.in_(wanted),
Rulebook.owner_user_id == user_id,
Rulebook.deleted_at.is_(None),
)
)
).all()
found = {rid: on for rid, on in rows}
missing = sorted(wanted - set(found))
found = {rid for (rid,) in rows}
missing = sorted(wanted - found)
if missing:
raise ValueError(f"rulebook(s) {missing} not found (or not yours)")
not_always = sorted(r for r in choices["exclude_always_on_rulebooks"] if not found[r])
if not_always:
raise ValueError(
f"rulebook(s) {not_always} are not always-on — only always-on rulebooks "
"can be excluded; a subscribed rulebook is simply not subscribed"
)
ds = choices["design_system_id"]
if ds is not None and not await access.can_read_design_system(user_id, ds):
raise ValueError(f"design system {ds} not found (or not readable)")
@@ -176,13 +168,12 @@ async def decide(
"""Record a project's inception decision and apply it (milestone 297).
Owner-only. Validates the choices (pure) and every target (owned /
readable) first; then, each idempotent: exclude the named always-on
rulebooks, subscribe the named rulebooks, point the project at the design
system (None = explicitly none), seed the standard Systems if asked and
the project has none; then write ``projects.inception`` LAST. Re-deciding
is additive for exclusions/subscriptions (nothing is silently dropped —
include/unsubscribe are explicit calls), replaces the design system, and
re-seeds nothing a project already has.
readable) first; then, each idempotent: subscribe the named rulebooks,
point the project at the design system (None = explicitly none), seed the
standard Systems if asked and the project has none; then write
``projects.inception`` LAST. Re-deciding is additive for subscriptions
(nothing is silently dropped — unsubscribe is an explicit call), replaces
the design system, and re-seeds nothing a project already has.
Returns {"inception": <record>, "effects": {excluded, subscribed,
design_system_id, systems_seeded}}.
@@ -203,8 +194,6 @@ async def decide(
raise ValueError(f"project {project_id} not found (or not yours)")
await _check_targets(user_id, choices)
for rb in choices["exclude_always_on_rulebooks"]:
await rulebooks_svc.exclude_always_on_rulebook_for_project(project_id, rb, user_id)
for rb in choices["subscribe_rulebooks"]:
await rulebooks_svc.subscribe_project(project_id, rb, user_id)
if not await design_systems_svc.set_project_design_system(
@@ -230,7 +219,6 @@ async def decide(
return {
"inception": record,
"effects": {
"excluded": choices["exclude_always_on_rulebooks"],
"subscribed": choices["subscribe_rulebooks"],
"design_system_id": choices["design_system_id"],
"systems_seeded": [sy.name for sy in seeded],
@@ -247,25 +235,24 @@ async def inception_ask(user_id: int, project_id: int) -> dict:
defaults = await current_defaults(user_id, project_id)
except Exception:
return {}
always = ", ".join(f"{r['title']} (#{r['id']})" for r in defaults["always_on_rulebooks"]) or "none"
others = ", ".join(f"{r['title']} (#{r['id']})" for r in defaults["other_rulebooks"]) or "none"
books = ", ".join(f"{r['title']} (#{r['id']})" for r in defaults["rulebooks"]) or "none"
designs = ", ".join(f"{d['title']} (#{d['id']})" for d in defaults["design_systems"]) or "none"
return {
"defaults": defaults,
"ask": (
"This project has no inception decision: nobody has said what it "
f"inherits. Today, by default: always-on rulebooks binding it{always}; "
f"rulebooks it could subscribe to — {others}; design system — "
f"inherits. Rulebooks it could subscribe to{books}; design system — "
f"{'#' + str(defaults['design_system_id']) if defaults['design_system_id'] else 'none'} "
f"(available: {designs}); Systems — {defaults['systems']}. Ask the operator, "
"once: which always-on rulebooks to EXCLUDE here (default: none), which "
"rulebooks to subscribe, which design system (or none), and whether to seed "
"once: which rulebooks to subscribe (default: none — a rulebook binds "
"a project only when it opts in), which design system (or none), and "
"whether to seed "
"the standard starter Systems — then record the answers. This ask repeats on "
"every enter_project until a decision is recorded."
),
"call": (
f"decide_project_inception(project_id={project_id}, "
"exclude_always_on_rulebooks=[...], subscribe_rulebooks=[...], "
"subscribe_rulebooks=[...], "
"design_system_id=<id | -1 for none>, seed_systems=<true|false>)"
),
}
+10 -28
View File
@@ -30,13 +30,13 @@ from __future__ import annotations
import asyncio
import logging
import traceback
from sqlalchemy import case, func, select
from scribe.models import async_session
from scribe.models.note_usage import PULLED, SURFACED, NoteUsageEvent
from scribe.models.base import iso
from scribe.services.background import report_telemetry_failure
logger = logging.getLogger(__name__)
@@ -46,37 +46,19 @@ logger = logging.getLogger(__name__)
# never lands. The done-callback discard keeps the set from growing.
_pending: set[asyncio.Task] = set()
# Sites that already dropped their once-per-process AppLog row. The readout
# runs on every snippet list render — without this, a broken table would turn
# the error log into a firehose that buries the finding it exists to surface.
_reported: set[str] = set()
async def _report_failure(site: str) -> None:
"""Make a swallowed telemetry failure visible. Called from an except block.
"""This subsystem's canary, now the shared one.
WARNING to the process log every time; one AppLog error row per process per
site so the admin UI shows the outage without host access. The AppLog write
is itself guarded — when the whole database is down it fails too, and that
is fine: the WARNING already said so, and a canary must never take down the
surface it watches.
The per-site dedup, the WARNING and the single AppLog row all moved to
`background.report_telemetry_failure` unchanged when `rule_usage` needed
the identical behaviour — two hand-kept copies of a thing whose whole job
is to be reliable is the wrong number. `retrieval_telemetry` deliberately
still has its own: its canary is a different shape (one process-wide flag,
no AppLog row), so repointing it would change behaviour rather than
consolidate it.
"""
logger.warning("note usage telemetry %s failed", site, exc_info=True)
if site in _reported:
return
_reported.add(site)
try:
from scribe.services.logging import log_error
await log_error(
endpoint="note_usage",
error_type=f"note_usage_{site}_failed",
error_message=f"note usage telemetry {site} is failing; "
"usage counters will read zero until this is fixed",
traceback=traceback.format_exc(),
)
except Exception:
logger.debug("note usage canary write failed", exc_info=True)
await report_telemetry_failure("note_usage", site)
async def _insert_events(rows: list[dict]) -> None:
+4
View File
@@ -76,6 +76,10 @@ def embed_note(note) -> None:
exceptions are swallowed because a record that saved must not fail on its
index refresh. No running loop (unit tests, scripts) is an ordinary case,
not an error.
Detaching also means this task races anything that deletes the note out
from under it. That is not handled here: `upsert_note_embedding` claims
the note's row before touching its vectors, and loses if it can't (#3262).
"""
try:
import asyncio
+1 -1
View File
@@ -60,7 +60,7 @@ async def start_planning(user_id: int, project_id: int, title: str) -> dict:
return {
"milestone": milestone.to_dict(),
**rulebooks_svc.rules_payload(applicable),
**rulebooks_svc.rules_payload(applicable, user_id=user_id, source="start_planning"),
"project_goal": getattr(project, "goal", "") or "",
"open_task_count": open_count,
}
File diff suppressed because it is too large Load Diff
+669 -20
View File
@@ -17,6 +17,7 @@ from __future__ import annotations
import asyncio
import logging
import re
from typing import Any
from datetime import datetime, timedelta, timezone
@@ -27,6 +28,10 @@ from scribe.models import async_session
from scribe.models.base import iso
from scribe.models.note import Note
from scribe.models.note_usage import PULLED, SURFACED, NoteUsageEvent
from scribe.models.rule_usage import PULLED as RULE_PULLED
from scribe.models.rule_usage import SURFACED as RULE_SURFACED
from scribe.models.rule_usage import RuleUsageEvent
from scribe.services.rule_usage import is_ambient
from scribe.models.retrieval_log import RetrievalLog
logger = logging.getLogger(__name__)
@@ -40,6 +45,84 @@ _pending: set[asyncio.Task] = set()
_reported = False
# ── secrets never reach the query column (#3925) ───────────────────────
#
# `pre_tool_rule` retrieves against the RAW COMMAND TEXT and `write_path_rule`
# against the code being written, so whatever was on the command line or in the
# buffer is what gets logged. A command that exports a token therefore stored
# the token — and worse than stored it: `near_miss_samples` is the readout the
# threshold docs tell you to open before moving a bar, so the value came back
# out into an agent's context on the next tuning pass. That is how this was
# found.
#
# SCRUBBED ON WRITE, NOT ON READ. A read-side filter leaves the secret in the
# table, where a backup, a debug query or a future readout still reaches it.
# The value must never land.
#
# REDACTED VISIBLY, AND THIS IS THE PART THAT KEEPS THE READOUT HONEST. The
# whole worth of a near-miss sample is reading the query that was actually
# refused; a scrubber that silently deleted spans would turn the one instrument
# for tuning a bar into unreadable stubs — the #2663 shape, where a surface
# looks fine and has quietly stopped saying anything. A `[redacted:<kind>]`
# marker keeps the sentence readable, keeps its shape and length roughly
# intact for the ranker's reader, and says plainly that something was removed.
#
# DELIBERATELY CONSERVATIVE. These patterns match things that are secrets by
# CONSTRUCTION — a vendor-prefixed credential, a value assigned to a
# secret-named variable, an auth header, a PEM header. Anything cleverer
# (entropy heuristics, long-opaque-string detection) starts eating real
# queries, and a query is evidence. Missing an exotic secret costs one
# redaction nobody made; eating a query costs the ability to tune the bar.
_SECRET_PATTERNS: tuple[tuple[str, "re.Pattern[str]"], ...] = (
# Vendor-prefixed credentials. The prefix IS the tell, so no entropy
# guessing is needed — `fmcp_` is Scribe's own API key format.
("token", re.compile(
r"\b(?:fmcp_|flt_|ghp_|gho_|ghs_|ghu_|github_pat_|glpat-|gitlab-ci-token:"
r"|xox[abprs]-|sk-[A-Za-z0-9]*-?|AKIA|ASIA)[A-Za-z0-9_\-]{12,}"
)),
# A value handed to a secret-NAMED variable, in shell, env files, YAML,
# JSON or a query string. The name is what identifies it, so the value can
# be anything.
("assigned", re.compile(
r"(?i)\b([A-Za-z0-9_]*"
# NO BARE "auth" HERE. It matched `--author=`, so a commit naming an
# address redacted the address — evidence eaten for a word that only
# LOOKS credential-shaped. `AUTH_TOKEN` is still caught, by `token`.
r"(?:token|secret|password|passwd|api[_-]?key|access[_-]?key)"
r"[A-Za-z0-9_]*)"
r"(\s*[:=]\s*[\"']?)"
r"([^\s\"'&]{8,})"
)),
("auth-header", re.compile(
r"(?i)(authorization\s*:\s*(?:bearer|basic|token)\s+)(\S+)"
)),
("private-key", re.compile(
r"-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----"
)),
)
def scrub_secrets(text: str | None) -> str | None:
"""Redact credential-shaped spans from a query before it is stored.
Pure and synchronous, so it is unit-testable and safe to run inline on the
write path. Returns the input unchanged when nothing matches, which is the
overwhelmingly common case and the one the patterns are tuned to protect.
"""
if not text:
return text
for kind, pattern in _SECRET_PATTERNS:
if kind == "assigned":
text = pattern.sub(
lambda m: f"{m.group(1)}{m.group(2)}[redacted:{kind}]", text)
elif kind == "auth-header":
text = pattern.sub(lambda m: f"{m.group(1)}[redacted:{kind}]", text)
else:
text = pattern.sub(f"[redacted:{kind}]", text)
return text
def _build_payload(
*,
user_id: int | None,
@@ -51,12 +134,27 @@ def _build_payload(
is_task: bool | None,
results: list[tuple[float, Note]],
duration_ms: float | None,
suppressed: int | None = None,
best_available: float | None = None,
best_available_id: int | None = None,
) -> dict:
"""Reduce a retrieval call to a flat, JSON-safe RetrievalLog payload.
Pure and synchronous (no DB, no event loop) so it is unit-testable and safe
to run inline before scheduling the write. `results` is the
`(score, Note)` list from semantic_search_notes, already highest-first.
`suppressed` is how many scored hits the caller dropped because the session
had already been shown them, and it stays None for callers that cannot
know. See the column's comment: None means "not measured here", which is a
different fact from 0 and must never render as one.
`best_available` is the highest score the ranker reached BEFORE the
threshold, and it carries the same null discipline for a sharper reason: it
is the only field that still says something on a call that returned
nothing, so a 0.0 standing in for "not measured" would read as "the corpus
held nothing remotely relevant" — a claim about the corpus invented out of
a caller's silence.
"""
items = [
{"id": int(note.id), "score": round(float(score), 5), "rank": rank}
@@ -66,14 +164,25 @@ def _build_payload(
return {
"user_id": user_id,
"source": source,
"query": query,
# Scrubbed HERE rather than at each caller: this is the only path to
# the column, and a per-caller scrub is three places for one of them
# to be forgotten by whoever adds the fourth arm.
"query": scrub_secrets(query),
"threshold": threshold,
"limit_n": limit,
"project_id": project_id,
"is_task": is_task,
"result_count": len(items),
"suppressed_count": (None if suppressed is None else int(suppressed)),
"top_score": (scores[0] if scores else None),
"min_score": (scores[-1] if scores else None),
"best_available_score": (
None if best_available is None else round(float(best_available), 5)
),
# The record that scored it, so a reader can go and look (#3807).
"best_available_id": (
None if best_available_id is None else int(best_available_id)
),
"result_ids": items,
"duration_ms": (round(duration_ms, 2) if duration_ms is not None else None),
}
@@ -111,6 +220,10 @@ def record_retrieval(
is_task: bool | None,
results: list[tuple[float, Any]],
duration_ms: float | None = None,
suppressed: int | None = None,
best_available: float | None = None,
best_available_id: int | None = None,
searched: bool = True,
) -> None:
"""Fire-and-forget: record one retrieval call.
@@ -122,9 +235,35 @@ def record_retrieval(
provide. retrieval_logs is not restored at all, so it has no such hazard,
and `source` already distinguishes the surfaces.
`searched=False` WRITES NO ROW, and that is the point rather than an
optimisation. A semantic search has three ways to return nothing without
having run — an empty query, an unavailable embedder, and the broad
`except` around the query itself — and each one currently arrives here
looking exactly like a ranker that declined. Logging it would report a
decline nobody made, drag `zero_result_calls` down with phantom evidence
about a threshold, and leave `best_available_score` null for a reason that
has nothing to do with the corpus. That last ambiguity is #3765: the field
added to judge a bar was null on four unrelated causes, one of them a
swallowed failure, and no reader could tell them apart.
Dropping the row is what makes the remaining nulls mean ONE thing —
"searched, and there was nothing".
The same convention already governs the pre-tool arm: a blank command costs
no embedding query, so it writes no row, because "a row here would report a
call that never happened and drag the clear-rate down with phantom
declines". This extends it from a case the caller could see in advance to
the ones only the search knows about.
A FAILURE IS NOT MADE INVISIBLE BY THIS. `semantic_search_notes` logs a
WARNING on a query failure, which is where a broken search belongs — a
counter cannot say "I am broken" without a reader already trusting it.
Builds the payload inline (synchronously) then schedules the insert so the
caller returns immediately. Never raises — telemetry must not affect search.
"""
if not searched:
return
try:
payload = _build_payload(
user_id=user_id,
@@ -136,6 +275,9 @@ def record_retrieval(
is_task=is_task,
results=results,
duration_ms=duration_ms,
suppressed=suppressed,
best_available=best_available,
best_available_id=best_available_id,
)
except Exception:
logger.debug("retrieval telemetry payload build failed", exc_info=True)
@@ -162,36 +304,147 @@ def record_retrieval(
def _bucket(rows: list) -> dict:
"""A score readout a human can act on, from one aggregate row."""
calls, zero, cleared, p10, p50, p90, lo, hi, avg_n, dur = rows
(calls, zero, p10, p50, p90, lo, hi, avg_n, dur,
measured, supp_calls, supp_zero,
miss_calls, miss_p50, miss_p90, miss_max) = rows
return {
"calls": int(calls or 0),
# A call that returned nothing is not a low-scoring call — it is a
# different failure (nothing indexed, filter too narrow), and averaging
# it into the score distribution would hide both.
"zero_result_calls": int(zero or 0),
# How often the best hit actually cleared the threshold in force for
# that call. THE precision-adjacent number: a surface that clears its
# bar on almost every call is either well-tuned or too loose, and the
# score spread below says which.
"cleared_threshold": int(cleared or 0),
# `cleared_threshold` USED TO LIVE HERE and it was a tautology (#3670).
# The search applies the bar before returning, so every returned result
# cleared it by construction and a call with nothing has no score to
# compare — the condition was true exactly when `result_count > 0`.
# `zero_result_calls + cleared_threshold == calls` held on all nineteen
# readings ever taken. It was `calls - zero_result_calls` wearing a name
# that promised a second opinion, and the docstring built a reading
# procedure on it that asked the reader to compare a number with itself.
# Its replacement is `near_misses` below, which the bar cannot fix by
# construction because it is measured on the calls the bar REJECTED.
# Of the zeros above, which were the RANKER declining and which were
# the reader having seen it already? `zero_result_calls` cannot say,
# and only the first kind is evidence about the threshold.
#
# None — not a zeroed dict — when no row in the window reported it. A
# surface that filters inside the search genuinely does not know, and
# rendering that as `{"calls": 0}` would state a measurement nobody
# made. That substitution is the whole of #3311.
"suppression": (
None if not int(measured or 0) else {
"measured_calls": int(measured or 0),
"calls_with_suppression": int(supp_calls or 0),
# Subtract from zero_result_calls for the true ranker declines.
"zero_because_already_shown": int(supp_zero or 0),
}
),
"top_score": {
"p10": _round(p10), "p50": _round(p50), "p90": _round(p90),
"min": _round(lo), "max": _round(hi),
},
# WHAT THE BAR TURNED AWAY, and the only figure here a threshold can
# actually be tuned from. Measured over the calls that returned
# NOTHING, on the best score the ranker reached before the filter.
#
# Read `p90` against the threshold in force. A bar at 0.72 rejecting a
# stream of 0.71s is set too high by a hair and the surface is losing
# hits it should have had; the same bar rejecting 0.30s is doing its
# job and the corpus simply had nothing. Both render as a zero-result
# call, and nothing else in this readout separates them.
#
# None — not a zeroed block — when no declining call in the window
# measured it. Old rows predate the column, and a 0.0 would assert that
# the corpus held nothing relevant, which is a claim about the corpus
# invented out of a caller's silence.
"near_misses": (
None if not int(miss_calls or 0) else {
"measured_calls": int(miss_calls or 0),
"p50": _round(miss_p50),
"p90": _round(miss_p90),
"max": _round(miss_max),
}
),
"avg_result_count": _round(avg_n),
"p90_duration_ms": _round(dur, 1),
}
# The aggregate row Postgres would have returned for a source with no rows in
# the window: nothing counted, nothing scored. Positional, matching the SELECT
# `_bucket` unpacks — calls, zero, p10, p50, p90, min, max, avg_n,
# dur, measured, supp_calls, supp_zero, miss_calls, miss_p50, miss_p90,
# miss_max. The counts are 0 because zero calls is a real observation;
# everything else is None because a distribution nobody sampled has no value,
# and rendering it as 0.0 would state one.
_NO_ROWS_IN_WINDOW = [0, 0, None, None, None, None, None, None, None,
0, 0, 0, 0, None, None, None]
def _round(v, places: int = 4):
return None if v is None else round(float(v), places)
async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
async def _complete_from(session, model, user_id) -> dict[str, Any]:
"""When each source in `model` started being recorded, and the instant the
WHOLE table is complete from. Returns {source: earliest_row, "*": latest}.
THE GRAIN IS THE SOURCE, and that is the whole point. `retrieval_logs` has
rows going back months, so a table-level "earliest row" says months and
tells a reader their window is fully covered — while a source added last
week has a week of rows and a counter that silently means something else.
Per-source is the only grain at which partial coverage is visible.
THE AGGREGATE USES THE LATEST, NOT THE EARLIEST. A number that sums several
sources is complete only once EVERY contributor was recording, so "*" is a
max over the sources, not a min. Taking the min here would reproduce the
exact reading this exists to prevent: the oldest source vouching for the
youngest.
All-time, deliberately unfiltered by the window — a query bounded by
`since` can only ever report something at or after `since`, which answers
nothing.
"""
rows = (
await session.execute(
select(model.source, func.min(model.created_at))
.where(model.user_id == user_id)
.group_by(model.source)
)
).all()
out: dict[str, Any] = {src: ts for src, ts in rows if ts is not None}
stamps = list(out.values())
out["*"] = max(stamps) if stamps else None
return out
def _coverage(complete_from, since) -> dict:
"""The two keys every counter block carries, from one timestamp.
`covers_window` is None — never False — when nothing was ever recorded.
"No rows at all" is not "partial coverage", it is no measurement, and the
null convention #3497 established for `suppression` holds here for the
same reason: absent must not read as a verdict.
"""
return {
# iso() already returns None for an unset value (#2845) — the guard
# belongs on covers_window, which is a verdict, not a serialisation.
"complete_from": iso(complete_from),
"covers_window": (
None if complete_from is None else complete_from <= since
),
}
async def retrieval_summary(
user_id: int | None, *, days: int = 30, near_miss_samples: int = 0,
) -> dict:
"""What the retrieval telemetry says, per surface, over a window.
Two aggregates side by side, each read from the table built for it — NOT a
join. `NoteUsageEvent`'s own docstring is explicit that the two are
Three aggregates side by side, each read from the table built for it — NOT
a join. `usage` is notes, `rule_usage` is rules, and they stay apart
because a few dozen eligible rules blended into thousands of notes is the
note ratio with noise on it (milestone 333). `NoteUsageEvent`'s own docstring is explicit that the two are
complements ("RetrievalLog tunes the threshold, this tunes the corpus") and
that RetrievalLog's JSONB `result_ids` "can't be indexed at" the per-note
grain. So the score distribution comes from `retrieval_logs` on its indexed
@@ -199,6 +452,12 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
it was built for. Reading each from its own table is both cheaper and more
honest than correlating them through JSONB.
`usage["by_source"]` is the one join, and it stays INSIDE
`note_usage_events` — surfaced rows against pulled rows on note_id. That
answers "of the notes this surface chose, how many were opened", which the
top-level ratio averages away. It does not cross into `retrieval_logs`, so
the sentence above still holds.
Scoped to one user's own telemetry. There is no sharing model for a
retrieval log — it records what THIS user's agent asked for, including the
query text — so an owner filter is the whole access rule here rather than a
@@ -214,23 +473,77 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
"since": iso(since),
"sources": {},
"usage": {},
"rule_usage": {},
"read_failed": False,
}
cleared = case(
(
(RetrievalLog.threshold.isnot(None))
& (RetrievalLog.top_score.isnot(None))
& (RetrievalLog.top_score >= RetrievalLog.threshold),
1,
),
zero = case((RetrievalLog.result_count == 0, 1), else_=0)
# THE NEAR-MISS POPULATION: calls that returned nothing BECAUSE THE BAR
# TURNED SOMETHING AWAY, and recorded what it was. Three conditions, and
# the third was missing for one deploy (#3739).
#
# Zero-result only: on a call that returned something,
# `best_available_score` equals `top_score` and adds nothing.
#
# Non-null only: rows written before #3670 genuinely do not know, and must
# not read as scoreless declines.
#
# AND NOT A REPEAT. A zero-result call is two unrelated events — the ranker
# found nothing above the bar, or it found only what this session had
# already been shown — and just the first says anything about the bar. That
# is the whole of #3497, and #3670 reintroduced the conflation one level up:
# the rule arms filter exclusions in PYTHON, after the search, so a rule
# that cleared the bar and was dropped as a repeat still reported a high
# `best_available_score` on a zero-result row. Live proof, first read after
# deploy: pre_tool_rule's near-miss max was 0.7457 while the lowest score it
# ever RETURNED was 0.7204 — a "rejection" that outscored acceptances.
#
# The NULL arm is principled, not permissive: `suppressed_count IS NULL`
# means the caller passed its exclusions INTO the search, which is exactly
# the case where the reported score is already post-exclusion and cannot be
# contaminated. Note arms stay measured; rule arms get cleaned.
#
# Deliberately conservative: a call carrying both a repeat and a lower
# genuine miss is dropped whole, losing that point. It undercounts; it
# cannot corrupt — the right way round for a number read against a bar.
#
# This also makes `near_misses.max < threshold` true BY CONSTRUCTION. An
# above-bar candidate that was not excluded would have been returned, so
# its call is not in this population at all.
declined = (
(RetrievalLog.result_count == 0)
& (RetrievalLog.best_available_score.isnot(None))
& (
RetrievalLog.suppressed_count.is_(None)
| (RetrievalLog.suppressed_count == 0)
)
)
miss = case((declined, 1), else_=0)
# `best_available_score` only for those rows; NULL elsewhere, and
# percentile_cont ignores NULLs, so the distribution is over the declines
# alone without a second pass over the table.
miss_score = case((declined, RetrievalLog.best_available_score), else_=None)
# Three sums rather than one, because "not measured" and "measured as zero"
# are different answers and a single counter cannot hold both.
measured = case((RetrievalLog.suppressed_count.isnot(None), 1), else_=0)
supp_calls = case((RetrievalLog.suppressed_count > 0, 1), else_=0)
supp_zero = case(
((RetrievalLog.result_count == 0) & (RetrievalLog.suppressed_count > 0), 1),
else_=0,
)
zero = case((RetrievalLog.result_count == 0, 1), else_=0)
def pct(p: float):
return func.percentile_cont(p).within_group(RetrievalLog.top_score.asc())
# Assigned inside the try below; named here so the readout can tell
# "this query failed" from "this window has no rows" (#2663).
by_source_rows = None
rule_rows = None
distinct_rules_surfaced = distinct_rules_pulled = 0
# None means the coverage read did not happen — distinct from a table with
# no rows, which is {"*": None}. Same reason `read_failed` exists.
note_complete = rule_complete = None
try:
async with async_session() as session:
rows = (
@@ -239,7 +552,6 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
RetrievalLog.source,
func.count().label("calls"),
func.sum(zero).label("zero"),
func.sum(cleared).label("cleared"),
pct(0.1), pct(0.5), pct(0.9),
func.min(RetrievalLog.top_score),
func.max(RetrievalLog.top_score),
@@ -247,6 +559,13 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
func.percentile_cont(0.9).within_group(
RetrievalLog.duration_ms.asc()
),
func.sum(measured).label("measured"),
func.sum(supp_calls).label("supp_calls"),
func.sum(supp_zero).label("supp_zero"),
func.sum(miss).label("miss_calls"),
func.percentile_cont(0.5).within_group(miss_score.asc()),
func.percentile_cont(0.9).within_group(miss_score.asc()),
func.max(miss_score),
)
.where(
RetrievalLog.created_at >= since,
@@ -255,8 +574,82 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
.group_by(RetrievalLog.source)
)
).all()
log_complete = await _complete_from(session, RetrievalLog, user_id)
for row in rows:
out["sources"][row[0]] = _bucket(list(row[1:]))
source = row[0]
bucket = _bucket(list(row[1:]))
# Per SOURCE, not per table: retrieval_logs goes back months
# while any individual arm may be days old, and the table's
# age would vouch for an arm that has barely started.
bucket.update(_coverage(log_complete.get(source), since))
out["sources"][source] = bucket
# A source with rows in the table but NONE in this window would
# otherwise be absent from the readout — and absent is exactly how
# a source that never existed renders, so a surface that WAS
# recording and went silent is unreadable (#3720). That is #2663
# one level up: the failure that looks like the correct answer.
#
# Zero here is a real measurement, not a manufactured one. The
# all-time query proves the source was recording, and it made no
# calls across a window it fully covers — which is why no
# `covers_window` special case is needed: a source whose first row
# fell after `since` would have that row IN the window and already
# hold a bucket, so anything reaching here began before it.
for src, first_row in log_complete.items():
if src == "*" or first_row is None or src in out["sources"]:
continue
quiet = _bucket(list(_NO_ROWS_IN_WINDOW))
quiet.update(_coverage(first_row, since))
out["sources"][src] = quiet
# WHAT THE BAR REFUSED, by name (#3807). Opt-in, because it is a
# LISTING and not a statistic: an id cannot be percentiled, and a
# reader tuning a threshold needs to go and read the records rather
# than see another number about them. Off by default so the
# ordinary readout keeps its size.
#
# Deliberately NOT a window function. This module's one production
# outage (#2663) was a grouped query the database rejected, swallowed
# by the broad except, every counter reading zero while the mocked
# tests passed — and the lesson recorded then was to group on a raw
# column and classify in Python rather than push cleverness into the
# SQL. So: one flat ordered query, overfetched, bucketed here.
if near_miss_samples > 0:
want = max(1, min(int(near_miss_samples), 20))
rows = (
await session.execute(
select(
RetrievalLog.source,
RetrievalLog.best_available_score,
RetrievalLog.best_available_id,
RetrievalLog.query,
)
.where(
declined,
RetrievalLog.created_at >= since,
RetrievalLog.user_id == user_id,
RetrievalLog.best_available_id.isnot(None),
)
.order_by(RetrievalLog.best_available_score.desc())
# Overfetch so every source can fill its own quota even
# when one of them holds all the highest scores.
.limit(want * 40)
)
).all()
for src, score, rec_id, q in rows:
bucket = out["sources"].get(src)
if bucket is None:
continue
samples = bucket.setdefault("near_miss_records", [])
if len(samples) >= want:
continue
samples.append({
"score": _round(score),
"record_id": int(rec_id),
# Enough to recognise the ask, not the whole prompt.
"query": (q or "")[:120],
})
# The corpus side, at its own grain. `ambient` mirrors
# note_usage.usage_for_notes: an ambient surfacing was not a scored
@@ -283,6 +676,7 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
.group_by(NoteUsageEvent.event, NoteUsageEvent.source)
)
).all()
note_complete = await _complete_from(session, NoteUsageEvent, user_id)
# Distinct-note counts need their OWN queries, and this is not
# fussiness: count(distinct note_id) per (event, source) group
@@ -310,6 +704,142 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
)
)
).scalar_one()
# Per-source pull-through, at the NOTE grain (#3311).
#
# The `urows` query above already groups by source and the loop
# below then throws the source away, so until now this readout
# could say what the corpus's overall pull-through was and nothing
# about WHICH surface earned it. The data was always here; only
# the aggregation discarded it.
#
# It cannot be had by grouping the PULLED rows by source: a pull
# records the door it came through (`mcp_get_note`), not the
# surface that put the record in front of the agent. Correlating
# those within a session is what #2085 ruled out — there is no
# session identity server-side and inventing one would mean
# threading a client-supplied token through every read path. The
# note grain answers the question without one: of the distinct
# notes surface X chose, how many did an agent open in this window?
#
# Guarded separately from the reads above, on #2663's actual
# lesson. That outage was a NOVEL SQL SHAPE the database rejected
# inside a broad except. This join is the novel shape here, and a
# failure in it must not take down two readouts that already work.
try:
pulled_ids = (
select(NoteUsageEvent.note_id)
.where(
NoteUsageEvent.created_at >= since,
NoteUsageEvent.user_id == user_id,
NoteUsageEvent.event == PULLED,
# autoescape because `_` is a LIKE wildcard: a bare
# like("mcp_%") also matches "mcpX…". The Python half
# of this readout uses str.startswith and has no such
# hazard; this is the SQL half's version of it.
NoteUsageEvent.source.startswith("mcp_", autoescape=True),
)
.distinct()
.subquery()
)
surfaced_pairs = (
select(NoteUsageEvent.source, NoteUsageEvent.note_id)
.where(
NoteUsageEvent.created_at >= since,
NoteUsageEvent.user_id == user_id,
NoteUsageEvent.event == SURFACED,
)
.distinct()
.subquery()
)
# DISTINCT on (source, note_id) FIRST, which is what lets the
# outer aggregate be a plain count(): the pairs are already
# unique, so the left join cannot multiply them and no
# count(DISTINCT) is needed to undo damage that never happens.
by_source_rows = (
await session.execute(
select(
surfaced_pairs.c.source,
func.count().label("notes_surfaced"),
func.count(pulled_ids.c.note_id).label("notes_pulled"),
)
.select_from(
surfaced_pairs.outerjoin(
pulled_ids,
pulled_ids.c.note_id == surfaced_pairs.c.note_id,
)
)
.group_by(surfaced_pairs.c.source)
)
).all()
except Exception:
logger.warning("per-source pull-through read failed", exc_info=True)
by_source_rows = None
# Rules, at their own grain and in their own block (milestone 333).
#
# Guarded separately from the reads above for the reason `by_source`
# is: this table is NEW, and an instance running upgraded code
# against un-migrated schema would otherwise take down two readouts
# that work perfectly in order to report a third that cannot.
#
# The queries themselves are the note block's shapes, not novel
# ones — a group-by on two indexed columns and two count(distinct).
# The distinct counts need their own queries for the same reason
# the note ones do: count(distinct rule_id) per group cannot be
# summed across groups without double-counting a rule two sources
# both touched.
try:
rule_rows = (
await session.execute(
select(
RuleUsageEvent.event,
RuleUsageEvent.source,
func.count().label("n"),
)
.where(
RuleUsageEvent.created_at >= since,
RuleUsageEvent.user_id == user_id,
)
.group_by(RuleUsageEvent.event, RuleUsageEvent.source)
)
).all()
rule_complete = await _complete_from(
session, RuleUsageEvent, user_id,
)
# The rows carry `source`, so the ranked/ambient split is done
# below rather than in SQL — the bulk surfaces started emitting
# on 2026-09-03 (#3473), so there IS an ambient class now.
#
# `distinct_rules_surfaced` deliberately counts BOTH classes. It
# answers "how many distinct rules did this install put in front
# of an agent at all", which is the denominator for dead weight
# — and a rule delivered by the preload a hundred times and
# never opened is the most important case that question has.
distinct_rules_surfaced = (
await session.execute(
select(func.count(func.distinct(RuleUsageEvent.rule_id)))
.where(
RuleUsageEvent.created_at >= since,
RuleUsageEvent.user_id == user_id,
RuleUsageEvent.event == RULE_SURFACED,
)
)
).scalar_one()
distinct_rules_pulled = (
await session.execute(
select(func.count(func.distinct(RuleUsageEvent.rule_id)))
.where(
RuleUsageEvent.created_at >= since,
RuleUsageEvent.user_id == user_id,
RuleUsageEvent.event == RULE_PULLED,
)
)
).scalar_one()
except Exception:
logger.warning("rule usage read failed", exc_info=True)
rule_rows = None
distinct_rules_surfaced = distinct_rules_pulled = 0
except Exception:
logger.warning("retrieval summary read failed", exc_info=True)
out["read_failed"] = True
@@ -350,5 +880,124 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
round(usage["pulled_by_agent"] / usage["surfaced"], 4)
if usage["surfaced"] else None
)
# The same question, per surface — which is the one the top-level ratio
# cannot answer. A corpus average of 0.05 is compatible with one surface
# earning its noise and another producing none, and tuning a threshold
# needs to know which.
#
# UPPER BOUND, and say so where it will be read: a pull records the door,
# not the surface that led to it, so a note surfaced by two surfaces and
# opened once counts as pulled for both. Attribution would need the session
# identity #2085 declined to invent. The bound is still decisive in the
# direction that matters — a surface reading near zero here is not being
# flattered by the double-count.
if by_source_rows is None:
usage["by_source"] = {}
# Distinct from an empty window, for the same reason `read_failed` is.
usage["by_source_failed"] = True
else:
by_source: dict[str, dict] = {}
for source, n_surfaced, n_pulled in by_source_rows:
n_surfaced, n_pulled = int(n_surfaced or 0), int(n_pulled or 0)
ambient = source in AMBIENT_SOURCES
by_source[source] = {
"notes_surfaced": n_surfaced,
"notes_pulled": n_pulled,
# None rather than a number on an ambient surface: nothing
# CHOSE those records, so "surfaced often, opened never" is not
# a judgment about them. The counts stay visible; the ratio
# that would be misread does not.
"pull_through": (
None if ambient or not n_surfaced
else round(n_pulled / n_surfaced, 4)
),
"ambient": ambient,
}
usage["by_source"] = by_source
# The SECTION's coverage, from the latest source to start recording — a
# figure that sums several sources is complete only once every one of them
# was being written. `_complete_from` computes that as "*".
usage.update(_coverage((note_complete or {}).get("*"), since))
out["usage"] = usage
# ── Rules, deliberately a SEPARATE block ────────────────────────────
#
# Not folded into `usage`, for two reasons and the second is the one that
# bites. The corpora differ by orders of magnitude — a few dozen eligible
# rules against thousands of notes — so one blended ratio would be the note
# ratio with a little noise on it, and the rule arm's own behaviour would
# be undetectable inside it. And `usage` is what existing callers already
# read: silently changing what it counts would move a number people have
# been comparing across windows, without telling them it now measures
# something else.
#
# `ambient` now carries the bulk deliveries — the SessionStart preload,
# and every `rules_payload` surface (#3473). Before
# they emitted, this block had no ambient key and said the absence was a
# fact about the data. It was, and it was also the thing that made the
# always-on set impossible to judge: the largest rule surface in the
# product was the one surface its own scoreboard could not see.
#
# READ THE TWO SEPARATELY, ALWAYS. `surfaced` is a claim a ranker made and
# a pull can settle. `ambient` is a delivery nobody chose, so a high count
# says the set is large and resident, never that it is useful.
rule_usage = {
"surfaced": 0, "ambient": 0,
"pulled": 0, "pulled_by_agent": 0, "pulled_by_human": 0,
"distinct_rules_surfaced": int(distinct_rules_surfaced or 0),
"distinct_rules_pulled": int(distinct_rules_pulled or 0),
}
if rule_rows is None:
# The FLAG is added, the shape is kept — matching `by_source_failed`
# one block up. A caller that renders this must not have to choose
# between crashing on a missing key and quietly showing zeros it has no
# right to: the keys let it render, and the flag tells it the zeros are
# "we could not find out" rather than "nothing happened" (#2663).
rule_usage["rule_usage_failed"] = True
else:
for event, source, n in rule_rows:
n = int(n)
if event == RULE_SURFACED:
# One definition of ranked-vs-ambient, imported rather than
# restated — the per-rule badge readout reads the same
# predicate, and two spellings of "what counts as surfaced" is
# precisely the uneven wiring #3246 found across this system.
if is_ambient(source):
rule_usage["ambient"] += n
else:
rule_usage["surfaced"] += n
elif event == RULE_PULLED:
rule_usage["pulled"] += n
# Same split, and it carries MORE weight here than for notes.
# The arm's whole claim is "this rule may apply to what you are
# writing", and only an agent opening it says the claim landed.
# A person browsing the rule list says nothing about the hint.
if source.startswith("mcp_"):
rule_usage["pulled_by_agent"] += n
else:
rule_usage["pulled_by_human"] += n
# None, not 0.0, when nothing was surfaced — matching the note block. A
# ratio of zero asserts "we showed rules and none were opened"; with an
# empty numerator AND denominator that is a claim the data does not
# support, and it is the reading that would make a brand-new install look
# like a broken one.
#
# RANKED SURFACINGS ONLY in the denominator, and this is the load-bearing
# line of the whole change. Pull-through asks "was that hint any use", and
# only a surface that CHOSE what it showed can be judged by it. Folding the
# preload in would divide the same pulls by a number that grows with every
# session and every rule added to the resident set — so enlarging the
# always-on set would DEPRESS the arm's measured precision, and trimming it
# would flatter it, neither for any reason to do with the arm. The ambient
# count sits beside it, unaveraged, and is read as size rather than skill.
rule_usage["pull_through"] = (
round(rule_usage["pulled_by_agent"] / rule_usage["surfaced"], 4)
if rule_usage["surfaced"] else None
)
rule_usage.update(_coverage((rule_complete or {}).get("*"), since))
out["rule_usage"] = rule_usage
return out
+303
View File
@@ -0,0 +1,303 @@
"""Rule usage telemetry — did a surfaced rule ever get read?
The sibling of `note_usage`, for the one retrieval surface in Scribe that
could not be measured at all.
Two event streams, deliberately independent:
- SURFACED: the write-path standing-rule arm put this rule in front of the
agent, unbidden, during a write.
- PULLED: someone then opened it in full (`get_rule`, or the REST detail
route).
WHY THIS ARM AND NOT ANOTHER. Every other surface declines most of the time —
`write_path` returns nothing on 78% of calls, `reuse_slot` on 79%, auto-inject
on 39%. The rule arm APPEARED never to have returned nothing (#3311), and this
docstring used to put that forward as the puzzle worth measuring: "either a
perfectly tuned surface or a bar it cannot fail to clear".
It was neither, and the correction belongs here rather than being quietly
deleted. The arm wrote its `retrieval_logs` row only on calls that FOUND
something (#3497), so `zero_result_calls` sat at 0 and `cleared_threshold` at
`calls` because of the shape of the code — at any threshold whatsoever. A
statistic that could not vary was read as a finding about the corpus. It is the
#2663 failure mode one level up: there the broken readout was a zero, here it
was a hundred percent, which is far better camouflage.
The reason to measure this arm survives the correction, and is stronger for it.
`retrieval_logs` records what the ranker scored, never whether the hint was any
use, so even an honest clear-rate would not settle the question. The ratio these
two streams produce is the missing half, and without it any threshold change is
a number picked off a histogram.
Design notes, mirroring `note_usage`:
- Writes are fire-and-forget through `background.spawn`, so telemetry never
adds latency to — or can break — the surface it observes. This module does
NOT carry its own copy of the strong-reference dance; `background` is the
one place that gets it right, and a fourth copy is how one of them drifts.
- Failures degrade, but never SILENTLY. `report_telemetry_failure` logs at
WARNING and drops one AppLog row per process per site. #2663 is the record
of this exact subsystem class running at zero for weeks — indistinguishable
from "nobody uses this" — because every failure went to `logger.debug`.
- Reads (`usage_for_rules`) are awaited and aggregated in one round-trip for
a whole page, never per row.
AMBIENT VS RANKED. The note twin splits ranked surfacings from ambient ones
because `enter_project` and the skill sync put records in front of the agent
without choosing them, and counting those as surfacings makes recency read as
popularity (#2477). Rules have exactly that shape: the SessionStart preload,
and every `rules_payload` surface hand over the whole
applicable set at once, chosen by nobody.
Until 2026-09-03 those bulk surfaces emitted nothing, and this module said so —
"an empty `AMBIENT_SOURCES` would be machinery pretending to a distinction the
data does not yet contain". True as far as it went, but it had a consequence
worth naming, because it is the reason the bucket exists now: the always-on
set's token cost was certain and its usefulness was UNFALSIFIABLE, permanently
and by construction. The one surface whose value was actually in question was
the one surface exempt from the scoreboard that judges every other.
They emit now. The split is the readout-level change the old note promised — a
`case()`, no migration, because `event` and `source` are plain Text with no
CHECK constraint. `source` stays granular so a reader can still tell the
preload from `enter_project` from the ranked arm.
WHY THIS NAMES THE RANKED SOURCES AND THE TWIN NAMES THE AMBIENT ONES. A
deliberate divergence, on the failure mode rather than on symmetry. Both shapes
fail silently when someone adds a surface and forgets the list, so the question
is which list changes more often — and here it is emphatically the ambient one:
there are TWO ranked rule sources (the write-path arm and the pre-tool arm)
against the seven bulk ones the preload alone contributes. Ranked sources are
added when somebody builds a ranker, which is rare and deliberate; bulk ones
appear whenever a surface hands rules over, which is most of them. Naming the
rare, slow-moving half means a newly-added bulk surface defaults to
`ambient`, which merely under-counts it, instead of defaulting to `ranked`,
which would quietly pad the pull-through denominator with surfacings nobody
chose and make the arm look imprecise. Same argument #3191 and #3430 make
against hand-kept lists: keep the list that must be remembered as short and as
slow-moving as possible.
"""
from __future__ import annotations
import logging
from sqlalchemy import case, func, select
from scribe.models import async_session
from scribe.models.base import iso
from scribe.models.rule_usage import PULLED, SURFACED, RuleUsageEvent
from scribe.services.background import report_telemetry_failure, spawn
logger = logging.getLogger(__name__)
# The surfaces that CHOSE the rules they showed. Everything else is ambient —
# see the module docstring for why the rare half is the half that gets named.
#
# Membership is the whole definition of the pull-through denominator: a ranked
# surfacing is a claim ("this rule may apply to what you are doing") that a pull
# can confirm or refute, while an ambient one is a delivery nobody decided on.
# Add a source here only when a ranker picked it.
RANKED_SOURCES = (
"write_path_rule", "pre_tool_rule", "prompt_rule",
# A reserved slot is a ranker's choice twice over — it ran a query AND
# decided a kind was worth guaranteeing a place. Left out, its line would
# be counted as bulk delivery and drop out of the denominator, so the one
# surface built because a record class kept losing would be the one whose
# hits nobody could confirm.
"preference_slot",
)
def is_ambient(source: str) -> bool:
"""Was this surfacing a bulk delivery rather than a ranked choice?
One definition, read by both the per-rule badge readout and the aggregate
in `retrieval_telemetry` — the two used to be able to disagree about what
"surfaced" counted, which is the class of drift #3246 found across the
rules system.
Sync and pure, per the service canon (#2860), but deliberately PUBLIC where
that canon says such helpers stay `_private`. The departure is the point:
a module-private copy in each caller is exactly the second definition this
exists to prevent.
"""
return source not in RANKED_SOURCES
async def _report_failure(site: str) -> None:
await report_telemetry_failure("rule_usage", site)
async def _insert_events(rows: list[dict]) -> None:
"""Persist usage rows. Best-effort: failures degrade, visibly."""
try:
async with async_session() as session:
session.add_all([RuleUsageEvent(**row) for row in rows])
await session.commit()
except Exception:
await _report_failure("write")
def _schedule(rows: list[dict]) -> None:
if not rows:
return
spawn(_insert_events(rows), site="rule_usage_write")
def record_rule_surfaced(
*, user_id: int | None, rule_ids: list[int] | set[int], source: str
) -> None:
"""Fire-and-forget: record that these rules were shown to the agent.
Takes the whole delivery at once — one insert per surfacing event, not per
rule — because a hint is a single decision and its rows should land
together.
Record what was actually SHOWN, never what was considered. For the ranked
arm that means the post-filter hits: it drops what the session already
holds (`exclude_rule_ids`) before it speaks, and a rule considered and not
shown was not surfaced. Counting those would inflate the denominator with
claims the agent never saw, which reads as a precision problem the arm does
not have.
Bulk surfaces pass their whole delivered set, which is the same rule read
from the other end — everything in a preload IS shown. `source` is what
separates the two afterwards (see `RANKED_SOURCES`); this function does not
care which kind it is recording.
"""
try:
rows = [
{
"user_id": user_id,
"rule_id": int(rid),
"event": SURFACED,
"source": source,
}
for rid in rule_ids
]
except Exception:
logger.debug("rule usage payload build failed", exc_info=True)
return
_schedule(rows)
def record_rule_pulled(*, user_id: int | None, rule_id: int, source: str) -> None:
"""Fire-and-forget: record that a rule was opened in full.
A PULL is somebody choosing to open one record. `enter_project` is NOT a
pull — they are bulk resident loads that hand over
every applicable rule at once, and counting them would swamp the signal
with the very ambient delivery the ratio exists to distinguish from.
"""
try:
rows = [
{
"user_id": user_id,
"rule_id": int(rule_id),
"event": PULLED,
"source": source,
}
]
except Exception:
logger.debug("rule usage payload build failed", exc_info=True)
return
_schedule(rows)
def empty_rule_usage() -> dict:
"""The zero readout — what a rule with no recorded events looks like.
Callers render this shape unconditionally, so a rule predating the table
reads as "never surfaced, never pulled" rather than as a missing key. That
distinction matters more here than for notes: every rule in an install
predates this table, so for a while "no events" is the normal state and it
must not look like a broken readout.
`surfaced_count` is RANKED surfacings only; `ambient_count` is the bulk
deliveries (see `RANKED_SOURCES`). The split is what keeps the badge's
"shown often, opened never → dead weight" reading honest: every rule in an
always-on set is delivered every session, so an unsplit counter would rank
the resident set as the most-surfaced rules in the install purely for being
resident.
"""
return {
"surfaced_count": 0,
"ambient_count": 0,
"pull_count": 0,
"last_surfaced_at": None,
"last_pulled_at": None,
}
async def usage_for_rules(rule_ids: list[int]) -> dict[int, dict]:
"""Aggregate usage for a set of rules: {rule_id: {counts + timestamps}}.
One GROUP BY for the whole page rather than a query per row — this feeds a
list view, so the per-row shape would be N+1 by construction. Rules with no
events come back with `empty_rule_usage()`, so the caller never has to tell
"no events" from "not in the result".
"""
ids = [int(r) for r in rule_ids]
out: dict[int, dict] = {rid: empty_rule_usage() for rid in ids}
if not ids:
return out
# Classified in SQL so the group stays small: per rule we get at most
# (surfaced-ranked, surfaced-ambient, pulled) rather than a row per distinct
# source. ONE labelled expression, bound to a variable and reused in the
# GROUP BY — a second `case()` instance there renders its own expanding-IN
# bind names under asyncpg, so the database sees two DIFFERENT expressions
# and rejects the query with a GroupingError. The note twin carries the
# same warning for the same reason, and #2663 is what it cost: the
# rejection was swallowed and every counter read zero in production while
# the writes were landing fine.
ambient = case(
(RuleUsageEvent.source.notin_(RANKED_SOURCES), True),
else_=False,
).label("ambient")
try:
async with async_session() as session:
rows = (
await session.execute(
select(
RuleUsageEvent.rule_id,
RuleUsageEvent.event,
func.count().label("n"),
func.max(RuleUsageEvent.created_at).label("last_at"),
ambient,
)
.where(RuleUsageEvent.rule_id.in_(ids))
.group_by(
RuleUsageEvent.rule_id,
RuleUsageEvent.event,
ambient,
)
)
).all()
except Exception:
# A telemetry readout must not be able to break the list it decorates —
# but it must say it failed, or a broken readout is indistinguishable
# from a corpus nobody uses (#2663).
await _report_failure("readout")
return out
for rule_id, event, n, last_at, is_amb in rows:
slot = out.get(int(rule_id))
if slot is None:
continue
if event == SURFACED and is_amb:
slot["ambient_count"] = int(n)
elif event == SURFACED:
slot["surfaced_count"] = int(n)
slot["last_surfaced_at"] = iso(last_at)
elif event == PULLED:
# Pulls are pulls regardless of what surfaced the rule — "did
# anyone ever open this?" does not depend on how it was found. Both
# halves accumulate, so this ADDS rather than assigns: a rule can
# now be pulled after a ranked hint and after a preload, and the
# split arrives as two rows.
slot["pull_count"] = slot["pull_count"] + int(n)
latest = iso(last_at)
if latest and (slot["last_pulled_at"] or "") < latest:
slot["last_pulled_at"] = latest
return out
+1 -1
View File
@@ -36,7 +36,7 @@ from scribe.models.rule_version import RuleVersion
# snapshots would bury the edits somebody is actually looking for.
SNAPSHOT_FIELDS = (
"title", "statement", "why", "how_to_apply", "when_to_apply",
"tier", "verify_with", "expires_when",
"kind", "verify_with", "expires_when",
)
+113 -261
View File
@@ -9,7 +9,6 @@ from __future__ import annotations
import logging
from collections.abc import Iterable
from datetime import datetime
from typing import Optional
from sqlalchemy import and_, delete as sql_delete, insert, or_, select
@@ -23,6 +22,7 @@ from scribe.services.verification import (
)
from scribe.services import rule_versions
from scribe.models.rule_version import RuleVersion
from scribe.services.rule_usage import record_rule_surfaced
logger = logging.getLogger(__name__)
@@ -81,7 +81,7 @@ async def update_rulebook(
rb = result.scalar_one_or_none()
if rb is None:
return None
allowed = {"title", "description", "always_on"}
allowed = {"title", "description"}
for key, value in fields.items():
if key in allowed and value is not None:
setattr(rb, key, value)
@@ -292,8 +292,10 @@ async def _assert_rulebook_rule_owned(session, rule_id: int, user_id: int) -> No
# The vocabularies migration 0088's CHECK constraints enforce. Named here so
# a caller can be corrected before the database refuses it (rule 36 keeps the
# two in step; this keeps the error readable).
TIERS = ("always_on", "conditional")
RELATION_KINDS = ("co_surfaces", "overrides", "elaborates")
# Migration 0098's CHECK. `rule` binds; `preference` is how the operator
# wants work done — see the model comment for why both live on one table.
KINDS = ("rule", "preference")
# The rule columns that are nullable, and therefore the ones where EMPTY has
@@ -307,15 +309,21 @@ NULLABLE_RULE_TEXT = (
)
def _valid_tier(tier: str) -> str:
"""An unrecognised tier falls back to always_on — the SAFE direction.
def _valid_kind(kind: str) -> str:
"""An unrecognised kind falls back to `rule` — the SAFE direction.
Getting this wrong the other way would silently stop a rule binding, which
is the one failure this whole milestone exists to prevent. A rule that
preloads when it did not need to costs context; a rule that quietly stops
preloading costs the behaviour it was written for.
The unrecognised value falls back to the binding one — the SAFE
direction, pointed at force instead
of delivery. A preference wrongly treated as binding costs a little
friction: the reader is told something is required that was only
preferred. A rule wrongly treated as a preference costs the thing the rule
was written to prevent, and costs it silently, because nothing downstream
can tell a softened rule from a preference that was always one.
Between a reader who is too careful and a reader who is not careful
enough, the typo should produce the first.
"""
return tier if tier in TIERS else "always_on"
return kind if kind in KINDS else "rule"
# Re-exported, not redefined. Notes gained the same trio in milestone 317 and
@@ -349,7 +357,13 @@ def rule_brief(rule: Rule, **extra) -> dict:
"title": rule.title,
"statement": rule.statement,
"topic_id": rule.topic_id,
"tier": rule.tier,
# Unconditional, and the payload cost is accepted deliberately. Every
# other optional key below is attached only when present, because an
# absent key should never read as a capability the record lacks. Force
# is the opposite case: a reader seeing no `kind` would have to assume
# one, and the assumption it would reach for — "this binds" — is the
# expensive one to get wrong in the other direction. Say it outright.
"kind": rule.kind or "rule",
"updated_at": rule.updated_at.date().isoformat() if rule.updated_at else None,
}
# Attached only when present (#2483: never a null key that reads as a
@@ -379,6 +393,10 @@ def _refresh_rule_embedding(rule: Rule) -> None:
swallowed because a rule that SAVED must not fail on its index refresh —
a stale vector costs a missed search hit, a raised exception costs the
write. No running loop (unit tests, scripts) is ordinary, not an error.
Detaching also means this task races anything that deletes the rule out
from under it. That is not handled here: `upsert_rule_embedding` claims
the rule's row before touching its vectors, and loses if it can't (#3262).
"""
try:
import asyncio
@@ -472,8 +490,8 @@ async def rule_detail(user_id: int, rule: Rule, system_ids: list[int] | None = N
async def create_rule(
topic_id: int, user_id: int, title: str, statement: str,
why: str = "", how_to_apply: str = "", order_index: int = 0,
when_to_apply: str = "", tier: str = "always_on", arose_from_id: int = 0,
verify_with: str = "", expires_when: str = "",
when_to_apply: str = "", arose_from_id: int = 0,
verify_with: str = "", expires_when: str = "", kind: str = "rule",
) -> Rule:
async with async_session() as session:
await _assert_topic_owned(session, topic_id, user_id)
@@ -482,7 +500,7 @@ async def create_rule(
title=title,
statement=statement,
when_to_apply=when_to_apply or None,
tier=_valid_tier(tier),
kind=_valid_kind(kind),
why=why or None,
how_to_apply=how_to_apply or None,
verify_with=verify_with or None,
@@ -500,8 +518,8 @@ async def create_rule(
async def create_project_rule(
project_id: int, user_id: int, title: str, statement: str,
why: str = "", how_to_apply: str = "", order_index: int = 0,
when_to_apply: str = "", tier: str = "always_on", arose_from_id: int = 0,
verify_with: str = "", expires_when: str = "",
when_to_apply: str = "", arose_from_id: int = 0,
verify_with: str = "", expires_when: str = "", kind: str = "rule",
) -> Rule:
"""Create a rule scoped to a single project (no rulebook ceremony).
@@ -516,7 +534,7 @@ async def create_project_rule(
title=title,
statement=statement,
when_to_apply=when_to_apply or None,
tier=_valid_tier(tier),
kind=_valid_kind(kind),
why=why or None,
how_to_apply=how_to_apply or None,
verify_with=verify_with or None,
@@ -599,89 +617,6 @@ async def list_rules(
return rulebook_rules + list(proj_result.scalars().all())
def _excluded_rulebook_ids_q(project_id: int):
"""Subquery: the always-on rulebooks this project opted out of at
inception (milestone 297) — used by every rule-resolution path so an
exclusion is total, not just cosmetic."""
from scribe.models.rulebook import project_rulebook_exclusions
return select(project_rulebook_exclusions.c.rulebook_id).where(
project_rulebook_exclusions.c.project_id == project_id
)
async def excluded_always_on_rulebooks(user_id: int, project_id: int) -> list[dict]:
"""[{id, title}] of the always-on rulebooks excluded for ``project_id``
(owner-scoped). Empty for an undecided or inherit-all project."""
from scribe.models.rulebook import project_rulebook_exclusions
if not project_id:
return []
async with async_session() as session:
rows = (
await session.execute(
select(Rulebook.id, Rulebook.title)
.join(project_rulebook_exclusions,
project_rulebook_exclusions.c.rulebook_id == Rulebook.id)
.where(
project_rulebook_exclusions.c.project_id == project_id,
Rulebook.owner_user_id == user_id,
Rulebook.deleted_at.is_(None),
)
.order_by(Rulebook.title)
)
).all()
return [{"id": rid, "title": title} for rid, title in rows]
async def list_always_on_rules(
user_id: int, limit: int = 100, project_id: int = 0,
) -> list[Rule]:
"""Return all rules from rulebooks flagged always_on for the user.
Called by the MCP tool of the same name at session start to load the
standing rules that apply regardless of which project (if any) is in
scope. Ordering matches list_rules so results are stable across calls.
``project_id`` (milestone 297): inside a project that excluded specific
always-on rulebooks at inception, those rulebooks' rules are NOT
returned — the project decided not to inherit them. 0 = the user-wide
set, which is what a session sees before a project is in scope.
"""
async with async_session() as session:
q = (
select(Rule)
.join(RulebookTopic, Rule.topic_id == RulebookTopic.id)
.join(Rulebook, RulebookTopic.rulebook_id == Rulebook.id)
.where(
Rulebook.owner_user_id == user_id,
Rulebook.always_on.is_(True),
Rule.deleted_at.is_(None),
RulebookTopic.deleted_at.is_(None),
Rulebook.deleted_at.is_(None),
# TIER (milestone 307). This is the SESSION-START call, made
# before any project is in scope — there is no area vocabulary
# to match a conditional rule against yet, so only the
# unconditional tier belongs here. A conditional rule reaches a
# session through enter_project (by area) or search (by
# meaning), not by being resident.
#
# Behaviour is unchanged until rules are actually re-tiered:
# `tier` defaults to always_on, so every existing rule still
# arrives exactly as it did.
Rule.tier == "always_on",
)
)
if project_id:
q = q.where(Rulebook.id.notin_(_excluded_rulebook_ids_q(project_id)))
result = await session.execute(
q.order_by(
Rulebook.id, RulebookTopic.order_index, Rule.order_index, Rule.title,
).limit(limit)
)
return list(result.scalars().all())
async def _fetch_owned_rule(session, rule_id: int, user_id: int) -> Optional[Rule]:
"""Fetch a rule by id, scoped to user owning either its rulebook
(via topic) or its project (via project_id). Honors soft-delete.
@@ -747,7 +682,7 @@ async def update_rule(
return None
allowed = {
"title", "statement", "why", "how_to_apply", "order_index",
"when_to_apply", "tier", "arose_from_id",
"when_to_apply", "kind", "arose_from_id",
"verify_with", "expires_when",
}
check_before = rule.verify_with
@@ -763,8 +698,8 @@ async def update_rule(
for key, value in fields.items():
if key not in allowed or value is None:
continue
if key == "tier":
value = _valid_tier(value)
if key == "kind":
value = _valid_kind(value)
elif key in NULLABLE_RULE_TEXT:
value = value or None
elif key == "arose_from_id":
@@ -773,9 +708,8 @@ async def update_rule(
# A verification stamp certifies A CHECK, not a rule. Rewrite or
# remove the check and the old stamp certifies something that no
# longer exists — so it is dropped, and the rule re-enters the sweep.
# The safe direction, for the same reason _valid_tier falls back to
# always_on: a rule wrongly listed as due costs one look, a rule
# wrongly vouched for costs the thing the sweep exists to catch.
# The safe direction: a rule wrongly listed as due costs one look, a
# rule wrongly vouched for costs the thing the sweep exists to catch.
if rule.verify_with != check_before:
rule.verified_at = None
# Same session as the edit, so the two commit together. The snapshot
@@ -998,9 +932,6 @@ async def delete_rule(rule_id: int, user_id: int) -> None:
# ── Subscriptions + get_applicable_rules ───────────────────────────────
from sqlalchemy.exc import IntegrityError
async def subscribe_project(
project_id: int, rulebook_id: int, user_id: int,
) -> None:
@@ -1078,51 +1009,6 @@ async def unsuppress_rule_for_project(
await session.commit()
async def exclude_always_on_rulebook_for_project(
project_id: int, rulebook_id: int, user_id: int,
) -> None:
"""Opt one project out of a whole ALWAYS-ON rulebook (milestone 297).
Owner-only on both sides; the rulebook must be always_on — a subscribed
rulebook is left by unsubscribing, not excluding. Idempotent."""
from scribe.models.rulebook import project_rulebook_exclusions
async with async_session() as session:
await _assert_project_owned(session, project_id, user_id)
await _assert_rulebook_owned(session, rulebook_id, user_id)
rb = await session.get(Rulebook, rulebook_id)
if rb is None or not rb.always_on:
raise ValueError(
f"rulebook {rulebook_id} is not always-on — it binds only by "
"subscription; unsubscribe_project_from_rulebook instead"
)
try:
await session.execute(
insert(project_rulebook_exclusions).values(
project_id=project_id, rulebook_id=rulebook_id,
)
)
await session.commit()
except IntegrityError:
await session.rollback() # already excluded — idempotent
async def include_always_on_rulebook_for_project(
project_id: int, rulebook_id: int, user_id: int,
) -> None:
"""Undo exclude_always_on_rulebook_for_project. Idempotent."""
from scribe.models.rulebook import project_rulebook_exclusions
async with async_session() as session:
await _assert_project_owned(session, project_id, user_id)
await session.execute(
sql_delete(project_rulebook_exclusions).where(
project_rulebook_exclusions.c.project_id == project_id,
project_rulebook_exclusions.c.rulebook_id == rulebook_id,
)
)
await session.commit()
async def suppress_topic_for_project(
project_id: int, topic_id: int, user_id: int,
) -> None:
@@ -1160,6 +1046,17 @@ async def unsuppress_topic_for_project(
await session.commit()
def _tagged_rule_ids():
"""Rules carrying at least one canonical area tag (milestone 394).
The complement is what matters: a rule NOT in this set was never narrowed
by its author, so it is general to its rulebook and applies wherever that
rulebook is subscribed. Expressed as a subquery rather than a fetched list
so the area test stays inside the one statement `limit` is counted on.
"""
return select(rule_systems.c.rule_id)
async def get_applicable_rules(
project_id: int, user_id: int, limit: int = 50,
) -> dict:
@@ -1291,7 +1188,6 @@ async def get_applicable_rules(
Rulebook.deleted_at.is_(None),
# An inception exclusion is total (milestone 297): a rulebook the
# project opted out of contributes nothing, subscribed or not.
Rulebook.id.notin_(_excluded_rulebook_ids_q(project_id)),
)
.order_by(
Rulebook.id, RulebookTopic.order_index, Rule.order_index, Rule.title,
@@ -1302,11 +1198,10 @@ async def get_applicable_rules(
rules_q = rules_q.where(Rule.id.notin_(suppressed_rule_ids))
if suppressed_topic_ids:
rules_q = rules_q.where(Rule.topic_id.notin_(suppressed_topic_ids))
# TIER (milestone 307). always_on rules are resident, as every rule was
# before tiers existed. A conditional rule is REACHABLE, and reaches
# this project only when it is tagged to an area this project actually
# works in — a deterministic tag match, never a similarity score, so
# bindingness never depends on a ranking (D7).
# AREA BINDING (milestone 307, narrowed by 394). A rule reaches this
# project when it is tagged to an area the project actually works in —
# a deterministic tag match, never a similarity score, so bindingness
# never depends on a ranking (D7).
#
# Applied in SQL rather than by filtering afterwards, so `limit` counts
# the rules that will actually be surfaced instead of counting rules
@@ -1322,10 +1217,32 @@ async def get_applicable_rules(
reachable = select(rule_systems.c.rule_id).where(
rule_systems.c.canonical_id.in_(project_area_ids)
) if project_area_ids else None
tier_clause = (Rule.tier == "always_on")
# SUBSCRIPTION IS THE SCOPE; AREAS NARROW ONLY WHERE AN AUTHOR ASKED.
#
# This read `always_on OR reachable` (milestone 307). The tier arm is
# gone, and the first attempt at 394 kept only the reachable arm — so
# a subscribed rulebook's untagged rules stopped arriving at all. That
# was wrong twice over: the query above is ALREADY scoped to rulebooks
# this project subscribed to, so the project opted in and was then
# handed a subset of what it asked for; and the milestone is explicit
# that subscription-derived rules are not what it removes. The
# integration suite caught it through a co_surfaces partner that never
# arrived because the rule it travels with had been filtered out.
#
# So: every rule in a subscribed rulebook applies, EXCEPT that a rule
# tagged to specific areas applies only to a project working in one of
# them. An untagged rule is general to its rulebook by construction —
# nobody narrowed it — while tagging is an author saying "this is
# about CI" and meaning it. That keeps D7's deterministic narrowing
# where it was asked for without inventing it where it was not.
if reachable is not None:
tier_clause = or_(tier_clause, Rule.id.in_(reachable))
rules_q = rules_q.where(tier_clause)
rules_q = rules_q.where(
or_(Rule.id.in_(reachable), Rule.id.notin_(_tagged_rule_ids())),
)
else:
# No canonical areas on this project: nothing can match by area,
# so only the untagged (general) rules apply.
rules_q = rules_q.where(Rule.id.notin_(_tagged_rule_ids()))
rule_rows = (await session.execute(rules_q)).all()
truncated = len(rule_rows) > limit
rules = [
@@ -1346,12 +1263,11 @@ async def get_applicable_rules(
)
.order_by(Rule.order_index, Rule.title)
)
if reachable is not None:
proj_rules_q = proj_rules_q.where(
or_(Rule.tier == "always_on", Rule.id.in_(reachable))
)
else:
proj_rules_q = proj_rules_q.where(Rule.tier == "always_on")
# A PROJECT'S OWN RULES ARE NOT FILTERED BY AREA, and the asymmetry
# with the family query above is the point. A family rule has to earn
# its way into this project; a rule written ON this project is scoped
# to it by construction, and filtering it again would drop rules whose
# only fault is that nobody tagged them to a System.
proj_rule_rows = (await session.execute(proj_rules_q)).all()
project_rules = [rule_brief(rule) for (rule,) in proj_rule_rows]
@@ -1387,11 +1303,10 @@ async def get_applicable_rules(
"suppressed_topics": suppressed_topics,
"truncated": truncated,
"subscribed_rulebooks": subscribed_rulebooks,
"excluded_always_on": await excluded_always_on_rulebooks(user_id, project_id),
}
def rules_payload(applicable: dict) -> dict:
def rules_payload(applicable: dict, *, user_id: int | None, source: str) -> dict:
"""The caller-facing shape of a get_applicable_rules() result.
Every surface that hands rules to an agent (enter_project, get_project,
@@ -1399,10 +1314,34 @@ def rules_payload(applicable: dict) -> dict:
same seven keys under the same names — so a reader learns them once. One
place renames `rules` → `applicable_rules` and `truncated` →
`applicable_rules_truncated`; the tools merge this into their payloads.
`excluded_always_on` (milestone 297) names the always-on rulebooks this
project decided NOT to inherit, so the departure is visible wherever the
rules are.
IT ALSO RECORDS THE SURFACING, which is why it now takes a caller and a
source. Every one of those surfaces is a bulk delivery — the applicable set
handed over whole, chosen by nobody — so this is the one place that has to
emit for all of them. Doing it per-caller instead would be five sites to
remember, and #3430 gap 2 is what that costs: the process→skill sync went
un-emitted through an entire dedicated telemetry survey because nothing
forced its surface to be accounted for.
`source` stays the CALLER's name rather than a constant, so the readout can
still separate the session handshake from a mid-session milestone read;
`RANKED_SOURCES` in `rule_usage` is what folds them back together.
Emitting from here is safe in a way emitting from `get_applicable_rules`
would not be: this function is only ever called to BUILD A REPLY. The two
other callers of the rules machinery — the write-path etag arm
(`plugin_context`) — computed a marker and showed nobody
anything, and counting those would put rules in the denominator that no
agent ever saw.
"""
record_rule_surfaced(
user_id=user_id,
rule_ids=(
[r["id"] for r in applicable.get("rules", [])]
+ [r["id"] for r in applicable.get("project_rules", [])]
),
source=source,
)
return {
"applicable_rules": applicable["rules"],
"applicable_rules_truncated": applicable["truncated"],
@@ -1410,7 +1349,6 @@ def rules_payload(applicable: dict) -> dict:
"project_rules": applicable.get("project_rules", []),
"suppressed_rules": applicable.get("suppressed_rules", []),
"suppressed_topics": applicable.get("suppressed_topics", []),
"excluded_always_on": applicable.get("excluded_always_on", []),
}
@@ -1435,86 +1373,11 @@ def rules_payload(applicable: dict) -> dict:
_ETAG_EMPTY = "empty|0"
def rules_etag(rules: list) -> str:
"""A marker for "is the set you are holding still the current one?".
`max(updated_at)` alone is not enough: DELETING a rule moves no timestamp,
and that is the single change that takes an instruction OUT of force —
the one a session most needs to hear about. The count catches it.
Instance-agnostic (rule 115): it knows nothing about any particular
rulebook, and an install with one rule or none produces a stable marker
rather than an error. "No rules" must read as a state, not as a change,
or every session on a fresh install would be told its rules had moved.
"""
if not rules:
return _ETAG_EMPTY
# A decoration must not be able to break what it decorates. This is
# computed on the SessionStart path, where raising would cost the whole
# context payload to save a hint — so a row with no usable timestamp is
# skipped rather than compared, and a set with none degrades to a
# count-only marker instead of failing. Count-only still catches a rule
# added or deleted; it just cannot see an edit, which is the right way
# round to lose information.
stamps = [
r.updated_at for r in rules
if isinstance(getattr(r, "updated_at", None), datetime)
]
if not stamps:
return f"unknown|{len(rules)}"
return f"{max(stamps).isoformat()}|{len(rules)}"
async def rules_etag_for(user_id: int, project_id: int = 0) -> str:
"""The current marker for the set a session at this scope would hold.
Deliberately built from `list_always_on_rules` rather than from a
`max()/count()` aggregate. An aggregate would be cheaper, and would have
to restate that function's definition of the set — the always_on flag,
the project's inception exclusions, the tier filter. Two definitions of
"the session's rules" is how the marker starts disagreeing with the
rules, which is worse than materialising a few dozen rows.
"""
rules = await list_always_on_rules(user_id, project_id=project_id)
return rules_etag(rules)
def rules_moved_since(rules: list, held_etag: str) -> list:
"""The rules whose text changed after `held_etag` was issued.
Returns [] when the marker matches, is unparseable, or is absent — a
caller cannot act on "something is different but I cannot say what", and
a garbled marker must not be reported as a change.
A count difference is real news that this list cannot show: a rule
DELETED since the marker was issued has no row left to return. Callers
compare counts separately.
"""
if not held_etag or held_etag == _ETAG_EMPTY:
return []
stamp, _, _count = held_etag.partition("|")
try:
held_at = datetime.fromisoformat(stamp)
except ValueError:
return []
return [r for r in rules if r.updated_at and r.updated_at > held_at]
def etag_count(held_etag: str) -> int | None:
"""How many rules the holder had. None when the marker cannot be read."""
_stamp, _, count = (held_etag or "").partition("|")
try:
return int(count)
except ValueError:
return None
# ── The staleness sweep (milestone 312) ────────────────────────────────
async def rules_due_for_verification(
user_id: int,
older_than_days: int = 0,
tier: str = "",
never_only: bool = False,
) -> list[Rule]:
"""Rules that carry a check, oldest verification first, never-checked top.
@@ -1543,20 +1406,12 @@ async def rules_due_for_verification(
older_than_days: only rules last verified longer ago than this.
Never-checked rules always qualify — they are the most overdue
thing there is. 0 = no age filter.
tier: "always_on" or "conditional" to narrow. Raises on anything else
rather than falling back: _valid_tier's silent always_on default
is right for a WRITE (the safe direction is to keep binding), and
wrong for a FILTER, where it would quietly answer a different
question than the one asked.
never_only: only rules that have never been verified.
"""
from datetime import datetime, timedelta, timezone
from scribe.models.project import Project
if tier and tier not in TIERS:
raise ValueError(f"tier must be one of {TIERS}, got {tier!r}")
async with async_session() as session:
stmt = (
select(Rule)
@@ -1579,8 +1434,6 @@ async def rules_due_for_verification(
),
)
)
if tier:
stmt = stmt.where(Rule.tier == tier)
if never_only:
stmt = stmt.where(Rule.verified_at.is_(None))
elif older_than_days > 0:
@@ -1606,7 +1459,6 @@ def verification_row(rule: Rule) -> dict:
"id": rule.id,
"title": rule.title,
"statement": rule.statement,
"tier": rule.tier,
"topic_id": rule.topic_id,
"project_id": rule.project_id,
"when_to_apply": rule.when_to_apply or "",
+32 -3
View File
@@ -59,14 +59,18 @@ def tool_doc(module: str, name: str) -> str:
return _re.sub(r"\s+", " ", fn.__doc__)
def compiled_sql(element) -> str:
def compiled_sql(element, dialect=None) -> str:
"""A SQLAlchemy clause or statement rendered as literal SQL text.
For asserting on the shape of a predicate without a database — which is how
the visibility clauses and the knowledge facets are both tested. Was a
private copy in each of those modules before #3128 needed a third.
Pass `dialect` when the assertion is about something only one backend
renders — a Postgres row-lock mode, say. The generic dialect is enough for
a predicate's shape and would quietly drop the rest.
"""
return str(element.compile(compile_kwargs={"literal_binds": True}))
return str(element.compile(dialect=dialect, compile_kwargs={"literal_binds": True}))
def make_mock_session() -> AsyncMock:
@@ -221,7 +225,13 @@ def fake_rule(**attrs) -> MagicMock:
# Named for the note-2109 reason the whole helper exists: unnamed,
# `when_to_apply` and `arose_from_id` would be truthy MagicMocks and
# rule_brief would attach both keys on every stand-in.
"when_to_apply": None, "tier": "always_on", "arose_from_id": None,
"when_to_apply": None, "arose_from_id": None,
# Named for the same reason one line up, and it bites harder here.
# `rule_brief` and `to_dict` both emit `kind or "rule"`, and a
# MagicMock is truthy — so an unnamed `kind` would put a MagicMock
# where every payload promises a force, and every stand-in rule would
# read as neither a rule nor a preference.
"kind": "rule",
# Same reason, and the same trap one field further on: an unnamed
# `verify_with` is a truthy MagicMock, so every stand-in rule would
# claim to carry a check and rule_brief would stamp a MagicMock date
@@ -231,6 +241,25 @@ def fake_rule(**attrs) -> MagicMock:
}, attrs)
def plain_rule_detail():
"""Stub `rulebooks_svc.rule_detail` down to the record's own dict.
Every rule-tool unit test needs it and none of them wants it: the real
`rule_detail` reads the rule's Systems and its typed edges from the
database, which a unit test has none of. What these tests assert is that
the TOOL forwarded the right arguments, so the seam is stubbed the same
way the create/update calls themselves already are.
Consolidated here on its second copy, per this module's own reason for
existing (#2825) — two stubs for one seam drift apart quietly, and a test
stubbing the seam slightly differently is a test asserting something
slightly different than it appears to.
"""
async def _detail(_uid, rule, _system_ids=None):
return rule.to_dict()
return patch("scribe.mcp.tools.rulebooks.rulebooks_svc.rule_detail", _detail)
class FakeMCP:
"""Stand-in for the FastMCP server a tool module's ``register(mcp)`` is
handed: records the ``name=`` of every ``@mcp.tool(...)`` registration in
+118
View File
@@ -0,0 +1,118 @@
"""The embedding refresh must LOSE to a delete, not race it (#3262).
Both upserts replace a record's vectors as delete-then-insert. That takes two
row locks — the chunk rows, then the parent row via the insert's foreign key —
in the exact reverse of the order a cascading delete of the parent takes them.
Postgres calls that a deadlock and kills one side at random, which sometimes
means killing the user's delete.
These pin the ORDER, not the outcome: the claim on the parent goes first, and
when the claim fails nothing else in the transaction runs. Compiling the
statement is the only way to assert on a lock mode without a database — the
integration twin (test_integration_embedding_yields_to_delete.py) proves the
behaviour against a real one.
"""
from unittest.mock import AsyncMock, MagicMock, patch
from sqlalchemy.dialects import postgresql
from sqlalchemy.exc import OperationalError
from scribe.services import embeddings as emb
from tests.helpers import compiled_sql
ONE_VECTOR = [[0.0] * 384]
# The lock mode is a Postgres extension — the generic dialect renders a plain
# FOR UPDATE and would pass an assertion that proves nothing.
PG = postgresql.dialect()
def _mock_session(lock_result: object = 7, execute_side_effect=None):
"""A session stand-in whose first execute answers the parent-row claim."""
session = MagicMock()
claimed = MagicMock()
claimed.scalar_one_or_none.return_value = lock_result
if execute_side_effect is not None:
session.execute = AsyncMock(side_effect=execute_side_effect)
else:
session.execute = AsyncMock(return_value=claimed)
session.commit = AsyncMock()
session.add = MagicMock()
ctx = MagicMock()
ctx.__aenter__ = AsyncMock(return_value=session)
ctx.__aexit__ = AsyncMock(return_value=False)
return session, ctx
def _lock_unavailable() -> OperationalError:
"""What asyncpg raises through SQLAlchemy when NOWAIT can't take the row."""
return OperationalError("SELECT ...", {}, Exception("lock not available"))
async def test_a_note_refresh_claims_the_row_before_rewriting_its_vectors():
"""The claim is FIRST, and it is FOR KEY SHARE NOWAIT.
FOR KEY SHARE because that is exactly the lock the insert's foreign key
takes anyway — it conflicts with a delete of the note and with nothing
else, so an ordinary edit is unaffected. NOWAIT because the whole point is
to lose immediately rather than queue up behind the delete and hold the
chunk rows while doing it.
"""
session, ctx = _mock_session()
with (
patch.object(emb, "async_session", return_value=ctx),
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
):
await emb.upsert_note_embedding(7, 42, "T", "a short body")
claim, replace = [c.args[0] for c in session.execute.call_args_list][:2]
assert compiled_sql(claim, dialect=PG).startswith("SELECT notes.id")
assert "FOR KEY SHARE NOWAIT" in compiled_sql(claim, dialect=PG)
assert compiled_sql(replace, dialect=PG).startswith("DELETE FROM note_embeddings")
session.add.assert_called()
async def test_a_rule_refresh_claims_the_row_before_rewriting_its_vectors():
"""The rule twin — the path the reported deadlock actually took."""
session, ctx = _mock_session()
with (
patch.object(emb, "async_session", return_value=ctx),
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
):
await emb.upsert_rule_embedding(9, "T", "a short statement", "on write")
claim, replace = [c.args[0] for c in session.execute.call_args_list][:2]
assert compiled_sql(claim, dialect=PG).startswith("SELECT rules.id")
assert "FOR KEY SHARE NOWAIT" in compiled_sql(claim, dialect=PG)
assert compiled_sql(replace, dialect=PG).startswith("DELETE FROM rule_embeddings")
session.add.assert_called()
async def test_a_record_being_deleted_is_left_alone_rather_than_raced():
"""The claim failing ends the write — it does not fall through to the
delete-and-insert that would take the locks in the losing order."""
session, ctx = _mock_session(execute_side_effect=_lock_unavailable())
with (
patch.object(emb, "async_session", return_value=ctx),
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
):
await emb.upsert_rule_embedding(9, "T", "a short statement", "on write")
assert session.execute.await_count == 1, "it stopped at the claim"
session.add.assert_not_called()
session.commit.assert_not_awaited()
async def test_a_record_already_gone_is_not_re_embedded():
"""A vector inserted for a row that no longer exists is either a foreign
key violation or, worse, a resurrected chunk. Nothing to refresh."""
session, ctx = _mock_session(lock_result=None)
with (
patch.object(emb, "async_session", return_value=ctx),
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
):
await emb.upsert_note_embedding(7, 42, "T", "a short body")
assert session.execute.await_count == 1
session.add.assert_not_called()
session.commit.assert_not_awaited()
+128
View File
@@ -0,0 +1,128 @@
"""Every request the web UI makes has a deadline (rule 156).
A source-inspection guard in the unit lane — there is no frontend test runner,
and this is a property of the source rather than of a rendered result, so
reading the source is the honest way to check it.
WHY. `fetch`'s default is to wait as long as the browser will. That is not a
long timeout, it is the absence of one, and there is no state a surface can
render for "pending forever" that is not a lie — the spinner that never
resolves is indistinguishable from work still in progress. Rule 156 names
`fetch` specifically:
When a library's default is "wait indefinitely" — `fetch`, most HTTP
clients, a bare `await` on a stream — supplying the deadline is part of
using it, not a hardening pass for later.
Before this guard, no request in the app carried one.
"""
from __future__ import annotations
import re
from pathlib import Path
FRONTEND = Path(__file__).resolve().parents[1] / "frontend" / "src"
CLIENT = FRONTEND / "api" / "client.ts"
def _call_text(src: str, start: int) -> str:
"""The source of one `fetch(...)` call, from its open paren to its close.
Naive paren balancing. Adequate because every call site here passes an
object literal, and a construct complex enough to defeat it is one worth
looking at by hand anyway.
"""
depth = 0
for i in range(start, len(src)):
if src[i] == "(":
depth += 1
elif src[i] == ")":
depth -= 1
if depth == 0:
return src[start:i + 1]
return src[start:]
def _fetch_calls() -> list[tuple[Path, str]]:
calls: list[tuple[Path, str]] = []
for path in list(FRONTEND.rglob("*.ts")) + list(FRONTEND.rglob("*.vue")):
src = path.read_text()
for m in re.finditer(r"\bfetch\(", src):
calls.append((path, _call_text(src, m.end() - 1)))
return calls
def test_every_fetch_passes_a_signal():
"""No bare `fetch` anywhere in the frontend.
Stated on the SIGNAL rather than on a timeout value, because the two
legitimate shapes here produce different values and only share this: an
ordinary call takes the client's default, a stream bounds its CONNECT and
then deliberately runs unbounded, and a bulk transfer passes minutes. What
they must all do is pass something.
"""
naked = [
f"{path.relative_to(FRONTEND)}: {call[:70]}"
for path, call in _fetch_calls()
if "signal:" not in call
]
assert not naked, (
"these fetch calls carry no AbortSignal, so they wait forever "
"(rule 156):\n " + "\n ".join(naked)
)
def test_the_client_applies_its_deadline_by_default():
"""The specific regression that would silently undo this.
An earlier pass (#3329) made `timeoutMs` OPT-IN and used it at exactly one
call site, which left ~330 others waiting forever while the mechanism
looked present. Reverting to that shape would not fail the guard above —
every call would still reach `fetch` through `request()` — so the default
is pinned here separately.
`??` is the operative character: `opts?.timeoutMs || DEFAULT` would treat
an explicit 0 as "use the default", and `opts?.timeoutMs` alone would
reinstate the opt-in bug.
"""
src = CLIENT.read_text()
assert "opts?.timeoutMs ?? DEFAULT_TIMEOUT_MS" in src, (
"request() must fall back to DEFAULT_TIMEOUT_MS — without it the "
"deadline is opt-in again and almost nothing opts in"
)
def test_a_timeout_arrives_as_the_error_shape_callers_already_handle():
"""Rule 156's second half: expiry surfaces as a NAMED failure.
A raw `DOMException: TimeoutError` reaches `apiErrorMessage(e, fallback)`
as an object with no `body`, so every catch site in the app would print its
generic fallback and the timeout would be invisible in exactly the
situation it exists to expose. Rethrowing as `ApiError` is what makes the
other ~330 call sites report it without being edited.
"""
src = CLIENT.read_text()
assert 'e.name === "TimeoutError"' in src, (
"request() must recognise a timeout specifically"
)
assert "new ApiError(CLIENT_TIMEOUT_STATUS" in src, (
"a timeout must be rethrown as ApiError so apiErrorMessage can read it"
)
def test_a_deliberate_cancellation_is_not_reported_as_a_timeout():
"""Only `TimeoutError` is converted, never `AbortError`.
A caller that cancelled its own request — a superseded search, a closed
stream — must not have that surfaced to the user as a server failure. The
guard is that the conversion is gated on the name, which the assertion
above already pins; this states the intent so the gate is not "simplified"
into catching every abort.
"""
src = CLIENT.read_text()
convert = src[src.index("async function request<"):]
convert = convert[:convert.index("\n}")]
assert "AbortError" not in convert, (
"request() must not convert AbortError — a deliberate cancellation is "
"not a timeout"
)
+17 -18
View File
@@ -1,12 +1,14 @@
"""Project inception (milestone 297) — step 1: the record's shape.
The WHY a project inherits what it does lives on projects.inception; the
opt-out of an always-on rulebook is its own association table. Pure
The WHY a project inherits what it does lives on projects.inception. Pure
validation is pinned here; the effects are step 3's integration tests.
The opt-out of an always-on rulebook had its own association table until
milestone 394. With no always-on tier there is nothing to opt out OF — a
rulebook binds a project only by subscription — so the table and the test
that pinned its shape both went with it.
"""
from scribe.models import Base
from scribe.models.project import Project
from scribe.models.rulebook import project_rulebook_exclusions
from scribe.services.inception import (
CHOICE_KEYS, INCEPTION_VIAS, is_decided, normalize_choices, validate_inception,
)
@@ -23,37 +25,34 @@ def test_project_carries_an_inception_record_and_to_dict_shows_it():
assert INCEPTION_VIAS == ("mcp", "ui", "legacy")
def test_exclusions_table_is_the_suppressions_sibling():
t = Base.metadata.tables["project_rulebook_exclusions"]
assert project_rulebook_exclusions is t
assert {c.name for c in t.primary_key.columns} == {"project_id", "rulebook_id"}
fks = {fk.column.table.name: fk.ondelete for c in t.columns for fk in c.foreign_keys}
assert fks == {"projects": "CASCADE", "rulebooks": "CASCADE"}
def test_validate_inception_pins_the_choice_vocabulary():
assert CHOICE_KEYS == ("exclude_always_on_rulebooks", "subscribe_rulebooks", "design_system_id", "seed_systems")
assert CHOICE_KEYS == ("subscribe_rulebooks", "design_system_id", "seed_systems")
assert validate_inception({}) is None
assert validate_inception({"exclude_always_on_rulebooks": [1], "subscribe_rulebooks": [2],
assert validate_inception({"subscribe_rulebooks": [2],
"design_system_id": 3, "seed_systems": True}) is None
assert validate_inception({"design_system_id": None}) is None
assert "must be an object" in validate_inception([])
assert "unknown inception choice" in validate_inception({"repo": "x"})
assert "list of rulebook ids" in validate_inception({"exclude_always_on_rulebooks": "1"})
# The retired exclusion key is now an UNKNOWN key rather than a typed one,
# which is the right error: a caller still passing it is asking for a
# choice that no longer exists, and silently ignoring it would let them
# believe a rulebook had been declined.
assert "unknown inception choice" in validate_inception(
{"exclude_always_on_rulebooks": [1]})
assert "list of rulebook ids" in validate_inception({"subscribe_rulebooks": [0]})
assert "list of rulebook ids" in validate_inception({"subscribe_rulebooks": [True]})
assert "both excluded and subscribed" in validate_inception(
{"exclude_always_on_rulebooks": [1, 2], "subscribe_rulebooks": [2]})
assert "positive id or null" in validate_inception({"design_system_id": 0})
assert "positive id or null" in validate_inception({"design_system_id": True})
assert "true or false" in validate_inception({"seed_systems": "yes"})
def test_normalize_choices_is_canonical_and_complete():
out = normalize_choices({"subscribe_rulebooks": [3, 1, 3], "exclude_always_on_rulebooks": [2]})
assert out == {"exclude_always_on_rulebooks": [2], "subscribe_rulebooks": [1, 3],
out = normalize_choices({"subscribe_rulebooks": [3, 1, 3]})
assert out == {"subscribe_rulebooks": [1, 3],
"design_system_id": None, "seed_systems": False}
assert normalize_choices(None) == {"exclude_always_on_rulebooks": [], "subscribe_rulebooks": [],
assert normalize_choices(None) == {"subscribe_rulebooks": [],
"design_system_id": None, "seed_systems": False}
-66
View File
@@ -1,66 +0,0 @@
"""Milestone 297 step 2 — always-on exclusions reach every rule surface.
The SQL is the integration lane's; here the contracts: rules_payload carries
the seventh key, list_always_on_rules takes project_id, the session-start
block names the excluded rulebooks, and the MCP tools mount.
"""
from unittest.mock import AsyncMock, patch
import pytest
from scribe.services.rulebooks import rules_payload
def test_rules_payload_carries_excluded_always_on_as_the_seventh_key():
out = rules_payload({
"rules": [], "truncated": False, "subscribed_rulebooks": [],
"excluded_always_on": [{"id": 1, "title": "Family"}],
})
assert set(out) == {
"applicable_rules", "applicable_rules_truncated", "subscribed_rulebooks",
"project_rules", "suppressed_rules", "suppressed_topics", "excluded_always_on",
}
assert out["excluded_always_on"] == [{"id": 1, "title": "Family"}]
# An older applicable dict without the key still renders (empty list).
assert rules_payload({"rules": [], "truncated": False, "subscribed_rulebooks": []})["excluded_always_on"] == []
def test_list_always_on_rules_service_and_tool_take_a_project_id():
import inspect
from scribe.mcp.tools import rulebooks as tools
from scribe.services import rulebooks as svc
assert "project_id" in inspect.signature(svc.list_always_on_rules).parameters
assert "project_id" in inspect.signature(tools.list_always_on_rules).parameters
@pytest.mark.asyncio
async def test_session_context_names_the_excluded_always_on_rulebooks():
from types import SimpleNamespace as NS
from scribe.services.plugin_context import build_session_context
rules = [NS(id=1, title="`dev` is home", topic_id=1, statement="x")]
project = NS(id=9, title="Widget", goal="", design_system_id=None)
with patch("scribe.services.plugin_context.rulebooks_svc.list_always_on_rules",
AsyncMock(return_value=rules)) as lao, \
patch("scribe.services.plugin_context.rulebooks_svc.excluded_always_on_rulebooks",
AsyncMock(return_value=[{"id": 5, "title": "Design standards"}])), \
patch("scribe.services.plugin_context._topic_titles", AsyncMock(return_value={1: "git"})), \
patch("scribe.services.plugin_context.projects_svc.get_project", AsyncMock(return_value=project)), \
patch("scribe.services.plugin_context.notes_svc.list_notes", AsyncMock(return_value=([], 0))), \
patch("scribe.services.plugin_context.rulebooks_svc.get_applicable_rules",
AsyncMock(return_value={"rules": [], "truncated": False, "subscribed_rulebooks": [],
"project_rules": [], "suppressed_rules": [],
"suppressed_topics": [], "excluded_always_on": []})):
out = await build_session_context(user_id=7, project_id=9)
# The always-on set was asked FOR THIS PROJECT, and the departure is named.
assert lao.await_args.kwargs.get("project_id") == 9
assert "Excluded for this project by its inception decision" in out["context"]
assert "Design standards (#5)" in out["context"]
def test_exclusion_routes_are_registered():
from scribe.app import create_app
rules = {r.rule for r in create_app().url_map.iter_rules()}
assert "/api/projects/<int:project_id>/exclusions/rulebooks/<int:rulebook_id>" in rules
+130 -20
View File
@@ -1,4 +1,4 @@
"""The instruction surfaces must agree that the agent pulls the rules itself.
"""The instruction surfaces must agree on how a rule reaches a session.
WHY THIS EXISTS
@@ -17,16 +17,23 @@ an extended period, and nothing announced it. An agent trusting the push would
have run with no binding rules and no signal — while those rules govern branch,
commit, push and other hard-to-reverse actions.
The asymmetry is the whole argument, and it is what these tests pin: pulling
when a push also arrived costs one redundant call; not pulling when the push
never came costs the operator's rules entirely.
The asymmetry is the whole argument, and it is what these tests pin: asking
when a rule had already arrived costs one redundant call; not asking when
nothing arrived costs the operator's rules entirely.
MILESTONE 394 SHARPENED IT RATHER THAN RETIRING IT. There is no longer a
resident set to pull, so "no rule in front of me" went from a rare and
suspicious state to the ordinary state of most turns. The instruction that
used to be supplementary — go and ask — is now the only route a rule has, and
the surfaces must additionally say what an EMPTY session means, or a session
reads silence as permission on nearly every turn.
WHAT THIS DOES NOT DO
It cannot tell whether two surfaces contradict each other in prose generally —
that needs a reader. It pins the one instruction whose absence is known to be
that needs a reader. It pins the instructions whose absence is known to be
load-bearing, and the specific shape the #2497 defect took: naming the push
without also stating the pull.
without also stating how to ask.
"""
from __future__ import annotations
@@ -34,8 +41,33 @@ import pathlib
ROOT = pathlib.Path(__file__).resolve().parents[1]
# The pull instruction, however a surface phrases the surrounding prose.
PULL = "list_always_on_rules"
# THE PULL IS NOW THE ASK (milestone 394). This was `list_always_on_rules`,
# the call that fetched the resident set. There is no resident set and no such
# call: a rule reaches a session by retrieval, and the only thing a session can
# DO about a rule it has not been handed is go looking for one.
#
# So the two halves this file used to pin separately — "pull the resident set"
# and "and retrieve the conditional ones too" — have collapsed into one
# instruction, and it is the load-bearing one rather than the supplementary
# one it used to be.
ASK = 'content_type="rule"'
# A surface must also say what an EMPTY session means, which is the half that
# is newly dangerous. Under residency, "no rule in front of me" was rare and
# suspicious. Under retrieval it is the ordinary state of most turns, so a
# session that reads it as "there is no rule" is wrong on nearly every turn
# rather than occasionally — the #3720 defect at session scale.
#
# Claim phrases, not a single word, for the reason BINDING_CLAIMS gives below:
# a bare "matched" or "silence" appears in prose that is not making this claim
# at all. A surface passes by asserting the distinction however it words it.
ABSENCE_CLAIMS = (
"nothing matched",
"is not the same as \"there is no rule",
"never \"there is no rule",
"silence is not absence",
"not evidence there is none",
)
# Surfaces a session loads before substantive work. Hand-written because
# "is this a session-start surface?" is an editorial fact, not a derivable one —
@@ -67,21 +99,51 @@ def _all_surfaces() -> list[tuple[str, str]]:
return found
def test_every_session_start_surface_states_the_pull():
def test_every_session_start_surface_states_the_ask():
"""Retrieval is the only delivery, so asking is the only recourse."""
missing = []
for path in SESSION_START_SURFACES:
assert path.exists(), (
f"{path.relative_to(ROOT)} is gone — it was one of the surfaces "
f"carrying the load-the-rules instruction. If it moved, update "
f"carrying the rules instruction. If it moved, update "
f"SESSION_START_SURFACES; if it was retired, check the instruction "
f"still lives somewhere a fresh session reads."
)
if PULL not in path.read_text():
if ASK not in path.read_text():
missing.append(str(path.relative_to(ROOT)))
assert not missing, (
f"these surfaces no longer tell the agent to call {PULL}(): {missing}. "
f"The rules are pull-only and the push is best-effort, so a surface "
f"that omits this leaves a session bound by nothing (#2198, #2497)."
f"these surfaces never tell the agent how to ask for a rule "
f"({ASK}): {missing}. Nothing is pushed and nothing is resident, so a "
f"surface that omits this leaves a session with no way to reach a rule "
f"it was not handed — bound by nothing (#2198, #2497, milestone 394)."
)
def test_every_session_start_surface_says_an_empty_session_is_not_an_empty_rulebook():
"""The half that got dangerous when residency went away.
Under the old model a session opened holding every applicable rule, so
"nothing is in front of me" was a rare state and a suspicious one. Under
retrieval it is the NORMAL state of most turns. A surface that describes
where rules come from, without also saying what their absence means, leaves
a session reading silence as permission — on nearly every turn rather than
occasionally.
That is #3720's defect ("absence reads as non-existence") moved from a
readout to the session itself, and this milestone is what makes every
session start in the absent state.
"""
missing = []
for path in SESSION_START_SURFACES:
text = path.read_text().lower()
if not any(c.lower() in text for c in ABSENCE_CLAIMS):
missing.append(str(path.relative_to(ROOT)))
assert not missing, (
f"these surfaces say how a rule arrives but never what it means when "
f"none does: {missing}. 'No rule arrived' means 'nothing matched', "
f"never 'there is no rule' — and only one of those has been checked. "
f"Say it however you like; one of {ABSENCE_CLAIMS} is what this looks "
f"for."
)
@@ -195,7 +257,7 @@ def test_displaced_topics_live_on_a_delivered_surface():
)
def test_no_surface_names_the_push_without_stating_the_pull():
def test_no_surface_names_the_push_without_stating_the_ask():
"""The exact shape #2497 took.
Mentioning the SessionStart hook is fine and often useful. Mentioning it
@@ -204,11 +266,59 @@ def test_no_surface_names_the_push_without_stating_the_pull():
"""
offenders = [
label for label, text in _all_surfaces()
if "SessionStart" in text and PULL not in text
if "SessionStart" in text and ASK not in text
]
assert not offenders, (
f"these surfaces describe the SessionStart push but never state the "
f"explicit pull: {offenders}. The push is a delivery optimisation, not "
f"the bridge — it can be absent without saying so. Name it if it helps, "
f"but say to call {PULL}() regardless."
f"these surfaces describe the SessionStart push but never state how to "
f"ask: {offenders}. The push is a delivery optimisation, not the "
f"bridge — it can be absent without saying so, and since milestone 394 "
f"it carries no rules at all. Name it if it helps, but say how to ask "
f"({ASK}) regardless."
)
# ── force: a surface that says rules bind must say what does not ────────
#
# Added with the preference kind (milestone 399). Before it, "rules bind" was
# the whole truth and every surface said so flatly. It is now half of one, and
# the half that is missing is the dangerous half to omit: a session reading
# only "rules bind" and then receiving a preference has been told, by the most
# authoritative surface it has, to treat it as binding.
#
# That failure is silent in both directions. Treating a preference as a rule
# produces a session that refuses to proceed over something the operator only
# preferred; and it removes the reason preferences exist, which is that they
# can be brought up to date rather than obeyed.
#
# Same bargain as every test in this file: STRUCTURE, not wording. A surface
# passes by mentioning the other kind at all, so the prose stays free.
#
# PINNED ON THE CLAIM, NOT THE WORD "bind". A bare substring also matches
# `bind_repo`, `list_repo_bindings` and the DNS-rebinding comment in
# server.py — so it would one day fail a skill that mentions repo binding and
# has nothing to do with force, which is rule 167's named failure: a guard
# raising a false alarm about the very thing it protects. These phrases are
# the ones that actually assert bindingness to a reader.
BINDING_CLAIMS = (
"rules bind",
"binding rules",
"rules are binding",
"treat every one as binding",
)
OTHER_KIND = "preference"
def test_a_surface_claiming_rules_bind_also_names_what_does_not():
offenders = [
label for label, text in _all_surfaces()
if any(c in text.lower() for c in BINDING_CLAIMS)
and OTHER_KIND not in text.lower()
]
assert not offenders, (
f"these surfaces tell a session that rules bind and never mention "
f"preferences: {offenders}. A preference arrives through the same arms "
f"and renders in the same line shape, so a surface that describes only "
f"the binding kind is read as covering both — and the session treats "
f"'how the operator likes this done' as something it may not proceed "
f"past. Name the other kind, however briefly."
)
@@ -0,0 +1,294 @@
"""Real-Postgres round trip for rule_usage_events (milestone 333 step 1).
**This file is the reason the table exists.** `rule_usage_events` could have
been a `rule_id` column on `note_usage_events` — the row carries no
note-specific field and the readout is the same shape, which is the strongest
case for sharing that note #3163 admits. What decided against it is identity at
restore, and that is a claim only a real round trip can support.
The failure it guards is the quiet kind. `note_usage_events`'s importer maps
`note_id` through `note_id_map`; a rule id parked in that column comes back
attached to whatever note happens to hold that number in the target database.
Not dropped — REATTACHED. The restore reports success, the counters are
populated, and every one of them is about the wrong record. Nothing downstream
can detect it, because a usage row has no other field to disagree with.
So the assertions below are about WHICH MAP resolved the id, and they are
written to fail if the answer ever becomes "the note one" or "neither".
Same shape as `test_integration_backup_rule_version_roundtrip.py`, which guards
`rule_versions` against #3182's `arose_from_id` trap on the same seam.
"""
import pytest
import pytest_asyncio
from sqlalchemy import select
from scribe.models import async_session
from scribe.models.note import Note
from scribe.models.rule_usage import PULLED, SURFACED, RuleUsageEvent
from scribe.models.rulebook import Rule, Rulebook, RulebookTopic
from scribe.models.user import User
from scribe.services import backup
from tests.helpers import ensure_user
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
OWNER_USERNAME = "rule_usage_roundtrip_owner"
RESTORED_USERNAME = "rule_usage_roundtrip_restored"
async def _purge_books(username: str) -> None:
"""user -> rulebook -> topic -> rule is ON DELETE CASCADE the whole way,
so dropping the books clears the rules this file made.
`rule_usage_events` is deliberately FK-FREE, so its rows do NOT cascade —
that is the property under test elsewhere (telemetry outlives what it
describes). They are cleared explicitly below.
"""
async with async_session() as s:
users = (await s.execute(
select(User).where(User.username == username)
)).scalars().all()
for user in users:
books = (await s.execute(
select(Rulebook).where(Rulebook.owner_user_id == user.id)
)).scalars().all()
for book in books:
await s.delete(book)
for note in (await s.execute(
select(Note).where(Note.user_id == user.id)
)).scalars().all():
await s.delete(note)
await s.commit()
async def _purge_usage(rule_ids: set[int]) -> None:
if not rule_ids:
return
async with async_session() as s:
for ev in (await s.execute(
select(RuleUsageEvent).where(RuleUsageEvent.rule_id.in_(rule_ids))
)).scalars().all():
await s.delete(ev)
await s.commit()
async def _purge_restored() -> None:
await _purge_books(RESTORED_USERNAME)
async with async_session() as s:
for user in (await s.execute(
select(User).where(User.username == RESTORED_USERNAME)
)).scalars().all():
await s.delete(user)
await s.commit()
@pytest_asyncio.fixture(autouse=True)
async def _no_leftovers():
"""SETUP ONLY — see the sibling file for why a database call after a
`yield` here orphans a pooled connection and breaks unrelated tests."""
await _purge_restored()
await _purge_books(OWNER_USERNAME)
@pytest_asyncio.fixture
async def source():
"""One rule with a surfaced/pulled pair — plus a NOTE that will hold the
rule's id in the restored database.
That note is the whole trick. Without it, a restore that ran rule ids
through `note_id_map` would simply drop them and the test would read as a
pass-by-absence. With it, the wrong map produces a plausible, populated,
entirely wrong result — which is the failure actually being guarded.
"""
async with async_session() as s:
owner = await ensure_user(s, OWNER_USERNAME)
uid = owner.id
await s.commit()
async with async_session() as s:
book = Rulebook(owner_user_id=uid, title="Environment facts")
s.add(book)
await s.flush()
topic = RulebookTopic(rulebook_id=book.id, title="ci")
s.add(topic)
await s.flush()
rule = Rule(
topic_id=topic.id,
title="A wait with no deadline is a bug",
statement="Every wait on something that can fail to answer carries one.",
)
s.add(rule)
# A note in the same export, so the target database has a note id to
# collide with. Its own id is irrelevant; what matters is that the
# note map is populated and would resolve to something.
note = Note(user_id=uid, title="a note that must not receive rule telemetry",
body="decoy")
s.add(note)
await s.flush()
s.add_all([
RuleUsageEvent(
user_id=uid, rule_id=rule.id,
event=SURFACED, source="write_path_rule",
),
RuleUsageEvent(
user_id=uid, rule_id=rule.id,
event=PULLED, source="mcp_get_rule",
),
# No actor. The arm can fire for an unauthenticated hook call, and
# a user who later leaves must not take the evidence with them.
RuleUsageEvent(
user_id=None, rule_id=rule.id,
event=SURFACED, source="write_path_rule",
),
])
await s.commit()
book_id, rule_id, note_id = book.id, rule.id, note.id
async with async_session() as s:
user_rows = backup._user_rows(
[(await s.execute(select(User).where(User.id == uid))).scalars().one()]
)
book_rows = backup._rulebook_rows(
[(await s.execute(select(Rulebook).where(Rulebook.id == book_id)))
.scalars().one()]
)
topic_rows = backup._topic_rows(
(await s.execute(
select(RulebookTopic).where(RulebookTopic.rulebook_id == book_id)
)).scalars().all()
)
rule_rows = backup._rule_rows(
[(await s.execute(select(Rule).where(Rule.id == rule_id))).scalars().one()]
)
note_rows = backup._note_rows(
[(await s.execute(select(Note).where(Note.id == note_id))).scalars().one()]
)
usage_rows = backup._rule_usage_event_rows(
(await s.execute(
select(RuleUsageEvent).where(RuleUsageEvent.rule_id == rule_id)
.order_by(RuleUsageEvent.id)
)).scalars().all()
)
user_rows[0]["username"] = RESTORED_USERNAME
yield {
"payload": {
"version": backup.BACKUP_VERSION,
"users": user_rows,
"rulebooks": book_rows,
"rulebook_topics": topic_rows,
"rules": rule_rows,
"notes": note_rows,
"rule_usage_events": usage_rows,
},
"source_rule_id": rule_id,
"source_user_id": uid,
}
await _purge_usage({rule_id})
async with async_session() as s:
book = await s.get(Rulebook, book_id)
if book is not None:
await s.delete(book)
note = await s.get(Note, note_id)
if note is not None:
await s.delete(note)
await s.commit()
@pytest_asyncio.fixture
async def restored(source):
await backup.restore_full_backup(source["payload"])
async with async_session() as s:
user = (await s.execute(
select(User).where(User.username == RESTORED_USERNAME)
)).scalars().first()
assert user is not None, "the payload's user was not restored"
book = (await s.execute(
select(Rulebook).where(Rulebook.owner_user_id == user.id)
)).scalars().one()
topic = (await s.execute(
select(RulebookTopic).where(RulebookTopic.rulebook_id == book.id)
)).scalars().one()
rule = (await s.execute(
select(Rule).where(Rule.topic_id == topic.id)
)).scalars().one()
note = (await s.execute(
select(Note).where(Note.user_id == user.id)
)).scalars().one()
events = (await s.execute(
select(RuleUsageEvent).where(RuleUsageEvent.rule_id == rule.id)
.order_by(RuleUsageEvent.id)
)).scalars().all()
yield {
"user": user, "rule": rule, "note": note,
"events": events, "source": source,
}
await _purge_usage({rule.id})
await _purge_restored()
async def test_every_event_comes_back(restored):
"""The count first: every shape assertion below reads the same on an empty
list, so without this a restore that dropped all three would pass them."""
assert len(restored["events"]) == 3
async def test_the_events_attach_to_the_RESTORED_rule(restored):
"""The remap, on the column that matters."""
new_rule_id = restored["rule"].id
source_rule_id = restored["source"]["source_rule_id"]
assert new_rule_id != source_rule_id, (
"the restore reused the source id, so this test cannot tell a remap "
"from a copy — the fixture is not proving what it claims"
)
assert {e.rule_id for e in restored["events"]} == {new_rule_id}
async def test_no_event_landed_on_the_note_id(restored):
"""THE ONE THIS TABLE EXISTS FOR.
If `rule_id` were ever resolved through `note_id_map` — the shape it would
have had as a column on `note_usage_events` — these rows would come back
pointing at the restored NOTE's id. Populated, plausible, and describing a
record that was never surfaced.
"""
note_id = restored["note"].id
landed_on_note = [e for e in restored["events"] if e.rule_id == note_id]
assert not landed_on_note, (
f"{len(landed_on_note)} usage event(s) resolved to the note's id "
f"({note_id}) instead of the rule's. The rule id went through the "
"note map — telemetry that is wrong rather than missing, and that "
"nothing downstream can detect."
)
async def test_the_actor_is_remapped_and_a_missing_one_survives(restored):
"""`user_id` is an id in the source database too — the same trap one
column over. And the actorless row must not be dropped: the arm can fire
for an unauthenticated hook call, so requiring an actor would discard the
surfacings of exactly the surface being measured."""
attributed = [e for e in restored["events"] if e.user_id is not None]
orphaned = [e for e in restored["events"] if e.user_id is None]
assert len(attributed) == 2
assert len(orphaned) == 1, (
"the event with no actor did not come back. Telemetry outlives the "
"account it was recorded for; dropping it silently lowers the "
"surfaced count that the pull-through ratio divides by."
)
assert {e.user_id for e in attributed} == {restored["user"].id}
assert restored["user"].id != restored["source"]["source_user_id"]
async def test_the_event_and_source_survive(restored):
"""The two fields the ratio is computed from. A restore that kept the rows
and lost these would preserve a count of nothing in particular."""
pairs = {(e.event, e.source) for e in restored["events"]}
assert pairs == {
(SURFACED, "write_path_rule"),
(PULLED, "mcp_get_rule"),
}
assert sum(1 for e in restored["events"] if e.event == SURFACED) == 2
assert sum(1 for e in restored["events"] if e.event == PULLED) == 1
@@ -117,14 +117,12 @@ async def source():
statement="Use sh.",
why="the image ships no bash",
verify_with="read the workflow's shell setting",
tier="always_on",
),
# The actor is already gone — what SET NULL leaves behind.
RuleVersion(
rule_id=rule.id, user_id=None,
title="The runner has no bash",
statement="Use POSIX sh in run steps.",
tier="always_on",
),
])
await s.commit()
@@ -263,4 +261,3 @@ async def test_the_text_survives(restored):
assert by_statement["Use sh."].verify_with == (
"read the workflow's shell setting"
)
assert by_statement["Use sh."].tier == "always_on"
@@ -0,0 +1,173 @@
"""#3262 against a real Postgres: the embedder loses the race, it doesn't run it.
The reported failure was a deadlock — `DELETE FROM rulebooks` killed by the
server while a detached `upsert_rule_embedding` held the other half of the
cycle. It cannot be reproduced with mocks, because there is nothing to
deadlock: the whole bug lives in the ORDER two transactions take two row
locks, which only a lock manager can adjudicate.
So each test here holds a real delete open in one transaction and calls the
embedder in another. What is being pinned is that the embedder RETURNS —
promptly, having written nothing. Before the fix it would sit on the chunk
rows waiting for a delete that is itself waiting on the insert's foreign key,
and the test would hang rather than fail, which is why every call carries a
deadline (rule 156).
"""
import asyncio
from unittest.mock import AsyncMock, patch
import pytest
import pytest_asyncio
from sqlalchemy import delete, select
from scribe.models import async_session
from scribe.models.embedding import NoteEmbedding, RuleEmbedding
from scribe.models.note import Note
from scribe.models.rulebook import Rulebook
from scribe.services import embeddings as emb
from scribe.services import rulebooks as rulebooks_svc
from tests.helpers import ensure_user
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
OWNER_USERNAME = "embed_lock_owner"
# Generous, because the assertion is "it did not block indefinitely", not "it
# was fast". A machine under load must not turn this into a flake; a genuinely
# blocked embedder never returns at all, so no honest run comes near this.
YIELD_DEADLINE_SECONDS = 20
# What the embedder would write if it wrongly went ahead. Distinct from the
# text the fixture's own create_* wrote, so the assertion cannot be satisfied
# by rows that were already there.
SENTINEL = "sentinelvector"
ONE_VECTOR = [[0.0] * 384]
# The fixture's own embedding task runs the REAL embedder, which either loads a
# model or gives up; both are bounded well inside this.
SETTLE_DEADLINE_SECONDS = 30
@pytest_asyncio.fixture
async def seeded():
"""A rule and a note to race against.
CLEANED AT SETUP, NOT TEARDOWN — the same constraint #3241 hit and the
reason this file exists. `create_rule` fires its own detached embedding
task; a teardown that deleted the rulebook would be racing exactly the
thing under test, on a loop that is closing.
"""
async with async_session() as s:
owner = await ensure_user(s, OWNER_USERNAME)
uid = owner.id
await s.commit()
for book in (await s.execute(
select(Rulebook).where(Rulebook.owner_user_id == uid)
)).scalars().all():
await s.delete(book)
for note in (await s.execute(
select(Note).where(Note.user_id == uid)
)).scalars().all():
await s.delete(note)
await s.commit()
book = await rulebooks_svc.create_rulebook(uid, "Lock fixtures")
topic = await rulebooks_svc.create_topic(book.id, uid, "locks")
rule = await rulebooks_svc.create_rule(
topic.id, uid, "A rule with vectors",
"Something for the embedder to index.",
)
async with async_session() as s:
note = Note(user_id=uid, title="A note with vectors", body="Body text.")
s.add(note)
await s.commit()
note_id = note.id
await _settle_detached_writes()
return {"uid": uid, "book_id": book.id, "rule_id": rule.id, "note_id": note_id}
async def _settle_detached_writes() -> None:
"""Let `create_rule`'s own fire-and-forget embedding task finish.
It is the same detached write these tests are about, aimed at the same
rule, and left in flight it would land in the middle of an assertion about
that rule's rows. Bounded, and a timeout is not a failure — the tests below
carry their own deadlines, and this is only tidying the start line.
"""
pending = [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
if pending:
await asyncio.wait(pending, timeout=SETTLE_DEADLINE_SECONDS)
async def _rule_chunks(rule_id: int) -> list[str]:
async with async_session() as s:
return list((await s.execute(
select(RuleEmbedding.chunk_text).where(RuleEmbedding.rule_id == rule_id)
)).scalars().all())
async def _note_chunks(note_id: int) -> list[str]:
async with async_session() as s:
return list((await s.execute(
select(NoteEmbedding.chunk_text).where(NoteEmbedding.note_id == note_id)
)).scalars().all())
async def test_a_rule_refresh_yields_to_a_delete_cascading_from_its_rulebook(seeded):
"""The reported case, exactly: the delete lands on the RULEBOOK and reaches
the rule through two cascades, which is why nothing on the rule's own write
path could have seen it coming."""
async with async_session() as blocker:
# Uncommitted on purpose — the cascade's locks are held for as long as
# this transaction stays open, which is the state the embedder must
# decline to fight over.
await blocker.execute(delete(Rulebook).where(Rulebook.id == seeded["book_id"]))
try:
with patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)):
await asyncio.wait_for(
emb.upsert_rule_embedding(
seeded["rule_id"], SENTINEL, f"{SENTINEL} statement",
),
timeout=YIELD_DEADLINE_SECONDS,
)
finally:
await blocker.rollback()
assert not any(SENTINEL in text for text in await _rule_chunks(seeded["rule_id"])), \
"the embedder wrote into a rule that was being deleted"
async def test_a_note_refresh_yields_to_a_delete_of_the_note(seeded):
"""The note twin, which #3262 recorded as unverified. Notes are soft-deleted
day to day, so the hard delete a trash purge issues is the one that can put
a lock on the row while a refresh is in flight."""
async with async_session() as blocker:
await blocker.execute(delete(Note).where(Note.id == seeded["note_id"]))
try:
with patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)):
await asyncio.wait_for(
emb.upsert_note_embedding(
seeded["note_id"], seeded["uid"], SENTINEL, f"{SENTINEL} body",
),
timeout=YIELD_DEADLINE_SECONDS,
)
finally:
await blocker.rollback()
assert not any(SENTINEL in text for text in await _note_chunks(seeded["note_id"])), \
"the embedder wrote into a note that was being deleted"
async def test_an_uncontended_refresh_still_writes(seeded):
"""The guard against the cheapest possible false pass: a claim that never
succeeds would satisfy both tests above while quietly ending semantic
search."""
with patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)):
await emb.upsert_rule_embedding(
seeded["rule_id"], SENTINEL, f"{SENTINEL} statement",
)
assert any(SENTINEL in text for text in await _rule_chunks(seeded["rule_id"])), \
"an unlocked rule was not embedded"
+20 -24
View File
@@ -10,7 +10,6 @@ import pytest_asyncio
from scribe.models import async_session
from scribe.models.project import Project
from scribe.models.rulebook import Rulebook
from scribe.services import inception as inception_svc
from scribe.services import rulebooks as rulebooks_svc
from scribe.services import canonical_systems as canonical_svc
@@ -31,12 +30,12 @@ async def seeded():
await s.flush()
ids = {"owner": owner.id, "pid": project.id}
await s.commit()
# Two ordinary rulebooks. One was flagged always-on until milestone 394
# removed the tier; a rulebook now reaches a project only by subscription,
# so what used to be "binds automatically" and "binds if you opt in" are
# the same kind of thing.
always = await rulebooks_svc.create_rulebook(ids["owner"], "Family standards")
other = await rulebooks_svc.create_rulebook(ids["owner"], "Optional practices")
async with async_session() as s:
rb = await s.get(Rulebook, always.id)
rb.always_on = True
await s.commit()
t1 = await rulebooks_svc.create_topic(always.id, ids["owner"], "git")
await rulebooks_svc.create_rule(t1.id, ids["owner"], "dev is home", "Work on dev.")
t2 = await rulebooks_svc.create_topic(other.id, ids["owner"], "docs")
@@ -48,20 +47,20 @@ async def seeded():
@pytest.mark.integration
async def test_decide_applies_every_effect_and_records_last(seeded):
owner, pid = seeded["owner"], seeded["pid"]
# Undecided: the always-on rulebook binds, nothing subscribed, no Systems.
assert [r.title for r in await rulebooks_svc.list_always_on_rules(owner, project_id=pid)] == ["dev is home"]
# Undecided: nothing binds, because nothing has been subscribed. Before
# milestone 394 the always-on rulebook bound here without being asked for,
# and this assertion read the other way round.
defaults = await inception_svc.current_defaults(owner, pid)
assert [r["id"] for r in defaults["always_on_rulebooks"]] == [seeded["always"]]
assert [r["id"] for r in defaults["other_rulebooks"]] == [seeded["other"]]
assert sorted(r["id"] for r in defaults["rulebooks"]) == sorted(
[seeded["always"], seeded["other"]])
assert defaults["systems"] == 0 and defaults["design_system_id"] is None
assert (await rulebooks_svc.get_applicable_rules(pid, owner))["rules"] == []
out = await inception_svc.decide(owner, pid, via="mcp", choices={
"exclude_always_on_rulebooks": [seeded["always"]],
"subscribe_rulebooks": [seeded["other"]],
"design_system_id": None,
"seed_systems": True,
})
assert out["effects"]["excluded"] == [seeded["always"]]
assert out["effects"]["subscribed"] == [seeded["other"]]
catalog = await canonical_svc.list_canonical_systems()
assert len(out["effects"]["systems_seeded"]) == len(catalog)
@@ -69,35 +68,32 @@ async def test_decide_applies_every_effect_and_records_last(seeded):
seeded_systems = await systems_svc.list_systems(owner, pid)
assert all(s.canonical_id is not None for s in seeded_systems)
# The exclusion is total: the project's always-on set is empty, the
# departure is named, the subscription binds.
assert await rulebooks_svc.list_always_on_rules(owner, project_id=pid) == []
assert len(await rulebooks_svc.list_always_on_rules(owner)) == 1 # user-wide unchanged
# The subscription is what binds, and it is the ONLY thing that does —
# the unsubscribed rulebook contributes nothing even though it used to
# bind every project by default.
applicable = await rulebooks_svc.get_applicable_rules(pid, owner)
assert [r["title"] for r in applicable["rules"]] == ["Write the why"]
assert [e["id"] for e in applicable["excluded_always_on"]] == [seeded["always"]]
assert [s["id"] for s in applicable["subscribed_rulebooks"]] == [seeded["other"]]
assert "dev is home" not in [r["title"] for r in applicable["rules"]]
# The record, written last, says why.
async with async_session() as s:
project = await s.get(Project, pid)
assert inception_svc.is_decided(project)
assert project.inception["via"] == "mcp" and project.inception["decided_by"] == owner
assert project.inception["choices"]["exclude_always_on_rulebooks"] == [seeded["always"]]
# Re-deciding with seed again mints nothing twice; include reverses the exclusion.
assert project.inception["choices"]["subscribe_rulebooks"] == [seeded["other"]]
# Re-deciding with seed again mints nothing twice.
again = await inception_svc.decide(owner, pid, via="ui", choices={"seed_systems": True})
assert again["effects"]["systems_seeded"] == []
assert len(await systems_svc.list_systems(owner, pid)) == len(catalog)
await rulebooks_svc.include_always_on_rulebook_for_project(pid, seeded["always"], owner)
assert [r.title for r in await rulebooks_svc.list_always_on_rules(owner, project_id=pid)] == ["dev is home"]
@pytest.mark.integration
async def test_a_bad_decision_applies_nothing(seeded):
owner, pid = seeded["owner"], seeded["pid"]
# Excluding a rulebook that is not always-on is refused BEFORE any effect.
with pytest.raises(ValueError, match="not always-on"):
# A subscription to a rulebook that is not yours is refused BEFORE any
# effect lands — the seed must not happen on a decision that fails.
with pytest.raises(ValueError, match="not found"):
await inception_svc.decide(owner, pid, via="mcp", choices={
"exclude_always_on_rulebooks": [seeded["other"]], "seed_systems": True,
"subscribe_rulebooks": [999999], "seed_systems": True,
})
assert await systems_svc.list_systems(owner, pid) == []
with pytest.raises(ValueError, match="not found"):
+165
View File
@@ -0,0 +1,165 @@
"""Real-Postgres tests for `rules.kind` and its CHECK (0098, #3849 step 1).
Rule 36 exists because a value and its constraint drift apart: the code
starts writing a new kind while the database still refuses it, and nothing
catches it until a write fails in front of someone. A mock cannot show that —
it has no CHECK — so the constraint gets a real-DB test, exactly as 0091's
`spike` did.
THREE HALVES, not one. The positive (a preference writes), the negative (a
typo is refused — without it every other assertion here would pass just as
happily against a table whose CHECK was dropped and never re-added), and the
DEFAULT.
The default is the one worth spelling out, because it is the half that has no
obvious failure. `kind` is NOT NULL with a server default, and the entire
safety argument for this migration is that every existing row keeps the force
it had. A row written without a kind must read back as `rule` — if the server
default did not apply, an upgraded install gets a NOT NULL violation on its
next rule write, or worse, a column that reads as something other than what
every rule in it has always been.
"""
import pytest
import pytest_asyncio
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from scribe.models import async_session
from scribe.models.rulebook import Rule, Rulebook
from scribe.services import rulebooks as rulebooks_svc
from tests.helpers import ensure_user
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
OWNER_USERNAME = "rule_kind_owner"
@pytest_asyncio.fixture
async def topic_id():
"""A topic to hang rules on.
CLEANED UP AT SETUP, NOT TEARDOWN, for the reason spelled out in
test_integration_rule_versions: the rule write path fires a detached
embedding task that opens its own connection and UPDATEs the row, and a
teardown deleting the rulebook deadlocks against it. Purging at setup runs
on a fresh loop, after the previous test's loop cancelled whatever it left
in flight.
"""
async with async_session() as s:
owner = await ensure_user(s, OWNER_USERNAME)
uid = owner.id
await s.commit()
for book in (await s.execute(
select(Rulebook).where(Rulebook.owner_user_id == uid)
)).scalars().all():
await s.delete(book)
await s.commit()
book = await rulebooks_svc.create_rulebook(uid, "Kind fixtures")
topic = await rulebooks_svc.create_topic(book.id, uid, "conventions")
return uid, topic.id
async def _write_raw(topic: int, **kw) -> int:
"""Straight to the model, bypassing `_valid_kind`.
The service coerces an unrecognised kind to `rule`, which is right for
callers and useless for testing the CHECK — it means no service call can
ever reach the database with a bad value. These tests are about the
constraint, so they go around the coercion.
"""
async with async_session() as s:
rule = Rule(topic_id=topic, title="t", statement="s", **kw)
s.add(rule)
await s.commit()
return rule.id
async def test_a_preference_can_be_written(topic_id):
_uid, topic = topic_id
rule_id = await _write_raw(topic, kind="preference")
async with async_session() as s:
assert (await s.get(Rule, rule_id)).kind == "preference"
async def test_a_rule_still_writes(topic_id):
"""0098 widens nothing — it adds a column — but it must not narrow either."""
_uid, topic = topic_id
rule_id = await _write_raw(topic, kind="rule")
async with async_session() as s:
assert (await s.get(Rule, rule_id)).kind == "rule"
async def test_an_unknown_kind_is_refused(topic_id):
"""The half that proves the constraint is there at all.
Without this, every other assertion in this file would pass against a
table whose CHECK had been dropped — the exact failure rule 36 is written
against.
"""
_uid, topic = topic_id
with pytest.raises(IntegrityError):
await _write_raw(topic, kind="suggestion")
async def test_a_rule_written_without_a_kind_defaults_to_rule(topic_id):
"""The migration's whole safety claim, asserted rather than assumed.
Every row that existed before 0098 was a rule and must stay one. This is
the closest a test can get to that: write the way code predating the
column would, and read back the force it should have.
"""
_uid, topic = topic_id
rule_id = await _write_raw(topic)
async with async_session() as s:
assert (await s.get(Rule, rule_id)).kind == "rule"
async def test_a_rule_can_become_a_preference_after_the_fact(topic_id):
"""The read-back is the whole test.
A version asserting only that the update call succeeded would pass against
code that accepted `kind` and dropped it — which is how the same bug
survived on `task_kind` long enough to be found by hand (#3129).
This is also the migration path the always-on triage needs: the rules that
turn out to be preferences change one column and keep their id, history
and relations.
"""
uid, topic = topic_id
rule_id = await _write_raw(topic)
await rulebooks_svc.update_rule(rule_id, uid, kind="preference")
async with async_session() as s:
assert (await s.get(Rule, rule_id)).kind == "preference"
async def test_an_unrecognised_kind_falls_back_rather_than_raising(topic_id):
"""`_valid_kind` coerces, matching `_valid_tier`, and the direction matters.
Deliberately NOT the `task_kind` behaviour, which raises a readable
ValueError. A rule's kind falls back to the binding value, because between
a reader who is too careful and one who is not careful enough, a typo
should produce the first. Asserted here so a later "make it consistent
with task_kind" change has to argue with a test rather than a comment.
"""
uid, topic = topic_id
rule_id = await _write_raw(topic, kind="preference")
await rulebooks_svc.update_rule(rule_id, uid, kind="prefrence")
async with async_session() as s:
assert (await s.get(Rule, rule_id)).kind == "rule"
async def test_kind_reaches_both_read_shapes(topic_id):
"""`to_dict` and `rule_brief` both carry it, unconditionally.
Force must never be inferred from an ABSENT key: "no kind field" and
"kind is rule" would be the same payload, and a reader would have to
assume one. Both shapes say it outright, so pin both — the two dicts are
built in different modules and have diverged before.
"""
_uid, topic = topic_id
rule_id = await _write_raw(topic, kind="preference")
async with async_session() as s:
rule = await s.get(Rule, rule_id)
assert rule.to_dict()["kind"] == "preference"
assert rulebooks_svc.rule_brief(rule)["kind"] == "preference"
+39 -53
View File
@@ -1,22 +1,27 @@
"""Real-Postgres tests for WHICH rules reach a session (milestone 307 step 5).
"""Real-Postgres tests for WHICH rules reach a session (milestone 307 step 5,
narrowed by 394).
What mocks can't prove, and what this milestone must not get wrong:
What mocks can't prove, and what this design must not get wrong:
1. **Nothing stops binding.** A rule with no tier, no areas and no edges
behaves exactly as it did before tiers existed. That is the one failure this
whole design must not produce, and it is asserted first.
2. A conditional rule is invisible to a project that doesn't work in its area,
and arrives — binding, not suggested — to one that does.
3. A `co_surfaces` partner arrives with its other half, which is the failure
1. A rule is invisible to a project that doesn't work in its area, and
arrives — binding, not suggested — to one that does. Area matching is
DETERMINISTIC: a tag comparison, never a similarity score.
2. A `co_surfaces` partner arrives with its other half, which is the failure
that made merging rule 144 into rule 46 look like the only fix.
4. An explicit suppression outranks an edge.
3. An explicit suppression outranks an edge.
TWO CLAIMS WERE DROPPED HERE BY MILESTONE 394, and it is worth saying which
rather than leaving a shorter list. "A rule with no tier binds exactly as
before" and "a conditional rule is reachable, not resident" were both about
the always-on tier. There is no tier and no resident payload, so neither
states anything that can now be true or false — they were not failing, they
had stopped being claims.
"""
import pytest
import pytest_asyncio
from scribe.models import async_session
from scribe.models.project import Project
from scribe.models.rulebook import Rulebook
from scribe.services import canonical_systems as canonical_svc
from scribe.services import rulebooks as rulebooks_svc
from scribe.services import systems as systems_svc
@@ -27,12 +32,13 @@ pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine"
@pytest_asyncio.fixture
async def world():
"""A project with TWO rulebooks, because the two payloads are different sets.
"""A project with two rulebooks — one subscribed, one not.
`list_always_on_rules` covers always-on rulebooks; `get_applicable_rules`
covers SUBSCRIBED ones. Conflating them is easy and would make these tests
assert nothing, so the fixture carries one of each and every test says
which payload it is about.
Both are ordinary rulebooks since milestone 394; the fixture used to flag
one always-on because that was a second, separate way to reach a project.
Keeping two is still worth it: a rulebook nobody subscribed to must
contribute nothing, and a fixture with only the subscribed one could not
tell "correctly scoped" from "returns everything".
"""
async with async_session() as s:
owner = await ensure_user(s, "surfacing_owner")
@@ -43,10 +49,6 @@ async def world():
await s.commit()
always = await rulebooks_svc.create_rulebook(ids["owner"], "Family standards")
async with async_session() as s:
rb = await s.get(Rulebook, always.id)
rb.always_on = True
await s.commit()
always_topic = await rulebooks_svc.create_topic(always.id, ids["owner"], "git")
await rulebooks_svc.create_rule(
always_topic.id, ids["owner"], "dev is home", "Work on dev.",
@@ -72,39 +74,8 @@ async def _titles(ids) -> set[str]:
return {r["title"] for r in applicable["rules"]}
@pytest.mark.integration
async def test_a_rule_with_no_tier_no_areas_and_no_edges_binds_exactly_as_before(world):
"""THE compatibility guarantee. An install upgrades and every rule it
already had keeps arriving — no tier set, no areas, no edges, still bound.
Getting this wrong would silently stop enforcing rules people rely on,
which is worse than any amount of payload bloat."""
always_on = await rulebooks_svc.list_always_on_rules(world["owner"])
assert "dev is home" in {r.title for r in always_on}
assert "Between batches, keep stacking" in await _titles(world)
@pytest.mark.integration
async def test_a_conditional_rule_is_reachable_not_resident(world):
"""It leaves the session-start payload entirely — that is the point of the
tier — and it does NOT reach a project with no matching area."""
# In the ALWAYS-ON book: the tier alone keeps it out of the session-start
# payload, which is the whole point of the tier.
resident = await rulebooks_svc.create_rule(
world["always_topic"], world["owner"], "Release tagging", "Derive the tag.",
when_to_apply="when cutting a release", tier="conditional",
)
assert resident.tier == "conditional"
always_on = await rulebooks_svc.list_always_on_rules(world["owner"])
assert "Release tagging" not in {r.title for r in always_on}
# In the SUBSCRIBED book, untagged: the project has no area to reach it by,
# so it stays out of the project payload too. Absent for a DIFFERENT reason
# than above, which is why both are asserted.
await rulebooks_svc.create_rule(
world["topic"], world["owner"], "Untagged conditional", "No area yet.",
when_to_apply="sometime", tier="conditional",
)
assert "Untagged conditional" not in await _titles(world)
@pytest.mark.integration
@@ -117,7 +88,7 @@ async def test_a_conditional_rule_binds_a_project_that_works_in_its_area(world):
rule = await rulebooks_svc.create_rule(
world["topic"], world["owner"], "Release tagging", "Derive the tag.",
when_to_apply="when cutting a release", tier="conditional",
when_to_apply="when cutting a release",
)
await rulebooks_svc.set_rule_systems(rule.id, world["owner"], [area.id])
@@ -145,8 +116,21 @@ async def test_co_surfaces_drags_in_the_half_that_would_have_been_missed(world):
partner = await rulebooks_svc.create_rule(
world["topic"], world["owner"], "Version names are labels",
"A name decides nothing.",
when_to_apply="when naming a build", tier="conditional",
when_to_apply="when naming a build",
)
# TAGGED TO AN AREA THIS PROJECT DOES NOT WORK IN, which is what makes the
# test able to fail at all. Since milestone 394 an UNTAGGED rule in a
# subscribed rulebook applies on its own, so an untagged partner arrives
# through the ordinary query and the edge is never exercised — the
# assertion below passed while proving nothing, which is how this was
# noticed. Tagging it puts it out of reach of everything except the edge.
area = await canonical_svc.find_by_name("CI & Release")
assert area is not None, "migration 0087 seeds the standard vocabulary"
await rulebooks_svc.set_rule_systems(partner.id, world["owner"], [area.id])
assert "Version names are labels" not in await _titles(world), (
"the partner must be unreachable on its own, or this test cannot fail"
)
await rulebooks_svc.add_rule_relation(
world["owner"], world["plain"], partner.id, "co_surfaces",
note="they fail together",
@@ -161,9 +145,11 @@ async def test_co_surfaces_drags_in_the_half_that_would_have_been_missed(world):
async def test_a_suppression_outranks_an_edge(world):
"""The edge says these belong together; the suppression says this project
does not want that one. An explicit decision beats an inferred one."""
# Untagged on purpose, unlike the partner above: this test is about the
# SUPPRESSION winning, so the partner should be one that would otherwise
# arrive by every available route — the ordinary query AND the edge.
partner = await rulebooks_svc.create_rule(
world["topic"], world["owner"], "Muted partner", "Should not arrive.",
tier="conditional",
)
await rulebooks_svc.add_rule_relation(
world["owner"], world["plain"], partner.id, "co_surfaces",
+1 -8
View File
@@ -99,7 +99,7 @@ async def test_rewording_the_check_drops_the_stamp(constraint):
"""A stamp certifies a check, not a rule.
The safe direction, for the same reason _valid_tier falls back to
always_on: a rule wrongly listed as due costs one look, a rule wrongly
the safe direction: a rule wrongly listed as due costs one look, a rule wrongly
vouched for costs exactly what the sweep exists to catch.
"""
await rulebooks_svc.update_rule(
@@ -162,7 +162,6 @@ async def rulebook_of_three():
stale = await rulebooks_svc.create_rule(
topic.id, uid, "Bumps need a dashboard tick", "Tick it first.",
verify_with="cat CI-runner/renovate/config.js",
tier="conditional",
)
async with async_session() as s:
row = await s.get(Rule, stale.id)
@@ -256,12 +255,6 @@ async def test_never_only_and_the_age_filter_narrow_to_what_they_say(rulebook_of
assert rulebook_of_three["never"] in aged
async def test_the_tier_filter_narrows_to_one_tier(rulebook_of_three):
ids = [r.id for r in await rulebooks_svc.rules_due_for_verification(
rulebook_of_three["uid"], tier="conditional",
)]
assert rulebook_of_three["stale"] in ids
assert rulebook_of_three["never"] not in ids
async def test_another_users_rules_are_not_in_your_sweep(rulebook_of_three):
+4 -4
View File
@@ -412,10 +412,10 @@ async def test_create_project_with_inception_args_decides_via_mcp():
decided = {"inception": {"via": "mcp", "choices": {}}, "effects": {"systems_seeded": []}}
with patch("scribe.mcp.tools.projects.projects_svc.create_project", AsyncMock(return_value=p)), \
patch("scribe.mcp.tools.projects.inception_svc.decide", AsyncMock(return_value=decided)) as decide:
out = await create_project(title="P", exclude_always_on_rulebooks=[1], design_system_id=-1, seed_systems=True)
out = await create_project(title="P", subscribe_rulebooks=[1], design_system_id=-1, seed_systems=True)
kw = decide.await_args.kwargs
assert decide.await_args.args[1] == 5 and kw["via"] == "mcp"
assert kw["choices"] == {"exclude_always_on_rulebooks": [1], "subscribe_rulebooks": [],
assert kw["choices"] == {"subscribe_rulebooks": [1],
"design_system_id": None, "seed_systems": True}
assert out["inception"]["via"] == "mcp" and "inception_effects" in out
@@ -433,7 +433,7 @@ async def test_decide_project_inception_tool_records_an_inherit_all_decision_whe
@pytest.mark.asyncio
async def test_enter_project_carries_the_inception_ask_only_for_an_undecided_own_project():
applicable = {"rules": [], "project_rules": [], "truncated": False,
"subscribed_rulebooks": [], "excluded_always_on": []}
"subscribed_rulebooks": []}
ask = {"defaults": {}, "ask": "decide", "call": "decide_project_inception(...)"}
async def run(project):
@@ -466,6 +466,6 @@ def test_inception_routes_and_tool_are_registered():
mcp = build_mcp_server()
assert mcp._tool_manager.get_tool("decide_project_inception") is not None
tool = mcp._tool_manager.get_tool("create_project")
for name in ("exclude_always_on_rulebooks", "subscribe_rulebooks", "design_system_id", "seed_systems"):
for name in ("subscribe_rulebooks", "design_system_id", "seed_systems"):
assert name in tool.parameters.get("properties", {}), name
+17 -64
View File
@@ -3,6 +3,7 @@ from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from tests.helpers import FakeMCP, fake_rule, fake_rulebook, fake_topic
from tests.helpers import plain_rule_detail as _plain_detail
pytestmark = pytest.mark.usefixtures("_bind_user")
@@ -48,18 +49,10 @@ async def test_get_rulebook_raises_when_not_found():
await get_rulebook(rulebook_id=999)
def _plain_detail():
"""Stub the rule_detail seam these tool tests are not about.
create/update/get_rule now return through services.rulebooks.rule_detail,
which reads the rule's areas and edges from the database. These are unit
tests with no database, and what they assert is that the TOOL forwards the
right arguments — so the seam is stubbed to the plain record, the same way
they already stub the create/update calls themselves.
"""
async def _detail(_uid, rule, _system_ids=None):
return rule.to_dict()
return patch("scribe.mcp.tools.rulebooks.rulebooks_svc.rule_detail", _detail)
# _plain_detail moved to tests/helpers on its second copy (#2825's own
# reason for existing): two stubs for one seam drift apart quietly, and a
# test stubbing it slightly differently asserts something slightly different
# than it appears to.
@pytest.mark.asyncio
@@ -230,16 +223,21 @@ def test_register_attaches_every_tool():
register(mcp)
# 26 through milestone 307, +2 for the staleness sweep (milestone 312),
# +1 for a rule's edit history (milestone 323).
assert len(mcp.names) == 29
# +1 for a rule's edit history (milestone 323), +2 for preferences
# (milestone 399).
# 28 since milestone 394 took list_always_on_rules and the two
# always-on exclusion tools with the tier they served.
assert len(mcp.names) == 28
# spot-check a few names
assert "list_rulebooks" in mcp.names
assert "create_rule" in mcp.names
# Preferences get their own WRITE door — create_rule's docstring is the
# approval gate, and a preference reached through it would be read
# through that prose. Reads stay shared deliberately, so there is no
# get_preference to look for here.
assert "create_preference" in mcp.names
assert "update_preference" in mcp.names
assert "subscribe_project_to_rulebook" in mcp.names
assert "list_always_on_rules" in mcp.names
# milestone 297: a project's opt-out of a whole always-on rulebook
assert "exclude_always_on_rulebook" in mcp.names
assert "include_always_on_rulebook" in mcp.names
assert "create_project_rule" in mcp.names
assert "suppress_rule_for_project" in mcp.names
# milestone 312: the sweep, and the stamp that answers it
@@ -252,57 +250,12 @@ def test_register_attaches_every_tool():
assert "unsuppress_topic_for_project" in mcp.names
@pytest.mark.asyncio
async def test_list_always_on_rules_returns_empty_when_no_always_on_rulebooks():
with patch(
"scribe.mcp.tools.rulebooks.rulebooks_svc.list_always_on_rules",
AsyncMock(return_value=[]),
):
from scribe.mcp.tools.rulebooks import list_always_on_rules
out = await list_always_on_rules()
# An install with no always-on rulebooks still gets a marker (milestone
# 323): "no rules" is a STATE, and a payload that omitted the key would
# make the write path read every session on a fresh install as a change.
assert out == {"rules": [], "total": 0, "rules_etag": "empty|0"}
@pytest.mark.asyncio
async def test_list_always_on_rules_projects_each_rule():
rules = [fake_rule(id=100, title="r", statement="s", topic_id=10), fake_rule(id=101, title="r", statement="s", topic_id=10)]
with patch(
"scribe.mcp.tools.rulebooks.rulebooks_svc.list_always_on_rules",
AsyncMock(return_value=rules),
):
from scribe.mcp.tools.rulebooks import list_always_on_rules
out = await list_always_on_rules()
assert out["total"] == 2
assert {r["id"] for r in out["rules"]} == {100, 101}
assert all("topic_id" in r for r in out["rules"])
@pytest.mark.asyncio
async def test_update_rulebook_forwards_always_on_when_set():
rb = fake_rulebook(id=1, title="t")
mock = AsyncMock(return_value=rb)
with patch("scribe.mcp.tools.rulebooks.rulebooks_svc.update_rulebook", mock):
from scribe.mcp.tools.rulebooks import update_rulebook
await update_rulebook(rulebook_id=1, always_on=True)
kwargs = mock.call_args.kwargs
assert kwargs.get("always_on") is True
assert "title" not in kwargs
assert "description" not in kwargs
@pytest.mark.asyncio
async def test_update_rulebook_omits_always_on_when_none():
rb = fake_rulebook(id=1, title="t")
mock = AsyncMock(return_value=rb)
with patch("scribe.mcp.tools.rulebooks.rulebooks_svc.update_rulebook", mock):
from scribe.mcp.tools.rulebooks import update_rulebook
await update_rulebook(rulebook_id=1, title="new title")
kwargs = mock.call_args.kwargs
assert "always_on" not in kwargs
assert kwargs["title"] == "new title"
@pytest.mark.asyncio
@@ -454,7 +407,7 @@ def _fake_version(**over):
"id": 5, "rule_id": 100, "user_id": 1,
"title": "The runner has no bash", "statement": "Use sh.",
"why": "the image ships no bash", "how_to_apply": None,
"when_to_apply": None, "tier": "always_on",
"when_to_apply": None,
"verify_with": "read the workflow's shell setting",
"expires_when": None,
"created_at": datetime(2026, 8, 29, tzinfo=timezone.utc),
+2 -1
View File
@@ -154,7 +154,8 @@ async def test_unscored_location_arms_are_recorded(lookups, expected_source):
patch.object(
plugin_context,
"get_writepath_config",
AsyncMock(return_value={"enabled": True, "threshold": 0.55, "top_k": 3}),
AsyncMock(return_value={"enabled": True, "threshold": 0.55, "top_k": 3,
"rule_threshold": 0.72}),
),
patch.object(
plugin_context.snippets_svc,
+250
View File
@@ -0,0 +1,250 @@
"""One definition of what SHIPS in the plugin, and the drift guards on it.
WHAT THIS IS ABOUT (#3127 §3, milestone 334 step 2). Scribe publishes two
artifacts. `plugin/` is not in the Docker image — installs fetch it from this
repo through `.claude-plugin/marketplace.json`, so **a push IS the release**,
with no build step in between. That makes "which files reach an install?" a
question with real consequences, and it has been answered wrong twice:
- #2198 — `plugin/**` was in no `paths:` filter, so four broken hooks
reached live installs having triggered no CI at all.
- #2209 — the fix for that shipped and still could not reach an install,
because the manifest version had not moved.
The set lives in `scripts/check_plugin.py`. Its second consumer is the
workflow's `paths:` trigger, which is YAML and cannot import Python — so the
"one definition" is held together by the drift tests here rather than by an
import. That is the honest shape, and it is why these tests exist at all.
The exclusion tests are the load-bearing half. Without the manifest-`version`
exclusion the version check is CIRCULAR: bumping the version edits a file
inside `plugin/`, which then reads as the content change that justifies the
bump. Every bump passes, no bump ever fails, and the check has proved nothing
while looking green.
"""
import json
import pathlib
import re
import pytest
from scripts.check_plugin import (
DERIVERS,
SHIPPED_PATHS,
manifest_differs_beyond_version,
)
ROOT = pathlib.Path(__file__).resolve().parents[1]
CI = ROOT / ".forgejo/workflows/ci.yml"
def trigger_paths() -> list[str]:
"""The `paths:` list under the workflow's push trigger.
Parsed with a regex rather than a YAML library, matching what
test_version_endpoint.py already does with this file — the alternative is
adding PyYAML as a dependency for one assertion. Raises rather than
returning empty: a silent no-op here would defeat the point of the file.
"""
text = CI.read_text()
block = re.search(r"^ paths:\n((?:(?: [-#].*)?\n)+)", text, re.M)
if block is None:
raise AssertionError("could not find the push trigger's `paths:` block")
found = re.findall(r'^ - "([^"]+)"', block.group(1), re.M)
if not found:
raise AssertionError("the `paths:` block parsed to zero entries")
return found
# ── The set itself ─────────────────────────────────────────────────────────
def test_every_shipped_path_exists():
"""A set naming something that isn't there is not a definition of anything."""
for path in SHIPPED_PATHS:
assert (ROOT / path).exists(), f"SHIPPED_PATHS names {path}, which does not exist"
def test_every_shipped_path_triggers_ci():
"""#2198's exact hole, stated as an assertion.
Directional on purpose: the trigger is a superset (it also fires on
`src/**`, `tests/**` and friends). What must never happen is a path that
reaches an install and fires no lane.
"""
triggers = trigger_paths()
for path in SHIPPED_PATHS:
covered = any(t == path or t.startswith(f"{path}/") for t in triggers)
assert covered, (
f"{path} ships to installs but no `paths:` entry covers it — "
f"changes there would reach a live install having run no CI (#2198)"
)
def test_the_checker_itself_triggers_ci():
"""Changing the checks must re-run them.
Not a member of the shipped set — a checker decides whether the lane goes
red, not what any artifact reports — but a change to it that runs no lane
is the same silence by a different route.
"""
assert "scripts/check_plugin.py" in trigger_paths()
def test_no_trigger_path_names_something_that_does_not_exist():
"""The guard that catches scaffolding outliving its subsystem.
`fable-mcp/**` sat in this list for three months after the directory was
deleted (commit 91bafb6, 2026-05-27), and `assets/**` named a path that
never existed at all. Neither ever failed anything — a `paths:` entry
matching nothing simply never fires — which is precisely why a list kept
by hand drifts and nobody finds out.
"""
missing = [
entry for entry in trigger_paths()
if not (ROOT / re.sub(r"/\*\*$", "", entry)).exists()
]
assert not missing, (
f"`paths:` names {missing}, which do not exist in the repo. A trigger "
f"that matches nothing is silent, so it survives every review."
)
def test_every_deriver_exists():
"""§3's table, kept honest.
The point of the table is that the next artifact is a one-line addition
(milestone 334 step 3 adds the plugin's mint script). A row pointing at a
file that has moved would make the table read as complete when it is not.
"""
for path, artifacts in DERIVERS.items():
assert (ROOT / path).exists(), f"DERIVERS names {path}, which does not exist"
assert artifacts, f"DERIVERS[{path}] names no artifact"
# ── The exclusion — the half that makes the version check mean anything ────
def manifest(**fields) -> str:
base = {
"name": "scribe",
"description": "d",
"version": "0.1.48",
"mcpServers": {"scribe": {"type": "http", "url": "${user_config.api_endpoint}/mcp"}},
"userConfig": {"api_endpoint": {"type": "string"}},
}
base.update(fields)
return json.dumps(base)
def test_a_version_only_change_is_NOT_a_content_change():
"""THE assertion. Without it the version check is self-satisfying: the
bump edits `plugin.json`, which lives inside `plugin/`, so the bump is its
own justification and every bump passes."""
assert manifest_differs_beyond_version(
manifest(version="2026.09.01.0512"), manifest(version="0.1.48")
) is False
def test_an_identical_manifest_is_not_a_change():
assert manifest_differs_beyond_version(manifest(), manifest()) is False
@pytest.mark.parametrize("field,value", [
("userConfig", {"api_endpoint": {"type": "string", "title": "changed"}}),
("mcpServers", {"scribe": {"type": "http", "url": "elsewhere"}}),
("description", "a different description"),
("name", "renamed"),
])
def test_every_OTHER_manifest_field_still_demands_a_new_version(field, value):
"""Why the exclusion is one FIELD and never the whole file.
`plugin.json` carries description, mcpServers and userConfig alongside the
version, and all of them reach an install. Excluding the file wholesale
would mean a userConfig-only edit computes an unchanged version and never
refreshes — #2209 again, with a narrower trigger and the same silence.
"""
assert manifest_differs_beyond_version(manifest(**{field: value}), manifest()) is True
def test_reformatting_is_not_a_content_change():
"""Parsed objects, not text. Whitespace and key order are not content, and
a check that treated them as such would demand a version for a re-indent."""
data = json.loads(manifest())
reordered = {k: data[k] for k in reversed(list(data))}
assert manifest_differs_beyond_version(
json.dumps(reordered, indent=4), json.dumps(data, separators=(",", ":"))
) is False
@pytest.mark.parametrize("bad", ["", "{not json", "[]", '"a string"', "null"])
def test_unreadable_input_demands_a_new_version(bad):
"""The conservative direction, chosen deliberately.
A spurious bump costs one cache refresh. A missed one is #2209 — the fix
reaches the repo and stops there, and the only detector is a human saying
"I don't think it updated."
"""
assert manifest_differs_beyond_version(bad, manifest()) is True
assert manifest_differs_beyond_version(manifest(), bad) is True
def test_a_manifest_appearing_or_vanishing_is_a_change():
"""None means the file is absent at that ref — a real difference, and not
the same thing as unreadable."""
assert manifest_differs_beyond_version(None, manifest()) is True
assert manifest_differs_beyond_version(manifest(), None) is True
# ── The reader that joins the exclusion to git ─────────────────────────────
def test_shipped_content_changed_reports_a_version_only_commit_as_unchanged(monkeypatch):
"""End to end through the git seam, with git stubbed.
The unit above proves the comparison; this proves it is actually WIRED to
the path that `check_version_is_minted` reads. A correct helper nobody calls
would leave the circular check exactly as it was.
"""
from scripts import check_plugin
monkeypatch.setattr(
check_plugin, "_git",
lambda *a: (0, "plugin/.claude-plugin/plugin.json"),
)
monkeypatch.setattr(
check_plugin, "manifest_text",
lambda ref=None: manifest(version="2026.09.01.0512" if ref is None else "0.1.48"),
)
changed, paths = check_plugin.shipped_content_changed("origin/main")
assert changed is False
assert paths == ["plugin/.claude-plugin/plugin.json"]
def test_shipped_content_changed_reports_a_hook_edit_as_changed(monkeypatch):
"""The guard against an exclusion that swallowed everything — a check that
can never fire is indistinguishable from one that is broken."""
from scripts import check_plugin
monkeypatch.setattr(
check_plugin, "_git",
lambda *a: (0, "plugin/hooks/scribe_session_context.sh"),
)
changed, paths = check_plugin.shipped_content_changed("origin/main")
assert changed is True
assert paths == ["plugin/hooks/scribe_session_context.sh"]
def test_a_failed_diff_is_None_and_never_False(monkeypatch):
"""Could-not-tell and nothing-changed must not collapse into one value.
#2663 is the precedent: a read that failed inside a broad except reported
the same zero as a genuinely empty window, and every counter read zero for
weeks with nothing to distinguish the two.
"""
from scripts import check_plugin
monkeypatch.setattr(check_plugin, "_git", lambda *a: (128, "fatal: bad revision"))
changed, paths = check_plugin.shipped_content_changed("origin/main")
assert changed is None
assert paths == []
+287
View File
@@ -0,0 +1,287 @@
"""The plugin's version is MINTED, and CI is the control that it moved.
WHAT THIS IS ABOUT (milestone 334 step 3). `plugin/` ships straight from this
git repo — no build step, so no moment at which CI could stamp a version in.
The value is therefore minted by a script before the commit, and CI's job is
not to produce it but to prove it moved when it had to.
THE DIVERGENCE THESE GUARD. Two artifacts in one repo derive their versions
from different clocks, on purpose:
server image name from COMMIT time, ordering key from BUILD time
plugin one value, from MINT time
#3127 §2 prescribes commit time so two lanes building one source report one
string. The plugin has one lane and no build, so that reason does not reach
it. "Let's make these consistent" is the obvious tidy-up and it breaks
whichever artifact loses — which is why the difference is pinned here rather
than only explained in a comment.
The trade mint time makes — you cannot recompute the value from history, only
verify it moved — is acceptable ONLY because #3325 established that the
installer's refresh test is `===` with no ordering anywhere. Where a
comparator orders, an unreproducible version would be unverifiable too.
"""
import ast
import json
import pathlib
import re
from datetime import datetime, timedelta, timezone
import pytest
from scripts import check_plugin
from scripts.mint_plugin_version import VERSION_RE, mint, rewrite
MINT_SRC = pathlib.Path(check_plugin.ROOT) / "scripts" / "mint_plugin_version.py"
@pytest.fixture(autouse=True)
def _reset_failures():
"""`check_plugin.fail` appends to a module global; without this a failing
assertion in one test would be visible from the next."""
check_plugin.failures.clear()
yield
check_plugin.failures.clear()
def fake_manifest(version: str = "2026.09.01.2252") -> str:
return json.dumps(
{"name": "scribe", "description": "d", "version": version,
"userConfig": {"api_endpoint": {"type": "string"}}},
indent=2,
)
# ── The mint ───────────────────────────────────────────────────────────────
@pytest.mark.parametrize("when,expected", [
# THE midnight case, which #3127 checklist 10 names by hand. An unpadded
# `%-H%M` renders this hour as `0` and silently shortens the string.
(datetime(2026, 1, 5, 0, 0, tzinfo=timezone.utc), "2026.01.05.0000"),
(datetime(2026, 1, 5, 0, 9, tzinfo=timezone.utc), "2026.01.05.0009"),
(datetime(2026, 12, 31, 23, 59, tzinfo=timezone.utc), "2026.12.31.2359"),
(datetime(2026, 9, 1, 22, 52, tzinfo=timezone.utc), "2026.09.01.2252"),
])
def test_the_mint_zero_pads_every_field(when, expected):
assert mint(when) == expected
assert VERSION_RE.match(mint(when))
def test_the_mint_is_UTC_not_local():
"""A local-time mint would make the value depend on who ran it — two people
minting the same minute would disagree, and the string is the artifact's
identity."""
utc = datetime(2026, 9, 1, 22, 52, tzinfo=timezone.utc)
east = utc.astimezone(timezone(timedelta(hours=9)))
assert mint(east) == mint(utc) == "2026.09.01.2252"
def test_the_mint_reads_a_CLOCK_and_never_git():
"""The clock divergence from the server image, asserted structurally.
Mint time is only meaningful if nothing consults history — the moment this
script shells out to git it has quietly become a commit-time deriver, and
the two artifacts' clocks have been "made consistent" without anyone
deciding to. That change would pass every other test in this file.
Asserted over the AST rather than the text, because the module docstring
discusses git at length explaining why it is absent. This looks for USE,
not mention.
"""
tree = ast.parse(MINT_SRC.read_text())
imported = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
imported |= {a.name.split(".")[0] for a in node.names}
elif isinstance(node, ast.ImportFrom) and node.module:
imported.add(node.module.split(".")[0])
assert "subprocess" not in imported, (
"the mint script imports subprocess — a mint that can read history is "
"a commit-time deriver wearing the wrong name"
)
called = {ast.unparse(n.func) for n in ast.walk(tree) if isinstance(n, ast.Call)}
assert "datetime.now" in called, "the mint script no longer reads a clock"
# ── The rewrite ────────────────────────────────────────────────────────────
def test_the_rewrite_touches_exactly_one_line():
"""Surgical, not a JSON round-trip. The manifest's formatting and key order
are not this script's to decide, and a whole-file reformat would make every
mint an unreadable diff."""
before = fake_manifest("0.1.48")
after = rewrite(before, "2026.09.01.2252")
b, a = before.splitlines(), after.splitlines()
assert len(b) == len(a)
differing = [i for i, (x, y) in enumerate(zip(b, a)) if x != y]
assert len(differing) == 1
assert '"version": "2026.09.01.2252"' in a[differing[0]]
def test_the_rewrite_preserves_indentation_and_key_order():
weird = '{\n\t"name": "scribe",\n\t"version": "0.1.48",\n\t"z": 1\n}\n'
out = rewrite(weird, "2026.09.01.2252")
assert out == '{\n\t"name": "scribe",\n\t"version": "2026.09.01.2252",\n\t"z": 1\n}\n'
def test_the_rewrite_refuses_a_manifest_it_cannot_match():
"""Raises rather than falling back to a JSON round-trip: a manifest this
cannot match is one whose shape changed, and quietly reformatting the file
to cope would be a far larger edit than the caller asked for."""
with pytest.raises(ValueError):
rewrite('{"name": "scribe"}', "2026.09.01.2252")
def test_the_rewrite_refuses_TWO_version_lines():
"""A capped `subn` would report one replacement and look clean while the
second `version` — possibly the real one — kept its old value."""
two = '{\n "version": "0.1.48",\n "nested": {\n "version": "9.9.9"\n }\n}\n'
with pytest.raises(ValueError):
rewrite(two, "2026.09.01.2252")
# ── The version-relevant set includes its own deriver ──────────────────────
def test_the_mint_script_is_version_relevant():
"""#3127 §3's asymmetry. A change to how the version is COMPUTED is
compared against nothing — leave the deriver out of the set and a format
change never forces a re-mint, so the manifest keeps a value in the old
format indefinitely and nothing says so."""
paths = check_plugin.version_relevant_paths()
assert "scripts/mint_plugin_version.py" in paths
for shipped in check_plugin.SHIPPED_PATHS:
assert shipped in paths
def test_the_checker_is_NOT_version_relevant():
"""The inverse, and it is the easy mistake. A checker decides whether the
lane goes red, not what the artifact reports — so its absence here is a
decision, not an oversight."""
assert "scripts/check_plugin.py" not in check_plugin.version_relevant_paths()
# ── The check ──────────────────────────────────────────────────────────────
def run_check(here: str, there: str | None, changed_paths: list[str], monkeypatch):
"""Drive `check_version_is_minted` with git stubbed. Returns the failures."""
monkeypatch.setattr(
check_plugin, "_git",
lambda *a: (0, "\n".join(changed_paths)) if a[0] == "diff" else (0, ""),
)
monkeypatch.setattr(
check_plugin, "manifest_version",
lambda ref=None: here if ref is None else there,
)
monkeypatch.setattr(
check_plugin, "manifest_text",
lambda ref=None: fake_manifest(here if ref is None else (there or "0.0.0.0000")),
)
check_plugin.check_version_is_minted("origin/main")
return list(check_plugin.failures)
def test_content_changed_and_the_version_did_not_FAILS(monkeypatch):
"""#2209, exactly. The headline, and the only reason the check exists."""
failures = run_check(
"2026.09.01.2252", "2026.09.01.2252",
["plugin/hooks/scribe_session_context.sh"], monkeypatch,
)
assert len(failures) == 1
assert "still 2026.09.01.2252" in failures[0]
assert "scribe_session_context.sh" in failures[0]
def test_content_changed_and_the_version_moved_PASSES(monkeypatch):
assert run_check(
"2026.09.01.2252", "2026.08.30.1200",
["plugin/hooks/scribe_session_context.sh"], monkeypatch,
) == []
def test_a_version_that_is_not_the_canonical_shape_FAILS(monkeypatch):
"""`K4` returns the manifest string verbatim, so a malformed value is not
rejected by the installer — it either sorts as an ordinary string or, when
unreadable, forces a reinstall every session. Neither is loud (#3325)."""
failures = run_check("0.1.48", "0.1.47", [], monkeypatch)
assert len(failures) == 1
assert "not YYYY.MM.DD.HHMM" in failures[0]
@pytest.mark.parametrize("bad", ["2026.9.1.2252", "2026.09.01.252", "2026.09.01"])
def test_an_UNPADDED_or_short_version_FAILS(bad, monkeypatch):
"""The padding is the contract, not cosmetics — one shape for every version
in the family (#3127 checklist 10)."""
assert run_check(bad, "2026.08.30.1200", [], monkeypatch) != []
def test_a_version_in_the_future_FAILS(monkeypatch):
ahead = (datetime.now(timezone.utc) + timedelta(days=400)).strftime("%Y.%m.%d.%H%M")
failures = run_check(ahead, "2026.08.30.1200", [], monkeypatch)
assert len(failures) == 1
assert "in the future" in failures[0]
def test_a_version_minted_minutes_ago_is_NOT_in_the_future(monkeypatch):
"""The guard has to tolerate ordinary skew: the mint happens on a
workstation and the lane runs later, on another machine's clock."""
now = datetime.now(timezone.utc).strftime("%Y.%m.%d.%H%M")
assert run_check(now, "2026.08.30.1200", [], monkeypatch) == []
def test_nothing_changed_and_nothing_minted_PASSES(monkeypatch):
assert run_check("2026.09.01.2252", "2026.09.01.2252", [], monkeypatch) == []
def test_a_version_that_moved_with_no_content_change_is_NOT_a_failure(monkeypatch):
"""Deliberately a pass. A needless re-mint costs one cache refresh; failing
the lane over a harmless act is how a check earns a `--no-version` in
somebody's muscle memory and stops running at all. The implication that
matters is one-directional: content changed IMPLIES version moved."""
assert run_check("2026.09.01.2252", "2026.08.30.1200", [], monkeypatch) == []
def test_a_failed_diff_FAILS_rather_than_passing_quietly(monkeypatch):
"""A check that cannot run must not report the same thing as a check that
passed — #2663's lesson, and the reason this file's siblings exist.
`rev-parse` is stubbed to SUCCEED so only the diff fails. Failing every git
call would trip the base-branch guard first and this would pass while
proving nothing about the diff arm.
"""
monkeypatch.setattr(
check_plugin, "_git",
lambda *a: (128, "fatal: bad object") if a[0] == "diff" else (0, ""),
)
monkeypatch.setattr(check_plugin, "manifest_version",
lambda ref=None: "2026.09.01.2252")
check_plugin.check_version_is_minted("origin/main")
assert len(check_plugin.failures) == 1
assert "git diff" in check_plugin.failures[0]
# ── The real manifest ──────────────────────────────────────────────────────
def test_the_shipped_manifest_carries_a_minted_version():
"""The end of the hand-bumped scheme, asserted on the real file. `0.1.48`
was the last of 48 numbers a person typed."""
version = json.loads(check_plugin.MANIFEST.read_text())["version"]
assert VERSION_RE.match(version), (
f"the shipped manifest says {version!r}, which is not a minted version"
)
def test_the_session_context_hook_still_reads_the_version_field():
"""The marker #2220 asked for. The value's SHAPE changed, not the field or
its reader — if this had to move, the derivation went somewhere it should
not have."""
hook = (check_plugin.HOOKS_DIR / "scribe_session_context.sh").read_text()
assert re.search(r"jq\s+-r\s+'\.version", hook)
+225
View File
@@ -0,0 +1,225 @@
"""The preference write path, and how it differs from a rule's (milestone 399).
WHY TWO DOORS AT ALL
`create_rule`'s docstring IS the approval gate (#3557): it tells its caller to
propose, offer three answers, and wait. That is right for a rule — the person
a rule binds should have agreed to be bound.
A preference inverts it. The operator's framing: *"preferences are rules that
scribe can and should update during use."* A preference that asks every time
never drifts, and drifting is the whole feature. Reaching one through
`create_rule(kind=...)` would mean reading it through the gate's prose, and
the caller would hesitate over exactly the act this kind exists to make
routine.
So the asymmetry is the product, and these tests pin it.
TWO PRESENCE CHECKS, NEVER AN ABSENCE
The tempting guard is "create_preference's docstring does NOT run the approval
loop". That is the shape snippet #3352 warns against: an absence check passes
against a docstring that has been deleted, emptied, or rewritten into
something else entirely, and it reads as coverage while proving nothing.
So the asymmetry is asserted as two PRESENCE facts — the rule door still asks,
the preference door still says write it — and each fails if its own side is
tidied away. Synonym families, structure not wording, the same bargain
test_rule_creation_asks_first strikes.
"""
from unittest.mock import AsyncMock, patch
import pytest
from tests.helpers import fake_rule, plain_rule_detail as _plain_detail
from tests.helpers import tool_doc as _doc
# The tool layer reads its caller from a ContextVar the HTTP transport sets.
# With no request in flight, the module binds it itself (snippet #2836).
pytestmark = pytest.mark.usefixtures("_bind_user")
MODULE = "scribe.mcp.tools.rulebooks"
# ── the required fields, and why each is required ───────────────────────
@pytest.mark.asyncio
async def test_a_preference_without_a_trigger_is_refused():
"""A preference with no `when_to_apply` is inert, not merely incomplete.
The trigger is two-thirds of the embedded document, so a record without
one never surfaces at the moment it applies. Refusing at the tool is the
difference between an error the writer can fix and a preference that is
written, stored, and silently never delivered — which looks identical to
one nobody wrote.
"""
create_mock = AsyncMock()
with patch(f"{MODULE}.rulebooks_svc.create_rule", create_mock):
from scribe.mcp.tools.rulebooks import create_preference
with pytest.raises(ValueError, match="when_to_apply is required"):
await create_preference(
topic_id=10, title="t", statement="s",
when_to_apply=" ", arose_from_id=42,
)
create_mock.assert_not_called()
@pytest.mark.asyncio
async def test_a_preference_without_provenance_is_refused():
"""Provenance is the price of the ungated write.
A preference is expected to change as the work teaches it. A corpus that
drifts with no record of what taught each change is one nobody can audit —
and the operator's veto over drift depends entirely on being able to read
why it happened.
"""
create_mock = AsyncMock()
with patch(f"{MODULE}.rulebooks_svc.create_rule", create_mock):
from scribe.mcp.tools.rulebooks import create_preference
with pytest.raises(ValueError, match="arose_from_id is required"):
await create_preference(
topic_id=10, title="t", statement="s",
when_to_apply="when x", arose_from_id=0,
)
create_mock.assert_not_called()
@pytest.mark.asyncio
async def test_create_preference_stores_the_preference_kind():
"""The tool's one irreducible job.
Asserted on the kwarg reaching the service rather than on the returned
payload: a tool that accepted the call and wrote a plain rule would
return something that reads correctly, and the force would be wrong.
"""
rule = fake_rule(id=100, kind="preference")
create_mock = AsyncMock(return_value=rule)
with patch(f"{MODULE}.rulebooks_svc.create_rule", create_mock), _plain_detail():
from scribe.mcp.tools.rulebooks import create_preference
await create_preference(
topic_id=10, title="Pace hard debugging",
statement="One step per turn.",
when_to_apply="during hard debugging",
arose_from_id=42,
)
kwargs = create_mock.call_args.kwargs
assert kwargs["kind"] == "preference"
assert kwargs["arose_from_id"] == 42
assert kwargs["when_to_apply"] == "during hard debugging"
@pytest.mark.asyncio
async def test_a_near_duplicate_preference_blocks():
"""The gate is what lets this corpus be written freely and stay small.
The second preference about a thing must UPDATE the first. Two that
quietly disagree are worse than none: retrieval surfaces whichever scores
higher, and nobody learns the other exists.
"""
from scribe.services.dedup import DuplicateMatch
dup = DuplicateMatch(id=47, title="Pace hard debugging", similarity=1.0, reason="title")
create_mock = AsyncMock()
with patch(f"{MODULE}.dedup_svc.find_duplicate_rule", AsyncMock(return_value=dup)), \
patch(f"{MODULE}.rulebooks_svc.create_rule", create_mock):
from scribe.mcp.tools.rulebooks import create_preference
out = await create_preference(
topic_id=10, title="Pace hard debugging", statement="s",
when_to_apply="when", arose_from_id=42,
)
assert out["duplicate"] is True
assert out["existing_id"] == 47
create_mock.assert_not_called()
@pytest.mark.asyncio
async def test_updating_a_preference_without_provenance_is_refused():
update_mock = AsyncMock()
with patch(f"{MODULE}.rulebooks_svc.update_rule", update_mock):
from scribe.mcp.tools.rulebooks import update_preference
with pytest.raises(ValueError, match="arose_from_id is required"):
await update_preference(rule_id=5, arose_from_id=0, statement="new")
update_mock.assert_not_called()
@pytest.mark.asyncio
async def test_update_preference_forwards_what_taught_the_change():
rule = fake_rule(id=5, kind="preference")
update_mock = AsyncMock(return_value=rule)
with patch(f"{MODULE}.rulebooks_svc.update_rule", update_mock), _plain_detail():
from scribe.mcp.tools.rulebooks import update_preference
await update_preference(
rule_id=5, arose_from_id=99, statement="the new way",
)
kwargs = update_mock.call_args.kwargs
assert kwargs["arose_from_id"] == 99
assert kwargs["statement"] == "the new way"
# ── the asymmetry, as two presence facts ────────────────────────────────
def test_the_rule_door_still_asks_before_writing():
"""Half one of the asymmetry. If this fails, the gate was tidied away and
preferences are no longer the exception — they are just the same thing.
"""
doc = _doc(MODULE, "create_rule").lower()
asks = ("approve", "propose", "ask", "question")
assert any(w in doc for w in asks), (
"create_rule's docstring no longer runs the propose-then-approve loop. "
"The preference path's whole justification is that it is the exception "
"to this; with the gate gone there is no asymmetry left to justify."
)
def test_the_preference_door_says_to_write_it():
"""Half two. The inverting instruction has to be PRESENT, not merely
unaccompanied by a gate.
An agent that has internalised #3557 will hesitate to write or rewrite a
preference unless told plainly that this door is different. Silence here
does not read as permission — it reads as an omission, and the caller
falls back on the behaviour it already knows.
"""
for tool in ("create_preference", "update_preference"):
doc = _doc(MODULE, tool).lower()
permits = ("expected", "no approval", "without asking", "ordinary work",
"write it", "not a liberty", "no proposal")
assert any(w in doc for w in permits), (
f"{tool}'s docstring no longer tells its caller that writing "
"without an approval loop is expected. A caller carrying "
"create_rule's gate will default to asking, and a preference "
"nothing ever updates is a rule nobody enforces."
)
def test_the_preference_door_names_the_force_distinction():
"""The routing test, stated positively (rule 165).
The confusion this milestone exists to fix is that a session cannot tell
which kind it is holding. If the docstring stops drawing the line, the
tool becomes a second way to write rules.
"""
doc = _doc(MODULE, "create_preference").lower()
assert "rule" in doc and any(
w in doc for w in ("followed", "binds", "breaks", "consistency")
), (
"create_preference's docstring no longer distinguishes a preference "
"from a rule by force. Without that line the tool is a second door "
"onto the rulebook with a weaker gate."
)
def test_the_preference_door_keeps_the_record_out_of_scope():
"""Preferences shape HOW work is done, never WHAT is recorded.
Worth pinning because "record it the way I like it" is the natural next
reach, and it would make dev-logs and issues idiosyncratic per author —
while the record is the one thing that has to outlive the person.
"""
doc = _doc(MODULE, "create_preference").lower()
assert "record" in doc, (
"create_preference's docstring no longer says that a preference does "
"not change what gets recorded. That boundary is the one a reader "
"would cross without noticing."
)
+174
View File
@@ -0,0 +1,174 @@
"""A logged query never carries a credential (#3925).
WHY THIS EXISTS
`pre_tool_rule` retrieves against the RAW COMMAND TEXT and `write_path_rule`
against the code being written, so whatever was on the command line or in the
buffer is what `record_retrieval` stores in `retrieval_logs.query`. A command
that exported a token therefore stored the token.
And storing it was not the worst of it. `near_miss_samples` is the readout the
threshold documentation tells you to open before moving a bar, so the value
came back OUT into an agent's context on the next tuning pass — which is
exactly how this was found, during #3853's threshold spike.
WHAT THIS PINS, IN BOTH DIRECTIONS, AND WHY THE SECOND HALF IS THE HARD ONE
A scrubber has two ways to fail and only one of them is obvious.
1. It misses a secret. Caught by the redaction cases below.
2. It eats the EVIDENCE. This is the failure that would do more damage,
because it is silent: the whole worth of a near-miss sample is reading the
query that was actually refused, and a scrubber that chewed up ordinary
commands would turn the one instrument for tuning a bar into unreadable
stubs while still looking like it worked. That is the #2663 shape — a
surface that reads fine and has quietly stopped saying anything.
So the second block is not padding. Its cases are REAL queries taken from this
install's `near_miss_samples` during #3853, and they must survive byte for
byte. If a future pattern is added and one of them changes, the pattern is too
greedy — tighten it rather than editing the expectation.
The secret cases use FABRICATED values in the real formats. Nothing here is or
was a live credential.
"""
import pytest
from scribe.services.retrieval_telemetry import scrub_secrets
# Fabricated, in the shapes that actually occur. The first is the shape that
# was found stored: a shell assignment of a vendor-prefixed token.
_SECRETS = [
("vendor-prefixed token in a shell assignment",
"TOK=flt_AAAABBBBCCCCDDDDEEEEFFFF\npython3 - <<'PY'",
"flt_AAAABBBBCCCCDDDDEEEEFFFF"),
("a Scribe fmcp_ key in an auth header",
"curl -H 'Authorization: Bearer fmcp_ZZZZYYYYXXXXWWWWVVVV' https://x",
"fmcp_ZZZZYYYYXXXXWWWWVVVV"),
("a forge token in an export",
"export GITHUB_TOKEN=ghp_1234567890abcdefghijABCDEF",
"ghp_1234567890abcdefghijABCDEF"),
("a value assigned to a secret-named variable",
'REGISTRY_PASSWORD="hunter2-correct-horse"',
"hunter2-correct-horse"),
("an api_key in a query string",
"curl 'https://api.example/v1/things?api_key=abcdef1234567890'",
"abcdef1234567890"),
("a private key block",
"-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKC\n-----END RSA PRIVATE KEY-----",
"MIIEowIBAAKC"),
]
# Real queries, from this install's near-miss samples during #3853.
_EVIDENCE = [
"git push origin dev",
'git pull --rebase origin dev 2>&1 | tail -3; echo "=== HEAD ==="; git log --oneline -2',
"python3 - <<'PY'\nimport pathlib\np = pathlib.Path(\"web/src/routes/admin/tuning/tuning.test.ts\")",
"docker compose up -d",
'package library\n\nimport (\n\t"context"\n\t"fmt"\n)',
"import {\n fetchTransfers,\n retryTransfer,\n} from './api'",
'grep -rn "useState" src/components/ | head -20',
# The word "token" in ordinary prose is not a token.
"explain how the token bucket rate limiter works",
"wc -l src/*.py && date",
# `--author=` contains "auth". A bare `auth` keyword in the assigned
# pattern redacted the address here, which is the evidence-eating failure
# this block exists to catch — and it shipped for one commit because the
# set did not contain a case with it. `AUTH_TOKEN=` is still caught, via
# `token`.
"git commit --author=bvandeusen@example.com -m 'x'",
"git log --author=\"Bryan Van Deusen\" --oneline",
]
@pytest.mark.parametrize(("label", "text", "secret"), _SECRETS,
ids=[c[0] for c in _SECRETS])
def test_a_credential_never_survives_into_the_query_column(label, text, secret):
"""The value goes; something visible stays in its place."""
out = scrub_secrets(text)
assert secret not in out, (
f"{label}: the credential is still in the text that would be stored"
)
assert "[redacted" in out, (
f"{label}: the span was removed without saying so. A silent deletion "
"leaves a reader unable to tell a scrubbed query from a short one, "
"which is the readout lying about itself rather than protecting you."
)
@pytest.mark.parametrize("query", _EVIDENCE)
def test_an_ordinary_query_is_stored_exactly_as_it_was(query):
"""Evidence survives byte for byte.
These came out of real `near_miss_samples`. A threshold is tuned by reading
them, so a pattern greedy enough to touch one has destroyed the instrument
it was meant to make safe — tighten the pattern, never this expectation.
"""
assert scrub_secrets(query) == query
def test_empty_and_missing_queries_pass_through():
"""Some sources log no query at all; scrubbing must not invent one."""
assert scrub_secrets(None) is None
assert scrub_secrets("") == ""
def test_the_write_path_scrubs_rather_than_the_read_path():
"""The payload built for storage carries the redacted text (#3925).
Pinned on `_build_payload` because that is the single seam every source
reaches the column through. A per-caller scrub would be three places for
one of them to be forgotten by whoever adds the fourth arm — and the one
forgotten would be the one that stored a secret.
"""
from scribe.services.retrieval_telemetry import _build_payload
payload = _build_payload(
user_id=1, source="pre_tool_rule",
query="export API_TOKEN=ghp_1234567890abcdefghijABCDEF && git push",
threshold=0.68, limit=5, project_id=0, is_task=None,
results=[], duration_ms=1.0,
)
assert "ghp_1234567890abcdefghijABCDEF" not in payload["query"]
assert "[redacted" in payload["query"]
# The rest of the command survives, or the row stops being evidence.
assert "git push" in payload["query"]
# ── the SQL twin has its own word-boundary spelling (#3925) ─────────────
#
# Migration 0099 carries an inlined copy of these patterns, deliberately: a
# migration is a frozen record of what already ran, and importing the live
# ones would mean it quietly did something different next year.
#
# Frozen is not the same as correct, and the first cut was neither. The
# boundary was dropped in the port, so `sk-` matched inside any word
# containing it — `<task-notification>` became `<ta[redacted:token]>` across
# thousands of rows on the one install that ran it. And writing `\b` would not
# have saved it: in Postgres ARE `\b` is a BACKSPACE, not a word boundary.
# `\m` (start of word) is the spelling that means what Python's `\b` means.
#
# So this pins the property no reader can eyeball, and it is a PRESENCE check
# on a token that must appear rather than an absence check on prose (#3352).
def test_the_migrations_patterns_anchor_to_a_word_start_the_postgres_way():
"""`\\m`, never `\\b` — the two are unrelated in Postgres."""
import pathlib
src = (pathlib.Path(__file__).resolve().parents[1]
/ "alembic" / "versions"
/ "0099_scrub_secrets_from_retrieval_logs.py").read_text()
for name in ("_TOKEN", "_ASSIGNED"):
line = src.split(f"{name} = (")[1].split(")")[0]
assert r"\m" in line, (
f"migration 0099's {name} no longer anchors to a word start. "
f"Without it a vendor prefix matches INSIDE a word — `sk-` in "
f"`task-notification` is the case that actually happened — and "
f"the UPDATE overwrites the only copy of the text it mangles."
)
assert r"\b" not in line, (
f"migration 0099's {name} uses `\\b`, which is a BACKSPACE in "
f"Postgres ARE rather than a word boundary. Python's `\\b` and "
f"Postgres's `\\m` look interchangeable and are not."
)
+25 -17
View File
@@ -46,7 +46,7 @@ def test_service_signatures_require_user_id():
"create_topic", "list_topics", "get_topic", "update_topic", "delete_topic",
"create_rule", "create_project_rule", "rule_detail",
"set_rule_systems", "add_rule_relation", "remove_rule_relation",
"list_rules", "list_always_on_rules",
"list_rules",
"get_rule", "update_rule", "delete_rule",
"subscribe_project", "unsubscribe_project", "get_applicable_rules",
"suppress_rule_for_project", "unsuppress_rule_for_project",
@@ -93,24 +93,8 @@ def test_suppression_association_tables_declared():
assert "rule_id" in cols or "topic_id" in cols
def test_rulebook_model_carries_always_on():
"""Migration 0058 added rulebooks.always_on — verify the model declares it."""
from scribe.models.rulebook import Rulebook
assert "always_on" in Rulebook.__table__.columns
col = Rulebook.__table__.columns["always_on"]
assert col.nullable is False
def test_update_rulebook_route_accepts_always_on():
"""PATCH /api/rulebooks/<id> must pass always_on through to the service.
The handler filters body keys against a whitelist; that whitelist needs to
include always_on or toggling from the UI silently drops the field.
"""
import inspect as _inspect
from scribe.routes import rulebooks as rb_routes
src = _inspect.getsource(rb_routes.update_rulebook)
assert "always_on" in src, "update_rulebook handler missing always_on in field whitelist"
def test_rule_and_subscription_handlers_callable():
@@ -122,3 +106,27 @@ def test_rule_and_subscription_handlers_callable():
"relate_rules", "unrelate_rules",
):
assert callable(getattr(rb_routes, name))
def test_the_rule_list_zero_fills_usage_on_every_row():
"""Milestone 333 step 5, asserted the only way this harness allows.
There is no live-HTTP fixture here (see this module's docstring), so this
reads the handler's source. What it can still prove is the property that
gets forgotten: the route must attach the key to EVERY row, zero-filled,
rather than only to rows that happen to have events. Every rule on every
existing install predates `rule_usage_events`, so a route that only
attached the key when it found something would leave the badge component
reading `undefined` on almost every row — and the difference between "no
events" and "no field" is exactly the distinction #2663 is about.
"""
import inspect
from scribe.routes import rulebooks as rb_routes
src = inspect.getsource(rb_routes.list_rules)
assert "usage_for_rules" in src, "the rule list does not read usage at all"
assert "empty_rule_usage()" in src, (
"the rule list does not zero-fill — a rule with no events would come "
"back without the key rather than with an empty one"
)

Some files were not shown because too many files have changed in this diff Show More