The derive check matches by name, so a renamed copy of a recorded snippet
passed it and showed only as "similar" prior art. When a write introduces a
definition the file does not hold (shape_ledger.new_shapes_at) and an unseen
snippet on the menu clears the write-path floor by WRITEPATH_COPY_MARGIN (0.80
at the default floor), the hint now says to reuse it or extract the shared
shape before another copy. Names the derive check or the stamping feed already
spoke for are left to them. Covers recorded snippets only: the ledger keeps no
bodies to compare an unrecorded sibling against.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 4xx with no S3 error body came from something else at that address, most
often a web server on the default port. The message now says so and to check
the scheme and port, beside the raw status. A 5xx without a body keeps its
own reason, since a busy S3 service can answer that way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pending() listed every suggested signal row, including pairs still gathering
evidence below the three-situation bar, so the queue held 200 "proposals"
where about 25 had crossed it. A signal pair now needs proposed_at in its
evidence, the same test the misfire half already applies; proposals from a
pass are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each moment gains by_rule: {rule_id, title, delivered, opened}, read from
the (moment, rule) grouping the readout already did. The moment's totals are
now summed from those rows, so the two cannot disagree. The Moments settings
page shows the list under each moment, closed by default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
surfaced_never_pulled told a rules reader there was no judged sample for the
rule arms. There is one now; the guard tying the text to REVIEWABLE caught it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule arm logs its query, floor, budget and project, which is everything
semantic_search_rules takes, so a logged call is re-run to a depth past its
budget and judged like an auto_inject menu. within_budget is the arm's own
cut: inside the budget and, for the banded command and write arms, inside the
band of the top hit. "Opened after" reads rule pulls, and rules created
after the call are dropped as notes are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"keep going", "yes please do", "is it deployed?": with no word of its own the
auto-inject query was the previous reply's tail alone, and the menu handed
back the records that reply was built from. In a sample of 59 logged calls,
28 came from short prompts; those with a subject of their own ("continue into
DRY 14", "fix the Windows gate") led with the record meant, while the ones
without were finished tasks already in context, with an open on 2 of about 15.
_has_subject checks the operator's words against a list of words that name
nothing; without one, the ranked menu is skipped and records named by number
are still shown. The prompt rule arm is unchanged and still searches with the
tail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
usage and rule_usage gain pulled_by_source, read from the per-source rows
the readout already had, and _silent_surfaces counts those doors as having
emitted. The rules-listing points were seen as of the previous commit,
through rule_usage.by_source.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_telemetry gains rule_usage.by_source, the twin of usage.by_source:
distinct rules each surface showed, how many an agent opened, and a ratio
that is null on a bulk delivery. list_rules(with_usage=True, days=N) adds
each rule's usage, windowed, and how many lessons are confirmed instances
of it, so linked and unlinked rules can be compared through MCP.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The write-path rule arm ran only past the notes guard: a write whose code
matched no snippet, issue or lesson returned before any rule was searched.
It now runs before that guard, on `code or path`, and a write that matched
nothing but a rule still returns the rule lines.
Operator ruling (2026-10-05): yes. A rule about the file being written does
not depend on a snippet resembling it.
Cost: one more rule search per write that matched no prior art, with its
own embedding (the query is `code or path`, not the notes arm's concept
query, so the per-request memo does not cover it). Plus the preference slot
from (c) on those writes.
Expected telemetry shift, to be read on its own once this reaches main:
write_path_rule calls rise toward write_path's count (14 days to 2026-10-22:
write_path_rule 4,634 against write_path 3,786 — the notes arm skips trivial
payloads, the rule arm does not, so the rule arm will now exceed it). Its
zero-result share rises too, since the newly covered writes are the ones
that matched nothing in the notes corpus.
Tests: the gate test is inverted (a write with no prior art still gets its
rule check); the design-only write now runs the rule arm on its path; two
notes-arm tests read the write_path row instead of any row, since the rule
arm writes its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PRE_TOOL_RULE and WRITE_PATH_RULE now take preference_slot=True, like
PROMPT_RULE. When no preference placed on score, the arm asks once more for
the best preference over the same bar, and shows it as one extra line.
Operator ruling (2026-10-05): yes. A preference about how to act ("let each
action land before the next") is ranked against every rule whose trigger
names the same command; the act is its last chance to shape anything.
Cost: one kind-filtered search per call that placed no preference. The
query embedding is memoised per request on both hook routes, so no extra
model call. Expect a new stream of preference_slot rows from the two act
arms; preference_slot's call count will rise roughly by pre_tool_rule's and
write_path_rule's (~23k calls over 14 days before this change), and its
zero-result share will rise with it, since most commands have no matching
preference.
Tests: the tool-arm call-row tests count rows under pre_tool_rule only (the
slot logs under its own name), and the write-arm bar test reads the arm's
own search, which is the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prompt_rule, pre_tool_rule and write_path_rule now share one settings key,
RULE_FLOOR_KEY = "kb_rule_threshold", default 0.70. The reply backstop keeps
its own 0.80, because there the floor is a stop bar. Moving the shared floor
through tune_retrieval or the Settings form records a tuning event on every
surface that reads it (set_dial, via retrieval_surfaces.sharing_floor), and
the result names the surfaces it carried as `moved_with`. Settings shows one
"Standing-rule confidence threshold" control where it showed three.
Operator ruling (2026-10-05): one shared bar for the rule arms, "especially
since we're only taking the first x number of them" -- the budget limits
what is shown; the bar only decides what is worth ranking. Kept a dial.
Why 0.70, from records read rather than percentiles:
- write_path_rule near-misses at 0.7199-0.72 were all false positives
(rule 149 against a Lidarr comment, 157 against a retention Svelte file,
121 against an Archives restore, 164 against a config test, 131 against a
Go package), so the old 0.72 was not hiding real hits -- but the band does
noise control on this arm, so dropping 0.02 costs little.
- prompt_rule: the operator set 0.70 by hand on 2026-09-18, after a 0.60
experiment admitted junk and was reverted. 0.70 is the number with a human
decision behind it.
- pre_tool_rule lines near 0.68 this session were mostly noise; rising to
0.70 trims them, and the band still narrows what clears.
Retired keys: kb_promptrule_threshold, kb_toolrule_threshold and
kb_rulehint_threshold are no longer read. Rows an install stored under them
stay in the settings table, inert. On this instance the effective bars move
pre_tool_rule 0.68 -> 0.70, write_path_rule 0.72 -> 0.70, prompt_rule stays
0.70; record_release_defaults writes those per-surface release rows on boot,
so the telemetry shift has an event to explain it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once the binding has moved, the old remote is no longer bound, and
move_binding refused. An install that renamed its repo before snippet
locations moved with it had no way to move them. When the old remote is
unbound and the new one is bound to this project, bind_repo now skips the
binding move and still moves the snippet locations that name the old
repo.
The snippet-move test gives each location its own path: rewritten to one
repo, identical locations merge into one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A repo renamed on its forge kept every snippet location naming the old
repo: the binding and the shape ledger moved, the snippets did not.
- snippets.move_location_repo rewrites, on every live snippet of the
project, each location whose repo is the old remote in any form (URL,
normalized key, "owner/Repo", bare "Repo", compared without case) to
the new normalized key. A snippet the caller can only read is listed
in read_only and left alone.
- Its loop is shared with move_locations through _rewrite_locations,
which saves each changed snippet through update_snippet.
- bind_repo calls it after the binding moves and reports snippets_moved
and snippets_read_only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With no SECRET_KEY set, Grimoire fell back to a fixed placeholder. The
quickstart compose substituted change-me-in-production, so the README's
own update step (pull, up -d) recreated the container on that placeholder,
and every credential sealed under the key the user started with stopped
opening.
- config: with neither SECRET_KEY nor SECRET_KEY_FILE, the first start
writes a random key to DATA_DIR/secret_key (mode 0600, published with a
hard link so concurrent starts agree), and later starts read it back.
An explicit key wins and leaves the file alone. A missing
SECRET_KEY_FILE or an empty key file stops the start instead of
switching keys. With no data directory (a run outside the container)
the process gets a temporary key and says so.
- app: warns on a public placeholder key, on a temporary key, and when
/data is not a mounted volume.
- Dockerfile creates /data.
- docker-compose.yml is now the deploy compose (published image, both
volumes, nothing to set). The build-from-source compose is
docker-compose.dev.yml, and docker-compose.quickstart.yml is gone. The
Swarm stack file gains the /data volume and keeps its explicit key,
because a Swarm can move the app away from a node-local volume.
- README: a Deploy section with the compose inline, SECRET_KEY optional,
"keep both volumes", and updates that keep the key. A test holds the
README's copy equal to the file.
- Installs that ran on the change-me-in-production placeholder get a key
of their own on the new compose; Settings then asks once for saved
forge tokens and the backup secret.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LICENSE carries the GNU Affero General Public License v3.0 text, the same
file the other Fabled projects ship. pyproject.toml, the frontend's
package.json and lockfile, and the plugin manifest declare it as the SPDX
expression AGPL-3.0-only, and the README says what it asks of anyone who
runs a modified version over a network. Reading `license` as an SPDX
expression needs setuptools 77, so the build floor rises to it. The
project description now matches the README's tagline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AppLogo still drew the old open book with a spark, so the new mark showed
only in the browser tab. It now draws the same clasped grimoire as
favicon.svg, in the mark's own fixed colours. That covers the header and
the sign-in, register and password pages.
The README follows the house order: the logo, name and tagline, a short
summary, quick start including the plugin install, the settings a reader
must set, setup pitfalls, updating and backups, features grouped by what
you do, troubleshooting by the error text, development, and licence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Snippet locations are keyed by path too, so a snippet recorded under a
moved directory reads `missing` at its next check. The tool is now
move_path(project_id, from_path, to_path). It moves the ledger rows as
before, and re-paths the project's snippet locations under from_path
through update_snippet, so each body and its indexed mirror stay one
record. A snippet the caller can only read is listed in
snippets_read_only. The tool refuses only when nothing at all is recorded
under from_path. The path test is the one snippets already uses
(_path_touches), and the plugin version is minted for the skill edit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ledger rows are keyed by path, so after a directory moves the next sync
marks every judged row vanished and lists the same shapes again, unjudged.
move_shapes(project_id, from_path, to_path) re-keys the rows under one path
to the other, in every repo of the project, keeping each row's judgment,
uses edges and history and adding a `moved` event that records the path it
left. A row already at the target is replaced when nobody judged it; a
judged one stands, and the old row is reported in `kept`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A binding and every row of its project's code-shape ledger are keyed by the
repo's remote. A rename on the forge changes the remote, so binding the new name
started an empty ledger beside the old one and left every judgment behind.
bind_repo now takes moved_from, the old remote. Its binding and the project's
ledger rows move to the new key in one transaction, and the reply says how many
shapes moved. The move is refused when:
- the old name isn't bound to that project;
- the new name is already bound;
- the project already has a ledger under the new name.
Each refusal leaves both names exactly as they were.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The forge repo is now bvandeusen/FabledGrimoire. The plugin marketplace default,
the install commands in the docs and plugin README, the CI requirements title and
the example repo URLs in comments and tests follow it. docs/development.md now
names the image the build actually publishes, fabledgrimoire.
Two test names still said "scribes_own"; they now say "grimoires_own".
What still says scribe does so on purpose:
- the backup reader's accepted archive names and prefix;
- the label every stored secret is sealed under;
- the notes on keeping an old install's database names;
- the old migrations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
favicon.svg is a violet book with gold corners and clasp and a four-pointed star
in a ring, replacing the indigo book with a spark. It is rendered to 192 and 512
PNG app icons and a 180 touch icon on the page surface.
manifest.json lists those icons instead of a favicon.ico that never existed. It
describes the product as the README does, and takes its colours from the design
system's page surface. index.html's theme-color follows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
theme.css loaded Fraunces, Inter and JetBrains Mono from Google Fonts. That sent
every visitor's address to Google on every page load, and an instance with no
internet drew in fallback type. All three families are now vendored under
frontend/src/assets/fonts with every subset Google publishes for them, plus their
OFL licences. The Content-Security-Policy no longer lists fonts.googleapis.com or
fonts.gstatic.com.
A new guard test fails if any stylesheet, page or component links a remote asset
or preconnects to one, or if the CSP names another host.
The theme.css header now describes the sheet as it is instead of how it used to
read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The step-3 sweep matched `scribe` only after a non-letter, and in these
three tests the shell call sits right after a `\n` escape in a Python
string, so the character before it was the letter n. They still called
scribe_recent_context, scribe_skip_prompt and scribe_checkpoint_allowed;
they now call the grimoire_ names. No other escape-adjacent occurrence
remains in the tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 3 of the Scribe -> Grimoire rename.
- Plugin `grimoire` in marketplace `grimoire-plugin`, MCP server key
`grimoire`, so tools reach a session as mcp__plugin_grimoire_grimoire__*.
- hooks/scribe_* -> hooks/grimoire_*, their shell functions and the TMPDIR
ledgers with them; SCRIBE_URL / SCRIBE_TOKEN / SCRIBE_HOOK_DIR /
SCRIBE_LEDGER_DIRS -> GRIMOIRE_*; the project marker is `.grimoire`.
- skills/using-scribe -> using-grimoire, /grimoire:sync, stored processes
arrive as grimoire-proc-<slug>; moment_actions names them the same way.
- Migration 0124 rewrites stored skill names (rule_moments,
rule_moment_judgments, moment_mappings) from scribe:/scribe-proc- to
grimoire:/grimoire-proc-, so a rule mounted on a skill keeps firing.
- Settings' install commands and MCP-only defaults, the docs and the
tests follow.
- scripts/check_plugin.py drops the smoke event for a report-check hook
that has no file in plugin/hooks.
Kept on purpose: the secret-sealing label, and the names that let backups
written as Scribe restore and be counted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 2 of the Scribe -> Grimoire rename.
- src/scribe -> src/grimoire, and every import, patch target, module path,
hypercorn entry point and `python -m grimoire.restore` with it; pyproject,
uv.lock and the frontend package are named grimoire.
- Image git.fabledsword.com/bvandeusen/fabledgrimoire; CI's Postgres service
runs as grimoire / grimoire_test.
- Compose files default the DB user, database and (prod) volume to grimoire,
each overridable from .env, so an install created as Scribe keeps its data
by setting POSTGRES_USER, POSTGRES_DB and PGDATA_VOLUME.
- Backups are written as grimoire-<ts>.tar with format grimoire-backup-tar.
Archives named scribe-* with the scribe-backup-tar manifest still restore
and are still counted at the destination (folder and S3), with tests.
- Exports, the MCP auth realm, the ASGI scope key and the persistent logger
carry the new name.
Kept on purpose: the secret-sealing key label stays "scribe.secrets.v1",
because every stored secret was derived under it.
Plugin identity (plugin name, hooks, env vars, .scribe marker, skills) is
step 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1 of the Scribe -> Grimoire rename: every capitalised "Scribe" in prose
becomes "Grimoire" ("Fabled Scribe" -> "Fabled Grimoire") across the UI,
the PWA manifest and page title, the MCP instructions and tool docstrings,
hook and skill text, docs and README, with the tests that assert on that
prose moving with it. Old migrations keep their wording.
Identifiers are untouched here: the package, plugin name, MCP server key,
hook file names, env vars, the .scribe marker and FabledScribe repo/image
names move in steps 2 and 3, along with prose that quotes them.
The sweep matched "Scribe" only as a whole word, so describe/subscribe and
friends are untouched. The longer name took the MCP instructions one
character over their 1,600 budget; two phrases are shortened to fit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The build job ran `docker system prune -af` on the runner's shared Docker
daemon. That deletes every image no container is using at that moment,
including images other repos' jobs had just pulled and were about to start.
Jobs with a service container wait ~14s for it to become healthy between
pull and create, and fail with "No such image" when a prune lands inside
that wait (CI-runner #16). The runner host prunes nightly instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Profile tab said an assistant used it to personalise chat and a daily
journal. Neither exists, and nothing read the profile: the page saved to
user_profiles and the backup carried it. Removed the tab, the client calls,
/api/profile, the service and the model, and migration 0123 drops the table
(downgrade recreates it empty).
Backup format v27 stops writing user_profiles; an older archive carrying the
section still restores, and the file-restore integration test now proves it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The acceptance greps found what the per-area passes missed:
- src/scribe: "Phase 7/8" removal notes in projects, settings and
user_profile; a paragraph promising an auto-summary side effect that no
longer exists; user_profile's claim that "upcoming MCP profile tools" read
the profile, which nothing does; an instance count and instance rule
titles in plugin_context; "MILESTONE 323 DECIDED" in rulebooks.
- scripts/: check_plugin's header and inline comments were a history of
the incidents behind each guard, with issue numbers in two failure
messages; check_dangling_styles named this app's classes as its examples;
check_design_tokens, measure_duplication and mint_plugin_version cited
issues and checklists.
- frontend/src: forty-odd comments with four-digit issue numbers that the
earlier frontend scan filtered out as hex colours, plus "used to" and
"Phase N" notes.
Kept: illustrative ids (`#4181` as a bare-id example, `PR #198`), migration
Create Date headers and one migration's dated data floor, and the plugin
skills' worked examples of how to cite a record.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twenty-eight test files described the code they guard by what it did
before a fix. Each is now the property and the failure it prevents, in the
present tense: the backfill snapshot, the per-dial calibration query, the
rule scope, the lesson trigger filter, the subshell global, the trash owner
scope, the version readout fallback, and others. A docstring line that
narrated the backup suite's own history is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings, section headers and assertion messages drop the record number,
milestone or step that introduced the behaviour, and incident accounts are
restated as the present-tense failure shape. A dead pointer to a design doc
that is not in the repo is removed.
Renamed to the property they check:
test_the_old_shape_would_be_caught -> test_a_rank_then_scope_query_is_caught
test_a_ledger_repeat_no_longer_produces_that_case_at_all -> test_a_ledger_repeat_is_counted_not_withheld
test_the_header_no_longer_promises_once_per_session -> test_the_header_explains_the_seen_marker
test_the_portal_case_no_longer_stamps_one_canon_onto_a_whole_file -> test_one_canon_is_not_stamped_onto_a_whole_file
test_the_repos_own_prose_no_longer_defines_with -> test_the_repos_own_prose_does_not_define_with
test_the_original_regression_create_note_with_content -> test_the_real_server_refuses_content_on_create_note
Fixture and example ids stay: "note 5" in a supersession chain, "rule 156"
in an illustrative question, `#2275` as the hex-versus-citation case,
`#3244` as the example of a bare number, fixed dates in date-filter tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Module and test docstrings state what is pinned without the record number
or step that introduced it. Incident accounts become the failure shape in
the present tense: the pipefail head, the code_shapes import, the export
order, the facet dialects, the cross-kind repeat, the write-once kind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings and comments state the property and its reason in the present
tense. The session-context ledger test no longer describes the decision it
"used to pin"; the supersession tests state demote-not-filter without the
quote; one-row measurements become the general shape with their figures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourteen more test files: docstrings state the failure each guard defends
as a present-tense shape instead of the incident that found it, and issue,
milestone and rule numbers leave comments and assertion messages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twelve more test files drop issue, milestone, rule, note, lesson and
snippet numbers and the dated or one-project figures that recorded how
each property was found; the docstrings state the property and why it
holds. The contract-around-the-change guard keeps the dict-key example as
the failure a signature-watcher misses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ten more test files: issue, milestone, rule, decision and snippet numbers
come out of docstrings, comments and messages; the preference door states
what a preference is instead of quoting the framing; the JSON reader says
why jq is not assumed instead of what every hook used to do; the backup
round trip's column warning no longer claims the import side is unguarded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state their properties without issue, milestone and
rule numbers or dated regressions. The version tests describe what folding
the channel into the version breaks instead of the date it did; the surface
registry and settings-defaults tests state that floors move rather than
quoting the decision that made them move.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state their properties without issue, milestone,
rule and decision numbers. The tuning and suspension notes describe the
symptom a shipped default or a late ledger produces instead of one
install's dated readings; the reply-discipline and checkpoint docstrings
state the failure they prevent instead of the session that measured it.
test_empty_selector_is_the_rule_115_baseline is renamed for what it
checks: the no-forge baseline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven more test files: docstrings and comments state the property and its
reason, the SQL-twin note states why `\m` and not `\b` without the install
it broke, and fixture provenance says "real near-miss queries" rather than
naming the spike that collected them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six more test files state their pinned properties without the issue and
milestone history: the pull-telemetry guard explains enumerating kinds
instead of listing two getter incidents, cited-record status explains why
a hint's id needs its status instead of recounting one misreported step,
and the trigger-shape guard keeps its reasoning about why prose heuristics
fail while dropping the dated corpus audit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state the property they pin as a present-tense
fact. Incident narratives (a dated storage stall, a week of zero tuning
events, a two-project poisoned ledger, a twenty-site owner audit) become
the failure shape each test defends; one-install figures that name a rule
become a general worked example with the calibration numbers kept. The
family-grouping regression guard is renamed for what it checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shape ledger, MCP projects/search/rulebooks tools, backup, chunking,
plugin context, derived mirror, rule band, shape form gate, plugin
version mint and note usage tests drop issue, milestone and rule numbers,
dates and one-install figures from docstrings, comments and messages.
Three shape-form-gate test names that told the discovery story now name
the property: a registry module's helpers are not service functions, the
divergence gate silences only a family contradiction, and callable helpers
beside a callable canon still prompt. The rulebook tool-count comment that
logged each milestone's additions is replaced by the reason it is a count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_pattern_coverage, test_guidance_ownership, test_integration_shape_classify,
test_instruction_surfaces_agree, test_retrieval_warnings, helpers.py and
conftest.py state the property each test or fixture holds. The issue and
milestone numbers, consolidation dates and one-install measurements that
recorded how each was found are dropped; fixture dates and ids stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings, comments and assertion messages in the three most-cited test
files now say which property each test holds and why, without the issue,
milestone or rule numbers, dates and one-install measurements that recorded
how each property was discovered. Fixture ids and calibration scores stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Components, stores, api clients, router, types and shared CSS now state what
each piece is and why, without the milestone that introduced it. diff.ts
names its consumers instead of the copy history; stores/notes and tasks state
their current scope instead of the cleanup that set it; theme.css states the
token practice instead of the sweep that established it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
components.css says why shared recipes live there without the five-copy
.btn-primary count; KnowledgeView's editor-routing and chip comments state
the present reason instead of the bug that once routed snippets wrong.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record and milestone citations go; the ACT-arm bar split and the per-arm
budgets state why they exist without the measurements that motivated them.
A comment about a removed think_enabled setting, pointing at a file that no
longer exists, is deleted, as is the story of a removed design-system picker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record, milestone, phase and plan citations go from every migration header;
the "end before start" fix (0043) and the suppressed-count column (0095)
state the condition they prevent rather than the incident. 0053 and 0055
pointed at design specs under docs/superpowers that are not in the repo;
those pointers are removed.
Kept: alembic's generated "Create Date:" headers (template metadata, not
rationale), 0121's ARM_BORN justification beside the constant, and 0051's
illustrative "update task 42" format example.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone/step labels, record ids, the rescoped-rule date and the 296-call
measurement go; each header states what the change protects. "rule 36"
citations become the DROP + ADD practice they named.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shipped tool docstring (mcp/tools/repos bind_repo) cited this instance's
"rule 1: dev is home"; it now says what a dev-branch project should do.
routes/knowledge, routes/tasks, routes/systems and models/retrieval_log state
the property each guards rather than the bug that once broke it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
routes/search states the derived-facets property rather than the silent
whole-corpus bug that motivated it; routes/rulebooks gives the preference
restore exception its reasoning without the milestone numbers it cited.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- api.py's build-version docstring states why a channel name is not a build
id, without the dated incident.
- rule_version states why rules need a history, without the rule that was
rescoped and the date.
- Capitalised "Rule N" / "Milestone N" citations the earlier lowercase grep
missed, in reply_shapes, supersession, services/family and models/family.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Corrected while reading: routes/notes.py said lessons and snippets pass the
owner to set_record_systems, "a real inconsistency". Every door passes the
caller; the comment now says so.
rule_usage's model docstring states what an outcome measures and why the
row cannot share the note table, without the measurement and record ids
that motivated it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule-embedding docstring carries the own-table test itself instead of
pointing at a record; the later siblings refer back to it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The deferred-backfill and inline-release-defaults comments describe the
lifespan-overrun hazard they guard without the dated incident; backfill
passes say what they fill rather than which milestone added the vectors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The relation kinds say what each prevents without the rule-46 split story;
the removed tier, subscriptions and suppressions are stated as what the
model does not have, with the migration that dropped them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pre-tool rules docstring says why action rules need their own surface
instead of narrating that they once lived in the preload; the write-path
checkpoint note states the reason the hook never blocks rather than whose
decision it was.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backup_s3 says what its signing is checked against instead of which of the
operator's projects it was ported from; rulebooks and shape_check state what
the code is for rather than which milestone it belonged to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 1-2 hit tail: access, background, backup, backup_job, canonical_systems,
design_stylesheet, forge_connections, moment_actions, notes, notifications,
placement, planning, platforms, projects, record_batch, recurrence,
report_check, retrieval_review, rule_scope, rule_versions, scheduler,
shape_check, supersession, system_rulings, system_usage, text, trash,
verification, version_pinning.
Also corrected while reading:
- backup.py said restore step 16 fills the System map; step 15 does.
- version_pinning.py pointed at a design doc that is not in the repo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_migration states what it lets a tuned floor do without quoting
the operator, and its order-of-operations refusal is described as enforced
rather than as history. logging states why raw date filters fail rather
than citing the two issues that found it. The rest drop milestone, step and
record citations; step references now name the code that does the job.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- db_maintenance_scheduler states the storage-stall hazard the startup
timeout guards, without the dated outage.
- design_starter_roles states why a missing role produces literals, without
the call-site count, the operator's call date or a project's name.
- The rest drop milestone/step labels and record citations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_surfaces' header states why the arms read their numbers from one
table and why a floor is moved by reading the refused record, without the
quoted decision, the retired arm or the declined-rule instance.
family_adoption drops milestone/step labels from its section headers and
states the no-ids reason instead of naming the trap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
inception states the legacy stamp and the retired rulebook keys as facts
about stored records; family keeps the repeat-trigger calibration numbers
without the date and record they were measured on, and drops the instance
that taught the per-kind comparison.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
record_refs states why a guessed id is refused without quoting the
operator; moment_delivery's docstrings name what each door does without the
milestone and step that built it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The name gate, catalog and starter-set docstrings drop milestone and record
citations and the history of the tuple constant; each states why it is
shaped this way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The silent-outage story (#2663), the ranked/ambient split (#2477), the
transfer evidence (#3735) and the attach_usage consolidation (#4230) are
stated as the property each guards, without the incident.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pool-exhaustion, handshake-size and stepless-roadmap comments state the
condition the code guards instead of the measured incident; citations go.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module header drops the milestone/step map and record citations; the
per-user optionality, the keyring model and the empty-selector baseline are
stated as present-tense facts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record and milestone citations go. The logs_unconditionally docstring, the
place arm's comment and the fan-out key note state the condition they guard
rather than the bug that once hit it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring states why the model moves dials and why a reason is
required, without the decision quote or the declined-rule story. Bare
record citations go; the release-default and ledger-gap docstrings describe
the condition they guard instead of the measurement that found it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, milestone, snippet and rule-number citations are removed; the scope
docstring states the leak it prevents without naming the milestone that
closed it for the direct arms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ACL header states the share behaviour without its date and decision ids;
the facet table, door map, floor constant and facet-count comments state why
they are single sources instead of retelling how copies drifted; this
instance's counts (legacy plan-tasks, issue share) are removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring keeps why the kind exists, where its trigger lives, what
it inherits and why it needs no migration, without milestone, decision and
note ids or this instance's snippet counts. The body-strip docstring states
why taking the note matters instead of retelling the defect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring keeps why rules need their own usage stream and why the
ranked list is the one that is named, and drops the dated story of the
bulk surfaces starting to emit, measured decline rates and the corrected
earlier wording. Outcome docstrings describe the four states without
milestone references.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, decision, milestone and operator-decision citations are removed;
"previously each route…" and "list_tasks once returned…" stories become the
present-tense reasons they taught. The note about the removed delete_note
now says plainly that there is deliberately none, and why.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, milestone and rule citations and dated measurement headers are
removed; calibration figures stay. Corrected the lesson-gate comment, which
said lessons are not in _REPORT_KINDS and so have no duplicate-report
backstop. They are, and the comment now says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone, issue, note and rule-number citations are removed. The
co_surfaces docstring states why paired rules need the edge instead of
retelling the instance that taught it; the list-rules and applicable-rules
docstrings drop the retired subscription history. The preference-restore
error string no longer cites a milestone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, decision, milestone and rule citations are removed; the snippet-mirror
docstring drops the paragraph about its own earlier wording, and the
pointer-model and freshness comments state the design rather than the plan
step that introduced it. Access notes point at services/access.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, note and milestone citations are removed along with first-deploy
counts and the measured phantom-definition story; "button B" jargon becomes
the divergence it names. Each block keeps its present-tense reason.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The recurring "#2663 shape" and similar issue citations become the property
they stand for (an outage that reads as a quiet window, a statistic that
cannot vary). Dated observations, milestone tags and rule numbers are
removed; the near-miss and pull-through reasoning stays in present tense.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The format-version log keeps what each version adds, because restore depends
on what an older archive can carry; dates, milestone and issue ids, and the
stories of what a missing column once cost are removed. Compatibility
comments now name the archive version instead of a milestone. Two leftover
history-phrased comments in embeddings and shape_ledger are restated as
present-tense reasons.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone tags, dated measurements and "used to" stories are
removed; the chunker version list now says what each stamped version
contains. Corrected two comments that said a failed search leaves
report["searched"] absent (it is stamped False first), and pointed the
rule-search docstring at _RULEHINT_BAND in retrieval_pipeline instead of a
tier note in plugin_context that no longer exists. plugin_context's own
dangling "note on tiers" pointer is reworded to match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone and step numbers, and the stories of copies drifting
apart are removed; each invariant and slot is justified in present tense.
The rule-band probe table stays as calibration. Section headers drop their
milestone tags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, snippet numbers, dated measurements and the proposer's version
history are removed; each block now states the design reason in the present
tense. Calibration figures that justify a floor stay, without the instance
they were measured on. One error string drops a note id. The density
threshold comment moves above the constants it describes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone numbers, dated telemetry readings and rule numbers
leave the retrieval-arm comments and docstrings. Each bar keeps its
present-tense reason and the probe-score calibrations, which describe the
model on any install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MCP tool docstrings and comments, the server's orientation notes, the plugin
skills, every hook script and the plugin README/PACKAGING drop record ids,
milestone numbers, dates and incident stories, and state each non-obvious why
as a present-tense fact. History stays in the task logs.
Corrections found on the way:
- plugin README/PACKAGING said every hook is a GET; moment, rule-opened and
reply-rules POST, so the plugin's key needs write scope. Both now say so,
name Settings -> MCP Access, and list every wired hook and endpoint.
- the contract-change hint printed "(lesson #4207)", this instance's record,
to every install. It no longer cites a record.
- tasks.py claimed there is no delete_task; there is.
- rulebooks.py cited delete_event and a design spec that no longer exist.
- scribe_after_write.sh carried an always-empty etag_q; removed.
- test_write_path_trigger pinned "| tac | scribe_defs | head -1", text that
existed only in a comment about the old code; it now pins the real tail -1
pipeline.
- docs/development.md: CI runs on Gitea Actions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
g t pushed "/", which now redirects to the dashboard, so it did what g h
does. It pushes /knowledge. The overlay gains g r (Rules) and g g (Graph),
calls g h "Dashboard", and docs/features.md lists g t.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone 544, step 6217 (follow-up to #1778, which rewrote features.md and
api-reference.md in July but left architecture.md and configuration.md alone).
- architecture.md, configuration.md, development.md: rewritten against the
code. They still described Ollama, chat, SSE generation, voice, project
summaries, image caching, Docker secrets that the compose files don't use,
a raw-SQL migration convention the migrations no longer follow, a 27-entry
migration list, CalVer YY.MM.DD.N tags and the removed infra/ files.
- features.md: adds what shipped since July (spikes, claims, moments, rule
versions and verification, lessons, reply shapes, snippets and the shape
ledger, forge integration, family canon, design systems, retrieval tuning)
and corrects the Settings tabs and keyboard shortcuts (g g is the graph).
- api-reference.md: adds the endpoints for snippets, lessons, design
systems, family canon, platforms, shared areas, retrieval, rule versions,
relations and verification, forge connections, webhooks, scheduled
backups and the newer plugin hooks.
- api-keys-and-mcp.md: Settings tab is "MCP Access"; a plugin install
section; the tool-family table matches the tools that exist.
- design-system.md: the dated "decisions in progress" changelog (April
PR log, chat/journal/calendar surfaces) becomes a short present-tense
section; the stored design system is named as the authority.
- sso-oauth.md: the app is Scribe, not "Fabled Assistant".
- README: feature summary and doc table brought current.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At the operator's request (milestone 544, step 6203):
- docs/2026-03-23-fable-mcp-{design,plan}.md, 2026-03-29-voice-s2s-design.md
and 2026-04-10-agentic-briefing-design.md were plans for work that has
since shipped or been retired (voice and briefings left in the pivot).
Plans live in Scribe milestones.
- infra/act-runner-config.yml and runner-compose.yml described one
install's CI runner (an /nfs path, the py3.12-node22 label, a
RUNNER_SETUP.md that no longer exists). The runner is the CI-Runner
repo's concern.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings the three compose files in line with rule 198 (milestone 544,
step 6203, at the operator's request):
- no healthcheck on the app or the database, so a slow pg_isready on
stalled storage can no longer get a working database killed;
- no deploy.restart_policy in the Swarm file, so Swarm's default
(condition: any) restarts clean exits too, instead of on-failure with
max_attempts leaving the app down;
- depends_on is plain start order; restart: unless-stopped stays on the
plain-Compose files (Swarm ignores it), and the dev app gains it.
The orchestrator used to be what retried a container that started before
Postgres was ready. That is now the app's job: alembic/env.py calls
services/db_ready.wait_for_database before migrating. It retries refused
connections, unresolved hosts, connect timeouts and "the database system
is starting up" with 1-2-4-...-10s spacing for up to 300s, then exits
non-zero; a wrong password or missing database fails at once.
Per-request recovery is the engine's pool_pre_ping.
Also fixes the dev compose database image: postgres:16-alpine has no
vector extension, so migrations could not run against it. It now uses
pgvector/pgvector:pg17 like the other two files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment-only pilot for the comment standard (rule 32, milestone 544).
Issue and rule numbers, dates, run ids and incident stories come out;
the constraint each one guarded stays as a present-tense sentence.
History removed from the files, kept here:
- Dockerfile/CI: installing from uv.lock replaced pip/uv-pip installs that
ignored the lock (#2194, the mcp 2.0.0 break on 2026-07-28); --locked
then caught pgvector missing from the lock.
- CI: plugin paths were added after #2198/#2209 reached a live install;
`integration` joined the build's needs after run 8212 published
:latest with integration red (#4958); the `[[ ]]` tag step failed
silently on dash (2a374d9); BUILD_VERSION carried the channel until
2026-08-31 (#3298).
- jq was removed from the hooks in #4107.
- prod compose: the lenient DB healthcheck followed a runc setns crash
loop; pgvector arrived with migration 0067 via a PG16->17 dump/restore.
Corrections to match the code: README names the Settings tab
"MCP Access"; ci-requirements says the Dockerfile builds on node:22
and lint covers scripts/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings -> Profile (name, role, expertise, response style, tone, interests,
working hours) is typed by its user and neither derived nor a credential, yet
_NOT_INCLUDED named it, so a restore dropped it without a word.
BACKUP_VERSION 26. Both exports carry it; the restore remaps only the user, and
a blank field stays blank rather than restoring as the page's default. The
column and import guards cover it, and the tar round-trip restores one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The column becomes token_sealed (migration 0122 seals existing rows with a
frozen copy of the v1 key), every reader opens it through connection_token,
a row this instance cannot open is skipped with a warning, and Settings asks
for that token again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The restore route takes the tar as a raw body beside the JSON download.
scribe/restore.py reads either format, refuses a database that already
has an account (a restore re-creates everything with new ids, so into a
live instance it would duplicate rather than replace), has no size
limit, and says to restart so search indexes rebuild. An integration
test restores a written tar through the real restore_full_backup and
checks every id is remapped. docs/backups.md explains the file, why it
is uncompressed, both destinations, Fabled Archives as the worked
example, and restoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BackupScheduleSettings.vue leads with whether the data is backed up
(last good backup, or Overdue), then what is wrong (last failure in
words, an S3 secret this SECRET_KEY cannot open, incomplete config),
then the form: daily toggle and UTC hour, folder or S3 bucket, Test
destination with unsaved values, Back up now (polls while running,
re-armed in finally), and the last runs. The secret is write-only.
formatBytes moves to utils/bytes.ts for both views. Restore from
Backup now also takes the .tar, and says to use the CLI on a 413.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/backup_job.py reads its config when it fires, so startup reads
nothing (#4181) and a changed hour needs no reschedule. A run is held to
a 60-minute deadline (a hung NFS mount would otherwise hold the lock and
skip every later backup), never runs twice at once, and keeps its outcome
- file, size, duration, or the reason in words - in the bounded
backup_runs setting, which no backup carries. summarise() gives the page
its verdict: overdue when on and no success for 26h.
Admin API: GET/PUT /api/admin/backups, POST /backups/check (tests unsaved
form values, 45s deadline), POST /backups/run (202, background).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/backup_s3.py is FabledSteward's SigV4 client ported (put + list,
never delete, so an append-only key is enough), with its tests against
AWS's published signatures. services/backup_destination.py holds the
admin config (backup_config JSON) and delivers a dated tar to a mounted
folder or a bucket, and checks either without sending a backup.
services/secrets.py seals the S3 secret with Fernet under a key derived
from SECRET_KEY; backup._SETTINGS_NOT_EXPORTED keeps it out of every
backup. cryptography becomes a direct dependency (already in the image
through pyjwt).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
export_full_backup orders every query by primary key: an UPDATE moves a
row to the end of the heap, so an unordered export shifted every byte
after an edited row. services/backup_file.py writes the payload as one
dated tar per run - manifest.json plus <section>.jsonl, sorted keys,
fixed member headers - atomically via a dot-prefixed .part file, and
reads it back into the dict restore_full_backup takes, refusing a
missing or short section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings > General > Reply shapes. One menu row per shape (core, completion,
asks, plan): its title and when it arrives, how many of the operator's
preferences adjust it, and its deliveries over 30 days (in full / as a
reminder, "—" when the counts cannot be read). A verdict line leads. Opening a
row shows the shipped text, read-only because it is product, and the
preferences mounted on its moment; each opens in the rule editor, and "Adjust
this" opens a new preference already mounted on the shape's moment with a
starting trigger.
The rule editor takes an optional preset (kind, moments, trigger) and, opened
outside the Rules view, asks where a new record lives instead of dropping it.
GET /api/retrieval/reply-shapes now returns reply_shapes.overview: the
catalog plus mounted preferences (the same lookup that delivers them) and
counts from the reply_shape delivery rows. #5497 (step 5 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reply shapes are server product now, delivered at their moments, so the
skill stops restating them. Gone from it: the "Every reply" list, the
per-kind tables, and "The operator's own shapes come first" (preferences
arrive beside the shape on the same moments). It opens by saying where the
shapes come from (list_reply_shapes, the delivered core's header) and keeps
the reasoning: sections chosen not filled, a settled decision acted on,
placement from the record, who decides what, the assumptions an option
carries, the completion report worked in full, and the second pass. 13.5k to
10.7k characters.
The core gains the Finding kind the skill's table carried (2,150 of 2,200).
Tests follow the content: the kind and Approval-row pins move to the shapes,
a new test holds the worked example and the completion shape to the same
sections, and the guidance-ownership registry reads the delivered shapes as a
surface, owning the preference-wins and length topics there. using-scribe
points at the delivery and list_reply_shapes. #5496.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
report_preference searched one constant string when a task closed and handed
matches back as update_task's reply_preferences. Milestone 500 step 3 delivers
the reply shapes and the preferences mounted beside them on the moments, so
the arm goes, whole (rule 22):
- services/reply_preferences.py, the REPORT_PREFERENCE RuleArm, its re-scorer
and corpus entries, and the REPORTPREF constants
- update_task's reply_preferences block and its cue; the docstring now points
at moment_rules
- the fixed_query field and the fixed_query_never_clears warning: this arm was
the only one that set it, so the concept and the cannot_decline exemption
go with it
- the two Settings fields for its floor and budget
- migration 0121 deletes its rows (logs, judgments, rule-usage events, tuning
history, settings keys), operator-approved 2026-10-09. Without it the
readout would call the source unregistered and its surfacings would count
as ambient. The unindexed rule_usage delete is bounded by created_at.
#5496 (step 4 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The marker scribe_reply_check.sh writes after a completion-section hold sat in
the swept ledger directory as `<sid>.reportcheck`, and the sweep matches
`.ids` - test_every_session_file_in_a_swept_directory_follows_the_convention
caught it (run 799). Renamed `<sid>.reportcheck.ids` and added to the roster.
#5496.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Stop hook sent the finished reply twice: scribe_report_check.sh checked a
task-closing reply for the completion sections in shell and reported to
/report-check, and scribe_reply_check.sh sent the same reply to /reply-rules
for the rule hold. Now the reply goes once. When the turn closed a task the
hook adds the close count and ids, and the server runs the section check
(services/report_check, the same three patterns) beside the reply hold,
folding both into one reason.
The report_check adherence log is still written for every checked reply
(milestone 409's number). A section hold marks the session, so its rewrite is
sent back once with rewrite:true to record how it came out, and is never held.
The block reason carries the completion shape's one line and points at
list_reply_shapes rather than at the skill. #5496 (step 4 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scribe_json_unescape wrote each escaped code point with sprintf("%c", n),
which writes the character only under gawk in a UTF-8 locale; gawk in the C
locale and mawk in any locale write the single byte n % 256. The server
escapes every non-ASCII character, so "·" reached the session as 0xB7, "—"
as 0x14 and an emoji as a NUL wherever a hook ran without a UTF-8 locale.
The decoder now encodes UTF-8 itself under LC_ALL=C. A new test sends what
the server sends from a bare environment, in three locales (#5495).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Every turn (/api/plugin/retrieve, UserPromptSubmit): the core reply shape
leads the payload - in full the first time, as its one-line reminder
after that - followed by whatever is mounted on reply.report, under the
shared rule ledger. Fresh keys come back as shape_keys.
- The ledger is <sid>.shapes.ids in scribe-priorart (scribe_shapes_file /
_seen / _append), so the compaction sweep that clears every .ids ledger
is what brings the full core back after one.
- At a moment (/api/plugin/moment): the slice for that reply - completion
on work.finish, asks on reply.ask, plan on work.plan - ahead of the
mounted rules. reachable_tools now lists tools reaching a shaped moment
even on an install with nothing mounted.
- Scribe's own tools (attach_moment_rules): reply_shape in the response,
in full, since that door has no ledger. enter_project carries the core
for clients with no prompt hook.
- Telemetry: one AppLog row per delivery (plugin / reply_shape), each
shape with full or pointer and the door (turn, hook, mcp).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The operator's line (2026-10-09), reconciling their 2026-09-21 ruling
(#4218, "you are the judge") with "not that Claude should make all
decisions": judging covers the work and its records - facts, findings,
measurements, whether work is done - and direction stays theirs:
priorities, what the thing should be, trade-offs only they can price,
anything they live with, and every act hard to undo or facing outward.
- reply_shapes: a one-line form in the core (every turn), the full text
in the asks slice (where a question gets handed back). Slice budget
1400 -> 1600 for it.
- reporting-back "You are the judge": "the operator reads the decision -
they do not make it" replaced; the line is drawn by who can see the
evidence, not by how hard the call is.
- plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/reply_shapes.py is the single source of the default reply shapes:
the core (every reply, ~1,800 chars against a 2,200 budget) and three
slices - completion on work.finish, asks on reply.ask, plan on work.plan.
Read through list_reply_shapes (MCP, read-only) and GET
/api/retrieval/reply-shapes, one service behind both doors.
Nothing delivers them yet; that is step 3. The skill still carries its
copy until step 4 shrinks it to the long-form reference.
The software-only vocabulary guard moves into tests/helpers.py
(DEV_ONLY, dev_only_hits) rather than becoming a fourth copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner's docker daemon is shared, so `--filter name=<job> | head -n1` can
pick another repo's Postgres when two jobs run side by side (Steward run 8358,
Inkwell run 8653). Scope the lookup to this job's own task prefix and require
exactly one match. Done from Inkwell #5313 with the operator's permission; the
recipe is rule 79.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lessons share one register, so any two score well above unrelated text and
a fixed similarity bar sat inside that band: every no-rule lesson
"resembled" most of the others and the named group grew with the pool.
A neighbour now counts only when it stands above the lesson's own
background (median and MAD of its similarity to every lesson it can reach,
in that register's spread), and a group is a clique: every pair clears the
bar from both sides, so one broad lesson cannot join unrelated ones. The
bar, the minimum background and the fetch sizes are stated as defaults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A multi-pass DRY audit could not answer whether it created duplication:
a pass's own shortening can leave two statements identical to a third,
and no per-pass scan sees a copy that did not exist when it ran. The
Librarian retrospective improvised this measure in a scratchpad; this is
it as a tool the DRY Pass process can name.
6-line windows over significant lines (comments, blanks and bare
punctuation dropped, strings folded), grouped by glob or extension.
Revisions are read through git archive, so nothing is checked out.
Stdlib only, so any project can run a scratch copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dated release log (a note) matched a signed-APK CI snippet at 0.83 on
shared release vocabulary and was opened as a family candidate, then came
back as a precedent. One chunk of a long narrative scores like the shape
it mentions, so a cross-kind score does not mean "the same thing made
twice". The search is filtered to the new record's kind and the loop
checks it too. Code built from an idea written elsewhere still reaches the
engine through the citation trigger and the shape ledger.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each decision stored its reason three times (reason, before.reason, after.reason)
and a promotion carried the whole criteria reasoning in evidence. Precedents
(assess, promote, get_family_adoption, get_family_idea) and get_family_idea's
history now serve FamilyDecision.to_brief(): reason once, before/after as status,
canon_version and platforms, evidence as the cited list. list_family_decisions
and the decision echoed back by a write keep to_dict. The precedent count was
already bounded at five.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1 added family_ideas.topic_id but nothing could set it, so the note<->topic
link the milestone promised existed only as a column. set_family_topic links or
unlinks (0) a topic in a rulebook the caller owns, one idea per topic; the
version does not move. get_family_idea and get_family_adoption list the topic's
rules; list_family_ideas and the Family page name the topic. The model
docstrings no longer claim the topic's rules are platform-scoped in retrieval.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- enter_project carries a `family` key, but only when the project has
something to answer: counts of unassessed, owed and to-recheck answers,
each with the list_family_adoptions call that lists it. It shows on every
entry, never by platform touch: entry is when work is chosen, and an
unanswered idea is otherwise invisible.
- Retrieval: a widened project search (include_global_kinds) now also
reaches the canon ideas on the project's platforms. It also reaches their
references in the project's languages, or all of them when none matches.
An off-platform project gets none, and the plain project filter (the
duplicate gate) is unchanged.
- Closing a task returns `family_owed`, the owed answers filed while it was
open, and the report cue asks for them to be named.
- New plugin skill family-canon (moment work.record) covers when to
evaluate a promotion, answering in order, what counts as a reason, the
precedent reflex and the conflict order. _INSTRUCTIONS, create_note,
create_snippet, classify_shapes and reporting-back point at it.
The plugin version is minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- classify_shapes takes idea_id: the shape is judged against the idea's
reference in its language, or its first.
- A shape classified against a canon idea's reference moves the project's
adoption row (instance -> adopted, variant -> variant). Withdrawing the
shapes that gave an answer returns it to unassessed. This runs on
classify_shapes, the sweep, confirm_proposals and the coverage refresh.
- assess and undo refuse an answer the shapes contradict. A conflict
resolution re-judges the variant shapes on both sides.
- The proposer's family arm offers a canon idea's reference on a shared
platform, in any language. It proposes only when the reference is the top
hit over every readable snippet and scores at least 0.70. The pairs this
was measured on are recorded beside _FAMILY_FLOOR. The proposer is now v6.
- The matrix cell shows the code that answers it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
assess_family_adoption took system_ids for the task it files, which made
the family tool module look like a System-tagging door and tripped the
parity registry. It is not one: the owed task is tagged to the project's
System matching the idea's canonical area, and update_task retags it.
The unit test for each conflict ground passed evidence/conditions twice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- services/family_adoption.py: assess one project against one canon idea by
the four outcomes in order (exempt, variant, adopted, owed). Every outcome
needs a reason and adopted needs evidence. The engine records the precedents
itself: this idea's answers elsewhere, and this project's answers to the
nearest ideas. The same answer given twice records nothing.
- owed files a task in the OWING project, tagged to the System matching the
idea's canonical area, naming the gap and the reference for that project's
language. The task follows the answer: adopted closes it, exempt or variant
cancels it, owed again reopens it. Each move is logged on the task.
- the conflict order is enforced: every ground above the deciding one must
say why it did not decide. The losing side is folded into the idea's note as
a trap, an alternative or a condition branch, the version moves, and both
rows are answered against the revision.
- recheck is derived (row version != idea version). family.revise moves the
version when substance changes. undo covers a project's latest answer too.
- set_family_references names the reference implementations.
- MCP: get/list/assess adoption, resolve_family_conflict, revise_family_idea,
set_family_references. Web: GET /api/family/matrix.
- UI: an adoption matrix on /family (platform filter, cell detail with reason,
recheck and owed-task link) and the same matrix narrowed to one project on
its Family tab. The decision log now reads project-level decisions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the undone decision had no recorded before-state, the synthesized prior copied the canon idea's current applies_when and platforms into a retired row. Before that decision the record was not an idea, so the restored state now has neither. Caught by test_undoing_a_promotion_restores_the_prior_state_and_keeps_judged_rows (run 8281).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent promotes a family idea when all three criteria hold, and no person approves it. The criteria are product text: services/family.py states them, and the promote tool's docstring names every criterion the service enforces.
- Criteria: each one vetoes on its own when its reasoning is blank. Platform terms also needs an applies-when and a platform scope; proven also needs named evidence. A veto keeps the idea a candidate and is logged, so it becomes precedent.
- Precedent: every promotion stores the decisions on the nearest ideas by meaning, plus any the caller names.
- Promotion sets canon, the applicability test and the platform scope, and opens an unassessed ledger row for each member project the promoter can write. Re-promotion moves the version past every version the idea has held.
- Retire and undo: undo reverses only the latest idea-level decision, restores its recorded before-state, and logs itself with the undone decision as its precedent. Settled here: leaving canon closes the unassessed rows but keeps the judged ones, which read as needing a recheck after a re-promotion.
- Triggers open evaluations but never promote:
- a cross-project lineage citation ("matching #N") on a note or task write;
- a same-meaning record in another project on a shared platform, on create, at 0.80 (measured: the known pattern's builds scored 0.79-0.82, an unrelated project's best match 0.65);
- a milestone closing on a platform.
Each fails open and rides the response as family_hint.
- Doors: seven MCP tools, /api/family REST endpoints, and a Family page (nav, /family) showing the criteria, the ideas, and the decision log with undo and retire.
- utils/recordHref.ts holds the one copy of "where a record opens", now shared with LessonDetailView.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 8271 failed in the image build: "[vite:vue] Cannot read properties of undefined (reading 'scoped')" on moments-shared.css. plugin-vue caches one descriptor per src file and answers ?index=N from whichever component registered it last. RuleEditorSlideOver had moments-shared at block 2, while its other two importers have it at block 1. Step 2's new imports changed the transform order and exposed the bug.
- RuleEditorSlideOver: moments-shared is now loaded with an @import instead of a third <style src>.
- rules-shared.css sat at block 0 in the five panes and at block 1 in LessonDetailView, LessonEditorView and RuleEditorSlideOver, which is the same latent hazard. The three move to block 0.
- tests/test_frontend_shared_styles.py: a guard that every <style src> sheet sits at one index, plus a test that the guard can fail. vue-tsc cannot see this hazard, and vite build only runs after every test has passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project's platforms decide which family ideas reach it. This step makes membership answerable from every door:
- services/platforms.py: the global catalog (writes are admin-only and duplicate-gated by slug); pure marker detection; and membership reads and writes. Detection only ADDS, and only where nobody has answered. It never overrides a declared or rejected row and never removes one.
- coverage: the archive scan now carries every path, and the refresh runs detection fail-open.
- inception: a platforms choice (slugs, or null for unanswered). The list is the whole answer: members left out of it become rejected.
- MCP: list_platforms and set_project_platforms; enter_project and get_project carry the project's platforms.
- REST: /api/platforms (admin writes) and /api/projects/<id>/platforms.
- UI: a platforms checklist on the inception card, a Family tab on ProjectView, and a Platforms admin tab in Settings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When one project solves something every project on the same platform will
meet, that solution becomes family canon and every other project on the
platform answers it. This is the storage for that.
- platforms: a global catalog in the canonical_systems shape, seeded with
generic technology names and the file markers step 2's detection reads.
- project_platforms: declared / detected / rejected. A rejected row is kept
so detection cannot re-add what a person said no to.
- family_ideas: a note's family state. No new record type; any note, snippet
or lesson becomes an idea. A canon idea must state when it applies (CHECK).
- family_idea_platforms: the only scope source. A linked rule topic takes its
scope from the idea, so the two cannot disagree.
- family_idea_references: reference implementations, explicit not inferred.
- family_adoptions: one answer per (project, idea). Variant and exempt require
a reason (CHECK). Recheck is derived from the two canon versions, never
stored.
- family_decisions: the append-only log, with a required reason and the
earlier decisions each one followed. The agent decides with no approval
step, so precedent is what keeps its calls consistent.
Backup v25 carries all seven: platforms by slug, precedent ids remapped
through the decision map. Both column guards cover the new tables, and a
real-Postgres test exercises the CHECKs and the restore remaps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ordered straight off a *_embeddings table, the planner walks the HNSW index,
takes ~ef_search (40) nearest chunks across every owner and project, and only
then applies the scope: an in-scope record behind 40 nearer ones the caller
cannot see was silently dropped. #4958 fixed the rule search alone; the note,
milestone and system searches kept the fault.
_scoped_chunks builds the in-scope chunks with their distance; _rank_scoped
materializes them as a CTE and ranks exactly. All four searches go through it,
and the row shape is unchanged, so callers and mocks are untouched.
Tests: a structural guard that every semantic_search_* ranks through
_rank_scoped and orders nothing itself (with a replay of the old shape), a
compiled-SQL check of the MATERIALIZED CTE, and integration crowd tests - 80
nearer out-of-scope records - for notes (another user; the reader own other
project), milestones and systems.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_prompt_rule_hint, build_tool_rule_hint and build_write_path_hint
each ran the rule arm and then the via-lesson step by hand, the first two
passing the query between them through a _via_query key on the payload.
They now call one composer, _rule_moment, and the split helpers
(_prompt_rule_hint, _tool_rule_hint, _add_rules_via_lessons) and the side
channel are deleted. Output shapes are unchanged: the prompt builder
returns no checkpoint key, the tool builder always does, and
shown_rule_ids stays the direct band.
A structural test pins that only _rule_moment runs a rule arm or the
via-lesson step, and that the three builders are its only callers.
plugin_context.py: 2,418 -> 2,361 lines this step; 3,558 when the
milestone began.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each arm spec now carries its tuning pair (Surface, moved verbatim into
retrieval_pipeline) and a Declared block - what the telemetry readout
must know and cannot read off its rows. The two ranked stages that are
not arms (preference_slot, rule_via_lesson) are RankedSource specs, and
the note slots carry their own declaration.
- retrieval_surfaces.SURFACES = the TUNED_ARMS tuning, same order
- retrieval_registry.POINTS ranked rows = one per spec in RANKED; the
lookups, asked, ambient and pull rows stay declared there
- rule_usage.RANKED_SOURCES = RULE_RANKED_SOURCES + moment_rule
Settings keys, defaults, prose and order are unchanged (checked field by
field against HEAD). tests/test_retrieval_specs.py pins the derivation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_pipeline gains the notes half: NoteArm / NoteSlot / NoteMoment /
NoteIO / NoteResult and run_note_arm, which writes once the stages both
notes arms copied: search, withhold this response's own menu (#3739),
fresh/repeat split, the call row before any return (#3497, #3752), the
band, the reserved slots in their order (reuse evicts, lesson extends),
and the surfacing rows. The note renderer (_record_kind, _menu_name,
_menu_passage, the seen pointer, menu_entry) moves with it, and
run_via_lesson_arm takes rule_via_lesson.
Behaviour-preserving, with flags for today's differences: notes still log
BEFORE the band and rules after it (step 7's question). One deliberate
change: a notes arm now fails open like the rule arms, so a failing
search costs its lines and no longer the whole hook response.
The I/O is resolved from plugin_context at call time (_note_io), so the
existing patches keep working. The review re-run reads the AUTO_INJECT
spec instead of restating it, and its guard now compares the two live
searches. The registry declares the pipeline's notes fan-out sites.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 8218: integration failure, Build & push skipped, :dev still package
version 11098 (23:36:29Z, run 8217), and no image tagged 9a76eb0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverted next. Expect: integration red, Build & push skipped, :dev unmoved,
no image tagged with this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The integration job was added after the build gate (c6211e5) and never
joined its needs, so main run 8212 published :latest with integration red.
Both jobs carry the same ref condition, so the gate cannot be satisfied by
a skipped integration run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ordered straight off rule_embeddings, the planner walked the HNSW index,
which returns about hnsw.ef_search (40) nearest chunks across every owner
and project and filters by home only afterwards. A reader's own rule ranked
past the 40th chunk overall was silently dropped - on a shared install,
other users' rules fill those 40. This is the likely cause of the
intermittent test_integration_rule_scope failures, whose axis-vector
fixtures sit far from every real embedding in the graph.
The in-scope chunks now go through a MATERIALIZED CTE, which the index
cannot order, so the ranking over them is exact. A rulebook is hundreds of
chunks; exact is cheap. New test: 80 nearer rules belonging to someone else
no longer hide the reader's one rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
semantic_search_rules fails open, so both failures on run 8209 and main
run 8212 read as set() with no trace. Assert report["searched"] and carry
the swallowed traceback into the failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The open-after-moment signal proposes a mount; nothing proposed taking one
off, so a wrong mount was noise at every occurrence until someone happened
to notice. rule_misfired(rule_id, moment, why, reached_by) records a report
against a MOUNTED pair, counted per distinct day (the MCP door carries no
session id) on a new rule_moment_judgments.misfire column (migration 0119,
backup v24). At three days the response carries a line asking the agent to
offer the operator the fix - reject takes the rule off, unmap_action stops
the action reaching the moment, confirm keeps the mount and stops the
asking - and Settings > Moments lists it as an unmount proposal with the
reasons and the actions that reached it. A re-mount clears the count.
Taught in moments.md, missed-retrieval.md and the reply hold's wording.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Until now only the tool arguments knew moments existed. The guidance
surfaces described rules as reached by resemblance alone:
- using-scribe: a short reflex paragraph and a new reference file,
moments.md. It covers reading "at <moment>, reached by <action>", the
reply held once at reply.report, map_action / unmap_action offered in
one line, and the step 7 proposal line answered with judge_rule_moments.
- writing-records: asks WHEN a rule applies as well as what it is about.
A rule, preference or process about a point in the work gets
moments=[...] as it is written, and the trigger stays as the net.
- missed-retrieval: a missed WHEN is mounted or mapped, not reworded. A
misfire is unmounted or unmapped.
- _INSTRUCTIONS: one clause (list_moments; mount rules about WHEN),
1594 of 1600 chars.
- static context: injected lines include the rules mounted on a moment
that was reached.
- test_guidance_ownership: three owned topics, so the text cannot quietly
drop out.
Plugin minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule written before moments existed is mounted on nothing. Step 7 records,
per (rule, moment), whether it belongs there and who said so:
- rule_moment_judgments (migration 0118, backup v23): suggested / confirmed /
rejected, from a pass, the signal, or an edit. Moment "" is "no moment fits".
- The pass: rules_to_mount lists unjudged rules; propose_rule_moments records
suggestions that mount nothing; rule_moment_proposals and
judge_rule_moments put them to the operator. A confirm mounts, a reject is
kept so the pair is never proposed again. Same service behind REST and a
"Waiting on you" panel in Settings > Moments.
- Edits are judgments: set_rule_moments, the one mount write path, confirms
what was added and rejects what was removed in the same transaction.
- The signal: scribe_moment.sh keeps a per-session acts ledger; when a rule
is opened, scribe_record_opened.sh sends the last three minutes of it to
/api/plugin/rule-opened. The acts resolve through the install's mappings;
work.run and work.change are not evidence. Counted per distinct session
with lesson_rules' evidence model, and once due the open returns one line
asking the reader to offer the mount.
- scribe_session_end.sh removes the session's scribe-moment files.
Plugin 2026.10.05.2003.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Everything an agent can do with moments, a person can now see and change
in the app.
- Rule editor: a moment picker beside the trigger. Catalog moments are
ticked; a named procedure's `skill.<name>` is typed and checked as the
server checks it. `moments` is always sent, so unticking the last moment
unmounts the rule.
- Settings, Moments section (General tab): for each moment, what it
means, the actions that reach it on this install (shipped ones can be
switched off, the install's own removed), how many rules are mounted on
it, and deliveries and agent opens over the window. Below that: named
procedures with mounts, switched-off defaults with Restore, and a form to
add an action.
- retrieval_telemetry.moment_usage: per moment, `delivered`, `rules`,
`opened` (agent pulls after the first delivery there; an upper bound, as
by_source is) and `last_delivered_at`. No ratio, because a mount is a
person's statement, not a ranker's guess. Guarded on its own, and also
reported in retrieval_summary as `moment_usage`.
- rulebooks.mount_counts; mounted_moments now derives from it.
- GET /api/retrieval/moments carries `mounted` and `usage` (?days=).
DELETE /moments/mappings also reads the mapping from query parameters,
since the browser's DELETE sends no body.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Loading a procedure now also reaches the moment it is for. Loading the
reporting procedure is a report; loading the release procedure is a delivery.
- Bundled skills: each SKILL.md declares `metadata: moments:`. The same
declaration ships as Skill defaults (BUNDLED_SKILL_MOMENTS), because the
server never sees the plugin's files. test_skill_moments holds the two
together and pins the plugin name that qualifies the skill.
- Stored processes: `moments` on create_process and update_process, stored
in the note's data and returned by get_process. A `scribe-proc-<slug>`
load resolves its process through the sync manifest at load time. The
moments are not copied into the stub, which would go stale mid-session.
- reachable_tools lists the skill loader whenever anything is mounted, since
a process's moments are known only when it loads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moment_delivery records the mounted half's surfacing under
source=rp.MOMENT_RULE_SOURCE. That is an attribute the registry's
extractor cannot resolve, so the site is now declared with the value it
emits, read from the pipeline's own constant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reply is the one act no tool call marks, and it is where "let me
know if it works" gets said. A new Stop hook (scribe_reply_check.sh)
sends the finished reply to POST /api/plugin/reply-rules, which checks
it twice:
- mounted: every unopened RULE on reply.report, plus reply.ask when the
reply asks a question. Deterministic.
- semantic: the reply's head and tail against every rule's trigger, on a
new ranked surface, reply_rule. It is the backstop for whatever the
earlier arms missed. Its floor is its stop bar (default 0.80, budget
1), with its own Settings dials. The new stop_only stage records
surfacing for the rule that holds and nothing else, because nothing
else reached anyone.
Following the operator's ruling from 456 step 8, a rule that holds blocks
once, in the server's words. The hook blocks only on a reason it was
given, so an unreachable instance never stops a session, and it never
holds the rewrite. The ledger is the act checkpoint's own, so a rule
holds a session once across both doors and the per-session cap counts
both.
The turn reader moved from the report check into scribe_defs.sh
(scribe_turn_facts / scribe_turn_fact), so the two Stop hooks read a
turn the same way. The output was checked identical on a real transcript.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two guards caught 7865313:
- test_mcp_tool_processes reads every coroutine in a tool module's
namespace as a tool, and a name-imported attach_moment_rules looked
like one. All seven modules now call moment_delivery.attach_moment_rules,
and the parity guard accepts the attribute form.
- The session-ledger convention: a file in a swept directory must be a
.ids ledger. The tool-list cache describes the install, not the
context, so it moves to its own directory, scribe-moment, where a
compaction does not sweep it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule mounted on a moment now reaches the session when an act reaches
that moment, with no semantic match involved:
- run_moment_arm on the pipeline: a lookup, not a ranked search. Each
line names the moment and the act that reached it ("at work.deliver,
reached by `git push`"), so a misfire is visible where it lands and
can be unmapped in-session. A repeat is cited, not quoted; fresh
rules are recorded surfaced under source moment_rule with the moment
in detail. No retrieval_logs row, as for the other lookups, so no
latency is persisted for this arm.
- rule_scope: a rule's home clause, moved out of semantic_search_rules
so the moment lookup scopes by the same one.
- rulebooks.rules_on_moments / mounted_moments.
- The plugin door: a catch-all PreToolUse hook (scribe_moment.sh). It
keeps /moment-tools' answer on disk for five minutes, so a call to a
tool that cannot reach a mounted rule sends nothing, and an install
that has mounted nothing sends one request per window. It shares the
rules ledger with the other arms and fails open silently.
- The MCP door: Scribe's own tools named by the shipped mappings carry
moment_rules in their response, so a client without the plugin gets
them too. The hook skips those tools. A guard pins the attach on
every one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rule_moments (migration 0117) records which moments a rule arrives at, by
catalog name, cascading with the rule. rule_detail, the one seam every
rule door already returns through, gains moments beside system_ids: None
leaves the mounts alone, a list replaces them. get_rule and both
list_rules doors read them back, batched per page.
All five MCP rule/preference writes and the three REST ones take moments
and validate them before their create or update. An unknown name is
refused with the catalog listed and leaves no half-made rule behind; a
parity test pins that ordering on every door.
Backup v22 carries the mounts as a join table remapped through the rule
map; a real-Postgres round trip checks they land on the restored rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moment_actions.resolve(tool, input) names every moment a call reaches and
the action that reached it. One call can reach several: kubectl apply is
a run, a deliver and a reach outside the workspace. Command tools match by
how each segment of the line starts, with a word boundary; other tools by
field=value arguments. The MCP server prefix and case are ignored.
56 shipped defaults cover the harness tools, Scribe tools and common
command shapes. moment_mappings (migration 0116) holds what an install
adds and the defaults it switches off. A removal is a stored row, so an
upgrade does not switch the default back on.
Per the operator ruling, corrections happen in the session: map_action
and unmap_action (write tools) return now_reaches so the fix can be
confirmed in the same reply. list_moments now shows each moment's
actions on this install. REST mirrors both doors, recorded as human.
Backup v21 carries the mappings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourteen generic moments of work (session.start, work.start … reply.ask)
plus the skill.<name> family, each with what it means and the kinds of
action that reach it, written for any kind of work rather than software
alone. The catalog is code because every install needs the same mount
points; which actions reach a moment is per-install data (step 2).
require_moment refuses an unknown name with the catalog listed, so a
typo cannot become a mount that never fires. list_moments (read-only)
and GET /api/retrieval/moments hand out the same catalog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
reply_preferences.completion_preferences was the last hand-written
rule search + record_retrieval + record_rule_surfaced triple outside the
pipeline. It is now REPORT_PREFERENCE, a RuleArm with kind="preference"
over COMPLETION_QUERY, read back as records rather than as lines.
- RuleArm gains `kind`. _ranked asks the ranker for that kind and drops
anything else on the way out, before the row is logged.
- RuleResult gains `shown`, the (score, rule) pairs behind the lines, for
callers that return records.
- RuleMoment.project_id may be None: logged as given, searched as
`project_id or None`. report_preference rows keep their NULL project.
- Guards: reply_preferences now has 0 direct rule searches. The registry
constant-source example moves from report_preference to preference_slot,
the pipeline slot that records through a module constant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 8137 had two failures, both source-scanning guards whose property
moved to the pipeline. The property itself still held:
- test_every_surface_name_is_a_real_telemetry_source looked for
source="<name>" literals. The rule arms now record under their spec field,
so the spec is the join key, read from RULE_ARMS.
- test_every_hook_rule_search_says_which_project_it_is_for counted 4 direct
searches in plugin_context. It now expects 0 there, so a copied arm fails
the count. It also asserts that the pipeline has exactly one search and
that its keyword literal carries project_id and never everywhere.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prompt_rule, pre_tool_rule and the rule half of the write path each wrote
the same steps out by hand: search, band, split fresh from repeats, log the
call before any early return, reserve a slot, render, record surfacings.
The copies drifted, and #3497, #3750 and #3752 were fixed one copy at a time.
- New services/retrieval_pipeline.py. run_rule_arm runs the stages in one
order. RuleArm is the spec (source, band, compact_tail, checkpoint,
preference_slot). RuleMoment is the query and the session ledger.
- The I/O (ranker and recorders) is passed in as RuleIO. plugin_context
resolves it from its own names at call time, so existing patch points
still apply.
- _rule_band, _rule_hint_line, checkpoint_for, checkpoint_reason, the band
constant and the preference slot moved into the pipeline unchanged.
plugin_context re-exports them.
- A recorder that raises now costs only its row, never a rendered line.
- The flags reproduce today exactly. Whether prompt_rule should band, and
whether the act arms should reserve a preference, are step 7.
- Registry: the pipeline call sites are declared in FAN_OUT_SITES, with
values read from the specs.
- The #3497 structural guard now checks the one implementation, and that
plugin_context writes no rule-source row of its own.
plugin_context.py: 3,558 -> 2,980 lines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single plugin request fans one query out to several ranked arms. The
operator message is searched by auto_inject, its reuse and lesson slots,
prompt_rule, the preference slot and rule_via_lesson, and each arm called
get_embedding on its own: up to six model calls for one vector.
- embeddings.query_embedding_memo(): a request-scoped ContextVar memo that
get_embedding consults. Outside a scope the default is None, so every
other caller is unchanged. A failed embedding is not remembered.
- memoized_query_embeddings decorates /retrieve, /tool-rules and
/prior-art.
- get_embeddings (document chunks) is untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4797 "A lesson's what takes a whole narrative, and the narrative becomes
its name". `what` is the title every listing and menu prints. Nothing
enforced its documented "one line", so lessons written with the incident
in `what` printed up to ~1,500 characters as a menu line, buried the
claim, and diluted the trigger in the embedded title. That was 14 of the
39 lessons on this install.
- lessons.require_claim refuses a `what` over WHAT_MAX_CHARS (240) or
running over several lines. The refusal says the story goes in
`insight`.
- Both doors call it before writing:
- MCP create_lesson / update_lesson;
- REST create / update.
An update checks only a NEW name, so a lesson stored with a long one can
still take the edit that repairs it.
- lessons.claim_line is the display half. _menu_name shows an over-long
stored name as its first sentence marked " …", because a door guard
does not undo rows already stored and an unrepaired install would keep
printing them.
- The tool docstrings state the bound.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4798 "The rules corpus's surfaced_never_pulled warning sends the reader to
menus_to_review, which can only review auto_inject". #4772 gave the warning
one remedy for both corpora: "judge a sample with menus_to_review". That is
right for notes, where a line carries its passage. For rules it is a dead
end: retrieval_review.REVIEWABLE holds only auto_inject, so the tool
refuses every rule arm.
- The reading is now per corpus (_NEVER_PULLED_READING). For rules, the
text says:
- the count includes rules that only arrived in a listing;
- a rule can rightly be set aside on its trigger alone;
- no judged sample exists for the rule arms;
- a rule set aside again and again is a trigger to fix (update_rule
when_to_apply, then what_might_apply), not a floor.
- The tool docstring says the same.
- The guard is tied to REVIEWABLE, so it can fail in both directions: if
the rules text names menus_to_review while no rule arm is reviewable, or
if a rule arm becomes reviewable and the text still says there is no
sample.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4796 "A record the operator names by number reaches auto-inject only if
its wording happens to match". "yes go ahead with 4448" says which record
is meant, but a number means nothing to an embedding, so the prompt menu
filled with records resembling the words around it.
- record_refs.named_record_ids reads the operator's raw prompt, never the
reply-enriched query:
- `#N`, unless the word before it marks another numbering (PR, CI, rule,
milestone, system, log...);
- a bare number of 3 or more digits that opens the message, follows a
reference word or continues a list one started;
- never a quantity ("300 seconds"), a date, version, path or fenced code.
- Each id is resolved through the ACL check; trashed or inaccessible ids
are dropped.
- A "Named in your message" block leads the menu: the kind, the System,
the name and the opening of the body, or the seen pointer if the record
is already on the ledger. It takes no share of top_k, and the semantic
lines leave those ids out.
- The block is booked under the new `named_ref` source, registered as an
unbidden lookup that is allowed to be quiet. A named id that also ranked
counts as suppressed in the auto_inject row, so #3668's identity holds.
- Named records now arrive even when the search finds nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SDK 2.x passes on only a ToolError's text. Any other exception becomes
UnexpectedToolError("Error executing tool X") and its message stays on the
server. ValueError is how every Scribe tool refuses — what was refused, why,
what to do instead — so every refusal reached the agent bare, and it could
only retry blind. Seen live: tune_retrieval(actor="operator") and
judge_menu(verdicts=[]) both answered "Error executing tool …" and nothing
else.
StrictArgsMCPServer.call_tool re-raises an UnexpectedToolError caused by a
ValueError as a ToolError carrying the message, in the SDK's own
"Error executing tool X: <reason>" shape. Other exceptions are crashes and
stay masked. The stale comment claiming the SDK returns ValueError text is
corrected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`<agent-message …>` turns were scored by auto-inject and the prompt rule
arm as if the operator had typed them, spending a retrieval and writing a
retrieval_logs row that reads as a real message (log 54355, found by the
#4772 review). Same class as #4142. Matched up to the tag name, since the
tag carries attributes; `<agent-messages>` and the tag named mid-sentence
are still retrieved against.
Plugin 2026.10.04.0125.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Judging live auto-inject menus (#4772) found "## Work log — 2026-08-21",
"## Findings worth carrying forward" and "Two dev→main PRs this session."
each embedded as a whole chunk. Text about nothing embeds close to
everything, so they took ranks 1–3 on vague queries and pushed real records
down.
Three ways the chunker made them, each closed:
- _split_paragraphs flushed a heading on its own when the paragraph under it
was long. A thin lead is now carried into the paragraph that follows, and
a hard cut never lands in the first quarter of the budget, where the
newline it finds closes a heading.
- A heading with no body (above its ### parts) became a section of its own
when the section before it was full. A thin section now takes the next.
- A short lead-in or a short last section stood alone. The lead-in joins
what follows; a thin tail joins what precedes it.
A continuation piece now repeats the LAST heading before it, since one
section can now hold several. CHUNKER_VERSION 2 → 3, so the startup backfill
re-embeds; tuned dials will report their shape as changed, which is true.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first live sample ranked records the call could never have been offered:
the session that made a call writes the decision, often quoting the message,
and that record tops the re-run. Judged, it inflates on_point exactly where
the budget is decided.
- _rerun over-fetches by POSTDATED_SLACK, drops records created after the
call before ranking, and names them in `postdated`.
- each line carries `changed_since_call` (updated_at or a work log after
the call) and `logged` (shown fresh then).
- judge_menu shares the re-run, so a post-dated record cannot be judged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An open rate cannot say whether a menu line related: every line carries its
matched passage (#4364), so "not opened" covers unrelated, enough as shown,
and already in context. #4772 "Injected notes are never judged".
- retrieval_judgments (0115): a reviewer verdict per line of a logged call,
on_point / adjacent / unrelated, with its reason, rank, budget side and
whether the agent opened it within the hour.
- menus_to_review re-runs a random sample of unjudged auto_inject calls with
the arm's own parameters, past its budget, passage on every line.
judge_menu records verdicts, re-deriving rank from a fresh re-run.
- retrieval_telemetry gains a judged block (by rank, within/beyond budget,
on_point_unopened). surfaced_never_pulled stops blaming titles.
- missed-retrieval guidance names the review before a budget move.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4769 "Rulings are counted where someone will read them": milestone 444
step 4 wrote system_usage_events and nothing read it.
- retrieval_telemetry gains a `system_usage` block: surfacings and opens by
source, distinct counts, and `by_system` naming the areas most shown.
There is deliberately no pull-through ratio, because rulings travel in full
in the line and opens are the exception.
- usage_for_systems (one GROUP BY) adds `usage` to the REST Systems list and
detail, and to MCP get_system. MCP list_systems is unchanged.
- The Systems UI shows a "rulings shown N×" chip.
- rulings_pre_tool, rulings_write_path and mcp_get_system are now declared
registry points; the registry guard covers their recorders.
- The Systems store merges a PATCH reply instead of replacing the row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4768 "Write-path prior-art route 500s when a nameless search hit carries
its excerpt under `snippet`": _prior_art_line read item["snippet"] as a
snippet record's field dict, but a search hit carries its matched passage
there as a string. Read the name only from a dict.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI 7939 went red on test_the_hook_and_the_route_agree_on_every_parameter_name:
the Bash hook now sends root and cwd, and the guard pins the exact set. Both
guards (tool-rules and prior-art) now also pin seen_ruling_systems, spelled
once in scribe_rulings_query, and check the routes read all three.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A System's rulings (the Rulings section of its description) now reach the
work by path, not by similarity. Both PreToolUse arms resolve the files a
command or edit names to the Systems whose path_patterns cover them, and the
first touch in a session shows each area's rulings in one line; a repeat is
a one-line reference. A lookup, so no floor, no budget, no retrieval_logs row.
- services/system_rulings: parse_rulings, command_paths (reads and writes,
relative to the repo root from any cwd; flags, URLs, globs skipped),
rulings_for_paths
- /tool-rules takes root, cwd and seen_ruling_systems; /prior-art takes
seen_ruling_systems; both return ruling_system_ids
- hooks share <sid>.rulings.ids (cleared on compaction by the ledger naming
convention); the Bash hook sends the repo root and cwd
- system_usage_events (migration 0114): surfacings by source, pulls from
get_system; carried by backup (v20) through the system map
- writing-records: rulings also arrive when the area's files are touched
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A System gains path_patterns: globs relative to the repo root (* within one
directory, ** across any depth, a plain directory covering everything under
it). One service validates them for every door, so the web UI and MCP refuse
the same bad pattern with the same message. systems_for_paths resolves paths
to every active System that covers them, which step 4 (#4757) uses to deliver
an area's rulings when its files are touched.
- schema: systems.path_patterns JSONB NOT NULL default [] (migration 0113)
- service: normalize_path_patterns, path_matches, systems_for_paths
- routes + MCP create_system/update_system accept it; [] clears
- web UI: a Files field in the create and edit forms, patterns on the card
- backup carries it through export and restore
- using-scribe reflex 7: tagging work keeps a System's files current
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Librarian session contradicted a decision the operator had made 13 days
earlier. The ruling ("retry, then replace, never give up on a book") was
kept only as a quote in a work log, beside a session's reading of it that
capped replacements at 3. Three later sessions built on the reading, and one
carried the cap into an option as a "known cost", which the operator then
approved without being asked about it.
- writing-records.md: "A ruling goes on the System it governs". What a
ruling is (the operator decided it; a later change could undo it), how it
differs from a rule, and where it goes: a Rulings section at the end of
the System description, one line each with who, when and the source
record. Written the turn the operator decides; holds what is in force,
not history; a charter line that contradicts a ruling is fixed in the
same edit.
- using-scribe SKILL.md: reflex 1 says code tells you what a thing does,
not what was wanted, and a limit read from code is unconfirmed until a
System's Rulings says otherwise. Reflex 9 points to the ruling section.
- reporting-back: an option that carries existing behaviour says whose call
it was (the operator's ruling, or a past session's never confirmed); one
that contradicts a ruling is a Conflict.
- create_system / update_system docstrings: the Rulings section, and that
description replaces the whole text.
- test_guidance_ownership: three topics pinned to their owners.
- Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lesson editor now asks the question the create tool asks, while the
writer still has the situation in mind. It offers three answers:
- an instance of a rule, ticked from the rules the lesson resembles;
- "No rule fits", with a required reason;
- leave it open.
The rules are fetched before the save from a new GET
/api/lessons/rule-candidates. It runs the same rule_candidates search the
create door replies with, and passes "search unavailable" through as null,
apart from "nothing resembles it".
An edit sends an answer only when it changed. Re-sending the same rules
would re-stamp their judgments. Worse, a shared editor who cannot see the
owner's rule would reject it by sending a list without it. Leaving a linked
lesson open sends rule_ids=[], which is what clears the links.
"Leave it open" is not offered once "no rule fits" is on file: the server
has no way to take that answer back except by naming a rule. When "no rule
fits" completes a convergence group, the editor says so in a toast, since
the page it goes to does not recompute it.
Guards check that the payload uses the names both routes read, that the
three answers are offered, that the candidates route sits above the id route
and keeps null apart from [], and that the editor reuses .rule-chip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lesson page gains an "Instance of" panel that holds one of three answers.
Unjudged is the fall-through, so it is stated rather than left blank:
- the rule(s) it was judged an instance of;
- "No rule — <why>";
- "Not yet judged".
Suggested links show what they rest on (distinct situations, and projects
when more than one), with Confirm / Not an instance for a reader who can
write. Rejected links stay listed with their reason.
The rule slide-over lists the lessons that are instances of it, plus the
suggestions waiting on a judgment. Each entry links through to the other
record, and kind and state wear the existing .rule-chip.
The write check moves to utils/permission.ts. The copy on the snippet page
looked for "edit", which the server never sends, so shared editors saw a
read-only page (#4640 "The snippet page hid its edit controls from shared
editors"). Guards pin the client's unions and write levels to the
service's, the model's and access.py's own values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a lesson is answered "no rule fits" (create_lesson / update_lesson on
both doors), the response looks for other no-rule lessons it resembles and,
once there are CONVERGENCE_LESSONS (3) of them, carries `convergence`: the
members, their incidents and projects, and a hint to draft the missing rule
with create_rule (operator approval as always) and point each lesson at it —
or to leave them as lessons when no single choice is right every time.
- convergence_group is the pure bar: distinct LESSONS count, incidents never
stand in for them (one broad lesson cannot trigger it), and a group whose
sources all point at one incident is one event written up several times.
- convergence_for searches lessons by the new one's claim + trigger
(trigger_title) at CONVERGENCE_THRESHOLD 0.65 — above the menu's "worth
showing", below the duplicate gate's "same record" — then keeps the ones
with a lesson_no_rule answer. Fail-open. No sweep, no timer (#4183).
- Defaults stated as defaults (rules 32, 115).
- Tests: the bar (pure), the search with stubs, the door, and the no-rule
filter against Postgres; conftest stubs convergence_for for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 712 failed test_neither_rule_arm_logs_its_call_behind_a_results_guard
with "min() iterable argument is empty": the guard walked the function named
build_tool_rule_hint, which since fd2ebf4 is a wrapper that searches nothing
— the arm's body moved to _tool_rule_hint. The property it guards (the call
is logged before any early return) still held; the guard had pinned a name.
It now finds the async function that calls semantic_search_rules under the
pre_tool_rule source, so a later rename moves the guard with the arm instead
of emptying it (rule 167).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On all three rule arms (prompt_rule, pre_tool_rule, write_path_rule), after
the direct match, a lesson matching the moment at the notes menu's own bar
brings the rule(s) it is CONFIRMED to be an instance of, rendered in rule
voice with "Reached through lesson #N “…”, a recorded instance of it."
- Confirmed links only (lesson_rules.confirmed_lessons /
confirmed_rules_in_scope). A suggested link carrying its rule would
manufacture the co-arrival #4637 counts and prove itself.
- Scope kept: a lesson surfaces everywhere, but the rule it brings must be
global or this project's own — milestone 414's boundary, not reopened
through a side door.
- Suppression is by RULE: anything the direct band named or the session
ledger holds is skipped, whichever lesson reached it.
- Its own slot (VIA_LESSON_LIMIT = 1), not a rule slot — a stated default,
since the plan's "decide by measurement" has nothing to measure until
links are confirmed (#4632). Its own source, rule_via_lesson: registered,
RANKED for pull-through, logged whenever it searches.
- Nothing searched at all while no lesson carries a confirmed link, which is
every install until one is judged.
- build_prompt_rule_hint / build_tool_rule_hint are now thin wrappers over
the direct arms (_prompt_rule_hint / _tool_rule_hint); the arm leaves its
query in `_via_query` only when it ran, so a disabled arm or a blank prompt
brings no rule in either, and the key never leaves the server.
- Via-lesson rules are not fed to the #4637 co-surfacing recorder: only
direct matches are evidence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When one hook response puts a lesson and a rule in front of the reader, the
pair is recorded as evidence on a SUGGESTED lesson_rule_links row; once the
pair has arrived together in PROPOSE_SITUATIONS (3) distinct situations, the
next co-arrival carries one line asking the reader to judge it with
judge_lesson_link. Nothing about surfacing changes: a suggested link carries
no rule anywhere (that is #4633, confirmed links only).
- lesson_rules: co_surfaced (fail-open; only pairs the reader could confirm —
a lesson they may write, a rule they own; judged pairs gather nothing; one
proposal per response; PROPOSE_COOLDOWN 6h between asks), plus the pure
counting rules: situation_key, add_evidence, proposal_due, evidence_summary.
A situation is the prompt on /retrieve (word tokens, sorted and
de-duplicated, so trivial rewordings count once) and the FILE on
/prior-art (every edit to one file is one situation).
- rules_for_lessons shows a suggested link's evidence counts.
- plugin_context: build_autoinject_hint returns lesson_ids,
build_prompt_rule_hint returns shown_rule_ids, build_write_path_hint records
its own pair; routes/plugin /retrieve records the prompt pair. Shown lines,
repeats included: relevance makes a co-arrival, not the session ledger.
- Evidence lives in the existing evidence column — no migration, and backup
already carries it.
- Tests: counting rules (unit), the recorder against Postgres (bar, repeat,
judged pairs, ownership, cooldown, evidence kept on confirm); conftest stubs
co_surfaced for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 707 failed test_the_trigger_separator_is_spelled_in_exactly_one_place:
rule_candidates joined the lesson's claim and trigger with an inline " — ",
a second spelling of embeddings.TRIGGER_SEP (#3207). It now calls
trigger_title — the same join a rule's own document title is built with,
which was the point of the query shape in the first place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Which rule is this lesson an instance of?" now has three recorded answers:
a rule named (a confirmed link, #4630), no rule fits (new), or unjudged.
- Model + migration 0112: lesson_no_rule (lesson_id PK, CASCADE from the
note; why; judged_at). A table rather than a key in notes.data, because
that mirror is re-composed from the body on every edit and would erase it.
- Service (lesson_rules): set_no_rule rejects any confirmed link with the
reason; a confirmation (set_lesson_rules or judge_link) deletes the answer;
require_one_answer refuses both answers in one call before any write;
judgments_for_lessons + attach_lesson_rules add rule_judgment (and no_rule)
to every lesson payload; list_unjudged lists the open ones; rule_candidates
searches rules with the lesson's claim + trigger at the explicit-search bar,
None when the search could not run.
- MCP: create_lesson/update_lesson take no_rule; an unanswered create returns
rule_candidates, rule_judgment and a rule_hint; list_lessons(unjudged=true).
- REST: the same on POST/PATCH /api/lessons and GET ?unjudged=1; create
returns rule_candidates.
- Backup v19: a lesson_no_rule section, export (full and per-user) and import.
- Guidance: create_lesson docstring, writing-records.md in using-scribe (owner,
pinned in test_guidance_ownership), create_rule docstring on linking the
lessons a new rule governs. Plugin version minted.
- Tests: door units, integration for the three states, the rejection reason,
scoping, cascade; backup registries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tool-classification guard (test_mcp_auth) failed CI run 705: the new
judge_lesson_link tool sat in no set, so a read key would have been silently
denied it. It changes a link's state, so it belongs in _WRITE_TOOLS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The link between a lesson (one concrete situation) and the rule that governs
it, with the operator's soft-then-hard design built into its state:
suggested while evidence accumulates, confirmed or rejected once judged. Only
confirmed will carry a rule in retrieval (#4633); rejected is kept so the pair
is never proposed again.
- models/lesson_rule_link.py + migration 0111: one row per (lesson, rule),
CASCADE on both ends, indexed both ways, CHECK on state (rule 36), evidence
JSONB and judged_at.
- services/lesson_rules.py: require_rules (validated before any write, so
a bad id leaves nothing half-linked), set_lesson_rules (set-semantics;
a dropped rule becomes rejected, not forgotten), judge_link, and the two
reads. ACL: write on the lesson (share-aware), ownership of the rule; a
reader sees only rules they own. Decorations are fail-open (#4286).
- MCP: create_lesson / update_lesson take rule_ids; get/create/update return
`rules`; new judge_lesson_link tool. REST: the same on /api/lessons plus
PUT /api/lessons/<id>/rules/<rule_id>. Rules: rule_detail carries `lessons`.
- Backup v18: export (full and user-scoped, both ends in scope), builder,
importer; both column guards register the table.
- Tests: integration (states, set-semantics, judge, ACL all-or-nothing,
cascade both ways, CHECK, one row per pair); unit (door wiring, judge
registered, migration/model state agreement, backup skip and unjudged
stays unjudged). conftest stubs the decorations for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone 385 step 8 (#3735 "a lesson is recalled on a project it was not
written on") is defined by opened-on-another-project. The reader's project has
been recorded on every usage event since 0c8e109, but nothing compared it with
the record's own, so the criterion was still unreadable.
- note_usage.usage_for_notes: a second aggregate in the same session joins
notes and counts surfaced_away_count / pulled_away_count. Counted only where
both projects are known and differ; ranked surfacings only (#2477). The
first aggregate is untouched, so events on deleted notes still count.
- empty_usage carries both keys zero-filled; every door that attaches usage
(list_lessons, get_lesson, snippets, knowledge) gets them through
attach_usage.
- UsageBadge tooltip says "On other projects: surfaced N×, opened M×" when it
happened, and nothing when it did not.
- Tests: the mocked split test feeds both aggregates; a unit test for the
away counters; a real-Postgres test that home, unreported and ambient
events are all left out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- reusing-code: "Before the turn ends — say what you built" — the four
verdicts, the one classify_shapes(repo=…) call, and the component file as a
shape. Description names the end-of-turn moment.
- shape-accounting: the writer judges; audits are the check that it held.
The write path SUGGESTS (no more hook instances); component file rows and
whole-file canon described; scoped covers Svelte too.
- _INSTRUCTIONS reuse line: "before the turn ends, say what you built
(create_snippet the reusable, classify_shapes the rest)" — 1570/1600.
- Coverage line: "written-shape check (7d): N turns checked, M asked, K left
unjudged", from the Stop hook's recorded outcomes; silent until the
question has been put.
- test_guidance_ownership pins the new topic on reusing-code.
- prior-art hook header no longer says it stamps instance rows.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single-file component defines `.card`, `.title` and a `Props`, never
anything named after itself — so the ledger could not hold "StatusChip is
canon" and could not say a new card was built where one already existed.
- coverage.is_file_unit: a file that RENDERS (its markup names classes, or it
opens a <script>/<template>/<style> block) and defines nothing named after
its stem gets one `file` row. Structural, not a framework list: Svelte and
Vue components qualify, a TSX component already has its function's row, a
module is accounted for by its definitions. Emitting every file would have
put every module of every project in the todo at once.
- No body fingerprint on file rows, so an edit never re-asks a judgment.
- `file` is its own form and family; derive grouping skips it (`index`,
`+page` repeat by convention). Divergence buckets it on its own, so a new
component where a component canon dominates is a fair question.
- mark_canonicals: a snippet recorded at a path with no symbol makes that
file row canonical; it still covers no definition inside.
- The write hooks apply the same test before noting a new file for the
end-of-turn question.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hook used to land its evidence as `instance` rows, classified_by=hook —
a permanent verdict nobody read, and the source of the weak stamps that
poisoned two ledgers (#4608). Now the agent that wrote the code judges it at
the end of the turn, and the hook's evidence is what it is shown.
- shape_ledger.suggest_write_path_instances (was stamp_write_path_instances):
same evidence and form gate, but it writes a PROPOSAL — proposed_snippet_id,
basis "reference" (named) or "semantic" (resembles), score — and never a
status. A judged row is left alone; a new shape gets an unclassified
provisional row so the suggestion reaches the end-of-turn question as
"looks like #N". By-name uses edges stay: a fact, not a verdict.
- The prior-art line says "looks like #N … when you judge this turn's
shapes, say whether it is", and the result key is `suggested`.
- write_time_divergence takes `suggested`; _RESEMBLE_MIN re-documented as the
suggestion bar and the weak-stamp line.
- reusing-code skill no longer says the pull stamps an instance.
- Tests moved to the new contract, plus a guard that nothing the hook does
writes classified_by="hook".
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recording used to be decided by machinery — the only "record it" prompt
fired when a same-named copy already existed (#2664), so a first instance of
a reusable piece was never asked about, and judgment arrived only through
audits. Now the question is asked where the knowledge is: the end of the
turn that wrote the code, of the agent that wrote it.
- Write hooks keep `<sid>.written.ids` (path, kind, name) for every
definition a write names; a new file adds a `file` line for its stem — a
candidate in any language without a framework rule (scribe_written_append).
- Stop hook scribe_shape_check.sh sends the ledger to GET
/api/plugin/shape-check and blocks once, in the server's words, when
anything is unjudged. Same discipline as the report check: never twice,
never without a recorded check, another hook's loop left alone; the ledger
is kept when the instance cannot be reached.
- shape_ledger.unjudged_shapes: no row, unclassified, scoped and hook stamps
are unjudged; an agent/audit/import verdict is not. A snippet recorded at
the shape answers for it until the refresh stamps it canonical.
- services/shape_check owns the reason text and records every outcome in
app_logs (passed / blocked / judged_after_block / left_after_block).
- classify_shapes(repo=…) judges a shape the ledger has not synced yet via a
provisional row under a bound repo; the sync confirms it, or vanishes and
revives it with the verdict intact. An unbound repo is refused.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Librarian and Stash had 132 and 488 write-path stamps written at 0.68-0.72,
before the 0.80 floor (#4204). Nothing re-judged them, and dominant_canon
counted them, so a YAML CI snippet (#3410) "dominated" Librarian's web
directory and every write there was told it diverged from it — 1408 flags
on one project, 902 on the other, all skimmed past.
- is_weak_stamp: one predicate for "hook-stamped below today's floor".
dominant_canon and canon_form skip such rows; stamps_to_review lists them
by the same predicate. No stored row changes — they stay for judgment.
- flag_divergence withdraws a standing flag whose canon no longer dominates
its directory. A flag is a mechanical prompt, not a judgment.
- The coverage line ends "N weak stamps · M incoherent canons to judge —
stamps_to_review" when either is nonzero, so a project's own session sees
the queue on arrival. The shape-accounting skill says what to do with it.
- scoped_definitions handles .svelte: every <style> is component-scoped
unless <style global> or :global(...); instance-script syms are scoped,
module-script ones stay ordinary.
Plugin version minted for the skill change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shape ledger's divergence readout flagged _gate_setting (#4385).
Reading it turned up a family with no canon: floor_for,
get_duplicate_threshold, get_plan_match_threshold and _gate_setting each
parsed a stored string, fell back to the default (never 0) and clamped
into [lo, 1].
- services/settings.bounded_float(raw, default, lo=0, hi=1) is the pure
parse, fallback and clamp. Each caller keeps its own get_setting read,
so tests patching get_setting per module still take effect, and
_gate_setting still fails open on an unreadable setting.
- SettingsView.saveKbInject: eleven inline Math.min/Math.max clamps and
the local gateAt become one asBar(v, d, lo = 0), mirroring the server.
No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Anthropic's skill guidance: keep SKILL.md under 500 lines, split into
reference files linked one level deep as it nears that. using-scribe was
478 and every new practice lands there.
- SKILL.md 478 -> 317 lines. It keeps orientation, one copy, the reflexes,
scope, the judge section, UI and the process-skill index, plus a "Read
these when the moment comes" list naming each file with its moment.
- projects.md: binding a non-git directory (.scribe) and project inception.
- writing-records.md: where a new rule goes, lesson growth, and notes that
carry their own check (reflex 10 keeps a pointer).
- missed-retrieval.md: the record-before-dial route, verbatim.
- Text moved, not rewritten, except for the seams and one cross-reference.
Tests:
- tests.helpers.skill_text reads SKILL.md plus its reference files. The
ownership registry, the miss-route and the verification tests use it, so
a topic stays owned by its skill whichever file holds it.
- The force test scans every skill .md on its own, since each file is read
on its own.
- New test_skill_structure: SKILL.md <= 350 lines, every reference file is
linked from SKILL.md, none links another, and one over 100 lines opens
with Contents. Each guard is shown to fail.
The plugin version is minted. That also clears 4fb53b8's red Plugin hooks
lane, which failed only because PACKAGING.md changed without a mint.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Spike #4389 read the spec, Claude's docs and a dozen servers: the field is
for how the tools fit together, and the field runs ~600-1,600 characters.
Ours sat at the 2,048 cap as a keyword index that also carried stance.
- JUDGE, REPORT and MISSED leave the index. They fire mid-work, not at
session start; using-scribe and reporting-back state them in full, and
`placement`/`report_back` cue reporting in-band. No skill text changes.
- The rest is rewritten as plain practices (1,503 chars) and keeps every
session-start marker the ownership registry pins.
- INSTRUCTIONS_BUDGET 2000 -> 1600; the three index markers are dropped
from the registry; the miss-route index test now checks that the index
keeps what_might_apply and stays off the route.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- server.py: `mcp.server.mcpserver.MCPServer`; StrictArgsFastMCP becomes
StrictArgsMCPServer, whose call_tool takes and forwards v2's `context`
and raises ToolError (the SDK logs anything else as an unexpected crash;
the message reaches the caller either way).
- stateless_http and transport_security moved from the constructor to
`streamable_http_app(...)` in mount_mcp, with their reasons.
- Per-request user identity is unchanged: the contextvar set around the
ASGI call reaches the handler on both v2 paths (legacy stateless spawns
from the request task; the 2026-07-28 modern path opens its task group
inside the request).
- pyproject: mcp[cli]>=2.2, no ceiling (installs are --locked). uv.lock
regenerated with --upgrade-package mcp in the ci-python image: only mcp
and its own dependencies moved (106 → 110 packages).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gate_bars(user_id, note_type) resolves the block bar and, for notes and
tasks, the overlap floor from kb_gate_* settings, with the old constants
as defaults. Fail-open on an unreadable value; a block bar clamps at 0.80
and the overlap floor at 0.70 and never above the bar. Five fields in
Settings beside the duplicate-report floors.
CLAIM_LEASE stays a constant, with the reason written at it: a per-user
lease would make one shared task live to one reader and dead to another.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
plugin-vue throws "Cannot read properties of undefined (reading 'scoped')"
when several SFCs share one src stylesheet. vue-tsc passed, so only the
image job caught it; :dev has not rebuilt since 22b7a92.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
create_log filtered on Note.user_id == user_id, a bare owner check, so a
collaborator with write access to a shared task was told it did not exist
— and, since a log now stamps the claim, could never be seen working it.
It now asks can_write_note. Editing and deleting a log still require its
author, which is authorship rather than access.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The note sweep, rule sweep, preference drift and rule history panes each
restated the same row recipe in a scoped block. It now lives once in
assets/sweep-shared.css under `sweep-` names (unprefixed globals would
leak into the unrelated scoped .row-title/.lede/.age/.actions elsewhere),
imported unscoped beside each pane's own scoped remainder.
Moved only what was shared: RuleHistoryPanel takes .sweep-state and keeps
its button row head; PreferenceDrift keeps its tiny-type footnote and
overrides the action row's layout. Lede and state now use the body-sm
token instead of 0.85rem/0.9rem (13px vs 13.6/14.4px) — one value, from
the design system.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Release is the mechanical half: scribe_session_end.sh sends the ending
session's id to /api/plugin/release-session, which releases the claims it
held. A tidy-up, not the guarantee (no SessionEnd on a crash; the lease
covers that), and skipped on /clear so SessionStart(clear) can still
hand the claimed work back.
Saying what happened is the half only the model can do. It is stated as a
practice where it is read: the using-scribe skill owns it ("Hand off before
this session's context stops existing", pinned in test_guidance_ownership),
the static context points at it for the wrap-up moment, and add_task_log's
docstring says a log claims the task. _INSTRUCTIONS is untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The claim now pays rent to the session that set it. The SessionStart hook
sends the host's `source` and the session id; the server renders a claim
section by source (task_claims.render_claims):
- compact / clear: the work this session had claimed, each with its two
latest log entries, so a compacted session resumes from the record
rather than from a count of open tasks.
- startup / clear: other sessions' live claims (may still be running) and
in-progress tasks whose claim went quiet (abandoned mid-task).
- fork: the same, framed as "two sessions may now hold this".
- resume, or no source sent: nothing.
The task sidebar shows a live claim as "Being worked" and a dead one on
open work as "Went quiet", so the operator sees a session die mid-task.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
status is durable and nothing clears it, so in_progress cannot also mean
"someone is on this now". The claim is that second meaning, stored as who
and when so it dies on read rather than needing to be cleared.
- notes.claimed_by / claimed_at / claim_touched_at / claim_session (0110).
- The server stamps it on the write that is the work: reaching in_progress
(update or create, via apply_status_transition) and a work log on an open
task. done/cancelled/todo release it. Live while touched within
CLAIM_LEASE (2h); dead on read past it, with no sweep.
- The plugin's PostToolUse hook on update_task/add_task_log binds the
harness's session_id (GET /api/plugin/claim-session). It binds only to a
live claim the caller holds; it cannot create one.
- to_dict carries `claim`. Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_note (create_note and create_records) set the status and nothing it
implies, so create_task(status="in_progress") wrote a started task with no
started_at, and done/cancelled had no completed_at or next recurrence.
The transition is now one function, apply_status_transition, called by
both paths; tests pin the invariant for every status.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The four embedding tables stamped chunker_version but not the model, and
vector(384) is a width, not an identity: a same-width model swap would
write a second geometry beside the first with no error.
- embedding_model on note/rule/milestone/system embeddings (0109; existing
rows stamped with the only model any install has ever run).
- Every write stamps EMBEDDING_MODEL; every backfill's "current" test is
is_current_stamp(), both halves of calibration_stamp().
- migrate_floor refuses while any row its surface searches is off the live
model, before sampling: re-embed, then migrate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on the live corpus, the 74 note pairs at or above the old 0.90 bar
were almost all distinct siblings — consecutive dev-logs, sub-notes of one
design, research parts — and the one clear copy sat at 0.997. The block
refused the next dev-log and taught force=true, as #4134 found for rules.
- The semantic arm blocks notes and tasks only at >= 0.98. The title block
stays; processes keep 0.90 (not measured).
- 0.87 to 0.98 comes back as `overlaps` on the create reply, from the same
per-chunk searches, with a note that leaves the call to the session:
fold in and delete if it is the same record, keep both if a sibling.
- create_note, create_task, create_records and start_planning's steps all
carry it; a batch names the record each overlap belongs to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snippet had no field for prose, so what a session learned about one went
into when_to_use — the trigger joined onto every chunk it is embedded as. A
sweep found write-ups of up to 3 KB there, headings and all.
- notes: stored after the code under `## Notes`, parsed back from the body,
carried by every path that rebuilds it (update, merge, un-merge). A
snippet with no notes composes the body it always did.
- create/update_snippet (MCP) take notes and return trigger_advice when
when_to_use is long, headed or multi-paragraph. Advice, not a refusal.
- Tool docs, the reusing-code skill and the editor hint describe the trigger
as the situation and point the explanation at notes.
- Editor gains a Notes field; the detail view renders it as markdown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The title was `subject — trigger` because the stored title WAS the
embedded one, and the join is what makes these kinds rank on the
situation they apply to (#2485). Every surface that shows a title then
showed the trigger too -- menus, lists and search rows ran to kilobytes.
- embeddings.document_title(title, note_type, data, body) joins the
trigger from `data` (body fallback) at embed time. Idempotent: an
un-migrated composed title comes out the same, never doubled. The
embed path, the startup backfill and the dedup gate's semantic signal
all use it, so the embedded text -- and every vector -- is unchanged.
- Writers store the subject: snippet create/update (service, REST, MCP)
and lesson_document. Both compose_title helpers are removed.
- Readers: dedup takes `data`; the menus strip the embedded title from a
passage; list rows project `when_to_use`, which SnippetListView reads.
- 0108 rewrites existing rows on an exact `' — ' || <own trigger>`
suffix with raw SQL, leaving updated_at alone so the backfill does not
re-embed the corpus for identical vectors. Downgrade recomposes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Injected lines rendered a snippet's or lesson's title, which carries its
whole trigger by construction (the embedding shape) and ran past 1,500
characters -- again on every `seen` repeat. The passage under a line was
cut to 200 chars from the middle, keeping its head (the title again) and
losing where the match was.
Now, on both the prompt menu and the write-path prior-art menu:
- the line shows the record's NAME (snippet data.name / lesson subject),
with its kind and System (`[issue (done) · Plugin & hooks]`);
- the passage is the whole matched chunk, title prefix stripped, on one
line so the blockquote holds; a title-only match hands over the trigger;
- a `seen` record is a one-line pointer to what is already in context.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The trailing newline became a space before the 600-char cut, so the cap
kept 599 characters of text; and under pipefail a grep that matched no
reply made the helper exit 1. Trim before cutting; return 0 explicitly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The context fetch folded a timeout, an HTTP error and a refused key into
one sentence that ended "enter_project() as needed" -- read as optional,
so a session could start with no recent milestones or open tasks and no
way to know what prior work existed. The fetch now names the cause and
the elapsed time, retries once (6s) only where a retry can change the
answer, and the fallback states enter_project as the first step, with
the marker's project id when there is one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prompt hook sent only the operator's message, so a mid-session
follow-up ("yes do that") named nothing a note, lesson or rule could
match. The hook now reads the tail of the last assistant reply from the
transcript and sends it as `ctx`; the notes and rule arms append it to a
short prompt (<= 280 chars), prompt first, capped at 600 chars. No model
tokens: it is embedding input, and the injected menu's budget is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured over 459 judged instance rows against every same-family canon:
true instances score 1.0 or ~0.795 (a model whose mixin list differs from
the canon's), and 0.8 cut the second band entirely. The proposal is read
before it is confirmed, so the floor admits it: 0.75 takes every measured
true row, at 35 wrong-canon pairings instead of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
0.8 was sized for proposals nobody reads; every semantic proposal is read
by a judge before it is confirmed. Measured, it proposed 0 of 150 while
judged instances of a canon score 0.68-0.71 - the write-path hint's own
floor asks the same question of the same documents at 0.68. The arm now
uses that floor, scans 8 hits instead of 3 (a true instance ranked 4th
behind snippets of other language families), and the proposer version
bumps to 5 so rows examined under the old floor are read again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Measured live: true instances of the service-unit canon score 0.68-0.71
against it at best, helpers 0.66-0.75 against unrelated snippets, and
nothing reaches the 0.8 floor. The "conclusive miss" fired for nearly every
body and silenced real divergences exactly as it silenced helpers. The
stored miss basis, its flag withdrawal and the report plumbing are removed;
the floor stays at 0.8 and the new-shapes-first ordering stays. False
prompts are answered by judgment (exempt with a reason).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Measured on the first live refresh after 91cde6c deployed: the version bump
queued 1,533 rows for the semantic arm, the cap read 150 of them in row
order, and all six shapes new since the previous refresh - the only rows
flag_divergence acts on, and the highest ids - were flagged before the arm
reached them. The gate silenced nothing because it never got to look, and
since a flag persists until judged, a later conclusive miss could not take
it back.
- propose_for_repo sorts the semantic todo by _semantic_priority:
unclassified before scoped, newest first.
- flag_divergence withdraws a standing flag when the row now carries a
conclusive miss; that evidence alone withdraws one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A design system binds like a rule but reached a session only through the
session-start block: complete for a session that asks, silent for one
writing a component. The write-path hint now carries a design arm.
A trigger, not a search: a project has one design system, so the question
is answered by the file being UI (.vue, .css, .tsx, ...) in a project that
has one. No vectors, no score, no slot from the ranked menu.
An index, not the prose: resolved guidance runs to ~8,000 chars (the house
style alone), near the hook's whole additionalContext cap. The line names
each inherited layer's section headings and inlines a layer short enough to
be a line - in practice the leaf's departure.
The other arms do not move. A design-only write returns on its own rather
than joining the prior-art guard, so the standing-rule arm still runs only
where it ran before. Once per session per system, on the hook's existing
token-keyed channel (exclude_derive, design:<id>) - no plugin change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
find_duplicate_rule was title-only, on the stated premise that rules are not
a semantic-retrieval surface - false since rules were embedded. A preference
restating a rule under another title passed untouched, and since both kinds
share one ranking, the weaker label could arrive alone.
find_overlapping_rules queries semantic_search_rules with the rule_document
shape, both kinds, in the scope the new record ranks in (global: every rule
the caller owns; project: global + that project). All three MCP create doors
call it before creating and return overlaps + overlap_note on the reply.
It advises rather than blocks, on measurement: across 16 sampled records the
nearest DISTINCT neighbour reached 0.853, while a true rewording scored 0.850.
No threshold separates the bands, so the floor (0.80) sits below the
restatement and the author judges. The stale docstring is corrected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7223 went red on seven tests in test_lesson_usage_surface.py. None of them
caught a regression. Every property they pin still holds; they pinned it at the
old LOCATION: they patched `usage_for_notes` on the door modules and read the
route source for `usage_for_notes(` and `empty_usage()`. 62f3a48 moved all
seven doors onto `note_usage.attach_usage`, so the name is gone from the doors.
This is rule 167's case: a guard that fails on code you believe is correct has
usually moved out from under its property. So the guards are relocated, not
deleted. Each one still has an assertion that can fail:
- MCP list / once-per-page / read-before-pull: patch `usage_for_notes` on
`note_usage`, the module every door now reads it through. The ordering test
still fails if a pull is recorded before the count is read.
- REST list: calls `attach_usage(items)` exactly once, and does NOT hand-roll
`it["usage"] =`, which is how a row could lose its zero-filled key again.
The seam's own zero-fill and single aggregate are asserted in
test_usage_attach_seam.py.
- REST detail: `attach_usage(` still precedes `record_pulled(`.
- Detail view: the advice moved into utils/deadWeight.ts, so assert BOTH that
the view reads DEAD_WEIGHT_ADVICE.lesson AND that the table's lesson entry
still points at `when_to_apply`, so the guard cannot pass by pointing at an
entry that has stopped saying the right thing.
Missed before pushing because I grepped src/ for callers of the old name and
not tests/ for patchers of it. A test that patches a name off a module is a
caller too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`usage_for_notes` is named for notes and works on every note row, yet the chip
reached snippets and rules only. Notes had it nowhere. Lessons had it collected
and shown nowhere a person could reach, because #4196 taught `/api/lessons` to
attach it and `KnowledgeView` — the only lesson list in the UI — browses through
`/api/knowledge`, so `listLessons` still has no consumer.
The cause was not a missing line. SEVEN call sites carried their own copy of the
same few lines: two REST lists, two REST details, two MCP lists, one MCP detail.
Each read perfectly well alone, so "which doors attach usage?" had no answer
anywhere in the code — the same asymmetry test_system_tagging_door_parity.py
records for System tagging (#4249), where whichever door nobody exercised for a
kind is the one that never grew the feature.
`attach_usage(rows, key="id")` is now that answer, and all seven go through it.
A detail payload is a one-row list, so the single-record doors share the seam
rather than keeping a second shape beside it. Deliberately NO try/except: the
fail-open already lives in `usage_for_notes`, which reports through
`_report_failure("readout")` and returns the zero-filled map. Wrapping it again
would swallow the REPORT as well as the error, and a silently-swallowed readout
failure is exactly #2663 — every counter reading zero in production for weeks
while the writes landed fine.
`/api/knowledge` now attaches usage, which closes both holes at once: it is how
notes, lessons and processes are all browsed. `KnowledgeView` renders the badge
on the card footer, looking the advice up per row because the feed is mixed.
The advice moves to utils/deadWeight.ts. Canon #3460 says each caller owns its
own const, and that held while each caller showed ONE kind; a mixed feed would
need five of its own and the next surface another five. The canon's actual
invariant — advice is kind-specific and never baked into the badge — is kept:
it is still a prop. The three existing callers now read the same table, so the
sentence has one home rather than four. Recorded against #3460 so the next
reader is not left re-litigating it.
`_row_id` rejects bools explicitly: `int(True)` is 1, so a row carrying a flag
under the key would be credited with note #1's counts, and a wrong chip is worse
than no chip because it reads as a measurement. A row with no usable id is
skipped rather than failing the page.
Tests pin the PROPERTY, not one route: no door calls the aggregate directly
(AST, so a comment naming it is not a false positive), and every door that shows
usage reaches the seam. Plus the N+1 guard — one aggregate per page, asserted on
await_count, because the per-row version reads more naturally and is invisible
in review.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4204 gave the divergence check a structural gate, which silenced one of the
five false prompts it was filed for. The other four are `def` helpers in a
directory whose canon is an `async def` service unit. No refinement of
`shape_form` reaches them: they differ from #2793's acceptance case — a
hand-rolled sync `confirmDanger` where an async confirm helper is canon —
only by the JOB they do, and a signature does not carry a job.
The proposer's semantic arm already reads bodies per symbol, which is the
comparison option 2 asked for and was thought to be missing. What it did not
do was record its MISSES: a hit became `proposal_basis = "semantic"`, a miss
left the row indistinguishable from one nobody had looked at. So
`flag_divergence` could ask the proposer "do you agree this is the canon?"
but never "did you check, and is it not?".
`_semantic_canon` now reports whether an empty answer is evidence, and
`flag_divergence` withholds the prompt when it is.
The whole risk is in the negative, so only a conclusive miss is stored. A
body too thin to embed, a row the per-refresh cap never reached, an arm that
threw, and a result set that came back full — and may therefore have hidden
the canon behind the limit — all stay "cannot tell" and still ask the
question. That is the discipline `FORM_UNKNOWN` already enforces here: not
knowing must make a check quieter, never more confident. `_SEMANTIC_LIMIT` is
named for that reason; the number is load-bearing, not a tuning knob.
No migration: `proposal_basis` is nullable Text with no CHECK constraint
(verified in the model and across alembic/versions), so rule 36 does not
bite. Nothing can mistake the miss for a proposal either — every reader keys
on `proposed_snippet_id` or `proposal_group`, and `confirm_shape_proposals`
requires the id non-NULL before it will confirm anything.
`_PROPOSER_VERSION` 3 -> 4, per its own contract: rows remember the ruleset
they were examined under, and without the bump no already-examined row would
ever acquire a miss.
Option 1 (widening `kind`) stays closed, on the merits rather than on cost:
bucketing density by exact form takes the async canon out of a sync
candidate's denominator and silences #2793's acceptance case by the identical
mechanism, one layer down. The reasoning is on #4208.
Tests: tests/test_divergence_meaning_gate.py pins the report contract, with
the truncation case tested hardest — reading a cut-off as a negative would
weaken the guard in proportion to how many snippets the operator has. The
end-to-end discrimination is in test_integration_shape_classify.py on
deliberately the SAME fixture as #2793's acceptance case, so the two runs
differ in exactly one thing: whether the arm claims to have looked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Derive-first, ahead of the tests shape pay-down. An AST pass over all 2628
definitions under `tests/` found exactly three helper bodies duplicated
across files. Two are real copies and are consolidated here; the third is
not, and is left alone.
`need_tools(*tools)` — byte-identical in three hook-test modules, each
skipping when `jq`/`awk`/`git` is absent from PATH. Now snippet #4277. The
`import shutil` each file carried existed only to serve it and goes with it.
`rule_row(rule_id)` — byte-identical in two integration modules, reading a
Rule back through a SEPARATE session so the assertion is about what Postgres
holds rather than what the writing session's identity map remembers. Now
snippet #4278. Its imports are lazy, because `tests/helpers.py` is imported
by unit tests that have no database, which is the same reason
`plugin_config` defers its service imports.
NOT consolidated: `_side(uid, k, d="")` in test_services_plugin_context and
test_write_path_trigger. The body is identical but it closes over a
module-local `stored` dict, so it is not self-contained and "moving" it
would mean inventing a parameter neither call site wants. That is convention
plumbing — two tests independently writing the same one-line side_effect —
and it is dismissed in the ledger rather than lifted.
Worth recording for the next pass: a repeated NAME is not a family. `_row`
is defined in five modules and only two of those share a body; the other
three (test_list_rows_brief, test_calibration_stamp, test_shape_ledger)
build entirely different objects. Grouping by name would have consolidated
three things that have nothing in common.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Run 7208 failed 5 of 23 in the new file, all with a `**Learned from:** #1`
that no shape asked for.
`lesson_sources` falls back to `arose_from_id` when neither the mirror nor
the body names a source. `fake_note` leaves that field to MagicMock, which
auto-creates it truthy and coerces to 1 — so every no-sources shape
recomposed with a provenance line and the round trip failed for a reason
with nothing to do with the trigger. `fake_lesson` pins `arose_from_id` to
None and its docstring says why; it is also the truthful stand-in, since
these are lessons.
Worth recording separately: the local harness that cleared this change used
a plain stub class, so it was kinder than the real fixture and could not
reproduce the truthiness. The stub has been given the same None. A
stand-in that is more forgiving than the one the suite uses will verify a
change that CI then rejects — note #2109's point, arriving from the other
direction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The module docstring said the defect grew the worst real body by 520
characters over ten edits. That number was recalled, not measured, and it is
wrong — as was the 2500 I then got by deriving it on paper instead of
running it.
Replayed lesson #4207's actual stored shape through the old code: 607
characters per edit, exactly the length of the trigger's paragraphs 2 and 3
with their separators, taking the body from 1291 to 7361 over ten edits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`update_lesson` is field-wise: an edit that touches only `system_ids`, or
only the title, passes `insight=None`. The service then derives the insight
from the stored body and hands it straight back to `compose_body`, which
re-adds the trigger and the provenance line. The derivation and the
composition have to be inverses, and they were not.
`_strip_composed_lines` filtered out body LINES matching the trigger
marker's pattern. That pattern is `re.M`, so it matches one line.
`when_to_apply` is free text and is routinely several paragraphs —
`compose_body` writes all of it after the marker, so paragraphs 2..n were
ordinary lines the filter had no reason to touch. They came back as part of
"the insight", the whole trigger was prepended again on top of them, and the
record gained (n-1) paragraphs per update. Silent, compounding, and in the
half of the document that does the retrieving: a lesson edited a few times
ends up stating when it applies several times over and burying the insight
below it.
No marker could fix this. There is no end-of-trigger marker in the body, and
adding one would not repair a row already written without it. What makes an
exact strip possible is that the trigger is mirrored in `data[TRIGGER_KEY]`,
so the composed head can be rebuilt byte for byte and removed by subtraction
rather than matched by pattern. Hence `lesson_insight(note)` rather than
`_strip_composed_lines(body)` — it takes the record because the body alone
does not say where the trigger ends. The line-wise removal survives as the
fallback for a row whose mirror is missing, where it is right for a
one-paragraph trigger and is the best available without one; a mirror that
disagrees with the body falls back rather than cutting into the insight.
tests/test_lesson_insight_round_trip.py asserts the property rather than the
strip: `lesson_document(what, trigger(n), insight(n), sources(n)) ==
(n.title, n.body)` across six shapes, and again over ten consecutive no-op
edits. Falsified against the old code before committing: 11 of 23 fail on
HEAD, the headline one reporting a body grown by 3230 characters over ten
edits.
The one damaged row in the corpus (#4207, two paragraphs duplicated by a
`system_ids` edit) was repaired by hand against its authored text. #4221 is
the only other lesson with a multi-paragraph trigger and has never been
updated, so its body is intact — its derived `insight` was wrong, its
storage was not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`set_record_systems` is not a dumb setter. It runs its own `can_write_note`
and then links only the Systems the given user can READ. Four of twenty
call sites handed it the record's owner instead of the acting user, which
did two quiet things at once: the access check became trivially true, since
an owner can always write their own record, and the System filter used the
owner's visibility rather than the actor's.
On a single-user install neither is observable. With a share it is an
editor acting with the owner's reach — the shape rule 47 exists to prevent,
and the same reasoning routes/notes.py already spells out for
`set_supersedes` two lines away.
This is NOT a permission change. Every one of the four sites establishes
the caller's write access first: routes/lessons.py and routes/snippets.py
call `can_write_note(uid, …)`, mcp/tools/processes.py does the same, and
mcp/tools/snippets.py reaches `set_record_systems` only after
`update_snippet` has raised PermissionError if the caller may not write. So
nobody gains or loses the ability to edit anything. What changes is whose
reach the tagging runs with, which is exactly the kind of difference that
survives review because every call site reads fine on its own.
The four:
routes/lessons.py:243 owner_uid -> uid
routes/snippets.py:211 owner_uid -> uid
mcp/tools/snippets.py:486 note.user_id -> uid
mcp/tools/processes.py:196 note.user_id -> uid
The last one was written earlier in this same session, an hour before the
sweep that found it, with a comment confidently explaining why the owner
was correct. That is the argument for the guard rather than for care: the
unified stance was known and still got it wrong at the next opportunity.
So the guard is the point again. `test_every_tagging_write_acts_as_the_caller`
walks every `set_record_systems` call in src/ and asserts the first argument
is a bare local named `uid` or `user_id` — an attribute access is a record's
owner by construction. Verified against `git show HEAD:` as well as the
working tree: clean now, four offenders on the code it replaces.
Reads are deliberately untouched. `list_record_systems(owner_uid, …)` gates
on reading the NOTE, which the caller can do anyway, so it returns the same
list either way; it is a different operation and churning it would add noise
without changing behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A System tag is how `list_system_records` gathers an area's pile, so a
record that cannot be tagged is reachable by search and by nothing else.
`update_lesson` did not take `system_ids` while `create_lesson` did, which
made tagging available exactly once — at the moment of least information.
A lesson is usually written at the end of a piece of work, which is
precisely when that argument gets dropped, and after that the record could
never be filed at all.
Filling in the rest of the table found three more gaps, and the issue's own
generalisation was wrong. It read as "the REST door can do something the
MCP door cannot", on three instances in a row. In fact:
* `create_preference` is the exact MIRROR of the lesson bug — update
takes `system_ids`, create does not. The same capability missing from
the opposite end of the same lifecycle.
* `routes/notes.py` handled `system_ids` NOWHERE, while the MCP door
handled both ends. That runs the opposite way round from the premise.
* `create_process` / `update_process` took it at neither door, though a
process is a note and has always been taggable in the data model.
The real pattern is that whichever door nobody exercised for a kind is the
one that never grew the parameter — which is a better statement of #4248
than the one recorded there, and is not something a reviewer reliably
notices, because each door is only ever read on its own.
Milestones are NOT a fifth gap. `RecordSystem.note_id` is a ForeignKey to
`notes.id` and milestones are their own table, so they cannot be tagged at
any door by construction. Pinned in the test so the next pass does not
re-open it.
So the guard is the point, not the four parameters. `update_lesson` alone
would have left the shape that produced it intact. The new test asserts the
TABLE — every kind taggable anywhere is taggable everywhere it is written —
and keys the registry-coverage check on the SIGNATURE rather than on a
grep, so a module that only names the argument in prose is not swept in and
no hand-kept skip list can go stale. A fifth kind fails there rather than
shipping half-wired, the same reasoning test_derived_mirror_generic_door.py
records for derived mirrors (#3734).
One inconsistency found and deliberately not changed here: for the same
operation `routes/tasks.py` scopes `set_record_systems` by the caller while
`routes/lessons.py` and `routes/snippets.py` scope it by the owner. The new
notes code follows tasks.py and says why in a comment (#47 — an editor-share
holder should tag from what they can see rather than inherit the owner's
reach). Recorded in #4249 rather than fixed as a drive-by.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`band_hugs_floor` compares an arm's weakest tenth against its floor, and
#4225 made it SUSPEND rather than soften when the floor moved inside the
window — because across a change the scores and the bar come from two
different populations. It reads `floor_moves_since`, which answers "did
this arm's floor move", and it read an empty answer as "no, it held
steady".
Those are the same answer only where the ledger was watching. Before an
arm's first floor row there is nothing to move, nothing to report, and no
way to tell a steady floor from an unrecorded one. The suspension was
reading absence of evidence as evidence of absence, and the symptom is the
one #4225 documented: a band "-0.0208 above its floor" — an impossible
negative distance, printed with the suspension silent.
Every install passes through this. The ledger's first row for an arm is
written when that install first boots the release that records baselines,
so any window longer than the install is old reaches back past it. The
lowered-floor direction hides: the gap comes out comfortably positive and
reads as a clean bill of health.
`floor_history_gaps(since)` answers the question its companion cannot:
which arms' floor history does not REACH the start of the window. Arms
come from `surface_names()`, not from the ledger — an arm the ledger has
never heard of is exactly the one at risk, so it cannot be the ledger that
decides which arms get asked about.
Baselines count here, which is the one place the two deliberately
disagree. A baseline records a default without changing it, so it is not a
move and `floor_moves_since` filters it out. It IS the ledger beginning to
observe the arm, and from that moment silence genuinely means the floor
held — filtering it out here would suspend the band check forever on every
install that has never tuned.
`floor_history_unknown` sits between the known move and the band, so an
arm with a date to give gives it. Two sentences, because the remedies
differ: a date says ask again with a smaller `days`; no history at all
says there is nothing to wait for, and names what starts the record.
Why now: #4261 measures the work-log change by reading these warnings, and
every window for the next month opens before this ledger's first row.
Measuring against an instrument that prints a number it cannot support is
the #4225 trap one level up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Every backfill selected the text alongside the id and then iterated that
snapshot, sleeping between records. Over a real corpus that is minutes, and
anything edited inside the window was re-embedded by its own write path and
then OVERWRITTEN with the pre-edit text the scan had captured — stamped at the
current chunker version, so the next boot considered it current and never
repaired it. The record stayed findable by what it used to say, indefinitely,
until someone happened to edit it again. Nothing reported it, and no existing
signal separated it from a correct record: the version was right and the
vectors were there.
All four backfills had it. Three were pre-existing; I wrote the fourth an hour
ago for #4251 by copying the third.
#4251 also widened the exposure. The race used to need a note-body edit; now
every add_task_log re-embeds its task, so any session recording work during a
backfill can hit it — which is exactly what "forward work continues while the
backfill runs" means.
TWO HALVES, SEPARATE ON PURPOSE.
Stopping it: the scan takes IDS ONLY and `_current_row` re-reads each record at
the moment it is embedded. One indexed primary-key read, negligible beside the
forward pass that follows, and it turns a backfill from "replay a snapshot"
into "repair to current truth". The residual race shrinks from the length of
the run to the width of one record, which the parent-row claim and the
single-transaction replacement narrow further.
Repairing it: `_vectors_older_than_their_record` adds a third staleness
condition beside "no vectors" and "old chunker" — vectors older than the text
they encode. A guard that refuses to write a bad value does not undo the bad
values already stored (#4202); the rows are what has to change.
That check earns its place beyond the bug that prompted it. It equally catches
a record whose `embed_note` never ran — no event loop, a swallowed refresh, a
process that died between the commit and the index. Every one of those looks
identical from the version column and identical from outside.
And a work log is part of a task's document while living in its own table, so
writing one never moves `notes.updated_at` and the timestamp comparison cannot
see it. `_tasks_logged_since_embedding` covers exactly the records the race was
most likely to have hit.
The live instance has already run #4251's backfill — the prior-art arm is
surfacing `## Work log —` passages, which is the thing itself rather than a
banner about it. Whether that run clobbered anything is unknown and was not
measured; the repair half picks up whatever it did on the next boot, without
needing it diagnosed first.
The structural guard in the tests was checked against the old shape, not just
the new one: reintroducing `select(Note.id, Note.title, Note.body)` makes it
fail, so it can catch the next copy of this rather than only agreeing with
today's code (#167).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 2 of #4251. A System's `description` is a charter — several hundred words
saying what belongs in that area and what does not — and it is the answer to
"which part of this codebase does X live in". There was no semantic path to
one: `list_systems` enumerates, and `search(system_id=…)` uses a System as a
FILTER over notes. So a System could narrow a search and could never be the
answer to one, and an agent asking where a record belonged had to read every
charter or guess.
ITS OWN SEARCH, not a `content_type` over notes, for the reason note 3163
gives about milestones: the row could be shared, the search cannot. A charter
competing with the whole note corpus for one top-k is outranked by the records
filed under it — the right answer crowded out by its own contents — and "where
does this belong?" is a different question from "what prior art is there?",
which a caller asking one should not have to read past answers to.
So `system_embeddings` (0107) joins note_, rule_ and milestone_embeddings as
the fourth sibling, with `system_document`, `upsert_system_embedding`,
`semantic_search_systems`, a startup backfill and `search(content_type=
"system")`. Scoped like milestones: with a project_id, that project's Systems
if the caller can read the project (rule 78); without one, the caller's own.
Archived Systems are excluded — an archived area is one the operator has said
is no longer where things go, which is exactly the question being asked.
`system_document` is the plainest of the four shapes on purpose. A charter is
already written as the thing this search has to match, in the words someone
asking would use — so there is no trigger to synthesise as `rule_document`
must, and no second record to gather as `task_document` must. The stored
charter IS the sharp document, the way a snippet's is. `color`, `status` and
`order_index` stay out: presentation and bookkeeping, and a vector carrying
them would be answering a question nobody asks of a charter.
The search publishes `report["best_chunk"]` from the start rather than being
retrofitted, which is what #4251 asked of any fourth search. It matters more
here than anywhere: a charter runs long and a result shows its NAME, so a match
on the paragraph that actually decides where a record belongs would otherwise
be previewed by two words that cannot say.
The id that comes back is the one `system_id`, `system_ids` and
`list_system_records` already take, so the answer to "where does this belong?"
is directly usable as "show me what is there" and as "file it here".
`embed_system` sits beside `notes.embed_note` at the service for #2056's
reason — every door gets it by construction. Not called on delete: that is a
soft delete and the search joins through `System`, so the vectors are already
unreachable, and leaving them means a restore is findable again immediately.
`system_embeddings` is declared in backup's `_NOT_INCLUDED` as derived, beside
its three siblings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 1 of #4251. A work log is the richest prose Scribe holds about WHY
something is the way it is — written during the work, recording what was tried
and ruled out. #4241 made it readable from the agent's door. It was still not
findable, so "has anyone tried this approach?" — precisely the question a log
answers — could not reach one. The cost is not hypothetical: #4208 was rebuilt
in this session because its logs were unreachable.
THE DESIGN QUESTION the issue left open was whether logs embed as part of their
task's document or as rows of their own. As their own rows, a hit has to be
resolved back to a task to be worth anything, and it needs a fourth search, a
fourth result shape and a fourth arm. As part of the task, the objection is
that a long log drowns a short title.
That objection was true before #280 and is not true now. Chunking made one
record into one vector per section, so each log becomes its own title-anchored
chunk, scored separately, and the task's own prose keeps the chunk it always
had — a task is as findable as its best-matching log rather than as the average
of everything in it. The other half is this session's other build: a search
hands back the chunk that won (#4243), so a hit earned by a log shows that
log's passage under the task's title. Without that a reader would have got
body[:240] of the task — the opening of a record whose relevance lives three
hundred lines further down.
So `task_document(title, body, logs)` sits beside `rule_document`: a
synthesised embed-time shape, because the stored record is the task row and the
logs live in their own table, so the document that should be searchable exists
nowhere until it is built. A task with no logs is returned untouched — most
notes are not tasks and most tasks carry no log, and their vectors are the
corpus every tuned number here was measured against.
CHUNKER_VERSION 1 → 2, and its comment now says what the version actually
means. It used to read "whenever chunk_document's output can change for the
same input", which this change would slip past: `chunk_document` is untouched
and every task with a log now embeds differently while its title, body and the
chunker all stand still. The invariant is the document a record is embedded as.
The startup backfill re-embeds on that.
Create, edit and delete of a log all refresh the task through `embed_note`, the
one path every writer shares — an edited log whose vectors still carry its old
wording keeps matching what it no longer says.
No new kind enters the auto-inject menu: tasks were always in it, and this
makes recall on them better rather than changing what the menu spans. The
calibration stamp will now report shape_version 2 against numbers measured at
1, which is exactly the report #4104 built it to make.
Also promoted `session_returning` into tests/helpers beside `make_mock_session`
(#2834) — two files had spelled it out identically and a third was about to.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The engine took `note_type` and `task_kind` all along. What was missing was a
way to say them: the MCP tool's `content_type` knew `note`, `task` and `all`,
and `/api/search` knew the same two — so an agent could not ask "has this
snippet already been recorded" or "what lessons apply here" without searching
everything and reading past the rest. Browse offered nine kinds from the same
data.
The cause is that each door kept its own map. `_FACETS` in services/knowledge
is where a kind is declared, and #3161 made adding one a single edit by
generating the SQL filter, the Python predicate and the door's validation from
it — but the two search doors were written before that and never joined. So
this adds the third dialect, `search_filters_for`, and one composition over it,
`content_type_filters`, and both doors now derive instead of listing.
Two names keep a meaning of their own, and the docstrings say so: `all` is no
filter, and `note` is BROAD — any non-task, snippets and lessons included —
where the browse facet of the same name is narrow (`note_type == 'note'`).
They are left different deliberately; narrowing this one would stop returning
snippets to every caller that already asks this way.
An unrecognised kind is now refused rather than answered. Both doors used to
fall through: the MCP tool into a filter matching no row, the route into no
filter at all, so `?content_type=snippets` returned the whole corpus while
looking like a narrowed search. An empty result set is a claim — "the corpus
holds nothing like this" — and an agent acts on that claim by building the
thing it could not find, so a typo must not be able to make it.
The docstring is the agent-facing contract (#2846), and a test now holds it to
the table: every kind `_FACETS` declares has to appear in it, because a filter
an agent has not been told about is unreachable however well it is wired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
test_milestone_search_is_its_own_shape asserts the result dict exactly, so
adding `matched` / `matched_is` / `body_length` broke it — which is the test
doing its job.
Its fake was a bare MagicMock, so `m.body` autovivified: `matched` came out as
a mock object and `body_length` as 0, and nothing in the old assertion touched
either. Lesson #2833 is about exactly this, and the failure output is what
surfaced it. The fake now carries a real body string and a report with the
chunk that matched, so the assertion is about the product rather than about
MagicMock's attribute behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4243 fixed one door. Scribe has three semantic searches over three chunk
tables, and all three collapsed chunk rows to the best one per record — each
of them KNEW which passage earned the hit, and each dropped it. Every surface
downstream then previewed the head of the document instead: a span the search
had already scored lower, with nothing saying so.
Mechanism, one place:
- embeddings.record_best_chunk publishes {id: {index, text}} into `report`.
Carried in `report`, NOT the return value: all three return
list[tuple[float, Record]] and ~30 sites unpack that pair (lesson #4207).
- semantic_search_rules and semantic_search_milestones now select
chunk_index/chunk_text and publish the winner, as notes already did.
semantic_search_milestones gains `report`, which it had no way to take.
- services/text.matched_excerpt is the one choice of span, and
excerpt_fields the one result block. Doors keep their own field names —
the web renders `snippet`, MCP returns `excerpt` — because renaming a
field a frontend reads is a different change from fixing what goes in it.
Surfaces:
- knowledge.query_knowledge, whose own comment calls it "the human's MAIN
search surface", was `(note.body or "")[:200]` on every row alike. Now the
matched passage on a search, the opening on a browse, and `snippet_is`
saying which. KnowledgeView renders that snippet, so this was live.
- search(content_type='milestone') gains `matched` — the plan body stays
out, but the passage that matched comes along, because recognising a plan
means recognising the part you asked about and a description written at
the start need not mention it.
- The auto-inject menu and the write-path prior-art menu put the passage
under their line. Both were title-only, which answers "does this apply?"
for a lesson or snippet (the trigger is IN the title) and not at all for
an issue or dev-log. No fallback to the body's opening: on a menu that is
preamble dressed as a reason, and once indented it cannot be told apart.
Left alone deliberately: the rule arms. A rule hint already renders the rule's
TRIGGER, which is written to answer exactly "does this apply to me" and beats
a matched chunk at it; and that line's budget was measured at #3851. Adding a
passage there would duplicate the trigger and spend the budget twice.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
test_chunking::test_search_collapses_chunk_rows_to_best_chunk_per_note feeds
rows straight into the real semantic_search_notes, so widening the select to
carry chunk_index/chunk_text broke its 2-tuple fakes. I had claimed no unit
test did this after grepping test_embeddings.py, which was the wrong file.
Rows are 4-tuples now, and the test additionally asserts the winning chunk is
REPORTED and not merely used for scoring — the property #4243 exists for, and
this is where it belongs, beside the collapse it comes from.
test_task_work_log_surface's ACL test asserted "task_logs.user_id" was absent
from the compiled statement. It is present in every SELECT as a projected
column; the claim was about the WHERE clause. Asserts on stmt.whereclause now,
which is what was actually meant and still fails if an owner filter returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Raised by the operator: are we limiting what comes back by character count,
and how do we verify the pertinent part is the part displayed?
We were not. mcp/tools/search.py sent (note.body or "")[:240] — a head cut,
with no marker that anything had been removed, so a 240-character preview of
a 4000-character record was indistinguishable from a complete short one.
The opening is the wrong span. The match is semantic and per chunk, and
semantic_search_notes collapses to best-chunk-per-note — its own comment at
the collapse says "the first appearance of a note is its best chunk". So the
system identified the passage that earned the hit and then discarded it:
select(Note, distance) kept no chunk column. A record could rank first on its
sixth paragraph, be previewed by its first, and be judged irrelevant on a
span the search had already scored lower. That biases against long records,
and it is self-concealing — the caller who does not open it never learns the
preview was misleading.
- embeddings: chunk_index/chunk_text ride along in the select, and the
collapse records the winner in report["best_chunk"]. Carried in `report`,
NOT by widening the return tuple: ten callers unpack (score, note) at
~18 sites and nothing would catch the misses (lesson #4207). `report` is
the side-channel this function already uses for best_available_score.
- search(): excerpt / excerpt_is / body_length, and read_full when there is
more. A caller that cannot tell a matched passage from a document opening
cannot judge whether to look deeper, which is the only decision the field
supports.
elide() moves to services/text.py so both callers share one copy, and it
keeps BOTH ends with a stated gap — it is the fallback for when nothing
identifies a better span than "all of it", not the goal.
Also fixes a guard that produced a false failure on the previous commit:
test_pull_telemetry checked `"project_id: int = 0" in body.split("\n")[0]`,
which sees only the first line, so wrapping get_task's signature over four
lines made it report a function that does take the project as one that does
not. Parsed with ast now, and proven to still reject an absent or
wrongly-typed parameter rather than being appeased by reflowing the code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The work log reached the web UI through routes/task_logs.py and nothing
else. get_task returned only the body — a claim written once, before the
work — with the record written during it invisible beside it. So a stale
body arrived with nothing to contradict it, and this session rebuilt work
that had already shipped, with the evidence sitting in the task's own logs.
Read side, scoped through the access layer (rule 78):
- logs_for_task / count_logs_for_task / log_counts_for_tasks in
services/task_logs.py. Scoped by who may read the TASK rather than by
who wrote the entry: list_logs filters TaskLog.user_id == user_id,
which hands a shared collaborator an empty list reading as "no work
has been done". The page query folds readable_notes_clause into the
same statement so the permission does not become an N+1.
- get_task returns work_log; list_tasks and get_milestone steps carry
log_count, zero-filled so "none" is a count and not a missing key.
Elision keeps both ends. The newest entry arrives whole to 4000 chars
because it answers "where does this stand"; older ones are shortened from
the MIDDLE, never the head. A head cut selects what a reader sees by
character position, which is uncorrelated with what matters — an entry
closing with "so this shipped in 04775c3" loses the one sentence that
answers the question, and a truncated flag says something went, never
whether it mattered. The gap states how many characters it covers.
conftest gains an autouse stub for the new read arm, same reasoning as
_no_rule_arm: three widely-called tools grew a database read, and the
existing call sites should not each have to learn about it.
Raised while reviewing this: search() has the same shape and worse —
body[:240] with no marker at all, while the chunk that actually matched
sits unused in the row that won. Filed as #4243, not fixed here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Found by reading a live `retrieval_telemetry` readout after milestone 419
deployed, not by inspection. The readout said:
cannot_decline / report_preference — "45 calls, 0 of them returned
nothing. An arm that fires unasked has to be able to say nothing; this one
never has. Check that it applies its floor at all."
And printed, beside it, that arm's band: p10 = p50 = p90 = min = max = 0.791.
FIVE IDENTICAL PERCENTILES IS THE TELL. That is not a ranking, it is one
record at one score on every call — because `report_preference` searches a
fixed string (`reply_preferences.COMPLETION_QUERY`, a module constant, and
deliberately so).
For a fixed query against a stable corpus the top score is a CONSTANT, so the
arm's decline rate is 0% or 100% and never in between; which of the two it is
depends only on where the bar sits relative to that one number. "Never
returned nothing" is therefore arithmetic, not evidence, and the warning's own
remedy — check whether it applies a floor — cannot be answered from it.
The arm already knew this about itself; the warning did not:
"a fixed query makes this arm's score a constant and a floor a hair above
it produces a dead arm no amount of traffic will ever reveal"
— services/reply_preferences.py
THIS CLASS OF BUG ALREADY HAS A GUARD, which is the argument for the shape of
the fix. `Point.logs_unconditionally` exists because of #3497: both rule arms
once logged only their hits, so their zero count was structurally 0 and this
same warning would have fired on a LOGGING property while sending the reader
to move a threshold that was never involved. This is that one step over — a
QUERY-SHAPE property — and gets the same treatment: a declared field on
`Point`, and exclusion rather than trust.
AND THE WARNING THAT WOULD BE INFORMATIVE HERE DID NOT EXIST. For a fixed-query
arm the dangerous state is the mirror image: every call empty, meaning the bar
is above the constant and no further traffic will ever move it. The arm is off
rather than quiet, and nothing in the readout said so — `expects_traffic`
covers an arm with NO calls, not one with calls and a 100% decline rate. That
state is real and reached: `report_preference` once logged 69 consecutive
declines at 0.0006 under its bar.
So `fixed_query_never_clears` sends the reader to `near_miss_samples` and not
to the dial — because that incident is also the one where the statistic and
the correct action pointed opposite ways. Every percentile said lower the
floor; opening the refused record showed it was rule 77 arriving as a false
positive, and lowering it would have delivered that rule on every completion
report ever written.
Guards in tests/test_retrieval_warnings.py, including the falsifier that
matters most here: `cannot_decline` must still fire on an arm whose query
varies, or this change is a disabled check wearing a narrowed one's clothes.
Both boundaries tested from both sides, per that module's own standard.
The new code is documented in the `retrieval_telemetry` tool docstring beside
the others (rule 33) — an undocumented code in a readout is a reader meeting a
verdict with no way to disagree with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4222 fixed one end of this defect — the extractors that decide what a payload
DEFINES now blank comment and string spans before any line matcher runs. This
is the other end: `scribe_local_dups`, which decides which OTHER files already
define that name, and which was still a plain `git grep`.
A grep sees lines, not spans, so the sentence
class with only modifier rules is a deletion that went half-way.
— real prose from a module docstring in this repo — matched the arm's pattern
for `name=with`. Writing a genuine `change`, `beside` or `wrapped` would be
told it already existed, and pointed at a docstring.
EVERY HIT IS NOW CONFIRMED by running `scribe_defs` over the candidate file and
keeping only names it actually reports. That is the only check that cannot
disagree with the other end of the pipe, which is the whole point.
MEASURED, NOT ASSUMED — both numbers the task reasoned from turned out wrong.
- WHAT IT REMOVES, across 141 payload files of this repo: 15 of 210 report
lines. Every one a string literal, a comment, a TypeScript `import { type
Foo }`, or Vue's `const emit = defineEmits()` boilerplate. No real
definition was lost. Where a name had both — `create_note` — the phantom
in a test's `shape_form("async def create_note(...)")` argument dropped out
and the definition in services/notes.py stayed. `with` went from four files
to none, which is the correct answer: nothing here defines it, and it is a
keyword in several of these languages.
- WHAT IT COSTS: mean 193ms -> 222ms, worst 569ms -> 572ms. The task feared
"over a second added to a PreToolUse hook" from 48 confirmations. It is
about 15%, because the arm was already dominated by its twelve `git grep`
calls, and because confirmation runs once per DISTINCT candidate file
rather than once per (name, file) pair. A deliberately pathological payload
— nine names that are ordinary English words — reaches 28 distinct files
and 947KB; `scribe_defs` runs at ~33ms per 250KB.
THE CANDIDATE CAP IS RAISED FROM FOUR TO TWELVE, and that is load-bearing.
Confirmation REMOVES hits, so capping before it runs lets phantom matches crowd
a real definition out of the window — hits dropped before anyone looked at
them, which is #4042's bug in a new place. The display cap stays at four and
now applies to CONFIRMED hits, which is where a cap belongs. #4042's own `||
true` inside the substitution is untouched, and its regression case still
passes.
The task's own advice not to fix this by tightening the grep pattern is
followed and written down: requiring `(` or `{` or `:` after the name rejects
`class with only…` and also `class Foo extends Bar {`, `class Foo : Base()` and
`type Foo struct {`. This arm exists because it works with no server, no index
and no binding (#2280, #2682), which makes a miss here invisible — a visible
false positive is the better failure.
Guards: tests/test_hook_duplicate_confirmation.py, against real git repos
because what is pinned is the interaction between `git grep`, `head` and the
extractor. Seven of the eight fail against the previous implementation; the
eighth is #4042's regression case, whose job is to keep passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The lesson slot has recorded surfaced-vs-opened since it shipped. Nothing
showed it. `get_lesson`'s REST door attached `usage` to the payload and no
view rendered it; the listing did not attach it at all, and neither MCP door
did.
#4196 asks when a lesson that keeps getting followed should become a rule, and
names the trap in the same breath: raw frequency cannot separate "this should
bind" from "this trigger is too broad", and the second is the commoner reading
by a wide margin. Surfaced-AND-opened can separate them. Neither question is
answerable by a reader who cannot see the numbers, which is why this is the
first step and not the threshold.
NO THRESHOLD IS PROPOSED HERE, deliberately. The corpus today is 10 lessons
with 7 recorded surfacings and 3 opens, over about fifteen hours of usage
data. A promotion rule fitted to that would be fitting noise — #3311's failure,
and the warning lesson #4228 was written to carry. `UsageBadge` already
declines to render a verdict under three surfacings for the same reason. So
#4196 stays open: its subject, the promotion path, is still unbuilt. What
lands is the evidence it needs.
- REST `GET /api/lessons` and MCP `list_lessons` attach `usage` to every
row, from one aggregate per page rather than a per-row read, which would
be N+1 by construction. Every row carries the key zero-filled, so "never
surfaced" is a state a reader can see rather than a missing field they
have to interpret.
- MCP `get_lesson` attaches it too, and reads it BEFORE recording its own
pull. That door records a pull on every open — it has to, or the kind sits
permanently at zero — which makes the order load-bearing in a way it is
not for a kind that only counts. The REST detail door already ordered it
this way; the two now agree about what the number means.
- `LessonDetailView` renders `UsageBadge` (snippet #3460) rather than
re-spelling the chip, with the advice keyed to this kind: a lesson that is
repeatedly offered and never opened is usually keyed to a situation nobody
is in, so it points at re-keying `when_to_apply`, not at deleting the
claim.
Guards, in the two styles this pair of doors already uses: the MCP side driven
behaviourally through mocks, including the call ORDER for `get_lesson`; the
REST side on structure like its siblings in test_lesson_rest_door.py, because
the route is decorated and returns a Quart response. Rule 167's falsifier is
included.
KNOWN GAP, not fixed here: `KnowledgeView` is the only lesson LIST in the UI
and it reads `/knowledge`, not `/lessons` — so the REST listing change reaches
`frontend/src/api/lessons.ts::listLessons`, which currently has no consumer.
The agent-facing listing does reach a reader today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The column guard is a claim about kwargs; "the value comes back" is a
different claim, and only a round trip settles it. This drives the real
`restore_full_backup` over a judged shape carrying all three of the columns
that were going missing.
`diverges_from` gets the harder assertion. It is a FK to notes.id, so the
tempting fix — carry the exported id across — produces a row pointing at
whatever note holds that number in the target database: not dropped,
REATTACHED, with the restore reporting success and the divergence about the
wrong snippet. So the test asserts WHOSE note the pointer landed on rather
than which integer it holds, and it refuses to run at all if the restore
happened to reuse the source id, which would let it pass without proving
anything. Same shape as the assertion in the rule_usage round trip, for the
same seam.
Also corrects this module's own docstring, which said the round-trip module
"is not written yet". Three existed; this is the fourth.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The column guard (#3182) makes a dropped column unexpressible on the way OUT.
Nothing watched the way back IN, and that is the worse half: an export gap
leaves an obviously thin backup, an import gap means holding a complete,
correct file and restoring an incomplete database from it, with a success
message.
WHAT IT FOUND, the first time it ran. `code_shapes` was exporting
`reason_code`, `recheck_at` and `diverges_from` and importing none of them.
A restored ledger would have carried every judgment's verdict and not the
code for WHY — the column the accounting reads to tell a scoped-css
exemption from convention-plumbing — with every recheck flag cleared and
every divergence pointer gone. `diverges_from` is a FK to notes.id, so it is
re-mapped rather than carried: the raw source id would point at whatever
snippet took that number in the destination, which is wrong rather than
missing and is the milestone 333 trap one table over.
WHY THE GUARD WAS ONE-SIDED. Not an oversight — the code was. Export goes
through per-table pure helpers, so a test can hand one a stand-in and read
which keys came out. `_restore_v2` built all 26 models inline in one 690-line
procedural function, and there was no per-table unit to hand anything to.
So the construction moved out, into a `_build_*` helper per table taking the
exported row plus the id maps built so far. What deliberately did NOT move is
the loops, the flushes and the id-map bookkeeping: that is the
order-dependent part, where a mistake is a restore that half-works, and it
gains nothing from being split. Returning None is "skip" and a None field is
"degrade" — which one a table wants stays the table's own call, because both
are right somewhere: a shape event without its project says nothing, while a
usage event without one is still a real pull and dropping it would deflate
the pull-through the table exists to report.
The refactor was checked to be behaviour-preserving before the guard went in:
all 26 constructor kwarg sets identical to HEAD, and all 28 skip conditions
accounted for — 25 now in builders, 3 in loops that construct nothing (the
rule_systems raw insert and the two in the final project patch).
The guard then composes the two halves end to end: export a stand-in row,
feed THAT dict to the builder, read which columns the model actually
received. `test_the_usage_importer_restores_the_reading_project` read the
source of `_restore_v2` with `inspect.getsource` because there was nothing to
call; it is replaced by tests that call the builders and assert the
skip/degrade behaviour directly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#2990 asked for three cases. The wrapped-comment-continuation one shipped
with #4222; these are the other two — a comment holding a whole rule defines
nothing, and an unterminated comment does not swallow the file. Both already
pass on both sides; the vectors are what stops a later change to the scan
quietly taking them back.
The phantom #2990 was filed for is among the twenty-two #4222 removes:
`.editor-body` in TaskEditorView.vue, a persisted `code_shapes` row whose
signature ends in `*/` and whose `used_by` count is zero — so it has been
sitting in the unused_css number the operator is meant to act on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The definition extractor is line-oriented and knows nothing about what a
line is INSIDE. A docstring that wraps onto a line starting with a keyword
announces a definition: `AND` reached a live session as a divergence prompt
asking it to justify a symbol that does not exist, and `is` reached it as a
repo-wide duplicate of four files that define nothing of the kind.
Measured, not assumed: running the extractor over every scannable file with
and without the scan differs by twenty-two phantoms. Two of them — `with`
and `nobody`, both out of the module docstring in check_dangling_styles.py —
are persisted `code_shapes` rows that have been judged. Those need no
migration: sync_shapes marks a row it no longer extracts as vanished.
`ast` would be the honest tool for .py and is not what this uses, because
the extractor is mirrored rule for rule by an awk program in the hook, awk
cannot parse Python, and a fix only one of the pair can run is the drift the
mirror exists to prevent. Both sides now run the same left-to-right scan and
blank comment and string spans to their own newlines before any matcher sees
a line. Three things the scan has to get right, each of which cost real
definitions while it was being written:
- a string that HOLDS a marker is not a marker. `"red /* "` in
test_design_stylesheet.py and a triple quote inside a single-quoted
regex in plugin_context.py each ate every definition below them.
- `#` is a colour in CSS and a comment in Python, and the extractor is
handed no path. An alphanumeric straight after it settles it.
- an unterminated opener blanks NOTHING. The scan rewinds past it and
continues, so a stray marker costs one span rather than the rest of the
file.
The comment claiming the two extractors agree has been the only thing
holding them together, and a comment cannot fail. The mirror test now RUNS
the hook's awk over the same vectors: with the old program it reports the
phantoms, which is what a guard that can fail looks like. Across all 631
scannable files in this repo the two now agree line for line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A CORRECTION TO THIS ISSUE'S OWN PLAN, first, because the plan was wrong in
a way that would have cost the acceptance case.
#4208 recommended widening `kind` past `css | sym` to carry the form, calling
the denominator the deeper win and a migration the price. Two things:
1. THE MIGRATION IS NOT NEEDED. The issue says density "is computed from
stored rows and `shape_form` only runs on read". `canon_density` does
`select(CodeShape)...scalars().all()` and filters to the directory in
PYTHON; `signature` is a stored column; `canon_form(siblings, ...)` on the
next line already derives form from those rows. Bucketing the denominator
costs a list comprehension.
2. WIDENING `kind` WOULD BREAK MILESTONE #2793's ACCEPTANCE CASE. If `kind`
separated `fn` from `async-fn`, density would bucket them apart — which
does silence #4204's four `def` helpers, as claimed. Apply it to a
hand-rolled SYNC `confirmDanger` in a directory where an async confirm
helper is canon: the canon leaves the candidate's denominator, nothing
dominates, no flag. That is the flag the milestone exists to produce, and
it is the same inversion the first form gate made, one layer down.
So option 1 is not expensive-but-right, it is wrong; and option 2 (compare
meaning) is not the interim, it is the only lever. `confirmDanger` beside an
async confirm helper and `is_registered` beside a service unit are identical
at every structural level — same family, same form contradiction, same
density. They differ only in whether the candidate does the canon's JOB, and
no signature carries that.
WHAT THIS SHIPS is the part that is right and provable: an honest
denominator. `comparable_siblings` narrows the count to rows whose family
does not contradict the candidate's, using `families_conflict` — the same
predicate the gate uses, so the count and the verdict cannot drift into
disagreeing about what comparable means. "372 judged siblings" stops counting
a dataclass, a constant and an async service unit as three comparable things.
NARROWED BY FAMILY, NOT FORM, for the reason above: `fn` beside `async-fn`
stays a fair question. An unreadable sibling STAYS COUNTED — dropping it
would shrink `judged`, raise the share, and fire the check more on the
directories it can read least. Every unknown-form decision in this module
goes that way.
Density is now per candidate rather than per kind, cached on (kind, form).
TWO BUGS THIS CHANGE HAD, both caught before CI and both pinned:
- I passed a FAMILY where `families_conflict` reads a FORM, so
`shape_family("callable")` returned "" and the narrowing was a silent
no-op that still read as applied. The regression guard deliberately uses a
callable: `type` is both a form and a family name, so testing with it
proves nothing and the bug hides.
- My new `_Row` in the test file shadowed the one already there — same
fields, different `snippet_id` default — silently breaking three passing
`canon_form` tests. Reused the existing class. A duplicate definition
quietly changing a neighbour's meaning is this file's own subject.
The residue test's docstring said separating the four needs "widen `kind` or
a comparison of meaning". Corrected: they are not alternatives.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`retrieval_telemetry(days=30)` reported, for write_path_rule:
"the weakest tenth of what this arm returns scores 0.6984, only -0.0216
above its floor of 0.72"
A negative distance above something. The tenth percentile of what an arm
RETURNED cannot sit below the floor that gates what it may return — not
inside one population.
MEASURED CAUSE. write_path_rule's floor was 0.68 until 2026-09-02, when
2385100 (#3318) raised the shipped default to 0.72. The window opened
2026-08-22, so six days of it are calls made under the old bar; top_score.min
for the surface is exactly 0.68, the old bar still in the sample.
AND THE CHANGE LEFT NO TRACE THE READOUT COULD SEE. retrieval_tuning_events
records dial turns — a person or a model choosing a number. It was silent
about the other way a floor moves: somebody edits floor_default and ships it.
retrieval_tuning_history returned {"events": []} and retrieval_surfaces said
last_change: {}, source: "shipped". All true, and all of it silent about a
floor that had in fact moved.
THE RAISE ANNOUNCED ITSELF. A LOWERED FLOOR WOULD NOT: the gap comes out
comfortably positive and reads as a clean bill of health on a sample that
half predates the bar being judged. Both directions are now pinned.
So the check is SUSPENDED, not softened. band_hugs_floor asks whether the
scores are piled on the bar; that needs the scores and the bar to come from
the same regime. Where they do not, the honest answer is that this sample
cannot say, plus the date after which one can — floor_moved_mid_window
replaces band_hugs_floor for that arm and never accompanies it. A reader told
a number is unavailable goes and gets one; a reader handed a qualified number
uses it.
NO MIGRATION. `actor` is Text with no CHECK precisely so a new kind of actor
is not one — the model's own comment says so, and this is the case it
anticipated. "release" joins "model" and "human". user_id is already
nullable, which is right: no user did this, a release acts on every account
that has not overridden the dial, and a row per user would both multiply and
misattribute it. Both readers now take the newest of (this user's change, the
release's).
THE FIRST SIGHTING IS A BASELINE, written with old_value NULL. Nothing moved;
the row exists so the next release has a predecessor. That null is
load-bearing: floor_moves_since asks for old_value IS NOT NULL, so a fresh
install's baseline does not silently retire the check on every new install.
UI: the tuning history rendered actor as `human ? 'you' : 'Claude'`, so a
release row would have told the operator that Claude moved a floor it never
touched — the one failure the actor column exists to prevent. Three-way now,
with an unknown value printing itself rather than guessing.
Recorded at startup, inline and awaited. What #4181 cost three hours was
concurrency — a background task racing the hook for the same pool. Sequential
creates no contention, and this is twelve single-row reads. It must finish
before serving because a readout served before the change was recorded is the
exact answer this exists to stop giving.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419's acceptance, and it failed the first time it was run — which
is the only reason this commit exists.
THE MEASUREMENT. Ran the step-5 readout against this session's real traffic
instead of a fixture. It reported 19 rules named by an arm and none opened.
That is false: the session had called `get_rule` 45 times. Across six real
sessions on this instance: 208 opens, 3 surviving ledger entries. 1.4%.
THE CAUSE. `.opened.ids` was doing two jobs with opposite lifetimes.
- "this context HOLDS rule 156" — false after a compaction, and three hooks
read it to decide whether to stay quiet. Clearing it is correct.
- "rule 156 WAS OPENED" — which no compaction makes untrue, and which the
session-end readout is built on.
`scribe_clear_session_ledgers` sweeps every `<sid>*.ids` on SessionStart
source=compact. Right for the first claim, and it was deleting the second.
The TTL did the same thing more quietly: `scribe_rules_live` ages an
exclusion ledger, which is right, and would have eaten the early part of any
long session's evidence too.
So the readout was reporting only the stretch since the last compaction while
reading as though it had reported the session — a statistic that cannot vary
being mistaken for a finding (#3311), which is the shape this whole milestone
exists to stop producing. It ran AT the seam it was blind to.
THE SPLIT. `scribe_rules_append` now writes both: the exclusion ledger it
always wrote, and `<kind>.keep.ids`, an evidence twin that is never aged and
never swept. The readout reads twins; the three `held` readers are untouched.
DERIVED, NOT LISTED, because a list is what broke this before — the comment
above the sweep says so about its own history. Every ledger written through
the appender gets a twin, including the next one somebody adds; a new ledger
is born on the swept side unless its name opts out. `scribe_checkpoint_allowed`
writes its twin explicitly since it bypasses the appender, and there the split
lands right on both sides: the cap counts the swept file, so a compaction
honestly restores the budget to stop an act the context can no longer justify,
while the record that a stop happened stays.
Removed `scribe_ledger_ids`, orphaned by the change — a dead helper beside a
live one is a thing the next reader trusts.
test_session_ledger_clear.py asserted every ledger dies and could not have
caught this: its fixture never created a twin, so the sweep was one glob away
from either mistake with only one of them guarded. Both sides now asserted.
The slippage tests build their ledgers through the real writers rather than
by hand, for the same reason — a fixture that writes the bytes itself keeps
passing after the writer stops.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 5 of milestone 419. The milestone's subject is that a rule read and
ignored is arithmetically identical to a rule read and followed, and the
compaction is where that identity becomes permanent — the turns holding the
evidence are summarised away, and the unjudged thing survives as nothing.
WHY THIS IS ASSEMBLED IN THE HOOK. `rule_usage_events` has no session column;
it is per user over a window. A session-scoped answer therefore cannot be
asked of the server, and has to be built where a session is a thing that
exists. Four ledgers four hooks already write:
.rules.ids an arm NAMED the rule
.opened.ids the session called get_rule (#4100)
.acted.ids the session called rule_outcome (new here)
.checkpoint.ids the rule HELD an act (#4214)
Every one is an observed tool call. Nothing asks the model what it followed —
milestone 386 ruled that out, because a model asked "did you apply rule 156?"
says yes. Two subtractions: named-minus-opened is the arm talking to nobody,
opened-minus-acted is the milestone's whole subject.
PreCompact stdout is the compaction's custom instructions (#3680), not a
message to the model, so the readout does not say "you slipped" — it says
which ids must be carried through, which is the one thing a summary can do
about an unjudged finding.
SILENT WHEN NOTHING HAPPENED, and the accusations are conditional on having
members. "0 rules unresolved" on every compaction is how a readout teaches
its reader to skip it. Traffic is still reported, because the static
instructions already ask for it in prose; these lines are the measured
version.
scribe_record_outcome.sh is the third ledger's writer, matched on
mcp__.*__rule_outcome and mirroring scribe_record_opened.sh: TMPDIR only,
silent, exit 0 on every path. A PostToolUse hook that spoke would put a line
after every rule_outcome call and give recording an outcome a cost.
Also: check_plugin.py skipped the new hook for want of a smoke event, which
would have left the newest of the three ledgers as the only one the plugin
lane never runs. Added, mirroring its sibling.
tests/test_precompact_hook.py now isolates TMPDIR — the hook reads session
ledgers from there, so without isolation a test would see whatever this real
session had accumulated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 4. Five of the milestone's seven misses were the same move:
acting on the thing in hand without reading the contract around it. Lesson
#4207 says so in words, and was written by its author hours before a
structurally identical mistake, having been surfaced twice in the turns
between. Text delivered at the moment of acting is too weak a carrier for a
reflex that has to change what the act IS. This looks it up instead.
Rule 33 one scope down: its checks are between layers, and the same question
exists between a definition and its callers.
THREE KINDS OF EXPOSED NAME, because a contract breaks three ways that look
nothing alike in source — the defined symbol (a rename or removal), its
parameter names (arity), and the quoted keys of its dict literals (the shape
of what it returns).
THE THIRD IS THE ONE A SIGNATURE-WATCHER MISSES, and it is in because of the
miss that produced this step. Two commits ago `get_writepath_config` gained
one dict key; three arms read that dict inside a fail-open `except`, every one
silently became a no-op, and ten tests went red with nothing pointing at the
cause. No signature changed. Run against that exact edit, the check now names
tests/helpers.py — the actual root cause — among six files, before the write.
TWO GATES, AND THE SECOND IS WHAT MAKES IT USABLE. A change to the exposed set
is necessary but not sufficient: a definition nothing else references has no
contract to break, so the readers lookup runs second and an empty result ends
it silently. Body-only edits say nothing, a subject is named once per session,
and the ledger lives in the swept directory under the `.ids` convention, so
the existing compaction-clear guards cover it — checked against
test_session_ledger_clear's own parsers rather than assumed.
LOCAL AND SERVERLESS, like the duplicate-name arm beside it. It needs the
working tree and nothing else; the server has no checkout, so this is the only
place the question can be asked. It is a NUDGE: scribe_prior_art.sh still
returns no permissionDecision, which is the operator's recorded decision that
a recall aid may not stand in the way of a write. A test asserts that here as
well as in test_write_path_trigger.py, because this is the arm most likely to
tempt someone into making it a gate — it reports something that may already be
broken.
The `sym` half delegates to `scribe_defs` rather than repeating its patterns:
those cover nine languages and have been corrected several times, and a second
copy would inherit today's version and quietly stop agreeing with it (#3497).
Verified by lifting the test file's own helpers and driving all 19 cases
against the real shell over a fixture git repo. Two of my own errors were
caught that way and are fixed: the fixtures were arriving as single lines
because Python `repr` inside bash single quotes leaves `\n` as two characters
(the extractor is line-oriented, so the tests would have gone green against
input no editor can produce), and the no-readers case put its subject in a
file that was not the excluded one, so it had a reader and tested the
opposite of its name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7126: 1 failed, 2227 passed — down from 14. The last one was mine, and a
different mistake from the batch before it: I widened four assertions on the
literal `{"context": "", "rule_ids": []}` by replacing the string, without
checking which arm each test was calling. Three are the tool arm. The fourth,
`test_the_prompt_arm_says_nothing_when_asked_nothing`, is not.
The distinction is real rather than an omission, so the assertion goes back
with it written down. The two ACT arms can hold a call because there is a
composed act to hold. The prompt arm fires on the operator's message, before
anything has been decided — there is nothing to put a rule in front of, and a
checkpoint there would have to guess at an act that does not exist yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7124 Python tests: 14 failed. One root cause behind ten of them, and the
failure was the exact one `tests/helpers.writepath_cfg`'s docstring already
warns about in prose — while being unable to prevent this instance of it.
Three arms read their numbers out of that config dict inside a fail-open
`except`. A missing key raises where nobody sees it, so the arm becomes a
silent no-op, indistinguishable from the arm working and finding nothing. The
helper derives its keys from `retrieval_surfaces.SURFACES` precisely to stop
that — and `checkpoint_threshold` is deliberately NOT a surface, because
everything in that table is a floor/budget pair belonging to one query and the
checkpoint runs none. The derivation therefore could not see it, the write-path
rule arm died before `record_retrieval`, and ten tests went red at once.
Fixed at the helper, from the module constant, so there is still exactly one
literal and it lives in the product. And the guard the docstring claimed now
exists: `test_the_config_stand_in_carries_every_key_the_real_one_does`
compares the stand-in's key set against the real `get_writepath_config`, so
the next key that is not a surface fails loudly here instead of quietly
disabling an arm under test. `test_retrieval_surfaces`'s hand-written key list
gains it for the same reason, spelled out in place.
The other four were the contract widening itself: `checkpoint` is present on
every return of the tool arm, including its early ones, so four assertions
comparing the whole dict needed it. That key is deliberately always present —
the two arms feed one shell reader where an absent key and an empty one are
read the same, so the difference is invisible exactly where it would bite.
Verified statically: every cfg in the suite now routes through `writepath_cfg`
(test_write_path_trigger's local `_cfg` delegates to it), no hand-written
config dict survives, and the real config's eight keys match the stand-in's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 3, the pre-act checkpoint. Every rule surface in this
plugin returns `additionalContext`, which Claude Code delivers alongside the
tool RESULT — so the rule is read after the call is written and lands as
commentary on a decision already made. That is the milestone's central
finding, measured over a session with seven misses, three caught by the
operator and none by this system.
The action arm can now return a `deny` instead. The act does not run, the
rule's text can be read before the call exists, and the remedy is one
`get_rule` call after which the act may be re-submitted unchanged. Nothing
reaches the operator: a deny is a message to the model.
"CONSEQUENTIAL" IS DERIVED, NOT ENUMERATED. The obvious implementation lists
act kinds — a write to product code, a schema change, a bulk classification, a
merge. Every one of those is consequential because THIS operator wrote rules
about it, and shipping that list is this instance's corpus hard-coded into the
product (rule 115). So the corpus decides: an act is consequential when the
install's own rules speak to it above the checkpoint bar. A fresh install with
no rules never stops anything.
FOUR CONDITIONS, EACH PREVENTING A DIFFERENT WRONG. Above the bar; a rule and
never a preference (which claims no such force); the band's top hit only (the
ranker's confidence claim attaches to its first element); and only a rule the
session has NOT opened — `held` is observable from the get_rule PostToolUse
hook (#4100), not self-report.
WHY "NOT OPENED" RATHER THAN "NO OUTCOME RECORDED". An outcome can be
satisfied with one cheap call asserting compliance without producing any, and
a checkpoint dismissible that way manufactures exactly the compliance data
step 2 was built to measure. Reading a rule cannot be faked in that direction:
after `get_rule` the statement is in context, which is the whole of what was
wanted.
THE BAR IS MEASURED. `retrieval_telemetry(days=30)`: write_path_rule p90
0.7628 max 0.8817; pre_tool_rule p90 0.7373 max 0.8293. 0.80 is above p90 on
both and below max on both, so it selects from the top decile of an already
selective arm and is still reachable. It ships as a setting with a Settings
card, because a cosine distance in one model's geometry over one corpus cannot
transfer.
TWO GUARDS ON THE WORST CASE: at most one hold per rule and five per session,
so a mis-set floor degrades to a noisy session rather than one that cannot
proceed. The ledger lives in the swept directory and is named `.ids`, so the
existing compaction-clear guards cover it.
WRITES ARE NOT HELD, AND THAT IS THE OPERATOR'S DECISION RATHER THAN MINE.
`scribe_prior_art.sh` carries a tested property that it never returns a
permissionDecision — a recall aid may not stand in the way of a write. Three
of the milestone's seven misses were file edits and none are reachable from
the command side, so there is a live argument for extending this; that
argument is exactly why the boundary is now asserted by a test rather than
left to memory. The write-path arm computes and returns the same block so the
decision can be revisited with evidence; the hook ignores it, and a change of
mind is a hook edit rather than a feature.
Verified by lifting `checkpoint_for` and `_rule_band` out of source with `ast`
and exercising the shipped functions over 17 populations, by running the
ledger and deny envelope in bash (10 cases, including that a refused hold is
not written and that a garbled rule id fails closed), and by
scripts/check_plugin.py — which caught the unminted plugin version, 0300 ->
0426.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 2. Step 1 made an outcome recordable; this makes it
readable. `retrieval_summary`'s rule block gains `applied`, `departed` and
`distinct_rules_acted`, and `_compute_warnings` gains two codes.
TWO CODES, NOT ONE WITH A ZERO IN IT. `read_and_unacted` reports rules that
were opened and left no outcome, against the ones that did. It only fires once
outcomes exist anywhere in the window, because a window with none cannot tell
"every rule was ignored" from "nothing calls `rule_outcome` yet" — and on
every install the day this ships, the truth is the second. Claiming the first
there would be #3311's failure exactly: a statistic that could not vary being
read as a fact about the corpus. The cold case gets its own code,
`outcomes_never_recorded`, whose prose says in as many words that it does NOT
mean the rules were ignored.
`applied` AND `departed` ARE NOT SUMMED. A departure carries the reason the
agent gave and is evidence about the RULE; an application is evidence about
the agent. Folded together they would say only "an outcome exists", which is
true of both and useful about neither. `distinct_rules_acted` counts either,
because for the unacted arithmetic the distinction does not matter.
An outcome is not a pull. The fold branches on OUTCOMES first and never routes
an outcome through the surfaced/ambient split: `source` on an outcome row
names the door the outcome came through, not a ranker, so the ambient
distinction has nothing to say about it. An integration test holds that line —
if an outcome leaked into the pull counters the silently-unchanged rule would
vanish into a compliant-looking total, which is the confusion #4212 was opened
to end.
Verified by lifting the shipped `_compute_warnings` out of source with `ast`
and exercising it against the six populations the new tests assert: cold
instrument, warm instrument, departures-only, full compliance, nothing opened,
and a failed read. The integration tests for the new counts run against real
Postgres in CI — count(distinct) with an IN over an unconstrained column is a
SQL shape a mock would agree with whatever it did, which is what #2663 was.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419's own subject, committed while building it. The module header
of mcp/tools/rulebooks.py says the tool count "lives in the registration
test, which fails when it drifts" — I read that line while adding the tool
and did not act on it, which is a rule surfaced, read, and silently
unchanged. The only reason it was caught is that the guard exists and CI ran
it; nothing about my process would have found it.
Worth stating plainly because the milestone is about exactly this class of
miss, and step 1 shipped the record that would let a future session SEE it:
a rule read and not followed is invisible unless something independent
notices. Here the something was a test written by whoever last changed this
count.
Integration was already green on the previous run, so the migration, the
backup round trip and the new `detail` column were never in question — this
was one integer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and
that it was PULLED. It could not say what happened next, so a rule that fires
constantly and is always obeyed and a rule that fires constantly and is never
obeyed left byte-identical telemetry. The second is far the more urgent and
was the one the readout could not name — measured on a session where three of
seven misses were caught by the operator and none by the system.
Two new events, `applied` and `departed`, and a `detail` column carrying the
why of a departure. No CHECK migration: `event` was created in 0094 as plain
Text with no constraint, verified in the migration rather than assumed from
the model, so rule 36 does not bite here — said in both places because the
next person adding a value will reach for it.
THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently-
unchanged is the failure this milestone was opened on, and it cannot be
reported: an agent that knew it was ignoring a rule would not be ignoring it.
So nothing here asks. `applied` and `departed` are reported; the third state
is a rule that was opened and left no trace. An `ignored` enum member would
collect nothing while reading as though it had measured something, which is
#3311's failure — a statistic that could not vary being taken for a finding.
`detail` is a column rather than two more bare event strings because a
departure stripped of its reason reads back as a miss, so the two states this
exists to separate would collapse again one layer down, in the readout, where
nobody would see it happen. Nullable: following a rule needs no argument, and
an expensive event is one that stops being recorded.
`outcome_state` is the single reading of the four states, taking the aggregate
`usage_for_rules` already returns, so the badge, the readout and any later
session summary cannot disagree about what "followed" means — the drift #3246
found across the rules system. A departure outranks an application: a rule
both applied and argued with is a rule someone argued with, and the argument
is the half worth surfacing.
`rule_outcome` is the MCP door, classed as a WRITE. The read-only set
tolerates getters that call record_pulled, but those are reads that leave a
trace; this tool's entire effect is the row, and the row carries prose the
agent authored. A read-scoped key that can put text in the operator's
database is not read-scoped, whatever table it lands in.
Backup carries `detail` on both sides. It is the one field here a fresh
install cannot re-earn — counts come back by being used again, a stated
reason exists once — and #4197 records that the column guard watches the
export side only, so the round-trip test is the thing that would catch a
one-sided add.
Delivery is deliberately not settled here: how an agent gets prompted to
record an outcome is step 3's subject, and the same record serves whichever
answer that step reaches.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The review surface shipped yesterday reported two canons on its first live
day and both were sound. Coherence was "do all judged rows share a form",
which #2844 failed at 37/62 = 0.597 for containing a model class and the
to_dict the canon's own text says the class must carry, and #2849 failed at
4/7 for pairing sync loop-starters with the async ticks they schedule. A
review surface whose whole output is noise is one that stops being read.
The obvious repair is a trap, and there is now a test standing in front of
it. Grouping by family and keeping a majority test makes the check BLIND:
before #2844 was cleaned by hand it held 37 classes and 56 callables, which
as families is 56/93 = 0.602 — a clean pass, and the 31 rows that had no
business being there (Vue functions, route handlers, a dozen tests) would
never have been reported at all. A looser bar in the same shape is worse
than the bug.
So the verdict is inverted. Instead of asking whether most rows agree, it
asks how many rows the canon CANNOT ACCOUNT FOR: a row in the majority
family is accounted for; a callable defined in a file that also holds a
majority-family `type` row is a method of a member, not a foreign body; and
strangers above a fifth of the readable rows make the canon incoherent. The
majority vote abstains those methods, so a class's own serialisers cannot
outvote the classes and turn the members into the strangers.
Measured on the real ledger before it was written, which is why it is this
rule and not a nudge to the share: clean #2844 has 0 strangers in 62,
#2849 has 0 in 7, and polluted #2844 had 31 in 93 — the same 31 withdrawn
by hand this morning, named exactly.
The entry now carries `families`, the majority `family`, `attached`,
`stranger_count`, `unattended`, and `strangers` — THE ROWS THAT DO NOT FIT,
replacing a sample of the first twelve members. The reader's question is
which rows are wrong, and a sample of the agreeing majority cannot answer
it. `unattended` is the discriminator between a check that is too strict and
a ledger full of junk: both canons flagged on day one were entirely
audit-judged, and nothing showed that without opening each one.
Scoped to the review surface. `canon_form` still answers at the precise form
level for stamping and divergence, where a sync helper beside an async canon
is a fair question; nothing here changes what the ledger writes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7098: unit tests red, everything else green. `_INSTRUCTIONS` was 2439
against a 2000 budget.
WHAT I DID NOT CHECK. That block is capped because Claude Code injects only
the first ~2,048 characters of a server's instructions and cuts the rest
mid-word (#2562, observed live — a 20k version delivered ~10% of itself and
the Systems guidance never reached a session). The cap is stated in a comment
directly above the literal I edited. It was at 1998/2000 before this batch:
a shared, nearly-exhausted resource, and I added a six-line entry to it.
THE JUDGE LINE STAYS, and paying for it is the decision rather than dropping
it. A client with no Agent Skills support receives this index and nothing
else, so of everything here, "you are the judge of record" is among the least
safe to leave past the fold — an agent that never learns it defers every call
to an operator who was never going to make them.
So the line is earned by compressing prose AROUND the existing markers, not by
removing anyone's entry: RULES loses a clause, RECORD and REPORT lose trailing
restatement, PLAN drops a sentence the two markers already imply, and the
opening paragraph tightens. Every index marker the ownership registry requires
survives verbatim — that is what test_the_index_names_each_reflex_it_points_at
checks, and it passes.
Back to 1998/2000: the same headroom as before, with one more reflex indexed.
The next addition pays the same way.
Three guidance modules run green locally (21 tests) — they read files and need
no database, so this one did not have to go to CI to be known.
Plugin version re-minted; the previous mint is on a commit that never went
green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
I recorded this as project rule 174 first. That was wrong twice over, and the
second reason is the one that matters.
RULE 119 SAYS THIS EXACTLY: guidance about how an agent should behave with
Scribe belongs in `_INSTRUCTIONS`, `plugin/skills/*` or the adapter's static
context, never in the corpus. I read 119 while writing the rule, decided it was
"about authority rather than about using Scribe", and wrote it anyway — which
is the reasoning preference 29 exists to catch, performed in full.
THE REASON THAT MATTERS: a rule in the corpus is true on ONE install. If the
agent being the judge is how Scribe works, every install gets it or none does.
Baked in, it ships. As a rule it was one operator's private note about a
product stance.
WHAT IT SAYS. The agent is the judge of record for the work — what a shape is,
whether a finding holds, whether something is done. Surfacing a finding for the
operator to rule on is the judgment NOT made, however well written up: it reads
as diligence and functions as a backlog. Escalate the acts that are genuinely
theirs — their money, their infrastructure, anything hard to reverse or facing
outward — and keep the decisions. A hard call is still yours; an irreversible
act is still theirs.
And the half that keeps this from becoming the previous defect: JUDGING IS
ATTENDED. An agent reading evidence and recording why is judgment; a threshold
or a sweep reclassifying in bulk with nobody reading is the thing that fills a
ledger with confident nonsense (#4208, and Portal's 35 rows). When the fix for
bad unattended writes is another unattended write, stop.
THE PRODUCT WAS TEACHING THE OPPOSITE. reporting-back's Finding row read
"Symptom · Cause · Size of the fix · **Offer to fix it**". So the behaviour I
was corrected for is the behaviour the skill prescribed — which is the better
argument for fixing it here than any rule could be.
Three surfaces, per 119 and the ownership registry (#4027): `_INSTRUCTIONS`
gets a one-line JUDGE index entry; using-scribe owns the authority and the
attended/unattended distinction; reporting-back owns the report shape. Two
topics rather than one, registered separately in test_guidance_ownership so
trimming one cannot quietly take the other.
Plugin version minted — skills only reach a session when the manifest moves
(#2209).
Rule 174 deleted (trash 074434a2, recoverable).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Lint caught an F821 that would have been a NameError the first time
`stamps_to_review` was called: `access` is imported locally inside each of the
seven functions in this module that need it — services/access reaches back
here, so a module-level import closes a cycle — and the new function used it
without one.
I wrote the function by pattern-matching its neighbours and did not check what
those neighbours do to make themselves work. Same shape as the tuple unpack
two commits ago (#4207): the mistake is not in the logic I was thinking about,
it is in the surrounding contract I did not read.
Unit and integration were both green on the failing run (7095); only lint was
red. Worth recording because the lane that caught it is the cheapest one and I
had read its command as covering tests — `ruff check src/ scripts/` does not
look at tests/ at all, so a clean test suite says nothing about it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
THE HALF THAT WAS MISSING. #4204 put a floor under what the write-path hook
may assert. A floor only guards new writes; every row already stored stands
(lesson #4202). Measured after that fix shipped: Portal carried 32 rows under
one canon and 3 under another, all stamped on scores of 0.69-0.77 — below the
0.80 floor, so none of them could be written today, and all of them were still
there. Scribe's own ledger carries 334 under #2860.
`stamps_to_review` reports two things and changes nothing:
weak — rows the hook stamped on a resemblance below the current floor,
each with its score, signature and derived form.
incoherent — canons whose own judged rows do not agree on a form. A canon
claims some shapes are the same sort of thing; when its members
are a class, three getters and a dozen tests, that claim has
stopped being true and every base-rate reading built on it is
reading noise. `canon_form` already made such a canon fall
silent — nothing made it VISIBLE.
IT DELIBERATELY CANNOT FIX ANYTHING, and that is the design, not an omission.
The first version of this commit was an automatic sweep that reset rows by
score. That is the original defect pointed the other way: what harmed the
ledger was not one wrong score, it was a machine recording permanent
classifications unattended. Un-recording them unattended is the same act with
a wider blast radius. An agent reads the evidence, judges, and records the
judgment under its own name through `classify_shapes`.
`test_the_service_carries_no_machinery_for_bulk_withdrawal` asserts that
structurally, so the next person to reach for an auto-retire has the argument
again on purpose rather than in a diff nobody reads.
A JUDGMENT IS NEVER LISTED AS WEAK, whatever its age. This is the measured
correction to an assumption I nearly shipped: of Scribe's 334 rows under
#2860, 302 are in `services/` — the canon's own home — and the ones sampled
there are `classified_by="audit"` with no score at all. The legitimate bulk
of that canon was never scored; it was judged by an agent in batch. Listing
those as weak would invite an agent to withdraw the only real judgments in the
ledger. An agent's decision is a different KIND of evidence, not a worse one.
THE SCORE NOW HAS A PARSER. It lived only inside a prose sentence, so nothing
could ask how strong the evidence for a row was without re-deriving it — which
is how 32 rows sat unexamined for nineteen days. Format and reader are one
constant apart (`_RESEMBLE_REASON` / `stamp_score`), with a round-trip test and
a test pinned to reason strings taken verbatim from the two poisoned ledgers.
`live_rows_for` is `live_rows` behind the project read gate, for callers that
arrive from outside rather than from a job that already knows who is asking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
[s_id for rank, _at, s_id, _why in bucket if rank == 2]
ValueError: too many values to unpack (expected 4, got 5)
`in_play` gained the canon's form as a fifth element so the stamp could be
decided per shape rather than per file. The `record_uses` call eighty lines
below still destructured four, and every stamp that reached it raised.
Both integration failures on runs 7090 and 7091 are this one line —
`test_write_path_stamp_is_evidence_that_yields_to_judgment` and
`test_a_brand_new_shape_gets_a_provisional_row_the_sync_settles`.
Now indexed rather than destructured, matching the candidate scan above it, so
the next widening cannot break it positionally.
WHY THE UNIT LANE STAYED GREEN THROUGH TWO PUSHES. `record_uses` is only
reached once a stamp is actually written, which needs a real snippet, a real
ledger row and the write ACL — so no unit test crosses that line. 139 local
assertions and the whole unit suite passed on code that raised on every
successful stamp. The integration lane was the only thing that could say so,
which is the case for it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 7090 caught this; the unit suite could not. Integration job 25763 —
`test_a_second_confirm_dialog_is_detected_and_named`, the acceptance case of
milestone #2793.
WHAT I GOT WRONG. The previous commit gated BOTH halves of the ledger on
`forms_agree`. That is right for stamping and backwards for divergence,
because the two assert opposite things:
STAMPING says "this IS that canon". Agreement in form is evidence FOR the
claim, so demanding it is correct.
DIVERGENCE says "this is NOT the canon that dominates here — did you mean
to?" A form MISMATCH is the PREMISE of that prompt. Requiring the candidate
to match the canon silences the check precisely where it belongs.
So #2793's case stopped firing: a hand-rolled sync `confirmDanger` in a
directory where an async confirm helper is canon read as `fn` against
`async-fn`, disagreed, and was dropped. `flag_divergence` returned 0 where the
test demands 1, and the write-time check returned nothing where it must name
the canon. That is a real flag the milestone exists to produce, and my change
removed it.
THE FIX. Divergence now gates at FAMILY level — callable {fn, async-fn},
type, value, css — and only on contradiction. A sync function beside an async
one is still a fair question. A frozen dataclass told to build from an async
service function is not a question at all.
WHAT THIS DOES NOT FIX, asserted rather than commented so it fails the day it
changes (`test_how_many_of_the_five_the_divergence_gate_actually_silences`):
of #4204's five false prompts this silences ONE. `Point` is a type against a
callable canon. `_p`, `get_point`, `is_registered` and
`sources_expected_to_emit` are callables like the canon and still ask — and
at the signature level they are indistinguishable from the #2793 case above,
so nothing readable here can separate them. That needs #4204 option 2 (widen
`kind` past `css | sym`) or a comparison of meaning rather than form.
The stamping half — `_RESEMBLE_MIN` 0.80 and the graded burden — is unchanged
and unaffected by this failure. It is also the half that matters more: loose
stamping is what manufactures the density the divergence check reads as
authority.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Two halves of one defect, found while writing #3431 and measured on a second
project.
THE DIVERGENCE CHECK WAS A BASE RATE. `dominant_canon` answers "what is most
common in this directory" and never "is this that" — the candidate's
signature was not examined at all. With `kind` carrying only `css | sym`, a
frozen dataclass, a module constant, a sync predicate, a class and an async
service function are all siblings, so the prior was not merely the best
signal, it was the only one. Writing a registry module of pure helpers
produced five prompts to build them from the `async_session` service canon.
THE AUTO-STAMP HAD NO FLOOR. `elif sid in resembles` took any score at all:
0.69 asserted as confidently as 0.95, and the number went into the reason
line without ever being compared to anything. Worse, the score is computed
against the WHOLE PAYLOAD, so one number spoke for every symbol in the file.
On Portal that recorded `class SessionAbsent`, `def build_channel`,
`async def attach` and a dozen test functions as instances of one snippet —
17 rows under #3283, which then made that directory "canon-dense" and started
instructing every later writer in it.
The two compound: loose stamping manufactures the density the divergence
check reads as authority. Both are fixed by one primitive.
`shape_form` derives a coarse form — css / type / async-fn / fn / binding —
from the signature, on READ. `kind` is part of the row identity, so widening
that column needs a migration and a re-extract (#4204 option 2, still the
principled fix); deriving costs nothing and is reversible. Every caller asks
`shape_form`, so the day the column carries the answer it returns that.
THE BURDEN SCALES WITH THE EVIDENCE, and getting this wrong was the first
version. A by-name reference — the payload names the canon's symbol — is
strong and needs only the absence of contradiction; demanding positive
agreement there silenced it whenever a shape's definition was not in the
payload (an Edit rather than a Write), turning strong evidence into none for
a reason unrelated to the code. A resemblance score is weak and must
positively agree.
`canon_form` reads the form a canon's own judged rows agree on, and returns
unknown when they disagree. That makes the halves cooperate: a canon already
poisoned by loose stamping — Portal's #3283 — falls silent instead of
flagging anyone else.
`forms_agree` requires BOTH sides known, so an unreadable signature makes the
checks quieter rather than more confident. `_RESEMBLE_MIN` is 0.80 rather
than the retrieval floors near 0.70: those decide whether to SHOW a record,
where being wrong costs a glance; this decides whether to RECORD a claim
unattended, where being wrong misinstructs everyone who writes there after.
Caught by the tests, not by review: a first pass required `const`/`let`/`var`
before a binding, so every Python module constant read as unreadable and a
whole form was silently excluded from both checks.
Not addressed: rows already carrying a wrong snippet_id are not undone by a
guard at the point of classification (lesson #4202). Portal's 17 keep
producing dominance until something re-judges them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The tool returned distributions and left the reading to the caller, so every
readout was the same four checks done by hand — #3430's baseline, #3835's rule
near-misses, the #1038 rerank gate. Mechanical, and therefore forgettable.
Tonight's acceptance pass on #3898 was the case for doing this. Reading it by
hand meant catching that two surfaces had `covers_window: false`, that
`prompt_rule`'s floor had moved three times inside the window (which made the
readout self-contradictory: deliveries at 0.622 beside refusals at 0.7199),
and that 15 of 20 near-misses were one record against text no operator wrote.
Miss any of those and the obvious conclusion was "the bar is too tight" — a
floor change that would have injected one preference into every notification.
`warnings` is always present and empty when clean, so its emptiness is an
answer rather than a gap. Each entry carries the numbers that produced it:
"345 calls, 0 declined" is the analysis, "check write_path_rule" is an
instruction to redo it. Five codes — cannot_decline, band_hugs_floor,
no_duration, surfaced_never_pulled, unregistered_source.
cannot_decline has three guards, each a bug it would otherwise cause. Asked
surfaces are exempt (a search returning a list every time is working). An arm
not known to log unconditionally is exempt — that is #3497 exactly, where both
rule arms recorded only their hits, so a decline count of zero was a LOGGING
defect and this warning would have sent the reader to a threshold that was
never involved. Unregistered sources get numbers but no verdict.
`silent_surfaces` is the half the rows cannot show: an arm that emitted
nothing is invisible to every row-based check and looks exactly like an arm
that does not exist. It is driven by a new declared registry,
`retrieval_registry.POINTS` — deliberately NOT `retrieval_surfaces.SURFACES`,
which answers "what can be tuned" and excludes the reserved slots because a
budget of 1 is their feature. This answers "what can be measured", and the
reserved slots belong in it precisely because they are judgeable without being
tunable. A test asserts the two cannot drift apart.
The registry test derives sources from the call sites with `ast`, not grep,
and the difference is not theoretical: `wide_net` and `report_preference`
reach their recorder as `source=SOURCE` through a module constant, so a grep
for `source="` is blind to both — the narrowing #3191 warns about. Three sites
pass `source` as a variable and are declared in FAN_OUT_SITES; the test pins
those sites but not the values they can pass, which is why the
`unregistered_source` warning exists to catch the rest at first fire.
Thresholds are settings (rule 25) defaulted so a fresh install with almost no
data produces no warnings at all (rule 115) — a new user's first readout
naming five broken things would be describing the emptiness.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Claude Code submits more than typed words through UserPromptSubmit. A task
notification, a slash-command echo and the caveat banner a local command
prints all arrive as user turns, reaching `.prompt` indistinguishable from
something the operator wrote. scribe_autoinject.sh believed all of them.
The cost that matters is not the wasted embedding — it is the log row. Every
such call counts in the denominator of every prompt-boundary surface, so
delivery rate reads low for a reason unrelated to retrieval; and each refusal
lands in `near_misses`, where a later tuning decision reads it as demand.
Measured while taking milestone 399's acceptance (#3898): 15 of the top 20
`preference_slot` near-misses were `<task-notification>` blocks, all matching
ONE record — #140 "Let each action land before starting the next" — all within
thousandths of the 0.70 floor. A notification that an action finished really
does resemble a preference about letting actions land. Lowering the floor to
serve that apparent demand would have injected that record into every
notification: the instrument arguing for the wrong fix, which is #379 again.
scribe_skip_prompt is a PREFIX test, not a substring one, and that is the
whole safety argument. A real prompt may contain one of these tags — an
operator pasting a transcript, or a `<system-reminder>` after typed words —
and must still be retrieved against. Nothing an operator types begins with a
client envelope. The compaction-resume injection is deliberately NOT filtered:
it is machine-written, but it summarises real work, and a resumed session is
where recalling a rule earns its keep.
Client-side only, so no server contract moves and lagging plugin caches keep
working. The tags are Claude Code protocol constructs, identical on every
install — instance-agnostic under rule 115.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
ci-requirements.md is the document that drives promoting a per-job dep into
the ci-python image, and it still recorded jq as installed in the plugin job
and "load-bearing for the smoke test specifically", on the grounds that every
hook opened `command -v jq || exit 0` and the test would otherwise pass while
exercising nothing.
That was the tail wagging the dog. The hooks ship to users; jq is absent by
default on macOS, the Debian/Ubuntu slim images, Alpine and most CI
containers, and a machine without it got no context, no rules, no prior art
and no process sync in silence. The answer was to remove the dependency, not
to install it harder.
Recorded as a promotion the entry now argues AGAINST rather than deleted: the
next person to read this file should find out why jq is absent, not merely
that it is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Three loose ends from a49e7ed, all found by CI or by re-reading it.
TWO TESTS PINNED THE jq SYNTAX, not the word `jq`, so grepping the suite for
the tool name missed them: test_write_path_trigger asserted the literal filter
strings `(.note_ids // []) - (.sync_note_ids // [])` and `(.rule_ids // [])[]?`
appear in the hook. Re-spelled with the readers that replaced them. The claim
each makes is unchanged — that the reuse and sync channels keep their own
state file, their own query parameter and their own write-back.
CI STILL INSTALLED jq, TWICE. The "Install jq for hook execution tests" step
existed because those tests skipped rather than failed without it; the Plugin
hooks job installed it alongside shellcheck for the same reason. Removing a
dependency from the product and leaving CI to install it anyway means the
smoke tests never run under the condition they exist to assert. Both gone —
the hook tests now run on a bare image.
`awk 'END { print $0 }'` IS UNDEFINED IN POSIX. gawk retains the last record
in END and mawk does too, but it is not guaranteed, and reaching for a
non-portable idiom inside the change whose entire purpose is portability is
the same mistake one layer down. Now `{ last = $0 } END { if (NR) print last }`.
Verified it still selects the same start line on a real transcript.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Every hook opened `command -v jq >/dev/null 2>&1 || exit 0`, so on a machine
without jq the operator got no session context, no rules, no prior art and no
process sync — and not one word saying why, because `exit 0` is
indistinguishable from "ran fine, nothing to say". jq is absent by default on
macOS, on the Debian/Ubuntu slim images, on Alpine and in most CI containers.
That is not a prerequisite to document; it is the plugin handing its own
packaging problem to whoever installs it.
`tac` was worse: GNU-only, so the prior-art hook's enclosing-definition arm
did nothing at all on every Mac, silently, from the day it shipped. It is not
replaced but removed — scribe_defs judges each line independently, so
extracting forward and taking `tail -1` is the same answer as reversing and
taking the head, and it drops the early-exit `head` that #4042 was filed for.
No server contract changed, so a lagging plugin cache keeps working.
scribe_json.awk JSON -> IDX<TAB>PATH<TAB>VALUE. Two modes: `whole` for an
event or a response body, `lines` for a transcript, where
an unparseable record is dropped and the rest still read —
the `map(try fromjson catch empty)` the jq program opened
with. Arrays also report their LENGTH at `[#]`, which is
what keeps "zero notes" distinct from "no answer" (#2932).
scribe_turn.awk the turn-bounding program, replacing the thirty lines of
jq in the Stop hook.
scribe_defs.sh scribe_json_flat / _pick / _list / _len / _list_minus read,
scribe_json_out writes the envelope (five copies of one
shape, gone), scribe_urlenc replaces `jq -sRr '@uri'`.
Percent-encoding goes through `od -tu1` rather than an awk character loop on
purpose: awk's idea of a character follows the locale, so gawk reads an
accented letter as one and mawk as two, and an encoder built on substr() would
emit a different URL depending on which awk is installed. Encoding is defined
on bytes. Verified byte-identical to `jq -sRr '@uri'`.
Measured, not assumed. The per-event path costs 8ms against jq's 3ms. The
transcript path was 70x slower until two fixes: the Stop hook now finds where
the turn starts with a fixed-string grep before parsing (a needle carrying
unescaped quotes cannot occur inside a JSON string, so it matches only at a
record's top level — checked against a full JSON parse of a 27MB transcript:
152 prompt records, 152 matches, no misses, no extras), and the parser reads
each token out of a 1024-byte window instead of copying the rest of the buffer
per token, which was quadratic in line length on the 400KB tool results a
transcript carries.
Differential-tested against the jq program it replaces over 724 windows cut
from three real transcripts — 724 identical, 0 mismatched, 45 of them
exercising a real task close and a real reply. That sweep is what caught
`scribe_turn.awk` never setting FS, which truncated every multi-word reply at
its first space and was invisible to a test whose replies were all empty.
check_plugin.py's `jq -R` lint becomes a guard against either binary coming
back, and three smoke checks lose their `shutil.which("jq")` skip. jq is not
in `ci-python` either, so those three announced a skip on every CI run and had
never once run there: removing the dependency from the product also closed a
permanent hole in its verification. They pass now across all ten hooks.
tests/test_hook_json_reader.py is a differential against Python's `json` over
nested objects, arrays, unicode, escapes, control characters, empty cases and
a value longer than the token window, plus the envelope, the encoder and the
turn analyzer. 139 cases.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Run 7068 failed two tests on a4aae97, and the second was a real bug the
guard caught before it shipped.
THE COLUMN GUARD (#3182) was right. `note_usage_events` excludes only
`id` from backup, so telemetry IS exported — and a new column that the
serialiser never learned about would have restored as null on every row.
That is the exact failure #3182 was built from, where nine columns
vanished from `notes` the same way: added to the model and the migration,
which fail loudly, and never to the serialiser, which fails silently.
So `project_id` is exported, not excluded, via the one `_usage_event_rows`
builder both exporters share.
The importer re-maps it through `project_id_map` and DEGRADES rather than
skipping. `code_shape_events` drops a row whose project will not map,
because a shape event without its project says nothing; a usage event is
not like that. The project is optional by design and null already means
"not reported", so an unmappable one restores as unreported. Dropping it
would lose a real pull and deflate the pull-through this table exists to
report.
BACKUP_VERSION deliberately unchanged: the suite states that column
additions do not bump it, only new sections do, and a third test asserts
the current number.
The second failure was mine and smaller — test_record_pulled_writes_a_
single_row compares exact dicts, so it now expects the null project. I
had claimed these assertions were key-based; they are not, and I had
read a summary line rather than the assertion.
Added an import-side guard while here. The column family checks the
EXPORT side only, so a column can be exported faithfully and dropped on
the way back in — a restore that reports success and has quietly lost a
dimension. No general guard for that direction exists yet; this covers
the column added here and names the gap for whoever builds the general
one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`RetrievalLog` has carried `project_id` since it existed, so "this record
was SURFACED on project B" was always answerable. `note_usage_events`
had none, so "this record was OPENED on project B" was not — and the two
cannot be joined to recover it, because there is deliberately no session
identity server-side. NoteUsageEvent's own docstring rules that out.
That gap sat exactly on the question milestone 385 exists to answer. A
lesson's whole claim is that it reaches a session on a project it was not
written on, and step 8's acceptance is "retrieved on a different project
AND opened". Each half was answerable; the conjunction was not.
WHICH project, because the name is ambiguous and the wrong reading makes
the column useless: it is the project the READER was in, never the one
the record belongs to. The record's own project is already on the note;
copying it here would answer a question nobody asked while looking like
it answered this one.
The surfacing half is free — every arm already holds the scope it just
searched, so auto_inject, lesson_slot, the write-path arms and
enter_project now record it. process_skill_sync does not and should not:
it installs every Process the operator can reach, which is not a
project-scoped question, so a project there would be a fiction.
The pull half needs the caller, since a getter knows only what it was
handed. The five single-record getters take `project_id: int = 0` and
pass it through, following the convention `search` and `create_*`
already set. Null stays an ordinary answer meaning "not reported" — a
pull with no project is still a pull and still counts toward dead
weight; it simply cannot speak to transfer. The four REST detail views
report none for now: a human opening a record in a browser is a
different event from an agent recalling one, and #2245 left that
asymmetry deliberately undecided.
Guarded the way #2245 and #2476 taught: by source inspection, because a
parameter that was never threaded through changes no return value and
shows up only as a column that is mysteriously always null. Three
guards — the signature, the pass-through, and the arms — plus the
can-fail test rule 167 asks for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The lesson kind shipped with every mechanism for growing and nothing
telling a session to use them. `learned_from` is a list on purpose, the
dedup gate hands back an existing id rather than minting a twin, and
`update_lesson` already names re-keying a bad trigger as the edit that
pays most. None of that was reachable as a habit.
The exclusivity claim was the bug. The skill said "a preference is the
one record you keep current yourself", and by naming only preferences it
put lessons outside the habit. That sentence is now "a preference is
yours to keep current", which says the same thing about preferences
without saying anything false about lessons.
Beside it, a paragraph on what growing a lesson means: another incident
added to what taught it, a claim stated more exactly, or a trigger
re-keyed to the situation that really fired. Written as a practice
rather than a prohibition (rule 165) — the reader is named as the one
person placed to judge the trigger, because they are standing in the
situation it claims to name.
`get_lesson` carries the same prompt at the moment it bites: a session
reading a lesson inside the situation it names is the only reader who
can tell whether the trigger is keyed to what actually fired.
The guidance-ownership registry gains the topic and re-points the
preference topic's statement, since the phrase it pinned is the sentence
this change rewrites — the module asks for exactly that, in the same
commit. No index marker: the index names session-start reflexes and this
one fires mid-work, so `_INSTRUCTIONS` stays at 1998/2000.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7059's typecheck, two errors on one line: `apiDelete` returns
`Promise<void>` and takes no type argument. I had given it the response body's
shape, which it discards.
Matched to how snippets delete, rather than adding a second delete helper to
carry the batch id — no caller has wanted it, and the second helper would be
the duplication rather than the feature.
Everything else in the UI batch typechecked clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 7's actual UI. Before this the frontend had zero lesson code — the kind
existed for agents only, which is rule 27 failing.
THE EDITOR ASKS FOR THE TRIGGER BY NAME, and leads with it. Three fields —
the trigger, the claim, the detail — never one markdown box. That is the
design step 1 settled, and the evidence is blunt: the snippet corpus carries
a trigger on every record with no guard anywhere, because a service composes
the title from a named parameter. What is at 100% is a named structured
field, not a writer remembering a convention. The trigger gets the most room,
its own explanation, and a save button that refuses without it and says why.
The form shows the composed title live, so the writer is agreeing to a
document they can read rather than one assembled out of sight. A 409 from the
duplicate gate is rendered as the record that already covers the moment, with
a link to improve it and an explicit override — not as a failure.
THE BROWSE VOCABULARY GAINS THE KIND, which #3161 warned this step not to get
wrong: a facet chip, a badge label, and routing to `/lessons/:id` rather than
the note editor, which cannot edit a trigger. The badge is neutral alongside
snippet and process — a hue would make the softest record in the corpus look
like the loudest, next to a rule that actually binds.
BOTH DIRECTIONS OF THE PROVENANCE. The detail page resolves `learned_from` to
titles rather than bare ids, because "#4181" tells a reader nothing about
whether it is worth opening. And `LessonsTaughtPanel` answers the reverse on
the record's own page — the direction the task body calls the one that gets
forgotten. It has no author to type it, which is exactly why it tends never
to get built. A component, not markup in the task editor, so the same panel
mounts on any record a lesson can cite instead of being written a second time
(#3207). Silent when empty: most records taught no lesson, and a panel that
says "None yet" everywhere is one people learn to skip.
GLOBAL-BY-DEFAULT IS MADE LEGIBLE. A lesson meeting you on a project it was
not written on reads as a bug unless the page says otherwise, so the origin
line says it as a property of the kind rather than as an apology.
Design system tokens throughout; no new raw hex. `--fs-error` rather than
`--fs-danger` — 31 uses against 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 7, part two. Milestone 385 built the lesson kind through the MCP tools,
which is the agent's surface. The Vue app speaks REST, so a lesson was a
record a person could not create, read, edit or retire — rule 27 failing at
the door rather than in the view.
`/api/lessons` now offers list, create, read, update and trash, plus
`/api/lessons/taught-by/<id>` — the reverse of `learned_from`, which the task
body calls the direction that gets forgotten and arguably the more useful one:
a reader opening an old issue wants to know what was learned from it, and
until now the relation was only navigable from the lesson's side.
`lessons_taught_by` reads `data[taught_by]` through `path_exists`, the same
jsonpath dialect the snippet location lookup uses, so both reverse lookups hit
the GIN index (0070) the same way rather than scanning bodies. Share-aware via
`readable_notes_clause`: it renders beside a record the caller can already
see, so a lesson shared with them belongs there exactly as their own does.
THE TRIGGER IS REFUSED WHEN EMPTY, at create and at update. This is the one
place the door is not a thin wrapper, and it is deliberate: the service will
store a triggerless lesson quite happily — it saves, reads correctly in every
listing, and never surfaces. There is nothing to notice afterwards, because it
looks exactly like a lesson that works. Better to refuse it than to hand back
a record that looks finished. The refusal says why, so the next reader does
not take it for a nag and delete it.
`lesson_to_dict` moves into the service and the MCP tool's `_to_dict` becomes
an alias for it. Both doors now return one shape — a payload spelled once per
door answers the two of them differently the first time a field is added — and
both compose through `services/lessons.py`, so a lesson written from the web
ranks identically to one written by an agent. The document IS what ranks, so
that parity is the whole reason the door is thin.
The dedup gate matches the MCP path: two lessons under one trigger compete in
a single ranked list for one reserved slot, so a duplicate here displaces
rather than merely clutters.
NOT DONE YET: this is the door, not the UI. #3734 stays in_progress until the
Vue views, the router entries, the Knowledge browse badge and the both-ways
sources panel exist — rule 27 is about the operator being able to touch it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Groundwork for step 7, and a data-integrity fix in its own right.
Two kinds keep a queryable mirror in `notes.data` derived from their body:
snippets and, since milestone 385, lessons. Every read prefers the mirror —
deliberately, because parsing markdown to answer what an index can answer is
how a hot path rots. So a write that moves the body must move the mirror.
#3128 found that hole for snippets and plugged it with a hard-coded
`if note.note_type == SNIPPET_NOTE_TYPE`. The plug was correct and did not
generalise: lessons arrived with the same design and none of the protection,
which is precisely the "don't add a fourth instance" defect #3734 was told to
avoid.
The cost is higher for a lesson. A stale snippet mirror reports the wrong
path. A stale lesson mirror reports the wrong TRIGGER, and the trigger is the
whole retrieval story — the lesson goes on firing for the situation it used
to name while displaying the one it now names. Silent, and confident.
So `update_note` now dispatches through `_mirror_recomposers()`, a
note_type -> recomposer table. A kind with a derived mirror is covered by
registering it, not by someone remembering to widen an if.
`lessons.recompose_data` is the lesson's entry. It recovers the subject with
`embeddings.untrigger_title` — new, and deliberately placed beside the join it
inverts rather than in the caller that wanted it, because a separator spelled
in two files is a separator that will one day be changed in one of them
(#3207). `TRIGGER_SEP` is now the one spelling, and `parse_snippet_fields`
uses it too; it had the third copy inline.
The two inverses stay distinct on purpose: a snippet partitions at the first
separator (its name is a symbol), a lesson strips an exact known suffix (its
subject may legitimately contain a dash). Different algorithms, one constant,
so they cannot disagree about where the seam is.
Provenance is DROPPED when the body drops it, which is the opposite call from
a snippet's `verification` — that is carried because it was never in the body
to delete. The body is the authority; carrying a value the reader just removed
is the failure the recompose exists to prevent.
Tests: test_snippet_mirror_generic_door.py becomes
test_derived_mirror_generic_door.py, since the concern is now plural. The
registry property is asserted directly (every kind with a mirror is in the
table; the dispatch names no kind inline), plus the lesson cases and the
join/inverse round-trip. `fake_lesson` moves to tests/helpers.py — it existed
in test_lesson_surfacing.py and a second copy was about to be written — and
gains the explicit `None`s `fake_snippet` carries, because update_note reads
`verify_with` and a MagicMock is truthy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The five-answer change touched plugin/skills/using-scribe/SKILL.md, and CI
7052 caught the manifest still reading 2026.09.18.1606. Per #2209 the
marketplace clone self-updates but the cache that actually executes only
refreshes on a version change, so without this the new guidance reaches the
repo and stops there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 6 of both milestone 385 (lessons) and 399 (preferences). #3896 asked for
the fourth and fifth answers in one pass, because two people each adding one
branch to a three-way distinction produce a list that does not read as a set.
create_rule now opens by asking what kind of thing is being held, with one
question that sorts it — what happens if someone doesn't do this? Something
breaks, a boundary is crossed: a rule. It gets done a way the operator didn't
want: a preference. They lose time rediscovering it: a lesson. The closing
question grew the two matching answers, and they are named as first-class
outcomes rather than places a proposal lands when it fails. An observation
that turns out to be a lesson has been routed, not dropped.
Stated as a practice, not a prohibition (rule 165). #3557's first cut opened
"NOT YOURS TO CALL UNPROMPTED" and cost the noticing; the wanted behaviour
here is still more proposals, and what changes is only which door they go
through.
create_note says the same from its side, so routing does not depend on having
opened create_rule first — and its existing rule test ("a mistake, not merely
uninformed") turned out to name the lesson exactly. create_lesson names the
fifth kind so the set is complete from every door. create_project_rule's
citation of the loop names five answers, since it cites rather than repeats.
The force axis has one owner (decision #4027): using-scribe states all three
strengths, the sorting question, that updating a preference mid-work is the
normal case, and that preferences shape how work is done and never what gets
recorded. _INSTRUCTIONS carries the pointer — "Rules bind; preferences guide
and you keep them current; lessons inform." It had 14 characters of headroom,
so the clause is paid for by trimming atmosphere from three other lines; 1998
of 2000 now.
Guards: the proposal-loop test learns the preference branch, the lesson
branch and the force question, on both rule surfaces; guidance-ownership gains
the force-axis topic (shared with the docstrings, for the moment a proposal is
actually written) and the preference-scope topic; a new guard pins that the
index names all three strengths and who keeps the middle one current, with its
can-fail case being the omission that actually happens — a kind added to the
product while the index still describes the corpus that came before it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
On 2026-09-19 a host storage stall made one Postgres checkpoint of 14
buffers take 281 seconds against a 1.3-second baseline. The app restarted
into the tail of it, `get_maintenance_hour()` — the first DB read in
`before_serving` — hung with no deadline, Hypercorn killed the worker at
its 60-second lifespan timeout, and nothing retries a failed lifespan. A
five-minute disk hiccup became a three-hour outage that only a human
restart could clear. Every MCP call returned 405, which reads like a
routing fault and was nothing of the kind: nothing was serving.
Three changes, none of which prevent a stall — they stop a transient one
becoming a permanent one.
1. THE STARTUP READ IS BOUNDED (rule 156). `get_maintenance_hour` already
answered `_DEFAULT_HOUR` for a value it could not parse; a database
that will not answer in three seconds is the same class of "no usable
value here". The failure is now a WARNING naming the symptom — the
breadcrumb whose absence meant this was only diagnosable from
Postgres's own log — and a default run-hour, instead of the app.
2. THE BACKFILL NO LONGER RACES STARTUP. Its comment said it "never
blocks the server from accepting requests": true of requests, false of
startup, because the task began while `before_serving` was still
running and competed for the same pool. Both of the incident's
cancelled statements were in flight together. It now waits on a flag
released on the hook's way out — in a `finally`, never after the work
(rule 157), because an undeadlined wait is only safe when the wake-up
cannot be missed.
3. THE ENGINE CANNOT WAIT FOREVER TO CONNECT. asyncpg's default is 60s,
the whole lifespan budget spent before a query is sent. `command_timeout`
is deliberately NOT set alongside it and the comment says why: it would
apply to every statement, and this app runs long ones on purpose.
tests/test_startup_survives_a_slow_database.py asserts the shape rather
than the stall: a read that never returns still yields an hour, the
warning names the symptom, a healthy read is unaffected, the backfill
does no work before release, the flag is released even when startup
raises, and the engine's connect args carry a deadline but no blanket
statement timeout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
WHICH ARM — the two note arms, and no new one.
`write_path` filters kinds, so a lesson was not outranked there but
unreachable, which is #3702's shape: an arm that never had the candidate
reports a healthy bar. It now asks for lessons alongside snippets and
issues. The founding example of the kind is a lesson about a code shape,
and this is the arm that fires when code is written.
`auto_inject` does not filter kinds, so lessons were already candidates —
but scoped to the bound project, which for a kind whose whole claim is
that it transfers is the same silence. Both arms now pass
`include_global_kinds` (#3730).
WHOSE BUDGET — a reserved slot in the prompt menu, none on the write path.
The step's premise needs a correction: the notes menu and the rule hints
are separate functions with separate budgets, so a line reserved here
displaces a note, never a rule. (`RULEHINT_LIMIT` is also 5, not 1, since
#4102 made it a default rather than a cap.)
The trade taken: a note crowded out is a lost convenience and a rule
crowded out still fires at an act arm, but a lesson crowded out is the
feature failing — a lesson exists only to be met at the moment it
applies, so the arm IS its delivery and the loss is total and silent.
That is `preference_slot`'s argument, and the rarity is `reuse_slot`'s.
It buys position, never a lower bar, and it EXTENDS rather than evicting:
a displaced hit sits in the general search's own log row, and evicting it
would make two tables disagree about one call (#3668, #379).
No slot on the write path: that arm fires before every Write and Edit,
where a guaranteed extra line is a guaranteed extra interruption, and its
field is already just snippets, issues and lessons rather than the whole
corpus.
`lesson_slot` logs its own retrieval and its own surfacing from the first
deploy, and the general contest stays open to the kind — otherwise "the
slot earns its line" would be true by construction.
THE VOICE — "they don't always have to be followed".
The menu's register is already the non-binding one. What it lacked is
that a lesson reads as one more title in a list of material when it is
advice someone paid for. One clause, in the header, only when a lesson is
on the menu: weigh it, use your judgement, it is not a rule and binds
nothing. It deliberately does not borrow the rule arms' "before deciding
it does not apply", and a guard asserts that phrase never appears.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`_REPORT_KINDS` gained `lesson` and `process` without `_KIND_SUGGESTION`
gaining either, so the report would have listed pairs with no advice —
and the suggestion is the report's point. CI's per-kind guard caught it.
Lesson: read both triggers first (a lesson is retrieved by the situation
it names, so alike insights under different triggers are two lessons);
same trigger means one lesson learned twice, so fold the `taught_by`
union into the survivor and delete the other. Process: a process arrives
as a skill, so a duplicate runs the wrong procedure rather than merely
cluttering a list — keep the one in use, fold the missing steps, delete
the loser.
The guard now asserts the property per kind rather than spot-checking
two: no non-snippet suggestion may propose merge_snippets, so a sixth
kind inherits the bar without anyone editing the test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Arising from #3731, which shipped a lesson with three of five tools and logged
the rest rather than widening its own scope. The operator's framing on reading
that: each kind deserves CRUD functions and to show up in the search and report
functions. So this fixes the property, not the two instances.
WHAT WAS MISSING FOR A LESSON: no delete, no list, and no duplicate report.
`delete_lesson` is the #2250 situation exactly — the trash is kind-agnostic so
`delete_note` always reached a lesson, but nothing said so, and a kind whose own
tools offer create/read/update reads as one you cannot retire. `list_lessons` is
the only way to ask what has been learned at all: `get_lesson` needs an id you
already have, and semantic search returns what resembles a query, never the set.
APPLYING THE RULE FOUND THE SAME REPORT GAP FOR PROCESSES, which have had full
CRUD for months and have never been in `_REPORT_KINDS` either. Both are in now,
each compared only against its own kind.
The lesson report default is 0.90 — the general semantic floor, deliberately
BELOW its own write-path bar of 0.96. The gate is permissive on purpose so it
does not refuse two genuinely different lessons whose triggers read alike, and
that tolerance is precisely what wants reviewing later, so the report looks at
the band the gate was told to let through. Safe there and not at the gate,
because a report proposes and the operator picks where the gate blocks a write.
A BUG CAUGHT BEFORE IT SHIPPED: `list_lessons` first read the trigger from
`it["data"]`, which `_note_to_item` does not carry — it projects named keys off
the mirror (`language`, `verification`) rather than the column. Every row would
have listed an empty trigger, which on a kind whose whole point is the trigger
is the failure looking like the feature. `when_to_apply` is now projected there
beside the others, so every listing surface gets it, including step 7's UI.
The guard asserts the PROPERTY rather than the instances: for each typed kind,
all five tools exist, are actually offered by register(), are classified for
auth, and the kind has a duplicate report. Derived from the kinds themselves, so
a fourth inherits the bar. A per-tool test cannot catch a missing tool, which is
why four steps of milestone 385 went green over this.
`find_duplicate_records` now validates against `_REPORT_KINDS` instead of its
own literal — the second copy is what would have refused a kind the service
already supported.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 385 step 4 — the write path.
ITS OWN TOOL MODULE, not create_note(note_type="lesson"), on the snippet and
process precedent and for the reason that precedent exists: a kind whose value
depends on one field being filled needs a door that ASKS for that field by
name. create_note would take a lesson through a generic body parameter and the
trigger — the whole of why a lesson is findable — would be something the writer
had to know to include.
THE TRIGGER IS REQUIRED, refused rather than flagged. Step 1 left the choice
open. Refusing is right for the same reason create_rule makes enforcement the
deciding question: a lesson with no trigger is not a weaker lesson, it is a
note that will never surface, and nothing downstream can tell the difference —
it saves, reads correctly in every listing, and is silently absent from the one
moment it was written for. A flag is a warning nobody is present to read; the
write path is where the writer still is. The message says SYMPTOM, because
"required" alone produces a topic where a situation was wanted.
The docstring carries the distinction this milestone exists to fix, in a line a
reader can apply: the difference between a lesson and a rule is FORCE, not
importance. If ignoring it would be a mistake it is a rule and needs the
operator's yes; if ignoring it just means someone re-derives it the slow way it
is a lesson, and nobody is bound.
CARDINALITY: a LIST, in notes.data under `taught_by`. The founding example
generalised three incidents into one claim about failure classes no CI lane can
see — generalising across incidents is the shape a good lesson HAS, and
arose_from_id holds one, so a single id keeps the first and drops two while
reading as complete. It lives in `data` rather than a join table for the reason
decision #4157 put the trigger there: a table would settle, for every note kind
at once, whether provenance is multi-valued — a question nothing has measured.
`arose_from_id` is filled only when there is exactly ONE source, because every
surface that renders it renders it as THE origin, and one of three would make
those surfaces state something false.
THE DUPLICATE GATE, which step 4 asked to check: a lesson is judged at a bar
ABOVE the sibling band, not the general 0.90. #2518 measured deliberately
parallel variants at 0.92 on a document that is mostly prose about the thing,
which is exactly a lesson's shape now — so at 0.90 two genuinely different
lessons about one area ("CI cannot see this class of failure") would refuse each
other. Its own constant rather than reusing the snippet's: the two are separate
facts that coincide today, and this number is inherited from a structurally
analogous corpus rather than measured on lessons, of which there are none yet.
Follows canon #2846 including the third registration point it names and this
change would otherwise have missed: get_lesson is in server._READ_ONLY_TOOLS
and the two writers in _WRITE_TOOLS, which test_mcp_auth requires.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Plugin reply shape, preferences UI, cited-record status, and the lesson kind
Merged to ship the plugin: the marketplace checkout tracks main, so
#4153's reply-shape rewrite and #4154's "a record you only mention is a
record to read" could not reach a session from dev. Manifest moves
2026.09.18.0436 -> 2026.09.18.1606, so the executing cache refreshes.
Riding along: #4154 (next_step on milestone summaries, status in the
injected kind marker), #3895 (preferences writable + drift visible), and
milestone 385 steps 2-3 (#3729, #3730) which are inert until a lesson can
be written in step 4.
CI 7023 green on 1361ed7, all six jobs.
Milestone 385 step 3 — the step where the kind either works or is cosmetic.
THE DOCUMENT, and why there is no `lesson_document()` beside `rule_document()`
in embeddings. The step expected one. The difference is where the sharp shape
LIVES. A rule keeps its trigger in a column and its title is a plain name, so
`{title} — {trigger}` has to be synthesised at embed time and exists nowhere
else. A snippet — the only sharp record in the corpus by #2485's measurement,
0.153 top-to-second against 0.010–0.023 — gets there the other way: its STORED
title is already the join and its stored body already opens with the trigger,
so the ordinary `title\nbody` join IS the sharp document. Step 1 chose the
snippet route and step 2 built it, so `lessons.lesson_document` composes what
is STORED and the generic chunker does the rest.
The consequence the step asked about: `chunk_document` is untouched, so
CHUNKER_VERSION does not move and NOTHING re-embeds. The step's "Re-embed"
section describes a change this design does not make.
THE NARRATIVE stays in the body, departing from the step's instruction to keep
it out. `rule_document` excludes `why` because long dated narrative made
sixteen dev-logs land on the centroid of "development" — but that finding
predates chunking (#280). A body over budget is now split, and every chunk is
prefixed with the title, which for a lesson carries the trigger. The story
occupies its own vectors instead of averaging itself into the trigger's, and
each of those is still anchored to when the lesson applies. A guard asserts
exactly that. Holding the story out would cost the reader the only part that
explains the insight, to buy a sharpness the chunker already provides.
GLOBAL IN THE SEARCH is the real new code: `GLOBAL_NOTE_TYPES` and
`include_global_kinds` on `semantic_search_notes`, widening the PROJECT filter
alone. Off by default, because two callers depend on that filter holding — the
near-duplicate gate compares a record only against its own project on purpose,
and a globally visible kind there would let a lesson block an unrelated note's
create on a project its author never touched. It composes with `note_type`
rather than overriding it, so narrowing to snippets does not quietly acquire
lessons, and it changes nothing about the ACL: `notes_visibility_clause` still
gates every row.
Wired into the explicit MCP search only — the operator asked, and there is no
budget to spend. The unasked-for injection arms are step 5's subject (#3732)
and the legibility of a lesson appearing on a foreign project is step 7's
(#3734), so neither is turned on here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7016. `test_non_task_facets_are_the_note_types_and_only_those` pins the
non-task vocabulary as a literal set, so adding `lesson` to `_FACETS` turned
it red — the guard working, not breaking. It stays a literal: derived from
_FACETS it would assert nothing, and rule 167 wants a guard that can fail.
The representative corpus had no lesson row, so `lesson` was reaching only
the two tests that iterate FACET_TYPES and never the one that asserts each
facet selects EXACTLY its own rows. It has one now, which is what pins the
half that matters: a lesson is not picked up by the Notes facet despite
both being non-task records.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 385 step 2, implementing decision #4157 from the step-1 spike.
The kind: `note_type='lesson'`, a note findable by WHEN IT APPLIES rather
than by what it is about. The trigger lives in `notes.data.when_to_apply`,
mirrored into the title and the head of the body — the shape snippets
already use, and the reason nothing re-embeds: chunk_document is untouched,
so CHUNKER_VERSION does not move.
NO MIGRATION, and the step assumed there would be one. `note_type` carries
no CHECK — only `task_kind` does (0056, 0065). Migration 0036 added it as
plain Text with a server default and nothing has gated it since, so rule 36
has no whitelist to expand and #3128's failure mode (a value the database
refuses) cannot arise for this column. The vocabulary that actually decides
what a reader can reach is services.knowledge._FACETS, which since #3161 is
one table feeding the door's validation, the counts and both dialects of
the type filter — so the kind lands there in a single edit.
ONE JOIN, not a fourth copy. `{subject} — {trigger}` had three
implementations: rule_document, snippets.compose_title, and this step
needed another. #3207 records what that costs, so the join moves to
embeddings.trigger_title beside embedding_text and all three delegate.
Behaviour is unchanged for rules and snippets; the guard calls each through
its own public name, so a re-implementation fails it.
The #3163 bill is stated in the service docstring rather than left to be
inferred: versions, supersession, trash, the share ACL, tags, project and
System tagging, chunked embeddings and the duplicate gate are all
inherited; status/task_kind/milestone_id and recurrence are not, and
verify_with/expires_when are available but outside the kind's contract.
The status cell is the one that matters — `is_task` IS `status is not
None`, so a lesson that acquired one would become a task.
The integration guard asserts the WRITE rather than the constraint: it
holds whether or not note_type is ever gated, and goes red only if it is
gated without this value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 399 step 5. Steps 1-4 put preferences into the backend: a kind
column, an inverted write path, a third register in the injected block, a
delivery slot. Nothing the operator could touch. Rule 27 forbids leaving it
there, and here it matters more than usual, because the UI is the only
guard against the risk the milestone named up front — an agent misreads one
session, rewrites a preference, and follows the rewritten version forever
while the operator never sees the moment it changed.
Four things ship.
A preference is DISTINGUISHABLE. `kind` reaches the client (the server has
always sent it in rule_brief) and a preference carries a chip. Force is the
one thing a list of instructions must not leave the reader to infer, and a
row that renders identically to a rule teaches the opposite of both facts
about a preference: it does not bind, and a session may rewrite it.
A preference is WRITABLE. The editor gains the kind as a first-class choice
with the test beside it — what happens when someone does not do this — and
says plainly, when preference is chosen, that sessions rewrite these
without asking and every rewrite is kept.
DRIFT ARRIVES. `GET /api/rules/drift` returns one row per rewritten
preference carrying its latest rewrite: what it said, what it says now, and
the record named by `arose_from_id` that taught the change. Both texts ride
along so the list shows the diff without a call per row. The new pane sits
beside the staleness sweep, because drift belongs to no one rulebook, and
it answers a question the operator would not have thought to ask.
REVERSION IS ONE ACTION, and this is the carve-out worth arguing with.
Milestone 323 refused a one-click restore for rules — "a binding
instruction should not be revertible in one click", because a silent revert
erases the only record of why the rewrite happened. That reasoning turns on
the rewrite being the operator's own decision. A preference's is not: the
agent makes it mid-work without asking, so reverting is a veto over someone
else's edit rather than an undo of your own, and a veto costing more than a
shrug is not supervision. The route refuses anything but a preference (409),
and nothing is erased: the restore goes through update_rule, so it snapshots
too and the history GAINS the revert. An integration test pins that, because
it is the whole basis for the exception.
Tested against real Postgres — every claim is about which rows come back
and in what order, which a stand-in session cannot judge.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The new fixtures fed (milestone_id, status, count) and the query also
carries max(updated_at), which last_touched_at is computed from. Six
tests in the new module died unpacking it; the product path was never
reached, and the rest of the suite was green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 1 made placement cheap for a task whose status CHANGES: create_task
and update_task return where it sits, and the report is written from
that. It did nothing for a task a reply merely cites.
This milestone's own step-6 review reported "#4014 is the open step of
milestone 409". #4014 had been done for four days; the open step was
#4015. The id did not come from a read — it came from a retrieval hint,
which carries an id, a kind and a title and says nothing about status,
while list_milestones said "8 of 9" and would not say which one. The
gap was there to be filled and the nearest-looking id filled it.
Two surfaces, one principle: the status arrives with the id.
1. get_project_milestone_summaries gains next_step — the earliest open
step, {id, title, status} or None — carried through _BRIEF_FIELDS to
enter_project, get_project and list_milestones. One extra flat query
for the whole batch, so #2384's fan-out does not come back.
OPEN_STEP_STATUSES moves to services/milestones.py and placement.py
imports it; both surfaces now answer "what is next" and must not
drift on what counts as open. Both step queries take the same
readable_notes_clause (rule 78), so a row cannot name a step its own
progress numbers exclude.
2. _record_kind renders a task's status: [task (done)], [issue (todo)].
A finished step and an open one read identically before, which is
exactly the line the misreport was taken from. Only tasks — is_task
IS status-is-not-None on the model, so there is no fallback branch.
reporting-back gains the practice, owned and registered in the guidance
ownership table: a record you only mention is a record to read.
The guards are structural and each fails on the regression it names:
the query count is asserted rather than the payload shape, and the two
surfaces' agreement is pinned on the rendered ORDER BY, since a mocked
session hands back whatever order the test chose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 409 step 6 measured the scaffold on live sessions and found its
two halves disagreeing: adherence passed and the read test failed.
Completion replies carried every section the table asks for and were
still hard to read.
The cause was in the skill, not in compliance with it. It said to pick a
kind of reply "then fill its sections... keep them even when one is
short", which is an instruction to complete a form, and nothing anywhere
set a ceiling. A faithful reply and an unreadable one were the same
reply.
Four changes to the discipline around the scaffold. The categories and
their sections are untouched.
- Sections are what to consider including, not a form to complete. A
section answering a standing question ("does anything need me?") is
always answered, even with "nothing"; a section that explains earns its
place only when it changes what the operator does. Otherwise it belongs
in the record's log, where it is available and not in the way.
- Write the shortest reply that carries the answer, with named exceptions
so this cannot be read as "always be terse".
- "Needs you" takes BOTH tests: theirs to decide, AND work is waiting on
it. A question answerable by reading something or taking an available
measurement is work not yet done, not a request — settle it, say which
way you went, and leave them free to overrule.
- A decision already made gets acted on. Re-arguing a settled question
reads as contradicting yourself rather than as being careful, and costs
the operator the decision twice.
"Before sending" gains a second pass for what can go, since the existing
check asks what is MISSING, which a bloated reply passes.
Guards in tests/test_reply_discipline.py, three topics registered for
ownership. Every guard was falsified against the pre-change text before
committing (rule 167): all five fail on it and pass on the fix, and the
sixth deliberately passes both since it guards the scaffold against
collateral damage. No absence checks — the skill legitimately discusses
filling in order to warn against it, so asserting "fill" is absent would
false-alarm on the corrected text (snippet #3352).
Instance-agnostic per rule 115: the added text carries no record ids, no
software-specific terms and no verbatim quotes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The tuning loop shipped in steps 4 and 6 and logged zero events in its
lifetime. `tune_retrieval`, `retrieval_telemetry` and `retrieval_surfaces`
appeared on no instruction surface at all — not the skills, not the hooks,
not the MCP instructions — so the decision that "the model should be the
thing handling it 9 times out of 10" could not begin to happen.
What was missing was not an auditor but a route. `using-scribe` now carries
it, ordered: read the refused records, fix the trigger, and only then
consider the dial. The order is the content. A rule's `when_to_apply` IS
the text its score is computed against, so a miss is evidence about that
text first; rewording one trigger changes one rule's reach, while moving a
floor changes what every record on the surface does and cannot tell a
badly-worded trigger from a genuinely distant one.
Measured, and the reason the order is asserted rather than suggested: rule
1 scored 0.6515 and ranked 5th for the moment it governed, behind three
rules that restrained the same act. Every percentile said "lower the
floor"; at 0.60 the arm delivered those three restraints and still not rule
1. Rewriting the trigger to lead with the symptom put it 1st at 0.7130.
Also: the create path gets a precondition. A new record is itself a
retrieval-affecting act, so before writing one, what_might_apply asks what
already covers that moment — fifty candidates and no bar, because a bar is
what lets the existing record hide.
`_INSTRUCTIONS` gets one index line, not the route: 1,986 of 2,000
characters, since Claude Code cuts the rest mid-word (#2562).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`current_settings` read a surface's history with one query and `limit(2)`, then
keyed the rows by dial. That is only the same thing while both dials have moved
equally often — and they do not. Floors get walked; budgets rarely move. Three
floor changes and one budget change returns two floor rows, and the budget
change vanishes.
Under #4102 that cost a missing reason. Since #4104 it is worse: the dial then
reports `source: "shipped"` — still on the value Scribe shipped — for a number
somebody deliberately tuned. A wrong calibration answer, in the direction a
reader has no cause to double-check.
One query per dial, `limit(1)` each.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Two failures in the tests added with the step, both mine.
`make_mock_session` is an AsyncMock and every child of an AsyncMock is one too,
so leaving `.all` as it came handed `migrate_floor` a coroutine where it reads a
list — the same trap the helper's own docstring already flags for `add`.
And the registration test enumerated three tools by name, which is exactly what
it is for: `migrate_retrieval_floor` made it four.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 416 step 6. A retrieval floor is a cosine similarity, which only
means something inside one embedding model's geometry over documents cut one
particular way. Change either and every floor on the install keeps applying
while describing nothing — and nothing anywhere says so, because the scores
simply come out different and the bar goes on cutting.
`CHUNKER_VERSION` already solved this for documents: stamped per row, so the
backfill re-embeds precisely what is stale. The same idea, applied to the
numbers:
- `calibration_stamp()` — embedding model + document shape, one definition.
TWO fields, never a fused string (rule 149): a mismatch has to say WHICH half
moved, because they call for different responses.
- `retrieval_tuning_events` gains `embedding_model` / `shape_version`
(migration 0104), stamped on every write. Nullable and NOT backfilled —
"unstamped" is the honest answer for a row written before this existed, and
it reports as `stale: null`, never as fine.
- `current_settings` reports calibration per dial: tuned rows from their event,
untouched dials from the registry default's own stamp.
- `retrieval_surfaces` and the Settings panel show the mismatch. The panel
renders ONLY when something is stale, so seeing it at all is the signal.
- `migrate_floor` / `migrate_retrieval_floor` answers "a path for thresholds to
be inherited by the next model so that they don't have to recalibrate a lot":
the raw cosine cannot cross models, but the PERCENTILE it represented can.
Measure what fraction of a surface's logged calls the old floor admitted,
re-score those queries under the current model, take the value admitting the
same fraction. Dry run by default; applying writes an ordinary tuning event
with the arithmetic in its reason.
Nothing auto-retunes. A stale stamp says a number is no longer a measurement;
it does not say what the number should be, and #4102 measured the one case
where the statistic and the correct action pointed opposite ways.
The load-bearing test is an ABSENCE: no chat-model identifier may appear
anywhere in the calibration path. Claude produces none of these scores, so a
Claude upgrade must trigger nothing — a false alarm here teaches the operator
to ignore the real one on the day bge-small becomes bge-base.
Backup v17 carries both columns, unfilled on the way out and on the way back:
a round trip must not turn "we don't know" into a stated fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 6973 — `test_guidance_ownership` on "rules are retrieved; ask before
a consequential act": the using-scribe skill was missing the marker
`content_type="rule"`.
It was there. I had hard-wrapped the new bullet across `content_type=`
/ `"rule"`, and that guard matches its MARKERS raw while normalising
whitespace only for the longer `statement` — which is why the failure
reported the statement present and the marker absent in the same
breath. Rewrapped so the token sits on one line.
Worth noting rather than just fixing: the markers are deliberately
raw-matched, since a marker IS a literal a reader copies. `tool_doc`
flattens for exactly the opposite reason one layer over. Both are
right; the trap is that they differ.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 416 step 5. The operator's compromise — "if you're worried
about excluding potentially important data let limit it to 50 entries"
— moved to the surface where it is safe. Fifty in the push would be
milestone 394 with extra steps; fifty in a pull crowds nothing out.
`what_might_apply(query)` returns ranked rule candidates with NO
threshold. The moment it serves is the one where the caller does not
trust a bar to decide for them, so it does not have one — every row
carries its score and the reader judges.
WHY IT IS NOT A BIGGER `limit` ON `search`
`_search_rules` returns statement, why and how_to_apply in full, on the
stated reasoning that a caller who went looking deserves the whole
record. That is the DEEP pull and should stay that way. This is the
SHALLOW one — many candidates, each just enough to decide whether to
open it. Opposite trade-offs, so it is a second tool.
THE PREMISE THE STEP GOT WRONG
The task said fifty "costs nothing". `_rule_hint_line` had already
measured otherwise: ~143 tokens per line once the trigger is rendered,
and #3855 tripled trigger lengths across the corpus. Fifty is ~7,000
tokens — cheap next to an arm firing before every Bash call, but not
free, and a tool promising a free wide net gets reached for casually
and then regretted.
So it reuses the graduated shape #3851 measured for the push: the top
few carry their trigger whole, the rest carry a cut of it. TRUNCATED,
never dropped — the trigger is what lets a reader judge without
opening, and a teaser without one is just an id. The cut borrows
`_goal_line`'s technique including the fallback that matters (#4036):
`textwrap.shorten` returns a bare "…" for one unbroken word.
TELEMETRY
Logged under its own pull source, asserted absent from both the tunable
push registry and AMBIENT_SOURCES. That guard is load-bearing right
now: the push arms' near-miss distributions are the evidence #4121
argues from, and a pull folded into them would move those numbers.
INSTRUCTION SURFACES
The using-scribe reflex and the MCP instructions both pointed at
`search(content_type="rule")` for the consequential moment — the deep
tool, at the moment you want breadth. They now point here, and keep
`search` for reading a rule you already suspect. Written as a practice
rather than a prohibition (rule 165).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Both said the repeated read-and-clamp was "canon #2860 across 295 of 372
siblings". Wrong on both halves, and checked rather than assumed:
#2860 is "Scribe service function — the async_session unit", the
service-layer canon, and its instance count is that canon's dominance in
src/scribe/services — nothing to do with a threshold helper. The
divergence hook flagged the new module against it because the new module
is a service, which is the hook working correctly and me misreading it.
The duplication itself was real and is what the registry consolidated:
`plugin_context` read and clamped the pair twice over, three rule arms
did it again, and `reply_preferences._threshold` once more. The comments
now say that, and claim nothing about a ledger count.
`rule_usage.py:122` cites #2860 correctly as the service canon and is
untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The last item in this step's done-when: no wording anywhere asks an
operator to tune for correctness.
Four Settings hints told them to do exactly that — "raise it if rules
keep arriving unread", "lower it if a git push arrives with nothing",
"lower this if genuine duplicates go unnoticed". Every one of those
asks the operator to diagnose a ranker from symptoms, which is the job
the model now does from the records: the telemetry says what each bar
refused, and reading those records is what separates a real miss from
a bar doing its job. The hints keep the explanation of WHAT each number
is — that is worth reading — and drop the homework.
`plugin_context.py` said the defaults "are meant to be tuned from
retrieval_logs once data accrues", which was true and had no owner.
It now names who does it and with what.
`retrieval_telemetry`'s docstring gained the warning that belongs
beside it: this readout has been measured pointing the wrong way, so
the ids it returns are the point, not its percentiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The other half of the bargain in milestone 416 step 4. The model moves
these dials; this is what makes that reviewable rather than merely
automatic.
THE HOLE THIS CLOSES
Every retrieval floor is an ordinary settings key, and `/api/settings`
accepts any key at all. A floor written through it landed correctly and
recorded nothing — a tuning history with holes in it, which is worse
than no history because it reads as complete.
So the generic endpoint now routes registry-owned keys through
`set_dial` instead of writing them as plain rows. ROUTED, not refused:
refusing would only work for callers that had been updated, while this
way the form, a script, and an old client all leave the trail, and
there is no version of "forgot to use the other endpoint". Clearing a
control is written as an explicit set back to the shipped default,
because the operator reverting something is the single most important
move this history can record.
`set_dial` now also refuses to record a no-op. The Settings form
re-sends every field on every save, so without that one press of Save
would write six rows saying the operator set six dials to the numbers
they were already on — and a history nobody can skim is one nobody
reads.
WHAT THE OPERATOR GETS
`/api/retrieval/surfaces`, `/surfaces/<name>` and `/tuning-history`,
with `actor` fixed server-side rather than taken from the payload: a
payload-supplied actor would let a model claim to be the operator, and
"did I do this, or did the session?" is the first question this list
is asked.
In Settings: the five missing BUDGETS (until now only auto-inject had
one, so the only control over a noisy surface was to raise its bar —
which discards that surface's best candidates along with its worst),
and a "What has been tuned" panel showing each change, who made it, and
the reason given. The operator's own changes are marked.
The MCP tool demands a reason; these endpoints do not. That asymmetry
is deliberate and stated in routes/retrieval.py: the requirement exists
to make the MODEL read the records before moving a number on someone
else's behalf, and the operator is that someone — a mandatory
justification box on every control would be friction charged to the one
participant who owes no explanation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 6962 — `test_every_registered_tool_is_classified_exactly_once`. The
backup gap is closed (1692 passed); this is the next guard, and the same
shape of one: a tool registered without a classification is silently
denied to a read key, with nothing to notice (#3191).
`retrieval_surfaces` and `retrieval_tuning_history` read. Read access
matters more than usual for these two — a session that cannot see the
bar in force, or the reason it was last moved, is one that will move it
again blind.
`tune_retrieval` writes in both senses: the number the arm reads, and
the reason appended to the audit trail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 416 step 4's write half. `retrieval_surfaces.py` made the six
push arms describe their `{floor, budget}` the same way; this adds the
three MCP tools that let the model READ that and change it, and the
backup sections that carry the reasons.
The operator's decision, which this implements:
"the floor should be chosen and adjusted by the model using it… the
user should be able to touch it but the model should be the thing
handling it 9 times out of 10."
WHY A REASON IS REQUIRED, AND WHY THE TOOL ARGUES AGAINST PERCENTILES
The milestone originally listed self-tuning as a non-goal on one
measured case, and that case is now the tool's docstring rather than a
prohibition: `report_preference` logged 69 consecutive declines with
the refused record 0.0006 under the bar, and every percentile said
"lower it". The refused record was rule 77 "Extract intent from loose
phrasing" matched against a query about report layout — a false
positive. Lowering would have attached that rule to every completion
report ever written.
What separated the statistic from the correct action was OPENING the
record. So `tune_retrieval` refuses a blank or perfunctory reason,
tells the caller to read `retrieval_telemetry(near_miss_samples=5)`
and the record ids it names, and carries that 69-decline example — an
abstract warning loses to a number. The non-goal that survives is
*statistical* auto-tuning; nothing here reads a percentile and picks a
value.
BACKUP (v16), which is what CI caught
`retrieval_tuning_events` was neither backed up nor excluded, and
#2293's guard said so. It is backed up: `settings` already carried the
numbers, so dropping this would restore an install with six moved
dials and no argument for any of them — precisely the state the table
exists to prevent, and worse now that the model is the one moving
them. One `_retrieval_tuning_event_rows` builder called from both
exporters (snippet #2851); `user_id` travels because a restore has to
remap it, which is why the row builder is not the model's `to_dict()`.
`surface` is a registry name rather than a foreign key, so the history
survives a restore into an install whose ids all differ.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI on 09b4845 caught both, and the second is an improvement rather than a
repair.
`test_services_reply_preferences` patched `reply_preferences.get_setting`,
which the registry refactor removed — its floor and budget are resolved through
`retrieval_surfaces` now. Its key-independence test also asserted the arm asks
for exactly one key; it asks for two, because both of its numbers are its own
since this step, so the assertion names both and adds a check that the registry
and the module constant still agree about the floor key. They writing different
keys is the failure where the Settings form saves one string and the arm reads
another.
`test_settings_defaults_agree` parsed `plugin_context.py` for a bare
module-level float, and those constants now alias the registry. Rather than
teach the regex about aliases, the six retrieval floors are keyed on their
SURFACE NAME and read from the registry directly — which is strictly better for
this guard: a surface name is also its telemetry source, so a row names the same
arm the readout does, and a floor cannot be checked against a stale constant
that happened to keep its old value. `PLAN_MATCH_DEFAULT_THRESHOLD` is not a
push surface and keeps the older shape, with a note saying why.
Added while there: `test_every_tunable_surface_has_a_control`, derived from the
registry, so a seventh surface arrives as a failing test rather than as a number
only the model can reach (rules 25, 27).
Also lands the audit trail this step needs — `retrieval_tuning_events` (model +
migration 0103) and `services/retrieval_tuning.py`. The tool layer on top is
the next commit; the table is here because `reason` being REQUIRED is the whole
guardrail, and the schema is where that starts. A number moved silently leaves
nothing for the operator to review or disagree with, and the operator's decision
is that the model moves these "9 times out of 10".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Groundwork for the step's real change. The operator's decision is that the
floor is chosen and adjusted by the model using it, not shipped as a value
somebody has to defend:
"we need a model consistent surface for the adjustment of these floor values.
the user should be able to touch it but the model should be the thing
handling it 9 times out of 10."
A tuning surface cannot be consistent across six arms that each spell their
configuration differently, so the arms stop owning their numbers.
`retrieval_surfaces.SURFACES` names each one, its floor key and default, its
budget key and default, and — because they are rendered by the tuning tool and
the Settings UI — what it asks, over what corpus, and how often it fires. A
floor cannot be moved responsibly by anyone who does not know those three.
Three things fall out:
- **`k` becomes a real budget everywhere.** Only auto-inject had a configurable
one; `RULEHINT_LIMIT`, `PROMPTRULE_LIMIT` and `reply_preferences.LIMIT` were
constants. `k` is what binds under a low floor, so it has to be settable per
surface — and per surface is the point, since `pre_tool_rule` fires before
every Bash call while `prompt_rule` fires once a turn.
- **`write_path` gets its own budget, inherited not reset.** It shared
auto-inject's outright on the argument that "how many titles at once" means
the same thing on both. It does not, for the same reason. Unset, it still
reads auto-inject's key, so an install that tuned the shared knob does not
silently drop to a new default.
- **The duplicated read-and-clamp goes.** That shape is canon #2860 across 295
of 372 judged siblings. Survivable while the numbers were constants; not once
they are meant to move.
The long measurement comments stay exactly where they are — #2223's noise-floor
probe, #3853's command-vs-code split, #3851's band measurement. The constants
they annotate now alias the registry, so there is one value and the reasoning
still sits beside it.
Tests build the write-path config from the registry (`helpers.writepath_cfg`)
instead of from hand-written dicts. That is not tidiness: the rule arms read
their numbers inside a fail-open `except`, so a dict missing one key does not
raise where a reader would see it — the arm silently becomes a no-op that reads
exactly like "fired and found nothing". Ten hand-written dicts each looked
complete on the day they were typed.
tests/test_retrieval_surfaces.py pins the identity everything rests on: a
surface's name IS its telemetry source. Nothing in the type system says so —
`record_retrieval(source="pre_tool_rule")` is a literal in another file — and
renaming one without the other yields an arm that can be tuned and not
measured, or measured and not tuned, with no symptom either way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Both were mine, and both show the change behaving as designed.
`test_session_dedup_marks_the_reuse_arms_rather_than_silencing_them` asserted
#12 stays out of the search's `exclude_ids`. It does not, and should not: the
place arm has just listed it, so the semantic arm must not list it again. That
is the same-call rule, not the ledger. The claim I meant — the LEDGER never
reaches the query — is asserted on its own against a record the call has not
otherwise rendered.
`test_a_pulled_snippet_already_seen_is_evidence_not_menu` expected #7 and #8.
It gets #7 alone, because #7 (0.91) now anchors the margin band at 0.81 and #8
scores 0.80. Withholding #7 used to promote a materially weaker hit into a slot
it had not earned, with nothing in the output saying so — the band measures
distance from the best answer, and letting the ledger decide which answer that
is was the axis confusion the rule band was written to avoid.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI on 5c64ea0 surfaced a half-measure I had shipped and three tests pinning the
semantics it replaces.
**The half-measure.** Only the semantic arm was marking repeats; the write
path's PLACE arm still dropped a nearby snippet that was on the ledger. One
menu, two rules, decided by which arm happened to find a record — and the
marker would then read as a complete account of what the session has met while
covering half the lines. `seen` now means this call's own menu and nothing
else, so both reuse arms mark and neither withholds.
**test_session_dedup_excludes_ids_from_the_reuse_arms** asserted the drop and
the ledger going into the query. Both are the defect; it is renamed and asserts
the marker instead.
**test_a_pulled_snippet_already_seen_is_evidence_not_menu** asserted #7 was
evidence and NOT menu, with the limit widened by one to pay for the drop. The
premise was the hard exclusion; #7 is now both, and the widening is no longer
needed for it.
**test_a_record_this_arm_withheld_itself_is_not_a_near_miss** is the
interesting one. Its property still holds and its setup no longer reaches it:
a ledger repeat comes back from the search and is rendered, so nothing is
withheld and `best_available` describes the bar. The one post-search filter
left is a PULLED record this same call already listed by place — kept in the
query because `resembles` needs its score, dropped from the menu because it is
already on it — so the test is rebuilt on that, and a sibling test pins the
direction that changed, so nulling the score whenever the ledger matches
cannot pass both.
Also stubs `superseded_ids` and `owner_names_for` in the new test file, which
were reaching for Postgres in a unit job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI caught two things in c61f730, and the second is the one that mattered.
1. `scribe_autoinject.sh` keeps its note ledger in `${TMPDIR}/scribe-autoinject`,
not `scribe-priorart`. Every ledger NAMED in the hooks was in the one
directory the sweep visited, so it read as complete — and the arm that fires
most (598 calls in five days) was the only one still carrying the bug. The
clear ran, found nothing to remove, and exited 0. `SCRIBE_LEDGER_DIRS` in
`scribe_defs.sh` is now the roster, and the guard reads that string rather
than a copy of it, so a test can no longer agree with itself forever.
2. The convention guard over-reached: it flagged `<sid>.snap` and
`<sid>.blocked`, which live in `scribe-afterwrite` and `scribe-reportcheck`
and can never be touched by the sweep. It is now two tests keyed on the
directory, because the two assumptions fail differently — one lets a ledger
sit where nothing sweeps, the other lets it sit in the right place under a
name the sweep does not match.
`test_only_the_rule_ledger_is_cleared_and_the_note_ledgers_are_left` also went
red, correctly: the note arms' exemption was a real decision, recorded in a
test, and my commit message said nothing about it had been decided. That was
wrong and the test was right to stop me. The decision is reversed rather than
ignored, and the reasoning is that #4101 removed its premise: it rested on a
note repeat being WITHHELD, so clearing the ledger meant re-injecting whole
menu lines the session already had. Now a repeat is rendered with a `seen`
marker, so the ledger only decides whether that marker is true — and across a
compaction it is false, telling a freshly-summarised session it has already
seen a record that is nowhere in its context. The test keeps its name and
records the reversal with the reason, rather than being deleted.
Verified by running the hook directly: all six ledgers across both directories
gone on `compact`, `<sid>.unreached` left standing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#3750 settled this for rules: a record the session was told about an hour ago
is not a record in front of the reader now, so the second time it is the best
answer it is rendered again with a tail saying so. The note and snippet arms
never got that fix, and theirs was worse — the ledger went into
`semantic_search_notes` as `exclude_ids`, so the repeat left the candidate set
entirely. Three things followed:
- the second time a note was the best answer the session got SILENCE,
indistinguishable from "nothing matched", on the arms that fire most
(`auto_inject` alone ran 598 calls in five days);
- a compaction made that permanent, since the ledger outlived the context it
described — fixed one layer down in c61f730;
- and `best_available` was measured against a candidate set the caller had
already edited, so the bar could be blamed for a record the caller
withheld (#3739, from the side its fix never reached).
The ledger is now a RENDERING fact. Every repeat is still ranked, still shown,
and carries a `seen` marker; the band is computed over all hits, because
letting the ledger move the cutoff would make "you were shown this" change what
counts as relevant. The marker is one word and deliberately not the rule arms'
phrasing — "before deciding it does not apply" is the voice of a record that
binds, and a dev-log borrowing it would claim authority it does not have.
Telemetry takes the rule arms' contract (#3752): `results` and `record_surfaced`
both take fresh only, the repeat is counted in `suppressed`, so this source's
surfaced set still matches its own log row (#3668). That makes a fact readable
that could not be stated here before — `result_count == 0` with
`suppressed_count > 0` is "everything that matched, the session has already
seen", which is a different claim about the bar from "nothing cleared it".
On the write path this also splits a variable that carried two claims. `seen`
was the ledger plus everything the call had already rendered, and both were
treated as reasons to withhold; `in_menu` keeps the same-call exclusion while
the ledger becomes a marker. That narrows the `best_available` compromise at
its old comment to the pulled-and-already-listed case, and retires the argument
that a suppression count here would be partial — nothing is hidden inside the
query any more.
Deliberately unchanged: the write-path SYNC class still shows once. Its claim
is about an edit in progress rather than a record's continuing relevance, and
repeating it every write to the same file would be nagging.
tests/test_ledger_references_not_silence.py pins both arms — the ledger never
reaching the search, the repeat rendered and distinguishable, the telemetry
split, the all-repeats call being readable, and the two exceptions (this call's
own menu, and the sync class).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`scribe_session_context.sh` cleared `.rules.ids` and `.opened.ids` by name and
left `.ids`, `.sync.ids` and `.derive.ids` standing, under a comment asserting
that was a decision. Reading the note arms says it was not: their exclusions go
straight into `semantic_search_notes`, so a surfaced note leaves the result set
rather than being rendered as a reference the way #3750 gave a repeated rule,
and unlike the rules ledger they never age. Hard, permanent, never cleared — a
note surfaced in a session's first minute is unreachable for the rest of it,
which is milestone 386's own defect alive on the arms that fire most often.
The list was the bug, so the fix is not a longer list. `scribe_clear_session_
ledgers` matches the naming convention instead — a per-session ledger is
`<sid>[.<kind>].ids` — which covers all five and covers the sixth on the day it
is written. `<sid>.unreached` is deliberately outside it: that records an
outage, not held context, and #2932 needs it to survive.
tests/test_session_ledger_clear.py runs the hook rather than grepping it for
`rm -f`, since grepping for the names is the pattern being removed. It pins
both directions — `compact`/`clear` take all five, `startup`/`resume` take
none — plus the convention the glob rests on, checked against the hooks
themselves so a ledger named outside it fails loudly instead of silently never
clearing.
Also drops a stale comment pointing at a rules-etag marker that milestone 394
retired.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 6945: 11 failures down to 1. The survivor is
test_shortening_a_line_does_not_decide_what_it_says_about_holding, which
asserted "no longer hold it" appears in the compact line for seen=True — the
phrase that now belongs to the OPENED state, not the named one.
Its subject is a property, not a string: `compact` and the ledger are
independent axes, and shortening a line must not change what it claims about
holding. So the fix follows the axis rather than swapping the phrase. The axis
grew from two states to three, and the test now checks all three are distinct
under compact — pinning only two would let the compact branch collapse the new
middle state into either neighbour, which is the same regression it was
written for with one more place to hide.
Added the assertion that matters most when room is short: a line the session
never opened must not imply it did.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 6943: 11 failures, two causes, both existing guards correctly catching
the behaviour change rather than defects in it.
TEN were `_SEEN_TAIL = "You saw it earlier this session"`. Every one of those
tests drives an arm with `exclude_rule_ids` alone — the NAMING ledger — which
since this change is the middle state, not the opened one. That is the whole
point of the step: `seen` stopped meaning "you saw it". The constant now holds
the middle tail, so each test keeps asserting exactly what it was written to
assert (a repeat gets a line distinct from a first surfacing) against the
wording that is now true.
Added `_HELD_TAIL` and a negative assertion with it: neither call names an
opened ledger, so neither may claim the session read anything. That catches
`held` defaulting true, which would have every repeat assert the strongest of
the three claims on no evidence — and none of the ten existing tests would
have noticed.
THE ELEVENTH was my own test edit being wrong, not the code. I asserted
`held_rule_ids` would appear in `re.findall(r"printf '([a-z_]+)=", defs)`, but
`scribe_held_query` spells it `printf '&held_rule_ids=`, and the leading `&`
means that regex never matches it. The regex enumerates SCOPE keys — the
alternatives that open a query — and the ledger key is appended to a query
that already has one. Two different contracts that happened to share a file,
so they are now asserted separately and the difference is written down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 386 made a repeat REFERENCED rather than withheld, and the line it
chose says "You saw it earlier this session". Nothing ever checked that. The
arms emit a TEASER — title, trigger, get_rule(N) — so a session can be shown a
rule twenty times and never read a word of it, and a compaction summarises the
teaser away leaving nothing behind. The server was asserting something about
the reader's context it had no way to know.
Three states now, where there were two:
never surfaced "it is not in this session's loaded set"
named, unopened "Mentioned earlier this session but not opened — read it…"
opened "You opened it earlier this session; pull it… again"
The middle one is the honest one and the one that was missing. It keeps the
full invitation, because a session that skipped a teaser is in nearly the
position of one never shown it.
HOW "OPENED" BECOMES OBSERVABLE. A new PostToolUse hook watches the get_rule
call itself and appends to `<sid>.opened.ids`. PostToolUse does fire for MCP
tools — the event's own output schema carries `updatedMCPToolOutput`, which
would be meaningless otherwise — and the matcher is `mcp__.*__get_rule` so the
server segment, which varies by install, is not pinned.
This is NOT the self-report 386 rejected. That objection was to ASKING a model
whether it holds a rule, which is unverifiable. A tool call is an event the
harness reports whether anyone asks. Recording what a session DID and believing
what it SAYS about itself are different kinds of evidence.
Both ledgers clear together on compact/clear. Keeping `.opened.ids` across a
compaction would have the arms telling a freshly-summarised session "you opened
it earlier" about a rule now nowhere in its context — a more confident version
of the bug being removed. Same reader (scribe_rules_live) for both, so ageing,
last-entry-wins and the bare-id format are defined once.
Also closes two smoke-coverage holes the checker was reporting as SKIP: the new
recorder, and scribe_precompact_preserve.sh from #3680. The latter needed
STATIC_FLOOR to become a set — PreCompact's contract is inverted, its stdout
BECOMES the summarizer's instructions, so silence is its failure mode and a
generic read of it looks like a leak.
Step 2 of milestone 416, and a hard prerequisite for step 4: while suppression
keys on shown, widening k marks records "seen" faster than they are read, and
the ledger would degrade in proportion to the improvement.
Plugin minted 2026.09.16.1232 -> 2026.09.16.2102.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 6937 red — 3 collection errors, `positional argument follows keyword
argument`, in the three integration fixtures that call the service positionally
(`create_rule(topic.id, uid, "title", "statement")`). The script that added
`when_to_apply=` to 15 fixtures inserted it as the FIRST argument, which is
valid only where every other argument is already a keyword.
Moved to the last argument in every call, which is legal in both styles, and
the continuation indent now matches the surrounding arguments.
Also repairs self-inflicted damage: the same script added a trigger to the two
tests in test_rule_trigger_required.py whose whole purpose is to call the
creators WITHOUT one. They would have stopped raising and the guard's own
proof would have inverted — a test that passes for the opposite reason than
the one it names, which is worse than a failing one.
Caught by ast.parse across tests/ rather than by the next CI round trip.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`when_to_apply` is not metadata. `rule_document` embeds a rule as
`{title} — {trigger}` / `When to apply: {trigger}\n\n{statement}`, the trigger
appearing twice so purpose dominates a short vector — the shape note 2485
measured on snippets (a 0.153 top-to-second gap against 0.010–0.023 for
everything else). Without one the document silently becomes title + statement:
a DIFFERENT shape, ranked against a corpus it does not match, with nothing to
report it. Every bar and every rank in the system assumes one shape.
`create_preference` has refused an empty trigger since it shipped. The two rule
creators defaulted it to "" — so the shape was enforced for the record kind
that guides and optional for the kind that binds.
The guard lives in the SERVICE, because both doors reach it: the MCP tools and
the frontend's fast path in routes/rulebooks.py. Written in either alone, the
other could still create a rule that never fires. The route keeps a matching
check for the STATUS CODE only (400, not the 404 it maps ValueError to).
update_rule refuses to EMPTY an existing trigger, checked after the mutation so
it covers `clear=[...]`, an emptied form input, and any route added later.
Deliberately asked as "did this edit remove one" rather than "does one exist":
a rule predating the guard has none, and refusing to save it would freeze
precisely the unreachable records that most need fixing.
Deliberately not following arose_from_id, which the human door exempts itself
from because provenance is about auditing what the AGENT changed. That reasoning
does not reach this field — a missing trigger is not a missing explanation, it
is a rule that does not work, and it fails an operator as badly as a session.
15 test fixtures across 6 files were creating rules with no trigger. They now
pass one; that they did not is the point — curation is not a guarantee.
Step 1 of milestone 416 "Retrieval stops guessing a bar". First because every
later step assumes one document shape, and it is much cheaper to guarantee
before a corpus grows than to backfill after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`report_preference` shipped reading PROMPTRULE_THRESHOLD_KEY, so the two arms
were one dial: tuning the bar for an operator's prose silently retuned the
lookup that runs when a task closes.
That coupling is worse on this arm than it would be anywhere else. Every other
retrieval arm scores a query that varies per call, so a mis-set bar shows up as
a changed clear-rate. COMPLETION_QUERY is a fixed string, so this arm's best
score for a given corpus is a CONSTANT — and a constant sitting under the bar is
a dead arm rather than a quiet one. No volume of traffic reveals it.
Found by the first live read for milestone 394 step 9: 69 calls, 69 declines,
every one naming the same record at the same score (0.7194 against a 0.72 bar).
Reading `best_available_id` (#3807) showed the record was about interpreting a
REQUEST, not about report shape — so the declines were correct and the arm is
healthy. The percentile alone would have said "lower the bar", which would have
delivered a false positive on every completion report ever written.
The bar does not move; the key does. Both defaults stay 0.72, so this changes no
behaviour on any install — it makes "leave this one where it is" expressible,
which it was not before. Settings grows the control (rules 25, 27), and the
default-agreement check grows a row.
Deliberately NOT included: a change to PROMPTRULE_DEFAULT_THRESHOLD. The
evidence for moving it is this install's near-miss table, and rule 115 keeps a
shipped default from being justified by one instance's corpus. That bar is a
per-user setting and belongs in the operator's Settings, not in the product.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Two interface-contract tests read each hook's own source for the query args it
sends. The project-scope key now comes from scribe_scope_query, so `repo=` was
no longer spelled in the hook and both read that as the hook having stopped
sending it.
They now pin the pair: the hook delegates to the helper, and the helper emits
exactly `repo=` and `project_id=`. That is a stronger assertion than the one it
replaces — it catches a hook rolling its own scope resolution again, and it
pins the new key, which the route must read for a non-git session to reach its
project at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
All six hooks scoped their requests one way: `git remote get-url origin`,
resolved server-side through the repo bindings. That key does not exist
outside a git repo, so a session in a plain directory was unscoped in every
hook at once — no project context, no prior-art scoping, no project rules —
and silently, because a missing remote is indistinguishable from a remote
nobody bound.
A `.scribe` file is the second key, read by the shared scribe_scope_query so a
directory scopes the same way everywhere:
{"instance": "https://scribe.example.com", "project_id": 2, "project": "…"}
`instance` is why the file is not just a number: an id is a different project
on every Scribe, so a marker that travels — a copied directory, a shared
machine, a repo someone else clones — would otherwise scope the session to the
wrong project without a word. Compared host-only, and a mismatch drops the id:
no project beats the wrong project. A bare integer is accepted too, since it
is what a person writes by hand. The marker beats a git remote — someone put
the file there on purpose — which is also how a directory overrides its
binding.
Two things it found on the way:
* An explicit project_id that did not resolve rendered NO message at all —
the branch hung off `if project_id` as an `elif`, so a caller holding a
pointer it believed in got a context that silently omitted the project it
had asked for. Now reported.
* The refusal reason was a global set inside a function every caller reads
through `$( )`. The assignment died with the subshell, leaving the caller
to read an unset variable under `set -u` — which aborts the hook and costs
the whole session's SessionStart context, to fetch a warning about a file.
It comes back through stdout with the id instead, and a test pins it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The hook's header quotes `{"decision":"block"}` while explaining why this hook
must never emit one, and the instructions it prints use "decision" in a
sentence. Grepping the whole file caught both and failed the guard on the file
doing its job.
Strips comments and the heredoc first, and guards the stripper: `_code()`
returning nothing would make all three static checks pass against an empty
string, which is the circularity rule 167 is about.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Spike #3680 asked whether a PreCompact hook can reach the model. Read out of
the installed Claude Code build (2.1.273), the answer is yes — through a
different channel than note #3679 assumed:
* `hookSpecificOutput.additionalContext` is NEVER read on PreCompact. The
hook-output schema has no PreCompact variant; the field is honoured for
SessionStart, SubagentStart and Stop, and silently dropped here.
* A PreCompact hook's STDOUT becomes `newCustomInstructions`, merged with
the operator's own `/compact` instructions and passed into the prompt that
writes the summary. Manual, auto and partial compaction all do this.
So the hook does not interrupt the compaction — it steers the summary, which
is what the next turn reads. Blocking is the thing not to do: a PreCompact
block SKIPS compaction, tells the model nothing, and leaves the session
running on uncompacted with no summary at all.
scribe_precompact_preserve.sh names what the summary is the only copy of:
Scribe record ids WITH titles, the in-progress task and its milestone, work
done but not yet recorded, governing rules, and unfinished operator asks.
No network and no config — what must survive is already in the conversation
being summarized; the hook only says which parts are load-bearing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
list_tasks returned every row's to_dict(), body included: a project's todo list
came to 93-165k characters, past what an MCP client accepts inline, so the list
arrived as a file to page through (the #4045 failure, one call over).
- notes.brief_row: id, title, type, project, tags, updated_at; for tasks,
status, kind, priority, milestone id and title; description, parent and due
date only when set.
- milestones.titles_for: one query for the milestone titles a page of rows
names.
- Brief rows on list_tasks, list_notes, get_milestone's steps, get_system and
list_system_records. get_task / get_note / get_snippet read a record in full,
and each docstring says so.
- tests/test_list_rows_brief.py pins the ceiling: 100 rows of ~5k-character
step plans stay under 40k characters.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A session's go-ahead request sat inside a completion list as "Blocked by the
permission check", and the operator read it as a fault rather than a question
waiting on them. The reporting-back skill now puts any action held for a yes
under a section headed "Approval requested", near the top of whatever reply
it is: each change numbered so part can be approved, why it needs them, how
it is undone, and what happens after. The Asks table gains an Approval row,
and the completion report's "Needs you" points at the section.
test_reporting_back_skill pins the heading and the row. Plugin version minted:
2026.09.15.1921.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Measured live after deploy: three plans reworded from existing active
milestones scored 0.83-0.87 against the milestone they restated, and the
nearest distinct plans 0.72-0.77. At the inherited 0.90 the semantic arm
matched nothing, so only an identical title was caught. 0.80 sits in the gap.
The Settings form's default moves with it (test_settings_defaults_agree).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 5 of milestone 415 "An existing plan is found before a new one is made".
Sessions opened a second milestone beside the roadmap milestone that already
covered the work, and filed related tasks loose, because no surface told them
to look first.
- writing-plans: a section on finding the plan that exists (enter_project's
unplanned_milestones, search(content_type="milestone"), list_milestones);
when an active milestone covers the work, add steps to it; a second
milestone only for a separate arc; the gate's existing_milestone reply.
- using-scribe: "when you plan" gains the same check and milestone_id on
related tasks.
- _INSTRUCTIONS PLAN line points at the milestone search (1,689 of 2,000).
- test_guidance_ownership pins the topic on writing-plans.
- Plugin version minted: 2026.09.15.1744.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
test_nothing_else_builds_the_embedding_document_itself caught start_planning
building f"{title}\n{body}" inline for the plan gate's candidate text. That is
the embedded-document shape; plan_candidate_text now takes (title, body) pairs
and calls embedding_text, so the candidate moves with the corpus it is ranked
against (#2486).
Also carries the create_task / create_records milestone_id docstring lines from
step 5 (#4080), which share the file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 4 of milestone 415 "An existing plan is found before a new one is made".
A session that could not see an existing plan made a second one beside it.
start_planning and create_milestone now ask first: an ACTIVE milestone in the
project with the same title, or one that reads as the same plan (title, design
and steps against milestone embeddings), is returned with its progress and a
pointer to create_records(milestone_id=...). Nothing is created; force=true
bypasses.
- dedup.find_matching_plan / plan_gate / plan_match_response; access-checked
before either arm (rule 78), fail-open like the other gates.
- Done milestones never block; the semantic arm needs 200+ chars of candidate.
- kb_plan_match_threshold (default 0.90) is a setting, in the Settings view,
and pinned against the Python default by test_settings_defaults_agree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`search` covered notes, tasks and rules, and a milestone — the record a plan
lives in — could not be found. A project whose roadmap was written as
milestones had every later plan opened beside the one that already described
it, because nothing could have told the session it existed.
- milestone_embeddings (migration 0102): the third sibling of note_ and
rule_embeddings, for note 3163's reason — the search is milestone-specific.
The document is title — description, then description and the plan body,
so a roadmap milestone with no description is still found by its design.
- Written on create, on a title/description/body update, and for a plan made
through start_planning / create_records, fire-and-forget with the parent-row
claim (#3262); a startup backfill covers every existing milestone. Derived,
so it joins _NOT_INCLUDED beside the other embeddings.
- semantic_search_milestones: a project's milestones when the caller can read
it (access.can_read_project), otherwise the caller's own; optional status.
- search(content_type="milestone"): id, title, description, status, project
and progress. Its own shape, and not part of "all", whose results are
note-shaped. The docstring says what it is for: ask before start_planning.
- Integration test on real Postgres: found in its project and not another,
status narrows, an unreadable project returns nothing.
Milestone 415 step 3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Operator, 2026-09-15: the unmilestoned group should collapse, "especially the
done column as it currently consume a lot of vertical space for [work] that's
already done."
- The "No Milestone" group gets the milestones' chevron and collapse, keyed as
0 in the same Set (no milestone id is 0), and starts collapsed on first load
when everything in it is finished — the rule finished milestones already
follow.
- Each group's Done column header becomes a button that folds its cards, with
the count still showing and aria-expanded set. Folded by default: done work is
the part of a board nobody is reading.
- Not persisted, matching the milestone collapse beside it.
Milestone 415 step 2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A plan written as a milestone with a description and no steps was invisible to
the session handshake: it lists the 5 most recently touched milestones (#4045),
and touching is a step changing, so a step-less milestone can never qualify.
FabledLibrarian's roadmap (nine such milestones) sat unseen while later plans
were opened as new milestones beside the ones that already described them.
enter_project adds `unplanned_milestones`: active milestones with no steps, in
roadmap order, id/title/description, up to 10 with an omitted count, none
repeated from the recent list, and absent when there are none. The docstring
says what they are for: check them before starting a new milestone, and add
steps to a match with create_records(milestone_id=...).
Milestone 415 step 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
start_planning on project 2 replied with 92,645 characters, 65k of them
applicable_rules. Milestone 414 made a project's listing every global rule
tagged to an area it works in (before, the rules of subscribed rulebooks, and
project 2 subscribed to none), and the non-brief rules_payload sent each as a
full rule_brief. get_milestone, get_project and get_task carried the same.
Every rules_payload form now lists: id, title, the topic a global rule sits in,
and `via` for a co_surfaces partner. get_rule reads one in full, and retrieval
delivers them in full when work matches — the reasoning #4045 applied to the
handshake. A test pins 81 full-length rules under 6k characters.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
rules_due_for_verification (the rule staleness sweep) and rule_history (what a
rule used to say) are pure reads, and a read-scoped API key was refused both:
neither is in _READ_ONLY_TOOLS, and the completeness test that should have
caught it only looked at tools whose NAMES start like a read (get_, list_,
search…). Neither does.
- Both join _READ_ONLY_TOOLS; the comment that pointed at this issue now says
why they sat unlisted.
- _WRITE_TOOLS declares every writing tool by name. Nothing reads it at
runtime — default-deny already refuses an unlisted tool — it exists so the
classification is total.
- test_every_registered_tool_is_classified_exactly_once takes its candidates
from what build_mcp_server() actually mounts, requires each in exactly one
of the three sets, and still flags a classified name that is no tool. The
decision stays explicit; only the candidate set widened.
- test_the_completeness_check_can_fail drops a real tool from its set and
asserts it is noticed (rule 167).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A rule's home is its reach: a rulebook topic makes it global, a project makes it
that project's. There was no way to change one, so a project rule decided to be
global could only be recreated and the original trashed — losing the id every
record cites, its edit history, its area tags and its relations.
- services.rulebooks.move_rule(rule_id, user_id, topic_id= | project_id=):
exactly one destination (the model's CHECK), owned by the caller, not the
rule's current home. A topic already holding a live rule with the same title
is refused with a message naming that rule, instead of uq_rule_per_topic
failing the commit. Someone else's rule reads as not found.
- Deliberately NOT done, and said in the docstring: no version (a version is
what a rule said, milestone 323 decision 4), no duplicate gate (nothing new
enters the corpus), no re-embed (retrieval reads the home at query time).
- Both doors: MCP move_rule, REST POST /api/rules/<id>/move (rule 33).
- UI: RuleHomePicker, one component in the rule editor (a global rule) and a
project's rules tab (a project rule), so the two cannot drift on what a
destination is.
- using-scribe names move_rule under "Where a new rule goes". Plugin
2026.09.15.1626.
Milestone 414 step 3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A rule's home is its scope now: a rule in a rulebook topic is global, a rule on
a project applies to that project, and retrieval reads that directly (#4074).
A subscription had stopped changing anything a session received; a suppression
muted rules from a subscription. Operator, 2026-09-15: "we have global and
project scoped rules, we don't need the subscriptions now."
What goes, whole (rule 22):
- Migration 0101 drops project_rulebook_subscriptions, project_rule_suppressions
and project_topic_suppressions, and strips subscribe_rulebooks (and 394's
leftover exclude_always_on_rulebooks) from stored inception choices.
- Service, MCP and REST: subscribe/unsubscribe and the four suppress/unsuppress
operations. The Subscribers checklist, the subscribe chips, the skip buttons
and the Suppressed section in the rules UI.
- Inception asks two questions (design system, seed Systems). create_project and
decide_project_inception lose subscribe_rulebooks.
- Backup v15 stops exporting the three sections; older archives still restore,
the keys simply unread. Trash no longer hard-deletes suppression rows.
What changes meaning:
- get_applicable_rules is a project's LISTING: its own rules, plus the global
rules tagged to an area it works in. Untagged global rules apply everywhere
and arrive by retrieval, so they are not listed. A co_surfaces partner on a
different project is not dragged in.
- list_rules(project_id) lists that project's own rules.
- rules_payload drops subscribed_rulebooks and suppressed_*; the handshake's
brief form is project_rules alone.
- using-scribe's "Where a new rule goes" and inception sections, tool
docstrings and docs say global vs project. Plugin 2026.09.15.1620.
Milestone 414 step 2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
semantic_search_rules searched every rule the user owned, and every hook arm
called it without a project, so each project's rules were injected into every
other project's sessions and a project rule meant nothing a session could feel.
The search now takes a scope: global rules by default (an unbound session, or a
caller that forgets to say), global plus project N when given project_id (N's
rules only if the caller can read that project, through access.can_read_project),
and every owned rule with everywhere=True. The four hook arms and the report
preference lookup pass the session's project; an explicit
search(content_type="rule") scopes to its project_id, or asks the whole rulebook
without one.
Milestone 414 step 1. Guarded by an AST walk that every hook call site passes
project_id, and an integration test on real Postgres that a rule is reached only
from its home.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The handshake carried the whole project record, every milestone's plan, full
rule text, the notes most recently edited and ~9k of design guidance. For
project 2 that was ~222k characters, past what an MCP client accepts as a tool
result. Each category was walked through with the operator and sized to what a
session needs on arrival; each names the call that has the rest.
- project: id, title, status and the full goal (session start's "full goal"
pointer still lands here). get_project keeps the whole record.
- milestone_summary: the 5 most recently touched milestones, any status, most
recent first, without plans. Summaries gain last_touched_at: the later of
the milestone's own edit and its newest step update, from the query that
already counts steps. milestone_summary_omitted counts the rest and points
to list_milestones. get_project and list_milestones list every milestone,
also without plans.
- open_tasks: the 10 most recently touched open tasks, with or without a
milestone, each naming its milestone. list_notes gains sort="touched"
(the later of updated_at and the newest work-log), because a log doesn't
bump updated_at.
- recent_notes: dropped. Retrieval surfaces notes by relevance, and
get_recent covers recency.
- systems: id and name.
- design_system: summary plus guidance_call. get_design_system gains
resolved_guidance, the chain-merged prose; its own guidance field is only
the departures, so session start's old pointer to it led to a fragment.
The session start pointer and using-scribe's "Building UI" section now
name resolved_guidance.
- rules: rules_payload(brief=True) gives project_rules as id and title plus
subscribed_rulebooks, and records only what it shows. Retrieval delivers
rules in full and ignores subscriptions (#4052). Other callers unchanged.
- pattern_coverage, inception and systems_bootstrap: unchanged.
Clients: the plugin's using-scribe skill, the compaction notice and session
start are updated here; the REST project summary only gains last_touched_at.
Plugin version minted.
Tests: a size ceiling on the handshake for a large project; milestone and
task selection and naming; brief rules; resolved_guidance; the session
start pointer; and a real-Postgres test that a work-log touches its task and
a step update touches its milestone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
enter_project returned every milestone's full plan body. On a project with 39
milestones the handshake came to ~222k characters, 168k of them bodies (110k
from done milestones). That is past what an MCP client accepts as a tool
result, so the call meant to orient a session arrived as a file to page
through. It grows with a project's history, so any long-lived project on any
install gets there.
- brief_milestone_summary (services/milestones.py) trims summary rows to the
listing fields: id, title, description, status, order_index and progress.
The plan is get_milestone's job. user_id, project_id and timestamps repeat
what the caller knows.
- enter_project and get_project share one block: every open milestone plus
the 5 most recently updated done ones, in order. milestone_summary_omitted
is attached only when older done ones were left out, and names
list_milestones and get_milestone.
- list_milestones lists every milestone, done included, without bodies. It is
the call the omitted line points to, and it had the same size problem.
- The REST project summary is unchanged; the web UI reads it.
Tests: trimming, the done cap and its order, the omitted key present and
absent, get_project and list_milestones, and a size ceiling on enter_project's
milestone block for a 200-milestone history.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
In a repo where 3,000 files define `slug`, scribe_local_dups printed
nothing; with 3 files it printed the duplicate line. Every hook runs under
`set -uo pipefail`, and `hits=$(git grep -l … | head -4) || hits=""` lost
head's four lines whenever git grep was still writing when head exited. That
is a SIGPIPE, the substitution fails, and the outer fallback wipes the result.
So the by-name duplicate arm went silent for exactly the most-duplicated
names. Found while fixing the same trap in the new Stop hook (#4041).
- The fallback moves inside the substitution, `$(… | head -N || true)`, at all
five sites: scribe_defs.sh (local dups), scribe_prior_art.sh (names,
old_first, shapes) and scribe_after_write.sh (names). A real upstream
failure still yields empty output.
- check_plugin gains a known-bad pattern for the shape, so no hook can bring
it back.
- Tests: the real function against a 3,000-file repo (past the pipe buffer),
and the pattern shown to flag the old shape and pass the fix. Plugin
version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI run #517 failed five of the new hook tests, all of them expecting a
request that never went out. The prefilter matched `"name":"…update_task"`
with no space after the colon, which is how Claude Code writes its
transcripts, while the tests wrote theirs with json.dumps defaults
(`"name": "…"`). The hook exited at the prefilter for every test transcript.
The silent-case tests passed for the same wrong reason.
- The prefilter now allows whitespace after the colon. The jq parse behind it
never depended on formatting.
- The test transcripts are written compact, matching the real file. The
silent-case tests now reach the turn parse rather than stopping at the
prefilter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Everything else Scribe gives an agent arrives before the reply is written. A
Stop hook is the one moment the finished reply exists, so it is the last
chance to fix a report the operator can't read, and the only place adherence
to the shape can be measured.
- plugin/hooks/scribe_report_check.sh (Stop): deterministic, no model call.
1. Did this turn close a task? That means an update_task/create_task call
with status "done" since the turn's prompt, whose tool_result is not an
error. Otherwise it stays silent, which covers most turns (one grep).
2. Does the reply that ends the turn say where the work sits (a record by
id and title, or step N of M), what needs the operator, and what comes
next? Matched on those words, not on exact headings.
3. If sections are missing, it blocks once. With stop_hook_active set, a
rewrite is recorded (passed_after_rewrite / missing_after_rewrite) and
never blocked again. A block loop started by another plugin (no marker
from this hook) is left alone.
- Measured: every checked reply is reported to GET /api/plugin/report-check
(passed / blocked / after rewrite). Turns that close nothing are not
reported; they would cost a request per turn and add nothing to the rate.
Outcomes go to app_logs as category "plugin", action "report_check".
- It blocks only when the block was recorded, and only in the server's words.
The endpoint returns the block reason, so the hook carries timing and
transport only (PACKAGING.md), and an unconfigured or unreachable instance
never stops a session.
- The transcript format is read from real transcripts and marked in the hook
as observed rather than documented. The Stop contract (transcript_path,
stop_hook_active, decision/reason, no matcher, SubagentStop separate) was
checked against the Claude Code hooks docs. A prompt-type hook was not
needed: the deterministic check passed a real completion report from this
session and blocked a stripped one.
- A pipefail trap was caught while exercising the hook: `tail | grep -q`
reports failure exactly when grep matches, because tail dies of SIGPIPE.
The prefilter reads through process substitution; the section checks use
here-strings.
- Tests: an end-to-end hook suite over synthetic transcripts and the shared
HTTP sink (silence, pass, server-worded block, rewrite recorded, foreign
loop, errored write, earlier turn, unwritten reply, no recorded check, bare
id), and service tests for the reason wording and the outcome record. Smoke
event added to check_plugin; README and PACKAGING list the hook and
endpoint. Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reporting-back skill ships default shapes; an operator's adjustments to
them are preference records. Prompt-time retrieval matches the operator's
message, and a shape preference is about the reply, so those preferences were
on file and never arrived. Operator's decision (logged on #4013): the server
delivers them for a completion report, and the skill asks for every other kind.
- Completion reports (option C): closing a task with update_task runs a
kind-filtered preference search for the moment "writing the completion
report after finishing a task" and returns matches as `reply_preferences`
({id, title, statement, kind}), with a sentence added to `report_back`
naming the key. A preference says it is about completion reports through
its own when_to_apply; no tag or column. Omitted when nothing matches, and
the lookup fails open.
- Telemetry: every call logs to retrieval_logs under `report_preference`
(empty calls included; a search that never ran writes no row) and hits are
recorded surfaced. The source is ranked, so it counts toward pull-through.
The bar is the prompt arm's setting until step 6 reads this source's near
misses.
- Every other reply (option A): reporting-back gains "The operator's own
shapes come first". Before a finding, decision, handoff or "where are we",
search(content_type="rule") in the words of that moment and follow what
comes back. Registered in the ownership guard with reporting-back as owner.
- Loading reply shapes at session start (option B) was rejected: it would be
a small copy of the preloading milestone 394 retired.
Domain-neutral query (pinned); works on an install with no preferences.
Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 7's live check (#4034 "a real session shows each topic stated once")
found nothing copied across surfaces, but found the owner repeating itself:
using-scribe said "rules are retrieved; ask before a consequential act; an
empty session is not an empty rulebook" in "Do this first", in reflex 2, and
twice more further down reflex 2.
- using-scribe: reflex 2 now holds the one full statement (how rules arrive,
why that is the surface working, what silence means, when to ask). "Do this
first" keeps the enter_project step and a short lead-in pointing at reflex
2. "A retrieved rule outranks a default habit" and "Ask hardest where you
feel most certain" stay, since each adds something.
- Adapter static context: "Lines injected beside your work" keeps only the
Claude Code timing and leaves "never the whole set" to the skill. "Keep one
copy" names CLAUDE.md and auto-memory and points at using-scribe, which
already says to leave a client's settings as they are.
- The ownership registry's markers and statement for the topic still sit on
using-scribe, and nothing else changed owner. Plugin version minted.
Sizes: using-scribe 18,823 -> 18,197 chars; scribe_static_context.md
1,797 -> 1,714.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A raw 200-char slice ended mid-word with nothing marking the cut, so a
reader took half a sentence for the whole goal. _goal_line flattens the
goal to one line, trims at a word break with an ellipsis, and points at
enter_project(id) when it cut.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 6 of milestone 410 "One owner per piece of guidance". CI now keeps the
shape decision #4027 set, so the next feature cannot quietly add a copy.
tests/test_guidance_ownership.py, 33 topics, each with an owner, markers, a
statement distinctive to the owner's full wording, and optional index markers:
- test_every_topic_is_stated_by_its_owner: markers and statement on the owner
- test_no_topic_is_stated_in_full_off_its_owner: the statement appears on no
other session surface (index, adapter static text and commands, live
context, other skills). Tool docstrings are not scanned; a contract may
elaborate the reflex that calls it.
- test_the_index_names_each_reflex_it_points_at: _INSTRUCTIONS keeps a
one-line pointer for each session-start reflex
- shared_with declares the one deliberate sharing: the note-check question
lives in create_note and in using-scribe for two different moments,
already pinned by test_verification_guidance_survives
- test_the_ownership_guards_can_fail shows each guard turning red (rule 167)
- the process topic now keys on get_process's "follow the returned body";
it had been passing on an unrelated "verbatim" in two other tools
tests/test_instruction_surfaces_agree.py keeps only what ownership cannot
enforce: the fold budget, rules-bind-names-preferences, and using-scribe's
pointer to reporting-back. Retired: the every-session-start-surface ask and
absence tests, the Systems and snippet owner pins (now registry topics), and
the SessionStart-without-ask test (#2497's shape). The push has carried no
rules since milestone 394, and requiring the ask beside every mention of it
would force a copy. The module docstring records where each protection went.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 5 of milestone 410 "One owner per piece of guidance". The operator
wants any attempt to package Scribe for another agent client to find the
repo already in the right shape.
plugin/PACKAGING.md (linked from the README) states:
- what every client package shares: plugin/skills/ (verbatim), the /mcp
endpoint and its in-band responses, the /api/plugin/* adapter endpoints
(context, retrieve, prior-art, tool-rules, processes), and one fmcp_ key
- what each client adds: a manifest; hooks limited to timing and transport;
optional commands; adapter static text that never copies a skill or the
index
- the Claude Code adapter file by file, as the worked example
- how Agent Plugins 1.0 clients (Codex, Cursor, Copilot/VS Code, Kiro,
ChatGPT) and Gemini CLI would map, marked researched-not-tested (#4023)
- four open questions for the second package: passing the key to the MCP
server, whether two manifests can share one folder, hook parity, and
where process skills go
Hook audit: every hook prints only server-provided text, status or outage
lines, the running version, or the compaction reload pointer. No guidance
copies, so nothing moved.
Guard: test_the_skills_reference_nothing_outside_their_folder fails on a
relative path upward or a reference to plugin/, hooks/, commands/ or a
manifest from inside a skill, with a companion test showing it can fail.
Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 4 of milestone 410 "One owner per piece of guidance". The plugin's
static session context was a second copy of using-scribe and the server
index. It now says only what Claude Code needs said: 8,705 -> 1,791 chars.
scribe_static_context.md:
- points at using-scribe for how to work with Scribe, and names the process
skills
- Claude Code specifics: keep one copy in Scribe rather than CLAUDE.md or
auto-memory (leave auto-memory at its default); injected lines are
retrieval, not the whole set; compact at clean seams; stored Processes
arrive as scribe-proc-* skills with /scribe:sync; say so when the tools
are unavailable
- retired: the restated reflexes, "how the surfaces divide the work", and
the "follow the surface that assumes least" precedence (decision #4027)
Hook behaviour is unchanged. The SessionStart hook header says what the
static tier now carries, and the unreachable-instance status points at the
using-scribe skill instead of "the standing guidance above".
README and manifest describe the plugin as the Claude Code adapter over the
shared, client-neutral skills. Fixed along the way: the README said the
SessionStart hook "injects your rules" and suggested disabling auto-memory,
both contradicting the product since milestone 394.
Tests: the session-start rules guards now pin the owner (using-scribe) and
the index (_INSTRUCTIONS) rather than every surface; the Systems-reflex and
snippet-trigger guards pin their owners; the reporting-reflex guard pins
using-scribe. The loss guard and client-neutral guard stay green. Plugin
version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 3 of milestone 410 "One owner per piece of guidance".
_INSTRUCTIONS is rewritten as an index for every MCP client (1,597 of 2,000
chars): orient, rules, recall, record, plan, ids, reuse, UI, report. Each
line names its tool, and the block says every reflex is stated in full in
the using-scribe skill and in each tool description.
- names no client: CLAUDE.md, auto-memory and "the client injects ~2k
chars" are gone
- gains the two reflexes it lacked: records that cite each other go through
create_records with {{ref:N}}, and reports start from `placement`
The comment block above it now explains ownership (decision #4027) instead of
accumulating per-milestone trade history, and keeps the budget and its
reason (#2562).
build_session_context states only what the server knows about this session:
the active project and open work, its design system, an unbound-repo hint,
or that no project is bound. Removed: the "you are not holding the
operator's rules" section, the closing "Reflex: search Scribe" line, the
design-system usage sentence, and the plugin-specific header. using-scribe
owns all of that. The truncation note no longer restates the rules ask.
Tests: a pin that the live context carries no rules reflex; the cap test
drives truncation through the unbound-repo hint, since a bare session is now
one line; the budget test message describes the index.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 2 of milestone 410 "One owner per piece of guidance". Skills are the
part of every client package shared verbatim (Agent Skills, decision #4027),
so they state each reflex in full and name no particular client.
using-scribe gains what only the static session context said:
- a retrieved rule outranks a default habit; ask when no rule speaks to it
- log on completing a task and on hitting a problem, not only successes
- the systems_hint on an untagged record is the tagging question, answered
at the moment of work
Client-specific text leaves the skills, rewritten as the universal idea:
- using-scribe: "keep one copy" no longer names CLAUDE.md, MEMORY.md, native
auto-memory or autoMemoryEnabled; "this plugin" becomes Scribe
- reusing-code / shape-accounting: Write/Edit and Bash become editor tools
and shell edits; the prior-art "hook" becomes the prior-art hint; a plugin
version number is dropped
tests/test_guidance_ownership.py:
- test_the_skills_name_no_particular_client fails on any Claude Code path,
memory file, slash command, hook event or tool name in a skill, each
marker commented with why it is client-specific; a companion test shows
it can fail
- three registry topics for what using-scribe now owns; the loss guard
stays green
Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 1 of milestone 410 "One owner per piece of guidance". The later steps
delete duplicate copies of agent guidance; this guard stops the last copy of
a topic going with them.
- tests/test_guidance_ownership.py holds the registry: 30 topics from the
ownership map in decision #4027, each with its owner and marker phrases,
and one definition of a delivered surface (_INSTRUCTIONS, tool docstrings,
each skill, the static context, the adapter commands, the live session
context).
- The loss guard: every topic is stated in full, with all its markers
together, on at least one delivered surface. A miss names the nearest
partial match.
- The owner column is recorded but not asserted yet; step 6 adds the
exactly-one-owner guard once the moves are done.
- test_the_loss_guard_can_fail proves split and absent markers are reported
(rule 167).
- The old DISPLACED_TOPICS list in test_instruction_surfaces_agree is folded
in, so there is one list rather than two.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Skill bodies and tool docstrings still taught the deleted model: using-scribe
said always-on rules "arrive whether or not you ask" and that SessionStart may
inject a rule index; create_project, decide_project_inception and enter_project
said an undecided project gets "every always-on rulebook"; create_rule pointed
standards at "the always-on one"; the verification sweeps and
retrieval_telemetry listed always-on paths and a live preload.
Every passage now describes the current model: every rule is retrieved, a
rulebook binds only by subscription, an undecided project inherits nothing,
and the preload survives only in telemetry rows older than 394.
Also repairs three sentences left half-replaced by the 394 edits: the static
context's "If you have not loaded the no rule has arrived", create_rule's
"an A subscribed rulebook", and create_project's doubled
subscribe_rulebooks entry. Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 3 of milestone 409 "Response shapes". Step 2's reporting-back skill
only helps if it fires, and only exists in the Claude Code plugin.
- scribe_static_context.md and using-scribe (new reflex 11) say: report back
in a shape the operator can read, placed from the `placement` block rather
than memory, and point at the reporting-back skill. using-scribe also lists
it among the sibling process-skills.
- update_task returns a one-line `report_back` cue when a task is closed
(done or cancelled). A tool response is the only surface every MCP client
sees, at the moment the report is about to be written.
- _INSTRUCTIONS takes no line: there is no budget without trading out a
session-start reflex. The decision is recorded in server.py's comment
block so it is not re-litigated blind.
- Guards: test_instruction_surfaces_agree pins the reflex and the placement
pointer on both plugin surfaces; a tool test pins the cue on closing
statuses and its absence on every other update.
Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 2 of milestone 409 "Response shapes". A reply written in the order the
work happened is accurate and still unreadable to someone who was not there.
The new bundled skill shapes it around where the work stands.
- Fires when an agent is about to report completion, hand off, ask the
operator something, answer "where are we", or propose an approach.
- Every reply: conclusion first, one topic per section, visible priority,
the ask in bold at the end, plain words, the work placed in Scribe.
- Placement is taken from the placement block step 1 returns (#4010), not
recalled; untracked work is said to be untracked.
- Four families of shapes: Reports, Asks, Answers, Proposals, with the
completion report written out in full.
- Domain-neutral: the worked example is a backup job, evidence is "what you
could open to check it". Written as practices, and naming no instance rule.
- A structural guard pins the completion sections, the from-the-record
placement, and the absence of software-only vocabulary.
Listed in the plugin README and manifest description; version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 1 of milestone 409 "Response shapes". An agent reporting finished work
is asked to say which milestone it belongs to, which step of how many, and
what is next. Without those facts to hand it reconstructs them, and a
reconstruction reads exactly like the truth when it is wrong.
create_task and update_task (MCP) and the REST create/update task routes now
return a placement block: project; and for a task in a milestone, the
milestone, position {step, of}, progress {completed, total, pct} and next
(the next open step, falling back to the earliest open one before it).
- Step order is creation order, not get_milestone listing order, which
reshuffles on every update.
- Siblings are read through readable_notes_clause; position and progress
are computed over that same readable set, so a collaborator is never shown
a step title they cannot open, and a note share alone reveals no plan.
- Fail-open and omitted when empty, like every in-band decoration.
- _no_embedding moves into conftest as one opt-in fixture for both
integration modules that need it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sessions predicted the ids their next creates would get and wrote them into
plan bodies and reference notes before the records existed. The database
never collides; the sequence is shared by every session and user, so any
concurrent create took the guessed numbers and the references pointed at
someone else's records.
- create_records (new MCP tool) and start_planning(body=, steps=) create
their records in ONE transaction: insert, flush for the real ids, rewrite
{{ref:N}} / {{ref:milestone}} placeholders as #id "title", commit. No
prediction, no waiting, no stub records left behind when a batch fails.
Ids need not be consecutive and nothing depends on it.
- Every MCP create/update of a note, task or milestone refuses a #N sitting
just above the highest assigned id (within 50): that can only be a guess.
Refusal, not warning. Numbers far above the max (PRs, forge issues) pass.
- notes.build_note splits validation out of create_note so the batch
validates records exactly as a single create does.
- writing-plans and using-scribe say to pass steps up front and never write
an unassigned id; plugin version minted.
Integration test runs six concurrent batches and checks each resolves its
placeholders to its own records, and that a failing batch writes nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#3852 and #3853 each added a threshold and neither added its input, so two
of the eleven retrieval settings were reachable only through the database.
The other nine have had UI all along.
That matters because the issue said otherwise. #3927 claimed none of the ten
appeared in the frontend, on the strength of a grep across `web/src` — a
directory this repo does not have. An empty result from a path that cannot
match was read as "no UI anywhere", and a rule-25 argument was written on top
of it. The issue is corrected rather than quietly rewritten: it is #3720's
defect, absence read as non-existence, committed while filing issues about
the product doing the same thing. A grep that returns nothing and a grep that
cannot match produce the same output, and only a positive control tells them
apart.
So this is small, which is the honest size:
- `kb_toolrule_threshold` — the command arm's bar. The hint says why it sits
BELOW the write-path one rather than leaving that looking like a mistake: a
shell command is short, so it scores lower for the same relevance, and at a
shared bar this arm spoke on 2% of calls against the write path's 37%.
- `kb_promptrule_threshold` — the prompt boundary, a third query shape again,
and the only moment that reaches a rule about how to ANSWER.
Both follow the five-site pattern the existing controls use: ref, clamp on
save, write-back, payload key, load. Separate keys, because the finding of
#3853 is that one number cannot serve arms whose queries differ in shape.
Also corrects copy that went stale this morning. The standing-rule hint still
said the arm surfaces "only rules marked conditional, since always-on ones
are already loaded" — describing a tier milestone 394 removed, on the surface
whose whole job is telling the operator what the bar does.
The guard is the part worth keeping. A form's initial value is a CLAIM about
the server's default, and nothing connected the two: retune the Python
constant and the input keeps rendering the old number, which the operator
reads as the bar in force. It pins the relationship across all five
thresholds, never the values, so retuning stays free as long as both move —
and it is falsified against a drifted form value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Found post-deploy, by reading the telemetry it had just rewritten.
The token it was written for IS gone — `TOK=[redacted:token]` where a
credential used to be. But `<task-notification>` came back as
`<ta[redacted:token]>`, across rows, because `sk-` matched INSIDE the word:
`sk-` + `notification` is a vendor prefix followed by twelve word characters.
TWO PORTING MISTAKES, COMPOUNDED. The live scrubber's pattern begins with
`\b`; the migration's inlined copy had no boundary at all, dropped when I
ported it to SQL. And `\b` would not have saved it either — in Postgres ARE
`\b` is a BACKSPACE, not a word boundary. `\m` (start of word) is the
spelling that means what Python's `\b` means. Two things that look
interchangeable, are not, and fail in the same direction.
The live scrubber was never affected, and the evidence says so cleanly: rows
written after the deploy carry `<task-notification>` intact, while
migration-rewritten ones are mangled. Only the frozen copy was wrong.
THE DAMAGE HERE IS PERMANENT. The UPDATE overwrote the only copy of that
text, so those rows cannot be restored. What this fixes is every OTHER
install: 0099 has run exactly once, on one instance, and shipping a known
evidence-destroying migration in the chain for everyone else would be the
worse half of the mistake. The docstring records what it cost rather than
tidying it away.
The guard pins the property no reader can eyeball — `\m` present, `\b`
absent, in both patterns — and is falsified against the shape that shipped.
This is the third time this scrubber has eaten evidence it should not have
(`--author=`, then `task-notification`), and the pattern is consistent: the
redaction half is easy to verify and the SURVIVAL half only fails on inputs
I did not think to include. The evidence set is where the work is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Last run left one failure, and it was the useful kind: the partner ARRIVED
(the scoping fix in 9c5ab1d worked) but carried no `via`, because it came
through the ordinary query rather than being dragged in by the edge.
The assertion was about to be read as "the edge is broken". It was not. The
edge was never exercised: since 394 an untagged rule in a subscribed rulebook
applies on its own, so the partner was already applicable and there was
nothing left for `co_surfaces` to do. The test had quietly stopped testing
anything — passing the first assertion for a reason unrelated to the
mechanism it names.
Rule 167's case exactly, so the fix is to restore its ability to fail rather
than to soften the assertion. The partner is now TAGGED to an area this
project does not work in, which puts it out of reach of everything except the
edge, and the test asserts that unreachability before drawing the relation.
If the edge ever stops dragging partners in, this fails again — which it
could not have done a commit ago.
The neighbouring suppression test keeps an UNTAGGED partner, deliberately and
now explicitly: that one is about a suppression outranking an edge, so its
partner should be reachable by every route, not none.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
I got this wrong in 0e10f6b and the integration suite caught it.
get_applicable_rules filtered `always_on OR area-reachable`. Removing the
tier, I kept only the reachable arm and argued that a project with no
canonical-tagged Systems should get no bulk rules and reach them by
retrieval instead.
Two things wrong with that. The query is ALREADY scoped to rulebooks the
project SUBSCRIBED to, so the project had opted in and was then handed a
subset of what it asked for — subscription is not bulk delivery, it is the
opt-in. And milestone 394 is explicit that subscription-derived rules are not
the always-on tier and are not what it removes; I narrowed something the
milestone said to leave alone.
The failure that surfaced it is a good one: a co_surfaces partner never
arrived, because the rule it travels with had been filtered out before the
edge could drag it in. A behaviour two steps from the change.
What ships instead is narrower than "drop the clause" and wider than what I
had: every rule in a subscribed rulebook applies, EXCEPT that a rule tagged
to specific areas applies only to a project working in one of them. An
untagged rule was never narrowed by anyone, so it is general to its rulebook
by construction; a tagged one is an author saying "this is about CI" and
meaning it. D7's deterministic narrowing is kept where it was asked for and
not invented where it was not.
Also: three tests covering the SessionStart preload, the always-on tool and
the two marker paths — all surfaces that no longer exist — and the backup's
declared-section list, which still named the section its table took with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Five validators failed on 0e10f6b; these are the ones the logs named.
COLLECTION died first and hid everything else: test_inception.py still
imported project_rulebook_exclusions, so pytest aborted before running a
single test in either the unit or the integration lane. The migrations
therefore never ran, which means 0099 and 0100 are still unverified — this
push is what puts them in front of real Postgres.
The obsolete table test went with the import, and its module docstring now
says why rather than just describing one fewer thing.
TypeScript: RulebookDetailPane's currentRulebook computed existed only to
feed the always-on toggle, so removing the toggle left it unread; the vue
'computed' import went with it.
Plugin version minted — plugin content changed and the manifest gates the
executing cache (#2209), so the hook check fails until it moves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Three defects from the sweep, all caught by ruff.
system_ids was REMOVED FROM create_rule AND create_project_rule — a real API
regression, not a lint nit. The parameter shared a signature line with tier,
so deleting the tier deleted it too, and the tools lost the ability to tag a
new rule to an area. Areas are what let a rule reach a project after this
milestone, so the one parameter that decides reach went missing from the two
tools that create reachable rules.
The other two are imports left holding nothing: services/rulebooks.py's
module-level datetime and its IntegrityError were used only by functions this
milestone deleted, and plugin_context lost four (select, async_session,
RulebookTopic, rulebooks_svc) with the preload and _topic_titles.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Milestone 394, steps 5-8. Operator: "remove the always on rule functionality
as the goal was to not have it at all since it didn't seem to work as
expected."
Unconditional preload had three failures the retrieval arms do not. It could
not be MEASURED — a resident rule is in the context whether or not it
mattered, so nothing distinguished "this governed the act" from "this was
scenery", and it was the one surface structurally exempt from the scoreboard
judging every other. It was SUMMARISED AWAY by compaction while the session
went on believing it held the rules. And it CROWDED OUT the few rules that
applied with the thirty that did not.
WHAT GOES
Schema (0100): rules.tier + ck_rules_tier, rule_versions.tier,
rulebooks.always_on, and project_rulebook_exclusions — a table recording a
project's opt-out of something that no longer binds it unasked.
Tools: list_always_on_rules, exclude_always_on_rulebook,
include_always_on_rulebook. Service: the same three plus rules_etag_for,
_valid_tier and the whole etag family. The SessionStart preload and the
write-path staleness arm go with them: nothing is resident, so nothing can
have drifted since a session loaded it.
THREE CALLS WORTH REVIEWING
enter_project got NARROWER, not wider. Its filter was `always_on OR
area-tagged`; dropping the tier arm leaves the deterministic half, so a
project with no canonical-tagged Systems gets no bulk rules and reaches them
by retrieval instead. Dropping the whole clause would have made that payload
bigger than the preload this milestone deletes.
Backups import tolerantly. A pre-394 archive carries tier, always_on and the
retired inception choice; none is read, and the exclusion key is DROPPED
rather than remapped, because restoring it would write data that
validate_inception now rejects as unknown.
The migration is irreversible in the way that matters and says so: downgrade
recreates the columns at their defaults and cannot restore which rules were
always-on. A value invented to fill a hole is not a measurement.
THE INSTRUCTION SURFACES SAY THE HARDER THING
Deleting "call list_always_on_rules()" is easy; replacing it is not, because
the new model asks a session to trust something it cannot see. All three
surfaces now say a session holds nothing, that rules arrive when work matches
them, and — the half that got dangerous — that "no rule arrived" means
"nothing matched", never "there is no rule". Under residency an empty session
was rare and suspicious; it is now the ordinary state of most turns, so
reading it as permission is wrong on nearly every turn rather than
occasionally. That is #3720's defect at session scale.
test_instruction_surfaces_agree is repointed rather than retired: its two
halves collapsed into one instruction, and it gains a guard that every
surface states what absence means. _INSTRUCTIONS is back at 1999/2000 —
the inception clause paid for the longer HOW line.
UI (rule 27, and the opportunity step 8 named)
The tier selector is gone, and what replaces it is the point: `when_to_apply`
is now the field that decides whether a rule is ever seen, so the editor
marks it required, warns while it is empty, and both rule lists badge a
trigger-less rule "never surfaces". A rule without one is not quiet, it is
unreachable.
TESTS
Two files deleted outright — test_rules_etag.py and test_inception_rules.py
tested subsystems that no longer exist. Elsewhere obsolete cases were removed
and the rest repointed. One deserves naming: the wiring test asserted the act
arms pass no `tier`, which had become an assertion that could not fail. It is
repointed onto `kind`, which does still exist and where the same claim is
live — a preference must reach a write exactly as a rule does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
pre_tool_rule retrieves against the RAW COMMAND TEXT and write_path_rule
against the code being written, so whatever was on the command line or in
the buffer is what record_retrieval stored in retrieval_logs.query. A
command that exported a token stored the token.
Storing it was not the worst of it. near_miss_samples is the readout the
threshold docs tell you to open before moving a bar, so the value came back
OUT into an agent's context on the next tuning pass — which is exactly how
this was found, mid-way through #3853's threshold spike.
SCRUBBED ON WRITE, at _build_payload — the single seam every source reaches
the column through. A read-side filter would leave the secret in the table
where a backup or a debug query still reaches it, and a per-caller scrub
would be three places for one to be forgotten by whoever adds the fourth
arm.
REDACTED VISIBLY. `[redacted:<kind>]` rather than a silent deletion: a
reader who cannot tell a scrubbed query from a short one is being lied to
by the readout itself.
DELIBERATELY CONSERVATIVE — vendor-prefixed credentials, values assigned to
secret-NAMED variables, auth headers, PEM blocks. Things that are secrets by
construction. Entropy heuristics and long-opaque-string detection start
eating real queries, and a query is evidence: missing an exotic secret costs
one redaction nobody made, while eating a query costs the ability to tune
the bar at all.
The guard pins BOTH directions, and the second half is the one that matters.
A scrubber that eats evidence fails silently — it keeps looking like it
works while turning the one instrument for tuning a threshold into
unreadable stubs, which is the #2663 shape in a new place. So nine REAL
queries from this install's near-miss samples must survive byte for byte. If
a future pattern touches one, the pattern is too greedy.
Verified against the real shapes before commit: six credential formats
redacted (fabricated values), nine real queries unchanged, and the payload
seam confirmed to store "export API_TOKEN=[redacted:assigned] && git push".
This does NOT scrub rows already written. Purging those is separate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
One threshold served both act arms. The telemetry says they are not the
same problem:
write_path_rule 2,325 calls, speaks on 37%, near-miss p50 0.6989
pre_tool_rule 11,768 calls, speaks on 2%, near-miss p50 0.6794
The second is not quiet, it is mute — 11,530 of 11,768 calls said nothing,
with near-miss p90 at 0.7097 against a 0.72 bar. Refused mass piled one
hundredth under the line is what a bar set too high leaves behind, and the
note arms are the control: auto_inject refuses at p90 0.5463, write_path at
0.6738, both far below theirs.
The cause is query shape, not corpus. A write-path query is a code payload,
long and rich — the case 0.72 was calibrated on. A pre-tool query is a shell
command, often under a dozen words: less text, less signal, lower scores for
the same relevance.
MEASURED. Eight replayed queries against the post-#3855 corpus, consequential
acts against innocuous ones:
0.7571 git push origin dev consequential
0.7245 cd ...; git fetch; git add -A consequential
0.7193 git pull --rebase origin dev consequential
0.6850 docker compose up -d consequential
------------------------------------- 0.68
0.6735 wc -l src/*.py && date innocuous
0.6544 grep -rn useState src/ innocuous
0.6099 sed -n '120,160p' package.json innocuous
0.6056 ls -la && cat README.md innocuous
At 0.72 three of four consequential acts retrieved nothing, including
`git pull --rebase origin dev`, where rules 153, 1 and 2 all ranked correctly
between 0.7126 and 0.7193 and were all refused.
The separation is 0.0115 wide. That is a direction, not a settled number, and
the comment says so — near_miss_samples on a few days of post-#3855 traffic
is what settles it.
This also corrects an assumption the old comment stated: it argued 0.68 sat
"below where this corpus's noise sits", inferring a higher floor from the
corpus being homogeneous. Measured, the command arm's noise ceiling is 0.6735,
so 0.68 clears it barely rather than sitting under it.
Lowering is safer now than it would have been. Until #3851 this arm had one
slot, so the bar was the only noise control; the band now filters downstream,
so the bar's job shrank and the bar can.
write_path_rule is unchanged — healthy at 0.72 on its own evidence.
Guards: the two bars parse independently, garbage falls back to its OWN
default rather than to the sibling's (which would silently re-merge them),
the command default stays below the write-path default as a direction check,
and each arm both SEARCHES and REPORTS at its own bar. That last one is a
failure the single-bar code could not have had: retrieval_logs.threshold is
what near-miss analysis is read against, so an arm searching at one number
and logging another misreports the refusal and invites moving the bar that
was already right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
CI run 6485 was red. Six failures, three causes, and only one of them was a
stale test.
THE REAL DEFECT. The compact branch dropped the `seen` TAIL along with the
trigger, so a rule the session had already been told rendered exactly like
one it had not. #3750's whole argument is that those are different claims —
a repeat is rendered precisely because the session may no longer HOLD what
it was told — and the tail is the entire difference a reader can act on.
test_a_rule_the_session_already_holds_is_referenced_not_re_offered caught it
within one commit, which is that guard working as intended.
Fixed by keeping the tail and dropping only the trigger, which is both the
cheaper and the safer cut: a trigger runs 300-400 characters after #3855, a
tail about 100. Re-measured on the real renderer — top-full-plus-references
is ~299 tokens against ~568 for five full lines, so about 2x the old single
line rather than the 1.4x claimed before, for four more rules and no lost
information. The comments carrying the old figure are corrected rather than
left to read as a decision nobody made.
THE FIXTURE THAT STRADDLED THE BAND. `_THREE_HITS` spanned 0.81-0.74 against
a 0.05 band, so the act arms dropped its lowest hit and four cases of
test_both_recorders_report_the_same_rules_for_one_call failed reporting a
count mismatch — under a message blaming the exclusion filter. A guard
pointing confidently at the wrong subsystem costs more than no guard,
because it is believed. Scores retightened to 0.81/0.80/0.79 and the
precondition is now asserted by a named test, so a future band change is
told where the problem is instead of through four confusing failures.
THE STALE CONSTANT GUARD. test_the_rule_arm_asks_for_one_rule_not_two pinned
RULEHINT_LIMIT == 1 — a real decision, correctly guarded, for a world with a
resident set. Rewritten to pin what replaced it, as relationships rather
than values (rule 115): the arm can return several, and rules are narrowed
HARDER than the notes menu because they measured flatter, not sharper.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
RULEHINT_LIMIT was 1. That was correct while retrieval SUPPLEMENTED a
33-rule resident set — one salient rule beside everything already loaded.
Milestone 394 removes residency, and then this arm is the whole delivery:
`git push origin dev` is governed by rules 1, 2, 9 and 140 simultaneously,
and each alone permits the mistake the others catch.
A cap plus a band, not a bigger cap. The old argument's real content is that
a fixed k invents lines — it fills slots whether or not anything deserves
them. A band keeps only what scored close to the top, so one clearly
relevant rule still shows one and four competing rules show four. The corpus
decides; the cap is a ceiling on the worst case, not the usual answer.
MEASURED, AND IT CORRECTED THE PREDICTION. The expectation was that rules
would rank sharply, since rule_document() shapes them like snippets and note
2485 measured snippets separating their top hit by 0.153 against 0.010-0.023
for every other kind. Three probes against real act queries say otherwise:
`git push origin dev` top 0.757, gap 0.022
`docker compose up -d` top 0.685, gap 0.016
a bare-owner-filter query top 0.656, gap 0.020
Dev-log territory, not snippet territory — rules arrive as a packed block,
so shaping alone did not buy separation. The band is therefore narrow: at
0.10 (the notes menu's value) every one of the top eight on the push probe
falls inside, including a CI-registry rule and another project's branch
policy. 0.05 admits about three ranks.
COST, MEASURED RATHER THAN ASSUMED. The old comment claimed a line costs
~40 tokens. It is ~143 once the trigger is rendered, and #3855 roughly
tripled trigger lengths, so five full lines are ~646 tokens before EVERY
Bash call. Hence rank decides volume: the top hit keeps the full rendering,
later hits are cited (~198 tokens total, 1.4x the old single line, for four
more rules). The old paragraph's instinct — a fourth voice at full volume is
where a reader stops reading — is answered by making later lines quieter
rather than by refusing to have them.
Band before dedup, deliberately. The band is a statement about scores;
letting the ledger reorder it would make "you were told this already" change
what counts as relevant. Same axis independence the renderer already keeps
between `kind` and `seen`, and `rule_ids` stays fresh-only (#3752) so
#3668's identity between logged results and surfacing rows survives.
`suppressed` now covers both causes and says so.
Both act arms take the same band: their score distributions are the same
shape, and only the query differs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Follows 8c9f947, which taught the trigger shape on the two update_*
surfaces but left the softer regression unguarded: guidance kept and
abstracted back to "name the moment in session vocabulary" — advice about
being concrete that is not itself concrete, which is the shape that was
already on file while the corpus filled with categories.
Two attempts to detect that in free prose were written and discarded:
- Counting quoted multi-word phrases anywhere in a docstring measured
ambient quotation rather than demonstrated triggers. It PASSED the
abstracted version by scoring unrelated prose, and text with an odd
number of quote characters produced matches spanning the gap BETWEEN two
unrelated phrases.
- Scoping that count to a window after each trigger mention then FAILED
create_preference in its CORRECT state, its examples sitting further from
the first mention than any defensible window reaches.
Both were proxies inferring demonstration from prose. Where a property
cannot be measured, changing the shape of the thing is cheaper than a
cleverer measurement — so all five trigger-writing surfaces now carry a
two-line labelled contrast:
RETRIEVES: "the migration failed with a check violation on a column we
just extended"
COLLAPSES: "when working on migrations"
Unambiguous to parse, free in its wording, and a better teaching form than
the sentences it replaces: the labels name the mechanism, so they do work
for the reader rather than only for the test.
The guard now pins both halves — the field is documented, and the contrast
is present, complete and non-identical. Falsified against three regressions
before committing: the paragraph stripped, the examples abstracted away,
and one half of the pair removed. All three fail; the current tree passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
A trigger is two-thirds of a rule's embedded document, so one naming a
CATEGORY rather than a moment collapses the record toward its title and it
never arrives. #3835 measured that across 113 rules; #3855 hit it again on
the eight preferences, where six named a category and two did not.
The split was not carelessness, it was an uneven contract. create_rule has
carried the full argument since c61925b (2026-08-27) and the two
preferences authored that day got good triggers; the six written weeks
earlier got categories. The guidance worked wherever it existed — and it
existed on three of five write surfaces. Both update_* tools were silent,
and the update path is where every RETROFITTED trigger is written, which is
most of them: a trigger that already reads fine as English is the one
nobody rewrites.
So:
- update_rule gains the retrofit case, which is a different trap from the
create case. There the field is empty and the instruction is "write one".
Here one exists, reads perfectly well, and the honest-looking verdict is
that it is fine.
- update_preference gains it too, plus why the field is load-bearing there
specifically: preferences get a reserved slot filled by a kind-filtered
query at limit=1, so the corpus ranks against ITSELF and the trigger is
nearly all that separates one from the next.
- create_preference and create_project_rule now SHOW a moment instead of
describing one. Advice about being concrete that is not itself concrete
is the shape that was already on file while the corpus filled up.
The guard pins one property: a tool taking when_to_apply mentions it. That
is exactly what update_preference failed. The surface list is derived from
register() rather than hand-kept, so a write tool added later is in scope
the day it lands.
Two stronger predicates were written for the softer regression — guidance
kept but abstracted — and both were discarded after falsification: counting
quoted phrases measured ambient quotation and passed the broken version,
and scoping that count to a window failed create_preference while correct.
Rule 167 settles it; the discarded attempts are recorded in the test
docstring so the next author does not repeat them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Milestone 399 step 4. A rule and a preference are not equally served by one
ranking, because their losses are not equal:
- a RULE crowded out at the prompt boundary still fires at an act arm. A
push reaches pre_tool_rule, a write reaches write_path_rule. The prompt
hit is a preview of a second chance.
- a PREFERENCE about how to answer has no second chance. The response IS
the act, so crowded out there it is never delivered at all.
A straight ranking therefore favours the record whose loss is recoverable
over the one whose loss is total, and does it INVISIBLY: the rule that won is
a legitimate hit, the telemetry reads healthy, and the only symptom is a
preference that quietly never arrives. reuse_slot exists for the same shape
one corpus over (#2463).
`semantic_search_rules` gains a `kind` filter, so the slot's query can only
answer with what the slot is for. Verifying afterwards would be weaker — an
unfiltered search that happened to return a rule would spend the slot on it,
and that line would be indistinguishable from one that earned its place.
THE SLOT BUYS POSITION, NOT A LOWER BAR, matching reuse_slot. A weak
preference cannot buy it, so silence stays the default. The task asked for a
separate threshold; I did not add one, and the reason is that the worry
behind it — reading a miss rate as a fact about preferences — is answered by
`preference_slot` being its own logged source, where best_available_id names
which preference was refused. A knob added on a guess is a way to
misconfigure the surface; a bar moved on evidence is an argument. The
evidence arrives on its own now.
IT EXTENDS, IT NEVER DISPLACES — and here it parts from reuse_slot, which
evicts its menu's weakest hit. A displaced hit sits in prompt_rule's
retrieval_logs row while never being surfaced, so that source's two tables
stop agreeing and #3668's identity breaks for a reason nothing in the data
explains. Milestone #379 is what losing that identity costs: five steps
planned against two counters disagreeing, not a write path dropping rows. One
extra line in a rare case is the cheaper price.
It also runs BEFORE the bail-out. An empty general result is not proof no
preference qualifies: that search overfetches by distance then collapses, so
a preference ranked below the window is invisible to it while a kind-filtered
query finds it at once. Bailing first would make the slot dead in exactly the
corpus it exists for.
One existing assertion repinned from a bare call_count to a per-source
filter: the slot logs its own query on the same call, and a count would pin
the number of arms rather than the property — going red the next time one is
added, which is rule 167's false alarm about the thing it protects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
The third rule arm, and the one the other two cannot reach. `write_path_rule`
is keyed on code, `pre_tool_rule` on a command — both things the session is
about to DO. A rule that governs what to SAY has no such trigger: extract
intent from loose phrasing, raise a conflict before acting, hand off an
action with its reason, end a finding with an offer all bind on a RESPONSE,
and no tool call precedes one.
The operator's message is the only query that exists before a response is
composed. That hook searched notes alone, so no rule had ever been retrieved
against a thing the operator actually said — and residency was the only
surface those rules had, which is what milestone 394 removes.
A SEPARATE FUNCTION, not a branch in build_autoinject_hint, because of its
early returns. That arm bails when auto-inject is disabled, when the query is
blank, when nothing clears the note bar — every one a statement about NOTES.
Folded in, an operator who turned the awareness menu off would silently lose
their rules, a coupling with no symptom since both look like a quiet hook.
Two functions, two sets of gates, composed in the route. Guarded as "the rule
arm never asks the notes arm's config", which is the structural fact.
Joins _ARMS rather than getting its own test file. #3497's history is that
the pre-tool arm inherited a defect from its sibling by being MODELLED on it
instead of sharing with it, and a third arm modelled on two is two chances to
repeat that. Repeat rendering, fresh-only counting, log-before-bailout, the
kind register and the two-recorders identity are properties of every arm or
of none.
The bar is INHERITED and says so. 0.72 was tuned against code and commands;
prose is a different query shape against the same documents, and triggers are
written in the vocabulary of the moment — which for most rules is act
vocabulary. Starting at the only number with evidence behind it and logging
every call from the first deploy is what makes it settleable; guessing lower
would put an unmeasured bar in front of a corpus that binds.
k=3, anchored on this hook's own budget rather than the act arms'.
RULEHINT_LIMIT is 1 because that arm fires before every Bash call; this one
fires once per turn, beside a notes menu already spending three slots. And a
prompt genuinely contains more than one act — "merge to main and then start
on X" is two — where a command is one thing.
`prompt_rule` added to RANKED_SOURCES: a ranker picked it, and a ranked
source missing from that tuple is silently counted as bulk delivery and drops
out of the pull-through denominator.
The hook reads and writes the SHARED rule ledger under scribe-priorart, not a
private one — one session keeps one list, aged (#3751), so a rule named here
is not re-announced before the next Bash call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Step 3 shipped a line a session can receive — "Preference that may apply
here …" — into surfaces that told it, in the most authoritative voice it
has, that anything arriving in that shape is binding. That is the confusion
milestone 399 exists to prevent, arriving through the one channel a session
has least reason to doubt.
Silent in both directions, which is why it could not wait for step 6. A
session treating a preference as a rule refuses to proceed over something
the operator merely preferred; and it loses the whole reason preferences
exist, which is that they are brought up to date rather than obeyed.
Three surfaces, each to its own budget:
- SKILL.md gets the full account: kind decides force, the injected line names
which in its opening words, and a preference is the one record a session
keeps current itself (update_preference, with what taught the change).
- scribe_static_context.md gets six lines — enough to tell the kinds apart
and to say a preference is yours to update.
- _INSTRUCTIONS gets four words. It is a MAP at 1978 of its 2000-char budget
(#2562), and the detail belongs in the surfaces above and in the tool
docstrings, which is what that budget exists to force.
Guarded so it cannot drift back: a surface that claims rules bind must name
the kind that does not. Pinned on the CLAIM rather than the word "bind",
because a bare substring also matches bind_repo, list_repo_bindings and
server.py's DNS-rebinding comment — a guard that would one day fail a skill
about repo binding is rule 167's named failure, raising a false alarm about
the very thing it protects. Falsified against all three surfaces losing the
mention.
Plugin version minted: the cache refreshes only on a version bump (#2209),
so a skill edit without one reaches no installed plugin.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Two independent axes on one hint line. `kind` decides the head, `seen`
decides the tail, and neither reads the other — which is what let a second
kind arrive without reopening #3750's repeat question. Whether a record is
already on the exclusion ledger has nothing to do with how much force it
carries, so the seen branch is shared verbatim.
The noun carries the whole visual difference, deliberately. A reader skimming
an injected block gets one word to place the register, so the word that moves
is the one naming force: "Standing rule" / "Preference". Everything
structural after it is identical, so the kinds read as one set rather than
two formats.
Force is asserted in exactly one other place, and that moves too. A rule's
line says to read it BEFORE DECIDING IT DOES NOT APPLY, because dismissing a
rule unread is how the thing it prevents happens. A preference makes no such
claim: it says where to find HOW THIS HAS BEEN DONE BEFORE, and following it
buys consistency rather than correctness.
Guarded on both places at once. Pinning the noun alone would pass a line
reading "Preference … before deciding it does not apply" — label swapped,
instruction kept — which is worse than not distinguishing them, because it
looks handled.
And a guard on the independence claim itself, exercising all four
combinations: the way this breaks silently is a seen branch that grows a kind
test, leaving one combination rendered by nobody's intention.
Noted, not fixed: plugin/skills/using-scribe/SKILL.md still says "Standing
rules are binding" with no room for a kind that does not. That surface is
step 6's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
The write path, and the step where a preference stops being a relabelled
rule. `create_preference` / `update_preference` on the MCP surface, plus
`kind` on update_rule and both HTTP doors.
SEPARATE TOOLS, NOT A `kind=` ARGUMENT. create_rule's docstring IS the
approval gate (#3557): propose, offer three answers, wait. That is right for
a rule — the person it binds should have agreed. A preference inverts it, and
one reached through create_rule would be read through that prose, so the
caller would hesitate over exactly the act this kind exists to make routine.
Two doors, two contracts, one table. Reads stay shared: a preference IS a
rule row, and "what governs this" wants both.
Two required fields, each buying something:
- `when_to_apply`, because the trigger is two-thirds of the embedded
document. Without one the record is written, stored, and silently never
delivered — indistinguishable from one nobody wrote.
- `arose_from_id`, the price of the ungated write. A corpus that drifts with
no record of what taught each change cannot be audited, and the operator's
veto over drift is worth exactly as much as their ability to read why it
happened.
The near-duplicate gate is what lets this corpus be written freely and stay
small: the second preference about a thing updates the first. It is
title-scoped and kind-blind, so it also catches a preference restating a rule
that already binds.
The asymmetry is guarded as two PRESENCE facts — the rule door still asks,
the preference door still says write it — never as an absence. An absence
check passes against a docstring that was deleted or rewritten into something
else, which is snippet #3352's warning and would read as coverage here while
proving nothing.
`_plain_detail` moved to tests/helpers on its second copy, per that module's
own reason for existing (#2825).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Two unit tests build a rule with SimpleNamespace rather than the model, so
adding a column broke them — the fixture has no `kind` for `_rule_rows` to
read. Fixture-only; the export itself was already right, which the existing
column-coverage guard confirmed by passing.
Adds the guard that coverage check cannot make. `_stand_in` walks
`__table__.columns` and proves the KEY is emitted; it cannot prove the VALUE
survives. A preference exported as a rule is a silent failure — the restored
rule reads fine and simply binds when it was only ever a preference — and a
fixture carrying the default would pass against a `_rule_rows` that dropped
the field and let the importer's `or "rule"` refill it. So the new test
asserts on `preference`, the one value that cannot be reconstructed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Adds `kind` to rules — `rule` binds, `preference` is how the operator wants
work done. One column, because the two differ in exactly one dimension and
everything else a preference needs already lives on `rules`: a trigger
column, a trigger-dominated embedding document, ownership-scoped search,
three retrieval arms with telemetry, typed relations, and versioning.
Defaults to `rule`, so nothing changes force on upgrade — 0088's argument
for `tier`, unchanged.
`rule_versions` gets the column too, and that half is not bookkeeping.
`record_if_changed` decides whether an edit deserves a snapshot by comparing
the fields a version carries, so a field absent from SNAPSHOT_FIELDS is a
field whose change records no history at all. Without it, turning a rule
into a preference — the moment something stops binding, and the single most
consequential edit either kind can undergo — would leave the history silent.
Backup carries it through all four seams. A missed one would have restored
every preference as a rule, quietly.
Guarded on real Postgres in three halves: a preference writes, a typo is
refused (without which every other assertion would pass against a table
whose CHECK had been dropped), and a row written with no kind reads back as
`rule` — the migration's whole safety claim, asserted rather than assumed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
#3749 clears the ledger when an EVENT destroys context — a compaction, a
/clear. This is the case with no event at all: a long session where a
rule was named two hundred turns ago and has simply fallen out of
attention. It is #3702's argument at the tier level — present in context
and salient at the moment are different properties — applied to time
instead of to tier.
FORMAT: `id<TAB>epoch`, per entry.
Not a whole-file mtime: that is one line of shell and wrong in exactly
the session that needs it, since a single recent write keeps every stale
id alive, and the ids that go stale first come from the rules that fire
most.
Not a turn counter, though it would be the truer model — an idle session
does not forget. A hook has no turn number without keeping its own, which
is a second piece of session state to write, read, clear on compaction
and get wrong. Wall time costs a `date` call. The failure it accepts is a
session left idle over lunch treating its rules as forgotten, worth one
extra full line per rule and nothing else.
TTL 2700s (45 minutes), reasoned rather than picked (rule 32). About one
working stretch on a single task: long enough that a rule does not
re-announce itself while you are still doing the thing it governs, short
enough that a multi-hour session gets a refresh rather than one 9am
mention. It leans short because since #3750 being wrong on the short side
is the cheaper error — an expired entry costs one full line instead of
one short one, and the exclusion re-arms the moment it is spent. There is
no data on this yet; #3807's near-miss listing is what should revise it.
BOTH READERS THROUGH ONE HELPER, in scribe_defs.sh. The two hooks share
one ledger so a rule named by one arm is not re-offered by the other; a
format only one of them understood would break that on the first read.
The flat `tr '\n' ','` read would now send `156<TAB>1789002860` as an
exclude id — verified, which is why this is not a per-hook edit.
THE LAST ENTRY FOR AN ID WINS. The file is append-only, so a rule that
ages out, is surfaced fresh and is appended again has two lines. Reading
the first leaves it permanently expired, and it then re-announces itself
on every call for the rest of the session — the mechanism meant to
quieten things becoming the loudest thing in the hint.
A BARE ID IS LIVE. That is the pre-#3751 format, and every session in
flight when this ships has a ledger full of them. Reading unknown as
EXPIRED would make all of those sessions re-announce every rule they had
already been told, at once — the exact noise this prevents, delivered by
the feature on the day it ships. Unknown means "not measured", never
"old", the same discipline the nullable retrieval_logs columns use.
GUARDS (tests/test_rule_ledger_ageing.py, real shell, no credentials)
- old ages out AND recent survives, in ONE assertion (rule 167): either
half alone passes against a broken helper — "old is gone" passes
against one returning nothing, "recent survives" passes against the
flat read this replaces, i.e. against the defect itself.
- the ping-pong case, which is the one that costs the most to get wrong.
- a bare id is live, including beside a stale stamped one.
- missing/empty ledger excludes nothing.
- an id repeated in the ledger appears once, with no empty list element.
- structural: neither hook reads the rule ledger flat again — pinned on
the flat-read SHAPE, so a rename of the helper is not a failure and a
hook that ages correctly some other way is not either.
The TTL's VALUE is deliberately not asserted. The tests read the constant
out of the shell and assert the property around it, so a later tuning
change stays a tuning change instead of a red build.
Dropped a boundary test (`ttl` vs `ttl + 1`) before committing: racy by
construction, since a ledger written at T is read at T+n and the two
cases swap. A one-second distinction on a 45-minute window is also not
observable behaviour, so it pinned a flake rather than a property.
Plugin version minted to 2026.09.10.0221 — this is entirely hook-side,
so without the bump the cache would never pick it up and the merge would
ship nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Both arms used to drop a hit already on the session's exclusion ledger
and emit nothing. That is correct only while the session still HOLDS
what it was told, and a compaction breaks exactly that: the earlier
injection is summarized away while the id stays on the ledger, leaving
the rule absent from context AND unreachable for the rest of the
session. #3749 closed the compaction half by clearing the ledger; this
closes the ordinary half, where a session simply stops holding a line
it read an hour ago.
Only ONE CLAUSE of the existing line is false on a repeat — the claim
that the rule is not in the session's loaded set — so only that clause
changes. The fresh line is byte-identical to what it was.
Both tails now come from one `_rule_hint_line`. The arms phrase their
heads differently on purpose; everything after must not differ, and
#3497's history is that the pre-tool arm inherited a defect by being
modelled on its sibling rather than sharing with it.
THE BUDGET DECISION, recorded at RULEHINT_LIMIT. A repeat competes for
the single slot on rank alone: nothing is fetched behind it, and it
never rides alongside as a second line. Promoting a fresh rule past a
better-ranked repeat would reinstate the withholding one rank deeper,
and a second line is the one thing the limit exists to forbid. The
consequence is deliberate — a rule that keeps ranking first for a
recurring situation keeps being referenced, and its decay belongs to
exclusion ageing (#3751), not to a first-place rule being demoted for
having won before.
THE TELEMETRY, decided before shipping rather than after a number moved
(#3752): nothing changes. A reference is a RENDERING decision, not a
retrieval outcome. `results` stays `fresh`, `suppressed` stays
len(hits) - len(fresh), and `record_rule_surfaced` still counts only
what the arm freshly chose. This matters more than it reads: the naive
implementation drops the `fresh` filter and takes suppressed_count to
zero everywhere — and #3739's near-miss fix identifies repeat-caused
zeros by `suppressed_count > 0`, so the contamination corrected on
2026-09-08 would return by a different route, in the same field, with
the fix still in the code and no longer working. A test asserts the
counters as unmoved, because "nothing changed" is only worth something
if it is checkable.
Also corrects two comments that outlived #3702 — both arms still
claimed CONDITIONAL ONLY while the module-level note above
RULEHINT_LIMIT said the opposite, in the exact code this change edits.
GUARDS
- an already-held hit produces a rule line at all (the regression),
asserted on `get_rule(<id>)` rather than a truthy context: the
write-path arm fills its context from four other sources, so
truthiness passes under the OLD behaviour and pins nothing there.
- the two tails are distinguishable, each excludes the other, and
neither injects the rule statement — the budget claim, both branches.
- the counters are unmoved, per #3752.
- test_a_rule_the_session_already_holds_is_not_re_offered asserted the
old contract (`"161" not in context`). Repinned rather than deleted:
the telemetry half of what it protected still holds.
Repinned the #3497 log-placement guard on structure (rule 167). It
read `body.index("if not fresh:")` — a local variable NAME, not the
property. This change renames that guard to `if not hits:`, so the
old assertion would have raised ValueError and reported #3497 as
back while the arm was entirely correct. Now walks the AST for the
first early return after the search and asserts the call row is
written before it. Falsified both ways: it fails on the #3497
mutation, and it refuses to pass when no early return exists at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
#3670 added `best_available_score` so a threshold could be judged from its
rejections. It records how CLOSE the bar came to firing and not WHAT it
refused, and that is the half a decision actually needs.
Live, pre_tool_rule sits at a ~0.72 bar with a near-miss p90 of 0.7071 —
about 117 declines a day within 0.013 of firing. Dropping to 0.707 would
take that arm from 22 hits a day to roughly 139: six-fold, on a surface
that runs before every Bash call. The percentile says the mass is there.
Nothing said whether it was worth showing.
NEITHER OBVIOUS INSTRUMENT ANSWERS IT. Pull-through cannot: the injected
rule line already carries title and trigger, so a session can comply
without ever calling get_rule, and rule pull-through understates
usefulness by construction. Reading the rejected records can — and
`result_ids` holds only what was RETURNED, so on a zero-result call the
near-missed record had no name at all.
So the id, from the SAME ranked candidate as the score. Both searches
unpack `best` once and read both fields off it, because splitting that
into two expressions is exactly how a later edit pairs a score with its
neighbour's id — and a score attached to the wrong record is worse than no
id, since it invites judging the wrong one and concluding the bar is fine.
write_path withholds the id on the same condition it withholds the score
(#3739): a surviving id beside a null score names a record without saying
what it scored, the pair disagreeing in the other direction.
THE READ PATH IS A LISTING, NOT A STATISTIC — an id cannot be percentiled,
and a reader tuning a bar needs to go and read the records. Opt-in via
`near_miss_samples` (0-20, default 0) so the ordinary readout keeps its
size, and deliberately NOT a window function: this module's one production
outage was a grouped query Postgres rejected, swallowed by the broad
except, every counter reading zero while the mocked tests passed (#2663).
One flat ordered query, overfetched, bucketed in Python — the shape that
lesson prescribes.
Migration 0097, nullable and unbackfilled. Not a foreign key: the table
spans record types and `source` says which, exactly as result_ids works.
The integration guard pins the listing as PER SOURCE. A global LIMIT would
let a noisy source eat the whole quota and leave the surface being tuned
showing nothing — which reads as "nothing was close", the misreading this
milestone has spent itself correcting.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
`best_available_score` was added by #3670 so a bar could be judged from
what it rejected, and it arrived null on four unrelated causes: the corpus
offered nothing, the query was empty, the embedder was down, or the
DATABASE QUERY FAILED. Only the first is a measurement. The fourth is the
#2663 shape — a swallowed failure rendering as a clean zero — inside the
field added to fix an instance of the #2663 shape.
Found while trying to explain why reuse_slot returned nothing on 45 of 45
calls, and auto_inject on 153 of 161. That investigation is still open;
what it established first is that the readout could not answer it.
THE FIX IS NOT A NEW COLUMN. A call that never searched writes no row, so
every remaining null means one thing: searched, and nothing came close.
That is the convention the pre-tool arm already follows for a blank
command — "a row here would report a call that never happened and drag the
clear-rate down with phantom declines" — extended from the case a caller
can see in advance to the ones only the search knows about.
Both searches stamp `report["searched"]` FALSE before anything can return
and True only where a real result set exists, so every early return leaves
it false. It has to be the first thing done to the dict: a return added
above that line would leave the key absent.
ABSENT IS A THIRD STATE AND IT DEFAULTS TO TRUE. A caller that passes no
report cannot know, and the safe reading there is the old behaviour. Only
a real search can report False, so absent means "nobody asked" and never
"it failed" — which is also why 66 existing mocked searches across twelve
test files keep working unchanged rather than being rewritten to simulate
a flag they do not care about.
A FAILURE IS NOT MADE INVISIBLE. semantic_search_notes already logs a
WARNING on a query failure, which is where a broken search belongs: a
counter cannot say "I am broken" without a reader already trusting it.
Three tests, and the middle one is what makes them discriminate — a
blanket `return` passes the first and fails the second, because a call
that searched and came back empty is the only evidence a threshold is too
high (#3497).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Found by verifying the previous fix on live data — the check that fix was
meant to make possible.
write_path near_misses.max 0.822 p90 0.7521
top_score.min 0.6857 so the bar is at or below this
A "rejection" that outscored every acceptance, and not one outlier: the p90
is above the bar too.
#3739's fix keyed on `suppressed_count`, and I justified its NULL branch as
"null means the caller passed its exclusions INTO the search, so the score
is already post-exclusion". That holds for auto_inject and reuse_slot, both
of which log the RAW search output and do their Python filtering after. It
does not hold for write_path, the one note arm that filters TWICE:
`exclude_ids` takes `seen - pulled_seen` into the search, but the
pulled-and-seen ids stay in the query on purpose — the arm's query doubles
as the resemblance test — and are dropped afterwards in Python. So the row
carries a POST-filter count beside a PRE-filter score.
The suppression column cannot rescue it the way it does for the rule arms.
This arm's count would be PARTIAL — covering the drops made here and not
the ones `exclude_ids` made inside the search — and a partial number under
a name that reads as complete is the substitution this milestone exists to
stop.
So it reports null whenever its own filter removed anything: not measured
on this call, because the bar was not the only thing that turned something
away. Calls that withheld nothing keep reporting, which is most of them.
Both directions are asserted. Without the second test, setting the field to
null unconditionally would pass the first while deleting the measurement
#3670 was built for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
The manifest gate caught this, which is what it is for:
FAIL plugin content changed but the version is still 2026.09.04.0140.
An install has two halves and only one self-updates. The marketplace clone
pulls on its own; the cache that actually EXECUTES refreshes only when this
string changes. So a hook edit shipped without a bump reaches the repo and
stops there — and the obvious debugging move, inspecting the clone, shows
the fix present while the broken copy keeps running. That is #2209, #1040
and #2220, and the only detector was the operator saying "I don't think it
updated".
Minted with scripts/mint_plugin_version.py rather than hand-edited: the
plugin ships straight from the repo with no build step, so there is no
moment at which CI could stamp a value, and the script is the path the
mint guard pins.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
The prior-art and tool-rule hooks record every rule id they have named in
<state>/<sid>.rules.ids and hand it back as exclude_rule_ids, so a rule is
surfaced once per session and then goes quiet. That is correct while the
session still HOLDS what it was told.
A compaction breaks it in the worst available way: it summarizes the
earlier injections out of context and does not touch the filesystem. The
rule ends up absent from context AND still excluded — unreachable for the
rest of the session. The compaction banner this hook already prints tells
the model to re-pull its ALWAYS-ON rules, but a rule an arm surfaced is
conditional and is not in that set, so it has no other way back. The rules
most likely to be in that state are the ones that fire most often.
The stale ledger is genuinely found again rather than orphaned: the etag
marker further down this same hook is rewritten on `compact` and keyed by
session_id, which is only meaningful if the id survives a compaction.
CLEARED ON THE SOURCES THAT DESTROY CONTEXT, AND ONLY THOSE. `compact` and
`clear` destroy it while the file survives. `resume` does not — the
context came back intact, so clearing there would re-surface every rule
after a restore that lost nothing, which is the same defect from the other
side. `startup` is a no-op against a new session id. `fork` keeps it, and
the answer holds whichever way forks are keyed: a fork carries the
conversation, so an inherited id means an accurate ledger and a new id
means an empty file.
Only the RULE ledger. The same directory holds .ids / .sync.ids /
.derive.ids for the note arms; whether a surfaced note should return after
a compaction is a different question with a different answer, and a `rm`
glob would have decided it silently.
The guard pins the DISCRIMINATION, not the deletion: the whole source
table is asserted in one statement, so a blanket delete (all False) and a
no-op (all True) both fail, and neither can be made to pass by editing one
case. A second test pins the scope against that glob, and a third proves
an event with no session id clears nothing rather than falling back to a
wildcard.
Runs with no SCRIBE_URL/SCRIBE_TOKEN on purpose — the clear is local,
keyless and networkless, and must still happen against an unreachable
instance. That is also why it sits above the config read rather than
inside the dynamic tier.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Caught on the first live read after deploying #3670. The readout
contradicted itself:
pre_tool_rule top_score.min 0.7204 the lowest score ever RETURNED
near_misses.max 0.7457 "rejected", but scored higher
`best_available_score` is measured pre-threshold, which is right, but for
the rule arms it is also PRE-EXCLUSION, which is not. The note arms pass
`exclude_ids` into semantic_search_notes so their score is already
post-exclusion and clean; `semantic_search_rules` takes no such parameter,
so the rule arms filter in Python after the search and a rule that cleared
the bar and was dropped as a repeat still reported its score on a
zero-result row.
That is #3497's distinction — a ranker decline versus a reader already
ahead of it — reintroduced one level up, inside the field built to replace
a tautology.
The population now also requires `suppressed_count IS NULL OR = 0`. The
NULL arm is principled rather than permissive: null means the caller
filtered INSIDE the search, which is exactly the case where the reported
score cannot be contaminated.
Deliberately conservative — a call carrying both a repeat and a lower
genuine miss is dropped whole, losing that point. It undercounts; it
cannot corrupt, which is the right way round for a number read against a
bar.
It also makes `near_misses.max < threshold` true BY CONSTRUCTION rather
than by fixture: an above-bar candidate nobody excluded would have been
returned, so its call is not in the population at all.
THE TEST DID NOT CATCH THIS, and that is the part worth keeping. The
assertion `nm["max"] < 0.72` was already there, with exactly the right
intent. It passed because the fixture contained no suppressed call — the
guard held because the breaking shape was absent, not because the code was
right. Rule 167's stated failure mode, in a test written while citing rule
167. The fixture now builds that shape: a 0.9 hit dropped as a repeat,
which lands in the population and drags `max` above the threshold unless
the predicate excludes it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
`cleared_threshold` was documented as the number to read first. It was a
tautology. The search applies the threshold before returning, so every
returned result cleared it by construction and a call with no results has
no top_score to compare — the condition was true exactly when
`result_count > 0`. It was `calls - zero_result_calls` under a name that
promised a second opinion, and `zero + cleared == calls` held on all
nineteen source/window readings ever taken, today's live seven included.
The reading procedure built on it asked the reader to compare a number
with itself, and a threshold change was unobservable through it: raise the
bar and both numbers move together, so the field could never show a bar
set too high.
REPLACED, NOT JUST REMOVED. The question the table exists to answer is
whether the bar is in the right place, and that is only answerable from
the calls that returned NOTHING: how close did the best rejected candidate
come? A 0.72 bar turning away a stream of 0.71s is set too high by a hair;
the same bar turning away 0.30s is working. Both render as a zero-result
call today and nothing separates them, because the losing score is
discarded inside the search.
So both searches now rank WITHOUT the bar and apply it in Python. The
qualifying set is provably identical — rows arrive ordered by distance, so
every above-bar row sorts ahead of every below-bar one, and an over-fetch
that returned N above-bar rows returns the same N plus some losers. What
changes is that the losers are visible instead of dropped in the query.
`report` carries the score out without changing what a search RETURNS:
eight of eleven call sites want hits and nothing else.
New column (migration 0096), nullable and unbackfilled. A row written
before this genuinely does not know, and a 0.0 would read as "the corpus
held nothing remotely relevant" — a claim invented out of a caller's
silence, which is the substitution this whole milestone corrects.
The new aggregate is a percentile_cont WITHIN GROUP over a CASE, one step
from the shape that produced #2663, where a rejected query was swallowed
by the broad except and every counter read zero. It carries an integration
guard for that reason: only real Postgres can say it parses, and the
symptom of failure is silence.
Also adds a guard that no int field in a bucket equals
`calls - zero_result_calls`. That identity is what `cleared_threshold`
satisfied for its whole life, and it survived because it had its own name
and nobody added the two numbers beside it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Reverts the deliberate break from 5e19a1b. `plugin_context.py` is now
byte-identical to before it; the only change against that baseline is the
guard itself.
FALSIFIED, not argued (rule 167). CI run 6046 with `results=hits` in both
arms failed exactly the predicted four cases —
[one-already-held-write_path] FAILED
[one-already-held-pre_tool] FAILED
[all-already-held-write_path] FAILED
[all-already-held-pre_tool] FAILED
[nothing-held-*] passed
— and the nothing-held cases passing is the point, not a gap: with no
exclusions both recorders see the same list however wrongly they are
wired, so that case can never discriminate and a guard built only from it
would read as coverage while catching nothing.
WHAT IS PINNED. Both arms build one `fresh` list and hand it to two
recorders in one function, so the call log and the surfacing log cannot
disagree about what a single call showed. Ids, not counts: equal counts
drawn from different lists is a real way for this to break, and a count
comparison would call it agreement.
Three hits, where production returns at most one. The identity holds at
any limit because both recorders read the same list, and stating it that
way survives RULEHINT_LIMIT moving again — it has moved once already
(2 → 1, 2385100), and that move is half of why the original
reconstruction misread its own numbers.
NOT DONE, deliberately: the readout-level self-check the task also
proposed. `cleared_threshold` counts CALLS that beat the bar while
`surfaced` counts RULES, so that identity holds only while the limit is
1 — it would fire on a healthy system the moment the limit rises. The
arm-level form has no such coupling, which is why the guard lives here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Deliberately broken, reverted in the next commit. Rule 167 requires every
guard be falsified against the regression it names before it is trusted,
and rule 10 puts CI as the only place that can run it — so the failure has
to be made to happen here rather than argued for.
The guard: both rule arms feed one `fresh` list to two recorders, so the
call log and the surfacing log cannot disagree about what one call showed.
The regression: `results=fresh` becomes `results=hits` in both arms, so
the call log counts what the ranker found while the surfacing log counts
what was shown. That is not a hypothetical shape. It is exactly the
divergence that would make a correct system report a lost write when the
two tables are later compared in aggregate — the reading that scoped this
milestone at five steps against a defect that did not exist.
Expected red: the one-already-held and all-already-held cases on both
arms. The nothing-held case must still PASS — with no exclusions both
recorders see the same list however wrongly they are wired, which is why
it could never have been the discriminating case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
`out["sources"]` was built only from the windowed aggregate, so a source
with rows in `retrieval_logs` but none inside the window got no bucket at
all. Absent is exactly how a source that never existed renders, so a
surface that WAS recording and went silent became unreadable — #2663 one
level up, the failure that looks like the correct answer.
Two queries at different scopes, and only one shaped the output.
`_complete_from` reads all-time and knows every source the table has ever
held; the windowed loop dropped whatever it did not return.
Every such source now gets a zero bucket. Zero is a real measurement here
rather than a manufactured one: the all-time query proves the source was
recording, and it made no calls across a window it fully covers. No
`covers_window` special case is needed either — a source whose first row
fell after `since` would have that row IN the window and already hold a
bucket, so anything reaching this branch began before it.
The counts are 0 and everything else is null. A sampled distribution is
not the same claim as a call count, and rendering p50 as 0.0 for a source
nobody sampled would assert a measurement — #3311's mistake, in the
readout built to prevent it.
Found while fixing #3712's fixture, which failed with KeyError for this
exact reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
The per-source grain test gave `auto_inject` a single row 90 days back and
`pre_tool_rule` one 2 days back, then asserted on both buckets. Only the
young arm got a bucket: `out["sources"]` is built from the WINDOWED query,
so a source with no rows inside the window is absent entirely, and the
assertion died on KeyError before it could test anything.
`complete_from` and the bucket come from different queries — all-time for
the first, windowed for the second — and the fixture only satisfied one of
them. Gave `auto_inject` a second row inside the window, which is also the
shape being described: an old surface that is STILL recording. The 90-day
row still sets its `complete_from`.
Still discriminating: auto_inject reads True and pre_tool_rule False, and a
per-table `_complete_from` would make both 90 days and fail the second
assertion — the regression this test is for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
`_complete_from` and `_coverage` landed between `retrieval_summary`'s
docstring and its body. Python does not object to that the way it looks
like it should: blank lines do not close a block, so the whole remaining
body — indented four spaces, sitting after `_coverage`'s `return` —
became unreachable code INSIDE `_coverage`, and `retrieval_summary`
became a function that is nothing but a docstring.
The error surfaced three ways at once, none of which named the cause:
fourteen F821s for `days` and `user_id` (real: those are
`retrieval_summary`'s parameters, and the body no longer lived there), a
SyntaxError on `async with` (real: `_coverage` is sync), and every test
module that imports this file failing to collect.
Moved both helpers above `retrieval_summary`, beside `_bucket` and
`_round`, where the file's other helpers already are. No behaviour
change — this is the code that was meant to be there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
A window that opens before a counter existed reports that counter as though
it had been measured throughout. The reader cannot tell "zero because
nothing happened" from "zero because nobody was counting yet", and — worse
— cannot tell a partial count from a complete one. That middle case yields
a plausible FRACTION rather than an obvious zero, which is what makes it
dangerous.
It is not hypothetical. A 7-day window opened while the ranked rule
surfacing recorders were four days old produced an apparent 64% write loss,
which survived a code review, four ruled-out alternative causes and a
five-step milestone before an identity check falsified it in one read.
Every counter block now carries `complete_from` and `covers_window`.
THE GRAIN IS THE SOURCE. retrieval_logs accumulates for months, so a
per-table earliest row says months for every source it holds — including an
arm added days ago whose counter means something else entirely. The old
source would vouch for the young one, which is the exact reading this
prevents.
A SECTION TAKES ITS LATEST CONTRIBUTOR, NOT ITS EARLIEST. A figure summing
several sources is complete only once every one of them was being written,
so "*" is a max. Using min would reproduce the original error in miniature.
`covers_window` is null, never false, when nothing was ever recorded: "no
measurement" is not "partial measurement" — the null convention #3497
established for `suppression`, one level up.
Also corrects a stale claim in the tool docstring: it still taught readers
that write_path_rule "has never once declined to fire" (#3311). That was
the arm writing its retrieval_logs row only on calls that found something;
#3497 fixed it, and the arm declines the large majority of its calls.
_complete_from takes the caller's session rather than opening its own,
departing from the services canon (#2860) because it runs inside an
existing block; to be recorded against the ledger once it ingests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Both arms passed tier="conditional", on the reasoning that an always-on
rule is already in the session so re-surfacing it is pure noise. That
conflates two different things:
PRESENT IN CONTEXT the rule was delivered at session start
SALIENT AT THE MOMENT the rule is in front of the reader when the action
it governs is about to be taken
A rule handed over in a list at turn zero is present while a session writes
a config value three hundred turns later. It is not surfaced. So the filter
did not skip a redundant hint — it made a whole class of rules permanently
ineligible for the only mechanism that puts a rule in front of an agent AT
the moment, and the more important a rule is, the likelier it sat in that
class.
Underneath, the filter was doing the THRESHOLD's job. Whether a rule belongs
in a hint is a relevance question and a similarity bar is the control for
relevance. A categorical exclusion standing in for a relevance judgment
cannot be tuned, cannot be measured, and cannot be wrong in a way anybody
notices.
MEASURED, NOT SETTLED. The old comment's fear is real: a hint that fires on
every write and says obvious things teaches the reader to skip the block. It
had simply never been checked, and retrieval_logs already records the scores
to check it with. Rules clearing often and high means the fear was justified
and the BAR is the work; rules clearing rarely in a thin band means relevance
was always sufficient.
Only eligibility moved. The bar stays at 0.72 and k stays at 1, so the
resulting distribution has one cause — and k=1 bounds the blast radius: a
wider pool can change which rule surfaces and how often, never how long a
single hint gets.
The threshold rationale's first premise ("the eligible corpus is TINY —
tier=conditional only") is updated rather than deleted: a larger pool makes
clearing the bar mean MORE, so that argument weakened, and the bar was left
alone anyway rather than move two variables at once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
The first cut opened "NOT YOURS TO CALL UNPROMPTED", and that is the wrong
instrument. A caller reading a prohibition stops NOTICING rule-shaped things
rather than noticing them and asking — which trades a small failure for a
larger one. The wanted behaviour is more proposals, not fewer.
So both docstrings now describe the practice: propose readily, state the
four things, and close with a question the operator answers in one word —
approve it as written / let's talk about it / no. Named options where the
interface has them, three written-out options where it does not.
"Approve it as written" is what makes element 1 load-bearing: they approved
TEXT, so that text is stored verbatim. "Let's talk about it" is framed as
the expected answer rather than a setback. "No" routes the observation to
create_note, which records without binding.
The argument for asking is also better than consent. The operator's yes is
the one moment the rule is certainly in front of them: afterwards a
conditional rule is not read aloud at session start, and a project rule is
absent from an unfiltered list_rules(). The proposal IS the review.
Guard gains the answers-offered-back element and drops the wording that
forbade; its header records why the framing changed, so the prohibition does
not get reintroduced as a tidy-up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Every gate on create_rule and create_project_rule was about SHAPE — rule vs
process vs snippet, one-thing-you-could-violate, general-enough, not-a-dupe.
All of them improve a rule someone has already decided to write. None asked
the prior question: has the person this will bind agreed to be bound by it?
Both docstrings now open with the gate, and with the four things a proposal
carries: what it would require in the words it would carry, its intent, why
now, and how it would be enforced.
The fourth is the one that decides it. "A test, a CI check, a hook, a schema
constraint... or nothing" is a question that sometimes dissolves the rule:
what a test can assert should BE that test, and a rule is what is left when
nothing mechanical can hold the thing. A rulebook grows by default and
shrinks only on purpose.
create_project_rule needs the gate more, not less, and says so: a project
rule is absent from an unfiltered list_rules(), and a conditional one is
absent from session start too, so one written there can bind for months
without ever having been in front of the person it binds.
test_rule_creation_asks_first pins structure, never wording — each element
matches a family of synonyms, and the gate must precede the Args: block,
because a caller who has decided to make the call reads the parameters and
not the prose under them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Every instruction surface told a session to load the always-on rules and
stopped there. None said the loaded set is partial, so an empty one read as
"no rule applies" when it only ever meant "none was pushed" — different
claims, and only one of them has been checked.
That is #2198's asymmetry one level in. The earlier defect was trusting the
SessionStart push over the explicit pull; this is trusting the resident TIER
as if it were the whole rulebook.
It is also why the always-on tier was the only one that worked, on any install
rather than this one (rule 115): a rule nothing retrieves must be resident to
bind at all, and a resident rule costs tokens in every session forever — so a
rulebook that only delivers cannot grow past what one session holds.
Retrieval lifts that ceiling, and it fires only if something asks. A
tool-choice reflex asks least of all (#3476, #161).
The same obligation now lands on all three session-start surfaces, because
rule 119 makes them the specification jointly and a surface stating it
differently IS the product behaving differently (#2497). Pinned by
test_every_session_start_surface_states_the_conditional_retrieval, mirroring
the pull test beside it.
THE BUDGET TRADE. _INSTRUCTIONS sat at 1978 against a 2000 test budget, and
its own comment says an addition there is a trade, never an append. Bought the
new clause by trading out "Processes are saved procedures (follow verbatim)"
and "Deletes are trash-recoverable" — both already in DISPLACED_TOPICS and
already stated on a delivered surface, and both per-tool guidance, which by
this block's doctrine belongs in the tool docstring. Now 1976. Recorded in the
comment above the block so it is not silently reversed.
Plugin version minted: shipped plugin content moved.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Making the rule arms log every call exposed a second ambiguity in the same
row. `result_count == 0` is two unrelated events wearing one number:
- the ranker found nothing above the bar — the only evidence a threshold is
set too high; and
- the ranker found only what this session had already been shown — which
says nothing whatever about the bar.
A long session excludes its way into the second, so the arm reads worse the
longer it runs correctly. Rows written now carry the ambiguity permanently,
which is why this lands before any watch period rather than after.
`retrieval_logs.suppressed_count` (0095, nullable) holds what the caller
dropped as already-shown. Both rule arms report it; they filter in Python and
always know. The note arms pass exclusions INTO semantic_search_notes and
never see what was dropped, so they store NULL.
THE NULL IS LOAD-BEARING. It means "not measured here", and the readout
renders it as `suppression: null` rather than a zeroed dict. Defaulting to 0
would let an unmeasured surface read as a perfectly clean one — the same
substitution of an artifact for a measurement that #3311 made. No backfill,
for the same reason: existing rows genuinely do not know.
`retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`,
`calls_with_suppression` and `zero_because_already_shown`; subtract the last
from `zero_result_calls` for the true ranker declines. The MCP tool docstring
says to read the two together and warns against reading the null as a zero.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
`assert_called_once` held only because the rule arm skipped its retrieval_logs
row when it found nothing. With the arm logging every call, the test now
asserts what it was always about — exactly one `write_path` row, no
`auto_inject`, and the rule arm keeping its own separate source.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
`write_path_rule` reported `zero_result_calls: 0` and `cleared_threshold:
133/133` — a perfect record no other surface comes near (`write_path` 421
zeroes of 613, `reuse_slot` 124/199, `auto_inject` 114/326). #3311 read that
as a measurement and milestone 333 was scoped on it.
It was an artifact. Both arms called `record_retrieval` inside a guard on
having results — the write-path arm behind `if fresh:`, the pre-tool arm
below `if not fresh: return out` — so a call that found nothing wrote no row.
The statistic was a fact about the shape of the code, true at any threshold
whatsoever.
The call log moves out of the guard in both arms. The surfacing log stays in
it: nothing was shown, so no surfacing occurred. `results=fresh` is kept
deliberately — the note arms pass exclusions into `semantic_search_notes`, so
what they log is already post-exclusion, and logging `hits` here would make
this row mean something other than every other row in the same readout.
The defect bites hardest on the pre-tool arm, which fires on every Bash call:
with no rows at all, a ranker that declined is indistinguishable from a hook
that never fired — the silent failure the arm exists to stop.
Tests cover both arms behaviourally (found nothing; found only what the
session already held; searched nothing at all, which must stay silent) plus a
structural guard, because this was one level of indentation and it appeared
independently in two places.
#3311 and the `rule_usage` docstring corrected rather than quietly rewritten.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
The only just-in-time rule surface was registered on `Write|Edit` and queried
with `code or path`, so a rule could be retrieved at the moment of a code
write and nowhere else. Every rule about which tool to reach for — don't curl
the forge, don't stand up a stack, don't run the suite locally, don't branch —
was unreachable exactly when it mattered, and residency in the always-on
preload was the only surface it had. That is the pressure that grew the
resident set to 31 against #3089's ceiling of ~23; it was never a judgment
anybody made.
A reflex generates no query, so an instruction to check the rules cannot catch
one. A mechanical trigger can: the tool call IS the query, and a reflex has to
become a tool call before it can do anything.
`build_tool_rule_hint` is deliberately tool-agnostic — a name and a string —
so widening the matcher later is a hooks.json edit with no server change. The
hook starts on Bash, which is where the action reflexes live.
The two pre-tool arms share ONE session ledger of already-named rules
(`<state>/<sid>.rules.ids`). Two ledgers would mean a rule named by one arm
gets re-offered by the other, and the hint that fires most often is exactly
the one that must not repeat itself. A test asserts both scripts build the
same path, and another checks the shell hook and the Python route agree on
every query-arg name (rule 33) — a rename there fails silently, looking like
a surface that never finds anything rather than a broken one.
Deliberately silent on outage, unlike the prior-art hook: a write is
occasional, a Bash call is not, and an outage line before every command is
what gets a channel muted.
`tier="conditional"` matches the write arm and is the transition point — an
always-on rule is already resident, so re-tier one and it starts arriving here
instead of in every session's preamble. `pre_tool_rule` joins RANKED_SOURCES:
this arm chose what it showed, so a pull can settle whether the choice landed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
The ranked rule arm became measurable in M333. The preload did not — and
that is the surface whose value is actually in question. `list_always_on_rules`,
the SessionStart block and every `rules_payload` caller handed rules over
wholesale and emitted nothing, so the resident set's token cost was certain
and its usefulness could not be tested even in principle.
Bulk deliveries now record as AMBIENT, beside the ranked count and never
inside pull-through. Folding them in would mean growing the always-on set
depressed the arm's measured precision and trimming it flattered the arm,
neither for any reason to do with the arm.
`RANKED_SOURCES` inverts the note twin's `AMBIENT_SOURCES` deliberately: there
is one ranked rule source and this change adds seven bulk ones, so naming the
rare half makes a forgotten surface default to ambient — under-counting it —
rather than padding the denominator with surfacings nobody chose.
Two lookalike call sites are deliberately left silent, with a test to keep
them that way: the write-path etag arm and `rules_etag_for` read the rules to
build or compare a MARKER and show nobody anything.
No migration — `event` and `source` are plain Text with no CHECK (rule 36
does not apply). Snippet #2858 updated to the new `rules_payload` contract.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
Milestone 333 step 5, and rule 27 — the counter had a tuning point from step
4 and no operator-facing one until now.
The task said to reuse the snippet badge's classes rather than mint a parallel
set, citing the eight duplicated CSS families the ledger already carries
(#3207). `.usage-tag` lived in SnippetListView's SCOPED block, so "reuse" was
not available: copying it into the rule pane would have been the ninth family,
and importing it is not a thing a scoped block permits. So it was promoted
rather than copied.
Three pieces, each of which existed once and now exists once:
- `components.css` gains `.usage-tag` / `.usage-dead`, geometry and colour
only, with the scoped original deleted rather than left behind.
- `UsageBadge.vue` holds the logic the two lists would otherwise duplicate —
the >=3 dead-weight threshold, the empty-string-renders-nothing rule, the
tooltip.
- `types/usage.ts` holds `RecordUsage`, one client type over two tables.
`SnippetUsage` becomes an alias, so no existing consumer changes.
THE ADVICE IS A PROP, and that is the substance rather than the plumbing. The
counts read identically for every kind; the remedy does not. A snippet offered
and never opened should probably be rewritten or deleted — one action. A rule
in the same position has TWO possible causes and the operator has to pick:
its trigger may fire on the wrong work, in which case `when_to_apply` wants
rewording, or it may genuinely not be wanted. Baking "delete it" into the
component would give the wrong nudge half the time on the surface where being
wrong is most expensive, since a deleted rule stops binding behaviour.
The route zero-fills every row through `usage_for_rules`, one aggregate per
page — per-row would be N+1 by construction. That matters more here than for
snippets: every rule on every existing install predates `rule_usage_events`,
so the zero-filled shape IS the common case for a while, and a route that
attached the key only where it found events would leave the badge reading
undefined on almost every row.
`usage_for_rules` had no test at all — step 1 covered the write path and the
zero shape and left the aggregate uncovered, which only became load-bearing
when a list started rendering it. It now has an integration test over real
Postgres, including that a rule with no events comes back zero-filled rather
than absent.
Recorded as snippet #3460, per the design system's own instruction that the
component layer lives as snippets rather than as prose.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
Milestone 333 step 4 — the split #2223 made one surface down, now made for the
third corpus. The arm inherited WRITEPATH_DEFAULT_THRESHOLD = 0.68, a number
measured against code-vs-note-PROSE and never re-derived for code-vs-RULE-TEXT.
THE DEFAULT IS ARGUED STRUCTURALLY, NOT READ OFF A HISTOGRAM (rule 115). Two
facts hold on any install, including one with six rules and no telemetry:
- The eligible corpus is tiny — conditional rules only, a handful to a few
dozen against thousands of notes. A top-k over forty candidates always
returns something, so "the best match cleared the bar" stops meaning "a good
match exists". A bar calibrated for best-of-thousands is cleared by
best-of-forty as arithmetic, not relevance.
- Rules are short imperative technical English, far more homogeneous than note
prose. #2223 put the code-vs-prose floor at 0.55-0.63 and set 0.68 above it;
a more homogeneous corpus has a HIGHER floor, so 0.68 is not merely
inherited, it sits below where this corpus's noise lives.
0.72 errs deliberately toward silence on an asymmetry that is also structural:
this hint fires on EVERY write. A missed rule is recoverable — it is still in
Scribe and the agent can search it. A hint that cries wolf is not: it teaches
the reader to skip the whole block, and the true positives go with it. The
arm's own comment already said "noise on a hint that fires on every write is
how a hint gets ignored".
Pinned as an INEQUALITY, not a value: test_the_rule_bar_defaults_above_the_code_bar
asserts RULEHINT > WRITEPATH, so tuning the number stays free while inverting
the relationship — which would silently reinstate #3311 — does not.
RULEHINT_LIMIT = 1, and deliberately not a knob. With a corpus this small, k=2
means the second line is almost always the second-best noise wearing the same
confident framing as the first; halving k halves that regardless of the bar.
It stays a constant because it is a decision about how loud one hint may be,
not a per-install tuning question — and a knob nobody turns only adds a way to
misconfigure the surface.
Reachable from Settings, no restart (rule 25), with copy that says which way to
move it and points at retrieval_telemetry's rule pull-through — which step 3
made readable — to tell "arriving unread" from "never arrived".
Every config stand-in in the suite gained the key, not just the one that
noticed. The arm reads `rule_threshold` while BUILDING its search arguments, so
a missing key raises inside its fail-open except and turns the arm into a
silent no-op — indistinguishable from it running and finding nothing. That is
the same vacuous-pass shape that bit step 2, one layer down (rule 33).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
Milestone 333 step 3, the read half. Steps 1 and 2 built the table and filled
it; until now nothing read it, and `usage` — sourced entirely from
note_usage_events — described notes only while `sources` happily listed a
write_path_rule row above it. A reader takes the aggregate as covering
everything named above it. It did not.
A SEPARATE `rule_usage` BLOCK, not folded into `usage`. Two reasons, and the
second is the one that bites: the corpora differ by orders of magnitude, so a
blended ratio would be the note ratio with noise on it and the rule arm would
stay invisible inside it; and `usage` is what existing callers already read and
compare across windows, so silently changing what it counts would move a number
nobody was told had changed meaning. There is a test asserting rule events stay
out of the note block.
No `ambient` key, unlike the twin. Nothing surfaces a rule un-ranked —
list_always_on_rules and enter_project hand rules over wholesale but emit no
event — so there is no ambient class to subtract. The absence is a fact about
the data, not an oversight, and it returns when a bulk loader starts emitting.
Guarded separately, like `by_source`. This table did not exist a commit ago,
and an instance running upgraded code against un-migrated schema would
otherwise take down two readouts that work perfectly in order to report a third
that cannot. On failure the FLAG is added and the SHAPE is kept — a caller must
not have to choose between crashing on a missing key and quietly rendering
zeros it has no right to.
`pull_through` is None rather than 0.0 on an empty window, matching the note
block. A ratio of zero asserts "rules were shown and none opened"; with an
empty numerator and denominator that is a claim the data does not support, and
it is the reading that would make a brand-new install look like a broken one.
Also fixed, from #3311: the rule arm never timed its search, so it was the one
source in the readout reporting a null p90_duration_ms — a gap that reads as
"this surface is somehow not measurable" rather than "nobody passed the
number".
Both docstrings updated in the same change. The tool's is the agent-facing
contract (rule 119) and it explicitly said rule surfacings were absent and had
"no usage counter at all". Leaving that would have had a reader conclude the
arm has zero pull-through rather than a separate one.
Tests are integration for the reason the block above them is: real GROUP BYs
and count(distinct) against a table a commit old, in a module whose one
production outage was a SQL shape the database rejected inside a broad except.
A mock would agree with whatever the code does, including nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
The product code was right; the test was wrong, and wrong in a way that made
two assertions fail and two others pass vacuously.
`build_write_path_hint` returns early when a write matched nothing at all — no
staleness, no synced record, no prior-art menu, no shape signal. The rule arm
sits deliberately on the FAR side of that guard, because it runs a semantic
search and hoisting it would mean an embedding query on every write in the
session. My fixture stubbed every other arm to empty, so it hit the early
return and the rule arm never ran: `record_rule_surfaced` was called zero
times, and "the recorder was not called" is also what two of the four tests
were asserting for their own reasons.
The fixture now supplies one prior-art hit — 0.72 against a 0.6 threshold, so
it clears the band and the top_k slice — with a comment saying the hit is the
arm's precondition rather than scenery.
And the gate got its own test, because the fixture now depends on it: a write
matching nothing must NOT reach the arm. Without that, a future change to the
guard would make every assertion in this file pass without exercising
anything. #3311 is explicit that the gate stays until the arm's precision is
fixed, so the test says to go read that issue rather than update the
assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
Milestone 333 step 2. Step 1 built the table; a counter nobody calls reads zero
and looks exactly like a surface nobody uses, which is #2663's shape.
SURFACED — the standing-rule arm in build_write_path_hint, beside the
record_retrieval it already made. Two tables, and the split is not arbitrary:
retrieval_logs is one row per CALL keyed on the score distribution a threshold
is tuned from; rule_usage_events is one row per RULE per event, the grain "was
this hint ever acted on" needs and the grain a JSONB result_ids array cannot be
indexed at.
The comment there said rule ids had nowhere to go — that note_usage_events
remaps ids on restore, so a rule id would return attached to whatever note took
that number. Still true of the NOTE table, and precisely why step 1 built its
own. Rewritten to say the gap is closed rather than leaving a stale rationale
that would have someone re-derive the same dead end.
Records `fresh`, i.e. AFTER exclude_rule_ids. A rule the session already holds
was considered and not shown; counting it would inflate the denominator with
claims the agent never saw, and the ratio would then fall for a reason that has
nothing to do with whether hints land.
PULLED — two doors, both after their access check so a refused read is not a
pull. mcp_get_rule is the one that matters: the arm's own message ends "Read it
with get_rule(N)", so that call is the exact action a landed hint produces.
rest_rule carries the other prefix, and the prefix is load-bearing — "is this
rule dead weight?" is served by any pull, "did that injected hint land?" by
agent pulls only.
NOT a pull: rule_history. It loads the rule for its title and its own output
says "The current wording is on the rule itself — get_rule(N)", so counting it
would credit a read of the history as a read of the rule and double-count
anyone who then follows that pointer. list_always_on_rules and enter_project
are likewise bulk resident loads, not somebody choosing to open one record.
tests/test_rule_usage_wiring.py is cross-cutting on purpose: the surfaced end
is in plugin_context, the pull end in two other modules, and "both ends meet"
is a property no module-shaped file asserts. It covers the exclusion boundary,
that a failing recorder cannot break the write, that a refused read records
nothing, and two completeness guards — every door records, and the bulk loaders
still do not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
Ruff F821, twice, on the same two lines. The query was placed next to its
note-usage counterpart — which reads `note_ids`, defined much earlier — while
`_rule_ids` is not built until forty lines further down, beside the rules
themselves. Moved to sit directly after the `rule_versions` query, which is the
other consumer of that variable and the block whose scoping argument this one
restates.
Worth noting what did NOT catch this. The integration round-trip passed on the
same commit: it drives `restore_full_backup` against a hand-built payload, so
it exercises the import side and the full export, and never calls
`export_user_backup` at all. A per-user export of any account owning a rule
would have raised NameError at runtime. The lint lane found it because a
static check does not need the path to be reachable by a test.
The comment moved with it and got sharper, since the hazard is that the
plausible column is the wrong one: `user_id` on a usage row is whoever the arm
fired FOR, not who owns the rule, so scoping a per-user export by it would
carry this user's surfacings of someone else's rule and drop the ones fired for
someone else on theirs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
Milestone 333 step 1. The write-path standing-rule arm is the only retrieval
surface in Scribe whose usefulness cannot be observed — and, not
coincidentally, the only one that has never declined to fire. 296 calls, zero
zero-result, 100% clearing its threshold, while every other surface declines
most of the time (#3311, and re-measured in note #3430). `retrieval_logs`
gives it scores; scores say what the ranker thought, never whether the hint
landed.
WHY A SIBLING TABLE AND NOT A COLUMN ON note_usage_events. The row carries no
note-specific field and the readout is the same shape, which is the strongest
case for sharing that note #3163 admits. What decides against it is identity at
RESTORE: the note importer maps note_id through note_id_map, so a rule id
parked in that column comes back attached to whatever note holds that number in
the target database. Not dropped — reattached. The restore reports success, the
counters are populated, and every one is about the wrong record, with no other
field to disagree with. rule_versions made the same call for the same reason;
this is the third rule-side sibling and it reads like the first two.
FK-free on rule_id and user_id, matching note_usage_events / retrieval_logs /
app_logs, and deliberately unlike rule_versions. A version belongs to a rule's
history and dies with it; telemetry outlives what it describes. Deleting a rule
must not erase the evidence that it was surfaced forty times and opened never,
because that evidence is the case for having deleted it.
The service uses `background.spawn` rather than a third copy of the
strong-reference dance — that module's own docstring says new callers should,
and a fourth copy is how one of them drifts. The AppLog canary #2663 demands is
kept, and since `rule_usage` needed exactly `note_usage`'s semantics, that
canary moved into `background.report_telemetry_failure` and note_usage now
calls it. `retrieval_telemetry` deliberately keeps its own: its canary is a
different shape (one process-wide flag, no AppLog row), so repointing it would
change behaviour rather than consolidate it.
No ambient bucket, and that is a decision. The note twin splits ranked from
ambient surfacings because enter_project and the skill sync deliver records
without choosing them (#2477). Rules have the same problem waiting —
list_always_on_rules loads them wholesale — but nothing emits here yet, so an
empty AMBIENT_SOURCES would be machinery pretending to a distinction the data
does not contain. `source` stays granular, so the split stays a readout-level
change needing no migration.
Backup carries it (v14). The round-trip test seeds a NOTE alongside the rule so
the target database has a note id to collide with — without that decoy, a
restore running rule ids through the wrong map would merely drop them and the
test would pass by absence, rather than failing on the populated-and-wrong
result that is the actual hazard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
Rule 156, across the whole client. `apiGet`, `apiPost`, `apiPut`, `apiPatch`
and `apiDelete` each called bare `fetch`, whose default is to wait as long as
the browser will — not a long timeout but the absence of one. The only
AbortController in the frontend belonged to the SSE stream and was for
cancellation. So every request in the app could hang forever, and there is no
state a surface can render for "pending forever" that is not a lie: the
spinner that never resolves looks exactly like work still in progress.
Found while building the version readout (#3329), which had to tell "the fetch
failed" apart from "still loading" and could not.
ONE REQUEST PATH. The five verbs were near-identical bodies; they now delegate
to a single `request()` that owns the deadline, so a sixth verb cannot be added
without one. 30s by default — long enough to clear a cold embedding call and a
list view under pool contention (#2384), so tripping it means something is
wrong rather than merely busy. Overridable per call via `timeoutMs`.
EXPIRY IS AN ApiError, which is the half of rule 156 that is easy to skip. A
raw `DOMException: TimeoutError` reaches `apiErrorMessage(e, fallback)` as an
object with no `body`, so all ~330 existing catch sites would have printed
their generic fallback and the timeout would have been invisible in exactly
the situation it exists to expose. Rethrown as `ApiError` with a 408 — a status
no Scribe route returns, so it unambiguously means the client gave up — every
one of those call sites now reports it correctly, untouched.
Only TimeoutError is converted. A deliberate cancellation aborts with
AbortError and passes through: a caller that cancelled its own request does not
want that surfaced as a server failure. Pinned by a test, because collapsing
the two is the obvious "simplification".
STREAMS RELOCATE THE DEADLINE RATHER THAN ESCAPING IT. A wall-clock timeout
would kill a long-lived SSE connection mid-flight, but two different waits are
involved and only one of them is the stream: the CONNECT can fail to answer and
now carries a 15s deadline, cleared the moment headers arrive; the BODY stays
unbounded on purpose, since its failure mode is going quiet, which a timeout
cannot distinguish from being idle — that is what reconnection and
Last-Event-ID are for. Reading the connect as exempt because "the stream is
long-lived" leaves an unreachable server looking like a quiet one.
BULK TRANSFERS get their own value, not the default. Backup, notes export and
admin restore walk the whole store and 30s would cut them off mid-work; they
carry 10 minutes. Bounded, not unbounded — rule 156 asks for a deadline, not a
short one, and no ceiling at all is what leaves a restore that died
server-side spinning forever.
Four source-inspection guards in the unit lane (no frontend test runner): no
bare fetch anywhere; the default is actually applied — pinning the specific
regression, since #3329's opt-in shape would pass every other check while
leaving 330 callers unbounded; expiry converts to ApiError; and cancellation
does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
#3127 checklist 12, plus rule 27 — a capability with no surface the operator
can touch is not shipped.
The step was planned on the premise that nothing read `/api/version`. Two
things did, and the state was worse than nothing:
- `App.vue` fetched it, wrote `version` into a ref initialised to the literal
`"dev"`, and swallowed the error. An instance that could not answer rendered
EXACTLY what a healthy local build renders. That is checklist 12's named
failure — a blank standing in for `unknown` — in the one readout whose whole
job is to say what is running, and it would have made #3298's debugging
session no cheaper.
- `SettingsView.vue` fetched the same endpoint again on every mount and wrote
the result into a local ref no template ever read. A duplicate request whose
answer was discarded.
So this is not "add a readout"; it is "make the existing one honest, and give
it the three fields nobody could see."
The readout — Settings → Config, first section, beside the other "what is this
instance doing" facts. Three states kept apart, because collapsing any two of
them is the defect:
not asked yet (tab unopened) nothing
answered the values, each ABSENT field as "unknown"
the fetch itself failed its own message, with a retry
`version` and `channel` prominent, `commit` in full with a copy button so it
can be pasted into a `:sha` lookup (rule 145 — the registry's identity and the
artifact's own must be checkable against each other), `build` kept because its
ABSENCE is the diagnostic part: no ordering key means this build is not in any
update order, which is what a local or hand-built image looks like.
Absence, not falsiness. The payload omits what it does not know rather than
sending `""` or `0` (see `build_version_payload`), so the renderer uses `??`
throughout — `build` is a number and `0` is a legitimate ordering key, which
`||` would report as unknown. `tests/test_version_readout.py` pins that
operator specifically, along with the "no plausible default" property, because
`||` is the form a person reaches for by habit.
Rule 156 — the fetch carries a deadline. This readout is consulted when an
instance is misbehaving, which is exactly when it may never answer; without one
the surface sits on "still loading" forever, which is the same blank arrived at
from the other direction. `apiGet` gains an OPT-IN `timeoutMs` rather than a
default, so no existing call site's behaviour moves. Every other call in the
client still has no deadline — reported separately, not fixed here.
No frontend test runner exists, so verification is the typecheck lane plus four
source-inspection guards in the unit lane, each pinning one property.
Also folded in: `plugin/README.md` now leads with the mint script and offers
`make` second, since `make` is not installed on every workstation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
#3127 checklist 19. The step's own deletion list turned out to be largely
spent: `check_version_bump()` came out with #3327, and the machinery the step
expected to delete alongside it is load-bearing for its replacement.
`manifest_version(ref=…)`, `--base`, `--no-version` and the `origin/main`
resolve path all STAY. Derivation makes the value right; it does not make the
comparison unnecessary. `check_version_is_minted` still has to ask "did the
version move when the shipped content did?", and that is a base-branch
question no matter who chose the number. The step was planned before #3327
landed, when the assumption was that these died with the guard.
What was actually still standing, all of it teaching or asserting the retired
scheme:
- `plugin/README.md` told the reader to "set a `version` bump per release."
A shipped file, instructing the exact act the mint replaced — this is how a
deleted control gets re-added by someone following the docs. Now says not to
hand-edit the field, names `make mint-plugin`, and says what a forgotten
mint costs. (`make` is not installed on every workstation, so the direct
script invocation is given too.)
- `test_plugin_version_bumped_with_the_hook` asserted `version >= (0,1,31)`
as a tuple of ints. Under a minted value it passes vacuously — every date
clears a floor of 0.1.31 — and `int("0415")` silently eats the padding the
format exists to keep. Superseded by
`test_the_shipped_manifest_carries_a_minted_version`, which asserts the
canonical shape instead of an ordering the comparator does not perform.
Removed whole (rule 22).
- The module preamble still ended on "a written rule that depends on being
remembered is not a control; this is" — true of the bump guard, and read as
a stronger claim than the mint can support. Replaced with what the change
did and did not remove: choosing a number is gone, running the mint is not,
and the difference is that forgetting is now loud rather than silent.
- An orphaned `# --- the version bump ---` section header with nothing under
it, and a test docstring still naming `check_version_bump`.
`--no-version` keeps its one legitimate case — on `main` the version is
measured against itself — and now says so in both the usage block and its
`--help`, so it does not read as an escape hatch. `check_session_context_
reports_its_version` stays untouched: a different check with a different job,
and the only thing that makes step 6 readable from a transcript (#2220).
Version minted 2026.09.02.0415 -> 2026.09.02.0438 for the README change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN
Run 5175 red on the Python tests lane. The failing assertion was
test_the_mint_is_UTC_not_local, and it was right: `strftime` renders the
offset the datetime carries, so mint() only produced UTC because its DEFAULT
argument happens to be datetime.now(timezone.utc). Hand it an aware datetime
in any other zone and it formats that zone's wall clock -- 22:52Z and its
+09:00 twin, the same instant, minted as 2026.09.01.2252 and 2026.09.02.0752.
The docstring already claimed "UTC, always", so this was a contract the code
did not hold rather than a test asking for something new. Two people minting
the same instant would disagree, and the string IS the artifact's identity.
Now converts explicitly. A naive datetime is read as UTC rather than as the
machine's zone: that is this function's stated contract, and guessing the
host's offset is how the same bug returns by another route.
Two things found while walking the rest of the module by hand:
- test_a_failed_diff_FAILS_rather_than_passing_quietly stubbed EVERY git call
to fail, so it tripped the base-branch guard first and passed while proving
nothing about the diff arm. rev-parse now succeeds and only the diff fails,
and the assertion names the diff message instead of the substring both
messages happen to share.
- the base-branch failure still said "version-bump check", a name that went
away with check_version_bump.
The mint script is in the version-relevant set, so fixing it is itself a
version-relevant change and forced a fresh mint -- 2026.09.02.0415. That is
the asymmetry in #3127 section 3 working as intended rather than a quirk: a
format change that did not re-mint would leave the manifest reporting a value
the current deriver can no longer produce.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN4zBVFWhBST9YqjCfQmPb
Milestone 334 step 3. 0.1.48 was the last of 48 numbers a person typed by
hand; forgetting to type the 49th is #2209, #1040 and #2220, three separate
times a shipped fix reached the repo and stopped there.
WHY A SCRIPT AND NOT A BUILD STEP. plugin/ is not in the image -- installs
fetch it from this repo via marketplace.json, so the push IS the release and
there is no moment at which CI could stamp a version in. Every other artifact
in the family derives during a build (#3127 section 2). This one has no build
to derive during, so the value is minted before the commit and CI's job is to
prove it moved when it had to.
MINT TIME, a fourth clock section 2 does not name. It prescribes commit time
so two lanes building one source report one string; the plugin has one lane
and no build, so that reason does not reach it. What is given up is
reproducibility-from-history -- you cannot recompute the value, only verify it
moved. That is acceptable ONLY because #3325 read the installer's code and
found the refresh test is `P.version === H`, plain equality, with zero
ordering comparisons anywhere. Where a comparator orders, an unreproducible
version would be unverifiable too.
Two artifacts in one repo now derive from different clocks on purpose, one
directory apart. "Let's make these consistent" is the obvious tidy-up and
breaks whichever loses, so the divergence is pinned in tests rather than only
explained in a comment -- including an AST assertion that the mint script
never imports subprocess, since a mint that can read history is a commit-time
deriver wearing the wrong name.
check_version_bump becomes check_version_is_minted. It gains the shape gate
and a future-value gate, and it keeps deliberately NOT failing when the
version moved without content changing: a needless re-mint costs one cache
refresh, and failing the lane over a harmless act is how a check earns a
--no-version in somebody's muscle memory and stops running at all. The
implication that matters is one-directional.
The mint script joins the version-relevant set, which is step 2's DERIVERS
table finally being read by something. Section 3's asymmetry is why it is not
optional: change the format string, change nothing else, and a diff over the
shipped paths alone says "no content change" while the manifest keeps a value
in the old format forever. Its own introduction demonstrates this -- adding
the deriver is itself the version-relevant change that forced this mint.
fetch-depth: 0 was NOT added, against this step's own brief. The plugin job
carries a comment refusing it, backed by an observed act_runner failure (any
`with:` block made checkout fail to extract, run 3027), and the reasoning
holds: the check diffs two trees and the workflow already fetches main at
depth 1. Checklist 6 is about jobs that derive; this one checks.
Verified live before pushing: the session-context marker reports
v2026.09.01.2252 keylessly, and both failure arms were probed by hand rather
than assumed. The shape gate fires first on a reverted 0.1.48, so the stale
arm is covered by unit test rather than by that probe.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN4zBVFWhBST9YqjCfQmPb
Milestone 334 step 2. The set of files that reach a plugin install lived in
two hand-kept copies -- SHIPPED in check_plugin.py and the workflow's paths:
filter -- with a comment asking a human to keep them in step. That is the
shape #3127 section 3 warns about, and both copies had drifted.
The load-bearing change is the exclusion. The version check reads "did
shipped content change against the base?", and plugin.json lives INSIDE
plugin/ -- so bumping the version is itself a change to the set, which then
reads as the change that justifies the bump. Every bump passed, no bump could
ever fail, and the check proved nothing while looking green.
manifest_differs_beyond_version compares parsed objects with `version`
dropped from both sides. One field, never the whole file: plugin.json also
carries description, mcpServers and userConfig, all of which reach an install,
and excluding the file wholesale would let a userConfig-only edit compute an
unchanged version and never refresh -- #2209 again with a narrower trigger.
Unreadable input answers "changed", because a spurious bump costs one cache
refresh while a missed one is the fix reaching the repo and stopping there.
shipped_content_changed returns None, not False, when the diff fails. #2663 is
why: a read that failed inside a broad except reported the same zero as an
empty window, and every counter read zero for weeks.
Two dead trigger paths removed, both found by writing the guard rather than by
review. fable-mcp/** outlived its directory by three months (deleted in
91bafb6, 2026-05-27) and assets/** named a path that never existed at all. A
paths: entry matching nothing never fires, so neither ever failed anything.
Their two orphaned bump scripts go with them -- a third manual-bump mechanism,
wired into no settings file.
DERIVERS is section 3's (deriver -> artifacts whose identity it decides) table.
The membership test is "can changing this file change what the artifact says
about itself?", not "is it copied in" -- a deriver is never in the COPY list.
A checker is not a deriver, which is why check_plugin.py is absent from it;
step 3's mint script adds its own row.
The workflow's paths: filter is YAML and cannot import Python, so "one
definition" is held by drift tests rather than an import. Said plainly in the
test module, because it is the honest shape rather than the ideal one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN4zBVFWhBST9YqjCfQmPb
The readout already grouped usage by source — `group_by(event, source)` —
and the loop directly below it threw the source away, collapsing every
surface into one corpus-wide ratio. So the question a threshold is
actually tuned against, "is THIS surface worth its noise", could not be
asked of any surface, while the data to answer it sat in the table.
`usage.by_source` reports notes_surfaced / notes_pulled / pull_through
per surface. The grain is the note, not the call: a pull records the
door it came through, not the surface that led there, so grouping the
pulled rows by source would answer a different question. Joining
surfaced rows to pulled rows on note_id answers this one without the
session identity #2085 declined to invent — at the cost of being an
upper bound per surface, which the docstring says where it is read.
Ambient surfaces report counts and a null ratio: nothing chose those
records, so "surfaced often, opened never" is not a judgment about them.
A surface that genuinely produced nothing reports 0.0, which must not
look like the null.
The join is guarded separately from the two reads above it. #2663 was a
novel SQL shape the database rejected inside a broad except; this is the
novel shape here, and it must not take down two readouts that work.
Tests are integration for that same reason — a mock passes on a query
Postgres refuses. They pin the distinct-first property (three surfacings
of one note are one note), the ambient null, and the LIKE escape, since
an unescaped `mcp_%` also matches `mcpXget_note` and nothing else in the
payload would show the difference.
The runner-facts step answered rule 81's check on its first run, and the
answer is the one the check itself warned about: `/bin/sh` resolves to
`/usr/bin/dash`, because ci-python is Debian-based. The constraint the
rule exists for is unchanged — dash has no /dev/tcp, no arrays, no
`[[ ]]` — but the shell has never been busybox, and this comment was
repeating the wrong name at the one place a reader would trust it.
Rule 81's own statement still says busybox; correcting it is a rulebook
edit and goes through propose -> approve -> apply.
Three conditional rules state facts about this act_runner — services are
not reachable by hostname (79), the service container's name is derived
from the job's truncated display name (80), and `run:` steps execute
under a shell without bash features (81). None had ever been verified,
because each check reads "add a step to a live CI job and read the log"
and nobody wants to arrange a throwaway run to do it.
So the step is not throwaway. Two lines on every integration run turn the
next sweep of these rules into a log read. Rule 80 needs nothing new: the
container listing the suite step already prints for the name filter is
its evidence, and run 5055's log already answers it.
Every command is guarded with a fallback. This observes the lane; it must
not be able to break it.
SQLAlchemy spells Postgres's four row locks as a read/key_share pair, so
`with_for_update(key_share=True)` renders FOR NO KEY UPDATE — an
exclusive lock that two refreshes of the same record would fight over,
and that an ordinary concurrent edit would block. The claim needs
`read=True` as well to be the FOR KEY SHARE the docstring describes.
Caught by the unit test that compiles the statement, which is the whole
reason it asserts on the rendered lock mode rather than on behaviour
that looks identical either way.
An embedding refresh replaces a record's vectors as delete-then-insert,
which takes the chunk rows first and the parent row second (via the
insert's foreign key). A cascading delete of the parent takes exactly
those two locks in the other order. Postgres calls the cycle a deadlock
and kills one side: sometimes the detached embedder, silently, and
sometimes the user's delete, as a 500 on an operation that should have
worked.
Both upserts now claim the parent row with FOR KEY SHARE NOWAIT before
touching any chunk row. That removes the cycle instead of narrowing it —
either the embedder is first and the delete queues behind it, or the
delete already holds the row and the embedder loses at once, which is
the side designed to lose. FOR KEY SHARE is the lock the insert would
take anyway, so an ordinary edit is unaffected.
The note twin, recorded as unverified on the issue, has the same shape
and the same fix; a trash purge is the hard delete that reaches it.
Unit tests pin the ORDER and the lock mode by compiling the statement;
the integration pair holds a real delete open in one transaction and
proves the embedder returns having written nothing, with a deadline so
a regression fails instead of hanging.
CI set BUILD_VERSION to the CHANNEL — literally "dev", "main", or the tag —
so a running instance answered "which build are you?" with the name of a
branch: {"version":"main"}. The cost was concrete rather than theoretical.
During #3244's live acceptance a deploy was behaving as though it held older
code, and the one endpoint whose job is to settle that could not.
Rule 149's three values, now three fields:
version the NAME, YYYY.MM.DD.HHMM from COMMIT time — "is this the same
code?", so two lanes carrying one commit report one string
build the ORDERING KEY, minutes since 2020-01-01 from BUILD time —
"may this be installed over that?", and the only value anything
may compare
channel its own field. Never a suffix, never a segment of the name
Plus `commit`, so the artifact's claim about itself can be checked against
the :<sha> it was published under (rule 145) — which is exactly the question
that could not be answered tonight.
THE TWO CLOCKS ARE DELIBERATE and look like an inconsistency. The name comes
from the commit so two lanes building one source agree; the key comes from
the build so it cannot go backwards when an older commit is rebuilt. A test
pins both derivations against being "tidied" into one.
ABSENT RATHER THAN EMPTY when unknown. A local build has no ordering key and
no channel; emitting "" or a placeholder would let it claim a position in an
update order it is not part of. A malformed key is dropped rather than passed
through — a reader that cannot order is correct, one that orders on garbage
is not. The key is an int, because a string ordering key is how a comparison
silently becomes lexicographic ("9" > "10").
The payload builder is extracted from the route so it can be tested as a
dict rather than through app startup and a request context.
Tests pin the SHAPE the lanes emit, not the values, including the midnight
leading-zero case rule 149 names specifically — and assert CI never stamps a
branch name as the version again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI caught the naming work reaching for Postgres from the unit lane, and the
connection error was the symptom of a real design fault rather than a test
gap: reading the title BEFORE the delete put a live query on the delete path,
so a lookup that failed would have stopped the delete happening.
That is a decoration breaking its payload — the same mistake just fixed in
rules_etag, made again two commits later. Every title lookup now fails open:
delete_task, delete_note, delete_milestone, delete_snippet and rule_history
lose the name, never the operation.
The five unit tests mock the lookup rather than reaching for a database, and
delete_note gains one asserting the delete still happens when the lookup
raises — the behaviour, not just the absence of a crash.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI caught two things, and the first is the feature not working rather than a
test being wrong.
THE SIGNAL WAS GATED ON A COINCIDENCE. build_write_path_hint returns early
when no prior art, stamp, divergence or derive matched, and that guard sat
ABOVE the new arm — so a session whose rules had changed was told only if the
file it happened to be editing also matched something else. A staleness
signal that fires on that coincidence is not a staleness signal. The arm now
runs above the guard, collecting into its own list that `lines` is seeded
from, and the guard accounts for it.
The standing-rule arm (milestone 307) is deliberately LEFT below that guard,
and this is a finding rather than a fix: it has the same gating and probably
should not, but it runs a SEMANTIC search, so lifting it would put an
embedding query on every write in every session. That is a cost decision, not
a bug fix, and not this task's to make.
THE MARKER MUST NOT BREAK THE PAYLOAD IT DECORATES. rules_etag is computed on
the SessionStart path, where `max()` raising costs the whole context payload
— every rule title, the project, all of it — to save a hint. A row with no
usable timestamp is now skipped and a set with none degrades to a count-only
marker, which still catches a rule added or deleted and only loses edits.
That is the right way round to lose information. CI found it because
build_session_context's tests pass MagicMock rules and `max()` over those
raises TypeError.
Also: list_always_on_rules on an install with no always-on rulebooks returns
`rules_etag: "empty|0"`. Its exact-dict test is updated rather than loosened
— the key being present on an empty install is the behaviour, not noise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rules payload carries a marker; the write-path hook hands it back; the
server says which rules moved. Nothing is said when nothing moved.
THE COUNT IS NOT DECORATION. max(updated_at) alone cannot see a DELETED rule
— it moves no timestamp — and that is the single change that takes an
instruction OUT of force, which is the one a session most needs to hear
about. The marker is `<max updated_at>|<count>`, and a deletion is reported
through the count because there is no row left to name.
THE HOOK IS THE CARRIER because it already fires before a write, which is the
moment acting on a stale rule costs something. One comparison, no payload,
no extra round trip.
WHERE THE MARKER IS CAPTURED, and it could not be anywhere else: the
SessionStart hook, from /api/plugin/context. The model also receives one from
list_always_on_rules, but a hook cannot see an MCP tool's result — so the
value the write path compares has to be stored where a shell script can
reach it. Keyed by session id in the state dir the prior-art hook already
uses, so "changed since" means since THIS session loaded its rules.
NOT ON rules_payload, against the task's letter. Those are applicable_rules —
a different, subscription-derived set. One key name over two sets is how a
comparison starts reporting phantom changes, and the write path compares
against the always-on set.
WHAT IT CANNOT SEE is stated in both the service and the write-path arm as a
table, because a reader who finds an etag will assume it covers staleness
generally:
another session edits a rule mid-flight | caught
the session is misremembering a rule read hours ago | caught
compaction summarised the rules out of context | NOT caught
The third is the most common, and the marker is blind to it — the etag was in
context too and went with the rules. The SessionStart nudge is that case's
only mechanism and must not be softened because this shipped. A test asserts
both modules still explain that.
Instance-agnostic (rule 115): an install with no rules produces a stable
marker rather than an error, and "no rules" reads as a state rather than as a
change. An unreadable or absent marker reports nothing — a signal that cries
wolf is worse than none, because it trains a reader to skip the line that
will one day be true. The arm fails open like every other arm on this hook.
The delivery is tested through the real build_write_path_hint rather than the
helper alone: the feature IS a line arriving in a session, and the arithmetic
being right proves nothing about that.
Live acceptance is deploy-gated and not yet recorded on the task.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Rule 27: a history nobody can read is not shipped. `RuleHistoryPanel.vue`
sits below the fields in `RuleEditorSlideOver`, where a rule is read in full
— not on the list row, where a history entry point would compete with the
row's job.
REUSE, DECIDED FIELD BY FIELD RATHER THAN ALL AT ONCE.
DiffView.vue is reused unchanged: it takes DiffLine[] and nothing
note-shaped.
HistoryPanel.vue is NOT, and its props are the reason — noteId +
currentBody, a NoteVersion carrying tags and pin columns, a fetch of
/api/notes/…, a restore emit, pin/unpin buttons. Rules have no tags, no
pins, and deliberately no restore, and a rule's text is EIGHT fields rather
than one body, which changes the reader's question from "what changed" to
"which fields moved". Recorded here rather than forked silently, per #3207.
THE FORK THAT WAS ALREADY THERE. The LCS walk existed three times —
privately in useAssist.ts, and again inside HistoryPanel.vue and
VersionHistorySection.vue — character-identical apart from quote style,
because computeDiff was never exported. Rather than add a fourth copy, it
moves to utils/diff.ts and the three become imports; the extraction was
verified equivalent to all three before anything was deleted. DiffLine is
re-exported from useAssist so its existing importers are untouched.
WHAT A ROW SHOWS: when, and which fields moved. A version holds the text the
edit REPLACED, so the edit is the step from a row to the next NEWER state —
the row above it, or, for the newest row, the rule as it stands now.
Comparing against the row below would attribute every change to the wrong
edit. A field nobody has fetched yet reads as neither changed nor unchanged.
An edit that touched verify_with is badged "check reset", because that edit
silently cleared verified_at (milestone 312) and put the rule back at the
top of the staleness sweep — a moment visible nowhere else.
The badge is a 12% color-mix TINT, not solid `--fs-warning`.
`--fs-warning-fg` is defined in theme.css as "warning TEXT on a warning
tint", so painting it over the solid token is exactly the same-hue contrast
failure #3141 records. Every var() the component references resolves against
theme.css, checked before pushing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`list_rule_versions` / `get_rule_version` in the rulebooks service, a pair of
REST routes beside the note-version ones, and an MCP `rule_history` tool.
The ACL-scoped reads live in services/rulebooks.py rather than in
services/rule_versions.py because rulebooks already imports rule_versions for
the write path and the reverse would be a cycle. It is also the honest split:
rule_versions owns what a version IS, rulebooks owns who may read one.
Scoping is through the RULE, never the version's user_id, and both directions
of that mistake are now pinned by tests. That column is the ACTOR — scoping
by it would hand someone the snapshots they personally wrote on a rule that
has since moved out of their reach, and would hide from the rule's owner
every edit anyone else made. `get_rule_version` takes the rule id as well as
the version id so the ownership check and the fetch agree about which rule is
in play; the test for that uses a second rule the caller genuinely owns,
because a nonexistent id would pass on the ownership check alone and prove
nothing.
An unreadable rule returns None, not an empty list. The two mean different
things — "not your rule" versus "never reworded" — and the MCP tool keeps
them apart: None raises, empty says so in band.
THE DIFF QUESTION, ANSWERED — and the task's premise was half wrong. It says
"notes have DiffView.vue and a diff endpoint already". The component exists
and is reusable as-is: it takes `DiffLine[]` and nothing note-shaped, so step
4 can render a rule diff with it unchanged. The ENDPOINT does not exist —
diffs are computed client-side by `computeDiff` in useAssist.ts. So no diff
route is needed here, and none was written.
For the MCP door the answer is different again: an agent has no client to
compute a diff, but it also does not need one. Each entry holds the text the
edit REPLACED, so "what did this say before the most recent change?" is the
first entry, and the text that change produced is the rule as it stands. The
docstring says so, and a test pins that sentence — read the other way round,
every diff comes out backwards.
No restore, per the task. Putting an old wording back goes through
update_rule, which snapshots what it replaces, so the undo stays visible like
any other edit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The integration lane failed on a deadlock, not an assertion — 103 passed, and
the one error was in teardown: `DELETE FROM rulebooks` blocked against
another process holding a lock on a rule row.
`update_rule` fires a detached asyncio.create_task(upsert_rule_embedding(...))
that opens its OWN connection and UPDATEs the rule it just saved. The
teardown's rulebook delete cascade-locks that same row, and Postgres resolves
the cycle by killing one of them. The sibling test_integration_rule_verification
never hit this because it calls update_rule but never deletes its rulebook.
Cleanup moves to setup, which runs on a fresh loop after the previous test's
loop has closed and cancelled whatever it left in flight. That also keeps the
#3240 constraint intact: no database call after a yield.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`update_rule` now snapshots the rule's text before it writes. Rescoping rule
79 meant hand-copying the superseded statement into a task log to keep it
(#3237); the history is that, done by the write path instead of by somebody
remembering.
The snapshot is taken BEFORE the field loop, which is the one ordering that
matters. `update_rule` drops `verified_at` when `verify_with` changes, and
rewriting a check is exactly the edit whose history is worth most — a
snapshot taken afterwards would file the NEW check against the OLD wording.
Taking it up front also covers `clear`, which is a separate argument from the
field loop and is how a rule that stops being a constraint loses its check
entirely.
Session-bound rather than opening its own like note_versions.create_version:
the version and the edit that caused it commit together, so a failed update
cannot leave a history entry for an edit that never happened.
Two guards from the sibling are deliberately absent, and both are now pinned
by tests rather than only by comments — "make it consistent with
note_versions" is a plausible-sounding change that would silently start
dropping history:
- No MIN_VERSION_INTERVAL_SECONDS. That 300-second gate exists because note
autosave fires every 60. Every version here comes from a deliberate
update_rule, so three edits in one second are three edits.
- No MAX_VERSIONS and no pruning. A rule is edited a handful of times in its
life; a cap could only ever discard the one edit somebody went looking for.
Kept from the sibling: the identical-content skip. Both doors resend every
field, so without it a form saved twice would file an identical snapshot.
`order_index` is excluded from the snapshot fields for the same reason —
reordering a rulebook is not an edit to what any rule says.
No delete-time snapshot, against the task's original scope and on the
operator's call. A delete goes through trash_svc and is SOFT: the rule row
keeps its full text and restores untouched, so there is nothing for a
snapshot to preserve. Anything that survived a purge would be data the
operator explicitly asked to be gone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`_dispose_engine` is a usefixtures entry, so it sets up AFTER an autouse
fixture and tears down BEFORE it. The purge running after this file's
`yield` therefore opened a fresh pooled connection that the closing loop
immediately orphaned, and the next test to touch Postgres died on "Future
attached to a different loop" — two of this file's own tests and
test_run_maintenance_vacuums_real_tables, which shares nothing with it but
the engine.
The autouse fixture is setup-only now, matching its sibling in
test_integration_backup_note_roundtrip.py, and the cleanup moved into
`restored`, whose teardown runs while the engine is still live.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The sibling notes already had. `note_versions` snapshots a note's every
meaningful edit; a RULE, which binds behaviour on every session that loads
it, had nothing — an edit destroyed the previous wording with no record
anywhere. Rescoping rule 79 meant hand-copying the superseded statement into
a task log to keep it (#3237). The more consequential record had the weaker
protection.
Schema and transport only. Nothing writes a version yet — that is step 2.
Three guards are deliberately NOT copied from note_versions, each defending
against autosave, which rules do not have: no pruning or MAX_VERSIONS, no
pin columns, no minimum interval. A rule is edited a handful of times in its
life, and capping invites losing the one edit somebody needed.
`user_id` is the ACTOR rather than the owner, and SET NULL rather than
CASCADE: deleting a user must not erase the history of the rules they
edited. The restore diverges from its NoteVersion sibling accordingly — an
unmappable user leaves the row with a null actor instead of dropping it,
which is the whole point of choosing SET NULL. The integration round trip
pins that, because nothing in the code says which of the two shapes is
intended and "make it match the sibling" would silently delete the record.
Backup goes to v13. Both export paths carry the table; the per-user one
scopes through the rule rather than the version's user_id, or it would carry
the versions this user wrote on someone else's rule and drop the ones
someone else wrote on theirs. The restore remaps rule_id through
rule_id_map — #3182's arose_from_id trap on a new table.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Step 6 measured the claim the whole milestone rests on — that the notes
corpus divides into norms and constraints, with constraints a minority worth
sweeping — against a stratified sample of the real thing. It holds: ~29% in
`reference`, ~8% general, ~0-5% in `decision`, 0% in `dev-log`; roughly 6-11%
of ~395 plain notes. Written up as note 3210. The step was allowed to return
"revert" and does not.
Two things the measurement found that the guidance did not say, now added to
both the skill and the create_note docstring:
A SHARPER TEST. Every note that earned a check was about somebody ELSE's
software — a signing service, a forge, a hub, an SDK, a model, a dependency
set. Not one was about the operator's own code. "Is the thing this note
describes yours to change?" is decidable from the title in nearly every case,
where the abstract form needs thought.
THREE FALSE CANDIDATES, one of them a live hazard. Resume pointers and
"current state" notes go stale faster than anything else in the corpus, which
is exactly why they tempt — but the cure is to update or delete them, not to
schedule a check, and a sweep full of pointers is a sweep nobody reads.
Measurements of our own system go false because we changed something and knew.
And a decision RESTING on someone else's behaviour is still a decision — the
check belongs on the note asserting the fact.
Also recorded, not fixed: the corpus already contains a note titled
"CONSTRAINT: software only — no DIY hardware", using the word for a
self-imposed scope limit — a NORM in this taxonomy, exactly backwards. Both
surfaces already lead with the question rather than the label, which is the
mitigation; note 3210 names the collision so it is not rediscovered.
The step that decides whether steps 1-4 were worth building. `verify_with` is
a free-text field on the highest-volume record kind in the product; described
only as "how to verify this note" it gets filled in on every note within a
week, and at that point the sweep returns the whole corpus and means nothing.
The signal is not "has a check" — it is "has a check AND almost nothing else
does".
Rule 119 puts this in the app's own instruction surfaces, never in a Scribe
rule. So:
- the using-scribe skill gains the reflex, next to "state updates in place;
chronicles don't" — its other half, since supersession only fires once
somebody has read a note and disagreed, which is the case where it was
already believed;
- update_note's docstring now states the DEFAULT rather than only deferring
to create_note for the test. Found by the new guard on its first run;
- plugin 0.1.48.
THE _INSTRUCTIONS BUDGET, decided rather than skipped. The payload is ~1980 of
the client's ~2048-char cap, so everything in it competes for the last ~68
characters. The operator declined a line for this milestone: the map's own
closing sentence says each tool's description carries the full contract, and
the sweep is a curation act rather than a session-start reflex like
enter_project or list_always_on_rules. That reasoning is now a comment beside
the constant, with the accepted cost named — an agent that never opens
create_note's docstring never learns the field exists — so the question is not
re-litigated blind.
The guard pins STRUCTURE, never wording, for the disambiguator's reason (a
test that punishes rewriting is a test that gets deleted): each write surface
must still draw the norm-vs-constraint distinction, say the empty case is
normal, and name where NOT to reach for it — plus that the skill carries the
one-question form, because the docstrings only reach a caller who already
opened the tool.
`_doc` moves to tests/helpers as `tool_doc`; it had been written twice.
Rule 27: no UI, no ship. Three surfaces.
THE EDITOR ASKS, but only where the answer can be saved: the fields appear
for a plain note and not for a task or a snippet, matching the service gate
from step 2 so the form never offers a write the save would reject. The
labels are phrased as the QUESTION rather than the field name — "how would
someone check this is still true?" and, underneath, "could this become false
without anyone editing it?". "Verify with" gets filled in on every note; the
question gets filled in on the few that can go stale. `expires_when` appears
only once a check exists, and asks for a state rather than a date in the
placeholder itself.
THE NOTE SHOWS ITS AGE beside the field — "checked 2026-08-28" or "never
checked", italic, and nothing at all when no check exists. No red/amber ramp,
matching RuleSweepPane: a colour scale would restate the sweep's ordering and
force an invented staleness threshold. "Never" is marked because it is
categorically different from a date, not a worse one.
THE SWEEP is a pane in the Knowledge view, not beside the rules sweep —
operator's call, taken over a unified "everything due" surface and over a
second pane under /rules. Notes stay where notes live. The cost, accepted
knowingly: no single screen shows every unconfirmed record. It REPLACES the
feed rather than filtering it, because a facet answers "show me this kind"
and this answers "show me what nobody has confirmed" — a question the type
chips cannot narrow without under-reporting.
Two REST routes for it, since step 3 built only the service and the MCP door.
Along the way: NoteEditorView spelled its write payload out at three call
sites (save, create, auto-save), so every new field had to be added three
times — which is how one of them ends up not carrying it. Now one `payload()`
and one `snapshot()`.
Known and filed, not fixed: NoteSweepPane copies ~12 scoped CSS rules from
RuleSweepPane (#3207). The clean extraction needs prefixed names, because
`.age`, `.row-title`, `.lede` and `.actions` all exist scoped in other
components and an unscoped global would leak into them — which means editing
the shipped rules sweep, blind, inside a step whose acceptance is the
operator looking at a different surface.
select(Note) names every column, so searching the whole statement for
"notes.note_type" always finds the projection, and sql.index("notes.id") finds
the first column rather than the ORDER BY tiebreak. Both tests were asking the
wrong string.
The ordering test now asserts on the END of the statement, and the filter test
reads the WHERE clause — extracted by regex rather than split on a literal,
because the exact whitespace SQLAlchemy puts around WHERE is not something a
test should depend on.
The product is unchanged: the two assertions that mattered — NULLS FIRST
present, and no legal-carrier filter in the predicate — were both already
true.
The read half. `notes_due_for_verification` + `mark_note_verified` + the MCP
pair, ordered `verified_at ASC NULLS FIRST`: never-checked outranks
checked-long-ago, because a note nobody has ever confirmed is a claim with no
evidence behind it at all. Postgres sorts NULLs LAST on ASC by default, so
getting this wrong would not error — it would silently invert the one signal
the sweep exists to carry, which is why it has a test of its own.
A SIBLING of rules_due_for_verification, not a shared implementation, and this
milestone is a deliberate self-application of note 3163: the row could have
been shared, the QUERY could not. That sweep scopes by rulebook ownership XOR
project ownership because rules have no sharing ACL at all; a note scopes by
the note ACL — browse, not read, so a record shared one-to-one never arrives
in a passive surface unasked (decision 2094).
What genuinely IS common moved to services/verification.py: how a stamp reads,
how old it is, and the three states `last_verified` distinguishes — None ("a
decision, the question does not apply"), "never" ("a fact nobody has
confirmed"), a date. Rulebooks now imports it rather than defining it, so this
is a consolidation and not a third copy.
A failed check writes NOTHING, carried over from 312: there is no "verified
false" state, because a note whose check failed is not in a special condition
— it is WRONG, and recording the failure as a flag would let it sit there
being false with the sweep quietly satisfied that somebody had looked.
Two decisions worth naming:
The sweep does NOT filter to non-task, non-snippet records even though the
write path permits a check on nothing else. Such a row would be in an ILLEGAL
state and this is the one surface that could say so; hiding it to match the
invariant would make the sweep agree with a database it had stopped
describing.
A negative `older_than_days` raises instead of meaning "everything" — silently
answering a different question is the failure shape this guards.
`notes_due_for_verification` is classified read-only in server.py, spelled out
because its name matches none of the prefixes the completeness test derives
from. `rules_due_for_verification` is in the same position and is NOT listed,
so it fails closed for read keys today — filed as #3191 rather than fixed
here, since widening an auth boundary on a tool I did not write is the
operator's call.
Four tests predating the helper built their note with MagicMock(), which is
truthy on every attribute nobody set. update_note now reads verify_with, so
the stand-in claimed to carry a check and the milestone-317 guard refused the
write.
That is note 2109 exactly, and the reason fake_note exists: a stand-in has to
be able to say NO. The product behaviour is right — a real column is None or a
string, so this cannot happen outside a test.
Observation, not changed here: fake_note sets is_task=False but leaves
`status` unset, so it too is a truthy mock on the column is_task is derived
FROM. Nothing depends on it today; worth making self-consistent when something
does.
The rules path's three lessons (#3096), inherited:
EMPTY MEANS NULL. The sweep's whole signal is `verify_with IS NULL` = "this is
a decision, there is nothing to go and check". A "" that is not NULL makes a
norm look like a constraint nobody has verified — and never-checked sorts
FIRST, so it would sit at the top of the sweep forever.
CLEARING IS EXPLICIT. At the MCP door "" means "leave this alone", so an agent
updating a body does not wipe a check it was never asked about — which leaves
no value meaning "remove it". `clear` names the field, and naming it cannot
happen by accident. The REST door, where a cleared form input arrives as "",
reaches the same place through normalisation: two idioms, one outcome.
THE STAMP CERTIFIES A CHECK, NOT A RECORD. Rewrite or clear `verify_with` and
`verified_at` is dropped, so the note re-enters the sweep. A note wrongly
listed as due costs one look; a note wrongly vouched for costs exactly what
the sweep exists to catch. `verified_at` is also no longer settable through an
ordinary edit — a stamp says somebody performed THIS check, and minting one
from a write that ran no check is the one thing that would make the sweep lie.
And one this path adds: not every record may carry a check. A task's decay is
its status — a done issue records what happened rather than asserting
something that can go false — and a snippet already has verify_snippet, which
compares its recorded location and code against the repo and expires its own
verdict. Both are refused with a message naming the alternative, never dropped
silently (minted_kind's reasoning, #3129), and the gate lives at the service
so the two doors cannot come to disagree.
Written as an INVARIANT over the resulting record, not a filter on which
fields were passed. That is what catches the sideways route — a checked note
being turned into a task, a write that names no check at all and would sail
past any per-field gate.
The MCP docstrings carry the norm-vs-constraint test, because at that door the
docstring IS the contract and a field described only as "how to verify this"
gets filled in on every note. Step 5 does this properly across the instruction
surfaces; this is the minimum that stops the field being misused on arrival.
tests/helpers gains `drive_update_note` — the patch stack for driving
update_note, written twice before this and now once. The note-shaped fakes
gain the trio explicitly, for fake_note's own stated reason: an unset
attribute is a truthy MagicMock, and a truthy verify_with reads as a check
that is there.
_BACKED_UP carries REAL table names — project_rulebook_subscriptions,
project_rule_suppressions, project_topic_suppressions,
project_rulebook_exclusions — not the shorter keys the payload uses for the
same sections. The registry-coverage assertion used the payload spelling and
reported four tables unguarded.
The integration round trip passed on this run, which is the half that matters:
the real restore_full_backup remaps arose_from_id onto the restored origin.
`_note_rows` carried 16 of the `notes` table's 27 columns. A backup -> restore
cycle reported success and handed back a corpus with every snippet and process
flattened into a plain note, every issue and spike into `work`, every
provenance edge gone, and recurring tasks no longer recurring. The record-type
and kind vocabulary is what #3128 and milestone 312 were about, and a restore
erased it.
Two more found by auditing every row helper rather than only the one being
edited: `_milestone_rows` dropped `body` — a milestone IS the plan (0066), so
every plan restored as a title with no reasoning behind it — and
`_repo_binding_rows` dropped `ref`, the branch a ledger follows (#2873), so a
restored binding silently accounts for a different tree.
`arose_from_id` is deferred to a second pass beside `parent_id`, never written
in the constructor: it is an id in the SOURCE database, so copying it through
lands the edge on whatever record happens to hold that number here. An edge
whose target did not survive stays NULL rather than being guessed at. This is
the trap that kept the fix out of milestone 317 step 1.
THE STRUCTURAL HALF. The coverage guard from #2293 checks TABLES against
Base.metadata; nothing checked COLUMNS, which is how nine went missing from a
table that had been "covered" for years — added to the model and the migration,
both of which fail loudly, and never to the serialiser, which fails silently.
`_COLUMN_EXCLUSIONS` now declares, per table, every column deliberately not
exported and why, and a parameterised guard walks all 23 helpers and asserts
the two agree. Forgetting is no longer expressible.
Reconciling all 23 turned up one more deliberate exclusion worth naming: the
`code_shapes` proposal columns are the machine's standing suggestion, cleared
by judgment and recomputed by every refresh, so carrying them would restore
stale guesses over a tree the proposer has not seen.
Tests: the round trip drives the REAL restore_full_backup against Postgres,
not a reimplementation of its loop — a test that re-derives the remap it is
checking would agree with whatever the product does, including nothing.
Backup v12.
The pin test caught its own inaccuracy on the first run, which is what it is
for. `deleted_batch_id` comes from SoftDeleteMixin alongside `deleted_at` and
is excluded for the same reason: trashed rows are not exported, so neither is
the batch id that groups them for restore(). The nine-field gap #3182 tracks
is unchanged.
The sibling of migration 0090, one table over. Same distinction: a NORM is a
decision with no truth value; a CONSTRAINT asserts a fact about someone
else's software and goes false with nobody watching. Notes hold far more
constraints than rules do and hold them longer — a cross-project reference
asserting what a signing service does on a duplicate upload is believed by
every project that reads it, and nothing in the record says when anyone last
looked. note_supersessions only fires once a human has already believed it.
Three nullable columns, no backfill, no index. The index margin is thinner
than 0090's — thousands of note rows against hundreds of rules — so the
comment says to decide it in step 3 against a real query plan rather than
guessing here.
The columns land on every row in `notes`, but only non-task, non-snippet
records will be OFFERED them (gated at the service in step 2): a task's decay
is its status, and a snippet already carries a richer location-aware verdict
in data.verification. A schema-level gate would have meant a CHECK across
three columns to say what the write path says in two lines.
Backup carries the trio (v11), with `verified_at` restored through
_dt_or_none — _dt substitutes now(), which would restore every never-checked
note as checked at the moment of the restore, inverting the one signal the
sweep reads.
Found while doing that, NOT fixed here, and now pinned by a test: `_note_rows`
carries 16 of the `notes` table's 26 columns. note_type, task_kind,
arose_from_id, the recurrence pair, the lifecycle stamps, description and data
have all been missing for a long time, so a restore flattens every snippet and
process into a plain note and every issue and spike into `work`. The coverage
guard cannot see it — it checks TABLES, not columns, which is #2293's failure
mode one level down. #3182 tracks it; arose_from_id needs the second
id-remapping pass parent_id gets, which is why it is not a drive-by fix.
Spike #3128's fourth question was whether a "when does a record type earn its
own table" rule was worth writing down. It turned out to already exist, in
full, in the RuleEmbedding docstring — the only written statement of a rule
Scribe applies to every record type, sitting where nobody would look for it.
Promoted to note 3163, with the three grounds (scoping different in kind,
machine-written at volume, edge-or-event-not-document), the worked cases
across the whole schema, and the bill: what `rules` had to re-import after
leaving `notes`, including the two cells it left empty on purpose.
The docstring stays put — it is where the decision was made — and now points
at the note.
Spike #3128 found the storage sound and the retrieval vocabulary frozen
before `issue` shipped (0065). Five things, in the order they had to land.
**The mirror (rec 5, the data-integrity one).** `notes.data` is DERIVED from
a snippet's body, but only `update_snippet` knew that. `update_note` is a
hasattr loop with no snippet awareness, and both doors reach it — so PATCH
/api/notes/<snippet_id> {body} rewrote the body and left the mirror behind.
`snippet_fields` PREFERS the mirror, so the row went on reporting its old
repo/path/symbol to the location reverse lookup and to prior-art recall while
displaying its new body: surfaced with full authority, and wrong.
`snippets.recompose_data` rebuilds it from the body, carrying `verification`
and `provenance` (neither is in the body to parse). An explicit `data` still
wins, so every snippet-service write is untouched.
**One facet table (rec 3), before adding any facet.** The type predicate was
written three times — SQL, Python over semantic candidates, and a ternary
computing the `is_task` pre-filter — and agreed only by luck. Adding `issue`
to the SQL arm alone would have set the pre-filter to is_task=False, handed
the Python arm a candidate set with no tasks in it, and returned an empty
semantic half for the Issues facet forever with nothing red. `_FACETS` now
generates all three. The Python arm also regains the `status IS NULL` half its
SQL twin always had.
**Issue and spike become facets (rec 2).** 435 issues — 17% of every task —
were filterable nowhere on the human surface, while retired `plan` (90 rows)
had a chip of its own. `_VALID_TYPES` was a hand-kept copy and is now derived.
`plan` stays a valid facet for its legacy rows; it loses its chip.
**Snippets stop being half-present in the feed (rec 4).** All 90 were in the
All list, in no count, wearing an empty badge, and opening in the note editor.
Counts now group by task_kind — every kind for the same two round-trips, which
is why `issue` had no number — and total includes snippets, so the All chip
matches the list it labels. Snippet cards route to /snippets/:id.
**The prose that excused it (rec 6).** `snippet_fields` and the `data` column
both still said pre-0070 rows were "never backfilled". True when 0070 landed,
false since `backfill_snippet_data` shipped, and it read as licence for a
stale mirror.
Tests: the pre-filter can never exclude a row its own facet accepts (the
regression, parameterised over every facet); both dialects select exactly
their own rows; an unknown facet matches nothing; the mirror follows a body or
title write, carries the verdict, and yields to an explicit `data`.
`compiled_sql` moves to tests/helpers rather than becoming a third copy.
Write-up: note #3161.
The badge fix (#3132) exposed the same defect everywhere: 48 rules painting a
token as TEXT on an inline color-mix tint of that same token. Worst raw
measurements, across every tint strength in use, both modes, over
page/raised/hover:
accent 1.53:1 · success 1.67:1 · text-tertiary 2.15:1
warning 2.32:1 · error 2.36:1 against AA's 4.5
THE DEFECT IS IN THE HOUSE, NOT IN SCRIBE. The semantic hues are shared
family-wide, and the accent case was measured against every app's real
accent, not assumed from Scribe's: Minstrel 1.81, Forge 1.87, Steward 1.65,
Roundtable 3.01 — all failing. So the six -fg tokens are recorded on
FabledSword (design system 1), where their parents live, rather than copied
into each app.
45% toward --fs-text-primary clears AA for ALL FIVE accents (4.56-5.00), so
this is one house token rather than five overrides, and it keeps deriving
from --fs-accent — an app that overrides its accent still gets a legible
tinted-text colour in its own colour, the same mechanism as
--fs-accent-soft. The tokens are additive: a sibling app is unaffected until
it regenerates its own stylesheet.
One token is honestly redundant. --fs-text-secondary already passes at
4.82:1, and --fs-text-secondary-fg barely moves it. It exists so the rule
has NO exceptions, because the alternative is a permanent allow-list entry
for the one case that happens to pass — and a guard with an invisible
exception is a guard that erodes.
46 substitutions across 18 files, each rewriting only the `color:` inside a
block that tints its own background.
THE CHECK NOW GATES BOTH SPELLINGS. It previously reported the inline form,
because a gate nobody can satisfy on the day it lands gets switched off.
Both are clean, so both fail the build now.
And the check had a false-positive bug worth naming: its `color\s*:` regex
matched the tail of `border-color`, `border-left-color` and `outline-color`,
so it flagged seven rules that were already correct. A border is a non-text
graphic with a 3:1 floor, not text at 4.5. A check that cries wolf on
correct code is one that gets muted, so that mattered more than the noise.
Verified by construction, not by passing: reintroduced each defect form
(exit 1 each), and confirmed a legitimate border-only rule still exits 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ITEM 1 — the dead canon. StatusBadge is recorded canon (#2960) and its only
consumer, TaskCard, has been unreachable since 2026-04-08, when
TasksListView was deleted in favour of the Knowledge view. Four and a half
months of a canon that rendered nowhere, which is worse than no canon: a
session pulls #2960, builds from it, and matches a component nobody has
seen. TaskCard is deleted (rule 22), and the canon is made real by adoption
rather than by being left as a museum piece.
ITEM 2 — MY OWN ISSUE OVERSTATED THIS, and the correction is the finding.
"Three scoped re-spellings" assumed one shape spelled thrice. Reading them:
KnowledgeView a task-status chip, just smaller -> a real duplicate
WorkspaceTaskPanel a CLICKABLE cycler: pointer,
outlined, transparent background -> a control, not a chip
ProjectView PROJECT lifecycle (active/paused/
completed/archived) -> a different vocabulary
Only the first was ever a duplicate. The others shared a class NAME and
nothing else — which is exactly what would make a future consolidation merge
three unrelated things. So: KnowledgeView adopts StatusBadge/PriorityBadge
via the `compact` variant the canon already anticipated ("interactive/compact
re-spellings are variants of it"); the cycler becomes `.status-cycler`; and
project status becomes its own vocabulary.
And there was a FOURTH, in ProjectListView — the genuine duplicate of
ProjectView's project pill, differing by the amounts two hands differ by:
0.68rem vs 0.7rem, a 14% tint vs 15%, one bordered and one not. Both now use
one ProjectStatusBadge. `statusLabel` went with its only caller.
ITEM 3 — weight. StatusBadge and PriorityBadge used font-weight 600; the
house style allows 400 and 500 only. Also "In Progress" -> "In progress",
which was invisible under `text-transform: uppercase` and becomes visible the
moment the compact variant turns that off.
THE GUARD MISSED FOUR LIVE SITES, which is the part worth keeping. The
project pills painted a hue on an inline `color-mix` tint of itself —
measured 1.61-2.39:1 — and the checker only knew the `--fs-X-bg` token form.
Widened, it finds 48 across the app, 26 of them --fs-accent.
That backlog is not this task, so the check now splits: it GATES the token
form, which is clean, and REPORTS the inline form with a count and its worst
offenders. A gate nobody can satisfy today gets switched off, and then it
guards nothing. Gate re-verified by reintroducing a defect — exit 1 with it,
exit 0 without.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every status and priority badge used its raw hue as TEXT on a 12% tint of
that same hue. Measured on the dark palette, all six pairs failed the kit's
own AA floor: todo 1.60:1, in-progress 1.97:1, done 2.06:1, low 2.02:1,
high 2.92:1, medium 2.97:1, against 4.5. Four also failed in light mode.
The cause is structural, not a bad colour pick. A 12% tint sits near the
surface it composites over, so the hue as text on it has almost nowhere to
go. Strengthening the tint was measured and REJECTED: on a dark palette a
heavier tint moves the chip toward the light text and makes it worse. 12%
was already optimal.
So each pair gains a `-fg` sibling: the hue mixed toward --fs-text-primary
until it clears 4.5:1 worst-case over surface-raised AND surface-hover in
BOTH modes. Mixing toward that token rather than a literal is what makes one
declaration cover both — it inverts by mode, so the text follows.
Recorded in the DESIGN SYSTEM, not hand-written into theme.css: seven tokens
on design system 2, each carrying its measurement and its reasoning, then the
sheet regenerated. theme.css says not to hand-edit the --fs-* block and it is
right — a hand-edit would be silently reverted by the next regeneration.
The ladder keeps its shape. High priority still holds 52% saturation and
medium 31% — the rungs that need to shout still shout. Low, todo and done
wash toward neutral, which is what their own rationales ask for: status-todo
is derived from the border colour precisely so not-yet-started recedes.
Receding and illegible are different things and the old value was the second.
--fs-status-cancelled-fg was found by measuring, not by reasoning. Cancelled
derives from --fs-text-tertiary, which looks like the obviously-correct
"quiet" choice and is a HINT colour tuned for plain surfaces — 2.63:1 on a
badge tint in light mode.
StatusBadge additionally dropped a `color-mix(..., #000 15%)` that darkened
the hue: a light-mode instinct that made these worse on a near-black surface,
and a literal besides.
THE GUARD IS THE POINT. check_design_tokens.py now FAILS on any rule that
paints text with a token on a tint of that same token, and names the -fg
sibling as the fix. Verified by reintroducing the defect: exit 1 with it,
exit 0 without. Unlike a raw literal there is nothing to weigh up, so it
gates rather than reports.
Two `border-top-color` uses keep the raw hue, correctly — a border is a
non-text graphic and needs 3:1, which is what the hue is for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
task_kind was only visible inside the task editor's Kind select, so every
list surface rendered work, issue and spike identically and a list of tasks
hid the fact that three different things were in it.
ONE component, not a fifth spelling. The badge layer had already drifted —
StatusBadge.vue is the recorded canon (#2960) but WorkspaceTaskPanel,
ProjectView and KnowledgeView each carry their own scoped `.status-badge`.
KindBadge is modelled on PriorityBadge, its closest sibling, which already
does the thing that matters here: the DEFAULT value renders nothing. `work`
is most tasks, so badging it would put a chip on nearly every row and say
nothing — the same reason RuleListPane marks only `conditional`.
COLOUR BY TEMPERATURE, measured rather than eyeballed. Issue and spike are
opposite in character — corrective vs exploratory — so they split warm
(warning) against cool (info), which survives being small and stays
distinguishable without reading the word. Neither uses the accent; kind is
not one of the places it is allowed.
The raw semantic colour FAILS the contrast floor on the dark palette:
warning on its own 12% tint measures 2.97:1 against AA's 4.5. So the text is
the hue mixed toward --fs-text-primary, which passes and, because that token
inverts by mode, follows light/dark for free. Measured both ways — issue
5.23:1 dark / 6.68:1 light, spike 5.33:1 / 9.26:1.
`plan` renders hue-free and italic: retired since 0066, so a legacy row
should read as archival rather than as a fourth kind competing for
attention. In KnowledgeView it is passed as null instead, because the type
badge beside it already says "Plan" and two chips reading the same word
would look like two facts.
Weight is 500, not the 600 the two older badges use — the house style allows
400 and 500 only, and copying 600 would spread it.
SERVER FIX, without which this was decorative: dashboard's `_task_row`
omitted task_kind entirely. The badge would have rendered nothing there
while working everywhere else, which reads as "this list has no issues"
rather than as a missing field. The guard is on the payload, where the
omission was.
Surfaces: ProjectView's three status columns, WorkspaceTaskPanel's two task
lists, DashboardView's milestone and no-milestone rows, KnowledgeView's
result rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Scribe's record kinds get reached for interchangeably, and the moment of
choice is the only moment a correction is cheap. Rule 119 puts product
guidance in the instruction surfaces, so the docstring is where this
belongs — but a docstring that only documents parameters answers "how do I
call this" and leaves "should I be calling this at all" unasked.
The gap was lopsided. create_rule and start_planning already carried real
disambiguators; create_note — far and away the highest-volume surface —
carried none at all. The guidance sat in the rarest tool and was missing
from the most common one.
Each surface now opens with ONE deciding question in its own terms rather
than a pasted block:
create_note WHAT ELSE COULD HOLD THIS? note is right when nothing
is owed and nothing enforces
create_task IS ANYTHING ACTUALLY OWED? nothing owed -> note;
an arc -> start_planning
create_snippet SHAPE, OR ADVICE? a snippet is code with a
LOCATION
create_process FOLLOWED, OR READ? applies uninvoked -> rule
create_project_rule now points at the entity check too; it had only ever
covered rule-vs-rule scope.
The guard asserts STRUCTURE, never wording: each surface must name at least
two siblings. Pinning phrasing would make every improvement a test failure,
and a test that punishes editing is a test that gets deleted. Its second
half asserts the Args: block survives — the first check is satisfiable by
turning a docstring into an essay about the other tools, which would be a
worse contract than the one being fixed.
The guard caught two gaps on its first run, one of them its own: "design
system" is hard-wrapped across a line break in create_rule, so matching the
raw docstring reported it absent. _doc() now flattens whitespace. It also
caught start_planning naming only one alternative, which was true and is
now fixed.
Deliberately NOT built: an intent-router tool. It has a bootstrapping
problem — it is itself a tool that must be reached for — and MCP clients
already list every tool's description. Recorded in #3123; build it only if
wrong-surface reaches survive this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PR #132 was rebase-merged, so main carries rewritten copies of dev's
milestone-312 commits under new SHAs. The merge base stayed at 02c1e37, so
a dev->main merge tried to replay all eight already-landed commits and
collided in the two test files both sides had touched.
Resolved by taking dev's version of each: dev's copy is main's content plus
the #3129 additions, verified as a strict superset before resolving rather
than assumed. The merged tree is identical to dev's — asserted below, not
eyeballed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts:
# tests/test_integration_task_kind_spike.py
# tests/test_mcp_tool_tasks_kind.py
CI 4682: test_create_task_issue_sets_kind_provenance_and_systems asserted
task_kind == "issue" and got a MagicMock. That test patches the whole
notes_svc module to keep the database out, so reaching validation through
`notes_svc.minted_kind(...)` handed back a mock — the guard approved
anything and returned nothing real.
The product was wrong, not the test. minted_kind is pure validation, not a
service call, so it is imported by name. A test that stubs the service to
avoid I/O now keeps the guard intact, which is the behaviour you want from
a guard: the only way to disable it should be to say so explicitly.
That test now exercises the real validation, so it doubles as the guard
against this recurring.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`kind` was accepted at CREATE on both doors and dropped at UPDATE on both:
update_task had no such parameter, and the REST PATCH allow-list never read
the field. So a task filed under the wrong kind could never be corrected.
The frontend made it worse by looking like it worked. TaskEditorView binds a
Kind select, marks the form dirty, and HAS ALWAYS SENT `kind` in the update
payload — the store even types it. The route ignored it, returned 200, the
view optimistically updated, the toast said "Task saved", and the old value
came back on reload. Silent success, same class as #2709.
Found by trying to re-file #3126 as a spike after deploying 0091. It could
not be done; the task had to be recreated as #3128 and the original
cancelled.
One seam, not two doors. `minted_kind()` lives in services/notes.py because
the REST route cannot import an MCP tool module and a second spelling of the
list is how the doors would come to disagree. Both create and update route
through it, so a bogus kind is now a readable error rather than a
CheckViolationError surfacing as a 500.
TaskKind joins TaskStatus and TaskPriority as a real enum, and update_note
validates task_kind exactly as it already validated those two — the field
had been reaching setattr through the hasattr guard with no validation at
all, unnoticed only because no door ever offered it.
The `-> plan` question #3129 raised is answered in code rather than left
implicit: MINTABLE_KINDS is work/issue/spike, deliberately NARROWER than the
column's CHECK. `plan` stays a valid stored value because historical
plan-tasks carry it and must stay writable; it is simply not a value any
door hands out, and the refusal names start_planning because a caller
reaching for it wants a plan. The whitelist and the policy answer different
questions and are not the same list.
Every new test reads the value BACK. One that only asserted the call
succeeded would have passed against the broken code — the route returned 200
while dropping the field, which is how this survived long enough to be found
by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The spike CHECK tests constructed Note(is_task=True). `is_task` is a derived
read-only property — `status is not None` — so SQLAlchemy raised
"property 'is_task' of 'Note' object has no setter" before any row reached
the database. All three failed for that, not for anything about migration
0091; the other 80 integration tests passed, including 0090's.
status="todo" is what makes a note a task. Noted inline, since the field
appears in to_dict output and reads like an ordinary column from there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A spike is a shape the other kinds cannot hold. `work` ships a change;
`issue` fixes something broken. A spike is time-boxed and its output is
KNOWLEDGE — it succeeds by producing an answer, and nothing ships at the end
of it. Filing one as `work` makes a finished investigation look like an
abandoned change, which is why the distinction earns a value rather than a
convention.
It is also the record a failed check asks for. This milestone gave rules a
verify_with; when one fails the rule is wrong, and the next move is often to
go and find out what replaced it. notes.arose_from_id already exists (0065),
so constraint -> spike provenance needed no schema at all — only a docstring
saying it is there.
Rule 36: the value and the widened CHECK land in the same migration, DROP
then ADD, exactly as 0065 did for 'issue'. The two whitelists live in one
tuple each so upgrade and downgrade cannot disagree about what the list was
on either side. The downgrade demotes existing spikes to 'work' first —
lossy, deliberately, because the alternative is a downgrade that fails on
real data, and one that says what it did beats one that cannot run.
'plan' stays whitelisted though retired: historical plan-tasks carry it, and
a row that cannot be rewritten cannot be edited, restored or migrated.
The integration test asserts both halves. A test that only proved 'spike' is
accepted would pass just as happily against a table whose CHECK had been
dropped and never re-added — which is the other way rule 36's failure
happens — so an unknown kind is asserted to still raise.
Not in scope, deliberately: any special lifecycle, time-box enforcement, or
gating relationship. It is a kind, not a workflow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rule 27 — the milestone was backend-only until this. Four surfaces:
RULE EDITOR — verify_with and expires_when under a legend that asks the
actual question ("Can this rule go stale?") and says empty is the normal
answer, because most rules are decisions and a form that implies a missing
field would get them filled in out of tidiness. When the SAVED rule carries
a check, the stamp shows with Still true / No longer true beside it. The
stamp reads the stored value, not the draft: an unsaved edit to the textarea
has not been run against anything.
SWEEP PANE — its own surface, not a filter on the rule list. That list can
only ever show one topic of one rulebook, and a rule that has gone false
belongs to no one rulebook; filtering it would under-report, which is the
failure this whole surface exists to catch. Reached from the rulebook list,
below the rulebooks, because that is where you go to look at rules.
RULE ROWS — a chip only on rules carrying a check, so its presence is the
signal. PROJECT RULES TAB — the check shows beside `why` when a rule has
one, read-only: that tab is the project's view of what binds it.
NO AGE-GRADED COLOUR anywhere, deliberately. The sweep is already ordered by
urgency, so a red/amber ramp would restate the ordering AND require an
invented "stale after N days" threshold — a magic number nobody could defend
and the first thing to go out of date. --fs-overdue is error red and reserved
for a broken promise like a missed due date; a verification age is not one,
and colouring it that way makes a rule someone just wrote look broken. Only
"never" is marked, because it is categorically different from a date rather
than a worse one — and it is marked by weight, not hue.
An empty sweep says "Nothing to check", not nothing: good news must not read
as a broken page.
Two chips (tier, then verification) turned out byte-identical, so .rule-chip
moves to rules-shared.css and snippet #2906 is updated to match rather than
left describing a file that has moved on. Its header comment counted the
panes it served; that count went stale the moment a fourth arrived, so it no
longer counts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three surfaces already agree on WHERE a rule goes — the using-scribe
skill's "Where a new rule goes" section and both tool docstrings frame it
as one question, who should this bind. What they did not say is that the
two homes want differently SHAPED rules, and one deferral was actively
misleading.
`create_project_rule` said `tier: "always_on" or "conditional" — see
create_rule`. That imports a bar calibrated for a different blast radius.
On a rulebook rule always_on means every session in every project, so the
test is severe: the trigger must be nameless. A project rule is already
scoped by construction, so always_on costs only that project's sessions —
and being specific, which the family test treats as the signal for
conditional, is what project rules are FOR. The instance's own data says
so: rules 78, 115 and 119 are all project rules and all always_on.
Not zero bar, a different one: conditional is right when the rule is about
one AREA of a large project, because forty always-on rules on one project
reproduces locally the preload bloat milestone 307 fixed globally.
Also:
- create_rule now says to write the general form WITHOUT hedging for
exceptions — a project needing to narrow it writes its own and links
with overrides/elaborates. A rulebook rule padded with "unless…" for two
projects is two project rules that were never written. Only the project
side mentioned that relationship; the side that benefits from it did not.
- arose_from_id: reach for it harder on a project rule, which usually comes
from one traceable incident in the repo, where a family rule is more
often a standing preference with no single origin.
- system_ids is worth setting on a project rule too — it is what lets a
conditional one arrive with its area.
- when_to_apply no longer claims to "decide" the tier here, which stopped
being true one entry down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The query the last two steps were storage for. `rules_due_for_verification`
returns every rule carrying a `verify_with`, ordered by `verified_at` ASC
NULLS FIRST, each row carrying the check IN FULL — the opposite call from
rule_brief, because the reader is about to go and run it.
NULLS FIRST is the ordering this turns on. Postgres sorts NULLs last on an
ASC ordering, which would put the rules nobody has ever confirmed BEHIND
every rule someone once looked at. Exactly backwards: a claim with no
evidence at all outranks an old one.
Rules with no check never appear, and that is the property that keeps the
list worth reading. Most rules are decisions — no truth value, nothing to go
and check. If they appeared here the sweep would be the rulebook.
`mark_rule_verified(rule_id, still_true)` closes the loop, asymmetrically:
passing writes a stamp, FAILING WRITES NOTHING. There is no "verified false"
state because a rule whose check failed is not in a special condition, it is
wrong — and recording the failure as a flag would let it sit there being
false with the sweep satisfied that someone had looked. So it stays at the
top until someone corrects or retires it, and the response says so.
An unrecognised `tier` filter raises rather than falling back. _valid_tier's
silent always_on default is right for a WRITE — a typo should leave a rule
binding — and wrong for a FILTER, where the same fallback quietly answers a
different question and returns a short list that reads as good news.
Deliberately NOT filterable by project: a project reaches rules through
project scope, subscriptions, always-on rulebooks and exclusions, and a
filter missing one of those paths would UNDER-report — the exact failure
this surface exists to prevent. Said so in the docstring rather than
shipping a half-correct filter.
Ownership-scoped like every other rule read (owned rulebook, or owned
project), in ONE statement with an OR across the XOR rather than two queries
merged in Python, so the ordering is the database's and cannot disagree with
itself. Note that rules have no sharing ACL in this schema — no rule_shares,
no rulebook_shares — so there is no wider set for access.py to consult here.
Also fixes a test title that had been lying for ten tools: "all sixteen
tools" asserted 26. The number now lives only in the assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_update_rule_only_sends_non_default_fields pins that the MCP door
forwards only what the caller actually gave. `clear` is now always
forwarded — an empty tuple is "clear nothing", a value rather than an
absent argument — so the expected kwargs gained it. The property under
test is unchanged: everything left at its default still stays out.
Two tests added beside it while the shape is in view: naming a field for
clearing reaches the service as `clear`, and the check fields are
forwarded when given.
CI 4630 otherwise green — the integration lane ran all six of the new
real-Postgres cases (72 selected, was 66) and applied 0089 -> 0090.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
verify_with / expires_when now reach a rule through both doors and come back
on every read. The open question this step existed to settle was how to
UNSET a nullable field, and the answer is one convention per door:
- MCP: "" still means "leave unchanged" — an agent filling three fields must
not wipe the other five — so clearing is explicit, clear_fields=["..."].
Naming the field is the one form that cannot happen by accident.
- REST: a cleared form input arrives as "", and the service normalises "" to
NULL for every nullable rule column, so an emptied input does what it looks
like it does.
Two idioms, one outcome, and the normalisation is what makes the step-3 sweep
correct: `verify_with IS NOT NULL` would otherwise be true for every rule ever
touched through the UI, and the sweep would list the whole rulebook and mean
nothing. to_dict renders "" and NULL identically, so this is only visible
against a real column — hence the integration module rather than a mock.
Editing verify_with drops verified_at. A stamp certifies A CHECK, not a rule;
reword the check and the old stamp vouches for something that no longer
exists. Safe direction, same asymmetry as _valid_tier: a rule wrongly listed
as due costs one look, a rule wrongly vouched for costs the thing the sweep
exists to catch. Editing anything else leaves the stamp alone, or a rulebook
tidy-up would reset every constraint and the ordering would carry nothing.
Reads: rule_brief attaches `last_verified` ONLY to a rule that carries a
check — its presence is the signal, and it says both "this asserts a fact
that can go false" and "here is how long ago anyone confirmed it". "never"
rather than null, per #2483. The check text itself stays in get_rule; a
listing needs to know which rules can rot, not how to test them. Search hits
carry the full trio, since a hit is exactly the moment someone is about to
act on a rule.
Also folds in the #3078 finding, which had been sitting as a note: create_rule
now teaches that when_to_apply is the retrieval surface and must carry the
SYMPTOM — the words you would type while stuck — not just the situation.
fake_rule gains the three fields as None for the reason the helper already
documents one line up: unnamed, verify_with is a truthy MagicMock and every
stand-in rule would claim a check it does not have.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A rulebook holds two kinds of row in one table. A NORM is a decision: no
truth value, changes only when its author changes it, and they know they
did. A CONSTRAINT asserts a fact about someone else's software, and goes
false with nobody present. Milestone 307's audit found nine stale sites;
every one was a constraint, and not one norm had rotted.
Three nullable columns so a rule can say how to check itself. expires_when
is a STATE, not a date — constraints expire when the ground moves, not on a
schedule. verified_at NULL means never checked and sorts FIRST in the sweep
to come: unexamined outranks examined-long-ago. Most rules set none of the
three; a null verify_with is the marker for "this is a decision, there is
nothing to go and check," and it only reads that way while it stays honest.
Nothing is backfilled and nothing is indexed. A migration cannot invent a
check any more than 0088 could invent a trigger, and the sweep reads a whole
rulebook — hundreds of rows, on operator demand, never on a request path.
Also, in the backup service the fields had to pass through:
- Restore now remaps arose_from_id through note_id_map. It has been exported
since 0088 and silently dropped on the way back in ever since, so every
restore lost every rule's provenance link.
- _dt_or_none, because _dt substitutes now() for an absent value. That is
right for created_at/updated_at and wrong here: a rule nobody ever checked
would restore looking freshly checked and fall to the bottom of the sweep
it should top.
Column additions do not move BACKUP_VERSION; only new sections do, as when
0088 added when_to_apply/tier/arose_from_id to the same helper.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The spike CHECK tests constructed Note(is_task=True). `is_task` is a derived
read-only property — `status is not None` — so SQLAlchemy raised
"property 'is_task' of 'Note' object has no setter" before any row reached
the database. All three failed for that, not for anything about migration
0091; the other 80 integration tests passed, including 0090's.
status="todo" is what makes a note a task. Noted inline, since the field
appears in to_dict output and reads like an ordinary column from there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A spike is a shape the other kinds cannot hold. `work` ships a change;
`issue` fixes something broken. A spike is time-boxed and its output is
KNOWLEDGE — it succeeds by producing an answer, and nothing ships at the end
of it. Filing one as `work` makes a finished investigation look like an
abandoned change, which is why the distinction earns a value rather than a
convention.
It is also the record a failed check asks for. This milestone gave rules a
verify_with; when one fails the rule is wrong, and the next move is often to
go and find out what replaced it. notes.arose_from_id already exists (0065),
so constraint -> spike provenance needed no schema at all — only a docstring
saying it is there.
Rule 36: the value and the widened CHECK land in the same migration, DROP
then ADD, exactly as 0065 did for 'issue'. The two whitelists live in one
tuple each so upgrade and downgrade cannot disagree about what the list was
on either side. The downgrade demotes existing spikes to 'work' first —
lossy, deliberately, because the alternative is a downgrade that fails on
real data, and one that says what it did beats one that cannot run.
'plan' stays whitelisted though retired: historical plan-tasks carry it, and
a row that cannot be rewritten cannot be edited, restored or migrated.
The integration test asserts both halves. A test that only proved 'spike' is
accepted would pass just as happily against a table whose CHECK had been
dropped and never re-added — which is the other way rule 36's failure
happens — so an unknown kind is asserted to still raise.
Not in scope, deliberately: any special lifecycle, time-box enforcement, or
gating relationship. It is a kind, not a workflow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rule 27 — the milestone was backend-only until this. Four surfaces:
RULE EDITOR — verify_with and expires_when under a legend that asks the
actual question ("Can this rule go stale?") and says empty is the normal
answer, because most rules are decisions and a form that implies a missing
field would get them filled in out of tidiness. When the SAVED rule carries
a check, the stamp shows with Still true / No longer true beside it. The
stamp reads the stored value, not the draft: an unsaved edit to the textarea
has not been run against anything.
SWEEP PANE — its own surface, not a filter on the rule list. That list can
only ever show one topic of one rulebook, and a rule that has gone false
belongs to no one rulebook; filtering it would under-report, which is the
failure this whole surface exists to catch. Reached from the rulebook list,
below the rulebooks, because that is where you go to look at rules.
RULE ROWS — a chip only on rules carrying a check, so its presence is the
signal. PROJECT RULES TAB — the check shows beside `why` when a rule has
one, read-only: that tab is the project's view of what binds it.
NO AGE-GRADED COLOUR anywhere, deliberately. The sweep is already ordered by
urgency, so a red/amber ramp would restate the ordering AND require an
invented "stale after N days" threshold — a magic number nobody could defend
and the first thing to go out of date. --fs-overdue is error red and reserved
for a broken promise like a missed due date; a verification age is not one,
and colouring it that way makes a rule someone just wrote look broken. Only
"never" is marked, because it is categorically different from a date rather
than a worse one — and it is marked by weight, not hue.
An empty sweep says "Nothing to check", not nothing: good news must not read
as a broken page.
Two chips (tier, then verification) turned out byte-identical, so .rule-chip
moves to rules-shared.css and snippet #2906 is updated to match rather than
left describing a file that has moved on. Its header comment counted the
panes it served; that count went stale the moment a fourth arrived, so it no
longer counts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three surfaces already agree on WHERE a rule goes — the using-scribe
skill's "Where a new rule goes" section and both tool docstrings frame it
as one question, who should this bind. What they did not say is that the
two homes want differently SHAPED rules, and one deferral was actively
misleading.
`create_project_rule` said `tier: "always_on" or "conditional" — see
create_rule`. That imports a bar calibrated for a different blast radius.
On a rulebook rule always_on means every session in every project, so the
test is severe: the trigger must be nameless. A project rule is already
scoped by construction, so always_on costs only that project's sessions —
and being specific, which the family test treats as the signal for
conditional, is what project rules are FOR. The instance's own data says
so: rules 78, 115 and 119 are all project rules and all always_on.
Not zero bar, a different one: conditional is right when the rule is about
one AREA of a large project, because forty always-on rules on one project
reproduces locally the preload bloat milestone 307 fixed globally.
Also:
- create_rule now says to write the general form WITHOUT hedging for
exceptions — a project needing to narrow it writes its own and links
with overrides/elaborates. A rulebook rule padded with "unless…" for two
projects is two project rules that were never written. Only the project
side mentioned that relationship; the side that benefits from it did not.
- arose_from_id: reach for it harder on a project rule, which usually comes
from one traceable incident in the repo, where a family rule is more
often a standing preference with no single origin.
- system_ids is worth setting on a project rule too — it is what lets a
conditional one arrive with its area.
- when_to_apply no longer claims to "decide" the tier here, which stopped
being true one entry down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The query the last two steps were storage for. `rules_due_for_verification`
returns every rule carrying a `verify_with`, ordered by `verified_at` ASC
NULLS FIRST, each row carrying the check IN FULL — the opposite call from
rule_brief, because the reader is about to go and run it.
NULLS FIRST is the ordering this turns on. Postgres sorts NULLs last on an
ASC ordering, which would put the rules nobody has ever confirmed BEHIND
every rule someone once looked at. Exactly backwards: a claim with no
evidence at all outranks an old one.
Rules with no check never appear, and that is the property that keeps the
list worth reading. Most rules are decisions — no truth value, nothing to go
and check. If they appeared here the sweep would be the rulebook.
`mark_rule_verified(rule_id, still_true)` closes the loop, asymmetrically:
passing writes a stamp, FAILING WRITES NOTHING. There is no "verified false"
state because a rule whose check failed is not in a special condition, it is
wrong — and recording the failure as a flag would let it sit there being
false with the sweep satisfied that someone had looked. So it stays at the
top until someone corrects or retires it, and the response says so.
An unrecognised `tier` filter raises rather than falling back. _valid_tier's
silent always_on default is right for a WRITE — a typo should leave a rule
binding — and wrong for a FILTER, where the same fallback quietly answers a
different question and returns a short list that reads as good news.
Deliberately NOT filterable by project: a project reaches rules through
project scope, subscriptions, always-on rulebooks and exclusions, and a
filter missing one of those paths would UNDER-report — the exact failure
this surface exists to prevent. Said so in the docstring rather than
shipping a half-correct filter.
Ownership-scoped like every other rule read (owned rulebook, or owned
project), in ONE statement with an OR across the XOR rather than two queries
merged in Python, so the ordering is the database's and cannot disagree with
itself. Note that rules have no sharing ACL in this schema — no rule_shares,
no rulebook_shares — so there is no wider set for access.py to consult here.
Also fixes a test title that had been lying for ten tools: "all sixteen
tools" asserted 26. The number now lives only in the assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_update_rule_only_sends_non_default_fields pins that the MCP door
forwards only what the caller actually gave. `clear` is now always
forwarded — an empty tuple is "clear nothing", a value rather than an
absent argument — so the expected kwargs gained it. The property under
test is unchanged: everything left at its default still stays out.
Two tests added beside it while the shape is in view: naming a field for
clearing reaches the service as `clear`, and the check fields are
forwarded when given.
CI 4630 otherwise green — the integration lane ran all six of the new
real-Postgres cases (72 selected, was 66) and applied 0089 -> 0090.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
verify_with / expires_when now reach a rule through both doors and come back
on every read. The open question this step existed to settle was how to
UNSET a nullable field, and the answer is one convention per door:
- MCP: "" still means "leave unchanged" — an agent filling three fields must
not wipe the other five — so clearing is explicit, clear_fields=["..."].
Naming the field is the one form that cannot happen by accident.
- REST: a cleared form input arrives as "", and the service normalises "" to
NULL for every nullable rule column, so an emptied input does what it looks
like it does.
Two idioms, one outcome, and the normalisation is what makes the step-3 sweep
correct: `verify_with IS NOT NULL` would otherwise be true for every rule ever
touched through the UI, and the sweep would list the whole rulebook and mean
nothing. to_dict renders "" and NULL identically, so this is only visible
against a real column — hence the integration module rather than a mock.
Editing verify_with drops verified_at. A stamp certifies A CHECK, not a rule;
reword the check and the old stamp vouches for something that no longer
exists. Safe direction, same asymmetry as _valid_tier: a rule wrongly listed
as due costs one look, a rule wrongly vouched for costs the thing the sweep
exists to catch. Editing anything else leaves the stamp alone, or a rulebook
tidy-up would reset every constraint and the ordering would carry nothing.
Reads: rule_brief attaches `last_verified` ONLY to a rule that carries a
check — its presence is the signal, and it says both "this asserts a fact
that can go false" and "here is how long ago anyone confirmed it". "never"
rather than null, per #2483. The check text itself stays in get_rule; a
listing needs to know which rules can rot, not how to test them. Search hits
carry the full trio, since a hit is exactly the moment someone is about to
act on a rule.
Also folds in the #3078 finding, which had been sitting as a note: create_rule
now teaches that when_to_apply is the retrieval surface and must carry the
SYMPTOM — the words you would type while stuck — not just the situation.
fake_rule gains the three fields as None for the reason the helper already
documents one line up: unnamed, verify_with is a truthy MagicMock and every
stand-in rule would claim a check it does not have.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A rulebook holds two kinds of row in one table. A NORM is a decision: no
truth value, changes only when its author changes it, and they know they
did. A CONSTRAINT asserts a fact about someone else's software, and goes
false with nobody present. Milestone 307's audit found nine stale sites;
every one was a constraint, and not one norm had rotted.
Three nullable columns so a rule can say how to check itself. expires_when
is a STATE, not a date — constraints expire when the ground moves, not on a
schedule. verified_at NULL means never checked and sorts FIRST in the sweep
to come: unexamined outranks examined-long-ago. Most rules set none of the
three; a null verify_with is the marker for "this is a decision, there is
nothing to go and check," and it only reads that way while it stays honest.
Nothing is backfilled and nothing is indexed. A migration cannot invent a
check any more than 0088 could invent a trigger, and the sweep reads a whole
rulebook — hundreds of rows, on operator demand, never on a request path.
Also, in the backup service the fields had to pass through:
- Restore now remaps arose_from_id through note_id_map. It has been exported
since 0088 and silently dropped on the way back in ever since, so every
restore lost every rule's provenance link.
- _dt_or_none, because _dt substitutes now() for an absent value. That is
right for created_at/updated_at and wrong here: a rule nobody ever checked
would restore looking freshly checked and fall to the bottom of the sweep
it should top.
Column additions do not move BACKUP_VERSION; only new sections do, as when
0088 added when_to_apply/tier/arose_from_id to the same helper.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Steps 1-5 of milestone 307. Design in note 3026; the step-6 true-up proposal is note 3061 and needs this deployed first.
Behaviour-neutral by construction: tier defaults to always_on, so every rule this instance already has keeps binding exactly as it did. That guarantee is the first case in tests/test_integration_rule_surfacing.py, against real Postgres.
Migrations 0087-0089 are additive. First boot backfills rule embeddings in the background. Plugin manifest at 0.1.47.
A conditional rule is not resident, so a session can be about to violate one it
was never handed. This arm notices: when what is being written resembles a
rule's trigger, the hint names it and says to read it before deciding it does
not apply.
A SUGGESTION, and the plan was wrong about why it could be more. It claimed the
hook "already resolves a path to an area" — it does not, and nothing in Scribe
maps a path to a System or a canonical area (build_write_path_hint resolves
paths against snippet LOCATIONS, a different index; the learned-alias idea
belongs to another project). Correction logged on the task. Rather than invent
path→area inference to make a stale claim true, the arm does what D7 already
decided and what this surface already IS: tags bind at enter_project, meaning
suggests here. The header of the hook says NEVER BLOCKS; dressing a hint up as
binding would have been the actual mistake.
CONDITIONAL RULES ONLY. An always-on rule is already in the session, so
re-offering it is noise — and noise on a hint that fires on every write is how
a hint gets ignored.
Telemetry goes to retrieval_logs, NOT note_usage_events, and that is a
correctness call rather than a preference: note_usage ids are REMAPPED on a
backup restore, so a rule id written there would come back attached to whatever
note took that number — silently corrupting the evidence the next true-up is
supposed to read. retrieval_logs is never restored and `source` already
separates surfaces. record_retrieval's `results` type widened to match what it
actually needs (an `.id`), instead of passing a Rule to something annotated Note.
The rule dedup gets its OWN state file and query parameter, like the three
channels before it — #2708's lesson was that one shared channel lets a hint of
one class silence a different class that had never been shown. Plugin version
bumped: a hook change clients cannot see did not ship (#1040).
The stub is autouse in conftest rather than added to forty-odd call sites: the
arm loads an embedding model, and every existing test that stubs the NOTES
search would otherwise pull a real model in through the one arm it had no way
to know about.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
NameError at runtime: the tier/area clause and co_surfaced_partners both build
an or_(), which rulebooks.py never imported. py_compile passes on this (it is
a name error, not a syntax error) and so did the whole unit suite.
Where it surfaced is the useful part. The unit tests mock the session, so the
project-areas query returns empty, `reachable` stays None, and the or_ branch
is never taken — they exercised the path that avoids the bug. Only the
integration lane, with a real project and real rows, went down the branch that
needed the name. Six integration tests caught it, including the inception one
that merely calls get_applicable_rules in passing.
A reminder about which lane proves what: mocking the thing that selects the
branch means the branch is untested by construction.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The payoff step: a rule stops having to be resident to be honoured.
- list_always_on_rules returns the ALWAYS-ON tier only. It is the session-start
call, made before any project is in scope, so there is no area vocabulary to
match a conditional rule against yet.
- get_applicable_rules carries a conditional rule when the project works in an
area the rule is tagged to — resolved through systems.canonical_id, so the
project's own NAME for the area is irrelevant, which is the entire reason the
catalog exists. The gate is applied IN SQL, so `limit` counts rules that will
actually surface rather than rules about to be dropped.
- Bindingness is a deterministic TAG match, never a similarity score (D7). The
vector channel stays a suggestion, in search.
co_surfaced_partners is the fix that rule 144 never had. It was split off rule
46 and folded back the same day because "either rule could surface without the
other and miss exposing a project to what the entire shape is intended to be" —
correct, and merging was the only remedy available. Now a partner ARRIVES with
its other half even when nothing else selected it, tagged `via: co_surfaces` so
the payload says why. Two limits, both deliberate: only rules the caller owns,
because an edge is not a back door into someone else's rulebook; and a
project's suppressions are passed as exclusions, because an explicit mute is a
decision and an edge does not outrank it.
COMPATIBILITY, asserted first in the integration test rather than reasoned
about: a rule with no tier, no areas and no edges binds exactly as it did
before any of this existed. `tier` defaults to always_on, so an install
upgrades and every rule it already had keeps arriving. Getting that backwards
would silently stop enforcing rules people rely on, which is worse than any
amount of payload bloat.
Four unit tests were coupled to the ORDER of a mocked session's execute()
calls, so a new query broke them. Rather than pad the sequence and deepen that
coupling, the three post-query lookups are stubbed by name — they have their
own coverage, and the real wiring is proven against Postgres.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
F821: the quoted `"Rule"` in semantic_search_rules' return annotation
evaluates fine at runtime — a string inside a subscript is a value, not a name
lookup — but it points at nothing a reader or a type checker can follow, which
is what ruff is objecting to and it is right to.
A TYPE_CHECKING import resolves it at zero runtime cost, while the real import
stays inside the function so this module still does not pull in the rulebook
models. Placed after the import block, where isort wants the guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rules were the only major record type with no vector, so `search` could never
return one and a rule could arrive only by being preloaded. That single fact is
what made every rule compete for one always-on budget.
THE DECISION THE TASK ASKED FOR, made explicitly: a sibling rule_embeddings
table, not a polymorphic embedding row. The ROW could have been generalised;
the SEARCH could not. semantic_search_notes is Note-specific scoping end to end
— the visibility clause, the supersession penalty, note_type/task_kind/system
filters — and a rule shares none of it, scoping instead by rulebook ownership
or project. Generalising the row while still needing two searches is the worst
of both: a key with referential integrity to neither table, on the path every
session start runs, to share four columns. What is genuinely common is
BEHAVIOUR — get_embedding, chunk_document, embedding_text, CHUNKER_VERSION —
and those are reused as-is. Sharing them is the DRY win; sharing the table
would have been the DRY costume.
The document shape is measured, not chosen (note 2485). That pass found the
snippet was the only discriminative record in the corpus — a 0.153
top-to-second gap against 0.010-0.023 — and that the cause was its SHAPE:
purpose stated twice in a short single-topic document. rule_document
reproduces it: the trigger in the title AND as the body's first line.
And it excludes `why`, which matters more than any of it. `why` is dated
incident narrative — rule 46's runs to 4,300 characters — and long multi-topic
prose is exactly what made sixteen dev-logs mutually indistinguishable. Adding
it would not give the vector more to work with; it would give every rule the
SAME thing to work with. rule_document takes no `why` parameter at all, so a
well-meaning caller cannot pass one.
A rule with no trigger degrades to title + statement — findable, less sharp.
That is an argument for backfilling triggers (step 6), not for padding the
document with whatever text is nearby.
search(content_type="rule") returns the rule WITH its why and how_to_apply:
they are its operational half, the session payload never carries them, and a
caller who went looking should not have to re-fetch. Writes re-index
fire-and-forget like notes; startup backfills in its own try block so neither
backfill can skip the other. rule_embeddings is derived, so it joins
note_embeddings in the backup's explicitly-NOT-included list.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vue-tsc caught what I missed: there were THREE places hand-building a rule
list row, not two. fetchRules mapped full rules down to the same four fields
in a spot far from the other two, so consolidating the pair I could see left
this one behind — which is precisely how the server side ended up with three
divergent trim dicts in the first place.
All three now go through toHeader.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rule 27 — the schema and both doors shipped with no human surface, so step 3
was not shippable until this.
RuleEditorSlideOver gains the trigger, the tier, the areas, and a read-only
view of the rule's edges. The tier is a radio pair carrying the test itself
rather than a bare toggle: can you name the trigger WITHOUT naming a system,
an artifact type or a moment? If the honest answer is "whenever you are
working", it is always on. It also says why conditional is not a demotion —
it costs nothing when irrelevant, which is what lets a rule be as long as it
needs to be. The relations block states the rule the whole milestone turns on:
rules that FAIL TOGETHER are linked, never merged.
RuleListPane shows the trigger and the LAST-CHANGED DATE on every row, and
marks conditional only — always_on is the default and badging every row would
say nothing. The date is the cheap triage the FabledCurator case wanted: a
rule whose age predates the capability it duplicates is visible at a glance
instead of needing a get_rule to find out.
ProjectRulesTab's inline create form gains the same two fields, because a
project rule bloats exactly the way a family one does — FabledCurator has 23
of them.
Two type fixes the new shapes forced, both worth keeping:
- toHeader() in the store: a list row is the server's rule_brief, so patching
a list locally has to mirror every field it carries or the two disagree.
There were two hand-built four-field literals doing that job.
- ApplicableRules.rules / .project_rules are now described AS RuleHeader
rather than as two more hand-written shapes — the same builder produces
them, so the same type should describe them.
groupByRulebookAndTopic skips a null-topic rule rather than widening
TopicGroup to accept one: a rule carries topic_id XOR project_id, so a null
topic in that list means something is wrong upstream, and a widened type would
hide it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MCP and REST both gain when_to_apply / tier / system_ids / arose_from_id on
create and update, plus relate_rules / unrelate_rules for the typed edges, and
get_rule now returns a rule's areas and relations alongside it.
rule_detail() is a SERVICE function, not one per door. It started as a copy in
each — identical, and the prior-art hook flagged it immediately, which is the
same lesson rules_payload (#2858) already recorded: a second copy drifts. Both
doors call the one seam, so create, update and get cannot disagree about what a
rule looks like coming back.
The authoring guidance lands in create_rule's docstring rather than in a rule,
per rule 119 as the operator described it: this is behaviour every instance
should inherit, not one operator's preference. It states the test —
ONE RULE = ONE THING YOU COULD VIOLATE. Rules that FAIL TOGETHER get linked
with relate_rules(kind="co_surfaces"), never merged into one row.
— and names why merging loses: a merged rule cannot be cited, surfaced or
suppressed a clause at a time, and it grows without limit because adding to it
is always cheaper than adding a rule. create_project_rule says the same about
"overrides", which is what FabledCurator's 85/86 should have been instead of
near-copies that drift from their parent.
The tier arg carries the test itself: can you name the trigger WITHOUT naming a
system, an artifact type or a moment? If the honest answer is "whenever you are
working", it is always_on.
Tests: the applicable-rules cases fabricated raw tuples matching the old column
lists, so they move to the entity shape via fake_rule; new cases pin rule_brief
(a DATE not a stamp, the depth left to get_rule, no null keys) and that an
unknown tier falls back to BINDING. fake_rule gains when_to_apply / tier /
arose_from_id for the note-2109 reason the helper exists: unnamed, they would
be truthy MagicMocks. The tool tests stub the new rule_detail seam — they are
about argument forwarding and have no database.
The module header's "Sixteen tools" had been wrong for two milestones; the
registration count test is what actually catches that, so the header now says
so instead of carrying a number.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A rule could not state its trigger, its area, or its siblings, so all three
were being written as prose instead: a System's charter restating rule text,
a `why` naming the note that caused it, and two halves of one shape merged
into a single row because either could surface without the other.
Migration 0088 adds the four fields those workarounds stood in for:
- `when_to_apply` — the trigger. Nullable in the DB and required at the
service layer: existing rules have none and a migration cannot invent one.
- `tier` — always_on | conditional, defaulting to always_on. This migration
therefore changes NOTHING about which rules bind; an install upgrades and
every rule keeps arriving exactly as before. Getting that backwards is the
one failure this milestone exists to prevent, so _valid_tier falls back to
always_on rather than silently un-binding a rule with a typo'd tier.
- `arose_from_id` — the record that caused the rule, the edge notes and tasks
already have. SET NULL: trashing the source does not repeal the rule.
- `rule_systems` / `rule_relations` — the canon tag and the typed edges
(co_surfaces / overrides / elaborates), each earned from a workaround its
absence forced.
rule_brief() replaces the THREE hand-written trim dicts that had already
diverged — two carried topic_id, one didn't, and none carried the timestamps
the model has held all along. That omission is why a rule written before the
capability it duplicates was indistinguishable at read time from one still
doing work. It now carries updated_at as a DATE: the question is "how old is
this", and a full stamp across the always-on set is ~2k characters for
precision nobody reads. The two callers select the ENTITY rather than a column
list, so rule_brief stays the single place deciding what a surfaced rule says.
Backup: both new tables carried, area tags by canonical SLUG (ids are
per-install). The rule-relation restore runs after ALL rules exist and after
the catalog, because an edge names two rules and a tag names a global row —
sections renumbered so the file reads in dependency order. A pre-0088 payload
restores with tier=always_on, i.e. binding exactly as when it was taken.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two tests further down test_mcp_tool_systems.py still drove the gate through
the tool — `svc.list_systems` stubbed, the tool doing the normalising — so the
new `await systems_svc.assess_system_name(...)` hit an unstubbed MagicMock.
Stubbing them at the tool would have kept testing the wrong layer. The
normalisation cases belong with the logic, so they move to
tests/test_services_systems.py as real coverage of assess_system_name: case and
whitespace folding, exact-beats-overlap (and that an exact hit short-circuits
the lesser lookup), no invented match, fail-open on both arms, and silence for a
nameless system.
What stays the tool's job — rendering a duplicate, applying an exact area,
offering an overlap — is already covered at the top of that file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 1 found the reason the standard names never held, and it is sharper than
"prose doesn't fire": the list WAS real and it WAS seeded — but only on the
inception path, for a project with zero Systems. Ad-hoc create_system never
consulted it, which is how Forge minted "CI and Release" and Portal minted
"CI & release" after the constant already existed. This wires the vocabulary to
the moment that mints a name.
- services/systems.assess_system_name: the local duplicate gate AND the
catalog lookup, in ONE service function both doors call. The gate lived only
in the MCP tool, which is exactly how the web UI shipped without a check the
agent surface enforced (#2482). REST now answers 409 with the System that
already covers the area.
- An `exact` catalog hit is APPLIED (mechanical — the names differ only in
spelling). An `overlap` is only OFFERED, on both doors: applying a judgment
call silently is how a cross-project rule surfaces in the wrong project.
- canonical_systems.best_overlap is the ONE scorer behind the create-time
offer and the review sweep, so the two surfaces can never name different
areas for one System. It also takes the catalog the caller already holds,
so the review is not an N+1.
UI (folded in from step 1 — rule 27, that step shipped with no human surface):
- SystemsSection: a Shared area picker on create and edit, the area on each
card, and a collapsed review of proposals that appears only when there is
something to decide. `exact` and `overlap` never share a style — one is
mechanical, the other is the reviewer's judgment, and presenting them alike
is how a wrong mapping gets waved through.
- Settings → Admin → Areas: the catalog itself, showing each entry's slug,
because the slug is what decides whether two names are the same area and a
rename moves it.
- A picker rather than a live matcher: reproducing the slug rule in TypeScript
would give this feature two matchers to keep in step — the exact drift the
catalog exists to end. The server stays authoritative.
tests/helpers.fake_system gains canonical_id=None: an unnamed attribute is an
auto-MagicMock and therefore truthy, which is the trap that helper exists for
(note 2109) and a nullable FK walks straight into it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI caught the seam the promotion opened: the standard names now come from an
async catalog read, and the unit test patches systems_svc wholesale — so the
read raised, the fail-open swallowed it, and the ask shipped without the names
it is supposed to carry.
Stub standard_systems in that test, and assert the wiring rather than the
vocabulary: THAT the seeded set is these eight is migration 0087's business and
belongs in the inception integration test, against a real database.
Adds the case the promotion actually created — an unreachable or empty catalog
must still produce the ask. The names are an aid to the question, not the
question; degrading to a weaker nudge is fine, going silent is not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The eight standard area names already existed — as STANDARD_SYSTEMS, a tuple in
services/systems.py that milestone 297 seeds at inception. A constant cannot be
a foreign key, so nothing outside a project could reference an area: systems.
project_id is NOT NULL, and a rule that spans projects would have to chain
itself to one project's row. And because the list only ever applied on the
inception-seed path, three spellings of one area reached this instance anyway
(CI & runners / CI and Release / CI & release).
- canonical_systems: global, no user_id — a shared project inherits the
vocabulary instead of re-earning it. Migration 0087 seeds the same eight.
- systems.canonical_id: nullable, SET NULL. Association only — no System is
renamed and record_systems is untouched, so no record's tags move.
- canonical_slug folds &/and, case and punctuation, so spelling variants map
mechanically and a real difference ("CI & runners") becomes a proposal a
human confirms. propose_mappings reports; set_system_canonical is the only
writer.
- seed_standard_systems now reads the catalog and maps as it mints, so a
project born standard never needs a reconciliation pass.
- Catalog writes are admin-only; reads are open — a global list anyone can
extend stops being shared.
- backup: carried by SLUG, not id (ids are per-install). Restore reuses the
target's own rows and only creates entries an admin added on the source; an
unknown slug restores unmapped rather than failing.
Rule 22: STANDARD_SYSTEMS is removed, not deprecated. Rule 115: nothing seeded
names an app, repo or house convention. Design in note 3026.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DRY pass 3's remainder. Both halves start from enumeration, because the task's
candidate list was hypotheses and the process requires counting before
proposing — and counting changed the answer twice.
## The list_* family: a limit with no offset
Enumerated all 19 `list_*` MCP tools first. They are genuinely heterogeneous —
8 take `project_id`, 6 take `limit`, six take no arguments at all — so a
common-parameter guard would invent a convention the API does not have, which
is the over-DRY trap (§5). One contract IS real: a `limit` without an `offset`
is a truncation with no continuation. The caller is told there are 250 results,
handed 50, and given no way to ask for the rest.
Two tools had it, and both were capped over a service that already accepted an
offset: `snippets_svc.list_snippets(offset=0)` was simply not exposed, and
`list_processes` passed a hardcoded `offset=0` into `query_knowledge`. The
capability existed one layer down in both; only the door was missing — the
missing-sibling shape exactly. Both now expose it.
`tests/test_mcp_list_family.py` guards it, with `list_tags` exempted for a
stated reason (a ranked top-N over a bounded vocabulary has no "rest" to page
into). Candidates derived, decision explicit, same design as test_mcp_auth —
plus the reverse checks: a stale exemption, and an offset with no limit, which
would page through an unbounded result set. Verified non-vacuous by running the
sweep against the pre-fix tree, where it fails naming both tools.
## The verb pairs: no finding, which is the finding
`preview`/`apply` and `dry_run`/`commit` do not exist anywhere in the 102
tools — those were guesses about a shape Scribe never adopted. `count_*` does
not exist either. Of the create/delete stems only `project_rule` lacks a
`delete_X`, and deliberately: a project rule IS a rule, `delete_rule` removes
it, and the docstring says so. `force` sits on 6 of 7 duplicate-gated creates;
the exception is `create_system`, whose gate is an exact normalized-NAME match
rather than a semantic near-match — forcing it would split one area's records
across two piles, which its own message explains. No guard added: it would
need a seven-entry exemption list to defend against a hypothetical. Recorded
on the leave-alone list instead, which the process asks for by name.
## The hook config preamble
Not 3 of 6 hooks as recorded — all FIVE carried their own copy, and of four
lines rather than two. The extra two are a guard treating an unexpanded
`${...}` placeholder as unset, so it is never sent as a garbage Bearer token:
precisely the correctness detail a sixth hook would omit with nothing failing
loudly. Now `scribe_config` in scribe_defs.sh, which also declares the two
names it owns. It sets globals rather than echoing, so a token never passes
through a subshell's output where xtrace or a log could catch it, and returns
a status so a caller can bail (`|| exit 0`) or continue degraded — the
session-context hook still owes its static floor when Scribe is unconfigured.
`check_plugin.py` now runs shellcheck with `-x`. Without it the shared helpers
were invisible: every variable they set read as unassigned and every bug inside
them went unlinted at the call site, which is the opposite of what sharing them
was for. All twelve fail-open scenarios still pass, and all five hooks were
probed live against the instance — prior_art and after_write both still name
canon, autoinject returns context, session_context serves 11k chars of rules,
sync_processes stays silent. Plugin 0.1.46 (#2209).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`retrieval_logs` was write-only. `record_retrieval` inserted rows and nothing
in the tree ever selected from them: the only `select()` over RetrievalLog
lived in a test. So #1038's gate — "build the reranker once telemetry shows
precision is the bottleneck" — was unsatisfiable by construction, and the one
real tuning decision on record (the 0.68 write-path threshold, #2223) had to
be reached by hand-probing the live instance with eight payloads. This adds
the half that was missing.
`retrieval_summary(user_id, days=30)` returns two aggregates side by side,
each read from the table built for it — NOT a join. NoteUsageEvent's docstring
is explicit that the two are complements ("RetrievalLog tunes the threshold,
this tunes the corpus") and that RetrievalLog's JSONB `result_ids` cannot be
indexed at the per-note grain, so correlating through it would be both slower
and less honest than reading each source directly. That corrects the approach
sketched on the task.
- `sources`, per surface: calls, zero_result_calls, cleared_threshold (how
often the best hit beat the threshold in force for THAT call), the
top_score spread as p10/p50/p90/min/max, avg_result_count, p90 duration.
Zero-result calls are counted apart from low-scoring ones — they are a
different failure and averaging them together would hide both.
- `usage`, from note_usage_events: ranked surfacings, ambient surfacings,
and pulls split into `pulled_by_agent` / `pulled_by_human`.
That split is not decoration. NoteUsageEvent's own comment says the mcp_/rest_
prefix is load-bearing and names #1038 while saying so: "is this dead weight?"
is answered by any pull, "was that injected line useful?" only by an agent
pull. `pull_through` exists to answer the second, so it counts agent pulls
over ranked surfacings; both halves ship so the first stays answerable.
Two things the code made me get right rather than guess:
- Distinct-note counts get their own queries. `count(distinct note_id)` per
(event, source) group cannot be summed across groups — a note surfaced by
two sources is one distinct note and would be counted twice. A wrong
number labelled "distinct" is worse than no number.
- No CASE in the GROUP BY. #2663 is the bug where a second case() rendered
its own expanding bind names, Postgres rejected the query, a broad except
swallowed it, and every counter read zero in production while mocked tests
passed. Grouping on raw `source` and classifying in Python cannot fail
that way. For the same reason the readout distinguishes `read_failed` from
an empty window, and its tests are integration against real Postgres —
percentile_cont ... WITHIN GROUP only proves it parses against a database.
Exposed as the `retrieval_telemetry` MCP tool, added to `_READ_ONLY_TOOLS`:
it mutates nothing, but its name carries no read prefix, so the completeness
test cannot derive it and it would otherwise have failed closed for read-only
keys in silence — the same reason `enter_project` is spelled out there. Docs
updated to name both exceptions rather than leave the rule looking derivable.
Scoped to the caller's own telemetry: a retrieval log records what one user's
agent asked for, query text included, and is not a shared record kind — the
owner filter is the whole access rule, not a shortcut past access.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pull-time freshness confirms a cached body by containment: normalised cached
code must appear inside the fetched file. That is right for a record kept
verbatim and permanently wrong for a deliberately annotated one. A record
whose job is to say WHY the shape is what it is carries commentary the source
does not, so containment fails on every pull, forever — #2508 was reading
`diverged` although its declarations match the source exactly, and always
would. Annotation is a sanctioned record style, so this was two deliberate
designs colliding, and it was quietly poisoning the one honest drift signal:
decision #2707's scoreboard watches body_freshness for `diverged` and was
accruing false positives it could never age out.
The escape hatch is the verdict itself. verify_snippet is exactly where a
human or agent already judged this body a faithful rendering of that source,
and `verification.commit_sha` records the repo commit they judged it at — a
field whose own docstring (#2688) anticipated this: "makes 'the REPO moved on
since the check' computable, once the forge integration can compare it against
the current head." This is that comparison. When containment fails, a standing
verdict can still vouch, on four conditions and no fewer:
- the verdict says `ok`;
- it has not EXPIRED — verification_view recomputes code_sha against the
record's current body, so editing the record retires the verdict;
- it was not INVALIDATED by a push touching the location (#2691);
- the file just fetched is at the very commit the verdict was stamped at.
That last one is what keeps it honest: the hatch vouches for a body against
ONE known commit, never against whatever the source became since. The moment
the file moves, containment resumes as the authority and the record reads
`diverged` until someone re-verifies — correct, because at that point nobody
has looked. The first three are checked by reusing verification_view rather
than restating its rule, so "expired" keeps meaning one thing.
Nothing is rewritten and no new freshness value is minted; `verification`
already travels in the same payload, so a reader can see the basis rather
than take "current" on faith. Verdicts predating commit stamping carry no
commit to compare and therefore do not vouch — they fall through to
containment rather than passing on age alone.
Tests pin the fix and, more usefully, every condition that switches it back
off: moved commit, expired verdict, push-invalidated verdict, non-ok verdict,
no verdict, and a pre-#2688 verdict with no commit_sha. Plus a regression
that a verbatim record still takes the containment path untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`.editor-body` was removed from editor-shared.css in c5faaf3 — nothing used
it once TaskEditorView replaced it — so "Replace .editor-body for task
editor" now names something a reader cannot find. Say what .task-body is and
record that its predecessor is gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Integration cover for the seam the unit tests only meet at either end: a
template that names no class at all (`<transition-group name="toast">`) and
one that builds its class (`` `status-${s}` ``) run through the real
extractor, through resolve_consumers, into `flag="unused-css"` — and the
rules they reach stop being reported, while a rule nothing can reach stays
listed. The prefix credits BOTH status rows, which is the documented
reading: the template does not say which one it built.
The manifest bump is what the last commit owed — it edited the
shape-accounting skill, and the installer compares versions to decide
whether to refresh the cache that actually executes (#2209), so plugin
edits that ship without a bump reach the repo and stop there. CI caught it
on 4323; 0.1.44 -> 0.1.45.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Paying down #2962 measured `flag="unused-css"` against a hand audit and
found it had a permanent false-positive floor: roughly sixty classes it
called unused are alive and always would be, because two ordinary authoring
forms produce names no reader of `class=` attributes can see. A flag whose
list you cannot act on line by line is worse than no flag — act on it and
you delete live UI.
- A transition `name=` IS a class reference. `<Transition name="toast">`
makes Vue apply `.toast-enter-active` and its siblings at runtime, and
React's `<CSSTransition classNames="fade">` does the same with a different
suffix set. Every spelling of the tag is read (`Transition`,
`TransitionGroup`, `transition-group`), and the emitted suffix set is the
union of Vue 3, Vue 2 and React: naming a class no rule defines costs
nothing, since it resolves to no row. A bound `:name` stays unknowable.
- A concatenated name contributes its static head as a PREFIX reference.
`` `status-${s}` ``, `'pri-' + p` and `class="card-{{ v }}"` all leave a
head behind once the hole is blanked — and `_CLASS_TOKEN_RE` accepts a
trailing hyphen, so until now the extractor emitted a junk token
`"status-"` that matched nothing. It is now `status-*`, and
resolve_consumers credits every row whose symbol starts with that head,
each under the same own-file-else-fan-out rule as an exact token. `*`
cannot occur in a class token, so the marker rides the existing
dict[str, int] with no schema change. A head shorter than two characters
says nothing and is dropped.
Crediting every candidate row is the honest reading: the template genuinely
does not say which one it built, and the alternative is reporting live rules
as dead. What the map still cannot see is a name assembled in a script —
`classList.add` — which stays deliberately out of scope; the skill, the
`flag=` docstring and the refresh payload docs all say so now.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first sweep asked "does this class token appear anywhere outside a
<style> block?". That is too generous for a `<style scoped>` rule, which can
only ever match its own template, the component's root element, or whatever
`:deep()` reaches — so a scoped rule whose name lives only in some OTHER file
is dead regardless. The server's map had this right and my local pass did
not; this closes the gap.
79 scoped classes are absent from their own file. 48 are names their own file
BUILDS — `status-${task.status}`, `pri-${p}`, `toast--${type}`,
`perm-${permission}`, `diff-${op}`, `is-${kind}` — and 12 more are Vue
transition classes. Those are the map's documented blind spot and they stay.
The remaining 19 match nothing:
- AppHeader (9, -39 lines): the whole connection-status indicator —
.status-indicator/.status-dot/.status-text and the five colour states,
plus @keyframes pulse-dot and status-pulse, which had no other user. Same
Phase 7 residue as the last commit. `.btn-icon.active` goes too: `active`
is never applied in this component (its nav uses router-link-active).
- SettingsView (6): .status-badge/.status-on/.status-off,
.perm-granted/.perm-denied, .location-row. This view renders no child
components at all, so nothing can inherit its scope.
- WorkspaceNoteEditor: .note-row:hover .btn-delete and .btn-suggest-tags —
those buttons are .btn-danger-outline/.btn-ghost now.
- ProjectListView .loading-msg; SnippetEditorView .field-row.three, whose
media-query companion keeps its live .field-row half.
Checked against child-component roots before deleting, since a child's root
element does inherit the parent's scope id — none of the 19 is one. Template
and script regions byte-identical; both style checks still pass with only
the six pre-existing reports.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Milestone 302's map flagged 195 css rows no template names. Re-derived the
list locally with a stricter test than the map's — a class is dead only if
its token appears NOWHERE outside a <style> block, in markup or script, and
is not the static half of a concatenated name (`priority-${p}`) — which
takes the map's known blind spot off the table. 123 survived that; 21 of
those are Vue <Transition>/<transition-group> classes generated at runtime
from a name= attribute (detail-fade, peek-slide, shortcuts-fade, tab-fade,
toast) and one is ProseMirror's vendor class. Those stay. The other 102 go.
- SettingsView.vue (80, -417 lines): whole features whose UI was removed —
Ollama model management (model-*, pull-bar-*, suggestion-chip), push
notifications (push-*), the voice library and voice blending (voice-*,
blend-*), geo status (geo-*), MCP package rows, retention, the learned
summary, and the form furniture that served them.
- editor-shared.css (16, -151): the standalone assist panel, superseded by
the sidebar assist section — streaming now renders as .stream-preview in
the main area, so .assist-panel*, .assist-sections*, .assist-preview-box
and .typing-indicator have no markup left. @keyframes blink went with the
last rule that animated it. .editor-body/.editor-main are dead too:
TaskEditorView replaced them with .task-body/.task-main.
- theme.css: .btn-new-conv/.btn-send dropped from the touch-target list
(names from another app), and the .hide-desktop utility no one used. The
generated --fs-* token block is untouched.
- ShareDialog .user-result-email, KnowledgeView .today-link,
ProjectView .edit-input (the remainder rule keeps its rationale comment
and its two live selectors).
A selector dies only when every class in its descendant chain is checked —
`.live .dead` matches nothing either — and a rule only when its whole comma
list is dead, so mixed lists keep their live half. Template and script
regions are byte-identical; check_dangling_styles.py and
check_design_tokens.py both pass, with only the six pre-existing reports.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/helpers.http_sink replaces three module-local _Sink handlers (the
write-path tests and the after-write test). ProjectView + SettingsView
parsed `(e as {body?:{error?}}).body?.error || fallback` by hand ten times
beside the apiErrorMessage canon (#2853) - all ten now call it. The
extractor (server + the hook awk mirror) no longer reads `import { type Foo }`
as a definition of Foo - that was the last "identical body" sym family.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
KnowledgeView's own comment asked for this promotion once a second view grew the
panel. The sheet carries .dup-panel / .dup-empty,.dup-head / .dup-group /
.dup-members / .dup-member(+:hover) / .dup-score; each view keeps only its
extras (.dup-claimed, .dup-action).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The derive queue said the biggest duplicate families were whole views:
TaskViewerView, UserManagementView and LogsView were imported nowhere — left
behind when tasks moved to the editor and users/logs became SettingsView
tabs. Deleted (rule 22). Note/TaskEditorView carried six identical scoped
rules -> editor-shared.css (the .tag-suggest-row gap the scoped copies
actually rendered wins). Three views wrapped the page under three names ->
.page-container in components.css. Three scoped input recipes -> the design
system fs-input recipe (snippet #2336), verbatim, in components.css; width/
box-sizing stay with the caller. The three rules panes share .pane and the
pane heading via rules-shared.css (the auth-shared pattern, #2852).
Ledger: a single-declaration CSS rule keeps its selector in its fingerprint,
so `color: var(--fs-text-tertiary)` under five different names is no longer
a five-file "identical body" family — the first pay-down found that most of
the 148 dup families were exactly this, and nobody would consolidate them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Run 4240: two pre-write hook tests pinned the skip case and scribe_defs()
inside scribe_prior_art.sh, which moved to the shared library; the after-write
test saw the same derive key appended once per changed file. Keep each token
once (sort -u after the appends) and point the pins at the library.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Edits made through sed/heredocs/scripts never reached the PreToolUse
Write|Edit hook, so a whole class of writes got no prior-art hint, no
ledger feed and no duplicate-family warning. scribe_after_write.sh asks git
what changed since it last looked (per-session path+blob snapshot; first
call = files touched in the last minute), extracts the definitions in the
added lines and calls /api/plugin/prior-art with the same three dedup
channels the pre hook keeps. Never blocks; silent on any failure. The
extractor, the prose/data skip list and the local by-name arm move to
scribe_defs.sh, sourced by both hooks. Version bump covers the step-2 hook
change too (run 4239 failed only on the bump check).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shape_ledger.write_time_derive asks the ledger what it knows about each
named (kind, symbol): a derive-grouped family (identical body / same name
in N other files) -> "derive it now, do not add a copy"; a canonical row
at another path -> "canon #N at <path>, reuse". Judged rows at the path and
the canon own file stay silent. Rendered by _derive_line beside the
divergence line; keyed (group id / canon:<id>) on a third per-session dedup
channel in the hook (.derive.ids -> exclude_derive=).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Since the scoped bucket (#2869) the ledger reads 100% accounted while 439
derive rows stand; the standing block was gated on unclassified > 0 and so
went silent. Build it whatever the todo count ("; standing: ..."), and add
derive_new — derive-grouped rows first seen after the previous refresh
stamp — so entering a project names the drift ("+2 new copies since last
refresh: .error-msg in InceptionCard.vue") instead of waiting for an audit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- components/InceptionCard.vue: the one form, two homes — mode="create" in
the New-project modal's second step (emits the choices; the create carries
`inception`), mode="decide" on ProjectView for the owner of an undecided
project (loads that project's defaults, records the decision). Always-on
rulebooks listed checked (uncheck = exclude), others unchecked (check =
subscribe), design system select, seed-Systems toggle (disabled once the
project has Systems). Tokens only; modal canon (#2855); .btn-* canon.
- ProjectView: the card while undecided, one "Inheritance decided <date> via
… · …" line after; onDecided refreshes the project.
- ProjectRulesTab: "Excluded always-on rulebooks" section with include-back.
- api/inception.ts (types, fetchInceptionDefaults, decideInception);
api/rulebooks.ts: ApplicableRules.excluded_always_on, exclude/include
wrappers; REST POST/DELETE /api/projects/<id>/exclusions/rulebooks/<rb>.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- MCP create_project(..., exclude_always_on_rulebooks, subscribe_rulebooks,
design_system_id (0 unstated / -1 none / n), seed_systems): any inception
arg → inception.decide(via="mcp") after the create; none → undecided with
an inception_hint. New decide_project_inception(project_id, …) records or
re-records; nothing given = an inherit-all decision, stated.
- enter_project carries `inception` ONLY for the caller's own, undecided
project: inception_ask() = the project's current defaults + what to ask the
operator once + the exact call (the #2683 ask shape). Absent otherwise.
- REST: POST /api/projects accepts `inception` (validated before the create);
POST /api/projects/<id>/inception decides/re-decides; GET …/inception/defaults
is the card's payload; GET project already carries inception via to_dict.
- _INSTRUCTIONS: ORIENT names the ask; START a project names the questions —
never create a project bare by default (product behaviour, P#119).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- inception.decide(user, project, choices=, via=): owner-only; validates the
choices (pure) and every target (owned rulebook / always-on for an
exclusion / readable design system) BEFORE any effect; then, each
idempotent: exclude always-on rulebooks, subscribe rulebooks, point the
design system (None = explicitly none), seed the standard Systems if asked
and the project has none; writes projects.inception LAST. Re-deciding is
additive for exclusions/subscriptions, replaces the design system, never
re-seeds.
- inception.current_defaults(): what binds if nobody decides — the ask's
payload (always-on / other rulebooks, standing exclusions + subscriptions,
design system + the choices, Systems count).
- services/systems.STANDARD_SYSTEMS (name + generic charter) + seed_standard_systems();
mcp/tools/systems names the same list in the bootstrap ask — one vocabulary.
- Integration tests: effects land and the record says why; bad targets apply
nothing; outsiders cannot decide.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A project that opted out of an always-on rulebook at inception must not see
it anywhere: list_always_on_rules(project_id=) and the subscription-derived
set skip it (an exclusion is total), get_applicable_rules / rules_payload
carry `excluded_always_on` as the seventh key so the departure is visible
wherever the rules are, and the SessionStart block built for a bound project
names it ("Excluded for this project by its inception decision …"). MCP:
list_always_on_rules takes project_id; exclude_always_on_rulebook /
include_always_on_rulebook mirror suppress/unsuppress (owner-only; the
rulebook must be always_on — subscribed rulebooks are left by unsubscribing).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A project's inheritance becomes a decision, not a default (milestone 297).
- projects.inception (JSONB, NULL = undecided): {decided_at, decided_by, via
mcp|ui|legacy, choices {exclude_always_on_rulebooks, subscribe_rulebooks,
design_system_id, seed_systems}}; on to_dict.
- project_rulebook_exclusions: a project's opt-out of a whole always-on
rulebook — the sibling of the rule/topic suppressions, CASCADE both ways.
- services/inception.py (first cut): the vocabulary, validate_inception
(pure, all-or-nothing), normalize_choices, is_decided. Effects come in
step 3.
- Migration 0085 backfills every existing project via="legacy" with its
current standing (no exclusions, its subscriptions, its design_system_id,
no seed) so the ask fires only for projects created after this ships.
- Backup v10: rulebook_exclusions section; project rows carry inception and
design_system_id, restored in a post-pass once rulebooks/design systems are
mapped (design_system_id was not restored before — fixed in passing).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 21:57:02 -04:00
770 changed files with 109175 additions and 24289 deletions
A self-hosted work system-of-record for software projects, built to be driven by Claude Code. Notes, tasks, issues, projects, milestones, rules, and stored processes — reachable from Claude via a built-in MCP endpoint and a bundled Claude Code plugin, with a clean web UI for humans. No in-app LLM; Claude is the sole assistant.
<h1 align="center">Fabled Grimoire</h1>
<p align="center">A self-hosted system of record for the work you do with Claude Code.</p>
Claude records what it learned, decided and built in Grimoire, and that record reaches the next session, so a solution is recalled instead of reinvented. Notes, tasks, plans, rules, lessons and reusable code live in one place. Claude reaches them through a built-in MCP endpoint and the bundled Claude Code plugin, and people read and edit the same records in the web UI. There is no language model inside Grimoire: Claude is the only assistant.
## Deploy
You need Docker with Compose. Save this as `docker-compose.yml` (it is the [`docker-compose.yml`](docker-compose.yml) in this repository), then run `docker compose up -d` beside it:
```yaml
# Fabled Grimoire: the app and its database.
# docker compose up -d
# Then open http://localhost:5000. The first account registered becomes admin.
#
# Keep both volumes. app_data holds the instance's own SECRET_KEY, which seals
# Postgres 17 with the vector extension Grimoire needs.
image:pgvector/pgvector:pg17
stop_grace_period:120s
volumes:
- pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER:${POSTGRES_USER:-grimoire}
POSTGRES_PASSWORD:${POSTGRES_PASSWORD:-grimoire}
POSTGRES_DB:${POSTGRES_DB:-grimoire}
restart:unless-stopped
volumes:
app_data:
pgdata:
```
Open `http://localhost:5000`. The first account you register becomes the admin.
Nothing has to be set first. On its first start Grimoire makes its own `SECRET_KEY` and keeps it in the `app_data` volume, so restarts, reboots and updates all use the same key.
To connect Claude, create an API key under **Settings → MCP Access**, then in Claude Code:
The plugin asks for your Grimoire URL and the API key. Settings → MCP Access shows the exact commands for your instance.
## Configuration
Everything is optional. Set these in the environment or in a `.env` file beside the compose file:
| Variable | When | What it does |
|----------|------|--------------|
| `BASE_URL` | Behind a proxy | The public URL, used in email links and SSO redirects |
| `SECURE_COOKIES` | Over HTTPS | `true` when Grimoire is served over HTTPS |
| `TRUST_PROXY_HEADERS` | Behind a proxy | `true` to read the client address from the proxy's headers |
| `POSTGRES_PASSWORD` | Before the first start | The database password (defaults to `grimoire`; the database is reachable only inside the compose network). Postgres takes it once, when the volume is created |
| `SECRET_KEY` | Rarely | A key you manage yourself, instead of the one Grimoire keeps. It wins over the kept key; remove it to go back |
Everything else (registration, email, SSO, git forge connections, retrieval tuning, backups) is in **Settings**, or listed in [Configuration](docs/configuration.md).
## Setup pitfalls
- **Keep both volumes.** `pgdata` holds everything Grimoire stores. `app_data` holds the instance's key, and forge tokens and the backup bucket's secret are sealed under it. If the key is lost, Settings asks you to enter those credentials again; nothing else is lost.
- **The embedding model downloads on first use**, into the `app_data` volume. Without internet access, semantic search falls back to keyword search until the model is in place.
- **Behind a reverse proxy, turn off response buffering** for `/mcp`, because MCP responses stream. [Configuration](docs/configuration.md#behind-a-reverse-proxy) has an Nginx example.
## Updating and backups
```bash
docker compose pull
docker compose up -d
```
Database migrations run when the app starts, and the key stays in its volume. Claude Code updates the plugin from its marketplace; run `/reload-plugins` after an update.
**Settings → Data → Scheduled backups** writes a daily backup of the database to a folder or an S3-compatible bucket, and `python -m grimoire.restore` brings a fresh instance back from one. See [Backups](docs/backups.md).
## Features
Notes and tasks with a Markdown editor, sub-tasks, milestones, issues, and kanban project workspaces. Stored processes, an engineering rulebook system, and semantic search with proactive knowledge-injection into Claude's context. A knowledge graph, per-user/group sharing, and a built-in MCP server (`/mcp`) plus a bundled Claude Code plugin so Claude can record and recall your work directly.
**Working with Claude**
- Claude enters a project and gets a short primer: its goal, current plans, open tasks and areas.
- Your rules reach Claude when its work matches them, or at a moment of work such as finishing a task or pushing code.
- Before Claude writes a file or runs a command, the plugin shows it the rules, lessons and reusable code that may apply.
- Stored procedures, such as a drift audit, become Claude Code skills.
## Quick Start
**Recording the work**
- Notes in a Markdown editor, with wikilinks, tags and version history.
- Tasks, issues and time-boxed investigations, with work logs that the next session reads first.
- Plans as milestones: a design plus ordered steps.
- Lessons learned the hard way, linked to the rules they illustrate.
- Duplicate checks, so a new record that matches an existing one updates it instead.
**Prerequisites:** Docker and Docker Compose. No GPU or local model needed — Claude is the sole assistant, reached over MCP.
**Reusing code**
- Snippets of reusable code, each with when to reach for it and where it lives in the repo.
- With a git forge connected, Grimoire tracks every function and style in your repos and shows which ones are still unaccounted for.
- Ideas shared across projects on the same platform, with each project's answer recorded.
- Design systems with tokens, inheritance and a generated stylesheet.
Download [`docker-compose.quickstart.yml`](docker-compose.quickstart.yml) from this repo, then:
**For people**
- A web UI to browse, search and edit everything, plus a graph of how notes connect.
- Sharing with users and groups, as viewer, editor or admin.
- Email and password sign-in, optional SSO through OIDC, and invite links.
- Personal export to Markdown or JSON, and admin backup and restore.
The full list is in [Features](docs/features.md).
## Troubleshooting
| What you see | Why | Fix |
|--------------|-----|-----|
| `SECRET_KEY is set to the public placeholder` at startup | `SECURE_COOKIES=true` while `SECRET_KEY` is `dev-secret-change-me` | Remove `SECRET_KEY`, or set a real one |
| Settings asks for forge tokens or the backup secret again | The instance's key changed: the `app_data` volume was replaced, or `SECRET_KEY` was set or removed | Enter them again, or restore the previous key |
| `Can't locate revision identified by '…'` at startup | An older image is running against a database a newer one has already migrated | Run the newer image; migrations only move forward |
| The plugin's tools are missing in Claude Code | The plugin isn't installed, or Claude Code hasn't reloaded it | `/plugin` to install or update it, then `/reload-plugins` |
| Search finds only exact words | The embedding model hasn't downloaded yet | Give the app internet access once, or wait for the first download to finish |
## Development
```bash
# Optional but recommended — set a secret key
exportSECRET_KEY=your-random-secret-here
docker compose -f docker-compose.quickstart.yml up -d
docker compose -f docker-compose.dev.yml up --build
```
Open `http://localhost:5000`. The first user to register becomes admin. To connect Claude, create an API key under **Settings → API Keys** and install the Claude Code plugin — see [API Keys & MCP](docs/api-keys-and-mcp.md).
[Development](docs/development.md) covers checks, migrations, the plugin and CI. [Architecture](docs/architecture.md) describes how it is built, and the [API reference](docs/api-reference.md) lists every REST endpoint.
> **Development:** To build from source, see [Development](docs/development.md).
| [Features](docs/features.md) | Detailed feature breakdown and keyboard shortcuts |
| [Development](docs/development.md) | Dev workflow, CI/CD, migrations, release process |
| [API Keys & MCP](docs/api-keys-and-mcp.md) | API key management and Fable MCP install guide |
| [SSO / OAuth](docs/sso-oauth.md) | OIDC setup for Authentik, Keycloak, and other providers |
| [API Reference](docs/api-reference.md) | All REST API endpoints |
## License
This project is privately maintained.
Fabled Grimoire is free software under the [GNU Affero General Public License v3.0 only](LICENSE) (`AGPL-3.0-only`). If you run a modified version for other people over a network, the licence requires you to offer them its source.
A Python MCP (Model Context Protocol) server that lets Claude directly interface with a running Fabled Assistant instance. The goal is two-fold: Claude's stronger reasoning handles planning, documentation, and analysis while Fable remains the authoritative store for notes, tasks, projects, and milestones; and direct API access enables process improvement by letting Claude observe and operate on real data rather than working from descriptions.
The work is split into two sub-projects:
1.**Fable API Key Feature** — additions to the main `scribe` project to support bearer token authentication
2.**Fable MCP Server** — a new standalone Python package at `fable-mcp/` in the same repo root
A third sub-project (Forgejo MCP for CI/CD automation) is planned as a follow-on after the Fable MCP is working.
---
## Sub-project 1: Fable API Key Feature
### Motivation
All existing Fable routes use session-based authentication (`session["user_id"]`). The MCP server runs as a local process and cannot maintain a browser session, so a stateless bearer token mechanism is required. API keys are user-scoped and carry a read/write permission level.
### Database
New table `api_keys` via migration `0027_add_api_keys.py`:
-`down_revision = "0026"` (the briefing tables migration)
Full keys are generated as `fmcp_<32 random url-safe chars>`, returned once at creation, never stored. Only the SHA-256 hash is persisted.
### Auth Middleware (`auth.py`)
`_check_auth` is updated to check the `Authorization: Bearer <token>` header before falling back to session auth:
1. If header present: hash the token, look up in `api_keys` where `revoked_at IS NULL`
2. If found: update `last_used_at`, set `g.user` and `g.api_key`
3. Scope enforcement: if `g.api_key.scope == "read"` and `request.method` is not `GET`, return 403
4. If header absent: existing session logic runs unchanged
Session auth is untouched — no regression risk for the web UI.
**Admin routes and API keys:** Routes decorated with `admin_required` check `user.role == "admin"` after auth resolves. API keys authenticate as the key owner — so a write-scoped key for a non-admin user will fail admin-protected routes with 403 (role check, not scope check). This is intentional: API keys cannot elevate privilege beyond the user's role.
### Routes (`/api/api-keys`)
New blueprint `api_keys_bp`, registered in `app.py`:
-`GET /api/api-keys` — list caller's keys (prefix, name, scope, last_used_at, created_at — never the hash or full key)
-`POST /api/api-keys` — create key; body: `{name, scope}`. Returns full key in response **once only**
-`DELETE /api/api-keys/:id` — revoke by setting `revoked_at = now()`
### Settings UI
New "API Keys" tab in `SettingsView.vue` (added to `VALID_TABS`):
Calls the existing `semantic_search_notes()` service (already in `services/embeddings.py`). The `content_type` parameter maps to the service's `is_task` argument:
This endpoint is needed by the MCP's `search.py` tool module. It reuses existing infrastructure with no new ML work.
### Conversation Type: `"mcp"`
A new conversation type `"mcp"` is added alongside `"chat"` and `"briefing"`. This requires changes in two places in the main app:
**`services/chat.py`** — `create_conversation(user_id, title, model)` gains an optional `conversation_type: str = "chat"` parameter, passed through to the `Conversation` constructor.
**`routes/chat.py`** — `POST /api/chat/conversations` accepts an optional `conversation_type` body field (defaults to `"chat"`), passed to `create_conversation`. `GET /api/chat/conversations` continues to default-filter to `conversation_type="chat"` (excluding `"mcp"` and `"briefing"`); pass `?type=mcp` to retrieve MCP conversations.
**Retention:**`cleanup_old_conversations` in `routes/chat.py` currently deletes conversations regardless of type. It must be updated to exclude `conversation_type="mcp"` from the sweep, so MCP audit-trail conversations are not automatically pruned.
MCP conversations:
- Are created with a caller-supplied name (e.g., `"MCP Session 2026-03-23"`) or auto-named
- Are excluded from the default chat list
- Are accessible via `GET /api/chat/conversations?type=mcp` for inspection
- Appear in the Fable UI if explicitly navigated to, providing an audit trail of MCP-driven interactions
### Chat SSE Wire Format
The MCP's `chat.py` tool must consume the existing SSE stream. The relevant endpoints and event schema:
-`{"type": "done", "content": "...", "tools_used": [...]}` — generation complete; this is the termination event
The MCP tool reads tokens until it receives `type: "done"`, then returns `response` (full content) and `tools_used` (list of tool names).
---
## Sub-project 2: Fable MCP Server
### Location
`fable-mcp/` at the repository root, alongside `src/`, `frontend/`, `alembic/`. It is **not** part of the main Docker build and has no import relationship with `scribe`. It will be extracted to its own Forgejo repo once stable.
`FableClient` is an async context manager wrapping `httpx.AsyncClient`. It reads `FABLE_URL` and `FABLE_API_KEY` from environment variables (with `python-dotenv` fallback to `.env`). All methods are `async def` and return parsed dicts. A `FableAPIError(status_code, message)` exception is raised for any non-2xx response.
A module-level singleton `_client: FableClient` is initialized at server startup and shared across all tool modules.
### `server.py`
Uses `mcp[cli]`'s `FastMCP` class. Imports all tool modules, which register their tools via decorators against the shared `FastMCP` instance. Entry point `main()` calls `mcp.run()` (stdio transport).
### Tool Modules
Each module imports `_client` from `client.py` and defines tools as `async def` functions decorated with `@mcp.tool()`. Tool docstrings serve as the MCP tool descriptions visible to Claude.
- MCP conversations are excluded from the normal chat UI list
### Error Handling
`FableAPIError` is caught at each tool boundary and returned as a descriptive string rather than propagating as an exception. This ensures Claude receives a readable error message (e.g., `"Fable API error 403: Read-only key cannot perform write operations."`) rather than a stack trace. Network errors (`httpx.RequestError`) are similarly caught and surfaced as strings.
Scope enforcement lives entirely in Fable's auth middleware — the MCP does not duplicate it.
### Claude Code Registration
After `pip install -e fable-mcp/` (or `uv tool install ./fable-mcp`), add to `~/.claude/settings.json`:
```json
{
"mcpServers":{
"fable":{
"command":"fable-mcp",
"env":{
"FABLE_URL":"http://localhost:8080",
"FABLE_API_KEY":"fmcp_your_key_here"
}
}
}
}
```
Claude Code spawns the process over stdio automatically. No Docker, no daemon.
---
## Build & Repo Plan
1. Implement and test within `scribe/fable-mcp/`
2. Once stable, extract to a new Forgejo repo (`bvandeusen/fable-mcp`)
3. Forgejo MCP (Gitea MCP) added as a second MCP server to automate build/push/config workflows — separate spec when ready
---
## Out of Scope (v1)
- MCP resources (browsable URI tree) — tools cover all use cases in Claude Code
- Forgejo MCP — follow-on spec
- Rate limiting on API key endpoints
- Key expiry / TTL
- Per-resource scope (e.g., key restricted to one project)
- In `startup()`: `asyncio.create_task(load_stt_model())` + `asyncio.create_task(load_tts_model())` when `VOICE_ENABLED`
### `src/scribe/config.py`
- Add 4 new env var attributes
- Add validation in `validate()`
### `src/scribe/services/llm.py`
- Add `voice_mode: bool = False` and `voice_speech_style: str = "conversational"` to `build_context()`
- When `voice_mode=True`, prepend: *"Respond naturally as if speaking aloud. No markdown, bullet points, headers, or code blocks. Complete sentences only."*
- Append style modifier based on `voice_speech_style`
### `src/scribe/services/generation_task.py`
- Add `voice_mode: bool = False` to `run_generation()`
- Read `voice_speech_style` from settings when voice_mode; pass both to `build_context()`
### `src/scribe/routes/chat.py`
- Allow `"voice"` in `conversation_type` whitelist
### `src/scribe/services/chat.py`
- Exclude `conversation_type == "voice"` from auto-cleanup retention
Floating PTT button (fixed bottom-right). Creates/reuses a `"voice"` conversation. Full flow: record → transcribe → send → stream → synthesise → play. Space bar hotkey (from `App.vue`). Mounted globally in `App.vue`.
The current briefing pipeline hallucinates calendar events, tasks, and news items that do not exist in the user's actual data. Observed examples from production:
- A morning briefing asserting "your dentist appointment is still in progress" when no such event existed
- An 8am check-in mentioning "a quick meeting at 10:30 AM" with no backing calendar entry
- A midday check-in inventing "a team huddle at 2:30 PM" and "the Q2 budget draft due by Friday"
- The model calling `search_images` in response to a clarifying question about a fabricated meeting
These are not bugs in data retrieval — the data gathering code (`_gather_internal`, `_gather_external`) returns correct values. They are a structural consequence of how the briefing context is assembled.
## Root cause — the "no receipt" problem
The current `run_compilation` pipeline does this:
1. Python code gathers data (tasks, events, weather, news) from the database
2. Python formats the data into a structured text blob as the user-role message: `TODAY'S EVENTS: ...`, `DUE TODAY: ...`, etc.
3. The LLM is called **once** with `[system_prompt, user_prompt]` and produces prose
4. Only the prose reply is written to the conversation. **The underlying data is never persisted in the conversation history.**
When the user later chats in the briefing conversation, the chat endpoint loads the full conversation history — which contains the model's *prose* from earlier but not the data that prose was derived from. The model's own prior output becomes the only source of "truth" available for follow-up questions. If that prose asserted a fact (real or hallucinated), the model has no way to distinguish it from ground truth when generating the next reply, and it will double down.
Compounding factors:
- **Empty sections are silently omitted.** If `calendar_events` is empty, the user prompt contains no `TODAY'S EVENTS:` line at all. The model has no explicit "zero events" signal — combined with an imperative system prompt ("note calendar events and tasks"), it interprets the silence as "I should mention some" and fabricates.
- **Scheduled slot injections append synthetic turns.** `run_slot_injection` writes a fake `[Midday briefing update]` user message and the assistant reply into the persistent conversation. By evening, the chat history contains three separate briefings, each potentially with errors, all treated as equal-weight context on follow-up.
- **The `search_images` tool is available during briefing chat**, with a negative-instruction description ("Not for factual questions"). Small and mid-sized models frequently ignore negative guidance in tool descriptions and call the tool anyway.
## Solution — agentic briefing (the receipt model)
Replace the one-shot synthesis with a tool-call loop. The briefing is no longer "a text blob synthesized from pre-gathered data." It is **a scheduled agent run**: the LLM is given a system prompt, a curated set of read-only data tools, and a trigger ("generate the morning briefing"). The model must call tools to see what exists. Every tool call and tool result becomes part of the conversation history, where it lives as a permanent, structured receipt.
### Why this fixes the hallucination
When the model calls `list_events(today) → []`, that empty array is now a persistent message in the conversation. On a follow-up question like "what meeting?", the chat endpoint loads the conversation and the model sees its own tool result from the morning showing no events. Answering "you had a meeting at 10:30" would require the model to directly contradict a tool result sitting two messages back — something LLMs are much more reliable at avoiding than contradicting their own prior prose.
In short: **the model cannot fabricate what has a visible receipt proving it does not exist.**
### What the model sees — before vs after
**Before** — the model sees one structured blob and produces prose. The blob is discarded.
```
[system] You are a personal assistant... weave together what matters...
[user] Date: 2026-04-10
WEATHER: Home — partly cloudy, 8–16°C
DUE TODAY: Fix briefing hallucinations
OVERDUE (2 tasks): ...
[assistant] Morning! Looks like a quiet day with two overdue items...
```
**After** — the model is given tools and iterates to ground truth.
```
[system] You are the user's personal assistant giving their morning briefing.
Use tools to see what's relevant. Only mention things you learned from
tool calls. If a tool returns nothing, say so honestly.
[assistant] Morning! Partly cloudy today, 8 to 16 — nothing on the calendar
so it's a clean run at the desk. Two things to keep in mind...
```
The conversation now contains verifiable receipts: `list_events(today)` returned `[]`, and that result sits in context forever (until pruned). Follow-up questions operate against those receipts.
## Architecture
### Existing infrastructure (reused, not rebuilt)
The codebase already has the agentic primitives — they're used for regular chat:
- **`llm.py::stream_chat_with_tools`** — the streaming tool-use loop that talks to Ollama with a `tools` parameter and yields tool-call chunks
- **`generation_task.py::_stream_with_retry`** — wraps `stream_chat_with_tools` with retry-on-500 behavior for cold-model races
- **`tools.py`** — defines 40+ tool schemas and an execution dispatcher
The briefing bypasses all of this and calls a one-shot `_llm_synthesise` helper. The refactor is mainly "route briefings through the same pipeline regular chat already uses."
### New modules
**`briefing_tools.py`** — a small wrapper exposing a curated read-only subset of `tools.py` for briefing runs. This is an **explicit allowlist**, not a blocklist, so newly-added tools must be opted in:
| Tool | Purpose |
|---|---|
| `list_tasks` (with filter args) | See what's actionable today, overdue, high priority |
| `list_events` (today / upcoming) | Know what's on the calendar |
| `get_weather` | Current/forecast weather |
| `get_rss_items` | Pull news themes filtered by user preferences |
| `list_projects` | Understand project context |
| `search_projects` | Surface active project summaries |
| `list_notes` (recent) | Capture follow-ups from yesterday |
**Explicitly omitted** from the briefing tool set:
-`search_images`, `search_web`, `research_topic`, `read_article` — external search is not a briefing concern, and `search_images` is the source of the "Peter Kyle Science Secretary" image-search bug
- All `create_*`, `update_*`, `delete_*` — briefings are read-only; a scheduled background job must not decide to mutate the user's data on its own
-`set_rag_scope`, `calculate` — not relevant to briefing content
### New briefing function
**`briefing_pipeline.py::run_agentic_briefing(user_id, slot, model, conversation_id)`** replaces `run_compilation`'s body (and eventually `run_slot_injection`). Internally:
1. Build a slot-specific system prompt (see below)
2. Load the curated briefing tools from `briefing_tools.py`
3. Seed messages with `[system, user]` where the user message is a simple trigger like `"Generate the morning briefing."`
4. Enter a tool-call loop (max 8 iterations):
- Call `stream_chat_with_tools`
- If the model returns `tool_calls`, execute them via the existing dispatcher, append tool results, continue
- If the model returns a final assistant message with no pending tool calls, break
The full message list is important: it's written to the conversation along with the final prose, so the tool-call receipts become part of the persistent record.
### Slot-specific system prompts
Compilation (full morning briefing):
```
You are the user's personal assistant giving their full morning briefing.
Use the tools available to see what's actually relevant today — tasks due,
overdue items, events on the calendar, weather, news themes, project state
— and weave it into a warm, natural-sounding summary.
Rules:
- Call tools to see the data. Never assert facts you didn't learn from a tool.
- If a tool returns nothing (no events today, no overdue tasks), say so
honestly. Don't fabricate items to fill space.
- Write flowing prose. No markdown, no headers, no bullet points.
- Aim for 6–10 sentences. Skip topics that have nothing interesting.
- Close on one or two concrete, actionable suggestions.
User profile (for tone and preferences):
{profile_body}
```
Check-ins (midday, afternoon):
```
You are the user's personal assistant giving a brief {slot} check-in.
Use tools to see what's changed since this morning. Focus on progress
and what's still unaddressed.
Rules:
- Call tools to see current state. Never assert facts without tool results.
- If nothing meaningful has changed, say so briefly — don't invent progress.
- 3–5 sentences, natural prose, no markdown.
```
### Conversation hygiene — removing the fake user messages
The current scheduler appends two fake messages on every slot injection:
Under the agentic model, we drop the fake `"user"` message entirely. Slot updates become plain assistant messages tagged with `metadata.briefing_slot = "midday"`. The chat endpoint's message loader is updated to filter these when building the LLM context on follow-ups, so a user chatting in the briefing conversation doesn't see three earlier briefings smashed into their history. The UI continues to show them as visible timeline entries.
(This is the Option A filter from the earlier discussion — a small, surgical change compared to migrating briefings to a separate sidecar table. Sidecar storage remains a possible future step.)
### Ollama setup compatibility
The existing Ollama deployment uses non-parallel mode across two GPUs, with a concern about context duplication between cards. This is the correct setup for agentic briefings:
- Each tool-call iteration shares the same KV cache on the same GPU, so appending tool results is cheap
- There is no race where a second iteration could land on a different card with a different cache state
- The trade-off is serialization: a briefing in progress will block a concurrent user chat request until it finishes, but scheduled briefings are rare enough (4× per day) that this is acceptable
If GPU contention becomes a problem later, the right lever is pinning specific models to specific GPUs (e.g., background tasks on GPU 1, interactive models on GPU 0) — not enabling parallelism.
## Cost & trade-offs
- **Latency:** a briefing now makes 5–7 inference calls (one per tool-call decision plus the final prose) instead of 1. On a local Ollama with a 7B model, expect 15–40s per briefing vs the current 5–10s. Acceptable for a background job; if it becomes painful at the user-facing check-in slots, investigate letting the model batch independent tool calls in a single turn (Ollama supports this).
- **Model requirements:** the default model must be reliable at tool calling. `qwen2.5:7b`, `llama3.1:8b`, `mistral-small-3:24b`, and similar handle it well. Models in the ≤3B class typically fail — they either emit no tool calls and return empty prose, or hallucinate invalid tool arguments. If a user's `default_model` is too small, agentic mode should fall back to legacy mode with a log warning.
- **Context growth:** tool results bloat the conversation. At 8 tool calls per compilation × 4 slots per day, a daily briefing conversation can reach 20-30 KB of JSON-heavy history. Fine for a day; aged briefings should be archived/rolled up after 7 days.
- **Tool subset drift:** someone adds a new mutating tool to `tools.py` and forgets to update the briefing allowlist. Mitigated by the allowlist model — the default for new tools is "not exposed to briefings."
- **Infinite loop safety:** a buggy model could tool-call forever. Hard cap at 8 iterations, log a warning if hit, return whatever prose was last produced (or a fallback message).
## Migration path
Ship incrementally, each step independently reversible:
**PR 1 — Agentic compilation behind a feature flag.** Add `briefing_tools.py`, add `run_agentic_briefing`, add a per-user setting `briefing_mode: "legacy" | "agentic"` (default legacy). Route only the 4am compilation through the new path when the flag is set. Keep slot-injection on the legacy path. Enable the flag on the author's account first, validate output quality over several days, then flip the default for all users. No DB migration required — the setting lives in the existing `settings` table.
**PR 2 — Agentic slot injections + conversation hygiene.** Migrate midday/afternoon check-ins to the same pipeline. Remove the fake `[Midday briefing update]` user-role message; slot updates become plain assistant messages tagged with `metadata.briefing_slot`. Add a chat-message-loader filter that excludes slot-tagged messages from the LLM context on follow-ups (they remain visible in the UI).
**PR 3 — UI polish.** Collapsed tool-call status row in the briefing card ("✓ checked calendar · ✓ looked at tasks · ✓ pulled weather"), expanding to show tool results on click. Tool-result cards (weather, news, task list) rendered inline where useful.
**PR 4 (optional) — Sidecar storage for briefing snapshots.** If the chat-filter approach in PR 2 feels too hacky, migrate briefings out of the conversations table and into a dedicated `briefing_snapshots` table. Frontend renders them as pinned timeline cards separate from chat. Larger refactor; defer until PR 1–3 prove the approach works.
## Secondary win — tightening the main chat system prompt
The regular chat is already agentic (it uses `stream_chat_with_tools`), but its system prompt does not explicitly require the model to ground factual claims in tool results. The prompt discipline introduced for briefings — *"Never assert facts you didn't learn from a tool. If a tool returns nothing, say so honestly."* — is worth applying to the main chat system prompt in a follow-up PR. The mechanism already works; only the framing needs tightening.
## Out of scope
- The Android Flutter client's failure to render `search_images` tool-result cards. This is a separate rendering gap in the mobile app and does not affect the server-side fix. Tracked separately.
- Re-evaluating the Ollama parallelism setting. The current non-parallel config is correct for this work.
- Replacing the background model for title/summary/observation extraction. That model's role is unrelated to briefings.
API keys let external tools access your Fable data without a browser session. Each key is scoped to a single user — it can only access data that user owns or has been shared with them.
API keys let external tools access your Grimoire data without a browser session. Each key is scoped to a single user — it can only access data that user owns or has been shared with them.
### Scopes
@@ -15,38 +15,51 @@ Admin-level operations (log access, user management) require a `write`-scoped ke
### Creating a Key
1. Go to **Settings → API Keys**
1. Go to **Settings → MCP Access**
2. Enter a name (e.g. "Claude MCP", "Home Server")
3. Choose scope
4. Click **Generate Key**
5. Copy the key immediately — it is shown only once (the token is `fmcp_`-prefixed)
Paste the key into the `Authorization: Bearer <key>` header of your MCP client
config (see **Scribe MCP Server** below).
config (see **Grimoire MCP Server** below).
### Revoking a Key
Click **Revoke** next to the key in the API Keys table and confirm. Revoked keys are deleted immediately.
Click **Revoke** next to the key in the keys table and confirm. Revoked keys are deleted immediately.
---
## Scribe MCP Server
## Grimoire MCP Server
Scribe exposes itself as a set of MCP tools that Claude (and other MCP clients)
Grimoire exposes itself as a set of MCP tools that Claude (and other MCP clients)
can use to read and write your notes, tasks, projects, rulebooks, and more. The
server is **built into the app** — it is mounted as a streamable-HTTP endpoint
at **`/mcp`** on the running Scribe instance (`src/scribe/mcp/server.py`). There
at **`/mcp`** on the running Grimoire instance (`src/grimoire/mcp/server.py`). There
is nothing to install: no wheel, no separate package, no CLI. You connect a
client straight to the URL with a Bearer token.
### Authentication
Authenticate with an API key generated from **Settings → API Keys** (see above),
Authenticate with an API key generated from **Settings → MCP Access** (see above),
sent as `Authorization: Bearer fmcp_<key>`. A `read`-scoped key may call only the
read tools (`get_*`, `list_*`, `search`, `enter_project`); any write/delete tool
is rejected with `403`. A `write`-scoped key may call everything.
any write/delete tool is rejected with `403`. The allow-list is explicit rather
than derived from the name — see `_READ_ONLY_TOOLS`, which is why the two reads
without a read-shaped name are spelled out here. A `write`-scoped key may call everything.
### Claude Code (Project-scoped)
### Claude Code: the plugin (recommended)
The plugin connects the MCP server and adds the hooks and skills that put Grimoire's records in front of Claude as it works. Install it from this repo:
It asks for your Grimoire base URL, an API key (kept in your OS keychain) and, optionally, a project id. Settings → MCP Access shows the exact command for your instance. [`plugin/README.md`](../plugin/README.md) describes each hook.
### Claude Code without the plugin
Add a `.mcp.json` at the project root. The server `type` is `http` and the URL is
your instance's `/mcp` endpoint:
@@ -54,9 +67,9 @@ your instance's `/mcp` endpoint:
| POST | `/api/projects/:id/inception` | Record what the project inherits `{choices: {design_system_id, seed_systems}}` (owner-only; `POST /api/projects` accepts the same under `inception`) |
| GET | `/api/projects/:id/inception/defaults` | What binds if nobody decides — the inception card's payload |
| GET | `/api/projects/:id/notes` | Notes + tasks in this project |
| GET / POST | `/api/projects/:id/milestones` | List / create milestones |
Grimoire is one app container and one PostgreSQL container. There is no language model in the stack: Claude works with Grimoire from outside, through the MCP endpoint and the Claude Code plugin, and people use the web UI.
## Stack
| Layer | Technology | Notes |
|-------|-----------|-------|
| Frontend | Vue 3 + TypeScript + Vite + Pinia + Vue Router | SPA served from the same container as the API |
Browser ──────── Vue SPA ────────── /api/* ┘ └── in-process jobs and embedder
```
## Project Structure
- **The web UI** is a Vue single-page app, built into the image and served by Quart. Every unknown path that isn't under `/api/` falls back to `index.html`; `/api/*` always answers JSON.
- **The REST API** (`src/grimoire/routes/`) backs the UI, one blueprint per resource.
- **The MCP server** (`src/grimoire/mcp/`) is how Claude reads and writes records. Each module in `mcp/tools/` registers its tools, and the server's instructions and each tool's docstring are the contract an agent reads. Requests authenticate with an API key as a Bearer token.
- **The plugin** (`plugin/`) runs in the user's Claude Code. Its hooks call `/api/plugin/*` to put the right context in front of Claude at the right moment: the project at session start, matching rules and prior art before a prompt or a tool call, the rules mounted on a moment of work, and a check on the reply that ends a turn. Its skills describe the working practices. See [API Keys & MCP](api-keys-and-mcp.md) to install it.
- **Services** (`src/grimoire/services/`) hold the logic both doors share. Routes and MCP tools stay thin: parse the request, check access, call a service, return a plain dict.
```
scribe/
├── docker-compose.yml # Development stack
├── docker-compose.prod.yml # Production stack (Docker Swarm)
Most records are rows in one `notes` table, told apart by kind:
**Single container for frontend + API.** Quart serves the Vue.js production build as static files and exposes the REST API under `/api/`. The SPA is built by Vite during the Docker image build.
- **Notes**, and the specialised kinds that are notes underneath: **snippets** (recorded prior art, with a canonical code location), **lessons** (transferable insights, linked to the rules they instance) and **processes** (stored procedures, synced to Claude Code as skills).
- **Tasks**: a note with a status. A task's kind is `work`, `issue` or `spike`. Tasks carry work logs, sub-tasks, recurrence and a session claim.
**SPA routing via 404 handler.**`app.py` uses `@app.errorhandler(404)` (not a catch-all route) to serve static files or fall back to `index.html`. API routes (`/api/*`) always return JSON 404. This avoids a catch-all `/<path:path>` route intercepting API GETs.
Around them:
**Unified note/task model.** A task is just a note with task attributes enabled. `status IS NOT NULL` means it's a task. "Convert to task" sets `status='todo'`; "convert to note" clears `status`, `priority`, `due_date`. No separate table, no cascade complexity.
- **Projects** group work and carry a goal. **Milestones** are plans: a design body plus ordered step tasks.
- **Systems** are a project's named areas; records are tagged with them, and each maps to a canonical system shared across projects.
- **Rulebooks → topics → rules.** A rule in a rulebook is global; a rule on a project applies there only. Rules have a trigger (`when_to_apply`), versions, relations to other rules, and can be mounted on **moments** of work (`work.change`, `reply.report`, ...), which plugin actions reach through moment mappings.
- **Family canon**: ideas shared across projects on the same platform, and each project's answer to them (adopted, variant, owed).
- **Design systems**: tokens and guidance, with inheritance; the stylesheet is generated from them.
- **The shape ledger**: code shapes extracted from bound repos and their classification against snippets.
- **Embeddings** for notes (chunked), rules, milestones and systems, in pgvector columns.
- **Telemetry**: retrieval logs, note and rule usage, and the tuning history of retrieval thresholds.
- **Users, groups, shares and API keys.** Access is decided in the app (`services/access.py`): owners, direct shares and group shares, with read or write permission.
**First-class tag column.** Tags live in a `tags ARRAY[text]` column and are explicitly set by the client — not auto-extracted from body text. Hierarchical tags (`project/webapp`) supported via SQL `unnest + LIKE` prefix matching.
Deleted records go to the trash first and can be restored until retention purges them.
**Background generation architecture.** LLM streaming runs in a detached `asyncio.Task` that writes into an in-memory `GenerationBuffer`. SSE clients tail the buffer and can reconnect mid-stream without data loss. Buffer has a `cancel_event` for user-initiated stop. Completed buffers are cleaned up after 60s grace period. Periodic DB flushes every 5s preserve partial content. Both chat and AI Assist use this architecture.
## Retrieval
**SSE over WebSockets for LLM streaming.** SSE clients connect via `GET /api/chat/conversations/:id/generation/stream` with `Last-Event-ID` reconnection support. Frontend uses `fetch()` + `ReadableStream`.
Search and the plugin's context injection run on the same pipeline (`services/retrieval_pipeline.py`): embed the query, rank stored chunks by cosine similarity in pgvector, keep what clears the surface's threshold, and log what was shown. Each surface (MCP search, auto-inject, the pre-tool rule arm, the write-path prior-art arm, ...) is a spec over those stages with its own thresholds and budget. The thresholds are settings an admin can tune, and every change is recorded with its reason. When the embedder is unavailable, search falls back to keyword matching.
**Context building is server-side.** Backend fetches URL content and searches notes. Frontend sends the message text + optional context note IDs. `build_context()` returns `(messages, context_meta)`; metadata includes auto-found note IDs/titles sent to frontend via a `context` SSE event beforestreaming begins.
The embedding model downloads into `FASTEMBED_CACHE_DIR` (default `/data/fastembed-cache`) on first use, so the `/data` volume keeps it across restarts.
**No blocking long-running operations.** Any slow operation (model pulls, LLM calls, URL fetching) must never block app startup or freeze the UI. Backend uses SSE streaming for incremental responses. Model pulls stream NDJSON progress to the frontend.
## Background work
**SSRF protection.**`services/llm.py` blocks requests to loopback, private, link-local, reserved, and multicast addresses before fetching. `follow_redirects=False` prevents redirect-based bypasses.
All in-process; there is no separate worker container.
**Session cookie security.**`HttpOnly` and `SameSite=Lax` always set. `Secure` flag controlled by `SECURE_COOKIES` env var.
- At startup the app records any shipped default that moved in this release, starts its schedulers, and only then backfills missing embeddings, so the backfill never competes with startup for the database.
- Recurring tasks spawn every 15 minutes. Daily jobs: version auto-pinning (03:00 UTC), trash purge (03:30 UTC), targeted `VACUUM ANALYZE` (04:00 UTC by default). Scheduled backups check hourly whether one is due ([Backups](backups.md)).
- Log, notification and auth-token retention run on their own loops.
- A diagnostics heartbeat writes to `/data/diagnostics`.
**Rate limiting.** In-memory sliding-window rate limiter (`rate_limit.py`) applied to auth endpoints: login (10/60s), register (5/300s), forgot-password (5/300s), reset-password (10/60s). Keys are per-IP. `TRUST_PROXY_HEADERS` env var enables `X-Forwarded-For` / `X-Real-IP` when behind a reverse proxy.
**Idempotent migrations.** All Alembic migrations use raw SQL with `CREATE TABLE IF NOT EXISTS`, `CREATE INDEX IF NOT EXISTS`, and `DO $$ BEGIN CREATE TYPE ... EXCEPTION WHEN duplicate_object` to allow safe re-runs.
## Data Model
### Users
| Column | Type | Notes |
|--------|------|-------|
| `id` | int PK | |
| `username` | text UNIQUE NOT NULL | |
| `email` | text nullable | |
| `password_hash` | text nullable | NULL for OAuth-only accounts |
`project_shares`, `note_shares`: each has `shared_with_user_id` OR `shared_with_group_id` (exclusive), `permission` (`viewer`/`editor`/`admin`), `invited_by`.
`groups`, `group_memberships`: platform-wide groups with `member`/`owner` roles.
Permission resolution is centralised in `services/access.py`. `get_project_permission(uid, project_id)` checks ownership → direct share → group-based share → note→project inheritance, returning the highest applicable permission.
### Journal-Related Tables
`weather_cache`: per-user, per-`location_key` cache. Columns: `user_id`, `location_key` (`home`/`work`/etc.), `location_label`, `forecast_json` (Open-Meteo response), `previous_json` (last forecast, used to detect changes), `fetched_at`. Lat/lon are *not* stored on the cache row — they live in the user's `journal_config.locations.{home|work}` setting and are used at refresh time.
### API Keys
`api_keys`: `id`, `user_id` FK CASCADE, `prefix` (first 8 chars, displayed in UI), `key_hash` (SHA-256 of full key — full key never stored), `name`, `scope` (`read`/`write`), `created_at`, `last_used_at`.
| `services/task_logs.py` | Append/list/delete work log entries on tasks |
| `services/journal_prep.py` | Deterministic data gather (tasks/events/weather/projects/recent moments/open threads) → LLM prose opener; persisted as the first assistant message of today's journal Conversation |
| `services/journal_pipeline.py` | System-prompt builder for journal conversations; calls `build_profile_context()` so the LLM sees the user's profile + learned summary |
| `services/journal_scheduler.py` | APScheduler `BackgroundScheduler`; per-user prep job; live-reschedule via `update_user_schedule()`; catch-up logic for missed runs |
| `services/journal_search.py`, `services/moments.py` | Moment recording + search across journal history |
| `services/user_profile.py` | `build_profile_context()` consolidates profile + observations + learned_summary into a system-prompt block |
| `services/research.py` | SearXNG research pipeline: sub-queries → parallel fetch → outline → section synthesis → executive summary → index note with linked section notes |
| `services/events.py` | Internal events CRUD: `list_events`, `create_event`, `update_event`, `delete_event`, `get_event`; source of truth for all event LLM tools |
| `services/rss.py` | feedparser-based fetch, per-feed DB cache, prune-to-100 |
Every database wait on startup has a deadline. Migrations wait for the database to accept connections (up to five minutes) before they run, so the app starts cleanly whatever order the containers come up in.
## Authentication
**Session cookies** —`HttpOnly`, `SameSite=Lax`, optionally`Secure`(`SECURE_COOKIES` env var). Session includes `session_version`; mismatch with DB value (after password change) results in 401 and session clear.
-**Sessions**:`HttpOnly`, `SameSite=Lax` cookies,`Secure`when `SECURE_COOKIES=true`. Changing a password bumps the user's sessionversion and signs out their other sessions.
- **API keys**: `Authorization: Bearer fmcp_...`, stored hashed. A `read` key may only call read endpoints and read MCP tools.
- **OIDC** single sign-on with PKCE, optionally replacing local logins ([SSO / OAuth](sso-oauth.md)).
- **Rate limits** on register, login and forgot-password, per client IP (`TRUST_PROXY_HEADERS` reads the IP from the proxy's headers).
**Bearer token / API keys** — `Authorization: Bearer <key>` accepted by `_check_auth()` as an alternative to session cookies. The raw key is SHA-256 hashed and looked up via `services/api_keys.py`. Keys with `scope=read` are rejected on non-safe methods (`POST`/`PATCH`/`DELETE`). Used by Fable MCP and any external API consumers.
Credentials Grimoire stores for outside services (forge tokens, the backup destination's S3 secret) are sealed under a key derived from `SECRET_KEY`, which the instance makes on first start and keeps in `/data/secret_key` unless one is set.
**Local auth + OIDC/OAuth (PKCE)** — `services/oauth.py` handles discovery and `find_or_create_oauth_user`. On OAuth login: checks existing `oauth_sub` → matching email → creates new user.
## Repository layout
See [sso-oauth.md](sso-oauth.md) for provider-specific setup instructions.
## LLM Pipeline Internals
### Tool Routing
No separate intent router — the main model handles all tool routing directly via Ollama's structured tool-calling output. The model receives the full tool schema list and decides whether to call a tool or respond conversationally. Extended reasoning (`think=True`) is always on for qwen3-class models: content-based gating was tried but exposed tool-call template fragility on short tool-intent prompts.
### Tool Loop
Multi-round tool loop (max 5 rounds). All implementations in `services/tools/` (decorator-based registry); `execute_tool(user_id, tool_name, arguments, conv_id=None, workspace_project_id=None)` is the dispatcher. `conv_id` and `workspace_project_id` are threaded in from `run_generation()` so tools like `set_rag_scope` can write to the current conversation.
**Unified `create_note` tool** — creates both notes and tasks. Setting `status` (e.g. `"todo"`) creates a task; omitting it creates a knowledge note. All task fields (due_date, priority, milestone, parent_task, recurrence_rule) are available on the single tool.
**Duplicate protection on `create_note`:**
1. Exact title match (case-insensitive) → hard block, redirect to `update_note`
2. Fuzzy title match (SequenceMatcher ≥ 82%; punctuation stripped before candidate search) → hard block
3. Semantic content similarity (threshold 0.90, body ≥ 80 chars) → soft block with `requires_confirmation: true`
**Project resolution** (`_helpers.resolve_project`): 4-step lookup — (1) exact DB match, (2) `query in title` substring, (3) `title in query` reverse substring, (4) SequenceMatcher ≥ 0.55.
### Context Window and Summarisation
`OLLAMA_NUM_CTX` (default 16384) controls the context window for all generation calls. Intent classification always uses `num_ctx=4096` to reduce VRAM pressure.
History summarisation threshold: 30 messages. Keeps 8 recent messages. Summary max 400 tokens.
### Web Research Pipeline
`services/research.py` implements a full autonomous research pipeline:
1. Intent model generates 5 focused sub-queries
2. All 5 SearXNG queries run in parallel (200ms stagger to avoid rate limiter)
3. Up to 15 unique URLs fetched in parallel
4. LLM generates an outline (2–8 sections with title + focus)
5. Each section synthesised in parallel from relevant sources
6. Executive summary generated from all section content
7. Index note created with executive summary + links to section notes; section notes linked back via `parent_id`
8. Falls back to single-note synthesis if outline generation fails
SearXNG tip: add the app server IP to `botdetection.ip_lists.pass_ip` in SearXNG `settings.yml` to bypass the rate limiter for trusted backend requests.
### Image Cache
`search_images` tool fetches images server-side via SearXNG, stores them on disk (SHA-256 dedup, content-type validation, 5 MB cap), and serves from `/api/images/<id>`. The user's browser never contacts the original image host.
1.`semantic_search_notes()` — cosine similarity via pgvector, threshold configurable per call; accepts `orphan_only` and `project_id` scope flags.
2. Notes ≥ 0.60 similarity auto-injected into system prompt (up to 3, 800 chars each).
3. Notes 0.45–0.60 surfaced in chat sidebar as "Suggested" (user clicks to include).
4. Explicitly included notes delivered full-body.
5.`excluded_note_ids` prevents the current note from being injected as its own context.
### RAG Scope (three-value system)
`conversations.rag_project_id` controls which notes are eligible for retrieval:
| Value | Behaviour |
|-------|-----------|
| `NULL` (default) | Orphan notes only — notes with `project_id IS NULL` |
| `-1` | All notes — opt-in to global search |
| Positive int | That project's notes only |
`build_context()` derives `orphan_only` + `effective_project_id` from this value before calling both the semantic and keyword search paths.
**Scope tools** — Two LLM tools let the model discover and switch scope mid-conversation:
-`search_projects` — SequenceMatcher scoring over title + description + `auto_summary`; returns top 5 matching projects.
-`set_rag_scope` — persists the new `rag_project_id` to DB immediately; blocked in workspace view; causes the SSE `done` event to include `new_rag_scope` + `new_rag_scope_label` so the frontend chip updates reactively.
**Project summaries** — `generate_project_summary()` calls Ollama (fire-and-forget) and stores the result in `projects.auto_summary`. Triggered on project update and note saves (debounced 1h). `backfill_project_summaries()` runs at startup.
Embedding model: `nomic-embed-text` via Ollama. Backfill runs 30s after startup (background task).
```
├── alembic/versions/ database migrations
├── src/grimoire/
│ ├── app.py app factory, blueprints, startup and shutdown
│ ├── config.py environment configuration
│ ├── models/ SQLAlchemy models
│ ├── routes/ REST blueprints
│ ├── mcp/ MCP server and its tool modules
│ ├── services/ shared logic
│ └── restore.py `python -m grimoire.restore` for a fresh instance
├── frontend/src/ views, components, stores, composables, api client
├── plugin/ Claude Code hooks, skills and commands
├── scripts/ CI checks and the plugin version minter
└── tests/ unit tests, plus `-m integration` against real Postgres
Configuration is via environment variables. The `docker-compose.yml` file sets defaults for local development; override them in a `.env` file (gitignored).
Grimoire reads its configuration from environment variables. Most settings people change day to day (registration, SMTP, retrieval thresholds, backups, forge connections) live in **Settings** in the web UI instead.
## Environment Variables
Any variable marked *secret* can also be read from a file: set `<NAME>_FILE` to the path, for example a Docker secret at `/run/secrets/grimoire_key`.
| `BASE_URL` | — | Public URL (e.g. `https://notes.example.com`) — required for OIDC redirect URIs and email links |
| `DATABASE_URL`*(secret)* | `postgresql+asyncpg://grimoire:grimoire@localhost:5432/grimoire` | PostgreSQL connection string, using the `asyncpg` driver |
| `SECRET_KEY`*(secret)* | kept in `DATA_DIR` | Signs sessions and seals stored credentials. Unset, the instance makes its own on first start and keeps it in `DATA_DIR/secret_key`. Set it only to manage the key yourself; it wins over the kept one (see below) |
| `DATA_DIR` | `/data` | Where the instance keeps its own `SECRET_KEY`. Mount a volume here |
| `BASE_URL` | `http://localhost:5000` | Public URL, used in email links and OIDC redirects |
| `SECURE_COOKIES` | `false` | `true` when Grimoire is served over HTTPS |
| `TRUST_PROXY_HEADERS` | `false` | `true` behind a reverse proxy, to read the client IP from `X-Forwarded-For` / `X-Real-IP` |
See [sso-oauth.md](sso-oauth.md) for provider-specific setup.
SSO turns on when the issuer, client ID and secret are all set. See [SSO / OAuth](sso-oauth.md).
### Security / Proxy
### Email
Admins can set SMTP in **Settings**; these are the defaults until they do.
| Variable | Default |
|----------|---------|
| `SMTP_HOST` | — |
| `SMTP_PORT` | `587` |
| `SMTP_USERNAME` | — |
| `SMTP_PASSWORD`*(secret)* | — |
| `SMTP_FROM_ADDRESS` | — |
| `SMTP_FROM_NAME` | `Fabled Grimoire` |
| `SMTP_USE_TLS` | `true` |
### Integrations
| Variable | Default | Description |
|----------|---------|-------------|
| `TRUST_PROXY_HEADERS` | `false` | Set `true` when behind a trusted reverse proxy to read real IP from `X-Forwarded-For` / `X-Real-IP` |
| `PLUGIN_MARKETPLACE_URL` | this repo's git URL | The git URL shown in the plugin install command under Settings → MCP Access. An admin can override it in Settings |
| `FORGE_KIND`, `FORGE_BASE_URL`, `FORGE_TOKEN`*(secret)* | — | A git forge connection for admins' projects, so a token can live in a Docker secret. Each user can add their own under Settings → Integrations; a stored connection for the same host wins |
| `FORGE_WEBHOOK_SECRET`*(secret)* | — | Verifies push webhooks from the forge |
| `SEARXNG_URL` | — | A SearXNG instance, listed as the web-search backend under Settings → Integrations |
### Web Search / Images
## Running it
| Variable | Default | Description |
|----------|---------|-------------|
| `SEARXNG_URL` | — | SearXNG base URL for web search and image search tools |
| `IMAGE_MAX_BYTES` | `5242880` (5 MB) | Maximum size for cached images |
### Docker Compose
### Data / Logging
| Variable | Default | Description |
|----------|---------|-------------|
| `LOG_RETENTION_DAYS` | `90` | Days to keep app logs before automatic pruning |
| `DATA_DIR` | `/data` | Root directory for persistent data (VAPID keys, backups) |
## Docker Compose Setup
### Development (`docker-compose.yml`)
[`docker-compose.yml`](../docker-compose.yml) runs the published image with its own database, and needs nothing set:
```bash
# Copy the example env file
cp .env.example .env
# Edit .env to set a real SECRET_KEY
# Start the stack
docker compose up --build
# App available at http://localhost:5000
docker compose up -d
```
The first user to register becomes admin. Registration auto-closes after that (re-enable from Settings → Users).
Open `http://localhost:5000`. The first account registered becomes the admin; after that, registration stays closed until an admin opens it in Settings or sends an invite.
### Production (`docker-compose.prod.yml`)
Update with `dockercompose pull && docker compose up -d`. The key and the data stay in their volumes.
The production compose file adds:
- Docker Secrets for `SECRET_KEY_FILE` and `DATABASE_URL_FILE`
- Network isolation (internal bridge for DB + Ollama; only the app is externally accessible)
docker compose -f docker-compose.dev.yml up --build
```
## Production Deployment
### Docker Swarm
### Reverse Proxy (Required)
[`docker-compose.prod.yml`](../docker-compose.prod.yml) is a stack file: the app and database on a private overlay network, with `SECURE_COOKIES` and `TRUST_PROXY_HEADERS` on. A Swarm can move the app to another node, away from a node-local `/data` volume, so the stack sets its own key: provide `DB_PASSWORD` and `SECRET_KEY` in the environment (or mount a Docker secret and set `SECRET_KEY_FILE`):
Fabled Assistant does **not** handle SSL/TLS. Run it behind a reverse proxy:
- **Nginx**, **Traefik**, or **Caddy** in front of the app container
- Terminate TLS at the proxy; forward to port 5000
- **Do not expose port 5000 directly to the internet**
- Rate-limit auth endpoints: ≤ 5 req/min per IP on `/api/auth/login` and `/api/auth/register`
The database user, database and volume default to `grimoire`. An install created under the name Scribe keeps its data by setting `POSTGRES_USER`, `POSTGRES_DB` and `PGDATA_VOLUME` to the names it was created with.
The shipped files leave out healthchecks and restart policies on purpose. The app waits for its database on startup, and the orchestrator's defaults restart it if it ever exits.
- **App data** (`/data`): the instance's own `SECRET_KEY`, the embedding model cache and diagnostics. Losing it costs the key: Settings then asks for saved forge tokens and the backup secret again, and the model downloads again.
Back up the database with **Settings → Data → Scheduled backups** ([Backups](backups.md)). A database backup doesn't carry the key; to move sealed credentials to a new instance, copy `/data/secret_key` with it.
## Behind a reverse proxy
Grimoire doesn't terminate TLS. Put it behind Nginx, Traefik or Caddy, forward to port 5000, and keep port 5000 off the internet. Set `SECURE_COOKIES=true`, `TRUST_PROXY_HEADERS=true` and `BASE_URL`.
- **`SECURE_COOKIES=true`** — must be set when running behind TLS.
- **`BASE_URL`** — set to your public URL; required for OIDC redirects and email links.
- **Registration** — auto-closes after the first user (admin). Re-enable from Settings → Users or send invite links.
- **Session invalidation** — changing or resetting a password bumps `session_version`, evicting all other active sessions. Button also available in Settings → Account.
- **Keep Ollama on an internal network** — both compose files keep Ollama off the host network. Never expose the Ollama port publicly.
- **Default SECRET_KEY warning** — the app logs a `WARNING` on startup if the default dev key is in use.
## Model Management
Models are managed through Settings → General → Model Management in the web UI. You can:
- Pull new models by name (streams progress via SSE)
- View installed models with size and loaded/unloaded state
- Delete unused models
The app auto-warms user-preferred models on startup (only models already installed — never auto-pulls). The embedding model (`nomic-embed-text`) is auto-pulled on startup if missing.
- **`llama3.1:8b`** — reliable baseline, widely tested with tools
- **Keep the key.** Forge tokens and the backup destination's S3 secret are sealed under a key derived from `SECRET_KEY`, so a copy of the database alone can't read them. Unset, the instance makes a random key and keeps it in `/data/secret_key`, readable by its owner only; keep that volume. If the key changes (say, a restore onto an instance with a different key), Settings marks those credentials and asks for them again; nothing else breaks. The app warns at startup when `SECRET_KEY` is a public placeholder or `/data` is not a mounted volume, and refuses to start with `dev-secret-change-me` when `SECURE_COOKIES=true`.
- **`SECURE_COOKIES=true`** whenever Grimoire is served over HTTPS.
- **Registration** is open only for the first account. Invite people, or open it briefly from Settings.
- **Signing everyone else out**: changing a password ends that user's other sessions; Settings → Account has a button for it too.
- **API keys** are per user, shown once, and can be read-only. Revoke any you no longer use under Settings → MCP Access.
**Critical rule:** Action button colors are universal across all apps. A Save button in Scribe and a Save button in Minstrel look identical. Per-app accents do not appear on buttons.
**Critical rule:** Action button colors are universal across all apps. A Save button in Grimoire and a Save button in Minstrel look identical. Per-app accents do not appear on buttons.
### Semantic colors
@@ -71,7 +71,7 @@ Parchment is intentionally not pure white — it's slightly warm to feel like ag
@@ -80,7 +80,7 @@ Parchment is intentionally not pure white — it's slightly warm to feel like ag
**Accent usage rules:**
- The accent appears on the app's wordmark and icon.
- The accent indicates active/current state in nav (the selected page, the active tab).
- The accent is the cursor color and text-selection color in long-form surfaces (Scribe notes, Forge story drafts).
- The accent is the cursor color and text-selection color in long-form surfaces (Grimoire notes, Forge story drafts).
- The accent does NOT appear on primary or secondary action buttons.
- The accent does NOT appear in body text or chrome.
- One accent per app. Don't mix accents within a single app.
@@ -101,6 +101,8 @@ All text-on-surface combinations meet WCAG AA at minimum. Parchment on Obsidian
| Inter | Body, UI, labels | Google Fonts |
| JetBrains Mono | Code, terminal output, monospaced data | Google Fonts |
All three are served by the app itself, from `frontend/src/assets/fonts/`, with every subset Google Fonts publishes for them. A page never asks a font host for anything, so an instance with no internet still draws in its own type. They are under the SIL Open Font License; the licences sit beside the files.
### Why this pairing
Fraunces is a contemporary serif with personality — it has the warmth and authority of a book serif without feeling like costume. It signals "this is considered" without signaling "this is a fantasy product." Inter is the workhorse — neutral, ubiquitous, designed for screens, doesn't compete with the serif. JetBrains Mono is the natural choice for any developer-adjacent product and supports ligatures.
@@ -124,7 +126,7 @@ Fraunces is a contemporary serif with personality — it has the warmth and auth
- **Sentence case everywhere.** Never Title Case for headings, never ALL CAPS except for the Tiny micro-label style.
- **Two weights only:** 400 regular and 500 medium. Never 600 or 700 — they read heavy in dark mode.
- **Fraunces only at 18px and above.** Below that it loses too much detail and feels fragile. For h3 and below, use Inter.
- **Line height** 1.5 for body, 1.3 for headings, 1.7 for long-form reading surfaces (Scribe notes, Forge drafts).
- **Line height** 1.5 for body, 1.3 for headings, 1.7 for long-form reading surfaces (Grimoire notes, Forge drafts).
- **Letter-spacing** at default for everything except the Tiny micro-label, which gets `0.08em` letter-spacing and uppercase styling.
---
@@ -173,7 +175,7 @@ Padding: `8px 16px` for default, `6px 12px` for compact, `10px 20px` for promine
Used for tags, hashtags, code spans, status badges. Background is the accent color at ~15% opacity, text is the accent at full strength. Border-radius 4px, padding `2px 8px`, Inter 11px/500.
In Scribe specifically, hashtags and tags use the dusty violet accent. In Minstrel, they'd use forest teal. The pattern is shared; the color follows the app.
In Grimoire specifically, hashtags and tags use the dusty violet accent. In Minstrel, they'd use forest teal. The pattern is shared; the color follows the app.
### Cards
@@ -272,9 +274,9 @@ Use **Lucide** (https://lucide.dev) as the base icon set — it matches this sty
## Per-app application
### Fabled Scribe (#5B4A8A — dusty violet)
### Fabled Grimoire (#5B4A8A — dusty violet)
A second-brain notes and task management tool. The accent appears in: the wordmark, hashtags and tag pills, the active nav item, text selection color, and the cursor in the editor. Notes are presented on Iron-surfaced cards with generous reading line-height. The hashtag system uses Scribe's accent for visual continuity.
A second-brain notes and task management tool. The accent appears in: the wordmark, hashtags and tag pills, the active nav item, text selection color, and the cursor in the editor. Notes are presented on Iron-surfaced cards with generous reading line-height. The hashtag system uses Grimoire's accent for visual continuity.
### Minstrel (#4A6B5C — forest teal)
@@ -332,7 +334,7 @@ Home server management. The accent appears on: the wordmark, healthy/online stat
}
/* Per-app accent — set ONE of these on the root for each app */
[data-app="scribe"]{--fs-accent:#5B4A8A;}
[data-app="grimoire"]{--fs-accent:#5B4A8A;}
[data-app="minstrel"]{--fs-accent:#4A6B5C;}
[data-app="forge"]{--fs-accent:#8B5A2B;}
[data-app="roundtable"]{--fs-accent:#4A5D7E;}
@@ -354,220 +356,13 @@ If using Tailwind, extend the theme with these tokens rather than relying on def
---
*Last updated: April 25, 2026. Iterate as the family of apps grows.*
# Grimoire
---
The design system Grimoire renders with is stored in Grimoire itself: **Grimoire**, which inherits **FabledSword**, edited at `/design-systems`. `frontend/src/assets/theme.css` is generated from it, and CI rejects a `var()` the sheet doesn't define. Where this page and the stored system differ, the stored system is right. Grimoire applies the baseline above with these choices:
# Scribe-specific decisions in progress
> This section tracks decisions made while adapting the FabledSword baseline above for Scribe specifically. Items here are *in progress* — once they feel solid, they get folded into the main body of the document (either as Scribe-specific extensions in the per-app section, or as updates to the universal rules where Scribe's needs reveal a gap in the baseline).
*Iteration started: 2026-04-26. Foundation pass shipped 2026-04-27 in `7a9a8b7` (palette, fonts, light mode, action tokens, hardcoded indigo cleanup, warm-gold deprecation). Surface phase shipped 2026-04-27 across `93a3beb` → `3c1ec40` (Lucide migration, Hybrid-rule button reclassification per surface, long-form line-height, two-weights-only). The system is now applied end-to-end; this section will fold into the main body once the result has had time to settle in real use.*
## Decisions made so far
### Accent footprint — Hybrid rule (not Strict)
The doc baseline says the per-app accent only appears on wordmark, active nav, cursor, and text selection — never on action buttons. Scribe currently uses indigo on essentially every interactive surface (CTAs, scrollbars, glows, borders, focus rings). Hard-cutting to the doc baseline would lose too much identity in one swing.
**Hybrid rule:** the accent reserves a slightly larger footprint than the doc baseline, but still much smaller than today.
- **Accent (dusty violet) lives on:** wordmark; active nav; cursor and text selection in editor surfaces; tags/pills/wikilinks; in-progress task badge; focus rings; **brand-moment CTAs** — chat Send, "Create note" empty-state CTA, journal Send, "Start journaling" empty-state.
- **Moss (sage-green primary) lives on:** Save / Submit / Confirm in forms and modals; generic affirmative actions where the button just means "do this thing" with no brand pretense.
- **Bronze (secondary):** Cancel-but-not-destructive, alternative paths.
- **Oxblood (destructive):** Delete / Remove (paired with an icon).
- **Pewter ghost:** tertiary actions, "later", "skip", "see also".
**Rule of thumb:** if the user is engaging with a *Scribe-feature moment* (sending a chat, opening a fresh note, jumping into the journal), accent. If they're just *operating the software* (saving an edit, confirming a dialog), Moss.
The doc is dark-only. Scribe today supports both light and dark, and we keep both. The light mode is derived to *match* the dark mode aesthetic rather than defaulting to system white-and-ink.
- Page background: in the `#F5F1E8` warm cream family (specific values TBD)
- Cards: near-white but slightly tinted
- Text: deep ink `#14171A` (mirroring Obsidian)
- Accent: same dusty violet `#5B4A8A` (works on both themes)
The metaphor stays consistent across themes: ink on aged paper (light) ↔ parchment text on graphite (dark). Light mode is *not* the system standard look.
**Known downside:** warm parchment backgrounds can fight with embedded color content. Mitigation: code blocks get a slight cool wash in light mode specifically, to keep syntax highlighting readable.
### Status and priority palette — extend the doc's semantic set
The doc's semantic colors (Success / Warning / Error / Info / Destructive) are leaner than what Scribe needs for task management. Rather than running a parallel palette, Scribe extends the doc by mapping its status/priority tokens onto doc primitives where they fit and defining new app-level tokens for the rest.
| `priority-none` | Vellum/Ash | doc | No signal |
The priority row reads as a clean cool→warm gradient (slate blue → golden brown → terracotta), which matches the semantic loudness — coherence the current ad-hoc palette doesn't have.
**Other functional tokens:**
| Token | Color | Logic |
|---|---|---|
| `wikilink` | dusty violet | Editorial brand moment per Hybrid |
| `overdue` | Error `#C04A1F` | Same as priority-high — overdue IS a priority signal |
| `toast-success` | Moss | doc semantic |
| `toast-error` | Error | doc semantic |
| `toast-info` | Info | doc semantic |
| `tag-bg` / `tag-text` | accent at 15% / accent | Per doc pill recipe |
Each token gets a `*-bg` companion at low alpha (matching the existing pattern in `theme.css`).
**Removed:** the warm gold accent (`--color-accent-warm: #b8860b`). Its two jobs split:
- Dates and timestamps (knowledge cards, event details, chat) → use `text-secondary` instead. Dates are metadata, not a brand surface; muted is the correct register.
- Paused project status → use the new `status-paused` (Warning `#8B6F1E`) row above. Same golden-brown family, semantically aligned.
### Typography — adopt the doc's stack and scale
Adopt the doc's type stack and scale verbatim, with one deferred verification (long-form line-height in practice).
- **Body font: Inter.** Replaces Scribe's current system-stack body font. Doc-defined; no Scribe-specific divergence.
- **Type scale:** as in the doc table — Display 40 / H1 32 / H2 24 / H3 18 / Body 15 / Body small 13 / Label 12 / Code 13 / Tiny 11.
- **Two weights only:** 400 regular, 500 medium. No 600/700 (reads heavy in dark mode and against the muted palette).
- **Family rules:** Fraunces at 18px+ only (Display, H1, H2). H3 and below = Inter. Code = JetBrains Mono.
**Code-block exception:** in light mode specifically, code blocks use a slight cool wash (e.g. `#EBEDF0`) instead of the warm inset bg, so syntax highlighting reads cleanly. This is the mitigation for the "warm bg fights colored content" downside.
The accent (`#5B4A8A` dusty violet), Moss, Bronze, Oxblood, and the semantic color set are **identical across themes** — only the surface and text palettes flip.
### Chat-bubble codification — keep the Illuminated Transcript pattern
The existing chat-bubble pattern (informally called "Illuminated Transcript") gets written into the design system as a documented chat component. Other apps in the family that add a chat surface inherit the pattern; Scribe's existing implementation continues to work with only color shifts.
**User bubble (whisper):**
- Background: transparent
- Border: 0.5px Pewter (was: indigo-tinted)
- Text color: secondary (Vellum dark / `#5A5852` light)
- Background: card surface (Iron dark / `#FBF8F0` light)
- Border: none on top/right/bottom; **2px solid accent (dusty violet) on left edge only**
- Box-shadow: accent-tinted glow + standard depth shadow (formula: `0 4px 28px rgba(<accent>, 0.14), 0 2px 8px rgba(0,0,0,0.4)` in dark; lower alphas in light)
- Text color: primary (Parchment dark / Obsidian-inverted light)
- Left-aligned, rounded except bottom-left
The 2px-accent left edge is the "illumination" — like an illuminated capital in a manuscript. The shadow is the lift. Together they make the assistant bubble read as the *primary* voice, while the user bubble is the *margin note*.
**Inline tool-call cards (`ToolCallCard`)** rendered inside an assistant bubble do NOT get their own border (per the border philosophy — the bubble already contains them). They use a slight surface tint to differentiate.
### Iconography — adopt Lucide, enforce a scale
Scribe currently hand-inlines SVG paths in 16+ Vue files, with 5 different stroke weights and 8+ different sizes. The visual style is already outline + rounded caps + `currentColor` stroke (matches the doc's intent), but there's no shared source and no scale discipline.
**Migration policy:**
1.**Install `lucide-vue-next`** as the icon source. Replace hand-inlined SVGs with imported components. Single source of truth.
2.**Strict size scale: 16px and 24px only.** Today's mix of 12/13/14/15/17/18/20 collapses to those two. 16 for inline-with-text and small affordances; 24 for nav and primary actions.
3.**Stroke weight per the doc: 1.5 at 24px, 1 at 16px.** Lighter than the current default of 2 — reads more refined, matches the muted palette philosophy. Overrides Lucide's default.
4.**Outline by default; filled only for active/selected state.** Introduces a new affordance Scribe doesn't currently use — bookmark/pin/star icons can switch outline → filled to indicate active state. Reserve filled style strictly for this.
5.**No emoji in chrome.** Replace the 3 files' emoji usage in UI labels/buttons/badges/empty states with Lucide equivalents. Emoji remain fine in *user content* (note bodies, chat messages the user typed).
Work cost: ~30-60 individual icon swaps across the 16 files. Mechanical; doesn't require redesign of any component.
### Voice and tone — adopt principles, defer formal audit
The doc's voice register applies to Scribe (understated mythic — plain for functional UI, flavored for empty/error/loading states). No formal sweep of every UI string yet.
**Approach:** apply the voice opportunistically as components are touched in the polish pass — when redesigning a settings tab, an empty state, or an error toast, rewrite the copy at the same time using the doc's register and examples table as the guide. A standalone audit pass is deferred unless drift becomes visible.
### Border philosophy — structural, not decorative
The doc treats borders as *structural* (Pewter neutral hairlines that say "boundary"), not decorative (Scribe today uses indigo-tinted borders that say "branded edge"). That principle suggests removing borders in places where surface tint and spacing already communicate separation.
**Borders to remove:**
- List rows (NotesListView, TasksListView, conversation history) — surface contrast + spacing should separate rows; current border reads as "boxed-in"
- Inline `ToolCallCard` inside chat bubbles — the bubble is already a container; an extra border feels like double-wrapping
- Filter chips and search-bar pills with a background tint — background does the work
- Empty-state callouts with dashed/bordered "nothing here yet" boxes — tinted background reads cleaner
**Borders to keep (genuinely structural):**
- Standalone card containers (Notes viewer, Task viewer, the new daily prep card)
- Modal / dialog edges
- Code blocks (separates content type, not just space)
- Focus rings (accessibility)
- Major section dividers within a panel
Border weight is not load-bearing for Scribe — happy to use the doc's 0.5px hairline default; the *placement* discipline matters more than the weight.
## Open threads (next iterations)
### Foundation pass — shipped 2026-04-27 (`7a9a8b7`)
Mechanical token + font + light-mode rewrite of `frontend/src/assets/theme.css`, plus a sweep of hardcoded indigo and `--color-accent-warm` references across ~14 component files. Action tokens (`--color-action-primary` Moss, `--color-action-secondary` Bronze, `--color-action-destructive` Oxblood, `--color-action-ghost-border` Pewter) are defined but not yet applied — buttons still flow through `--color-primary` and read as dusty-violet gradients in the meantime, by design. Spec lives at `docs/superpowers/specs/2026-04-27-design-system-polish-foundation-design.md` (gitignored, local-only).
Bundled as Hybrid (option C from the brainstorm): Lucide cross-cutting first, then surface-by-surface for the judgment work. Spec lives at `docs/superpowers/specs/2026-04-27-design-system-polish-surface-design.md` (gitignored, local-only). Seven PRs landed on `dev`:
| PR | Commit | Surface | Notes |
|---|---|---|---|
| 1 | `93a3beb` | Lucide cross-cutting | 60 hand-inlined SVGs across 15 files → `lucide-vue-next`. Every chrome icon at 16 or 24. Emoji-as-icons (`✕`, `✓`, `🎤`, `📎`, `↻`, `↑`, `×`, `☀`/`☾`) swept across the chrome. AppLogo wordmark and the GraphView D3 mount kept as the legitimate exceptions. |
**Cross-cutting changes folded in as the work touched files:**
- Long-form 1.7 line-height on `.prose` (PR 4) — applies to Note viewer, Task viewer, chat assistant bubbles, anywhere markdown renders into a reading surface.
- Two-weights-only (400 + 500) — every `font-weight: 600` and `700` snapped to `500` across all surface PRs.
- Hardcoded `--color-danger` in destructive button contexts → `--color-action-destructive` (Oxblood). `--color-danger` (Error terracotta) preserved for validation/error messages, per the doc's distinction between Error and Destructive.
- Adjacent `×` / `✕` / unicode-arrow emoji swept opportunistically as files were touched (PRs 5, 7).
### Out of scope — deferred indefinitely
Items deliberately not addressed in this round; revisit when a real need surfaces:
- Lucide stroke-weight overrides (doc spec: 1.5 at 24, 1 at 16; current: Lucide default 2). Touched components if they read too heavy in practice.
- Filled-as-active icon state — no current affordance uses it; introduce when bookmark/pin/star toggles are added.
- Token rename to `--fs-*` namespace — Scribe is the only FabledSword app sharing this codebase.
- FabledSword lockup placement — waiting on the actual heraldic mark to be drawn.
- Standalone voice/tone audit across every UI string — opportunistic-only; full sweep deferred unless drift becomes visible.
- A handful of editor utility buttons (`.btn-suggest-tags`, `.btn-link-all`, AI assist generate/proofread/accept/reject set, etc.) — currently ghost-styled and visually compliant; revisited only if they read off in practice.
### Open threads
*New threads will accumulate here as gaps surface in real use.*
- **Accent footprint.** Dusty violet marks Grimoire's own moments: the wordmark, active navigation, the editor's cursor and selection, tags and wikilinks, in-progress status, focus rings, and the call to action that starts something new (a first note, say). Operating the software uses the action colours: Moss to save or confirm, Bronze for alternatives, Oxblood with an icon to delete, Pewter ghost for tertiary actions.
- **Light and dark.** Both are supported. Light mode is warm parchment (`#F5F1E8` page, `#FBF8F0` cards, `#14171A` text) rather than white, with a cool wash on code blocks so syntax colours stay readable. The accent, action and semantic colours are the same in both.
- **Status and priority.** Status: todo Pewter, in progress the accent, done Moss, cancelled Ash, paused Warning. Priority runs cool to warm: low Info, medium Warning, high Error. Overdue uses Error. Each has a low-alpha `-bg` companion.
- **Type.** Inter for UI and body, Fraunces at 18px and up (display, H1, H2), JetBrains Mono for code. Two weights only, 400 and 500. Line height 1.5 for body, 1.3 for headings, 1.7 on reading surfaces such as a note.
- **Icons.** Lucide, at 16px inline and 24px for navigation and primary actions; no emoji in the interface.
- **Borders are structural.** Cards, dialogs, code blocks, focus rings and major dividers get borders; list rows, chips and empty states are separated by surface tint and spacing instead.
All development is Docker-based. Do not install Python or Node dependencies locally.
## Running it
```bash
# Start the full stack (app + PostgreSQL + Ollama)
docker compose up --build
docker compose -f docker-compose.dev.yml up --build # app + PostgreSQL at http://localhost:5000
docker compose -f docker-compose.dev.yml up --build app # rebuild after a change
docker compose -f docker-compose.dev.yml down -v # stop and wipe the database
```
# Rebuild after backend changes (frontend changes require rebuild too)
docker compose up --build app
The image builds the frontend, so a frontend change needs a rebuild too. For quicker frontend work, run Vite against the running app; it proxies `/api` to port 5000:
# Reset everything (wipes database)
docker compose down -v && docker compose up --build
```bash
cd frontend && npm ci && npm run dev # http://localhost:5173
```
# Run checks (lint, format, typecheck, tests)
make check
## Checks
# Individual checks
CI runs these on every push; locally they need `uv sync --extra dev` and `cd frontend && npm ci`.
```bash
make lint # ruff check src/
make fmt # ruff format src/
make typecheck # vue-tsc --noEmit
make test# pytest tests/
make check # all three
```
## Frontend Hot Reload
Unit tests run without a database. Tests marked `integration` need a real PostgreSQL and run in CI's integration job, against a schema built by the real migrations.
The Docker setup does not include Vite's hot-reload dev server. After frontend changes, rebuild the image. For faster iteration during active frontend work, you can run Vite locally:
## Database migrations
The container runs `alembic upgrade head` on every start, after waiting for the database to accept connections.
```bash
cd frontend
npm install
npm run dev # Vite dev server at http://localhost:5173
Point the Vite dev server at the backend by setting `VITE_API_BASE_URL=http://localhost:5000` (or configure `vite.config.ts` proxy).
- Number revisions sequentially (`0123_...`). Revision ids must fit in 32 characters, the width of `alembic_version.version_num`.
- Write both `upgrade()` and `downgrade()`.
- Adding a value to a `CHECK`-constrained column means dropping and re-adding the constraint in the same migration.
- Keep migrations to schema changes. Filling a derived table from a large one belongs in a background backfill after startup, where its cost doesn't hold up the boot.
- A migration that depends on application code (a key derivation, a serialiser) copies what it needs rather than importing it, so a later edit can't change what an old migration does.
## Database Migrations
## The plugin
Alembic migrations run automatically on container startup (`alembic upgrade head` in the `CMD`).
`plugin/` ships straight from git: installs fetch it through `.claude-plugin/marketplace.json`, and there is no build step. After changing anything under `plugin/` or `.claude-plugin/`, run `make mint-plugin` before committing. Claude Code only refreshes an installed plugin when its version changes, and CI fails a plugin change that didn't bump it.
The hooks must run on a bare machine: POSIX shell, awk, sed and the usual coreutils, plus `git` and `curl`. They fail open, so a slow or unreachable Grimoire never blocks Claude Code.
Migration conventions:
- Use raw SQL with `IF NOT EXISTS` guards for idempotency
- Use `DO $$ BEGIN CREATE TYPE … EXCEPTION WHEN duplicate_object THEN NULL; END $$` for enum types
- Number migrations sequentially (e.g. `0027_add_something.py`)
- Always provide both `upgrade()` and `downgrade()`
## CI and releases
## CI/CD
CI runs on Gitea Actions (workflow in `.forgejo/workflows/ci.yml`) in the shared `ci-python:3.14` image ([`ci-requirements.md`](../ci-requirements.md) lists what it relies on). Six jobs: TypeScript typecheck, plugin hooks, Python lint (plus design-token and dangling-style checks), unit tests, integration tests, and the image build, which waits on all of them except the plugin job.
### Pipeline
| Push | Image tags |
|------|-----------|
| `dev` | `:dev`, `:<sha>` |
| `main` | `:latest`, `:<sha>` |
| a `v*` tag | `:latest`, `:<version>`, `:<sha>` |
`:<sha>` never moves, so it is the rollback point. Images publish to `git.fabledsword.com/bvandeusen/fabledgrimoire`, with a registry layer cache at `:cache`. The build needs the `REGISTRY_USER` and `REGISTRY_TOKEN` (a token with `write:packages`) repository secrets.
| Trigger | Jobs | Docker tags pushed |
|---------|------|--------------------|
| Push to `dev` | typecheck + lint + test → build | `:dev`, `:<sha>` |
| Tag `v*` on `main` | typecheck + lint + test → build | `:latest`, `:<version>`, `:<sha>` |
| Push to `main` | typecheck + lint + test | (no build) |
Work happens on `dev`. A release merges `dev` into `main` with a plain merge commit; a dated `vYYYY.MM.DD.HHMM` tag, matching the version the image reports, can mark it.
### Release Process
1. Work on `dev` branch — CI validates on every push
2. When ready, open a PR from `dev` → `main` in Forgejo
3. Merge the PR
4. Create a release via the Forgejo UI on `main` with a `v*` tag (e.g. `v26.03.23.1` — CalVer: `YY.MM.DD.N`)
5. The tag push triggers CI → build job pushes `:latest` + `:<version>` Docker images
6. After merging to main, sync dev back:
```bash
git checkout dev && git merge main && git push origin dev
```
### Runner
CI jobs schedule against the `python-ci` runner label and run inside the
shared `git.fabledsword.com/bvandeusen/ci-python:3.14` image (see
`ci-requirements.md` for what this project relies on from the image).
The runner deployment lives outside this repo; image bumps happen in
[CI-Runner](https://git.fabledsword.com/bvandeusen/CI-runner) via Renovate.
`infra/runner-compose.yml` + `infra/act-runner-config.yml` document the
runner-host deployment shape; the source of truth is the deployed
config on the runner host.
### Docker Registry
Images pushed to: `git.fabledsword.com/bvandeusen/scribe`
**Important:** Do NOT use `op.create_table()` or `sa.Enum()` — SQLAlchemy's event system can fire `CREATE TYPE` even with `create_type=False`, causing failures on re-run. Always use raw SQL with `IF NOT EXISTS` / `DO $$ BEGIN ... EXCEPTION WHEN duplicate_object` guards.
## Project Conventions
## Conventions
### Backend
- Services: `async with async_session() as session:` — import from `scribe.models`
- No `scribe.database` module
- Blueprint per resource: `routes/notes.py`, `routes/tasks.py`, etc.
- All business logic in `services/`; routes are thin wrappers
- Permission checks via `services/access.py` — never inline ownership checks in routes
- Services open their own session: `async with async_session() as session:` (from `grimoire.models`).
- Routes and MCP tools stay thin; the logic lives in `services/`, shared by both.
- Access checks go through `services/access.py`, never inline ownership checks.
- Every wait on something outside the process has a deadline.
### Frontend
- API calls via `frontend/src/api/client.ts` typed helpers (`apiGet`, `apiPost`, `apiPatch`, `apiDelete`)
- Pinia stores for shared state; local `ref()` for component-only state
- Composables in `composables/` for reusable behaviour (autosave, keyboard nav, tag suggestions, …)
- Views are page-level components in `views/`; reusable UI in `components/`
- API calls go through the typed helpers in `frontend/src/api/client.ts`.
- Pinia stores for shared state, `ref()` for state local to a component; composables for reusable behaviour.
- Colours, spacing and type come from the design system's tokens in `theme.css`, which is generated, so CI rejects an unknown `var()`.
### Commit Style
### Commits
```
type(scope): short description
type(scope): what the change does
Longer body if needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Why, and anything a reader tracing this change later needs.
Tests are in `tests/`. They run against a real PostgreSQL instance in CI (not mocked). Keep tests integration-style where possible — mock failures have historically masked real migration bugs.
Types: `feat`, `fix`, `refactor`, `docs`, `chore`, `test`. Comments in the code describe how it works today; the history of a change belongs in its commit message.
Scribe is a self-hosted work system-of-record for software projects, built to be
driven by Claude Code. There is **no in-app LLM** — Claude is the sole assistant,
reaching Scribe through a built-in MCP endpoint and a bundled Claude Code plugin. The
web UI is a clean surface for humans to read and edit the same data.
Grimoire is a self-hosted systemofrecord for the work you do with Claude Code. Claude records what it learned, decided and built, and that record reaches the next session, so a solution is recalled rather than reinvented. There is no language model inside Grimoire: Claude reaches it through the MCP endpoint and the Claude Code plugin, and the web UI is where people read and edit the same records.
## Notes
Write in Markdown with a live-preview editor (Tiptap/ProseMirror). Headings, bold,
italic, lists, code blocks, and task checklists render inline. A slash-command menu
(`/`) inserts common blocks.
A Markdown editor (Tiptap) with headings, lists, code blocks and checklists inline, and a `/` menu for blocks.
- **Wikilinks** — Link notes with`[[Title]]` or `[[Title|Display Text]]`. Clicking
navigates to (or auto-creates) the referenced note; the editor suggests existing
titles while you type `[[`. Backlinks appear in the note viewer sidebar.
- **Tags** — First-class `ARRAY[text]` column with autocomplete. Hierarchical tags
(`area/backend`) supported — filtering by `area` matches all `area/*` children.
- **Version history** — Every body edit snapshots a version (up to 20 per note).
Browse, diff, and restore from the editor's History panel.
- **Draft recovery** — In-progress edits persist across page loads and are restored
when you reopen a note.
- **Convert freely** — Turn a note into a task (sets `status=todo`) or back again.
- **Wikilinks**:`[[Title]]` or `[[Title|text]]`, suggested as you type; backlinks show in the viewer.
- **Tags**, with autocomplete. Hierarchical tags (`area/backend`) match their parent's filter.
- **Version history**: browse, diff and restore earlier versions of a note.
- **Draft recovery**: unsaved edits survive a reload.
## Tasks and Issues
## Tasks
Tasks carry status (`todo` →`in_progress` →`done`/`cancelled`), priority
(`none`/`low`/`medium`/`high`), due date, milestone assignment, and a parent task
(sub-tasks). Notes and tasks share one model — a task is a note with a status.
A task is a note with a status (`todo`,`in_progress`,`done`, `cancelled`), plus priority, due date, milestone and parent task.
- **Work logs** — Append timestamped progress entries (with optional duration) to a
task without rewriting its body; shown chronologically in the task view.
- **Issues** — A task whose `kind` is corrective: a problem you fixed or are fixing,
with symptom → root cause → fix in the body. An issue can link the task it arose
from and the System(s) it touches.
- **Recurring tasks** — An interval or calendar recurrence rule spawns the next
occurrence when a task is completed (a background job drains due spawns).
- **Sub-tasks** — Any task can have children via `parent_id`; the viewer shows them
inline.
- **Kinds**: `work`; `issue`, for corrective work, which can link the task it arose from; and `spike`, a time-boxed investigation whose output is an answer.
- **Work logs**: timestamped entries that record progress without rewriting the body. They are what the next session reads first.
- **Claims**: a session that logs on a task holds it, so after a context compaction the work comes back to that session, and other sessions can see who is on it.
- **Recurring tasks** spawn their next occurrence on a schedule.
## Projects and Milestones
## Projects, milestones and systems
- **Projects** — Group related notes and tasks. Title, description, goal, status
(`active`/`paused`/`completed`/`archived`), and a colour.
- **Milestones** — Ordered stages within a project. A milestone is also the home of a
**plan** — its body holds the design (Goal/Approach/Verification) and its child
tasks are the steps. Completion percentage is shown on the project page.
- **Kanban view** — `/projects/:id` groups tasks by milestone in a column layout with
status-advance buttons on the cards.
- **Projects** carry a goal, a status and a colour, and group everything else. Entering a project gives Claude a short primer: its goal, recent plans and their next steps, open tasks, its systems and its design system.
- **Milestones are plans**: a body holding the design (goal, approach, verification) and ordered step tasks. Starting a plan that matches an active milestone hands back that milestone instead of opening a second one.
- **Systems** are a project's named areas ("auth", "retrieval"). Records are tagged with them, each has a charter saying what belongs there, and each maps to a canonical area shared across projects.
- **Duplicate gates**: creating a record that reads like an existing one returns the existing one to update instead.
## Systems
## Rules, lessons and moments
A **System** is a per-project, reusable, self-describing subsystem or area (e.g.
"auth", "billing"). Associate any note, task, or issue with a System so research,
build-work, and fixes for the same area line up and recurring problem-spots surface.
- **Rulebooks → topics → rules.** A rule in a rulebook applies in every project; a project rule applies only there. A project that departs from a global rule says so with an `overrides` relation.
- **Rules arrive by retrieval.** Nothing is preloaded: a rule reaches a session when what Claude is about to do matches its trigger (`when_to_apply`), whether that's the operator's message, a command or a file being written.
- **Moments**: a rule about *when* something is done (finishing a task, reporting back, pushing) can be mounted on a moment of work, and arrives whenever that moment happens, whatever the words look like. Settings → General shows which actions reach each moment.
- **Versions and verification**: every edit to a rule keeps the text it replaced. A rule that asserts a fact about someone else's software can say how to check it and when it stops being true, and a sweep lists rules due for a check.
- **Preferences** are rules with the force of guidance rather than obligation.
- **Lessons** record something learned the hard way, keyed to the situation where it applies, and link to the rules they are instances of.
- **Reply shapes**: the default shape of Claude's replies (completion report, plan for review, question) arrives at the moment it applies. Settings → General shows them.
## Rules and Rulebooks
## Snippets and the shape ledger
Scribe stores the operator's engineering and workflow **rules** so Claude follows them
across sessions.
- **Snippets** are recorded prior art: a reusable piece of code with when to reach for it, its signature and its canonical location in a repo. Claude searches them before building and is shown matching ones before it writes a file.
- **Forge integration**: with a git forge connected, Grimoire checks a snippet's cached body against its source and marks it current or diverged. Push webhooks keep that fresh.
- **The shape ledger**: Grimoire extracts the code shapes from a bound repo and tracks how each is accounted for: an instance of a snippet, a deliberate variant with its reason, or a one-off. Coverage shows what is still unclassified.
- **Rulebooks → topics → rules** — Rules are grouped by topic inside a rulebook.
- **Always-on rules** — A rulebook can be flagged always-on; its rules load at the
start of every session through the plugin's push channel.
- **Per-project scope** — A project subscribes to rulebooks, and can add
project-scoped rules or suppress individual inherited rules/topics.
## Family canon
## Stored Processes
Projects on the same platform (Android apps, Go services, ...) share good ideas. An idea promoted to the family's canon reaches every project on that platform, and each answers it: adopted, a variant with its reason, or owed (filed as a task). `/family` shows the ideas and every project's answers.
Reusable saved prompts (a note with `note_type=process`) — e.g. a drift-audit or a
DRY pass. The bundled plugin syncs each Process into a local Claude Code skill stub
(`/scribe:sync`) that auto-surfaces by relevance and fetches the live procedure on
demand.
## Design systems
## Search and Knowledge Injection
Tokens (colour, type, space, motion, ...) and written guidance, with inheritance: a project's design system can extend a house style and record where it departs. Grimoire generates a stylesheet from it, checks code snippets against it, and gives Claude the values to use before it writes UI. `/design-systems` edits them.
- **Semantic search** — pgvector-backed similarity search over notes and tasks
(in-process `fastembed` embeddings; no external model).
- **Proactive knowledge-injection** — the plugin's `UserPromptSubmit` hook surfaces a
short, high-confidence menu of maybe-relevant note *titles* into Claude's context
each turn; Claude pulls a full body only when it judges it relevant. Gated so it
stays quiet on most turns and never repeats within a session.
## Stored processes
## Knowledge Graph
Saved procedures, such as a drift audit or a DRY pass. The plugin syncs each one into a Claude Code skill (`/grimoire:sync`), which fetches the live procedure when it is invoked.
`/graph` renders notes, tasks, and tags as a D3 force-directed graph. Tag nodes
controls (repulsion, link distance/strength, hub pull, gravity); click a node to peek,
click a tag to filter.
## Search and context injection
- **Semantic search** over notes, tasks, rules, milestones and systems, with embeddings computed in-process (no external model service). It falls back to keyword search if the embedder is unavailable.
- **Context injection**: the plugin's hooks show Claude a short menu of possibly relevant records at the moments they matter: rules and prior art when a prompt arrives, before a command or a file write, and when a moment of work is reached. Claude opens a full record only when it judges it relevant.
- **Tuning**: each surface has its own thresholds and budget, in Settings → General. Every change is logged with its reason, and retrieval telemetry shows what was surfaced and what was opened.
## Claude via MCP and the plugin
The whole store is reachable by Claude through a built-in **MCP endpoint at `/mcp`**
(Bearer-auth with an API key). The **Scribe Claude Code plugin** (shipped in this
repo) wires it up:
The whole store is reachable through the MCP endpoint at `/mcp`, authenticated with an API key. The Claude Code plugin in this repo adds:
-a `SessionStart` hook that injects the operator's always-on rules + active-project
context so Scribe surfaces without being asked (fail-open if Scribe is unreachable);
brainstorming — that route their output into Scribe;
- your saved Processes auto-surfaced as skills.
-hooks for session start, each prompt, before and after tool calls, the end of a turn, compaction and session end, all failing open if Grimoire is unreachable;
- skills for the working practices: using Grimoire, writing plans, systematic debugging, verification, brainstorming, reporting back, reusing code, shape accounting and family canon;
-your stored processes as skills.
See [API Keys & MCP](api-keys-and-mcp.md).
## Sharing and Collaboration
## The web UI
- **Share** — Share any project, note, or task with users or groups at
`viewer`/`editor`/`admin` levels from the viewer/project toolbar.
- **Groups** — Admins create platform-wide groups and assign `member`/`owner` roles;
share a resource with a group in one action.
- **Shared with me** — `/shared` lists incoming shares with permission badges.
- **Notifications** — An in-app bell (unread count, polled) fires when a project or
note is shared with you or you're added to a group.
- **Dashboard**: what to work on next. **Knowledge** browses everything.
- **Graph** (`/graph`): notes, tasks and tags as a force-directed graph, with physics controls and filtering by tag.
- **Trash**: deleted records wait here until retention purges them, and can be restored.
- **Installable** as a desktop or mobile app.
Every read and mutation is scoped by owner + direct shares + group shares.
## Sharing
## Data Export and Backup
Share a project, note or task with users or groups, as viewer, editor or admin. Admins manage groups. `/shared` lists what has been shared with you, and an in-app bell notifies you of new shares. Every read and write is checked against ownership and shares.
- **Personal export** — download all your notes/tasks as a Markdown ZIP (YAML
frontmatter) or a JSON array.
- **Admin backup** — full application backup/restore (format v4) with ID remapping on
restore for cross-instance migration.
## Export and backup
## Progressive Web App
- **Personal export**: your notes and tasks as a Markdown ZIP (with YAML frontmatter) or JSON.
- **Admin backup and restore**, with ids remapped so a backup restores onto another instance.
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. `python -m grimoire.restore` brings a fresh instance back from one. See [Backups](backups.md).
Installable as a desktop or mobile app; a service worker caches the shell.
that links to a matching local account. Invite links, a registration toggle, password
reset, and session invalidation are included. See [SSO / OAuth](sso-oauth.md).
Email and password, plus optional OIDC single sign-on (Authentik, Keycloak, ...) that links to a matching local account. Invite links, password reset and signing out other sessions are included. See [SSO / OAuth](sso-oauth.md).
Add the following environment variables to the `app` service in `docker-compose.yml`:
@@ -64,7 +64,7 @@ docker compose up --build -d
## 4. Account Linking
When a user logs in via OAuth for the first time, Fabled checks in this order:
When a user logs in via OAuth for the first time, Grimoire checks in this order:
1. **Existing OAuth sub** — returns that user immediately.
2. **Matching email** — if a local account already exists with the same email, the OAuth identity is linked to it automatically. The user retains all their notes and tasks.
@@ -92,4 +92,4 @@ The backend will reject any `POST /api/auth/login` or `POST /api/auth/register`
The OIDC discovery endpoint (`<issuer>/.well-known/openid-configuration`) must be publicly reachable from the Fabled container (server-to-server call at login time).
The OIDC discovery endpoint (`<issuer>/.well-known/openid-configuration`) must be publicly reachable from the Grimoire container (server-to-server call at login time).
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
bvandeusen
