classify_shapes(repo=...) and the write-path hook looked only at live rows,
so a shape whose row had vanished got a second provisional row under the
same (project, repo, path, symbol, kind). The unique constraint refused it,
and the whole classify batch was lost as a tool error.
provisional_row() now finds the row under that identity, vanished or not.
A vanished row is revived with a reappeared event and keeps its judgment.
A new row is inserted inside a savepoint, so a concurrent insert of the
same identity falls back to that row instead of failing the transaction.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Configuration, the README env table and the features page say Settings
-> Config -> Reverse proxy is where the count is checked and set,
that a saved count wins over TRUSTED_PROXY_HOPS, that Cloudflare in
front of your own proxy is 2, and what a too-low count does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The section opens with a verdict on the admin's own request: the
address Grimoire reads them as, or, through Cloudflare, the count to set
with a button that sets it. Below it are the count with where it came
from, a table of what counts 0-3 would read, and the raw forwarding
chain. A saved count can be cleared back to the environment's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An admin can save how many reverse proxies stand in front of Grimoire.
A saved count wins over TRUSTED_PROXY_HOPS, which stays the starting
value. Each worker loads it at startup (under a deadline) and re-reads
it every 30s; the worker that saves applies it at once.
GET/PUT /api/admin/proxy returns the count, where it came from, and a
readout of the admin's own request: the socket peer, the forwarding
chains, what counts 0-3 would read, and the count that reads
Cloudflare's visitor when Cloudflare's headers are present. Saves are
audit-logged. A "too many attempts" reply now says where to look when
everyone is being blocked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
using-grimoire, which owns cross-project conduct, now says the one change
to make toward another project is filing an issue with how it was found
and the evidence, and that the plugin asks before anything more. The
features page describes the fence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The project a session entered now lives in grimoire-fence, which the
compaction clear never visits, and session end removes it. Arrays of ids
in a tool's input were sent as [,"3","4"] under mawk, which creates an
array element as soon as it is assigned to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
plugin/hooks/grimoire_project_fence.sh runs as PreToolUse on editor writes,
Bash and Grimoire's tools. A session belongs to the project its starting
directory resolves to (CLAUDE_PROJECT_DIR: a .grimoire marker, else the git
remote's binding). A session that starts outside any project belongs to the
first one it enters, which PostToolUse on enter_project records.
These acts stop on permissionDecision "ask", with the server's reason naming
the other project:
- an edit in another project's repo or a worktree of it;
- a git commit, push, merge, rebase, pull, cherry-pick, revert, tag, am, reset
or worktree add there, found through git -C, a cd before it, or the shell's
current directory;
- a Grimoire write that lands in another project.
A live probe showed "ask" holds in bypassPermissions. Filing an issue and
every read go ahead. An edit or git command in the session's own repo, and any
tool named like a read, never calls the server. Grimoire writes send only
their short arguments, so the URL stays small.
When the instance doesn't answer, the act goes ahead and the hook says once
per outage that it couldn't check. check_plugin.py now holds one outage line
per hook and has a smoke event for the fence.
GET /api/plugin/write-scope takes fallback_project_id for the entered project.
tests/test_project_fence.py drives the hook against real git repos and a
stand-in server. tests/test_write_scope.py holds the hook's
FENCE_READ_PREFIXES equal to services/write_scope.READ_PREFIXES. The plugin
README and PACKAGING describe the fence, and the plugin is minted to
2026.11.05.0245.
Milestone 583, step 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guard caught seven write-tool arguments with a locating name that are not
where the write lands:
- snippet_id on classify_shapes_by_rule and confirm_shape_proposals names the
canon, not the rows written;
- rule_outcome, rule_misfired and judge_lesson_link record judgments.
REFERENCES lists each one with its reason, and the guard reads it.
READ_PREFIXES names the tool prefixes the fence answers without the server,
and a test holds every write tool to a name outside them.
plugin.json is minted to 2026.11.05.0242 for the PACKAGING.md row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /api/plugin/write-scope answers the plugin's fence. The session's project
comes from repo / project_id, as every plugin endpoint reads it. The act is
either:
- a directory (target_repo / target_project_id), for a file edit or a git
command; or
- an MCP call (tool, plus input: the call's short arguments).
It returns allowed, the other projects the act would change, and the reason
the operator is asked. It is a GET so a read-scoped key still fences file
edits and git.
services/write_scope.WRITE_TARGETS lists every write tool with the arguments
that locate its write: a project id, or a task, note, milestone, System, rule,
relation, trash batch, repo or family decision, each traced to its project. A
write that lands in no project (install settings, user-level records, the
judgments a session is asked for anywhere) has an empty entry. The one
cross-project write allowed is create_task(kind="issue") as a new, unplanned,
open issue. A project the caller cannot read is never named.
tests/test_write_scope.py:
- requires an entry for every write tool;
- requires every declared argument to exist on the tool;
- refuses a locating argument (project_id, task_id, milestone_id and the
rest) that a write tool takes without declaring it.
The integration test traces each locator from real rows.
Milestone 583, step 2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScrollSentinel sits at the foot of a server-paged list and asks for the
next page a screen before the reader reaches it. It asks once per cursor,
forgets the cursor when its element goes, and shows Try again on a failed
page instead of retrying in a loop.
FamilyView's decision log loses its "Older decisions" button. A failed
page shows in place rather than as a toast, and rows already shown are
skipped when a new decision shifts the offset.
KnowledgeView drops its inline IntersectionObserver for the component.
The cursor is the ids consumed, not the items rendered, so a batch that
returns fewer records than it asked for still moves the feed on. A failed
batch puts its ids back and offers Try again, where it used to fail
silently and stop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two integration tests in test_integration_backup_rollback.py drive
grimoire.restore.run on the scheduled backup's tar, written from a full
export of this instance:
- without --yes it exits 4 and nothing changes;
- with --yes it takes the safety backup, then puts back the edited and the
deleted task and removes the new one.
The file-restore test that expected a refusal "because the database has
accounts" now names the reason it actually refuses: on a live instance, a
backup that records no instance id can't be rolled back to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
python -m grimoire.restore <file> prints what the restore would change (the
same preview as Settings) and stops. With --yes it runs
backup_job.rollback_to_backup: the current state is saved first, then the
whole instance goes back to the backup. An empty instance takes any full
backup. A live one only takes a backup of its own, and otherwise exits 3 with
the reason. A v1 backup keeps the copy-in path into an empty instance.
docs/backups.md Restoring is rewritten:
- what a restore puts back and removes;
- the same-instance rule;
- the safety backup;
- what survives;
- the web and command-line ways in.
Its S3 key advice now asks for read as well as write when restoring from the
bucket. README, features, architecture and the API reference follow.
Milestone 582, step 5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The web restore no longer copies a backup in beside the live data.
- GET /api/admin/restore/sources lists the backups at the backup destination,
newest first.
- POST /api/admin/restore/preview says what a restore would do, changing
nothing: fresh or rollback, when the backup was taken, and counts now and
after per kind. A backup that cannot apply here gets a 409 with the reason.
- POST /api/admin/restore?confirm=rollback runs backup_job.rollback_to_backup:
safety backup, replace, then search rebuilt in the background.
Each takes ?source=<name> (a backup at the destination) or an upload. The
destination path exists because uploads are held to the request-size limit,
and a real instance's backup is far over it. backup_s3.get_file downloads
from a bucket. backup_destination.list_backups/open_backup only accept names
the scheduled backup writes, so a request cannot reach any other file.
Settings -> Data -> Restore from Backup opens RestoreBackupDialog. The dialog:
1. Pick a backup at the destination or a file.
2. Read the preview.
3. Type "roll back".
4. Run it. The dialog then says where the replaced state was saved, and sends
the user to sign in again.
Closes#6840. Milestone 582, step 4.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backup_job.rollback_to_backup:
1. Checks the backup can apply here (backup.check_replace, which shares
_replace_mode with replace_from_backup), refusing before anything is written.
2. Takes the backup lock, so no scheduled backup runs during the rollback.
3. Saves the current store with save_state_before_restore.
4. Replaces the store.
The safety file goes to the backup destination when backups are on and it
takes it. Otherwise it goes to DATA_DIR/pre-restore, with a note when the
destination refused. If neither takes it the rollback does not run. The
result names where the file went.
Milestone 582, step 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One transaction:
- Refuse unless the backup is a full one and, on a live instance, carries
this instance's id. A fresh instance (no records, at most the restoring
account) takes any full backup and adopts its id.
- Clear first: search vectors and the CSS consumer map (derived); every
backed-up table nothing outside the backup points at, whole; the settings
the backup carries; and the identity-table rows the backup lacks. A share
granted by a removed user goes before the user.
- Write the backup through the same _build_* helpers as the copy-in restore,
now _load_v2(keep_ids=...). Identity tables (users, projects, milestones,
notes, rulebooks, topics, rules, systems, design systems, code shapes,
family decisions) go through _place. That keeps the backup's id and
merges, so API keys, shares and forge connections of a surviving row stay.
- Reset id sequences past the restored max, and sign every session out.
reindex_after_restore runs the four embedding backfills afterwards. A unit
guard checks that no row kept for outside references cascades from a table
replaced whole. Integration tests cover edits, deletions and new rows put
back, a surviving API key, sessions ending, fresh ids afterwards, foreign or
id-less backups refused, a user export refused, a failure part-way changing
nothing, and a fresh instance adopting the id.
Milestone 582, step 2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
instance_identity is one row made by migration 0127 with a random id, so an
instance has it before its first account. A full backup carries it in its
header as instance_id (the tar manifest header and the JSON top level);
BACKUP_VERSION is 28. The table rides in _NOT_INCLUDED: a restore decides from
the header value rather than copying a row.
Two stores with the same id number their rows the same way. That is what lets
a later step roll a live instance back by original id, refuse a backup whose
ids would land on unrelated rows, and have a fresh-instance restore adopt the
backup's id (adopt_instance_id).
Milestone 582, step 1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Quart takes remote_addr from a Remote-Addr header its ASGI layer writes from
the scope (replacing any the client sent). A test request context does not
read scope_base for it, so the two socket-peer tests saw an empty address.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
grimoire/proxy.py is the one reader of X-Forwarded-For and X-Forwarded-Proto.
It takes the entry TRUSTED_PROXY_HOPS in from the right (default 1; a shorter
header reads its leftmost entry; 0 trusts no proxy), so an entry a client puts
left of the proxy's is never believed. Before this, with TRUST_PROXY_HEADERS
on, the leftmost X-Forwarded-For entry keyed the login, register, forgot and
reset rate limits and the audit log, and a client could choose it.
The session cookie is Secure, and HSTS (max-age=31536000) is sent, exactly
when the request is HTTPS by the trusted proxy's word. Plain HTTP is
untouched and nothing redirects. SECURE_COOKIES is gone (a startup warning says
so when it is still set); TRUST_PROXY_HEADERS=false still maps to 0 hops, with
a warning to move to TRUSTED_PROXY_HOPS. Admin audit lines now use the same
client address instead of the socket peer.
The placeholder-key refusal keyed on SECURE_COOKIES, so it now refuses either
placeholder key outright; the instance makes its own key when none is set.
Family idea #5004 (IsHTTPS, trusted-hop X-Forwarded-Proto), canon v2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A by-name reference edge could not tell two definitions of one name apart:
in a Go repo that defines `file` as importRun.file (internal/files, the
canon) and as sftpRun.file (internal/replication), a mirror test calling
`m.file(` was recorded as a use of the canon.
names_canon now takes defined_at, every repo path that defines each name,
in place of the shape file's own names. The own-file rule is unchanged.
When the repo also defines the canon's name somewhere the canon is not
recorded, a mention counts only from the canon's own package: the
directory of a file location exactly, or under a directory location.
A name defined once resolves as before. Proposer version 8 -> 9, so the
next coverage refresh re-reads every row and replaces its reference edges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read against Fabled Archives after the cleanup: of the 9 rows still linked
to snippet #5448 (symbol `file`), all but one matched prose in a comment,
"a file (not a directory)", "the file (ScanWaiting". The only real call
was `r.file(ctx, ...)`. Formatted code never puts a space before a call's
parenthesis and prose does, so the plain-word test is now `word(` with no
whitespace. _PROPOSER_VERSION 8 re-reads every row once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Migration 0125 removed the proposer's cross-project `reference` edges, but
the write path records its by-name evidence with basis `hook`, so those
survived it. A hook edge across projects is impossible for the same reason a
reference edge is, and the write path now applies the same by-name test
(names_canon), so 0126 removes the ones written before. Judged edges keep
their own basis and are untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The proposer's reference basis, and every uses edge it writes, counted any
word-bounded mention of a canon's symbol. Three holes made that wrong at
scale (snippet #5448, symbol `file`, held 2,161 uses edges):
- Go, Rust, Kotlin, Swift, Dart, C, C#, Ruby and PHP were outside the
language-family maps, so their canons read as "unknown, no gate" and were
matched in every Python, TS and CSS body.
- Canons in other projects were matched by name, though one repo never
calls a function defined in another's.
- A plain word (`file`, `seen`) or a module's own helper (`_run`) matched
wherever it appeared.
names_canon is now the one by-name test, used by match_canon,
reference_canons and the write-path hint: same project, not a name the file
defines itself (unless the canon is recorded there), no generic verb with or
without a leading underscore, and a bare lowercase word only where it is
called. The write path keeps the payload's own names in play, since a
redefinition of a pulled canon is the copy it exists to flag.
Repair: record_uses only adds, so replace_reference_uses now makes a row's
machine-written reference edges exactly what the current rule finds,
leaving judged and hook edges alone. _PROPOSER_VERSION 7 re-reads every row
on its project's next refresh. Migration 0125 deletes the cross-project
reference edges already stored.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HistoryPanel, InlineAssistPanel, NoteCard, WorkspaceNoteEditor and
WorkspaceTaskPanel had no importer, lazy import or global registration
anywhere in frontend/src. The features they served were replaced; token
sweeps kept restyling them because they match a glob. The two doc comments
that named HistoryPanel.vue now name VersionHistorySection.vue, the note
history that is live.
SearchBar.vue stays: it is the recorded home of snippet #2961 "SearchBar -
the search input recipe", and the two live search boxes are judged variants
of it. Deleting it would leave the recipe pointing at nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The signal counted every get_rule open against every moment from the
previous three minutes, so a rule a search arm had just shown was
proposed for whatever moment happened to fire. All nine pending signal
proposals were wrong.
- The hook skips an open of a rule a hook named this session, and sends
the last three acts from 90 seconds instead of 30 from 180.
- A rule already mounted anywhere gathers no signal evidence.
- A signal pair keeps the actions that reached its moment and shows them
as its why.
- The proposal line says it is co-occurrence, not a recommendation, and
asks the reader to check the rule against the moment before offering
it; moments.md says the same.
Plugin 2026.10.26.2029.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The write hooks recorded at most 12 definitions per write (alphabetical,
so test functions were the ones dropped) and the post-Bash hook stopped
after 4 files, so the Stop hook never asked about the rest. The caps
bound the prior-art query; the written ledger now gets the full list,
up to 40 files per Bash call, while the query stays at 4 files x 12 names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The derive check matches by name, so a renamed copy of a recorded snippet
passed it and showed only as "similar" prior art. When a write introduces a
definition the file does not hold (shape_ledger.new_shapes_at) and an unseen
snippet on the menu clears the write-path floor by WRITEPATH_COPY_MARGIN (0.80
at the default floor), the hint now says to reuse it or extract the shared
shape before another copy. Names the derive check or the stamping feed already
spoke for are left to them. Covers recorded snippets only: the ledger keeps no
bodies to compare an unrecorded sibling against.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 4xx with no S3 error body came from something else at that address, most
often a web server on the default port. The message now says so and to check
the scheme and port, beside the raw status. A 5xx without a body keeps its
own reason, since a busy S3 service can answer that way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pending() listed every suggested signal row, including pairs still gathering
evidence below the three-situation bar, so the queue held 200 "proposals"
where about 25 had crossed it. A signal pair now needs proposed_at in its
evidence, the same test the misfire half already applies; proposals from a
pass are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each moment gains by_rule: {rule_id, title, delivered, opened}, read from
the (moment, rule) grouping the readout already did. The moment's totals are
now summed from those rows, so the two cannot disagree. The Moments settings
page shows the list under each moment, closed by default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
surfaced_never_pulled told a rules reader there was no judged sample for the
rule arms. There is one now; the guard tying the text to REVIEWABLE caught it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule arm logs its query, floor, budget and project, which is everything
semantic_search_rules takes, so a logged call is re-run to a depth past its
budget and judged like an auto_inject menu. within_budget is the arm's own
cut: inside the budget and, for the banded command and write arms, inside the
band of the top hit. "Opened after" reads rule pulls, and rules created
after the call are dropped as notes are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"keep going", "yes please do", "is it deployed?": with no word of its own the
auto-inject query was the previous reply's tail alone, and the menu handed
back the records that reply was built from. In a sample of 59 logged calls,
28 came from short prompts; those with a subject of their own ("continue into
DRY 14", "fix the Windows gate") led with the record meant, while the ones
without were finished tasks already in context, with an open on 2 of about 15.
_has_subject checks the operator's words against a list of words that name
nothing; without one, the ranked menu is skipped and records named by number
are still shown. The prompt rule arm is unchanged and still searches with the
tail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
usage and rule_usage gain pulled_by_source, read from the per-source rows
the readout already had, and _silent_surfaces counts those doors as having
emitted. The rules-listing points were seen as of the previous commit,
through rule_usage.by_source.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_telemetry gains rule_usage.by_source, the twin of usage.by_source:
distinct rules each surface showed, how many an agent opened, and a ratio
that is null on a bulk delivery. list_rules(with_usage=True, days=N) adds
each rule's usage, windowed, and how many lessons are confirmed instances
of it, so linked and unlinked rules can be compared through MCP.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The write-path rule arm ran only past the notes guard: a write whose code
matched no snippet, issue or lesson returned before any rule was searched.
It now runs before that guard, on `code or path`, and a write that matched
nothing but a rule still returns the rule lines.
Operator ruling (2026-10-05): yes. A rule about the file being written does
not depend on a snippet resembling it.
Cost: one more rule search per write that matched no prior art, with its
own embedding (the query is `code or path`, not the notes arm's concept
query, so the per-request memo does not cover it). Plus the preference slot
from (c) on those writes.
Expected telemetry shift, to be read on its own once this reaches main:
write_path_rule calls rise toward write_path's count (14 days to 2026-10-22:
write_path_rule 4,634 against write_path 3,786 — the notes arm skips trivial
payloads, the rule arm does not, so the rule arm will now exceed it). Its
zero-result share rises too, since the newly covered writes are the ones
that matched nothing in the notes corpus.
Tests: the gate test is inverted (a write with no prior art still gets its
rule check); the design-only write now runs the rule arm on its path; two
notes-arm tests read the write_path row instead of any row, since the rule
arm writes its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PRE_TOOL_RULE and WRITE_PATH_RULE now take preference_slot=True, like
PROMPT_RULE. When no preference placed on score, the arm asks once more for
the best preference over the same bar, and shows it as one extra line.
Operator ruling (2026-10-05): yes. A preference about how to act ("let each
action land before the next") is ranked against every rule whose trigger
names the same command; the act is its last chance to shape anything.
Cost: one kind-filtered search per call that placed no preference. The
query embedding is memoised per request on both hook routes, so no extra
model call. Expect a new stream of preference_slot rows from the two act
arms; preference_slot's call count will rise roughly by pre_tool_rule's and
write_path_rule's (~23k calls over 14 days before this change), and its
zero-result share will rise with it, since most commands have no matching
preference.
Tests: the tool-arm call-row tests count rows under pre_tool_rule only (the
slot logs under its own name), and the write-arm bar test reads the arm's
own search, which is the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prompt_rule, pre_tool_rule and write_path_rule now share one settings key,
RULE_FLOOR_KEY = "kb_rule_threshold", default 0.70. The reply backstop keeps
its own 0.80, because there the floor is a stop bar. Moving the shared floor
through tune_retrieval or the Settings form records a tuning event on every
surface that reads it (set_dial, via retrieval_surfaces.sharing_floor), and
the result names the surfaces it carried as `moved_with`. Settings shows one
"Standing-rule confidence threshold" control where it showed three.
Operator ruling (2026-10-05): one shared bar for the rule arms, "especially
since we're only taking the first x number of them" -- the budget limits
what is shown; the bar only decides what is worth ranking. Kept a dial.
Why 0.70, from records read rather than percentiles:
- write_path_rule near-misses at 0.7199-0.72 were all false positives
(rule 149 against a Lidarr comment, 157 against a retention Svelte file,
121 against an Archives restore, 164 against a config test, 131 against a
Go package), so the old 0.72 was not hiding real hits -- but the band does
noise control on this arm, so dropping 0.02 costs little.
- prompt_rule: the operator set 0.70 by hand on 2026-09-18, after a 0.60
experiment admitted junk and was reverted. 0.70 is the number with a human
decision behind it.
- pre_tool_rule lines near 0.68 this session were mostly noise; rising to
0.70 trims them, and the band still narrows what clears.
Retired keys: kb_promptrule_threshold, kb_toolrule_threshold and
kb_rulehint_threshold are no longer read. Rows an install stored under them
stay in the settings table, inert. On this instance the effective bars move
pre_tool_rule 0.68 -> 0.70, write_path_rule 0.72 -> 0.70, prompt_rule stays
0.70; record_release_defaults writes those per-surface release rows on boot,
so the telemetry shift has an event to explain it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once the binding has moved, the old remote is no longer bound, and
move_binding refused. An install that renamed its repo before snippet
locations moved with it had no way to move them. When the old remote is
unbound and the new one is bound to this project, bind_repo now skips the
binding move and still moves the snippet locations that name the old
repo.
The snippet-move test gives each location its own path: rewritten to one
repo, identical locations merge into one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A repo renamed on its forge kept every snippet location naming the old
repo: the binding and the shape ledger moved, the snippets did not.
- snippets.move_location_repo rewrites, on every live snippet of the
project, each location whose repo is the old remote in any form (URL,
normalized key, "owner/Repo", bare "Repo", compared without case) to
the new normalized key. A snippet the caller can only read is listed
in read_only and left alone.
- Its loop is shared with move_locations through _rewrite_locations,
which saves each changed snippet through update_snippet.
- bind_repo calls it after the binding moves and reports snippets_moved
and snippets_read_only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With no SECRET_KEY set, Grimoire fell back to a fixed placeholder. The
quickstart compose substituted change-me-in-production, so the README's
own update step (pull, up -d) recreated the container on that placeholder,
and every credential sealed under the key the user started with stopped
opening.
- config: with neither SECRET_KEY nor SECRET_KEY_FILE, the first start
writes a random key to DATA_DIR/secret_key (mode 0600, published with a
hard link so concurrent starts agree), and later starts read it back.
An explicit key wins and leaves the file alone. A missing
SECRET_KEY_FILE or an empty key file stops the start instead of
switching keys. With no data directory (a run outside the container)
the process gets a temporary key and says so.
- app: warns on a public placeholder key, on a temporary key, and when
/data is not a mounted volume.
- Dockerfile creates /data.
- docker-compose.yml is now the deploy compose (published image, both
volumes, nothing to set). The build-from-source compose is
docker-compose.dev.yml, and docker-compose.quickstart.yml is gone. The
Swarm stack file gains the /data volume and keeps its explicit key,
because a Swarm can move the app away from a node-local volume.
- README: a Deploy section with the compose inline, SECRET_KEY optional,
"keep both volumes", and updates that keep the key. A test holds the
README's copy equal to the file.
- Installs that ran on the change-me-in-production placeholder get a key
of their own on the new compose; Settings then asks once for saved
forge tokens and the backup secret.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LICENSE carries the GNU Affero General Public License v3.0 text, the same
file the other Fabled projects ship. pyproject.toml, the frontend's
package.json and lockfile, and the plugin manifest declare it as the SPDX
expression AGPL-3.0-only, and the README says what it asks of anyone who
runs a modified version over a network. Reading `license` as an SPDX
expression needs setuptools 77, so the build floor rises to it. The
project description now matches the README's tagline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AppLogo still drew the old open book with a spark, so the new mark showed
only in the browser tab. It now draws the same clasped grimoire as
favicon.svg, in the mark's own fixed colours. That covers the header and
the sign-in, register and password pages.
The README follows the house order: the logo, name and tagline, a short
summary, quick start including the plugin install, the settings a reader
must set, setup pitfalls, updating and backups, features grouped by what
you do, troubleshooting by the error text, development, and licence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Snippet locations are keyed by path too, so a snippet recorded under a
moved directory reads `missing` at its next check. The tool is now
move_path(project_id, from_path, to_path). It moves the ledger rows as
before, and re-paths the project's snippet locations under from_path
through update_snippet, so each body and its indexed mirror stay one
record. A snippet the caller can only read is listed in
snippets_read_only. The tool refuses only when nothing at all is recorded
under from_path. The path test is the one snippets already uses
(_path_touches), and the plugin version is minted for the skill edit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ledger rows are keyed by path, so after a directory moves the next sync
marks every judged row vanished and lists the same shapes again, unjudged.
move_shapes(project_id, from_path, to_path) re-keys the rows under one path
to the other, in every repo of the project, keeping each row's judgment,
uses edges and history and adding a `moved` event that records the path it
left. A row already at the target is replaced when nobody judged it; a
judged one stands, and the old row is reported in `kept`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A binding and every row of its project's code-shape ledger are keyed by the
repo's remote. A rename on the forge changes the remote, so binding the new name
started an empty ledger beside the old one and left every judgment behind.
bind_repo now takes moved_from, the old remote. Its binding and the project's
ledger rows move to the new key in one transaction, and the reply says how many
shapes moved. The move is refused when:
- the old name isn't bound to that project;
- the new name is already bound;
- the project already has a ledger under the new name.
Each refusal leaves both names exactly as they were.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The forge repo is now bvandeusen/FabledGrimoire. The plugin marketplace default,
the install commands in the docs and plugin README, the CI requirements title and
the example repo URLs in comments and tests follow it. docs/development.md now
names the image the build actually publishes, fabledgrimoire.
Two test names still said "scribes_own"; they now say "grimoires_own".
What still says scribe does so on purpose:
- the backup reader's accepted archive names and prefix;
- the label every stored secret is sealed under;
- the notes on keeping an old install's database names;
- the old migrations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
favicon.svg is a violet book with gold corners and clasp and a four-pointed star
in a ring, replacing the indigo book with a spark. It is rendered to 192 and 512
PNG app icons and a 180 touch icon on the page surface.
manifest.json lists those icons instead of a favicon.ico that never existed. It
describes the product as the README does, and takes its colours from the design
system's page surface. index.html's theme-color follows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
theme.css loaded Fraunces, Inter and JetBrains Mono from Google Fonts. That sent
every visitor's address to Google on every page load, and an instance with no
internet drew in fallback type. All three families are now vendored under
frontend/src/assets/fonts with every subset Google publishes for them, plus their
OFL licences. The Content-Security-Policy no longer lists fonts.googleapis.com or
fonts.gstatic.com.
A new guard test fails if any stylesheet, page or component links a remote asset
or preconnects to one, or if the CSP names another host.
The theme.css header now describes the sheet as it is instead of how it used to
read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The step-3 sweep matched `scribe` only after a non-letter, and in these
three tests the shell call sits right after a `\n` escape in a Python
string, so the character before it was the letter n. They still called
scribe_recent_context, scribe_skip_prompt and scribe_checkpoint_allowed;
they now call the grimoire_ names. No other escape-adjacent occurrence
remains in the tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 3 of the Scribe -> Grimoire rename.
- Plugin `grimoire` in marketplace `grimoire-plugin`, MCP server key
`grimoire`, so tools reach a session as mcp__plugin_grimoire_grimoire__*.
- hooks/scribe_* -> hooks/grimoire_*, their shell functions and the TMPDIR
ledgers with them; SCRIBE_URL / SCRIBE_TOKEN / SCRIBE_HOOK_DIR /
SCRIBE_LEDGER_DIRS -> GRIMOIRE_*; the project marker is `.grimoire`.
- skills/using-scribe -> using-grimoire, /grimoire:sync, stored processes
arrive as grimoire-proc-<slug>; moment_actions names them the same way.
- Migration 0124 rewrites stored skill names (rule_moments,
rule_moment_judgments, moment_mappings) from scribe:/scribe-proc- to
grimoire:/grimoire-proc-, so a rule mounted on a skill keeps firing.
- Settings' install commands and MCP-only defaults, the docs and the
tests follow.
- scripts/check_plugin.py drops the smoke event for a report-check hook
that has no file in plugin/hooks.
Kept on purpose: the secret-sealing label, and the names that let backups
written as Scribe restore and be counted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 2 of the Scribe -> Grimoire rename.
- src/scribe -> src/grimoire, and every import, patch target, module path,
hypercorn entry point and `python -m grimoire.restore` with it; pyproject,
uv.lock and the frontend package are named grimoire.
- Image git.fabledsword.com/bvandeusen/fabledgrimoire; CI's Postgres service
runs as grimoire / grimoire_test.
- Compose files default the DB user, database and (prod) volume to grimoire,
each overridable from .env, so an install created as Scribe keeps its data
by setting POSTGRES_USER, POSTGRES_DB and PGDATA_VOLUME.
- Backups are written as grimoire-<ts>.tar with format grimoire-backup-tar.
Archives named scribe-* with the scribe-backup-tar manifest still restore
and are still counted at the destination (folder and S3), with tests.
- Exports, the MCP auth realm, the ASGI scope key and the persistent logger
carry the new name.
Kept on purpose: the secret-sealing key label stays "scribe.secrets.v1",
because every stored secret was derived under it.
Plugin identity (plugin name, hooks, env vars, .scribe marker, skills) is
step 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1 of the Scribe -> Grimoire rename: every capitalised "Scribe" in prose
becomes "Grimoire" ("Fabled Scribe" -> "Fabled Grimoire") across the UI,
the PWA manifest and page title, the MCP instructions and tool docstrings,
hook and skill text, docs and README, with the tests that assert on that
prose moving with it. Old migrations keep their wording.
Identifiers are untouched here: the package, plugin name, MCP server key,
hook file names, env vars, the .scribe marker and FabledScribe repo/image
names move in steps 2 and 3, along with prose that quotes them.
The sweep matched "Scribe" only as a whole word, so describe/subscribe and
friends are untouched. The longer name took the MCP instructions one
character over their 1,600 budget; two phrases are shortened to fit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The build job ran `docker system prune -af` on the runner's shared Docker
daemon. That deletes every image no container is using at that moment,
including images other repos' jobs had just pulled and were about to start.
Jobs with a service container wait ~14s for it to become healthy between
pull and create, and fail with "No such image" when a prune lands inside
that wait (CI-runner #16). The runner host prunes nightly instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Profile tab said an assistant used it to personalise chat and a daily
journal. Neither exists, and nothing read the profile: the page saved to
user_profiles and the backup carried it. Removed the tab, the client calls,
/api/profile, the service and the model, and migration 0123 drops the table
(downgrade recreates it empty).
Backup format v27 stops writing user_profiles; an older archive carrying the
section still restores, and the file-restore integration test now proves it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The acceptance greps found what the per-area passes missed:
- src/scribe: "Phase 7/8" removal notes in projects, settings and
user_profile; a paragraph promising an auto-summary side effect that no
longer exists; user_profile's claim that "upcoming MCP profile tools" read
the profile, which nothing does; an instance count and instance rule
titles in plugin_context; "MILESTONE 323 DECIDED" in rulebooks.
- scripts/: check_plugin's header and inline comments were a history of
the incidents behind each guard, with issue numbers in two failure
messages; check_dangling_styles named this app's classes as its examples;
check_design_tokens, measure_duplication and mint_plugin_version cited
issues and checklists.
- frontend/src: forty-odd comments with four-digit issue numbers that the
earlier frontend scan filtered out as hex colours, plus "used to" and
"Phase N" notes.
Kept: illustrative ids (`#4181` as a bare-id example, `PR #198`), migration
Create Date headers and one migration's dated data floor, and the plugin
skills' worked examples of how to cite a record.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twenty-eight test files described the code they guard by what it did
before a fix. Each is now the property and the failure it prevents, in the
present tense: the backfill snapshot, the per-dial calibration query, the
rule scope, the lesson trigger filter, the subshell global, the trash owner
scope, the version readout fallback, and others. A docstring line that
narrated the backup suite's own history is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings, section headers and assertion messages drop the record number,
milestone or step that introduced the behaviour, and incident accounts are
restated as the present-tense failure shape. A dead pointer to a design doc
that is not in the repo is removed.
Renamed to the property they check:
test_the_old_shape_would_be_caught -> test_a_rank_then_scope_query_is_caught
test_a_ledger_repeat_no_longer_produces_that_case_at_all -> test_a_ledger_repeat_is_counted_not_withheld
test_the_header_no_longer_promises_once_per_session -> test_the_header_explains_the_seen_marker
test_the_portal_case_no_longer_stamps_one_canon_onto_a_whole_file -> test_one_canon_is_not_stamped_onto_a_whole_file
test_the_repos_own_prose_no_longer_defines_with -> test_the_repos_own_prose_does_not_define_with
test_the_original_regression_create_note_with_content -> test_the_real_server_refuses_content_on_create_note
Fixture and example ids stay: "note 5" in a supersession chain, "rule 156"
in an illustrative question, `#2275` as the hex-versus-citation case,
`#3244` as the example of a bare number, fixed dates in date-filter tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Module and test docstrings state what is pinned without the record number
or step that introduced it. Incident accounts become the failure shape in
the present tense: the pipefail head, the code_shapes import, the export
order, the facet dialects, the cross-kind repeat, the write-once kind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings and comments state the property and its reason in the present
tense. The session-context ledger test no longer describes the decision it
"used to pin"; the supersession tests state demote-not-filter without the
quote; one-row measurements become the general shape with their figures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourteen more test files: docstrings state the failure each guard defends
as a present-tense shape instead of the incident that found it, and issue,
milestone and rule numbers leave comments and assertion messages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twelve more test files drop issue, milestone, rule, note, lesson and
snippet numbers and the dated or one-project figures that recorded how
each property was found; the docstrings state the property and why it
holds. The contract-around-the-change guard keeps the dict-key example as
the failure a signature-watcher misses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ten more test files: issue, milestone, rule, decision and snippet numbers
come out of docstrings, comments and messages; the preference door states
what a preference is instead of quoting the framing; the JSON reader says
why jq is not assumed instead of what every hook used to do; the backup
round trip's column warning no longer claims the import side is unguarded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state their properties without issue, milestone and
rule numbers or dated regressions. The version tests describe what folding
the channel into the version breaks instead of the date it did; the surface
registry and settings-defaults tests state that floors move rather than
quoting the decision that made them move.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state their properties without issue, milestone,
rule and decision numbers. The tuning and suspension notes describe the
symptom a shipped default or a late ledger produces instead of one
install's dated readings; the reply-discipline and checkpoint docstrings
state the failure they prevent instead of the session that measured it.
test_empty_selector_is_the_rule_115_baseline is renamed for what it
checks: the no-forge baseline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven more test files: docstrings and comments state the property and its
reason, the SQL-twin note states why `\m` and not `\b` without the install
it broke, and fixture provenance says "real near-miss queries" rather than
naming the spike that collected them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six more test files state their pinned properties without the issue and
milestone history: the pull-telemetry guard explains enumerating kinds
instead of listing two getter incidents, cited-record status explains why
a hint's id needs its status instead of recounting one misreported step,
and the trigger-shape guard keeps its reasoning about why prose heuristics
fail while dropping the dated corpus audit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state the property they pin as a present-tense
fact. Incident narratives (a dated storage stall, a week of zero tuning
events, a two-project poisoned ledger, a twenty-site owner audit) become
the failure shape each test defends; one-install figures that name a rule
become a general worked example with the calibration numbers kept. The
family-grouping regression guard is renamed for what it checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shape ledger, MCP projects/search/rulebooks tools, backup, chunking,
plugin context, derived mirror, rule band, shape form gate, plugin
version mint and note usage tests drop issue, milestone and rule numbers,
dates and one-install figures from docstrings, comments and messages.
Three shape-form-gate test names that told the discovery story now name
the property: a registry module's helpers are not service functions, the
divergence gate silences only a family contradiction, and callable helpers
beside a callable canon still prompt. The rulebook tool-count comment that
logged each milestone's additions is replaced by the reason it is a count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_pattern_coverage, test_guidance_ownership, test_integration_shape_classify,
test_instruction_surfaces_agree, test_retrieval_warnings, helpers.py and
conftest.py state the property each test or fixture holds. The issue and
milestone numbers, consolidation dates and one-install measurements that
recorded how each was found are dropped; fixture dates and ids stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings, comments and assertion messages in the three most-cited test
files now say which property each test holds and why, without the issue,
milestone or rule numbers, dates and one-install measurements that recorded
how each property was discovered. Fixture ids and calibration scores stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Components, stores, api clients, router, types and shared CSS now state what
each piece is and why, without the milestone that introduced it. diff.ts
names its consumers instead of the copy history; stores/notes and tasks state
their current scope instead of the cleanup that set it; theme.css states the
token practice instead of the sweep that established it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
components.css says why shared recipes live there without the five-copy
.btn-primary count; KnowledgeView's editor-routing and chip comments state
the present reason instead of the bug that once routed snippets wrong.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record and milestone citations go; the ACT-arm bar split and the per-arm
budgets state why they exist without the measurements that motivated them.
A comment about a removed think_enabled setting, pointing at a file that no
longer exists, is deleted, as is the story of a removed design-system picker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record, milestone, phase and plan citations go from every migration header;
the "end before start" fix (0043) and the suppressed-count column (0095)
state the condition they prevent rather than the incident. 0053 and 0055
pointed at design specs under docs/superpowers that are not in the repo;
those pointers are removed.
Kept: alembic's generated "Create Date:" headers (template metadata, not
rationale), 0121's ARM_BORN justification beside the constant, and 0051's
illustrative "update task 42" format example.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone/step labels, record ids, the rescoped-rule date and the 296-call
measurement go; each header states what the change protects. "rule 36"
citations become the DROP + ADD practice they named.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shipped tool docstring (mcp/tools/repos bind_repo) cited this instance's
"rule 1: dev is home"; it now says what a dev-branch project should do.
routes/knowledge, routes/tasks, routes/systems and models/retrieval_log state
the property each guards rather than the bug that once broke it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
routes/search states the derived-facets property rather than the silent
whole-corpus bug that motivated it; routes/rulebooks gives the preference
restore exception its reasoning without the milestone numbers it cited.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- api.py's build-version docstring states why a channel name is not a build
id, without the dated incident.
- rule_version states why rules need a history, without the rule that was
rescoped and the date.
- Capitalised "Rule N" / "Milestone N" citations the earlier lowercase grep
missed, in reply_shapes, supersession, services/family and models/family.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Corrected while reading: routes/notes.py said lessons and snippets pass the
owner to set_record_systems, "a real inconsistency". Every door passes the
caller; the comment now says so.
rule_usage's model docstring states what an outcome measures and why the
row cannot share the note table, without the measurement and record ids
that motivated it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule-embedding docstring carries the own-table test itself instead of
pointing at a record; the later siblings refer back to it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The deferred-backfill and inline-release-defaults comments describe the
lifespan-overrun hazard they guard without the dated incident; backfill
passes say what they fill rather than which milestone added the vectors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The relation kinds say what each prevents without the rule-46 split story;
the removed tier, subscriptions and suppressions are stated as what the
model does not have, with the migration that dropped them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pre-tool rules docstring says why action rules need their own surface
instead of narrating that they once lived in the preload; the write-path
checkpoint note states the reason the hook never blocks rather than whose
decision it was.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backup_s3 says what its signing is checked against instead of which of the
operator's projects it was ported from; rulebooks and shape_check state what
the code is for rather than which milestone it belonged to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 1-2 hit tail: access, background, backup, backup_job, canonical_systems,
design_stylesheet, forge_connections, moment_actions, notes, notifications,
placement, planning, platforms, projects, record_batch, recurrence,
report_check, retrieval_review, rule_scope, rule_versions, scheduler,
shape_check, supersession, system_rulings, system_usage, text, trash,
verification, version_pinning.
Also corrected while reading:
- backup.py said restore step 16 fills the System map; step 15 does.
- version_pinning.py pointed at a design doc that is not in the repo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_migration states what it lets a tuned floor do without quoting
the operator, and its order-of-operations refusal is described as enforced
rather than as history. logging states why raw date filters fail rather
than citing the two issues that found it. The rest drop milestone, step and
record citations; step references now name the code that does the job.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- db_maintenance_scheduler states the storage-stall hazard the startup
timeout guards, without the dated outage.
- design_starter_roles states why a missing role produces literals, without
the call-site count, the operator's call date or a project's name.
- The rest drop milestone/step labels and record citations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_surfaces' header states why the arms read their numbers from one
table and why a floor is moved by reading the refused record, without the
quoted decision, the retired arm or the declined-rule instance.
family_adoption drops milestone/step labels from its section headers and
states the no-ids reason instead of naming the trap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
inception states the legacy stamp and the retired rulebook keys as facts
about stored records; family keeps the repeat-trigger calibration numbers
without the date and record they were measured on, and drops the instance
that taught the per-kind comparison.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
record_refs states why a guessed id is refused without quoting the
operator; moment_delivery's docstrings name what each door does without the
milestone and step that built it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The name gate, catalog and starter-set docstrings drop milestone and record
citations and the history of the tuple constant; each states why it is
shaped this way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The silent-outage story (#2663), the ranked/ambient split (#2477), the
transfer evidence (#3735) and the attach_usage consolidation (#4230) are
stated as the property each guards, without the incident.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pool-exhaustion, handshake-size and stepless-roadmap comments state the
condition the code guards instead of the measured incident; citations go.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module header drops the milestone/step map and record citations; the
per-user optionality, the keyring model and the empty-selector baseline are
stated as present-tense facts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record and milestone citations go. The logs_unconditionally docstring, the
place arm's comment and the fan-out key note state the condition they guard
rather than the bug that once hit it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring states why the model moves dials and why a reason is
required, without the decision quote or the declined-rule story. Bare
record citations go; the release-default and ledger-gap docstrings describe
the condition they guard instead of the measurement that found it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, milestone, snippet and rule-number citations are removed; the scope
docstring states the leak it prevents without naming the milestone that
closed it for the direct arms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ACL header states the share behaviour without its date and decision ids;
the facet table, door map, floor constant and facet-count comments state why
they are single sources instead of retelling how copies drifted; this
instance's counts (legacy plan-tasks, issue share) are removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring keeps why the kind exists, where its trigger lives, what
it inherits and why it needs no migration, without milestone, decision and
note ids or this instance's snippet counts. The body-strip docstring states
why taking the note matters instead of retelling the defect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring keeps why rules need their own usage stream and why the
ranked list is the one that is named, and drops the dated story of the
bulk surfaces starting to emit, measured decline rates and the corrected
earlier wording. Outcome docstrings describe the four states without
milestone references.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, decision, milestone and operator-decision citations are removed;
"previously each route…" and "list_tasks once returned…" stories become the
present-tense reasons they taught. The note about the removed delete_note
now says plainly that there is deliberately none, and why.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, milestone and rule citations and dated measurement headers are
removed; calibration figures stay. Corrected the lesson-gate comment, which
said lessons are not in _REPORT_KINDS and so have no duplicate-report
backstop. They are, and the comment now says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone, issue, note and rule-number citations are removed. The
co_surfaces docstring states why paired rules need the edge instead of
retelling the instance that taught it; the list-rules and applicable-rules
docstrings drop the retired subscription history. The preference-restore
error string no longer cites a milestone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, decision, milestone and rule citations are removed; the snippet-mirror
docstring drops the paragraph about its own earlier wording, and the
pointer-model and freshness comments state the design rather than the plan
step that introduced it. Access notes point at services/access.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, note and milestone citations are removed along with first-deploy
counts and the measured phantom-definition story; "button B" jargon becomes
the divergence it names. Each block keeps its present-tense reason.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The recurring "#2663 shape" and similar issue citations become the property
they stand for (an outage that reads as a quiet window, a statistic that
cannot vary). Dated observations, milestone tags and rule numbers are
removed; the near-miss and pull-through reasoning stays in present tense.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The format-version log keeps what each version adds, because restore depends
on what an older archive can carry; dates, milestone and issue ids, and the
stories of what a missing column once cost are removed. Compatibility
comments now name the archive version instead of a milestone. Two leftover
history-phrased comments in embeddings and shape_ledger are restated as
present-tense reasons.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone tags, dated measurements and "used to" stories are
removed; the chunker version list now says what each stamped version
contains. Corrected two comments that said a failed search leaves
report["searched"] absent (it is stamped False first), and pointed the
rule-search docstring at _RULEHINT_BAND in retrieval_pipeline instead of a
tier note in plugin_context that no longer exists. plugin_context's own
dangling "note on tiers" pointer is reworded to match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone and step numbers, and the stories of copies drifting
apart are removed; each invariant and slot is justified in present tense.
The rule-band probe table stays as calibration. Section headers drop their
milestone tags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, snippet numbers, dated measurements and the proposer's version
history are removed; each block now states the design reason in the present
tense. Calibration figures that justify a floor stay, without the instance
they were measured on. One error string drops a note id. The density
threshold comment moves above the constants it describes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone numbers, dated telemetry readings and rule numbers
leave the retrieval-arm comments and docstrings. Each bar keeps its
present-tense reason and the probe-score calibrations, which describe the
model on any install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MCP tool docstrings and comments, the server's orientation notes, the plugin
skills, every hook script and the plugin README/PACKAGING drop record ids,
milestone numbers, dates and incident stories, and state each non-obvious why
as a present-tense fact. History stays in the task logs.
Corrections found on the way:
- plugin README/PACKAGING said every hook is a GET; moment, rule-opened and
reply-rules POST, so the plugin's key needs write scope. Both now say so,
name Settings -> MCP Access, and list every wired hook and endpoint.
- the contract-change hint printed "(lesson #4207)", this instance's record,
to every install. It no longer cites a record.
- tasks.py claimed there is no delete_task; there is.
- rulebooks.py cited delete_event and a design spec that no longer exist.
- scribe_after_write.sh carried an always-empty etag_q; removed.
- test_write_path_trigger pinned "| tac | scribe_defs | head -1", text that
existed only in a comment about the old code; it now pins the real tail -1
pipeline.
- docs/development.md: CI runs on Gitea Actions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
g t pushed "/", which now redirects to the dashboard, so it did what g h
does. It pushes /knowledge. The overlay gains g r (Rules) and g g (Graph),
calls g h "Dashboard", and docs/features.md lists g t.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone 544, step 6217 (follow-up to #1778, which rewrote features.md and
api-reference.md in July but left architecture.md and configuration.md alone).
- architecture.md, configuration.md, development.md: rewritten against the
code. They still described Ollama, chat, SSE generation, voice, project
summaries, image caching, Docker secrets that the compose files don't use,
a raw-SQL migration convention the migrations no longer follow, a 27-entry
migration list, CalVer YY.MM.DD.N tags and the removed infra/ files.
- features.md: adds what shipped since July (spikes, claims, moments, rule
versions and verification, lessons, reply shapes, snippets and the shape
ledger, forge integration, family canon, design systems, retrieval tuning)
and corrects the Settings tabs and keyboard shortcuts (g g is the graph).
- api-reference.md: adds the endpoints for snippets, lessons, design
systems, family canon, platforms, shared areas, retrieval, rule versions,
relations and verification, forge connections, webhooks, scheduled
backups and the newer plugin hooks.
- api-keys-and-mcp.md: Settings tab is "MCP Access"; a plugin install
section; the tool-family table matches the tools that exist.
- design-system.md: the dated "decisions in progress" changelog (April
PR log, chat/journal/calendar surfaces) becomes a short present-tense
section; the stored design system is named as the authority.
- sso-oauth.md: the app is Scribe, not "Fabled Assistant".
- README: feature summary and doc table brought current.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At the operator's request (milestone 544, step 6203):
- docs/2026-03-23-fable-mcp-{design,plan}.md, 2026-03-29-voice-s2s-design.md
and 2026-04-10-agentic-briefing-design.md were plans for work that has
since shipped or been retired (voice and briefings left in the pivot).
Plans live in Scribe milestones.
- infra/act-runner-config.yml and runner-compose.yml described one
install's CI runner (an /nfs path, the py3.12-node22 label, a
RUNNER_SETUP.md that no longer exists). The runner is the CI-Runner
repo's concern.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings the three compose files in line with rule 198 (milestone 544,
step 6203, at the operator's request):
- no healthcheck on the app or the database, so a slow pg_isready on
stalled storage can no longer get a working database killed;
- no deploy.restart_policy in the Swarm file, so Swarm's default
(condition: any) restarts clean exits too, instead of on-failure with
max_attempts leaving the app down;
- depends_on is plain start order; restart: unless-stopped stays on the
plain-Compose files (Swarm ignores it), and the dev app gains it.
The orchestrator used to be what retried a container that started before
Postgres was ready. That is now the app's job: alembic/env.py calls
services/db_ready.wait_for_database before migrating. It retries refused
connections, unresolved hosts, connect timeouts and "the database system
is starting up" with 1-2-4-...-10s spacing for up to 300s, then exits
non-zero; a wrong password or missing database fails at once.
Per-request recovery is the engine's pool_pre_ping.
Also fixes the dev compose database image: postgres:16-alpine has no
vector extension, so migrations could not run against it. It now uses
pgvector/pgvector:pg17 like the other two files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment-only pilot for the comment standard (rule 32, milestone 544).
Issue and rule numbers, dates, run ids and incident stories come out;
the constraint each one guarded stays as a present-tense sentence.
History removed from the files, kept here:
- Dockerfile/CI: installing from uv.lock replaced pip/uv-pip installs that
ignored the lock (#2194, the mcp 2.0.0 break on 2026-07-28); --locked
then caught pgvector missing from the lock.
- CI: plugin paths were added after #2198/#2209 reached a live install;
`integration` joined the build's needs after run 8212 published
:latest with integration red (#4958); the `[[ ]]` tag step failed
silently on dash (2a374d9); BUILD_VERSION carried the channel until
2026-08-31 (#3298).
- jq was removed from the hooks in #4107.
- prod compose: the lenient DB healthcheck followed a runc setns crash
loop; pgvector arrived with migration 0067 via a PG16->17 dump/restore.
Corrections to match the code: README names the Settings tab
"MCP Access"; ci-requirements says the Dockerfile builds on node:22
and lint covers scripts/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings -> Profile (name, role, expertise, response style, tone, interests,
working hours) is typed by its user and neither derived nor a credential, yet
_NOT_INCLUDED named it, so a restore dropped it without a word.
BACKUP_VERSION 26. Both exports carry it; the restore remaps only the user, and
a blank field stays blank rather than restoring as the page's default. The
column and import guards cover it, and the tar round-trip restores one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The column becomes token_sealed (migration 0122 seals existing rows with a
frozen copy of the v1 key), every reader opens it through connection_token,
a row this instance cannot open is skipped with a warning, and Settings asks
for that token again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The restore route takes the tar as a raw body beside the JSON download.
scribe/restore.py reads either format, refuses a database that already
has an account (a restore re-creates everything with new ids, so into a
live instance it would duplicate rather than replace), has no size
limit, and says to restart so search indexes rebuild. An integration
test restores a written tar through the real restore_full_backup and
checks every id is remapped. docs/backups.md explains the file, why it
is uncompressed, both destinations, Fabled Archives as the worked
example, and restoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BackupScheduleSettings.vue leads with whether the data is backed up
(last good backup, or Overdue), then what is wrong (last failure in
words, an S3 secret this SECRET_KEY cannot open, incomplete config),
then the form: daily toggle and UTC hour, folder or S3 bucket, Test
destination with unsaved values, Back up now (polls while running,
re-armed in finally), and the last runs. The secret is write-only.
formatBytes moves to utils/bytes.ts for both views. Restore from
Backup now also takes the .tar, and says to use the CLI on a 413.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/backup_job.py reads its config when it fires, so startup reads
nothing (#4181) and a changed hour needs no reschedule. A run is held to
a 60-minute deadline (a hung NFS mount would otherwise hold the lock and
skip every later backup), never runs twice at once, and keeps its outcome
- file, size, duration, or the reason in words - in the bounded
backup_runs setting, which no backup carries. summarise() gives the page
its verdict: overdue when on and no success for 26h.
Admin API: GET/PUT /api/admin/backups, POST /backups/check (tests unsaved
form values, 45s deadline), POST /backups/run (202, background).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/backup_s3.py is FabledSteward's SigV4 client ported (put + list,
never delete, so an append-only key is enough), with its tests against
AWS's published signatures. services/backup_destination.py holds the
admin config (backup_config JSON) and delivers a dated tar to a mounted
folder or a bucket, and checks either without sending a backup.
services/secrets.py seals the S3 secret with Fernet under a key derived
from SECRET_KEY; backup._SETTINGS_NOT_EXPORTED keeps it out of every
backup. cryptography becomes a direct dependency (already in the image
through pyjwt).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
export_full_backup orders every query by primary key: an UPDATE moves a
row to the end of the heap, so an unordered export shifted every byte
after an edited row. services/backup_file.py writes the payload as one
dated tar per run - manifest.json plus <section>.jsonl, sorted keys,
fixed member headers - atomically via a dot-prefixed .part file, and
reads it back into the dict restore_full_backup takes, refusing a
missing or short section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings > General > Reply shapes. One menu row per shape (core, completion,
asks, plan): its title and when it arrives, how many of the operator's
preferences adjust it, and its deliveries over 30 days (in full / as a
reminder, "—" when the counts cannot be read). A verdict line leads. Opening a
row shows the shipped text, read-only because it is product, and the
preferences mounted on its moment; each opens in the rule editor, and "Adjust
this" opens a new preference already mounted on the shape's moment with a
starting trigger.
The rule editor takes an optional preset (kind, moments, trigger) and, opened
outside the Rules view, asks where a new record lives instead of dropping it.
GET /api/retrieval/reply-shapes now returns reply_shapes.overview: the
catalog plus mounted preferences (the same lookup that delivers them) and
counts from the reply_shape delivery rows. #5497 (step 5 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reply shapes are server product now, delivered at their moments, so the
skill stops restating them. Gone from it: the "Every reply" list, the
per-kind tables, and "The operator's own shapes come first" (preferences
arrive beside the shape on the same moments). It opens by saying where the
shapes come from (list_reply_shapes, the delivered core's header) and keeps
the reasoning: sections chosen not filled, a settled decision acted on,
placement from the record, who decides what, the assumptions an option
carries, the completion report worked in full, and the second pass. 13.5k to
10.7k characters.
The core gains the Finding kind the skill's table carried (2,150 of 2,200).
Tests follow the content: the kind and Approval-row pins move to the shapes,
a new test holds the worked example and the completion shape to the same
sections, and the guidance-ownership registry reads the delivered shapes as a
surface, owning the preference-wins and length topics there. using-scribe
points at the delivery and list_reply_shapes. #5496.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
report_preference searched one constant string when a task closed and handed
matches back as update_task's reply_preferences. Milestone 500 step 3 delivers
the reply shapes and the preferences mounted beside them on the moments, so
the arm goes, whole (rule 22):
- services/reply_preferences.py, the REPORT_PREFERENCE RuleArm, its re-scorer
and corpus entries, and the REPORTPREF constants
- update_task's reply_preferences block and its cue; the docstring now points
at moment_rules
- the fixed_query field and the fixed_query_never_clears warning: this arm was
the only one that set it, so the concept and the cannot_decline exemption
go with it
- the two Settings fields for its floor and budget
- migration 0121 deletes its rows (logs, judgments, rule-usage events, tuning
history, settings keys), operator-approved 2026-10-09. Without it the
readout would call the source unregistered and its surfacings would count
as ambient. The unindexed rule_usage delete is bounded by created_at.
#5496 (step 4 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The marker scribe_reply_check.sh writes after a completion-section hold sat in
the swept ledger directory as `<sid>.reportcheck`, and the sweep matches
`.ids` - test_every_session_file_in_a_swept_directory_follows_the_convention
caught it (run 799). Renamed `<sid>.reportcheck.ids` and added to the roster.
#5496.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Stop hook sent the finished reply twice: scribe_report_check.sh checked a
task-closing reply for the completion sections in shell and reported to
/report-check, and scribe_reply_check.sh sent the same reply to /reply-rules
for the rule hold. Now the reply goes once. When the turn closed a task the
hook adds the close count and ids, and the server runs the section check
(services/report_check, the same three patterns) beside the reply hold,
folding both into one reason.
The report_check adherence log is still written for every checked reply
(milestone 409's number). A section hold marks the session, so its rewrite is
sent back once with rewrite:true to record how it came out, and is never held.
The block reason carries the completion shape's one line and points at
list_reply_shapes rather than at the skill. #5496 (step 4 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scribe_json_unescape wrote each escaped code point with sprintf("%c", n),
which writes the character only under gawk in a UTF-8 locale; gawk in the C
locale and mawk in any locale write the single byte n % 256. The server
escapes every non-ASCII character, so "·" reached the session as 0xB7, "—"
as 0x14 and an emoji as a NUL wherever a hook ran without a UTF-8 locale.
The decoder now encodes UTF-8 itself under LC_ALL=C. A new test sends what
the server sends from a bare environment, in three locales (#5495).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Every turn (/api/plugin/retrieve, UserPromptSubmit): the core reply shape
leads the payload - in full the first time, as its one-line reminder
after that - followed by whatever is mounted on reply.report, under the
shared rule ledger. Fresh keys come back as shape_keys.
- The ledger is <sid>.shapes.ids in scribe-priorart (scribe_shapes_file /
_seen / _append), so the compaction sweep that clears every .ids ledger
is what brings the full core back after one.
- At a moment (/api/plugin/moment): the slice for that reply - completion
on work.finish, asks on reply.ask, plan on work.plan - ahead of the
mounted rules. reachable_tools now lists tools reaching a shaped moment
even on an install with nothing mounted.
- Scribe's own tools (attach_moment_rules): reply_shape in the response,
in full, since that door has no ledger. enter_project carries the core
for clients with no prompt hook.
- Telemetry: one AppLog row per delivery (plugin / reply_shape), each
shape with full or pointer and the door (turn, hook, mcp).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The operator's line (2026-10-09), reconciling their 2026-09-21 ruling
(#4218, "you are the judge") with "not that Claude should make all
decisions": judging covers the work and its records - facts, findings,
measurements, whether work is done - and direction stays theirs:
priorities, what the thing should be, trade-offs only they can price,
anything they live with, and every act hard to undo or facing outward.
- reply_shapes: a one-line form in the core (every turn), the full text
in the asks slice (where a question gets handed back). Slice budget
1400 -> 1600 for it.
- reporting-back "You are the judge": "the operator reads the decision -
they do not make it" replaced; the line is drawn by who can see the
evidence, not by how hard the call is.
- plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/reply_shapes.py is the single source of the default reply shapes:
the core (every reply, ~1,800 chars against a 2,200 budget) and three
slices - completion on work.finish, asks on reply.ask, plan on work.plan.
Read through list_reply_shapes (MCP, read-only) and GET
/api/retrieval/reply-shapes, one service behind both doors.
Nothing delivers them yet; that is step 3. The skill still carries its
copy until step 4 shrinks it to the long-form reference.
The software-only vocabulary guard moves into tests/helpers.py
(DEV_ONLY, dev_only_hits) rather than becoming a fourth copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner's docker daemon is shared, so `--filter name=<job> | head -n1` can
pick another repo's Postgres when two jobs run side by side (Steward run 8358,
Inkwell run 8653). Scope the lookup to this job's own task prefix and require
exactly one match. Done from Inkwell #5313 with the operator's permission; the
recipe is rule 79.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lessons share one register, so any two score well above unrelated text and
a fixed similarity bar sat inside that band: every no-rule lesson
"resembled" most of the others and the named group grew with the pool.
A neighbour now counts only when it stands above the lesson's own
background (median and MAD of its similarity to every lesson it can reach,
in that register's spread), and a group is a clique: every pair clears the
bar from both sides, so one broad lesson cannot join unrelated ones. The
bar, the minimum background and the fetch sizes are stated as defaults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A multi-pass DRY audit could not answer whether it created duplication:
a pass's own shortening can leave two statements identical to a third,
and no per-pass scan sees a copy that did not exist when it ran. The
Librarian retrospective improvised this measure in a scratchpad; this is
it as a tool the DRY Pass process can name.
6-line windows over significant lines (comments, blanks and bare
punctuation dropped, strings folded), grouped by glob or extension.
Revisions are read through git archive, so nothing is checked out.
Stdlib only, so any project can run a scratch copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dated release log (a note) matched a signed-APK CI snippet at 0.83 on
shared release vocabulary and was opened as a family candidate, then came
back as a precedent. One chunk of a long narrative scores like the shape
it mentions, so a cross-kind score does not mean "the same thing made
twice". The search is filtered to the new record's kind and the loop
checks it too. Code built from an idea written elsewhere still reaches the
engine through the citation trigger and the shape ledger.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each decision stored its reason three times (reason, before.reason, after.reason)
and a promotion carried the whole criteria reasoning in evidence. Precedents
(assess, promote, get_family_adoption, get_family_idea) and get_family_idea's
history now serve FamilyDecision.to_brief(): reason once, before/after as status,
canon_version and platforms, evidence as the cited list. list_family_decisions
and the decision echoed back by a write keep to_dict. The precedent count was
already bounded at five.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1 added family_ideas.topic_id but nothing could set it, so the note<->topic
link the milestone promised existed only as a column. set_family_topic links or
unlinks (0) a topic in a rulebook the caller owns, one idea per topic; the
version does not move. get_family_idea and get_family_adoption list the topic's
rules; list_family_ideas and the Family page name the topic. The model
docstrings no longer claim the topic's rules are platform-scoped in retrieval.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- enter_project carries a `family` key, but only when the project has
something to answer: counts of unassessed, owed and to-recheck answers,
each with the list_family_adoptions call that lists it. It shows on every
entry, never by platform touch: entry is when work is chosen, and an
unanswered idea is otherwise invisible.
- Retrieval: a widened project search (include_global_kinds) now also
reaches the canon ideas on the project's platforms. It also reaches their
references in the project's languages, or all of them when none matches.
An off-platform project gets none, and the plain project filter (the
duplicate gate) is unchanged.
- Closing a task returns `family_owed`, the owed answers filed while it was
open, and the report cue asks for them to be named.
- New plugin skill family-canon (moment work.record) covers when to
evaluate a promotion, answering in order, what counts as a reason, the
precedent reflex and the conflict order. _INSTRUCTIONS, create_note,
create_snippet, classify_shapes and reporting-back point at it.
The plugin version is minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- classify_shapes takes idea_id: the shape is judged against the idea's
reference in its language, or its first.
- A shape classified against a canon idea's reference moves the project's
adoption row (instance -> adopted, variant -> variant). Withdrawing the
shapes that gave an answer returns it to unassessed. This runs on
classify_shapes, the sweep, confirm_proposals and the coverage refresh.
- assess and undo refuse an answer the shapes contradict. A conflict
resolution re-judges the variant shapes on both sides.
- The proposer's family arm offers a canon idea's reference on a shared
platform, in any language. It proposes only when the reference is the top
hit over every readable snippet and scores at least 0.70. The pairs this
was measured on are recorded beside _FAMILY_FLOOR. The proposer is now v6.
- The matrix cell shows the code that answers it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
assess_family_adoption took system_ids for the task it files, which made
the family tool module look like a System-tagging door and tripped the
parity registry. It is not one: the owed task is tagged to the project's
System matching the idea's canonical area, and update_task retags it.
The unit test for each conflict ground passed evidence/conditions twice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- services/family_adoption.py: assess one project against one canon idea by
the four outcomes in order (exempt, variant, adopted, owed). Every outcome
needs a reason and adopted needs evidence. The engine records the precedents
itself: this idea's answers elsewhere, and this project's answers to the
nearest ideas. The same answer given twice records nothing.
- owed files a task in the OWING project, tagged to the System matching the
idea's canonical area, naming the gap and the reference for that project's
language. The task follows the answer: adopted closes it, exempt or variant
cancels it, owed again reopens it. Each move is logged on the task.
- the conflict order is enforced: every ground above the deciding one must
say why it did not decide. The losing side is folded into the idea's note as
a trap, an alternative or a condition branch, the version moves, and both
rows are answered against the revision.
- recheck is derived (row version != idea version). family.revise moves the
version when substance changes. undo covers a project's latest answer too.
- set_family_references names the reference implementations.
- MCP: get/list/assess adoption, resolve_family_conflict, revise_family_idea,
set_family_references. Web: GET /api/family/matrix.
- UI: an adoption matrix on /family (platform filter, cell detail with reason,
recheck and owed-task link) and the same matrix narrowed to one project on
its Family tab. The decision log now reads project-level decisions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the undone decision had no recorded before-state, the synthesized prior copied the canon idea's current applies_when and platforms into a retired row. Before that decision the record was not an idea, so the restored state now has neither. Caught by test_undoing_a_promotion_restores_the_prior_state_and_keeps_judged_rows (run 8281).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent promotes a family idea when all three criteria hold, and no person approves it. The criteria are product text: services/family.py states them, and the promote tool's docstring names every criterion the service enforces.
- Criteria: each one vetoes on its own when its reasoning is blank. Platform terms also needs an applies-when and a platform scope; proven also needs named evidence. A veto keeps the idea a candidate and is logged, so it becomes precedent.
- Precedent: every promotion stores the decisions on the nearest ideas by meaning, plus any the caller names.
- Promotion sets canon, the applicability test and the platform scope, and opens an unassessed ledger row for each member project the promoter can write. Re-promotion moves the version past every version the idea has held.
- Retire and undo: undo reverses only the latest idea-level decision, restores its recorded before-state, and logs itself with the undone decision as its precedent. Settled here: leaving canon closes the unassessed rows but keeps the judged ones, which read as needing a recheck after a re-promotion.
- Triggers open evaluations but never promote:
- a cross-project lineage citation ("matching #N") on a note or task write;
- a same-meaning record in another project on a shared platform, on create, at 0.80 (measured: the known pattern's builds scored 0.79-0.82, an unrelated project's best match 0.65);
- a milestone closing on a platform.
Each fails open and rides the response as family_hint.
- Doors: seven MCP tools, /api/family REST endpoints, and a Family page (nav, /family) showing the criteria, the ideas, and the decision log with undo and retire.
- utils/recordHref.ts holds the one copy of "where a record opens", now shared with LessonDetailView.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 8271 failed in the image build: "[vite:vue] Cannot read properties of undefined (reading 'scoped')" on moments-shared.css. plugin-vue caches one descriptor per src file and answers ?index=N from whichever component registered it last. RuleEditorSlideOver had moments-shared at block 2, while its other two importers have it at block 1. Step 2's new imports changed the transform order and exposed the bug.
- RuleEditorSlideOver: moments-shared is now loaded with an @import instead of a third <style src>.
- rules-shared.css sat at block 0 in the five panes and at block 1 in LessonDetailView, LessonEditorView and RuleEditorSlideOver, which is the same latent hazard. The three move to block 0.
- tests/test_frontend_shared_styles.py: a guard that every <style src> sheet sits at one index, plus a test that the guard can fail. vue-tsc cannot see this hazard, and vite build only runs after every test has passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project's platforms decide which family ideas reach it. This step makes membership answerable from every door:
- services/platforms.py: the global catalog (writes are admin-only and duplicate-gated by slug); pure marker detection; and membership reads and writes. Detection only ADDS, and only where nobody has answered. It never overrides a declared or rejected row and never removes one.
- coverage: the archive scan now carries every path, and the refresh runs detection fail-open.
- inception: a platforms choice (slugs, or null for unanswered). The list is the whole answer: members left out of it become rejected.
- MCP: list_platforms and set_project_platforms; enter_project and get_project carry the project's platforms.
- REST: /api/platforms (admin writes) and /api/projects/<id>/platforms.
- UI: a platforms checklist on the inception card, a Family tab on ProjectView, and a Platforms admin tab in Settings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When one project solves something every project on the same platform will
meet, that solution becomes family canon and every other project on the
platform answers it. This is the storage for that.
- platforms: a global catalog in the canonical_systems shape, seeded with
generic technology names and the file markers step 2's detection reads.
- project_platforms: declared / detected / rejected. A rejected row is kept
so detection cannot re-add what a person said no to.
- family_ideas: a note's family state. No new record type; any note, snippet
or lesson becomes an idea. A canon idea must state when it applies (CHECK).
- family_idea_platforms: the only scope source. A linked rule topic takes its
scope from the idea, so the two cannot disagree.
- family_idea_references: reference implementations, explicit not inferred.
- family_adoptions: one answer per (project, idea). Variant and exempt require
a reason (CHECK). Recheck is derived from the two canon versions, never
stored.
- family_decisions: the append-only log, with a required reason and the
earlier decisions each one followed. The agent decides with no approval
step, so precedent is what keeps its calls consistent.
Backup v25 carries all seven: platforms by slug, precedent ids remapped
through the decision map. Both column guards cover the new tables, and a
real-Postgres test exercises the CHECKs and the restore remaps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ordered straight off a *_embeddings table, the planner walks the HNSW index,
takes ~ef_search (40) nearest chunks across every owner and project, and only
then applies the scope: an in-scope record behind 40 nearer ones the caller
cannot see was silently dropped. #4958 fixed the rule search alone; the note,
milestone and system searches kept the fault.
_scoped_chunks builds the in-scope chunks with their distance; _rank_scoped
materializes them as a CTE and ranks exactly. All four searches go through it,
and the row shape is unchanged, so callers and mocks are untouched.
Tests: a structural guard that every semantic_search_* ranks through
_rank_scoped and orders nothing itself (with a replay of the old shape), a
compiled-SQL check of the MATERIALIZED CTE, and integration crowd tests - 80
nearer out-of-scope records - for notes (another user; the reader own other
project), milestones and systems.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_prompt_rule_hint, build_tool_rule_hint and build_write_path_hint
each ran the rule arm and then the via-lesson step by hand, the first two
passing the query between them through a _via_query key on the payload.
They now call one composer, _rule_moment, and the split helpers
(_prompt_rule_hint, _tool_rule_hint, _add_rules_via_lessons) and the side
channel are deleted. Output shapes are unchanged: the prompt builder
returns no checkpoint key, the tool builder always does, and
shown_rule_ids stays the direct band.
A structural test pins that only _rule_moment runs a rule arm or the
via-lesson step, and that the three builders are its only callers.
plugin_context.py: 2,418 -> 2,361 lines this step; 3,558 when the
milestone began.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each arm spec now carries its tuning pair (Surface, moved verbatim into
retrieval_pipeline) and a Declared block - what the telemetry readout
must know and cannot read off its rows. The two ranked stages that are
not arms (preference_slot, rule_via_lesson) are RankedSource specs, and
the note slots carry their own declaration.
- retrieval_surfaces.SURFACES = the TUNED_ARMS tuning, same order
- retrieval_registry.POINTS ranked rows = one per spec in RANKED; the
lookups, asked, ambient and pull rows stay declared there
- rule_usage.RANKED_SOURCES = RULE_RANKED_SOURCES + moment_rule
Settings keys, defaults, prose and order are unchanged (checked field by
field against HEAD). tests/test_retrieval_specs.py pins the derivation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_pipeline gains the notes half: NoteArm / NoteSlot / NoteMoment /
NoteIO / NoteResult and run_note_arm, which writes once the stages both
notes arms copied: search, withhold this response's own menu (#3739),
fresh/repeat split, the call row before any return (#3497, #3752), the
band, the reserved slots in their order (reuse evicts, lesson extends),
and the surfacing rows. The note renderer (_record_kind, _menu_name,
_menu_passage, the seen pointer, menu_entry) moves with it, and
run_via_lesson_arm takes rule_via_lesson.
Behaviour-preserving, with flags for today's differences: notes still log
BEFORE the band and rules after it (step 7's question). One deliberate
change: a notes arm now fails open like the rule arms, so a failing
search costs its lines and no longer the whole hook response.
The I/O is resolved from plugin_context at call time (_note_io), so the
existing patches keep working. The review re-run reads the AUTO_INJECT
spec instead of restating it, and its guard now compares the two live
searches. The registry declares the pipeline's notes fan-out sites.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 8218: integration failure, Build & push skipped, :dev still package
version 11098 (23:36:29Z, run 8217), and no image tagged 9a76eb0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverted next. Expect: integration red, Build & push skipped, :dev unmoved,
no image tagged with this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The integration job was added after the build gate (c6211e5) and never
joined its needs, so main run 8212 published :latest with integration red.
Both jobs carry the same ref condition, so the gate cannot be satisfied by
a skipped integration run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ordered straight off rule_embeddings, the planner walked the HNSW index,
which returns about hnsw.ef_search (40) nearest chunks across every owner
and project and filters by home only afterwards. A reader's own rule ranked
past the 40th chunk overall was silently dropped - on a shared install,
other users' rules fill those 40. This is the likely cause of the
intermittent test_integration_rule_scope failures, whose axis-vector
fixtures sit far from every real embedding in the graph.
The in-scope chunks now go through a MATERIALIZED CTE, which the index
cannot order, so the ranking over them is exact. A rulebook is hundreds of
chunks; exact is cheap. New test: 80 nearer rules belonging to someone else
no longer hide the reader's one rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
semantic_search_rules fails open, so both failures on run 8209 and main
run 8212 read as set() with no trace. Assert report["searched"] and carry
the swallowed traceback into the failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The open-after-moment signal proposes a mount; nothing proposed taking one
off, so a wrong mount was noise at every occurrence until someone happened
to notice. rule_misfired(rule_id, moment, why, reached_by) records a report
against a MOUNTED pair, counted per distinct day (the MCP door carries no
session id) on a new rule_moment_judgments.misfire column (migration 0119,
backup v24). At three days the response carries a line asking the agent to
offer the operator the fix - reject takes the rule off, unmap_action stops
the action reaching the moment, confirm keeps the mount and stops the
asking - and Settings > Moments lists it as an unmount proposal with the
reasons and the actions that reached it. A re-mount clears the count.
Taught in moments.md, missed-retrieval.md and the reply hold's wording.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Until now only the tool arguments knew moments existed. The guidance
surfaces described rules as reached by resemblance alone:
- using-scribe: a short reflex paragraph and a new reference file,
moments.md. It covers reading "at <moment>, reached by <action>", the
reply held once at reply.report, map_action / unmap_action offered in
one line, and the step 7 proposal line answered with judge_rule_moments.
- writing-records: asks WHEN a rule applies as well as what it is about.
A rule, preference or process about a point in the work gets
moments=[...] as it is written, and the trigger stays as the net.
- missed-retrieval: a missed WHEN is mounted or mapped, not reworded. A
misfire is unmounted or unmapped.
- _INSTRUCTIONS: one clause (list_moments; mount rules about WHEN),
1594 of 1600 chars.
- static context: injected lines include the rules mounted on a moment
that was reached.
- test_guidance_ownership: three owned topics, so the text cannot quietly
drop out.
Plugin minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule written before moments existed is mounted on nothing. Step 7 records,
per (rule, moment), whether it belongs there and who said so:
- rule_moment_judgments (migration 0118, backup v23): suggested / confirmed /
rejected, from a pass, the signal, or an edit. Moment "" is "no moment fits".
- The pass: rules_to_mount lists unjudged rules; propose_rule_moments records
suggestions that mount nothing; rule_moment_proposals and
judge_rule_moments put them to the operator. A confirm mounts, a reject is
kept so the pair is never proposed again. Same service behind REST and a
"Waiting on you" panel in Settings > Moments.
- Edits are judgments: set_rule_moments, the one mount write path, confirms
what was added and rejects what was removed in the same transaction.
- The signal: scribe_moment.sh keeps a per-session acts ledger; when a rule
is opened, scribe_record_opened.sh sends the last three minutes of it to
/api/plugin/rule-opened. The acts resolve through the install's mappings;
work.run and work.change are not evidence. Counted per distinct session
with lesson_rules' evidence model, and once due the open returns one line
asking the reader to offer the mount.
- scribe_session_end.sh removes the session's scribe-moment files.
Plugin 2026.10.05.2003.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Everything an agent can do with moments, a person can now see and change
in the app.
- Rule editor: a moment picker beside the trigger. Catalog moments are
ticked; a named procedure's `skill.<name>` is typed and checked as the
server checks it. `moments` is always sent, so unticking the last moment
unmounts the rule.
- Settings, Moments section (General tab): for each moment, what it
means, the actions that reach it on this install (shipped ones can be
switched off, the install's own removed), how many rules are mounted on
it, and deliveries and agent opens over the window. Below that: named
procedures with mounts, switched-off defaults with Restore, and a form to
add an action.
- retrieval_telemetry.moment_usage: per moment, `delivered`, `rules`,
`opened` (agent pulls after the first delivery there; an upper bound, as
by_source is) and `last_delivered_at`. No ratio, because a mount is a
person's statement, not a ranker's guess. Guarded on its own, and also
reported in retrieval_summary as `moment_usage`.
- rulebooks.mount_counts; mounted_moments now derives from it.
- GET /api/retrieval/moments carries `mounted` and `usage` (?days=).
DELETE /moments/mappings also reads the mapping from query parameters,
since the browser's DELETE sends no body.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Loading a procedure now also reaches the moment it is for. Loading the
reporting procedure is a report; loading the release procedure is a delivery.
- Bundled skills: each SKILL.md declares `metadata: moments:`. The same
declaration ships as Skill defaults (BUNDLED_SKILL_MOMENTS), because the
server never sees the plugin's files. test_skill_moments holds the two
together and pins the plugin name that qualifies the skill.
- Stored processes: `moments` on create_process and update_process, stored
in the note's data and returned by get_process. A `scribe-proc-<slug>`
load resolves its process through the sync manifest at load time. The
moments are not copied into the stub, which would go stale mid-session.
- reachable_tools lists the skill loader whenever anything is mounted, since
a process's moments are known only when it loads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moment_delivery records the mounted half's surfacing under
source=rp.MOMENT_RULE_SOURCE. That is an attribute the registry's
extractor cannot resolve, so the site is now declared with the value it
emits, read from the pipeline's own constant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reply is the one act no tool call marks, and it is where "let me
know if it works" gets said. A new Stop hook (scribe_reply_check.sh)
sends the finished reply to POST /api/plugin/reply-rules, which checks
it twice:
- mounted: every unopened RULE on reply.report, plus reply.ask when the
reply asks a question. Deterministic.
- semantic: the reply's head and tail against every rule's trigger, on a
new ranked surface, reply_rule. It is the backstop for whatever the
earlier arms missed. Its floor is its stop bar (default 0.80, budget
1), with its own Settings dials. The new stop_only stage records
surfacing for the rule that holds and nothing else, because nothing
else reached anyone.
Following the operator's ruling from 456 step 8, a rule that holds blocks
once, in the server's words. The hook blocks only on a reason it was
given, so an unreachable instance never stops a session, and it never
holds the rewrite. The ledger is the act checkpoint's own, so a rule
holds a session once across both doors and the per-session cap counts
both.
The turn reader moved from the report check into scribe_defs.sh
(scribe_turn_facts / scribe_turn_fact), so the two Stop hooks read a
turn the same way. The output was checked identical on a real transcript.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two guards caught 7865313:
- test_mcp_tool_processes reads every coroutine in a tool module's
namespace as a tool, and a name-imported attach_moment_rules looked
like one. All seven modules now call moment_delivery.attach_moment_rules,
and the parity guard accepts the attribute form.
- The session-ledger convention: a file in a swept directory must be a
.ids ledger. The tool-list cache describes the install, not the
context, so it moves to its own directory, scribe-moment, where a
compaction does not sweep it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule mounted on a moment now reaches the session when an act reaches
that moment, with no semantic match involved:
- run_moment_arm on the pipeline: a lookup, not a ranked search. Each
line names the moment and the act that reached it ("at work.deliver,
reached by `git push`"), so a misfire is visible where it lands and
can be unmapped in-session. A repeat is cited, not quoted; fresh
rules are recorded surfaced under source moment_rule with the moment
in detail. No retrieval_logs row, as for the other lookups, so no
latency is persisted for this arm.
- rule_scope: a rule's home clause, moved out of semantic_search_rules
so the moment lookup scopes by the same one.
- rulebooks.rules_on_moments / mounted_moments.
- The plugin door: a catch-all PreToolUse hook (scribe_moment.sh). It
keeps /moment-tools' answer on disk for five minutes, so a call to a
tool that cannot reach a mounted rule sends nothing, and an install
that has mounted nothing sends one request per window. It shares the
rules ledger with the other arms and fails open silently.
- The MCP door: Scribe's own tools named by the shipped mappings carry
moment_rules in their response, so a client without the plugin gets
them too. The hook skips those tools. A guard pins the attach on
every one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rule_moments (migration 0117) records which moments a rule arrives at, by
catalog name, cascading with the rule. rule_detail, the one seam every
rule door already returns through, gains moments beside system_ids: None
leaves the mounts alone, a list replaces them. get_rule and both
list_rules doors read them back, batched per page.
All five MCP rule/preference writes and the three REST ones take moments
and validate them before their create or update. An unknown name is
refused with the catalog listed and leaves no half-made rule behind; a
parity test pins that ordering on every door.
Backup v22 carries the mounts as a join table remapped through the rule
map; a real-Postgres round trip checks they land on the restored rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moment_actions.resolve(tool, input) names every moment a call reaches and
the action that reached it. One call can reach several: kubectl apply is
a run, a deliver and a reach outside the workspace. Command tools match by
how each segment of the line starts, with a word boundary; other tools by
field=value arguments. The MCP server prefix and case are ignored.
56 shipped defaults cover the harness tools, Scribe tools and common
command shapes. moment_mappings (migration 0116) holds what an install
adds and the defaults it switches off. A removal is a stored row, so an
upgrade does not switch the default back on.
Per the operator ruling, corrections happen in the session: map_action
and unmap_action (write tools) return now_reaches so the fix can be
confirmed in the same reply. list_moments now shows each moment's
actions on this install. REST mirrors both doors, recorded as human.
Backup v21 carries the mappings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourteen generic moments of work (session.start, work.start … reply.ask)
plus the skill.<name> family, each with what it means and the kinds of
action that reach it, written for any kind of work rather than software
alone. The catalog is code because every install needs the same mount
points; which actions reach a moment is per-install data (step 2).
require_moment refuses an unknown name with the catalog listed, so a
typo cannot become a mount that never fires. list_moments (read-only)
and GET /api/retrieval/moments hand out the same catalog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
reply_preferences.completion_preferences was the last hand-written
rule search + record_retrieval + record_rule_surfaced triple outside the
pipeline. It is now REPORT_PREFERENCE, a RuleArm with kind="preference"
over COMPLETION_QUERY, read back as records rather than as lines.
- RuleArm gains `kind`. _ranked asks the ranker for that kind and drops
anything else on the way out, before the row is logged.
- RuleResult gains `shown`, the (score, rule) pairs behind the lines, for
callers that return records.
- RuleMoment.project_id may be None: logged as given, searched as
`project_id or None`. report_preference rows keep their NULL project.
- Guards: reply_preferences now has 0 direct rule searches. The registry
constant-source example moves from report_preference to preference_slot,
the pipeline slot that records through a module constant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 8137 had two failures, both source-scanning guards whose property
moved to the pipeline. The property itself still held:
- test_every_surface_name_is_a_real_telemetry_source looked for
source="<name>" literals. The rule arms now record under their spec field,
so the spec is the join key, read from RULE_ARMS.
- test_every_hook_rule_search_says_which_project_it_is_for counted 4 direct
searches in plugin_context. It now expects 0 there, so a copied arm fails
the count. It also asserts that the pipeline has exactly one search and
that its keyword literal carries project_id and never everywhere.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prompt_rule, pre_tool_rule and the rule half of the write path each wrote
the same steps out by hand: search, band, split fresh from repeats, log the
call before any early return, reserve a slot, render, record surfacings.
The copies drifted, and #3497, #3750 and #3752 were fixed one copy at a time.
- New services/retrieval_pipeline.py. run_rule_arm runs the stages in one
order. RuleArm is the spec (source, band, compact_tail, checkpoint,
preference_slot). RuleMoment is the query and the session ledger.
- The I/O (ranker and recorders) is passed in as RuleIO. plugin_context
resolves it from its own names at call time, so existing patch points
still apply.
- _rule_band, _rule_hint_line, checkpoint_for, checkpoint_reason, the band
constant and the preference slot moved into the pipeline unchanged.
plugin_context re-exports them.
- A recorder that raises now costs only its row, never a rendered line.
- The flags reproduce today exactly. Whether prompt_rule should band, and
whether the act arms should reserve a preference, are step 7.
- Registry: the pipeline call sites are declared in FAN_OUT_SITES, with
values read from the specs.
- The #3497 structural guard now checks the one implementation, and that
plugin_context writes no rule-source row of its own.
plugin_context.py: 3,558 -> 2,980 lines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single plugin request fans one query out to several ranked arms. The
operator message is searched by auto_inject, its reuse and lesson slots,
prompt_rule, the preference slot and rule_via_lesson, and each arm called
get_embedding on its own: up to six model calls for one vector.
- embeddings.query_embedding_memo(): a request-scoped ContextVar memo that
get_embedding consults. Outside a scope the default is None, so every
other caller is unchanged. A failed embedding is not remembered.
- memoized_query_embeddings decorates /retrieve, /tool-rules and
/prior-art.
- get_embeddings (document chunks) is untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4797 "A lesson's what takes a whole narrative, and the narrative becomes
its name". `what` is the title every listing and menu prints. Nothing
enforced its documented "one line", so lessons written with the incident
in `what` printed up to ~1,500 characters as a menu line, buried the
claim, and diluted the trigger in the embedded title. That was 14 of the
39 lessons on this install.
- lessons.require_claim refuses a `what` over WHAT_MAX_CHARS (240) or
running over several lines. The refusal says the story goes in
`insight`.
- Both doors call it before writing:
- MCP create_lesson / update_lesson;
- REST create / update.
An update checks only a NEW name, so a lesson stored with a long one can
still take the edit that repairs it.
- lessons.claim_line is the display half. _menu_name shows an over-long
stored name as its first sentence marked " …", because a door guard
does not undo rows already stored and an unrepaired install would keep
printing them.
- The tool docstrings state the bound.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4798 "The rules corpus's surfaced_never_pulled warning sends the reader to
menus_to_review, which can only review auto_inject". #4772 gave the warning
one remedy for both corpora: "judge a sample with menus_to_review". That is
right for notes, where a line carries its passage. For rules it is a dead
end: retrieval_review.REVIEWABLE holds only auto_inject, so the tool
refuses every rule arm.
- The reading is now per corpus (_NEVER_PULLED_READING). For rules, the
text says:
- the count includes rules that only arrived in a listing;
- a rule can rightly be set aside on its trigger alone;
- no judged sample exists for the rule arms;
- a rule set aside again and again is a trigger to fix (update_rule
when_to_apply, then what_might_apply), not a floor.
- The tool docstring says the same.
- The guard is tied to REVIEWABLE, so it can fail in both directions: if
the rules text names menus_to_review while no rule arm is reviewable, or
if a rule arm becomes reviewable and the text still says there is no
sample.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4796 "A record the operator names by number reaches auto-inject only if
its wording happens to match". "yes go ahead with 4448" says which record
is meant, but a number means nothing to an embedding, so the prompt menu
filled with records resembling the words around it.
- record_refs.named_record_ids reads the operator's raw prompt, never the
reply-enriched query:
- `#N`, unless the word before it marks another numbering (PR, CI, rule,
milestone, system, log...);
- a bare number of 3 or more digits that opens the message, follows a
reference word or continues a list one started;
- never a quantity ("300 seconds"), a date, version, path or fenced code.
- Each id is resolved through the ACL check; trashed or inaccessible ids
are dropped.
- A "Named in your message" block leads the menu: the kind, the System,
the name and the opening of the body, or the seen pointer if the record
is already on the ledger. It takes no share of top_k, and the semantic
lines leave those ids out.
- The block is booked under the new `named_ref` source, registered as an
unbidden lookup that is allowed to be quiet. A named id that also ranked
counts as suppressed in the auto_inject row, so #3668's identity holds.
- Named records now arrive even when the search finds nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SDK 2.x passes on only a ToolError's text. Any other exception becomes
UnexpectedToolError("Error executing tool X") and its message stays on the
server. ValueError is how every Scribe tool refuses — what was refused, why,
what to do instead — so every refusal reached the agent bare, and it could
only retry blind. Seen live: tune_retrieval(actor="operator") and
judge_menu(verdicts=[]) both answered "Error executing tool …" and nothing
else.
StrictArgsMCPServer.call_tool re-raises an UnexpectedToolError caused by a
ValueError as a ToolError carrying the message, in the SDK's own
"Error executing tool X: <reason>" shape. Other exceptions are crashes and
stay masked. The stale comment claiming the SDK returns ValueError text is
corrected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`<agent-message …>` turns were scored by auto-inject and the prompt rule
arm as if the operator had typed them, spending a retrieval and writing a
retrieval_logs row that reads as a real message (log 54355, found by the
#4772 review). Same class as #4142. Matched up to the tag name, since the
tag carries attributes; `<agent-messages>` and the tag named mid-sentence
are still retrieved against.
Plugin 2026.10.04.0125.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Judging live auto-inject menus (#4772) found "## Work log — 2026-08-21",
"## Findings worth carrying forward" and "Two dev→main PRs this session."
each embedded as a whole chunk. Text about nothing embeds close to
everything, so they took ranks 1–3 on vague queries and pushed real records
down.
Three ways the chunker made them, each closed:
- _split_paragraphs flushed a heading on its own when the paragraph under it
was long. A thin lead is now carried into the paragraph that follows, and
a hard cut never lands in the first quarter of the budget, where the
newline it finds closes a heading.
- A heading with no body (above its ### parts) became a section of its own
when the section before it was full. A thin section now takes the next.
- A short lead-in or a short last section stood alone. The lead-in joins
what follows; a thin tail joins what precedes it.
A continuation piece now repeats the LAST heading before it, since one
section can now hold several. CHUNKER_VERSION 2 → 3, so the startup backfill
re-embeds; tuned dials will report their shape as changed, which is true.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first live sample ranked records the call could never have been offered:
the session that made a call writes the decision, often quoting the message,
and that record tops the re-run. Judged, it inflates on_point exactly where
the budget is decided.
- _rerun over-fetches by POSTDATED_SLACK, drops records created after the
call before ranking, and names them in `postdated`.
- each line carries `changed_since_call` (updated_at or a work log after
the call) and `logged` (shown fresh then).
- judge_menu shares the re-run, so a post-dated record cannot be judged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An open rate cannot say whether a menu line related: every line carries its
matched passage (#4364), so "not opened" covers unrelated, enough as shown,
and already in context. #4772 "Injected notes are never judged".
- retrieval_judgments (0115): a reviewer verdict per line of a logged call,
on_point / adjacent / unrelated, with its reason, rank, budget side and
whether the agent opened it within the hour.
- menus_to_review re-runs a random sample of unjudged auto_inject calls with
the arm's own parameters, past its budget, passage on every line.
judge_menu records verdicts, re-deriving rank from a fresh re-run.
- retrieval_telemetry gains a judged block (by rank, within/beyond budget,
on_point_unopened). surfaced_never_pulled stops blaming titles.
- missed-retrieval guidance names the review before a budget move.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4769 "Rulings are counted where someone will read them": milestone 444
step 4 wrote system_usage_events and nothing read it.
- retrieval_telemetry gains a `system_usage` block: surfacings and opens by
source, distinct counts, and `by_system` naming the areas most shown.
There is deliberately no pull-through ratio, because rulings travel in full
in the line and opens are the exception.
- usage_for_systems (one GROUP BY) adds `usage` to the REST Systems list and
detail, and to MCP get_system. MCP list_systems is unchanged.
- The Systems UI shows a "rulings shown N×" chip.
- rulings_pre_tool, rulings_write_path and mcp_get_system are now declared
registry points; the registry guard covers their recorders.
- The Systems store merges a PATCH reply instead of replacing the row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4768 "Write-path prior-art route 500s when a nameless search hit carries
its excerpt under `snippet`": _prior_art_line read item["snippet"] as a
snippet record's field dict, but a search hit carries its matched passage
there as a string. Read the name only from a dict.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI 7939 went red on test_the_hook_and_the_route_agree_on_every_parameter_name:
the Bash hook now sends root and cwd, and the guard pins the exact set. Both
guards (tool-rules and prior-art) now also pin seen_ruling_systems, spelled
once in scribe_rulings_query, and check the routes read all three.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A System's rulings (the Rulings section of its description) now reach the
work by path, not by similarity. Both PreToolUse arms resolve the files a
command or edit names to the Systems whose path_patterns cover them, and the
first touch in a session shows each area's rulings in one line; a repeat is
a one-line reference. A lookup, so no floor, no budget, no retrieval_logs row.
- services/system_rulings: parse_rulings, command_paths (reads and writes,
relative to the repo root from any cwd; flags, URLs, globs skipped),
rulings_for_paths
- /tool-rules takes root, cwd and seen_ruling_systems; /prior-art takes
seen_ruling_systems; both return ruling_system_ids
- hooks share <sid>.rulings.ids (cleared on compaction by the ledger naming
convention); the Bash hook sends the repo root and cwd
- system_usage_events (migration 0114): surfacings by source, pulls from
get_system; carried by backup (v20) through the system map
- writing-records: rulings also arrive when the area's files are touched
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A System gains path_patterns: globs relative to the repo root (* within one
directory, ** across any depth, a plain directory covering everything under
it). One service validates them for every door, so the web UI and MCP refuse
the same bad pattern with the same message. systems_for_paths resolves paths
to every active System that covers them, which step 4 (#4757) uses to deliver
an area's rulings when its files are touched.
- schema: systems.path_patterns JSONB NOT NULL default [] (migration 0113)
- service: normalize_path_patterns, path_matches, systems_for_paths
- routes + MCP create_system/update_system accept it; [] clears
- web UI: a Files field in the create and edit forms, patterns on the card
- backup carries it through export and restore
- using-scribe reflex 7: tagging work keeps a System's files current
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Librarian session contradicted a decision the operator had made 13 days
earlier. The ruling ("retry, then replace, never give up on a book") was
kept only as a quote in a work log, beside a session's reading of it that
capped replacements at 3. Three later sessions built on the reading, and one
carried the cap into an option as a "known cost", which the operator then
approved without being asked about it.
- writing-records.md: "A ruling goes on the System it governs". What a
ruling is (the operator decided it; a later change could undo it), how it
differs from a rule, and where it goes: a Rulings section at the end of
the System description, one line each with who, when and the source
record. Written the turn the operator decides; holds what is in force,
not history; a charter line that contradicts a ruling is fixed in the
same edit.
- using-scribe SKILL.md: reflex 1 says code tells you what a thing does,
not what was wanted, and a limit read from code is unconfirmed until a
System's Rulings says otherwise. Reflex 9 points to the ruling section.
- reporting-back: an option that carries existing behaviour says whose call
it was (the operator's ruling, or a past session's never confirmed); one
that contradicts a ruling is a Conflict.
- create_system / update_system docstrings: the Rulings section, and that
description replaces the whole text.
- test_guidance_ownership: three topics pinned to their owners.
- Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lesson editor now asks the question the create tool asks, while the
writer still has the situation in mind. It offers three answers:
- an instance of a rule, ticked from the rules the lesson resembles;
- "No rule fits", with a required reason;
- leave it open.
The rules are fetched before the save from a new GET
/api/lessons/rule-candidates. It runs the same rule_candidates search the
create door replies with, and passes "search unavailable" through as null,
apart from "nothing resembles it".
An edit sends an answer only when it changed. Re-sending the same rules
would re-stamp their judgments. Worse, a shared editor who cannot see the
owner's rule would reject it by sending a list without it. Leaving a linked
lesson open sends rule_ids=[], which is what clears the links.
"Leave it open" is not offered once "no rule fits" is on file: the server
has no way to take that answer back except by naming a rule. When "no rule
fits" completes a convergence group, the editor says so in a toast, since
the page it goes to does not recompute it.
Guards check that the payload uses the names both routes read, that the
three answers are offered, that the candidates route sits above the id route
and keeps null apart from [], and that the editor reuses .rule-chip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lesson page gains an "Instance of" panel that holds one of three answers.
Unjudged is the fall-through, so it is stated rather than left blank:
- the rule(s) it was judged an instance of;
- "No rule — <why>";
- "Not yet judged".
Suggested links show what they rest on (distinct situations, and projects
when more than one), with Confirm / Not an instance for a reader who can
write. Rejected links stay listed with their reason.
The rule slide-over lists the lessons that are instances of it, plus the
suggestions waiting on a judgment. Each entry links through to the other
record, and kind and state wear the existing .rule-chip.
The write check moves to utils/permission.ts. The copy on the snippet page
looked for "edit", which the server never sends, so shared editors saw a
read-only page (#4640 "The snippet page hid its edit controls from shared
editors"). Guards pin the client's unions and write levels to the
service's, the model's and access.py's own values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a lesson is answered "no rule fits" (create_lesson / update_lesson on
both doors), the response looks for other no-rule lessons it resembles and,
once there are CONVERGENCE_LESSONS (3) of them, carries `convergence`: the
members, their incidents and projects, and a hint to draft the missing rule
with create_rule (operator approval as always) and point each lesson at it —
or to leave them as lessons when no single choice is right every time.
- convergence_group is the pure bar: distinct LESSONS count, incidents never
stand in for them (one broad lesson cannot trigger it), and a group whose
sources all point at one incident is one event written up several times.
- convergence_for searches lessons by the new one's claim + trigger
(trigger_title) at CONVERGENCE_THRESHOLD 0.65 — above the menu's "worth
showing", below the duplicate gate's "same record" — then keeps the ones
with a lesson_no_rule answer. Fail-open. No sweep, no timer (#4183).
- Defaults stated as defaults (rules 32, 115).
- Tests: the bar (pure), the search with stubs, the door, and the no-rule
filter against Postgres; conftest stubs convergence_for for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 712 failed test_neither_rule_arm_logs_its_call_behind_a_results_guard
with "min() iterable argument is empty": the guard walked the function named
build_tool_rule_hint, which since fd2ebf4 is a wrapper that searches nothing
— the arm's body moved to _tool_rule_hint. The property it guards (the call
is logged before any early return) still held; the guard had pinned a name.
It now finds the async function that calls semantic_search_rules under the
pre_tool_rule source, so a later rename moves the guard with the arm instead
of emptying it (rule 167).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On all three rule arms (prompt_rule, pre_tool_rule, write_path_rule), after
the direct match, a lesson matching the moment at the notes menu's own bar
brings the rule(s) it is CONFIRMED to be an instance of, rendered in rule
voice with "Reached through lesson #N “…”, a recorded instance of it."
- Confirmed links only (lesson_rules.confirmed_lessons /
confirmed_rules_in_scope). A suggested link carrying its rule would
manufacture the co-arrival #4637 counts and prove itself.
- Scope kept: a lesson surfaces everywhere, but the rule it brings must be
global or this project's own — milestone 414's boundary, not reopened
through a side door.
- Suppression is by RULE: anything the direct band named or the session
ledger holds is skipped, whichever lesson reached it.
- Its own slot (VIA_LESSON_LIMIT = 1), not a rule slot — a stated default,
since the plan's "decide by measurement" has nothing to measure until
links are confirmed (#4632). Its own source, rule_via_lesson: registered,
RANKED for pull-through, logged whenever it searches.
- Nothing searched at all while no lesson carries a confirmed link, which is
every install until one is judged.
- build_prompt_rule_hint / build_tool_rule_hint are now thin wrappers over
the direct arms (_prompt_rule_hint / _tool_rule_hint); the arm leaves its
query in `_via_query` only when it ran, so a disabled arm or a blank prompt
brings no rule in either, and the key never leaves the server.
- Via-lesson rules are not fed to the #4637 co-surfacing recorder: only
direct matches are evidence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When one hook response puts a lesson and a rule in front of the reader, the
pair is recorded as evidence on a SUGGESTED lesson_rule_links row; once the
pair has arrived together in PROPOSE_SITUATIONS (3) distinct situations, the
next co-arrival carries one line asking the reader to judge it with
judge_lesson_link. Nothing about surfacing changes: a suggested link carries
no rule anywhere (that is #4633, confirmed links only).
- lesson_rules: co_surfaced (fail-open; only pairs the reader could confirm —
a lesson they may write, a rule they own; judged pairs gather nothing; one
proposal per response; PROPOSE_COOLDOWN 6h between asks), plus the pure
counting rules: situation_key, add_evidence, proposal_due, evidence_summary.
A situation is the prompt on /retrieve (word tokens, sorted and
de-duplicated, so trivial rewordings count once) and the FILE on
/prior-art (every edit to one file is one situation).
- rules_for_lessons shows a suggested link's evidence counts.
- plugin_context: build_autoinject_hint returns lesson_ids,
build_prompt_rule_hint returns shown_rule_ids, build_write_path_hint records
its own pair; routes/plugin /retrieve records the prompt pair. Shown lines,
repeats included: relevance makes a co-arrival, not the session ledger.
- Evidence lives in the existing evidence column — no migration, and backup
already carries it.
- Tests: counting rules (unit), the recorder against Postgres (bar, repeat,
judged pairs, ownership, cooldown, evidence kept on confirm); conftest stubs
co_surfaced for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 707 failed test_the_trigger_separator_is_spelled_in_exactly_one_place:
rule_candidates joined the lesson's claim and trigger with an inline " — ",
a second spelling of embeddings.TRIGGER_SEP (#3207). It now calls
trigger_title — the same join a rule's own document title is built with,
which was the point of the query shape in the first place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Which rule is this lesson an instance of?" now has three recorded answers:
a rule named (a confirmed link, #4630), no rule fits (new), or unjudged.
- Model + migration 0112: lesson_no_rule (lesson_id PK, CASCADE from the
note; why; judged_at). A table rather than a key in notes.data, because
that mirror is re-composed from the body on every edit and would erase it.
- Service (lesson_rules): set_no_rule rejects any confirmed link with the
reason; a confirmation (set_lesson_rules or judge_link) deletes the answer;
require_one_answer refuses both answers in one call before any write;
judgments_for_lessons + attach_lesson_rules add rule_judgment (and no_rule)
to every lesson payload; list_unjudged lists the open ones; rule_candidates
searches rules with the lesson's claim + trigger at the explicit-search bar,
None when the search could not run.
- MCP: create_lesson/update_lesson take no_rule; an unanswered create returns
rule_candidates, rule_judgment and a rule_hint; list_lessons(unjudged=true).
- REST: the same on POST/PATCH /api/lessons and GET ?unjudged=1; create
returns rule_candidates.
- Backup v19: a lesson_no_rule section, export (full and per-user) and import.
- Guidance: create_lesson docstring, writing-records.md in using-scribe (owner,
pinned in test_guidance_ownership), create_rule docstring on linking the
lessons a new rule governs. Plugin version minted.
- Tests: door units, integration for the three states, the rejection reason,
scoping, cascade; backup registries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tool-classification guard (test_mcp_auth) failed CI run 705: the new
judge_lesson_link tool sat in no set, so a read key would have been silently
denied it. It changes a link's state, so it belongs in _WRITE_TOOLS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The link between a lesson (one concrete situation) and the rule that governs
it, with the operator's soft-then-hard design built into its state:
suggested while evidence accumulates, confirmed or rejected once judged. Only
confirmed will carry a rule in retrieval (#4633); rejected is kept so the pair
is never proposed again.
- models/lesson_rule_link.py + migration 0111: one row per (lesson, rule),
CASCADE on both ends, indexed both ways, CHECK on state (rule 36), evidence
JSONB and judged_at.
- services/lesson_rules.py: require_rules (validated before any write, so
a bad id leaves nothing half-linked), set_lesson_rules (set-semantics;
a dropped rule becomes rejected, not forgotten), judge_link, and the two
reads. ACL: write on the lesson (share-aware), ownership of the rule; a
reader sees only rules they own. Decorations are fail-open (#4286).
- MCP: create_lesson / update_lesson take rule_ids; get/create/update return
`rules`; new judge_lesson_link tool. REST: the same on /api/lessons plus
PUT /api/lessons/<id>/rules/<rule_id>. Rules: rule_detail carries `lessons`.
- Backup v18: export (full and user-scoped, both ends in scope), builder,
importer; both column guards register the table.
- Tests: integration (states, set-semantics, judge, ACL all-or-nothing,
cascade both ways, CHECK, one row per pair); unit (door wiring, judge
registered, migration/model state agreement, backup skip and unjudged
stays unjudged). conftest stubs the decorations for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone 385 step 8 (#3735 "a lesson is recalled on a project it was not
written on") is defined by opened-on-another-project. The reader's project has
been recorded on every usage event since 0c8e109, but nothing compared it with
the record's own, so the criterion was still unreadable.
- note_usage.usage_for_notes: a second aggregate in the same session joins
notes and counts surfaced_away_count / pulled_away_count. Counted only where
both projects are known and differ; ranked surfacings only (#2477). The
first aggregate is untouched, so events on deleted notes still count.
- empty_usage carries both keys zero-filled; every door that attaches usage
(list_lessons, get_lesson, snippets, knowledge) gets them through
attach_usage.
- UsageBadge tooltip says "On other projects: surfaced N×, opened M×" when it
happened, and nothing when it did not.
- Tests: the mocked split test feeds both aggregates; a unit test for the
away counters; a real-Postgres test that home, unreported and ambient
events are all left out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- reusing-code: "Before the turn ends — say what you built" — the four
verdicts, the one classify_shapes(repo=…) call, and the component file as a
shape. Description names the end-of-turn moment.
- shape-accounting: the writer judges; audits are the check that it held.
The write path SUGGESTS (no more hook instances); component file rows and
whole-file canon described; scoped covers Svelte too.
- _INSTRUCTIONS reuse line: "before the turn ends, say what you built
(create_snippet the reusable, classify_shapes the rest)" — 1570/1600.
- Coverage line: "written-shape check (7d): N turns checked, M asked, K left
unjudged", from the Stop hook's recorded outcomes; silent until the
question has been put.
- test_guidance_ownership pins the new topic on reusing-code.
- prior-art hook header no longer says it stamps instance rows.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single-file component defines `.card`, `.title` and a `Props`, never
anything named after itself — so the ledger could not hold "StatusChip is
canon" and could not say a new card was built where one already existed.
- coverage.is_file_unit: a file that RENDERS (its markup names classes, or it
opens a <script>/<template>/<style> block) and defines nothing named after
its stem gets one `file` row. Structural, not a framework list: Svelte and
Vue components qualify, a TSX component already has its function's row, a
module is accounted for by its definitions. Emitting every file would have
put every module of every project in the todo at once.
- No body fingerprint on file rows, so an edit never re-asks a judgment.
- `file` is its own form and family; derive grouping skips it (`index`,
`+page` repeat by convention). Divergence buckets it on its own, so a new
component where a component canon dominates is a fair question.
- mark_canonicals: a snippet recorded at a path with no symbol makes that
file row canonical; it still covers no definition inside.
- The write hooks apply the same test before noting a new file for the
end-of-turn question.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hook used to land its evidence as `instance` rows, classified_by=hook —
a permanent verdict nobody read, and the source of the weak stamps that
poisoned two ledgers (#4608). Now the agent that wrote the code judges it at
the end of the turn, and the hook's evidence is what it is shown.
- shape_ledger.suggest_write_path_instances (was stamp_write_path_instances):
same evidence and form gate, but it writes a PROPOSAL — proposed_snippet_id,
basis "reference" (named) or "semantic" (resembles), score — and never a
status. A judged row is left alone; a new shape gets an unclassified
provisional row so the suggestion reaches the end-of-turn question as
"looks like #N". By-name uses edges stay: a fact, not a verdict.
- The prior-art line says "looks like #N … when you judge this turn's
shapes, say whether it is", and the result key is `suggested`.
- write_time_divergence takes `suggested`; _RESEMBLE_MIN re-documented as the
suggestion bar and the weak-stamp line.
- reusing-code skill no longer says the pull stamps an instance.
- Tests moved to the new contract, plus a guard that nothing the hook does
writes classified_by="hook".
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recording used to be decided by machinery — the only "record it" prompt
fired when a same-named copy already existed (#2664), so a first instance of
a reusable piece was never asked about, and judgment arrived only through
audits. Now the question is asked where the knowledge is: the end of the
turn that wrote the code, of the agent that wrote it.
- Write hooks keep `<sid>.written.ids` (path, kind, name) for every
definition a write names; a new file adds a `file` line for its stem — a
candidate in any language without a framework rule (scribe_written_append).
- Stop hook scribe_shape_check.sh sends the ledger to GET
/api/plugin/shape-check and blocks once, in the server's words, when
anything is unjudged. Same discipline as the report check: never twice,
never without a recorded check, another hook's loop left alone; the ledger
is kept when the instance cannot be reached.
- shape_ledger.unjudged_shapes: no row, unclassified, scoped and hook stamps
are unjudged; an agent/audit/import verdict is not. A snippet recorded at
the shape answers for it until the refresh stamps it canonical.
- services/shape_check owns the reason text and records every outcome in
app_logs (passed / blocked / judged_after_block / left_after_block).
- classify_shapes(repo=…) judges a shape the ledger has not synced yet via a
provisional row under a bound repo; the sync confirms it, or vanishes and
revives it with the verdict intact. An unbound repo is refused.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Librarian and Stash had 132 and 488 write-path stamps written at 0.68-0.72,
before the 0.80 floor (#4204). Nothing re-judged them, and dominant_canon
counted them, so a YAML CI snippet (#3410) "dominated" Librarian's web
directory and every write there was told it diverged from it — 1408 flags
on one project, 902 on the other, all skimmed past.
- is_weak_stamp: one predicate for "hook-stamped below today's floor".
dominant_canon and canon_form skip such rows; stamps_to_review lists them
by the same predicate. No stored row changes — they stay for judgment.
- flag_divergence withdraws a standing flag whose canon no longer dominates
its directory. A flag is a mechanical prompt, not a judgment.
- The coverage line ends "N weak stamps · M incoherent canons to judge —
stamps_to_review" when either is nonzero, so a project's own session sees
the queue on arrival. The shape-accounting skill says what to do with it.
- scoped_definitions handles .svelte: every <style> is component-scoped
unless <style global> or :global(...); instance-script syms are scoped,
module-script ones stay ordinary.
Plugin version minted for the skill change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shape ledger's divergence readout flagged _gate_setting (#4385).
Reading it turned up a family with no canon: floor_for,
get_duplicate_threshold, get_plan_match_threshold and _gate_setting each
parsed a stored string, fell back to the default (never 0) and clamped
into [lo, 1].
- services/settings.bounded_float(raw, default, lo=0, hi=1) is the pure
parse, fallback and clamp. Each caller keeps its own get_setting read,
so tests patching get_setting per module still take effect, and
_gate_setting still fails open on an unreadable setting.
- SettingsView.saveKbInject: eleven inline Math.min/Math.max clamps and
the local gateAt become one asBar(v, d, lo = 0), mirroring the server.
No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Anthropic's skill guidance: keep SKILL.md under 500 lines, split into
reference files linked one level deep as it nears that. using-scribe was
478 and every new practice lands there.
- SKILL.md 478 -> 317 lines. It keeps orientation, one copy, the reflexes,
scope, the judge section, UI and the process-skill index, plus a "Read
these when the moment comes" list naming each file with its moment.
- projects.md: binding a non-git directory (.scribe) and project inception.
- writing-records.md: where a new rule goes, lesson growth, and notes that
carry their own check (reflex 10 keeps a pointer).
- missed-retrieval.md: the record-before-dial route, verbatim.
- Text moved, not rewritten, except for the seams and one cross-reference.
Tests:
- tests.helpers.skill_text reads SKILL.md plus its reference files. The
ownership registry, the miss-route and the verification tests use it, so
a topic stays owned by its skill whichever file holds it.
- The force test scans every skill .md on its own, since each file is read
on its own.
- New test_skill_structure: SKILL.md <= 350 lines, every reference file is
linked from SKILL.md, none links another, and one over 100 lines opens
with Contents. Each guard is shown to fail.
The plugin version is minted. That also clears 4fb53b8's red Plugin hooks
lane, which failed only because PACKAGING.md changed without a mint.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Spike #4389 read the spec, Claude's docs and a dozen servers: the field is
for how the tools fit together, and the field runs ~600-1,600 characters.
Ours sat at the 2,048 cap as a keyword index that also carried stance.
- JUDGE, REPORT and MISSED leave the index. They fire mid-work, not at
session start; using-scribe and reporting-back state them in full, and
`placement`/`report_back` cue reporting in-band. No skill text changes.
- The rest is rewritten as plain practices (1,503 chars) and keeps every
session-start marker the ownership registry pins.
- INSTRUCTIONS_BUDGET 2000 -> 1600; the three index markers are dropped
from the registry; the miss-route index test now checks that the index
keeps what_might_apply and stays off the route.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- server.py: `mcp.server.mcpserver.MCPServer`; StrictArgsFastMCP becomes
StrictArgsMCPServer, whose call_tool takes and forwards v2's `context`
and raises ToolError (the SDK logs anything else as an unexpected crash;
the message reaches the caller either way).
- stateless_http and transport_security moved from the constructor to
`streamable_http_app(...)` in mount_mcp, with their reasons.
- Per-request user identity is unchanged: the contextvar set around the
ASGI call reaches the handler on both v2 paths (legacy stateless spawns
from the request task; the 2026-07-28 modern path opens its task group
inside the request).
- pyproject: mcp[cli]>=2.2, no ceiling (installs are --locked). uv.lock
regenerated with --upgrade-package mcp in the ci-python image: only mcp
and its own dependencies moved (106 → 110 packages).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gate_bars(user_id, note_type) resolves the block bar and, for notes and
tasks, the overlap floor from kb_gate_* settings, with the old constants
as defaults. Fail-open on an unreadable value; a block bar clamps at 0.80
and the overlap floor at 0.70 and never above the bar. Five fields in
Settings beside the duplicate-report floors.
CLAIM_LEASE stays a constant, with the reason written at it: a per-user
lease would make one shared task live to one reader and dead to another.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
plugin-vue throws "Cannot read properties of undefined (reading 'scoped')"
when several SFCs share one src stylesheet. vue-tsc passed, so only the
image job caught it; :dev has not rebuilt since 22b7a92.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
create_log filtered on Note.user_id == user_id, a bare owner check, so a
collaborator with write access to a shared task was told it did not exist
— and, since a log now stamps the claim, could never be seen working it.
It now asks can_write_note. Editing and deleting a log still require its
author, which is authorship rather than access.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The note sweep, rule sweep, preference drift and rule history panes each
restated the same row recipe in a scoped block. It now lives once in
assets/sweep-shared.css under `sweep-` names (unprefixed globals would
leak into the unrelated scoped .row-title/.lede/.age/.actions elsewhere),
imported unscoped beside each pane's own scoped remainder.
Moved only what was shared: RuleHistoryPanel takes .sweep-state and keeps
its button row head; PreferenceDrift keeps its tiny-type footnote and
overrides the action row's layout. Lede and state now use the body-sm
token instead of 0.85rem/0.9rem (13px vs 13.6/14.4px) — one value, from
the design system.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Release is the mechanical half: scribe_session_end.sh sends the ending
session's id to /api/plugin/release-session, which releases the claims it
held. A tidy-up, not the guarantee (no SessionEnd on a crash; the lease
covers that), and skipped on /clear so SessionStart(clear) can still
hand the claimed work back.
Saying what happened is the half only the model can do. It is stated as a
practice where it is read: the using-scribe skill owns it ("Hand off before
this session's context stops existing", pinned in test_guidance_ownership),
the static context points at it for the wrap-up moment, and add_task_log's
docstring says a log claims the task. _INSTRUCTIONS is untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The claim now pays rent to the session that set it. The SessionStart hook
sends the host's `source` and the session id; the server renders a claim
section by source (task_claims.render_claims):
- compact / clear: the work this session had claimed, each with its two
latest log entries, so a compacted session resumes from the record
rather than from a count of open tasks.
- startup / clear: other sessions' live claims (may still be running) and
in-progress tasks whose claim went quiet (abandoned mid-task).
- fork: the same, framed as "two sessions may now hold this".
- resume, or no source sent: nothing.
The task sidebar shows a live claim as "Being worked" and a dead one on
open work as "Went quiet", so the operator sees a session die mid-task.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
status is durable and nothing clears it, so in_progress cannot also mean
"someone is on this now". The claim is that second meaning, stored as who
and when so it dies on read rather than needing to be cleared.
- notes.claimed_by / claimed_at / claim_touched_at / claim_session (0110).
- The server stamps it on the write that is the work: reaching in_progress
(update or create, via apply_status_transition) and a work log on an open
task. done/cancelled/todo release it. Live while touched within
CLAIM_LEASE (2h); dead on read past it, with no sweep.
- The plugin's PostToolUse hook on update_task/add_task_log binds the
harness's session_id (GET /api/plugin/claim-session). It binds only to a
live claim the caller holds; it cannot create one.
- to_dict carries `claim`. Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_note (create_note and create_records) set the status and nothing it
implies, so create_task(status="in_progress") wrote a started task with no
started_at, and done/cancelled had no completed_at or next recurrence.
The transition is now one function, apply_status_transition, called by
both paths; tests pin the invariant for every status.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The four embedding tables stamped chunker_version but not the model, and
vector(384) is a width, not an identity: a same-width model swap would
write a second geometry beside the first with no error.
- embedding_model on note/rule/milestone/system embeddings (0109; existing
rows stamped with the only model any install has ever run).
- Every write stamps EMBEDDING_MODEL; every backfill's "current" test is
is_current_stamp(), both halves of calibration_stamp().
- migrate_floor refuses while any row its surface searches is off the live
model, before sampling: re-embed, then migrate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on the live corpus, the 74 note pairs at or above the old 0.90 bar
were almost all distinct siblings — consecutive dev-logs, sub-notes of one
design, research parts — and the one clear copy sat at 0.997. The block
refused the next dev-log and taught force=true, as #4134 found for rules.
- The semantic arm blocks notes and tasks only at >= 0.98. The title block
stays; processes keep 0.90 (not measured).
- 0.87 to 0.98 comes back as `overlaps` on the create reply, from the same
per-chunk searches, with a note that leaves the call to the session:
fold in and delete if it is the same record, keep both if a sibling.
- create_note, create_task, create_records and start_planning's steps all
carry it; a batch names the record each overlap belongs to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snippet had no field for prose, so what a session learned about one went
into when_to_use — the trigger joined onto every chunk it is embedded as. A
sweep found write-ups of up to 3 KB there, headings and all.
- notes: stored after the code under `## Notes`, parsed back from the body,
carried by every path that rebuilds it (update, merge, un-merge). A
snippet with no notes composes the body it always did.
- create/update_snippet (MCP) take notes and return trigger_advice when
when_to_use is long, headed or multi-paragraph. Advice, not a refusal.
- Tool docs, the reusing-code skill and the editor hint describe the trigger
as the situation and point the explanation at notes.
- Editor gains a Notes field; the detail view renders it as markdown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The title was `subject — trigger` because the stored title WAS the
embedded one, and the join is what makes these kinds rank on the
situation they apply to (#2485). Every surface that shows a title then
showed the trigger too -- menus, lists and search rows ran to kilobytes.
- embeddings.document_title(title, note_type, data, body) joins the
trigger from `data` (body fallback) at embed time. Idempotent: an
un-migrated composed title comes out the same, never doubled. The
embed path, the startup backfill and the dedup gate's semantic signal
all use it, so the embedded text -- and every vector -- is unchanged.
- Writers store the subject: snippet create/update (service, REST, MCP)
and lesson_document. Both compose_title helpers are removed.
- Readers: dedup takes `data`; the menus strip the embedded title from a
passage; list rows project `when_to_use`, which SnippetListView reads.
- 0108 rewrites existing rows on an exact `' — ' || <own trigger>`
suffix with raw SQL, leaving updated_at alone so the backfill does not
re-embed the corpus for identical vectors. Downgrade recomposes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Injected lines rendered a snippet's or lesson's title, which carries its
whole trigger by construction (the embedding shape) and ran past 1,500
characters -- again on every `seen` repeat. The passage under a line was
cut to 200 chars from the middle, keeping its head (the title again) and
losing where the match was.
Now, on both the prompt menu and the write-path prior-art menu:
- the line shows the record's NAME (snippet data.name / lesson subject),
with its kind and System (`[issue (done) · Plugin & hooks]`);
- the passage is the whole matched chunk, title prefix stripped, on one
line so the blockquote holds; a title-only match hands over the trigger;
- a `seen` record is a one-line pointer to what is already in context.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The trailing newline became a space before the 600-char cut, so the cap
kept 599 characters of text; and under pipefail a grep that matched no
reply made the helper exit 1. Trim before cutting; return 0 explicitly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The context fetch folded a timeout, an HTTP error and a refused key into
one sentence that ended "enter_project() as needed" -- read as optional,
so a session could start with no recent milestones or open tasks and no
way to know what prior work existed. The fetch now names the cause and
the elapsed time, retries once (6s) only where a retry can change the
answer, and the fallback states enter_project as the first step, with
the marker's project id when there is one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prompt hook sent only the operator's message, so a mid-session
follow-up ("yes do that") named nothing a note, lesson or rule could
match. The hook now reads the tail of the last assistant reply from the
transcript and sends it as `ctx`; the notes and rule arms append it to a
short prompt (<= 280 chars), prompt first, capped at 600 chars. No model
tokens: it is embedding input, and the injected menu's budget is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured over 459 judged instance rows against every same-family canon:
true instances score 1.0 or ~0.795 (a model whose mixin list differs from
the canon's), and 0.8 cut the second band entirely. The proposal is read
before it is confirmed, so the floor admits it: 0.75 takes every measured
true row, at 35 wrong-canon pairings instead of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
0.8 was sized for proposals nobody reads; every semantic proposal is read
by a judge before it is confirmed. Measured, it proposed 0 of 150 while
judged instances of a canon score 0.68-0.71 - the write-path hint's own
floor asks the same question of the same documents at 0.68. The arm now
uses that floor, scans 8 hits instead of 3 (a true instance ranked 4th
behind snippets of other language families), and the proposer version
bumps to 5 so rows examined under the old floor are read again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Measured live: true instances of the service-unit canon score 0.68-0.71
against it at best, helpers 0.66-0.75 against unrelated snippets, and
nothing reaches the 0.8 floor. The "conclusive miss" fired for nearly every
body and silenced real divergences exactly as it silenced helpers. The
stored miss basis, its flag withdrawal and the report plumbing are removed;
the floor stays at 0.8 and the new-shapes-first ordering stays. False
prompts are answered by judgment (exempt with a reason).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Measured on the first live refresh after 91cde6c deployed: the version bump
queued 1,533 rows for the semantic arm, the cap read 150 of them in row
order, and all six shapes new since the previous refresh - the only rows
flag_divergence acts on, and the highest ids - were flagged before the arm
reached them. The gate silenced nothing because it never got to look, and
since a flag persists until judged, a later conclusive miss could not take
it back.
- propose_for_repo sorts the semantic todo by _semantic_priority:
unclassified before scoped, newest first.
- flag_divergence withdraws a standing flag when the row now carries a
conclusive miss; that evidence alone withdraws one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A design system binds like a rule but reached a session only through the
session-start block: complete for a session that asks, silent for one
writing a component. The write-path hint now carries a design arm.
A trigger, not a search: a project has one design system, so the question
is answered by the file being UI (.vue, .css, .tsx, ...) in a project that
has one. No vectors, no score, no slot from the ranked menu.
An index, not the prose: resolved guidance runs to ~8,000 chars (the house
style alone), near the hook's whole additionalContext cap. The line names
each inherited layer's section headings and inlines a layer short enough to
be a line - in practice the leaf's departure.
The other arms do not move. A design-only write returns on its own rather
than joining the prior-art guard, so the standing-rule arm still runs only
where it ran before. Once per session per system, on the hook's existing
token-keyed channel (exclude_derive, design:<id>) - no plugin change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
find_duplicate_rule was title-only, on the stated premise that rules are not
a semantic-retrieval surface - false since rules were embedded. A preference
restating a rule under another title passed untouched, and since both kinds
share one ranking, the weaker label could arrive alone.
find_overlapping_rules queries semantic_search_rules with the rule_document
shape, both kinds, in the scope the new record ranks in (global: every rule
the caller owns; project: global + that project). All three MCP create doors
call it before creating and return overlaps + overlap_note on the reply.
It advises rather than blocks, on measurement: across 16 sampled records the
nearest DISTINCT neighbour reached 0.853, while a true rewording scored 0.850.
No threshold separates the bands, so the floor (0.80) sits below the
restatement and the author judges. The stale docstring is corrected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7223 went red on seven tests in test_lesson_usage_surface.py. None of them
caught a regression. Every property they pin still holds; they pinned it at the
old LOCATION: they patched `usage_for_notes` on the door modules and read the
route source for `usage_for_notes(` and `empty_usage()`. 62f3a48 moved all
seven doors onto `note_usage.attach_usage`, so the name is gone from the doors.
This is rule 167's case: a guard that fails on code you believe is correct has
usually moved out from under its property. So the guards are relocated, not
deleted. Each one still has an assertion that can fail:
- MCP list / once-per-page / read-before-pull: patch `usage_for_notes` on
`note_usage`, the module every door now reads it through. The ordering test
still fails if a pull is recorded before the count is read.
- REST list: calls `attach_usage(items)` exactly once, and does NOT hand-roll
`it["usage"] =`, which is how a row could lose its zero-filled key again.
The seam's own zero-fill and single aggregate are asserted in
test_usage_attach_seam.py.
- REST detail: `attach_usage(` still precedes `record_pulled(`.
- Detail view: the advice moved into utils/deadWeight.ts, so assert BOTH that
the view reads DEAD_WEIGHT_ADVICE.lesson AND that the table's lesson entry
still points at `when_to_apply`, so the guard cannot pass by pointing at an
entry that has stopped saying the right thing.
Missed before pushing because I grepped src/ for callers of the old name and
not tests/ for patchers of it. A test that patches a name off a module is a
caller too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`usage_for_notes` is named for notes and works on every note row, yet the chip
reached snippets and rules only. Notes had it nowhere. Lessons had it collected
and shown nowhere a person could reach, because #4196 taught `/api/lessons` to
attach it and `KnowledgeView` — the only lesson list in the UI — browses through
`/api/knowledge`, so `listLessons` still has no consumer.
The cause was not a missing line. SEVEN call sites carried their own copy of the
same few lines: two REST lists, two REST details, two MCP lists, one MCP detail.
Each read perfectly well alone, so "which doors attach usage?" had no answer
anywhere in the code — the same asymmetry test_system_tagging_door_parity.py
records for System tagging (#4249), where whichever door nobody exercised for a
kind is the one that never grew the feature.
`attach_usage(rows, key="id")` is now that answer, and all seven go through it.
A detail payload is a one-row list, so the single-record doors share the seam
rather than keeping a second shape beside it. Deliberately NO try/except: the
fail-open already lives in `usage_for_notes`, which reports through
`_report_failure("readout")` and returns the zero-filled map. Wrapping it again
would swallow the REPORT as well as the error, and a silently-swallowed readout
failure is exactly #2663 — every counter reading zero in production for weeks
while the writes landed fine.
`/api/knowledge` now attaches usage, which closes both holes at once: it is how
notes, lessons and processes are all browsed. `KnowledgeView` renders the badge
on the card footer, looking the advice up per row because the feed is mixed.
The advice moves to utils/deadWeight.ts. Canon #3460 says each caller owns its
own const, and that held while each caller showed ONE kind; a mixed feed would
need five of its own and the next surface another five. The canon's actual
invariant — advice is kind-specific and never baked into the badge — is kept:
it is still a prop. The three existing callers now read the same table, so the
sentence has one home rather than four. Recorded against #3460 so the next
reader is not left re-litigating it.
`_row_id` rejects bools explicitly: `int(True)` is 1, so a row carrying a flag
under the key would be credited with note #1's counts, and a wrong chip is worse
than no chip because it reads as a measurement. A row with no usable id is
skipped rather than failing the page.
Tests pin the PROPERTY, not one route: no door calls the aggregate directly
(AST, so a comment naming it is not a false positive), and every door that shows
usage reaches the seam. Plus the N+1 guard — one aggregate per page, asserted on
await_count, because the per-row version reads more naturally and is invisible
in review.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4204 gave the divergence check a structural gate, which silenced one of the
five false prompts it was filed for. The other four are `def` helpers in a
directory whose canon is an `async def` service unit. No refinement of
`shape_form` reaches them: they differ from #2793's acceptance case — a
hand-rolled sync `confirmDanger` where an async confirm helper is canon —
only by the JOB they do, and a signature does not carry a job.
The proposer's semantic arm already reads bodies per symbol, which is the
comparison option 2 asked for and was thought to be missing. What it did not
do was record its MISSES: a hit became `proposal_basis = "semantic"`, a miss
left the row indistinguishable from one nobody had looked at. So
`flag_divergence` could ask the proposer "do you agree this is the canon?"
but never "did you check, and is it not?".
`_semantic_canon` now reports whether an empty answer is evidence, and
`flag_divergence` withholds the prompt when it is.
The whole risk is in the negative, so only a conclusive miss is stored. A
body too thin to embed, a row the per-refresh cap never reached, an arm that
threw, and a result set that came back full — and may therefore have hidden
the canon behind the limit — all stay "cannot tell" and still ask the
question. That is the discipline `FORM_UNKNOWN` already enforces here: not
knowing must make a check quieter, never more confident. `_SEMANTIC_LIMIT` is
named for that reason; the number is load-bearing, not a tuning knob.
No migration: `proposal_basis` is nullable Text with no CHECK constraint
(verified in the model and across alembic/versions), so rule 36 does not
bite. Nothing can mistake the miss for a proposal either — every reader keys
on `proposed_snippet_id` or `proposal_group`, and `confirm_shape_proposals`
requires the id non-NULL before it will confirm anything.
`_PROPOSER_VERSION` 3 -> 4, per its own contract: rows remember the ruleset
they were examined under, and without the bump no already-examined row would
ever acquire a miss.
Option 1 (widening `kind`) stays closed, on the merits rather than on cost:
bucketing density by exact form takes the async canon out of a sync
candidate's denominator and silences #2793's acceptance case by the identical
mechanism, one layer down. The reasoning is on #4208.
Tests: tests/test_divergence_meaning_gate.py pins the report contract, with
the truncation case tested hardest — reading a cut-off as a negative would
weaken the guard in proportion to how many snippets the operator has. The
end-to-end discrimination is in test_integration_shape_classify.py on
deliberately the SAME fixture as #2793's acceptance case, so the two runs
differ in exactly one thing: whether the arm claims to have looked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Derive-first, ahead of the tests shape pay-down. An AST pass over all 2628
definitions under `tests/` found exactly three helper bodies duplicated
across files. Two are real copies and are consolidated here; the third is
not, and is left alone.
`need_tools(*tools)` — byte-identical in three hook-test modules, each
skipping when `jq`/`awk`/`git` is absent from PATH. Now snippet #4277. The
`import shutil` each file carried existed only to serve it and goes with it.
`rule_row(rule_id)` — byte-identical in two integration modules, reading a
Rule back through a SEPARATE session so the assertion is about what Postgres
holds rather than what the writing session's identity map remembers. Now
snippet #4278. Its imports are lazy, because `tests/helpers.py` is imported
by unit tests that have no database, which is the same reason
`plugin_config` defers its service imports.
NOT consolidated: `_side(uid, k, d="")` in test_services_plugin_context and
test_write_path_trigger. The body is identical but it closes over a
module-local `stored` dict, so it is not self-contained and "moving" it
would mean inventing a parameter neither call site wants. That is convention
plumbing — two tests independently writing the same one-line side_effect —
and it is dismissed in the ledger rather than lifted.
Worth recording for the next pass: a repeated NAME is not a family. `_row`
is defined in five modules and only two of those share a body; the other
three (test_list_rows_brief, test_calibration_stamp, test_shape_ledger)
build entirely different objects. Grouping by name would have consolidated
three things that have nothing in common.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Run 7208 failed 5 of 23 in the new file, all with a `**Learned from:** #1`
that no shape asked for.
`lesson_sources` falls back to `arose_from_id` when neither the mirror nor
the body names a source. `fake_note` leaves that field to MagicMock, which
auto-creates it truthy and coerces to 1 — so every no-sources shape
recomposed with a provenance line and the round trip failed for a reason
with nothing to do with the trigger. `fake_lesson` pins `arose_from_id` to
None and its docstring says why; it is also the truthful stand-in, since
these are lessons.
Worth recording separately: the local harness that cleared this change used
a plain stub class, so it was kinder than the real fixture and could not
reproduce the truthiness. The stub has been given the same None. A
stand-in that is more forgiving than the one the suite uses will verify a
change that CI then rejects — note #2109's point, arriving from the other
direction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The module docstring said the defect grew the worst real body by 520
characters over ten edits. That number was recalled, not measured, and it is
wrong — as was the 2500 I then got by deriving it on paper instead of
running it.
Replayed lesson #4207's actual stored shape through the old code: 607
characters per edit, exactly the length of the trigger's paragraphs 2 and 3
with their separators, taking the body from 1291 to 7361 over ten edits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`update_lesson` is field-wise: an edit that touches only `system_ids`, or
only the title, passes `insight=None`. The service then derives the insight
from the stored body and hands it straight back to `compose_body`, which
re-adds the trigger and the provenance line. The derivation and the
composition have to be inverses, and they were not.
`_strip_composed_lines` filtered out body LINES matching the trigger
marker's pattern. That pattern is `re.M`, so it matches one line.
`when_to_apply` is free text and is routinely several paragraphs —
`compose_body` writes all of it after the marker, so paragraphs 2..n were
ordinary lines the filter had no reason to touch. They came back as part of
"the insight", the whole trigger was prepended again on top of them, and the
record gained (n-1) paragraphs per update. Silent, compounding, and in the
half of the document that does the retrieving: a lesson edited a few times
ends up stating when it applies several times over and burying the insight
below it.
No marker could fix this. There is no end-of-trigger marker in the body, and
adding one would not repair a row already written without it. What makes an
exact strip possible is that the trigger is mirrored in `data[TRIGGER_KEY]`,
so the composed head can be rebuilt byte for byte and removed by subtraction
rather than matched by pattern. Hence `lesson_insight(note)` rather than
`_strip_composed_lines(body)` — it takes the record because the body alone
does not say where the trigger ends. The line-wise removal survives as the
fallback for a row whose mirror is missing, where it is right for a
one-paragraph trigger and is the best available without one; a mirror that
disagrees with the body falls back rather than cutting into the insight.
tests/test_lesson_insight_round_trip.py asserts the property rather than the
strip: `lesson_document(what, trigger(n), insight(n), sources(n)) ==
(n.title, n.body)` across six shapes, and again over ten consecutive no-op
edits. Falsified against the old code before committing: 11 of 23 fail on
HEAD, the headline one reporting a body grown by 3230 characters over ten
edits.
The one damaged row in the corpus (#4207, two paragraphs duplicated by a
`system_ids` edit) was repaired by hand against its authored text. #4221 is
the only other lesson with a multi-paragraph trigger and has never been
updated, so its body is intact — its derived `insight` was wrong, its
storage was not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`set_record_systems` is not a dumb setter. It runs its own `can_write_note`
and then links only the Systems the given user can READ. Four of twenty
call sites handed it the record's owner instead of the acting user, which
did two quiet things at once: the access check became trivially true, since
an owner can always write their own record, and the System filter used the
owner's visibility rather than the actor's.
On a single-user install neither is observable. With a share it is an
editor acting with the owner's reach — the shape rule 47 exists to prevent,
and the same reasoning routes/notes.py already spells out for
`set_supersedes` two lines away.
This is NOT a permission change. Every one of the four sites establishes
the caller's write access first: routes/lessons.py and routes/snippets.py
call `can_write_note(uid, …)`, mcp/tools/processes.py does the same, and
mcp/tools/snippets.py reaches `set_record_systems` only after
`update_snippet` has raised PermissionError if the caller may not write. So
nobody gains or loses the ability to edit anything. What changes is whose
reach the tagging runs with, which is exactly the kind of difference that
survives review because every call site reads fine on its own.
The four:
routes/lessons.py:243 owner_uid -> uid
routes/snippets.py:211 owner_uid -> uid
mcp/tools/snippets.py:486 note.user_id -> uid
mcp/tools/processes.py:196 note.user_id -> uid
The last one was written earlier in this same session, an hour before the
sweep that found it, with a comment confidently explaining why the owner
was correct. That is the argument for the guard rather than for care: the
unified stance was known and still got it wrong at the next opportunity.
So the guard is the point again. `test_every_tagging_write_acts_as_the_caller`
walks every `set_record_systems` call in src/ and asserts the first argument
is a bare local named `uid` or `user_id` — an attribute access is a record's
owner by construction. Verified against `git show HEAD:` as well as the
working tree: clean now, four offenders on the code it replaces.
Reads are deliberately untouched. `list_record_systems(owner_uid, …)` gates
on reading the NOTE, which the caller can do anyway, so it returns the same
list either way; it is a different operation and churning it would add noise
without changing behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A System tag is how `list_system_records` gathers an area's pile, so a
record that cannot be tagged is reachable by search and by nothing else.
`update_lesson` did not take `system_ids` while `create_lesson` did, which
made tagging available exactly once — at the moment of least information.
A lesson is usually written at the end of a piece of work, which is
precisely when that argument gets dropped, and after that the record could
never be filed at all.
Filling in the rest of the table found three more gaps, and the issue's own
generalisation was wrong. It read as "the REST door can do something the
MCP door cannot", on three instances in a row. In fact:
* `create_preference` is the exact MIRROR of the lesson bug — update
takes `system_ids`, create does not. The same capability missing from
the opposite end of the same lifecycle.
* `routes/notes.py` handled `system_ids` NOWHERE, while the MCP door
handled both ends. That runs the opposite way round from the premise.
* `create_process` / `update_process` took it at neither door, though a
process is a note and has always been taggable in the data model.
The real pattern is that whichever door nobody exercised for a kind is the
one that never grew the parameter — which is a better statement of #4248
than the one recorded there, and is not something a reviewer reliably
notices, because each door is only ever read on its own.
Milestones are NOT a fifth gap. `RecordSystem.note_id` is a ForeignKey to
`notes.id` and milestones are their own table, so they cannot be tagged at
any door by construction. Pinned in the test so the next pass does not
re-open it.
So the guard is the point, not the four parameters. `update_lesson` alone
would have left the shape that produced it intact. The new test asserts the
TABLE — every kind taggable anywhere is taggable everywhere it is written —
and keys the registry-coverage check on the SIGNATURE rather than on a
grep, so a module that only names the argument in prose is not swept in and
no hand-kept skip list can go stale. A fifth kind fails there rather than
shipping half-wired, the same reasoning test_derived_mirror_generic_door.py
records for derived mirrors (#3734).
One inconsistency found and deliberately not changed here: for the same
operation `routes/tasks.py` scopes `set_record_systems` by the caller while
`routes/lessons.py` and `routes/snippets.py` scope it by the owner. The new
notes code follows tasks.py and says why in a comment (#47 — an editor-share
holder should tag from what they can see rather than inherit the owner's
reach). Recorded in #4249 rather than fixed as a drive-by.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`band_hugs_floor` compares an arm's weakest tenth against its floor, and
#4225 made it SUSPEND rather than soften when the floor moved inside the
window — because across a change the scores and the bar come from two
different populations. It reads `floor_moves_since`, which answers "did
this arm's floor move", and it read an empty answer as "no, it held
steady".
Those are the same answer only where the ledger was watching. Before an
arm's first floor row there is nothing to move, nothing to report, and no
way to tell a steady floor from an unrecorded one. The suspension was
reading absence of evidence as evidence of absence, and the symptom is the
one #4225 documented: a band "-0.0208 above its floor" — an impossible
negative distance, printed with the suspension silent.
Every install passes through this. The ledger's first row for an arm is
written when that install first boots the release that records baselines,
so any window longer than the install is old reaches back past it. The
lowered-floor direction hides: the gap comes out comfortably positive and
reads as a clean bill of health.
`floor_history_gaps(since)` answers the question its companion cannot:
which arms' floor history does not REACH the start of the window. Arms
come from `surface_names()`, not from the ledger — an arm the ledger has
never heard of is exactly the one at risk, so it cannot be the ledger that
decides which arms get asked about.
Baselines count here, which is the one place the two deliberately
disagree. A baseline records a default without changing it, so it is not a
move and `floor_moves_since` filters it out. It IS the ledger beginning to
observe the arm, and from that moment silence genuinely means the floor
held — filtering it out here would suspend the band check forever on every
install that has never tuned.
`floor_history_unknown` sits between the known move and the band, so an
arm with a date to give gives it. Two sentences, because the remedies
differ: a date says ask again with a smaller `days`; no history at all
says there is nothing to wait for, and names what starts the record.
Why now: #4261 measures the work-log change by reading these warnings, and
every window for the next month opens before this ledger's first row.
Measuring against an instrument that prints a number it cannot support is
the #4225 trap one level up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Every backfill selected the text alongside the id and then iterated that
snapshot, sleeping between records. Over a real corpus that is minutes, and
anything edited inside the window was re-embedded by its own write path and
then OVERWRITTEN with the pre-edit text the scan had captured — stamped at the
current chunker version, so the next boot considered it current and never
repaired it. The record stayed findable by what it used to say, indefinitely,
until someone happened to edit it again. Nothing reported it, and no existing
signal separated it from a correct record: the version was right and the
vectors were there.
All four backfills had it. Three were pre-existing; I wrote the fourth an hour
ago for #4251 by copying the third.
#4251 also widened the exposure. The race used to need a note-body edit; now
every add_task_log re-embeds its task, so any session recording work during a
backfill can hit it — which is exactly what "forward work continues while the
backfill runs" means.
TWO HALVES, SEPARATE ON PURPOSE.
Stopping it: the scan takes IDS ONLY and `_current_row` re-reads each record at
the moment it is embedded. One indexed primary-key read, negligible beside the
forward pass that follows, and it turns a backfill from "replay a snapshot"
into "repair to current truth". The residual race shrinks from the length of
the run to the width of one record, which the parent-row claim and the
single-transaction replacement narrow further.
Repairing it: `_vectors_older_than_their_record` adds a third staleness
condition beside "no vectors" and "old chunker" — vectors older than the text
they encode. A guard that refuses to write a bad value does not undo the bad
values already stored (#4202); the rows are what has to change.
That check earns its place beyond the bug that prompted it. It equally catches
a record whose `embed_note` never ran — no event loop, a swallowed refresh, a
process that died between the commit and the index. Every one of those looks
identical from the version column and identical from outside.
And a work log is part of a task's document while living in its own table, so
writing one never moves `notes.updated_at` and the timestamp comparison cannot
see it. `_tasks_logged_since_embedding` covers exactly the records the race was
most likely to have hit.
The live instance has already run #4251's backfill — the prior-art arm is
surfacing `## Work log —` passages, which is the thing itself rather than a
banner about it. Whether that run clobbered anything is unknown and was not
measured; the repair half picks up whatever it did on the next boot, without
needing it diagnosed first.
The structural guard in the tests was checked against the old shape, not just
the new one: reintroducing `select(Note.id, Note.title, Note.body)` makes it
fail, so it can catch the next copy of this rather than only agreeing with
today's code (#167).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 2 of #4251. A System's `description` is a charter — several hundred words
saying what belongs in that area and what does not — and it is the answer to
"which part of this codebase does X live in". There was no semantic path to
one: `list_systems` enumerates, and `search(system_id=…)` uses a System as a
FILTER over notes. So a System could narrow a search and could never be the
answer to one, and an agent asking where a record belonged had to read every
charter or guess.
ITS OWN SEARCH, not a `content_type` over notes, for the reason note 3163
gives about milestones: the row could be shared, the search cannot. A charter
competing with the whole note corpus for one top-k is outranked by the records
filed under it — the right answer crowded out by its own contents — and "where
does this belong?" is a different question from "what prior art is there?",
which a caller asking one should not have to read past answers to.
So `system_embeddings` (0107) joins note_, rule_ and milestone_embeddings as
the fourth sibling, with `system_document`, `upsert_system_embedding`,
`semantic_search_systems`, a startup backfill and `search(content_type=
"system")`. Scoped like milestones: with a project_id, that project's Systems
if the caller can read the project (rule 78); without one, the caller's own.
Archived Systems are excluded — an archived area is one the operator has said
is no longer where things go, which is exactly the question being asked.
`system_document` is the plainest of the four shapes on purpose. A charter is
already written as the thing this search has to match, in the words someone
asking would use — so there is no trigger to synthesise as `rule_document`
must, and no second record to gather as `task_document` must. The stored
charter IS the sharp document, the way a snippet's is. `color`, `status` and
`order_index` stay out: presentation and bookkeeping, and a vector carrying
them would be answering a question nobody asks of a charter.
The search publishes `report["best_chunk"]` from the start rather than being
retrofitted, which is what #4251 asked of any fourth search. It matters more
here than anywhere: a charter runs long and a result shows its NAME, so a match
on the paragraph that actually decides where a record belongs would otherwise
be previewed by two words that cannot say.
The id that comes back is the one `system_id`, `system_ids` and
`list_system_records` already take, so the answer to "where does this belong?"
is directly usable as "show me what is there" and as "file it here".
`embed_system` sits beside `notes.embed_note` at the service for #2056's
reason — every door gets it by construction. Not called on delete: that is a
soft delete and the search joins through `System`, so the vectors are already
unreachable, and leaving them means a restore is findable again immediately.
`system_embeddings` is declared in backup's `_NOT_INCLUDED` as derived, beside
its three siblings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 1 of #4251. A work log is the richest prose Scribe holds about WHY
something is the way it is — written during the work, recording what was tried
and ruled out. #4241 made it readable from the agent's door. It was still not
findable, so "has anyone tried this approach?" — precisely the question a log
answers — could not reach one. The cost is not hypothetical: #4208 was rebuilt
in this session because its logs were unreachable.
THE DESIGN QUESTION the issue left open was whether logs embed as part of their
task's document or as rows of their own. As their own rows, a hit has to be
resolved back to a task to be worth anything, and it needs a fourth search, a
fourth result shape and a fourth arm. As part of the task, the objection is
that a long log drowns a short title.
That objection was true before #280 and is not true now. Chunking made one
record into one vector per section, so each log becomes its own title-anchored
chunk, scored separately, and the task's own prose keeps the chunk it always
had — a task is as findable as its best-matching log rather than as the average
of everything in it. The other half is this session's other build: a search
hands back the chunk that won (#4243), so a hit earned by a log shows that
log's passage under the task's title. Without that a reader would have got
body[:240] of the task — the opening of a record whose relevance lives three
hundred lines further down.
So `task_document(title, body, logs)` sits beside `rule_document`: a
synthesised embed-time shape, because the stored record is the task row and the
logs live in their own table, so the document that should be searchable exists
nowhere until it is built. A task with no logs is returned untouched — most
notes are not tasks and most tasks carry no log, and their vectors are the
corpus every tuned number here was measured against.
CHUNKER_VERSION 1 → 2, and its comment now says what the version actually
means. It used to read "whenever chunk_document's output can change for the
same input", which this change would slip past: `chunk_document` is untouched
and every task with a log now embeds differently while its title, body and the
chunker all stand still. The invariant is the document a record is embedded as.
The startup backfill re-embeds on that.
Create, edit and delete of a log all refresh the task through `embed_note`, the
one path every writer shares — an edited log whose vectors still carry its old
wording keeps matching what it no longer says.
No new kind enters the auto-inject menu: tasks were always in it, and this
makes recall on them better rather than changing what the menu spans. The
calibration stamp will now report shape_version 2 against numbers measured at
1, which is exactly the report #4104 built it to make.
Also promoted `session_returning` into tests/helpers beside `make_mock_session`
(#2834) — two files had spelled it out identically and a third was about to.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The engine took `note_type` and `task_kind` all along. What was missing was a
way to say them: the MCP tool's `content_type` knew `note`, `task` and `all`,
and `/api/search` knew the same two — so an agent could not ask "has this
snippet already been recorded" or "what lessons apply here" without searching
everything and reading past the rest. Browse offered nine kinds from the same
data.
The cause is that each door kept its own map. `_FACETS` in services/knowledge
is where a kind is declared, and #3161 made adding one a single edit by
generating the SQL filter, the Python predicate and the door's validation from
it — but the two search doors were written before that and never joined. So
this adds the third dialect, `search_filters_for`, and one composition over it,
`content_type_filters`, and both doors now derive instead of listing.
Two names keep a meaning of their own, and the docstrings say so: `all` is no
filter, and `note` is BROAD — any non-task, snippets and lessons included —
where the browse facet of the same name is narrow (`note_type == 'note'`).
They are left different deliberately; narrowing this one would stop returning
snippets to every caller that already asks this way.
An unrecognised kind is now refused rather than answered. Both doors used to
fall through: the MCP tool into a filter matching no row, the route into no
filter at all, so `?content_type=snippets` returned the whole corpus while
looking like a narrowed search. An empty result set is a claim — "the corpus
holds nothing like this" — and an agent acts on that claim by building the
thing it could not find, so a typo must not be able to make it.
The docstring is the agent-facing contract (#2846), and a test now holds it to
the table: every kind `_FACETS` declares has to appear in it, because a filter
an agent has not been told about is unreachable however well it is wired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
test_milestone_search_is_its_own_shape asserts the result dict exactly, so
adding `matched` / `matched_is` / `body_length` broke it — which is the test
doing its job.
Its fake was a bare MagicMock, so `m.body` autovivified: `matched` came out as
a mock object and `body_length` as 0, and nothing in the old assertion touched
either. Lesson #2833 is about exactly this, and the failure output is what
surfaced it. The fake now carries a real body string and a report with the
chunk that matched, so the assertion is about the product rather than about
MagicMock's attribute behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4243 fixed one door. Scribe has three semantic searches over three chunk
tables, and all three collapsed chunk rows to the best one per record — each
of them KNEW which passage earned the hit, and each dropped it. Every surface
downstream then previewed the head of the document instead: a span the search
had already scored lower, with nothing saying so.
Mechanism, one place:
- embeddings.record_best_chunk publishes {id: {index, text}} into `report`.
Carried in `report`, NOT the return value: all three return
list[tuple[float, Record]] and ~30 sites unpack that pair (lesson #4207).
- semantic_search_rules and semantic_search_milestones now select
chunk_index/chunk_text and publish the winner, as notes already did.
semantic_search_milestones gains `report`, which it had no way to take.
- services/text.matched_excerpt is the one choice of span, and
excerpt_fields the one result block. Doors keep their own field names —
the web renders `snippet`, MCP returns `excerpt` — because renaming a
field a frontend reads is a different change from fixing what goes in it.
Surfaces:
- knowledge.query_knowledge, whose own comment calls it "the human's MAIN
search surface", was `(note.body or "")[:200]` on every row alike. Now the
matched passage on a search, the opening on a browse, and `snippet_is`
saying which. KnowledgeView renders that snippet, so this was live.
- search(content_type='milestone') gains `matched` — the plan body stays
out, but the passage that matched comes along, because recognising a plan
means recognising the part you asked about and a description written at
the start need not mention it.
- The auto-inject menu and the write-path prior-art menu put the passage
under their line. Both were title-only, which answers "does this apply?"
for a lesson or snippet (the trigger is IN the title) and not at all for
an issue or dev-log. No fallback to the body's opening: on a menu that is
preamble dressed as a reason, and once indented it cannot be told apart.
Left alone deliberately: the rule arms. A rule hint already renders the rule's
TRIGGER, which is written to answer exactly "does this apply to me" and beats
a matched chunk at it; and that line's budget was measured at #3851. Adding a
passage there would duplicate the trigger and spend the budget twice.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
test_chunking::test_search_collapses_chunk_rows_to_best_chunk_per_note feeds
rows straight into the real semantic_search_notes, so widening the select to
carry chunk_index/chunk_text broke its 2-tuple fakes. I had claimed no unit
test did this after grepping test_embeddings.py, which was the wrong file.
Rows are 4-tuples now, and the test additionally asserts the winning chunk is
REPORTED and not merely used for scoring — the property #4243 exists for, and
this is where it belongs, beside the collapse it comes from.
test_task_work_log_surface's ACL test asserted "task_logs.user_id" was absent
from the compiled statement. It is present in every SELECT as a projected
column; the claim was about the WHERE clause. Asserts on stmt.whereclause now,
which is what was actually meant and still fails if an owner filter returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Raised by the operator: are we limiting what comes back by character count,
and how do we verify the pertinent part is the part displayed?
We were not. mcp/tools/search.py sent (note.body or "")[:240] — a head cut,
with no marker that anything had been removed, so a 240-character preview of
a 4000-character record was indistinguishable from a complete short one.
The opening is the wrong span. The match is semantic and per chunk, and
semantic_search_notes collapses to best-chunk-per-note — its own comment at
the collapse says "the first appearance of a note is its best chunk". So the
system identified the passage that earned the hit and then discarded it:
select(Note, distance) kept no chunk column. A record could rank first on its
sixth paragraph, be previewed by its first, and be judged irrelevant on a
span the search had already scored lower. That biases against long records,
and it is self-concealing — the caller who does not open it never learns the
preview was misleading.
- embeddings: chunk_index/chunk_text ride along in the select, and the
collapse records the winner in report["best_chunk"]. Carried in `report`,
NOT by widening the return tuple: ten callers unpack (score, note) at
~18 sites and nothing would catch the misses (lesson #4207). `report` is
the side-channel this function already uses for best_available_score.
- search(): excerpt / excerpt_is / body_length, and read_full when there is
more. A caller that cannot tell a matched passage from a document opening
cannot judge whether to look deeper, which is the only decision the field
supports.
elide() moves to services/text.py so both callers share one copy, and it
keeps BOTH ends with a stated gap — it is the fallback for when nothing
identifies a better span than "all of it", not the goal.
Also fixes a guard that produced a false failure on the previous commit:
test_pull_telemetry checked `"project_id: int = 0" in body.split("\n")[0]`,
which sees only the first line, so wrapping get_task's signature over four
lines made it report a function that does take the project as one that does
not. Parsed with ast now, and proven to still reject an absent or
wrongly-typed parameter rather than being appeased by reflowing the code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The work log reached the web UI through routes/task_logs.py and nothing
else. get_task returned only the body — a claim written once, before the
work — with the record written during it invisible beside it. So a stale
body arrived with nothing to contradict it, and this session rebuilt work
that had already shipped, with the evidence sitting in the task's own logs.
Read side, scoped through the access layer (rule 78):
- logs_for_task / count_logs_for_task / log_counts_for_tasks in
services/task_logs.py. Scoped by who may read the TASK rather than by
who wrote the entry: list_logs filters TaskLog.user_id == user_id,
which hands a shared collaborator an empty list reading as "no work
has been done". The page query folds readable_notes_clause into the
same statement so the permission does not become an N+1.
- get_task returns work_log; list_tasks and get_milestone steps carry
log_count, zero-filled so "none" is a count and not a missing key.
Elision keeps both ends. The newest entry arrives whole to 4000 chars
because it answers "where does this stand"; older ones are shortened from
the MIDDLE, never the head. A head cut selects what a reader sees by
character position, which is uncorrelated with what matters — an entry
closing with "so this shipped in 04775c3" loses the one sentence that
answers the question, and a truncated flag says something went, never
whether it mattered. The gap states how many characters it covers.
conftest gains an autouse stub for the new read arm, same reasoning as
_no_rule_arm: three widely-called tools grew a database read, and the
existing call sites should not each have to learn about it.
Raised while reviewing this: search() has the same shape and worse —
body[:240] with no marker at all, while the chunk that actually matched
sits unused in the row that won. Filed as #4243, not fixed here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Found by reading a live `retrieval_telemetry` readout after milestone 419
deployed, not by inspection. The readout said:
cannot_decline / report_preference — "45 calls, 0 of them returned
nothing. An arm that fires unasked has to be able to say nothing; this one
never has. Check that it applies its floor at all."
And printed, beside it, that arm's band: p10 = p50 = p90 = min = max = 0.791.
FIVE IDENTICAL PERCENTILES IS THE TELL. That is not a ranking, it is one
record at one score on every call — because `report_preference` searches a
fixed string (`reply_preferences.COMPLETION_QUERY`, a module constant, and
deliberately so).
For a fixed query against a stable corpus the top score is a CONSTANT, so the
arm's decline rate is 0% or 100% and never in between; which of the two it is
depends only on where the bar sits relative to that one number. "Never
returned nothing" is therefore arithmetic, not evidence, and the warning's own
remedy — check whether it applies a floor — cannot be answered from it.
The arm already knew this about itself; the warning did not:
"a fixed query makes this arm's score a constant and a floor a hair above
it produces a dead arm no amount of traffic will ever reveal"
— services/reply_preferences.py
THIS CLASS OF BUG ALREADY HAS A GUARD, which is the argument for the shape of
the fix. `Point.logs_unconditionally` exists because of #3497: both rule arms
once logged only their hits, so their zero count was structurally 0 and this
same warning would have fired on a LOGGING property while sending the reader
to move a threshold that was never involved. This is that one step over — a
QUERY-SHAPE property — and gets the same treatment: a declared field on
`Point`, and exclusion rather than trust.
AND THE WARNING THAT WOULD BE INFORMATIVE HERE DID NOT EXIST. For a fixed-query
arm the dangerous state is the mirror image: every call empty, meaning the bar
is above the constant and no further traffic will ever move it. The arm is off
rather than quiet, and nothing in the readout said so — `expects_traffic`
covers an arm with NO calls, not one with calls and a 100% decline rate. That
state is real and reached: `report_preference` once logged 69 consecutive
declines at 0.0006 under its bar.
So `fixed_query_never_clears` sends the reader to `near_miss_samples` and not
to the dial — because that incident is also the one where the statistic and
the correct action pointed opposite ways. Every percentile said lower the
floor; opening the refused record showed it was rule 77 arriving as a false
positive, and lowering it would have delivered that rule on every completion
report ever written.
Guards in tests/test_retrieval_warnings.py, including the falsifier that
matters most here: `cannot_decline` must still fire on an arm whose query
varies, or this change is a disabled check wearing a narrowed one's clothes.
Both boundaries tested from both sides, per that module's own standard.
The new code is documented in the `retrieval_telemetry` tool docstring beside
the others (rule 33) — an undocumented code in a readout is a reader meeting a
verdict with no way to disagree with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4222 fixed one end of this defect — the extractors that decide what a payload
DEFINES now blank comment and string spans before any line matcher runs. This
is the other end: `scribe_local_dups`, which decides which OTHER files already
define that name, and which was still a plain `git grep`.
A grep sees lines, not spans, so the sentence
class with only modifier rules is a deletion that went half-way.
— real prose from a module docstring in this repo — matched the arm's pattern
for `name=with`. Writing a genuine `change`, `beside` or `wrapped` would be
told it already existed, and pointed at a docstring.
EVERY HIT IS NOW CONFIRMED by running `scribe_defs` over the candidate file and
keeping only names it actually reports. That is the only check that cannot
disagree with the other end of the pipe, which is the whole point.
MEASURED, NOT ASSUMED — both numbers the task reasoned from turned out wrong.
- WHAT IT REMOVES, across 141 payload files of this repo: 15 of 210 report
lines. Every one a string literal, a comment, a TypeScript `import { type
Foo }`, or Vue's `const emit = defineEmits()` boilerplate. No real
definition was lost. Where a name had both — `create_note` — the phantom
in a test's `shape_form("async def create_note(...)")` argument dropped out
and the definition in services/notes.py stayed. `with` went from four files
to none, which is the correct answer: nothing here defines it, and it is a
keyword in several of these languages.
- WHAT IT COSTS: mean 193ms -> 222ms, worst 569ms -> 572ms. The task feared
"over a second added to a PreToolUse hook" from 48 confirmations. It is
about 15%, because the arm was already dominated by its twelve `git grep`
calls, and because confirmation runs once per DISTINCT candidate file
rather than once per (name, file) pair. A deliberately pathological payload
— nine names that are ordinary English words — reaches 28 distinct files
and 947KB; `scribe_defs` runs at ~33ms per 250KB.
THE CANDIDATE CAP IS RAISED FROM FOUR TO TWELVE, and that is load-bearing.
Confirmation REMOVES hits, so capping before it runs lets phantom matches crowd
a real definition out of the window — hits dropped before anyone looked at
them, which is #4042's bug in a new place. The display cap stays at four and
now applies to CONFIRMED hits, which is where a cap belongs. #4042's own `||
true` inside the substitution is untouched, and its regression case still
passes.
The task's own advice not to fix this by tightening the grep pattern is
followed and written down: requiring `(` or `{` or `:` after the name rejects
`class with only…` and also `class Foo extends Bar {`, `class Foo : Base()` and
`type Foo struct {`. This arm exists because it works with no server, no index
and no binding (#2280, #2682), which makes a miss here invisible — a visible
false positive is the better failure.
Guards: tests/test_hook_duplicate_confirmation.py, against real git repos
because what is pinned is the interaction between `git grep`, `head` and the
extractor. Seven of the eight fail against the previous implementation; the
eighth is #4042's regression case, whose job is to keep passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The lesson slot has recorded surfaced-vs-opened since it shipped. Nothing
showed it. `get_lesson`'s REST door attached `usage` to the payload and no
view rendered it; the listing did not attach it at all, and neither MCP door
did.
#4196 asks when a lesson that keeps getting followed should become a rule, and
names the trap in the same breath: raw frequency cannot separate "this should
bind" from "this trigger is too broad", and the second is the commoner reading
by a wide margin. Surfaced-AND-opened can separate them. Neither question is
answerable by a reader who cannot see the numbers, which is why this is the
first step and not the threshold.
NO THRESHOLD IS PROPOSED HERE, deliberately. The corpus today is 10 lessons
with 7 recorded surfacings and 3 opens, over about fifteen hours of usage
data. A promotion rule fitted to that would be fitting noise — #3311's failure,
and the warning lesson #4228 was written to carry. `UsageBadge` already
declines to render a verdict under three surfacings for the same reason. So
#4196 stays open: its subject, the promotion path, is still unbuilt. What
lands is the evidence it needs.
- REST `GET /api/lessons` and MCP `list_lessons` attach `usage` to every
row, from one aggregate per page rather than a per-row read, which would
be N+1 by construction. Every row carries the key zero-filled, so "never
surfaced" is a state a reader can see rather than a missing field they
have to interpret.
- MCP `get_lesson` attaches it too, and reads it BEFORE recording its own
pull. That door records a pull on every open — it has to, or the kind sits
permanently at zero — which makes the order load-bearing in a way it is
not for a kind that only counts. The REST detail door already ordered it
this way; the two now agree about what the number means.
- `LessonDetailView` renders `UsageBadge` (snippet #3460) rather than
re-spelling the chip, with the advice keyed to this kind: a lesson that is
repeatedly offered and never opened is usually keyed to a situation nobody
is in, so it points at re-keying `when_to_apply`, not at deleting the
claim.
Guards, in the two styles this pair of doors already uses: the MCP side driven
behaviourally through mocks, including the call ORDER for `get_lesson`; the
REST side on structure like its siblings in test_lesson_rest_door.py, because
the route is decorated and returns a Quart response. Rule 167's falsifier is
included.
KNOWN GAP, not fixed here: `KnowledgeView` is the only lesson LIST in the UI
and it reads `/knowledge`, not `/lessons` — so the REST listing change reaches
`frontend/src/api/lessons.ts::listLessons`, which currently has no consumer.
The agent-facing listing does reach a reader today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The column guard is a claim about kwargs; "the value comes back" is a
different claim, and only a round trip settles it. This drives the real
`restore_full_backup` over a judged shape carrying all three of the columns
that were going missing.
`diverges_from` gets the harder assertion. It is a FK to notes.id, so the
tempting fix — carry the exported id across — produces a row pointing at
whatever note holds that number in the target database: not dropped,
REATTACHED, with the restore reporting success and the divergence about the
wrong snippet. So the test asserts WHOSE note the pointer landed on rather
than which integer it holds, and it refuses to run at all if the restore
happened to reuse the source id, which would let it pass without proving
anything. Same shape as the assertion in the rule_usage round trip, for the
same seam.
Also corrects this module's own docstring, which said the round-trip module
"is not written yet". Three existed; this is the fourth.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The column guard (#3182) makes a dropped column unexpressible on the way OUT.
Nothing watched the way back IN, and that is the worse half: an export gap
leaves an obviously thin backup, an import gap means holding a complete,
correct file and restoring an incomplete database from it, with a success
message.
WHAT IT FOUND, the first time it ran. `code_shapes` was exporting
`reason_code`, `recheck_at` and `diverges_from` and importing none of them.
A restored ledger would have carried every judgment's verdict and not the
code for WHY — the column the accounting reads to tell a scoped-css
exemption from convention-plumbing — with every recheck flag cleared and
every divergence pointer gone. `diverges_from` is a FK to notes.id, so it is
re-mapped rather than carried: the raw source id would point at whatever
snippet took that number in the destination, which is wrong rather than
missing and is the milestone 333 trap one table over.
WHY THE GUARD WAS ONE-SIDED. Not an oversight — the code was. Export goes
through per-table pure helpers, so a test can hand one a stand-in and read
which keys came out. `_restore_v2` built all 26 models inline in one 690-line
procedural function, and there was no per-table unit to hand anything to.
So the construction moved out, into a `_build_*` helper per table taking the
exported row plus the id maps built so far. What deliberately did NOT move is
the loops, the flushes and the id-map bookkeeping: that is the
order-dependent part, where a mistake is a restore that half-works, and it
gains nothing from being split. Returning None is "skip" and a None field is
"degrade" — which one a table wants stays the table's own call, because both
are right somewhere: a shape event without its project says nothing, while a
usage event without one is still a real pull and dropping it would deflate
the pull-through the table exists to report.
The refactor was checked to be behaviour-preserving before the guard went in:
all 26 constructor kwarg sets identical to HEAD, and all 28 skip conditions
accounted for — 25 now in builders, 3 in loops that construct nothing (the
rule_systems raw insert and the two in the final project patch).
The guard then composes the two halves end to end: export a stand-in row,
feed THAT dict to the builder, read which columns the model actually
received. `test_the_usage_importer_restores_the_reading_project` read the
source of `_restore_v2` with `inspect.getsource` because there was nothing to
call; it is replaced by tests that call the builders and assert the
skip/degrade behaviour directly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#2990 asked for three cases. The wrapped-comment-continuation one shipped
with #4222; these are the other two — a comment holding a whole rule defines
nothing, and an unterminated comment does not swallow the file. Both already
pass on both sides; the vectors are what stops a later change to the scan
quietly taking them back.
The phantom #2990 was filed for is among the twenty-two #4222 removes:
`.editor-body` in TaskEditorView.vue, a persisted `code_shapes` row whose
signature ends in `*/` and whose `used_by` count is zero — so it has been
sitting in the unused_css number the operator is meant to act on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The definition extractor is line-oriented and knows nothing about what a
line is INSIDE. A docstring that wraps onto a line starting with a keyword
announces a definition: `AND` reached a live session as a divergence prompt
asking it to justify a symbol that does not exist, and `is` reached it as a
repo-wide duplicate of four files that define nothing of the kind.
Measured, not assumed: running the extractor over every scannable file with
and without the scan differs by twenty-two phantoms. Two of them — `with`
and `nobody`, both out of the module docstring in check_dangling_styles.py —
are persisted `code_shapes` rows that have been judged. Those need no
migration: sync_shapes marks a row it no longer extracts as vanished.
`ast` would be the honest tool for .py and is not what this uses, because
the extractor is mirrored rule for rule by an awk program in the hook, awk
cannot parse Python, and a fix only one of the pair can run is the drift the
mirror exists to prevent. Both sides now run the same left-to-right scan and
blank comment and string spans to their own newlines before any matcher sees
a line. Three things the scan has to get right, each of which cost real
definitions while it was being written:
- a string that HOLDS a marker is not a marker. `"red /* "` in
test_design_stylesheet.py and a triple quote inside a single-quoted
regex in plugin_context.py each ate every definition below them.
- `#` is a colour in CSS and a comment in Python, and the extractor is
handed no path. An alphanumeric straight after it settles it.
- an unterminated opener blanks NOTHING. The scan rewinds past it and
continues, so a stray marker costs one span rather than the rest of the
file.
The comment claiming the two extractors agree has been the only thing
holding them together, and a comment cannot fail. The mirror test now RUNS
the hook's awk over the same vectors: with the old program it reports the
phantoms, which is what a guard that can fail looks like. Across all 631
scannable files in this repo the two now agree line for line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A CORRECTION TO THIS ISSUE'S OWN PLAN, first, because the plan was wrong in
a way that would have cost the acceptance case.
#4208 recommended widening `kind` past `css | sym` to carry the form, calling
the denominator the deeper win and a migration the price. Two things:
1. THE MIGRATION IS NOT NEEDED. The issue says density "is computed from
stored rows and `shape_form` only runs on read". `canon_density` does
`select(CodeShape)...scalars().all()` and filters to the directory in
PYTHON; `signature` is a stored column; `canon_form(siblings, ...)` on the
next line already derives form from those rows. Bucketing the denominator
costs a list comprehension.
2. WIDENING `kind` WOULD BREAK MILESTONE #2793's ACCEPTANCE CASE. If `kind`
separated `fn` from `async-fn`, density would bucket them apart — which
does silence #4204's four `def` helpers, as claimed. Apply it to a
hand-rolled SYNC `confirmDanger` in a directory where an async confirm
helper is canon: the canon leaves the candidate's denominator, nothing
dominates, no flag. That is the flag the milestone exists to produce, and
it is the same inversion the first form gate made, one layer down.
So option 1 is not expensive-but-right, it is wrong; and option 2 (compare
meaning) is not the interim, it is the only lever. `confirmDanger` beside an
async confirm helper and `is_registered` beside a service unit are identical
at every structural level — same family, same form contradiction, same
density. They differ only in whether the candidate does the canon's JOB, and
no signature carries that.
WHAT THIS SHIPS is the part that is right and provable: an honest
denominator. `comparable_siblings` narrows the count to rows whose family
does not contradict the candidate's, using `families_conflict` — the same
predicate the gate uses, so the count and the verdict cannot drift into
disagreeing about what comparable means. "372 judged siblings" stops counting
a dataclass, a constant and an async service unit as three comparable things.
NARROWED BY FAMILY, NOT FORM, for the reason above: `fn` beside `async-fn`
stays a fair question. An unreadable sibling STAYS COUNTED — dropping it
would shrink `judged`, raise the share, and fire the check more on the
directories it can read least. Every unknown-form decision in this module
goes that way.
Density is now per candidate rather than per kind, cached on (kind, form).
TWO BUGS THIS CHANGE HAD, both caught before CI and both pinned:
- I passed a FAMILY where `families_conflict` reads a FORM, so
`shape_family("callable")` returned "" and the narrowing was a silent
no-op that still read as applied. The regression guard deliberately uses a
callable: `type` is both a form and a family name, so testing with it
proves nothing and the bug hides.
- My new `_Row` in the test file shadowed the one already there — same
fields, different `snippet_id` default — silently breaking three passing
`canon_form` tests. Reused the existing class. A duplicate definition
quietly changing a neighbour's meaning is this file's own subject.
The residue test's docstring said separating the four needs "widen `kind` or
a comparison of meaning". Corrected: they are not alternatives.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`retrieval_telemetry(days=30)` reported, for write_path_rule:
"the weakest tenth of what this arm returns scores 0.6984, only -0.0216
above its floor of 0.72"
A negative distance above something. The tenth percentile of what an arm
RETURNED cannot sit below the floor that gates what it may return — not
inside one population.
MEASURED CAUSE. write_path_rule's floor was 0.68 until 2026-09-02, when
2385100 (#3318) raised the shipped default to 0.72. The window opened
2026-08-22, so six days of it are calls made under the old bar; top_score.min
for the surface is exactly 0.68, the old bar still in the sample.
AND THE CHANGE LEFT NO TRACE THE READOUT COULD SEE. retrieval_tuning_events
records dial turns — a person or a model choosing a number. It was silent
about the other way a floor moves: somebody edits floor_default and ships it.
retrieval_tuning_history returned {"events": []} and retrieval_surfaces said
last_change: {}, source: "shipped". All true, and all of it silent about a
floor that had in fact moved.
THE RAISE ANNOUNCED ITSELF. A LOWERED FLOOR WOULD NOT: the gap comes out
comfortably positive and reads as a clean bill of health on a sample that
half predates the bar being judged. Both directions are now pinned.
So the check is SUSPENDED, not softened. band_hugs_floor asks whether the
scores are piled on the bar; that needs the scores and the bar to come from
the same regime. Where they do not, the honest answer is that this sample
cannot say, plus the date after which one can — floor_moved_mid_window
replaces band_hugs_floor for that arm and never accompanies it. A reader told
a number is unavailable goes and gets one; a reader handed a qualified number
uses it.
NO MIGRATION. `actor` is Text with no CHECK precisely so a new kind of actor
is not one — the model's own comment says so, and this is the case it
anticipated. "release" joins "model" and "human". user_id is already
nullable, which is right: no user did this, a release acts on every account
that has not overridden the dial, and a row per user would both multiply and
misattribute it. Both readers now take the newest of (this user's change, the
release's).
THE FIRST SIGHTING IS A BASELINE, written with old_value NULL. Nothing moved;
the row exists so the next release has a predecessor. That null is
load-bearing: floor_moves_since asks for old_value IS NOT NULL, so a fresh
install's baseline does not silently retire the check on every new install.
UI: the tuning history rendered actor as `human ? 'you' : 'Claude'`, so a
release row would have told the operator that Claude moved a floor it never
touched — the one failure the actor column exists to prevent. Three-way now,
with an unknown value printing itself rather than guessing.
Recorded at startup, inline and awaited. What #4181 cost three hours was
concurrency — a background task racing the hook for the same pool. Sequential
creates no contention, and this is twelve single-row reads. It must finish
before serving because a readout served before the change was recorded is the
exact answer this exists to stop giving.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419's acceptance, and it failed the first time it was run — which
is the only reason this commit exists.
THE MEASUREMENT. Ran the step-5 readout against this session's real traffic
instead of a fixture. It reported 19 rules named by an arm and none opened.
That is false: the session had called `get_rule` 45 times. Across six real
sessions on this instance: 208 opens, 3 surviving ledger entries. 1.4%.
THE CAUSE. `.opened.ids` was doing two jobs with opposite lifetimes.
- "this context HOLDS rule 156" — false after a compaction, and three hooks
read it to decide whether to stay quiet. Clearing it is correct.
- "rule 156 WAS OPENED" — which no compaction makes untrue, and which the
session-end readout is built on.
`scribe_clear_session_ledgers` sweeps every `<sid>*.ids` on SessionStart
source=compact. Right for the first claim, and it was deleting the second.
The TTL did the same thing more quietly: `scribe_rules_live` ages an
exclusion ledger, which is right, and would have eaten the early part of any
long session's evidence too.
So the readout was reporting only the stretch since the last compaction while
reading as though it had reported the session — a statistic that cannot vary
being mistaken for a finding (#3311), which is the shape this whole milestone
exists to stop producing. It ran AT the seam it was blind to.
THE SPLIT. `scribe_rules_append` now writes both: the exclusion ledger it
always wrote, and `<kind>.keep.ids`, an evidence twin that is never aged and
never swept. The readout reads twins; the three `held` readers are untouched.
DERIVED, NOT LISTED, because a list is what broke this before — the comment
above the sweep says so about its own history. Every ledger written through
the appender gets a twin, including the next one somebody adds; a new ledger
is born on the swept side unless its name opts out. `scribe_checkpoint_allowed`
writes its twin explicitly since it bypasses the appender, and there the split
lands right on both sides: the cap counts the swept file, so a compaction
honestly restores the budget to stop an act the context can no longer justify,
while the record that a stop happened stays.
Removed `scribe_ledger_ids`, orphaned by the change — a dead helper beside a
live one is a thing the next reader trusts.
test_session_ledger_clear.py asserted every ledger dies and could not have
caught this: its fixture never created a twin, so the sweep was one glob away
from either mistake with only one of them guarded. Both sides now asserted.
The slippage tests build their ledgers through the real writers rather than
by hand, for the same reason — a fixture that writes the bytes itself keeps
passing after the writer stops.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 5 of milestone 419. The milestone's subject is that a rule read and
ignored is arithmetically identical to a rule read and followed, and the
compaction is where that identity becomes permanent — the turns holding the
evidence are summarised away, and the unjudged thing survives as nothing.
WHY THIS IS ASSEMBLED IN THE HOOK. `rule_usage_events` has no session column;
it is per user over a window. A session-scoped answer therefore cannot be
asked of the server, and has to be built where a session is a thing that
exists. Four ledgers four hooks already write:
.rules.ids an arm NAMED the rule
.opened.ids the session called get_rule (#4100)
.acted.ids the session called rule_outcome (new here)
.checkpoint.ids the rule HELD an act (#4214)
Every one is an observed tool call. Nothing asks the model what it followed —
milestone 386 ruled that out, because a model asked "did you apply rule 156?"
says yes. Two subtractions: named-minus-opened is the arm talking to nobody,
opened-minus-acted is the milestone's whole subject.
PreCompact stdout is the compaction's custom instructions (#3680), not a
message to the model, so the readout does not say "you slipped" — it says
which ids must be carried through, which is the one thing a summary can do
about an unjudged finding.
SILENT WHEN NOTHING HAPPENED, and the accusations are conditional on having
members. "0 rules unresolved" on every compaction is how a readout teaches
its reader to skip it. Traffic is still reported, because the static
instructions already ask for it in prose; these lines are the measured
version.
scribe_record_outcome.sh is the third ledger's writer, matched on
mcp__.*__rule_outcome and mirroring scribe_record_opened.sh: TMPDIR only,
silent, exit 0 on every path. A PostToolUse hook that spoke would put a line
after every rule_outcome call and give recording an outcome a cost.
Also: check_plugin.py skipped the new hook for want of a smoke event, which
would have left the newest of the three ledgers as the only one the plugin
lane never runs. Added, mirroring its sibling.
tests/test_precompact_hook.py now isolates TMPDIR — the hook reads session
ledgers from there, so without isolation a test would see whatever this real
session had accumulated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 4. Five of the milestone's seven misses were the same move:
acting on the thing in hand without reading the contract around it. Lesson
#4207 says so in words, and was written by its author hours before a
structurally identical mistake, having been surfaced twice in the turns
between. Text delivered at the moment of acting is too weak a carrier for a
reflex that has to change what the act IS. This looks it up instead.
Rule 33 one scope down: its checks are between layers, and the same question
exists between a definition and its callers.
THREE KINDS OF EXPOSED NAME, because a contract breaks three ways that look
nothing alike in source — the defined symbol (a rename or removal), its
parameter names (arity), and the quoted keys of its dict literals (the shape
of what it returns).
THE THIRD IS THE ONE A SIGNATURE-WATCHER MISSES, and it is in because of the
miss that produced this step. Two commits ago `get_writepath_config` gained
one dict key; three arms read that dict inside a fail-open `except`, every one
silently became a no-op, and ten tests went red with nothing pointing at the
cause. No signature changed. Run against that exact edit, the check now names
tests/helpers.py — the actual root cause — among six files, before the write.
TWO GATES, AND THE SECOND IS WHAT MAKES IT USABLE. A change to the exposed set
is necessary but not sufficient: a definition nothing else references has no
contract to break, so the readers lookup runs second and an empty result ends
it silently. Body-only edits say nothing, a subject is named once per session,
and the ledger lives in the swept directory under the `.ids` convention, so
the existing compaction-clear guards cover it — checked against
test_session_ledger_clear's own parsers rather than assumed.
LOCAL AND SERVERLESS, like the duplicate-name arm beside it. It needs the
working tree and nothing else; the server has no checkout, so this is the only
place the question can be asked. It is a NUDGE: scribe_prior_art.sh still
returns no permissionDecision, which is the operator's recorded decision that
a recall aid may not stand in the way of a write. A test asserts that here as
well as in test_write_path_trigger.py, because this is the arm most likely to
tempt someone into making it a gate — it reports something that may already be
broken.
The `sym` half delegates to `scribe_defs` rather than repeating its patterns:
those cover nine languages and have been corrected several times, and a second
copy would inherit today's version and quietly stop agreeing with it (#3497).
Verified by lifting the test file's own helpers and driving all 19 cases
against the real shell over a fixture git repo. Two of my own errors were
caught that way and are fixed: the fixtures were arriving as single lines
because Python `repr` inside bash single quotes leaves `\n` as two characters
(the extractor is line-oriented, so the tests would have gone green against
input no editor can produce), and the no-readers case put its subject in a
file that was not the excluded one, so it had a reader and tested the
opposite of its name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7126: 1 failed, 2227 passed — down from 14. The last one was mine, and a
different mistake from the batch before it: I widened four assertions on the
literal `{"context": "", "rule_ids": []}` by replacing the string, without
checking which arm each test was calling. Three are the tool arm. The fourth,
`test_the_prompt_arm_says_nothing_when_asked_nothing`, is not.
The distinction is real rather than an omission, so the assertion goes back
with it written down. The two ACT arms can hold a call because there is a
composed act to hold. The prompt arm fires on the operator's message, before
anything has been decided — there is nothing to put a rule in front of, and a
checkpoint there would have to guess at an act that does not exist yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7124 Python tests: 14 failed. One root cause behind ten of them, and the
failure was the exact one `tests/helpers.writepath_cfg`'s docstring already
warns about in prose — while being unable to prevent this instance of it.
Three arms read their numbers out of that config dict inside a fail-open
`except`. A missing key raises where nobody sees it, so the arm becomes a
silent no-op, indistinguishable from the arm working and finding nothing. The
helper derives its keys from `retrieval_surfaces.SURFACES` precisely to stop
that — and `checkpoint_threshold` is deliberately NOT a surface, because
everything in that table is a floor/budget pair belonging to one query and the
checkpoint runs none. The derivation therefore could not see it, the write-path
rule arm died before `record_retrieval`, and ten tests went red at once.
Fixed at the helper, from the module constant, so there is still exactly one
literal and it lives in the product. And the guard the docstring claimed now
exists: `test_the_config_stand_in_carries_every_key_the_real_one_does`
compares the stand-in's key set against the real `get_writepath_config`, so
the next key that is not a surface fails loudly here instead of quietly
disabling an arm under test. `test_retrieval_surfaces`'s hand-written key list
gains it for the same reason, spelled out in place.
The other four were the contract widening itself: `checkpoint` is present on
every return of the tool arm, including its early ones, so four assertions
comparing the whole dict needed it. That key is deliberately always present —
the two arms feed one shell reader where an absent key and an empty one are
read the same, so the difference is invisible exactly where it would bite.
Verified statically: every cfg in the suite now routes through `writepath_cfg`
(test_write_path_trigger's local `_cfg` delegates to it), no hand-written
config dict survives, and the real config's eight keys match the stand-in's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 3, the pre-act checkpoint. Every rule surface in this
plugin returns `additionalContext`, which Claude Code delivers alongside the
tool RESULT — so the rule is read after the call is written and lands as
commentary on a decision already made. That is the milestone's central
finding, measured over a session with seven misses, three caught by the
operator and none by this system.
The action arm can now return a `deny` instead. The act does not run, the
rule's text can be read before the call exists, and the remedy is one
`get_rule` call after which the act may be re-submitted unchanged. Nothing
reaches the operator: a deny is a message to the model.
"CONSEQUENTIAL" IS DERIVED, NOT ENUMERATED. The obvious implementation lists
act kinds — a write to product code, a schema change, a bulk classification, a
merge. Every one of those is consequential because THIS operator wrote rules
about it, and shipping that list is this instance's corpus hard-coded into the
product (rule 115). So the corpus decides: an act is consequential when the
install's own rules speak to it above the checkpoint bar. A fresh install with
no rules never stops anything.
FOUR CONDITIONS, EACH PREVENTING A DIFFERENT WRONG. Above the bar; a rule and
never a preference (which claims no such force); the band's top hit only (the
ranker's confidence claim attaches to its first element); and only a rule the
session has NOT opened — `held` is observable from the get_rule PostToolUse
hook (#4100), not self-report.
WHY "NOT OPENED" RATHER THAN "NO OUTCOME RECORDED". An outcome can be
satisfied with one cheap call asserting compliance without producing any, and
a checkpoint dismissible that way manufactures exactly the compliance data
step 2 was built to measure. Reading a rule cannot be faked in that direction:
after `get_rule` the statement is in context, which is the whole of what was
wanted.
THE BAR IS MEASURED. `retrieval_telemetry(days=30)`: write_path_rule p90
0.7628 max 0.8817; pre_tool_rule p90 0.7373 max 0.8293. 0.80 is above p90 on
both and below max on both, so it selects from the top decile of an already
selective arm and is still reachable. It ships as a setting with a Settings
card, because a cosine distance in one model's geometry over one corpus cannot
transfer.
TWO GUARDS ON THE WORST CASE: at most one hold per rule and five per session,
so a mis-set floor degrades to a noisy session rather than one that cannot
proceed. The ledger lives in the swept directory and is named `.ids`, so the
existing compaction-clear guards cover it.
WRITES ARE NOT HELD, AND THAT IS THE OPERATOR'S DECISION RATHER THAN MINE.
`scribe_prior_art.sh` carries a tested property that it never returns a
permissionDecision — a recall aid may not stand in the way of a write. Three
of the milestone's seven misses were file edits and none are reachable from
the command side, so there is a live argument for extending this; that
argument is exactly why the boundary is now asserted by a test rather than
left to memory. The write-path arm computes and returns the same block so the
decision can be revisited with evidence; the hook ignores it, and a change of
mind is a hook edit rather than a feature.
Verified by lifting `checkpoint_for` and `_rule_band` out of source with `ast`
and exercising the shipped functions over 17 populations, by running the
ledger and deny envelope in bash (10 cases, including that a refused hold is
not written and that a garbled rule id fails closed), and by
scripts/check_plugin.py — which caught the unminted plugin version, 0300 ->
0426.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 2. Step 1 made an outcome recordable; this makes it
readable. `retrieval_summary`'s rule block gains `applied`, `departed` and
`distinct_rules_acted`, and `_compute_warnings` gains two codes.
TWO CODES, NOT ONE WITH A ZERO IN IT. `read_and_unacted` reports rules that
were opened and left no outcome, against the ones that did. It only fires once
outcomes exist anywhere in the window, because a window with none cannot tell
"every rule was ignored" from "nothing calls `rule_outcome` yet" — and on
every install the day this ships, the truth is the second. Claiming the first
there would be #3311's failure exactly: a statistic that could not vary being
read as a fact about the corpus. The cold case gets its own code,
`outcomes_never_recorded`, whose prose says in as many words that it does NOT
mean the rules were ignored.
`applied` AND `departed` ARE NOT SUMMED. A departure carries the reason the
agent gave and is evidence about the RULE; an application is evidence about
the agent. Folded together they would say only "an outcome exists", which is
true of both and useful about neither. `distinct_rules_acted` counts either,
because for the unacted arithmetic the distinction does not matter.
An outcome is not a pull. The fold branches on OUTCOMES first and never routes
an outcome through the surfaced/ambient split: `source` on an outcome row
names the door the outcome came through, not a ranker, so the ambient
distinction has nothing to say about it. An integration test holds that line —
if an outcome leaked into the pull counters the silently-unchanged rule would
vanish into a compliant-looking total, which is the confusion #4212 was opened
to end.
Verified by lifting the shipped `_compute_warnings` out of source with `ast`
and exercising it against the six populations the new tests assert: cold
instrument, warm instrument, departures-only, full compliance, nothing opened,
and a failed read. The integration tests for the new counts run against real
Postgres in CI — count(distinct) with an IN over an unconstrained column is a
SQL shape a mock would agree with whatever it did, which is what #2663 was.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419's own subject, committed while building it. The module header
of mcp/tools/rulebooks.py says the tool count "lives in the registration
test, which fails when it drifts" — I read that line while adding the tool
and did not act on it, which is a rule surfaced, read, and silently
unchanged. The only reason it was caught is that the guard exists and CI ran
it; nothing about my process would have found it.
Worth stating plainly because the milestone is about exactly this class of
miss, and step 1 shipped the record that would let a future session SEE it:
a rule read and not followed is invisible unless something independent
notices. Here the something was a test written by whoever last changed this
count.
Integration was already green on the previous run, so the migration, the
backup round trip and the new `detail` column were never in question — this
was one integer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and
that it was PULLED. It could not say what happened next, so a rule that fires
constantly and is always obeyed and a rule that fires constantly and is never
obeyed left byte-identical telemetry. The second is far the more urgent and
was the one the readout could not name — measured on a session where three of
seven misses were caught by the operator and none by the system.
Two new events, `applied` and `departed`, and a `detail` column carrying the
why of a departure. No CHECK migration: `event` was created in 0094 as plain
Text with no constraint, verified in the migration rather than assumed from
the model, so rule 36 does not bite here — said in both places because the
next person adding a value will reach for it.
THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently-
unchanged is the failure this milestone was opened on, and it cannot be
reported: an agent that knew it was ignoring a rule would not be ignoring it.
So nothing here asks. `applied` and `departed` are reported; the third state
is a rule that was opened and left no trace. An `ignored` enum member would
collect nothing while reading as though it had measured something, which is
#3311's failure — a statistic that could not vary being taken for a finding.
`detail` is a column rather than two more bare event strings because a
departure stripped of its reason reads back as a miss, so the two states this
exists to separate would collapse again one layer down, in the readout, where
nobody would see it happen. Nullable: following a rule needs no argument, and
an expensive event is one that stops being recorded.
`outcome_state` is the single reading of the four states, taking the aggregate
`usage_for_rules` already returns, so the badge, the readout and any later
session summary cannot disagree about what "followed" means — the drift #3246
found across the rules system. A departure outranks an application: a rule
both applied and argued with is a rule someone argued with, and the argument
is the half worth surfacing.
`rule_outcome` is the MCP door, classed as a WRITE. The read-only set
tolerates getters that call record_pulled, but those are reads that leave a
trace; this tool's entire effect is the row, and the row carries prose the
agent authored. A read-scoped key that can put text in the operator's
database is not read-scoped, whatever table it lands in.
Backup carries `detail` on both sides. It is the one field here a fresh
install cannot re-earn — counts come back by being used again, a stated
reason exists once — and #4197 records that the column guard watches the
export side only, so the round-trip test is the thing that would catch a
one-sided add.
Delivery is deliberately not settled here: how an agent gets prompted to
record an outcome is step 3's subject, and the same record serves whichever
answer that step reaches.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The review surface shipped yesterday reported two canons on its first live
day and both were sound. Coherence was "do all judged rows share a form",
which #2844 failed at 37/62 = 0.597 for containing a model class and the
to_dict the canon's own text says the class must carry, and #2849 failed at
4/7 for pairing sync loop-starters with the async ticks they schedule. A
review surface whose whole output is noise is one that stops being read.
The obvious repair is a trap, and there is now a test standing in front of
it. Grouping by family and keeping a majority test makes the check BLIND:
before #2844 was cleaned by hand it held 37 classes and 56 callables, which
as families is 56/93 = 0.602 — a clean pass, and the 31 rows that had no
business being there (Vue functions, route handlers, a dozen tests) would
never have been reported at all. A looser bar in the same shape is worse
than the bug.
So the verdict is inverted. Instead of asking whether most rows agree, it
asks how many rows the canon CANNOT ACCOUNT FOR: a row in the majority
family is accounted for; a callable defined in a file that also holds a
majority-family `type` row is a method of a member, not a foreign body; and
strangers above a fifth of the readable rows make the canon incoherent. The
majority vote abstains those methods, so a class's own serialisers cannot
outvote the classes and turn the members into the strangers.
Measured on the real ledger before it was written, which is why it is this
rule and not a nudge to the share: clean #2844 has 0 strangers in 62,
#2849 has 0 in 7, and polluted #2844 had 31 in 93 — the same 31 withdrawn
by hand this morning, named exactly.
The entry now carries `families`, the majority `family`, `attached`,
`stranger_count`, `unattended`, and `strangers` — THE ROWS THAT DO NOT FIT,
replacing a sample of the first twelve members. The reader's question is
which rows are wrong, and a sample of the agreeing majority cannot answer
it. `unattended` is the discriminator between a check that is too strict and
a ledger full of junk: both canons flagged on day one were entirely
audit-judged, and nothing showed that without opening each one.
Scoped to the review surface. `canon_form` still answers at the precise form
level for stamping and divergence, where a sync helper beside an async canon
is a fair question; nothing here changes what the ledger writes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7098: unit tests red, everything else green. `_INSTRUCTIONS` was 2439
against a 2000 budget.
WHAT I DID NOT CHECK. That block is capped because Claude Code injects only
the first ~2,048 characters of a server's instructions and cuts the rest
mid-word (#2562, observed live — a 20k version delivered ~10% of itself and
the Systems guidance never reached a session). The cap is stated in a comment
directly above the literal I edited. It was at 1998/2000 before this batch:
a shared, nearly-exhausted resource, and I added a six-line entry to it.
THE JUDGE LINE STAYS, and paying for it is the decision rather than dropping
it. A client with no Agent Skills support receives this index and nothing
else, so of everything here, "you are the judge of record" is among the least
safe to leave past the fold — an agent that never learns it defers every call
to an operator who was never going to make them.
So the line is earned by compressing prose AROUND the existing markers, not by
removing anyone's entry: RULES loses a clause, RECORD and REPORT lose trailing
restatement, PLAN drops a sentence the two markers already imply, and the
opening paragraph tightens. Every index marker the ownership registry requires
survives verbatim — that is what test_the_index_names_each_reflex_it_points_at
checks, and it passes.
Back to 1998/2000: the same headroom as before, with one more reflex indexed.
The next addition pays the same way.
Three guidance modules run green locally (21 tests) — they read files and need
no database, so this one did not have to go to CI to be known.
Plugin version re-minted; the previous mint is on a commit that never went
green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
I recorded this as project rule 174 first. That was wrong twice over, and the
second reason is the one that matters.
RULE 119 SAYS THIS EXACTLY: guidance about how an agent should behave with
Scribe belongs in `_INSTRUCTIONS`, `plugin/skills/*` or the adapter's static
context, never in the corpus. I read 119 while writing the rule, decided it was
"about authority rather than about using Scribe", and wrote it anyway — which
is the reasoning preference 29 exists to catch, performed in full.
THE REASON THAT MATTERS: a rule in the corpus is true on ONE install. If the
agent being the judge is how Scribe works, every install gets it or none does.
Baked in, it ships. As a rule it was one operator's private note about a
product stance.
WHAT IT SAYS. The agent is the judge of record for the work — what a shape is,
whether a finding holds, whether something is done. Surfacing a finding for the
operator to rule on is the judgment NOT made, however well written up: it reads
as diligence and functions as a backlog. Escalate the acts that are genuinely
theirs — their money, their infrastructure, anything hard to reverse or facing
outward — and keep the decisions. A hard call is still yours; an irreversible
act is still theirs.
And the half that keeps this from becoming the previous defect: JUDGING IS
ATTENDED. An agent reading evidence and recording why is judgment; a threshold
or a sweep reclassifying in bulk with nobody reading is the thing that fills a
ledger with confident nonsense (#4208, and Portal's 35 rows). When the fix for
bad unattended writes is another unattended write, stop.
THE PRODUCT WAS TEACHING THE OPPOSITE. reporting-back's Finding row read
"Symptom · Cause · Size of the fix · **Offer to fix it**". So the behaviour I
was corrected for is the behaviour the skill prescribed — which is the better
argument for fixing it here than any rule could be.
Three surfaces, per 119 and the ownership registry (#4027): `_INSTRUCTIONS`
gets a one-line JUDGE index entry; using-scribe owns the authority and the
attended/unattended distinction; reporting-back owns the report shape. Two
topics rather than one, registered separately in test_guidance_ownership so
trimming one cannot quietly take the other.
Plugin version minted — skills only reach a session when the manifest moves
(#2209).
Rule 174 deleted (trash 074434a2, recoverable).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Lint caught an F821 that would have been a NameError the first time
`stamps_to_review` was called: `access` is imported locally inside each of the
seven functions in this module that need it — services/access reaches back
here, so a module-level import closes a cycle — and the new function used it
without one.
I wrote the function by pattern-matching its neighbours and did not check what
those neighbours do to make themselves work. Same shape as the tuple unpack
two commits ago (#4207): the mistake is not in the logic I was thinking about,
it is in the surrounding contract I did not read.
Unit and integration were both green on the failing run (7095); only lint was
red. Worth recording because the lane that caught it is the cheapest one and I
had read its command as covering tests — `ruff check src/ scripts/` does not
look at tests/ at all, so a clean test suite says nothing about it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
THE HALF THAT WAS MISSING. #4204 put a floor under what the write-path hook
may assert. A floor only guards new writes; every row already stored stands
(lesson #4202). Measured after that fix shipped: Portal carried 32 rows under
one canon and 3 under another, all stamped on scores of 0.69-0.77 — below the
0.80 floor, so none of them could be written today, and all of them were still
there. Scribe's own ledger carries 334 under #2860.
`stamps_to_review` reports two things and changes nothing:
weak — rows the hook stamped on a resemblance below the current floor,
each with its score, signature and derived form.
incoherent — canons whose own judged rows do not agree on a form. A canon
claims some shapes are the same sort of thing; when its members
are a class, three getters and a dozen tests, that claim has
stopped being true and every base-rate reading built on it is
reading noise. `canon_form` already made such a canon fall
silent — nothing made it VISIBLE.
IT DELIBERATELY CANNOT FIX ANYTHING, and that is the design, not an omission.
The first version of this commit was an automatic sweep that reset rows by
score. That is the original defect pointed the other way: what harmed the
ledger was not one wrong score, it was a machine recording permanent
classifications unattended. Un-recording them unattended is the same act with
a wider blast radius. An agent reads the evidence, judges, and records the
judgment under its own name through `classify_shapes`.
`test_the_service_carries_no_machinery_for_bulk_withdrawal` asserts that
structurally, so the next person to reach for an auto-retire has the argument
again on purpose rather than in a diff nobody reads.
A JUDGMENT IS NEVER LISTED AS WEAK, whatever its age. This is the measured
correction to an assumption I nearly shipped: of Scribe's 334 rows under
#2860, 302 are in `services/` — the canon's own home — and the ones sampled
there are `classified_by="audit"` with no score at all. The legitimate bulk
of that canon was never scored; it was judged by an agent in batch. Listing
those as weak would invite an agent to withdraw the only real judgments in the
ledger. An agent's decision is a different KIND of evidence, not a worse one.
THE SCORE NOW HAS A PARSER. It lived only inside a prose sentence, so nothing
could ask how strong the evidence for a row was without re-deriving it — which
is how 32 rows sat unexamined for nineteen days. Format and reader are one
constant apart (`_RESEMBLE_REASON` / `stamp_score`), with a round-trip test and
a test pinned to reason strings taken verbatim from the two poisoned ledgers.
`live_rows_for` is `live_rows` behind the project read gate, for callers that
arrive from outside rather than from a job that already knows who is asking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
[s_id for rank, _at, s_id, _why in bucket if rank == 2]
ValueError: too many values to unpack (expected 4, got 5)
`in_play` gained the canon's form as a fifth element so the stamp could be
decided per shape rather than per file. The `record_uses` call eighty lines
below still destructured four, and every stamp that reached it raised.
Both integration failures on runs 7090 and 7091 are this one line —
`test_write_path_stamp_is_evidence_that_yields_to_judgment` and
`test_a_brand_new_shape_gets_a_provisional_row_the_sync_settles`.
Now indexed rather than destructured, matching the candidate scan above it, so
the next widening cannot break it positionally.
WHY THE UNIT LANE STAYED GREEN THROUGH TWO PUSHES. `record_uses` is only
reached once a stamp is actually written, which needs a real snippet, a real
ledger row and the write ACL — so no unit test crosses that line. 139 local
assertions and the whole unit suite passed on code that raised on every
successful stamp. The integration lane was the only thing that could say so,
which is the case for it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 7090 caught this; the unit suite could not. Integration job 25763 —
`test_a_second_confirm_dialog_is_detected_and_named`, the acceptance case of
milestone #2793.
WHAT I GOT WRONG. The previous commit gated BOTH halves of the ledger on
`forms_agree`. That is right for stamping and backwards for divergence,
because the two assert opposite things:
STAMPING says "this IS that canon". Agreement in form is evidence FOR the
claim, so demanding it is correct.
DIVERGENCE says "this is NOT the canon that dominates here — did you mean
to?" A form MISMATCH is the PREMISE of that prompt. Requiring the candidate
to match the canon silences the check precisely where it belongs.
So #2793's case stopped firing: a hand-rolled sync `confirmDanger` in a
directory where an async confirm helper is canon read as `fn` against
`async-fn`, disagreed, and was dropped. `flag_divergence` returned 0 where the
test demands 1, and the write-time check returned nothing where it must name
the canon. That is a real flag the milestone exists to produce, and my change
removed it.
THE FIX. Divergence now gates at FAMILY level — callable {fn, async-fn},
type, value, css — and only on contradiction. A sync function beside an async
one is still a fair question. A frozen dataclass told to build from an async
service function is not a question at all.
WHAT THIS DOES NOT FIX, asserted rather than commented so it fails the day it
changes (`test_how_many_of_the_five_the_divergence_gate_actually_silences`):
of #4204's five false prompts this silences ONE. `Point` is a type against a
callable canon. `_p`, `get_point`, `is_registered` and
`sources_expected_to_emit` are callables like the canon and still ask — and
at the signature level they are indistinguishable from the #2793 case above,
so nothing readable here can separate them. That needs #4204 option 2 (widen
`kind` past `css | sym`) or a comparison of meaning rather than form.
The stamping half — `_RESEMBLE_MIN` 0.80 and the graded burden — is unchanged
and unaffected by this failure. It is also the half that matters more: loose
stamping is what manufactures the density the divergence check reads as
authority.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Two halves of one defect, found while writing #3431 and measured on a second
project.
THE DIVERGENCE CHECK WAS A BASE RATE. `dominant_canon` answers "what is most
common in this directory" and never "is this that" — the candidate's
signature was not examined at all. With `kind` carrying only `css | sym`, a
frozen dataclass, a module constant, a sync predicate, a class and an async
service function are all siblings, so the prior was not merely the best
signal, it was the only one. Writing a registry module of pure helpers
produced five prompts to build them from the `async_session` service canon.
THE AUTO-STAMP HAD NO FLOOR. `elif sid in resembles` took any score at all:
0.69 asserted as confidently as 0.95, and the number went into the reason
line without ever being compared to anything. Worse, the score is computed
against the WHOLE PAYLOAD, so one number spoke for every symbol in the file.
On Portal that recorded `class SessionAbsent`, `def build_channel`,
`async def attach` and a dozen test functions as instances of one snippet —
17 rows under #3283, which then made that directory "canon-dense" and started
instructing every later writer in it.
The two compound: loose stamping manufactures the density the divergence
check reads as authority. Both are fixed by one primitive.
`shape_form` derives a coarse form — css / type / async-fn / fn / binding —
from the signature, on READ. `kind` is part of the row identity, so widening
that column needs a migration and a re-extract (#4204 option 2, still the
principled fix); deriving costs nothing and is reversible. Every caller asks
`shape_form`, so the day the column carries the answer it returns that.
THE BURDEN SCALES WITH THE EVIDENCE, and getting this wrong was the first
version. A by-name reference — the payload names the canon's symbol — is
strong and needs only the absence of contradiction; demanding positive
agreement there silenced it whenever a shape's definition was not in the
payload (an Edit rather than a Write), turning strong evidence into none for
a reason unrelated to the code. A resemblance score is weak and must
positively agree.
`canon_form` reads the form a canon's own judged rows agree on, and returns
unknown when they disagree. That makes the halves cooperate: a canon already
poisoned by loose stamping — Portal's #3283 — falls silent instead of
flagging anyone else.
`forms_agree` requires BOTH sides known, so an unreadable signature makes the
checks quieter rather than more confident. `_RESEMBLE_MIN` is 0.80 rather
than the retrieval floors near 0.70: those decide whether to SHOW a record,
where being wrong costs a glance; this decides whether to RECORD a claim
unattended, where being wrong misinstructs everyone who writes there after.
Caught by the tests, not by review: a first pass required `const`/`let`/`var`
before a binding, so every Python module constant read as unreadable and a
whole form was silently excluded from both checks.
Not addressed: rows already carrying a wrong snippet_id are not undone by a
guard at the point of classification (lesson #4202). Portal's 17 keep
producing dominance until something re-judges them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The tool returned distributions and left the reading to the caller, so every
readout was the same four checks done by hand — #3430's baseline, #3835's rule
near-misses, the #1038 rerank gate. Mechanical, and therefore forgettable.
Tonight's acceptance pass on #3898 was the case for doing this. Reading it by
hand meant catching that two surfaces had `covers_window: false`, that
`prompt_rule`'s floor had moved three times inside the window (which made the
readout self-contradictory: deliveries at 0.622 beside refusals at 0.7199),
and that 15 of 20 near-misses were one record against text no operator wrote.
Miss any of those and the obvious conclusion was "the bar is too tight" — a
floor change that would have injected one preference into every notification.
`warnings` is always present and empty when clean, so its emptiness is an
answer rather than a gap. Each entry carries the numbers that produced it:
"345 calls, 0 declined" is the analysis, "check write_path_rule" is an
instruction to redo it. Five codes — cannot_decline, band_hugs_floor,
no_duration, surfaced_never_pulled, unregistered_source.
cannot_decline has three guards, each a bug it would otherwise cause. Asked
surfaces are exempt (a search returning a list every time is working). An arm
not known to log unconditionally is exempt — that is #3497 exactly, where both
rule arms recorded only their hits, so a decline count of zero was a LOGGING
defect and this warning would have sent the reader to a threshold that was
never involved. Unregistered sources get numbers but no verdict.
`silent_surfaces` is the half the rows cannot show: an arm that emitted
nothing is invisible to every row-based check and looks exactly like an arm
that does not exist. It is driven by a new declared registry,
`retrieval_registry.POINTS` — deliberately NOT `retrieval_surfaces.SURFACES`,
which answers "what can be tuned" and excludes the reserved slots because a
budget of 1 is their feature. This answers "what can be measured", and the
reserved slots belong in it precisely because they are judgeable without being
tunable. A test asserts the two cannot drift apart.
The registry test derives sources from the call sites with `ast`, not grep,
and the difference is not theoretical: `wide_net` and `report_preference`
reach their recorder as `source=SOURCE` through a module constant, so a grep
for `source="` is blind to both — the narrowing #3191 warns about. Three sites
pass `source` as a variable and are declared in FAN_OUT_SITES; the test pins
those sites but not the values they can pass, which is why the
`unregistered_source` warning exists to catch the rest at first fire.
Thresholds are settings (rule 25) defaulted so a fresh install with almost no
data produces no warnings at all (rule 115) — a new user's first readout
naming five broken things would be describing the emptiness.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Claude Code submits more than typed words through UserPromptSubmit. A task
notification, a slash-command echo and the caveat banner a local command
prints all arrive as user turns, reaching `.prompt` indistinguishable from
something the operator wrote. scribe_autoinject.sh believed all of them.
The cost that matters is not the wasted embedding — it is the log row. Every
such call counts in the denominator of every prompt-boundary surface, so
delivery rate reads low for a reason unrelated to retrieval; and each refusal
lands in `near_misses`, where a later tuning decision reads it as demand.
Measured while taking milestone 399's acceptance (#3898): 15 of the top 20
`preference_slot` near-misses were `<task-notification>` blocks, all matching
ONE record — #140 "Let each action land before starting the next" — all within
thousandths of the 0.70 floor. A notification that an action finished really
does resemble a preference about letting actions land. Lowering the floor to
serve that apparent demand would have injected that record into every
notification: the instrument arguing for the wrong fix, which is #379 again.
scribe_skip_prompt is a PREFIX test, not a substring one, and that is the
whole safety argument. A real prompt may contain one of these tags — an
operator pasting a transcript, or a `<system-reminder>` after typed words —
and must still be retrieved against. Nothing an operator types begins with a
client envelope. The compaction-resume injection is deliberately NOT filtered:
it is machine-written, but it summarises real work, and a resumed session is
where recalling a rule earns its keep.
Client-side only, so no server contract moves and lagging plugin caches keep
working. The tags are Claude Code protocol constructs, identical on every
install — instance-agnostic under rule 115.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
ci-requirements.md is the document that drives promoting a per-job dep into
the ci-python image, and it still recorded jq as installed in the plugin job
and "load-bearing for the smoke test specifically", on the grounds that every
hook opened `command -v jq || exit 0` and the test would otherwise pass while
exercising nothing.
That was the tail wagging the dog. The hooks ship to users; jq is absent by
default on macOS, the Debian/Ubuntu slim images, Alpine and most CI
containers, and a machine without it got no context, no rules, no prior art
and no process sync in silence. The answer was to remove the dependency, not
to install it harder.
Recorded as a promotion the entry now argues AGAINST rather than deleted: the
next person to read this file should find out why jq is absent, not merely
that it is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Three loose ends from a49e7ed, all found by CI or by re-reading it.
TWO TESTS PINNED THE jq SYNTAX, not the word `jq`, so grepping the suite for
the tool name missed them: test_write_path_trigger asserted the literal filter
strings `(.note_ids // []) - (.sync_note_ids // [])` and `(.rule_ids // [])[]?`
appear in the hook. Re-spelled with the readers that replaced them. The claim
each makes is unchanged — that the reuse and sync channels keep their own
state file, their own query parameter and their own write-back.
CI STILL INSTALLED jq, TWICE. The "Install jq for hook execution tests" step
existed because those tests skipped rather than failed without it; the Plugin
hooks job installed it alongside shellcheck for the same reason. Removing a
dependency from the product and leaving CI to install it anyway means the
smoke tests never run under the condition they exist to assert. Both gone —
the hook tests now run on a bare image.
`awk 'END { print $0 }'` IS UNDEFINED IN POSIX. gawk retains the last record
in END and mawk does too, but it is not guaranteed, and reaching for a
non-portable idiom inside the change whose entire purpose is portability is
the same mistake one layer down. Now `{ last = $0 } END { if (NR) print last }`.
Verified it still selects the same start line on a real transcript.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Every hook opened `command -v jq >/dev/null 2>&1 || exit 0`, so on a machine
without jq the operator got no session context, no rules, no prior art and no
process sync — and not one word saying why, because `exit 0` is
indistinguishable from "ran fine, nothing to say". jq is absent by default on
macOS, on the Debian/Ubuntu slim images, on Alpine and in most CI containers.
That is not a prerequisite to document; it is the plugin handing its own
packaging problem to whoever installs it.
`tac` was worse: GNU-only, so the prior-art hook's enclosing-definition arm
did nothing at all on every Mac, silently, from the day it shipped. It is not
replaced but removed — scribe_defs judges each line independently, so
extracting forward and taking `tail -1` is the same answer as reversing and
taking the head, and it drops the early-exit `head` that #4042 was filed for.
No server contract changed, so a lagging plugin cache keeps working.
scribe_json.awk JSON -> IDX<TAB>PATH<TAB>VALUE. Two modes: `whole` for an
event or a response body, `lines` for a transcript, where
an unparseable record is dropped and the rest still read —
the `map(try fromjson catch empty)` the jq program opened
with. Arrays also report their LENGTH at `[#]`, which is
what keeps "zero notes" distinct from "no answer" (#2932).
scribe_turn.awk the turn-bounding program, replacing the thirty lines of
jq in the Stop hook.
scribe_defs.sh scribe_json_flat / _pick / _list / _len / _list_minus read,
scribe_json_out writes the envelope (five copies of one
shape, gone), scribe_urlenc replaces `jq -sRr '@uri'`.
Percent-encoding goes through `od -tu1` rather than an awk character loop on
purpose: awk's idea of a character follows the locale, so gawk reads an
accented letter as one and mawk as two, and an encoder built on substr() would
emit a different URL depending on which awk is installed. Encoding is defined
on bytes. Verified byte-identical to `jq -sRr '@uri'`.
Measured, not assumed. The per-event path costs 8ms against jq's 3ms. The
transcript path was 70x slower until two fixes: the Stop hook now finds where
the turn starts with a fixed-string grep before parsing (a needle carrying
unescaped quotes cannot occur inside a JSON string, so it matches only at a
record's top level — checked against a full JSON parse of a 27MB transcript:
152 prompt records, 152 matches, no misses, no extras), and the parser reads
each token out of a 1024-byte window instead of copying the rest of the buffer
per token, which was quadratic in line length on the 400KB tool results a
transcript carries.
Differential-tested against the jq program it replaces over 724 windows cut
from three real transcripts — 724 identical, 0 mismatched, 45 of them
exercising a real task close and a real reply. That sweep is what caught
`scribe_turn.awk` never setting FS, which truncated every multi-word reply at
its first space and was invisible to a test whose replies were all empty.
check_plugin.py's `jq -R` lint becomes a guard against either binary coming
back, and three smoke checks lose their `shutil.which("jq")` skip. jq is not
in `ci-python` either, so those three announced a skip on every CI run and had
never once run there: removing the dependency from the product also closed a
permanent hole in its verification. They pass now across all ten hooks.
tests/test_hook_json_reader.py is a differential against Python's `json` over
nested objects, arrays, unicode, escapes, control characters, empty cases and
a value longer than the token window, plus the envelope, the encoder and the
turn analyzer. 139 cases.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Run 7068 failed two tests on a4aae97, and the second was a real bug the
guard caught before it shipped.
THE COLUMN GUARD (#3182) was right. `note_usage_events` excludes only
`id` from backup, so telemetry IS exported — and a new column that the
serialiser never learned about would have restored as null on every row.
That is the exact failure #3182 was built from, where nine columns
vanished from `notes` the same way: added to the model and the migration,
which fail loudly, and never to the serialiser, which fails silently.
So `project_id` is exported, not excluded, via the one `_usage_event_rows`
builder both exporters share.
The importer re-maps it through `project_id_map` and DEGRADES rather than
skipping. `code_shape_events` drops a row whose project will not map,
because a shape event without its project says nothing; a usage event is
not like that. The project is optional by design and null already means
"not reported", so an unmappable one restores as unreported. Dropping it
would lose a real pull and deflate the pull-through this table exists to
report.
BACKUP_VERSION deliberately unchanged: the suite states that column
additions do not bump it, only new sections do, and a third test asserts
the current number.
The second failure was mine and smaller — test_record_pulled_writes_a_
single_row compares exact dicts, so it now expects the null project. I
had claimed these assertions were key-based; they are not, and I had
read a summary line rather than the assertion.
Added an import-side guard while here. The column family checks the
EXPORT side only, so a column can be exported faithfully and dropped on
the way back in — a restore that reports success and has quietly lost a
dimension. No general guard for that direction exists yet; this covers
the column added here and names the gap for whoever builds the general
one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`RetrievalLog` has carried `project_id` since it existed, so "this record
was SURFACED on project B" was always answerable. `note_usage_events`
had none, so "this record was OPENED on project B" was not — and the two
cannot be joined to recover it, because there is deliberately no session
identity server-side. NoteUsageEvent's own docstring rules that out.
That gap sat exactly on the question milestone 385 exists to answer. A
lesson's whole claim is that it reaches a session on a project it was not
written on, and step 8's acceptance is "retrieved on a different project
AND opened". Each half was answerable; the conjunction was not.
WHICH project, because the name is ambiguous and the wrong reading makes
the column useless: it is the project the READER was in, never the one
the record belongs to. The record's own project is already on the note;
copying it here would answer a question nobody asked while looking like
it answered this one.
The surfacing half is free — every arm already holds the scope it just
searched, so auto_inject, lesson_slot, the write-path arms and
enter_project now record it. process_skill_sync does not and should not:
it installs every Process the operator can reach, which is not a
project-scoped question, so a project there would be a fiction.
The pull half needs the caller, since a getter knows only what it was
handed. The five single-record getters take `project_id: int = 0` and
pass it through, following the convention `search` and `create_*`
already set. Null stays an ordinary answer meaning "not reported" — a
pull with no project is still a pull and still counts toward dead
weight; it simply cannot speak to transfer. The four REST detail views
report none for now: a human opening a record in a browser is a
different event from an agent recalling one, and #2245 left that
asymmetry deliberately undecided.
Guarded the way #2245 and #2476 taught: by source inspection, because a
parameter that was never threaded through changes no return value and
shows up only as a column that is mysteriously always null. Three
guards — the signature, the pass-through, and the arms — plus the
can-fail test rule 167 asks for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The lesson kind shipped with every mechanism for growing and nothing
telling a session to use them. `learned_from` is a list on purpose, the
dedup gate hands back an existing id rather than minting a twin, and
`update_lesson` already names re-keying a bad trigger as the edit that
pays most. None of that was reachable as a habit.
The exclusivity claim was the bug. The skill said "a preference is the
one record you keep current yourself", and by naming only preferences it
put lessons outside the habit. That sentence is now "a preference is
yours to keep current", which says the same thing about preferences
without saying anything false about lessons.
Beside it, a paragraph on what growing a lesson means: another incident
added to what taught it, a claim stated more exactly, or a trigger
re-keyed to the situation that really fired. Written as a practice
rather than a prohibition (rule 165) — the reader is named as the one
person placed to judge the trigger, because they are standing in the
situation it claims to name.
`get_lesson` carries the same prompt at the moment it bites: a session
reading a lesson inside the situation it names is the only reader who
can tell whether the trigger is keyed to what actually fired.
The guidance-ownership registry gains the topic and re-points the
preference topic's statement, since the phrase it pinned is the sentence
this change rewrites — the module asks for exactly that, in the same
commit. No index marker: the index names session-start reflexes and this
one fires mid-work, so `_INSTRUCTIONS` stays at 1998/2000.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7059's typecheck, two errors on one line: `apiDelete` returns
`Promise<void>` and takes no type argument. I had given it the response body's
shape, which it discards.
Matched to how snippets delete, rather than adding a second delete helper to
carry the batch id — no caller has wanted it, and the second helper would be
the duplication rather than the feature.
Everything else in the UI batch typechecked clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 7's actual UI. Before this the frontend had zero lesson code — the kind
existed for agents only, which is rule 27 failing.
THE EDITOR ASKS FOR THE TRIGGER BY NAME, and leads with it. Three fields —
the trigger, the claim, the detail — never one markdown box. That is the
design step 1 settled, and the evidence is blunt: the snippet corpus carries
a trigger on every record with no guard anywhere, because a service composes
the title from a named parameter. What is at 100% is a named structured
field, not a writer remembering a convention. The trigger gets the most room,
its own explanation, and a save button that refuses without it and says why.
The form shows the composed title live, so the writer is agreeing to a
document they can read rather than one assembled out of sight. A 409 from the
duplicate gate is rendered as the record that already covers the moment, with
a link to improve it and an explicit override — not as a failure.
THE BROWSE VOCABULARY GAINS THE KIND, which #3161 warned this step not to get
wrong: a facet chip, a badge label, and routing to `/lessons/:id` rather than
the note editor, which cannot edit a trigger. The badge is neutral alongside
snippet and process — a hue would make the softest record in the corpus look
like the loudest, next to a rule that actually binds.
BOTH DIRECTIONS OF THE PROVENANCE. The detail page resolves `learned_from` to
titles rather than bare ids, because "#4181" tells a reader nothing about
whether it is worth opening. And `LessonsTaughtPanel` answers the reverse on
the record's own page — the direction the task body calls the one that gets
forgotten. It has no author to type it, which is exactly why it tends never
to get built. A component, not markup in the task editor, so the same panel
mounts on any record a lesson can cite instead of being written a second time
(#3207). Silent when empty: most records taught no lesson, and a panel that
says "None yet" everywhere is one people learn to skip.
GLOBAL-BY-DEFAULT IS MADE LEGIBLE. A lesson meeting you on a project it was
not written on reads as a bug unless the page says otherwise, so the origin
line says it as a property of the kind rather than as an apology.
Design system tokens throughout; no new raw hex. `--fs-error` rather than
`--fs-danger` — 31 uses against 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 7, part two. Milestone 385 built the lesson kind through the MCP tools,
which is the agent's surface. The Vue app speaks REST, so a lesson was a
record a person could not create, read, edit or retire — rule 27 failing at
the door rather than in the view.
`/api/lessons` now offers list, create, read, update and trash, plus
`/api/lessons/taught-by/<id>` — the reverse of `learned_from`, which the task
body calls the direction that gets forgotten and arguably the more useful one:
a reader opening an old issue wants to know what was learned from it, and
until now the relation was only navigable from the lesson's side.
`lessons_taught_by` reads `data[taught_by]` through `path_exists`, the same
jsonpath dialect the snippet location lookup uses, so both reverse lookups hit
the GIN index (0070) the same way rather than scanning bodies. Share-aware via
`readable_notes_clause`: it renders beside a record the caller can already
see, so a lesson shared with them belongs there exactly as their own does.
THE TRIGGER IS REFUSED WHEN EMPTY, at create and at update. This is the one
place the door is not a thin wrapper, and it is deliberate: the service will
store a triggerless lesson quite happily — it saves, reads correctly in every
listing, and never surfaces. There is nothing to notice afterwards, because it
looks exactly like a lesson that works. Better to refuse it than to hand back
a record that looks finished. The refusal says why, so the next reader does
not take it for a nag and delete it.
`lesson_to_dict` moves into the service and the MCP tool's `_to_dict` becomes
an alias for it. Both doors now return one shape — a payload spelled once per
door answers the two of them differently the first time a field is added — and
both compose through `services/lessons.py`, so a lesson written from the web
ranks identically to one written by an agent. The document IS what ranks, so
that parity is the whole reason the door is thin.
The dedup gate matches the MCP path: two lessons under one trigger compete in
a single ranked list for one reserved slot, so a duplicate here displaces
rather than merely clutters.
NOT DONE YET: this is the door, not the UI. #3734 stays in_progress until the
Vue views, the router entries, the Knowledge browse badge and the both-ways
sources panel exist — rule 27 is about the operator being able to touch it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Groundwork for step 7, and a data-integrity fix in its own right.
Two kinds keep a queryable mirror in `notes.data` derived from their body:
snippets and, since milestone 385, lessons. Every read prefers the mirror —
deliberately, because parsing markdown to answer what an index can answer is
how a hot path rots. So a write that moves the body must move the mirror.
#3128 found that hole for snippets and plugged it with a hard-coded
`if note.note_type == SNIPPET_NOTE_TYPE`. The plug was correct and did not
generalise: lessons arrived with the same design and none of the protection,
which is precisely the "don't add a fourth instance" defect #3734 was told to
avoid.
The cost is higher for a lesson. A stale snippet mirror reports the wrong
path. A stale lesson mirror reports the wrong TRIGGER, and the trigger is the
whole retrieval story — the lesson goes on firing for the situation it used
to name while displaying the one it now names. Silent, and confident.
So `update_note` now dispatches through `_mirror_recomposers()`, a
note_type -> recomposer table. A kind with a derived mirror is covered by
registering it, not by someone remembering to widen an if.
`lessons.recompose_data` is the lesson's entry. It recovers the subject with
`embeddings.untrigger_title` — new, and deliberately placed beside the join it
inverts rather than in the caller that wanted it, because a separator spelled
in two files is a separator that will one day be changed in one of them
(#3207). `TRIGGER_SEP` is now the one spelling, and `parse_snippet_fields`
uses it too; it had the third copy inline.
The two inverses stay distinct on purpose: a snippet partitions at the first
separator (its name is a symbol), a lesson strips an exact known suffix (its
subject may legitimately contain a dash). Different algorithms, one constant,
so they cannot disagree about where the seam is.
Provenance is DROPPED when the body drops it, which is the opposite call from
a snippet's `verification` — that is carried because it was never in the body
to delete. The body is the authority; carrying a value the reader just removed
is the failure the recompose exists to prevent.
Tests: test_snippet_mirror_generic_door.py becomes
test_derived_mirror_generic_door.py, since the concern is now plural. The
registry property is asserted directly (every kind with a mirror is in the
table; the dispatch names no kind inline), plus the lesson cases and the
join/inverse round-trip. `fake_lesson` moves to tests/helpers.py — it existed
in test_lesson_surfacing.py and a second copy was about to be written — and
gains the explicit `None`s `fake_snippet` carries, because update_note reads
`verify_with` and a MagicMock is truthy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The five-answer change touched plugin/skills/using-scribe/SKILL.md, and CI
7052 caught the manifest still reading 2026.09.18.1606. Per #2209 the
marketplace clone self-updates but the cache that actually executes only
refreshes on a version change, so without this the new guidance reaches the
repo and stops there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 6 of both milestone 385 (lessons) and 399 (preferences). #3896 asked for
the fourth and fifth answers in one pass, because two people each adding one
branch to a three-way distinction produce a list that does not read as a set.
create_rule now opens by asking what kind of thing is being held, with one
question that sorts it — what happens if someone doesn't do this? Something
breaks, a boundary is crossed: a rule. It gets done a way the operator didn't
want: a preference. They lose time rediscovering it: a lesson. The closing
question grew the two matching answers, and they are named as first-class
outcomes rather than places a proposal lands when it fails. An observation
that turns out to be a lesson has been routed, not dropped.
Stated as a practice, not a prohibition (rule 165). #3557's first cut opened
"NOT YOURS TO CALL UNPROMPTED" and cost the noticing; the wanted behaviour
here is still more proposals, and what changes is only which door they go
through.
create_note says the same from its side, so routing does not depend on having
opened create_rule first — and its existing rule test ("a mistake, not merely
uninformed") turned out to name the lesson exactly. create_lesson names the
fifth kind so the set is complete from every door. create_project_rule's
citation of the loop names five answers, since it cites rather than repeats.
The force axis has one owner (decision #4027): using-scribe states all three
strengths, the sorting question, that updating a preference mid-work is the
normal case, and that preferences shape how work is done and never what gets
recorded. _INSTRUCTIONS carries the pointer — "Rules bind; preferences guide
and you keep them current; lessons inform." It had 14 characters of headroom,
so the clause is paid for by trimming atmosphere from three other lines; 1998
of 2000 now.
Guards: the proposal-loop test learns the preference branch, the lesson
branch and the force question, on both rule surfaces; guidance-ownership gains
the force-axis topic (shared with the docstrings, for the moment a proposal is
actually written) and the preference-scope topic; a new guard pins that the
index names all three strengths and who keeps the middle one current, with its
can-fail case being the omission that actually happens — a kind added to the
product while the index still describes the corpus that came before it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
On 2026-09-19 a host storage stall made one Postgres checkpoint of 14
buffers take 281 seconds against a 1.3-second baseline. The app restarted
into the tail of it, `get_maintenance_hour()` — the first DB read in
`before_serving` — hung with no deadline, Hypercorn killed the worker at
its 60-second lifespan timeout, and nothing retries a failed lifespan. A
five-minute disk hiccup became a three-hour outage that only a human
restart could clear. Every MCP call returned 405, which reads like a
routing fault and was nothing of the kind: nothing was serving.
Three changes, none of which prevent a stall — they stop a transient one
becoming a permanent one.
1. THE STARTUP READ IS BOUNDED (rule 156). `get_maintenance_hour` already
answered `_DEFAULT_HOUR` for a value it could not parse; a database
that will not answer in three seconds is the same class of "no usable
value here". The failure is now a WARNING naming the symptom — the
breadcrumb whose absence meant this was only diagnosable from
Postgres's own log — and a default run-hour, instead of the app.
2. THE BACKFILL NO LONGER RACES STARTUP. Its comment said it "never
blocks the server from accepting requests": true of requests, false of
startup, because the task began while `before_serving` was still
running and competed for the same pool. Both of the incident's
cancelled statements were in flight together. It now waits on a flag
released on the hook's way out — in a `finally`, never after the work
(rule 157), because an undeadlined wait is only safe when the wake-up
cannot be missed.
3. THE ENGINE CANNOT WAIT FOREVER TO CONNECT. asyncpg's default is 60s,
the whole lifespan budget spent before a query is sent. `command_timeout`
is deliberately NOT set alongside it and the comment says why: it would
apply to every statement, and this app runs long ones on purpose.
tests/test_startup_survives_a_slow_database.py asserts the shape rather
than the stall: a read that never returns still yields an hour, the
warning names the symptom, a healthy read is unaffected, the backfill
does no work before release, the flag is released even when startup
raises, and the engine's connect args carry a deadline but no blanket
statement timeout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
WHICH ARM — the two note arms, and no new one.
`write_path` filters kinds, so a lesson was not outranked there but
unreachable, which is #3702's shape: an arm that never had the candidate
reports a healthy bar. It now asks for lessons alongside snippets and
issues. The founding example of the kind is a lesson about a code shape,
and this is the arm that fires when code is written.
`auto_inject` does not filter kinds, so lessons were already candidates —
but scoped to the bound project, which for a kind whose whole claim is
that it transfers is the same silence. Both arms now pass
`include_global_kinds` (#3730).
WHOSE BUDGET — a reserved slot in the prompt menu, none on the write path.
The step's premise needs a correction: the notes menu and the rule hints
are separate functions with separate budgets, so a line reserved here
displaces a note, never a rule. (`RULEHINT_LIMIT` is also 5, not 1, since
#4102 made it a default rather than a cap.)
The trade taken: a note crowded out is a lost convenience and a rule
crowded out still fires at an act arm, but a lesson crowded out is the
feature failing — a lesson exists only to be met at the moment it
applies, so the arm IS its delivery and the loss is total and silent.
That is `preference_slot`'s argument, and the rarity is `reuse_slot`'s.
It buys position, never a lower bar, and it EXTENDS rather than evicting:
a displaced hit sits in the general search's own log row, and evicting it
would make two tables disagree about one call (#3668, #379).
No slot on the write path: that arm fires before every Write and Edit,
where a guaranteed extra line is a guaranteed extra interruption, and its
field is already just snippets, issues and lessons rather than the whole
corpus.
`lesson_slot` logs its own retrieval and its own surfacing from the first
deploy, and the general contest stays open to the kind — otherwise "the
slot earns its line" would be true by construction.
THE VOICE — "they don't always have to be followed".
The menu's register is already the non-binding one. What it lacked is
that a lesson reads as one more title in a list of material when it is
advice someone paid for. One clause, in the header, only when a lesson is
on the menu: weigh it, use your judgement, it is not a rule and binds
nothing. It deliberately does not borrow the rule arms' "before deciding
it does not apply", and a guard asserts that phrase never appears.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`_REPORT_KINDS` gained `lesson` and `process` without `_KIND_SUGGESTION`
gaining either, so the report would have listed pairs with no advice —
and the suggestion is the report's point. CI's per-kind guard caught it.
Lesson: read both triggers first (a lesson is retrieved by the situation
it names, so alike insights under different triggers are two lessons);
same trigger means one lesson learned twice, so fold the `taught_by`
union into the survivor and delete the other. Process: a process arrives
as a skill, so a duplicate runs the wrong procedure rather than merely
cluttering a list — keep the one in use, fold the missing steps, delete
the loser.
The guard now asserts the property per kind rather than spot-checking
two: no non-snippet suggestion may propose merge_snippets, so a sixth
kind inherits the bar without anyone editing the test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Arising from #3731, which shipped a lesson with three of five tools and logged
the rest rather than widening its own scope. The operator's framing on reading
that: each kind deserves CRUD functions and to show up in the search and report
functions. So this fixes the property, not the two instances.
WHAT WAS MISSING FOR A LESSON: no delete, no list, and no duplicate report.
`delete_lesson` is the #2250 situation exactly — the trash is kind-agnostic so
`delete_note` always reached a lesson, but nothing said so, and a kind whose own
tools offer create/read/update reads as one you cannot retire. `list_lessons` is
the only way to ask what has been learned at all: `get_lesson` needs an id you
already have, and semantic search returns what resembles a query, never the set.
APPLYING THE RULE FOUND THE SAME REPORT GAP FOR PROCESSES, which have had full
CRUD for months and have never been in `_REPORT_KINDS` either. Both are in now,
each compared only against its own kind.
The lesson report default is 0.90 — the general semantic floor, deliberately
BELOW its own write-path bar of 0.96. The gate is permissive on purpose so it
does not refuse two genuinely different lessons whose triggers read alike, and
that tolerance is precisely what wants reviewing later, so the report looks at
the band the gate was told to let through. Safe there and not at the gate,
because a report proposes and the operator picks where the gate blocks a write.
A BUG CAUGHT BEFORE IT SHIPPED: `list_lessons` first read the trigger from
`it["data"]`, which `_note_to_item` does not carry — it projects named keys off
the mirror (`language`, `verification`) rather than the column. Every row would
have listed an empty trigger, which on a kind whose whole point is the trigger
is the failure looking like the feature. `when_to_apply` is now projected there
beside the others, so every listing surface gets it, including step 7's UI.
The guard asserts the PROPERTY rather than the instances: for each typed kind,
all five tools exist, are actually offered by register(), are classified for
auth, and the kind has a duplicate report. Derived from the kinds themselves, so
a fourth inherits the bar. A per-tool test cannot catch a missing tool, which is
why four steps of milestone 385 went green over this.
`find_duplicate_records` now validates against `_REPORT_KINDS` instead of its
own literal — the second copy is what would have refused a kind the service
already supported.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 385 step 4 — the write path.
ITS OWN TOOL MODULE, not create_note(note_type="lesson"), on the snippet and
process precedent and for the reason that precedent exists: a kind whose value
depends on one field being filled needs a door that ASKS for that field by
name. create_note would take a lesson through a generic body parameter and the
trigger — the whole of why a lesson is findable — would be something the writer
had to know to include.
THE TRIGGER IS REQUIRED, refused rather than flagged. Step 1 left the choice
open. Refusing is right for the same reason create_rule makes enforcement the
deciding question: a lesson with no trigger is not a weaker lesson, it is a
note that will never surface, and nothing downstream can tell the difference —
it saves, reads correctly in every listing, and is silently absent from the one
moment it was written for. A flag is a warning nobody is present to read; the
write path is where the writer still is. The message says SYMPTOM, because
"required" alone produces a topic where a situation was wanted.
The docstring carries the distinction this milestone exists to fix, in a line a
reader can apply: the difference between a lesson and a rule is FORCE, not
importance. If ignoring it would be a mistake it is a rule and needs the
operator's yes; if ignoring it just means someone re-derives it the slow way it
is a lesson, and nobody is bound.
CARDINALITY: a LIST, in notes.data under `taught_by`. The founding example
generalised three incidents into one claim about failure classes no CI lane can
see — generalising across incidents is the shape a good lesson HAS, and
arose_from_id holds one, so a single id keeps the first and drops two while
reading as complete. It lives in `data` rather than a join table for the reason
decision #4157 put the trigger there: a table would settle, for every note kind
at once, whether provenance is multi-valued — a question nothing has measured.
`arose_from_id` is filled only when there is exactly ONE source, because every
surface that renders it renders it as THE origin, and one of three would make
those surfaces state something false.
THE DUPLICATE GATE, which step 4 asked to check: a lesson is judged at a bar
ABOVE the sibling band, not the general 0.90. #2518 measured deliberately
parallel variants at 0.92 on a document that is mostly prose about the thing,
which is exactly a lesson's shape now — so at 0.90 two genuinely different
lessons about one area ("CI cannot see this class of failure") would refuse each
other. Its own constant rather than reusing the snippet's: the two are separate
facts that coincide today, and this number is inherited from a structurally
analogous corpus rather than measured on lessons, of which there are none yet.
Follows canon #2846 including the third registration point it names and this
change would otherwise have missed: get_lesson is in server._READ_ONLY_TOOLS
and the two writers in _WRITE_TOOLS, which test_mcp_auth requires.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Plugin reply shape, preferences UI, cited-record status, and the lesson kind
Merged to ship the plugin: the marketplace checkout tracks main, so
#4153's reply-shape rewrite and #4154's "a record you only mention is a
record to read" could not reach a session from dev. Manifest moves
2026.09.18.0436 -> 2026.09.18.1606, so the executing cache refreshes.
Riding along: #4154 (next_step on milestone summaries, status in the
injected kind marker), #3895 (preferences writable + drift visible), and
milestone 385 steps 2-3 (#3729, #3730) which are inert until a lesson can
be written in step 4.
CI 7023 green on 1361ed7, all six jobs.
Milestone 385 step 3 — the step where the kind either works or is cosmetic.
THE DOCUMENT, and why there is no `lesson_document()` beside `rule_document()`
in embeddings. The step expected one. The difference is where the sharp shape
LIVES. A rule keeps its trigger in a column and its title is a plain name, so
`{title} — {trigger}` has to be synthesised at embed time and exists nowhere
else. A snippet — the only sharp record in the corpus by #2485's measurement,
0.153 top-to-second against 0.010–0.023 — gets there the other way: its STORED
title is already the join and its stored body already opens with the trigger,
so the ordinary `title\nbody` join IS the sharp document. Step 1 chose the
snippet route and step 2 built it, so `lessons.lesson_document` composes what
is STORED and the generic chunker does the rest.
The consequence the step asked about: `chunk_document` is untouched, so
CHUNKER_VERSION does not move and NOTHING re-embeds. The step's "Re-embed"
section describes a change this design does not make.
THE NARRATIVE stays in the body, departing from the step's instruction to keep
it out. `rule_document` excludes `why` because long dated narrative made
sixteen dev-logs land on the centroid of "development" — but that finding
predates chunking (#280). A body over budget is now split, and every chunk is
prefixed with the title, which for a lesson carries the trigger. The story
occupies its own vectors instead of averaging itself into the trigger's, and
each of those is still anchored to when the lesson applies. A guard asserts
exactly that. Holding the story out would cost the reader the only part that
explains the insight, to buy a sharpness the chunker already provides.
GLOBAL IN THE SEARCH is the real new code: `GLOBAL_NOTE_TYPES` and
`include_global_kinds` on `semantic_search_notes`, widening the PROJECT filter
alone. Off by default, because two callers depend on that filter holding — the
near-duplicate gate compares a record only against its own project on purpose,
and a globally visible kind there would let a lesson block an unrelated note's
create on a project its author never touched. It composes with `note_type`
rather than overriding it, so narrowing to snippets does not quietly acquire
lessons, and it changes nothing about the ACL: `notes_visibility_clause` still
gates every row.
Wired into the explicit MCP search only — the operator asked, and there is no
budget to spend. The unasked-for injection arms are step 5's subject (#3732)
and the legibility of a lesson appearing on a foreign project is step 7's
(#3734), so neither is turned on here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7016. `test_non_task_facets_are_the_note_types_and_only_those` pins the
non-task vocabulary as a literal set, so adding `lesson` to `_FACETS` turned
it red — the guard working, not breaking. It stays a literal: derived from
_FACETS it would assert nothing, and rule 167 wants a guard that can fail.
The representative corpus had no lesson row, so `lesson` was reaching only
the two tests that iterate FACET_TYPES and never the one that asserts each
facet selects EXACTLY its own rows. It has one now, which is what pins the
half that matters: a lesson is not picked up by the Notes facet despite
both being non-task records.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 385 step 2, implementing decision #4157 from the step-1 spike.
The kind: `note_type='lesson'`, a note findable by WHEN IT APPLIES rather
than by what it is about. The trigger lives in `notes.data.when_to_apply`,
mirrored into the title and the head of the body — the shape snippets
already use, and the reason nothing re-embeds: chunk_document is untouched,
so CHUNKER_VERSION does not move.
NO MIGRATION, and the step assumed there would be one. `note_type` carries
no CHECK — only `task_kind` does (0056, 0065). Migration 0036 added it as
plain Text with a server default and nothing has gated it since, so rule 36
has no whitelist to expand and #3128's failure mode (a value the database
refuses) cannot arise for this column. The vocabulary that actually decides
what a reader can reach is services.knowledge._FACETS, which since #3161 is
one table feeding the door's validation, the counts and both dialects of
the type filter — so the kind lands there in a single edit.
ONE JOIN, not a fourth copy. `{subject} — {trigger}` had three
implementations: rule_document, snippets.compose_title, and this step
needed another. #3207 records what that costs, so the join moves to
embeddings.trigger_title beside embedding_text and all three delegate.
Behaviour is unchanged for rules and snippets; the guard calls each through
its own public name, so a re-implementation fails it.
The #3163 bill is stated in the service docstring rather than left to be
inferred: versions, supersession, trash, the share ACL, tags, project and
System tagging, chunked embeddings and the duplicate gate are all
inherited; status/task_kind/milestone_id and recurrence are not, and
verify_with/expires_when are available but outside the kind's contract.
The status cell is the one that matters — `is_task` IS `status is not
None`, so a lesson that acquired one would become a task.
The integration guard asserts the WRITE rather than the constraint: it
holds whether or not note_type is ever gated, and goes red only if it is
gated without this value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 399 step 5. Steps 1-4 put preferences into the backend: a kind
column, an inverted write path, a third register in the injected block, a
delivery slot. Nothing the operator could touch. Rule 27 forbids leaving it
there, and here it matters more than usual, because the UI is the only
guard against the risk the milestone named up front — an agent misreads one
session, rewrites a preference, and follows the rewritten version forever
while the operator never sees the moment it changed.
Four things ship.
A preference is DISTINGUISHABLE. `kind` reaches the client (the server has
always sent it in rule_brief) and a preference carries a chip. Force is the
one thing a list of instructions must not leave the reader to infer, and a
row that renders identically to a rule teaches the opposite of both facts
about a preference: it does not bind, and a session may rewrite it.
A preference is WRITABLE. The editor gains the kind as a first-class choice
with the test beside it — what happens when someone does not do this — and
says plainly, when preference is chosen, that sessions rewrite these
without asking and every rewrite is kept.
DRIFT ARRIVES. `GET /api/rules/drift` returns one row per rewritten
preference carrying its latest rewrite: what it said, what it says now, and
the record named by `arose_from_id` that taught the change. Both texts ride
along so the list shows the diff without a call per row. The new pane sits
beside the staleness sweep, because drift belongs to no one rulebook, and
it answers a question the operator would not have thought to ask.
REVERSION IS ONE ACTION, and this is the carve-out worth arguing with.
Milestone 323 refused a one-click restore for rules — "a binding
instruction should not be revertible in one click", because a silent revert
erases the only record of why the rewrite happened. That reasoning turns on
the rewrite being the operator's own decision. A preference's is not: the
agent makes it mid-work without asking, so reverting is a veto over someone
else's edit rather than an undo of your own, and a veto costing more than a
shrug is not supervision. The route refuses anything but a preference (409),
and nothing is erased: the restore goes through update_rule, so it snapshots
too and the history GAINS the revert. An integration test pins that, because
it is the whole basis for the exception.
Tested against real Postgres — every claim is about which rows come back
and in what order, which a stand-in session cannot judge.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The new fixtures fed (milestone_id, status, count) and the query also
carries max(updated_at), which last_touched_at is computed from. Six
tests in the new module died unpacking it; the product path was never
reached, and the rest of the suite was green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 1 made placement cheap for a task whose status CHANGES: create_task
and update_task return where it sits, and the report is written from
that. It did nothing for a task a reply merely cites.
This milestone's own step-6 review reported "#4014 is the open step of
milestone 409". #4014 had been done for four days; the open step was
#4015. The id did not come from a read — it came from a retrieval hint,
which carries an id, a kind and a title and says nothing about status,
while list_milestones said "8 of 9" and would not say which one. The
gap was there to be filled and the nearest-looking id filled it.
Two surfaces, one principle: the status arrives with the id.
1. get_project_milestone_summaries gains next_step — the earliest open
step, {id, title, status} or None — carried through _BRIEF_FIELDS to
enter_project, get_project and list_milestones. One extra flat query
for the whole batch, so #2384's fan-out does not come back.
OPEN_STEP_STATUSES moves to services/milestones.py and placement.py
imports it; both surfaces now answer "what is next" and must not
drift on what counts as open. Both step queries take the same
readable_notes_clause (rule 78), so a row cannot name a step its own
progress numbers exclude.
2. _record_kind renders a task's status: [task (done)], [issue (todo)].
A finished step and an open one read identically before, which is
exactly the line the misreport was taken from. Only tasks — is_task
IS status-is-not-None on the model, so there is no fallback branch.
reporting-back gains the practice, owned and registered in the guidance
ownership table: a record you only mention is a record to read.
The guards are structural and each fails on the regression it names:
the query count is asserted rather than the payload shape, and the two
surfaces' agreement is pinned on the rendered ORDER BY, since a mocked
session hands back whatever order the test chose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 409 step 6 measured the scaffold on live sessions and found its
two halves disagreeing: adherence passed and the read test failed.
Completion replies carried every section the table asks for and were
still hard to read.
The cause was in the skill, not in compliance with it. It said to pick a
kind of reply "then fill its sections... keep them even when one is
short", which is an instruction to complete a form, and nothing anywhere
set a ceiling. A faithful reply and an unreadable one were the same
reply.
Four changes to the discipline around the scaffold. The categories and
their sections are untouched.
- Sections are what to consider including, not a form to complete. A
section answering a standing question ("does anything need me?") is
always answered, even with "nothing"; a section that explains earns its
place only when it changes what the operator does. Otherwise it belongs
in the record's log, where it is available and not in the way.
- Write the shortest reply that carries the answer, with named exceptions
so this cannot be read as "always be terse".
- "Needs you" takes BOTH tests: theirs to decide, AND work is waiting on
it. A question answerable by reading something or taking an available
measurement is work not yet done, not a request — settle it, say which
way you went, and leave them free to overrule.
- A decision already made gets acted on. Re-arguing a settled question
reads as contradicting yourself rather than as being careful, and costs
the operator the decision twice.
"Before sending" gains a second pass for what can go, since the existing
check asks what is MISSING, which a bloated reply passes.
Guards in tests/test_reply_discipline.py, three topics registered for
ownership. Every guard was falsified against the pre-change text before
committing (rule 167): all five fail on it and pass on the fix, and the
sixth deliberately passes both since it guards the scaffold against
collateral damage. No absence checks — the skill legitimately discusses
filling in order to warn against it, so asserting "fill" is absent would
false-alarm on the corrected text (snippet #3352).
Instance-agnostic per rule 115: the added text carries no record ids, no
software-specific terms and no verbatim quotes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The tuning loop shipped in steps 4 and 6 and logged zero events in its
lifetime. `tune_retrieval`, `retrieval_telemetry` and `retrieval_surfaces`
appeared on no instruction surface at all — not the skills, not the hooks,
not the MCP instructions — so the decision that "the model should be the
thing handling it 9 times out of 10" could not begin to happen.
What was missing was not an auditor but a route. `using-scribe` now carries
it, ordered: read the refused records, fix the trigger, and only then
consider the dial. The order is the content. A rule's `when_to_apply` IS
the text its score is computed against, so a miss is evidence about that
text first; rewording one trigger changes one rule's reach, while moving a
floor changes what every record on the surface does and cannot tell a
badly-worded trigger from a genuinely distant one.
Measured, and the reason the order is asserted rather than suggested: rule
1 scored 0.6515 and ranked 5th for the moment it governed, behind three
rules that restrained the same act. Every percentile said "lower the
floor"; at 0.60 the arm delivered those three restraints and still not rule
1. Rewriting the trigger to lead with the symptom put it 1st at 0.7130.
Also: the create path gets a precondition. A new record is itself a
retrieval-affecting act, so before writing one, what_might_apply asks what
already covers that moment — fifty candidates and no bar, because a bar is
what lets the existing record hide.
`_INSTRUCTIONS` gets one index line, not the route: 1,986 of 2,000
characters, since Claude Code cuts the rest mid-word (#2562).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`current_settings` read a surface's history with one query and `limit(2)`, then
keyed the rows by dial. That is only the same thing while both dials have moved
equally often — and they do not. Floors get walked; budgets rarely move. Three
floor changes and one budget change returns two floor rows, and the budget
change vanishes.
Under #4102 that cost a missing reason. Since #4104 it is worse: the dial then
reports `source: "shipped"` — still on the value Scribe shipped — for a number
somebody deliberately tuned. A wrong calibration answer, in the direction a
reader has no cause to double-check.
One query per dial, `limit(1)` each.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Two failures in the tests added with the step, both mine.
`make_mock_session` is an AsyncMock and every child of an AsyncMock is one too,
so leaving `.all` as it came handed `migrate_floor` a coroutine where it reads a
list — the same trap the helper's own docstring already flags for `add`.
And the registration test enumerated three tools by name, which is exactly what
it is for: `migrate_retrieval_floor` made it four.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 416 step 6. A retrieval floor is a cosine similarity, which only
means something inside one embedding model's geometry over documents cut one
particular way. Change either and every floor on the install keeps applying
while describing nothing — and nothing anywhere says so, because the scores
simply come out different and the bar goes on cutting.
`CHUNKER_VERSION` already solved this for documents: stamped per row, so the
backfill re-embeds precisely what is stale. The same idea, applied to the
numbers:
- `calibration_stamp()` — embedding model + document shape, one definition.
TWO fields, never a fused string (rule 149): a mismatch has to say WHICH half
moved, because they call for different responses.
- `retrieval_tuning_events` gains `embedding_model` / `shape_version`
(migration 0104), stamped on every write. Nullable and NOT backfilled —
"unstamped" is the honest answer for a row written before this existed, and
it reports as `stale: null`, never as fine.
- `current_settings` reports calibration per dial: tuned rows from their event,
untouched dials from the registry default's own stamp.
- `retrieval_surfaces` and the Settings panel show the mismatch. The panel
renders ONLY when something is stale, so seeing it at all is the signal.
- `migrate_floor` / `migrate_retrieval_floor` answers "a path for thresholds to
be inherited by the next model so that they don't have to recalibrate a lot":
the raw cosine cannot cross models, but the PERCENTILE it represented can.
Measure what fraction of a surface's logged calls the old floor admitted,
re-score those queries under the current model, take the value admitting the
same fraction. Dry run by default; applying writes an ordinary tuning event
with the arithmetic in its reason.
Nothing auto-retunes. A stale stamp says a number is no longer a measurement;
it does not say what the number should be, and #4102 measured the one case
where the statistic and the correct action pointed opposite ways.
The load-bearing test is an ABSENCE: no chat-model identifier may appear
anywhere in the calibration path. Claude produces none of these scores, so a
Claude upgrade must trigger nothing — a false alarm here teaches the operator
to ignore the real one on the day bge-small becomes bge-base.
Backup v17 carries both columns, unfilled on the way out and on the way back:
a round trip must not turn "we don't know" into a stated fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 6973 — `test_guidance_ownership` on "rules are retrieved; ask before
a consequential act": the using-scribe skill was missing the marker
`content_type="rule"`.
It was there. I had hard-wrapped the new bullet across `content_type=`
/ `"rule"`, and that guard matches its MARKERS raw while normalising
whitespace only for the longer `statement` — which is why the failure
reported the statement present and the marker absent in the same
breath. Rewrapped so the token sits on one line.
Worth noting rather than just fixing: the markers are deliberately
raw-matched, since a marker IS a literal a reader copies. `tool_doc`
flattens for exactly the opposite reason one layer over. Both are
right; the trap is that they differ.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 416 step 5. The operator's compromise — "if you're worried
about excluding potentially important data let limit it to 50 entries"
— moved to the surface where it is safe. Fifty in the push would be
milestone 394 with extra steps; fifty in a pull crowds nothing out.
`what_might_apply(query)` returns ranked rule candidates with NO
threshold. The moment it serves is the one where the caller does not
trust a bar to decide for them, so it does not have one — every row
carries its score and the reader judges.
WHY IT IS NOT A BIGGER `limit` ON `search`
`_search_rules` returns statement, why and how_to_apply in full, on the
stated reasoning that a caller who went looking deserves the whole
record. That is the DEEP pull and should stay that way. This is the
SHALLOW one — many candidates, each just enough to decide whether to
open it. Opposite trade-offs, so it is a second tool.
THE PREMISE THE STEP GOT WRONG
The task said fifty "costs nothing". `_rule_hint_line` had already
measured otherwise: ~143 tokens per line once the trigger is rendered,
and #3855 tripled trigger lengths across the corpus. Fifty is ~7,000
tokens — cheap next to an arm firing before every Bash call, but not
free, and a tool promising a free wide net gets reached for casually
and then regretted.
So it reuses the graduated shape #3851 measured for the push: the top
few carry their trigger whole, the rest carry a cut of it. TRUNCATED,
never dropped — the trigger is what lets a reader judge without
opening, and a teaser without one is just an id. The cut borrows
`_goal_line`'s technique including the fallback that matters (#4036):
`textwrap.shorten` returns a bare "…" for one unbroken word.
TELEMETRY
Logged under its own pull source, asserted absent from both the tunable
push registry and AMBIENT_SOURCES. That guard is load-bearing right
now: the push arms' near-miss distributions are the evidence #4121
argues from, and a pull folded into them would move those numbers.
INSTRUCTION SURFACES
The using-scribe reflex and the MCP instructions both pointed at
`search(content_type="rule")` for the consequential moment — the deep
tool, at the moment you want breadth. They now point here, and keep
`search` for reading a rule you already suspect. Written as a practice
rather than a prohibition (rule 165).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Both said the repeated read-and-clamp was "canon #2860 across 295 of 372
siblings". Wrong on both halves, and checked rather than assumed:
#2860 is "Scribe service function — the async_session unit", the
service-layer canon, and its instance count is that canon's dominance in
src/scribe/services — nothing to do with a threshold helper. The
divergence hook flagged the new module against it because the new module
is a service, which is the hook working correctly and me misreading it.
The duplication itself was real and is what the registry consolidated:
`plugin_context` read and clamped the pair twice over, three rule arms
did it again, and `reply_preferences._threshold` once more. The comments
now say that, and claim nothing about a ledger count.
`rule_usage.py:122` cites #2860 correctly as the service canon and is
untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The last item in this step's done-when: no wording anywhere asks an
operator to tune for correctness.
Four Settings hints told them to do exactly that — "raise it if rules
keep arriving unread", "lower it if a git push arrives with nothing",
"lower this if genuine duplicates go unnoticed". Every one of those
asks the operator to diagnose a ranker from symptoms, which is the job
the model now does from the records: the telemetry says what each bar
refused, and reading those records is what separates a real miss from
a bar doing its job. The hints keep the explanation of WHAT each number
is — that is worth reading — and drop the homework.
`plugin_context.py` said the defaults "are meant to be tuned from
retrieval_logs once data accrues", which was true and had no owner.
It now names who does it and with what.
`retrieval_telemetry`'s docstring gained the warning that belongs
beside it: this readout has been measured pointing the wrong way, so
the ids it returns are the point, not its percentiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The other half of the bargain in milestone 416 step 4. The model moves
these dials; this is what makes that reviewable rather than merely
automatic.
THE HOLE THIS CLOSES
Every retrieval floor is an ordinary settings key, and `/api/settings`
accepts any key at all. A floor written through it landed correctly and
recorded nothing — a tuning history with holes in it, which is worse
than no history because it reads as complete.
So the generic endpoint now routes registry-owned keys through
`set_dial` instead of writing them as plain rows. ROUTED, not refused:
refusing would only work for callers that had been updated, while this
way the form, a script, and an old client all leave the trail, and
there is no version of "forgot to use the other endpoint". Clearing a
control is written as an explicit set back to the shipped default,
because the operator reverting something is the single most important
move this history can record.
`set_dial` now also refuses to record a no-op. The Settings form
re-sends every field on every save, so without that one press of Save
would write six rows saying the operator set six dials to the numbers
they were already on — and a history nobody can skim is one nobody
reads.
WHAT THE OPERATOR GETS
`/api/retrieval/surfaces`, `/surfaces/<name>` and `/tuning-history`,
with `actor` fixed server-side rather than taken from the payload: a
payload-supplied actor would let a model claim to be the operator, and
"did I do this, or did the session?" is the first question this list
is asked.
In Settings: the five missing BUDGETS (until now only auto-inject had
one, so the only control over a noisy surface was to raise its bar —
which discards that surface's best candidates along with its worst),
and a "What has been tuned" panel showing each change, who made it, and
the reason given. The operator's own changes are marked.
The MCP tool demands a reason; these endpoints do not. That asymmetry
is deliberate and stated in routes/retrieval.py: the requirement exists
to make the MODEL read the records before moving a number on someone
else's behalf, and the operator is that someone — a mandatory
justification box on every control would be friction charged to the one
participant who owes no explanation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 6962 — `test_every_registered_tool_is_classified_exactly_once`. The
backup gap is closed (1692 passed); this is the next guard, and the same
shape of one: a tool registered without a classification is silently
denied to a read key, with nothing to notice (#3191).
`retrieval_surfaces` and `retrieval_tuning_history` read. Read access
matters more than usual for these two — a session that cannot see the
bar in force, or the reason it was last moved, is one that will move it
again blind.
`tune_retrieval` writes in both senses: the number the arm reads, and
the reason appended to the audit trail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 416 step 4's write half. `retrieval_surfaces.py` made the six
push arms describe their `{floor, budget}` the same way; this adds the
three MCP tools that let the model READ that and change it, and the
backup sections that carry the reasons.
The operator's decision, which this implements:
"the floor should be chosen and adjusted by the model using it… the
user should be able to touch it but the model should be the thing
handling it 9 times out of 10."
WHY A REASON IS REQUIRED, AND WHY THE TOOL ARGUES AGAINST PERCENTILES
The milestone originally listed self-tuning as a non-goal on one
measured case, and that case is now the tool's docstring rather than a
prohibition: `report_preference` logged 69 consecutive declines with
the refused record 0.0006 under the bar, and every percentile said
"lower it". The refused record was rule 77 "Extract intent from loose
phrasing" matched against a query about report layout — a false
positive. Lowering would have attached that rule to every completion
report ever written.
What separated the statistic from the correct action was OPENING the
record. So `tune_retrieval` refuses a blank or perfunctory reason,
tells the caller to read `retrieval_telemetry(near_miss_samples=5)`
and the record ids it names, and carries that 69-decline example — an
abstract warning loses to a number. The non-goal that survives is
*statistical* auto-tuning; nothing here reads a percentile and picks a
value.
BACKUP (v16), which is what CI caught
`retrieval_tuning_events` was neither backed up nor excluded, and
#2293's guard said so. It is backed up: `settings` already carried the
numbers, so dropping this would restore an install with six moved
dials and no argument for any of them — precisely the state the table
exists to prevent, and worse now that the model is the one moving
them. One `_retrieval_tuning_event_rows` builder called from both
exporters (snippet #2851); `user_id` travels because a restore has to
remap it, which is why the row builder is not the model's `to_dict()`.
`surface` is a registry name rather than a foreign key, so the history
survives a restore into an install whose ids all differ.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI on 09b4845 caught both, and the second is an improvement rather than a
repair.
`test_services_reply_preferences` patched `reply_preferences.get_setting`,
which the registry refactor removed — its floor and budget are resolved through
`retrieval_surfaces` now. Its key-independence test also asserted the arm asks
for exactly one key; it asks for two, because both of its numbers are its own
since this step, so the assertion names both and adds a check that the registry
and the module constant still agree about the floor key. They writing different
keys is the failure where the Settings form saves one string and the arm reads
another.
`test_settings_defaults_agree` parsed `plugin_context.py` for a bare
module-level float, and those constants now alias the registry. Rather than
teach the regex about aliases, the six retrieval floors are keyed on their
SURFACE NAME and read from the registry directly — which is strictly better for
this guard: a surface name is also its telemetry source, so a row names the same
arm the readout does, and a floor cannot be checked against a stale constant
that happened to keep its old value. `PLAN_MATCH_DEFAULT_THRESHOLD` is not a
push surface and keeps the older shape, with a note saying why.
Added while there: `test_every_tunable_surface_has_a_control`, derived from the
registry, so a seventh surface arrives as a failing test rather than as a number
only the model can reach (rules 25, 27).
Also lands the audit trail this step needs — `retrieval_tuning_events` (model +
migration 0103) and `services/retrieval_tuning.py`. The tool layer on top is
the next commit; the table is here because `reason` being REQUIRED is the whole
guardrail, and the schema is where that starts. A number moved silently leaves
nothing for the operator to review or disagree with, and the operator's decision
is that the model moves these "9 times out of 10".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Groundwork for the step's real change. The operator's decision is that the
floor is chosen and adjusted by the model using it, not shipped as a value
somebody has to defend:
"we need a model consistent surface for the adjustment of these floor values.
the user should be able to touch it but the model should be the thing
handling it 9 times out of 10."
A tuning surface cannot be consistent across six arms that each spell their
configuration differently, so the arms stop owning their numbers.
`retrieval_surfaces.SURFACES` names each one, its floor key and default, its
budget key and default, and — because they are rendered by the tuning tool and
the Settings UI — what it asks, over what corpus, and how often it fires. A
floor cannot be moved responsibly by anyone who does not know those three.
Three things fall out:
- **`k` becomes a real budget everywhere.** Only auto-inject had a configurable
one; `RULEHINT_LIMIT`, `PROMPTRULE_LIMIT` and `reply_preferences.LIMIT` were
constants. `k` is what binds under a low floor, so it has to be settable per
surface — and per surface is the point, since `pre_tool_rule` fires before
every Bash call while `prompt_rule` fires once a turn.
- **`write_path` gets its own budget, inherited not reset.** It shared
auto-inject's outright on the argument that "how many titles at once" means
the same thing on both. It does not, for the same reason. Unset, it still
reads auto-inject's key, so an install that tuned the shared knob does not
silently drop to a new default.
- **The duplicated read-and-clamp goes.** That shape is canon #2860 across 295
of 372 judged siblings. Survivable while the numbers were constants; not once
they are meant to move.
The long measurement comments stay exactly where they are — #2223's noise-floor
probe, #3853's command-vs-code split, #3851's band measurement. The constants
they annotate now alias the registry, so there is one value and the reasoning
still sits beside it.
Tests build the write-path config from the registry (`helpers.writepath_cfg`)
instead of from hand-written dicts. That is not tidiness: the rule arms read
their numbers inside a fail-open `except`, so a dict missing one key does not
raise where a reader would see it — the arm silently becomes a no-op that reads
exactly like "fired and found nothing". Ten hand-written dicts each looked
complete on the day they were typed.
tests/test_retrieval_surfaces.py pins the identity everything rests on: a
surface's name IS its telemetry source. Nothing in the type system says so —
`record_retrieval(source="pre_tool_rule")` is a literal in another file — and
renaming one without the other yields an arm that can be tuned and not
measured, or measured and not tuned, with no symptom either way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Both were mine, and both show the change behaving as designed.
`test_session_dedup_marks_the_reuse_arms_rather_than_silencing_them` asserted
#12 stays out of the search's `exclude_ids`. It does not, and should not: the
place arm has just listed it, so the semantic arm must not list it again. That
is the same-call rule, not the ledger. The claim I meant — the LEDGER never
reaches the query — is asserted on its own against a record the call has not
otherwise rendered.
`test_a_pulled_snippet_already_seen_is_evidence_not_menu` expected #7 and #8.
It gets #7 alone, because #7 (0.91) now anchors the margin band at 0.81 and #8
scores 0.80. Withholding #7 used to promote a materially weaker hit into a slot
it had not earned, with nothing in the output saying so — the band measures
distance from the best answer, and letting the ledger decide which answer that
is was the axis confusion the rule band was written to avoid.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI on 5c64ea0 surfaced a half-measure I had shipped and three tests pinning the
semantics it replaces.
**The half-measure.** Only the semantic arm was marking repeats; the write
path's PLACE arm still dropped a nearby snippet that was on the ledger. One
menu, two rules, decided by which arm happened to find a record — and the
marker would then read as a complete account of what the session has met while
covering half the lines. `seen` now means this call's own menu and nothing
else, so both reuse arms mark and neither withholds.
**test_session_dedup_excludes_ids_from_the_reuse_arms** asserted the drop and
the ledger going into the query. Both are the defect; it is renamed and asserts
the marker instead.
**test_a_pulled_snippet_already_seen_is_evidence_not_menu** asserted #7 was
evidence and NOT menu, with the limit widened by one to pay for the drop. The
premise was the hard exclusion; #7 is now both, and the widening is no longer
needed for it.
**test_a_record_this_arm_withheld_itself_is_not_a_near_miss** is the
interesting one. Its property still holds and its setup no longer reaches it:
a ledger repeat comes back from the search and is rendered, so nothing is
withheld and `best_available` describes the bar. The one post-search filter
left is a PULLED record this same call already listed by place — kept in the
query because `resembles` needs its score, dropped from the menu because it is
already on it — so the test is rebuilt on that, and a sibling test pins the
direction that changed, so nulling the score whenever the ledger matches
cannot pass both.
Also stubs `superseded_ids` and `owner_names_for` in the new test file, which
were reaching for Postgres in a unit job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI caught two things in c61f730, and the second is the one that mattered.
1. `scribe_autoinject.sh` keeps its note ledger in `${TMPDIR}/scribe-autoinject`,
not `scribe-priorart`. Every ledger NAMED in the hooks was in the one
directory the sweep visited, so it read as complete — and the arm that fires
most (598 calls in five days) was the only one still carrying the bug. The
clear ran, found nothing to remove, and exited 0. `SCRIBE_LEDGER_DIRS` in
`scribe_defs.sh` is now the roster, and the guard reads that string rather
than a copy of it, so a test can no longer agree with itself forever.
2. The convention guard over-reached: it flagged `<sid>.snap` and
`<sid>.blocked`, which live in `scribe-afterwrite` and `scribe-reportcheck`
and can never be touched by the sweep. It is now two tests keyed on the
directory, because the two assumptions fail differently — one lets a ledger
sit where nothing sweeps, the other lets it sit in the right place under a
name the sweep does not match.
`test_only_the_rule_ledger_is_cleared_and_the_note_ledgers_are_left` also went
red, correctly: the note arms' exemption was a real decision, recorded in a
test, and my commit message said nothing about it had been decided. That was
wrong and the test was right to stop me. The decision is reversed rather than
ignored, and the reasoning is that #4101 removed its premise: it rested on a
note repeat being WITHHELD, so clearing the ledger meant re-injecting whole
menu lines the session already had. Now a repeat is rendered with a `seen`
marker, so the ledger only decides whether that marker is true — and across a
compaction it is false, telling a freshly-summarised session it has already
seen a record that is nowhere in its context. The test keeps its name and
records the reversal with the reason, rather than being deleted.
Verified by running the hook directly: all six ledgers across both directories
gone on `compact`, `<sid>.unreached` left standing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#3750 settled this for rules: a record the session was told about an hour ago
is not a record in front of the reader now, so the second time it is the best
answer it is rendered again with a tail saying so. The note and snippet arms
never got that fix, and theirs was worse — the ledger went into
`semantic_search_notes` as `exclude_ids`, so the repeat left the candidate set
entirely. Three things followed:
- the second time a note was the best answer the session got SILENCE,
indistinguishable from "nothing matched", on the arms that fire most
(`auto_inject` alone ran 598 calls in five days);
- a compaction made that permanent, since the ledger outlived the context it
described — fixed one layer down in c61f730;
- and `best_available` was measured against a candidate set the caller had
already edited, so the bar could be blamed for a record the caller
withheld (#3739, from the side its fix never reached).
The ledger is now a RENDERING fact. Every repeat is still ranked, still shown,
and carries a `seen` marker; the band is computed over all hits, because
letting the ledger move the cutoff would make "you were shown this" change what
counts as relevant. The marker is one word and deliberately not the rule arms'
phrasing — "before deciding it does not apply" is the voice of a record that
binds, and a dev-log borrowing it would claim authority it does not have.
Telemetry takes the rule arms' contract (#3752): `results` and `record_surfaced`
both take fresh only, the repeat is counted in `suppressed`, so this source's
surfaced set still matches its own log row (#3668). That makes a fact readable
that could not be stated here before — `result_count == 0` with
`suppressed_count > 0` is "everything that matched, the session has already
seen", which is a different claim about the bar from "nothing cleared it".
On the write path this also splits a variable that carried two claims. `seen`
was the ledger plus everything the call had already rendered, and both were
treated as reasons to withhold; `in_menu` keeps the same-call exclusion while
the ledger becomes a marker. That narrows the `best_available` compromise at
its old comment to the pulled-and-already-listed case, and retires the argument
that a suppression count here would be partial — nothing is hidden inside the
query any more.
Deliberately unchanged: the write-path SYNC class still shows once. Its claim
is about an edit in progress rather than a record's continuing relevance, and
repeating it every write to the same file would be nagging.
tests/test_ledger_references_not_silence.py pins both arms — the ledger never
reaching the search, the repeat rendered and distinguishable, the telemetry
split, the all-repeats call being readable, and the two exceptions (this call's
own menu, and the sync class).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`scribe_session_context.sh` cleared `.rules.ids` and `.opened.ids` by name and
left `.ids`, `.sync.ids` and `.derive.ids` standing, under a comment asserting
that was a decision. Reading the note arms says it was not: their exclusions go
straight into `semantic_search_notes`, so a surfaced note leaves the result set
rather than being rendered as a reference the way #3750 gave a repeated rule,
and unlike the rules ledger they never age. Hard, permanent, never cleared — a
note surfaced in a session's first minute is unreachable for the rest of it,
which is milestone 386's own defect alive on the arms that fire most often.
The list was the bug, so the fix is not a longer list. `scribe_clear_session_
ledgers` matches the naming convention instead — a per-session ledger is
`<sid>[.<kind>].ids` — which covers all five and covers the sixth on the day it
is written. `<sid>.unreached` is deliberately outside it: that records an
outage, not held context, and #2932 needs it to survive.
tests/test_session_ledger_clear.py runs the hook rather than grepping it for
`rm -f`, since grepping for the names is the pattern being removed. It pins
both directions — `compact`/`clear` take all five, `startup`/`resume` take
none — plus the convention the glob rests on, checked against the hooks
themselves so a ledger named outside it fails loudly instead of silently never
clearing.
Also drops a stale comment pointing at a rules-etag marker that milestone 394
retired.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 6945: 11 failures down to 1. The survivor is
test_shortening_a_line_does_not_decide_what_it_says_about_holding, which
asserted "no longer hold it" appears in the compact line for seen=True — the
phrase that now belongs to the OPENED state, not the named one.
Its subject is a property, not a string: `compact` and the ledger are
independent axes, and shortening a line must not change what it claims about
holding. So the fix follows the axis rather than swapping the phrase. The axis
grew from two states to three, and the test now checks all three are distinct
under compact — pinning only two would let the compact branch collapse the new
middle state into either neighbour, which is the same regression it was
written for with one more place to hide.
Added the assertion that matters most when room is short: a line the session
never opened must not imply it did.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 6943: 11 failures, two causes, both existing guards correctly catching
the behaviour change rather than defects in it.
TEN were `_SEEN_TAIL = "You saw it earlier this session"`. Every one of those
tests drives an arm with `exclude_rule_ids` alone — the NAMING ledger — which
since this change is the middle state, not the opened one. That is the whole
point of the step: `seen` stopped meaning "you saw it". The constant now holds
the middle tail, so each test keeps asserting exactly what it was written to
assert (a repeat gets a line distinct from a first surfacing) against the
wording that is now true.
Added `_HELD_TAIL` and a negative assertion with it: neither call names an
opened ledger, so neither may claim the session read anything. That catches
`held` defaulting true, which would have every repeat assert the strongest of
the three claims on no evidence — and none of the ten existing tests would
have noticed.
THE ELEVENTH was my own test edit being wrong, not the code. I asserted
`held_rule_ids` would appear in `re.findall(r"printf '([a-z_]+)=", defs)`, but
`scribe_held_query` spells it `printf '&held_rule_ids=`, and the leading `&`
means that regex never matches it. The regex enumerates SCOPE keys — the
alternatives that open a query — and the ledger key is appended to a query
that already has one. Two different contracts that happened to share a file,
so they are now asserted separately and the difference is written down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 386 made a repeat REFERENCED rather than withheld, and the line it
chose says "You saw it earlier this session". Nothing ever checked that. The
arms emit a TEASER — title, trigger, get_rule(N) — so a session can be shown a
rule twenty times and never read a word of it, and a compaction summarises the
teaser away leaving nothing behind. The server was asserting something about
the reader's context it had no way to know.
Three states now, where there were two:
never surfaced "it is not in this session's loaded set"
named, unopened "Mentioned earlier this session but not opened — read it…"
opened "You opened it earlier this session; pull it… again"
The middle one is the honest one and the one that was missing. It keeps the
full invitation, because a session that skipped a teaser is in nearly the
position of one never shown it.
HOW "OPENED" BECOMES OBSERVABLE. A new PostToolUse hook watches the get_rule
call itself and appends to `<sid>.opened.ids`. PostToolUse does fire for MCP
tools — the event's own output schema carries `updatedMCPToolOutput`, which
would be meaningless otherwise — and the matcher is `mcp__.*__get_rule` so the
server segment, which varies by install, is not pinned.
This is NOT the self-report 386 rejected. That objection was to ASKING a model
whether it holds a rule, which is unverifiable. A tool call is an event the
harness reports whether anyone asks. Recording what a session DID and believing
what it SAYS about itself are different kinds of evidence.
Both ledgers clear together on compact/clear. Keeping `.opened.ids` across a
compaction would have the arms telling a freshly-summarised session "you opened
it earlier" about a rule now nowhere in its context — a more confident version
of the bug being removed. Same reader (scribe_rules_live) for both, so ageing,
last-entry-wins and the bare-id format are defined once.
Also closes two smoke-coverage holes the checker was reporting as SKIP: the new
recorder, and scribe_precompact_preserve.sh from #3680. The latter needed
STATIC_FLOOR to become a set — PreCompact's contract is inverted, its stdout
BECOMES the summarizer's instructions, so silence is its failure mode and a
generic read of it looks like a leak.
Step 2 of milestone 416, and a hard prerequisite for step 4: while suppression
keys on shown, widening k marks records "seen" faster than they are read, and
the ledger would degrade in proportion to the improvement.
Plugin minted 2026.09.16.1232 -> 2026.09.16.2102.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 6937 red — 3 collection errors, `positional argument follows keyword
argument`, in the three integration fixtures that call the service positionally
(`create_rule(topic.id, uid, "title", "statement")`). The script that added
`when_to_apply=` to 15 fixtures inserted it as the FIRST argument, which is
valid only where every other argument is already a keyword.
Moved to the last argument in every call, which is legal in both styles, and
the continuation indent now matches the surrounding arguments.
Also repairs self-inflicted damage: the same script added a trigger to the two
tests in test_rule_trigger_required.py whose whole purpose is to call the
creators WITHOUT one. They would have stopped raising and the guard's own
proof would have inverted — a test that passes for the opposite reason than
the one it names, which is worse than a failing one.
Caught by ast.parse across tests/ rather than by the next CI round trip.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`when_to_apply` is not metadata. `rule_document` embeds a rule as
`{title} — {trigger}` / `When to apply: {trigger}\n\n{statement}`, the trigger
appearing twice so purpose dominates a short vector — the shape note 2485
measured on snippets (a 0.153 top-to-second gap against 0.010–0.023 for
everything else). Without one the document silently becomes title + statement:
a DIFFERENT shape, ranked against a corpus it does not match, with nothing to
report it. Every bar and every rank in the system assumes one shape.
`create_preference` has refused an empty trigger since it shipped. The two rule
creators defaulted it to "" — so the shape was enforced for the record kind
that guides and optional for the kind that binds.
The guard lives in the SERVICE, because both doors reach it: the MCP tools and
the frontend's fast path in routes/rulebooks.py. Written in either alone, the
other could still create a rule that never fires. The route keeps a matching
check for the STATUS CODE only (400, not the 404 it maps ValueError to).
update_rule refuses to EMPTY an existing trigger, checked after the mutation so
it covers `clear=[...]`, an emptied form input, and any route added later.
Deliberately asked as "did this edit remove one" rather than "does one exist":
a rule predating the guard has none, and refusing to save it would freeze
precisely the unreachable records that most need fixing.
Deliberately not following arose_from_id, which the human door exempts itself
from because provenance is about auditing what the AGENT changed. That reasoning
does not reach this field — a missing trigger is not a missing explanation, it
is a rule that does not work, and it fails an operator as badly as a session.
15 test fixtures across 6 files were creating rules with no trigger. They now
pass one; that they did not is the point — curation is not a guarantee.
Step 1 of milestone 416 "Retrieval stops guessing a bar". First because every
later step assumes one document shape, and it is much cheaper to guarantee
before a corpus grows than to backfill after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`report_preference` shipped reading PROMPTRULE_THRESHOLD_KEY, so the two arms
were one dial: tuning the bar for an operator's prose silently retuned the
lookup that runs when a task closes.
That coupling is worse on this arm than it would be anywhere else. Every other
retrieval arm scores a query that varies per call, so a mis-set bar shows up as
a changed clear-rate. COMPLETION_QUERY is a fixed string, so this arm's best
score for a given corpus is a CONSTANT — and a constant sitting under the bar is
a dead arm rather than a quiet one. No volume of traffic reveals it.
Found by the first live read for milestone 394 step 9: 69 calls, 69 declines,
every one naming the same record at the same score (0.7194 against a 0.72 bar).
Reading `best_available_id` (#3807) showed the record was about interpreting a
REQUEST, not about report shape — so the declines were correct and the arm is
healthy. The percentile alone would have said "lower the bar", which would have
delivered a false positive on every completion report ever written.
The bar does not move; the key does. Both defaults stay 0.72, so this changes no
behaviour on any install — it makes "leave this one where it is" expressible,
which it was not before. Settings grows the control (rules 25, 27), and the
default-agreement check grows a row.
Deliberately NOT included: a change to PROMPTRULE_DEFAULT_THRESHOLD. The
evidence for moving it is this install's near-miss table, and rule 115 keeps a
shipped default from being justified by one instance's corpus. That bar is a
per-user setting and belongs in the operator's Settings, not in the product.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Two interface-contract tests read each hook's own source for the query args it
sends. The project-scope key now comes from scribe_scope_query, so `repo=` was
no longer spelled in the hook and both read that as the hook having stopped
sending it.
They now pin the pair: the hook delegates to the helper, and the helper emits
exactly `repo=` and `project_id=`. That is a stronger assertion than the one it
replaces — it catches a hook rolling its own scope resolution again, and it
pins the new key, which the route must read for a non-git session to reach its
project at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
All six hooks scoped their requests one way: `git remote get-url origin`,
resolved server-side through the repo bindings. That key does not exist
outside a git repo, so a session in a plain directory was unscoped in every
hook at once — no project context, no prior-art scoping, no project rules —
and silently, because a missing remote is indistinguishable from a remote
nobody bound.
A `.scribe` file is the second key, read by the shared scribe_scope_query so a
directory scopes the same way everywhere:
{"instance": "https://scribe.example.com", "project_id": 2, "project": "…"}
`instance` is why the file is not just a number: an id is a different project
on every Scribe, so a marker that travels — a copied directory, a shared
machine, a repo someone else clones — would otherwise scope the session to the
wrong project without a word. Compared host-only, and a mismatch drops the id:
no project beats the wrong project. A bare integer is accepted too, since it
is what a person writes by hand. The marker beats a git remote — someone put
the file there on purpose — which is also how a directory overrides its
binding.
Two things it found on the way:
* An explicit project_id that did not resolve rendered NO message at all —
the branch hung off `if project_id` as an `elif`, so a caller holding a
pointer it believed in got a context that silently omitted the project it
had asked for. Now reported.
* The refusal reason was a global set inside a function every caller reads
through `$( )`. The assignment died with the subshell, leaving the caller
to read an unset variable under `set -u` — which aborts the hook and costs
the whole session's SessionStart context, to fetch a warning about a file.
It comes back through stdout with the id instead, and a test pins it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The hook's header quotes `{"decision":"block"}` while explaining why this hook
must never emit one, and the instructions it prints use "decision" in a
sentence. Grepping the whole file caught both and failed the guard on the file
doing its job.
Strips comments and the heredoc first, and guards the stripper: `_code()`
returning nothing would make all three static checks pass against an empty
string, which is the circularity rule 167 is about.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Spike #3680 asked whether a PreCompact hook can reach the model. Read out of
the installed Claude Code build (2.1.273), the answer is yes — through a
different channel than note #3679 assumed:
* `hookSpecificOutput.additionalContext` is NEVER read on PreCompact. The
hook-output schema has no PreCompact variant; the field is honoured for
SessionStart, SubagentStart and Stop, and silently dropped here.
* A PreCompact hook's STDOUT becomes `newCustomInstructions`, merged with
the operator's own `/compact` instructions and passed into the prompt that
writes the summary. Manual, auto and partial compaction all do this.
So the hook does not interrupt the compaction — it steers the summary, which
is what the next turn reads. Blocking is the thing not to do: a PreCompact
block SKIPS compaction, tells the model nothing, and leaves the session
running on uncompacted with no summary at all.
scribe_precompact_preserve.sh names what the summary is the only copy of:
Scribe record ids WITH titles, the in-progress task and its milestone, work
done but not yet recorded, governing rules, and unfinished operator asks.
No network and no config — what must survive is already in the conversation
being summarized; the hook only says which parts are load-bearing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
list_tasks returned every row's to_dict(), body included: a project's todo list
came to 93-165k characters, past what an MCP client accepts inline, so the list
arrived as a file to page through (the #4045 failure, one call over).
- notes.brief_row: id, title, type, project, tags, updated_at; for tasks,
status, kind, priority, milestone id and title; description, parent and due
date only when set.
- milestones.titles_for: one query for the milestone titles a page of rows
names.
- Brief rows on list_tasks, list_notes, get_milestone's steps, get_system and
list_system_records. get_task / get_note / get_snippet read a record in full,
and each docstring says so.
- tests/test_list_rows_brief.py pins the ceiling: 100 rows of ~5k-character
step plans stay under 40k characters.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A session's go-ahead request sat inside a completion list as "Blocked by the
permission check", and the operator read it as a fault rather than a question
waiting on them. The reporting-back skill now puts any action held for a yes
under a section headed "Approval requested", near the top of whatever reply
it is: each change numbered so part can be approved, why it needs them, how
it is undone, and what happens after. The Asks table gains an Approval row,
and the completion report's "Needs you" points at the section.
test_reporting_back_skill pins the heading and the row. Plugin version minted:
2026.09.15.1921.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Measured live after deploy: three plans reworded from existing active
milestones scored 0.83-0.87 against the milestone they restated, and the
nearest distinct plans 0.72-0.77. At the inherited 0.90 the semantic arm
matched nothing, so only an identical title was caught. 0.80 sits in the gap.
The Settings form's default moves with it (test_settings_defaults_agree).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 5 of milestone 415 "An existing plan is found before a new one is made".
Sessions opened a second milestone beside the roadmap milestone that already
covered the work, and filed related tasks loose, because no surface told them
to look first.
- writing-plans: a section on finding the plan that exists (enter_project's
unplanned_milestones, search(content_type="milestone"), list_milestones);
when an active milestone covers the work, add steps to it; a second
milestone only for a separate arc; the gate's existing_milestone reply.
- using-scribe: "when you plan" gains the same check and milestone_id on
related tasks.
- _INSTRUCTIONS PLAN line points at the milestone search (1,689 of 2,000).
- test_guidance_ownership pins the topic on writing-plans.
- Plugin version minted: 2026.09.15.1744.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
test_nothing_else_builds_the_embedding_document_itself caught start_planning
building f"{title}\n{body}" inline for the plan gate's candidate text. That is
the embedded-document shape; plan_candidate_text now takes (title, body) pairs
and calls embedding_text, so the candidate moves with the corpus it is ranked
against (#2486).
Also carries the create_task / create_records milestone_id docstring lines from
step 5 (#4080), which share the file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 4 of milestone 415 "An existing plan is found before a new one is made".
A session that could not see an existing plan made a second one beside it.
start_planning and create_milestone now ask first: an ACTIVE milestone in the
project with the same title, or one that reads as the same plan (title, design
and steps against milestone embeddings), is returned with its progress and a
pointer to create_records(milestone_id=...). Nothing is created; force=true
bypasses.
- dedup.find_matching_plan / plan_gate / plan_match_response; access-checked
before either arm (rule 78), fail-open like the other gates.
- Done milestones never block; the semantic arm needs 200+ chars of candidate.
- kb_plan_match_threshold (default 0.90) is a setting, in the Settings view,
and pinned against the Python default by test_settings_defaults_agree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`search` covered notes, tasks and rules, and a milestone — the record a plan
lives in — could not be found. A project whose roadmap was written as
milestones had every later plan opened beside the one that already described
it, because nothing could have told the session it existed.
- milestone_embeddings (migration 0102): the third sibling of note_ and
rule_embeddings, for note 3163's reason — the search is milestone-specific.
The document is title — description, then description and the plan body,
so a roadmap milestone with no description is still found by its design.
- Written on create, on a title/description/body update, and for a plan made
through start_planning / create_records, fire-and-forget with the parent-row
claim (#3262); a startup backfill covers every existing milestone. Derived,
so it joins _NOT_INCLUDED beside the other embeddings.
- semantic_search_milestones: a project's milestones when the caller can read
it (access.can_read_project), otherwise the caller's own; optional status.
- search(content_type="milestone"): id, title, description, status, project
and progress. Its own shape, and not part of "all", whose results are
note-shaped. The docstring says what it is for: ask before start_planning.
- Integration test on real Postgres: found in its project and not another,
status narrows, an unreadable project returns nothing.
Milestone 415 step 3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Operator, 2026-09-15: the unmilestoned group should collapse, "especially the
done column as it currently consume a lot of vertical space for [work] that's
already done."
- The "No Milestone" group gets the milestones' chevron and collapse, keyed as
0 in the same Set (no milestone id is 0), and starts collapsed on first load
when everything in it is finished — the rule finished milestones already
follow.
- Each group's Done column header becomes a button that folds its cards, with
the count still showing and aria-expanded set. Folded by default: done work is
the part of a board nobody is reading.
- Not persisted, matching the milestone collapse beside it.
Milestone 415 step 2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A plan written as a milestone with a description and no steps was invisible to
the session handshake: it lists the 5 most recently touched milestones (#4045),
and touching is a step changing, so a step-less milestone can never qualify.
FabledLibrarian's roadmap (nine such milestones) sat unseen while later plans
were opened as new milestones beside the ones that already described them.
enter_project adds `unplanned_milestones`: active milestones with no steps, in
roadmap order, id/title/description, up to 10 with an omitted count, none
repeated from the recent list, and absent when there are none. The docstring
says what they are for: check them before starting a new milestone, and add
steps to a match with create_records(milestone_id=...).
Milestone 415 step 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
start_planning on project 2 replied with 92,645 characters, 65k of them
applicable_rules. Milestone 414 made a project's listing every global rule
tagged to an area it works in (before, the rules of subscribed rulebooks, and
project 2 subscribed to none), and the non-brief rules_payload sent each as a
full rule_brief. get_milestone, get_project and get_task carried the same.
Every rules_payload form now lists: id, title, the topic a global rule sits in,
and `via` for a co_surfaces partner. get_rule reads one in full, and retrieval
delivers them in full when work matches — the reasoning #4045 applied to the
handshake. A test pins 81 full-length rules under 6k characters.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
rules_due_for_verification (the rule staleness sweep) and rule_history (what a
rule used to say) are pure reads, and a read-scoped API key was refused both:
neither is in _READ_ONLY_TOOLS, and the completeness test that should have
caught it only looked at tools whose NAMES start like a read (get_, list_,
search…). Neither does.
- Both join _READ_ONLY_TOOLS; the comment that pointed at this issue now says
why they sat unlisted.
- _WRITE_TOOLS declares every writing tool by name. Nothing reads it at
runtime — default-deny already refuses an unlisted tool — it exists so the
classification is total.
- test_every_registered_tool_is_classified_exactly_once takes its candidates
from what build_mcp_server() actually mounts, requires each in exactly one
of the three sets, and still flags a classified name that is no tool. The
decision stays explicit; only the candidate set widened.
- test_the_completeness_check_can_fail drops a real tool from its set and
asserts it is noticed (rule 167).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A rule's home is its reach: a rulebook topic makes it global, a project makes it
that project's. There was no way to change one, so a project rule decided to be
global could only be recreated and the original trashed — losing the id every
record cites, its edit history, its area tags and its relations.
- services.rulebooks.move_rule(rule_id, user_id, topic_id= | project_id=):
exactly one destination (the model's CHECK), owned by the caller, not the
rule's current home. A topic already holding a live rule with the same title
is refused with a message naming that rule, instead of uq_rule_per_topic
failing the commit. Someone else's rule reads as not found.
- Deliberately NOT done, and said in the docstring: no version (a version is
what a rule said, milestone 323 decision 4), no duplicate gate (nothing new
enters the corpus), no re-embed (retrieval reads the home at query time).
- Both doors: MCP move_rule, REST POST /api/rules/<id>/move (rule 33).
- UI: RuleHomePicker, one component in the rule editor (a global rule) and a
project's rules tab (a project rule), so the two cannot drift on what a
destination is.
- using-scribe names move_rule under "Where a new rule goes". Plugin
2026.09.15.1626.
Milestone 414 step 3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A rule's home is its scope now: a rule in a rulebook topic is global, a rule on
a project applies to that project, and retrieval reads that directly (#4074).
A subscription had stopped changing anything a session received; a suppression
muted rules from a subscription. Operator, 2026-09-15: "we have global and
project scoped rules, we don't need the subscriptions now."
What goes, whole (rule 22):
- Migration 0101 drops project_rulebook_subscriptions, project_rule_suppressions
and project_topic_suppressions, and strips subscribe_rulebooks (and 394's
leftover exclude_always_on_rulebooks) from stored inception choices.
- Service, MCP and REST: subscribe/unsubscribe and the four suppress/unsuppress
operations. The Subscribers checklist, the subscribe chips, the skip buttons
and the Suppressed section in the rules UI.
- Inception asks two questions (design system, seed Systems). create_project and
decide_project_inception lose subscribe_rulebooks.
- Backup v15 stops exporting the three sections; older archives still restore,
the keys simply unread. Trash no longer hard-deletes suppression rows.
What changes meaning:
- get_applicable_rules is a project's LISTING: its own rules, plus the global
rules tagged to an area it works in. Untagged global rules apply everywhere
and arrive by retrieval, so they are not listed. A co_surfaces partner on a
different project is not dragged in.
- list_rules(project_id) lists that project's own rules.
- rules_payload drops subscribed_rulebooks and suppressed_*; the handshake's
brief form is project_rules alone.
- using-scribe's "Where a new rule goes" and inception sections, tool
docstrings and docs say global vs project. Plugin 2026.09.15.1620.
Milestone 414 step 2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
semantic_search_rules searched every rule the user owned, and every hook arm
called it without a project, so each project's rules were injected into every
other project's sessions and a project rule meant nothing a session could feel.
The search now takes a scope: global rules by default (an unbound session, or a
caller that forgets to say), global plus project N when given project_id (N's
rules only if the caller can read that project, through access.can_read_project),
and every owned rule with everywhere=True. The four hook arms and the report
preference lookup pass the session's project; an explicit
search(content_type="rule") scopes to its project_id, or asks the whole rulebook
without one.
Milestone 414 step 1. Guarded by an AST walk that every hook call site passes
project_id, and an integration test on real Postgres that a rule is reached only
from its home.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The handshake carried the whole project record, every milestone's plan, full
rule text, the notes most recently edited and ~9k of design guidance. For
project 2 that was ~222k characters, past what an MCP client accepts as a tool
result. Each category was walked through with the operator and sized to what a
session needs on arrival; each names the call that has the rest.
- project: id, title, status and the full goal (session start's "full goal"
pointer still lands here). get_project keeps the whole record.
- milestone_summary: the 5 most recently touched milestones, any status, most
recent first, without plans. Summaries gain last_touched_at: the later of
the milestone's own edit and its newest step update, from the query that
already counts steps. milestone_summary_omitted counts the rest and points
to list_milestones. get_project and list_milestones list every milestone,
also without plans.
- open_tasks: the 10 most recently touched open tasks, with or without a
milestone, each naming its milestone. list_notes gains sort="touched"
(the later of updated_at and the newest work-log), because a log doesn't
bump updated_at.
- recent_notes: dropped. Retrieval surfaces notes by relevance, and
get_recent covers recency.
- systems: id and name.
- design_system: summary plus guidance_call. get_design_system gains
resolved_guidance, the chain-merged prose; its own guidance field is only
the departures, so session start's old pointer to it led to a fragment.
The session start pointer and using-scribe's "Building UI" section now
name resolved_guidance.
- rules: rules_payload(brief=True) gives project_rules as id and title plus
subscribed_rulebooks, and records only what it shows. Retrieval delivers
rules in full and ignores subscriptions (#4052). Other callers unchanged.
- pattern_coverage, inception and systems_bootstrap: unchanged.
Clients: the plugin's using-scribe skill, the compaction notice and session
start are updated here; the REST project summary only gains last_touched_at.
Plugin version minted.
Tests: a size ceiling on the handshake for a large project; milestone and
task selection and naming; brief rules; resolved_guidance; the session
start pointer; and a real-Postgres test that a work-log touches its task and
a step update touches its milestone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
enter_project returned every milestone's full plan body. On a project with 39
milestones the handshake came to ~222k characters, 168k of them bodies (110k
from done milestones). That is past what an MCP client accepts as a tool
result, so the call meant to orient a session arrived as a file to page
through. It grows with a project's history, so any long-lived project on any
install gets there.
- brief_milestone_summary (services/milestones.py) trims summary rows to the
listing fields: id, title, description, status, order_index and progress.
The plan is get_milestone's job. user_id, project_id and timestamps repeat
what the caller knows.
- enter_project and get_project share one block: every open milestone plus
the 5 most recently updated done ones, in order. milestone_summary_omitted
is attached only when older done ones were left out, and names
list_milestones and get_milestone.
- list_milestones lists every milestone, done included, without bodies. It is
the call the omitted line points to, and it had the same size problem.
- The REST project summary is unchanged; the web UI reads it.
Tests: trimming, the done cap and its order, the omitted key present and
absent, get_project and list_milestones, and a size ceiling on enter_project's
milestone block for a 200-milestone history.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
In a repo where 3,000 files define `slug`, scribe_local_dups printed
nothing; with 3 files it printed the duplicate line. Every hook runs under
`set -uo pipefail`, and `hits=$(git grep -l … | head -4) || hits=""` lost
head's four lines whenever git grep was still writing when head exited. That
is a SIGPIPE, the substitution fails, and the outer fallback wipes the result.
So the by-name duplicate arm went silent for exactly the most-duplicated
names. Found while fixing the same trap in the new Stop hook (#4041).
- The fallback moves inside the substitution, `$(… | head -N || true)`, at all
five sites: scribe_defs.sh (local dups), scribe_prior_art.sh (names,
old_first, shapes) and scribe_after_write.sh (names). A real upstream
failure still yields empty output.
- check_plugin gains a known-bad pattern for the shape, so no hook can bring
it back.
- Tests: the real function against a 3,000-file repo (past the pipe buffer),
and the pattern shown to flag the old shape and pass the fix. Plugin
version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI run #517 failed five of the new hook tests, all of them expecting a
request that never went out. The prefilter matched `"name":"…update_task"`
with no space after the colon, which is how Claude Code writes its
transcripts, while the tests wrote theirs with json.dumps defaults
(`"name": "…"`). The hook exited at the prefilter for every test transcript.
The silent-case tests passed for the same wrong reason.
- The prefilter now allows whitespace after the colon. The jq parse behind it
never depended on formatting.
- The test transcripts are written compact, matching the real file. The
silent-case tests now reach the turn parse rather than stopping at the
prefilter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Everything else Scribe gives an agent arrives before the reply is written. A
Stop hook is the one moment the finished reply exists, so it is the last
chance to fix a report the operator can't read, and the only place adherence
to the shape can be measured.
- plugin/hooks/scribe_report_check.sh (Stop): deterministic, no model call.
1. Did this turn close a task? That means an update_task/create_task call
with status "done" since the turn's prompt, whose tool_result is not an
error. Otherwise it stays silent, which covers most turns (one grep).
2. Does the reply that ends the turn say where the work sits (a record by
id and title, or step N of M), what needs the operator, and what comes
next? Matched on those words, not on exact headings.
3. If sections are missing, it blocks once. With stop_hook_active set, a
rewrite is recorded (passed_after_rewrite / missing_after_rewrite) and
never blocked again. A block loop started by another plugin (no marker
from this hook) is left alone.
- Measured: every checked reply is reported to GET /api/plugin/report-check
(passed / blocked / after rewrite). Turns that close nothing are not
reported; they would cost a request per turn and add nothing to the rate.
Outcomes go to app_logs as category "plugin", action "report_check".
- It blocks only when the block was recorded, and only in the server's words.
The endpoint returns the block reason, so the hook carries timing and
transport only (PACKAGING.md), and an unconfigured or unreachable instance
never stops a session.
- The transcript format is read from real transcripts and marked in the hook
as observed rather than documented. The Stop contract (transcript_path,
stop_hook_active, decision/reason, no matcher, SubagentStop separate) was
checked against the Claude Code hooks docs. A prompt-type hook was not
needed: the deterministic check passed a real completion report from this
session and blocked a stripped one.
- A pipefail trap was caught while exercising the hook: `tail | grep -q`
reports failure exactly when grep matches, because tail dies of SIGPIPE.
The prefilter reads through process substitution; the section checks use
here-strings.
- Tests: an end-to-end hook suite over synthetic transcripts and the shared
HTTP sink (silence, pass, server-worded block, rewrite recorded, foreign
loop, errored write, earlier turn, unwritten reply, no recorded check, bare
id), and service tests for the reason wording and the outcome record. Smoke
event added to check_plugin; README and PACKAGING list the hook and
endpoint. Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reporting-back skill ships default shapes; an operator's adjustments to
them are preference records. Prompt-time retrieval matches the operator's
message, and a shape preference is about the reply, so those preferences were
on file and never arrived. Operator's decision (logged on #4013): the server
delivers them for a completion report, and the skill asks for every other kind.
- Completion reports (option C): closing a task with update_task runs a
kind-filtered preference search for the moment "writing the completion
report after finishing a task" and returns matches as `reply_preferences`
({id, title, statement, kind}), with a sentence added to `report_back`
naming the key. A preference says it is about completion reports through
its own when_to_apply; no tag or column. Omitted when nothing matches, and
the lookup fails open.
- Telemetry: every call logs to retrieval_logs under `report_preference`
(empty calls included; a search that never ran writes no row) and hits are
recorded surfaced. The source is ranked, so it counts toward pull-through.
The bar is the prompt arm's setting until step 6 reads this source's near
misses.
- Every other reply (option A): reporting-back gains "The operator's own
shapes come first". Before a finding, decision, handoff or "where are we",
search(content_type="rule") in the words of that moment and follow what
comes back. Registered in the ownership guard with reporting-back as owner.
- Loading reply shapes at session start (option B) was rejected: it would be
a small copy of the preloading milestone 394 retired.
Domain-neutral query (pinned); works on an install with no preferences.
Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 7's live check (#4034 "a real session shows each topic stated once")
found nothing copied across surfaces, but found the owner repeating itself:
using-scribe said "rules are retrieved; ask before a consequential act; an
empty session is not an empty rulebook" in "Do this first", in reflex 2, and
twice more further down reflex 2.
- using-scribe: reflex 2 now holds the one full statement (how rules arrive,
why that is the surface working, what silence means, when to ask). "Do this
first" keeps the enter_project step and a short lead-in pointing at reflex
2. "A retrieved rule outranks a default habit" and "Ask hardest where you
feel most certain" stay, since each adds something.
- Adapter static context: "Lines injected beside your work" keeps only the
Claude Code timing and leaves "never the whole set" to the skill. "Keep one
copy" names CLAUDE.md and auto-memory and points at using-scribe, which
already says to leave a client's settings as they are.
- The ownership registry's markers and statement for the topic still sit on
using-scribe, and nothing else changed owner. Plugin version minted.
Sizes: using-scribe 18,823 -> 18,197 chars; scribe_static_context.md
1,797 -> 1,714.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A raw 200-char slice ended mid-word with nothing marking the cut, so a
reader took half a sentence for the whole goal. _goal_line flattens the
goal to one line, trims at a word break with an ellipsis, and points at
enter_project(id) when it cut.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 6 of milestone 410 "One owner per piece of guidance". CI now keeps the
shape decision #4027 set, so the next feature cannot quietly add a copy.
tests/test_guidance_ownership.py, 33 topics, each with an owner, markers, a
statement distinctive to the owner's full wording, and optional index markers:
- test_every_topic_is_stated_by_its_owner: markers and statement on the owner
- test_no_topic_is_stated_in_full_off_its_owner: the statement appears on no
other session surface (index, adapter static text and commands, live
context, other skills). Tool docstrings are not scanned; a contract may
elaborate the reflex that calls it.
- test_the_index_names_each_reflex_it_points_at: _INSTRUCTIONS keeps a
one-line pointer for each session-start reflex
- shared_with declares the one deliberate sharing: the note-check question
lives in create_note and in using-scribe for two different moments,
already pinned by test_verification_guidance_survives
- test_the_ownership_guards_can_fail shows each guard turning red (rule 167)
- the process topic now keys on get_process's "follow the returned body";
it had been passing on an unrelated "verbatim" in two other tools
tests/test_instruction_surfaces_agree.py keeps only what ownership cannot
enforce: the fold budget, rules-bind-names-preferences, and using-scribe's
pointer to reporting-back. Retired: the every-session-start-surface ask and
absence tests, the Systems and snippet owner pins (now registry topics), and
the SessionStart-without-ask test (#2497's shape). The push has carried no
rules since milestone 394, and requiring the ask beside every mention of it
would force a copy. The module docstring records where each protection went.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 5 of milestone 410 "One owner per piece of guidance". The operator
wants any attempt to package Scribe for another agent client to find the
repo already in the right shape.
plugin/PACKAGING.md (linked from the README) states:
- what every client package shares: plugin/skills/ (verbatim), the /mcp
endpoint and its in-band responses, the /api/plugin/* adapter endpoints
(context, retrieve, prior-art, tool-rules, processes), and one fmcp_ key
- what each client adds: a manifest; hooks limited to timing and transport;
optional commands; adapter static text that never copies a skill or the
index
- the Claude Code adapter file by file, as the worked example
- how Agent Plugins 1.0 clients (Codex, Cursor, Copilot/VS Code, Kiro,
ChatGPT) and Gemini CLI would map, marked researched-not-tested (#4023)
- four open questions for the second package: passing the key to the MCP
server, whether two manifests can share one folder, hook parity, and
where process skills go
Hook audit: every hook prints only server-provided text, status or outage
lines, the running version, or the compaction reload pointer. No guidance
copies, so nothing moved.
Guard: test_the_skills_reference_nothing_outside_their_folder fails on a
relative path upward or a reference to plugin/, hooks/, commands/ or a
manifest from inside a skill, with a companion test showing it can fail.
Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 4 of milestone 410 "One owner per piece of guidance". The plugin's
static session context was a second copy of using-scribe and the server
index. It now says only what Claude Code needs said: 8,705 -> 1,791 chars.
scribe_static_context.md:
- points at using-scribe for how to work with Scribe, and names the process
skills
- Claude Code specifics: keep one copy in Scribe rather than CLAUDE.md or
auto-memory (leave auto-memory at its default); injected lines are
retrieval, not the whole set; compact at clean seams; stored Processes
arrive as scribe-proc-* skills with /scribe:sync; say so when the tools
are unavailable
- retired: the restated reflexes, "how the surfaces divide the work", and
the "follow the surface that assumes least" precedence (decision #4027)
Hook behaviour is unchanged. The SessionStart hook header says what the
static tier now carries, and the unreachable-instance status points at the
using-scribe skill instead of "the standing guidance above".
README and manifest describe the plugin as the Claude Code adapter over the
shared, client-neutral skills. Fixed along the way: the README said the
SessionStart hook "injects your rules" and suggested disabling auto-memory,
both contradicting the product since milestone 394.
Tests: the session-start rules guards now pin the owner (using-scribe) and
the index (_INSTRUCTIONS) rather than every surface; the Systems-reflex and
snippet-trigger guards pin their owners; the reporting-reflex guard pins
using-scribe. The loss guard and client-neutral guard stay green. Plugin
version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 3 of milestone 410 "One owner per piece of guidance".
_INSTRUCTIONS is rewritten as an index for every MCP client (1,597 of 2,000
chars): orient, rules, recall, record, plan, ids, reuse, UI, report. Each
line names its tool, and the block says every reflex is stated in full in
the using-scribe skill and in each tool description.
- names no client: CLAUDE.md, auto-memory and "the client injects ~2k
chars" are gone
- gains the two reflexes it lacked: records that cite each other go through
create_records with {{ref:N}}, and reports start from `placement`
The comment block above it now explains ownership (decision #4027) instead of
accumulating per-milestone trade history, and keeps the budget and its
reason (#2562).
build_session_context states only what the server knows about this session:
the active project and open work, its design system, an unbound-repo hint,
or that no project is bound. Removed: the "you are not holding the
operator's rules" section, the closing "Reflex: search Scribe" line, the
design-system usage sentence, and the plugin-specific header. using-scribe
owns all of that. The truncation note no longer restates the rules ask.
Tests: a pin that the live context carries no rules reflex; the cap test
drives truncation through the unbound-repo hint, since a bare session is now
one line; the budget test message describes the index.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 2 of milestone 410 "One owner per piece of guidance". Skills are the
part of every client package shared verbatim (Agent Skills, decision #4027),
so they state each reflex in full and name no particular client.
using-scribe gains what only the static session context said:
- a retrieved rule outranks a default habit; ask when no rule speaks to it
- log on completing a task and on hitting a problem, not only successes
- the systems_hint on an untagged record is the tagging question, answered
at the moment of work
Client-specific text leaves the skills, rewritten as the universal idea:
- using-scribe: "keep one copy" no longer names CLAUDE.md, MEMORY.md, native
auto-memory or autoMemoryEnabled; "this plugin" becomes Scribe
- reusing-code / shape-accounting: Write/Edit and Bash become editor tools
and shell edits; the prior-art "hook" becomes the prior-art hint; a plugin
version number is dropped
tests/test_guidance_ownership.py:
- test_the_skills_name_no_particular_client fails on any Claude Code path,
memory file, slash command, hook event or tool name in a skill, each
marker commented with why it is client-specific; a companion test shows
it can fail
- three registry topics for what using-scribe now owns; the loss guard
stays green
Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 1 of milestone 410 "One owner per piece of guidance". The later steps
delete duplicate copies of agent guidance; this guard stops the last copy of
a topic going with them.
- tests/test_guidance_ownership.py holds the registry: 30 topics from the
ownership map in decision #4027, each with its owner and marker phrases,
and one definition of a delivered surface (_INSTRUCTIONS, tool docstrings,
each skill, the static context, the adapter commands, the live session
context).
- The loss guard: every topic is stated in full, with all its markers
together, on at least one delivered surface. A miss names the nearest
partial match.
- The owner column is recorded but not asserted yet; step 6 adds the
exactly-one-owner guard once the moves are done.
- test_the_loss_guard_can_fail proves split and absent markers are reported
(rule 167).
- The old DISPLACED_TOPICS list in test_instruction_surfaces_agree is folded
in, so there is one list rather than two.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Skill bodies and tool docstrings still taught the deleted model: using-scribe
said always-on rules "arrive whether or not you ask" and that SessionStart may
inject a rule index; create_project, decide_project_inception and enter_project
said an undecided project gets "every always-on rulebook"; create_rule pointed
standards at "the always-on one"; the verification sweeps and
retrieval_telemetry listed always-on paths and a live preload.
Every passage now describes the current model: every rule is retrieved, a
rulebook binds only by subscription, an undecided project inherits nothing,
and the preload survives only in telemetry rows older than 394.
Also repairs three sentences left half-replaced by the 394 edits: the static
context's "If you have not loaded the no rule has arrived", create_rule's
"an A subscribed rulebook", and create_project's doubled
subscribe_rulebooks entry. Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 3 of milestone 409 "Response shapes". Step 2's reporting-back skill
only helps if it fires, and only exists in the Claude Code plugin.
- scribe_static_context.md and using-scribe (new reflex 11) say: report back
in a shape the operator can read, placed from the `placement` block rather
than memory, and point at the reporting-back skill. using-scribe also lists
it among the sibling process-skills.
- update_task returns a one-line `report_back` cue when a task is closed
(done or cancelled). A tool response is the only surface every MCP client
sees, at the moment the report is about to be written.
- _INSTRUCTIONS takes no line: there is no budget without trading out a
session-start reflex. The decision is recorded in server.py's comment
block so it is not re-litigated blind.
- Guards: test_instruction_surfaces_agree pins the reflex and the placement
pointer on both plugin surfaces; a tool test pins the cue on closing
statuses and its absence on every other update.
Plugin version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 2 of milestone 409 "Response shapes". A reply written in the order the
work happened is accurate and still unreadable to someone who was not there.
The new bundled skill shapes it around where the work stands.
- Fires when an agent is about to report completion, hand off, ask the
operator something, answer "where are we", or propose an approach.
- Every reply: conclusion first, one topic per section, visible priority,
the ask in bold at the end, plain words, the work placed in Scribe.
- Placement is taken from the placement block step 1 returns (#4010), not
recalled; untracked work is said to be untracked.
- Four families of shapes: Reports, Asks, Answers, Proposals, with the
completion report written out in full.
- Domain-neutral: the worked example is a backup job, evidence is "what you
could open to check it". Written as practices, and naming no instance rule.
- A structural guard pins the completion sections, the from-the-record
placement, and the absence of software-only vocabulary.
Listed in the plugin README and manifest description; version minted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Step 1 of milestone 409 "Response shapes". An agent reporting finished work
is asked to say which milestone it belongs to, which step of how many, and
what is next. Without those facts to hand it reconstructs them, and a
reconstruction reads exactly like the truth when it is wrong.
create_task and update_task (MCP) and the REST create/update task routes now
return a placement block: project; and for a task in a milestone, the
milestone, position {step, of}, progress {completed, total, pct} and next
(the next open step, falling back to the earliest open one before it).
- Step order is creation order, not get_milestone listing order, which
reshuffles on every update.
- Siblings are read through readable_notes_clause; position and progress
are computed over that same readable set, so a collaborator is never shown
a step title they cannot open, and a note share alone reveals no plan.
- Fail-open and omitted when empty, like every in-band decoration.
- _no_embedding moves into conftest as one opt-in fixture for both
integration modules that need it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sessions predicted the ids their next creates would get and wrote them into
plan bodies and reference notes before the records existed. The database
never collides; the sequence is shared by every session and user, so any
concurrent create took the guessed numbers and the references pointed at
someone else's records.
- create_records (new MCP tool) and start_planning(body=, steps=) create
their records in ONE transaction: insert, flush for the real ids, rewrite
{{ref:N}} / {{ref:milestone}} placeholders as #id "title", commit. No
prediction, no waiting, no stub records left behind when a batch fails.
Ids need not be consecutive and nothing depends on it.
- Every MCP create/update of a note, task or milestone refuses a #N sitting
just above the highest assigned id (within 50): that can only be a guess.
Refusal, not warning. Numbers far above the max (PRs, forge issues) pass.
- notes.build_note splits validation out of create_note so the batch
validates records exactly as a single create does.
- writing-plans and using-scribe say to pass steps up front and never write
an unassigned id; plugin version minted.
Integration test runs six concurrent batches and checks each resolves its
placeholders to its own records, and that a failing batch writes nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 08:33:56 -04:00
763 changed files with 90588 additions and 25486 deletions
A self-hosted work system-of-record for software projects, built to be driven by Claude Code. Notes, tasks, issues, projects, milestones, rules, and stored processes — reachable from Claude via a built-in MCP endpoint and a bundled Claude Code plugin, with a clean web UI for humans. No in-app LLM; Claude is the sole assistant.
<h1 align="center">Fabled Grimoire</h1>
<p align="center">A self-hosted system of record for the work you do with Claude Code.</p>
Claude records what it learned, decided and built in Grimoire, and that record reaches the next session, so a solution is recalled instead of reinvented. Notes, tasks, plans, rules, lessons and reusable code live in one place. Claude reaches them through a built-in MCP endpoint and the bundled Claude Code plugin, and people read and edit the same records in the web UI. There is no language model inside Grimoire: Claude is the only assistant.
## Deploy
You need Docker with Compose. Save this as `docker-compose.yml` (it is the [`docker-compose.yml`](docker-compose.yml) in this repository), then run `docker compose up -d` beside it:
```yaml
# Fabled Grimoire: the app and its database.
# docker compose up -d
# Then open http://localhost:5000. The first account registered becomes admin.
#
# Keep both volumes. app_data holds the instance's own SECRET_KEY, which seals
# Port 5000 is published straight to clients, so no proxy is trusted.
# Behind a reverse proxy, set 1 (one per proxy) and BASE_URL:
TRUSTED_PROXY_HOPS:"0"
# BASE_URL: "https://grimoire.example.com"
volumes:
- app_data:/data
depends_on:
- db
restart:unless-stopped
db:
# Postgres 17 with the vector extension Grimoire needs.
image:pgvector/pgvector:pg17
stop_grace_period:120s
volumes:
- pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER:${POSTGRES_USER:-grimoire}
POSTGRES_PASSWORD:${POSTGRES_PASSWORD:-grimoire}
POSTGRES_DB:${POSTGRES_DB:-grimoire}
restart:unless-stopped
volumes:
app_data:
pgdata:
```
Open `http://localhost:5000`. The first account you register becomes the admin.
Nothing has to be set first. On its first start Grimoire makes its own `SECRET_KEY` and keeps it in the `app_data` volume, so restarts, reboots and updates all use the same key.
To connect Claude, create an API key under **Settings → MCP Access**, then in Claude Code:
The plugin asks for your Grimoire URL and the API key. Settings → MCP Access shows the exact commands for your instance.
## Configuration
Everything is optional. Set these in the environment or in a `.env` file beside the compose file:
| Variable | When | What it does |
|----------|------|--------------|
| `BASE_URL` | Behind a proxy | The public URL, used in email links and SSO redirects |
| `TRUSTED_PROXY_HOPS` | Rarely | The starting count of reverse proxies in front of Grimoire (default `1`; the compose file above sets `0` because it publishes the port directly). Check and change it in **Settings → Config → Reverse proxy**, which shows what each count reads for your own request; behind Cloudflare it is usually `2` |
| `POSTGRES_PASSWORD` | Before the first start | The database password (defaults to `grimoire`; the database is reachable only inside the compose network). Postgres takes it once, when the volume is created |
| `SECRET_KEY` | Rarely | A key you manage yourself, instead of the one Grimoire keeps. It wins over the kept key; remove it to go back |
Everything else (registration, email, SSO, git forge connections, retrieval tuning, backups) is in **Settings**, or listed in [Configuration](docs/configuration.md).
## Setup pitfalls
- **Keep both volumes.** `pgdata` holds everything Grimoire stores. `app_data` holds the instance's key, and forge tokens and the backup bucket's secret are sealed under it. If the key is lost, Settings asks you to enter those credentials again; nothing else is lost.
- **The embedding model downloads on first use**, into the `app_data` volume. Without internet access, semantic search falls back to keyword search until the model is in place.
- **Behind a reverse proxy, turn off response buffering** for `/mcp`, because MCP responses stream. [Configuration](docs/configuration.md#behind-a-reverse-proxy) has an Nginx example.
## Updating and backups
```bash
docker compose pull
docker compose up -d
```
Database migrations run when the app starts, and the key stays in its volume. Claude Code updates the plugin from its marketplace; run `/reload-plugins` after an update.
**Settings → Data → Scheduled backups** writes a daily backup of the database to a folder or an S3-compatible bucket, and Settings → Data → Restore from Backup rolls the instance back to one, or brings a new instance back from it. See [Backups](docs/backups.md).
## Features
Notes and tasks with a Markdown editor, sub-tasks, milestones, issues, and kanban project workspaces. Stored processes, an engineering rulebook system (with an inception step that decides what each project inherits), and semantic search with proactive knowledge-injection into Claude's context. A knowledge graph, per-user/group sharing, and a built-in MCP server (`/mcp`) plus a bundled Claude Code plugin so Claude can record and recall your work directly.
**Working with Claude**
- Claude enters a project and gets a short primer: its goal, current plans, open tasks and areas.
- Your rules reach Claude when its work matches them, or at a moment of work such as finishing a task or pushing code.
- Before Claude writes a file or runs a command, the plugin shows it the rules, lessons and reusable code that may apply.
- Stored procedures, such as a drift audit, become Claude Code skills.
## Quick Start
**Recording the work**
- Notes in a Markdown editor, with wikilinks, tags and version history.
- Tasks, issues and time-boxed investigations, with work logs that the next session reads first.
- Plans as milestones: a design plus ordered steps.
- Lessons learned the hard way, linked to the rules they illustrate.
- Duplicate checks, so a new record that matches an existing one updates it instead.
**Prerequisites:** Docker and Docker Compose. No GPU or local model needed — Claude is the sole assistant, reached over MCP.
**Reusing code**
- Snippets of reusable code, each with when to reach for it and where it lives in the repo.
- With a git forge connected, Grimoire tracks every function and style in your repos and shows which ones are still unaccounted for.
- Ideas shared across projects on the same platform, with each project's answer recorded.
- Design systems with tokens, inheritance and a generated stylesheet.
Download [`docker-compose.quickstart.yml`](docker-compose.quickstart.yml) from this repo, then:
**For people**
- A web UI to browse, search and edit everything, plus a graph of how notes connect.
- Sharing with users and groups, as viewer, editor or admin.
- Email and password sign-in, optional SSO through OIDC, and invite links.
- Personal export to Markdown or JSON, and admin backup and restore.
The full list is in [Features](docs/features.md).
## Troubleshooting
| What you see | Why | Fix |
|--------------|-----|-----|
| `SECRET_KEY is set to the public placeholder` at startup | `SECRET_KEY` is `dev-secret-change-me` or `change-me-in-production` | Remove `SECRET_KEY`, or set a real one |
| Settings asks for forge tokens or the backup secret again | The instance's key changed: the `app_data` volume was replaced, or `SECRET_KEY` was set or removed | Enter them again, or restore the previous key |
| `Can't locate revision identified by '…'` at startup | An older image is running against a database a newer one has already migrated | Run the newer image; migrations only move forward |
| The plugin's tools are missing in Claude Code | The plugin isn't installed, or Claude Code hasn't reloaded it | `/plugin` to install or update it, then `/reload-plugins` |
| Search finds only exact words | The embedding model hasn't downloaded yet | Give the app internet access once, or wait for the first download to finish |
## Development
```bash
# Optional but recommended — set a secret key
exportSECRET_KEY=your-random-secret-here
docker compose -f docker-compose.quickstart.yml up -d
docker compose -f docker-compose.dev.yml up --build
```
Open `http://localhost:5000`. The first user to register becomes admin. To connect Claude, create an API key under **Settings → API Keys** and install the Claude Code plugin — see [API Keys & MCP](docs/api-keys-and-mcp.md).
[Development](docs/development.md) covers checks, migrations, the plugin and CI. [Architecture](docs/architecture.md) describes how it is built, and the [API reference](docs/api-reference.md) lists every REST endpoint.
> **Development:** To build from source, see [Development](docs/development.md).
| [Features](docs/features.md) | Detailed feature breakdown and keyboard shortcuts |
| [Development](docs/development.md) | Dev workflow, CI/CD, migrations, release process |
| [API Keys & MCP](docs/api-keys-and-mcp.md) | API key management and Fable MCP install guide |
| [SSO / OAuth](docs/sso-oauth.md) | OIDC setup for Authentik, Keycloak, and other providers |
| [API Reference](docs/api-reference.md) | All REST API endpoints |
## License
This project is privately maintained.
Fabled Grimoire is free software under the [GNU Affero General Public License v3.0 only](LICENSE) (`AGPL-3.0-only`). If you run a modified version for other people over a network, the licence requires you to offer them its source.
A Python MCP (Model Context Protocol) server that lets Claude directly interface with a running Fabled Assistant instance. The goal is two-fold: Claude's stronger reasoning handles planning, documentation, and analysis while Fable remains the authoritative store for notes, tasks, projects, and milestones; and direct API access enables process improvement by letting Claude observe and operate on real data rather than working from descriptions.
The work is split into two sub-projects:
1.**Fable API Key Feature** — additions to the main `scribe` project to support bearer token authentication
2.**Fable MCP Server** — a new standalone Python package at `fable-mcp/` in the same repo root
A third sub-project (Forgejo MCP for CI/CD automation) is planned as a follow-on after the Fable MCP is working.
---
## Sub-project 1: Fable API Key Feature
### Motivation
All existing Fable routes use session-based authentication (`session["user_id"]`). The MCP server runs as a local process and cannot maintain a browser session, so a stateless bearer token mechanism is required. API keys are user-scoped and carry a read/write permission level.
### Database
New table `api_keys` via migration `0027_add_api_keys.py`:
-`down_revision = "0026"` (the briefing tables migration)
Full keys are generated as `fmcp_<32 random url-safe chars>`, returned once at creation, never stored. Only the SHA-256 hash is persisted.
### Auth Middleware (`auth.py`)
`_check_auth` is updated to check the `Authorization: Bearer <token>` header before falling back to session auth:
1. If header present: hash the token, look up in `api_keys` where `revoked_at IS NULL`
2. If found: update `last_used_at`, set `g.user` and `g.api_key`
3. Scope enforcement: if `g.api_key.scope == "read"` and `request.method` is not `GET`, return 403
4. If header absent: existing session logic runs unchanged
Session auth is untouched — no regression risk for the web UI.
**Admin routes and API keys:** Routes decorated with `admin_required` check `user.role == "admin"` after auth resolves. API keys authenticate as the key owner — so a write-scoped key for a non-admin user will fail admin-protected routes with 403 (role check, not scope check). This is intentional: API keys cannot elevate privilege beyond the user's role.
### Routes (`/api/api-keys`)
New blueprint `api_keys_bp`, registered in `app.py`:
-`GET /api/api-keys` — list caller's keys (prefix, name, scope, last_used_at, created_at — never the hash or full key)
-`POST /api/api-keys` — create key; body: `{name, scope}`. Returns full key in response **once only**
-`DELETE /api/api-keys/:id` — revoke by setting `revoked_at = now()`
### Settings UI
New "API Keys" tab in `SettingsView.vue` (added to `VALID_TABS`):
Calls the existing `semantic_search_notes()` service (already in `services/embeddings.py`). The `content_type` parameter maps to the service's `is_task` argument:
This endpoint is needed by the MCP's `search.py` tool module. It reuses existing infrastructure with no new ML work.
### Conversation Type: `"mcp"`
A new conversation type `"mcp"` is added alongside `"chat"` and `"briefing"`. This requires changes in two places in the main app:
**`services/chat.py`** — `create_conversation(user_id, title, model)` gains an optional `conversation_type: str = "chat"` parameter, passed through to the `Conversation` constructor.
**`routes/chat.py`** — `POST /api/chat/conversations` accepts an optional `conversation_type` body field (defaults to `"chat"`), passed to `create_conversation`. `GET /api/chat/conversations` continues to default-filter to `conversation_type="chat"` (excluding `"mcp"` and `"briefing"`); pass `?type=mcp` to retrieve MCP conversations.
**Retention:**`cleanup_old_conversations` in `routes/chat.py` currently deletes conversations regardless of type. It must be updated to exclude `conversation_type="mcp"` from the sweep, so MCP audit-trail conversations are not automatically pruned.
MCP conversations:
- Are created with a caller-supplied name (e.g., `"MCP Session 2026-03-23"`) or auto-named
- Are excluded from the default chat list
- Are accessible via `GET /api/chat/conversations?type=mcp` for inspection
- Appear in the Fable UI if explicitly navigated to, providing an audit trail of MCP-driven interactions
### Chat SSE Wire Format
The MCP's `chat.py` tool must consume the existing SSE stream. The relevant endpoints and event schema:
-`{"type": "done", "content": "...", "tools_used": [...]}` — generation complete; this is the termination event
The MCP tool reads tokens until it receives `type: "done"`, then returns `response` (full content) and `tools_used` (list of tool names).
---
## Sub-project 2: Fable MCP Server
### Location
`fable-mcp/` at the repository root, alongside `src/`, `frontend/`, `alembic/`. It is **not** part of the main Docker build and has no import relationship with `scribe`. It will be extracted to its own Forgejo repo once stable.
`FableClient` is an async context manager wrapping `httpx.AsyncClient`. It reads `FABLE_URL` and `FABLE_API_KEY` from environment variables (with `python-dotenv` fallback to `.env`). All methods are `async def` and return parsed dicts. A `FableAPIError(status_code, message)` exception is raised for any non-2xx response.
A module-level singleton `_client: FableClient` is initialized at server startup and shared across all tool modules.
### `server.py`
Uses `mcp[cli]`'s `FastMCP` class. Imports all tool modules, which register their tools via decorators against the shared `FastMCP` instance. Entry point `main()` calls `mcp.run()` (stdio transport).
### Tool Modules
Each module imports `_client` from `client.py` and defines tools as `async def` functions decorated with `@mcp.tool()`. Tool docstrings serve as the MCP tool descriptions visible to Claude.
- MCP conversations are excluded from the normal chat UI list
### Error Handling
`FableAPIError` is caught at each tool boundary and returned as a descriptive string rather than propagating as an exception. This ensures Claude receives a readable error message (e.g., `"Fable API error 403: Read-only key cannot perform write operations."`) rather than a stack trace. Network errors (`httpx.RequestError`) are similarly caught and surfaced as strings.
Scope enforcement lives entirely in Fable's auth middleware — the MCP does not duplicate it.
### Claude Code Registration
After `pip install -e fable-mcp/` (or `uv tool install ./fable-mcp`), add to `~/.claude/settings.json`:
```json
{
"mcpServers":{
"fable":{
"command":"fable-mcp",
"env":{
"FABLE_URL":"http://localhost:8080",
"FABLE_API_KEY":"fmcp_your_key_here"
}
}
}
}
```
Claude Code spawns the process over stdio automatically. No Docker, no daemon.
---
## Build & Repo Plan
1. Implement and test within `scribe/fable-mcp/`
2. Once stable, extract to a new Forgejo repo (`bvandeusen/fable-mcp`)
3. Forgejo MCP (Gitea MCP) added as a second MCP server to automate build/push/config workflows — separate spec when ready
---
## Out of Scope (v1)
- MCP resources (browsable URI tree) — tools cover all use cases in Claude Code
- Forgejo MCP — follow-on spec
- Rate limiting on API key endpoints
- Key expiry / TTL
- Per-resource scope (e.g., key restricted to one project)
- In `startup()`: `asyncio.create_task(load_stt_model())` + `asyncio.create_task(load_tts_model())` when `VOICE_ENABLED`
### `src/scribe/config.py`
- Add 4 new env var attributes
- Add validation in `validate()`
### `src/scribe/services/llm.py`
- Add `voice_mode: bool = False` and `voice_speech_style: str = "conversational"` to `build_context()`
- When `voice_mode=True`, prepend: *"Respond naturally as if speaking aloud. No markdown, bullet points, headers, or code blocks. Complete sentences only."*
- Append style modifier based on `voice_speech_style`
### `src/scribe/services/generation_task.py`
- Add `voice_mode: bool = False` to `run_generation()`
- Read `voice_speech_style` from settings when voice_mode; pass both to `build_context()`
### `src/scribe/routes/chat.py`
- Allow `"voice"` in `conversation_type` whitelist
### `src/scribe/services/chat.py`
- Exclude `conversation_type == "voice"` from auto-cleanup retention
Floating PTT button (fixed bottom-right). Creates/reuses a `"voice"` conversation. Full flow: record → transcribe → send → stream → synthesise → play. Space bar hotkey (from `App.vue`). Mounted globally in `App.vue`.
The current briefing pipeline hallucinates calendar events, tasks, and news items that do not exist in the user's actual data. Observed examples from production:
- A morning briefing asserting "your dentist appointment is still in progress" when no such event existed
- An 8am check-in mentioning "a quick meeting at 10:30 AM" with no backing calendar entry
- A midday check-in inventing "a team huddle at 2:30 PM" and "the Q2 budget draft due by Friday"
- The model calling `search_images` in response to a clarifying question about a fabricated meeting
These are not bugs in data retrieval — the data gathering code (`_gather_internal`, `_gather_external`) returns correct values. They are a structural consequence of how the briefing context is assembled.
## Root cause — the "no receipt" problem
The current `run_compilation` pipeline does this:
1. Python code gathers data (tasks, events, weather, news) from the database
2. Python formats the data into a structured text blob as the user-role message: `TODAY'S EVENTS: ...`, `DUE TODAY: ...`, etc.
3. The LLM is called **once** with `[system_prompt, user_prompt]` and produces prose
4. Only the prose reply is written to the conversation. **The underlying data is never persisted in the conversation history.**
When the user later chats in the briefing conversation, the chat endpoint loads the full conversation history — which contains the model's *prose* from earlier but not the data that prose was derived from. The model's own prior output becomes the only source of "truth" available for follow-up questions. If that prose asserted a fact (real or hallucinated), the model has no way to distinguish it from ground truth when generating the next reply, and it will double down.
Compounding factors:
- **Empty sections are silently omitted.** If `calendar_events` is empty, the user prompt contains no `TODAY'S EVENTS:` line at all. The model has no explicit "zero events" signal — combined with an imperative system prompt ("note calendar events and tasks"), it interprets the silence as "I should mention some" and fabricates.
- **Scheduled slot injections append synthetic turns.** `run_slot_injection` writes a fake `[Midday briefing update]` user message and the assistant reply into the persistent conversation. By evening, the chat history contains three separate briefings, each potentially with errors, all treated as equal-weight context on follow-up.
- **The `search_images` tool is available during briefing chat**, with a negative-instruction description ("Not for factual questions"). Small and mid-sized models frequently ignore negative guidance in tool descriptions and call the tool anyway.
## Solution — agentic briefing (the receipt model)
Replace the one-shot synthesis with a tool-call loop. The briefing is no longer "a text blob synthesized from pre-gathered data." It is **a scheduled agent run**: the LLM is given a system prompt, a curated set of read-only data tools, and a trigger ("generate the morning briefing"). The model must call tools to see what exists. Every tool call and tool result becomes part of the conversation history, where it lives as a permanent, structured receipt.
### Why this fixes the hallucination
When the model calls `list_events(today) → []`, that empty array is now a persistent message in the conversation. On a follow-up question like "what meeting?", the chat endpoint loads the conversation and the model sees its own tool result from the morning showing no events. Answering "you had a meeting at 10:30" would require the model to directly contradict a tool result sitting two messages back — something LLMs are much more reliable at avoiding than contradicting their own prior prose.
In short: **the model cannot fabricate what has a visible receipt proving it does not exist.**
### What the model sees — before vs after
**Before** — the model sees one structured blob and produces prose. The blob is discarded.
```
[system] You are a personal assistant... weave together what matters...
[user] Date: 2026-04-10
WEATHER: Home — partly cloudy, 8–16°C
DUE TODAY: Fix briefing hallucinations
OVERDUE (2 tasks): ...
[assistant] Morning! Looks like a quiet day with two overdue items...
```
**After** — the model is given tools and iterates to ground truth.
```
[system] You are the user's personal assistant giving their morning briefing.
Use tools to see what's relevant. Only mention things you learned from
tool calls. If a tool returns nothing, say so honestly.
[assistant] Morning! Partly cloudy today, 8 to 16 — nothing on the calendar
so it's a clean run at the desk. Two things to keep in mind...
```
The conversation now contains verifiable receipts: `list_events(today)` returned `[]`, and that result sits in context forever (until pruned). Follow-up questions operate against those receipts.
## Architecture
### Existing infrastructure (reused, not rebuilt)
The codebase already has the agentic primitives — they're used for regular chat:
- **`llm.py::stream_chat_with_tools`** — the streaming tool-use loop that talks to Ollama with a `tools` parameter and yields tool-call chunks
- **`generation_task.py::_stream_with_retry`** — wraps `stream_chat_with_tools` with retry-on-500 behavior for cold-model races
- **`tools.py`** — defines 40+ tool schemas and an execution dispatcher
The briefing bypasses all of this and calls a one-shot `_llm_synthesise` helper. The refactor is mainly "route briefings through the same pipeline regular chat already uses."
### New modules
**`briefing_tools.py`** — a small wrapper exposing a curated read-only subset of `tools.py` for briefing runs. This is an **explicit allowlist**, not a blocklist, so newly-added tools must be opted in:
| Tool | Purpose |
|---|---|
| `list_tasks` (with filter args) | See what's actionable today, overdue, high priority |
| `list_events` (today / upcoming) | Know what's on the calendar |
| `get_weather` | Current/forecast weather |
| `get_rss_items` | Pull news themes filtered by user preferences |
| `list_projects` | Understand project context |
| `search_projects` | Surface active project summaries |
| `list_notes` (recent) | Capture follow-ups from yesterday |
**Explicitly omitted** from the briefing tool set:
-`search_images`, `search_web`, `research_topic`, `read_article` — external search is not a briefing concern, and `search_images` is the source of the "Peter Kyle Science Secretary" image-search bug
- All `create_*`, `update_*`, `delete_*` — briefings are read-only; a scheduled background job must not decide to mutate the user's data on its own
-`set_rag_scope`, `calculate` — not relevant to briefing content
### New briefing function
**`briefing_pipeline.py::run_agentic_briefing(user_id, slot, model, conversation_id)`** replaces `run_compilation`'s body (and eventually `run_slot_injection`). Internally:
1. Build a slot-specific system prompt (see below)
2. Load the curated briefing tools from `briefing_tools.py`
3. Seed messages with `[system, user]` where the user message is a simple trigger like `"Generate the morning briefing."`
4. Enter a tool-call loop (max 8 iterations):
- Call `stream_chat_with_tools`
- If the model returns `tool_calls`, execute them via the existing dispatcher, append tool results, continue
- If the model returns a final assistant message with no pending tool calls, break
The full message list is important: it's written to the conversation along with the final prose, so the tool-call receipts become part of the persistent record.
### Slot-specific system prompts
Compilation (full morning briefing):
```
You are the user's personal assistant giving their full morning briefing.
Use the tools available to see what's actually relevant today — tasks due,
overdue items, events on the calendar, weather, news themes, project state
— and weave it into a warm, natural-sounding summary.
Rules:
- Call tools to see the data. Never assert facts you didn't learn from a tool.
- If a tool returns nothing (no events today, no overdue tasks), say so
honestly. Don't fabricate items to fill space.
- Write flowing prose. No markdown, no headers, no bullet points.
- Aim for 6–10 sentences. Skip topics that have nothing interesting.
- Close on one or two concrete, actionable suggestions.
User profile (for tone and preferences):
{profile_body}
```
Check-ins (midday, afternoon):
```
You are the user's personal assistant giving a brief {slot} check-in.
Use tools to see what's changed since this morning. Focus on progress
and what's still unaddressed.
Rules:
- Call tools to see current state. Never assert facts without tool results.
- If nothing meaningful has changed, say so briefly — don't invent progress.
- 3–5 sentences, natural prose, no markdown.
```
### Conversation hygiene — removing the fake user messages
The current scheduler appends two fake messages on every slot injection:
Under the agentic model, we drop the fake `"user"` message entirely. Slot updates become plain assistant messages tagged with `metadata.briefing_slot = "midday"`. The chat endpoint's message loader is updated to filter these when building the LLM context on follow-ups, so a user chatting in the briefing conversation doesn't see three earlier briefings smashed into their history. The UI continues to show them as visible timeline entries.
(This is the Option A filter from the earlier discussion — a small, surgical change compared to migrating briefings to a separate sidecar table. Sidecar storage remains a possible future step.)
### Ollama setup compatibility
The existing Ollama deployment uses non-parallel mode across two GPUs, with a concern about context duplication between cards. This is the correct setup for agentic briefings:
- Each tool-call iteration shares the same KV cache on the same GPU, so appending tool results is cheap
- There is no race where a second iteration could land on a different card with a different cache state
- The trade-off is serialization: a briefing in progress will block a concurrent user chat request until it finishes, but scheduled briefings are rare enough (4× per day) that this is acceptable
If GPU contention becomes a problem later, the right lever is pinning specific models to specific GPUs (e.g., background tasks on GPU 1, interactive models on GPU 0) — not enabling parallelism.
## Cost & trade-offs
- **Latency:** a briefing now makes 5–7 inference calls (one per tool-call decision plus the final prose) instead of 1. On a local Ollama with a 7B model, expect 15–40s per briefing vs the current 5–10s. Acceptable for a background job; if it becomes painful at the user-facing check-in slots, investigate letting the model batch independent tool calls in a single turn (Ollama supports this).
- **Model requirements:** the default model must be reliable at tool calling. `qwen2.5:7b`, `llama3.1:8b`, `mistral-small-3:24b`, and similar handle it well. Models in the ≤3B class typically fail — they either emit no tool calls and return empty prose, or hallucinate invalid tool arguments. If a user's `default_model` is too small, agentic mode should fall back to legacy mode with a log warning.
- **Context growth:** tool results bloat the conversation. At 8 tool calls per compilation × 4 slots per day, a daily briefing conversation can reach 20-30 KB of JSON-heavy history. Fine for a day; aged briefings should be archived/rolled up after 7 days.
- **Tool subset drift:** someone adds a new mutating tool to `tools.py` and forgets to update the briefing allowlist. Mitigated by the allowlist model — the default for new tools is "not exposed to briefings."
- **Infinite loop safety:** a buggy model could tool-call forever. Hard cap at 8 iterations, log a warning if hit, return whatever prose was last produced (or a fallback message).
## Migration path
Ship incrementally, each step independently reversible:
**PR 1 — Agentic compilation behind a feature flag.** Add `briefing_tools.py`, add `run_agentic_briefing`, add a per-user setting `briefing_mode: "legacy" | "agentic"` (default legacy). Route only the 4am compilation through the new path when the flag is set. Keep slot-injection on the legacy path. Enable the flag on the author's account first, validate output quality over several days, then flip the default for all users. No DB migration required — the setting lives in the existing `settings` table.
**PR 2 — Agentic slot injections + conversation hygiene.** Migrate midday/afternoon check-ins to the same pipeline. Remove the fake `[Midday briefing update]` user-role message; slot updates become plain assistant messages tagged with `metadata.briefing_slot`. Add a chat-message-loader filter that excludes slot-tagged messages from the LLM context on follow-ups (they remain visible in the UI).
**PR 3 — UI polish.** Collapsed tool-call status row in the briefing card ("✓ checked calendar · ✓ looked at tasks · ✓ pulled weather"), expanding to show tool results on click. Tool-result cards (weather, news, task list) rendered inline where useful.
**PR 4 (optional) — Sidecar storage for briefing snapshots.** If the chat-filter approach in PR 2 feels too hacky, migrate briefings out of the conversations table and into a dedicated `briefing_snapshots` table. Frontend renders them as pinned timeline cards separate from chat. Larger refactor; defer until PR 1–3 prove the approach works.
## Secondary win — tightening the main chat system prompt
The regular chat is already agentic (it uses `stream_chat_with_tools`), but its system prompt does not explicitly require the model to ground factual claims in tool results. The prompt discipline introduced for briefings — *"Never assert facts you didn't learn from a tool. If a tool returns nothing, say so honestly."* — is worth applying to the main chat system prompt in a follow-up PR. The mechanism already works; only the framing needs tightening.
## Out of scope
- The Android Flutter client's failure to render `search_images` tool-result cards. This is a separate rendering gap in the mobile app and does not affect the server-side fix. Tracked separately.
- Re-evaluating the Ollama parallelism setting. The current non-parallel config is correct for this work.
- Replacing the background model for title/summary/observation extraction. That model's role is unrelated to briefings.
API keys let external tools access your Fable data without a browser session. Each key is scoped to a single user — it can only access data that user owns or has been shared with them.
API keys let external tools access your Grimoire data without a browser session. Each key is scoped to a single user — it can only access data that user owns or has been shared with them.
### Scopes
@@ -15,40 +15,51 @@ Admin-level operations (log access, user management) require a `write`-scoped ke
### Creating a Key
1. Go to **Settings → API Keys**
1. Go to **Settings → MCP Access**
2. Enter a name (e.g. "Claude MCP", "Home Server")
3. Choose scope
4. Click **Generate Key**
5. Copy the key immediately — it is shown only once (the token is `fmcp_`-prefixed)
Paste the key into the `Authorization: Bearer <key>` header of your MCP client
config (see **Scribe MCP Server** below).
config (see **Grimoire MCP Server** below).
### Revoking a Key
Click **Revoke** next to the key in the API Keys table and confirm. Revoked keys are deleted immediately.
Click **Revoke** next to the key in the keys table and confirm. Revoked keys are deleted immediately.
---
## Scribe MCP Server
## Grimoire MCP Server
Scribe exposes itself as a set of MCP tools that Claude (and other MCP clients)
Grimoire exposes itself as a set of MCP tools that Claude (and other MCP clients)
can use to read and write your notes, tasks, projects, rulebooks, and more. The
server is **built into the app** — it is mounted as a streamable-HTTP endpoint
at **`/mcp`** on the running Scribe instance (`src/scribe/mcp/server.py`). There
at **`/mcp`** on the running Grimoire instance (`src/grimoire/mcp/server.py`). There
is nothing to install: no wheel, no separate package, no CLI. You connect a
client straight to the URL with a Bearer token.
### Authentication
Authenticate with an API key generated from **Settings → API Keys** (see above),
Authenticate with an API key generated from **Settings → MCP Access** (see above),
sent as `Authorization: Bearer fmcp_<key>`. A `read`-scoped key may call only the
any write/delete tool is rejected with `403`. The allow-list is explicit rather
than derived from the name — see `_READ_ONLY_TOOLS`, which is why the two reads
without a read-shaped name are spelled out here. A `write`-scoped key may call everything.
### Claude Code (Project-scoped)
### Claude Code: the plugin (recommended)
The plugin connects the MCP server and adds the hooks and skills that put Grimoire's records in front of Claude as it works. Install it from this repo:
It asks for your Grimoire base URL, an API key (kept in your OS keychain) and, optionally, a project id. Settings → MCP Access shows the exact command for your instance. [`plugin/README.md`](../plugin/README.md) describes each hook.
### Claude Code without the plugin
Add a `.mcp.json` at the project root. The server `type` is `http` and the URL is
your instance's `/mcp` endpoint:
@@ -56,9 +67,9 @@ your instance's `/mcp` endpoint:
| Search / Recall | `search`, `get_recent`, `list_tags`, `retrieval_telemetry` | Semantic + structured recall, and the readout its thresholds are tuned from |
| POST | `/api/projects/:id/inception` | Record what the project inherits `{choices: {subscribe_rulebooks, design_system_id, seed_systems}}` (owner-only; `POST /api/projects` accepts the same under `inception`) |
| POST | `/api/projects/:id/inception` | Record what the project inherits `{choices: {design_system_id, seed_systems}}` (owner-only; `POST /api/projects` accepts the same under `inception`) |
| GET | `/api/projects/:id/inception/defaults` | What binds if nobody decides — the inception card's payload |
| GET | `/api/projects/:id/notes` | Notes + tasks in this project |
| GET / POST | `/api/projects/:id/milestones` | List / create milestones |
@@ -86,7 +89,55 @@ endpoint at `/mcp`, not these REST routes.
| GET / POST | `/api/projects/:id/systems` | List / create systems |
| GET | `/api/plugin/retrieve` | Title-first knowledge-injection candidates |
| GET | `/api/plugin/context` | Session-start context: the activeproject and its state |
| GET | `/api/plugin/retrieve` | Records and rules matching the operator's prompt |
| GET | `/api/plugin/tool-rules` | Rules matching a tool call about to run |
| GET | `/api/plugin/moment-tools` | The tools whose calls can reach a moment |
| POST | `/api/plugin/moment` | The rules mounted on the moments a tool call reaches |
| POST | `/api/plugin/reply-rules` | Whether the reply ending a turn is held for one read |
| POST | `/api/plugin/rule-opened` | Records that a session opened a rule it was shown |
| GET | `/api/plugin/shape-check` | The end-of-turn question about what this turn wrote |
| GET | `/api/plugin/claim-session` · `/release-session` | Bind a session to its task claims / release them at session end |
| GET | `/api/plugin/write-scope` | Which projects a pending act would change (a directory, or an MCP call's `tool` and `input`), and whether a session in its own project may do it. Filing an issue elsewhere is allowed; every other cross-project write is answered `allowed: false` with the reason the operator is asked |
| GET | `/api/plugin/processes` | Stored Processes for skill-stub sync |
| GET | `/api/plugin/prior-art` | Write-path hint for the plugin hooks (params: `path`, `code`, `repo`, `shapes`, `exclude_ids`, `exclude_sync_ids`, `exclude_derive`); returns `context`, `note_ids`, `sync_note_ids`, `stamped`, `divergence`, `derive`, `derive_keys` |
| GET / POST | `/api/projects/<id>/coverage`, `…/coverage/refresh` | Shape-ledger accounting (`pattern_coverage` line, counts, `derive_groups` — css groups carry `consumers`, `derive_new`, `unused_css`, `divergence`, `recheck`) |
@@ -191,19 +273,32 @@ endpoint at `/mcp`, not these REST routes.
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/admin/backup` | Export backup (format v4; `?scope=user` for own data) |
| POST | `/api/admin/restore` | Restore from a backup |
| GET | `/api/admin/restore/sources` | The backups at the backup destination, newest first |
| POST | `/api/admin/restore/preview` | What restoring a backup (`?source=<name>` or an uploaded file) would change; 409 with the reason when it can't apply here |
| POST | `/api/admin/restore?confirm=rollback` | Put the whole instance back to that backup, after saving the current state |
| GET / DELETE | `/api/admin/users` · `/api/admin/users/:id` | List / delete users |
| GET / PUT | `/api/admin/registration` | Get / toggle registration |
| GET / PUT | `/api/admin/smtp` · POST `/api/admin/smtp/test` | SMTP config + test email |
| GET / PUT | `/api/admin/base-url` | Get / set the public base URL |
| GET / PUT | `/api/admin/proxy` | Get / set how many reverse proxies stand in front of Grimoire (`{"hops": n}`, or `null` for the environment's count), with what each count would read for this request |
| GET / PUT | `/api/admin/db-maintenance` (+ `/health`, POST `/run`) | VACUUM schedule, health, manual run |
| POST / GET / DELETE | `/api/admin/invitations` (+ `/:id`) | Create / list / revoke invite links |
| GET / PUT | `/api/admin/backups` | Scheduled backups: destination, schedule and recent runs |
| POST | `/api/admin/backups/check` | Ask the destination whether it would accept a backup |
| POST | `/api/admin/backups/run` | Start a backup now |
| GET / PUT | `/api/admin/forge-webhook` | The forge push-webhook settings |
## Scribe MCP
## Webhooks
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/webhooks/forge` | Push events from a git forge, verified with `FORGE_WEBHOOK_SECRET`**(public)** |
## Grimoire MCP
Claude clients connect to the built-in MCP server at `POST /mcp` (streamable HTTP,
Bearer auth with an `fmcp_` key), served by `src/scribe/mcp/`. It is not a REST
Bearer auth with an `fmcp_` key), served by `src/grimoire/mcp/`. It is not a REST
surface — it exposes the same data as typed tools (`create_note`, `create_task`,
`start_planning`, `search`, `enter_project`, …) with
server-level usage guidance delivered in the MCP `instructions` block. See
Grimoire is one app container and one PostgreSQL container. There is no language model in the stack: Claude works with Grimoire from outside, through the MCP endpoint and the Claude Code plugin, and people use the web UI.
## Stack
| Layer | Technology | Notes |
|-------|-----------|-------|
| Frontend | Vue 3 + TypeScript + Vite + Pinia + Vue Router | SPA served from the same container as the API |
Browser ──────── Vue SPA ────────── /api/* ┘ └── in-process jobs and embedder
```
## Project Structure
- **The web UI** is a Vue single-page app, built into the image and served by Quart. Every unknown path that isn't under `/api/` falls back to `index.html`; `/api/*` always answers JSON.
- **The REST API** (`src/grimoire/routes/`) backs the UI, one blueprint per resource.
- **The MCP server** (`src/grimoire/mcp/`) is how Claude reads and writes records. Each module in `mcp/tools/` registers its tools, and the server's instructions and each tool's docstring are the contract an agent reads. Requests authenticate with an API key as a Bearer token.
- **The plugin** (`plugin/`) runs in the user's Claude Code. Its hooks call `/api/plugin/*` to put the right context in front of Claude at the right moment: the project at session start, matching rules and prior art before a prompt or a tool call, the rules mounted on a moment of work, and a check on the reply that ends a turn. Its skills describe the working practices. See [API Keys & MCP](api-keys-and-mcp.md) to install it.
- **Services** (`src/grimoire/services/`) hold the logic both doors share. Routes and MCP tools stay thin: parse the request, check access, call a service, return a plain dict.
```
scribe/
├── docker-compose.yml # Development stack
├── docker-compose.prod.yml # Production stack (Docker Swarm)
Most records are rows in one `notes` table, told apart by kind:
**Single container for frontend + API.** Quart serves the Vue.js production build as static files and exposes the REST API under `/api/`. The SPA is built by Vite during the Docker image build.
- **Notes**, and the specialised kinds that are notes underneath: **snippets** (recorded prior art, with a canonical code location), **lessons** (transferable insights, linked to the rules they instance) and **processes** (stored procedures, synced to Claude Code as skills).
- **Tasks**: a note with a status. A task's kind is `work`, `issue` or `spike`. Tasks carry work logs, sub-tasks, recurrence and a session claim.
**SPA routing via 404 handler.**`app.py` uses `@app.errorhandler(404)` (not a catch-all route) to serve static files or fall back to `index.html`. API routes (`/api/*`) always return JSON 404. This avoids a catch-all `/<path:path>` route intercepting API GETs.
Around them:
**Unified note/task model.** A task is just a note with task attributes enabled. `status IS NOT NULL` means it's a task. "Convert to task" sets `status='todo'`; "convert to note" clears `status`, `priority`, `due_date`. No separate table, no cascade complexity.
- **Projects** group work and carry a goal. **Milestones** are plans: a design body plus ordered step tasks.
- **Systems** are a project's named areas; records are tagged with them, and each maps to a canonical system shared across projects.
- **Rulebooks → topics → rules.** A rule in a rulebook is global; a rule on a project applies there only. Rules have a trigger (`when_to_apply`), versions, relations to other rules, and can be mounted on **moments** of work (`work.change`, `reply.report`, ...), which plugin actions reach through moment mappings.
- **Family canon**: ideas shared across projects on the same platform, and each project's answer to them (adopted, variant, owed).
- **Design systems**: tokens and guidance, with inheritance; the stylesheet is generated from them.
- **The shape ledger**: code shapes extracted from bound repos and their classification against snippets.
- **Embeddings** for notes (chunked), rules, milestones and systems, in pgvector columns.
- **Telemetry**: retrieval logs, note and rule usage, and the tuning history of retrieval thresholds.
- **Users, groups, shares and API keys.** Access is decided in the app (`services/access.py`): owners, direct shares and group shares, with read or write permission.
**First-class tag column.** Tags live in a `tags ARRAY[text]` column and are explicitly set by the client — not auto-extracted from body text. Hierarchical tags (`project/webapp`) supported via SQL `unnest + LIKE` prefix matching.
Deleted records go to the trash first and can be restored until retention purges them.
**Background generation architecture.** LLM streaming runs in a detached `asyncio.Task` that writes into an in-memory `GenerationBuffer`. SSE clients tail the buffer and can reconnect mid-stream without data loss. Buffer has a `cancel_event` for user-initiated stop. Completed buffers are cleaned up after 60s grace period. Periodic DB flushes every 5s preserve partial content. Both chat and AI Assist use this architecture.
## Retrieval
**SSE over WebSockets for LLM streaming.** SSE clients connect via `GET /api/chat/conversations/:id/generation/stream` with `Last-Event-ID` reconnection support. Frontend uses `fetch()` + `ReadableStream`.
Search and the plugin's context injection run on the same pipeline (`services/retrieval_pipeline.py`): embed the query, rank stored chunks by cosine similarity in pgvector, keep what clears the surface's threshold, and log what was shown. Each surface (MCP search, auto-inject, the pre-tool rule arm, the write-path prior-art arm, ...) is a spec over those stages with its own thresholds and budget. The thresholds are settings an admin can tune, and every change is recorded with its reason. When the embedder is unavailable, search falls back to keyword matching.
**Context building is server-side.** Backend fetches URL content and searches notes. Frontend sends the message text + optional context note IDs. `build_context()` returns `(messages, context_meta)`; metadata includes auto-found note IDs/titles sent to frontend via a `context` SSE event beforestreaming begins.
The embedding model downloads into `FASTEMBED_CACHE_DIR` (default `/data/fastembed-cache`) on first use, so the `/data` volume keeps it across restarts.
**No blocking long-running operations.** Any slow operation (model pulls, LLM calls, URL fetching) must never block app startup or freeze the UI. Backend uses SSE streaming for incremental responses. Model pulls stream NDJSON progress to the frontend.
## Background work
**SSRF protection.**`services/llm.py` blocks requests to loopback, private, link-local, reserved, and multicast addresses before fetching. `follow_redirects=False` prevents redirect-based bypasses.
All in-process; there is no separate worker container.
**Session cookie security.**`HttpOnly` and `SameSite=Lax` always set. `Secure` flag controlled by `SECURE_COOKIES` env var.
- At startup the app records any shipped default that moved in this release, starts its schedulers, and only then backfills missing embeddings, so the backfill never competes with startup for the database.
- Recurring tasks spawn every 15 minutes. Daily jobs: version auto-pinning (03:00 UTC), trash purge (03:30 UTC), targeted `VACUUM ANALYZE` (04:00 UTC by default). Scheduled backups check hourly whether one is due ([Backups](backups.md)).
- Log, notification and auth-token retention run on their own loops.
- A diagnostics heartbeat writes to `/data/diagnostics`.
**Rate limiting.** In-memory sliding-window rate limiter (`rate_limit.py`) applied to auth endpoints: login (10/60s), register (5/300s), forgot-password (5/300s), reset-password (10/60s). Keys are per-IP. `TRUST_PROXY_HEADERS` env var enables `X-Forwarded-For` / `X-Real-IP` when behind a reverse proxy.
**Idempotent migrations.** All Alembic migrations use raw SQL with `CREATE TABLE IF NOT EXISTS`, `CREATE INDEX IF NOT EXISTS`, and `DO $$ BEGIN CREATE TYPE ... EXCEPTION WHEN duplicate_object` to allow safe re-runs.
## Data Model
### Users
| Column | Type | Notes |
|--------|------|-------|
| `id` | int PK | |
| `username` | text UNIQUE NOT NULL | |
| `email` | text nullable | |
| `password_hash` | text nullable | NULL for OAuth-only accounts |
`project_shares`, `note_shares`: each has `shared_with_user_id` OR `shared_with_group_id` (exclusive), `permission` (`viewer`/`editor`/`admin`), `invited_by`.
`groups`, `group_memberships`: platform-wide groups with `member`/`owner` roles.
Permission resolution is centralised in `services/access.py`. `get_project_permission(uid, project_id)` checks ownership → direct share → group-based share → note→project inheritance, returning the highest applicable permission.
### Journal-Related Tables
`weather_cache`: per-user, per-`location_key` cache. Columns: `user_id`, `location_key` (`home`/`work`/etc.), `location_label`, `forecast_json` (Open-Meteo response), `previous_json` (last forecast, used to detect changes), `fetched_at`. Lat/lon are *not* stored on the cache row — they live in the user's `journal_config.locations.{home|work}` setting and are used at refresh time.
### API Keys
`api_keys`: `id`, `user_id` FK CASCADE, `prefix` (first 8 chars, displayed in UI), `key_hash` (SHA-256 of full key — full key never stored), `name`, `scope` (`read`/`write`), `created_at`, `last_used_at`.
| `services/task_logs.py` | Append/list/delete work log entries on tasks |
| `services/journal_prep.py` | Deterministic data gather (tasks/events/weather/projects/recent moments/open threads) → LLM prose opener; persisted as the first assistant message of today's journal Conversation |
| `services/journal_pipeline.py` | System-prompt builder for journal conversations; calls `build_profile_context()` so the LLM sees the user's profile + learned summary |
| `services/journal_scheduler.py` | APScheduler `BackgroundScheduler`; per-user prep job; live-reschedule via `update_user_schedule()`; catch-up logic for missed runs |
| `services/journal_search.py`, `services/moments.py` | Moment recording + search across journal history |
| `services/user_profile.py` | `build_profile_context()` consolidates profile + observations + learned_summary into a system-prompt block |
| `services/research.py` | SearXNG research pipeline: sub-queries → parallel fetch → outline → section synthesis → executive summary → index note with linked section notes |
| `services/events.py` | Internal events CRUD: `list_events`, `create_event`, `update_event`, `delete_event`, `get_event`; source of truth for all event LLM tools |
| `services/rss.py` | feedparser-based fetch, per-feed DB cache, prune-to-100 |
Every database wait on startup has a deadline. Migrations wait for the database to accept connections (up to five minutes) before they run, so the app starts cleanly whatever order the containers come up in.
## Authentication
**Session cookies** —`HttpOnly`, `SameSite=Lax`, optionally `Secure` (`SECURE_COOKIES` env var). Session includes `session_version`; mismatch with DB value (after password change) results in 401 and session clear.
-**Sessions**:`HttpOnly`, `SameSite=Lax` cookies, `Secure` on a request the trusted proxy reports as HTTPS, which also gets HSTS (`grimoire/proxy.py`). Changing a password bumps the user's session version and signs out their other sessions.
- **API keys**: `Authorization: Bearer fmcp_...`, stored hashed. A `read` key may only call read endpoints and read MCP tools.
- **OIDC** single sign-on with PKCE, optionally replacing local logins ([SSO / OAuth](sso-oauth.md)).
- **Rate limits** on register, login and forgot-password, per client IP, read from `X-Forwarded-For` by `TRUSTED_PROXY_HOPS`.
**Bearer token / API keys** — `Authorization: Bearer <key>` accepted by `_check_auth()` as an alternative to session cookies. The raw key is SHA-256 hashed and looked up via `services/api_keys.py`. Keys with `scope=read` are rejected on non-safe methods (`POST`/`PATCH`/`DELETE`). Used by Fable MCP and any external API consumers.
Credentials Grimoire stores for outside services (forge tokens, the backup destination's S3 secret) are sealed under a key derived from `SECRET_KEY`, which the instance makes on first start and keeps in `/data/secret_key` unless one is set.
**Local auth + OIDC/OAuth (PKCE)** — `services/oauth.py` handles discovery and `find_or_create_oauth_user`. On OAuth login: checks existing `oauth_sub` → matching email → creates new user.
## Repository layout
See [sso-oauth.md](sso-oauth.md) for provider-specific setup instructions.
## LLM Pipeline Internals
### Tool Routing
No separate intent router — the main model handles all tool routing directly via Ollama's structured tool-calling output. The model receives the full tool schema list and decides whether to call a tool or respond conversationally. Extended reasoning (`think=True`) is always on for qwen3-class models: content-based gating was tried but exposed tool-call template fragility on short tool-intent prompts.
### Tool Loop
Multi-round tool loop (max 5 rounds). All implementations in `services/tools/` (decorator-based registry); `execute_tool(user_id, tool_name, arguments, conv_id=None, workspace_project_id=None)` is the dispatcher. `conv_id` and `workspace_project_id` are threaded in from `run_generation()` so tools like `set_rag_scope` can write to the current conversation.
**Unified `create_note` tool** — creates both notes and tasks. Setting `status` (e.g. `"todo"`) creates a task; omitting it creates a knowledge note. All task fields (due_date, priority, milestone, parent_task, recurrence_rule) are available on the single tool.
**Duplicate protection on `create_note`:**
1. Exact title match (case-insensitive) → hard block, redirect to `update_note`
2. Fuzzy title match (SequenceMatcher ≥ 82%; punctuation stripped before candidate search) → hard block
3. Semantic content similarity (threshold 0.90, body ≥ 80 chars) → soft block with `requires_confirmation: true`
**Project resolution** (`_helpers.resolve_project`): 4-step lookup — (1) exact DB match, (2) `query in title` substring, (3) `title in query` reverse substring, (4) SequenceMatcher ≥ 0.55.
### Context Window and Summarisation
`OLLAMA_NUM_CTX` (default 16384) controls the context window for all generation calls. Intent classification always uses `num_ctx=4096` to reduce VRAM pressure.
History summarisation threshold: 30 messages. Keeps 8 recent messages. Summary max 400 tokens.
### Web Research Pipeline
`services/research.py` implements a full autonomous research pipeline:
1. Intent model generates 5 focused sub-queries
2. All 5 SearXNG queries run in parallel (200ms stagger to avoid rate limiter)
3. Up to 15 unique URLs fetched in parallel
4. LLM generates an outline (2–8 sections with title + focus)
5. Each section synthesised in parallel from relevant sources
6. Executive summary generated from all section content
7. Index note created with executive summary + links to section notes; section notes linked back via `parent_id`
8. Falls back to single-note synthesis if outline generation fails
SearXNG tip: add the app server IP to `botdetection.ip_lists.pass_ip` in SearXNG `settings.yml` to bypass the rate limiter for trusted backend requests.
### Image Cache
`search_images` tool fetches images server-side via SearXNG, stores them on disk (SHA-256 dedup, content-type validation, 5 MB cap), and serves from `/api/images/<id>`. The user's browser never contacts the original image host.
1.`semantic_search_notes()` — cosine similarity via pgvector, threshold configurable per call; accepts `orphan_only` and `project_id` scope flags.
2. Notes ≥ 0.60 similarity auto-injected into system prompt (up to 3, 800 chars each).
3. Notes 0.45–0.60 surfaced in chat sidebar as "Suggested" (user clicks to include).
4. Explicitly included notes delivered full-body.
5.`excluded_note_ids` prevents the current note from being injected as its own context.
### RAG Scope (three-value system)
`conversations.rag_project_id` controls which notes are eligible for retrieval:
| Value | Behaviour |
|-------|-----------|
| `NULL` (default) | Orphan notes only — notes with `project_id IS NULL` |
| `-1` | All notes — opt-in to global search |
| Positive int | That project's notes only |
`build_context()` derives `orphan_only` + `effective_project_id` from this value before calling both the semantic and keyword search paths.
**Scope tools** — Two LLM tools let the model discover and switch scope mid-conversation:
-`search_projects` — SequenceMatcher scoring over title + description + `auto_summary`; returns top 5 matching projects.
-`set_rag_scope` — persists the new `rag_project_id` to DB immediately; blocked in workspace view; causes the SSE `done` event to include `new_rag_scope` + `new_rag_scope_label` so the frontend chip updates reactively.
**Project summaries** — `generate_project_summary()` calls Ollama (fire-and-forget) and stores the result in `projects.auto_summary`. Triggered on project update and note saves (debounced 1h). `backfill_project_summaries()` runs at startup.
Embedding model: `nomic-embed-text` via Ollama. Backfill runs 30s after startup (background task).
```
├── alembic/versions/ database migrations
├── src/grimoire/
│ ├── app.py app factory, blueprints, startup and shutdown
│ ├── config.py environment configuration
│ ├── models/ SQLAlchemy models
│ ├── routes/ REST blueprints
│ ├── mcp/ MCP server and its tool modules
│ ├── services/ shared logic
│ └── restore.py `python -m grimoire.restore`: the restore, from the command line
├── frontend/src/ views, components, stores, composables, api client
├── plugin/ Claude Code hooks, skills and commands
├── scripts/ CI checks and the plugin version minter
└── tests/ unit tests, plus `-m integration` against real Postgres
Configuration is via environment variables. The `docker-compose.yml` file sets defaults for local development; override them in a `.env` file (gitignored).
Grimoire reads its configuration from environment variables. Most settings people change day to day (registration, SMTP, retrieval thresholds, backups, forge connections) live in **Settings** in the web UI instead.
## Environment Variables
Any variable marked *secret* can also be read from a file: set `<NAME>_FILE` to the path, for example a Docker secret at `/run/secrets/grimoire_key`.
| `SECURE_COOKIES` | `false` | Set `true` when running behind TLS |
| `BASE_URL` | — | Public URL (e.g. `https://notes.example.com`) — required for OIDC redirect URIs and email links |
| `DATABASE_URL`*(secret)* | `postgresql+asyncpg://grimoire:grimoire@localhost:5432/grimoire` | PostgreSQL connection string, using the `asyncpg` driver |
| `SECRET_KEY`*(secret)* | kept in `DATA_DIR` | Signs sessions and seals stored credentials. Unset, the instance makes its own on first start and keeps it in `DATA_DIR/secret_key`. Set it only to manage the key yourself; it wins over the kept one (see below) |
| `DATA_DIR` | `/data` | Where the instance keeps its own `SECRET_KEY`. Mount a volume here |
| `BASE_URL` | `http://localhost:5000` | Public URL, used in email links and OIDC redirects |
| `TRUSTED_PROXY_HOPS` | `1` | The starting count of reverse proxies in front of Grimoire. **Settings → Config → Reverse proxy** is where the count is checked and set, and a count saved there wins over this. The client address and the scheme are read that many entries in from the right of `X-Forwarded-For` and `X-Forwarded-Proto`; `0` ignores both headers. See [Behind a reverse proxy](#behind-a-reverse-proxy) |
See [sso-oauth.md](sso-oauth.md) for provider-specific setup.
SSO turns on when the issuer, client ID and secret are all set. See [SSO / OAuth](sso-oauth.md).
### Security / Proxy
### Email
Admins can set SMTP in **Settings**; these are the defaults until they do.
| Variable | Default |
|----------|---------|
| `SMTP_HOST` | — |
| `SMTP_PORT` | `587` |
| `SMTP_USERNAME` | — |
| `SMTP_PASSWORD`*(secret)* | — |
| `SMTP_FROM_ADDRESS` | — |
| `SMTP_FROM_NAME` | `Fabled Grimoire` |
| `SMTP_USE_TLS` | `true` |
### Integrations
| Variable | Default | Description |
|----------|---------|-------------|
| `TRUST_PROXY_HEADERS` | `false` | Set `true` when behind a trusted reverse proxy to read real IP from `X-Forwarded-For` / `X-Real-IP` |
| `PLUGIN_MARKETPLACE_URL` | this repo's git URL | The git URL shown in the plugin install command under Settings → MCP Access. An admin can override it in Settings |
| `FORGE_KIND`, `FORGE_BASE_URL`, `FORGE_TOKEN`*(secret)* | — | A git forge connection for admins' projects, so a token can live in a Docker secret. Each user can add their own under Settings → Integrations; a stored connection for the same host wins |
| `FORGE_WEBHOOK_SECRET`*(secret)* | — | Verifies push webhooks from the forge |
| `SEARXNG_URL` | — | A SearXNG instance, listed as the web-search backend under Settings → Integrations |
### Web Search / Images
## Running it
| Variable | Default | Description |
|----------|---------|-------------|
| `SEARXNG_URL` | — | SearXNG base URL for web search and image search tools |
| `IMAGE_MAX_BYTES` | `5242880` (5 MB) | Maximum size for cached images |
### Docker Compose
### Data / Logging
| Variable | Default | Description |
|----------|---------|-------------|
| `LOG_RETENTION_DAYS` | `90` | Days to keep app logs before automatic pruning |
| `DATA_DIR` | `/data` | Root directory for persistent data (VAPID keys, backups) |
## Docker Compose Setup
### Development (`docker-compose.yml`)
[`docker-compose.yml`](../docker-compose.yml) runs the published image with its own database, and needs nothing set:
```bash
# Copy the example env file
cp .env.example .env
# Edit .env to set a real SECRET_KEY
# Start the stack
docker compose up --build
# App available at http://localhost:5000
docker compose up -d
```
The first user to register becomes admin. Registration auto-closes after that (re-enable from Settings → Users).
Open `http://localhost:5000`. The first account registered becomes the admin; after that, registration stays closed until an admin opens it in Settings or sends an invite.
### Production (`docker-compose.prod.yml`)
Update with `dockercompose pull && docker compose up -d`. The key and the data stay in their volumes.
The production compose file adds:
- Docker Secrets for `SECRET_KEY_FILE` and `DATABASE_URL_FILE`
- Network isolation (internal bridge for DB + Ollama; only the app is externally accessible)
docker compose -f docker-compose.dev.yml up --build
```
## Production Deployment
### Docker Swarm
### Reverse Proxy (Required)
[`docker-compose.prod.yml`](../docker-compose.prod.yml) is a stack file: the app and database on a private overlay network, trusting one proxy (`TRUSTED_PROXY_HOPS: "1"`). A Swarm can move the app to another node, away from a node-local `/data` volume, so the stack sets its own key: provide `DB_PASSWORD` and `SECRET_KEY` in the environment (or mount a Docker secret and set `SECRET_KEY_FILE`):
Fabled Assistant does **not** handle SSL/TLS. Run it behind a reverse proxy:
- **Nginx**, **Traefik**, or **Caddy** in front of the app container
- Terminate TLS at the proxy; forward to port 5000
- **Do not expose port 5000 directly to the internet**
- Rate-limit auth endpoints: ≤ 5 req/min per IP on `/api/auth/login` and `/api/auth/register`
The database user, database and volume default to `grimoire`. An install created under the name Scribe keeps its data by setting `POSTGRES_USER`, `POSTGRES_DB` and `PGDATA_VOLUME` to the names it was created with.
The shipped files leave out healthchecks and restart policies on purpose. The app waits for its database on startup, and the orchestrator's defaults restart it if it ever exits.
- **App data** (`/data`): the instance's own `SECRET_KEY`, the embedding model cache and diagnostics. Losing it costs the key: Settings then asks for saved forge tokens and the backup secret again, and the model downloads again.
Back up the database with **Settings → Data → Scheduled backups** ([Backups](backups.md)). A database backup doesn't carry the key; to move sealed credentials to a new instance, copy `/data/secret_key` with it.
## Behind a reverse proxy
Grimoire doesn't terminate TLS. Put it behind Nginx, Traefik or Caddy, forward to port 5000, and keep port 5000 off the internet. Set `BASE_URL`, then set the proxy count to the number of proxies a request passes through: one proxy is `1`, the default; Cloudflare's proxied DNS in front of your own proxy is `2`.
**Settings → Config → Reverse proxy** sets the count and checks it against your own request. It shows the address Grimoire reads you as, and what each count from 0 to 3 would read. Pick the count that shows your own address. When a request comes through Cloudflare, the section names the count that reads the address Cloudflare reports, with a button that sets it. `TRUSTED_PROXY_HOPS` in the environment is only the starting value: a count saved in Settings wins, and **Use the environment's** goes back to it.
A count that is too low makes every visitor read as the proxy, so they share one sign-in rate limit and a few bad sign-ins block everyone. The "too many attempts" message says so and points here. The block on sign-ins lifts after a minute; an admin who is already signed in can fix the count at once.
If using `TRUST_PROXY_HEADERS=true`, ensure your proxy strips any client-supplied `X-Forwarded-For` headers before adding its own.
Each proxy appends its entry to `X-Forwarded-For` and `X-Forwarded-Proto`, so Grimoire reads the entry that many in from the right: the one your own proxy wrote. Whatever a client puts to the left of it is ignored. A header with fewer entries than that (a LAN road with one proxy beside a Cloudflare road with two) is read at its leftmost entry, the one the first proxy wrote.
### Security Checklist
When that entry says `https`, the session cookie is marked `Secure` and the response carries `Strict-Transport-Security: max-age=31536000`. Over plain HTTP neither is sent, and Grimoire never redirects to HTTPS, so a LAN install keeps working.
- **`SECURE_COOKIES=true`** — must be set when running behind TLS.
- **`BASE_URL`** — set to your public URL; required for OIDC redirects and email links.
- **Registration** — auto-closes after the first user (admin). Re-enable from Settings → Users or send invite links.
- **Session invalidation** — changing or resetting a password bumps `session_version`, evicting all other active sessions. Button also available in Settings → Account.
- **Keep Ollama on an internal network** — both compose files keep Ollama off the host network. Never expose the Ollama port publicly.
- **Default SECRET_KEY warning** — the app logs a `WARNING` on startup if the default dev key is in use.
A hop count higher than the real number of proxies lets a client choose its own address, so count them. With `0`, the headers are ignored and the socket peer is the client: use it when clients reach port 5000 directly.
## Model Management
## Security checklist
Models are managed through Settings → General → Model Management in the web UI. You can:
- Pull new models by name (streams progress via SSE)
- View installed models with size and loaded/unloaded state
- Delete unused models
The app auto-warms user-preferred models on startup (only models already installed — never auto-pulls). The embedding model (`nomic-embed-text`) is auto-pulled on startup if missing.
- **`llama3.1:8b`** — reliable baseline, widely tested with tools
- **Keep the key.** Forge tokens and the backup destination's S3 secret are sealed under a key derived from `SECRET_KEY`, so a copy of the database alone can't read them. Unset, the instance makes a random key and keeps it in `/data/secret_key`, readable by its owner only; keep that volume. If the key changes (say, a restore onto an instance with a different key), Settings marks those credentials and asks for them again; nothing else breaks. The app warns at startup when `SECRET_KEY` is a public placeholder or `/data` is not a mounted volume, and refuses to start with a placeholder key (`dev-secret-change-me`, `change-me-in-production`).
- **The proxy count** (Settings → Config → Reverse proxy) matches the number of proxies in front of Grimoire, so rate limits and the audit log see the real client and the cookie is `Secure` over HTTPS. The section shows whether it does.
- **Registration** is open only for the first account. Invite people, or open it briefly from Settings.
- **Signing everyone else out**: changing a password ends that user's other sessions; Settings → Account has a button for it too.
- **API keys** are per user, shown once, and can be read-only. Revoke any you no longer use under Settings → MCP Access.
**Critical rule:** Action button colors are universal across all apps. A Save button in Scribe and a Save button in Minstrel look identical. Per-app accents do not appear on buttons.
**Critical rule:** Action button colors are universal across all apps. A Save button in Grimoire and a Save button in Minstrel look identical. Per-app accents do not appear on buttons.
### Semantic colors
@@ -71,7 +71,7 @@ Parchment is intentionally not pure white — it's slightly warm to feel like ag
@@ -80,7 +80,7 @@ Parchment is intentionally not pure white — it's slightly warm to feel like ag
**Accent usage rules:**
- The accent appears on the app's wordmark and icon.
- The accent indicates active/current state in nav (the selected page, the active tab).
- The accent is the cursor color and text-selection color in long-form surfaces (Scribe notes, Forge story drafts).
- The accent is the cursor color and text-selection color in long-form surfaces (Grimoire notes, Forge story drafts).
- The accent does NOT appear on primary or secondary action buttons.
- The accent does NOT appear in body text or chrome.
- One accent per app. Don't mix accents within a single app.
@@ -101,6 +101,8 @@ All text-on-surface combinations meet WCAG AA at minimum. Parchment on Obsidian
| Inter | Body, UI, labels | Google Fonts |
| JetBrains Mono | Code, terminal output, monospaced data | Google Fonts |
All three are served by the app itself, from `frontend/src/assets/fonts/`, with every subset Google Fonts publishes for them. A page never asks a font host for anything, so an instance with no internet still draws in its own type. They are under the SIL Open Font License; the licences sit beside the files.
### Why this pairing
Fraunces is a contemporary serif with personality — it has the warmth and authority of a book serif without feeling like costume. It signals "this is considered" without signaling "this is a fantasy product." Inter is the workhorse — neutral, ubiquitous, designed for screens, doesn't compete with the serif. JetBrains Mono is the natural choice for any developer-adjacent product and supports ligatures.
@@ -124,7 +126,7 @@ Fraunces is a contemporary serif with personality — it has the warmth and auth
- **Sentence case everywhere.** Never Title Case for headings, never ALL CAPS except for the Tiny micro-label style.
- **Two weights only:** 400 regular and 500 medium. Never 600 or 700 — they read heavy in dark mode.
- **Fraunces only at 18px and above.** Below that it loses too much detail and feels fragile. For h3 and below, use Inter.
- **Line height** 1.5 for body, 1.3 for headings, 1.7 for long-form reading surfaces (Scribe notes, Forge drafts).
- **Line height** 1.5 for body, 1.3 for headings, 1.7 for long-form reading surfaces (Grimoire notes, Forge drafts).
- **Letter-spacing** at default for everything except the Tiny micro-label, which gets `0.08em` letter-spacing and uppercase styling.
---
@@ -173,7 +175,7 @@ Padding: `8px 16px` for default, `6px 12px` for compact, `10px 20px` for promine
Used for tags, hashtags, code spans, status badges. Background is the accent color at ~15% opacity, text is the accent at full strength. Border-radius 4px, padding `2px 8px`, Inter 11px/500.
In Scribe specifically, hashtags and tags use the dusty violet accent. In Minstrel, they'd use forest teal. The pattern is shared; the color follows the app.
In Grimoire specifically, hashtags and tags use the dusty violet accent. In Minstrel, they'd use forest teal. The pattern is shared; the color follows the app.
### Cards
@@ -272,9 +274,9 @@ Use **Lucide** (https://lucide.dev) as the base icon set — it matches this sty
## Per-app application
### Fabled Scribe (#5B4A8A — dusty violet)
### Fabled Grimoire (#5B4A8A — dusty violet)
A second-brain notes and task management tool. The accent appears in: the wordmark, hashtags and tag pills, the active nav item, text selection color, and the cursor in the editor. Notes are presented on Iron-surfaced cards with generous reading line-height. The hashtag system uses Scribe's accent for visual continuity.
A second-brain notes and task management tool. The accent appears in: the wordmark, hashtags and tag pills, the active nav item, text selection color, and the cursor in the editor. Notes are presented on Iron-surfaced cards with generous reading line-height. The hashtag system uses Grimoire's accent for visual continuity.
### Minstrel (#4A6B5C — forest teal)
@@ -332,7 +334,7 @@ Home server management. The accent appears on: the wordmark, healthy/online stat
}
/* Per-app accent — set ONE of these on the root for each app */
[data-app="scribe"]{--fs-accent:#5B4A8A;}
[data-app="grimoire"]{--fs-accent:#5B4A8A;}
[data-app="minstrel"]{--fs-accent:#4A6B5C;}
[data-app="forge"]{--fs-accent:#8B5A2B;}
[data-app="roundtable"]{--fs-accent:#4A5D7E;}
@@ -354,220 +356,13 @@ If using Tailwind, extend the theme with these tokens rather than relying on def
---
*Last updated: April 25, 2026. Iterate as the family of apps grows.*
# Grimoire
---
The design system Grimoire renders with is stored in Grimoire itself: **Grimoire**, which inherits **FabledSword**, edited at `/design-systems`. `frontend/src/assets/theme.css` is generated from it, and CI rejects a `var()` the sheet doesn't define. Where this page and the stored system differ, the stored system is right. Grimoire applies the baseline above with these choices:
# Scribe-specific decisions in progress
> This section tracks decisions made while adapting the FabledSword baseline above for Scribe specifically. Items here are *in progress* — once they feel solid, they get folded into the main body of the document (either as Scribe-specific extensions in the per-app section, or as updates to the universal rules where Scribe's needs reveal a gap in the baseline).
*Iteration started: 2026-04-26. Foundation pass shipped 2026-04-27 in `7a9a8b7` (palette, fonts, light mode, action tokens, hardcoded indigo cleanup, warm-gold deprecation). Surface phase shipped 2026-04-27 across `93a3beb` → `3c1ec40` (Lucide migration, Hybrid-rule button reclassification per surface, long-form line-height, two-weights-only). The system is now applied end-to-end; this section will fold into the main body once the result has had time to settle in real use.*
## Decisions made so far
### Accent footprint — Hybrid rule (not Strict)
The doc baseline says the per-app accent only appears on wordmark, active nav, cursor, and text selection — never on action buttons. Scribe currently uses indigo on essentially every interactive surface (CTAs, scrollbars, glows, borders, focus rings). Hard-cutting to the doc baseline would lose too much identity in one swing.
**Hybrid rule:** the accent reserves a slightly larger footprint than the doc baseline, but still much smaller than today.
- **Accent (dusty violet) lives on:** wordmark; active nav; cursor and text selection in editor surfaces; tags/pills/wikilinks; in-progress task badge; focus rings; **brand-moment CTAs** — chat Send, "Create note" empty-state CTA, journal Send, "Start journaling" empty-state.
- **Moss (sage-green primary) lives on:** Save / Submit / Confirm in forms and modals; generic affirmative actions where the button just means "do this thing" with no brand pretense.
- **Bronze (secondary):** Cancel-but-not-destructive, alternative paths.
- **Oxblood (destructive):** Delete / Remove (paired with an icon).
- **Pewter ghost:** tertiary actions, "later", "skip", "see also".
**Rule of thumb:** if the user is engaging with a *Scribe-feature moment* (sending a chat, opening a fresh note, jumping into the journal), accent. If they're just *operating the software* (saving an edit, confirming a dialog), Moss.
The doc is dark-only. Scribe today supports both light and dark, and we keep both. The light mode is derived to *match* the dark mode aesthetic rather than defaulting to system white-and-ink.
- Page background: in the `#F5F1E8` warm cream family (specific values TBD)
- Cards: near-white but slightly tinted
- Text: deep ink `#14171A` (mirroring Obsidian)
- Accent: same dusty violet `#5B4A8A` (works on both themes)
The metaphor stays consistent across themes: ink on aged paper (light) ↔ parchment text on graphite (dark). Light mode is *not* the system standard look.
**Known downside:** warm parchment backgrounds can fight with embedded color content. Mitigation: code blocks get a slight cool wash in light mode specifically, to keep syntax highlighting readable.
### Status and priority palette — extend the doc's semantic set
The doc's semantic colors (Success / Warning / Error / Info / Destructive) are leaner than what Scribe needs for task management. Rather than running a parallel palette, Scribe extends the doc by mapping its status/priority tokens onto doc primitives where they fit and defining new app-level tokens for the rest.
| `priority-none` | Vellum/Ash | doc | No signal |
The priority row reads as a clean cool→warm gradient (slate blue → golden brown → terracotta), which matches the semantic loudness — coherence the current ad-hoc palette doesn't have.
**Other functional tokens:**
| Token | Color | Logic |
|---|---|---|
| `wikilink` | dusty violet | Editorial brand moment per Hybrid |
| `overdue` | Error `#C04A1F` | Same as priority-high — overdue IS a priority signal |
| `toast-success` | Moss | doc semantic |
| `toast-error` | Error | doc semantic |
| `toast-info` | Info | doc semantic |
| `tag-bg` / `tag-text` | accent at 15% / accent | Per doc pill recipe |
Each token gets a `*-bg` companion at low alpha (matching the existing pattern in `theme.css`).
**Removed:** the warm gold accent (`--color-accent-warm: #b8860b`). Its two jobs split:
- Dates and timestamps (knowledge cards, event details, chat) → use `text-secondary` instead. Dates are metadata, not a brand surface; muted is the correct register.
- Paused project status → use the new `status-paused` (Warning `#8B6F1E`) row above. Same golden-brown family, semantically aligned.
### Typography — adopt the doc's stack and scale
Adopt the doc's type stack and scale verbatim, with one deferred verification (long-form line-height in practice).
- **Body font: Inter.** Replaces Scribe's current system-stack body font. Doc-defined; no Scribe-specific divergence.
- **Type scale:** as in the doc table — Display 40 / H1 32 / H2 24 / H3 18 / Body 15 / Body small 13 / Label 12 / Code 13 / Tiny 11.
- **Two weights only:** 400 regular, 500 medium. No 600/700 (reads heavy in dark mode and against the muted palette).
- **Family rules:** Fraunces at 18px+ only (Display, H1, H2). H3 and below = Inter. Code = JetBrains Mono.
**Code-block exception:** in light mode specifically, code blocks use a slight cool wash (e.g. `#EBEDF0`) instead of the warm inset bg, so syntax highlighting reads cleanly. This is the mitigation for the "warm bg fights colored content" downside.
The accent (`#5B4A8A` dusty violet), Moss, Bronze, Oxblood, and the semantic color set are **identical across themes** — only the surface and text palettes flip.
### Chat-bubble codification — keep the Illuminated Transcript pattern
The existing chat-bubble pattern (informally called "Illuminated Transcript") gets written into the design system as a documented chat component. Other apps in the family that add a chat surface inherit the pattern; Scribe's existing implementation continues to work with only color shifts.
**User bubble (whisper):**
- Background: transparent
- Border: 0.5px Pewter (was: indigo-tinted)
- Text color: secondary (Vellum dark / `#5A5852` light)
- Background: card surface (Iron dark / `#FBF8F0` light)
- Border: none on top/right/bottom; **2px solid accent (dusty violet) on left edge only**
- Box-shadow: accent-tinted glow + standard depth shadow (formula: `0 4px 28px rgba(<accent>, 0.14), 0 2px 8px rgba(0,0,0,0.4)` in dark; lower alphas in light)
- Text color: primary (Parchment dark / Obsidian-inverted light)
- Left-aligned, rounded except bottom-left
The 2px-accent left edge is the "illumination" — like an illuminated capital in a manuscript. The shadow is the lift. Together they make the assistant bubble read as the *primary* voice, while the user bubble is the *margin note*.
**Inline tool-call cards (`ToolCallCard`)** rendered inside an assistant bubble do NOT get their own border (per the border philosophy — the bubble already contains them). They use a slight surface tint to differentiate.
### Iconography — adopt Lucide, enforce a scale
Scribe currently hand-inlines SVG paths in 16+ Vue files, with 5 different stroke weights and 8+ different sizes. The visual style is already outline + rounded caps + `currentColor` stroke (matches the doc's intent), but there's no shared source and no scale discipline.
**Migration policy:**
1.**Install `lucide-vue-next`** as the icon source. Replace hand-inlined SVGs with imported components. Single source of truth.
2.**Strict size scale: 16px and 24px only.** Today's mix of 12/13/14/15/17/18/20 collapses to those two. 16 for inline-with-text and small affordances; 24 for nav and primary actions.
3.**Stroke weight per the doc: 1.5 at 24px, 1 at 16px.** Lighter than the current default of 2 — reads more refined, matches the muted palette philosophy. Overrides Lucide's default.
4.**Outline by default; filled only for active/selected state.** Introduces a new affordance Scribe doesn't currently use — bookmark/pin/star icons can switch outline → filled to indicate active state. Reserve filled style strictly for this.
5.**No emoji in chrome.** Replace the 3 files' emoji usage in UI labels/buttons/badges/empty states with Lucide equivalents. Emoji remain fine in *user content* (note bodies, chat messages the user typed).
Work cost: ~30-60 individual icon swaps across the 16 files. Mechanical; doesn't require redesign of any component.
### Voice and tone — adopt principles, defer formal audit
The doc's voice register applies to Scribe (understated mythic — plain for functional UI, flavored for empty/error/loading states). No formal sweep of every UI string yet.
**Approach:** apply the voice opportunistically as components are touched in the polish pass — when redesigning a settings tab, an empty state, or an error toast, rewrite the copy at the same time using the doc's register and examples table as the guide. A standalone audit pass is deferred unless drift becomes visible.
### Border philosophy — structural, not decorative
The doc treats borders as *structural* (Pewter neutral hairlines that say "boundary"), not decorative (Scribe today uses indigo-tinted borders that say "branded edge"). That principle suggests removing borders in places where surface tint and spacing already communicate separation.
**Borders to remove:**
- List rows (NotesListView, TasksListView, conversation history) — surface contrast + spacing should separate rows; current border reads as "boxed-in"
- Inline `ToolCallCard` inside chat bubbles — the bubble is already a container; an extra border feels like double-wrapping
- Filter chips and search-bar pills with a background tint — background does the work
- Empty-state callouts with dashed/bordered "nothing here yet" boxes — tinted background reads cleaner
**Borders to keep (genuinely structural):**
- Standalone card containers (Notes viewer, Task viewer, the new daily prep card)
- Modal / dialog edges
- Code blocks (separates content type, not just space)
- Focus rings (accessibility)
- Major section dividers within a panel
Border weight is not load-bearing for Scribe — happy to use the doc's 0.5px hairline default; the *placement* discipline matters more than the weight.
## Open threads (next iterations)
### Foundation pass — shipped 2026-04-27 (`7a9a8b7`)
Mechanical token + font + light-mode rewrite of `frontend/src/assets/theme.css`, plus a sweep of hardcoded indigo and `--color-accent-warm` references across ~14 component files. Action tokens (`--color-action-primary` Moss, `--color-action-secondary` Bronze, `--color-action-destructive` Oxblood, `--color-action-ghost-border` Pewter) are defined but not yet applied — buttons still flow through `--color-primary` and read as dusty-violet gradients in the meantime, by design. Spec lives at `docs/superpowers/specs/2026-04-27-design-system-polish-foundation-design.md` (gitignored, local-only).
Bundled as Hybrid (option C from the brainstorm): Lucide cross-cutting first, then surface-by-surface for the judgment work. Spec lives at `docs/superpowers/specs/2026-04-27-design-system-polish-surface-design.md` (gitignored, local-only). Seven PRs landed on `dev`:
| PR | Commit | Surface | Notes |
|---|---|---|---|
| 1 | `93a3beb` | Lucide cross-cutting | 60 hand-inlined SVGs across 15 files → `lucide-vue-next`. Every chrome icon at 16 or 24. Emoji-as-icons (`✕`, `✓`, `🎤`, `📎`, `↻`, `↑`, `×`, `☀`/`☾`) swept across the chrome. AppLogo wordmark and the GraphView D3 mount kept as the legitimate exceptions. |
**Cross-cutting changes folded in as the work touched files:**
- Long-form 1.7 line-height on `.prose` (PR 4) — applies to Note viewer, Task viewer, chat assistant bubbles, anywhere markdown renders into a reading surface.
- Two-weights-only (400 + 500) — every `font-weight: 600` and `700` snapped to `500` across all surface PRs.
- Hardcoded `--color-danger` in destructive button contexts → `--color-action-destructive` (Oxblood). `--color-danger` (Error terracotta) preserved for validation/error messages, per the doc's distinction between Error and Destructive.
- Adjacent `×` / `✕` / unicode-arrow emoji swept opportunistically as files were touched (PRs 5, 7).
### Out of scope — deferred indefinitely
Items deliberately not addressed in this round; revisit when a real need surfaces:
- Lucide stroke-weight overrides (doc spec: 1.5 at 24, 1 at 16; current: Lucide default 2). Touched components if they read too heavy in practice.
- Filled-as-active icon state — no current affordance uses it; introduce when bookmark/pin/star toggles are added.
- Token rename to `--fs-*` namespace — Scribe is the only FabledSword app sharing this codebase.
- FabledSword lockup placement — waiting on the actual heraldic mark to be drawn.
- Standalone voice/tone audit across every UI string — opportunistic-only; full sweep deferred unless drift becomes visible.
- A handful of editor utility buttons (`.btn-suggest-tags`, `.btn-link-all`, AI assist generate/proofread/accept/reject set, etc.) — currently ghost-styled and visually compliant; revisited only if they read off in practice.
### Open threads
*New threads will accumulate here as gaps surface in real use.*
- **Accent footprint.** Dusty violet marks Grimoire's own moments: the wordmark, active navigation, the editor's cursor and selection, tags and wikilinks, in-progress status, focus rings, and the call to action that starts something new (a first note, say). Operating the software uses the action colours: Moss to save or confirm, Bronze for alternatives, Oxblood with an icon to delete, Pewter ghost for tertiary actions.
- **Light and dark.** Both are supported. Light mode is warm parchment (`#F5F1E8` page, `#FBF8F0` cards, `#14171A` text) rather than white, with a cool wash on code blocks so syntax colours stay readable. The accent, action and semantic colours are the same in both.
- **Status and priority.** Status: todo Pewter, in progress the accent, done Moss, cancelled Ash, paused Warning. Priority runs cool to warm: low Info, medium Warning, high Error. Overdue uses Error. Each has a low-alpha `-bg` companion.
- **Type.** Inter for UI and body, Fraunces at 18px and up (display, H1, H2), JetBrains Mono for code. Two weights only, 400 and 500. Line height 1.5 for body, 1.3 for headings, 1.7 on reading surfaces such as a note.
- **Icons.** Lucide, at 16px inline and 24px for navigation and primary actions; no emoji in the interface.
- **Borders are structural.** Cards, dialogs, code blocks, focus rings and major dividers get borders; list rows, chips and empty states are separated by surface tint and spacing instead.
All development is Docker-based. Do not install Python or Node dependencies locally.
## Running it
```bash
# Start the full stack (app + PostgreSQL + Ollama)
docker compose up --build
docker compose -f docker-compose.dev.yml up --build # app + PostgreSQL at http://localhost:5000
docker compose -f docker-compose.dev.yml up --build app # rebuild after a change
docker compose -f docker-compose.dev.yml down -v # stop and wipe the database
```
# Rebuild after backend changes (frontend changes require rebuild too)
docker compose up --build app
The image builds the frontend, so a frontend change needs a rebuild too. For quicker frontend work, run Vite against the running app; it proxies `/api` to port 5000:
# Reset everything (wipes database)
docker compose down -v && docker compose up --build
```bash
cd frontend && npm ci && npm run dev # http://localhost:5173
```
# Run checks (lint, format, typecheck, tests)
make check
## Checks
# Individual checks
CI runs these on every push; locally they need `uv sync --extra dev` and `cd frontend && npm ci`.
```bash
make lint # ruff check src/
make fmt # ruff format src/
make typecheck # vue-tsc --noEmit
make test# pytest tests/
make check # all three
```
## Frontend Hot Reload
Unit tests run without a database. Tests marked `integration` need a real PostgreSQL and run in CI's integration job, against a schema built by the real migrations.
The Docker setup does not include Vite's hot-reload dev server. After frontend changes, rebuild the image. For faster iteration during active frontend work, you can run Vite locally:
## Database migrations
The container runs `alembic upgrade head` on every start, after waiting for the database to accept connections.
```bash
cd frontend
npm install
npm run dev # Vite dev server at http://localhost:5173
Point the Vite dev server at the backend by setting `VITE_API_BASE_URL=http://localhost:5000` (or configure `vite.config.ts` proxy).
- Number revisions sequentially (`0123_...`). Revision ids must fit in 32 characters, the width of `alembic_version.version_num`.
- Write both `upgrade()` and `downgrade()`.
- Adding a value to a `CHECK`-constrained column means dropping and re-adding the constraint in the same migration.
- Keep migrations to schema changes. Filling a derived table from a large one belongs in a background backfill after startup, where its cost doesn't hold up the boot.
- A migration that depends on application code (a key derivation, a serialiser) copies what it needs rather than importing it, so a later edit can't change what an old migration does.
## Database Migrations
## The plugin
Alembic migrations run automatically on container startup (`alembic upgrade head` in the `CMD`).
`plugin/` ships straight from git: installs fetch it through `.claude-plugin/marketplace.json`, and there is no build step. After changing anything under `plugin/` or `.claude-plugin/`, run `make mint-plugin` before committing. Claude Code only refreshes an installed plugin when its version changes, and CI fails a plugin change that didn't bump it.
The hooks must run on a bare machine: POSIX shell, awk, sed and the usual coreutils, plus `git` and `curl`. They fail open, so a slow or unreachable Grimoire never blocks Claude Code.
Migration conventions:
- Use raw SQL with `IF NOT EXISTS` guards for idempotency
- Use `DO $$ BEGIN CREATE TYPE … EXCEPTION WHEN duplicate_object THEN NULL; END $$` for enum types
- Number migrations sequentially (e.g. `0027_add_something.py`)
- Always provide both `upgrade()` and `downgrade()`
## CI and releases
## CI/CD
CI runs on Gitea Actions (workflow in `.forgejo/workflows/ci.yml`) in the shared `ci-python:3.14` image ([`ci-requirements.md`](../ci-requirements.md) lists what it relies on). Six jobs: TypeScript typecheck, plugin hooks, Python lint (plus design-token and dangling-style checks), unit tests, integration tests, and the image build, which waits on all of them except the plugin job.
### Pipeline
| Push | Image tags |
|------|-----------|
| `dev` | `:dev`, `:<sha>` |
| `main` | `:latest`, `:<sha>` |
| a `v*` tag | `:latest`, `:<version>`, `:<sha>` |
`:<sha>` never moves, so it is the rollback point. Images publish to `git.fabledsword.com/bvandeusen/fabledgrimoire`, with a registry layer cache at `:cache`. The build needs the `REGISTRY_USER` and `REGISTRY_TOKEN` (a token with `write:packages`) repository secrets.
| Trigger | Jobs | Docker tags pushed |
|---------|------|--------------------|
| Push to `dev` | typecheck + lint + test → build | `:dev`, `:<sha>` |
| Tag `v*` on `main` | typecheck + lint + test → build | `:latest`, `:<version>`, `:<sha>` |
| Push to `main` | typecheck + lint + test | (no build) |
Work happens on `dev`. A release merges `dev` into `main` with a plain merge commit; a dated `vYYYY.MM.DD.HHMM` tag, matching the version the image reports, can mark it.
### Release Process
1. Work on `dev` branch — CI validates on every push
2. When ready, open a PR from `dev` → `main` in Forgejo
3. Merge the PR
4. Create a release via the Forgejo UI on `main` with a `v*` tag (e.g. `v26.03.23.1` — CalVer: `YY.MM.DD.N`)
5. The tag push triggers CI → build job pushes `:latest` + `:<version>` Docker images
6. After merging to main, sync dev back:
```bash
git checkout dev && git merge main && git push origin dev
```
### Runner
CI jobs schedule against the `python-ci` runner label and run inside the
shared `git.fabledsword.com/bvandeusen/ci-python:3.14` image (see
`ci-requirements.md` for what this project relies on from the image).
The runner deployment lives outside this repo; image bumps happen in
[CI-Runner](https://git.fabledsword.com/bvandeusen/CI-runner) via Renovate.
`infra/runner-compose.yml` + `infra/act-runner-config.yml` document the
runner-host deployment shape; the source of truth is the deployed
config on the runner host.
### Docker Registry
Images pushed to: `git.fabledsword.com/bvandeusen/scribe`
**Important:** Do NOT use `op.create_table()` or `sa.Enum()` — SQLAlchemy's event system can fire `CREATE TYPE` even with `create_type=False`, causing failures on re-run. Always use raw SQL with `IF NOT EXISTS` / `DO $$ BEGIN ... EXCEPTION WHEN duplicate_object` guards.
## Project Conventions
## Conventions
### Backend
- Services: `async with async_session() as session:` — import from `scribe.models`
- No `scribe.database` module
- Blueprint per resource: `routes/notes.py`, `routes/tasks.py`, etc.
- All business logic in `services/`; routes are thin wrappers
- Permission checks via `services/access.py` — never inline ownership checks in routes
- Services open their own session: `async with async_session() as session:` (from `grimoire.models`).
- Routes and MCP tools stay thin; the logic lives in `services/`, shared by both.
- Access checks go through `services/access.py`, never inline ownership checks.
- Every wait on something outside the process has a deadline.
### Frontend
- API calls via `frontend/src/api/client.ts` typed helpers (`apiGet`, `apiPost`, `apiPatch`, `apiDelete`)
- Pinia stores for shared state; local `ref()` for component-only state
- Composables in `composables/` for reusable behaviour (autosave, keyboard nav, tag suggestions, …)
- Views are page-level components in `views/`; reusable UI in `components/`
- API calls go through the typed helpers in `frontend/src/api/client.ts`.
- Pinia stores for shared state, `ref()` for state local to a component; composables for reusable behaviour.
- Colours, spacing and type come from the design system's tokens in `theme.css`, which is generated, so CI rejects an unknown `var()`.
### Commit Style
### Commits
```
type(scope): short description
type(scope): what the change does
Longer body if needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Why, and anything a reader tracing this change later needs.
Tests are in `tests/`. They run against a real PostgreSQL instance in CI (not mocked). Keep tests integration-style where possible — mock failures have historically masked real migration bugs.
Types: `feat`, `fix`, `refactor`, `docs`, `chore`, `test`. Comments in the code describe how it works today; the history of a change belongs in its commit message.
Scribe is a self-hosted work system-of-record for software projects, built to be
driven by Claude Code. There is **no in-app LLM** — Claude is the sole assistant,
reaching Scribe through a built-in MCP endpoint and a bundled Claude Code plugin. The
web UI is a clean surface for humans to read and edit the same data.
Grimoire is a self-hosted systemofrecord for the work you do with Claude Code. Claude records what it learned, decided and built, and that record reaches the next session, so a solution is recalled rather than reinvented. There is no language model inside Grimoire: Claude reaches it through the MCP endpoint and the Claude Code plugin, and the web UI is where people read and edit the same records.
## Notes
Write in Markdown with a live-preview editor (Tiptap/ProseMirror). Headings, bold,
italic, lists, code blocks, and task checklists render inline. A slash-command menu
(`/`) inserts common blocks.
A Markdown editor (Tiptap) with headings, lists, code blocks and checklists inline, and a `/` menu for blocks.
- **Wikilinks** — Link notes with`[[Title]]` or `[[Title|Display Text]]`. Clicking
navigates to (or auto-creates) the referenced note; the editor suggests existing
titles while you type `[[`. Backlinks appear in the note viewer sidebar.
- **Tags** — First-class `ARRAY[text]` column with autocomplete. Hierarchical tags
(`area/backend`) supported — filtering by `area` matches all `area/*` children.
- **Version history** — Every body edit snapshots a version (up to 20 per note).
Browse, diff, and restore from the editor's History panel.
- **Draft recovery** — In-progress edits persist across page loads and are restored
when you reopen a note.
- **Convert freely** — Turn a note into a task (sets `status=todo`) or back again.
- **Wikilinks**:`[[Title]]` or `[[Title|text]]`, suggested as you type; backlinks show in the viewer.
- **Tags**, with autocomplete. Hierarchical tags (`area/backend`) match their parent's filter.
- **Version history**: browse, diff and restore earlier versions of a note.
- **Draft recovery**: unsaved edits survive a reload.
## Tasks and Issues
## Tasks
Tasks carry status (`todo` →`in_progress` →`done`/`cancelled`), priority
(`none`/`low`/`medium`/`high`), due date, milestone assignment, and a parent task
(sub-tasks). Notes and tasks share one model — a task is a note with a status.
A task is a note with a status (`todo`,`in_progress`,`done`, `cancelled`), plus priority, due date, milestone and parent task.
- **Work logs** — Append timestamped progress entries (with optional duration) to a
task without rewriting its body; shown chronologically in the task view.
- **Issues** — A task whose `kind` is corrective: a problem you fixed or are fixing,
with symptom → root cause → fix in the body. An issue can link the task it arose
from and the System(s) it touches.
- **Recurring tasks** — An interval or calendar recurrence rule spawns the next
occurrence when a task is completed (a background job drains due spawns).
- **Sub-tasks** — Any task can have children via `parent_id`; the viewer shows them
inline.
- **Kinds**: `work`; `issue`, for corrective work, which can link the task it arose from; and `spike`, a time-boxed investigation whose output is an answer.
- **Work logs**: timestamped entries that record progress without rewriting the body. They are what the next session reads first.
- **Claims**: a session that logs on a task holds it, so after a context compaction the work comes back to that session, and other sessions can see who is on it.
- **Recurring tasks** spawn their next occurrence on a schedule.
## Projects and Milestones
## Projects, milestones and systems
- **Projects** — Group related notes and tasks. Title, description, goal, status
(`active`/`paused`/`completed`/`archived`), and a colour.
- **Milestones** — Ordered stages within a project. A milestone is also the home of a
**plan** — its body holds the design (Goal/Approach/Verification) and its child
tasks are the steps. Completion percentage is shown on the project page.
- **Kanban view** — `/projects/:id` groups tasks by milestone in a column layout with
status-advance buttons on the cards.
- **Projects** carry a goal, a status and a colour, and group everything else. Entering a project gives Claude a short primer: its goal, recent plans and their next steps, open tasks, its systems and its design system.
- **Milestones are plans**: a body holding the design (goal, approach, verification) and ordered step tasks. Starting a plan that matches an active milestone hands back that milestone instead of opening a second one.
- **Systems** are a project's named areas ("auth", "retrieval"). Records are tagged with them, each has a charter saying what belongs there, and each maps to a canonical area shared across projects.
- **Duplicate gates**: creating a record that reads like an existing one returns the existing one to update instead.
## Systems
## Rules, lessons and moments
A **System** is a per-project, reusable, self-describing subsystem or area (e.g.
"auth", "billing"). Associate any note, task, or issue with a System so research,
build-work, and fixes for the same area line up and recurring problem-spots surface.
- **Rulebooks → topics → rules.** A rule in a rulebook applies in every project; a project rule applies only there. A project that departs from a global rule says so with an `overrides` relation.
- **Rules arrive by retrieval.** Nothing is preloaded: a rule reaches a session when what Claude is about to do matches its trigger (`when_to_apply`), whether that's the operator's message, a command or a file being written.
- **Moments**: a rule about *when* something is done (finishing a task, reporting back, pushing) can be mounted on a moment of work, and arrives whenever that moment happens, whatever the words look like. Settings → General shows which actions reach each moment.
- **Versions and verification**: every edit to a rule keeps the text it replaced. A rule that asserts a fact about someone else's software can say how to check it and when it stops being true, and a sweep lists rules due for a check.
- **Preferences** are rules with the force of guidance rather than obligation.
- **Lessons** record something learned the hard way, keyed to the situation where it applies, and link to the rules they are instances of.
- **Reply shapes**: the default shape of Claude's replies (completion report, plan for review, question) arrives at the moment it applies. Settings → General shows them.
## Rules and Rulebooks
## Snippets and the shape ledger
Scribe stores the operator's engineering and workflow **rules** so Claude follows them
across sessions.
- **Snippets** are recorded prior art: a reusable piece of code with when to reach for it, its signature and its canonical location in a repo. Claude searches them before building and is shown matching ones before it writes a file.
- **Forge integration**: with a git forge connected, Grimoire checks a snippet's cached body against its source and marks it current or diverged. Push webhooks keep that fresh.
- **The shape ledger**: Grimoire extracts the code shapes from a bound repo and tracks how each is accounted for: an instance of a snippet, a deliberate variant with its reason, or a one-off. Coverage shows what is still unclassified.
- **Rulebooks → topics → rules** — Rules are grouped by topic inside a rulebook.
- **Rules arrive by retrieval** — Nothing is preloaded. A rule reaches a
session when what the agent is about to do matches its trigger: a command,
a file being written, or the operator's own message. `when_to_apply` is
therefore the field that decides whether a rule is ever seen.
- **Per-project scope** — A project subscribes to rulebooks, and can add
project-scoped rules or suppress individual inherited rules/topics.
## Family canon
## Stored Processes
Projects on the same platform (Android apps, Go services, ...) share good ideas. An idea promoted to the family's canon reaches every project on that platform, and each answers it: adopted, a variant with its reason, or owed (filed as a task). `/family` shows the ideas and every project's answers.
Reusable saved prompts (a note with `note_type=process`) — e.g. a drift-audit or a
DRY pass. The bundled plugin syncs each Process into a local Claude Code skill stub
(`/scribe:sync`) that auto-surfaces by relevance and fetches the live procedure on
demand.
## Design systems
## Search and Knowledge Injection
Tokens (colour, type, space, motion, ...) and written guidance, with inheritance: a project's design system can extend a house style and record where it departs. Grimoire generates a stylesheet from it, checks code snippets against it, and gives Claude the values to use before it writes UI. `/design-systems` edits them.
- **Semantic search** — pgvector-backed similarity search over notes and tasks
(in-process `fastembed` embeddings; no external model).
- **Proactive knowledge-injection** — the plugin's `UserPromptSubmit` hook surfaces a
short, high-confidence menu of maybe-relevant note *titles* into Claude's context
each turn; Claude pulls a full body only when it judges it relevant. Gated so it
stays quiet on most turns and never repeats within a session.
## Stored processes
## Knowledge Graph
Saved procedures, such as a drift audit or a DRY pass. The plugin syncs each one into a Claude Code skill (`/grimoire:sync`), which fetches the live procedure when it is invoked.
`/graph` renders notes, tasks, and tags as a D3 force-directed graph. Tag nodes
controls (repulsion, link distance/strength, hub pull, gravity); click a node to peek,
click a tag to filter.
## Search and context injection
- **Semantic search** over notes, tasks, rules, milestones and systems, with embeddings computed in-process (no external model service). It falls back to keyword search if the embedder is unavailable.
- **Context injection**: the plugin's hooks show Claude a short menu of possibly relevant records at the moments they matter: rules and prior art when a prompt arrives, before a command or a file write, and when a moment of work is reached. Claude opens a full record only when it judges it relevant.
- **Tuning**: each surface has its own thresholds and budget, in Settings → General. Every change is logged with its reason, and retrieval telemetry shows what was surfaced and what was opened.
## Claude via MCP and the plugin
The whole store is reachable by Claude through a built-in **MCP endpoint at `/mcp`**
(Bearer-auth with an API key). The **Scribe Claude Code plugin** (shipped in this
repo) wires it up:
The whole store is reachable through the MCP endpoint at `/mcp`, authenticated with an API key. The Claude Code plugin in this repo adds:
-a `SessionStart` hook that injects active-project
context so Scribe surfaces without being asked (fail-open if Scribe is unreachable);
brainstorming — that route their output into Scribe;
- your saved Processes auto-surfaced as skills.
-hooks for session start, each prompt, before and after tool calls, the end of a turn, compaction and session end, all failing open if Grimoire is unreachable;
- a project fence: a session belongs to the project its starting directory is bound to. Toward any other project it can read anything and file an issue. Editing that project's files, committing or pushing in its repo, or changing its records first asks you, in every permission mode;
-skills for the working practices: using Grimoire, writingplans, systematicdebugging, verification, brainstorming, reporting back, reusing code, shape accounting and family canon;
- your stored processes as skills.
See [API Keys & MCP](api-keys-and-mcp.md).
## Sharing and Collaboration
## The web UI
- **Share** — Share any project, note, or task with users or groups at
`viewer`/`editor`/`admin` levels from the viewer/project toolbar.
- **Groups** — Admins create platform-wide groups and assign `member`/`owner` roles;
share a resource with a group in one action.
- **Shared with me** — `/shared` lists incoming shares with permission badges.
- **Notifications** — An in-app bell (unread count, polled) fires when a project or
note is shared with you or you're added to a group.
- **Dashboard**: what to work on next. **Knowledge** browses everything.
- **Graph** (`/graph`): notes, tasks and tags as a force-directed graph, with physics controls and filtering by tag.
- **Trash**: deleted records wait here until retention purges them, and can be restored.
- **Installable** as a desktop or mobile app.
Every read and mutation is scoped by owner + direct shares + group shares.
## Sharing
## Data Export and Backup
Share a project, note or task with users or groups, as viewer, editor or admin. Admins manage groups. `/shared` lists what has been shared with you, and an in-app bell notifies you of new shares. Every read and write is checked against ownership and shares.
- **Personal export** — download all your notes/tasks as a Markdown ZIP (YAML
frontmatter) or a JSON array.
- **Admin backup** — full application backup/restore (format v4) with ID remapping on
restore for cross-instance migration.
## Export and backup
## Progressive Web App
- **Personal export**: your notes and tasks as a Markdown ZIP (with YAML frontmatter) or JSON.
- **Admin backup and restore**: a restore rolls the whole instance back to one of its own backups, after saving the current state, or brings a new instance back from any backup.
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. Restoring is in Settings, or `python -m grimoire.restore`. See [Backups](backups.md).
Installable as a desktop or mobile app; a service worker caches the shell.
that links to a matching local account. Invite links, a registration toggle, password
reset, and session invalidation are included. See [SSO / OAuth](sso-oauth.md).
Email and password, plus optional OIDC single sign-on (Authentik, Keycloak, ...) that links to a matching local account. Invite links, password reset and signing out other sessions are included. See [SSO / OAuth](sso-oauth.md).
Add the following environment variables to the `app` service in `docker-compose.yml`:
@@ -64,7 +64,7 @@ docker compose up --build -d
## 4. Account Linking
When a user logs in via OAuth for the first time, Fabled checks in this order:
When a user logs in via OAuth for the first time, Grimoire checks in this order:
1. **Existing OAuth sub** — returns that user immediately.
2. **Matching email** — if a local account already exists with the same email, the OAuth identity is linked to it automatically. The user retains all their notes and tasks.
@@ -92,4 +92,4 @@ The backend will reject any `POST /api/auth/login` or `POST /api/auth/register`
The OIDC discovery endpoint (`<issuer>/.well-known/openid-configuration`) must be publicly reachable from the Fabled container (server-to-server call at login time).
The OIDC discovery endpoint (`<issuer>/.well-known/openid-configuration`) must be publicly reachable from the Grimoire container (server-to-server call at login time).
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.