classify_shapes(repo=...) and the write-path hook looked only at live rows,
so a shape whose row had vanished got a second provisional row under the
same (project, repo, path, symbol, kind). The unique constraint refused it,
and the whole classify batch was lost as a tool error.
provisional_row() now finds the row under that identity, vanished or not.
A vanished row is revived with a reappeared event and keeps its judgment.
A new row is inserted inside a savepoint, so a concurrent insert of the
same identity falls back to that row instead of failing the transaction.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Configuration, the README env table and the features page say Settings
-> Config -> Reverse proxy is where the count is checked and set,
that a saved count wins over TRUSTED_PROXY_HOPS, that Cloudflare in
front of your own proxy is 2, and what a too-low count does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The section opens with a verdict on the admin's own request: the
address Grimoire reads them as, or, through Cloudflare, the count to set
with a button that sets it. Below it are the count with where it came
from, a table of what counts 0-3 would read, and the raw forwarding
chain. A saved count can be cleared back to the environment's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An admin can save how many reverse proxies stand in front of Grimoire.
A saved count wins over TRUSTED_PROXY_HOPS, which stays the starting
value. Each worker loads it at startup (under a deadline) and re-reads
it every 30s; the worker that saves applies it at once.
GET/PUT /api/admin/proxy returns the count, where it came from, and a
readout of the admin's own request: the socket peer, the forwarding
chains, what counts 0-3 would read, and the count that reads
Cloudflare's visitor when Cloudflare's headers are present. Saves are
audit-logged. A "too many attempts" reply now says where to look when
everyone is being blocked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
using-grimoire, which owns cross-project conduct, now says the one change
to make toward another project is filing an issue with how it was found
and the evidence, and that the plugin asks before anything more. The
features page describes the fence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The project a session entered now lives in grimoire-fence, which the
compaction clear never visits, and session end removes it. Arrays of ids
in a tool's input were sent as [,"3","4"] under mawk, which creates an
array element as soon as it is assigned to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
plugin/hooks/grimoire_project_fence.sh runs as PreToolUse on editor writes,
Bash and Grimoire's tools. A session belongs to the project its starting
directory resolves to (CLAUDE_PROJECT_DIR: a .grimoire marker, else the git
remote's binding). A session that starts outside any project belongs to the
first one it enters, which PostToolUse on enter_project records.
These acts stop on permissionDecision "ask", with the server's reason naming
the other project:
- an edit in another project's repo or a worktree of it;
- a git commit, push, merge, rebase, pull, cherry-pick, revert, tag, am, reset
or worktree add there, found through git -C, a cd before it, or the shell's
current directory;
- a Grimoire write that lands in another project.
A live probe showed "ask" holds in bypassPermissions. Filing an issue and
every read go ahead. An edit or git command in the session's own repo, and any
tool named like a read, never calls the server. Grimoire writes send only
their short arguments, so the URL stays small.
When the instance doesn't answer, the act goes ahead and the hook says once
per outage that it couldn't check. check_plugin.py now holds one outage line
per hook and has a smoke event for the fence.
GET /api/plugin/write-scope takes fallback_project_id for the entered project.
tests/test_project_fence.py drives the hook against real git repos and a
stand-in server. tests/test_write_scope.py holds the hook's
FENCE_READ_PREFIXES equal to services/write_scope.READ_PREFIXES. The plugin
README and PACKAGING describe the fence, and the plugin is minted to
2026.11.05.0245.
Milestone 583, step 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guard caught seven write-tool arguments with a locating name that are not
where the write lands:
- snippet_id on classify_shapes_by_rule and confirm_shape_proposals names the
canon, not the rows written;
- rule_outcome, rule_misfired and judge_lesson_link record judgments.
REFERENCES lists each one with its reason, and the guard reads it.
READ_PREFIXES names the tool prefixes the fence answers without the server,
and a test holds every write tool to a name outside them.
plugin.json is minted to 2026.11.05.0242 for the PACKAGING.md row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /api/plugin/write-scope answers the plugin's fence. The session's project
comes from repo / project_id, as every plugin endpoint reads it. The act is
either:
- a directory (target_repo / target_project_id), for a file edit or a git
command; or
- an MCP call (tool, plus input: the call's short arguments).
It returns allowed, the other projects the act would change, and the reason
the operator is asked. It is a GET so a read-scoped key still fences file
edits and git.
services/write_scope.WRITE_TARGETS lists every write tool with the arguments
that locate its write: a project id, or a task, note, milestone, System, rule,
relation, trash batch, repo or family decision, each traced to its project. A
write that lands in no project (install settings, user-level records, the
judgments a session is asked for anywhere) has an empty entry. The one
cross-project write allowed is create_task(kind="issue") as a new, unplanned,
open issue. A project the caller cannot read is never named.
tests/test_write_scope.py:
- requires an entry for every write tool;
- requires every declared argument to exist on the tool;
- refuses a locating argument (project_id, task_id, milestone_id and the
rest) that a write tool takes without declaring it.
The integration test traces each locator from real rows.
Milestone 583, step 2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScrollSentinel sits at the foot of a server-paged list and asks for the
next page a screen before the reader reaches it. It asks once per cursor,
forgets the cursor when its element goes, and shows Try again on a failed
page instead of retrying in a loop.
FamilyView's decision log loses its "Older decisions" button. A failed
page shows in place rather than as a toast, and rows already shown are
skipped when a new decision shifts the offset.
KnowledgeView drops its inline IntersectionObserver for the component.
The cursor is the ids consumed, not the items rendered, so a batch that
returns fewer records than it asked for still moves the feed on. A failed
batch puts its ids back and offers Try again, where it used to fail
silently and stop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two integration tests in test_integration_backup_rollback.py drive
grimoire.restore.run on the scheduled backup's tar, written from a full
export of this instance:
- without --yes it exits 4 and nothing changes;
- with --yes it takes the safety backup, then puts back the edited and the
deleted task and removes the new one.
The file-restore test that expected a refusal "because the database has
accounts" now names the reason it actually refuses: on a live instance, a
backup that records no instance id can't be rolled back to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
python -m grimoire.restore <file> prints what the restore would change (the
same preview as Settings) and stops. With --yes it runs
backup_job.rollback_to_backup: the current state is saved first, then the
whole instance goes back to the backup. An empty instance takes any full
backup. A live one only takes a backup of its own, and otherwise exits 3 with
the reason. A v1 backup keeps the copy-in path into an empty instance.
docs/backups.md Restoring is rewritten:
- what a restore puts back and removes;
- the same-instance rule;
- the safety backup;
- what survives;
- the web and command-line ways in.
Its S3 key advice now asks for read as well as write when restoring from the
bucket. README, features, architecture and the API reference follow.
Milestone 582, step 5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The web restore no longer copies a backup in beside the live data.
- GET /api/admin/restore/sources lists the backups at the backup destination,
newest first.
- POST /api/admin/restore/preview says what a restore would do, changing
nothing: fresh or rollback, when the backup was taken, and counts now and
after per kind. A backup that cannot apply here gets a 409 with the reason.
- POST /api/admin/restore?confirm=rollback runs backup_job.rollback_to_backup:
safety backup, replace, then search rebuilt in the background.
Each takes ?source=<name> (a backup at the destination) or an upload. The
destination path exists because uploads are held to the request-size limit,
and a real instance's backup is far over it. backup_s3.get_file downloads
from a bucket. backup_destination.list_backups/open_backup only accept names
the scheduled backup writes, so a request cannot reach any other file.
Settings -> Data -> Restore from Backup opens RestoreBackupDialog. The dialog:
1. Pick a backup at the destination or a file.
2. Read the preview.
3. Type "roll back".
4. Run it. The dialog then says where the replaced state was saved, and sends
the user to sign in again.
Closes#6840. Milestone 582, step 4.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backup_job.rollback_to_backup:
1. Checks the backup can apply here (backup.check_replace, which shares
_replace_mode with replace_from_backup), refusing before anything is written.
2. Takes the backup lock, so no scheduled backup runs during the rollback.
3. Saves the current store with save_state_before_restore.
4. Replaces the store.
The safety file goes to the backup destination when backups are on and it
takes it. Otherwise it goes to DATA_DIR/pre-restore, with a note when the
destination refused. If neither takes it the rollback does not run. The
result names where the file went.
Milestone 582, step 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One transaction:
- Refuse unless the backup is a full one and, on a live instance, carries
this instance's id. A fresh instance (no records, at most the restoring
account) takes any full backup and adopts its id.
- Clear first: search vectors and the CSS consumer map (derived); every
backed-up table nothing outside the backup points at, whole; the settings
the backup carries; and the identity-table rows the backup lacks. A share
granted by a removed user goes before the user.
- Write the backup through the same _build_* helpers as the copy-in restore,
now _load_v2(keep_ids=...). Identity tables (users, projects, milestones,
notes, rulebooks, topics, rules, systems, design systems, code shapes,
family decisions) go through _place. That keeps the backup's id and
merges, so API keys, shares and forge connections of a surviving row stay.
- Reset id sequences past the restored max, and sign every session out.
reindex_after_restore runs the four embedding backfills afterwards. A unit
guard checks that no row kept for outside references cascades from a table
replaced whole. Integration tests cover edits, deletions and new rows put
back, a surviving API key, sessions ending, fresh ids afterwards, foreign or
id-less backups refused, a user export refused, a failure part-way changing
nothing, and a fresh instance adopting the id.
Milestone 582, step 2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
instance_identity is one row made by migration 0127 with a random id, so an
instance has it before its first account. A full backup carries it in its
header as instance_id (the tar manifest header and the JSON top level);
BACKUP_VERSION is 28. The table rides in _NOT_INCLUDED: a restore decides from
the header value rather than copying a row.
Two stores with the same id number their rows the same way. That is what lets
a later step roll a live instance back by original id, refuse a backup whose
ids would land on unrelated rows, and have a fresh-instance restore adopt the
backup's id (adopt_instance_id).
Milestone 582, step 1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Quart takes remote_addr from a Remote-Addr header its ASGI layer writes from
the scope (replacing any the client sent). A test request context does not
read scope_base for it, so the two socket-peer tests saw an empty address.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
grimoire/proxy.py is the one reader of X-Forwarded-For and X-Forwarded-Proto.
It takes the entry TRUSTED_PROXY_HOPS in from the right (default 1; a shorter
header reads its leftmost entry; 0 trusts no proxy), so an entry a client puts
left of the proxy's is never believed. Before this, with TRUST_PROXY_HEADERS
on, the leftmost X-Forwarded-For entry keyed the login, register, forgot and
reset rate limits and the audit log, and a client could choose it.
The session cookie is Secure, and HSTS (max-age=31536000) is sent, exactly
when the request is HTTPS by the trusted proxy's word. Plain HTTP is
untouched and nothing redirects. SECURE_COOKIES is gone (a startup warning says
so when it is still set); TRUST_PROXY_HEADERS=false still maps to 0 hops, with
a warning to move to TRUSTED_PROXY_HOPS. Admin audit lines now use the same
client address instead of the socket peer.
The placeholder-key refusal keyed on SECURE_COOKIES, so it now refuses either
placeholder key outright; the instance makes its own key when none is set.
Family idea #5004 (IsHTTPS, trusted-hop X-Forwarded-Proto), canon v2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A by-name reference edge could not tell two definitions of one name apart:
in a Go repo that defines `file` as importRun.file (internal/files, the
canon) and as sftpRun.file (internal/replication), a mirror test calling
`m.file(` was recorded as a use of the canon.
names_canon now takes defined_at, every repo path that defines each name,
in place of the shape file's own names. The own-file rule is unchanged.
When the repo also defines the canon's name somewhere the canon is not
recorded, a mention counts only from the canon's own package: the
directory of a file location exactly, or under a directory location.
A name defined once resolves as before. Proposer version 8 -> 9, so the
next coverage refresh re-reads every row and replaces its reference edges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read against Fabled Archives after the cleanup: of the 9 rows still linked
to snippet #5448 (symbol `file`), all but one matched prose in a comment,
"a file (not a directory)", "the file (ScanWaiting". The only real call
was `r.file(ctx, ...)`. Formatted code never puts a space before a call's
parenthesis and prose does, so the plain-word test is now `word(` with no
whitespace. _PROPOSER_VERSION 8 re-reads every row once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Migration 0125 removed the proposer's cross-project `reference` edges, but
the write path records its by-name evidence with basis `hook`, so those
survived it. A hook edge across projects is impossible for the same reason a
reference edge is, and the write path now applies the same by-name test
(names_canon), so 0126 removes the ones written before. Judged edges keep
their own basis and are untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The proposer's reference basis, and every uses edge it writes, counted any
word-bounded mention of a canon's symbol. Three holes made that wrong at
scale (snippet #5448, symbol `file`, held 2,161 uses edges):
- Go, Rust, Kotlin, Swift, Dart, C, C#, Ruby and PHP were outside the
language-family maps, so their canons read as "unknown, no gate" and were
matched in every Python, TS and CSS body.
- Canons in other projects were matched by name, though one repo never
calls a function defined in another's.
- A plain word (`file`, `seen`) or a module's own helper (`_run`) matched
wherever it appeared.
names_canon is now the one by-name test, used by match_canon,
reference_canons and the write-path hint: same project, not a name the file
defines itself (unless the canon is recorded there), no generic verb with or
without a leading underscore, and a bare lowercase word only where it is
called. The write path keeps the payload's own names in play, since a
redefinition of a pulled canon is the copy it exists to flag.
Repair: record_uses only adds, so replace_reference_uses now makes a row's
machine-written reference edges exactly what the current rule finds,
leaving judged and hook edges alone. _PROPOSER_VERSION 7 re-reads every row
on its project's next refresh. Migration 0125 deletes the cross-project
reference edges already stored.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HistoryPanel, InlineAssistPanel, NoteCard, WorkspaceNoteEditor and
WorkspaceTaskPanel had no importer, lazy import or global registration
anywhere in frontend/src. The features they served were replaced; token
sweeps kept restyling them because they match a glob. The two doc comments
that named HistoryPanel.vue now name VersionHistorySection.vue, the note
history that is live.
SearchBar.vue stays: it is the recorded home of snippet #2961 "SearchBar -
the search input recipe", and the two live search boxes are judged variants
of it. Deleting it would leave the recipe pointing at nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The signal counted every get_rule open against every moment from the
previous three minutes, so a rule a search arm had just shown was
proposed for whatever moment happened to fire. All nine pending signal
proposals were wrong.
- The hook skips an open of a rule a hook named this session, and sends
the last three acts from 90 seconds instead of 30 from 180.
- A rule already mounted anywhere gathers no signal evidence.
- A signal pair keeps the actions that reached its moment and shows them
as its why.
- The proposal line says it is co-occurrence, not a recommendation, and
asks the reader to check the rule against the moment before offering
it; moments.md says the same.
Plugin 2026.10.26.2029.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The write hooks recorded at most 12 definitions per write (alphabetical,
so test functions were the ones dropped) and the post-Bash hook stopped
after 4 files, so the Stop hook never asked about the rest. The caps
bound the prior-art query; the written ledger now gets the full list,
up to 40 files per Bash call, while the query stays at 4 files x 12 names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The derive check matches by name, so a renamed copy of a recorded snippet
passed it and showed only as "similar" prior art. When a write introduces a
definition the file does not hold (shape_ledger.new_shapes_at) and an unseen
snippet on the menu clears the write-path floor by WRITEPATH_COPY_MARGIN (0.80
at the default floor), the hint now says to reuse it or extract the shared
shape before another copy. Names the derive check or the stamping feed already
spoke for are left to them. Covers recorded snippets only: the ledger keeps no
bodies to compare an unrecorded sibling against.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 4xx with no S3 error body came from something else at that address, most
often a web server on the default port. The message now says so and to check
the scheme and port, beside the raw status. A 5xx without a body keeps its
own reason, since a busy S3 service can answer that way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pending() listed every suggested signal row, including pairs still gathering
evidence below the three-situation bar, so the queue held 200 "proposals"
where about 25 had crossed it. A signal pair now needs proposed_at in its
evidence, the same test the misfire half already applies; proposals from a
pass are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each moment gains by_rule: {rule_id, title, delivered, opened}, read from
the (moment, rule) grouping the readout already did. The moment's totals are
now summed from those rows, so the two cannot disagree. The Moments settings
page shows the list under each moment, closed by default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
surfaced_never_pulled told a rules reader there was no judged sample for the
rule arms. There is one now; the guard tying the text to REVIEWABLE caught it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule arm logs its query, floor, budget and project, which is everything
semantic_search_rules takes, so a logged call is re-run to a depth past its
budget and judged like an auto_inject menu. within_budget is the arm's own
cut: inside the budget and, for the banded command and write arms, inside the
band of the top hit. "Opened after" reads rule pulls, and rules created
after the call are dropped as notes are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"keep going", "yes please do", "is it deployed?": with no word of its own the
auto-inject query was the previous reply's tail alone, and the menu handed
back the records that reply was built from. In a sample of 59 logged calls,
28 came from short prompts; those with a subject of their own ("continue into
DRY 14", "fix the Windows gate") led with the record meant, while the ones
without were finished tasks already in context, with an open on 2 of about 15.
_has_subject checks the operator's words against a list of words that name
nothing; without one, the ranked menu is skipped and records named by number
are still shown. The prompt rule arm is unchanged and still searches with the
tail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
usage and rule_usage gain pulled_by_source, read from the per-source rows
the readout already had, and _silent_surfaces counts those doors as having
emitted. The rules-listing points were seen as of the previous commit,
through rule_usage.by_source.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_telemetry gains rule_usage.by_source, the twin of usage.by_source:
distinct rules each surface showed, how many an agent opened, and a ratio
that is null on a bulk delivery. list_rules(with_usage=True, days=N) adds
each rule's usage, windowed, and how many lessons are confirmed instances
of it, so linked and unlinked rules can be compared through MCP.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The write-path rule arm ran only past the notes guard: a write whose code
matched no snippet, issue or lesson returned before any rule was searched.
It now runs before that guard, on `code or path`, and a write that matched
nothing but a rule still returns the rule lines.
Operator ruling (2026-10-05): yes. A rule about the file being written does
not depend on a snippet resembling it.
Cost: one more rule search per write that matched no prior art, with its
own embedding (the query is `code or path`, not the notes arm's concept
query, so the per-request memo does not cover it). Plus the preference slot
from (c) on those writes.
Expected telemetry shift, to be read on its own once this reaches main:
write_path_rule calls rise toward write_path's count (14 days to 2026-10-22:
write_path_rule 4,634 against write_path 3,786 — the notes arm skips trivial
payloads, the rule arm does not, so the rule arm will now exceed it). Its
zero-result share rises too, since the newly covered writes are the ones
that matched nothing in the notes corpus.
Tests: the gate test is inverted (a write with no prior art still gets its
rule check); the design-only write now runs the rule arm on its path; two
notes-arm tests read the write_path row instead of any row, since the rule
arm writes its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PRE_TOOL_RULE and WRITE_PATH_RULE now take preference_slot=True, like
PROMPT_RULE. When no preference placed on score, the arm asks once more for
the best preference over the same bar, and shows it as one extra line.
Operator ruling (2026-10-05): yes. A preference about how to act ("let each
action land before the next") is ranked against every rule whose trigger
names the same command; the act is its last chance to shape anything.
Cost: one kind-filtered search per call that placed no preference. The
query embedding is memoised per request on both hook routes, so no extra
model call. Expect a new stream of preference_slot rows from the two act
arms; preference_slot's call count will rise roughly by pre_tool_rule's and
write_path_rule's (~23k calls over 14 days before this change), and its
zero-result share will rise with it, since most commands have no matching
preference.
Tests: the tool-arm call-row tests count rows under pre_tool_rule only (the
slot logs under its own name), and the write-arm bar test reads the arm's
own search, which is the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prompt_rule, pre_tool_rule and write_path_rule now share one settings key,
RULE_FLOOR_KEY = "kb_rule_threshold", default 0.70. The reply backstop keeps
its own 0.80, because there the floor is a stop bar. Moving the shared floor
through tune_retrieval or the Settings form records a tuning event on every
surface that reads it (set_dial, via retrieval_surfaces.sharing_floor), and
the result names the surfaces it carried as `moved_with`. Settings shows one
"Standing-rule confidence threshold" control where it showed three.
Operator ruling (2026-10-05): one shared bar for the rule arms, "especially
since we're only taking the first x number of them" -- the budget limits
what is shown; the bar only decides what is worth ranking. Kept a dial.
Why 0.70, from records read rather than percentiles:
- write_path_rule near-misses at 0.7199-0.72 were all false positives
(rule 149 against a Lidarr comment, 157 against a retention Svelte file,
121 against an Archives restore, 164 against a config test, 131 against a
Go package), so the old 0.72 was not hiding real hits -- but the band does
noise control on this arm, so dropping 0.02 costs little.
- prompt_rule: the operator set 0.70 by hand on 2026-09-18, after a 0.60
experiment admitted junk and was reverted. 0.70 is the number with a human
decision behind it.
- pre_tool_rule lines near 0.68 this session were mostly noise; rising to
0.70 trims them, and the band still narrows what clears.
Retired keys: kb_promptrule_threshold, kb_toolrule_threshold and
kb_rulehint_threshold are no longer read. Rows an install stored under them
stay in the settings table, inert. On this instance the effective bars move
pre_tool_rule 0.68 -> 0.70, write_path_rule 0.72 -> 0.70, prompt_rule stays
0.70; record_release_defaults writes those per-surface release rows on boot,
so the telemetry shift has an event to explain it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Once the binding has moved, the old remote is no longer bound, and
move_binding refused. An install that renamed its repo before snippet
locations moved with it had no way to move them. When the old remote is
unbound and the new one is bound to this project, bind_repo now skips the
binding move and still moves the snippet locations that name the old
repo.
The snippet-move test gives each location its own path: rewritten to one
repo, identical locations merge into one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A repo renamed on its forge kept every snippet location naming the old
repo: the binding and the shape ledger moved, the snippets did not.
- snippets.move_location_repo rewrites, on every live snippet of the
project, each location whose repo is the old remote in any form (URL,
normalized key, "owner/Repo", bare "Repo", compared without case) to
the new normalized key. A snippet the caller can only read is listed
in read_only and left alone.
- Its loop is shared with move_locations through _rewrite_locations,
which saves each changed snippet through update_snippet.
- bind_repo calls it after the binding moves and reports snippets_moved
and snippets_read_only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With no SECRET_KEY set, Grimoire fell back to a fixed placeholder. The
quickstart compose substituted change-me-in-production, so the README's
own update step (pull, up -d) recreated the container on that placeholder,
and every credential sealed under the key the user started with stopped
opening.
- config: with neither SECRET_KEY nor SECRET_KEY_FILE, the first start
writes a random key to DATA_DIR/secret_key (mode 0600, published with a
hard link so concurrent starts agree), and later starts read it back.
An explicit key wins and leaves the file alone. A missing
SECRET_KEY_FILE or an empty key file stops the start instead of
switching keys. With no data directory (a run outside the container)
the process gets a temporary key and says so.
- app: warns on a public placeholder key, on a temporary key, and when
/data is not a mounted volume.
- Dockerfile creates /data.
- docker-compose.yml is now the deploy compose (published image, both
volumes, nothing to set). The build-from-source compose is
docker-compose.dev.yml, and docker-compose.quickstart.yml is gone. The
Swarm stack file gains the /data volume and keeps its explicit key,
because a Swarm can move the app away from a node-local volume.
- README: a Deploy section with the compose inline, SECRET_KEY optional,
"keep both volumes", and updates that keep the key. A test holds the
README's copy equal to the file.
- Installs that ran on the change-me-in-production placeholder get a key
of their own on the new compose; Settings then asks once for saved
forge tokens and the backup secret.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LICENSE carries the GNU Affero General Public License v3.0 text, the same
file the other Fabled projects ship. pyproject.toml, the frontend's
package.json and lockfile, and the plugin manifest declare it as the SPDX
expression AGPL-3.0-only, and the README says what it asks of anyone who
runs a modified version over a network. Reading `license` as an SPDX
expression needs setuptools 77, so the build floor rises to it. The
project description now matches the README's tagline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AppLogo still drew the old open book with a spark, so the new mark showed
only in the browser tab. It now draws the same clasped grimoire as
favicon.svg, in the mark's own fixed colours. That covers the header and
the sign-in, register and password pages.
The README follows the house order: the logo, name and tagline, a short
summary, quick start including the plugin install, the settings a reader
must set, setup pitfalls, updating and backups, features grouped by what
you do, troubleshooting by the error text, development, and licence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Snippet locations are keyed by path too, so a snippet recorded under a
moved directory reads `missing` at its next check. The tool is now
move_path(project_id, from_path, to_path). It moves the ledger rows as
before, and re-paths the project's snippet locations under from_path
through update_snippet, so each body and its indexed mirror stay one
record. A snippet the caller can only read is listed in
snippets_read_only. The tool refuses only when nothing at all is recorded
under from_path. The path test is the one snippets already uses
(_path_touches), and the plugin version is minted for the skill edit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ledger rows are keyed by path, so after a directory moves the next sync
marks every judged row vanished and lists the same shapes again, unjudged.
move_shapes(project_id, from_path, to_path) re-keys the rows under one path
to the other, in every repo of the project, keeping each row's judgment,
uses edges and history and adding a `moved` event that records the path it
left. A row already at the target is replaced when nobody judged it; a
judged one stands, and the old row is reported in `kept`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A binding and every row of its project's code-shape ledger are keyed by the
repo's remote. A rename on the forge changes the remote, so binding the new name
started an empty ledger beside the old one and left every judgment behind.
bind_repo now takes moved_from, the old remote. Its binding and the project's
ledger rows move to the new key in one transaction, and the reply says how many
shapes moved. The move is refused when:
- the old name isn't bound to that project;
- the new name is already bound;
- the project already has a ledger under the new name.
Each refusal leaves both names exactly as they were.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The forge repo is now bvandeusen/FabledGrimoire. The plugin marketplace default,
the install commands in the docs and plugin README, the CI requirements title and
the example repo URLs in comments and tests follow it. docs/development.md now
names the image the build actually publishes, fabledgrimoire.
Two test names still said "scribes_own"; they now say "grimoires_own".
What still says scribe does so on purpose:
- the backup reader's accepted archive names and prefix;
- the label every stored secret is sealed under;
- the notes on keeping an old install's database names;
- the old migrations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
favicon.svg is a violet book with gold corners and clasp and a four-pointed star
in a ring, replacing the indigo book with a spark. It is rendered to 192 and 512
PNG app icons and a 180 touch icon on the page surface.
manifest.json lists those icons instead of a favicon.ico that never existed. It
describes the product as the README does, and takes its colours from the design
system's page surface. index.html's theme-color follows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
theme.css loaded Fraunces, Inter and JetBrains Mono from Google Fonts. That sent
every visitor's address to Google on every page load, and an instance with no
internet drew in fallback type. All three families are now vendored under
frontend/src/assets/fonts with every subset Google publishes for them, plus their
OFL licences. The Content-Security-Policy no longer lists fonts.googleapis.com or
fonts.gstatic.com.
A new guard test fails if any stylesheet, page or component links a remote asset
or preconnects to one, or if the CSP names another host.
The theme.css header now describes the sheet as it is instead of how it used to
read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The step-3 sweep matched `scribe` only after a non-letter, and in these
three tests the shell call sits right after a `\n` escape in a Python
string, so the character before it was the letter n. They still called
scribe_recent_context, scribe_skip_prompt and scribe_checkpoint_allowed;
they now call the grimoire_ names. No other escape-adjacent occurrence
remains in the tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 3 of the Scribe -> Grimoire rename.
- Plugin `grimoire` in marketplace `grimoire-plugin`, MCP server key
`grimoire`, so tools reach a session as mcp__plugin_grimoire_grimoire__*.
- hooks/scribe_* -> hooks/grimoire_*, their shell functions and the TMPDIR
ledgers with them; SCRIBE_URL / SCRIBE_TOKEN / SCRIBE_HOOK_DIR /
SCRIBE_LEDGER_DIRS -> GRIMOIRE_*; the project marker is `.grimoire`.
- skills/using-scribe -> using-grimoire, /grimoire:sync, stored processes
arrive as grimoire-proc-<slug>; moment_actions names them the same way.
- Migration 0124 rewrites stored skill names (rule_moments,
rule_moment_judgments, moment_mappings) from scribe:/scribe-proc- to
grimoire:/grimoire-proc-, so a rule mounted on a skill keeps firing.
- Settings' install commands and MCP-only defaults, the docs and the
tests follow.
- scripts/check_plugin.py drops the smoke event for a report-check hook
that has no file in plugin/hooks.
Kept on purpose: the secret-sealing label, and the names that let backups
written as Scribe restore and be counted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 2 of the Scribe -> Grimoire rename.
- src/scribe -> src/grimoire, and every import, patch target, module path,
hypercorn entry point and `python -m grimoire.restore` with it; pyproject,
uv.lock and the frontend package are named grimoire.
- Image git.fabledsword.com/bvandeusen/fabledgrimoire; CI's Postgres service
runs as grimoire / grimoire_test.
- Compose files default the DB user, database and (prod) volume to grimoire,
each overridable from .env, so an install created as Scribe keeps its data
by setting POSTGRES_USER, POSTGRES_DB and PGDATA_VOLUME.
- Backups are written as grimoire-<ts>.tar with format grimoire-backup-tar.
Archives named scribe-* with the scribe-backup-tar manifest still restore
and are still counted at the destination (folder and S3), with tests.
- Exports, the MCP auth realm, the ASGI scope key and the persistent logger
carry the new name.
Kept on purpose: the secret-sealing key label stays "scribe.secrets.v1",
because every stored secret was derived under it.
Plugin identity (plugin name, hooks, env vars, .scribe marker, skills) is
step 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1 of the Scribe -> Grimoire rename: every capitalised "Scribe" in prose
becomes "Grimoire" ("Fabled Scribe" -> "Fabled Grimoire") across the UI,
the PWA manifest and page title, the MCP instructions and tool docstrings,
hook and skill text, docs and README, with the tests that assert on that
prose moving with it. Old migrations keep their wording.
Identifiers are untouched here: the package, plugin name, MCP server key,
hook file names, env vars, the .scribe marker and FabledScribe repo/image
names move in steps 2 and 3, along with prose that quotes them.
The sweep matched "Scribe" only as a whole word, so describe/subscribe and
friends are untouched. The longer name took the MCP instructions one
character over their 1,600 budget; two phrases are shortened to fit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The build job ran `docker system prune -af` on the runner's shared Docker
daemon. That deletes every image no container is using at that moment,
including images other repos' jobs had just pulled and were about to start.
Jobs with a service container wait ~14s for it to become healthy between
pull and create, and fail with "No such image" when a prune lands inside
that wait (CI-runner #16). The runner host prunes nightly instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Profile tab said an assistant used it to personalise chat and a daily
journal. Neither exists, and nothing read the profile: the page saved to
user_profiles and the backup carried it. Removed the tab, the client calls,
/api/profile, the service and the model, and migration 0123 drops the table
(downgrade recreates it empty).
Backup format v27 stops writing user_profiles; an older archive carrying the
section still restores, and the file-restore integration test now proves it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The acceptance greps found what the per-area passes missed:
- src/scribe: "Phase 7/8" removal notes in projects, settings and
user_profile; a paragraph promising an auto-summary side effect that no
longer exists; user_profile's claim that "upcoming MCP profile tools" read
the profile, which nothing does; an instance count and instance rule
titles in plugin_context; "MILESTONE 323 DECIDED" in rulebooks.
- scripts/: check_plugin's header and inline comments were a history of
the incidents behind each guard, with issue numbers in two failure
messages; check_dangling_styles named this app's classes as its examples;
check_design_tokens, measure_duplication and mint_plugin_version cited
issues and checklists.
- frontend/src: forty-odd comments with four-digit issue numbers that the
earlier frontend scan filtered out as hex colours, plus "used to" and
"Phase N" notes.
Kept: illustrative ids (`#4181` as a bare-id example, `PR #198`), migration
Create Date headers and one migration's dated data floor, and the plugin
skills' worked examples of how to cite a record.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twenty-eight test files described the code they guard by what it did
before a fix. Each is now the property and the failure it prevents, in the
present tense: the backfill snapshot, the per-dial calibration query, the
rule scope, the lesson trigger filter, the subshell global, the trash owner
scope, the version readout fallback, and others. A docstring line that
narrated the backup suite's own history is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings, section headers and assertion messages drop the record number,
milestone or step that introduced the behaviour, and incident accounts are
restated as the present-tense failure shape. A dead pointer to a design doc
that is not in the repo is removed.
Renamed to the property they check:
test_the_old_shape_would_be_caught -> test_a_rank_then_scope_query_is_caught
test_a_ledger_repeat_no_longer_produces_that_case_at_all -> test_a_ledger_repeat_is_counted_not_withheld
test_the_header_no_longer_promises_once_per_session -> test_the_header_explains_the_seen_marker
test_the_portal_case_no_longer_stamps_one_canon_onto_a_whole_file -> test_one_canon_is_not_stamped_onto_a_whole_file
test_the_repos_own_prose_no_longer_defines_with -> test_the_repos_own_prose_does_not_define_with
test_the_original_regression_create_note_with_content -> test_the_real_server_refuses_content_on_create_note
Fixture and example ids stay: "note 5" in a supersession chain, "rule 156"
in an illustrative question, `#2275` as the hex-versus-citation case,
`#3244` as the example of a bare number, fixed dates in date-filter tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Module and test docstrings state what is pinned without the record number
or step that introduced it. Incident accounts become the failure shape in
the present tense: the pipefail head, the code_shapes import, the export
order, the facet dialects, the cross-kind repeat, the write-once kind.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings and comments state the property and its reason in the present
tense. The session-context ledger test no longer describes the decision it
"used to pin"; the supersession tests state demote-not-filter without the
quote; one-row measurements become the general shape with their figures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourteen more test files: docstrings state the failure each guard defends
as a present-tense shape instead of the incident that found it, and issue,
milestone and rule numbers leave comments and assertion messages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twelve more test files drop issue, milestone, rule, note, lesson and
snippet numbers and the dated or one-project figures that recorded how
each property was found; the docstrings state the property and why it
holds. The contract-around-the-change guard keeps the dict-key example as
the failure a signature-watcher misses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ten more test files: issue, milestone, rule, decision and snippet numbers
come out of docstrings, comments and messages; the preference door states
what a preference is instead of quoting the framing; the JSON reader says
why jq is not assumed instead of what every hook used to do; the backup
round trip's column warning no longer claims the import side is unguarded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state their properties without issue, milestone and
rule numbers or dated regressions. The version tests describe what folding
the channel into the version breaks instead of the date it did; the surface
registry and settings-defaults tests state that floors move rather than
quoting the decision that made them move.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state their properties without issue, milestone,
rule and decision numbers. The tuning and suspension notes describe the
symptom a shipped default or a late ledger produces instead of one
install's dated readings; the reply-discipline and checkpoint docstrings
state the failure they prevent instead of the session that measured it.
test_empty_selector_is_the_rule_115_baseline is renamed for what it
checks: the no-forge baseline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven more test files: docstrings and comments state the property and its
reason, the SQL-twin note states why `\m` and not `\b` without the install
it broke, and fixture provenance says "real near-miss queries" rather than
naming the spike that collected them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six more test files state their pinned properties without the issue and
milestone history: the pull-telemetry guard explains enumerating kinds
instead of listing two getter incidents, cited-record status explains why
a hint's id needs its status instead of recounting one misreported step,
and the trigger-shape guard keeps its reasoning about why prose heuristics
fail while dropping the dated corpus audit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight more test files state the property they pin as a present-tense
fact. Incident narratives (a dated storage stall, a week of zero tuning
events, a two-project poisoned ledger, a twenty-site owner audit) become
the failure shape each test defends; one-install figures that name a rule
become a general worked example with the calibration numbers kept. The
family-grouping regression guard is renamed for what it checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shape ledger, MCP projects/search/rulebooks tools, backup, chunking,
plugin context, derived mirror, rule band, shape form gate, plugin
version mint and note usage tests drop issue, milestone and rule numbers,
dates and one-install figures from docstrings, comments and messages.
Three shape-form-gate test names that told the discovery story now name
the property: a registry module's helpers are not service functions, the
divergence gate silences only a family contradiction, and callable helpers
beside a callable canon still prompt. The rulebook tool-count comment that
logged each milestone's additions is replaced by the reason it is a count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_pattern_coverage, test_guidance_ownership, test_integration_shape_classify,
test_instruction_surfaces_agree, test_retrieval_warnings, helpers.py and
conftest.py state the property each test or fixture holds. The issue and
milestone numbers, consolidation dates and one-install measurements that
recorded how each was found are dropped; fixture dates and ids stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings, comments and assertion messages in the three most-cited test
files now say which property each test holds and why, without the issue,
milestone or rule numbers, dates and one-install measurements that recorded
how each property was discovered. Fixture ids and calibration scores stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Components, stores, api clients, router, types and shared CSS now state what
each piece is and why, without the milestone that introduced it. diff.ts
names its consumers instead of the copy history; stores/notes and tasks state
their current scope instead of the cleanup that set it; theme.css states the
token practice instead of the sweep that established it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
components.css says why shared recipes live there without the five-copy
.btn-primary count; KnowledgeView's editor-routing and chip comments state
the present reason instead of the bug that once routed snippets wrong.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record and milestone citations go; the ACT-arm bar split and the per-arm
budgets state why they exist without the measurements that motivated them.
A comment about a removed think_enabled setting, pointing at a file that no
longer exists, is deleted, as is the story of a removed design-system picker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record, milestone, phase and plan citations go from every migration header;
the "end before start" fix (0043) and the suppressed-count column (0095)
state the condition they prevent rather than the incident. 0053 and 0055
pointed at design specs under docs/superpowers that are not in the repo;
those pointers are removed.
Kept: alembic's generated "Create Date:" headers (template metadata, not
rationale), 0121's ARM_BORN justification beside the constant, and 0051's
illustrative "update task 42" format example.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone/step labels, record ids, the rescoped-rule date and the 296-call
measurement go; each header states what the change protects. "rule 36"
citations become the DROP + ADD practice they named.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shipped tool docstring (mcp/tools/repos bind_repo) cited this instance's
"rule 1: dev is home"; it now says what a dev-branch project should do.
routes/knowledge, routes/tasks, routes/systems and models/retrieval_log state
the property each guards rather than the bug that once broke it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
routes/search states the derived-facets property rather than the silent
whole-corpus bug that motivated it; routes/rulebooks gives the preference
restore exception its reasoning without the milestone numbers it cited.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- api.py's build-version docstring states why a channel name is not a build
id, without the dated incident.
- rule_version states why rules need a history, without the rule that was
rescoped and the date.
- Capitalised "Rule N" / "Milestone N" citations the earlier lowercase grep
missed, in reply_shapes, supersession, services/family and models/family.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Corrected while reading: routes/notes.py said lessons and snippets pass the
owner to set_record_systems, "a real inconsistency". Every door passes the
caller; the comment now says so.
rule_usage's model docstring states what an outcome measures and why the
row cannot share the note table, without the measurement and record ids
that motivated it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule-embedding docstring carries the own-table test itself instead of
pointing at a record; the later siblings refer back to it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The deferred-backfill and inline-release-defaults comments describe the
lifespan-overrun hazard they guard without the dated incident; backfill
passes say what they fill rather than which milestone added the vectors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The relation kinds say what each prevents without the rule-46 split story;
the removed tier, subscriptions and suppressions are stated as what the
model does not have, with the migration that dropped them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pre-tool rules docstring says why action rules need their own surface
instead of narrating that they once lived in the preload; the write-path
checkpoint note states the reason the hook never blocks rather than whose
decision it was.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backup_s3 says what its signing is checked against instead of which of the
operator's projects it was ported from; rulebooks and shape_check state what
the code is for rather than which milestone it belonged to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 1-2 hit tail: access, background, backup, backup_job, canonical_systems,
design_stylesheet, forge_connections, moment_actions, notes, notifications,
placement, planning, platforms, projects, record_batch, recurrence,
report_check, retrieval_review, rule_scope, rule_versions, scheduler,
shape_check, supersession, system_rulings, system_usage, text, trash,
verification, version_pinning.
Also corrected while reading:
- backup.py said restore step 16 fills the System map; step 15 does.
- version_pinning.py pointed at a design doc that is not in the repo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_migration states what it lets a tuned floor do without quoting
the operator, and its order-of-operations refusal is described as enforced
rather than as history. logging states why raw date filters fail rather
than citing the two issues that found it. The rest drop milestone, step and
record citations; step references now name the code that does the job.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- db_maintenance_scheduler states the storage-stall hazard the startup
timeout guards, without the dated outage.
- design_starter_roles states why a missing role produces literals, without
the call-site count, the operator's call date or a project's name.
- The rest drop milestone/step labels and record citations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_surfaces' header states why the arms read their numbers from one
table and why a floor is moved by reading the refused record, without the
quoted decision, the retired arm or the declined-rule instance.
family_adoption drops milestone/step labels from its section headers and
states the no-ids reason instead of naming the trap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
inception states the legacy stamp and the retired rulebook keys as facts
about stored records; family keeps the repeat-trigger calibration numbers
without the date and record they were measured on, and drops the instance
that taught the per-kind comparison.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
record_refs states why a guessed id is refused without quoting the
operator; moment_delivery's docstrings name what each door does without the
milestone and step that built it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The name gate, catalog and starter-set docstrings drop milestone and record
citations and the history of the tuple constant; each states why it is
shaped this way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The silent-outage story (#2663), the ranked/ambient split (#2477), the
transfer evidence (#3735) and the attach_usage consolidation (#4230) are
stated as the property each guards, without the incident.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pool-exhaustion, handshake-size and stepless-roadmap comments state the
condition the code guards instead of the measured incident; citations go.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module header drops the milestone/step map and record citations; the
per-user optionality, the keyring model and the empty-selector baseline are
stated as present-tense facts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record and milestone citations go. The logs_unconditionally docstring, the
place arm's comment and the fan-out key note state the condition they guard
rather than the bug that once hit it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring states why the model moves dials and why a reason is
required, without the decision quote or the declined-rule story. Bare
record citations go; the release-default and ledger-gap docstrings describe
the condition they guard instead of the measurement that found it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, milestone, snippet and rule-number citations are removed; the scope
docstring states the leak it prevents without naming the milestone that
closed it for the direct arms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ACL header states the share behaviour without its date and decision ids;
the facet table, door map, floor constant and facet-count comments state why
they are single sources instead of retelling how copies drifted; this
instance's counts (legacy plan-tasks, issue share) are removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring keeps why the kind exists, where its trigger lives, what
it inherits and why it needs no migration, without milestone, decision and
note ids or this instance's snippet counts. The body-strip docstring states
why taking the note matters instead of retelling the defect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module docstring keeps why rules need their own usage stream and why the
ranked list is the one that is named, and drops the dated story of the
bulk surfaces starting to emit, measured decline rates and the corrected
earlier wording. Outcome docstrings describe the four states without
milestone references.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, decision, milestone and operator-decision citations are removed;
"previously each route…" and "list_tasks once returned…" stories become the
present-tense reasons they taught. The note about the removed delete_note
now says plainly that there is deliberately none, and why.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, milestone and rule citations and dated measurement headers are
removed; calibration figures stay. Corrected the lesson-gate comment, which
said lessons are not in _REPORT_KINDS and so have no duplicate-report
backstop. They are, and the comment now says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone, issue, note and rule-number citations are removed. The
co_surfaces docstring states why paired rules need the edge instead of
retelling the instance that taught it; the list-rules and applicable-rules
docstrings drop the retired subscription history. The preference-restore
error string no longer cites a milestone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, decision, milestone and rule citations are removed; the snippet-mirror
docstring drops the paragraph about its own earlier wording, and the
pointer-model and freshness comments state the design rather than the plan
step that introduced it. Access notes point at services/access.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue, note and milestone citations are removed along with first-deploy
counts and the measured phantom-definition story; "button B" jargon becomes
the divergence it names. Each block keeps its present-tense reason.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The recurring "#2663 shape" and similar issue citations become the property
they stand for (an outage that reads as a quiet window, a statistic that
cannot vary). Dated observations, milestone tags and rule numbers are
removed; the near-miss and pull-through reasoning stays in present tense.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The format-version log keeps what each version adds, because restore depends
on what an older archive can carry; dates, milestone and issue ids, and the
stories of what a missing column once cost are removed. Compatibility
comments now name the archive version instead of a milestone. Two leftover
history-phrased comments in embeddings and shape_ledger are restated as
present-tense reasons.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone tags, dated measurements and "used to" stories are
removed; the chunker version list now says what each stamped version
contains. Corrected two comments that said a failed search leaves
report["searched"] absent (it is stamped False first), and pointed the
rule-search docstring at _RULEHINT_BAND in retrieval_pipeline instead of a
tier note in plugin_context that no longer exists. plugin_context's own
dangling "note on tiers" pointer is reworded to match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone and step numbers, and the stories of copies drifting
apart are removed; each invariant and slot is justified in present tense.
The rule-band probe table stays as calibration. Section headers drop their
milestone tags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, snippet numbers, dated measurements and the proposer's version
history are removed; each block now states the design reason in the present
tense. Calibration figures that justify a floor stay, without the instance
they were measured on. One error string drops a note id. The density
threshold comment moves above the constants it describes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record ids, milestone numbers, dated telemetry readings and rule numbers
leave the retrieval-arm comments and docstrings. Each bar keeps its
present-tense reason and the probe-score calibrations, which describe the
model on any install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MCP tool docstrings and comments, the server's orientation notes, the plugin
skills, every hook script and the plugin README/PACKAGING drop record ids,
milestone numbers, dates and incident stories, and state each non-obvious why
as a present-tense fact. History stays in the task logs.
Corrections found on the way:
- plugin README/PACKAGING said every hook is a GET; moment, rule-opened and
reply-rules POST, so the plugin's key needs write scope. Both now say so,
name Settings -> MCP Access, and list every wired hook and endpoint.
- the contract-change hint printed "(lesson #4207)", this instance's record,
to every install. It no longer cites a record.
- tasks.py claimed there is no delete_task; there is.
- rulebooks.py cited delete_event and a design spec that no longer exist.
- scribe_after_write.sh carried an always-empty etag_q; removed.
- test_write_path_trigger pinned "| tac | scribe_defs | head -1", text that
existed only in a comment about the old code; it now pins the real tail -1
pipeline.
- docs/development.md: CI runs on Gitea Actions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
g t pushed "/", which now redirects to the dashboard, so it did what g h
does. It pushes /knowledge. The overlay gains g r (Rules) and g g (Graph),
calls g h "Dashboard", and docs/features.md lists g t.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone 544, step 6217 (follow-up to #1778, which rewrote features.md and
api-reference.md in July but left architecture.md and configuration.md alone).
- architecture.md, configuration.md, development.md: rewritten against the
code. They still described Ollama, chat, SSE generation, voice, project
summaries, image caching, Docker secrets that the compose files don't use,
a raw-SQL migration convention the migrations no longer follow, a 27-entry
migration list, CalVer YY.MM.DD.N tags and the removed infra/ files.
- features.md: adds what shipped since July (spikes, claims, moments, rule
versions and verification, lessons, reply shapes, snippets and the shape
ledger, forge integration, family canon, design systems, retrieval tuning)
and corrects the Settings tabs and keyboard shortcuts (g g is the graph).
- api-reference.md: adds the endpoints for snippets, lessons, design
systems, family canon, platforms, shared areas, retrieval, rule versions,
relations and verification, forge connections, webhooks, scheduled
backups and the newer plugin hooks.
- api-keys-and-mcp.md: Settings tab is "MCP Access"; a plugin install
section; the tool-family table matches the tools that exist.
- design-system.md: the dated "decisions in progress" changelog (April
PR log, chat/journal/calendar surfaces) becomes a short present-tense
section; the stored design system is named as the authority.
- sso-oauth.md: the app is Scribe, not "Fabled Assistant".
- README: feature summary and doc table brought current.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At the operator's request (milestone 544, step 6203):
- docs/2026-03-23-fable-mcp-{design,plan}.md, 2026-03-29-voice-s2s-design.md
and 2026-04-10-agentic-briefing-design.md were plans for work that has
since shipped or been retired (voice and briefings left in the pivot).
Plans live in Scribe milestones.
- infra/act-runner-config.yml and runner-compose.yml described one
install's CI runner (an /nfs path, the py3.12-node22 label, a
RUNNER_SETUP.md that no longer exists). The runner is the CI-Runner
repo's concern.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings the three compose files in line with rule 198 (milestone 544,
step 6203, at the operator's request):
- no healthcheck on the app or the database, so a slow pg_isready on
stalled storage can no longer get a working database killed;
- no deploy.restart_policy in the Swarm file, so Swarm's default
(condition: any) restarts clean exits too, instead of on-failure with
max_attempts leaving the app down;
- depends_on is plain start order; restart: unless-stopped stays on the
plain-Compose files (Swarm ignores it), and the dev app gains it.
The orchestrator used to be what retried a container that started before
Postgres was ready. That is now the app's job: alembic/env.py calls
services/db_ready.wait_for_database before migrating. It retries refused
connections, unresolved hosts, connect timeouts and "the database system
is starting up" with 1-2-4-...-10s spacing for up to 300s, then exits
non-zero; a wrong password or missing database fails at once.
Per-request recovery is the engine's pool_pre_ping.
Also fixes the dev compose database image: postgres:16-alpine has no
vector extension, so migrations could not run against it. It now uses
pgvector/pgvector:pg17 like the other two files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment-only pilot for the comment standard (rule 32, milestone 544).
Issue and rule numbers, dates, run ids and incident stories come out;
the constraint each one guarded stays as a present-tense sentence.
History removed from the files, kept here:
- Dockerfile/CI: installing from uv.lock replaced pip/uv-pip installs that
ignored the lock (#2194, the mcp 2.0.0 break on 2026-07-28); --locked
then caught pgvector missing from the lock.
- CI: plugin paths were added after #2198/#2209 reached a live install;
`integration` joined the build's needs after run 8212 published
:latest with integration red (#4958); the `[[ ]]` tag step failed
silently on dash (2a374d9); BUILD_VERSION carried the channel until
2026-08-31 (#3298).
- jq was removed from the hooks in #4107.
- prod compose: the lenient DB healthcheck followed a runc setns crash
loop; pgvector arrived with migration 0067 via a PG16->17 dump/restore.
Corrections to match the code: README names the Settings tab
"MCP Access"; ci-requirements says the Dockerfile builds on node:22
and lint covers scripts/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings -> Profile (name, role, expertise, response style, tone, interests,
working hours) is typed by its user and neither derived nor a credential, yet
_NOT_INCLUDED named it, so a restore dropped it without a word.
BACKUP_VERSION 26. Both exports carry it; the restore remaps only the user, and
a blank field stays blank rather than restoring as the page's default. The
column and import guards cover it, and the tar round-trip restores one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The column becomes token_sealed (migration 0122 seals existing rows with a
frozen copy of the v1 key), every reader opens it through connection_token,
a row this instance cannot open is skipped with a warning, and Settings asks
for that token again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The restore route takes the tar as a raw body beside the JSON download.
scribe/restore.py reads either format, refuses a database that already
has an account (a restore re-creates everything with new ids, so into a
live instance it would duplicate rather than replace), has no size
limit, and says to restart so search indexes rebuild. An integration
test restores a written tar through the real restore_full_backup and
checks every id is remapped. docs/backups.md explains the file, why it
is uncompressed, both destinations, Fabled Archives as the worked
example, and restoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BackupScheduleSettings.vue leads with whether the data is backed up
(last good backup, or Overdue), then what is wrong (last failure in
words, an S3 secret this SECRET_KEY cannot open, incomplete config),
then the form: daily toggle and UTC hour, folder or S3 bucket, Test
destination with unsaved values, Back up now (polls while running,
re-armed in finally), and the last runs. The secret is write-only.
formatBytes moves to utils/bytes.ts for both views. Restore from
Backup now also takes the .tar, and says to use the CLI on a 413.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/backup_job.py reads its config when it fires, so startup reads
nothing (#4181) and a changed hour needs no reschedule. A run is held to
a 60-minute deadline (a hung NFS mount would otherwise hold the lock and
skip every later backup), never runs twice at once, and keeps its outcome
- file, size, duration, or the reason in words - in the bounded
backup_runs setting, which no backup carries. summarise() gives the page
its verdict: overdue when on and no success for 26h.
Admin API: GET/PUT /api/admin/backups, POST /backups/check (tests unsaved
form values, 45s deadline), POST /backups/run (202, background).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/backup_s3.py is FabledSteward's SigV4 client ported (put + list,
never delete, so an append-only key is enough), with its tests against
AWS's published signatures. services/backup_destination.py holds the
admin config (backup_config JSON) and delivers a dated tar to a mounted
folder or a bucket, and checks either without sending a backup.
services/secrets.py seals the S3 secret with Fernet under a key derived
from SECRET_KEY; backup._SETTINGS_NOT_EXPORTED keeps it out of every
backup. cryptography becomes a direct dependency (already in the image
through pyjwt).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
export_full_backup orders every query by primary key: an UPDATE moves a
row to the end of the heap, so an unordered export shifted every byte
after an edited row. services/backup_file.py writes the payload as one
dated tar per run - manifest.json plus <section>.jsonl, sorted keys,
fixed member headers - atomically via a dot-prefixed .part file, and
reads it back into the dict restore_full_backup takes, refusing a
missing or short section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings > General > Reply shapes. One menu row per shape (core, completion,
asks, plan): its title and when it arrives, how many of the operator's
preferences adjust it, and its deliveries over 30 days (in full / as a
reminder, "—" when the counts cannot be read). A verdict line leads. Opening a
row shows the shipped text, read-only because it is product, and the
preferences mounted on its moment; each opens in the rule editor, and "Adjust
this" opens a new preference already mounted on the shape's moment with a
starting trigger.
The rule editor takes an optional preset (kind, moments, trigger) and, opened
outside the Rules view, asks where a new record lives instead of dropping it.
GET /api/retrieval/reply-shapes now returns reply_shapes.overview: the
catalog plus mounted preferences (the same lookup that delivers them) and
counts from the reply_shape delivery rows. #5497 (step 5 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reply shapes are server product now, delivered at their moments, so the
skill stops restating them. Gone from it: the "Every reply" list, the
per-kind tables, and "The operator's own shapes come first" (preferences
arrive beside the shape on the same moments). It opens by saying where the
shapes come from (list_reply_shapes, the delivered core's header) and keeps
the reasoning: sections chosen not filled, a settled decision acted on,
placement from the record, who decides what, the assumptions an option
carries, the completion report worked in full, and the second pass. 13.5k to
10.7k characters.
The core gains the Finding kind the skill's table carried (2,150 of 2,200).
Tests follow the content: the kind and Approval-row pins move to the shapes,
a new test holds the worked example and the completion shape to the same
sections, and the guidance-ownership registry reads the delivered shapes as a
surface, owning the preference-wins and length topics there. using-scribe
points at the delivery and list_reply_shapes. #5496.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
report_preference searched one constant string when a task closed and handed
matches back as update_task's reply_preferences. Milestone 500 step 3 delivers
the reply shapes and the preferences mounted beside them on the moments, so
the arm goes, whole (rule 22):
- services/reply_preferences.py, the REPORT_PREFERENCE RuleArm, its re-scorer
and corpus entries, and the REPORTPREF constants
- update_task's reply_preferences block and its cue; the docstring now points
at moment_rules
- the fixed_query field and the fixed_query_never_clears warning: this arm was
the only one that set it, so the concept and the cannot_decline exemption
go with it
- the two Settings fields for its floor and budget
- migration 0121 deletes its rows (logs, judgments, rule-usage events, tuning
history, settings keys), operator-approved 2026-10-09. Without it the
readout would call the source unregistered and its surfacings would count
as ambient. The unindexed rule_usage delete is bounded by created_at.
#5496 (step 4 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The marker scribe_reply_check.sh writes after a completion-section hold sat in
the swept ledger directory as `<sid>.reportcheck`, and the sweep matches
`.ids` - test_every_session_file_in_a_swept_directory_follows_the_convention
caught it (run 799). Renamed `<sid>.reportcheck.ids` and added to the roster.
#5496.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Stop hook sent the finished reply twice: scribe_report_check.sh checked a
task-closing reply for the completion sections in shell and reported to
/report-check, and scribe_reply_check.sh sent the same reply to /reply-rules
for the rule hold. Now the reply goes once. When the turn closed a task the
hook adds the close count and ids, and the server runs the section check
(services/report_check, the same three patterns) beside the reply hold,
folding both into one reason.
The report_check adherence log is still written for every checked reply
(milestone 409's number). A section hold marks the session, so its rewrite is
sent back once with rewrite:true to record how it came out, and is never held.
The block reason carries the completion shape's one line and points at
list_reply_shapes rather than at the skill. #5496 (step 4 of milestone 500).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scribe_json_unescape wrote each escaped code point with sprintf("%c", n),
which writes the character only under gawk in a UTF-8 locale; gawk in the C
locale and mawk in any locale write the single byte n % 256. The server
escapes every non-ASCII character, so "·" reached the session as 0xB7, "—"
as 0x14 and an emoji as a NUL wherever a hook ran without a UTF-8 locale.
The decoder now encodes UTF-8 itself under LC_ALL=C. A new test sends what
the server sends from a bare environment, in three locales (#5495).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Every turn (/api/plugin/retrieve, UserPromptSubmit): the core reply shape
leads the payload - in full the first time, as its one-line reminder
after that - followed by whatever is mounted on reply.report, under the
shared rule ledger. Fresh keys come back as shape_keys.
- The ledger is <sid>.shapes.ids in scribe-priorart (scribe_shapes_file /
_seen / _append), so the compaction sweep that clears every .ids ledger
is what brings the full core back after one.
- At a moment (/api/plugin/moment): the slice for that reply - completion
on work.finish, asks on reply.ask, plan on work.plan - ahead of the
mounted rules. reachable_tools now lists tools reaching a shaped moment
even on an install with nothing mounted.
- Scribe's own tools (attach_moment_rules): reply_shape in the response,
in full, since that door has no ledger. enter_project carries the core
for clients with no prompt hook.
- Telemetry: one AppLog row per delivery (plugin / reply_shape), each
shape with full or pointer and the door (turn, hook, mcp).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The operator's line (2026-10-09), reconciling their 2026-09-21 ruling
(#4218, "you are the judge") with "not that Claude should make all
decisions": judging covers the work and its records - facts, findings,
measurements, whether work is done - and direction stays theirs:
priorities, what the thing should be, trade-offs only they can price,
anything they live with, and every act hard to undo or facing outward.
- reply_shapes: a one-line form in the core (every turn), the full text
in the asks slice (where a question gets handed back). Slice budget
1400 -> 1600 for it.
- reporting-back "You are the judge": "the operator reads the decision -
they do not make it" replaced; the line is drawn by who can see the
evidence, not by how hard the call is.
- plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services/reply_shapes.py is the single source of the default reply shapes:
the core (every reply, ~1,800 chars against a 2,200 budget) and three
slices - completion on work.finish, asks on reply.ask, plan on work.plan.
Read through list_reply_shapes (MCP, read-only) and GET
/api/retrieval/reply-shapes, one service behind both doors.
Nothing delivers them yet; that is step 3. The skill still carries its
copy until step 4 shrinks it to the long-form reference.
The software-only vocabulary guard moves into tests/helpers.py
(DEV_ONLY, dev_only_hits) rather than becoming a fourth copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner's docker daemon is shared, so `--filter name=<job> | head -n1` can
pick another repo's Postgres when two jobs run side by side (Steward run 8358,
Inkwell run 8653). Scope the lookup to this job's own task prefix and require
exactly one match. Done from Inkwell #5313 with the operator's permission; the
recipe is rule 79.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lessons share one register, so any two score well above unrelated text and
a fixed similarity bar sat inside that band: every no-rule lesson
"resembled" most of the others and the named group grew with the pool.
A neighbour now counts only when it stands above the lesson's own
background (median and MAD of its similarity to every lesson it can reach,
in that register's spread), and a group is a clique: every pair clears the
bar from both sides, so one broad lesson cannot join unrelated ones. The
bar, the minimum background and the fetch sizes are stated as defaults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A multi-pass DRY audit could not answer whether it created duplication:
a pass's own shortening can leave two statements identical to a third,
and no per-pass scan sees a copy that did not exist when it ran. The
Librarian retrospective improvised this measure in a scratchpad; this is
it as a tool the DRY Pass process can name.
6-line windows over significant lines (comments, blanks and bare
punctuation dropped, strings folded), grouped by glob or extension.
Revisions are read through git archive, so nothing is checked out.
Stdlib only, so any project can run a scratch copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dated release log (a note) matched a signed-APK CI snippet at 0.83 on
shared release vocabulary and was opened as a family candidate, then came
back as a precedent. One chunk of a long narrative scores like the shape
it mentions, so a cross-kind score does not mean "the same thing made
twice". The search is filtered to the new record's kind and the loop
checks it too. Code built from an idea written elsewhere still reaches the
engine through the citation trigger and the shape ledger.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each decision stored its reason three times (reason, before.reason, after.reason)
and a promotion carried the whole criteria reasoning in evidence. Precedents
(assess, promote, get_family_adoption, get_family_idea) and get_family_idea's
history now serve FamilyDecision.to_brief(): reason once, before/after as status,
canon_version and platforms, evidence as the cited list. list_family_decisions
and the decision echoed back by a write keep to_dict. The precedent count was
already bounded at five.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1 added family_ideas.topic_id but nothing could set it, so the note<->topic
link the milestone promised existed only as a column. set_family_topic links or
unlinks (0) a topic in a rulebook the caller owns, one idea per topic; the
version does not move. get_family_idea and get_family_adoption list the topic's
rules; list_family_ideas and the Family page name the topic. The model
docstrings no longer claim the topic's rules are platform-scoped in retrieval.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- enter_project carries a `family` key, but only when the project has
something to answer: counts of unassessed, owed and to-recheck answers,
each with the list_family_adoptions call that lists it. It shows on every
entry, never by platform touch: entry is when work is chosen, and an
unanswered idea is otherwise invisible.
- Retrieval: a widened project search (include_global_kinds) now also
reaches the canon ideas on the project's platforms. It also reaches their
references in the project's languages, or all of them when none matches.
An off-platform project gets none, and the plain project filter (the
duplicate gate) is unchanged.
- Closing a task returns `family_owed`, the owed answers filed while it was
open, and the report cue asks for them to be named.
- New plugin skill family-canon (moment work.record) covers when to
evaluate a promotion, answering in order, what counts as a reason, the
precedent reflex and the conflict order. _INSTRUCTIONS, create_note,
create_snippet, classify_shapes and reporting-back point at it.
The plugin version is minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- classify_shapes takes idea_id: the shape is judged against the idea's
reference in its language, or its first.
- A shape classified against a canon idea's reference moves the project's
adoption row (instance -> adopted, variant -> variant). Withdrawing the
shapes that gave an answer returns it to unassessed. This runs on
classify_shapes, the sweep, confirm_proposals and the coverage refresh.
- assess and undo refuse an answer the shapes contradict. A conflict
resolution re-judges the variant shapes on both sides.
- The proposer's family arm offers a canon idea's reference on a shared
platform, in any language. It proposes only when the reference is the top
hit over every readable snippet and scores at least 0.70. The pairs this
was measured on are recorded beside _FAMILY_FLOOR. The proposer is now v6.
- The matrix cell shows the code that answers it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
assess_family_adoption took system_ids for the task it files, which made
the family tool module look like a System-tagging door and tripped the
parity registry. It is not one: the owed task is tagged to the project's
System matching the idea's canonical area, and update_task retags it.
The unit test for each conflict ground passed evidence/conditions twice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- services/family_adoption.py: assess one project against one canon idea by
the four outcomes in order (exempt, variant, adopted, owed). Every outcome
needs a reason and adopted needs evidence. The engine records the precedents
itself: this idea's answers elsewhere, and this project's answers to the
nearest ideas. The same answer given twice records nothing.
- owed files a task in the OWING project, tagged to the System matching the
idea's canonical area, naming the gap and the reference for that project's
language. The task follows the answer: adopted closes it, exempt or variant
cancels it, owed again reopens it. Each move is logged on the task.
- the conflict order is enforced: every ground above the deciding one must
say why it did not decide. The losing side is folded into the idea's note as
a trap, an alternative or a condition branch, the version moves, and both
rows are answered against the revision.
- recheck is derived (row version != idea version). family.revise moves the
version when substance changes. undo covers a project's latest answer too.
- set_family_references names the reference implementations.
- MCP: get/list/assess adoption, resolve_family_conflict, revise_family_idea,
set_family_references. Web: GET /api/family/matrix.
- UI: an adoption matrix on /family (platform filter, cell detail with reason,
recheck and owed-task link) and the same matrix narrowed to one project on
its Family tab. The decision log now reads project-level decisions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the undone decision had no recorded before-state, the synthesized prior copied the canon idea's current applies_when and platforms into a retired row. Before that decision the record was not an idea, so the restored state now has neither. Caught by test_undoing_a_promotion_restores_the_prior_state_and_keeps_judged_rows (run 8281).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent promotes a family idea when all three criteria hold, and no person approves it. The criteria are product text: services/family.py states them, and the promote tool's docstring names every criterion the service enforces.
- Criteria: each one vetoes on its own when its reasoning is blank. Platform terms also needs an applies-when and a platform scope; proven also needs named evidence. A veto keeps the idea a candidate and is logged, so it becomes precedent.
- Precedent: every promotion stores the decisions on the nearest ideas by meaning, plus any the caller names.
- Promotion sets canon, the applicability test and the platform scope, and opens an unassessed ledger row for each member project the promoter can write. Re-promotion moves the version past every version the idea has held.
- Retire and undo: undo reverses only the latest idea-level decision, restores its recorded before-state, and logs itself with the undone decision as its precedent. Settled here: leaving canon closes the unassessed rows but keeps the judged ones, which read as needing a recheck after a re-promotion.
- Triggers open evaluations but never promote:
- a cross-project lineage citation ("matching #N") on a note or task write;
- a same-meaning record in another project on a shared platform, on create, at 0.80 (measured: the known pattern's builds scored 0.79-0.82, an unrelated project's best match 0.65);
- a milestone closing on a platform.
Each fails open and rides the response as family_hint.
- Doors: seven MCP tools, /api/family REST endpoints, and a Family page (nav, /family) showing the criteria, the ideas, and the decision log with undo and retire.
- utils/recordHref.ts holds the one copy of "where a record opens", now shared with LessonDetailView.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 8271 failed in the image build: "[vite:vue] Cannot read properties of undefined (reading 'scoped')" on moments-shared.css. plugin-vue caches one descriptor per src file and answers ?index=N from whichever component registered it last. RuleEditorSlideOver had moments-shared at block 2, while its other two importers have it at block 1. Step 2's new imports changed the transform order and exposed the bug.
- RuleEditorSlideOver: moments-shared is now loaded with an @import instead of a third <style src>.
- rules-shared.css sat at block 0 in the five panes and at block 1 in LessonDetailView, LessonEditorView and RuleEditorSlideOver, which is the same latent hazard. The three move to block 0.
- tests/test_frontend_shared_styles.py: a guard that every <style src> sheet sits at one index, plus a test that the guard can fail. vue-tsc cannot see this hazard, and vite build only runs after every test has passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project's platforms decide which family ideas reach it. This step makes membership answerable from every door:
- services/platforms.py: the global catalog (writes are admin-only and duplicate-gated by slug); pure marker detection; and membership reads and writes. Detection only ADDS, and only where nobody has answered. It never overrides a declared or rejected row and never removes one.
- coverage: the archive scan now carries every path, and the refresh runs detection fail-open.
- inception: a platforms choice (slugs, or null for unanswered). The list is the whole answer: members left out of it become rejected.
- MCP: list_platforms and set_project_platforms; enter_project and get_project carry the project's platforms.
- REST: /api/platforms (admin writes) and /api/projects/<id>/platforms.
- UI: a platforms checklist on the inception card, a Family tab on ProjectView, and a Platforms admin tab in Settings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When one project solves something every project on the same platform will
meet, that solution becomes family canon and every other project on the
platform answers it. This is the storage for that.
- platforms: a global catalog in the canonical_systems shape, seeded with
generic technology names and the file markers step 2's detection reads.
- project_platforms: declared / detected / rejected. A rejected row is kept
so detection cannot re-add what a person said no to.
- family_ideas: a note's family state. No new record type; any note, snippet
or lesson becomes an idea. A canon idea must state when it applies (CHECK).
- family_idea_platforms: the only scope source. A linked rule topic takes its
scope from the idea, so the two cannot disagree.
- family_idea_references: reference implementations, explicit not inferred.
- family_adoptions: one answer per (project, idea). Variant and exempt require
a reason (CHECK). Recheck is derived from the two canon versions, never
stored.
- family_decisions: the append-only log, with a required reason and the
earlier decisions each one followed. The agent decides with no approval
step, so precedent is what keeps its calls consistent.
Backup v25 carries all seven: platforms by slug, precedent ids remapped
through the decision map. Both column guards cover the new tables, and a
real-Postgres test exercises the CHECKs and the restore remaps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ordered straight off a *_embeddings table, the planner walks the HNSW index,
takes ~ef_search (40) nearest chunks across every owner and project, and only
then applies the scope: an in-scope record behind 40 nearer ones the caller
cannot see was silently dropped. #4958 fixed the rule search alone; the note,
milestone and system searches kept the fault.
_scoped_chunks builds the in-scope chunks with their distance; _rank_scoped
materializes them as a CTE and ranks exactly. All four searches go through it,
and the row shape is unchanged, so callers and mocks are untouched.
Tests: a structural guard that every semantic_search_* ranks through
_rank_scoped and orders nothing itself (with a replay of the old shape), a
compiled-SQL check of the MATERIALIZED CTE, and integration crowd tests - 80
nearer out-of-scope records - for notes (another user; the reader own other
project), milestones and systems.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_prompt_rule_hint, build_tool_rule_hint and build_write_path_hint
each ran the rule arm and then the via-lesson step by hand, the first two
passing the query between them through a _via_query key on the payload.
They now call one composer, _rule_moment, and the split helpers
(_prompt_rule_hint, _tool_rule_hint, _add_rules_via_lessons) and the side
channel are deleted. Output shapes are unchanged: the prompt builder
returns no checkpoint key, the tool builder always does, and
shown_rule_ids stays the direct band.
A structural test pins that only _rule_moment runs a rule arm or the
via-lesson step, and that the three builders are its only callers.
plugin_context.py: 2,418 -> 2,361 lines this step; 3,558 when the
milestone began.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each arm spec now carries its tuning pair (Surface, moved verbatim into
retrieval_pipeline) and a Declared block - what the telemetry readout
must know and cannot read off its rows. The two ranked stages that are
not arms (preference_slot, rule_via_lesson) are RankedSource specs, and
the note slots carry their own declaration.
- retrieval_surfaces.SURFACES = the TUNED_ARMS tuning, same order
- retrieval_registry.POINTS ranked rows = one per spec in RANKED; the
lookups, asked, ambient and pull rows stay declared there
- rule_usage.RANKED_SOURCES = RULE_RANKED_SOURCES + moment_rule
Settings keys, defaults, prose and order are unchanged (checked field by
field against HEAD). tests/test_retrieval_specs.py pins the derivation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
retrieval_pipeline gains the notes half: NoteArm / NoteSlot / NoteMoment /
NoteIO / NoteResult and run_note_arm, which writes once the stages both
notes arms copied: search, withhold this response's own menu (#3739),
fresh/repeat split, the call row before any return (#3497, #3752), the
band, the reserved slots in their order (reuse evicts, lesson extends),
and the surfacing rows. The note renderer (_record_kind, _menu_name,
_menu_passage, the seen pointer, menu_entry) moves with it, and
run_via_lesson_arm takes rule_via_lesson.
Behaviour-preserving, with flags for today's differences: notes still log
BEFORE the band and rules after it (step 7's question). One deliberate
change: a notes arm now fails open like the rule arms, so a failing
search costs its lines and no longer the whole hook response.
The I/O is resolved from plugin_context at call time (_note_io), so the
existing patches keep working. The review re-run reads the AUTO_INJECT
spec instead of restating it, and its guard now compares the two live
searches. The registry declares the pipeline's notes fan-out sites.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run 8218: integration failure, Build & push skipped, :dev still package
version 11098 (23:36:29Z, run 8217), and no image tagged 9a76eb0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverted next. Expect: integration red, Build & push skipped, :dev unmoved,
no image tagged with this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The integration job was added after the build gate (c6211e5) and never
joined its needs, so main run 8212 published :latest with integration red.
Both jobs carry the same ref condition, so the gate cannot be satisfied by
a skipped integration run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ordered straight off rule_embeddings, the planner walked the HNSW index,
which returns about hnsw.ef_search (40) nearest chunks across every owner
and project and filters by home only afterwards. A reader's own rule ranked
past the 40th chunk overall was silently dropped - on a shared install,
other users' rules fill those 40. This is the likely cause of the
intermittent test_integration_rule_scope failures, whose axis-vector
fixtures sit far from every real embedding in the graph.
The in-scope chunks now go through a MATERIALIZED CTE, which the index
cannot order, so the ranking over them is exact. A rulebook is hundreds of
chunks; exact is cheap. New test: 80 nearer rules belonging to someone else
no longer hide the reader's one rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
semantic_search_rules fails open, so both failures on run 8209 and main
run 8212 read as set() with no trace. Assert report["searched"] and carry
the swallowed traceback into the failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The open-after-moment signal proposes a mount; nothing proposed taking one
off, so a wrong mount was noise at every occurrence until someone happened
to notice. rule_misfired(rule_id, moment, why, reached_by) records a report
against a MOUNTED pair, counted per distinct day (the MCP door carries no
session id) on a new rule_moment_judgments.misfire column (migration 0119,
backup v24). At three days the response carries a line asking the agent to
offer the operator the fix - reject takes the rule off, unmap_action stops
the action reaching the moment, confirm keeps the mount and stops the
asking - and Settings > Moments lists it as an unmount proposal with the
reasons and the actions that reached it. A re-mount clears the count.
Taught in moments.md, missed-retrieval.md and the reply hold's wording.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Until now only the tool arguments knew moments existed. The guidance
surfaces described rules as reached by resemblance alone:
- using-scribe: a short reflex paragraph and a new reference file,
moments.md. It covers reading "at <moment>, reached by <action>", the
reply held once at reply.report, map_action / unmap_action offered in
one line, and the step 7 proposal line answered with judge_rule_moments.
- writing-records: asks WHEN a rule applies as well as what it is about.
A rule, preference or process about a point in the work gets
moments=[...] as it is written, and the trigger stays as the net.
- missed-retrieval: a missed WHEN is mounted or mapped, not reworded. A
misfire is unmounted or unmapped.
- _INSTRUCTIONS: one clause (list_moments; mount rules about WHEN),
1594 of 1600 chars.
- static context: injected lines include the rules mounted on a moment
that was reached.
- test_guidance_ownership: three owned topics, so the text cannot quietly
drop out.
Plugin minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule written before moments existed is mounted on nothing. Step 7 records,
per (rule, moment), whether it belongs there and who said so:
- rule_moment_judgments (migration 0118, backup v23): suggested / confirmed /
rejected, from a pass, the signal, or an edit. Moment "" is "no moment fits".
- The pass: rules_to_mount lists unjudged rules; propose_rule_moments records
suggestions that mount nothing; rule_moment_proposals and
judge_rule_moments put them to the operator. A confirm mounts, a reject is
kept so the pair is never proposed again. Same service behind REST and a
"Waiting on you" panel in Settings > Moments.
- Edits are judgments: set_rule_moments, the one mount write path, confirms
what was added and rejects what was removed in the same transaction.
- The signal: scribe_moment.sh keeps a per-session acts ledger; when a rule
is opened, scribe_record_opened.sh sends the last three minutes of it to
/api/plugin/rule-opened. The acts resolve through the install's mappings;
work.run and work.change are not evidence. Counted per distinct session
with lesson_rules' evidence model, and once due the open returns one line
asking the reader to offer the mount.
- scribe_session_end.sh removes the session's scribe-moment files.
Plugin 2026.10.05.2003.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Everything an agent can do with moments, a person can now see and change
in the app.
- Rule editor: a moment picker beside the trigger. Catalog moments are
ticked; a named procedure's `skill.<name>` is typed and checked as the
server checks it. `moments` is always sent, so unticking the last moment
unmounts the rule.
- Settings, Moments section (General tab): for each moment, what it
means, the actions that reach it on this install (shipped ones can be
switched off, the install's own removed), how many rules are mounted on
it, and deliveries and agent opens over the window. Below that: named
procedures with mounts, switched-off defaults with Restore, and a form to
add an action.
- retrieval_telemetry.moment_usage: per moment, `delivered`, `rules`,
`opened` (agent pulls after the first delivery there; an upper bound, as
by_source is) and `last_delivered_at`. No ratio, because a mount is a
person's statement, not a ranker's guess. Guarded on its own, and also
reported in retrieval_summary as `moment_usage`.
- rulebooks.mount_counts; mounted_moments now derives from it.
- GET /api/retrieval/moments carries `mounted` and `usage` (?days=).
DELETE /moments/mappings also reads the mapping from query parameters,
since the browser's DELETE sends no body.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Loading a procedure now also reaches the moment it is for. Loading the
reporting procedure is a report; loading the release procedure is a delivery.
- Bundled skills: each SKILL.md declares `metadata: moments:`. The same
declaration ships as Skill defaults (BUNDLED_SKILL_MOMENTS), because the
server never sees the plugin's files. test_skill_moments holds the two
together and pins the plugin name that qualifies the skill.
- Stored processes: `moments` on create_process and update_process, stored
in the note's data and returned by get_process. A `scribe-proc-<slug>`
load resolves its process through the sync manifest at load time. The
moments are not copied into the stub, which would go stale mid-session.
- reachable_tools lists the skill loader whenever anything is mounted, since
a process's moments are known only when it loads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moment_delivery records the mounted half's surfacing under
source=rp.MOMENT_RULE_SOURCE. That is an attribute the registry's
extractor cannot resolve, so the site is now declared with the value it
emits, read from the pipeline's own constant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reply is the one act no tool call marks, and it is where "let me
know if it works" gets said. A new Stop hook (scribe_reply_check.sh)
sends the finished reply to POST /api/plugin/reply-rules, which checks
it twice:
- mounted: every unopened RULE on reply.report, plus reply.ask when the
reply asks a question. Deterministic.
- semantic: the reply's head and tail against every rule's trigger, on a
new ranked surface, reply_rule. It is the backstop for whatever the
earlier arms missed. Its floor is its stop bar (default 0.80, budget
1), with its own Settings dials. The new stop_only stage records
surfacing for the rule that holds and nothing else, because nothing
else reached anyone.
Following the operator's ruling from 456 step 8, a rule that holds blocks
once, in the server's words. The hook blocks only on a reason it was
given, so an unreachable instance never stops a session, and it never
holds the rewrite. The ledger is the act checkpoint's own, so a rule
holds a session once across both doors and the per-session cap counts
both.
The turn reader moved from the report check into scribe_defs.sh
(scribe_turn_facts / scribe_turn_fact), so the two Stop hooks read a
turn the same way. The output was checked identical on a real transcript.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two guards caught 7865313:
- test_mcp_tool_processes reads every coroutine in a tool module's
namespace as a tool, and a name-imported attach_moment_rules looked
like one. All seven modules now call moment_delivery.attach_moment_rules,
and the parity guard accepts the attribute form.
- The session-ledger convention: a file in a swept directory must be a
.ids ledger. The tool-list cache describes the install, not the
context, so it moves to its own directory, scribe-moment, where a
compaction does not sweep it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A rule mounted on a moment now reaches the session when an act reaches
that moment, with no semantic match involved:
- run_moment_arm on the pipeline: a lookup, not a ranked search. Each
line names the moment and the act that reached it ("at work.deliver,
reached by `git push`"), so a misfire is visible where it lands and
can be unmapped in-session. A repeat is cited, not quoted; fresh
rules are recorded surfaced under source moment_rule with the moment
in detail. No retrieval_logs row, as for the other lookups, so no
latency is persisted for this arm.
- rule_scope: a rule's home clause, moved out of semantic_search_rules
so the moment lookup scopes by the same one.
- rulebooks.rules_on_moments / mounted_moments.
- The plugin door: a catch-all PreToolUse hook (scribe_moment.sh). It
keeps /moment-tools' answer on disk for five minutes, so a call to a
tool that cannot reach a mounted rule sends nothing, and an install
that has mounted nothing sends one request per window. It shares the
rules ledger with the other arms and fails open silently.
- The MCP door: Scribe's own tools named by the shipped mappings carry
moment_rules in their response, so a client without the plugin gets
them too. The hook skips those tools. A guard pins the attach on
every one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rule_moments (migration 0117) records which moments a rule arrives at, by
catalog name, cascading with the rule. rule_detail, the one seam every
rule door already returns through, gains moments beside system_ids: None
leaves the mounts alone, a list replaces them. get_rule and both
list_rules doors read them back, batched per page.
All five MCP rule/preference writes and the three REST ones take moments
and validate them before their create or update. An unknown name is
refused with the catalog listed and leaves no half-made rule behind; a
parity test pins that ordering on every door.
Backup v22 carries the mounts as a join table remapped through the rule
map; a real-Postgres round trip checks they land on the restored rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moment_actions.resolve(tool, input) names every moment a call reaches and
the action that reached it. One call can reach several: kubectl apply is
a run, a deliver and a reach outside the workspace. Command tools match by
how each segment of the line starts, with a word boundary; other tools by
field=value arguments. The MCP server prefix and case are ignored.
56 shipped defaults cover the harness tools, Scribe tools and common
command shapes. moment_mappings (migration 0116) holds what an install
adds and the defaults it switches off. A removal is a stored row, so an
upgrade does not switch the default back on.
Per the operator ruling, corrections happen in the session: map_action
and unmap_action (write tools) return now_reaches so the fix can be
confirmed in the same reply. list_moments now shows each moment's
actions on this install. REST mirrors both doors, recorded as human.
Backup v21 carries the mappings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fourteen generic moments of work (session.start, work.start … reply.ask)
plus the skill.<name> family, each with what it means and the kinds of
action that reach it, written for any kind of work rather than software
alone. The catalog is code because every install needs the same mount
points; which actions reach a moment is per-install data (step 2).
require_moment refuses an unknown name with the catalog listed, so a
typo cannot become a mount that never fires. list_moments (read-only)
and GET /api/retrieval/moments hand out the same catalog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
reply_preferences.completion_preferences was the last hand-written
rule search + record_retrieval + record_rule_surfaced triple outside the
pipeline. It is now REPORT_PREFERENCE, a RuleArm with kind="preference"
over COMPLETION_QUERY, read back as records rather than as lines.
- RuleArm gains `kind`. _ranked asks the ranker for that kind and drops
anything else on the way out, before the row is logged.
- RuleResult gains `shown`, the (score, rule) pairs behind the lines, for
callers that return records.
- RuleMoment.project_id may be None: logged as given, searched as
`project_id or None`. report_preference rows keep their NULL project.
- Guards: reply_preferences now has 0 direct rule searches. The registry
constant-source example moves from report_preference to preference_slot,
the pipeline slot that records through a module constant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 8137 had two failures, both source-scanning guards whose property
moved to the pipeline. The property itself still held:
- test_every_surface_name_is_a_real_telemetry_source looked for
source="<name>" literals. The rule arms now record under their spec field,
so the spec is the join key, read from RULE_ARMS.
- test_every_hook_rule_search_says_which_project_it_is_for counted 4 direct
searches in plugin_context. It now expects 0 there, so a copied arm fails
the count. It also asserts that the pipeline has exactly one search and
that its keyword literal carries project_id and never everywhere.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prompt_rule, pre_tool_rule and the rule half of the write path each wrote
the same steps out by hand: search, band, split fresh from repeats, log the
call before any early return, reserve a slot, render, record surfacings.
The copies drifted, and #3497, #3750 and #3752 were fixed one copy at a time.
- New services/retrieval_pipeline.py. run_rule_arm runs the stages in one
order. RuleArm is the spec (source, band, compact_tail, checkpoint,
preference_slot). RuleMoment is the query and the session ledger.
- The I/O (ranker and recorders) is passed in as RuleIO. plugin_context
resolves it from its own names at call time, so existing patch points
still apply.
- _rule_band, _rule_hint_line, checkpoint_for, checkpoint_reason, the band
constant and the preference slot moved into the pipeline unchanged.
plugin_context re-exports them.
- A recorder that raises now costs only its row, never a rendered line.
- The flags reproduce today exactly. Whether prompt_rule should band, and
whether the act arms should reserve a preference, are step 7.
- Registry: the pipeline call sites are declared in FAN_OUT_SITES, with
values read from the specs.
- The #3497 structural guard now checks the one implementation, and that
plugin_context writes no rule-source row of its own.
plugin_context.py: 3,558 -> 2,980 lines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single plugin request fans one query out to several ranked arms. The
operator message is searched by auto_inject, its reuse and lesson slots,
prompt_rule, the preference slot and rule_via_lesson, and each arm called
get_embedding on its own: up to six model calls for one vector.
- embeddings.query_embedding_memo(): a request-scoped ContextVar memo that
get_embedding consults. Outside a scope the default is None, so every
other caller is unchanged. A failed embedding is not remembered.
- memoized_query_embeddings decorates /retrieve, /tool-rules and
/prior-art.
- get_embeddings (document chunks) is untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4797 "A lesson's what takes a whole narrative, and the narrative becomes
its name". `what` is the title every listing and menu prints. Nothing
enforced its documented "one line", so lessons written with the incident
in `what` printed up to ~1,500 characters as a menu line, buried the
claim, and diluted the trigger in the embedded title. That was 14 of the
39 lessons on this install.
- lessons.require_claim refuses a `what` over WHAT_MAX_CHARS (240) or
running over several lines. The refusal says the story goes in
`insight`.
- Both doors call it before writing:
- MCP create_lesson / update_lesson;
- REST create / update.
An update checks only a NEW name, so a lesson stored with a long one can
still take the edit that repairs it.
- lessons.claim_line is the display half. _menu_name shows an over-long
stored name as its first sentence marked " …", because a door guard
does not undo rows already stored and an unrepaired install would keep
printing them.
- The tool docstrings state the bound.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4798 "The rules corpus's surfaced_never_pulled warning sends the reader to
menus_to_review, which can only review auto_inject". #4772 gave the warning
one remedy for both corpora: "judge a sample with menus_to_review". That is
right for notes, where a line carries its passage. For rules it is a dead
end: retrieval_review.REVIEWABLE holds only auto_inject, so the tool
refuses every rule arm.
- The reading is now per corpus (_NEVER_PULLED_READING). For rules, the
text says:
- the count includes rules that only arrived in a listing;
- a rule can rightly be set aside on its trigger alone;
- no judged sample exists for the rule arms;
- a rule set aside again and again is a trigger to fix (update_rule
when_to_apply, then what_might_apply), not a floor.
- The tool docstring says the same.
- The guard is tied to REVIEWABLE, so it can fail in both directions: if
the rules text names menus_to_review while no rule arm is reviewable, or
if a rule arm becomes reviewable and the text still says there is no
sample.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4796 "A record the operator names by number reaches auto-inject only if
its wording happens to match". "yes go ahead with 4448" says which record
is meant, but a number means nothing to an embedding, so the prompt menu
filled with records resembling the words around it.
- record_refs.named_record_ids reads the operator's raw prompt, never the
reply-enriched query:
- `#N`, unless the word before it marks another numbering (PR, CI, rule,
milestone, system, log...);
- a bare number of 3 or more digits that opens the message, follows a
reference word or continues a list one started;
- never a quantity ("300 seconds"), a date, version, path or fenced code.
- Each id is resolved through the ACL check; trashed or inaccessible ids
are dropped.
- A "Named in your message" block leads the menu: the kind, the System,
the name and the opening of the body, or the seen pointer if the record
is already on the ledger. It takes no share of top_k, and the semantic
lines leave those ids out.
- The block is booked under the new `named_ref` source, registered as an
unbidden lookup that is allowed to be quiet. A named id that also ranked
counts as suppressed in the auto_inject row, so #3668's identity holds.
- Named records now arrive even when the search finds nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SDK 2.x passes on only a ToolError's text. Any other exception becomes
UnexpectedToolError("Error executing tool X") and its message stays on the
server. ValueError is how every Scribe tool refuses — what was refused, why,
what to do instead — so every refusal reached the agent bare, and it could
only retry blind. Seen live: tune_retrieval(actor="operator") and
judge_menu(verdicts=[]) both answered "Error executing tool …" and nothing
else.
StrictArgsMCPServer.call_tool re-raises an UnexpectedToolError caused by a
ValueError as a ToolError carrying the message, in the SDK's own
"Error executing tool X: <reason>" shape. Other exceptions are crashes and
stay masked. The stale comment claiming the SDK returns ValueError text is
corrected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`<agent-message …>` turns were scored by auto-inject and the prompt rule
arm as if the operator had typed them, spending a retrieval and writing a
retrieval_logs row that reads as a real message (log 54355, found by the
#4772 review). Same class as #4142. Matched up to the tag name, since the
tag carries attributes; `<agent-messages>` and the tag named mid-sentence
are still retrieved against.
Plugin 2026.10.04.0125.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Judging live auto-inject menus (#4772) found "## Work log — 2026-08-21",
"## Findings worth carrying forward" and "Two dev→main PRs this session."
each embedded as a whole chunk. Text about nothing embeds close to
everything, so they took ranks 1–3 on vague queries and pushed real records
down.
Three ways the chunker made them, each closed:
- _split_paragraphs flushed a heading on its own when the paragraph under it
was long. A thin lead is now carried into the paragraph that follows, and
a hard cut never lands in the first quarter of the budget, where the
newline it finds closes a heading.
- A heading with no body (above its ### parts) became a section of its own
when the section before it was full. A thin section now takes the next.
- A short lead-in or a short last section stood alone. The lead-in joins
what follows; a thin tail joins what precedes it.
A continuation piece now repeats the LAST heading before it, since one
section can now hold several. CHUNKER_VERSION 2 → 3, so the startup backfill
re-embeds; tuned dials will report their shape as changed, which is true.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first live sample ranked records the call could never have been offered:
the session that made a call writes the decision, often quoting the message,
and that record tops the re-run. Judged, it inflates on_point exactly where
the budget is decided.
- _rerun over-fetches by POSTDATED_SLACK, drops records created after the
call before ranking, and names them in `postdated`.
- each line carries `changed_since_call` (updated_at or a work log after
the call) and `logged` (shown fresh then).
- judge_menu shares the re-run, so a post-dated record cannot be judged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An open rate cannot say whether a menu line related: every line carries its
matched passage (#4364), so "not opened" covers unrelated, enough as shown,
and already in context. #4772 "Injected notes are never judged".
- retrieval_judgments (0115): a reviewer verdict per line of a logged call,
on_point / adjacent / unrelated, with its reason, rank, budget side and
whether the agent opened it within the hour.
- menus_to_review re-runs a random sample of unjudged auto_inject calls with
the arm's own parameters, past its budget, passage on every line.
judge_menu records verdicts, re-deriving rank from a fresh re-run.
- retrieval_telemetry gains a judged block (by rank, within/beyond budget,
on_point_unopened). surfaced_never_pulled stops blaming titles.
- missed-retrieval guidance names the review before a budget move.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4769 "Rulings are counted where someone will read them": milestone 444
step 4 wrote system_usage_events and nothing read it.
- retrieval_telemetry gains a `system_usage` block: surfacings and opens by
source, distinct counts, and `by_system` naming the areas most shown.
There is deliberately no pull-through ratio, because rulings travel in full
in the line and opens are the exception.
- usage_for_systems (one GROUP BY) adds `usage` to the REST Systems list and
detail, and to MCP get_system. MCP list_systems is unchanged.
- The Systems UI shows a "rulings shown N×" chip.
- rulings_pre_tool, rulings_write_path and mcp_get_system are now declared
registry points; the registry guard covers their recorders.
- The Systems store merges a PATCH reply instead of replacing the row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4768 "Write-path prior-art route 500s when a nameless search hit carries
its excerpt under `snippet`": _prior_art_line read item["snippet"] as a
snippet record's field dict, but a search hit carries its matched passage
there as a string. Read the name only from a dict.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI 7939 went red on test_the_hook_and_the_route_agree_on_every_parameter_name:
the Bash hook now sends root and cwd, and the guard pins the exact set. Both
guards (tool-rules and prior-art) now also pin seen_ruling_systems, spelled
once in scribe_rulings_query, and check the routes read all three.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A System's rulings (the Rulings section of its description) now reach the
work by path, not by similarity. Both PreToolUse arms resolve the files a
command or edit names to the Systems whose path_patterns cover them, and the
first touch in a session shows each area's rulings in one line; a repeat is
a one-line reference. A lookup, so no floor, no budget, no retrieval_logs row.
- services/system_rulings: parse_rulings, command_paths (reads and writes,
relative to the repo root from any cwd; flags, URLs, globs skipped),
rulings_for_paths
- /tool-rules takes root, cwd and seen_ruling_systems; /prior-art takes
seen_ruling_systems; both return ruling_system_ids
- hooks share <sid>.rulings.ids (cleared on compaction by the ledger naming
convention); the Bash hook sends the repo root and cwd
- system_usage_events (migration 0114): surfacings by source, pulls from
get_system; carried by backup (v20) through the system map
- writing-records: rulings also arrive when the area's files are touched
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A System gains path_patterns: globs relative to the repo root (* within one
directory, ** across any depth, a plain directory covering everything under
it). One service validates them for every door, so the web UI and MCP refuse
the same bad pattern with the same message. systems_for_paths resolves paths
to every active System that covers them, which step 4 (#4757) uses to deliver
an area's rulings when its files are touched.
- schema: systems.path_patterns JSONB NOT NULL default [] (migration 0113)
- service: normalize_path_patterns, path_matches, systems_for_paths
- routes + MCP create_system/update_system accept it; [] clears
- web UI: a Files field in the create and edit forms, patterns on the card
- backup carries it through export and restore
- using-scribe reflex 7: tagging work keeps a System's files current
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Librarian session contradicted a decision the operator had made 13 days
earlier. The ruling ("retry, then replace, never give up on a book") was
kept only as a quote in a work log, beside a session's reading of it that
capped replacements at 3. Three later sessions built on the reading, and one
carried the cap into an option as a "known cost", which the operator then
approved without being asked about it.
- writing-records.md: "A ruling goes on the System it governs". What a
ruling is (the operator decided it; a later change could undo it), how it
differs from a rule, and where it goes: a Rulings section at the end of
the System description, one line each with who, when and the source
record. Written the turn the operator decides; holds what is in force,
not history; a charter line that contradicts a ruling is fixed in the
same edit.
- using-scribe SKILL.md: reflex 1 says code tells you what a thing does,
not what was wanted, and a limit read from code is unconfirmed until a
System's Rulings says otherwise. Reflex 9 points to the ruling section.
- reporting-back: an option that carries existing behaviour says whose call
it was (the operator's ruling, or a past session's never confirmed); one
that contradicts a ruling is a Conflict.
- create_system / update_system docstrings: the Rulings section, and that
description replaces the whole text.
- test_guidance_ownership: three topics pinned to their owners.
- Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lesson editor now asks the question the create tool asks, while the
writer still has the situation in mind. It offers three answers:
- an instance of a rule, ticked from the rules the lesson resembles;
- "No rule fits", with a required reason;
- leave it open.
The rules are fetched before the save from a new GET
/api/lessons/rule-candidates. It runs the same rule_candidates search the
create door replies with, and passes "search unavailable" through as null,
apart from "nothing resembles it".
An edit sends an answer only when it changed. Re-sending the same rules
would re-stamp their judgments. Worse, a shared editor who cannot see the
owner's rule would reject it by sending a list without it. Leaving a linked
lesson open sends rule_ids=[], which is what clears the links.
"Leave it open" is not offered once "no rule fits" is on file: the server
has no way to take that answer back except by naming a rule. When "no rule
fits" completes a convergence group, the editor says so in a toast, since
the page it goes to does not recompute it.
Guards check that the payload uses the names both routes read, that the
three answers are offered, that the candidates route sits above the id route
and keeps null apart from [], and that the editor reuses .rule-chip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lesson page gains an "Instance of" panel that holds one of three answers.
Unjudged is the fall-through, so it is stated rather than left blank:
- the rule(s) it was judged an instance of;
- "No rule — <why>";
- "Not yet judged".
Suggested links show what they rest on (distinct situations, and projects
when more than one), with Confirm / Not an instance for a reader who can
write. Rejected links stay listed with their reason.
The rule slide-over lists the lessons that are instances of it, plus the
suggestions waiting on a judgment. Each entry links through to the other
record, and kind and state wear the existing .rule-chip.
The write check moves to utils/permission.ts. The copy on the snippet page
looked for "edit", which the server never sends, so shared editors saw a
read-only page (#4640 "The snippet page hid its edit controls from shared
editors"). Guards pin the client's unions and write levels to the
service's, the model's and access.py's own values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a lesson is answered "no rule fits" (create_lesson / update_lesson on
both doors), the response looks for other no-rule lessons it resembles and,
once there are CONVERGENCE_LESSONS (3) of them, carries `convergence`: the
members, their incidents and projects, and a hint to draft the missing rule
with create_rule (operator approval as always) and point each lesson at it —
or to leave them as lessons when no single choice is right every time.
- convergence_group is the pure bar: distinct LESSONS count, incidents never
stand in for them (one broad lesson cannot trigger it), and a group whose
sources all point at one incident is one event written up several times.
- convergence_for searches lessons by the new one's claim + trigger
(trigger_title) at CONVERGENCE_THRESHOLD 0.65 — above the menu's "worth
showing", below the duplicate gate's "same record" — then keeps the ones
with a lesson_no_rule answer. Fail-open. No sweep, no timer (#4183).
- Defaults stated as defaults (rules 32, 115).
- Tests: the bar (pure), the search with stubs, the door, and the no-rule
filter against Postgres; conftest stubs convergence_for for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 712 failed test_neither_rule_arm_logs_its_call_behind_a_results_guard
with "min() iterable argument is empty": the guard walked the function named
build_tool_rule_hint, which since fd2ebf4 is a wrapper that searches nothing
— the arm's body moved to _tool_rule_hint. The property it guards (the call
is logged before any early return) still held; the guard had pinned a name.
It now finds the async function that calls semantic_search_rules under the
pre_tool_rule source, so a later rename moves the guard with the arm instead
of emptying it (rule 167).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On all three rule arms (prompt_rule, pre_tool_rule, write_path_rule), after
the direct match, a lesson matching the moment at the notes menu's own bar
brings the rule(s) it is CONFIRMED to be an instance of, rendered in rule
voice with "Reached through lesson #N “…”, a recorded instance of it."
- Confirmed links only (lesson_rules.confirmed_lessons /
confirmed_rules_in_scope). A suggested link carrying its rule would
manufacture the co-arrival #4637 counts and prove itself.
- Scope kept: a lesson surfaces everywhere, but the rule it brings must be
global or this project's own — milestone 414's boundary, not reopened
through a side door.
- Suppression is by RULE: anything the direct band named or the session
ledger holds is skipped, whichever lesson reached it.
- Its own slot (VIA_LESSON_LIMIT = 1), not a rule slot — a stated default,
since the plan's "decide by measurement" has nothing to measure until
links are confirmed (#4632). Its own source, rule_via_lesson: registered,
RANKED for pull-through, logged whenever it searches.
- Nothing searched at all while no lesson carries a confirmed link, which is
every install until one is judged.
- build_prompt_rule_hint / build_tool_rule_hint are now thin wrappers over
the direct arms (_prompt_rule_hint / _tool_rule_hint); the arm leaves its
query in `_via_query` only when it ran, so a disabled arm or a blank prompt
brings no rule in either, and the key never leaves the server.
- Via-lesson rules are not fed to the #4637 co-surfacing recorder: only
direct matches are evidence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When one hook response puts a lesson and a rule in front of the reader, the
pair is recorded as evidence on a SUGGESTED lesson_rule_links row; once the
pair has arrived together in PROPOSE_SITUATIONS (3) distinct situations, the
next co-arrival carries one line asking the reader to judge it with
judge_lesson_link. Nothing about surfacing changes: a suggested link carries
no rule anywhere (that is #4633, confirmed links only).
- lesson_rules: co_surfaced (fail-open; only pairs the reader could confirm —
a lesson they may write, a rule they own; judged pairs gather nothing; one
proposal per response; PROPOSE_COOLDOWN 6h between asks), plus the pure
counting rules: situation_key, add_evidence, proposal_due, evidence_summary.
A situation is the prompt on /retrieve (word tokens, sorted and
de-duplicated, so trivial rewordings count once) and the FILE on
/prior-art (every edit to one file is one situation).
- rules_for_lessons shows a suggested link's evidence counts.
- plugin_context: build_autoinject_hint returns lesson_ids,
build_prompt_rule_hint returns shown_rule_ids, build_write_path_hint records
its own pair; routes/plugin /retrieve records the prompt pair. Shown lines,
repeats included: relevance makes a co-arrival, not the session ledger.
- Evidence lives in the existing evidence column — no migration, and backup
already carries it.
- Tests: counting rules (unit), the recorder against Postgres (bar, repeat,
judged pairs, ownership, cooldown, evidence kept on confirm); conftest stubs
co_surfaced for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 707 failed test_the_trigger_separator_is_spelled_in_exactly_one_place:
rule_candidates joined the lesson's claim and trigger with an inline " — ",
a second spelling of embeddings.TRIGGER_SEP (#3207). It now calls
trigger_title — the same join a rule's own document title is built with,
which was the point of the query shape in the first place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Which rule is this lesson an instance of?" now has three recorded answers:
a rule named (a confirmed link, #4630), no rule fits (new), or unjudged.
- Model + migration 0112: lesson_no_rule (lesson_id PK, CASCADE from the
note; why; judged_at). A table rather than a key in notes.data, because
that mirror is re-composed from the body on every edit and would erase it.
- Service (lesson_rules): set_no_rule rejects any confirmed link with the
reason; a confirmation (set_lesson_rules or judge_link) deletes the answer;
require_one_answer refuses both answers in one call before any write;
judgments_for_lessons + attach_lesson_rules add rule_judgment (and no_rule)
to every lesson payload; list_unjudged lists the open ones; rule_candidates
searches rules with the lesson's claim + trigger at the explicit-search bar,
None when the search could not run.
- MCP: create_lesson/update_lesson take no_rule; an unanswered create returns
rule_candidates, rule_judgment and a rule_hint; list_lessons(unjudged=true).
- REST: the same on POST/PATCH /api/lessons and GET ?unjudged=1; create
returns rule_candidates.
- Backup v19: a lesson_no_rule section, export (full and per-user) and import.
- Guidance: create_lesson docstring, writing-records.md in using-scribe (owner,
pinned in test_guidance_ownership), create_rule docstring on linking the
lessons a new rule governs. Plugin version minted.
- Tests: door units, integration for the three states, the rejection reason,
scoping, cascade; backup registries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tool-classification guard (test_mcp_auth) failed CI run 705: the new
judge_lesson_link tool sat in no set, so a read key would have been silently
denied it. It changes a link's state, so it belongs in _WRITE_TOOLS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The link between a lesson (one concrete situation) and the rule that governs
it, with the operator's soft-then-hard design built into its state:
suggested while evidence accumulates, confirmed or rejected once judged. Only
confirmed will carry a rule in retrieval (#4633); rejected is kept so the pair
is never proposed again.
- models/lesson_rule_link.py + migration 0111: one row per (lesson, rule),
CASCADE on both ends, indexed both ways, CHECK on state (rule 36), evidence
JSONB and judged_at.
- services/lesson_rules.py: require_rules (validated before any write, so
a bad id leaves nothing half-linked), set_lesson_rules (set-semantics;
a dropped rule becomes rejected, not forgotten), judge_link, and the two
reads. ACL: write on the lesson (share-aware), ownership of the rule; a
reader sees only rules they own. Decorations are fail-open (#4286).
- MCP: create_lesson / update_lesson take rule_ids; get/create/update return
`rules`; new judge_lesson_link tool. REST: the same on /api/lessons plus
PUT /api/lessons/<id>/rules/<rule_id>. Rules: rule_detail carries `lessons`.
- Backup v18: export (full and user-scoped, both ends in scope), builder,
importer; both column guards register the table.
- Tests: integration (states, set-semantics, judge, ACL all-or-nothing,
cascade both ways, CHECK, one row per pair); unit (door wiring, judge
registered, migration/model state agreement, backup skip and unjudged
stays unjudged). conftest stubs the decorations for unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Milestone 385 step 8 (#3735 "a lesson is recalled on a project it was not
written on") is defined by opened-on-another-project. The reader's project has
been recorded on every usage event since 0c8e109, but nothing compared it with
the record's own, so the criterion was still unreadable.
- note_usage.usage_for_notes: a second aggregate in the same session joins
notes and counts surfaced_away_count / pulled_away_count. Counted only where
both projects are known and differ; ranked surfacings only (#2477). The
first aggregate is untouched, so events on deleted notes still count.
- empty_usage carries both keys zero-filled; every door that attaches usage
(list_lessons, get_lesson, snippets, knowledge) gets them through
attach_usage.
- UsageBadge tooltip says "On other projects: surfaced N×, opened M×" when it
happened, and nothing when it did not.
- Tests: the mocked split test feeds both aggregates; a unit test for the
away counters; a real-Postgres test that home, unreported and ambient
events are all left out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- reusing-code: "Before the turn ends — say what you built" — the four
verdicts, the one classify_shapes(repo=…) call, and the component file as a
shape. Description names the end-of-turn moment.
- shape-accounting: the writer judges; audits are the check that it held.
The write path SUGGESTS (no more hook instances); component file rows and
whole-file canon described; scoped covers Svelte too.
- _INSTRUCTIONS reuse line: "before the turn ends, say what you built
(create_snippet the reusable, classify_shapes the rest)" — 1570/1600.
- Coverage line: "written-shape check (7d): N turns checked, M asked, K left
unjudged", from the Stop hook's recorded outcomes; silent until the
question has been put.
- test_guidance_ownership pins the new topic on reusing-code.
- prior-art hook header no longer says it stamps instance rows.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single-file component defines `.card`, `.title` and a `Props`, never
anything named after itself — so the ledger could not hold "StatusChip is
canon" and could not say a new card was built where one already existed.
- coverage.is_file_unit: a file that RENDERS (its markup names classes, or it
opens a <script>/<template>/<style> block) and defines nothing named after
its stem gets one `file` row. Structural, not a framework list: Svelte and
Vue components qualify, a TSX component already has its function's row, a
module is accounted for by its definitions. Emitting every file would have
put every module of every project in the todo at once.
- No body fingerprint on file rows, so an edit never re-asks a judgment.
- `file` is its own form and family; derive grouping skips it (`index`,
`+page` repeat by convention). Divergence buckets it on its own, so a new
component where a component canon dominates is a fair question.
- mark_canonicals: a snippet recorded at a path with no symbol makes that
file row canonical; it still covers no definition inside.
- The write hooks apply the same test before noting a new file for the
end-of-turn question.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hook used to land its evidence as `instance` rows, classified_by=hook —
a permanent verdict nobody read, and the source of the weak stamps that
poisoned two ledgers (#4608). Now the agent that wrote the code judges it at
the end of the turn, and the hook's evidence is what it is shown.
- shape_ledger.suggest_write_path_instances (was stamp_write_path_instances):
same evidence and form gate, but it writes a PROPOSAL — proposed_snippet_id,
basis "reference" (named) or "semantic" (resembles), score — and never a
status. A judged row is left alone; a new shape gets an unclassified
provisional row so the suggestion reaches the end-of-turn question as
"looks like #N". By-name uses edges stay: a fact, not a verdict.
- The prior-art line says "looks like #N … when you judge this turn's
shapes, say whether it is", and the result key is `suggested`.
- write_time_divergence takes `suggested`; _RESEMBLE_MIN re-documented as the
suggestion bar and the weak-stamp line.
- reusing-code skill no longer says the pull stamps an instance.
- Tests moved to the new contract, plus a guard that nothing the hook does
writes classified_by="hook".
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recording used to be decided by machinery — the only "record it" prompt
fired when a same-named copy already existed (#2664), so a first instance of
a reusable piece was never asked about, and judgment arrived only through
audits. Now the question is asked where the knowledge is: the end of the
turn that wrote the code, of the agent that wrote it.
- Write hooks keep `<sid>.written.ids` (path, kind, name) for every
definition a write names; a new file adds a `file` line for its stem — a
candidate in any language without a framework rule (scribe_written_append).
- Stop hook scribe_shape_check.sh sends the ledger to GET
/api/plugin/shape-check and blocks once, in the server's words, when
anything is unjudged. Same discipline as the report check: never twice,
never without a recorded check, another hook's loop left alone; the ledger
is kept when the instance cannot be reached.
- shape_ledger.unjudged_shapes: no row, unclassified, scoped and hook stamps
are unjudged; an agent/audit/import verdict is not. A snippet recorded at
the shape answers for it until the refresh stamps it canonical.
- services/shape_check owns the reason text and records every outcome in
app_logs (passed / blocked / judged_after_block / left_after_block).
- classify_shapes(repo=…) judges a shape the ledger has not synced yet via a
provisional row under a bound repo; the sync confirms it, or vanishes and
revives it with the verdict intact. An unbound repo is refused.
Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Librarian and Stash had 132 and 488 write-path stamps written at 0.68-0.72,
before the 0.80 floor (#4204). Nothing re-judged them, and dominant_canon
counted them, so a YAML CI snippet (#3410) "dominated" Librarian's web
directory and every write there was told it diverged from it — 1408 flags
on one project, 902 on the other, all skimmed past.
- is_weak_stamp: one predicate for "hook-stamped below today's floor".
dominant_canon and canon_form skip such rows; stamps_to_review lists them
by the same predicate. No stored row changes — they stay for judgment.
- flag_divergence withdraws a standing flag whose canon no longer dominates
its directory. A flag is a mechanical prompt, not a judgment.
- The coverage line ends "N weak stamps · M incoherent canons to judge —
stamps_to_review" when either is nonzero, so a project's own session sees
the queue on arrival. The shape-accounting skill says what to do with it.
- scoped_definitions handles .svelte: every <style> is component-scoped
unless <style global> or :global(...); instance-script syms are scoped,
module-script ones stay ordinary.
Plugin version minted for the skill change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shape ledger's divergence readout flagged _gate_setting (#4385).
Reading it turned up a family with no canon: floor_for,
get_duplicate_threshold, get_plan_match_threshold and _gate_setting each
parsed a stored string, fell back to the default (never 0) and clamped
into [lo, 1].
- services/settings.bounded_float(raw, default, lo=0, hi=1) is the pure
parse, fallback and clamp. Each caller keeps its own get_setting read,
so tests patching get_setting per module still take effect, and
_gate_setting still fails open on an unreadable setting.
- SettingsView.saveKbInject: eleven inline Math.min/Math.max clamps and
the local gateAt become one asBar(v, d, lo = 0), mirroring the server.
No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Anthropic's skill guidance: keep SKILL.md under 500 lines, split into
reference files linked one level deep as it nears that. using-scribe was
478 and every new practice lands there.
- SKILL.md 478 -> 317 lines. It keeps orientation, one copy, the reflexes,
scope, the judge section, UI and the process-skill index, plus a "Read
these when the moment comes" list naming each file with its moment.
- projects.md: binding a non-git directory (.scribe) and project inception.
- writing-records.md: where a new rule goes, lesson growth, and notes that
carry their own check (reflex 10 keeps a pointer).
- missed-retrieval.md: the record-before-dial route, verbatim.
- Text moved, not rewritten, except for the seams and one cross-reference.
Tests:
- tests.helpers.skill_text reads SKILL.md plus its reference files. The
ownership registry, the miss-route and the verification tests use it, so
a topic stays owned by its skill whichever file holds it.
- The force test scans every skill .md on its own, since each file is read
on its own.
- New test_skill_structure: SKILL.md <= 350 lines, every reference file is
linked from SKILL.md, none links another, and one over 100 lines opens
with Contents. Each guard is shown to fail.
The plugin version is minted. That also clears 4fb53b8's red Plugin hooks
lane, which failed only because PACKAGING.md changed without a mint.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Spike #4389 read the spec, Claude's docs and a dozen servers: the field is
for how the tools fit together, and the field runs ~600-1,600 characters.
Ours sat at the 2,048 cap as a keyword index that also carried stance.
- JUDGE, REPORT and MISSED leave the index. They fire mid-work, not at
session start; using-scribe and reporting-back state them in full, and
`placement`/`report_back` cue reporting in-band. No skill text changes.
- The rest is rewritten as plain practices (1,503 chars) and keeps every
session-start marker the ownership registry pins.
- INSTRUCTIONS_BUDGET 2000 -> 1600; the three index markers are dropped
from the registry; the miss-route index test now checks that the index
keeps what_might_apply and stays off the route.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- server.py: `mcp.server.mcpserver.MCPServer`; StrictArgsFastMCP becomes
StrictArgsMCPServer, whose call_tool takes and forwards v2's `context`
and raises ToolError (the SDK logs anything else as an unexpected crash;
the message reaches the caller either way).
- stateless_http and transport_security moved from the constructor to
`streamable_http_app(...)` in mount_mcp, with their reasons.
- Per-request user identity is unchanged: the contextvar set around the
ASGI call reaches the handler on both v2 paths (legacy stateless spawns
from the request task; the 2026-07-28 modern path opens its task group
inside the request).
- pyproject: mcp[cli]>=2.2, no ceiling (installs are --locked). uv.lock
regenerated with --upgrade-package mcp in the ci-python image: only mcp
and its own dependencies moved (106 → 110 packages).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gate_bars(user_id, note_type) resolves the block bar and, for notes and
tasks, the overlap floor from kb_gate_* settings, with the old constants
as defaults. Fail-open on an unreadable value; a block bar clamps at 0.80
and the overlap floor at 0.70 and never above the bar. Five fields in
Settings beside the duplicate-report floors.
CLAIM_LEASE stays a constant, with the reason written at it: a per-user
lease would make one shared task live to one reader and dead to another.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
plugin-vue throws "Cannot read properties of undefined (reading 'scoped')"
when several SFCs share one src stylesheet. vue-tsc passed, so only the
image job caught it; :dev has not rebuilt since 22b7a92.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
create_log filtered on Note.user_id == user_id, a bare owner check, so a
collaborator with write access to a shared task was told it did not exist
— and, since a log now stamps the claim, could never be seen working it.
It now asks can_write_note. Editing and deleting a log still require its
author, which is authorship rather than access.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The note sweep, rule sweep, preference drift and rule history panes each
restated the same row recipe in a scoped block. It now lives once in
assets/sweep-shared.css under `sweep-` names (unprefixed globals would
leak into the unrelated scoped .row-title/.lede/.age/.actions elsewhere),
imported unscoped beside each pane's own scoped remainder.
Moved only what was shared: RuleHistoryPanel takes .sweep-state and keeps
its button row head; PreferenceDrift keeps its tiny-type footnote and
overrides the action row's layout. Lede and state now use the body-sm
token instead of 0.85rem/0.9rem (13px vs 13.6/14.4px) — one value, from
the design system.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Release is the mechanical half: scribe_session_end.sh sends the ending
session's id to /api/plugin/release-session, which releases the claims it
held. A tidy-up, not the guarantee (no SessionEnd on a crash; the lease
covers that), and skipped on /clear so SessionStart(clear) can still
hand the claimed work back.
Saying what happened is the half only the model can do. It is stated as a
practice where it is read: the using-scribe skill owns it ("Hand off before
this session's context stops existing", pinned in test_guidance_ownership),
the static context points at it for the wrap-up moment, and add_task_log's
docstring says a log claims the task. _INSTRUCTIONS is untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The claim now pays rent to the session that set it. The SessionStart hook
sends the host's `source` and the session id; the server renders a claim
section by source (task_claims.render_claims):
- compact / clear: the work this session had claimed, each with its two
latest log entries, so a compacted session resumes from the record
rather than from a count of open tasks.
- startup / clear: other sessions' live claims (may still be running) and
in-progress tasks whose claim went quiet (abandoned mid-task).
- fork: the same, framed as "two sessions may now hold this".
- resume, or no source sent: nothing.
The task sidebar shows a live claim as "Being worked" and a dead one on
open work as "Went quiet", so the operator sees a session die mid-task.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
status is durable and nothing clears it, so in_progress cannot also mean
"someone is on this now". The claim is that second meaning, stored as who
and when so it dies on read rather than needing to be cleared.
- notes.claimed_by / claimed_at / claim_touched_at / claim_session (0110).
- The server stamps it on the write that is the work: reaching in_progress
(update or create, via apply_status_transition) and a work log on an open
task. done/cancelled/todo release it. Live while touched within
CLAIM_LEASE (2h); dead on read past it, with no sweep.
- The plugin's PostToolUse hook on update_task/add_task_log binds the
harness's session_id (GET /api/plugin/claim-session). It binds only to a
live claim the caller holds; it cannot create one.
- to_dict carries `claim`. Plugin version minted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_note (create_note and create_records) set the status and nothing it
implies, so create_task(status="in_progress") wrote a started task with no
started_at, and done/cancelled had no completed_at or next recurrence.
The transition is now one function, apply_status_transition, called by
both paths; tests pin the invariant for every status.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The four embedding tables stamped chunker_version but not the model, and
vector(384) is a width, not an identity: a same-width model swap would
write a second geometry beside the first with no error.
- embedding_model on note/rule/milestone/system embeddings (0109; existing
rows stamped with the only model any install has ever run).
- Every write stamps EMBEDDING_MODEL; every backfill's "current" test is
is_current_stamp(), both halves of calibration_stamp().
- migrate_floor refuses while any row its surface searches is off the live
model, before sampling: re-embed, then migrate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on the live corpus, the 74 note pairs at or above the old 0.90 bar
were almost all distinct siblings — consecutive dev-logs, sub-notes of one
design, research parts — and the one clear copy sat at 0.997. The block
refused the next dev-log and taught force=true, as #4134 found for rules.
- The semantic arm blocks notes and tasks only at >= 0.98. The title block
stays; processes keep 0.90 (not measured).
- 0.87 to 0.98 comes back as `overlaps` on the create reply, from the same
per-chunk searches, with a note that leaves the call to the session:
fold in and delete if it is the same record, keep both if a sibling.
- create_note, create_task, create_records and start_planning's steps all
carry it; a batch names the record each overlap belongs to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snippet had no field for prose, so what a session learned about one went
into when_to_use — the trigger joined onto every chunk it is embedded as. A
sweep found write-ups of up to 3 KB there, headings and all.
- notes: stored after the code under `## Notes`, parsed back from the body,
carried by every path that rebuilds it (update, merge, un-merge). A
snippet with no notes composes the body it always did.
- create/update_snippet (MCP) take notes and return trigger_advice when
when_to_use is long, headed or multi-paragraph. Advice, not a refusal.
- Tool docs, the reusing-code skill and the editor hint describe the trigger
as the situation and point the explanation at notes.
- Editor gains a Notes field; the detail view renders it as markdown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The title was `subject — trigger` because the stored title WAS the
embedded one, and the join is what makes these kinds rank on the
situation they apply to (#2485). Every surface that shows a title then
showed the trigger too -- menus, lists and search rows ran to kilobytes.
- embeddings.document_title(title, note_type, data, body) joins the
trigger from `data` (body fallback) at embed time. Idempotent: an
un-migrated composed title comes out the same, never doubled. The
embed path, the startup backfill and the dedup gate's semantic signal
all use it, so the embedded text -- and every vector -- is unchanged.
- Writers store the subject: snippet create/update (service, REST, MCP)
and lesson_document. Both compose_title helpers are removed.
- Readers: dedup takes `data`; the menus strip the embedded title from a
passage; list rows project `when_to_use`, which SnippetListView reads.
- 0108 rewrites existing rows on an exact `' — ' || <own trigger>`
suffix with raw SQL, leaving updated_at alone so the backfill does not
re-embed the corpus for identical vectors. Downgrade recomposes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Injected lines rendered a snippet's or lesson's title, which carries its
whole trigger by construction (the embedding shape) and ran past 1,500
characters -- again on every `seen` repeat. The passage under a line was
cut to 200 chars from the middle, keeping its head (the title again) and
losing where the match was.
Now, on both the prompt menu and the write-path prior-art menu:
- the line shows the record's NAME (snippet data.name / lesson subject),
with its kind and System (`[issue (done) · Plugin & hooks]`);
- the passage is the whole matched chunk, title prefix stripped, on one
line so the blockquote holds; a title-only match hands over the trigger;
- a `seen` record is a one-line pointer to what is already in context.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The trailing newline became a space before the 600-char cut, so the cap
kept 599 characters of text; and under pipefail a grep that matched no
reply made the helper exit 1. Trim before cutting; return 0 explicitly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The context fetch folded a timeout, an HTTP error and a refused key into
one sentence that ended "enter_project() as needed" -- read as optional,
so a session could start with no recent milestones or open tasks and no
way to know what prior work existed. The fetch now names the cause and
the elapsed time, retries once (6s) only where a retry can change the
answer, and the fallback states enter_project as the first step, with
the marker's project id when there is one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prompt hook sent only the operator's message, so a mid-session
follow-up ("yes do that") named nothing a note, lesson or rule could
match. The hook now reads the tail of the last assistant reply from the
transcript and sends it as `ctx`; the notes and rule arms append it to a
short prompt (<= 280 chars), prompt first, capped at 600 chars. No model
tokens: it is embedding input, and the injected menu's budget is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured over 459 judged instance rows against every same-family canon:
true instances score 1.0 or ~0.795 (a model whose mixin list differs from
the canon's), and 0.8 cut the second band entirely. The proposal is read
before it is confirmed, so the floor admits it: 0.75 takes every measured
true row, at 35 wrong-canon pairings instead of 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
0.8 was sized for proposals nobody reads; every semantic proposal is read
by a judge before it is confirmed. Measured, it proposed 0 of 150 while
judged instances of a canon score 0.68-0.71 - the write-path hint's own
floor asks the same question of the same documents at 0.68. The arm now
uses that floor, scans 8 hits instead of 3 (a true instance ranked 4th
behind snippets of other language families), and the proposer version
bumps to 5 so rows examined under the old floor are read again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Measured live: true instances of the service-unit canon score 0.68-0.71
against it at best, helpers 0.66-0.75 against unrelated snippets, and
nothing reaches the 0.8 floor. The "conclusive miss" fired for nearly every
body and silenced real divergences exactly as it silenced helpers. The
stored miss basis, its flag withdrawal and the report plumbing are removed;
the floor stays at 0.8 and the new-shapes-first ordering stays. False
prompts are answered by judgment (exempt with a reason).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Measured on the first live refresh after 91cde6c deployed: the version bump
queued 1,533 rows for the semantic arm, the cap read 150 of them in row
order, and all six shapes new since the previous refresh - the only rows
flag_divergence acts on, and the highest ids - were flagged before the arm
reached them. The gate silenced nothing because it never got to look, and
since a flag persists until judged, a later conclusive miss could not take
it back.
- propose_for_repo sorts the semantic todo by _semantic_priority:
unclassified before scoped, newest first.
- flag_divergence withdraws a standing flag when the row now carries a
conclusive miss; that evidence alone withdraws one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A design system binds like a rule but reached a session only through the
session-start block: complete for a session that asks, silent for one
writing a component. The write-path hint now carries a design arm.
A trigger, not a search: a project has one design system, so the question
is answered by the file being UI (.vue, .css, .tsx, ...) in a project that
has one. No vectors, no score, no slot from the ranked menu.
An index, not the prose: resolved guidance runs to ~8,000 chars (the house
style alone), near the hook's whole additionalContext cap. The line names
each inherited layer's section headings and inlines a layer short enough to
be a line - in practice the leaf's departure.
The other arms do not move. A design-only write returns on its own rather
than joining the prior-art guard, so the standing-rule arm still runs only
where it ran before. Once per session per system, on the hook's existing
token-keyed channel (exclude_derive, design:<id>) - no plugin change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
find_duplicate_rule was title-only, on the stated premise that rules are not
a semantic-retrieval surface - false since rules were embedded. A preference
restating a rule under another title passed untouched, and since both kinds
share one ranking, the weaker label could arrive alone.
find_overlapping_rules queries semantic_search_rules with the rule_document
shape, both kinds, in the scope the new record ranks in (global: every rule
the caller owns; project: global + that project). All three MCP create doors
call it before creating and return overlaps + overlap_note on the reply.
It advises rather than blocks, on measurement: across 16 sampled records the
nearest DISTINCT neighbour reached 0.853, while a true rewording scored 0.850.
No threshold separates the bands, so the floor (0.80) sits below the
restatement and the author judges. The stale docstring is corrected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7223 went red on seven tests in test_lesson_usage_surface.py. None of them
caught a regression. Every property they pin still holds; they pinned it at the
old LOCATION: they patched `usage_for_notes` on the door modules and read the
route source for `usage_for_notes(` and `empty_usage()`. 62f3a48 moved all
seven doors onto `note_usage.attach_usage`, so the name is gone from the doors.
This is rule 167's case: a guard that fails on code you believe is correct has
usually moved out from under its property. So the guards are relocated, not
deleted. Each one still has an assertion that can fail:
- MCP list / once-per-page / read-before-pull: patch `usage_for_notes` on
`note_usage`, the module every door now reads it through. The ordering test
still fails if a pull is recorded before the count is read.
- REST list: calls `attach_usage(items)` exactly once, and does NOT hand-roll
`it["usage"] =`, which is how a row could lose its zero-filled key again.
The seam's own zero-fill and single aggregate are asserted in
test_usage_attach_seam.py.
- REST detail: `attach_usage(` still precedes `record_pulled(`.
- Detail view: the advice moved into utils/deadWeight.ts, so assert BOTH that
the view reads DEAD_WEIGHT_ADVICE.lesson AND that the table's lesson entry
still points at `when_to_apply`, so the guard cannot pass by pointing at an
entry that has stopped saying the right thing.
Missed before pushing because I grepped src/ for callers of the old name and
not tests/ for patchers of it. A test that patches a name off a module is a
caller too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`usage_for_notes` is named for notes and works on every note row, yet the chip
reached snippets and rules only. Notes had it nowhere. Lessons had it collected
and shown nowhere a person could reach, because #4196 taught `/api/lessons` to
attach it and `KnowledgeView` — the only lesson list in the UI — browses through
`/api/knowledge`, so `listLessons` still has no consumer.
The cause was not a missing line. SEVEN call sites carried their own copy of the
same few lines: two REST lists, two REST details, two MCP lists, one MCP detail.
Each read perfectly well alone, so "which doors attach usage?" had no answer
anywhere in the code — the same asymmetry test_system_tagging_door_parity.py
records for System tagging (#4249), where whichever door nobody exercised for a
kind is the one that never grew the feature.
`attach_usage(rows, key="id")` is now that answer, and all seven go through it.
A detail payload is a one-row list, so the single-record doors share the seam
rather than keeping a second shape beside it. Deliberately NO try/except: the
fail-open already lives in `usage_for_notes`, which reports through
`_report_failure("readout")` and returns the zero-filled map. Wrapping it again
would swallow the REPORT as well as the error, and a silently-swallowed readout
failure is exactly #2663 — every counter reading zero in production for weeks
while the writes landed fine.
`/api/knowledge` now attaches usage, which closes both holes at once: it is how
notes, lessons and processes are all browsed. `KnowledgeView` renders the badge
on the card footer, looking the advice up per row because the feed is mixed.
The advice moves to utils/deadWeight.ts. Canon #3460 says each caller owns its
own const, and that held while each caller showed ONE kind; a mixed feed would
need five of its own and the next surface another five. The canon's actual
invariant — advice is kind-specific and never baked into the badge — is kept:
it is still a prop. The three existing callers now read the same table, so the
sentence has one home rather than four. Recorded against #3460 so the next
reader is not left re-litigating it.
`_row_id` rejects bools explicitly: `int(True)` is 1, so a row carrying a flag
under the key would be credited with note #1's counts, and a wrong chip is worse
than no chip because it reads as a measurement. A row with no usable id is
skipped rather than failing the page.
Tests pin the PROPERTY, not one route: no door calls the aggregate directly
(AST, so a comment naming it is not a false positive), and every door that shows
usage reaches the seam. Plus the N+1 guard — one aggregate per page, asserted on
await_count, because the per-row version reads more naturally and is invisible
in review.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4204 gave the divergence check a structural gate, which silenced one of the
five false prompts it was filed for. The other four are `def` helpers in a
directory whose canon is an `async def` service unit. No refinement of
`shape_form` reaches them: they differ from #2793's acceptance case — a
hand-rolled sync `confirmDanger` where an async confirm helper is canon —
only by the JOB they do, and a signature does not carry a job.
The proposer's semantic arm already reads bodies per symbol, which is the
comparison option 2 asked for and was thought to be missing. What it did not
do was record its MISSES: a hit became `proposal_basis = "semantic"`, a miss
left the row indistinguishable from one nobody had looked at. So
`flag_divergence` could ask the proposer "do you agree this is the canon?"
but never "did you check, and is it not?".
`_semantic_canon` now reports whether an empty answer is evidence, and
`flag_divergence` withholds the prompt when it is.
The whole risk is in the negative, so only a conclusive miss is stored. A
body too thin to embed, a row the per-refresh cap never reached, an arm that
threw, and a result set that came back full — and may therefore have hidden
the canon behind the limit — all stay "cannot tell" and still ask the
question. That is the discipline `FORM_UNKNOWN` already enforces here: not
knowing must make a check quieter, never more confident. `_SEMANTIC_LIMIT` is
named for that reason; the number is load-bearing, not a tuning knob.
No migration: `proposal_basis` is nullable Text with no CHECK constraint
(verified in the model and across alembic/versions), so rule 36 does not
bite. Nothing can mistake the miss for a proposal either — every reader keys
on `proposed_snippet_id` or `proposal_group`, and `confirm_shape_proposals`
requires the id non-NULL before it will confirm anything.
`_PROPOSER_VERSION` 3 -> 4, per its own contract: rows remember the ruleset
they were examined under, and without the bump no already-examined row would
ever acquire a miss.
Option 1 (widening `kind`) stays closed, on the merits rather than on cost:
bucketing density by exact form takes the async canon out of a sync
candidate's denominator and silences #2793's acceptance case by the identical
mechanism, one layer down. The reasoning is on #4208.
Tests: tests/test_divergence_meaning_gate.py pins the report contract, with
the truncation case tested hardest — reading a cut-off as a negative would
weaken the guard in proportion to how many snippets the operator has. The
end-to-end discrimination is in test_integration_shape_classify.py on
deliberately the SAME fixture as #2793's acceptance case, so the two runs
differ in exactly one thing: whether the arm claims to have looked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Derive-first, ahead of the tests shape pay-down. An AST pass over all 2628
definitions under `tests/` found exactly three helper bodies duplicated
across files. Two are real copies and are consolidated here; the third is
not, and is left alone.
`need_tools(*tools)` — byte-identical in three hook-test modules, each
skipping when `jq`/`awk`/`git` is absent from PATH. Now snippet #4277. The
`import shutil` each file carried existed only to serve it and goes with it.
`rule_row(rule_id)` — byte-identical in two integration modules, reading a
Rule back through a SEPARATE session so the assertion is about what Postgres
holds rather than what the writing session's identity map remembers. Now
snippet #4278. Its imports are lazy, because `tests/helpers.py` is imported
by unit tests that have no database, which is the same reason
`plugin_config` defers its service imports.
NOT consolidated: `_side(uid, k, d="")` in test_services_plugin_context and
test_write_path_trigger. The body is identical but it closes over a
module-local `stored` dict, so it is not self-contained and "moving" it
would mean inventing a parameter neither call site wants. That is convention
plumbing — two tests independently writing the same one-line side_effect —
and it is dismissed in the ledger rather than lifted.
Worth recording for the next pass: a repeated NAME is not a family. `_row`
is defined in five modules and only two of those share a body; the other
three (test_list_rows_brief, test_calibration_stamp, test_shape_ledger)
build entirely different objects. Grouping by name would have consolidated
three things that have nothing in common.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Run 7208 failed 5 of 23 in the new file, all with a `**Learned from:** #1`
that no shape asked for.
`lesson_sources` falls back to `arose_from_id` when neither the mirror nor
the body names a source. `fake_note` leaves that field to MagicMock, which
auto-creates it truthy and coerces to 1 — so every no-sources shape
recomposed with a provenance line and the round trip failed for a reason
with nothing to do with the trigger. `fake_lesson` pins `arose_from_id` to
None and its docstring says why; it is also the truthful stand-in, since
these are lessons.
Worth recording separately: the local harness that cleared this change used
a plain stub class, so it was kinder than the real fixture and could not
reproduce the truthiness. The stub has been given the same None. A
stand-in that is more forgiving than the one the suite uses will verify a
change that CI then rejects — note #2109's point, arriving from the other
direction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The module docstring said the defect grew the worst real body by 520
characters over ten edits. That number was recalled, not measured, and it is
wrong — as was the 2500 I then got by deriving it on paper instead of
running it.
Replayed lesson #4207's actual stored shape through the old code: 607
characters per edit, exactly the length of the trigger's paragraphs 2 and 3
with their separators, taking the body from 1291 to 7361 over ten edits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`update_lesson` is field-wise: an edit that touches only `system_ids`, or
only the title, passes `insight=None`. The service then derives the insight
from the stored body and hands it straight back to `compose_body`, which
re-adds the trigger and the provenance line. The derivation and the
composition have to be inverses, and they were not.
`_strip_composed_lines` filtered out body LINES matching the trigger
marker's pattern. That pattern is `re.M`, so it matches one line.
`when_to_apply` is free text and is routinely several paragraphs —
`compose_body` writes all of it after the marker, so paragraphs 2..n were
ordinary lines the filter had no reason to touch. They came back as part of
"the insight", the whole trigger was prepended again on top of them, and the
record gained (n-1) paragraphs per update. Silent, compounding, and in the
half of the document that does the retrieving: a lesson edited a few times
ends up stating when it applies several times over and burying the insight
below it.
No marker could fix this. There is no end-of-trigger marker in the body, and
adding one would not repair a row already written without it. What makes an
exact strip possible is that the trigger is mirrored in `data[TRIGGER_KEY]`,
so the composed head can be rebuilt byte for byte and removed by subtraction
rather than matched by pattern. Hence `lesson_insight(note)` rather than
`_strip_composed_lines(body)` — it takes the record because the body alone
does not say where the trigger ends. The line-wise removal survives as the
fallback for a row whose mirror is missing, where it is right for a
one-paragraph trigger and is the best available without one; a mirror that
disagrees with the body falls back rather than cutting into the insight.
tests/test_lesson_insight_round_trip.py asserts the property rather than the
strip: `lesson_document(what, trigger(n), insight(n), sources(n)) ==
(n.title, n.body)` across six shapes, and again over ten consecutive no-op
edits. Falsified against the old code before committing: 11 of 23 fail on
HEAD, the headline one reporting a body grown by 3230 characters over ten
edits.
The one damaged row in the corpus (#4207, two paragraphs duplicated by a
`system_ids` edit) was repaired by hand against its authored text. #4221 is
the only other lesson with a multi-paragraph trigger and has never been
updated, so its body is intact — its derived `insight` was wrong, its
storage was not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`set_record_systems` is not a dumb setter. It runs its own `can_write_note`
and then links only the Systems the given user can READ. Four of twenty
call sites handed it the record's owner instead of the acting user, which
did two quiet things at once: the access check became trivially true, since
an owner can always write their own record, and the System filter used the
owner's visibility rather than the actor's.
On a single-user install neither is observable. With a share it is an
editor acting with the owner's reach — the shape rule 47 exists to prevent,
and the same reasoning routes/notes.py already spells out for
`set_supersedes` two lines away.
This is NOT a permission change. Every one of the four sites establishes
the caller's write access first: routes/lessons.py and routes/snippets.py
call `can_write_note(uid, …)`, mcp/tools/processes.py does the same, and
mcp/tools/snippets.py reaches `set_record_systems` only after
`update_snippet` has raised PermissionError if the caller may not write. So
nobody gains or loses the ability to edit anything. What changes is whose
reach the tagging runs with, which is exactly the kind of difference that
survives review because every call site reads fine on its own.
The four:
routes/lessons.py:243 owner_uid -> uid
routes/snippets.py:211 owner_uid -> uid
mcp/tools/snippets.py:486 note.user_id -> uid
mcp/tools/processes.py:196 note.user_id -> uid
The last one was written earlier in this same session, an hour before the
sweep that found it, with a comment confidently explaining why the owner
was correct. That is the argument for the guard rather than for care: the
unified stance was known and still got it wrong at the next opportunity.
So the guard is the point again. `test_every_tagging_write_acts_as_the_caller`
walks every `set_record_systems` call in src/ and asserts the first argument
is a bare local named `uid` or `user_id` — an attribute access is a record's
owner by construction. Verified against `git show HEAD:` as well as the
working tree: clean now, four offenders on the code it replaces.
Reads are deliberately untouched. `list_record_systems(owner_uid, …)` gates
on reading the NOTE, which the caller can do anyway, so it returns the same
list either way; it is a different operation and churning it would add noise
without changing behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A System tag is how `list_system_records` gathers an area's pile, so a
record that cannot be tagged is reachable by search and by nothing else.
`update_lesson` did not take `system_ids` while `create_lesson` did, which
made tagging available exactly once — at the moment of least information.
A lesson is usually written at the end of a piece of work, which is
precisely when that argument gets dropped, and after that the record could
never be filed at all.
Filling in the rest of the table found three more gaps, and the issue's own
generalisation was wrong. It read as "the REST door can do something the
MCP door cannot", on three instances in a row. In fact:
* `create_preference` is the exact MIRROR of the lesson bug — update
takes `system_ids`, create does not. The same capability missing from
the opposite end of the same lifecycle.
* `routes/notes.py` handled `system_ids` NOWHERE, while the MCP door
handled both ends. That runs the opposite way round from the premise.
* `create_process` / `update_process` took it at neither door, though a
process is a note and has always been taggable in the data model.
The real pattern is that whichever door nobody exercised for a kind is the
one that never grew the parameter — which is a better statement of #4248
than the one recorded there, and is not something a reviewer reliably
notices, because each door is only ever read on its own.
Milestones are NOT a fifth gap. `RecordSystem.note_id` is a ForeignKey to
`notes.id` and milestones are their own table, so they cannot be tagged at
any door by construction. Pinned in the test so the next pass does not
re-open it.
So the guard is the point, not the four parameters. `update_lesson` alone
would have left the shape that produced it intact. The new test asserts the
TABLE — every kind taggable anywhere is taggable everywhere it is written —
and keys the registry-coverage check on the SIGNATURE rather than on a
grep, so a module that only names the argument in prose is not swept in and
no hand-kept skip list can go stale. A fifth kind fails there rather than
shipping half-wired, the same reasoning test_derived_mirror_generic_door.py
records for derived mirrors (#3734).
One inconsistency found and deliberately not changed here: for the same
operation `routes/tasks.py` scopes `set_record_systems` by the caller while
`routes/lessons.py` and `routes/snippets.py` scope it by the owner. The new
notes code follows tasks.py and says why in a comment (#47 — an editor-share
holder should tag from what they can see rather than inherit the owner's
reach). Recorded in #4249 rather than fixed as a drive-by.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`band_hugs_floor` compares an arm's weakest tenth against its floor, and
#4225 made it SUSPEND rather than soften when the floor moved inside the
window — because across a change the scores and the bar come from two
different populations. It reads `floor_moves_since`, which answers "did
this arm's floor move", and it read an empty answer as "no, it held
steady".
Those are the same answer only where the ledger was watching. Before an
arm's first floor row there is nothing to move, nothing to report, and no
way to tell a steady floor from an unrecorded one. The suspension was
reading absence of evidence as evidence of absence, and the symptom is the
one #4225 documented: a band "-0.0208 above its floor" — an impossible
negative distance, printed with the suspension silent.
Every install passes through this. The ledger's first row for an arm is
written when that install first boots the release that records baselines,
so any window longer than the install is old reaches back past it. The
lowered-floor direction hides: the gap comes out comfortably positive and
reads as a clean bill of health.
`floor_history_gaps(since)` answers the question its companion cannot:
which arms' floor history does not REACH the start of the window. Arms
come from `surface_names()`, not from the ledger — an arm the ledger has
never heard of is exactly the one at risk, so it cannot be the ledger that
decides which arms get asked about.
Baselines count here, which is the one place the two deliberately
disagree. A baseline records a default without changing it, so it is not a
move and `floor_moves_since` filters it out. It IS the ledger beginning to
observe the arm, and from that moment silence genuinely means the floor
held — filtering it out here would suspend the band check forever on every
install that has never tuned.
`floor_history_unknown` sits between the known move and the band, so an
arm with a date to give gives it. Two sentences, because the remedies
differ: a date says ask again with a smaller `days`; no history at all
says there is nothing to wait for, and names what starts the record.
Why now: #4261 measures the work-log change by reading these warnings, and
every window for the next month opens before this ledger's first row.
Measuring against an instrument that prints a number it cannot support is
the #4225 trap one level up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Every backfill selected the text alongside the id and then iterated that
snapshot, sleeping between records. Over a real corpus that is minutes, and
anything edited inside the window was re-embedded by its own write path and
then OVERWRITTEN with the pre-edit text the scan had captured — stamped at the
current chunker version, so the next boot considered it current and never
repaired it. The record stayed findable by what it used to say, indefinitely,
until someone happened to edit it again. Nothing reported it, and no existing
signal separated it from a correct record: the version was right and the
vectors were there.
All four backfills had it. Three were pre-existing; I wrote the fourth an hour
ago for #4251 by copying the third.
#4251 also widened the exposure. The race used to need a note-body edit; now
every add_task_log re-embeds its task, so any session recording work during a
backfill can hit it — which is exactly what "forward work continues while the
backfill runs" means.
TWO HALVES, SEPARATE ON PURPOSE.
Stopping it: the scan takes IDS ONLY and `_current_row` re-reads each record at
the moment it is embedded. One indexed primary-key read, negligible beside the
forward pass that follows, and it turns a backfill from "replay a snapshot"
into "repair to current truth". The residual race shrinks from the length of
the run to the width of one record, which the parent-row claim and the
single-transaction replacement narrow further.
Repairing it: `_vectors_older_than_their_record` adds a third staleness
condition beside "no vectors" and "old chunker" — vectors older than the text
they encode. A guard that refuses to write a bad value does not undo the bad
values already stored (#4202); the rows are what has to change.
That check earns its place beyond the bug that prompted it. It equally catches
a record whose `embed_note` never ran — no event loop, a swallowed refresh, a
process that died between the commit and the index. Every one of those looks
identical from the version column and identical from outside.
And a work log is part of a task's document while living in its own table, so
writing one never moves `notes.updated_at` and the timestamp comparison cannot
see it. `_tasks_logged_since_embedding` covers exactly the records the race was
most likely to have hit.
The live instance has already run #4251's backfill — the prior-art arm is
surfacing `## Work log —` passages, which is the thing itself rather than a
banner about it. Whether that run clobbered anything is unknown and was not
measured; the repair half picks up whatever it did on the next boot, without
needing it diagnosed first.
The structural guard in the tests was checked against the old shape, not just
the new one: reintroducing `select(Note.id, Note.title, Note.body)` makes it
fail, so it can catch the next copy of this rather than only agreeing with
today's code (#167).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 2 of #4251. A System's `description` is a charter — several hundred words
saying what belongs in that area and what does not — and it is the answer to
"which part of this codebase does X live in". There was no semantic path to
one: `list_systems` enumerates, and `search(system_id=…)` uses a System as a
FILTER over notes. So a System could narrow a search and could never be the
answer to one, and an agent asking where a record belonged had to read every
charter or guess.
ITS OWN SEARCH, not a `content_type` over notes, for the reason note 3163
gives about milestones: the row could be shared, the search cannot. A charter
competing with the whole note corpus for one top-k is outranked by the records
filed under it — the right answer crowded out by its own contents — and "where
does this belong?" is a different question from "what prior art is there?",
which a caller asking one should not have to read past answers to.
So `system_embeddings` (0107) joins note_, rule_ and milestone_embeddings as
the fourth sibling, with `system_document`, `upsert_system_embedding`,
`semantic_search_systems`, a startup backfill and `search(content_type=
"system")`. Scoped like milestones: with a project_id, that project's Systems
if the caller can read the project (rule 78); without one, the caller's own.
Archived Systems are excluded — an archived area is one the operator has said
is no longer where things go, which is exactly the question being asked.
`system_document` is the plainest of the four shapes on purpose. A charter is
already written as the thing this search has to match, in the words someone
asking would use — so there is no trigger to synthesise as `rule_document`
must, and no second record to gather as `task_document` must. The stored
charter IS the sharp document, the way a snippet's is. `color`, `status` and
`order_index` stay out: presentation and bookkeeping, and a vector carrying
them would be answering a question nobody asks of a charter.
The search publishes `report["best_chunk"]` from the start rather than being
retrofitted, which is what #4251 asked of any fourth search. It matters more
here than anywhere: a charter runs long and a result shows its NAME, so a match
on the paragraph that actually decides where a record belongs would otherwise
be previewed by two words that cannot say.
The id that comes back is the one `system_id`, `system_ids` and
`list_system_records` already take, so the answer to "where does this belong?"
is directly usable as "show me what is there" and as "file it here".
`embed_system` sits beside `notes.embed_note` at the service for #2056's
reason — every door gets it by construction. Not called on delete: that is a
soft delete and the search joins through `System`, so the vectors are already
unreachable, and leaving them means a restore is findable again immediately.
`system_embeddings` is declared in backup's `_NOT_INCLUDED` as derived, beside
its three siblings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 1 of #4251. A work log is the richest prose Scribe holds about WHY
something is the way it is — written during the work, recording what was tried
and ruled out. #4241 made it readable from the agent's door. It was still not
findable, so "has anyone tried this approach?" — precisely the question a log
answers — could not reach one. The cost is not hypothetical: #4208 was rebuilt
in this session because its logs were unreachable.
THE DESIGN QUESTION the issue left open was whether logs embed as part of their
task's document or as rows of their own. As their own rows, a hit has to be
resolved back to a task to be worth anything, and it needs a fourth search, a
fourth result shape and a fourth arm. As part of the task, the objection is
that a long log drowns a short title.
That objection was true before #280 and is not true now. Chunking made one
record into one vector per section, so each log becomes its own title-anchored
chunk, scored separately, and the task's own prose keeps the chunk it always
had — a task is as findable as its best-matching log rather than as the average
of everything in it. The other half is this session's other build: a search
hands back the chunk that won (#4243), so a hit earned by a log shows that
log's passage under the task's title. Without that a reader would have got
body[:240] of the task — the opening of a record whose relevance lives three
hundred lines further down.
So `task_document(title, body, logs)` sits beside `rule_document`: a
synthesised embed-time shape, because the stored record is the task row and the
logs live in their own table, so the document that should be searchable exists
nowhere until it is built. A task with no logs is returned untouched — most
notes are not tasks and most tasks carry no log, and their vectors are the
corpus every tuned number here was measured against.
CHUNKER_VERSION 1 → 2, and its comment now says what the version actually
means. It used to read "whenever chunk_document's output can change for the
same input", which this change would slip past: `chunk_document` is untouched
and every task with a log now embeds differently while its title, body and the
chunker all stand still. The invariant is the document a record is embedded as.
The startup backfill re-embeds on that.
Create, edit and delete of a log all refresh the task through `embed_note`, the
one path every writer shares — an edited log whose vectors still carry its old
wording keeps matching what it no longer says.
No new kind enters the auto-inject menu: tasks were always in it, and this
makes recall on them better rather than changing what the menu spans. The
calibration stamp will now report shape_version 2 against numbers measured at
1, which is exactly the report #4104 built it to make.
Also promoted `session_returning` into tests/helpers beside `make_mock_session`
(#2834) — two files had spelled it out identically and a third was about to.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The engine took `note_type` and `task_kind` all along. What was missing was a
way to say them: the MCP tool's `content_type` knew `note`, `task` and `all`,
and `/api/search` knew the same two — so an agent could not ask "has this
snippet already been recorded" or "what lessons apply here" without searching
everything and reading past the rest. Browse offered nine kinds from the same
data.
The cause is that each door kept its own map. `_FACETS` in services/knowledge
is where a kind is declared, and #3161 made adding one a single edit by
generating the SQL filter, the Python predicate and the door's validation from
it — but the two search doors were written before that and never joined. So
this adds the third dialect, `search_filters_for`, and one composition over it,
`content_type_filters`, and both doors now derive instead of listing.
Two names keep a meaning of their own, and the docstrings say so: `all` is no
filter, and `note` is BROAD — any non-task, snippets and lessons included —
where the browse facet of the same name is narrow (`note_type == 'note'`).
They are left different deliberately; narrowing this one would stop returning
snippets to every caller that already asks this way.
An unrecognised kind is now refused rather than answered. Both doors used to
fall through: the MCP tool into a filter matching no row, the route into no
filter at all, so `?content_type=snippets` returned the whole corpus while
looking like a narrowed search. An empty result set is a claim — "the corpus
holds nothing like this" — and an agent acts on that claim by building the
thing it could not find, so a typo must not be able to make it.
The docstring is the agent-facing contract (#2846), and a test now holds it to
the table: every kind `_FACETS` declares has to appear in it, because a filter
an agent has not been told about is unreachable however well it is wired.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
test_milestone_search_is_its_own_shape asserts the result dict exactly, so
adding `matched` / `matched_is` / `body_length` broke it — which is the test
doing its job.
Its fake was a bare MagicMock, so `m.body` autovivified: `matched` came out as
a mock object and `body_length` as 0, and nothing in the old assertion touched
either. Lesson #2833 is about exactly this, and the failure output is what
surfaced it. The fake now carries a real body string and a report with the
chunk that matched, so the assertion is about the product rather than about
MagicMock's attribute behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4243 fixed one door. Scribe has three semantic searches over three chunk
tables, and all three collapsed chunk rows to the best one per record — each
of them KNEW which passage earned the hit, and each dropped it. Every surface
downstream then previewed the head of the document instead: a span the search
had already scored lower, with nothing saying so.
Mechanism, one place:
- embeddings.record_best_chunk publishes {id: {index, text}} into `report`.
Carried in `report`, NOT the return value: all three return
list[tuple[float, Record]] and ~30 sites unpack that pair (lesson #4207).
- semantic_search_rules and semantic_search_milestones now select
chunk_index/chunk_text and publish the winner, as notes already did.
semantic_search_milestones gains `report`, which it had no way to take.
- services/text.matched_excerpt is the one choice of span, and
excerpt_fields the one result block. Doors keep their own field names —
the web renders `snippet`, MCP returns `excerpt` — because renaming a
field a frontend reads is a different change from fixing what goes in it.
Surfaces:
- knowledge.query_knowledge, whose own comment calls it "the human's MAIN
search surface", was `(note.body or "")[:200]` on every row alike. Now the
matched passage on a search, the opening on a browse, and `snippet_is`
saying which. KnowledgeView renders that snippet, so this was live.
- search(content_type='milestone') gains `matched` — the plan body stays
out, but the passage that matched comes along, because recognising a plan
means recognising the part you asked about and a description written at
the start need not mention it.
- The auto-inject menu and the write-path prior-art menu put the passage
under their line. Both were title-only, which answers "does this apply?"
for a lesson or snippet (the trigger is IN the title) and not at all for
an issue or dev-log. No fallback to the body's opening: on a menu that is
preamble dressed as a reason, and once indented it cannot be told apart.
Left alone deliberately: the rule arms. A rule hint already renders the rule's
TRIGGER, which is written to answer exactly "does this apply to me" and beats
a matched chunk at it; and that line's budget was measured at #3851. Adding a
passage there would duplicate the trigger and spend the budget twice.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
test_chunking::test_search_collapses_chunk_rows_to_best_chunk_per_note feeds
rows straight into the real semantic_search_notes, so widening the select to
carry chunk_index/chunk_text broke its 2-tuple fakes. I had claimed no unit
test did this after grepping test_embeddings.py, which was the wrong file.
Rows are 4-tuples now, and the test additionally asserts the winning chunk is
REPORTED and not merely used for scoring — the property #4243 exists for, and
this is where it belongs, beside the collapse it comes from.
test_task_work_log_surface's ACL test asserted "task_logs.user_id" was absent
from the compiled statement. It is present in every SELECT as a projected
column; the claim was about the WHERE clause. Asserts on stmt.whereclause now,
which is what was actually meant and still fails if an owner filter returns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Raised by the operator: are we limiting what comes back by character count,
and how do we verify the pertinent part is the part displayed?
We were not. mcp/tools/search.py sent (note.body or "")[:240] — a head cut,
with no marker that anything had been removed, so a 240-character preview of
a 4000-character record was indistinguishable from a complete short one.
The opening is the wrong span. The match is semantic and per chunk, and
semantic_search_notes collapses to best-chunk-per-note — its own comment at
the collapse says "the first appearance of a note is its best chunk". So the
system identified the passage that earned the hit and then discarded it:
select(Note, distance) kept no chunk column. A record could rank first on its
sixth paragraph, be previewed by its first, and be judged irrelevant on a
span the search had already scored lower. That biases against long records,
and it is self-concealing — the caller who does not open it never learns the
preview was misleading.
- embeddings: chunk_index/chunk_text ride along in the select, and the
collapse records the winner in report["best_chunk"]. Carried in `report`,
NOT by widening the return tuple: ten callers unpack (score, note) at
~18 sites and nothing would catch the misses (lesson #4207). `report` is
the side-channel this function already uses for best_available_score.
- search(): excerpt / excerpt_is / body_length, and read_full when there is
more. A caller that cannot tell a matched passage from a document opening
cannot judge whether to look deeper, which is the only decision the field
supports.
elide() moves to services/text.py so both callers share one copy, and it
keeps BOTH ends with a stated gap — it is the fallback for when nothing
identifies a better span than "all of it", not the goal.
Also fixes a guard that produced a false failure on the previous commit:
test_pull_telemetry checked `"project_id: int = 0" in body.split("\n")[0]`,
which sees only the first line, so wrapping get_task's signature over four
lines made it report a function that does take the project as one that does
not. Parsed with ast now, and proven to still reject an absent or
wrongly-typed parameter rather than being appeased by reflowing the code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The work log reached the web UI through routes/task_logs.py and nothing
else. get_task returned only the body — a claim written once, before the
work — with the record written during it invisible beside it. So a stale
body arrived with nothing to contradict it, and this session rebuilt work
that had already shipped, with the evidence sitting in the task's own logs.
Read side, scoped through the access layer (rule 78):
- logs_for_task / count_logs_for_task / log_counts_for_tasks in
services/task_logs.py. Scoped by who may read the TASK rather than by
who wrote the entry: list_logs filters TaskLog.user_id == user_id,
which hands a shared collaborator an empty list reading as "no work
has been done". The page query folds readable_notes_clause into the
same statement so the permission does not become an N+1.
- get_task returns work_log; list_tasks and get_milestone steps carry
log_count, zero-filled so "none" is a count and not a missing key.
Elision keeps both ends. The newest entry arrives whole to 4000 chars
because it answers "where does this stand"; older ones are shortened from
the MIDDLE, never the head. A head cut selects what a reader sees by
character position, which is uncorrelated with what matters — an entry
closing with "so this shipped in 04775c3" loses the one sentence that
answers the question, and a truncated flag says something went, never
whether it mattered. The gap states how many characters it covers.
conftest gains an autouse stub for the new read arm, same reasoning as
_no_rule_arm: three widely-called tools grew a database read, and the
existing call sites should not each have to learn about it.
Raised while reviewing this: search() has the same shape and worse —
body[:240] with no marker at all, while the chunk that actually matched
sits unused in the row that won. Filed as #4243, not fixed here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Found by reading a live `retrieval_telemetry` readout after milestone 419
deployed, not by inspection. The readout said:
cannot_decline / report_preference — "45 calls, 0 of them returned
nothing. An arm that fires unasked has to be able to say nothing; this one
never has. Check that it applies its floor at all."
And printed, beside it, that arm's band: p10 = p50 = p90 = min = max = 0.791.
FIVE IDENTICAL PERCENTILES IS THE TELL. That is not a ranking, it is one
record at one score on every call — because `report_preference` searches a
fixed string (`reply_preferences.COMPLETION_QUERY`, a module constant, and
deliberately so).
For a fixed query against a stable corpus the top score is a CONSTANT, so the
arm's decline rate is 0% or 100% and never in between; which of the two it is
depends only on where the bar sits relative to that one number. "Never
returned nothing" is therefore arithmetic, not evidence, and the warning's own
remedy — check whether it applies a floor — cannot be answered from it.
The arm already knew this about itself; the warning did not:
"a fixed query makes this arm's score a constant and a floor a hair above
it produces a dead arm no amount of traffic will ever reveal"
— services/reply_preferences.py
THIS CLASS OF BUG ALREADY HAS A GUARD, which is the argument for the shape of
the fix. `Point.logs_unconditionally` exists because of #3497: both rule arms
once logged only their hits, so their zero count was structurally 0 and this
same warning would have fired on a LOGGING property while sending the reader
to move a threshold that was never involved. This is that one step over — a
QUERY-SHAPE property — and gets the same treatment: a declared field on
`Point`, and exclusion rather than trust.
AND THE WARNING THAT WOULD BE INFORMATIVE HERE DID NOT EXIST. For a fixed-query
arm the dangerous state is the mirror image: every call empty, meaning the bar
is above the constant and no further traffic will ever move it. The arm is off
rather than quiet, and nothing in the readout said so — `expects_traffic`
covers an arm with NO calls, not one with calls and a 100% decline rate. That
state is real and reached: `report_preference` once logged 69 consecutive
declines at 0.0006 under its bar.
So `fixed_query_never_clears` sends the reader to `near_miss_samples` and not
to the dial — because that incident is also the one where the statistic and
the correct action pointed opposite ways. Every percentile said lower the
floor; opening the refused record showed it was rule 77 arriving as a false
positive, and lowering it would have delivered that rule on every completion
report ever written.
Guards in tests/test_retrieval_warnings.py, including the falsifier that
matters most here: `cannot_decline` must still fire on an arm whose query
varies, or this change is a disabled check wearing a narrowed one's clothes.
Both boundaries tested from both sides, per that module's own standard.
The new code is documented in the `retrieval_telemetry` tool docstring beside
the others (rule 33) — an undocumented code in a readout is a reader meeting a
verdict with no way to disagree with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#4222 fixed one end of this defect — the extractors that decide what a payload
DEFINES now blank comment and string spans before any line matcher runs. This
is the other end: `scribe_local_dups`, which decides which OTHER files already
define that name, and which was still a plain `git grep`.
A grep sees lines, not spans, so the sentence
class with only modifier rules is a deletion that went half-way.
— real prose from a module docstring in this repo — matched the arm's pattern
for `name=with`. Writing a genuine `change`, `beside` or `wrapped` would be
told it already existed, and pointed at a docstring.
EVERY HIT IS NOW CONFIRMED by running `scribe_defs` over the candidate file and
keeping only names it actually reports. That is the only check that cannot
disagree with the other end of the pipe, which is the whole point.
MEASURED, NOT ASSUMED — both numbers the task reasoned from turned out wrong.
- WHAT IT REMOVES, across 141 payload files of this repo: 15 of 210 report
lines. Every one a string literal, a comment, a TypeScript `import { type
Foo }`, or Vue's `const emit = defineEmits()` boilerplate. No real
definition was lost. Where a name had both — `create_note` — the phantom
in a test's `shape_form("async def create_note(...)")` argument dropped out
and the definition in services/notes.py stayed. `with` went from four files
to none, which is the correct answer: nothing here defines it, and it is a
keyword in several of these languages.
- WHAT IT COSTS: mean 193ms -> 222ms, worst 569ms -> 572ms. The task feared
"over a second added to a PreToolUse hook" from 48 confirmations. It is
about 15%, because the arm was already dominated by its twelve `git grep`
calls, and because confirmation runs once per DISTINCT candidate file
rather than once per (name, file) pair. A deliberately pathological payload
— nine names that are ordinary English words — reaches 28 distinct files
and 947KB; `scribe_defs` runs at ~33ms per 250KB.
THE CANDIDATE CAP IS RAISED FROM FOUR TO TWELVE, and that is load-bearing.
Confirmation REMOVES hits, so capping before it runs lets phantom matches crowd
a real definition out of the window — hits dropped before anyone looked at
them, which is #4042's bug in a new place. The display cap stays at four and
now applies to CONFIRMED hits, which is where a cap belongs. #4042's own `||
true` inside the substitution is untouched, and its regression case still
passes.
The task's own advice not to fix this by tightening the grep pattern is
followed and written down: requiring `(` or `{` or `:` after the name rejects
`class with only…` and also `class Foo extends Bar {`, `class Foo : Base()` and
`type Foo struct {`. This arm exists because it works with no server, no index
and no binding (#2280, #2682), which makes a miss here invisible — a visible
false positive is the better failure.
Guards: tests/test_hook_duplicate_confirmation.py, against real git repos
because what is pinned is the interaction between `git grep`, `head` and the
extractor. Seven of the eight fail against the previous implementation; the
eighth is #4042's regression case, whose job is to keep passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The lesson slot has recorded surfaced-vs-opened since it shipped. Nothing
showed it. `get_lesson`'s REST door attached `usage` to the payload and no
view rendered it; the listing did not attach it at all, and neither MCP door
did.
#4196 asks when a lesson that keeps getting followed should become a rule, and
names the trap in the same breath: raw frequency cannot separate "this should
bind" from "this trigger is too broad", and the second is the commoner reading
by a wide margin. Surfaced-AND-opened can separate them. Neither question is
answerable by a reader who cannot see the numbers, which is why this is the
first step and not the threshold.
NO THRESHOLD IS PROPOSED HERE, deliberately. The corpus today is 10 lessons
with 7 recorded surfacings and 3 opens, over about fifteen hours of usage
data. A promotion rule fitted to that would be fitting noise — #3311's failure,
and the warning lesson #4228 was written to carry. `UsageBadge` already
declines to render a verdict under three surfacings for the same reason. So
#4196 stays open: its subject, the promotion path, is still unbuilt. What
lands is the evidence it needs.
- REST `GET /api/lessons` and MCP `list_lessons` attach `usage` to every
row, from one aggregate per page rather than a per-row read, which would
be N+1 by construction. Every row carries the key zero-filled, so "never
surfaced" is a state a reader can see rather than a missing field they
have to interpret.
- MCP `get_lesson` attaches it too, and reads it BEFORE recording its own
pull. That door records a pull on every open — it has to, or the kind sits
permanently at zero — which makes the order load-bearing in a way it is
not for a kind that only counts. The REST detail door already ordered it
this way; the two now agree about what the number means.
- `LessonDetailView` renders `UsageBadge` (snippet #3460) rather than
re-spelling the chip, with the advice keyed to this kind: a lesson that is
repeatedly offered and never opened is usually keyed to a situation nobody
is in, so it points at re-keying `when_to_apply`, not at deleting the
claim.
Guards, in the two styles this pair of doors already uses: the MCP side driven
behaviourally through mocks, including the call ORDER for `get_lesson`; the
REST side on structure like its siblings in test_lesson_rest_door.py, because
the route is decorated and returns a Quart response. Rule 167's falsifier is
included.
KNOWN GAP, not fixed here: `KnowledgeView` is the only lesson LIST in the UI
and it reads `/knowledge`, not `/lessons` — so the REST listing change reaches
`frontend/src/api/lessons.ts::listLessons`, which currently has no consumer.
The agent-facing listing does reach a reader today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The column guard is a claim about kwargs; "the value comes back" is a
different claim, and only a round trip settles it. This drives the real
`restore_full_backup` over a judged shape carrying all three of the columns
that were going missing.
`diverges_from` gets the harder assertion. It is a FK to notes.id, so the
tempting fix — carry the exported id across — produces a row pointing at
whatever note holds that number in the target database: not dropped,
REATTACHED, with the restore reporting success and the divergence about the
wrong snippet. So the test asserts WHOSE note the pointer landed on rather
than which integer it holds, and it refuses to run at all if the restore
happened to reuse the source id, which would let it pass without proving
anything. Same shape as the assertion in the rule_usage round trip, for the
same seam.
Also corrects this module's own docstring, which said the round-trip module
"is not written yet". Three existed; this is the fourth.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The column guard (#3182) makes a dropped column unexpressible on the way OUT.
Nothing watched the way back IN, and that is the worse half: an export gap
leaves an obviously thin backup, an import gap means holding a complete,
correct file and restoring an incomplete database from it, with a success
message.
WHAT IT FOUND, the first time it ran. `code_shapes` was exporting
`reason_code`, `recheck_at` and `diverges_from` and importing none of them.
A restored ledger would have carried every judgment's verdict and not the
code for WHY — the column the accounting reads to tell a scoped-css
exemption from convention-plumbing — with every recheck flag cleared and
every divergence pointer gone. `diverges_from` is a FK to notes.id, so it is
re-mapped rather than carried: the raw source id would point at whatever
snippet took that number in the destination, which is wrong rather than
missing and is the milestone 333 trap one table over.
WHY THE GUARD WAS ONE-SIDED. Not an oversight — the code was. Export goes
through per-table pure helpers, so a test can hand one a stand-in and read
which keys came out. `_restore_v2` built all 26 models inline in one 690-line
procedural function, and there was no per-table unit to hand anything to.
So the construction moved out, into a `_build_*` helper per table taking the
exported row plus the id maps built so far. What deliberately did NOT move is
the loops, the flushes and the id-map bookkeeping: that is the
order-dependent part, where a mistake is a restore that half-works, and it
gains nothing from being split. Returning None is "skip" and a None field is
"degrade" — which one a table wants stays the table's own call, because both
are right somewhere: a shape event without its project says nothing, while a
usage event without one is still a real pull and dropping it would deflate
the pull-through the table exists to report.
The refactor was checked to be behaviour-preserving before the guard went in:
all 26 constructor kwarg sets identical to HEAD, and all 28 skip conditions
accounted for — 25 now in builders, 3 in loops that construct nothing (the
rule_systems raw insert and the two in the final project patch).
The guard then composes the two halves end to end: export a stand-in row,
feed THAT dict to the builder, read which columns the model actually
received. `test_the_usage_importer_restores_the_reading_project` read the
source of `_restore_v2` with `inspect.getsource` because there was nothing to
call; it is replaced by tests that call the builders and assert the
skip/degrade behaviour directly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
#2990 asked for three cases. The wrapped-comment-continuation one shipped
with #4222; these are the other two — a comment holding a whole rule defines
nothing, and an unterminated comment does not swallow the file. Both already
pass on both sides; the vectors are what stops a later change to the scan
quietly taking them back.
The phantom #2990 was filed for is among the twenty-two #4222 removes:
`.editor-body` in TaskEditorView.vue, a persisted `code_shapes` row whose
signature ends in `*/` and whose `used_by` count is zero — so it has been
sitting in the unused_css number the operator is meant to act on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The definition extractor is line-oriented and knows nothing about what a
line is INSIDE. A docstring that wraps onto a line starting with a keyword
announces a definition: `AND` reached a live session as a divergence prompt
asking it to justify a symbol that does not exist, and `is` reached it as a
repo-wide duplicate of four files that define nothing of the kind.
Measured, not assumed: running the extractor over every scannable file with
and without the scan differs by twenty-two phantoms. Two of them — `with`
and `nobody`, both out of the module docstring in check_dangling_styles.py —
are persisted `code_shapes` rows that have been judged. Those need no
migration: sync_shapes marks a row it no longer extracts as vanished.
`ast` would be the honest tool for .py and is not what this uses, because
the extractor is mirrored rule for rule by an awk program in the hook, awk
cannot parse Python, and a fix only one of the pair can run is the drift the
mirror exists to prevent. Both sides now run the same left-to-right scan and
blank comment and string spans to their own newlines before any matcher sees
a line. Three things the scan has to get right, each of which cost real
definitions while it was being written:
- a string that HOLDS a marker is not a marker. `"red /* "` in
test_design_stylesheet.py and a triple quote inside a single-quoted
regex in plugin_context.py each ate every definition below them.
- `#` is a colour in CSS and a comment in Python, and the extractor is
handed no path. An alphanumeric straight after it settles it.
- an unterminated opener blanks NOTHING. The scan rewinds past it and
continues, so a stray marker costs one span rather than the rest of the
file.
The comment claiming the two extractors agree has been the only thing
holding them together, and a comment cannot fail. The mirror test now RUNS
the hook's awk over the same vectors: with the old program it reports the
phantoms, which is what a guard that can fail looks like. Across all 631
scannable files in this repo the two now agree line for line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
A CORRECTION TO THIS ISSUE'S OWN PLAN, first, because the plan was wrong in
a way that would have cost the acceptance case.
#4208 recommended widening `kind` past `css | sym` to carry the form, calling
the denominator the deeper win and a migration the price. Two things:
1. THE MIGRATION IS NOT NEEDED. The issue says density "is computed from
stored rows and `shape_form` only runs on read". `canon_density` does
`select(CodeShape)...scalars().all()` and filters to the directory in
PYTHON; `signature` is a stored column; `canon_form(siblings, ...)` on the
next line already derives form from those rows. Bucketing the denominator
costs a list comprehension.
2. WIDENING `kind` WOULD BREAK MILESTONE #2793's ACCEPTANCE CASE. If `kind`
separated `fn` from `async-fn`, density would bucket them apart — which
does silence #4204's four `def` helpers, as claimed. Apply it to a
hand-rolled SYNC `confirmDanger` in a directory where an async confirm
helper is canon: the canon leaves the candidate's denominator, nothing
dominates, no flag. That is the flag the milestone exists to produce, and
it is the same inversion the first form gate made, one layer down.
So option 1 is not expensive-but-right, it is wrong; and option 2 (compare
meaning) is not the interim, it is the only lever. `confirmDanger` beside an
async confirm helper and `is_registered` beside a service unit are identical
at every structural level — same family, same form contradiction, same
density. They differ only in whether the candidate does the canon's JOB, and
no signature carries that.
WHAT THIS SHIPS is the part that is right and provable: an honest
denominator. `comparable_siblings` narrows the count to rows whose family
does not contradict the candidate's, using `families_conflict` — the same
predicate the gate uses, so the count and the verdict cannot drift into
disagreeing about what comparable means. "372 judged siblings" stops counting
a dataclass, a constant and an async service unit as three comparable things.
NARROWED BY FAMILY, NOT FORM, for the reason above: `fn` beside `async-fn`
stays a fair question. An unreadable sibling STAYS COUNTED — dropping it
would shrink `judged`, raise the share, and fire the check more on the
directories it can read least. Every unknown-form decision in this module
goes that way.
Density is now per candidate rather than per kind, cached on (kind, form).
TWO BUGS THIS CHANGE HAD, both caught before CI and both pinned:
- I passed a FAMILY where `families_conflict` reads a FORM, so
`shape_family("callable")` returned "" and the narrowing was a silent
no-op that still read as applied. The regression guard deliberately uses a
callable: `type` is both a form and a family name, so testing with it
proves nothing and the bug hides.
- My new `_Row` in the test file shadowed the one already there — same
fields, different `snippet_id` default — silently breaking three passing
`canon_form` tests. Reused the existing class. A duplicate definition
quietly changing a neighbour's meaning is this file's own subject.
The residue test's docstring said separating the four needs "widen `kind` or
a comparison of meaning". Corrected: they are not alternatives.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`retrieval_telemetry(days=30)` reported, for write_path_rule:
"the weakest tenth of what this arm returns scores 0.6984, only -0.0216
above its floor of 0.72"
A negative distance above something. The tenth percentile of what an arm
RETURNED cannot sit below the floor that gates what it may return — not
inside one population.
MEASURED CAUSE. write_path_rule's floor was 0.68 until 2026-09-02, when
2385100 (#3318) raised the shipped default to 0.72. The window opened
2026-08-22, so six days of it are calls made under the old bar; top_score.min
for the surface is exactly 0.68, the old bar still in the sample.
AND THE CHANGE LEFT NO TRACE THE READOUT COULD SEE. retrieval_tuning_events
records dial turns — a person or a model choosing a number. It was silent
about the other way a floor moves: somebody edits floor_default and ships it.
retrieval_tuning_history returned {"events": []} and retrieval_surfaces said
last_change: {}, source: "shipped". All true, and all of it silent about a
floor that had in fact moved.
THE RAISE ANNOUNCED ITSELF. A LOWERED FLOOR WOULD NOT: the gap comes out
comfortably positive and reads as a clean bill of health on a sample that
half predates the bar being judged. Both directions are now pinned.
So the check is SUSPENDED, not softened. band_hugs_floor asks whether the
scores are piled on the bar; that needs the scores and the bar to come from
the same regime. Where they do not, the honest answer is that this sample
cannot say, plus the date after which one can — floor_moved_mid_window
replaces band_hugs_floor for that arm and never accompanies it. A reader told
a number is unavailable goes and gets one; a reader handed a qualified number
uses it.
NO MIGRATION. `actor` is Text with no CHECK precisely so a new kind of actor
is not one — the model's own comment says so, and this is the case it
anticipated. "release" joins "model" and "human". user_id is already
nullable, which is right: no user did this, a release acts on every account
that has not overridden the dial, and a row per user would both multiply and
misattribute it. Both readers now take the newest of (this user's change, the
release's).
THE FIRST SIGHTING IS A BASELINE, written with old_value NULL. Nothing moved;
the row exists so the next release has a predecessor. That null is
load-bearing: floor_moves_since asks for old_value IS NOT NULL, so a fresh
install's baseline does not silently retire the check on every new install.
UI: the tuning history rendered actor as `human ? 'you' : 'Claude'`, so a
release row would have told the operator that Claude moved a floor it never
touched — the one failure the actor column exists to prevent. Three-way now,
with an unknown value printing itself rather than guessing.
Recorded at startup, inline and awaited. What #4181 cost three hours was
concurrency — a background task racing the hook for the same pool. Sequential
creates no contention, and this is twelve single-row reads. It must finish
before serving because a readout served before the change was recorded is the
exact answer this exists to stop giving.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419's acceptance, and it failed the first time it was run — which
is the only reason this commit exists.
THE MEASUREMENT. Ran the step-5 readout against this session's real traffic
instead of a fixture. It reported 19 rules named by an arm and none opened.
That is false: the session had called `get_rule` 45 times. Across six real
sessions on this instance: 208 opens, 3 surviving ledger entries. 1.4%.
THE CAUSE. `.opened.ids` was doing two jobs with opposite lifetimes.
- "this context HOLDS rule 156" — false after a compaction, and three hooks
read it to decide whether to stay quiet. Clearing it is correct.
- "rule 156 WAS OPENED" — which no compaction makes untrue, and which the
session-end readout is built on.
`scribe_clear_session_ledgers` sweeps every `<sid>*.ids` on SessionStart
source=compact. Right for the first claim, and it was deleting the second.
The TTL did the same thing more quietly: `scribe_rules_live` ages an
exclusion ledger, which is right, and would have eaten the early part of any
long session's evidence too.
So the readout was reporting only the stretch since the last compaction while
reading as though it had reported the session — a statistic that cannot vary
being mistaken for a finding (#3311), which is the shape this whole milestone
exists to stop producing. It ran AT the seam it was blind to.
THE SPLIT. `scribe_rules_append` now writes both: the exclusion ledger it
always wrote, and `<kind>.keep.ids`, an evidence twin that is never aged and
never swept. The readout reads twins; the three `held` readers are untouched.
DERIVED, NOT LISTED, because a list is what broke this before — the comment
above the sweep says so about its own history. Every ledger written through
the appender gets a twin, including the next one somebody adds; a new ledger
is born on the swept side unless its name opts out. `scribe_checkpoint_allowed`
writes its twin explicitly since it bypasses the appender, and there the split
lands right on both sides: the cap counts the swept file, so a compaction
honestly restores the budget to stop an act the context can no longer justify,
while the record that a stop happened stays.
Removed `scribe_ledger_ids`, orphaned by the change — a dead helper beside a
live one is a thing the next reader trusts.
test_session_ledger_clear.py asserted every ledger dies and could not have
caught this: its fixture never created a twin, so the sweep was one glob away
from either mistake with only one of them guarded. Both sides now asserted.
The slippage tests build their ledgers through the real writers rather than
by hand, for the same reason — a fixture that writes the bytes itself keeps
passing after the writer stops.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Step 5 of milestone 419. The milestone's subject is that a rule read and
ignored is arithmetically identical to a rule read and followed, and the
compaction is where that identity becomes permanent — the turns holding the
evidence are summarised away, and the unjudged thing survives as nothing.
WHY THIS IS ASSEMBLED IN THE HOOK. `rule_usage_events` has no session column;
it is per user over a window. A session-scoped answer therefore cannot be
asked of the server, and has to be built where a session is a thing that
exists. Four ledgers four hooks already write:
.rules.ids an arm NAMED the rule
.opened.ids the session called get_rule (#4100)
.acted.ids the session called rule_outcome (new here)
.checkpoint.ids the rule HELD an act (#4214)
Every one is an observed tool call. Nothing asks the model what it followed —
milestone 386 ruled that out, because a model asked "did you apply rule 156?"
says yes. Two subtractions: named-minus-opened is the arm talking to nobody,
opened-minus-acted is the milestone's whole subject.
PreCompact stdout is the compaction's custom instructions (#3680), not a
message to the model, so the readout does not say "you slipped" — it says
which ids must be carried through, which is the one thing a summary can do
about an unjudged finding.
SILENT WHEN NOTHING HAPPENED, and the accusations are conditional on having
members. "0 rules unresolved" on every compaction is how a readout teaches
its reader to skip it. Traffic is still reported, because the static
instructions already ask for it in prose; these lines are the measured
version.
scribe_record_outcome.sh is the third ledger's writer, matched on
mcp__.*__rule_outcome and mirroring scribe_record_opened.sh: TMPDIR only,
silent, exit 0 on every path. A PostToolUse hook that spoke would put a line
after every rule_outcome call and give recording an outcome a cost.
Also: check_plugin.py skipped the new hook for want of a smoke event, which
would have left the newest of the three ledgers as the only one the plugin
lane never runs. Added, mirroring its sibling.
tests/test_precompact_hook.py now isolates TMPDIR — the hook reads session
ledgers from there, so without isolation a test would see whatever this real
session had accumulated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 4. Five of the milestone's seven misses were the same move:
acting on the thing in hand without reading the contract around it. Lesson
#4207 says so in words, and was written by its author hours before a
structurally identical mistake, having been surfaced twice in the turns
between. Text delivered at the moment of acting is too weak a carrier for a
reflex that has to change what the act IS. This looks it up instead.
Rule 33 one scope down: its checks are between layers, and the same question
exists between a definition and its callers.
THREE KINDS OF EXPOSED NAME, because a contract breaks three ways that look
nothing alike in source — the defined symbol (a rename or removal), its
parameter names (arity), and the quoted keys of its dict literals (the shape
of what it returns).
THE THIRD IS THE ONE A SIGNATURE-WATCHER MISSES, and it is in because of the
miss that produced this step. Two commits ago `get_writepath_config` gained
one dict key; three arms read that dict inside a fail-open `except`, every one
silently became a no-op, and ten tests went red with nothing pointing at the
cause. No signature changed. Run against that exact edit, the check now names
tests/helpers.py — the actual root cause — among six files, before the write.
TWO GATES, AND THE SECOND IS WHAT MAKES IT USABLE. A change to the exposed set
is necessary but not sufficient: a definition nothing else references has no
contract to break, so the readers lookup runs second and an empty result ends
it silently. Body-only edits say nothing, a subject is named once per session,
and the ledger lives in the swept directory under the `.ids` convention, so
the existing compaction-clear guards cover it — checked against
test_session_ledger_clear's own parsers rather than assumed.
LOCAL AND SERVERLESS, like the duplicate-name arm beside it. It needs the
working tree and nothing else; the server has no checkout, so this is the only
place the question can be asked. It is a NUDGE: scribe_prior_art.sh still
returns no permissionDecision, which is the operator's recorded decision that
a recall aid may not stand in the way of a write. A test asserts that here as
well as in test_write_path_trigger.py, because this is the arm most likely to
tempt someone into making it a gate — it reports something that may already be
broken.
The `sym` half delegates to `scribe_defs` rather than repeating its patterns:
those cover nine languages and have been corrected several times, and a second
copy would inherit today's version and quietly stop agreeing with it (#3497).
Verified by lifting the test file's own helpers and driving all 19 cases
against the real shell over a fixture git repo. Two of my own errors were
caught that way and are fixed: the fixtures were arriving as single lines
because Python `repr` inside bash single quotes leaves `\n` as two characters
(the extractor is line-oriented, so the tests would have gone green against
input no editor can produce), and the no-readers case put its subject in a
file that was not the excluded one, so it had a reader and tested the
opposite of its name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7126: 1 failed, 2227 passed — down from 14. The last one was mine, and a
different mistake from the batch before it: I widened four assertions on the
literal `{"context": "", "rule_ids": []}` by replacing the string, without
checking which arm each test was calling. Three are the tool arm. The fourth,
`test_the_prompt_arm_says_nothing_when_asked_nothing`, is not.
The distinction is real rather than an omission, so the assertion goes back
with it written down. The two ACT arms can hold a call because there is a
composed act to hold. The prompt arm fires on the operator's message, before
anything has been decided — there is nothing to put a rule in front of, and a
checkpoint there would have to guess at an act that does not exist yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7124 Python tests: 14 failed. One root cause behind ten of them, and the
failure was the exact one `tests/helpers.writepath_cfg`'s docstring already
warns about in prose — while being unable to prevent this instance of it.
Three arms read their numbers out of that config dict inside a fail-open
`except`. A missing key raises where nobody sees it, so the arm becomes a
silent no-op, indistinguishable from the arm working and finding nothing. The
helper derives its keys from `retrieval_surfaces.SURFACES` precisely to stop
that — and `checkpoint_threshold` is deliberately NOT a surface, because
everything in that table is a floor/budget pair belonging to one query and the
checkpoint runs none. The derivation therefore could not see it, the write-path
rule arm died before `record_retrieval`, and ten tests went red at once.
Fixed at the helper, from the module constant, so there is still exactly one
literal and it lives in the product. And the guard the docstring claimed now
exists: `test_the_config_stand_in_carries_every_key_the_real_one_does`
compares the stand-in's key set against the real `get_writepath_config`, so
the next key that is not a surface fails loudly here instead of quietly
disabling an arm under test. `test_retrieval_surfaces`'s hand-written key list
gains it for the same reason, spelled out in place.
The other four were the contract widening itself: `checkpoint` is present on
every return of the tool arm, including its early ones, so four assertions
comparing the whole dict needed it. That key is deliberately always present —
the two arms feed one shell reader where an absent key and an empty one are
read the same, so the difference is invisible exactly where it would bite.
Verified statically: every cfg in the suite now routes through `writepath_cfg`
(test_write_path_trigger's local `_cfg` delegates to it), no hand-written
config dict survives, and the real config's eight keys match the stand-in's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 3, the pre-act checkpoint. Every rule surface in this
plugin returns `additionalContext`, which Claude Code delivers alongside the
tool RESULT — so the rule is read after the call is written and lands as
commentary on a decision already made. That is the milestone's central
finding, measured over a session with seven misses, three caught by the
operator and none by this system.
The action arm can now return a `deny` instead. The act does not run, the
rule's text can be read before the call exists, and the remedy is one
`get_rule` call after which the act may be re-submitted unchanged. Nothing
reaches the operator: a deny is a message to the model.
"CONSEQUENTIAL" IS DERIVED, NOT ENUMERATED. The obvious implementation lists
act kinds — a write to product code, a schema change, a bulk classification, a
merge. Every one of those is consequential because THIS operator wrote rules
about it, and shipping that list is this instance's corpus hard-coded into the
product (rule 115). So the corpus decides: an act is consequential when the
install's own rules speak to it above the checkpoint bar. A fresh install with
no rules never stops anything.
FOUR CONDITIONS, EACH PREVENTING A DIFFERENT WRONG. Above the bar; a rule and
never a preference (which claims no such force); the band's top hit only (the
ranker's confidence claim attaches to its first element); and only a rule the
session has NOT opened — `held` is observable from the get_rule PostToolUse
hook (#4100), not self-report.
WHY "NOT OPENED" RATHER THAN "NO OUTCOME RECORDED". An outcome can be
satisfied with one cheap call asserting compliance without producing any, and
a checkpoint dismissible that way manufactures exactly the compliance data
step 2 was built to measure. Reading a rule cannot be faked in that direction:
after `get_rule` the statement is in context, which is the whole of what was
wanted.
THE BAR IS MEASURED. `retrieval_telemetry(days=30)`: write_path_rule p90
0.7628 max 0.8817; pre_tool_rule p90 0.7373 max 0.8293. 0.80 is above p90 on
both and below max on both, so it selects from the top decile of an already
selective arm and is still reachable. It ships as a setting with a Settings
card, because a cosine distance in one model's geometry over one corpus cannot
transfer.
TWO GUARDS ON THE WORST CASE: at most one hold per rule and five per session,
so a mis-set floor degrades to a noisy session rather than one that cannot
proceed. The ledger lives in the swept directory and is named `.ids`, so the
existing compaction-clear guards cover it.
WRITES ARE NOT HELD, AND THAT IS THE OPERATOR'S DECISION RATHER THAN MINE.
`scribe_prior_art.sh` carries a tested property that it never returns a
permissionDecision — a recall aid may not stand in the way of a write. Three
of the milestone's seven misses were file edits and none are reachable from
the command side, so there is a live argument for extending this; that
argument is exactly why the boundary is now asserted by a test rather than
left to memory. The write-path arm computes and returns the same block so the
decision can be revisited with evidence; the hook ignores it, and a change of
mind is a hook edit rather than a feature.
Verified by lifting `checkpoint_for` and `_rule_band` out of source with `ast`
and exercising the shipped functions over 17 populations, by running the
ledger and deny envelope in bash (10 cases, including that a refused hold is
not written and that a garbled rule id fails closed), and by
scripts/check_plugin.py — which caught the unminted plugin version, 0300 ->
0426.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 2. Step 1 made an outcome recordable; this makes it
readable. `retrieval_summary`'s rule block gains `applied`, `departed` and
`distinct_rules_acted`, and `_compute_warnings` gains two codes.
TWO CODES, NOT ONE WITH A ZERO IN IT. `read_and_unacted` reports rules that
were opened and left no outcome, against the ones that did. It only fires once
outcomes exist anywhere in the window, because a window with none cannot tell
"every rule was ignored" from "nothing calls `rule_outcome` yet" — and on
every install the day this ships, the truth is the second. Claiming the first
there would be #3311's failure exactly: a statistic that could not vary being
read as a fact about the corpus. The cold case gets its own code,
`outcomes_never_recorded`, whose prose says in as many words that it does NOT
mean the rules were ignored.
`applied` AND `departed` ARE NOT SUMMED. A departure carries the reason the
agent gave and is evidence about the RULE; an application is evidence about
the agent. Folded together they would say only "an outcome exists", which is
true of both and useful about neither. `distinct_rules_acted` counts either,
because for the unacted arithmetic the distinction does not matter.
An outcome is not a pull. The fold branches on OUTCOMES first and never routes
an outcome through the surfaced/ambient split: `source` on an outcome row
names the door the outcome came through, not a ranker, so the ambient
distinction has nothing to say about it. An integration test holds that line —
if an outcome leaked into the pull counters the silently-unchanged rule would
vanish into a compliant-looking total, which is the confusion #4212 was opened
to end.
Verified by lifting the shipped `_compute_warnings` out of source with `ast`
and exercising it against the six populations the new tests assert: cold
instrument, warm instrument, departures-only, full compliance, nothing opened,
and a failed read. The integration tests for the new counts run against real
Postgres in CI — count(distinct) with an IN over an unconstrained column is a
SQL shape a mock would agree with whatever it did, which is what #2663 was.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419's own subject, committed while building it. The module header
of mcp/tools/rulebooks.py says the tool count "lives in the registration
test, which fails when it drifts" — I read that line while adding the tool
and did not act on it, which is a rule surfaced, read, and silently
unchanged. The only reason it was caught is that the guard exists and CI ran
it; nothing about my process would have found it.
Worth stating plainly because the milestone is about exactly this class of
miss, and step 1 shipped the record that would let a future session SEE it:
a rule read and not followed is invisible unless something independent
notices. Here the something was a test written by whoever last changed this
count.
Integration was already green on the previous run, so the migration, the
backup round trip and the new `detail` column were never in question — this
was one integer.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and
that it was PULLED. It could not say what happened next, so a rule that fires
constantly and is always obeyed and a rule that fires constantly and is never
obeyed left byte-identical telemetry. The second is far the more urgent and
was the one the readout could not name — measured on a session where three of
seven misses were caught by the operator and none by the system.
Two new events, `applied` and `departed`, and a `detail` column carrying the
why of a departure. No CHECK migration: `event` was created in 0094 as plain
Text with no constraint, verified in the migration rather than assumed from
the model, so rule 36 does not bite here — said in both places because the
next person adding a value will reach for it.
THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently-
unchanged is the failure this milestone was opened on, and it cannot be
reported: an agent that knew it was ignoring a rule would not be ignoring it.
So nothing here asks. `applied` and `departed` are reported; the third state
is a rule that was opened and left no trace. An `ignored` enum member would
collect nothing while reading as though it had measured something, which is
#3311's failure — a statistic that could not vary being taken for a finding.
`detail` is a column rather than two more bare event strings because a
departure stripped of its reason reads back as a miss, so the two states this
exists to separate would collapse again one layer down, in the readout, where
nobody would see it happen. Nullable: following a rule needs no argument, and
an expensive event is one that stops being recorded.
`outcome_state` is the single reading of the four states, taking the aggregate
`usage_for_rules` already returns, so the badge, the readout and any later
session summary cannot disagree about what "followed" means — the drift #3246
found across the rules system. A departure outranks an application: a rule
both applied and argued with is a rule someone argued with, and the argument
is the half worth surfacing.
`rule_outcome` is the MCP door, classed as a WRITE. The read-only set
tolerates getters that call record_pulled, but those are reads that leave a
trace; this tool's entire effect is the row, and the row carries prose the
agent authored. A read-scoped key that can put text in the operator's
database is not read-scoped, whatever table it lands in.
Backup carries `detail` on both sides. It is the one field here a fresh
install cannot re-earn — counts come back by being used again, a stated
reason exists once — and #4197 records that the column guard watches the
export side only, so the round-trip test is the thing that would catch a
one-sided add.
Delivery is deliberately not settled here: how an agent gets prompted to
record an outcome is step 3's subject, and the same record serves whichever
answer that step reaches.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The review surface shipped yesterday reported two canons on its first live
day and both were sound. Coherence was "do all judged rows share a form",
which #2844 failed at 37/62 = 0.597 for containing a model class and the
to_dict the canon's own text says the class must carry, and #2849 failed at
4/7 for pairing sync loop-starters with the async ticks they schedule. A
review surface whose whole output is noise is one that stops being read.
The obvious repair is a trap, and there is now a test standing in front of
it. Grouping by family and keeping a majority test makes the check BLIND:
before #2844 was cleaned by hand it held 37 classes and 56 callables, which
as families is 56/93 = 0.602 — a clean pass, and the 31 rows that had no
business being there (Vue functions, route handlers, a dozen tests) would
never have been reported at all. A looser bar in the same shape is worse
than the bug.
So the verdict is inverted. Instead of asking whether most rows agree, it
asks how many rows the canon CANNOT ACCOUNT FOR: a row in the majority
family is accounted for; a callable defined in a file that also holds a
majority-family `type` row is a method of a member, not a foreign body; and
strangers above a fifth of the readable rows make the canon incoherent. The
majority vote abstains those methods, so a class's own serialisers cannot
outvote the classes and turn the members into the strangers.
Measured on the real ledger before it was written, which is why it is this
rule and not a nudge to the share: clean #2844 has 0 strangers in 62,
#2849 has 0 in 7, and polluted #2844 had 31 in 93 — the same 31 withdrawn
by hand this morning, named exactly.
The entry now carries `families`, the majority `family`, `attached`,
`stranger_count`, `unattended`, and `strangers` — THE ROWS THAT DO NOT FIT,
replacing a sample of the first twelve members. The reader's question is
which rows are wrong, and a sample of the agreeing majority cannot answer
it. `unattended` is the discriminator between a check that is too strict and
a ledger full of junk: both canons flagged on day one were entirely
audit-judged, and nothing showed that without opening each one.
Scoped to the review surface. `canon_form` still answers at the precise form
level for stamping and divergence, where a sync helper beside an async canon
is a fair question; nothing here changes what the ledger writes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI 7098: unit tests red, everything else green. `_INSTRUCTIONS` was 2439
against a 2000 budget.
WHAT I DID NOT CHECK. That block is capped because Claude Code injects only
the first ~2,048 characters of a server's instructions and cuts the rest
mid-word (#2562, observed live — a 20k version delivered ~10% of itself and
the Systems guidance never reached a session). The cap is stated in a comment
directly above the literal I edited. It was at 1998/2000 before this batch:
a shared, nearly-exhausted resource, and I added a six-line entry to it.
THE JUDGE LINE STAYS, and paying for it is the decision rather than dropping
it. A client with no Agent Skills support receives this index and nothing
else, so of everything here, "you are the judge of record" is among the least
safe to leave past the fold — an agent that never learns it defers every call
to an operator who was never going to make them.
So the line is earned by compressing prose AROUND the existing markers, not by
removing anyone's entry: RULES loses a clause, RECORD and REPORT lose trailing
restatement, PLAN drops a sentence the two markers already imply, and the
opening paragraph tightens. Every index marker the ownership registry requires
survives verbatim — that is what test_the_index_names_each_reflex_it_points_at
checks, and it passes.
Back to 1998/2000: the same headroom as before, with one more reflex indexed.
The next addition pays the same way.
Three guidance modules run green locally (21 tests) — they read files and need
no database, so this one did not have to go to CI to be known.
Plugin version re-minted; the previous mint is on a commit that never went
green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
I recorded this as project rule 174 first. That was wrong twice over, and the
second reason is the one that matters.
RULE 119 SAYS THIS EXACTLY: guidance about how an agent should behave with
Scribe belongs in `_INSTRUCTIONS`, `plugin/skills/*` or the adapter's static
context, never in the corpus. I read 119 while writing the rule, decided it was
"about authority rather than about using Scribe", and wrote it anyway — which
is the reasoning preference 29 exists to catch, performed in full.
THE REASON THAT MATTERS: a rule in the corpus is true on ONE install. If the
agent being the judge is how Scribe works, every install gets it or none does.
Baked in, it ships. As a rule it was one operator's private note about a
product stance.
WHAT IT SAYS. The agent is the judge of record for the work — what a shape is,
whether a finding holds, whether something is done. Surfacing a finding for the
operator to rule on is the judgment NOT made, however well written up: it reads
as diligence and functions as a backlog. Escalate the acts that are genuinely
theirs — their money, their infrastructure, anything hard to reverse or facing
outward — and keep the decisions. A hard call is still yours; an irreversible
act is still theirs.
And the half that keeps this from becoming the previous defect: JUDGING IS
ATTENDED. An agent reading evidence and recording why is judgment; a threshold
or a sweep reclassifying in bulk with nobody reading is the thing that fills a
ledger with confident nonsense (#4208, and Portal's 35 rows). When the fix for
bad unattended writes is another unattended write, stop.
THE PRODUCT WAS TEACHING THE OPPOSITE. reporting-back's Finding row read
"Symptom · Cause · Size of the fix · **Offer to fix it**". So the behaviour I
was corrected for is the behaviour the skill prescribed — which is the better
argument for fixing it here than any rule could be.
Three surfaces, per 119 and the ownership registry (#4027): `_INSTRUCTIONS`
gets a one-line JUDGE index entry; using-scribe owns the authority and the
attended/unattended distinction; reporting-back owns the report shape. Two
topics rather than one, registered separately in test_guidance_ownership so
trimming one cannot quietly take the other.
Plugin version minted — skills only reach a session when the manifest moves
(#2209).
Rule 174 deleted (trash 074434a2, recoverable).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Lint caught an F821 that would have been a NameError the first time
`stamps_to_review` was called: `access` is imported locally inside each of the
seven functions in this module that need it — services/access reaches back
here, so a module-level import closes a cycle — and the new function used it
without one.
I wrote the function by pattern-matching its neighbours and did not check what
those neighbours do to make themselves work. Same shape as the tuple unpack
two commits ago (#4207): the mistake is not in the logic I was thinking about,
it is in the surrounding contract I did not read.
Unit and integration were both green on the failing run (7095); only lint was
red. Worth recording because the lane that caught it is the cheapest one and I
had read its command as covering tests — `ruff check src/ scripts/` does not
look at tests/ at all, so a clean test suite says nothing about it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
THE HALF THAT WAS MISSING. #4204 put a floor under what the write-path hook
may assert. A floor only guards new writes; every row already stored stands
(lesson #4202). Measured after that fix shipped: Portal carried 32 rows under
one canon and 3 under another, all stamped on scores of 0.69-0.77 — below the
0.80 floor, so none of them could be written today, and all of them were still
there. Scribe's own ledger carries 334 under #2860.
`stamps_to_review` reports two things and changes nothing:
weak — rows the hook stamped on a resemblance below the current floor,
each with its score, signature and derived form.
incoherent — canons whose own judged rows do not agree on a form. A canon
claims some shapes are the same sort of thing; when its members
are a class, three getters and a dozen tests, that claim has
stopped being true and every base-rate reading built on it is
reading noise. `canon_form` already made such a canon fall
silent — nothing made it VISIBLE.
IT DELIBERATELY CANNOT FIX ANYTHING, and that is the design, not an omission.
The first version of this commit was an automatic sweep that reset rows by
score. That is the original defect pointed the other way: what harmed the
ledger was not one wrong score, it was a machine recording permanent
classifications unattended. Un-recording them unattended is the same act with
a wider blast radius. An agent reads the evidence, judges, and records the
judgment under its own name through `classify_shapes`.
`test_the_service_carries_no_machinery_for_bulk_withdrawal` asserts that
structurally, so the next person to reach for an auto-retire has the argument
again on purpose rather than in a diff nobody reads.
A JUDGMENT IS NEVER LISTED AS WEAK, whatever its age. This is the measured
correction to an assumption I nearly shipped: of Scribe's 334 rows under
#2860, 302 are in `services/` — the canon's own home — and the ones sampled
there are `classified_by="audit"` with no score at all. The legitimate bulk
of that canon was never scored; it was judged by an agent in batch. Listing
those as weak would invite an agent to withdraw the only real judgments in the
ledger. An agent's decision is a different KIND of evidence, not a worse one.
THE SCORE NOW HAS A PARSER. It lived only inside a prose sentence, so nothing
could ask how strong the evidence for a row was without re-deriving it — which
is how 32 rows sat unexamined for nineteen days. Format and reader are one
constant apart (`_RESEMBLE_REASON` / `stamp_score`), with a round-trip test and
a test pinned to reason strings taken verbatim from the two poisoned ledgers.
`live_rows_for` is `live_rows` behind the project read gate, for callers that
arrive from outside rather than from a job that already knows who is asking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
[s_id for rank, _at, s_id, _why in bucket if rank == 2]
ValueError: too many values to unpack (expected 4, got 5)
`in_play` gained the canon's form as a fifth element so the stamp could be
decided per shape rather than per file. The `record_uses` call eighty lines
below still destructured four, and every stamp that reached it raised.
Both integration failures on runs 7090 and 7091 are this one line —
`test_write_path_stamp_is_evidence_that_yields_to_judgment` and
`test_a_brand_new_shape_gets_a_provisional_row_the_sync_settles`.
Now indexed rather than destructured, matching the candidate scan above it, so
the next widening cannot break it positionally.
WHY THE UNIT LANE STAYED GREEN THROUGH TWO PUSHES. `record_uses` is only
reached once a stamp is actually written, which needs a real snippet, a real
ledger row and the write ACL — so no unit test crosses that line. 139 local
assertions and the whole unit suite passed on code that raised on every
successful stamp. The integration lane was the only thing that could say so,
which is the case for it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
CI run 7090 caught this; the unit suite could not. Integration job 25763 —
`test_a_second_confirm_dialog_is_detected_and_named`, the acceptance case of
milestone #2793.
WHAT I GOT WRONG. The previous commit gated BOTH halves of the ledger on
`forms_agree`. That is right for stamping and backwards for divergence,
because the two assert opposite things:
STAMPING says "this IS that canon". Agreement in form is evidence FOR the
claim, so demanding it is correct.
DIVERGENCE says "this is NOT the canon that dominates here — did you mean
to?" A form MISMATCH is the PREMISE of that prompt. Requiring the candidate
to match the canon silences the check precisely where it belongs.
So #2793's case stopped firing: a hand-rolled sync `confirmDanger` in a
directory where an async confirm helper is canon read as `fn` against
`async-fn`, disagreed, and was dropped. `flag_divergence` returned 0 where the
test demands 1, and the write-time check returned nothing where it must name
the canon. That is a real flag the milestone exists to produce, and my change
removed it.
THE FIX. Divergence now gates at FAMILY level — callable {fn, async-fn},
type, value, css — and only on contradiction. A sync function beside an async
one is still a fair question. A frozen dataclass told to build from an async
service function is not a question at all.
WHAT THIS DOES NOT FIX, asserted rather than commented so it fails the day it
changes (`test_how_many_of_the_five_the_divergence_gate_actually_silences`):
of #4204's five false prompts this silences ONE. `Point` is a type against a
callable canon. `_p`, `get_point`, `is_registered` and
`sources_expected_to_emit` are callables like the canon and still ask — and
at the signature level they are indistinguishable from the #2793 case above,
so nothing readable here can separate them. That needs #4204 option 2 (widen
`kind` past `css | sym`) or a comparison of meaning rather than form.
The stamping half — `_RESEMBLE_MIN` 0.80 and the graded burden — is unchanged
and unaffected by this failure. It is also the half that matters more: loose
stamping is what manufactures the density the divergence check reads as
authority.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Two halves of one defect, found while writing #3431 and measured on a second
project.
THE DIVERGENCE CHECK WAS A BASE RATE. `dominant_canon` answers "what is most
common in this directory" and never "is this that" — the candidate's
signature was not examined at all. With `kind` carrying only `css | sym`, a
frozen dataclass, a module constant, a sync predicate, a class and an async
service function are all siblings, so the prior was not merely the best
signal, it was the only one. Writing a registry module of pure helpers
produced five prompts to build them from the `async_session` service canon.
THE AUTO-STAMP HAD NO FLOOR. `elif sid in resembles` took any score at all:
0.69 asserted as confidently as 0.95, and the number went into the reason
line without ever being compared to anything. Worse, the score is computed
against the WHOLE PAYLOAD, so one number spoke for every symbol in the file.
On Portal that recorded `class SessionAbsent`, `def build_channel`,
`async def attach` and a dozen test functions as instances of one snippet —
17 rows under #3283, which then made that directory "canon-dense" and started
instructing every later writer in it.
The two compound: loose stamping manufactures the density the divergence
check reads as authority. Both are fixed by one primitive.
`shape_form` derives a coarse form — css / type / async-fn / fn / binding —
from the signature, on READ. `kind` is part of the row identity, so widening
that column needs a migration and a re-extract (#4204 option 2, still the
principled fix); deriving costs nothing and is reversible. Every caller asks
`shape_form`, so the day the column carries the answer it returns that.
THE BURDEN SCALES WITH THE EVIDENCE, and getting this wrong was the first
version. A by-name reference — the payload names the canon's symbol — is
strong and needs only the absence of contradiction; demanding positive
agreement there silenced it whenever a shape's definition was not in the
payload (an Edit rather than a Write), turning strong evidence into none for
a reason unrelated to the code. A resemblance score is weak and must
positively agree.
`canon_form` reads the form a canon's own judged rows agree on, and returns
unknown when they disagree. That makes the halves cooperate: a canon already
poisoned by loose stamping — Portal's #3283 — falls silent instead of
flagging anyone else.
`forms_agree` requires BOTH sides known, so an unreadable signature makes the
checks quieter rather than more confident. `_RESEMBLE_MIN` is 0.80 rather
than the retrieval floors near 0.70: those decide whether to SHOW a record,
where being wrong costs a glance; this decides whether to RECORD a claim
unattended, where being wrong misinstructs everyone who writes there after.
Caught by the tests, not by review: a first pass required `const`/`let`/`var`
before a binding, so every Python module constant read as unreadable and a
whole form was silently excluded from both checks.
Not addressed: rows already carrying a wrong snippet_id are not undone by a
guard at the point of classification (lesson #4202). Portal's 17 keep
producing dominance until something re-judges them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The tool returned distributions and left the reading to the caller, so every
readout was the same four checks done by hand — #3430's baseline, #3835's rule
near-misses, the #1038 rerank gate. Mechanical, and therefore forgettable.
Tonight's acceptance pass on #3898 was the case for doing this. Reading it by
hand meant catching that two surfaces had `covers_window: false`, that
`prompt_rule`'s floor had moved three times inside the window (which made the
readout self-contradictory: deliveries at 0.622 beside refusals at 0.7199),
and that 15 of 20 near-misses were one record against text no operator wrote.
Miss any of those and the obvious conclusion was "the bar is too tight" — a
floor change that would have injected one preference into every notification.
`warnings` is always present and empty when clean, so its emptiness is an
answer rather than a gap. Each entry carries the numbers that produced it:
"345 calls, 0 declined" is the analysis, "check write_path_rule" is an
instruction to redo it. Five codes — cannot_decline, band_hugs_floor,
no_duration, surfaced_never_pulled, unregistered_source.
cannot_decline has three guards, each a bug it would otherwise cause. Asked
surfaces are exempt (a search returning a list every time is working). An arm
not known to log unconditionally is exempt — that is #3497 exactly, where both
rule arms recorded only their hits, so a decline count of zero was a LOGGING
defect and this warning would have sent the reader to a threshold that was
never involved. Unregistered sources get numbers but no verdict.
`silent_surfaces` is the half the rows cannot show: an arm that emitted
nothing is invisible to every row-based check and looks exactly like an arm
that does not exist. It is driven by a new declared registry,
`retrieval_registry.POINTS` — deliberately NOT `retrieval_surfaces.SURFACES`,
which answers "what can be tuned" and excludes the reserved slots because a
budget of 1 is their feature. This answers "what can be measured", and the
reserved slots belong in it precisely because they are judgeable without being
tunable. A test asserts the two cannot drift apart.
The registry test derives sources from the call sites with `ast`, not grep,
and the difference is not theoretical: `wide_net` and `report_preference`
reach their recorder as `source=SOURCE` through a module constant, so a grep
for `source="` is blind to both — the narrowing #3191 warns about. Three sites
pass `source` as a variable and are declared in FAN_OUT_SITES; the test pins
those sites but not the values they can pass, which is why the
`unregistered_source` warning exists to catch the rest at first fire.
Thresholds are settings (rule 25) defaulted so a fresh install with almost no
data produces no warnings at all (rule 115) — a new user's first readout
naming five broken things would be describing the emptiness.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Claude Code submits more than typed words through UserPromptSubmit. A task
notification, a slash-command echo and the caveat banner a local command
prints all arrive as user turns, reaching `.prompt` indistinguishable from
something the operator wrote. scribe_autoinject.sh believed all of them.
The cost that matters is not the wasted embedding — it is the log row. Every
such call counts in the denominator of every prompt-boundary surface, so
delivery rate reads low for a reason unrelated to retrieval; and each refusal
lands in `near_misses`, where a later tuning decision reads it as demand.
Measured while taking milestone 399's acceptance (#3898): 15 of the top 20
`preference_slot` near-misses were `<task-notification>` blocks, all matching
ONE record — #140 "Let each action land before starting the next" — all within
thousandths of the 0.70 floor. A notification that an action finished really
does resemble a preference about letting actions land. Lowering the floor to
serve that apparent demand would have injected that record into every
notification: the instrument arguing for the wrong fix, which is #379 again.
scribe_skip_prompt is a PREFIX test, not a substring one, and that is the
whole safety argument. A real prompt may contain one of these tags — an
operator pasting a transcript, or a `<system-reminder>` after typed words —
and must still be retrieved against. Nothing an operator types begins with a
client envelope. The compaction-resume injection is deliberately NOT filtered:
it is machine-written, but it summarises real work, and a resumed session is
where recalling a rule earns its keep.
Client-side only, so no server contract moves and lagging plugin caches keep
working. The tags are Claude Code protocol constructs, identical on every
install — instance-agnostic under rule 115.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
ci-requirements.md is the document that drives promoting a per-job dep into
the ci-python image, and it still recorded jq as installed in the plugin job
and "load-bearing for the smoke test specifically", on the grounds that every
hook opened `command -v jq || exit 0` and the test would otherwise pass while
exercising nothing.
That was the tail wagging the dog. The hooks ship to users; jq is absent by
default on macOS, the Debian/Ubuntu slim images, Alpine and most CI
containers, and a machine without it got no context, no rules, no prior art
and no process sync in silence. The answer was to remove the dependency, not
to install it harder.
Recorded as a promotion the entry now argues AGAINST rather than deleted: the
next person to read this file should find out why jq is absent, not merely
that it is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Three loose ends from a49e7ed, all found by CI or by re-reading it.
TWO TESTS PINNED THE jq SYNTAX, not the word `jq`, so grepping the suite for
the tool name missed them: test_write_path_trigger asserted the literal filter
strings `(.note_ids // []) - (.sync_note_ids // [])` and `(.rule_ids // [])[]?`
appear in the hook. Re-spelled with the readers that replaced them. The claim
each makes is unchanged — that the reuse and sync channels keep their own
state file, their own query parameter and their own write-back.
CI STILL INSTALLED jq, TWICE. The "Install jq for hook execution tests" step
existed because those tests skipped rather than failed without it; the Plugin
hooks job installed it alongside shellcheck for the same reason. Removing a
dependency from the product and leaving CI to install it anyway means the
smoke tests never run under the condition they exist to assert. Both gone —
the hook tests now run on a bare image.
`awk 'END { print $0 }'` IS UNDEFINED IN POSIX. gawk retains the last record
in END and mawk does too, but it is not guaranteed, and reaching for a
non-portable idiom inside the change whose entire purpose is portability is
the same mistake one layer down. Now `{ last = $0 } END { if (NR) print last }`.
Verified it still selects the same start line on a real transcript.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Every hook opened `command -v jq >/dev/null 2>&1 || exit 0`, so on a machine
without jq the operator got no session context, no rules, no prior art and no
process sync — and not one word saying why, because `exit 0` is
indistinguishable from "ran fine, nothing to say". jq is absent by default on
macOS, on the Debian/Ubuntu slim images, on Alpine and in most CI containers.
That is not a prerequisite to document; it is the plugin handing its own
packaging problem to whoever installs it.
`tac` was worse: GNU-only, so the prior-art hook's enclosing-definition arm
did nothing at all on every Mac, silently, from the day it shipped. It is not
replaced but removed — scribe_defs judges each line independently, so
extracting forward and taking `tail -1` is the same answer as reversing and
taking the head, and it drops the early-exit `head` that #4042 was filed for.
No server contract changed, so a lagging plugin cache keeps working.
scribe_json.awk JSON -> IDX<TAB>PATH<TAB>VALUE. Two modes: `whole` for an
event or a response body, `lines` for a transcript, where
an unparseable record is dropped and the rest still read —
the `map(try fromjson catch empty)` the jq program opened
with. Arrays also report their LENGTH at `[#]`, which is
what keeps "zero notes" distinct from "no answer" (#2932).
scribe_turn.awk the turn-bounding program, replacing the thirty lines of
jq in the Stop hook.
scribe_defs.sh scribe_json_flat / _pick / _list / _len / _list_minus read,
scribe_json_out writes the envelope (five copies of one
shape, gone), scribe_urlenc replaces `jq -sRr '@uri'`.
Percent-encoding goes through `od -tu1` rather than an awk character loop on
purpose: awk's idea of a character follows the locale, so gawk reads an
accented letter as one and mawk as two, and an encoder built on substr() would
emit a different URL depending on which awk is installed. Encoding is defined
on bytes. Verified byte-identical to `jq -sRr '@uri'`.
Measured, not assumed. The per-event path costs 8ms against jq's 3ms. The
transcript path was 70x slower until two fixes: the Stop hook now finds where
the turn starts with a fixed-string grep before parsing (a needle carrying
unescaped quotes cannot occur inside a JSON string, so it matches only at a
record's top level — checked against a full JSON parse of a 27MB transcript:
152 prompt records, 152 matches, no misses, no extras), and the parser reads
each token out of a 1024-byte window instead of copying the rest of the buffer
per token, which was quadratic in line length on the 400KB tool results a
transcript carries.
Differential-tested against the jq program it replaces over 724 windows cut
from three real transcripts — 724 identical, 0 mismatched, 45 of them
exercising a real task close and a real reply. That sweep is what caught
`scribe_turn.awk` never setting FS, which truncated every multi-word reply at
its first space and was invisible to a test whose replies were all empty.
check_plugin.py's `jq -R` lint becomes a guard against either binary coming
back, and three smoke checks lose their `shutil.which("jq")` skip. jq is not
in `ci-python` either, so those three announced a skip on every CI run and had
never once run there: removing the dependency from the product also closed a
permanent hole in its verification. They pass now across all ten hooks.
tests/test_hook_json_reader.py is a differential against Python's `json` over
nested objects, arrays, unicode, escapes, control characters, empty cases and
a value longer than the token window, plus the envelope, the encoder and the
turn analyzer. 139 cases.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Run 7068 failed two tests on a4aae97, and the second was a real bug the
guard caught before it shipped.
THE COLUMN GUARD (#3182) was right. `note_usage_events` excludes only
`id` from backup, so telemetry IS exported — and a new column that the
serialiser never learned about would have restored as null on every row.
That is the exact failure #3182 was built from, where nine columns
vanished from `notes` the same way: added to the model and the migration,
which fail loudly, and never to the serialiser, which fails silently.
So `project_id` is exported, not excluded, via the one `_usage_event_rows`
builder both exporters share.
The importer re-maps it through `project_id_map` and DEGRADES rather than
skipping. `code_shape_events` drops a row whose project will not map,
because a shape event without its project says nothing; a usage event is
not like that. The project is optional by design and null already means
"not reported", so an unmappable one restores as unreported. Dropping it
would lose a real pull and deflate the pull-through this table exists to
report.
BACKUP_VERSION deliberately unchanged: the suite states that column
additions do not bump it, only new sections do, and a third test asserts
the current number.
The second failure was mine and smaller — test_record_pulled_writes_a_
single_row compares exact dicts, so it now expects the null project. I
had claimed these assertions were key-based; they are not, and I had
read a summary line rather than the assertion.
Added an import-side guard while here. The column family checks the
EXPORT side only, so a column can be exported faithfully and dropped on
the way back in — a restore that reports success and has quietly lost a
dimension. No general guard for that direction exists yet; this covers
the column added here and names the gap for whoever builds the general
one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
`RetrievalLog` has carried `project_id` since it existed, so "this record
was SURFACED on project B" was always answerable. `note_usage_events`
had none, so "this record was OPENED on project B" was not — and the two
cannot be joined to recover it, because there is deliberately no session
identity server-side. NoteUsageEvent's own docstring rules that out.
That gap sat exactly on the question milestone 385 exists to answer. A
lesson's whole claim is that it reaches a session on a project it was not
written on, and step 8's acceptance is "retrieved on a different project
AND opened". Each half was answerable; the conjunction was not.
WHICH project, because the name is ambiguous and the wrong reading makes
the column useless: it is the project the READER was in, never the one
the record belongs to. The record's own project is already on the note;
copying it here would answer a question nobody asked while looking like
it answered this one.
The surfacing half is free — every arm already holds the scope it just
searched, so auto_inject, lesson_slot, the write-path arms and
enter_project now record it. process_skill_sync does not and should not:
it installs every Process the operator can reach, which is not a
project-scoped question, so a project there would be a fiction.
The pull half needs the caller, since a getter knows only what it was
handed. The five single-record getters take `project_id: int = 0` and
pass it through, following the convention `search` and `create_*`
already set. Null stays an ordinary answer meaning "not reported" — a
pull with no project is still a pull and still counts toward dead
weight; it simply cannot speak to transfer. The four REST detail views
report none for now: a human opening a record in a browser is a
different event from an agent recalling one, and #2245 left that
asymmetry deliberately undecided.
Guarded the way #2245 and #2476 taught: by source inspection, because a
parameter that was never threaded through changes no return value and
shows up only as a column that is mysteriously always null. Three
guards — the signature, the pass-through, and the arms — plus the
can-fail test rule 167 asks for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
The lesson kind shipped with every mechanism for growing and nothing
telling a session to use them. `learned_from` is a list on purpose, the
dedup gate hands back an existing id rather than minting a twin, and
`update_lesson` already names re-keying a bad trigger as the edit that
pays most. None of that was reachable as a habit.
The exclusivity claim was the bug. The skill said "a preference is the
one record you keep current yourself", and by naming only preferences it
put lessons outside the habit. That sentence is now "a preference is
yours to keep current", which says the same thing about preferences
without saying anything false about lessons.
Beside it, a paragraph on what growing a lesson means: another incident
added to what taught it, a claim stated more exactly, or a trigger
re-keyed to the situation that really fired. Written as a practice
rather than a prohibition (rule 165) — the reader is named as the one
person placed to judge the trigger, because they are standing in the
situation it claims to name.
`get_lesson` carries the same prompt at the moment it bites: a session
reading a lesson inside the situation it names is the only reader who
can tell whether the trigger is keyed to what actually fired.
The guidance-ownership registry gains the topic and re-points the
preference topic's statement, since the phrase it pinned is the sentence
this change rewrites — the module asks for exactly that, in the same
commit. No index marker: the index names session-start reflexes and this
one fires mid-work, so `_INSTRUCTIONS` stays at 1998/2000.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
2026-09-19 23:17:20 -04:00
759 changed files with 75100 additions and 26141 deletions
A self-hosted work system-of-record for software projects, built to be driven by Claude Code. Notes, tasks, issues, projects, milestones, rules, and stored processes — reachable from Claude via a built-in MCP endpoint and a bundled Claude Code plugin, with a clean web UI for humans. No in-app LLM; Claude is the sole assistant.
<h1 align="center">Fabled Grimoire</h1>
<p align="center">A self-hosted system of record for the work you do with Claude Code.</p>
Claude records what it learned, decided and built in Grimoire, and that record reaches the next session, so a solution is recalled instead of reinvented. Notes, tasks, plans, rules, lessons and reusable code live in one place. Claude reaches them through a built-in MCP endpoint and the bundled Claude Code plugin, and people read and edit the same records in the web UI. There is no language model inside Grimoire: Claude is the only assistant.
## Deploy
You need Docker with Compose. Save this as `docker-compose.yml` (it is the [`docker-compose.yml`](docker-compose.yml) in this repository), then run `docker compose up -d` beside it:
```yaml
# Fabled Grimoire: the app and its database.
# docker compose up -d
# Then open http://localhost:5000. The first account registered becomes admin.
#
# Keep both volumes. app_data holds the instance's own SECRET_KEY, which seals
# Port 5000 is published straight to clients, so no proxy is trusted.
# Behind a reverse proxy, set 1 (one per proxy) and BASE_URL:
TRUSTED_PROXY_HOPS:"0"
# BASE_URL: "https://grimoire.example.com"
volumes:
- app_data:/data
depends_on:
- db
restart:unless-stopped
db:
# Postgres 17 with the vector extension Grimoire needs.
image:pgvector/pgvector:pg17
stop_grace_period:120s
volumes:
- pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER:${POSTGRES_USER:-grimoire}
POSTGRES_PASSWORD:${POSTGRES_PASSWORD:-grimoire}
POSTGRES_DB:${POSTGRES_DB:-grimoire}
restart:unless-stopped
volumes:
app_data:
pgdata:
```
Open `http://localhost:5000`. The first account you register becomes the admin.
Nothing has to be set first. On its first start Grimoire makes its own `SECRET_KEY` and keeps it in the `app_data` volume, so restarts, reboots and updates all use the same key.
To connect Claude, create an API key under **Settings → MCP Access**, then in Claude Code:
The plugin asks for your Grimoire URL and the API key. Settings → MCP Access shows the exact commands for your instance.
## Configuration
Everything is optional. Set these in the environment or in a `.env` file beside the compose file:
| Variable | When | What it does |
|----------|------|--------------|
| `BASE_URL` | Behind a proxy | The public URL, used in email links and SSO redirects |
| `TRUSTED_PROXY_HOPS` | Rarely | The starting count of reverse proxies in front of Grimoire (default `1`; the compose file above sets `0` because it publishes the port directly). Check and change it in **Settings → Config → Reverse proxy**, which shows what each count reads for your own request; behind Cloudflare it is usually `2` |
| `POSTGRES_PASSWORD` | Before the first start | The database password (defaults to `grimoire`; the database is reachable only inside the compose network). Postgres takes it once, when the volume is created |
| `SECRET_KEY` | Rarely | A key you manage yourself, instead of the one Grimoire keeps. It wins over the kept key; remove it to go back |
Everything else (registration, email, SSO, git forge connections, retrieval tuning, backups) is in **Settings**, or listed in [Configuration](docs/configuration.md).
## Setup pitfalls
- **Keep both volumes.** `pgdata` holds everything Grimoire stores. `app_data` holds the instance's key, and forge tokens and the backup bucket's secret are sealed under it. If the key is lost, Settings asks you to enter those credentials again; nothing else is lost.
- **The embedding model downloads on first use**, into the `app_data` volume. Without internet access, semantic search falls back to keyword search until the model is in place.
- **Behind a reverse proxy, turn off response buffering** for `/mcp`, because MCP responses stream. [Configuration](docs/configuration.md#behind-a-reverse-proxy) has an Nginx example.
## Updating and backups
```bash
docker compose pull
docker compose up -d
```
Database migrations run when the app starts, and the key stays in its volume. Claude Code updates the plugin from its marketplace; run `/reload-plugins` after an update.
**Settings → Data → Scheduled backups** writes a daily backup of the database to a folder or an S3-compatible bucket, and Settings → Data → Restore from Backup rolls the instance back to one, or brings a new instance back from it. See [Backups](docs/backups.md).
## Features
Notes and tasks with a Markdown editor, sub-tasks, milestones, issues, and kanban project workspaces. Stored processes, an engineering rulebook system (with an inception step that decides what each project inherits), and semantic search with proactive knowledge-injection into Claude's context. A knowledge graph, per-user/group sharing, and a built-in MCP server (`/mcp`) plus a bundled Claude Code plugin so Claude can record and recall your work directly.
**Working with Claude**
- Claude enters a project and gets a short primer: its goal, current plans, open tasks and areas.
- Your rules reach Claude when its work matches them, or at a moment of work such as finishing a task or pushing code.
- Before Claude writes a file or runs a command, the plugin shows it the rules, lessons and reusable code that may apply.
- Stored procedures, such as a drift audit, become Claude Code skills.
## Quick Start
**Recording the work**
- Notes in a Markdown editor, with wikilinks, tags and version history.
- Tasks, issues and time-boxed investigations, with work logs that the next session reads first.
- Plans as milestones: a design plus ordered steps.
- Lessons learned the hard way, linked to the rules they illustrate.
- Duplicate checks, so a new record that matches an existing one updates it instead.
**Prerequisites:** Docker and Docker Compose. No GPU or local model needed — Claude is the sole assistant, reached over MCP.
**Reusing code**
- Snippets of reusable code, each with when to reach for it and where it lives in the repo.
- With a git forge connected, Grimoire tracks every function and style in your repos and shows which ones are still unaccounted for.
- Ideas shared across projects on the same platform, with each project's answer recorded.
- Design systems with tokens, inheritance and a generated stylesheet.
Download [`docker-compose.quickstart.yml`](docker-compose.quickstart.yml) from this repo, then:
**For people**
- A web UI to browse, search and edit everything, plus a graph of how notes connect.
- Sharing with users and groups, as viewer, editor or admin.
- Email and password sign-in, optional SSO through OIDC, and invite links.
- Personal export to Markdown or JSON, and admin backup and restore.
The full list is in [Features](docs/features.md).
## Troubleshooting
| What you see | Why | Fix |
|--------------|-----|-----|
| `SECRET_KEY is set to the public placeholder` at startup | `SECRET_KEY` is `dev-secret-change-me` or `change-me-in-production` | Remove `SECRET_KEY`, or set a real one |
| Settings asks for forge tokens or the backup secret again | The instance's key changed: the `app_data` volume was replaced, or `SECRET_KEY` was set or removed | Enter them again, or restore the previous key |
| `Can't locate revision identified by '…'` at startup | An older image is running against a database a newer one has already migrated | Run the newer image; migrations only move forward |
| The plugin's tools are missing in Claude Code | The plugin isn't installed, or Claude Code hasn't reloaded it | `/plugin` to install or update it, then `/reload-plugins` |
| Search finds only exact words | The embedding model hasn't downloaded yet | Give the app internet access once, or wait for the first download to finish |
## Development
```bash
# Optional but recommended — set a secret key
exportSECRET_KEY=your-random-secret-here
docker compose -f docker-compose.quickstart.yml up -d
docker compose -f docker-compose.dev.yml up --build
```
Open `http://localhost:5000`. The first user to register becomes admin. To connect Claude, create an API key under **Settings → API Keys** and install the Claude Code plugin — see [API Keys & MCP](docs/api-keys-and-mcp.md).
[Development](docs/development.md) covers checks, migrations, the plugin and CI. [Architecture](docs/architecture.md) describes how it is built, and the [API reference](docs/api-reference.md) lists every REST endpoint.
> **Development:** To build from source, see [Development](docs/development.md).
| [Features](docs/features.md) | Detailed feature breakdown and keyboard shortcuts |
| [Development](docs/development.md) | Dev workflow, CI/CD, migrations, release process |
| [API Keys & MCP](docs/api-keys-and-mcp.md) | API key management and Fable MCP install guide |
| [SSO / OAuth](docs/sso-oauth.md) | OIDC setup for Authentik, Keycloak, and other providers |
| [API Reference](docs/api-reference.md) | All REST API endpoints |
## License
This project is privately maintained.
Fabled Grimoire is free software under the [GNU Affero General Public License v3.0 only](LICENSE) (`AGPL-3.0-only`). If you run a modified version for other people over a network, the licence requires you to offer them its source.
A Python MCP (Model Context Protocol) server that lets Claude directly interface with a running Fabled Assistant instance. The goal is two-fold: Claude's stronger reasoning handles planning, documentation, and analysis while Fable remains the authoritative store for notes, tasks, projects, and milestones; and direct API access enables process improvement by letting Claude observe and operate on real data rather than working from descriptions.
The work is split into two sub-projects:
1.**Fable API Key Feature** — additions to the main `scribe` project to support bearer token authentication
2.**Fable MCP Server** — a new standalone Python package at `fable-mcp/` in the same repo root
A third sub-project (Forgejo MCP for CI/CD automation) is planned as a follow-on after the Fable MCP is working.
---
## Sub-project 1: Fable API Key Feature
### Motivation
All existing Fable routes use session-based authentication (`session["user_id"]`). The MCP server runs as a local process and cannot maintain a browser session, so a stateless bearer token mechanism is required. API keys are user-scoped and carry a read/write permission level.
### Database
New table `api_keys` via migration `0027_add_api_keys.py`:
-`down_revision = "0026"` (the briefing tables migration)
Full keys are generated as `fmcp_<32 random url-safe chars>`, returned once at creation, never stored. Only the SHA-256 hash is persisted.
### Auth Middleware (`auth.py`)
`_check_auth` is updated to check the `Authorization: Bearer <token>` header before falling back to session auth:
1. If header present: hash the token, look up in `api_keys` where `revoked_at IS NULL`
2. If found: update `last_used_at`, set `g.user` and `g.api_key`
3. Scope enforcement: if `g.api_key.scope == "read"` and `request.method` is not `GET`, return 403
4. If header absent: existing session logic runs unchanged
Session auth is untouched — no regression risk for the web UI.
**Admin routes and API keys:** Routes decorated with `admin_required` check `user.role == "admin"` after auth resolves. API keys authenticate as the key owner — so a write-scoped key for a non-admin user will fail admin-protected routes with 403 (role check, not scope check). This is intentional: API keys cannot elevate privilege beyond the user's role.
### Routes (`/api/api-keys`)
New blueprint `api_keys_bp`, registered in `app.py`:
-`GET /api/api-keys` — list caller's keys (prefix, name, scope, last_used_at, created_at — never the hash or full key)
-`POST /api/api-keys` — create key; body: `{name, scope}`. Returns full key in response **once only**
-`DELETE /api/api-keys/:id` — revoke by setting `revoked_at = now()`
### Settings UI
New "API Keys" tab in `SettingsView.vue` (added to `VALID_TABS`):
Calls the existing `semantic_search_notes()` service (already in `services/embeddings.py`). The `content_type` parameter maps to the service's `is_task` argument:
This endpoint is needed by the MCP's `search.py` tool module. It reuses existing infrastructure with no new ML work.
### Conversation Type: `"mcp"`
A new conversation type `"mcp"` is added alongside `"chat"` and `"briefing"`. This requires changes in two places in the main app:
**`services/chat.py`** — `create_conversation(user_id, title, model)` gains an optional `conversation_type: str = "chat"` parameter, passed through to the `Conversation` constructor.
**`routes/chat.py`** — `POST /api/chat/conversations` accepts an optional `conversation_type` body field (defaults to `"chat"`), passed to `create_conversation`. `GET /api/chat/conversations` continues to default-filter to `conversation_type="chat"` (excluding `"mcp"` and `"briefing"`); pass `?type=mcp` to retrieve MCP conversations.
**Retention:**`cleanup_old_conversations` in `routes/chat.py` currently deletes conversations regardless of type. It must be updated to exclude `conversation_type="mcp"` from the sweep, so MCP audit-trail conversations are not automatically pruned.
MCP conversations:
- Are created with a caller-supplied name (e.g., `"MCP Session 2026-03-23"`) or auto-named
- Are excluded from the default chat list
- Are accessible via `GET /api/chat/conversations?type=mcp` for inspection
- Appear in the Fable UI if explicitly navigated to, providing an audit trail of MCP-driven interactions
### Chat SSE Wire Format
The MCP's `chat.py` tool must consume the existing SSE stream. The relevant endpoints and event schema:
-`{"type": "done", "content": "...", "tools_used": [...]}` — generation complete; this is the termination event
The MCP tool reads tokens until it receives `type: "done"`, then returns `response` (full content) and `tools_used` (list of tool names).
---
## Sub-project 2: Fable MCP Server
### Location
`fable-mcp/` at the repository root, alongside `src/`, `frontend/`, `alembic/`. It is **not** part of the main Docker build and has no import relationship with `scribe`. It will be extracted to its own Forgejo repo once stable.
`FableClient` is an async context manager wrapping `httpx.AsyncClient`. It reads `FABLE_URL` and `FABLE_API_KEY` from environment variables (with `python-dotenv` fallback to `.env`). All methods are `async def` and return parsed dicts. A `FableAPIError(status_code, message)` exception is raised for any non-2xx response.
A module-level singleton `_client: FableClient` is initialized at server startup and shared across all tool modules.
### `server.py`
Uses `mcp[cli]`'s `FastMCP` class. Imports all tool modules, which register their tools via decorators against the shared `FastMCP` instance. Entry point `main()` calls `mcp.run()` (stdio transport).
### Tool Modules
Each module imports `_client` from `client.py` and defines tools as `async def` functions decorated with `@mcp.tool()`. Tool docstrings serve as the MCP tool descriptions visible to Claude.
- MCP conversations are excluded from the normal chat UI list
### Error Handling
`FableAPIError` is caught at each tool boundary and returned as a descriptive string rather than propagating as an exception. This ensures Claude receives a readable error message (e.g., `"Fable API error 403: Read-only key cannot perform write operations."`) rather than a stack trace. Network errors (`httpx.RequestError`) are similarly caught and surfaced as strings.
Scope enforcement lives entirely in Fable's auth middleware — the MCP does not duplicate it.
### Claude Code Registration
After `pip install -e fable-mcp/` (or `uv tool install ./fable-mcp`), add to `~/.claude/settings.json`:
```json
{
"mcpServers":{
"fable":{
"command":"fable-mcp",
"env":{
"FABLE_URL":"http://localhost:8080",
"FABLE_API_KEY":"fmcp_your_key_here"
}
}
}
}
```
Claude Code spawns the process over stdio automatically. No Docker, no daemon.
---
## Build & Repo Plan
1. Implement and test within `scribe/fable-mcp/`
2. Once stable, extract to a new Forgejo repo (`bvandeusen/fable-mcp`)
3. Forgejo MCP (Gitea MCP) added as a second MCP server to automate build/push/config workflows — separate spec when ready
---
## Out of Scope (v1)
- MCP resources (browsable URI tree) — tools cover all use cases in Claude Code
- Forgejo MCP — follow-on spec
- Rate limiting on API key endpoints
- Key expiry / TTL
- Per-resource scope (e.g., key restricted to one project)
- In `startup()`: `asyncio.create_task(load_stt_model())` + `asyncio.create_task(load_tts_model())` when `VOICE_ENABLED`
### `src/scribe/config.py`
- Add 4 new env var attributes
- Add validation in `validate()`
### `src/scribe/services/llm.py`
- Add `voice_mode: bool = False` and `voice_speech_style: str = "conversational"` to `build_context()`
- When `voice_mode=True`, prepend: *"Respond naturally as if speaking aloud. No markdown, bullet points, headers, or code blocks. Complete sentences only."*
- Append style modifier based on `voice_speech_style`
### `src/scribe/services/generation_task.py`
- Add `voice_mode: bool = False` to `run_generation()`
- Read `voice_speech_style` from settings when voice_mode; pass both to `build_context()`
### `src/scribe/routes/chat.py`
- Allow `"voice"` in `conversation_type` whitelist
### `src/scribe/services/chat.py`
- Exclude `conversation_type == "voice"` from auto-cleanup retention
Floating PTT button (fixed bottom-right). Creates/reuses a `"voice"` conversation. Full flow: record → transcribe → send → stream → synthesise → play. Space bar hotkey (from `App.vue`). Mounted globally in `App.vue`.
The current briefing pipeline hallucinates calendar events, tasks, and news items that do not exist in the user's actual data. Observed examples from production:
- A morning briefing asserting "your dentist appointment is still in progress" when no such event existed
- An 8am check-in mentioning "a quick meeting at 10:30 AM" with no backing calendar entry
- A midday check-in inventing "a team huddle at 2:30 PM" and "the Q2 budget draft due by Friday"
- The model calling `search_images` in response to a clarifying question about a fabricated meeting
These are not bugs in data retrieval — the data gathering code (`_gather_internal`, `_gather_external`) returns correct values. They are a structural consequence of how the briefing context is assembled.
## Root cause — the "no receipt" problem
The current `run_compilation` pipeline does this:
1. Python code gathers data (tasks, events, weather, news) from the database
2. Python formats the data into a structured text blob as the user-role message: `TODAY'S EVENTS: ...`, `DUE TODAY: ...`, etc.
3. The LLM is called **once** with `[system_prompt, user_prompt]` and produces prose
4. Only the prose reply is written to the conversation. **The underlying data is never persisted in the conversation history.**
When the user later chats in the briefing conversation, the chat endpoint loads the full conversation history — which contains the model's *prose* from earlier but not the data that prose was derived from. The model's own prior output becomes the only source of "truth" available for follow-up questions. If that prose asserted a fact (real or hallucinated), the model has no way to distinguish it from ground truth when generating the next reply, and it will double down.
Compounding factors:
- **Empty sections are silently omitted.** If `calendar_events` is empty, the user prompt contains no `TODAY'S EVENTS:` line at all. The model has no explicit "zero events" signal — combined with an imperative system prompt ("note calendar events and tasks"), it interprets the silence as "I should mention some" and fabricates.
- **Scheduled slot injections append synthetic turns.** `run_slot_injection` writes a fake `[Midday briefing update]` user message and the assistant reply into the persistent conversation. By evening, the chat history contains three separate briefings, each potentially with errors, all treated as equal-weight context on follow-up.
- **The `search_images` tool is available during briefing chat**, with a negative-instruction description ("Not for factual questions"). Small and mid-sized models frequently ignore negative guidance in tool descriptions and call the tool anyway.
## Solution — agentic briefing (the receipt model)
Replace the one-shot synthesis with a tool-call loop. The briefing is no longer "a text blob synthesized from pre-gathered data." It is **a scheduled agent run**: the LLM is given a system prompt, a curated set of read-only data tools, and a trigger ("generate the morning briefing"). The model must call tools to see what exists. Every tool call and tool result becomes part of the conversation history, where it lives as a permanent, structured receipt.
### Why this fixes the hallucination
When the model calls `list_events(today) → []`, that empty array is now a persistent message in the conversation. On a follow-up question like "what meeting?", the chat endpoint loads the conversation and the model sees its own tool result from the morning showing no events. Answering "you had a meeting at 10:30" would require the model to directly contradict a tool result sitting two messages back — something LLMs are much more reliable at avoiding than contradicting their own prior prose.
In short: **the model cannot fabricate what has a visible receipt proving it does not exist.**
### What the model sees — before vs after
**Before** — the model sees one structured blob and produces prose. The blob is discarded.
```
[system] You are a personal assistant... weave together what matters...
[user] Date: 2026-04-10
WEATHER: Home — partly cloudy, 8–16°C
DUE TODAY: Fix briefing hallucinations
OVERDUE (2 tasks): ...
[assistant] Morning! Looks like a quiet day with two overdue items...
```
**After** — the model is given tools and iterates to ground truth.
```
[system] You are the user's personal assistant giving their morning briefing.
Use tools to see what's relevant. Only mention things you learned from
tool calls. If a tool returns nothing, say so honestly.
[assistant] Morning! Partly cloudy today, 8 to 16 — nothing on the calendar
so it's a clean run at the desk. Two things to keep in mind...
```
The conversation now contains verifiable receipts: `list_events(today)` returned `[]`, and that result sits in context forever (until pruned). Follow-up questions operate against those receipts.
## Architecture
### Existing infrastructure (reused, not rebuilt)
The codebase already has the agentic primitives — they're used for regular chat:
- **`llm.py::stream_chat_with_tools`** — the streaming tool-use loop that talks to Ollama with a `tools` parameter and yields tool-call chunks
- **`generation_task.py::_stream_with_retry`** — wraps `stream_chat_with_tools` with retry-on-500 behavior for cold-model races
- **`tools.py`** — defines 40+ tool schemas and an execution dispatcher
The briefing bypasses all of this and calls a one-shot `_llm_synthesise` helper. The refactor is mainly "route briefings through the same pipeline regular chat already uses."
### New modules
**`briefing_tools.py`** — a small wrapper exposing a curated read-only subset of `tools.py` for briefing runs. This is an **explicit allowlist**, not a blocklist, so newly-added tools must be opted in:
| Tool | Purpose |
|---|---|
| `list_tasks` (with filter args) | See what's actionable today, overdue, high priority |
| `list_events` (today / upcoming) | Know what's on the calendar |
| `get_weather` | Current/forecast weather |
| `get_rss_items` | Pull news themes filtered by user preferences |
| `list_projects` | Understand project context |
| `search_projects` | Surface active project summaries |
| `list_notes` (recent) | Capture follow-ups from yesterday |
**Explicitly omitted** from the briefing tool set:
-`search_images`, `search_web`, `research_topic`, `read_article` — external search is not a briefing concern, and `search_images` is the source of the "Peter Kyle Science Secretary" image-search bug
- All `create_*`, `update_*`, `delete_*` — briefings are read-only; a scheduled background job must not decide to mutate the user's data on its own
-`set_rag_scope`, `calculate` — not relevant to briefing content
### New briefing function
**`briefing_pipeline.py::run_agentic_briefing(user_id, slot, model, conversation_id)`** replaces `run_compilation`'s body (and eventually `run_slot_injection`). Internally:
1. Build a slot-specific system prompt (see below)
2. Load the curated briefing tools from `briefing_tools.py`
3. Seed messages with `[system, user]` where the user message is a simple trigger like `"Generate the morning briefing."`
4. Enter a tool-call loop (max 8 iterations):
- Call `stream_chat_with_tools`
- If the model returns `tool_calls`, execute them via the existing dispatcher, append tool results, continue
- If the model returns a final assistant message with no pending tool calls, break
The full message list is important: it's written to the conversation along with the final prose, so the tool-call receipts become part of the persistent record.
### Slot-specific system prompts
Compilation (full morning briefing):
```
You are the user's personal assistant giving their full morning briefing.
Use the tools available to see what's actually relevant today — tasks due,
overdue items, events on the calendar, weather, news themes, project state
— and weave it into a warm, natural-sounding summary.
Rules:
- Call tools to see the data. Never assert facts you didn't learn from a tool.
- If a tool returns nothing (no events today, no overdue tasks), say so
honestly. Don't fabricate items to fill space.
- Write flowing prose. No markdown, no headers, no bullet points.
- Aim for 6–10 sentences. Skip topics that have nothing interesting.
- Close on one or two concrete, actionable suggestions.
User profile (for tone and preferences):
{profile_body}
```
Check-ins (midday, afternoon):
```
You are the user's personal assistant giving a brief {slot} check-in.
Use tools to see what's changed since this morning. Focus on progress
and what's still unaddressed.
Rules:
- Call tools to see current state. Never assert facts without tool results.
- If nothing meaningful has changed, say so briefly — don't invent progress.
- 3–5 sentences, natural prose, no markdown.
```
### Conversation hygiene — removing the fake user messages
The current scheduler appends two fake messages on every slot injection:
Under the agentic model, we drop the fake `"user"` message entirely. Slot updates become plain assistant messages tagged with `metadata.briefing_slot = "midday"`. The chat endpoint's message loader is updated to filter these when building the LLM context on follow-ups, so a user chatting in the briefing conversation doesn't see three earlier briefings smashed into their history. The UI continues to show them as visible timeline entries.
(This is the Option A filter from the earlier discussion — a small, surgical change compared to migrating briefings to a separate sidecar table. Sidecar storage remains a possible future step.)
### Ollama setup compatibility
The existing Ollama deployment uses non-parallel mode across two GPUs, with a concern about context duplication between cards. This is the correct setup for agentic briefings:
- Each tool-call iteration shares the same KV cache on the same GPU, so appending tool results is cheap
- There is no race where a second iteration could land on a different card with a different cache state
- The trade-off is serialization: a briefing in progress will block a concurrent user chat request until it finishes, but scheduled briefings are rare enough (4× per day) that this is acceptable
If GPU contention becomes a problem later, the right lever is pinning specific models to specific GPUs (e.g., background tasks on GPU 1, interactive models on GPU 0) — not enabling parallelism.
## Cost & trade-offs
- **Latency:** a briefing now makes 5–7 inference calls (one per tool-call decision plus the final prose) instead of 1. On a local Ollama with a 7B model, expect 15–40s per briefing vs the current 5–10s. Acceptable for a background job; if it becomes painful at the user-facing check-in slots, investigate letting the model batch independent tool calls in a single turn (Ollama supports this).
- **Model requirements:** the default model must be reliable at tool calling. `qwen2.5:7b`, `llama3.1:8b`, `mistral-small-3:24b`, and similar handle it well. Models in the ≤3B class typically fail — they either emit no tool calls and return empty prose, or hallucinate invalid tool arguments. If a user's `default_model` is too small, agentic mode should fall back to legacy mode with a log warning.
- **Context growth:** tool results bloat the conversation. At 8 tool calls per compilation × 4 slots per day, a daily briefing conversation can reach 20-30 KB of JSON-heavy history. Fine for a day; aged briefings should be archived/rolled up after 7 days.
- **Tool subset drift:** someone adds a new mutating tool to `tools.py` and forgets to update the briefing allowlist. Mitigated by the allowlist model — the default for new tools is "not exposed to briefings."
- **Infinite loop safety:** a buggy model could tool-call forever. Hard cap at 8 iterations, log a warning if hit, return whatever prose was last produced (or a fallback message).
## Migration path
Ship incrementally, each step independently reversible:
**PR 1 — Agentic compilation behind a feature flag.** Add `briefing_tools.py`, add `run_agentic_briefing`, add a per-user setting `briefing_mode: "legacy" | "agentic"` (default legacy). Route only the 4am compilation through the new path when the flag is set. Keep slot-injection on the legacy path. Enable the flag on the author's account first, validate output quality over several days, then flip the default for all users. No DB migration required — the setting lives in the existing `settings` table.
**PR 2 — Agentic slot injections + conversation hygiene.** Migrate midday/afternoon check-ins to the same pipeline. Remove the fake `[Midday briefing update]` user-role message; slot updates become plain assistant messages tagged with `metadata.briefing_slot`. Add a chat-message-loader filter that excludes slot-tagged messages from the LLM context on follow-ups (they remain visible in the UI).
**PR 3 — UI polish.** Collapsed tool-call status row in the briefing card ("✓ checked calendar · ✓ looked at tasks · ✓ pulled weather"), expanding to show tool results on click. Tool-result cards (weather, news, task list) rendered inline where useful.
**PR 4 (optional) — Sidecar storage for briefing snapshots.** If the chat-filter approach in PR 2 feels too hacky, migrate briefings out of the conversations table and into a dedicated `briefing_snapshots` table. Frontend renders them as pinned timeline cards separate from chat. Larger refactor; defer until PR 1–3 prove the approach works.
## Secondary win — tightening the main chat system prompt
The regular chat is already agentic (it uses `stream_chat_with_tools`), but its system prompt does not explicitly require the model to ground factual claims in tool results. The prompt discipline introduced for briefings — *"Never assert facts you didn't learn from a tool. If a tool returns nothing, say so honestly."* — is worth applying to the main chat system prompt in a follow-up PR. The mechanism already works; only the framing needs tightening.
## Out of scope
- The Android Flutter client's failure to render `search_images` tool-result cards. This is a separate rendering gap in the mobile app and does not affect the server-side fix. Tracked separately.
- Re-evaluating the Ollama parallelism setting. The current non-parallel config is correct for this work.
- Replacing the background model for title/summary/observation extraction. That model's role is unrelated to briefings.
API keys let external tools access your Fable data without a browser session. Each key is scoped to a single user — it can only access data that user owns or has been shared with them.
API keys let external tools access your Grimoire data without a browser session. Each key is scoped to a single user — it can only access data that user owns or has been shared with them.
### Scopes
@@ -15,40 +15,51 @@ Admin-level operations (log access, user management) require a `write`-scoped ke
### Creating a Key
1. Go to **Settings → API Keys**
1. Go to **Settings → MCP Access**
2. Enter a name (e.g. "Claude MCP", "Home Server")
3. Choose scope
4. Click **Generate Key**
5. Copy the key immediately — it is shown only once (the token is `fmcp_`-prefixed)
Paste the key into the `Authorization: Bearer <key>` header of your MCP client
config (see **Scribe MCP Server** below).
config (see **Grimoire MCP Server** below).
### Revoking a Key
Click **Revoke** next to the key in the API Keys table and confirm. Revoked keys are deleted immediately.
Click **Revoke** next to the key in the keys table and confirm. Revoked keys are deleted immediately.
---
## Scribe MCP Server
## Grimoire MCP Server
Scribe exposes itself as a set of MCP tools that Claude (and other MCP clients)
Grimoire exposes itself as a set of MCP tools that Claude (and other MCP clients)
can use to read and write your notes, tasks, projects, rulebooks, and more. The
server is **built into the app** — it is mounted as a streamable-HTTP endpoint
at **`/mcp`** on the running Scribe instance (`src/scribe/mcp/server.py`). There
at **`/mcp`** on the running Grimoire instance (`src/grimoire/mcp/server.py`). There
is nothing to install: no wheel, no separate package, no CLI. You connect a
client straight to the URL with a Bearer token.
### Authentication
Authenticate with an API key generated from **Settings → API Keys** (see above),
Authenticate with an API key generated from **Settings → MCP Access** (see above),
sent as `Authorization: Bearer fmcp_<key>`. A `read`-scoped key may call only the
any write/delete tool is rejected with `403`. The allow-list is explicit rather
than derived from the name — see `_READ_ONLY_TOOLS`, which is why the two reads
without a read-shaped name are spelled out here. A `write`-scoped key may call everything.
### Claude Code (Project-scoped)
### Claude Code: the plugin (recommended)
The plugin connects the MCP server and adds the hooks and skills that put Grimoire's records in front of Claude as it works. Install it from this repo:
It asks for your Grimoire base URL, an API key (kept in your OS keychain) and, optionally, a project id. Settings → MCP Access shows the exact command for your instance. [`plugin/README.md`](../plugin/README.md) describes each hook.
### Claude Code without the plugin
Add a `.mcp.json` at the project root. The server `type` is `http` and the URL is
your instance's `/mcp` endpoint:
@@ -56,9 +67,9 @@ your instance's `/mcp` endpoint:
| Search / Recall | `search`, `get_recent`, `list_tags`, `retrieval_telemetry` | Semantic + structured recall, and the readout its thresholds are tuned from |
| GET | `/api/plugin/retrieve` | Title-first knowledge-injection candidates |
| GET | `/api/plugin/context` | Session-start context: the activeproject and its state |
| GET | `/api/plugin/retrieve` | Records and rules matching the operator's prompt |
| GET | `/api/plugin/tool-rules` | Rules matching a tool call about to run |
| GET | `/api/plugin/moment-tools` | The tools whose calls can reach a moment |
| POST | `/api/plugin/moment` | The rules mounted on the moments a tool call reaches |
| POST | `/api/plugin/reply-rules` | Whether the reply ending a turn is held for one read |
| POST | `/api/plugin/rule-opened` | Records that a session opened a rule it was shown |
| GET | `/api/plugin/shape-check` | The end-of-turn question about what this turn wrote |
| GET | `/api/plugin/claim-session` · `/release-session` | Bind a session to its task claims / release them at session end |
| GET | `/api/plugin/write-scope` | Which projects a pending act would change (a directory, or an MCP call's `tool` and `input`), and whether a session in its own project may do it. Filing an issue elsewhere is allowed; every other cross-project write is answered `allowed: false` with the reason the operator is asked |
| GET | `/api/plugin/processes` | Stored Processes for skill-stub sync |
| GET | `/api/plugin/prior-art` | Write-path hint for the plugin hooks (params: `path`, `code`, `repo`, `shapes`, `exclude_ids`, `exclude_sync_ids`, `exclude_derive`); returns `context`, `note_ids`, `sync_note_ids`, `stamped`, `divergence`, `derive`, `derive_keys` |
| GET / POST | `/api/projects/<id>/coverage`, `…/coverage/refresh` | Shape-ledger accounting (`pattern_coverage` line, counts, `derive_groups` — css groups carry `consumers`, `derive_new`, `unused_css`, `divergence`, `recheck`) |
@@ -189,19 +273,32 @@ endpoint at `/mcp`, not these REST routes.
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/admin/backup` | Export backup (format v4; `?scope=user` for own data) |
| POST | `/api/admin/restore` | Restore from a backup |
| GET | `/api/admin/restore/sources` | The backups at the backup destination, newest first |
| POST | `/api/admin/restore/preview` | What restoring a backup (`?source=<name>` or an uploaded file) would change; 409 with the reason when it can't apply here |
| POST | `/api/admin/restore?confirm=rollback` | Put the whole instance back to that backup, after saving the current state |
| GET / DELETE | `/api/admin/users` · `/api/admin/users/:id` | List / delete users |
| GET / PUT | `/api/admin/registration` | Get / toggle registration |
| GET / PUT | `/api/admin/smtp` · POST `/api/admin/smtp/test` | SMTP config + test email |
| GET / PUT | `/api/admin/base-url` | Get / set the public base URL |
| GET / PUT | `/api/admin/proxy` | Get / set how many reverse proxies stand in front of Grimoire (`{"hops": n}`, or `null` for the environment's count), with what each count would read for this request |
| GET / PUT | `/api/admin/db-maintenance` (+ `/health`, POST `/run`) | VACUUM schedule, health, manual run |
| POST / GET / DELETE | `/api/admin/invitations` (+ `/:id`) | Create / list / revoke invite links |
| GET / PUT | `/api/admin/backups` | Scheduled backups: destination, schedule and recent runs |
| POST | `/api/admin/backups/check` | Ask the destination whether it would accept a backup |
| POST | `/api/admin/backups/run` | Start a backup now |
| GET / PUT | `/api/admin/forge-webhook` | The forge push-webhook settings |
## Scribe MCP
## Webhooks
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/webhooks/forge` | Push events from a git forge, verified with `FORGE_WEBHOOK_SECRET`**(public)** |
## Grimoire MCP
Claude clients connect to the built-in MCP server at `POST /mcp` (streamable HTTP,
Bearer auth with an `fmcp_` key), served by `src/scribe/mcp/`. It is not a REST
Bearer auth with an `fmcp_` key), served by `src/grimoire/mcp/`. It is not a REST
surface — it exposes the same data as typed tools (`create_note`, `create_task`,
`start_planning`, `search`, `enter_project`, …) with
server-level usage guidance delivered in the MCP `instructions` block. See
Grimoire is one app container and one PostgreSQL container. There is no language model in the stack: Claude works with Grimoire from outside, through the MCP endpoint and the Claude Code plugin, and people use the web UI.
## Stack
| Layer | Technology | Notes |
|-------|-----------|-------|
| Frontend | Vue 3 + TypeScript + Vite + Pinia + Vue Router | SPA served from the same container as the API |
Browser ──────── Vue SPA ────────── /api/* ┘ └── in-process jobs and embedder
```
## Project Structure
- **The web UI** is a Vue single-page app, built into the image and served by Quart. Every unknown path that isn't under `/api/` falls back to `index.html`; `/api/*` always answers JSON.
- **The REST API** (`src/grimoire/routes/`) backs the UI, one blueprint per resource.
- **The MCP server** (`src/grimoire/mcp/`) is how Claude reads and writes records. Each module in `mcp/tools/` registers its tools, and the server's instructions and each tool's docstring are the contract an agent reads. Requests authenticate with an API key as a Bearer token.
- **The plugin** (`plugin/`) runs in the user's Claude Code. Its hooks call `/api/plugin/*` to put the right context in front of Claude at the right moment: the project at session start, matching rules and prior art before a prompt or a tool call, the rules mounted on a moment of work, and a check on the reply that ends a turn. Its skills describe the working practices. See [API Keys & MCP](api-keys-and-mcp.md) to install it.
- **Services** (`src/grimoire/services/`) hold the logic both doors share. Routes and MCP tools stay thin: parse the request, check access, call a service, return a plain dict.
```
scribe/
├── docker-compose.yml # Development stack
├── docker-compose.prod.yml # Production stack (Docker Swarm)
Most records are rows in one `notes` table, told apart by kind:
**Single container for frontend + API.** Quart serves the Vue.js production build as static files and exposes the REST API under `/api/`. The SPA is built by Vite during the Docker image build.
- **Notes**, and the specialised kinds that are notes underneath: **snippets** (recorded prior art, with a canonical code location), **lessons** (transferable insights, linked to the rules they instance) and **processes** (stored procedures, synced to Claude Code as skills).
- **Tasks**: a note with a status. A task's kind is `work`, `issue` or `spike`. Tasks carry work logs, sub-tasks, recurrence and a session claim.
**SPA routing via 404 handler.**`app.py` uses `@app.errorhandler(404)` (not a catch-all route) to serve static files or fall back to `index.html`. API routes (`/api/*`) always return JSON 404. This avoids a catch-all `/<path:path>` route intercepting API GETs.
Around them:
**Unified note/task model.** A task is just a note with task attributes enabled. `status IS NOT NULL` means it's a task. "Convert to task" sets `status='todo'`; "convert to note" clears `status`, `priority`, `due_date`. No separate table, no cascade complexity.
- **Projects** group work and carry a goal. **Milestones** are plans: a design body plus ordered step tasks.
- **Systems** are a project's named areas; records are tagged with them, and each maps to a canonical system shared across projects.
- **Rulebooks → topics → rules.** A rule in a rulebook is global; a rule on a project applies there only. Rules have a trigger (`when_to_apply`), versions, relations to other rules, and can be mounted on **moments** of work (`work.change`, `reply.report`, ...), which plugin actions reach through moment mappings.
- **Family canon**: ideas shared across projects on the same platform, and each project's answer to them (adopted, variant, owed).
- **Design systems**: tokens and guidance, with inheritance; the stylesheet is generated from them.
- **The shape ledger**: code shapes extracted from bound repos and their classification against snippets.
- **Embeddings** for notes (chunked), rules, milestones and systems, in pgvector columns.
- **Telemetry**: retrieval logs, note and rule usage, and the tuning history of retrieval thresholds.
- **Users, groups, shares and API keys.** Access is decided in the app (`services/access.py`): owners, direct shares and group shares, with read or write permission.
**First-class tag column.** Tags live in a `tags ARRAY[text]` column and are explicitly set by the client — not auto-extracted from body text. Hierarchical tags (`project/webapp`) supported via SQL `unnest + LIKE` prefix matching.
Deleted records go to the trash first and can be restored until retention purges them.
**Background generation architecture.** LLM streaming runs in a detached `asyncio.Task` that writes into an in-memory `GenerationBuffer`. SSE clients tail the buffer and can reconnect mid-stream without data loss. Buffer has a `cancel_event` for user-initiated stop. Completed buffers are cleaned up after 60s grace period. Periodic DB flushes every 5s preserve partial content. Both chat and AI Assist use this architecture.
## Retrieval
**SSE over WebSockets for LLM streaming.** SSE clients connect via `GET /api/chat/conversations/:id/generation/stream` with `Last-Event-ID` reconnection support. Frontend uses `fetch()` + `ReadableStream`.
Search and the plugin's context injection run on the same pipeline (`services/retrieval_pipeline.py`): embed the query, rank stored chunks by cosine similarity in pgvector, keep what clears the surface's threshold, and log what was shown. Each surface (MCP search, auto-inject, the pre-tool rule arm, the write-path prior-art arm, ...) is a spec over those stages with its own thresholds and budget. The thresholds are settings an admin can tune, and every change is recorded with its reason. When the embedder is unavailable, search falls back to keyword matching.
**Context building is server-side.** Backend fetches URL content and searches notes. Frontend sends the message text + optional context note IDs. `build_context()` returns `(messages, context_meta)`; metadata includes auto-found note IDs/titles sent to frontend via a `context` SSE event beforestreaming begins.
The embedding model downloads into `FASTEMBED_CACHE_DIR` (default `/data/fastembed-cache`) on first use, so the `/data` volume keeps it across restarts.
**No blocking long-running operations.** Any slow operation (model pulls, LLM calls, URL fetching) must never block app startup or freeze the UI. Backend uses SSE streaming for incremental responses. Model pulls stream NDJSON progress to the frontend.
## Background work
**SSRF protection.**`services/llm.py` blocks requests to loopback, private, link-local, reserved, and multicast addresses before fetching. `follow_redirects=False` prevents redirect-based bypasses.
All in-process; there is no separate worker container.
**Session cookie security.**`HttpOnly` and `SameSite=Lax` always set. `Secure` flag controlled by `SECURE_COOKIES` env var.
- At startup the app records any shipped default that moved in this release, starts its schedulers, and only then backfills missing embeddings, so the backfill never competes with startup for the database.
- Recurring tasks spawn every 15 minutes. Daily jobs: version auto-pinning (03:00 UTC), trash purge (03:30 UTC), targeted `VACUUM ANALYZE` (04:00 UTC by default). Scheduled backups check hourly whether one is due ([Backups](backups.md)).
- Log, notification and auth-token retention run on their own loops.
- A diagnostics heartbeat writes to `/data/diagnostics`.
**Rate limiting.** In-memory sliding-window rate limiter (`rate_limit.py`) applied to auth endpoints: login (10/60s), register (5/300s), forgot-password (5/300s), reset-password (10/60s). Keys are per-IP. `TRUST_PROXY_HEADERS` env var enables `X-Forwarded-For` / `X-Real-IP` when behind a reverse proxy.
**Idempotent migrations.** All Alembic migrations use raw SQL with `CREATE TABLE IF NOT EXISTS`, `CREATE INDEX IF NOT EXISTS`, and `DO $$ BEGIN CREATE TYPE ... EXCEPTION WHEN duplicate_object` to allow safe re-runs.
## Data Model
### Users
| Column | Type | Notes |
|--------|------|-------|
| `id` | int PK | |
| `username` | text UNIQUE NOT NULL | |
| `email` | text nullable | |
| `password_hash` | text nullable | NULL for OAuth-only accounts |
`project_shares`, `note_shares`: each has `shared_with_user_id` OR `shared_with_group_id` (exclusive), `permission` (`viewer`/`editor`/`admin`), `invited_by`.
`groups`, `group_memberships`: platform-wide groups with `member`/`owner` roles.
Permission resolution is centralised in `services/access.py`. `get_project_permission(uid, project_id)` checks ownership → direct share → group-based share → note→project inheritance, returning the highest applicable permission.
### Journal-Related Tables
`weather_cache`: per-user, per-`location_key` cache. Columns: `user_id`, `location_key` (`home`/`work`/etc.), `location_label`, `forecast_json` (Open-Meteo response), `previous_json` (last forecast, used to detect changes), `fetched_at`. Lat/lon are *not* stored on the cache row — they live in the user's `journal_config.locations.{home|work}` setting and are used at refresh time.
### API Keys
`api_keys`: `id`, `user_id` FK CASCADE, `prefix` (first 8 chars, displayed in UI), `key_hash` (SHA-256 of full key — full key never stored), `name`, `scope` (`read`/`write`), `created_at`, `last_used_at`.
| `services/task_logs.py` | Append/list/delete work log entries on tasks |
| `services/journal_prep.py` | Deterministic data gather (tasks/events/weather/projects/recent moments/open threads) → LLM prose opener; persisted as the first assistant message of today's journal Conversation |
| `services/journal_pipeline.py` | System-prompt builder for journal conversations; calls `build_profile_context()` so the LLM sees the user's profile + learned summary |
| `services/journal_scheduler.py` | APScheduler `BackgroundScheduler`; per-user prep job; live-reschedule via `update_user_schedule()`; catch-up logic for missed runs |
| `services/journal_search.py`, `services/moments.py` | Moment recording + search across journal history |
| `services/user_profile.py` | `build_profile_context()` consolidates profile + observations + learned_summary into a system-prompt block |
| `services/research.py` | SearXNG research pipeline: sub-queries → parallel fetch → outline → section synthesis → executive summary → index note with linked section notes |
| `services/events.py` | Internal events CRUD: `list_events`, `create_event`, `update_event`, `delete_event`, `get_event`; source of truth for all event LLM tools |
| `services/rss.py` | feedparser-based fetch, per-feed DB cache, prune-to-100 |
Every database wait on startup has a deadline. Migrations wait for the database to accept connections (up to five minutes) before they run, so the app starts cleanly whatever order the containers come up in.
## Authentication
**Session cookies** —`HttpOnly`, `SameSite=Lax`, optionally `Secure` (`SECURE_COOKIES` env var). Session includes `session_version`; mismatch with DB value (after password change) results in 401 and session clear.
-**Sessions**:`HttpOnly`, `SameSite=Lax` cookies, `Secure` on a request the trusted proxy reports as HTTPS, which also gets HSTS (`grimoire/proxy.py`). Changing a password bumps the user's session version and signs out their other sessions.
- **API keys**: `Authorization: Bearer fmcp_...`, stored hashed. A `read` key may only call read endpoints and read MCP tools.
- **OIDC** single sign-on with PKCE, optionally replacing local logins ([SSO / OAuth](sso-oauth.md)).
- **Rate limits** on register, login and forgot-password, per client IP, read from `X-Forwarded-For` by `TRUSTED_PROXY_HOPS`.
**Bearer token / API keys** — `Authorization: Bearer <key>` accepted by `_check_auth()` as an alternative to session cookies. The raw key is SHA-256 hashed and looked up via `services/api_keys.py`. Keys with `scope=read` are rejected on non-safe methods (`POST`/`PATCH`/`DELETE`). Used by Fable MCP and any external API consumers.
Credentials Grimoire stores for outside services (forge tokens, the backup destination's S3 secret) are sealed under a key derived from `SECRET_KEY`, which the instance makes on first start and keeps in `/data/secret_key` unless one is set.
**Local auth + OIDC/OAuth (PKCE)** — `services/oauth.py` handles discovery and `find_or_create_oauth_user`. On OAuth login: checks existing `oauth_sub` → matching email → creates new user.
## Repository layout
See [sso-oauth.md](sso-oauth.md) for provider-specific setup instructions.
## LLM Pipeline Internals
### Tool Routing
No separate intent router — the main model handles all tool routing directly via Ollama's structured tool-calling output. The model receives the full tool schema list and decides whether to call a tool or respond conversationally. Extended reasoning (`think=True`) is always on for qwen3-class models: content-based gating was tried but exposed tool-call template fragility on short tool-intent prompts.
### Tool Loop
Multi-round tool loop (max 5 rounds). All implementations in `services/tools/` (decorator-based registry); `execute_tool(user_id, tool_name, arguments, conv_id=None, workspace_project_id=None)` is the dispatcher. `conv_id` and `workspace_project_id` are threaded in from `run_generation()` so tools like `set_rag_scope` can write to the current conversation.
**Unified `create_note` tool** — creates both notes and tasks. Setting `status` (e.g. `"todo"`) creates a task; omitting it creates a knowledge note. All task fields (due_date, priority, milestone, parent_task, recurrence_rule) are available on the single tool.
**Duplicate protection on `create_note`:**
1. Exact title match (case-insensitive) → hard block, redirect to `update_note`
2. Fuzzy title match (SequenceMatcher ≥ 82%; punctuation stripped before candidate search) → hard block
3. Semantic content similarity (threshold 0.90, body ≥ 80 chars) → soft block with `requires_confirmation: true`
**Project resolution** (`_helpers.resolve_project`): 4-step lookup — (1) exact DB match, (2) `query in title` substring, (3) `title in query` reverse substring, (4) SequenceMatcher ≥ 0.55.
### Context Window and Summarisation
`OLLAMA_NUM_CTX` (default 16384) controls the context window for all generation calls. Intent classification always uses `num_ctx=4096` to reduce VRAM pressure.
History summarisation threshold: 30 messages. Keeps 8 recent messages. Summary max 400 tokens.
### Web Research Pipeline
`services/research.py` implements a full autonomous research pipeline:
1. Intent model generates 5 focused sub-queries
2. All 5 SearXNG queries run in parallel (200ms stagger to avoid rate limiter)
3. Up to 15 unique URLs fetched in parallel
4. LLM generates an outline (2–8 sections with title + focus)
5. Each section synthesised in parallel from relevant sources
6. Executive summary generated from all section content
7. Index note created with executive summary + links to section notes; section notes linked back via `parent_id`
8. Falls back to single-note synthesis if outline generation fails
SearXNG tip: add the app server IP to `botdetection.ip_lists.pass_ip` in SearXNG `settings.yml` to bypass the rate limiter for trusted backend requests.
### Image Cache
`search_images` tool fetches images server-side via SearXNG, stores them on disk (SHA-256 dedup, content-type validation, 5 MB cap), and serves from `/api/images/<id>`. The user's browser never contacts the original image host.
1.`semantic_search_notes()` — cosine similarity via pgvector, threshold configurable per call; accepts `orphan_only` and `project_id` scope flags.
2. Notes ≥ 0.60 similarity auto-injected into system prompt (up to 3, 800 chars each).
3. Notes 0.45–0.60 surfaced in chat sidebar as "Suggested" (user clicks to include).
4. Explicitly included notes delivered full-body.
5.`excluded_note_ids` prevents the current note from being injected as its own context.
### RAG Scope (three-value system)
`conversations.rag_project_id` controls which notes are eligible for retrieval:
| Value | Behaviour |
|-------|-----------|
| `NULL` (default) | Orphan notes only — notes with `project_id IS NULL` |
| `-1` | All notes — opt-in to global search |
| Positive int | That project's notes only |
`build_context()` derives `orphan_only` + `effective_project_id` from this value before calling both the semantic and keyword search paths.
**Scope tools** — Two LLM tools let the model discover and switch scope mid-conversation:
-`search_projects` — SequenceMatcher scoring over title + description + `auto_summary`; returns top 5 matching projects.
-`set_rag_scope` — persists the new `rag_project_id` to DB immediately; blocked in workspace view; causes the SSE `done` event to include `new_rag_scope` + `new_rag_scope_label` so the frontend chip updates reactively.
**Project summaries** — `generate_project_summary()` calls Ollama (fire-and-forget) and stores the result in `projects.auto_summary`. Triggered on project update and note saves (debounced 1h). `backfill_project_summaries()` runs at startup.
Embedding model: `nomic-embed-text` via Ollama. Backfill runs 30s after startup (background task).
```
├── alembic/versions/ database migrations
├── src/grimoire/
│ ├── app.py app factory, blueprints, startup and shutdown
│ ├── config.py environment configuration
│ ├── models/ SQLAlchemy models
│ ├── routes/ REST blueprints
│ ├── mcp/ MCP server and its tool modules
│ ├── services/ shared logic
│ └── restore.py `python -m grimoire.restore`: the restore, from the command line
├── frontend/src/ views, components, stores, composables, api client
├── plugin/ Claude Code hooks, skills and commands
├── scripts/ CI checks and the plugin version minter
└── tests/ unit tests, plus `-m integration` against real Postgres
Configuration is via environment variables. The `docker-compose.yml` file sets defaults for local development; override them in a `.env` file (gitignored).
Grimoire reads its configuration from environment variables. Most settings people change day to day (registration, SMTP, retrieval thresholds, backups, forge connections) live in **Settings** in the web UI instead.
## Environment Variables
Any variable marked *secret* can also be read from a file: set `<NAME>_FILE` to the path, for example a Docker secret at `/run/secrets/grimoire_key`.
| `SECURE_COOKIES` | `false` | Set `true` when running behind TLS |
| `BASE_URL` | — | Public URL (e.g. `https://notes.example.com`) — required for OIDC redirect URIs and email links |
| `DATABASE_URL`*(secret)* | `postgresql+asyncpg://grimoire:grimoire@localhost:5432/grimoire` | PostgreSQL connection string, using the `asyncpg` driver |
| `SECRET_KEY`*(secret)* | kept in `DATA_DIR` | Signs sessions and seals stored credentials. Unset, the instance makes its own on first start and keeps it in `DATA_DIR/secret_key`. Set it only to manage the key yourself; it wins over the kept one (see below) |
| `DATA_DIR` | `/data` | Where the instance keeps its own `SECRET_KEY`. Mount a volume here |
| `BASE_URL` | `http://localhost:5000` | Public URL, used in email links and OIDC redirects |
| `TRUSTED_PROXY_HOPS` | `1` | The starting count of reverse proxies in front of Grimoire. **Settings → Config → Reverse proxy** is where the count is checked and set, and a count saved there wins over this. The client address and the scheme are read that many entries in from the right of `X-Forwarded-For` and `X-Forwarded-Proto`; `0` ignores both headers. See [Behind a reverse proxy](#behind-a-reverse-proxy) |
See [sso-oauth.md](sso-oauth.md) for provider-specific setup.
SSO turns on when the issuer, client ID and secret are all set. See [SSO / OAuth](sso-oauth.md).
### Security / Proxy
### Email
Admins can set SMTP in **Settings**; these are the defaults until they do.
| Variable | Default |
|----------|---------|
| `SMTP_HOST` | — |
| `SMTP_PORT` | `587` |
| `SMTP_USERNAME` | — |
| `SMTP_PASSWORD`*(secret)* | — |
| `SMTP_FROM_ADDRESS` | — |
| `SMTP_FROM_NAME` | `Fabled Grimoire` |
| `SMTP_USE_TLS` | `true` |
### Integrations
| Variable | Default | Description |
|----------|---------|-------------|
| `TRUST_PROXY_HEADERS` | `false` | Set `true` when behind a trusted reverse proxy to read real IP from `X-Forwarded-For` / `X-Real-IP` |
| `PLUGIN_MARKETPLACE_URL` | this repo's git URL | The git URL shown in the plugin install command under Settings → MCP Access. An admin can override it in Settings |
| `FORGE_KIND`, `FORGE_BASE_URL`, `FORGE_TOKEN`*(secret)* | — | A git forge connection for admins' projects, so a token can live in a Docker secret. Each user can add their own under Settings → Integrations; a stored connection for the same host wins |
| `FORGE_WEBHOOK_SECRET`*(secret)* | — | Verifies push webhooks from the forge |
| `SEARXNG_URL` | — | A SearXNG instance, listed as the web-search backend under Settings → Integrations |
### Web Search / Images
## Running it
| Variable | Default | Description |
|----------|---------|-------------|
| `SEARXNG_URL` | — | SearXNG base URL for web search and image search tools |
| `IMAGE_MAX_BYTES` | `5242880` (5 MB) | Maximum size for cached images |
### Docker Compose
### Data / Logging
| Variable | Default | Description |
|----------|---------|-------------|
| `LOG_RETENTION_DAYS` | `90` | Days to keep app logs before automatic pruning |
| `DATA_DIR` | `/data` | Root directory for persistent data (VAPID keys, backups) |
## Docker Compose Setup
### Development (`docker-compose.yml`)
[`docker-compose.yml`](../docker-compose.yml) runs the published image with its own database, and needs nothing set:
```bash
# Copy the example env file
cp .env.example .env
# Edit .env to set a real SECRET_KEY
# Start the stack
docker compose up --build
# App available at http://localhost:5000
docker compose up -d
```
The first user to register becomes admin. Registration auto-closes after that (re-enable from Settings → Users).
Open `http://localhost:5000`. The first account registered becomes the admin; after that, registration stays closed until an admin opens it in Settings or sends an invite.
### Production (`docker-compose.prod.yml`)
Update with `dockercompose pull && docker compose up -d`. The key and the data stay in their volumes.
The production compose file adds:
- Docker Secrets for `SECRET_KEY_FILE` and `DATABASE_URL_FILE`
- Network isolation (internal bridge for DB + Ollama; only the app is externally accessible)
docker compose -f docker-compose.dev.yml up --build
```
## Production Deployment
### Docker Swarm
### Reverse Proxy (Required)
[`docker-compose.prod.yml`](../docker-compose.prod.yml) is a stack file: the app and database on a private overlay network, trusting one proxy (`TRUSTED_PROXY_HOPS: "1"`). A Swarm can move the app to another node, away from a node-local `/data` volume, so the stack sets its own key: provide `DB_PASSWORD` and `SECRET_KEY` in the environment (or mount a Docker secret and set `SECRET_KEY_FILE`):
Fabled Assistant does **not** handle SSL/TLS. Run it behind a reverse proxy:
- **Nginx**, **Traefik**, or **Caddy** in front of the app container
- Terminate TLS at the proxy; forward to port 5000
- **Do not expose port 5000 directly to the internet**
- Rate-limit auth endpoints: ≤ 5 req/min per IP on `/api/auth/login` and `/api/auth/register`
The database user, database and volume default to `grimoire`. An install created under the name Scribe keeps its data by setting `POSTGRES_USER`, `POSTGRES_DB` and `PGDATA_VOLUME` to the names it was created with.
The shipped files leave out healthchecks and restart policies on purpose. The app waits for its database on startup, and the orchestrator's defaults restart it if it ever exits.
- **App data** (`/data`): the instance's own `SECRET_KEY`, the embedding model cache and diagnostics. Losing it costs the key: Settings then asks for saved forge tokens and the backup secret again, and the model downloads again.
Back up the database with **Settings → Data → Scheduled backups** ([Backups](backups.md)). A database backup doesn't carry the key; to move sealed credentials to a new instance, copy `/data/secret_key` with it.
## Behind a reverse proxy
Grimoire doesn't terminate TLS. Put it behind Nginx, Traefik or Caddy, forward to port 5000, and keep port 5000 off the internet. Set `BASE_URL`, then set the proxy count to the number of proxies a request passes through: one proxy is `1`, the default; Cloudflare's proxied DNS in front of your own proxy is `2`.
**Settings → Config → Reverse proxy** sets the count and checks it against your own request. It shows the address Grimoire reads you as, and what each count from 0 to 3 would read. Pick the count that shows your own address. When a request comes through Cloudflare, the section names the count that reads the address Cloudflare reports, with a button that sets it. `TRUSTED_PROXY_HOPS` in the environment is only the starting value: a count saved in Settings wins, and **Use the environment's** goes back to it.
A count that is too low makes every visitor read as the proxy, so they share one sign-in rate limit and a few bad sign-ins block everyone. The "too many attempts" message says so and points here. The block on sign-ins lifts after a minute; an admin who is already signed in can fix the count at once.
If using `TRUST_PROXY_HEADERS=true`, ensure your proxy strips any client-supplied `X-Forwarded-For` headers before adding its own.
Each proxy appends its entry to `X-Forwarded-For` and `X-Forwarded-Proto`, so Grimoire reads the entry that many in from the right: the one your own proxy wrote. Whatever a client puts to the left of it is ignored. A header with fewer entries than that (a LAN road with one proxy beside a Cloudflare road with two) is read at its leftmost entry, the one the first proxy wrote.
### Security Checklist
When that entry says `https`, the session cookie is marked `Secure` and the response carries `Strict-Transport-Security: max-age=31536000`. Over plain HTTP neither is sent, and Grimoire never redirects to HTTPS, so a LAN install keeps working.
- **`SECURE_COOKIES=true`** — must be set when running behind TLS.
- **`BASE_URL`** — set to your public URL; required for OIDC redirects and email links.
- **Registration** — auto-closes after the first user (admin). Re-enable from Settings → Users or send invite links.
- **Session invalidation** — changing or resetting a password bumps `session_version`, evicting all other active sessions. Button also available in Settings → Account.
- **Keep Ollama on an internal network** — both compose files keep Ollama off the host network. Never expose the Ollama port publicly.
- **Default SECRET_KEY warning** — the app logs a `WARNING` on startup if the default dev key is in use.
A hop count higher than the real number of proxies lets a client choose its own address, so count them. With `0`, the headers are ignored and the socket peer is the client: use it when clients reach port 5000 directly.
## Model Management
## Security checklist
Models are managed through Settings → General → Model Management in the web UI. You can:
- Pull new models by name (streams progress via SSE)
- View installed models with size and loaded/unloaded state
- Delete unused models
The app auto-warms user-preferred models on startup (only models already installed — never auto-pulls). The embedding model (`nomic-embed-text`) is auto-pulled on startup if missing.
- **`llama3.1:8b`** — reliable baseline, widely tested with tools
- **Keep the key.** Forge tokens and the backup destination's S3 secret are sealed under a key derived from `SECRET_KEY`, so a copy of the database alone can't read them. Unset, the instance makes a random key and keeps it in `/data/secret_key`, readable by its owner only; keep that volume. If the key changes (say, a restore onto an instance with a different key), Settings marks those credentials and asks for them again; nothing else breaks. The app warns at startup when `SECRET_KEY` is a public placeholder or `/data` is not a mounted volume, and refuses to start with a placeholder key (`dev-secret-change-me`, `change-me-in-production`).
- **The proxy count** (Settings → Config → Reverse proxy) matches the number of proxies in front of Grimoire, so rate limits and the audit log see the real client and the cookie is `Secure` over HTTPS. The section shows whether it does.
- **Registration** is open only for the first account. Invite people, or open it briefly from Settings.
- **Signing everyone else out**: changing a password ends that user's other sessions; Settings → Account has a button for it too.
- **API keys** are per user, shown once, and can be read-only. Revoke any you no longer use under Settings → MCP Access.
**Critical rule:** Action button colors are universal across all apps. A Save button in Scribe and a Save button in Minstrel look identical. Per-app accents do not appear on buttons.
**Critical rule:** Action button colors are universal across all apps. A Save button in Grimoire and a Save button in Minstrel look identical. Per-app accents do not appear on buttons.
### Semantic colors
@@ -71,7 +71,7 @@ Parchment is intentionally not pure white — it's slightly warm to feel like ag
@@ -80,7 +80,7 @@ Parchment is intentionally not pure white — it's slightly warm to feel like ag
**Accent usage rules:**
- The accent appears on the app's wordmark and icon.
- The accent indicates active/current state in nav (the selected page, the active tab).
- The accent is the cursor color and text-selection color in long-form surfaces (Scribe notes, Forge story drafts).
- The accent is the cursor color and text-selection color in long-form surfaces (Grimoire notes, Forge story drafts).
- The accent does NOT appear on primary or secondary action buttons.
- The accent does NOT appear in body text or chrome.
- One accent per app. Don't mix accents within a single app.
@@ -101,6 +101,8 @@ All text-on-surface combinations meet WCAG AA at minimum. Parchment on Obsidian
| Inter | Body, UI, labels | Google Fonts |
| JetBrains Mono | Code, terminal output, monospaced data | Google Fonts |
All three are served by the app itself, from `frontend/src/assets/fonts/`, with every subset Google Fonts publishes for them. A page never asks a font host for anything, so an instance with no internet still draws in its own type. They are under the SIL Open Font License; the licences sit beside the files.
### Why this pairing
Fraunces is a contemporary serif with personality — it has the warmth and authority of a book serif without feeling like costume. It signals "this is considered" without signaling "this is a fantasy product." Inter is the workhorse — neutral, ubiquitous, designed for screens, doesn't compete with the serif. JetBrains Mono is the natural choice for any developer-adjacent product and supports ligatures.
@@ -124,7 +126,7 @@ Fraunces is a contemporary serif with personality — it has the warmth and auth
- **Sentence case everywhere.** Never Title Case for headings, never ALL CAPS except for the Tiny micro-label style.
- **Two weights only:** 400 regular and 500 medium. Never 600 or 700 — they read heavy in dark mode.
- **Fraunces only at 18px and above.** Below that it loses too much detail and feels fragile. For h3 and below, use Inter.
- **Line height** 1.5 for body, 1.3 for headings, 1.7 for long-form reading surfaces (Scribe notes, Forge drafts).
- **Line height** 1.5 for body, 1.3 for headings, 1.7 for long-form reading surfaces (Grimoire notes, Forge drafts).
- **Letter-spacing** at default for everything except the Tiny micro-label, which gets `0.08em` letter-spacing and uppercase styling.
---
@@ -173,7 +175,7 @@ Padding: `8px 16px` for default, `6px 12px` for compact, `10px 20px` for promine
Used for tags, hashtags, code spans, status badges. Background is the accent color at ~15% opacity, text is the accent at full strength. Border-radius 4px, padding `2px 8px`, Inter 11px/500.
In Scribe specifically, hashtags and tags use the dusty violet accent. In Minstrel, they'd use forest teal. The pattern is shared; the color follows the app.
In Grimoire specifically, hashtags and tags use the dusty violet accent. In Minstrel, they'd use forest teal. The pattern is shared; the color follows the app.
### Cards
@@ -272,9 +274,9 @@ Use **Lucide** (https://lucide.dev) as the base icon set — it matches this sty
## Per-app application
### Fabled Scribe (#5B4A8A — dusty violet)
### Fabled Grimoire (#5B4A8A — dusty violet)
A second-brain notes and task management tool. The accent appears in: the wordmark, hashtags and tag pills, the active nav item, text selection color, and the cursor in the editor. Notes are presented on Iron-surfaced cards with generous reading line-height. The hashtag system uses Scribe's accent for visual continuity.
A second-brain notes and task management tool. The accent appears in: the wordmark, hashtags and tag pills, the active nav item, text selection color, and the cursor in the editor. Notes are presented on Iron-surfaced cards with generous reading line-height. The hashtag system uses Grimoire's accent for visual continuity.
### Minstrel (#4A6B5C — forest teal)
@@ -332,7 +334,7 @@ Home server management. The accent appears on: the wordmark, healthy/online stat
}
/* Per-app accent — set ONE of these on the root for each app */
[data-app="scribe"]{--fs-accent:#5B4A8A;}
[data-app="grimoire"]{--fs-accent:#5B4A8A;}
[data-app="minstrel"]{--fs-accent:#4A6B5C;}
[data-app="forge"]{--fs-accent:#8B5A2B;}
[data-app="roundtable"]{--fs-accent:#4A5D7E;}
@@ -354,220 +356,13 @@ If using Tailwind, extend the theme with these tokens rather than relying on def
---
*Last updated: April 25, 2026. Iterate as the family of apps grows.*
# Grimoire
---
The design system Grimoire renders with is stored in Grimoire itself: **Grimoire**, which inherits **FabledSword**, edited at `/design-systems`. `frontend/src/assets/theme.css` is generated from it, and CI rejects a `var()` the sheet doesn't define. Where this page and the stored system differ, the stored system is right. Grimoire applies the baseline above with these choices:
# Scribe-specific decisions in progress
> This section tracks decisions made while adapting the FabledSword baseline above for Scribe specifically. Items here are *in progress* — once they feel solid, they get folded into the main body of the document (either as Scribe-specific extensions in the per-app section, or as updates to the universal rules where Scribe's needs reveal a gap in the baseline).
*Iteration started: 2026-04-26. Foundation pass shipped 2026-04-27 in `7a9a8b7` (palette, fonts, light mode, action tokens, hardcoded indigo cleanup, warm-gold deprecation). Surface phase shipped 2026-04-27 across `93a3beb` → `3c1ec40` (Lucide migration, Hybrid-rule button reclassification per surface, long-form line-height, two-weights-only). The system is now applied end-to-end; this section will fold into the main body once the result has had time to settle in real use.*
## Decisions made so far
### Accent footprint — Hybrid rule (not Strict)
The doc baseline says the per-app accent only appears on wordmark, active nav, cursor, and text selection — never on action buttons. Scribe currently uses indigo on essentially every interactive surface (CTAs, scrollbars, glows, borders, focus rings). Hard-cutting to the doc baseline would lose too much identity in one swing.
**Hybrid rule:** the accent reserves a slightly larger footprint than the doc baseline, but still much smaller than today.
- **Accent (dusty violet) lives on:** wordmark; active nav; cursor and text selection in editor surfaces; tags/pills/wikilinks; in-progress task badge; focus rings; **brand-moment CTAs** — chat Send, "Create note" empty-state CTA, journal Send, "Start journaling" empty-state.
- **Moss (sage-green primary) lives on:** Save / Submit / Confirm in forms and modals; generic affirmative actions where the button just means "do this thing" with no brand pretense.
- **Bronze (secondary):** Cancel-but-not-destructive, alternative paths.
- **Oxblood (destructive):** Delete / Remove (paired with an icon).
- **Pewter ghost:** tertiary actions, "later", "skip", "see also".
**Rule of thumb:** if the user is engaging with a *Scribe-feature moment* (sending a chat, opening a fresh note, jumping into the journal), accent. If they're just *operating the software* (saving an edit, confirming a dialog), Moss.
The doc is dark-only. Scribe today supports both light and dark, and we keep both. The light mode is derived to *match* the dark mode aesthetic rather than defaulting to system white-and-ink.
- Page background: in the `#F5F1E8` warm cream family (specific values TBD)
- Cards: near-white but slightly tinted
- Text: deep ink `#14171A` (mirroring Obsidian)
- Accent: same dusty violet `#5B4A8A` (works on both themes)
The metaphor stays consistent across themes: ink on aged paper (light) ↔ parchment text on graphite (dark). Light mode is *not* the system standard look.
**Known downside:** warm parchment backgrounds can fight with embedded color content. Mitigation: code blocks get a slight cool wash in light mode specifically, to keep syntax highlighting readable.
### Status and priority palette — extend the doc's semantic set
The doc's semantic colors (Success / Warning / Error / Info / Destructive) are leaner than what Scribe needs for task management. Rather than running a parallel palette, Scribe extends the doc by mapping its status/priority tokens onto doc primitives where they fit and defining new app-level tokens for the rest.
| `priority-none` | Vellum/Ash | doc | No signal |
The priority row reads as a clean cool→warm gradient (slate blue → golden brown → terracotta), which matches the semantic loudness — coherence the current ad-hoc palette doesn't have.
**Other functional tokens:**
| Token | Color | Logic |
|---|---|---|
| `wikilink` | dusty violet | Editorial brand moment per Hybrid |
| `overdue` | Error `#C04A1F` | Same as priority-high — overdue IS a priority signal |
| `toast-success` | Moss | doc semantic |
| `toast-error` | Error | doc semantic |
| `toast-info` | Info | doc semantic |
| `tag-bg` / `tag-text` | accent at 15% / accent | Per doc pill recipe |
Each token gets a `*-bg` companion at low alpha (matching the existing pattern in `theme.css`).
**Removed:** the warm gold accent (`--color-accent-warm: #b8860b`). Its two jobs split:
- Dates and timestamps (knowledge cards, event details, chat) → use `text-secondary` instead. Dates are metadata, not a brand surface; muted is the correct register.
- Paused project status → use the new `status-paused` (Warning `#8B6F1E`) row above. Same golden-brown family, semantically aligned.
### Typography — adopt the doc's stack and scale
Adopt the doc's type stack and scale verbatim, with one deferred verification (long-form line-height in practice).
- **Body font: Inter.** Replaces Scribe's current system-stack body font. Doc-defined; no Scribe-specific divergence.
- **Type scale:** as in the doc table — Display 40 / H1 32 / H2 24 / H3 18 / Body 15 / Body small 13 / Label 12 / Code 13 / Tiny 11.
- **Two weights only:** 400 regular, 500 medium. No 600/700 (reads heavy in dark mode and against the muted palette).
- **Family rules:** Fraunces at 18px+ only (Display, H1, H2). H3 and below = Inter. Code = JetBrains Mono.
**Code-block exception:** in light mode specifically, code blocks use a slight cool wash (e.g. `#EBEDF0`) instead of the warm inset bg, so syntax highlighting reads cleanly. This is the mitigation for the "warm bg fights colored content" downside.
The accent (`#5B4A8A` dusty violet), Moss, Bronze, Oxblood, and the semantic color set are **identical across themes** — only the surface and text palettes flip.
### Chat-bubble codification — keep the Illuminated Transcript pattern
The existing chat-bubble pattern (informally called "Illuminated Transcript") gets written into the design system as a documented chat component. Other apps in the family that add a chat surface inherit the pattern; Scribe's existing implementation continues to work with only color shifts.
**User bubble (whisper):**
- Background: transparent
- Border: 0.5px Pewter (was: indigo-tinted)
- Text color: secondary (Vellum dark / `#5A5852` light)
- Background: card surface (Iron dark / `#FBF8F0` light)
- Border: none on top/right/bottom; **2px solid accent (dusty violet) on left edge only**
- Box-shadow: accent-tinted glow + standard depth shadow (formula: `0 4px 28px rgba(<accent>, 0.14), 0 2px 8px rgba(0,0,0,0.4)` in dark; lower alphas in light)
- Text color: primary (Parchment dark / Obsidian-inverted light)
- Left-aligned, rounded except bottom-left
The 2px-accent left edge is the "illumination" — like an illuminated capital in a manuscript. The shadow is the lift. Together they make the assistant bubble read as the *primary* voice, while the user bubble is the *margin note*.
**Inline tool-call cards (`ToolCallCard`)** rendered inside an assistant bubble do NOT get their own border (per the border philosophy — the bubble already contains them). They use a slight surface tint to differentiate.
### Iconography — adopt Lucide, enforce a scale
Scribe currently hand-inlines SVG paths in 16+ Vue files, with 5 different stroke weights and 8+ different sizes. The visual style is already outline + rounded caps + `currentColor` stroke (matches the doc's intent), but there's no shared source and no scale discipline.
**Migration policy:**
1.**Install `lucide-vue-next`** as the icon source. Replace hand-inlined SVGs with imported components. Single source of truth.
2.**Strict size scale: 16px and 24px only.** Today's mix of 12/13/14/15/17/18/20 collapses to those two. 16 for inline-with-text and small affordances; 24 for nav and primary actions.
3.**Stroke weight per the doc: 1.5 at 24px, 1 at 16px.** Lighter than the current default of 2 — reads more refined, matches the muted palette philosophy. Overrides Lucide's default.
4.**Outline by default; filled only for active/selected state.** Introduces a new affordance Scribe doesn't currently use — bookmark/pin/star icons can switch outline → filled to indicate active state. Reserve filled style strictly for this.
5.**No emoji in chrome.** Replace the 3 files' emoji usage in UI labels/buttons/badges/empty states with Lucide equivalents. Emoji remain fine in *user content* (note bodies, chat messages the user typed).
Work cost: ~30-60 individual icon swaps across the 16 files. Mechanical; doesn't require redesign of any component.
### Voice and tone — adopt principles, defer formal audit
The doc's voice register applies to Scribe (understated mythic — plain for functional UI, flavored for empty/error/loading states). No formal sweep of every UI string yet.
**Approach:** apply the voice opportunistically as components are touched in the polish pass — when redesigning a settings tab, an empty state, or an error toast, rewrite the copy at the same time using the doc's register and examples table as the guide. A standalone audit pass is deferred unless drift becomes visible.
### Border philosophy — structural, not decorative
The doc treats borders as *structural* (Pewter neutral hairlines that say "boundary"), not decorative (Scribe today uses indigo-tinted borders that say "branded edge"). That principle suggests removing borders in places where surface tint and spacing already communicate separation.
**Borders to remove:**
- List rows (NotesListView, TasksListView, conversation history) — surface contrast + spacing should separate rows; current border reads as "boxed-in"
- Inline `ToolCallCard` inside chat bubbles — the bubble is already a container; an extra border feels like double-wrapping
- Filter chips and search-bar pills with a background tint — background does the work
- Empty-state callouts with dashed/bordered "nothing here yet" boxes — tinted background reads cleaner
**Borders to keep (genuinely structural):**
- Standalone card containers (Notes viewer, Task viewer, the new daily prep card)
- Modal / dialog edges
- Code blocks (separates content type, not just space)
- Focus rings (accessibility)
- Major section dividers within a panel
Border weight is not load-bearing for Scribe — happy to use the doc's 0.5px hairline default; the *placement* discipline matters more than the weight.
## Open threads (next iterations)
### Foundation pass — shipped 2026-04-27 (`7a9a8b7`)
Mechanical token + font + light-mode rewrite of `frontend/src/assets/theme.css`, plus a sweep of hardcoded indigo and `--color-accent-warm` references across ~14 component files. Action tokens (`--color-action-primary` Moss, `--color-action-secondary` Bronze, `--color-action-destructive` Oxblood, `--color-action-ghost-border` Pewter) are defined but not yet applied — buttons still flow through `--color-primary` and read as dusty-violet gradients in the meantime, by design. Spec lives at `docs/superpowers/specs/2026-04-27-design-system-polish-foundation-design.md` (gitignored, local-only).
Bundled as Hybrid (option C from the brainstorm): Lucide cross-cutting first, then surface-by-surface for the judgment work. Spec lives at `docs/superpowers/specs/2026-04-27-design-system-polish-surface-design.md` (gitignored, local-only). Seven PRs landed on `dev`:
| PR | Commit | Surface | Notes |
|---|---|---|---|
| 1 | `93a3beb` | Lucide cross-cutting | 60 hand-inlined SVGs across 15 files → `lucide-vue-next`. Every chrome icon at 16 or 24. Emoji-as-icons (`✕`, `✓`, `🎤`, `📎`, `↻`, `↑`, `×`, `☀`/`☾`) swept across the chrome. AppLogo wordmark and the GraphView D3 mount kept as the legitimate exceptions. |
**Cross-cutting changes folded in as the work touched files:**
- Long-form 1.7 line-height on `.prose` (PR 4) — applies to Note viewer, Task viewer, chat assistant bubbles, anywhere markdown renders into a reading surface.
- Two-weights-only (400 + 500) — every `font-weight: 600` and `700` snapped to `500` across all surface PRs.
- Hardcoded `--color-danger` in destructive button contexts → `--color-action-destructive` (Oxblood). `--color-danger` (Error terracotta) preserved for validation/error messages, per the doc's distinction between Error and Destructive.
- Adjacent `×` / `✕` / unicode-arrow emoji swept opportunistically as files were touched (PRs 5, 7).
### Out of scope — deferred indefinitely
Items deliberately not addressed in this round; revisit when a real need surfaces:
- Lucide stroke-weight overrides (doc spec: 1.5 at 24, 1 at 16; current: Lucide default 2). Touched components if they read too heavy in practice.
- Filled-as-active icon state — no current affordance uses it; introduce when bookmark/pin/star toggles are added.
- Token rename to `--fs-*` namespace — Scribe is the only FabledSword app sharing this codebase.
- FabledSword lockup placement — waiting on the actual heraldic mark to be drawn.
- Standalone voice/tone audit across every UI string — opportunistic-only; full sweep deferred unless drift becomes visible.
- A handful of editor utility buttons (`.btn-suggest-tags`, `.btn-link-all`, AI assist generate/proofread/accept/reject set, etc.) — currently ghost-styled and visually compliant; revisited only if they read off in practice.
### Open threads
*New threads will accumulate here as gaps surface in real use.*
- **Accent footprint.** Dusty violet marks Grimoire's own moments: the wordmark, active navigation, the editor's cursor and selection, tags and wikilinks, in-progress status, focus rings, and the call to action that starts something new (a first note, say). Operating the software uses the action colours: Moss to save or confirm, Bronze for alternatives, Oxblood with an icon to delete, Pewter ghost for tertiary actions.
- **Light and dark.** Both are supported. Light mode is warm parchment (`#F5F1E8` page, `#FBF8F0` cards, `#14171A` text) rather than white, with a cool wash on code blocks so syntax colours stay readable. The accent, action and semantic colours are the same in both.
- **Status and priority.** Status: todo Pewter, in progress the accent, done Moss, cancelled Ash, paused Warning. Priority runs cool to warm: low Info, medium Warning, high Error. Overdue uses Error. Each has a low-alpha `-bg` companion.
- **Type.** Inter for UI and body, Fraunces at 18px and up (display, H1, H2), JetBrains Mono for code. Two weights only, 400 and 500. Line height 1.5 for body, 1.3 for headings, 1.7 on reading surfaces such as a note.
- **Icons.** Lucide, at 16px inline and 24px for navigation and primary actions; no emoji in the interface.
- **Borders are structural.** Cards, dialogs, code blocks, focus rings and major dividers get borders; list rows, chips and empty states are separated by surface tint and spacing instead.
All development is Docker-based. Do not install Python or Node dependencies locally.
## Running it
```bash
# Start the full stack (app + PostgreSQL + Ollama)
docker compose up --build
docker compose -f docker-compose.dev.yml up --build # app + PostgreSQL at http://localhost:5000
docker compose -f docker-compose.dev.yml up --build app # rebuild after a change
docker compose -f docker-compose.dev.yml down -v # stop and wipe the database
```
# Rebuild after backend changes (frontend changes require rebuild too)
docker compose up --build app
The image builds the frontend, so a frontend change needs a rebuild too. For quicker frontend work, run Vite against the running app; it proxies `/api` to port 5000:
# Reset everything (wipes database)
docker compose down -v && docker compose up --build
```bash
cd frontend && npm ci && npm run dev # http://localhost:5173
```
# Run checks (lint, format, typecheck, tests)
make check
## Checks
# Individual checks
CI runs these on every push; locally they need `uv sync --extra dev` and `cd frontend && npm ci`.
```bash
make lint # ruff check src/
make fmt # ruff format src/
make typecheck # vue-tsc --noEmit
make test# pytest tests/
make check # all three
```
## Frontend Hot Reload
Unit tests run without a database. Tests marked `integration` need a real PostgreSQL and run in CI's integration job, against a schema built by the real migrations.
The Docker setup does not include Vite's hot-reload dev server. After frontend changes, rebuild the image. For faster iteration during active frontend work, you can run Vite locally:
## Database migrations
The container runs `alembic upgrade head` on every start, after waiting for the database to accept connections.
```bash
cd frontend
npm install
npm run dev # Vite dev server at http://localhost:5173
Point the Vite dev server at the backend by setting `VITE_API_BASE_URL=http://localhost:5000` (or configure `vite.config.ts` proxy).
- Number revisions sequentially (`0123_...`). Revision ids must fit in 32 characters, the width of `alembic_version.version_num`.
- Write both `upgrade()` and `downgrade()`.
- Adding a value to a `CHECK`-constrained column means dropping and re-adding the constraint in the same migration.
- Keep migrations to schema changes. Filling a derived table from a large one belongs in a background backfill after startup, where its cost doesn't hold up the boot.
- A migration that depends on application code (a key derivation, a serialiser) copies what it needs rather than importing it, so a later edit can't change what an old migration does.
## Database Migrations
## The plugin
Alembic migrations run automatically on container startup (`alembic upgrade head` in the `CMD`).
`plugin/` ships straight from git: installs fetch it through `.claude-plugin/marketplace.json`, and there is no build step. After changing anything under `plugin/` or `.claude-plugin/`, run `make mint-plugin` before committing. Claude Code only refreshes an installed plugin when its version changes, and CI fails a plugin change that didn't bump it.
The hooks must run on a bare machine: POSIX shell, awk, sed and the usual coreutils, plus `git` and `curl`. They fail open, so a slow or unreachable Grimoire never blocks Claude Code.
Migration conventions:
- Use raw SQL with `IF NOT EXISTS` guards for idempotency
- Use `DO $$ BEGIN CREATE TYPE … EXCEPTION WHEN duplicate_object THEN NULL; END $$` for enum types
- Number migrations sequentially (e.g. `0027_add_something.py`)
- Always provide both `upgrade()` and `downgrade()`
## CI and releases
## CI/CD
CI runs on Gitea Actions (workflow in `.forgejo/workflows/ci.yml`) in the shared `ci-python:3.14` image ([`ci-requirements.md`](../ci-requirements.md) lists what it relies on). Six jobs: TypeScript typecheck, plugin hooks, Python lint (plus design-token and dangling-style checks), unit tests, integration tests, and the image build, which waits on all of them except the plugin job.
### Pipeline
| Push | Image tags |
|------|-----------|
| `dev` | `:dev`, `:<sha>` |
| `main` | `:latest`, `:<sha>` |
| a `v*` tag | `:latest`, `:<version>`, `:<sha>` |
`:<sha>` never moves, so it is the rollback point. Images publish to `git.fabledsword.com/bvandeusen/fabledgrimoire`, with a registry layer cache at `:cache`. The build needs the `REGISTRY_USER` and `REGISTRY_TOKEN` (a token with `write:packages`) repository secrets.
| Trigger | Jobs | Docker tags pushed |
|---------|------|--------------------|
| Push to `dev` | typecheck + lint + test → build | `:dev`, `:<sha>` |
| Tag `v*` on `main` | typecheck + lint + test → build | `:latest`, `:<version>`, `:<sha>` |
| Push to `main` | typecheck + lint + test | (no build) |
Work happens on `dev`. A release merges `dev` into `main` with a plain merge commit; a dated `vYYYY.MM.DD.HHMM` tag, matching the version the image reports, can mark it.
### Release Process
1. Work on `dev` branch — CI validates on every push
2. When ready, open a PR from `dev` → `main` in Forgejo
3. Merge the PR
4. Create a release via the Forgejo UI on `main` with a `v*` tag (e.g. `v26.03.23.1` — CalVer: `YY.MM.DD.N`)
5. The tag push triggers CI → build job pushes `:latest` + `:<version>` Docker images
6. After merging to main, sync dev back:
```bash
git checkout dev && git merge main && git push origin dev
```
### Runner
CI jobs schedule against the `python-ci` runner label and run inside the
shared `git.fabledsword.com/bvandeusen/ci-python:3.14` image (see
`ci-requirements.md` for what this project relies on from the image).
The runner deployment lives outside this repo; image bumps happen in
[CI-Runner](https://git.fabledsword.com/bvandeusen/CI-runner) via Renovate.
`infra/runner-compose.yml` + `infra/act-runner-config.yml` document the
runner-host deployment shape; the source of truth is the deployed
config on the runner host.
### Docker Registry
Images pushed to: `git.fabledsword.com/bvandeusen/scribe`
**Important:** Do NOT use `op.create_table()` or `sa.Enum()` — SQLAlchemy's event system can fire `CREATE TYPE` even with `create_type=False`, causing failures on re-run. Always use raw SQL with `IF NOT EXISTS` / `DO $$ BEGIN ... EXCEPTION WHEN duplicate_object` guards.
## Project Conventions
## Conventions
### Backend
- Services: `async with async_session() as session:` — import from `scribe.models`
- No `scribe.database` module
- Blueprint per resource: `routes/notes.py`, `routes/tasks.py`, etc.
- All business logic in `services/`; routes are thin wrappers
- Permission checks via `services/access.py` — never inline ownership checks in routes
- Services open their own session: `async with async_session() as session:` (from `grimoire.models`).
- Routes and MCP tools stay thin; the logic lives in `services/`, shared by both.
- Access checks go through `services/access.py`, never inline ownership checks.
- Every wait on something outside the process has a deadline.
### Frontend
- API calls via `frontend/src/api/client.ts` typed helpers (`apiGet`, `apiPost`, `apiPatch`, `apiDelete`)
- Pinia stores for shared state; local `ref()` for component-only state
- Composables in `composables/` for reusable behaviour (autosave, keyboard nav, tag suggestions, …)
- Views are page-level components in `views/`; reusable UI in `components/`
- API calls go through the typed helpers in `frontend/src/api/client.ts`.
- Pinia stores for shared state, `ref()` for state local to a component; composables for reusable behaviour.
- Colours, spacing and type come from the design system's tokens in `theme.css`, which is generated, so CI rejects an unknown `var()`.
### Commit Style
### Commits
```
type(scope): short description
type(scope): what the change does
Longer body if needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Why, and anything a reader tracing this change later needs.
Tests are in `tests/`. They run against a real PostgreSQL instance in CI (not mocked). Keep tests integration-style where possible — mock failures have historically masked real migration bugs.
Types: `feat`, `fix`, `refactor`, `docs`, `chore`, `test`. Comments in the code describe how it works today; the history of a change belongs in its commit message.
Scribe is a self-hosted work system-of-record for software projects, built to be
driven by Claude Code. There is **no in-app LLM** — Claude is the sole assistant,
reaching Scribe through a built-in MCP endpoint and a bundled Claude Code plugin. The
web UI is a clean surface for humans to read and edit the same data.
Grimoire is a self-hosted systemofrecord for the work you do with Claude Code. Claude records what it learned, decided and built, and that record reaches the next session, so a solution is recalled rather than reinvented. There is no language model inside Grimoire: Claude reaches it through the MCP endpoint and the Claude Code plugin, and the web UI is where people read and edit the same records.
## Notes
Write in Markdown with a live-preview editor (Tiptap/ProseMirror). Headings, bold,
italic, lists, code blocks, and task checklists render inline. A slash-command menu
(`/`) inserts common blocks.
A Markdown editor (Tiptap) with headings, lists, code blocks and checklists inline, and a `/` menu for blocks.
- **Wikilinks** — Link notes with`[[Title]]` or `[[Title|Display Text]]`. Clicking
navigates to (or auto-creates) the referenced note; the editor suggests existing
titles while you type `[[`. Backlinks appear in the note viewer sidebar.
- **Tags** — First-class `ARRAY[text]` column with autocomplete. Hierarchical tags
(`area/backend`) supported — filtering by `area` matches all `area/*` children.
- **Version history** — Every body edit snapshots a version (up to 20 per note).
Browse, diff, and restore from the editor's History panel.
- **Draft recovery** — In-progress edits persist across page loads and are restored
when you reopen a note.
- **Convert freely** — Turn a note into a task (sets `status=todo`) or back again.
- **Wikilinks**:`[[Title]]` or `[[Title|text]]`, suggested as you type; backlinks show in the viewer.
- **Tags**, with autocomplete. Hierarchical tags (`area/backend`) match their parent's filter.
- **Version history**: browse, diff and restore earlier versions of a note.
- **Draft recovery**: unsaved edits survive a reload.
## Tasks and Issues
## Tasks
Tasks carry status (`todo` →`in_progress` →`done`/`cancelled`), priority
(`none`/`low`/`medium`/`high`), due date, milestone assignment, and a parent task
(sub-tasks). Notes and tasks share one model — a task is a note with a status.
A task is a note with a status (`todo`,`in_progress`,`done`, `cancelled`), plus priority, due date, milestone and parent task.
- **Work logs** — Append timestamped progress entries (with optional duration) to a
task without rewriting its body; shown chronologically in the task view.
- **Issues** — A task whose `kind` is corrective: a problem you fixed or are fixing,
with symptom → root cause → fix in the body. An issue can link the task it arose
from and the System(s) it touches.
- **Recurring tasks** — An interval or calendar recurrence rule spawns the next
occurrence when a task is completed (a background job drains due spawns).
- **Sub-tasks** — Any task can have children via `parent_id`; the viewer shows them
inline.
- **Kinds**: `work`; `issue`, for corrective work, which can link the task it arose from; and `spike`, a time-boxed investigation whose output is an answer.
- **Work logs**: timestamped entries that record progress without rewriting the body. They are what the next session reads first.
- **Claims**: a session that logs on a task holds it, so after a context compaction the work comes back to that session, and other sessions can see who is on it.
- **Recurring tasks** spawn their next occurrence on a schedule.
## Projects and Milestones
## Projects, milestones and systems
- **Projects** — Group related notes and tasks. Title, description, goal, status
(`active`/`paused`/`completed`/`archived`), and a colour.
- **Milestones** — Ordered stages within a project. A milestone is also the home of a
**plan** — its body holds the design (Goal/Approach/Verification) and its child
tasks are the steps. Completion percentage is shown on the project page.
Milestones are searchable by meaning (`search(content_type="milestone")`), and an
agent starting a plan (`start_planning`, `create_milestone`) is handed the active
milestone that already has its title or reads as the same plan, so steps are added
there rather than to a parallel plan. The match threshold is in Settings.
- **Kanban view** — `/projects/:id` groups tasks by milestone in a column layout with
status-advance buttons on the cards.
- **Projects** carry a goal, a status and a colour, and group everything else. Entering a project gives Claude a short primer: its goal, recent plans and their next steps, open tasks, its systems and its design system.
- **Milestones are plans**: a body holding the design (goal, approach, verification) and ordered step tasks. Starting a plan that matches an active milestone hands back that milestone instead of opening a second one.
- **Systems** are a project's named areas ("auth", "retrieval"). Records are tagged with them, each has a charter saying what belongs there, and each maps to a canonical area shared across projects.
- **Duplicate gates**: creating a record that reads like an existing one returns the existing one to update instead.
## Systems
## Rules, lessons and moments
A **System** is a per-project, reusable, self-describing subsystem or area (e.g.
"auth", "billing"). Associate any note, task, or issue with a System so research,
build-work, and fixes for the same area line up and recurring problem-spots surface.
- **Rulebooks → topics → rules.** A rule in a rulebook applies in every project; a project rule applies only there. A project that departs from a global rule says so with an `overrides` relation.
- **Rules arrive by retrieval.** Nothing is preloaded: a rule reaches a session when what Claude is about to do matches its trigger (`when_to_apply`), whether that's the operator's message, a command or a file being written.
- **Moments**: a rule about *when* something is done (finishing a task, reporting back, pushing) can be mounted on a moment of work, and arrives whenever that moment happens, whatever the words look like. Settings → General shows which actions reach each moment.
- **Versions and verification**: every edit to a rule keeps the text it replaced. A rule that asserts a fact about someone else's software can say how to check it and when it stops being true, and a sweep lists rules due for a check.
- **Preferences** are rules with the force of guidance rather than obligation.
- **Lessons** record something learned the hard way, keyed to the situation where it applies, and link to the rules they are instances of.
- **Reply shapes**: the default shape of Claude's replies (completion report, plan for review, question) arrives at the moment it applies. Settings → General shows them.
## Rules and Rulebooks
## Snippets and the shape ledger
Scribe stores the operator's engineering and workflow **rules** so Claude follows them
across sessions.
- **Snippets** are recorded prior art: a reusable piece of code with when to reach for it, its signature and its canonical location in a repo. Claude searches them before building and is shown matching ones before it writes a file.
- **Forge integration**: with a git forge connected, Grimoire checks a snippet's cached body against its source and marks it current or diverged. Push webhooks keep that fresh.
- **The shape ledger**: Grimoire extracts the code shapes from a bound repo and tracks how each is accounted for: an instance of a snippet, a deliberate variant with its reason, or a one-off. Coverage shows what is still unclassified.
- **Rulebooks → topics → rules** — Rules are grouped by topic inside a rulebook.
- **Rules arrive by retrieval** — Nothing is preloaded. A rule reaches a
session when what the agent is about to do matches its trigger: a command,
a file being written, or the operator's own message. `when_to_apply` is
therefore the field that decides whether a rule is ever seen.
- **Global or project scope** — A rule in a rulebook is global: it applies in
every project. A project rule applies to that project only. Retrieval honours
the difference, so a session sees global rules plus its own project's, never
another project's. A project that departs from a global rule writes its own
and links it with an `overrides` relation.
## Family canon
## Stored Processes
Projects on the same platform (Android apps, Go services, ...) share good ideas. An idea promoted to the family's canon reaches every project on that platform, and each answers it: adopted, a variant with its reason, or owed (filed as a task). `/family` shows the ideas and every project's answers.
Reusable saved prompts (a note with `note_type=process`) — e.g. a drift-audit or a
DRY pass. The bundled plugin syncs each Process into a local Claude Code skill stub
(`/scribe:sync`) that auto-surfaces by relevance and fetches the live procedure on
demand.
## Design systems
## Search and Knowledge Injection
Tokens (colour, type, space, motion, ...) and written guidance, with inheritance: a project's design system can extend a house style and record where it departs. Grimoire generates a stylesheet from it, checks code snippets against it, and gives Claude the values to use before it writes UI. `/design-systems` edits them.
- **Semantic search** — pgvector-backed similarity search over notes and tasks
(in-process `fastembed` embeddings; no external model).
- **Proactive knowledge-injection** — the plugin's `UserPromptSubmit` hook surfaces a
short, high-confidence menu of maybe-relevant note *titles* into Claude's context
each turn; Claude pulls a full body only when it judges it relevant. Gated so it
stays quiet on most turns and never repeats within a session.
## Stored processes
## Knowledge Graph
Saved procedures, such as a drift audit or a DRY pass. The plugin syncs each one into a Claude Code skill (`/grimoire:sync`), which fetches the live procedure when it is invoked.
`/graph` renders notes, tasks, and tags as a D3 force-directed graph. Tag nodes
controls (repulsion, link distance/strength, hub pull, gravity); click a node to peek,
click a tag to filter.
## Search and context injection
- **Semantic search** over notes, tasks, rules, milestones and systems, with embeddings computed in-process (no external model service). It falls back to keyword search if the embedder is unavailable.
- **Context injection**: the plugin's hooks show Claude a short menu of possibly relevant records at the moments they matter: rules and prior art when a prompt arrives, before a command or a file write, and when a moment of work is reached. Claude opens a full record only when it judges it relevant.
- **Tuning**: each surface has its own thresholds and budget, in Settings → General. Every change is logged with its reason, and retrieval telemetry shows what was surfaced and what was opened.
## Claude via MCP and the plugin
The whole store is reachable by Claude through a built-in **MCP endpoint at `/mcp`**
(Bearer-auth with an API key). The **Scribe Claude Code plugin** (shipped in this
repo) wires it up:
The whole store is reachable through the MCP endpoint at `/mcp`, authenticated with an API key. The Claude Code plugin in this repo adds:
-a `SessionStart` hook that injects active-project
context so Scribe surfaces without being asked (fail-open if Scribe is unreachable);
brainstorming — that route their output into Scribe;
- your saved Processes auto-surfaced as skills.
-hooks for session start, each prompt, before and after tool calls, the end of a turn, compaction and session end, all failing open if Grimoire is unreachable;
- a project fence: a session belongs to the project its starting directory is bound to. Toward any other project it can read anything and file an issue. Editing that project's files, committing or pushing in its repo, or changing its records first asks you, in every permission mode;
-skills for the working practices: using Grimoire, writingplans, systematicdebugging, verification, brainstorming, reporting back, reusing code, shape accounting and family canon;
- your stored processes as skills.
See [API Keys & MCP](api-keys-and-mcp.md).
## Sharing and Collaboration
## The web UI
- **Share** — Share any project, note, or task with users or groups at
`viewer`/`editor`/`admin` levels from the viewer/project toolbar.
- **Groups** — Admins create platform-wide groups and assign `member`/`owner` roles;
share a resource with a group in one action.
- **Shared with me** — `/shared` lists incoming shares with permission badges.
- **Notifications** — An in-app bell (unread count, polled) fires when a project or
note is shared with you or you're added to a group.
- **Dashboard**: what to work on next. **Knowledge** browses everything.
- **Graph** (`/graph`): notes, tasks and tags as a force-directed graph, with physics controls and filtering by tag.
- **Trash**: deleted records wait here until retention purges them, and can be restored.
- **Installable** as a desktop or mobile app.
Every read and mutation is scoped by owner + direct shares + group shares.
## Sharing
## Data Export and Backup
Share a project, note or task with users or groups, as viewer, editor or admin. Admins manage groups. `/shared` lists what has been shared with you, and an in-app bell notifies you of new shares. Every read and write is checked against ownership and shares.
- **Personal export** — download all your notes/tasks as a Markdown ZIP (YAML
frontmatter) or a JSON array.
- **Admin backup** — full application backup/restore (format v4) with ID remapping on
restore for cross-instance migration.
## Export and backup
## Progressive Web App
- **Personal export**: your notes and tasks as a Markdown ZIP (with YAML frontmatter) or JSON.
- **Admin backup and restore**: a restore rolls the whole instance back to one of its own backups, after saving the current state, or brings a new instance back from any backup.
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. Restoring is in Settings, or `python -m grimoire.restore`. See [Backups](backups.md).
Installable as a desktop or mobile app; a service worker caches the shell.
that links to a matching local account. Invite links, a registration toggle, password
reset, and session invalidation are included. See [SSO / OAuth](sso-oauth.md).
Email and password, plus optional OIDC single sign-on (Authentik, Keycloak, ...) that links to a matching local account. Invite links, password reset and signing out other sessions are included. See [SSO / OAuth](sso-oauth.md).
Add the following environment variables to the `app` service in `docker-compose.yml`:
@@ -64,7 +64,7 @@ docker compose up --build -d
## 4. Account Linking
When a user logs in via OAuth for the first time, Fabled checks in this order:
When a user logs in via OAuth for the first time, Grimoire checks in this order:
1. **Existing OAuth sub** — returns that user immediately.
2. **Matching email** — if a local account already exists with the same email, the OAuth identity is linked to it automatically. The user retains all their notes and tasks.
@@ -92,4 +92,4 @@ The backend will reject any `POST /api/auth/login` or `POST /api/auth/register`
The OIDC discovery endpoint (`<issuer>/.well-known/openid-configuration`) must be publicly reachable from the Fabled container (server-to-server call at login time).
The OIDC discovery endpoint (`<issuer>/.well-known/openid-configuration`) must be publicly reachable from the Grimoire container (server-to-server call at login time).
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.