Compare commits

...
50 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 6be44013bb fix(6872): a verdict or write-path stamp on a vanished shape revives its row instead of inserting a duplicate
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m46s
CI & Build / integration (push) Successful in 1m42s
CI & Build / Build & push image (push) Successful in 18s
classify_shapes(repo=...) and the write-path hook looked only at live rows,
so a shape whose row had vanished got a second provisional row under the
same (project, repo, path, symbol, kind). The unique constraint refused it,
and the whole classify batch was lost as a tool error.

provisional_row() now finds the row under that identity, vanished or not.
A vanished row is revived with a reappeared event and keeps its judgment.
A new row is inserted inside a savepoint, so a concurrent insert of the
same identity falls back to that row instead of failing the transaction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 12:02:13 -05:00
bvandeusenandClaude Opus 5.5 219bfeac42 docs(6868): the proxy count is set in Settings, with the env var as the starting value
Configuration, the README env table and the features page say Settings
-> Config -> Reverse proxy is where the count is checked and set,
that a saved count wins over TRUSTED_PROXY_HOPS, that Cloudflare in
front of your own proxy is 2, and what a too-low count does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 10:03:22 -05:00
bvandeusenandClaude Opus 5.5 d2267b615f feat(6867): Settings → Config has a Reverse proxy section that shows which count is right
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m46s
CI & Build / integration (push) Successful in 2m8s
CI & Build / Build & push image (push) Successful in 33s
The section opens with a verdict on the admin's own request: the
address Grimoire reads them as, or, through Cloudflare, the count to set
with a button that sets it. Below it are the count with where it came
from, a table of what counts 0-3 would read, and the raw forwarding
chain. A saved count can be cleared back to the environment's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 09:59:05 -05:00
bvandeusenandClaude Opus 5.5 753a6ccb31 feat(6866): the proxy hop count is an admin setting, and the server shows what each count reads
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / integration (push) Successful in 2m1s
CI & Build / Build & push image (push) Successful in 20s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / Python tests (push) Successful in 1m47s
An admin can save how many reverse proxies stand in front of Grimoire.
A saved count wins over TRUSTED_PROXY_HOPS, which stays the starting
value. Each worker loads it at startup (under a deadline) and re-reads
it every 30s; the worker that saves applies it at once.

GET/PUT /api/admin/proxy returns the count, where it came from, and a
readout of the admin's own request: the socket peer, the forwarding
chains, what counts 0-3 would read, and the count that reads
Cloudflare's visitor when Cloudflare's headers are present. Saves are
audit-logged. A "too many attempts" reply now says where to look when
everyone is being blocked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 09:55:42 -05:00
bvandeusenandClaude Opus 5.5 49b5ceee41 docs(6856): the scope section says the cross-project rule in fewer lines, so using-grimoire fits its 350-line budget
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 17s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m52s
CI & Build / integration (push) Successful in 2m26s
CI & Build / Build & push image (push) Successful in 17s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 22:01:22 -05:00
bvandeusenandClaude Opus 5.5 8f8263b347 docs(6856): the plugin's guidance and the docs say what a session may do in another project
CI & Build / TypeScript typecheck (push) Successful in 31s
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / Python tests (push) Failing after 1m29s
CI & Build / integration (push) Successful in 2m21s
CI & Build / Build & push image (push) Skipped
using-grimoire, which owns cross-project conduct, now says the one change
to make toward another project is filing an issue with how it was found
and the evidence, and that the plugin asks before anything more. The
features page describes the fence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:52:09 -05:00
bvandeusenandClaude Opus 5.5 ddcac96542 fix(6855): the fence keeps its files where no compaction sweeps them, and builds id lists under mawk
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / Python tests (push) Successful in 1m52s
CI & Build / integration (push) Successful in 2m21s
CI & Build / Build & push image (push) Successful in 22s
The project a session entered now lives in grimoire-fence, which the
compaction clear never visits, and session end removes it. Arrays of ids
in a tool's input were sent as [,"3","4"] under mawk, which creates an
array element as soon as it is assigned to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:51:34 -05:00
bvandeusenandClaude Opus 5.5 805fda5cb1 feat(6855): the plugin asks before a session writes to a project other than its own
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Failing after 1m27s
CI & Build / Build & push image (push) Skipped
CI & Build / integration (push) Successful in 2m1s
plugin/hooks/grimoire_project_fence.sh runs as PreToolUse on editor writes,
Bash and Grimoire's tools. A session belongs to the project its starting
directory resolves to (CLAUDE_PROJECT_DIR: a .grimoire marker, else the git
remote's binding). A session that starts outside any project belongs to the
first one it enters, which PostToolUse on enter_project records.

These acts stop on permissionDecision "ask", with the server's reason naming
the other project:
- an edit in another project's repo or a worktree of it;
- a git commit, push, merge, rebase, pull, cherry-pick, revert, tag, am, reset
  or worktree add there, found through git -C, a cd before it, or the shell's
  current directory;
- a Grimoire write that lands in another project.

A live probe showed "ask" holds in bypassPermissions. Filing an issue and
every read go ahead. An edit or git command in the session's own repo, and any
tool named like a read, never calls the server. Grimoire writes send only
their short arguments, so the URL stays small.

When the instance doesn't answer, the act goes ahead and the hook says once
per outage that it couldn't check. check_plugin.py now holds one outage line
per hook and has a smoke event for the fence.

GET /api/plugin/write-scope takes fallback_project_id for the entered project.

tests/test_project_fence.py drives the hook against real git repos and a
stand-in server. tests/test_write_scope.py holds the hook's
FENCE_READ_PREFIXES equal to services/write_scope.READ_PREFIXES. The plugin
README and PACKAGING describe the fence, and the plugin is minted to
2026.11.05.0245.

Milestone 583, step 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:45:37 -05:00
bvandeusenandClaude Opus 5.5 16b1d68ebd fix(6854): the locating arguments that only point are declared as references, and the plugin version is minted
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / Python tests (push) Successful in 1m43s
CI & Build / integration (push) Successful in 2m2s
CI & Build / Build & push image (push) Successful in 22s
The guard caught seven write-tool arguments with a locating name that are not
where the write lands:
- snippet_id on classify_shapes_by_rule and confirm_shape_proposals names the
  canon, not the rows written;
- rule_outcome, rule_misfired and judge_lesson_link record judgments.

REFERENCES lists each one with its reason, and the guard reads it.
READ_PREFIXES names the tool prefixes the fence answers without the server,
and a test holds every write tool to a name outside them.

plugin.json is minted to 2026.11.05.0242 for the PACKAGING.md row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:42:14 -05:00
bvandeusenandClaude Opus 5.5 55f0f96016 feat(6854): the server says which project a pending act would change, and whether this session may do it
CI & Build / Python lint (push) Successful in 2s
CI & Build / Python tests (push) Failing after 1m23s
CI & Build / Build & push image (push) Skipped
CI & Build / Plugin hooks (push) Failing after 11s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / integration (push) Successful in 1m53s
GET /api/plugin/write-scope answers the plugin's fence. The session's project
comes from repo / project_id, as every plugin endpoint reads it. The act is
either:
- a directory (target_repo / target_project_id), for a file edit or a git
  command; or
- an MCP call (tool, plus input: the call's short arguments).

It returns allowed, the other projects the act would change, and the reason
the operator is asked. It is a GET so a read-scoped key still fences file
edits and git.

services/write_scope.WRITE_TARGETS lists every write tool with the arguments
that locate its write: a project id, or a task, note, milestone, System, rule,
relation, trash batch, repo or family decision, each traced to its project. A
write that lands in no project (install settings, user-level records, the
judgments a session is asked for anywhere) has an empty entry. The one
cross-project write allowed is create_task(kind="issue") as a new, unplanned,
open issue. A project the caller cannot read is never named.

tests/test_write_scope.py:
- requires an entry for every write tool;
- requires every declared argument to exist on the tool;
- refuses a locating argument (project_id, task_id, milestone_id and the
  rest) that a write tool takes without declaring it.

The integration test traces each locator from real rows.

Milestone 583, step 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 21:39:03 -05:00
bvandeusenandClaude Opus 5.5 373f5bd454 feat(6839): the decision log and the knowledge feed load as you scroll, through one ScrollSentinel
CI & Build / Python lint (push) Successful in 2s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / integration (push) Successful in 2m9s
CI & Build / Build & push image (push) Successful in 36s
CI & Build / Plugin hooks (push) Successful in 17s
CI & Build / TypeScript typecheck (push) Successful in 33s
ScrollSentinel sits at the foot of a server-paged list and asks for the
next page a screen before the reader reaches it. It asks once per cursor,
forgets the cursor when its element goes, and shows Try again on a failed
page instead of retrying in a loop.

FamilyView's decision log loses its "Older decisions" button. A failed
page shows in place rather than as a toast, and rows already shown are
skipped when a new decision shifts the offset.

KnowledgeView drops its inline IntersectionObserver for the component.
The cursor is the ids consumed, not the items rendered, so a batch that
returns fewer records than it asked for still moves the feed on. A failed
batch puts its ids back and offers Try again, where it used to fail
silently and stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 20:14:41 -05:00
bvandeusenandClaude Opus 5.5 846c87629f test(6849): the restore command previews without --yes and rolls back from the tar with it
CI & Build / Python lint (push) Successful in 1s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m39s
CI & Build / integration (push) Successful in 1m51s
CI & Build / Build & push image (push) Successful in 16s
Two integration tests in test_integration_backup_rollback.py drive
grimoire.restore.run on the scheduled backup's tar, written from a full
export of this instance:
- without --yes it exits 4 and nothing changes;
- with --yes it takes the safety backup, then puts back the edited and the
  deleted task and removes the new one.

The file-restore test that expected a refusal "because the database has
accounts" now names the reason it actually refuses: on a live instance, a
backup that records no instance id can't be rolled back to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 20:12:30 -05:00
bvandeusenandClaude Opus 5.5 37be3b0bff feat(6849): the command-line restore uses the same rollback, and the docs say how restore works now
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m39s
CI & Build / integration (push) Successful in 1m49s
CI & Build / Build & push image (push) Successful in 31s
python -m grimoire.restore <file> prints what the restore would change (the
same preview as Settings) and stops. With --yes it runs
backup_job.rollback_to_backup: the current state is saved first, then the
whole instance goes back to the backup. An empty instance takes any full
backup. A live one only takes a backup of its own, and otherwise exits 3 with
the reason. A v1 backup keeps the copy-in path into an empty instance.

docs/backups.md Restoring is rewritten:
- what a restore puts back and removes;
- the same-instance rule;
- the safety backup;
- what survives;
- the web and command-line ways in.

Its S3 key advice now asks for read as well as write when restoring from the
bucket. README, features, architecture and the API reference follow.

Milestone 582, step 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 20:08:31 -05:00
bvandeusenandClaude Opus 5.5 1f5e963c35 feat(6848): Settings restore rolls back, from the destination or a file, after a preview
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / Python lint (push) Successful in 1s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m42s
CI & Build / integration (push) Successful in 1m48s
CI & Build / Build & push image (push) Canceled after 22s
The web restore no longer copies a backup in beside the live data.

- GET /api/admin/restore/sources lists the backups at the backup destination,
  newest first.
- POST /api/admin/restore/preview says what a restore would do, changing
  nothing: fresh or rollback, when the backup was taken, and counts now and
  after per kind. A backup that cannot apply here gets a 409 with the reason.
- POST /api/admin/restore?confirm=rollback runs backup_job.rollback_to_backup:
  safety backup, replace, then search rebuilt in the background.

Each takes ?source=<name> (a backup at the destination) or an upload. The
destination path exists because uploads are held to the request-size limit,
and a real instance's backup is far over it. backup_s3.get_file downloads
from a bucket. backup_destination.list_backups/open_backup only accept names
the scheduled backup writes, so a request cannot reach any other file.

Settings -> Data -> Restore from Backup opens RestoreBackupDialog. The dialog:
1. Pick a backup at the destination or a file.
2. Read the preview.
3. Type "roll back".
4. Run it. The dialog then says where the replaced state was saved, and sends
   the user to sign in again.

Closes #6840. Milestone 582, step 4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 20:06:07 -05:00
bvandeusenandClaude Opus 5.5 34f8d7060b feat(6847): a rollback first backs up the state it replaces
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / integration (push) Successful in 1m49s
CI & Build / Python tests (push) Successful in 1m40s
CI & Build / Build & push image (push) Successful in 24s
backup_job.rollback_to_backup:
1. Checks the backup can apply here (backup.check_replace, which shares
   _replace_mode with replace_from_backup), refusing before anything is written.
2. Takes the backup lock, so no scheduled backup runs during the rollback.
3. Saves the current store with save_state_before_restore.
4. Replaces the store.

The safety file goes to the backup destination when backups are on and it
takes it. Otherwise it goes to DATA_DIR/pre-restore, with a note when the
destination refused. If neither takes it the rollback does not run. The
result names where the file went.

Milestone 582, step 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 20:02:26 -05:00
bvandeusenandClaude Opus 5.5 8806ab85b4 feat(6846): replace_from_backup rolls the store back to a full backup by id
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / integration (push) Successful in 2m1s
CI & Build / Build & push image (push) Successful in 20s
One transaction:
- Refuse unless the backup is a full one and, on a live instance, carries
  this instance's id. A fresh instance (no records, at most the restoring
  account) takes any full backup and adopts its id.
- Clear first: search vectors and the CSS consumer map (derived); every
  backed-up table nothing outside the backup points at, whole; the settings
  the backup carries; and the identity-table rows the backup lacks. A share
  granted by a removed user goes before the user.
- Write the backup through the same _build_* helpers as the copy-in restore,
  now _load_v2(keep_ids=...). Identity tables (users, projects, milestones,
  notes, rulebooks, topics, rules, systems, design systems, code shapes,
  family decisions) go through _place. That keeps the backup's id and
  merges, so API keys, shares and forge connections of a surviving row stay.
- Reset id sequences past the restored max, and sign every session out.

reindex_after_restore runs the four embedding backfills afterwards. A unit
guard checks that no row kept for outside references cascades from a table
replaced whole. Integration tests cover edits, deletions and new rows put
back, a surviving API key, sessions ending, fresh ids afterwards, foreign or
id-less backups refused, a user export refused, a failure part-way changing
nothing, and a fresh instance adopting the id.

Milestone 582, step 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 19:59:48 -05:00
bvandeusenandClaude Opus 5.5 6d8db0bb65 feat(6845): every instance has an id lineage, and every full backup carries it
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / Python tests (push) Successful in 1m37s
CI & Build / integration (push) Successful in 2m7s
CI & Build / Build & push image (push) Successful in 22s
instance_identity is one row made by migration 0127 with a random id, so an
instance has it before its first account. A full backup carries it in its
header as instance_id (the tar manifest header and the JSON top level);
BACKUP_VERSION is 28. The table rides in _NOT_INCLUDED: a restore decides from
the header value rather than copying a row.

Two stores with the same id number their rows the same way. That is what lets
a later step roll a live instance back by original id, refuse a backup whose
ids would land on unrelated rows, and have a fresh-instance restore adopt the
backup's id (adopt_instance_id).

Milestone 582, step 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 19:53:45 -05:00
bvandeusenandClaude Opus 5.5 3c01e17559 test(6838): give the test request its socket peer the way Quart reads it
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / integration (push) Successful in 2m16s
CI & Build / Build & push image (push) Successful in 38s
Quart takes remote_addr from a Remote-Addr header its ASGI layer writes from
the scope (replacing any the client sent). A test request context does not
read scope_base for it, so the two socket-peer tests saw an empty address.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 19:37:28 -05:00
bvandeusenandClaude Opus 5.5 dbd6d5b8b5 feat(6838): read the client address and HTTPS from the trusted proxy hop
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Failing after 1m20s
CI & Build / integration (push) Successful in 2m27s
CI & Build / Build & push image (push) Skipped
grimoire/proxy.py is the one reader of X-Forwarded-For and X-Forwarded-Proto.
It takes the entry TRUSTED_PROXY_HOPS in from the right (default 1; a shorter
header reads its leftmost entry; 0 trusts no proxy), so an entry a client puts
left of the proxy's is never believed. Before this, with TRUST_PROXY_HEADERS
on, the leftmost X-Forwarded-For entry keyed the login, register, forgot and
reset rate limits and the audit log, and a client could choose it.

The session cookie is Secure, and HSTS (max-age=31536000) is sent, exactly
when the request is HTTPS by the trusted proxy's word. Plain HTTP is
untouched and nothing redirects. SECURE_COOKIES is gone (a startup warning says
so when it is still set); TRUST_PROXY_HEADERS=false still maps to 0 hops, with
a warning to move to TRUSTED_PROXY_HOPS. Admin audit lines now use the same
client address instead of the socket peer.

The placeholder-key refusal keyed on SECURE_COOKIES, so it now refuses either
placeholder key outright; the instance makes its own key when none is set.

Family idea #5004 (IsHTTPS, trusted-hop X-Forwarded-Proto), canon v2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 19:32:20 -05:00
bvandeusenandClaude Opus 5.5 b87178d261 fix(6495): a name defined in two packages counts only in the canon's own
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / integration (push) Successful in 1m22s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Successful in 1m59s
CI & Build / Build & push image (push) Successful in 43s
A by-name reference edge could not tell two definitions of one name apart:
in a Go repo that defines `file` as importRun.file (internal/files, the
canon) and as sftpRun.file (internal/replication), a mirror test calling
`m.file(` was recorded as a use of the canon.

names_canon now takes defined_at, every repo path that defines each name,
in place of the shape file's own names. The own-file rule is unchanged.
When the repo also defines the canon's name somewhere the canon is not
recorded, a mention counts only from the canon's own package: the
directory of a file location exactly, or under a directory location.
A name defined once resolves as before. Proposer version 8 -> 9, so the
next coverage refresh re-reads every row and replaces its reference edges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 20:05:09 -04:00
bvandeusenandClaude Opus 5.5 2a6d9080ba fix(6477): a plain-word call needs the parenthesis right after the word
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / integration (push) Successful in 59s
CI & Build / Python tests (push) Successful in 1m37s
CI & Build / Build & push image (push) Successful in 26s
Read against Fabled Archives after the cleanup: of the 9 rows still linked
to snippet #5448 (symbol `file`), all but one matched prose in a comment,
"a file (not a directory)", "the file (ScanWaiting". The only real call
was `r.file(ctx, ...)`. Formatted code never puts a space before a call's
parenthesis and prose does, so the plain-word test is now `word(` with no
whitespace. _PROPOSER_VERSION 8 re-reads every row once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 19:07:03 -04:00
bvandeusenandClaude Opus 5.5 3e2227e8c5 fix(6483): drop the cross-project uses edges the write path wrote
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / integration (push) Successful in 1m0s
CI & Build / Python tests (push) Successful in 1m41s
CI & Build / Build & push image (push) Successful in 19s
Migration 0125 removed the proposer's cross-project `reference` edges, but
the write path records its by-name evidence with basis `hook`, so those
survived it. A hook edge across projects is impossible for the same reason a
reference edge is, and the write path now applies the same by-name test
(names_canon), so 0126 removes the ones written before. Judged edges keep
their own basis and are untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 18:29:54 -04:00
bvandeusenandClaude Opus 5.5 cdb71e64c4 fix(6477): a body uses a canon by name only when the name can resolve to it
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 43s
CI & Build / integration (push) Successful in 1m36s
CI & Build / Python tests (push) Successful in 2m7s
CI & Build / Build & push image (push) Successful in 42s
The proposer's reference basis, and every uses edge it writes, counted any
word-bounded mention of a canon's symbol. Three holes made that wrong at
scale (snippet #5448, symbol `file`, held 2,161 uses edges):

- Go, Rust, Kotlin, Swift, Dart, C, C#, Ruby and PHP were outside the
  language-family maps, so their canons read as "unknown, no gate" and were
  matched in every Python, TS and CSS body.
- Canons in other projects were matched by name, though one repo never
  calls a function defined in another's.
- A plain word (`file`, `seen`) or a module's own helper (`_run`) matched
  wherever it appeared.

names_canon is now the one by-name test, used by match_canon,
reference_canons and the write-path hint: same project, not a name the file
defines itself (unless the canon is recorded there), no generic verb with or
without a leading underscore, and a bare lowercase word only where it is
called. The write path keeps the payload's own names in play, since a
redefinition of a pulled canon is the copy it exists to flag.

Repair: record_uses only adds, so replace_reference_uses now makes a row's
machine-written reference edges exactly what the current rule finds,
leaving judged and hook edges alone. _PROPOSER_VERSION 7 re-reads every row
on its project's next refresh. Migration 0125 deletes the cross-project
reference edges already stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 17:41:21 -04:00
bvandeusenandClaude Opus 5.5 0b98a3a5cd chore(6480): remove five Vue components nothing imports
HistoryPanel, InlineAssistPanel, NoteCard, WorkspaceNoteEditor and
WorkspaceTaskPanel had no importer, lazy import or global registration
anywhere in frontend/src. The features they served were replaced; token
sweeps kept restyling them because they match a glob. The two doc comments
that named HistoryPanel.vue now name VersionHistorySection.vue, the note
history that is live.

SearchBar.vue stays: it is the recorded home of snippet #2961 "SearchBar -
the search input recipe", and the two live search boxes are judged variants
of it. Deleting it would leave the recipe pointing at nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 17:36:03 -04:00
bvandeusenandClaude Opus 5.5 7faaa311d7 fix(6470): a moment proposal counts only opens no hook explained, and asks the reader to judge it
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Successful in 1m46s
CI & Build / Build & push image (push) Successful in 37s
The signal counted every get_rule open against every moment from the
previous three minutes, so a rule a search arm had just shown was
proposed for whatever moment happened to fire. All nine pending signal
proposals were wrong.

- The hook skips an open of a rule a hook named this session, and sends
  the last three acts from 90 seconds instead of 30 from 180.
- A rule already mounted anywhere gathers no signal evidence.
- A signal pair keeps the actions that reached its moment and shows them
  as its why.
- The proposal line says it is co-occurrence, not a recommendation, and
  asks the reader to check the rule against the moment before offering
  it; moments.md says the same.

Plugin 2026.10.26.2029.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 16:29:40 -04:00
bvandeusenandClaude Opus 5.5 83cb6420e2 fix(6471): the end-of-turn shape check hears every definition a write made
The write hooks recorded at most 12 definitions per write (alphabetical,
so test functions were the ones dropped) and the post-Bash hook stopped
after 4 files, so the Stop hook never asked about the rest. The caps
bound the prior-art query; the written ledger now gets the full list,
up to 40 files per Bash call, while the query stays at 4 files x 12 names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 16:26:15 -04:00
bvandeusenandClaude Opus 5.5 db2427a3dc feat(6249): the write-path hint asks whether a new definition copies a close snippet
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m2s
CI & Build / Python tests (push) Successful in 1m56s
CI & Build / Build & push image (push) Successful in 22s
The derive check matches by name, so a renamed copy of a recorded snippet
passed it and showed only as "similar" prior art. When a write introduces a
definition the file does not hold (shape_ledger.new_shapes_at) and an unseen
snippet on the menu clears the write-path floor by WRITEPATH_COPY_MARGIN (0.80
at the default floor), the hint now says to reuse it or extract the shared
shape before another copy. Names the derive check or the stamping feed already
spoke for are left to them. Covers recorded snippets only: the ledger keeps no
bodies to compare an unrecorded sibling against.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 20:14:41 -04:00
bvandeusenandClaude Opus 5.5 0a4eb313fe fix(6255): Test destination names an address that did not answer as S3
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m5s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 23s
A 4xx with no S3 error body came from something else at that address, most
often a web server on the default port. The message now says so and to check
the scheme and port, beside the raw status. A 5xx without a body keeps its
own reason, since a busy S3 service can answer that way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 20:03:52 -04:00
bvandeusenandClaude Opus 5.5 d51ccd9b69 fix(6253): the moment-proposal queue lists a signal pair only once it has proposed
pending() listed every suggested signal row, including pairs still gathering
evidence below the three-situation bar, so the queue held 200 "proposals"
where about 25 had crossed it. A signal pair now needs proposed_at in its
evidence, the same test the misfire half already applies; proposals from a
pass are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 20:03:52 -04:00
bvandeusenandClaude Opus 5.5 67d3c68fbf feat(6254): moment_usage says which rule each moment delivered, and whether it was opened
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 1m5s
CI & Build / Python tests (push) Successful in 1m57s
CI & Build / Build & push image (push) Successful in 33s
Each moment gains by_rule: {rule_id, title, delivered, opened}, read from
the (moment, rule) grouping the readout already did. The moment's totals are
now summed from those rows, so the two cannot disagree. The Moments settings
page shows the list under each moment, closed by default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:59:02 -04:00
bvandeusenandClaude Opus 5.5 95ef4af746 fix(6251): the rules warning sends its reader to the rule-arm review
surfaced_never_pulled told a rules reader there was no judged sample for the
rule arms. There is one now; the guard tying the text to REVIEWABLE caught it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:59:02 -04:00
bvandeusenandClaude Opus 5.5 7c1d98dd2a feat(6251): the rule arms can be reviewed — menus_to_review re-runs prompt_rule, pre_tool_rule and write_path_rule
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m2s
CI & Build / Python tests (push) Failing after 1m26s
CI & Build / Build & push image (push) Skipped
A rule arm logs its query, floor, budget and project, which is everything
semantic_search_rules takes, so a logged call is re-run to a depth past its
budget and judged like an auto_inject menu. within_budget is the arm's own
cut: inside the budget and, for the banded command and write arms, inside the
band of the top hit. "Opened after" reads rule pulls, and rules created
after the call are dropped as notes are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:56:17 -04:00
bvandeusenandClaude Opus 5.5 4297c88cd2 fix(6252): the named-record test keeps its number after "with"
"in 4448" is not read as a reference, so the subject went after the number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:56:17 -04:00
bvandeusenandClaude Opus 5.5 dac4c59e07 fix(6252): a prompt that names nothing gets no notes menu
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Failing after 1m26s
CI & Build / Build & push image (push) Skipped
"keep going", "yes please do", "is it deployed?": with no word of its own the
auto-inject query was the previous reply's tail alone, and the menu handed
back the records that reply was built from. In a sample of 59 logged calls,
28 came from short prompts; those with a subject of their own ("continue into
DRY 14", "fix the Windows gate") led with the record meant, while the ones
without were finished tasks already in context, with an open on 2 of about 15.

_has_subject checks the operator's words against a list of words that name
nothing; without one, the ranked menu is skipped and records named by number
are still shown. The prompt rule arm is unchanged and still searches with the
tail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:51:58 -04:00
bvandeusenandClaude Opus 5.5 1fa9245bbf fix(6247): a getter is seen by its pulls, so the silent list stops naming every pull door
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m5s
CI & Build / Python tests (push) Successful in 1m56s
CI & Build / Build & push image (push) Successful in 22s
usage and rule_usage gain pulled_by_source, read from the per-source rows
the readout already had, and _silent_surfaces counts those doors as having
emitted. The rules-listing points were seen as of the previous commit,
through rule_usage.by_source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:28:29 -04:00
bvandeusenandClaude Opus 5.5 d05d4f3632 feat(4636): rule pull-through per surface, and per-rule usage with confirmed lessons on list_rules
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Successful in 1m56s
CI & Build / Build & push image (push) Successful in 22s
retrieval_telemetry gains rule_usage.by_source, the twin of usage.by_source:
distinct rules each surface showed, how many an agent opened, and a ratio
that is null on a bulk delivery. list_rules(with_usage=True, days=N) adds
each rule's usage, windowed, and how many lessons are confirmed instances
of it, so linked and unlinked rules can be compared through MCP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 19:20:32 -04:00
bvandeusenandClaude Opus 5.5 71260a28e2 align(4909b): every write with a path gets its rule check, whether or not prior art matched
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Successful in 1m56s
CI & Build / Build & push image (push) Successful in 22s
The write-path rule arm ran only past the notes guard: a write whose code
matched no snippet, issue or lesson returned before any rule was searched.
It now runs before that guard, on `code or path`, and a write that matched
nothing but a rule still returns the rule lines.

Operator ruling (2026-10-05): yes. A rule about the file being written does
not depend on a snippet resembling it.

Cost: one more rule search per write that matched no prior art, with its
own embedding (the query is `code or path`, not the notes arm's concept
query, so the per-request memo does not cover it). Plus the preference slot
from (c) on those writes.

Expected telemetry shift, to be read on its own once this reaches main:
write_path_rule calls rise toward write_path's count (14 days to 2026-10-22:
write_path_rule 4,634 against write_path 3,786 — the notes arm skips trivial
payloads, the rule arm does not, so the rule arm will now exceed it). Its
zero-result share rises too, since the newly covered writes are the ones
that matched nothing in the notes corpus.

Tests: the gate test is inverted (a write with no prior art still gets its
rule check); the design-only write now runs the rule arm on its path; two
notes-arm tests read the write_path row instead of any row, since the rule
arm writes its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:42:14 -04:00
bvandeusenandClaude Opus 5.5 407f594a53 fix(4909c): the tool-arm suppression tests read the arm's own row, not the slot's
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Successful in 1m54s
CI & Build / Build & push image (push) Successful in 26s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:39:27 -04:00
bvandeusenandClaude Opus 5.5 c3933507c9 align(4909c): the command and write rule arms reserve a line for a preference
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Failing after 1m26s
CI & Build / Build & push image (push) Skipped
PRE_TOOL_RULE and WRITE_PATH_RULE now take preference_slot=True, like
PROMPT_RULE. When no preference placed on score, the arm asks once more for
the best preference over the same bar, and shows it as one extra line.

Operator ruling (2026-10-05): yes. A preference about how to act ("let each
action land before the next") is ranked against every rule whose trigger
names the same command; the act is its last chance to shape anything.

Cost: one kind-filtered search per call that placed no preference. The
query embedding is memoised per request on both hook routes, so no extra
model call. Expect a new stream of preference_slot rows from the two act
arms; preference_slot's call count will rise roughly by pre_tool_rule's and
write_path_rule's (~23k calls over 14 days before this change), and its
zero-result share will rise with it, since most commands have no matching
preference.

Tests: the tool-arm call-row tests count rows under pre_tool_rule only (the
slot logs under its own name), and the write-arm bar test reads the arm's
own search, which is the first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:37:22 -04:00
bvandeusenandClaude Opus 5.5 6299335254 fix(4909d): a shared floor move records the named surface first, then its siblings
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m9s
CI & Build / Python tests (push) Successful in 1m58s
CI & Build / Build & push image (push) Successful in 33s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:34:16 -04:00
bvandeusenandClaude Opus 5.5 c8ede427ca align(4909d): every rule arm that annotates reads one bar, kb_rule_threshold at 0.70
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Failing after 1m30s
CI & Build / Build & push image (push) Skipped
prompt_rule, pre_tool_rule and write_path_rule now share one settings key,
RULE_FLOOR_KEY = "kb_rule_threshold", default 0.70. The reply backstop keeps
its own 0.80, because there the floor is a stop bar. Moving the shared floor
through tune_retrieval or the Settings form records a tuning event on every
surface that reads it (set_dial, via retrieval_surfaces.sharing_floor), and
the result names the surfaces it carried as `moved_with`. Settings shows one
"Standing-rule confidence threshold" control where it showed three.

Operator ruling (2026-10-05): one shared bar for the rule arms, "especially
since we're only taking the first x number of them" -- the budget limits
what is shown; the bar only decides what is worth ranking. Kept a dial.

Why 0.70, from records read rather than percentiles:
- write_path_rule near-misses at 0.7199-0.72 were all false positives
  (rule 149 against a Lidarr comment, 157 against a retention Svelte file,
  121 against an Archives restore, 164 against a config test, 131 against a
  Go package), so the old 0.72 was not hiding real hits -- but the band does
  noise control on this arm, so dropping 0.02 costs little.
- prompt_rule: the operator set 0.70 by hand on 2026-09-18, after a 0.60
  experiment admitted junk and was reverted. 0.70 is the number with a human
  decision behind it.
- pre_tool_rule lines near 0.68 this session were mostly noise; rising to
  0.70 trims them, and the band still narrows what clears.

Retired keys: kb_promptrule_threshold, kb_toolrule_threshold and
kb_rulehint_threshold are no longer read. Rows an install stored under them
stay in the settings table, inert. On this instance the effective bars move
pre_tool_rule 0.68 -> 0.70, write_path_rule 0.72 -> 0.70, prompt_rule stays
0.70; record_release_defaults writes those per-surface release rows on boot,
so the telemetry shift has an event to explain it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:32:09 -04:00
bvandeusenandClaude Opus 5.5 f77d18cb63 fix(6245): a re-run of bind_repo(moved_from=…) finishes an earlier move
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 59s
CI & Build / integration (push) Successful in 1m25s
CI & Build / Python tests (push) Successful in 2m20s
CI & Build / Build & push image (push) Successful in 32s
Once the binding has moved, the old remote is no longer bound, and
move_binding refused. An install that renamed its repo before snippet
locations moved with it had no way to move them. When the old remote is
unbound and the new one is bound to this project, bind_repo now skips the
binding move and still moves the snippet locations that name the old
repo.

The snippet-move test gives each location its own path: rewritten to one
repo, identical locations merge into one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 16:21:19 -04:00
bvandeusenandClaude Opus 5.5 e89e6a24de fix(6245): bind_repo(moved_from=…) moves the snippet locations recorded in the old repo
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 58s
CI & Build / integration (push) Failing after 1m16s
CI & Build / Python tests (push) Successful in 2m33s
CI & Build / Build & push image (push) Skipped
A repo renamed on its forge kept every snippet location naming the old
repo: the binding and the shape ledger moved, the snippets did not.

- snippets.move_location_repo rewrites, on every live snippet of the
  project, each location whose repo is the old remote in any form (URL,
  normalized key, "owner/Repo", bare "Repo", compared without case) to
  the new normalized key. A snippet the caller can only read is listed
  in read_only and left alone.
- Its loop is shared with move_locations through _rewrite_locations,
  which saves each changed snippet through update_snippet.
- bind_repo calls it after the binding moves and reports snippets_moved
  and snippets_read_only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 16:18:03 -04:00
bvandeusenandClaude Opus 5.5 787685ac63 fix(6244): the instance keeps its own SECRET_KEY in /data, and docker-compose.yml is the compose users deploy from
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m9s
CI & Build / Python tests (push) Successful in 2m7s
CI & Build / Build & push image (push) Successful in 30s
With no SECRET_KEY set, Grimoire fell back to a fixed placeholder. The
quickstart compose substituted change-me-in-production, so the README's
own update step (pull, up -d) recreated the container on that placeholder,
and every credential sealed under the key the user started with stopped
opening.

- config: with neither SECRET_KEY nor SECRET_KEY_FILE, the first start
  writes a random key to DATA_DIR/secret_key (mode 0600, published with a
  hard link so concurrent starts agree), and later starts read it back.
  An explicit key wins and leaves the file alone. A missing
  SECRET_KEY_FILE or an empty key file stops the start instead of
  switching keys. With no data directory (a run outside the container)
  the process gets a temporary key and says so.
- app: warns on a public placeholder key, on a temporary key, and when
  /data is not a mounted volume.
- Dockerfile creates /data.
- docker-compose.yml is now the deploy compose (published image, both
  volumes, nothing to set). The build-from-source compose is
  docker-compose.dev.yml, and docker-compose.quickstart.yml is gone. The
  Swarm stack file gains the /data volume and keeps its explicit key,
  because a Swarm can move the app away from a node-local volume.
- README: a Deploy section with the compose inline, SECRET_KEY optional,
  "keep both volumes", and updates that keep the key. A test holds the
  README's copy equal to the file.
- Installs that ran on the change-me-in-production placeholder get a key
  of their own on the new compose; Settings then asks once for saved
  forge tokens and the backup secret.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 16:13:43 -04:00
bvandeusenandClaude Opus 5.5 81fe851509 licence: Fabled Grimoire is AGPL-3.0-only
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Successful in 2m4s
CI & Build / Build & push image (push) Successful in 47s
LICENSE carries the GNU Affero General Public License v3.0 text, the same
file the other Fabled projects ship. pyproject.toml, the frontend's
package.json and lockfile, and the plugin manifest declare it as the SPDX
expression AGPL-3.0-only, and the README says what it asks of anyone who
runs a modified version over a network. Reading `license` as an SPDX
expression needs setuptools 77, so the build floor rises to it. The
project description now matches the README's tagline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:45:11 -04:00
bvandeusenandClaude Opus 5.5 92a15d3e1a rename(547): the header draws the clasped grimoire, and the README takes the standard shape
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Successful in 2m9s
CI & Build / Build & push image (push) Successful in 42s
AppLogo still drew the old open book with a spark, so the new mark showed
only in the browser tab. It now draws the same clasped grimoire as
favicon.svg, in the mark's own fixed colours. That covers the header and
the sign-in, register and password pages.

The README follows the house order: the logo, name and tagline, a short
summary, quick start including the plugin install, the settings a reader
must set, setup pitfalls, updating and backups, features grouped by what
you do, troubleshooting by the error text, development, and licence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:44:03 -04:00
bvandeusenandClaude Opus 5.5 979efe6662 feat(6243): move_path carries a moved path's snippets with its shapes
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 17s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m9s
CI & Build / Python tests (push) Successful in 2m12s
CI & Build / Build & push image (push) Successful in 31s
Snippet locations are keyed by path too, so a snippet recorded under a
moved directory reads `missing` at its next check. The tool is now
move_path(project_id, from_path, to_path). It moves the ledger rows as
before, and re-paths the project's snippet locations under from_path
through update_snippet, so each body and its indexed mirror stay one
record. A snippet the caller can only read is listed in
snippets_read_only. The tool refuses only when nothing at all is recorded
under from_path. The path test is the one snippets already uses
(_path_touches), and the plugin version is minted for the skill edit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:07:32 -04:00
bvandeusenandClaude Opus 5.5 47b52197aa feat(6243): a directory moved in the tree keeps its shape judgments
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Failing after 15s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m15s
CI & Build / Python tests (push) Successful in 2m9s
CI & Build / Build & push image (push) Successful in 30s
Ledger rows are keyed by path, so after a directory moves the next sync
marks every judged row vanished and lists the same shapes again, unjudged.
move_shapes(project_id, from_path, to_path) re-keys the rows under one path
to the other, in every repo of the project, keeping each row's judgment,
uses edges and history and adding a `moved` event that records the path it
left. A row already at the target is replaced when nobody judged it; a
judged one stands, and the old row is reported in `kept`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:03:03 -04:00
bvandeusenandClaude Opus 5.5 5ad1df8e0f feat(6242): a repo renamed on its forge keeps its binding and shape ledger
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 1m13s
CI & Build / integration (push) Successful in 1m15s
CI & Build / Python tests (push) Successful in 2m8s
CI & Build / Build & push image (push) Successful in 28s
A binding and every row of its project's code-shape ledger are keyed by the
repo's remote. A rename on the forge changes the remote, so binding the new name
started an empty ledger beside the old one and left every judgment behind.

bind_repo now takes moved_from, the old remote. Its binding and the project's
ledger rows move to the new key in one transaction, and the reply says how many
shapes moved. The move is refused when:
- the old name isn't bound to that project;
- the new name is already bound;
- the project already has a ledger under the new name.

Each refusal leaves both names exactly as they were.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 13:53:57 -04:00
bvandeusenandClaude Opus 5.5 a07547447f rename(547): the repo is FabledGrimoire, and no stray "scribe" is left
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m1s
CI & Build / Python tests (push) Successful in 2m2s
CI & Build / Build & push image (push) Successful in 28s
The forge repo is now bvandeusen/FabledGrimoire. The plugin marketplace default,
the install commands in the docs and plugin README, the CI requirements title and
the example repo URLs in comments and tests follow it. docs/development.md now
names the image the build actually publishes, fabledgrimoire.

Two test names still said "scribes_own"; they now say "grimoires_own".

What still says scribe does so on purpose:
- the backup reader's accepted archive names and prefix;
- the label every stored secret is sealed under;
- the notes on keeping an old install's database names;
- the old migrations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 13:29:00 -04:00
141 changed files with 8436 additions and 3539 deletions
+3 -1
View File
@@ -4,4 +4,6 @@ POSTGRES_USER=fabled
POSTGRES_PASSWORD=fabled
POSTGRES_DB=grimoire
# PGDATA_VOLUME=grimoire_pgdata
SECRET_KEY=dev-secret-change-me
# Grimoire keeps its own SECRET_KEY in the /data volume. Set one here only to
# use a key you manage yourself; it then wins over the kept one.
# SECRET_KEY=
+4
View File
@@ -56,5 +56,9 @@ ENV APP_BUILD_KEY=$BUILD_KEY
ENV APP_CHANNEL=$BUILD_CHANNEL
ENV APP_COMMIT=$BUILD_COMMIT
# The instance's own SECRET_KEY, the model cache and diagnostics live here.
# Mount a volume on it so they outlive the container.
RUN mkdir -p /data
EXPOSE 5000
CMD ["sh", "-c", "alembic upgrade head && hypercorn 'grimoire.app:create_app()' --bind 0.0.0.0:5000 --keep-alive 600"]
+661
View File
@@ -0,0 +1,661 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
+5 -5
View File
@@ -3,22 +3,22 @@
# --- Docker ---
build:
docker compose build
docker compose -f docker-compose.dev.yml build
up:
docker compose up -d
docker compose -f docker-compose.dev.yml up -d
down:
docker compose down
docker compose -f docker-compose.dev.yml down
logs:
docker compose logs -f
docker compose -f docker-compose.dev.yml logs -f
health:
curl -s http://localhost:5000/api/health
migrate:
docker compose exec app alembic upgrade head
docker compose -f docker-compose.dev.yml exec app alembic upgrade head
# --- Local dev (requires: uv sync --extra dev and cd frontend && npm ci) ---
+137 -29
View File
@@ -1,42 +1,150 @@
# Fabled Grimoire
<p align="center">
<img src="frontend/public/favicon.svg" width="96" alt="">
</p>
A self-hosted work system-of-record for software projects, built to be driven by Claude Code. Notes, tasks, issues, projects, milestones, rules, and stored processes — reachable from Claude via a built-in MCP endpoint and a bundled Claude Code plugin, with a clean web UI for humans. No in-app LLM; Claude is the sole assistant.
<h1 align="center">Fabled Grimoire</h1>
<p align="center">A self-hosted system of record for the work you do with Claude Code.</p>
Claude records what it learned, decided and built in Grimoire, and that record reaches the next session, so a solution is recalled instead of reinvented. Notes, tasks, plans, rules, lessons and reusable code live in one place. Claude reaches them through a built-in MCP endpoint and the bundled Claude Code plugin, and people read and edit the same records in the web UI. There is no language model inside Grimoire: Claude is the only assistant.
## Deploy
You need Docker with Compose. Save this as `docker-compose.yml` (it is the [`docker-compose.yml`](docker-compose.yml) in this repository), then run `docker compose up -d` beside it:
```yaml
# Fabled Grimoire: the app and its database.
# docker compose up -d
# Then open http://localhost:5000. The first account registered becomes admin.
#
# Keep both volumes. app_data holds the instance's own SECRET_KEY, which seals
# saved credentials; pgdata holds everything else.
services:
app:
image: git.fabledsword.com/bvandeusen/fabledgrimoire:latest
ports:
- "5000:5000"
environment:
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-grimoire}:${POSTGRES_PASSWORD:-grimoire}@db:5432/${POSTGRES_DB:-grimoire}"
LOG_LEVEL: "${LOG_LEVEL:-INFO}"
# Port 5000 is published straight to clients, so no proxy is trusted.
# Behind a reverse proxy, set 1 (one per proxy) and BASE_URL:
TRUSTED_PROXY_HOPS: "0"
# BASE_URL: "https://grimoire.example.com"
volumes:
- app_data:/data
depends_on:
- db
restart: unless-stopped
db:
# Postgres 17 with the vector extension Grimoire needs.
image: pgvector/pgvector:pg17
stop_grace_period: 120s
volumes:
- pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: ${POSTGRES_USER:-grimoire}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-grimoire}
POSTGRES_DB: ${POSTGRES_DB:-grimoire}
restart: unless-stopped
volumes:
app_data:
pgdata:
```
Open `http://localhost:5000`. The first account you register becomes the admin.
Nothing has to be set first. On its first start Grimoire makes its own `SECRET_KEY` and keeps it in the `app_data` volume, so restarts, reboots and updates all use the same key.
To connect Claude, create an API key under **Settings → MCP Access**, then in Claude Code:
```
/plugin marketplace add https://git.fabledsword.com/bvandeusen/FabledGrimoire.git
/plugin install grimoire@grimoire-plugin
```
The plugin asks for your Grimoire URL and the API key. Settings → MCP Access shows the exact commands for your instance.
## Configuration
Everything is optional. Set these in the environment or in a `.env` file beside the compose file:
| Variable | When | What it does |
|----------|------|--------------|
| `BASE_URL` | Behind a proxy | The public URL, used in email links and SSO redirects |
| `TRUSTED_PROXY_HOPS` | Rarely | The starting count of reverse proxies in front of Grimoire (default `1`; the compose file above sets `0` because it publishes the port directly). Check and change it in **Settings → Config → Reverse proxy**, which shows what each count reads for your own request; behind Cloudflare it is usually `2` |
| `POSTGRES_PASSWORD` | Before the first start | The database password (defaults to `grimoire`; the database is reachable only inside the compose network). Postgres takes it once, when the volume is created |
| `SECRET_KEY` | Rarely | A key you manage yourself, instead of the one Grimoire keeps. It wins over the kept key; remove it to go back |
Everything else (registration, email, SSO, git forge connections, retrieval tuning, backups) is in **Settings**, or listed in [Configuration](docs/configuration.md).
## Setup pitfalls
- **Keep both volumes.** `pgdata` holds everything Grimoire stores. `app_data` holds the instance's key, and forge tokens and the backup bucket's secret are sealed under it. If the key is lost, Settings asks you to enter those credentials again; nothing else is lost.
- **The embedding model downloads on first use**, into the `app_data` volume. Without internet access, semantic search falls back to keyword search until the model is in place.
- **Behind a reverse proxy, turn off response buffering** for `/mcp`, because MCP responses stream. [Configuration](docs/configuration.md#behind-a-reverse-proxy) has an Nginx example.
## Updating and backups
```bash
docker compose pull
docker compose up -d
```
Database migrations run when the app starts, and the key stays in its volume. Claude Code updates the plugin from its marketplace; run `/reload-plugins` after an update.
**Settings → Data → Scheduled backups** writes a daily backup of the database to a folder or an S3-compatible bucket, and Settings → Data → Restore from Backup rolls the instance back to one, or brings a new instance back from it. See [Backups](docs/backups.md).
## Features
Notes, tasks and issues; projects with milestones as plans; rules that reach Claude when its work matches them or at a moment of work; lessons, snippets of prior art and a ledger of the code they account for; design systems; ideas shared across projects on the same platform; and stored processes. Semantic search feeds both Claude's recall and the plugin's context injection. A knowledge graph, sharing with users and groups, and scheduled backups round it out. See [Features](docs/features.md).
**Working with Claude**
- Claude enters a project and gets a short primer: its goal, current plans, open tasks and areas.
- Your rules reach Claude when its work matches them, or at a moment of work such as finishing a task or pushing code.
- Before Claude writes a file or runs a command, the plugin shows it the rules, lessons and reusable code that may apply.
- Stored procedures, such as a drift audit, become Claude Code skills.
## Quick Start
**Recording the work**
- Notes in a Markdown editor, with wikilinks, tags and version history.
- Tasks, issues and time-boxed investigations, with work logs that the next session reads first.
- Plans as milestones: a design plus ordered steps.
- Lessons learned the hard way, linked to the rules they illustrate.
- Duplicate checks, so a new record that matches an existing one updates it instead.
**Prerequisites:** Docker and Docker Compose. No GPU or local model needed — Claude is the sole assistant, reached over MCP.
**Reusing code**
- Snippets of reusable code, each with when to reach for it and where it lives in the repo.
- With a git forge connected, Grimoire tracks every function and style in your repos and shows which ones are still unaccounted for.
- Ideas shared across projects on the same platform, with each project's answer recorded.
- Design systems with tokens, inheritance and a generated stylesheet.
Download [`docker-compose.quickstart.yml`](docker-compose.quickstart.yml) from this repo, then:
**For people**
- A web UI to browse, search and edit everything, plus a graph of how notes connect.
- Sharing with users and groups, as viewer, editor or admin.
- Email and password sign-in, optional SSO through OIDC, and invite links.
- Personal export to Markdown or JSON, and admin backup and restore.
The full list is in [Features](docs/features.md).
## Troubleshooting
| What you see | Why | Fix |
|--------------|-----|-----|
| `SECRET_KEY is set to the public placeholder` at startup | `SECRET_KEY` is `dev-secret-change-me` or `change-me-in-production` | Remove `SECRET_KEY`, or set a real one |
| Settings asks for forge tokens or the backup secret again | The instance's key changed: the `app_data` volume was replaced, or `SECRET_KEY` was set or removed | Enter them again, or restore the previous key |
| `Can't locate revision identified by '…'` at startup | An older image is running against a database a newer one has already migrated | Run the newer image; migrations only move forward |
| The plugin's tools are missing in Claude Code | The plugin isn't installed, or Claude Code hasn't reloaded it | `/plugin` to install or update it, then `/reload-plugins` |
| Search finds only exact words | The embedding model hasn't downloaded yet | Give the app internet access once, or wait for the first download to finish |
## Development
```bash
# Optional but recommended — set a secret key
export SECRET_KEY=your-random-secret-here
docker compose -f docker-compose.quickstart.yml up -d
docker compose -f docker-compose.dev.yml up --build
```
Open `http://localhost:5000`. The first user to register becomes admin. To connect Claude, create an API key under **Settings → MCP Access** and install the Claude Code plugin — see [API Keys & MCP](docs/api-keys-and-mcp.md).
[Development](docs/development.md) covers checks, migrations, the plugin and CI. [Architecture](docs/architecture.md) describes how it is built, and the [API reference](docs/api-reference.md) lists every REST endpoint.
> **Development:** To build from source, see [Development](docs/development.md).
## Licence
## Documentation
| Doc | Contents |
|-----|----------|
| [Architecture](docs/architecture.md) | Stack, design decisions, data models, key services |
| [Configuration](docs/configuration.md) | Environment variables, Docker Compose, production setup, security |
| [Features](docs/features.md) | Detailed feature breakdown and keyboard shortcuts |
| [Development](docs/development.md) | Dev workflow, CI/CD, migrations, release process |
| [API Keys & MCP](docs/api-keys-and-mcp.md) | API keys and installing the Claude Code plugin |
| [SSO / OAuth](docs/sso-oauth.md) | OIDC setup for Authentik, Keycloak, and other providers |
| [Backups](docs/backups.md) | Scheduled backups to a folder or S3, and restoring from one |
| [Design system](docs/design-system.md) | The FabledSword house style and how Grimoire applies it |
| [API Reference](docs/api-reference.md) | All REST API endpoints |
## License
This project is privately maintained.
Fabled Grimoire is free software under the [GNU Affero General Public License v3.0 only](LICENSE) (`AGPL-3.0-only`). If you run a modified version for other people over a network, the licence requires you to offer them its source.
@@ -0,0 +1,35 @@
"""drop machine-written uses edges that cross projects
Revision ID: 0125
Revises: 0124
Create Date: 2026-10-26
A `uses` edge with basis `reference` is the proposer's by-name reading: the
shape's body names a canon's symbol. A body never calls a function defined in
another project's repository, so such an edge across projects is a common
word matched, not a call. The proposer no longer writes them, and this removes
the ones it wrote before. Edges a person or an audit judged keep their own
basis and are left alone. Rows within one project are re-read on the next
coverage refresh, which replaces their reference edges under the same rule.
Downgrade restores nothing: the removed edges were never facts.
"""
from alembic import op
revision = "0125"
down_revision = "0124"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute(
"DELETE FROM code_shape_uses u USING code_shapes s, notes n "
"WHERE u.shape_id = s.id AND u.snippet_id = n.id "
"AND u.basis = 'reference' "
"AND n.project_id IS NOT NULL AND n.project_id <> s.project_id"
)
def downgrade() -> None:
pass
@@ -0,0 +1,34 @@
"""drop write-path uses edges that cross projects
Revision ID: 0126
Revises: 0125
Create Date: 2026-10-26
The write path records a `uses` edge with basis `hook` when a session pulled a
snippet and then wrote code naming its symbol. That is by-name evidence, like
the proposer's `reference` edges 0125 removed, and a body never calls a
function defined in another project's repository. The write path now applies
the same by-name test, so it writes no such edge; this removes the ones it
wrote before. Judged edges keep their own basis and are left alone.
Downgrade restores nothing: the removed edges were never facts.
"""
from alembic import op
revision = "0126"
down_revision = "0125"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.execute(
"DELETE FROM code_shape_uses u USING code_shapes s, notes n "
"WHERE u.shape_id = s.id AND u.snippet_id = n.id "
"AND u.basis = 'hook' "
"AND n.project_id IS NOT NULL AND n.project_id <> s.project_id"
)
def downgrade() -> None:
pass
@@ -0,0 +1,39 @@
"""instance identity — the id lineage a backup carries
Revision ID: 0127
Revises: 0126
Create Date: 2026-11-05
One row naming this database's id lineage. Every full backup carries it, a
restore into a fresh instance adopts it (the restore keeps the original ids),
and rolling a live instance back requires the backup's to match, so a backup's
ids never land on unrelated rows. Made here with a random value, so every
instance has one before its first account exists.
"""
import sqlalchemy as sa
from alembic import op
revision = "0127"
down_revision = "0126"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"instance_identity",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("instance_id", sa.Text(), nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), nullable=False,
server_default=sa.func.now(),
),
sa.CheckConstraint("id = 1", name="ck_instance_identity_single_row"),
)
op.execute(
"INSERT INTO instance_identity (id, instance_id) VALUES (1, gen_random_uuid()::text)"
)
def downgrade() -> None:
op.drop_table("instance_identity")
+1 -1
View File
@@ -1,4 +1,4 @@
# CI Requirements — FabledScribe
# CI Requirements — FabledGrimoire
> Spec lives in [`docs/process.md`](https://git.fabledsword.com/bvandeusen/CI-runner/src/branch/main/docs/process.md)
> in the CI-Runner repo.
@@ -1,36 +1,35 @@
# Fabled Grimoire from the pre-built image:
# docker compose -f docker-compose.quickstart.yml up -d
# Then open http://localhost:5000 (the first account registered becomes admin)
# and connect Claude from Settings → MCP Access.
# Grimoire built from this checkout, for development:
# docker compose -f docker-compose.dev.yml up --build
services:
app:
image: git.fabledsword.com/bvandeusen/fabledgrimoire:latest
build: .
ports:
- "5000:5000"
environment:
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-grimoire}:${POSTGRES_PASSWORD:-grimoire}@db:5432/${POSTGRES_DB:-grimoire}"
# Set SECRET_KEY in the environment or a .env file beside this one.
SECRET_KEY: "${SECRET_KEY:-change-me-in-production}"
LOG_LEVEL: "${LOG_LEVEL:-INFO}"
volumes:
- app_data:/data
depends_on:
- db
restart: unless-stopped
volumes:
- app_data:/data
# Or a bind mount, for direct host access to the app's data:
# - ./data:/data
environment:
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-grimoire}:${POSTGRES_PASSWORD:-grimoire}@db:5432/${POSTGRES_DB:-grimoire}"
# Optional: a SearXNG instance to list as the web-search backend in Integrations.
# SEARXNG_URL: "http://searxng:8080"
db:
# Postgres 17 with the vector extension Grimoire needs.
image: pgvector/pgvector:pg17
stop_grace_period: 120s
restart: unless-stopped
volumes:
- pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: ${POSTGRES_USER:-grimoire}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-grimoire}
POSTGRES_DB: ${POSTGRES_DB:-grimoire}
restart: unless-stopped
volumes:
app_data:
pgdata:
app_data:
+9 -2
View File
@@ -3,10 +3,16 @@ services:
image: git.fabledsword.com/bvandeusen/fabledgrimoire:latest
environment:
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-grimoire}:${DB_PASSWORD}@db:5432/${POSTGRES_DB:-grimoire}"
# A Swarm can start the app on another node, away from the key kept in
# a node-local /data volume, so a stack sets its key itself (or mounts
# a Docker secret and sets SECRET_KEY_FILE to its path).
SECRET_KEY: "${SECRET_KEY}"
LOG_LEVEL: "${LOG_LEVEL:-INFO}"
TRUST_PROXY_HEADERS: "true"
SECURE_COOKIES: "true"
# Proxies in front of the app (Traefik = 1). The client address and
# HTTPS are read from that far in from the right of X-Forwarded-*.
TRUSTED_PROXY_HOPS: "1"
volumes:
- app_data:/data
networks:
- backend
@@ -26,6 +32,7 @@ services:
volumes:
pgdata:
name: ${PGDATA_VOLUME:-grimoire_pgdata}
app_data:
networks:
backend:
+19 -12
View File
@@ -1,33 +1,40 @@
# Fabled Grimoire: the app and its database.
# docker compose up -d
# Then open http://localhost:5000. The first account registered becomes admin.
#
# Keep both volumes. app_data holds the instance's own SECRET_KEY, which seals
# saved credentials; pgdata holds everything else.
services:
app:
build: .
image: git.fabledsword.com/bvandeusen/fabledgrimoire:latest
ports:
- "5000:5000"
environment:
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-grimoire}:${POSTGRES_PASSWORD:-grimoire}@db:5432/${POSTGRES_DB:-grimoire}"
LOG_LEVEL: "${LOG_LEVEL:-INFO}"
# Port 5000 is published straight to clients, so no proxy is trusted.
# Behind a reverse proxy, set 1 (one per proxy) and BASE_URL:
TRUSTED_PROXY_HOPS: "0"
# BASE_URL: "https://grimoire.example.com"
volumes:
- app_data:/data
depends_on:
- db
restart: unless-stopped
volumes:
- app_data:/data
# Or a bind mount, for direct host access to the app's data:
# - ./data:/data
environment:
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-grimoire}:${POSTGRES_PASSWORD:-grimoire}@db:5432/${POSTGRES_DB:-grimoire}"
SECRET_KEY: "${SECRET_KEY:-dev-secret-change-me}"
# Optional: a SearXNG instance to list as the web-search backend in Integrations.
# SEARXNG_URL: "http://searxng:8080"
db:
# Postgres 17 with the vector extension Grimoire needs.
image: pgvector/pgvector:pg17
stop_grace_period: 120s
restart: unless-stopped
volumes:
- pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: ${POSTGRES_USER:-grimoire}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-grimoire}
POSTGRES_DB: ${POSTGRES_DB:-grimoire}
restart: unless-stopped
volumes:
pgdata:
app_data:
pgdata:
+1 -1
View File
@@ -53,7 +53,7 @@ without a read-shaped name are spelled out here. A `write`-scoped key may call e
The plugin connects the MCP server and adds the hooks and skills that put Grimoire's records in front of Claude as it works. Install it from this repo:
```
/plugin marketplace add https://git.fabledsword.com/bvandeusen/FabledScribe.git
/plugin marketplace add https://git.fabledsword.com/bvandeusen/FabledGrimoire.git
/plugin install grimoire@grimoire-plugin
```
+5 -1
View File
@@ -251,6 +251,7 @@ These serve the plugin's hooks.
| POST | `/api/plugin/rule-opened` | Records that a session opened a rule it was shown |
| GET | `/api/plugin/shape-check` | The end-of-turn question about what this turn wrote |
| GET | `/api/plugin/claim-session` · `/release-session` | Bind a session to its task claims / release them at session end |
| GET | `/api/plugin/write-scope` | Which projects a pending act would change (a directory, or an MCP call's `tool` and `input`), and whether a session in its own project may do it. Filing an issue elsewhere is allowed; every other cross-project write is answered `allowed: false` with the reason the operator is asked |
| GET | `/api/plugin/processes` | Stored Processes for skill-stub sync |
| GET | `/api/plugin/prior-art` | Write-path hint for the plugin hooks (params: `path`, `code`, `repo`, `shapes`, `exclude_ids`, `exclude_sync_ids`, `exclude_derive`); returns `context`, `note_ids`, `sync_note_ids`, `stamped`, `divergence`, `derive`, `derive_keys` |
| GET / POST | `/api/projects/<id>/coverage`, `…/coverage/refresh` | Shape-ledger accounting (`pattern_coverage` line, counts, `derive_groups` — css groups carry `consumers`, `derive_new`, `unused_css`, `divergence`, `recheck`) |
@@ -272,12 +273,15 @@ These serve the plugin's hooks.
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/admin/backup` | Export backup (format v4; `?scope=user` for own data) |
| POST | `/api/admin/restore` | Restore from a backup |
| GET | `/api/admin/restore/sources` | The backups at the backup destination, newest first |
| POST | `/api/admin/restore/preview` | What restoring a backup (`?source=<name>` or an uploaded file) would change; 409 with the reason when it can't apply here |
| POST | `/api/admin/restore?confirm=rollback` | Put the whole instance back to that backup, after saving the current state |
| GET / DELETE | `/api/admin/users` · `/api/admin/users/:id` | List / delete users |
| GET / PUT | `/api/admin/registration` | Get / toggle registration |
| GET / PUT | `/api/admin/smtp` · POST `/api/admin/smtp/test` | SMTP config + test email |
| GET | `/api/admin/logs` · `/api/admin/logs/stats` | Log entries + category counts |
| GET / PUT | `/api/admin/base-url` | Get / set the public base URL |
| GET / PUT | `/api/admin/proxy` | Get / set how many reverse proxies stand in front of Grimoire (`{"hops": n}`, or `null` for the environment's count), with what each count would read for this request |
| GET / PUT | `/api/admin/db-maintenance` (+ `/health`, POST `/run`) | VACUUM schedule, health, manual run |
| POST / GET / DELETE | `/api/admin/invitations` (+ `/:id`) | Create / list / revoke invite links |
| GET / PUT | `/api/admin/backups` | Scheduled backups: destination, schedule and recent runs |
+4 -4
View File
@@ -69,12 +69,12 @@ Every database wait on startup has a deadline. Migrations wait for the database
## Authentication
- **Sessions**: `HttpOnly`, `SameSite=Lax` cookies, `Secure` when `SECURE_COOKIES=true`. Changing a password bumps the user's session version and signs out their other sessions.
- **Sessions**: `HttpOnly`, `SameSite=Lax` cookies, `Secure` on a request the trusted proxy reports as HTTPS, which also gets HSTS (`grimoire/proxy.py`). Changing a password bumps the user's session version and signs out their other sessions.
- **API keys**: `Authorization: Bearer fmcp_...`, stored hashed. A `read` key may only call read endpoints and read MCP tools.
- **OIDC** single sign-on with PKCE, optionally replacing local logins ([SSO / OAuth](sso-oauth.md)).
- **Rate limits** on register, login and forgot-password, per client IP (`TRUST_PROXY_HEADERS` reads the IP from the proxy's headers).
- **Rate limits** on register, login and forgot-password, per client IP, read from `X-Forwarded-For` by `TRUSTED_PROXY_HOPS`.
Credentials Grimoire stores for outside services (forge tokens, the backup destination's S3 secret) are sealed under a key derived from `SECRET_KEY`.
Credentials Grimoire stores for outside services (forge tokens, the backup destination's S3 secret) are sealed under a key derived from `SECRET_KEY`, which the instance makes on first start and keeps in `/data/secret_key` unless one is set.
## Repository layout
@@ -87,7 +87,7 @@ Credentials Grimoire stores for outside services (forge tokens, the backup desti
│ ├── routes/ REST blueprints
│ ├── mcp/ MCP server and its tool modules
│ ├── services/ shared logic
│ └── restore.py `python -m grimoire.restore` for a fresh instance
│ └── restore.py `python -m grimoire.restore`: the restore, from the command line
├── frontend/src/ views, components, stores, composables, api client
├── plugin/ Claude Code hooks, skills and commands
├── scripts/ CI checks and the plugin version minter
+54 -22
View File
@@ -31,7 +31,8 @@ differently. It does **not** hold:
- **Credentials:** API keys and forge connections. After a restore, issue new
API keys and add forge connections again. (Forge tokens are also sealed under
`SECRET_KEY` in the database itself, so a copy of the database doesn't carry
them in readable form either.)
them in readable form either. The key lives in `/data/secret_key`, outside
the database.)
- **Logs and notifications.**
- **The backup destination's own S3 secret.**
@@ -87,9 +88,12 @@ You need four things:
- **Access key.**
- **Secret key.**
Grimoire uploads and lists; it never deletes. A key that may **only add
objects** is therefore enough, and it is the safer key to hand out: a stolen
one cannot wipe the history.
Grimoire uploads and lists, and reads a backup back when you restore from the
bucket; it never deletes. A key that may **add and read objects but not
delete them** is therefore enough, and it is the safer key to hand out: a
stolen one cannot wipe the history. A key that may only add objects still
takes backups; restore then works by uploading a file or from the command
line.
The secret is stored encrypted, under a key derived from `SECRET_KEY`. It is
never shown again, and never included in a backup. If an instance's
@@ -121,8 +125,10 @@ keeps service backups as "backup targets". Either destination works with it.
1. On Archives' **Backups** page, make a target for Grimoire, named `grimoire`
for example.
2. Choose how Grimoire's files arrive:
- **S3**: turn on the S3 way in for the target and make an **append-only**
S3 key. In Grimoire, choose **An S3 bucket**:
- **S3**: turn on the S3 way in for the target and make an S3 key that
may add and read objects but not delete them (an **append-only** key
takes backups too, but then Grimoire can't restore from the bucket). In
Grimoire, choose **An S3 bucket**:
- address: Archives' backup address (`http://<archives>:8081`);
- bucket: the target's name;
- access key and secret key: the key you just made.
@@ -138,26 +144,52 @@ keeps service backups as "backup targets". Either destination works with it.
## Restoring
### Into a fresh instance
A restore puts the **whole instance** back to the moment a backup was taken,
for every user:
Start Grimoire on an empty database, copy the backup file into the container,
and run:
- records edited since go back to how they were;
- records deleted since come back;
- anything made since is removed.
It matches records by their ids, so a live instance only restores a backup **of
its own**. Each instance has an id, and every backup records the one it came
from. A backup from another instance, or one made before backups recorded it,
only goes into a new, empty instance, which takes it as it is.
Before changing anything, the restore saves the current state as one more
backup. It goes to the backup destination when scheduled backups are on, and
otherwise to `/data/pre-restore`. The result says which file it is, so a
restore can itself be undone.
What a backup doesn't carry stays as it is wherever its owner survives the
restore: API keys, forge connections, sharing, logs, and the destination's S3
secret. The trash isn't in backups, so anything in the trash when the backup
was taken is gone afterwards. Everyone is signed out, and search is rebuilt.
### From the web
**Settings → Data → Restore from Backup** lists the backups at the backup
destination. You can pick one of those, or upload a `.tar` or the JSON
**Full Backup** download (uploads are held to the server's request-size
limit). The restore then shows what would change: when the backup was taken,
and how many accounts, projects, plans, notes and rules there are now and will
be afterwards. Type `roll back` to run it. Then sign in again.
To read backups back from an S3 bucket, the key needs read access as well as
write. Grimoire never deletes from the bucket.
### From the command line
For a file with no size limit, or a new instance with no backup destination
yet, copy the file into the container and run:
```sh
docker exec -it grimoire python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar
```
- The command refuses a database that already has an account. A restore
re-creates everything with new ids, so on a live instance it would add a
second copy instead of replacing anything.
- Nothing limits the file's size.
- It also reads the JSON from the admin **Full Backup** download.
It prints what would change and stops. Add `--yes` to restore. It reads the
JSON download too. Afterwards, **restart Grimoire** so the search indexes are
rebuilt.
Afterwards, **restart Grimoire** so the search indexes are rebuilt. Then issue
new API keys, add forge connections again, and set up any sharing.
### From the web
**Settings → Data → Restore from Backup** takes the `.tar` as well as the
JSON download. The upload is held to the server's request-size limit. For a
bigger file, use the command above.
After restoring into a new instance, issue new API keys, add forge connections
again, and set up any sharing: none of them travel in a backup.
+24 -17
View File
@@ -11,10 +11,10 @@ Any variable marked *secret* can also be read from a file: set `<NAME>_FILE` to
| Variable | Default | Description |
|----------|---------|-------------|
| `DATABASE_URL` *(secret)* | `postgresql+asyncpg://grimoire:grimoire@localhost:5432/grimoire` | PostgreSQL connection string, using the `asyncpg` driver |
| `SECRET_KEY` *(secret)* | `dev-secret-change-me` | Signs sessions and seals stored credentials. Set it before first use and keep it (see below) |
| `SECRET_KEY` *(secret)* | kept in `DATA_DIR` | Signs sessions and seals stored credentials. Unset, the instance makes its own on first start and keeps it in `DATA_DIR/secret_key`. Set it only to manage the key yourself; it wins over the kept one (see below) |
| `DATA_DIR` | `/data` | Where the instance keeps its own `SECRET_KEY`. Mount a volume here |
| `BASE_URL` | `http://localhost:5000` | Public URL, used in email links and OIDC redirects |
| `SECURE_COOKIES` | `false` | `true` when Grimoire is served over HTTPS |
| `TRUST_PROXY_HEADERS` | `false` | `true` behind a reverse proxy, to read the client IP from `X-Forwarded-For` / `X-Real-IP` |
| `TRUSTED_PROXY_HOPS` | `1` | The starting count of reverse proxies in front of Grimoire. **Settings → Config → Reverse proxy** is where the count is checked and set, and a count saved there wins over this. The client address and the scheme are read that many entries in from the right of `X-Forwarded-For` and `X-Forwarded-Proto`; `0` ignores both headers. See [Behind a reverse proxy](#behind-a-reverse-proxy) |
| `LOG_LEVEL` | `INFO` | `DEBUG`, `INFO`, `WARNING` or `ERROR` |
| `LOG_RETENTION_DAYS` | `90` | Days of app logs to keep |
| `FASTEMBED_CACHE_DIR` | `/data/fastembed-cache` | Where the embedding model is stored after its first download |
@@ -56,27 +56,27 @@ Admins can set SMTP in **Settings**; these are the defaults until they do.
## Running it
### Quick start
### Docker Compose
[`docker-compose.quickstart.yml`](../docker-compose.quickstart.yml) runs the published image with its own database:
[`docker-compose.yml`](../docker-compose.yml) runs the published image with its own database, and needs nothing set:
```bash
SECRET_KEY=$(python3 -c 'import secrets; print(secrets.token_hex(32))') \
docker compose -f docker-compose.quickstart.yml up -d
docker compose up -d
```
Open `http://localhost:5000`. The first account registered becomes the admin; after that, registration stays closed until an admin opens it in Settings or sends an invite.
Update with `docker compose pull && docker compose up -d`. The key and the data stay in their volumes.
### From source
```bash
cp .env.example .env # then set SECRET_KEY
docker compose up --build
docker compose -f docker-compose.dev.yml up --build
```
### Docker Swarm
[`docker-compose.prod.yml`](../docker-compose.prod.yml) is a stack file: the app and database on a private overlay network, with `SECURE_COOKIES` and `TRUST_PROXY_HEADERS` on. Provide `DB_PASSWORD` and `SECRET_KEY` in the environment:
[`docker-compose.prod.yml`](../docker-compose.prod.yml) is a stack file: the app and database on a private overlay network, trusting one proxy (`TRUSTED_PROXY_HOPS: "1"`). A Swarm can move the app to another node, away from a node-local `/data` volume, so the stack sets its own key: provide `DB_PASSWORD` and `SECRET_KEY` in the environment (or mount a Docker secret and set `SECRET_KEY_FILE`):
```bash
DB_PASSWORD=... SECRET_KEY=... docker stack deploy -c docker-compose.prod.yml grimoire
@@ -89,19 +89,22 @@ The shipped files leave out healthchecks and restart policies on purpose. The ap
### Volumes
- **Database** (`pgdata`): everything Grimoire stores.
- **App data** (`/data`): the embedding model cache and diagnostics. Losing it costs a re-download, nothing else.
- **App data** (`/data`): the instance's own `SECRET_KEY`, the embedding model cache and diagnostics. Losing it costs the key: Settings then asks for saved forge tokens and the backup secret again, and the model downloads again.
Back up the database with **Settings → Data → Scheduled backups** ([Backups](backups.md)).
Back up the database with **Settings → Data → Scheduled backups** ([Backups](backups.md)). A database backup doesn't carry the key; to move sealed credentials to a new instance, copy `/data/secret_key` with it.
## Behind a reverse proxy
Grimoire doesn't terminate TLS. Put it behind Nginx, Traefik or Caddy, forward to port 5000, and keep port 5000 off the internet. Set `SECURE_COOKIES=true`, `TRUST_PROXY_HEADERS=true` and `BASE_URL`.
Grimoire doesn't terminate TLS. Put it behind Nginx, Traefik or Caddy, forward to port 5000, and keep port 5000 off the internet. Set `BASE_URL`, then set the proxy count to the number of proxies a request passes through: one proxy is `1`, the default; Cloudflare's proxied DNS in front of your own proxy is `2`.
**Settings → Config → Reverse proxy** sets the count and checks it against your own request. It shows the address Grimoire reads you as, and what each count from 0 to 3 would read. Pick the count that shows your own address. When a request comes through Cloudflare, the section names the count that reads the address Cloudflare reports, with a button that sets it. `TRUSTED_PROXY_HOPS` in the environment is only the starting value: a count saved in Settings wins, and **Use the environment's** goes back to it.
A count that is too low makes every visitor read as the proxy, so they share one sign-in rate limit and a few bad sign-ins block everyone. The "too many attempts" message says so and points here. The block on sign-ins lifts after a minute; an admin who is already signed in can fix the count at once.
```nginx
location / {
proxy_pass http://127.0.0.1:5000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
@@ -111,12 +114,16 @@ location / {
}
```
With `TRUST_PROXY_HEADERS=true`, make sure the proxy replaces any `X-Forwarded-For` the client sends.
Each proxy appends its entry to `X-Forwarded-For` and `X-Forwarded-Proto`, so Grimoire reads the entry that many in from the right: the one your own proxy wrote. Whatever a client puts to the left of it is ignored. A header with fewer entries than that (a LAN road with one proxy beside a Cloudflare road with two) is read at its leftmost entry, the one the first proxy wrote.
When that entry says `https`, the session cookie is marked `Secure` and the response carries `Strict-Transport-Security: max-age=31536000`. Over plain HTTP neither is sent, and Grimoire never redirects to HTTPS, so a LAN install keeps working.
A hop count higher than the real number of proxies lets a client choose its own address, so count them. With `0`, the headers are ignored and the socket peer is the client: use it when clients reach port 5000 directly.
## Security checklist
- **Set a strong `SECRET_KEY`, and keep it.** Forge tokens and the backup destination's S3 secret are sealed under a key derived from it, so a copy of the database alone can't read them. If the key changes (say, a restore onto an instance with a different key), Settings marks those credentials and asks for them again; nothing else breaks. The app logs a warning at startup while the default key is in use, and refuses to start with it when `SECURE_COOKIES=true`.
- **`SECURE_COOKIES=true`** whenever Grimoire is served over HTTPS.
- **Keep the key.** Forge tokens and the backup destination's S3 secret are sealed under a key derived from `SECRET_KEY`, so a copy of the database alone can't read them. Unset, the instance makes a random key and keeps it in `/data/secret_key`, readable by its owner only; keep that volume. If the key changes (say, a restore onto an instance with a different key), Settings marks those credentials and asks for them again; nothing else breaks. The app warns at startup when `SECRET_KEY` is a public placeholder or `/data` is not a mounted volume, and refuses to start with a placeholder key (`dev-secret-change-me`, `change-me-in-production`).
- **The proxy count** (Settings → Config → Reverse proxy) matches the number of proxies in front of Grimoire, so rate limits and the audit log see the real client and the cookie is `Secure` over HTTPS. The section shows whether it does.
- **Registration** is open only for the first account. Invite people, or open it briefly from Settings.
- **Signing everyone else out**: changing a password ends that user's other sessions; Settings → Account has a button for it too.
- **API keys** are per user, shown once, and can be read-only. Revoke any you no longer use under Settings → MCP Access.
+5 -5
View File
@@ -3,9 +3,9 @@
## Running it
```bash
docker compose up --build # app + PostgreSQL at http://localhost:5000
docker compose up --build app # rebuild after a change
docker compose down -v # stop and wipe the database
docker compose -f docker-compose.dev.yml up --build # app + PostgreSQL at http://localhost:5000
docker compose -f docker-compose.dev.yml up --build app # rebuild after a change
docker compose -f docker-compose.dev.yml down -v # stop and wipe the database
```
The image builds the frontend, so a frontend change needs a rebuild too. For quicker frontend work, run Vite against the running app; it proxies `/api` to port 5000:
@@ -32,7 +32,7 @@ Unit tests run without a database. Tests marked `integration` need a real Postgr
The container runs `alembic upgrade head` on every start, after waiting for the database to accept connections.
```bash
docker compose exec app alembic revision -m "short_description"
docker compose -f docker-compose.dev.yml exec app alembic revision -m "short_description"
```
- Number revisions sequentially (`0123_...`). Revision ids must fit in 32 characters, the width of `alembic_version.version_num`.
@@ -57,7 +57,7 @@ CI runs on Gitea Actions (workflow in `.forgejo/workflows/ci.yml`) in the shared
| `main` | `:latest`, `:<sha>` |
| a `v*` tag | `:latest`, `:<version>`, `:<sha>` |
`:<sha>` never moves, so it is the rollback point. Images publish to `git.fabledsword.com/bvandeusen/fabledscribe`, with a registry layer cache at `:cache`. The build needs the `REGISTRY_USER` and `REGISTRY_TOKEN` (a token with `write:packages`) repository secrets.
`:<sha>` never moves, so it is the rollback point. Images publish to `git.fabledsword.com/bvandeusen/fabledgrimoire`, with a registry layer cache at `:cache`. The build needs the `REGISTRY_USER` and `REGISTRY_TOKEN` (a token with `write:packages`) repository secrets.
Work happens on `dev`. A release merges `dev` into `main` with a plain merge commit; a dated `vYYYY.MM.DD.HHMM` tag, matching the version the image reports, can mark it.
+4 -3
View File
@@ -66,6 +66,7 @@ Saved procedures, such as a drift audit or a DRY pass. The plugin syncs each one
The whole store is reachable through the MCP endpoint at `/mcp`, authenticated with an API key. The Claude Code plugin in this repo adds:
- hooks for session start, each prompt, before and after tool calls, the end of a turn, compaction and session end, all failing open if Grimoire is unreachable;
- a project fence: a session belongs to the project its starting directory is bound to. Toward any other project it can read anything and file an issue. Editing that project's files, committing or pushing in its repo, or changing its records first asks you, in every permission mode;
- skills for the working practices: using Grimoire, writing plans, systematic debugging, verification, brainstorming, reporting back, reusing code, shape accounting and family canon;
- your stored processes as skills.
@@ -85,8 +86,8 @@ Share a project, note or task with users or groups, as viewer, editor or admin.
## Export and backup
- **Personal export**: your notes and tasks as a Markdown ZIP (with YAML frontmatter) or JSON.
- **Admin backup and restore**, with ids remapped so a backup restores onto another instance.
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. `python -m grimoire.restore` brings a fresh instance back from one. See [Backups](backups.md).
- **Admin backup and restore**: a restore rolls the whole instance back to one of its own backups, after saving the current state, or brings a new instance back from any backup.
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. Restoring is in Settings, or `python -m grimoire.restore`. See [Backups](backups.md).
## Sign-in
@@ -102,7 +103,7 @@ Email and password, plus optional OIDC single sign-on (Authentik, Keycloak, ...)
| Integrations | SearXNG, git forge connections |
| Data | Export, backup and restore, scheduled backups (admin) |
| MCP Access | API keys and the command to connect Claude |
| Config (admin) | What's running, application URL, plugin marketplace, database maintenance, SMTP, forge webhook |
| Config (admin) | What's running, reverse proxy count (checked against your own request), application URL, plugin marketplace, database maintenance, SMTP, forge webhook |
| Shared areas (admin) | The canonical areas systems map to |
| Platforms (admin) | The platform catalogue family canon is grouped by |
| Users (admin) | Registration, invites, users |
+1
View File
@@ -7,6 +7,7 @@
"": {
"name": "grimoire-frontend",
"version": "0.1.0",
"license": "AGPL-3.0-only",
"dependencies": {
"@fullcalendar/core": "^6.1.20",
"@fullcalendar/daygrid": "^6.1.20",
+1
View File
@@ -1,6 +1,7 @@
{
"name": "grimoire-frontend",
"private": true,
"license": "AGPL-3.0-only",
"version": "0.1.0",
"type": "module",
"scripts": {
+70 -1
View File
@@ -1,4 +1,4 @@
import { apiGet, apiPost, apiPut } from "@/api/client";
import { apiGet, apiPost, apiPostRaw, apiPut } from "@/api/client";
/**
* Scheduled backups — `/api/admin/backups`. One dated,
@@ -71,3 +71,72 @@ export function checkBackupDestination(
export function runBackupNow(): Promise<{ status: string }> {
return apiPost<{ status: string }>("/api/admin/backups/run", {});
}
// ── Restore: put the whole instance back to a full backup ─────────────────
export interface StoredBackup {
name: string;
size_bytes: number;
}
export interface RestoreCount {
kind: string;
now: number;
after: number;
}
export interface RestorePreview {
/** "fresh": an empty instance takes the backup. "rollback": this instance
* goes back to its own earlier backup. */
mode: "fresh" | "rollback";
exported_at: string | null;
counts: RestoreCount[];
}
export interface RestoreResult extends RestorePreview {
status: "ok";
restored: Record<string, number>;
safety_backup: { file: string; size_bytes: number; where: string; note?: string };
}
/** A backup at the destination, by name, or a file picked from this computer. */
export type RestoreFrom = { source: string } | { file: File };
// Reading, checking and writing the whole store is slow by design; bounded,
// but in minutes.
const RESTORE_TIMEOUT_MS = 10 * 60 * 1000;
export function listRestoreSources(): Promise<{ backups: StoredBackup[]; error?: string }> {
return apiGet("/api/admin/restore/sources", { timeoutMs: 60_000 });
}
async function restoreCall<T>(path: string, from: RestoreFrom, query: Record<string, string> = {}): Promise<T> {
const params = new URLSearchParams(query);
if ("source" in from) {
params.set("source", from.source);
return apiPostRaw<T>(`${path}?${params}`, null, null, { timeoutMs: RESTORE_TIMEOUT_MS });
}
const qs = params.toString() ? `?${params}` : "";
const isTar = from.file.name.toLowerCase().endsWith(".tar");
// The JSON download is parsed first, so a damaged file fails here, before
// the upload.
const body = isTar ? from.file : JSON.stringify(JSON.parse(await from.file.text()));
return apiPostRaw<T>(`${path}${qs}`, body, isTar ? "application/x-tar" : "application/json",
{ timeoutMs: RESTORE_TIMEOUT_MS });
}
export function previewRestore(from: RestoreFrom): Promise<RestorePreview> {
return restoreCall<RestorePreview>("/api/admin/restore/preview", from);
}
export function restoreBackup(from: RestoreFrom): Promise<RestoreResult> {
return restoreCall<RestoreResult>("/api/admin/restore", from, { confirm: "rollback" });
}
/** When a backup named `grimoire-20261015T020000Z.tar` was taken. */
export function backupTakenAt(name: string): Date | null {
const m = /(\d{4})(\d{2})(\d{2})T(\d{2})(\d{2})(\d{2})Z/.exec(name);
if (!m) return null;
return new Date(Date.UTC(+m[1], +m[2] - 1, +m[3], +m[4], +m[5], +m[6]));
}
+9
View File
@@ -156,6 +156,15 @@ export function apiPost<T>(path: string, body: unknown, opts?: RequestOpts): Pro
return request<T>(path, { method: "POST", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
}
/** A POST whose body is sent as it is (a file, or text already serialised),
* under the same deadline and error handling as every other call. */
export function apiPostRaw<T>(
path: string, body: BodyInit | null, contentType: string | null, opts?: RequestOpts,
): Promise<T> {
const headers = contentType ? { "Content-Type": contentType } : undefined;
return request<T>(path, { method: "POST", headers, body }, opts);
}
export function apiPut<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
return request<T>(path, { method: "PUT", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
}
+10
View File
@@ -44,6 +44,14 @@ export interface RemovedDefault {
created_at: string | null;
}
export interface MomentRuleUsage {
rule_id: number;
/** Null for a rule deleted since it was delivered. */
title: string | null;
delivered: number;
opened: boolean;
}
export interface MomentUsage {
/** How many times a mounted rule arrived at this moment. */
delivered: number;
@@ -53,6 +61,8 @@ export interface MomentUsage {
* the door, not the line that prompted it. */
opened: number;
last_delivered_at: string | null;
/** Each rule delivered here, most delivered first. */
by_rule: MomentRuleUsage[];
}
export interface MomentsPayload {
+18 -40
View File
@@ -3,6 +3,8 @@ defineProps<{ size?: number }>();
</script>
<template>
<!-- The clasped grimoire, the same drawing as public/favicon.svg. Its colours
are the mark's own and stay fixed in both themes, like the tab icon. -->
<svg
class="app-logo"
xmlns="http://www.w3.org/2000/svg"
@@ -11,45 +13,21 @@ defineProps<{ size?: number }>();
:height="size ?? 24"
aria-hidden="true"
>
<defs>
<linearGradient id="logo-gradient" x1="0" y1="0" x2="1" y2="1">
<stop offset="0%" stop-color="var(--fs-accent)" />
<stop offset="100%" stop-color="var(--fs-accent-deep)" />
</linearGradient>
</defs>
<!-- Book body -->
<path class="logo-book" d="M4 7 C4 7 8 5 16 6 C24 5 28 7 28 7 L28 26 C28 26 24 24 16 25 C8 24 4 26 4 26 Z" stroke-width="0.5"/>
<!-- Book spine -->
<line class="logo-spine" x1="16" y1="6" x2="16" y2="25" stroke-width="0.8"/>
<!-- Left page lines -->
<line class="logo-lines" x1="7" y1="11" x2="14" y2="10.5" stroke-width="0.6" stroke-linecap="round"/>
<line class="logo-lines" x1="7" y1="14" x2="14" y2="13.5" stroke-width="0.6" stroke-linecap="round"/>
<line class="logo-lines" x1="7" y1="17" x2="14" y2="16.5" stroke-width="0.6" stroke-linecap="round"/>
<line class="logo-lines" x1="7" y1="20" x2="12" y2="19.5" stroke-width="0.6" stroke-linecap="round"/>
<!-- Right page lines -->
<line class="logo-lines" x1="18" y1="10.5" x2="25" y2="11" stroke-width="0.6" stroke-linecap="round"/>
<line class="logo-lines" x1="18" y1="13.5" x2="25" y2="14" stroke-width="0.6" stroke-linecap="round"/>
<line class="logo-lines" x1="18" y1="16.5" x2="25" y2="17" stroke-width="0.6" stroke-linecap="round"/>
<!-- Magic spark -->
<g transform="translate(24, 5)">
<path d="M0 -4 L1 -1 L4 0 L1 1 L0 4 L-1 1 L-4 0 L-1 -1 Z" fill="#f6ad55"/>
<path d="M0 -2.5 L0.6 -0.6 L2.5 0 L0.6 0.6 L0 2.5 L-0.6 0.6 L-2.5 0 L-0.6 -0.6 Z" fill="#fbd38d"/>
</g>
<!-- Secondary sparkles -->
<circle cx="20" cy="3" r="0.7" fill="#f6ad55" opacity="0.7"/>
<circle cx="27" cy="8" r="0.5" fill="#fbd38d" opacity="0.6"/>
<!-- page block, seen at the fore-edge -->
<path d="M8.5 4.5 H25.5 Q27 4.5 27 6 V27 Q27 28.5 25.5 28.5 H8.5 Z" fill="#E9DFC6"/>
<path d="M25 7 V26.5" stroke="#C9BB97" stroke-width="0.6"/>
<!-- cover -->
<path d="M5 4 Q5 3 6 3 H24 Q25.5 3 25.5 4.5 V26.5 Q25.5 28 24 28 H6 Q5 28 5 27 Z" fill="#5B4A8A"/>
<!-- spine band -->
<rect x="5" y="3" width="3.2" height="25" rx="0.6" fill="#47396E"/>
<!-- corner fittings -->
<path d="M21.5 3 H24 Q25.5 3 25.5 4.5 V7 Z" fill="#D4A94E"/>
<path d="M21.5 28 H24 Q25.5 28 25.5 26.5 V24 Z" fill="#D4A94E"/>
<!-- clasp across the fore-edge -->
<rect x="22.5" y="14" width="5.5" height="3.6" rx="0.8" fill="#D4A94E"/>
<circle cx="26.4" cy="15.8" r="0.7" fill="#8A6A22"/>
<!-- sigil: a four-pointed star in a ring -->
<circle cx="15.5" cy="15.5" r="5.2" fill="none" stroke="#D4A94E" stroke-width="0.9"/>
<path d="M15.5 11.4 L16.5 14.5 L19.6 15.5 L16.5 16.5 L15.5 19.6 L14.5 16.5 L11.4 15.5 L14.5 14.5 Z" fill="#F1D48A"/>
</svg>
</template>
<style scoped>
.logo-book {
fill: url(#logo-gradient);
stroke: color-mix(in srgb, var(--fs-accent) 70%, transparent);
}
.logo-spine {
stroke: var(--fs-text-secondary);
}
.logo-lines {
stroke: var(--fs-text-tertiary);
}
</style>
-466
View File
@@ -1,466 +0,0 @@
<script setup lang="ts">
import { ref, computed, onMounted } from "vue";
import { apiGet, pinNoteVersion, unpinNoteVersion } from "@/api/client";
import DiffView from "@/components/DiffView.vue";
import { computeDiff, type DiffLine } from "@/utils/diff";
import { fmtStamp } from "@/utils/dateFormat";
interface NoteVersion {
id: number;
note_id: number;
title: string;
tags: string[];
body?: string;
pin_kind: "auto" | "manual" | null;
pin_label: string | null;
created_at: string;
}
const props = defineProps<{
noteId: number;
currentBody: string;
}>();
const emit = defineEmits<{
(e: 'restore', body: string, tags: string[]): void;
(e: 'close'): void;
}>();
const versions = ref<NoteVersion[]>([]);
const selectedVersion = ref<NoteVersion | null>(null);
const loading = ref(false);
const loadingDetail = ref(false);
const diff = computed<DiffLine[]>(() => {
if (!selectedVersion.value?.body) return [];
return computeDiff(props.currentBody, selectedVersion.value.body);
});
async function loadVersions() {
loading.value = true;
try {
const data = await apiGet<{ versions: NoteVersion[] }>(
`/api/notes/${props.noteId}/versions`
);
versions.value = data.versions;
if (versions.value.length > 0) {
await selectVersionItem(versions.value[0]);
}
} catch {
// silent
} finally {
loading.value = false;
}
}
async function selectVersionItem(v: NoteVersion) {
if (v.body !== undefined) {
selectedVersion.value = v;
return;
}
loadingDetail.value = true;
try {
const full = await apiGet<NoteVersion>(
`/api/notes/${props.noteId}/versions/${v.id}`
);
// Cache body back onto list item
const listItem = versions.value.find(x => x.id === v.id);
if (listItem) listItem.body = full.body;
selectedVersion.value = full;
} catch {
// silent
} finally {
loadingDetail.value = false;
}
}
function restore() {
if (!selectedVersion.value?.body) return;
emit('restore', selectedVersion.value.body, selectedVersion.value.tags ?? []);
}
// ── Pin / unpin / edit-label state ──────────────────────────────────────────
const editingLabel = ref(false);
const draftLabel = ref("");
const pinSaving = ref(false);
function startEditLabel() {
if (!selectedVersion.value) return;
draftLabel.value = selectedVersion.value.pin_label ?? "";
editingLabel.value = true;
}
function cancelEditLabel() {
editingLabel.value = false;
draftLabel.value = "";
}
async function savePin() {
if (!selectedVersion.value || pinSaving.value) return;
pinSaving.value = true;
try {
const updated = await pinNoteVersion(
props.noteId, selectedVersion.value.id, draftLabel.value || null,
);
// Reflect server state in the local list + selected version.
const idx = versions.value.findIndex(v => v.id === updated.id);
if (idx >= 0) {
versions.value[idx].pin_kind = updated.pin_kind;
versions.value[idx].pin_label = updated.pin_label;
}
if (selectedVersion.value) {
selectedVersion.value.pin_kind = updated.pin_kind;
selectedVersion.value.pin_label = updated.pin_label;
}
editingLabel.value = false;
} catch {
// silent — leaving the form open lets the user retry
} finally {
pinSaving.value = false;
}
}
async function unpinSelected() {
if (!selectedVersion.value || pinSaving.value) return;
pinSaving.value = true;
try {
await unpinNoteVersion(props.noteId, selectedVersion.value.id);
const idx = versions.value.findIndex(v => v.id === selectedVersion.value!.id);
if (idx >= 0) {
versions.value[idx].pin_kind = null;
versions.value[idx].pin_label = null;
}
if (selectedVersion.value) {
selectedVersion.value.pin_kind = null;
selectedVersion.value.pin_label = null;
}
} catch {
// silent
} finally {
pinSaving.value = false;
}
}
onMounted(loadVersions);
</script>
<template>
<teleport to="body">
<div class="modal-overlay" @click.self="emit('close')">
<div class="modal-card history-modal">
<div class="history-header">
<h3 class="history-title">Version History</h3>
<button class="history-close" aria-label="Close version history" @click="emit('close')">&times;</button>
</div>
<div class="history-body">
<!-- Left: version list -->
<div class="history-list">
<div v-if="loading" class="history-empty">Loading...</div>
<div v-else-if="!versions.length" class="history-empty">No versions yet.</div>
<div
v-for="v in versions"
:key="v.id"
:class="['history-item', { selected: selectedVersion?.id === v.id }]"
@click="selectVersionItem(v)"
>
<div class="history-item-title">
<span
v-if="v.pin_kind === 'manual'"
class="pin-badge pin-badge-manual"
:title="v.pin_label || 'Pinned'"
aria-label="manually pinned"
>●</span>
<span
v-else-if="v.pin_kind === 'auto'"
class="pin-badge pin-badge-auto"
:title="v.pin_label || 'Auto-pinned (stable)'"
aria-label="auto-pinned"
>◐</span>
{{ v.title || 'Untitled' }}
</div>
<div
v-if="v.pin_kind === 'manual' && v.pin_label"
class="history-item-label"
>{{ v.pin_label }}</div>
<div class="history-item-date">{{ fmtStamp(v.created_at) }}</div>
</div>
</div>
<!-- Right: diff -->
<div class="history-diff">
<div v-if="loadingDetail" class="history-empty">Loading version...</div>
<div v-else-if="!selectedVersion" class="history-empty">Select a version to compare.</div>
<template v-else>
<div class="version-pin-controls">
<!-- Label editor -->
<div v-if="editingLabel" class="pin-label-form">
<input
v-model="draftLabel"
maxlength="500"
type="text"
class="pin-label-input"
placeholder="Optional label (e.g. 'post-network-refresh runbook')"
/>
<button class="btn-pin-save" :disabled="pinSaving" @click="savePin">
{{ pinSaving ? "Saving…" : "Save" }}
</button>
<button class="btn-pin-cancel" @click="cancelEditLabel">Cancel</button>
</div>
<!-- Default: kind-aware action row -->
<div v-else class="pin-actions">
<span v-if="selectedVersion.pin_kind === 'manual'" class="pin-state">
<span class="pin-badge pin-badge-manual">●</span>
Manually pinned{{ selectedVersion.pin_label ? `: ${selectedVersion.pin_label}` : '' }}
</span>
<span v-else-if="selectedVersion.pin_kind === 'auto'" class="pin-state">
<span class="pin-badge pin-badge-auto">◐</span>
Auto-pinned{{ selectedVersion.pin_label ? `: ${selectedVersion.pin_label}` : '' }}
</span>
<button
v-if="selectedVersion.pin_kind === null"
class="btn-pin"
:disabled="pinSaving"
@click="startEditLabel"
>Pin version</button>
<button
v-else-if="selectedVersion.pin_kind === 'manual'"
class="btn-pin-edit"
:disabled="pinSaving"
@click="startEditLabel"
>Edit label</button>
<button
v-else
class="btn-pin"
:disabled="pinSaving"
@click="startEditLabel"
>Pin permanently</button>
<button
v-if="selectedVersion.pin_kind === 'manual'"
class="btn-unpin"
:disabled="pinSaving"
@click="unpinSelected"
>Unpin</button>
</div>
</div>
<DiffView :diff="diff" />
</template>
</div>
</div>
<div class="history-footer">
<button
class="btn-primary"
:disabled="!selectedVersion?.body"
@click="restore"
>Restore this version</button>
<button class="modal-btn" @click="emit('close')">Close</button>
</div>
</div>
</div>
</teleport>
</template>
<style scoped>
.history-modal {
max-width: 900px;
width: 95%;
height: 80vh;
max-height: 700px;
display: flex;
flex-direction: column;
padding: 0;
}
.history-header {
flex-shrink: 0;
display: flex;
align-items: center;
justify-content: space-between;
padding: 0.9rem 1.25rem;
border-bottom: 1px solid var(--fs-border-color);
}
.history-title {
margin: 0;
font-size: 1rem;
}
.history-close {
background: none;
border: none;
font-size: 1.25rem;
cursor: pointer;
color: var(--fs-text-tertiary);
line-height: 1;
padding: 0.1rem 0.3rem;
}
.history-close:hover { color: var(--fs-text-primary); }
.history-body {
flex: 1;
min-height: 0;
display: flex;
overflow: hidden;
}
.history-list {
width: 220px;
flex-shrink: 0;
border-right: 1px solid var(--fs-border-color);
overflow-y: auto;
}
.history-item {
padding: 0.6rem 0.9rem;
cursor: pointer;
border-left: 3px solid transparent;
border-bottom: 1px solid var(--fs-border-color);
}
.history-item:hover { background: var(--fs-surface-raised); }
.history-item.selected {
border-left-color: var(--fs-accent);
background: var(--fs-surface-raised);
}
.history-item-title {
font-size: 0.85rem;
font-weight: 500;
color: var(--fs-text-primary);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.history-item-date {
font-size: 0.75rem;
color: var(--fs-text-tertiary);
margin-top: 0.15rem;
}
.history-diff {
flex: 1;
min-width: 0;
overflow: hidden;
display: flex;
flex-direction: column;
padding: 0.5rem;
}
.history-empty {
padding: 1rem;
font-size: 0.85rem;
color: var(--fs-text-tertiary);
}
.history-footer {
flex-shrink: 0;
display: flex;
gap: 0.5rem;
justify-content: flex-end;
padding: 0.75rem 1.25rem;
border-top: 1px solid var(--fs-border-color);
}
/* ── Pin badges + label rendering ───────────────────────────────────────── */
.pin-badge {
display: inline-block;
width: 0.7rem;
text-align: center;
margin-right: 0.35rem;
font-size: 0.85em;
line-height: 1;
}
.pin-badge-manual { color: var(--fs-accent); }
.pin-badge-auto { color: var(--fs-text-tertiary); }
.history-item-label {
font-size: 0.72rem;
color: var(--fs-accent);
font-style: italic;
margin-top: 0.15rem;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
/* ── Pin controls above the diff ────────────────────────────────────────── */
.version-pin-controls {
padding: 0.4rem 0.5rem 0.5rem;
border-bottom: 1px solid var(--fs-border-color);
font-size: 0.82rem;
}
.pin-actions {
display: flex;
align-items: center;
gap: 0.5rem;
flex-wrap: wrap;
}
.pin-state {
font-style: italic;
color: var(--fs-text-tertiary);
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.btn-pin, .btn-pin-edit, .btn-unpin {
padding: 0.25rem 0.7rem;
font-size: 0.78rem;
background: transparent;
color: inherit;
border: 1px solid var(--fs-border-color);
border-radius: 999px;
cursor: pointer;
}
.btn-pin:hover:not(:disabled), .btn-pin-edit:hover:not(:disabled) {
background: rgba(99, 102, 241, 0.12);
border-color: var(--fs-accent);
}
.btn-unpin:hover:not(:disabled) {
background: rgba(239, 68, 68, 0.10);
border-color: rgba(239, 68, 68, 0.5);
}
.pin-label-form {
display: flex;
gap: 0.4rem;
align-items: center;
}
.pin-label-input {
flex: 1;
padding: 0.3rem 0.5rem;
font-size: 0.85rem;
background: var(--fs-surface-page);
border: 1px solid var(--fs-border-color);
border-radius: var(--fs-radius-sm);
color: inherit;
}
.pin-label-input:focus {
outline: none;
border-color: var(--fs-accent);
}
.btn-pin-save, .btn-pin-cancel {
padding: 0.3rem 0.7rem;
font-size: 0.78rem;
background: transparent;
color: inherit;
border: 1px solid var(--fs-border-color);
border-radius: var(--fs-radius-sm);
cursor: pointer;
}
.btn-pin-save:hover:not(:disabled) {
background: rgba(99, 102, 241, 0.12);
border-color: var(--fs-accent);
}
.btn-pin-save:disabled, .btn-pin-cancel:disabled,
.btn-pin:disabled, .btn-pin-edit:disabled, .btn-unpin:disabled {
opacity: 0.5;
cursor: progress;
}
</style>
@@ -1,258 +0,0 @@
<script setup lang="ts">
import { ref, computed } from "vue";
import { renderMarkdown } from "@/utils/markdown";
import type { DiffLine } from "@/composables/useAssist";
const props = defineProps<{
phase: "streaming" | "review";
label: string;
streamingText: string;
diff: DiffLine[];
proposedText: string;
}>();
const emit = defineEmits<{
accept: [];
reject: [];
cancel: [];
}>();
const showFullText = ref(false);
const renderedStreaming = computed(() => renderMarkdown(props.streamingText));
const renderedProposed = computed(() => renderMarkdown(props.proposedText));
const markers: Record<DiffLine["type"], string> = {
equal: " ",
delete: "−",
insert: "+",
};
</script>
<template>
<!-- ── Streaming ───────────────────────────────────────────────── -->
<div v-if="phase === 'streaming'" class="iap iap-streaming">
<div class="iap-header">
<span class="iap-pulse"></span>
<span class="iap-label">{{ label }}</span>
<button class="iap-btn-cancel" @click="emit('cancel')">Cancel</button>
</div>
<div class="iap-stream-preview prose" v-html="renderedStreaming"></div>
<div v-if="!streamingText" class="iap-waiting">Waiting for model…</div>
</div>
<!-- ── Review ─────────────────────────────────────────────────── -->
<div v-else-if="phase === 'review'" class="iap iap-review">
<div class="iap-header">
<span class="iap-review-title">Proposed changes</span>
<button class="iap-btn-toggle" @click="showFullText = !showFullText">
{{ showFullText ? "Show diff" : "Show full text" }}
</button>
<div class="iap-actions">
<button class="iap-btn-accept" @click="emit('accept')">✓ Accept</button>
<button class="iap-btn-reject" @click="emit('reject')">✗ Reject</button>
</div>
</div>
<!-- Diff view -->
<div v-if="!showFullText" class="iap-diff">
<div
v-for="(line, i) in diff"
:key="i"
:class="['iap-diff-line', `iap-diff-${line.type}`]"
>
<span class="iap-diff-marker">{{ markers[line.type] }}</span>
<span class="iap-diff-text">{{ line.text }}</span>
</div>
<div v-if="!diff.length" class="iap-diff-empty">No changes.</div>
</div>
<!-- Full text view -->
<div v-else class="iap-full-text prose" v-html="renderedProposed"></div>
</div>
</template>
<style scoped>
.iap {
border-radius: var(--fs-radius-sm);
margin-bottom: 0.75rem;
overflow: hidden;
border: 1px solid var(--fs-border-color);
background: var(--fs-surface-page);
}
/* ── Header ── */
.iap-header {
display: flex;
align-items: center;
gap: 0.5rem;
padding: 0.45rem 0.75rem;
font-size: 0.85rem;
border-bottom: 1px solid var(--fs-border-color);
background: var(--fs-surface-raised);
}
/* ── Streaming ── */
.iap-streaming {
border-color: var(--fs-accent);
}
.iap-streaming .iap-header {
background: color-mix(in srgb, var(--fs-accent) 8%, var(--fs-surface-raised));
}
.iap-pulse {
display: inline-block;
width: 8px;
height: 8px;
border-radius: 50%;
background: var(--fs-accent);
flex-shrink: 0;
animation: iap-pulse 1.2s ease-in-out infinite;
}
@keyframes iap-pulse {
0%, 100% { opacity: 1; transform: scale(1); }
50% { opacity: 0.4; transform: scale(0.8); }
}
.iap-label {
flex: 1;
font-weight: 500;
color: var(--fs-text-primary);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.iap-btn-cancel {
background: none;
border: 1px solid var(--fs-border-color);
color: var(--fs-text-secondary);
border-radius: var(--fs-radius-sm);
padding: 0.15rem 0.5rem;
cursor: pointer;
font-size: 0.8rem;
font-family: inherit;
flex-shrink: 0;
}
.iap-btn-cancel:hover {
border-color: var(--fs-error);
color: var(--fs-error);
}
.iap-stream-preview {
padding: 0.75rem;
font-size: 0.9rem;
min-height: 2.5rem;
max-height: 320px;
overflow-y: auto;
}
.iap-waiting {
padding: 0.75rem;
font-size: 0.85rem;
color: var(--fs-text-tertiary);
}
/* ── Review ── */
.iap-review-title {
flex: 1;
font-weight: 600;
color: var(--fs-text-primary);
}
.iap-btn-toggle {
background: none;
border: 1px solid var(--fs-border-color);
color: var(--fs-text-secondary);
border-radius: var(--fs-radius-sm);
padding: 0.15rem 0.5rem;
cursor: pointer;
font-size: 0.78rem;
font-family: inherit;
}
.iap-btn-toggle:hover {
border-color: var(--fs-accent);
color: var(--fs-accent);
}
.iap-actions {
display: flex;
gap: 0.35rem;
}
.iap-btn-accept,
.iap-btn-reject {
border: none;
border-radius: var(--fs-radius-sm);
padding: 0.2rem 0.65rem;
cursor: pointer;
font-size: 0.8rem;
font-family: inherit;
font-weight: var(--fs-weight-medium);
}
.iap-btn-accept {
background: var(--fs-success);
color: var(--fs-text-on-action);
}
.iap-btn-accept:hover { opacity: 0.85; }
.iap-btn-reject {
background: var(--fs-surface-raised);
color: var(--fs-text-secondary);
border: 1px solid var(--fs-border-color);
}
.iap-btn-reject:hover {
border-color: var(--fs-error);
color: var(--fs-error);
}
/* ── Diff ── */
.iap-diff {
font-family: ui-monospace, "Cascadia Code", "Fira Mono", monospace;
font-size: 0.82rem;
line-height: 1.6;
max-height: 440px;
overflow-y: auto;
padding: 0.4rem 0;
}
.iap-diff-line {
display: flex;
gap: 0.5rem;
padding: 0 0.75rem;
white-space: pre-wrap;
word-break: break-word;
}
.iap-diff-equal { color: var(--fs-text-tertiary); }
.iap-diff-delete {
background: color-mix(in srgb, var(--fs-error) 10%, transparent);
color: var(--fs-error-fg);
}
.iap-diff-insert {
background: color-mix(in srgb, var(--fs-success) 10%, transparent);
color: var(--fs-success-fg);
}
.iap-diff-marker {
width: 1ch;
flex-shrink: 0;
font-weight: 700;
user-select: none;
}
.iap-diff-empty {
padding: 0.75rem;
color: var(--fs-text-tertiary);
font-size: 0.85rem;
font-family: inherit;
}
/* ── Full text ── */
.iap-full-text {
padding: 0.75rem;
font-size: 0.9rem;
max-height: 440px;
overflow-y: auto;
}
</style>
@@ -0,0 +1,37 @@
<script setup lang="ts">
import type { MomentRuleUsage } from "@/api/moments";
/**
* Which rules a moment delivered, and whether an agent went on to read each.
* The moment's own counts cannot say this once two rules are mounted there.
* Closed by default: the counts in the row above are what most visits want.
*/
defineProps<{ rules: MomentRuleUsage[] }>();
</script>
<template>
<details v-if="rules.length" class="moment-rules">
<summary>Rules delivered here</summary>
<ul class="delivered-list">
<li v-for="r in rules" :key="r.rule_id" class="delivered-row">
<span class="delivered-title">{{ r.title ?? `Rule ${r.rule_id} (deleted)` }}</span>
<span class="delivered-count">delivered {{ r.delivered }}</span>
<span :class="r.opened ? 'delivered-opened' : 'delivered-unopened'">
{{ r.opened ? "opened" : "not opened" }}
</span>
</li>
</ul>
</details>
</template>
<style scoped>
.moment-rules { margin-top: var(--fs-space-2); font-size: var(--fs-size-tiny); }
.moment-rules summary { cursor: pointer; color: var(--fs-text-tertiary); }
.delivered-list { list-style: none; margin: var(--fs-space-1) 0 0; padding: 0; display: flex; flex-direction: column; gap: var(--fs-space-1); }
.delivered-row { display: flex; flex-wrap: wrap; align-items: baseline; gap: var(--fs-space-2); font-variant-numeric: tabular-nums; }
.delivered-title { color: var(--fs-text-primary); font-size: var(--fs-size-body-sm); }
.delivered-count { color: var(--fs-text-secondary); }
.delivered-opened { color: var(--fs-text-secondary); }
/* Not read is a fact about this window, not a fault: quiet, like "nothing mounted". */
.delivered-unopened { color: var(--fs-text-tertiary); font-style: italic; }
</style>
@@ -2,6 +2,7 @@
import { computed, onMounted, ref } from "vue";
import { useMomentsStore } from "@/stores/moments";
import MomentProposals from "@/components/MomentProposals.vue";
import MomentRuleUsage from "@/components/MomentRuleUsage.vue";
import type { MomentAction, MomentUsage } from "@/api/moments";
import { fmtDate } from "@/utils/dateFormat";
@@ -167,6 +168,7 @@ onMounted(() => store.load(true));
>×</button>
</span>
</div>
<MomentRuleUsage v-if="usage(m.name)" :rules="usage(m.name)!.by_rule" />
</li>
</ul>
@@ -186,6 +188,7 @@ onMounted(() => store.load(true));
</template>
</span>
</div>
<MomentRuleUsage v-if="usage(name)" :rules="usage(name)!.by_rule" />
</li>
</ul>
</template>
-168
View File
@@ -1,168 +0,0 @@
<script setup lang="ts">
import { useRouter } from "vue-router";
import type { Note } from "@/types/note";
import TagPill from "@/components/TagPill.vue";
import { relativeTime } from "@/composables/useRelativeTime";
import { renderPreview } from "@/utils/markdown";
const props = defineProps<{ note: Note; compact?: boolean }>();
const emit = defineEmits<{ "tag-click": [tag: string] }>();
const router = useRouter();
function onTagClick(tag: string) {
emit("tag-click", tag);
}
function goEdit() {
router.push(`/notes/${props.note.id}/edit`);
}
</script>
<template>
<router-link :to="`/notes/${note.id}`" :class="['note-card', { compact }]">
<!-- Compact: single row -->
<template v-if="compact">
<span class="note-title-compact">{{ note.title || "Untitled" }}</span>
<div class="note-tags-compact">
<TagPill
v-for="tag in note.tags.slice(0, 3)"
:key="tag"
:tag="tag"
@click.stop="onTagClick(tag)"
/>
</div>
<span class="timestamp-compact">{{ relativeTime(note.updated_at) }}</span>
</template>
<!-- Full: original layout -->
<template v-else>
<div class="note-top">
<h3 class="note-title">{{ note.title || "Untitled" }}</h3>
<button class="btn-edit" title="Edit" @click.prevent.stop="goEdit">Edit</button>
</div>
<div v-if="note.body" class="note-preview prose" v-html="renderPreview(note.body)"></div>
<div class="note-meta">
<TagPill
v-for="tag in note.tags"
:key="tag"
:tag="tag"
@click.stop="onTagClick(tag)"
/>
<span class="timestamp">{{ relativeTime(note.updated_at) }}</span>
</div>
</template>
</router-link>
</template>
<style scoped>
.note-card {
display: block;
padding: 1rem;
border-radius: var(--fs-radius-lg);
text-decoration: none;
color: inherit;
background: var(--fs-surface-raised);
box-shadow: 0 1px 4px rgba(0, 0, 0, 0.06), 0 0 0 1px color-mix(in srgb, var(--fs-accent) 6%, transparent);
transition: box-shadow 0.2s, transform 0.18s ease;
}
.note-card:hover {
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.10), 0 0 0 1px color-mix(in srgb, var(--fs-accent) 14.0%, transparent);
transform: translateY(-2px);
}
/* Compact single-row layout */
.note-card.compact {
display: flex;
align-items: center;
gap: 0.6rem;
padding: 0.45rem 0.75rem;
background: var(--fs-surface-raised);
box-shadow: none;
border-radius: 0;
border-bottom: 1px solid var(--fs-border-color);
transform: none !important;
}
.note-card.compact:first-child {
border-top: 1px solid var(--fs-border-color);
}
.note-card.compact:hover {
box-shadow: none;
background: color-mix(in srgb, var(--fs-accent) 4%, transparent);
transform: none;
}
.note-title-compact {
font-size: 0.9rem;
font-weight: 500;
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.note-tags-compact {
display: flex;
gap: 0.3rem;
flex-shrink: 0;
}
.timestamp-compact {
font-size: 0.72rem;
color: var(--fs-text-tertiary);
flex-shrink: 0;
white-space: nowrap;
}
/* Full layout */
.note-top {
display: flex;
align-items: center;
justify-content: space-between;
gap: 0.5rem;
margin-bottom: 0.25rem;
}
.note-title {
margin: 0;
font-size: 1.1rem;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.btn-edit {
flex-shrink: 0;
padding: 0.25rem 0.6rem;
font-size: 0.8rem;
background: var(--fs-surface-raised);
color: var(--fs-text-secondary);
border: 1px solid var(--fs-border-color);
border-radius: var(--fs-radius-sm);
cursor: pointer;
transition: color 0.15s, border-color 0.15s;
}
.btn-edit:hover {
color: var(--fs-accent);
border-color: var(--fs-accent);
}
.note-preview {
margin: 0 0 0.5rem;
color: var(--fs-text-secondary);
font-size: 0.9rem;
max-height: 7.5em;
overflow: hidden;
position: relative;
-webkit-mask-image: linear-gradient(to bottom, black 60%, transparent 100%);
mask-image: linear-gradient(to bottom, black 60%, transparent 100%);
}
.note-meta {
display: flex;
align-items: center;
gap: 0.5rem;
flex-wrap: wrap;
}
.timestamp {
margin-left: auto;
font-size: 0.75rem;
color: var(--fs-text-tertiary);
}
</style>
@@ -0,0 +1,278 @@
<script setup lang="ts">
/**
* Restore: put the whole instance back to a full backup.
*
* Choose a backup (one at the backup destination, or a file from this
* computer), see what it would change, type the confirmation, run it. On a
* live instance this is a rollback to the instance's own earlier backup; on a
* new, empty one it takes the backup in. Either way the server backs up the
* current state first and signs everyone out afterwards.
*/
import { computed, onMounted, ref } from "vue";
import { apiErrorMessage } from "@/api/client";
import {
backupTakenAt, listRestoreSources, previewRestore, restoreBackup,
type RestoreFrom, type RestorePreview, type RestoreResult, type StoredBackup,
} from "@/api/backups";
import { formatBytes } from "@/utils/bytes";
const emit = defineEmits<{ close: [] }>();
const CONFIRM_WORDS = "roll back";
type Step = "choose" | "checking" | "preview" | "running" | "done";
const step = ref<Step>("choose");
const sources = ref<StoredBackup[]>([]);
const sourcesError = ref("");
const sourcesLoading = ref(true);
const chosen = ref<RestoreFrom | null>(null);
const chosenLabel = ref("");
const preview = ref<RestorePreview | null>(null);
const result = ref<RestoreResult | null>(null);
const error = ref("");
const typed = ref("");
const fileInput = ref<HTMLInputElement | null>(null);
const confirmed = computed(() => typed.value.trim().toLowerCase() === CONFIRM_WORDS);
function when(value: Date | string | null | undefined): string {
if (!value) return "an unknown date";
const date = typeof value === "string" ? new Date(value) : value;
return Number.isNaN(date.getTime()) ? "an unknown date" : date.toLocaleString();
}
onMounted(async () => {
try {
const answer = await listRestoreSources();
sources.value = answer.backups;
sourcesError.value = answer.error ?? "";
} catch (e) {
sourcesError.value = apiErrorMessage(e, "Could not list the backups at the destination.");
} finally {
sourcesLoading.value = false;
}
});
async function check(from: RestoreFrom, label: string) {
chosen.value = from;
chosenLabel.value = label;
error.value = "";
typed.value = "";
step.value = "checking";
try {
preview.value = await previewRestore(from);
step.value = "preview";
} catch (e) {
error.value = apiErrorMessage(e, "That backup could not be read.");
step.value = "choose";
}
}
function pickFile(event: Event) {
const file = (event.target as HTMLInputElement).files?.[0];
if (fileInput.value) fileInput.value.value = "";
if (file) check({ file }, file.name);
}
async function run() {
if (!chosen.value || !confirmed.value) return;
error.value = "";
step.value = "running";
try {
result.value = await restoreBackup(chosen.value);
step.value = "done";
} catch (e) {
error.value = apiErrorMessage(e, "The restore failed. Nothing was changed.");
step.value = "preview";
}
}
function finish() {
// Every session ended with the restore, this one included.
window.location.assign("/login");
}
</script>
<template>
<teleport to="body">
<div class="modal-overlay" @click="step === 'running' || step === 'done' ? null : emit('close')">
<div class="modal-card restore-card" @click.stop>
<template v-if="step === 'choose' || step === 'checking'">
<h3 class="modal-title">Restore from a backup</h3>
<p class="modal-message">
Puts the whole instance back to the moment the backup was taken, for every user.
You'll see what changes before anything happens.
</p>
<p v-if="error" class="restore-error" role="alert">{{ error }}</p>
<h4 class="restore-subhead">At the backup destination</h4>
<p v-if="sourcesLoading" class="restore-muted">Looking for backups…</p>
<p v-else-if="sourcesError" class="restore-muted">{{ sourcesError }}</p>
<p v-else-if="!sources.length" class="restore-muted">No backups there yet.</p>
<ul v-else class="restore-sources">
<li v-for="b in sources" :key="b.name">
<button
class="restore-source"
:disabled="step === 'checking'"
@click="check({ source: b.name }, when(backupTakenAt(b.name)))"
>
<span>{{ when(backupTakenAt(b.name)) }}</span>
<span class="restore-muted">{{ formatBytes(b.size_bytes) }}</span>
</button>
</li>
</ul>
<h4 class="restore-subhead">From this computer</h4>
<p class="restore-muted">
A backup file (.tar) or the Full Backup download (.json). Uploads are size-limited;
a large file restores from the destination above, or with
<code>python -m grimoire.restore</code>.
</p>
<div class="modal-actions restore-actions-left">
<button class="modal-btn" :disabled="step === 'checking'" @click="fileInput?.click()">
Choose a file…
</button>
<input ref="fileInput" type="file" accept=".json,.tar" class="hidden-file-input" @change="pickFile" />
</div>
<p v-if="step === 'checking'" class="restore-muted">Reading {{ chosenLabel }}…</p>
<div class="modal-actions">
<button class="modal-btn" @click="emit('close')">Cancel</button>
</div>
</template>
<template v-else-if="(step === 'preview' || step === 'running') && preview">
<h3 class="modal-title">
{{ preview.mode === "rollback" ? "Roll back to this backup?" : "Restore this backup?" }}
</h3>
<p class="modal-message">
<template v-if="preview.mode === 'rollback'">
Everything goes back to {{ when(preview.exported_at) }}, for every user. Anything made
since is removed, edits since are undone, and records deleted since come back.
</template>
<template v-else>
This instance has no records yet, so it takes the backup from {{ when(preview.exported_at) }} as it is.
</template>
The current state is backed up first, and everyone is signed out afterwards.
</p>
<table class="restore-counts">
<thead><tr><th></th><th>Now</th><th>After</th></tr></thead>
<tbody>
<tr v-for="c in preview.counts" :key="c.kind">
<td>{{ c.kind }}</td><td>{{ c.now }}</td><td>{{ c.after }}</td>
</tr>
</tbody>
</table>
<p v-if="error" class="restore-error" role="alert">{{ error }}</p>
<label class="restore-confirm">
Type <strong>{{ CONFIRM_WORDS }}</strong> to continue
<input v-model="typed" class="fs-input input" :disabled="step === 'running'" autocomplete="off" />
</label>
<div class="modal-actions">
<button class="modal-btn" :disabled="step === 'running'" @click="step = 'choose'">Back</button>
<button
class="modal-btn modal-btn-danger"
:disabled="!confirmed || step === 'running'"
@click="run"
>{{ step === "running" ? "Restoring…" : preview.mode === "rollback" ? "Roll back" : "Restore" }}</button>
</div>
</template>
<template v-else-if="step === 'done' && result">
<h3 class="modal-title">Restored</h3>
<p class="modal-message">
The instance is back to {{ when(result.exported_at) }}.
The state it replaced was saved as <code>{{ result.safety_backup.file }}</code>
in {{ result.safety_backup.where }}.
<template v-if="result.safety_backup.note"> {{ result.safety_backup.note }}</template>
Search is being rebuilt in the background. Everyone has been signed out, so sign in again.
</p>
<div class="modal-actions">
<button class="modal-btn modal-btn-primary" @click="finish">Sign in again</button>
</div>
</template>
</div>
</div>
</teleport>
</template>
<style scoped>
.restore-card {
max-width: 34rem;
max-height: 85vh;
overflow-y: auto;
}
.restore-subhead {
margin: 1rem 0 0.4rem;
font-size: 0.9rem;
}
.restore-muted {
font-size: 0.85rem;
color: var(--fs-text-secondary);
margin: 0 0 0.5rem;
}
.restore-error {
font-size: 0.85rem;
color: var(--fs-action-destructive);
margin: 0 0 0.75rem;
}
.restore-sources {
list-style: none;
margin: 0 0 0.5rem;
padding: 0;
max-height: 14rem;
overflow-y: auto;
border: 1px solid var(--fs-border-color);
border-radius: var(--fs-radius-sm);
}
.restore-source {
display: flex;
justify-content: space-between;
width: 100%;
padding: 0.45rem 0.7rem;
border: none;
border-bottom: 1px solid var(--fs-border-color);
background: none;
color: var(--fs-text-primary);
font: inherit;
font-size: 0.875rem;
text-align: left;
cursor: pointer;
}
.restore-sources li:last-child .restore-source {
border-bottom: none;
}
.restore-source:hover:not(:disabled) {
background: var(--fs-surface-hover);
}
.restore-actions-left {
justify-content: flex-start;
margin-bottom: 1rem;
}
.restore-counts {
width: 100%;
border-collapse: collapse;
font-size: 0.875rem;
margin: 0 0 1rem;
}
.restore-counts th,
.restore-counts td {
text-align: right;
padding: 0.25rem 0.5rem;
border-bottom: 1px solid var(--fs-border-color);
}
.restore-counts th:first-child,
.restore-counts td:first-child {
text-align: left;
}
.hidden-file-input {
display: none;
}
.restore-confirm {
display: flex;
flex-direction: column;
gap: 0.35rem;
font-size: 0.875rem;
margin: 0 0 1rem;
}
</style>
@@ -0,0 +1,309 @@
<script setup lang="ts">
import { computed, onMounted, ref } from "vue";
import { apiGet, apiPut, apiErrorMessage } from "@/api/client";
import { useToastStore } from "@/stores/toast";
/**
* How many reverse proxies stand in front of Grimoire, checked against the
* admin's own request.
*
* Each proxy appends to X-Forwarded-For, and Grimoire reads the entry this many
* in from the right as the visitor's address. Every sign-in rate limit and
* audit line keys on that address, and the scheme read the same way decides
* whether the session cookie is Secure. Too low a count makes every visitor
* read as the proxy, so they share one rate limit and one bad sign-in run
* blocks everyone. Behind Cloudflare's proxied DNS the usual count is 2.
*/
interface ReadoutRow { hops: number; client: string; https: boolean }
interface Readout {
peer: string;
forwarded_for: string[];
forwarded_proto: string[];
cloudflare: boolean;
cloudflare_client: string | null;
rows: ReadoutRow[];
suggested: number | null;
}
interface ProxyConfig {
hops: number;
source: "setting" | "env" | "default";
env_hops: number;
max_hops: number;
readout: Readout;
}
const toast = useToastStore();
const config = ref<ProxyConfig | null>(null);
const loadError = ref("");
const draft = ref<number>(1);
const saving = ref(false);
const saved = ref(false);
async function load() {
loadError.value = "";
try {
config.value = await apiGet<ProxyConfig>("/api/admin/proxy");
draft.value = config.value.hops;
} catch (e) {
loadError.value = apiErrorMessage(e, "The proxy settings could not be loaded.");
}
}
async function save(hops: number | null) {
saving.value = true;
saved.value = false;
try {
config.value = await apiPut<ProxyConfig>("/api/admin/proxy", { hops });
draft.value = config.value.hops;
saved.value = true;
setTimeout(() => (saved.value = false), 2000);
} catch (e) {
toast.show(apiErrorMessage(e, "Failed to save the proxy count"), "error");
} finally {
saving.value = false;
}
}
const readout = computed(() => config.value?.readout ?? null);
const inForce = computed(() => readout.value?.rows.find((r) => r.hops === config.value?.hops) ?? null);
// The verdict that leads the section. A Cloudflare request names the count to
// set, since Cloudflare reports the visitor's address; any other road can only
// show the address read and let the admin judge whether it is theirs.
const verdict = computed<"set-suggested" | "cloudflare-unmatched" | "check" | null>(() => {
const r = readout.value;
if (!r || !config.value) return null;
if (r.cloudflare && r.suggested !== null && r.suggested !== config.value.hops) return "set-suggested";
if (r.cloudflare && r.suggested === null) return "cloudflare-unmatched";
return "check";
});
const sourceLabel = computed(() => {
const c = config.value;
if (!c) return "";
if (c.source === "setting") return "Saved here.";
if (c.source === "env") return `From TRUSTED_PROXY_HOPS in the environment (${c.env_hops}).`;
return "The default.";
});
const draftValid = computed(() =>
Number.isInteger(draft.value) && draft.value >= 0 && draft.value <= (config.value?.max_hops ?? 5),
);
onMounted(load);
</script>
<template>
<div class="proxy-settings">
<p v-if="loadError" class="error-msg">
{{ loadError }}
<button type="button" class="btn-ghost btn-compact" @click="load">Try again</button>
</p>
<p v-else-if="!config" class="state-msg">Reading this request&hellip;</p>
<template v-else-if="readout">
<div v-if="verdict === 'set-suggested'" class="proxy-warning" role="alert">
<p class="proxy-warning-title">Set the proxy count to {{ readout.suggested }}</p>
<p class="proxy-warning-body">
This request came through Cloudflare, which reports your address as
<code>{{ readout.cloudflare_client }}</code>. At {{ config.hops }}, Grimoire reads
<code>{{ inForce?.client }}</code> instead, so visitors share Cloudflare's addresses
and one sign-in limit.
</p>
<button type="button" class="btn-primary" :disabled="saving" @click="save(readout.suggested)">
Use {{ readout.suggested }}
</button>
</div>
<div v-else-if="verdict === 'cloudflare-unmatched'" class="proxy-warning" role="alert">
<p class="proxy-warning-title">This request came through Cloudflare</p>
<p class="proxy-warning-body">
No count reads the address Cloudflare reports for you. Check the forwarding chain below,
and that your own proxy passes Cloudflare's <code>X-Forwarded-For</code> on.
</p>
</div>
<p v-else class="proxy-verdict">
Grimoire reads this request as coming from <code>{{ inForce?.client || "an unknown address" }}</code>
<template v-if="inForce?.https"> over HTTPS</template>. If that is your address, the count
is right. If it is your proxy's, raise the count.
</p>
<div class="field proxy-field">
<label for="proxy-hops">Proxies in front of Grimoire</label>
<div class="proxy-input-row">
<input
id="proxy-hops"
v-model.number="draft"
type="number"
min="0"
:max="config.max_hops"
step="1"
class="fs-input input proxy-hops-input"
/>
<button
type="button"
class="btn-primary"
:disabled="saving || !draftValid || draft === config.hops"
@click="save(draft)"
>
{{ saving ? "Saving..." : "Save" }}
</button>
<button
v-if="config.source === 'setting'"
type="button"
class="btn-ghost"
:disabled="saving"
@click="save(null)"
>
Use the environment's ({{ config.env_hops }})
</button>
<span v-if="saved" class="saved-msg">Saved. It applies to the next request.</span>
</div>
<p class="field-hint">
{{ sourceLabel }} One proxy (Traefik, Nginx, Caddy) is 1. Cloudflare in front of your own
proxy is 2. 0 ignores the forwarding headers and uses the connecting address.
</p>
</div>
<table class="proxy-table">
<caption>What each count makes of this request</caption>
<thead>
<tr><th class="num">Count</th><th>Reads you as</th><th>HTTPS</th><th></th></tr>
</thead>
<tbody>
<tr
v-for="row in readout.rows"
:key="row.hops"
:class="{ 'proxy-row-current': row.hops === config.hops }"
>
<td class="num">{{ row.hops }}</td>
<td><code>{{ row.client || "unknown" }}</code></td>
<td>{{ row.https ? "yes" : "no" }}</td>
<td class="proxy-row-tags">
<span v-if="row.hops === config.hops" class="proxy-tag">in force</span>
<span v-if="row.hops === readout.suggested && row.hops !== config.hops" class="proxy-tag proxy-tag-suggested">
suggested
</span>
</td>
</tr>
</tbody>
</table>
<dl class="proxy-chain">
<dt>Connecting address</dt>
<dd><code>{{ readout.peer || "unknown" }}</code></dd>
<dt>X-Forwarded-For</dt>
<dd v-if="readout.forwarded_for.length"><code>{{ readout.forwarded_for.join(", ") }}</code></dd>
<dd v-else class="proxy-absent">not sent</dd>
<dt>X-Forwarded-Proto</dt>
<dd v-if="readout.forwarded_proto.length"><code>{{ readout.forwarded_proto.join(", ") }}</code></dd>
<dd v-else class="proxy-absent">not sent</dd>
</dl>
</template>
</div>
</template>
<style scoped>
.proxy-warning {
margin-bottom: var(--fs-space-4);
padding: var(--fs-space-3);
border: 1px solid var(--fs-warning);
border-radius: var(--fs-radius-md);
background: color-mix(in srgb, var(--fs-warning) 12%, var(--fs-surface-raised));
}
.proxy-warning-title {
margin: 0 0 var(--fs-space-2);
font-weight: 600;
color: var(--fs-warning-fg);
}
.proxy-warning-body {
margin: 0 0 var(--fs-space-3);
font-size: var(--fs-size-body-sm);
color: var(--fs-text-primary);
}
.proxy-verdict {
margin: 0 0 var(--fs-space-4);
font-size: var(--fs-size-body-sm);
color: var(--fs-text-primary);
}
.proxy-field {
margin-bottom: var(--fs-space-4);
}
.proxy-input-row {
display: flex;
align-items: center;
gap: var(--fs-space-3);
flex-wrap: wrap;
}
.proxy-hops-input {
width: 5rem;
}
.proxy-table {
width: 100%;
max-width: 560px;
border-collapse: collapse;
font-size: var(--fs-size-body-sm);
margin-bottom: var(--fs-space-4);
}
.proxy-table caption {
text-align: left;
font-size: var(--fs-size-label);
color: var(--fs-text-secondary);
margin-bottom: var(--fs-space-2);
}
.proxy-table th,
.proxy-table td {
text-align: left;
padding: var(--fs-space-1) var(--fs-space-3);
border-bottom: 1px solid var(--fs-border-color);
}
.proxy-table th {
font-size: var(--fs-size-tiny);
text-transform: uppercase;
letter-spacing: 0.04em;
color: var(--fs-text-tertiary);
font-weight: 500;
}
.proxy-table .num {
text-align: right;
font-variant-numeric: tabular-nums;
width: 4rem;
}
.proxy-row-current td {
background: var(--fs-surface-hover);
font-weight: 600;
}
.proxy-row-tags {
white-space: nowrap;
}
.proxy-tag {
display: inline-block;
padding: 0 var(--fs-space-2);
border-radius: var(--fs-radius-pill);
font-size: var(--fs-size-tiny);
font-weight: 500;
color: var(--fs-text-secondary);
border: 1px solid var(--fs-border-active);
}
.proxy-tag-suggested {
color: var(--fs-warning-fg);
border-color: var(--fs-warning);
}
.proxy-chain {
display: grid;
grid-template-columns: max-content 1fr;
gap: var(--fs-space-1) var(--fs-space-4);
margin: 0;
font-size: var(--fs-size-body-sm);
}
.proxy-chain dt {
color: var(--fs-text-secondary);
}
.proxy-chain dd {
margin: 0;
overflow-wrap: anywhere;
}
.proxy-absent {
font-style: italic;
color: var(--fs-text-tertiary);
}
</style>
@@ -0,0 +1,96 @@
<script setup lang="ts">
/**
* The end of a server-paged list: asks for the next page as the reader nears
* it, in place of a "Load more" button. The parent keeps the paging — what the
* next page is keyed on, whether one is in flight, whether the last one
* failed — and this owns the observing.
*
* <ScrollSentinel :cursor="more ? String(rows.length) : ''"
* :loading="loadingMore" :failed="moreFailed" @more="loadMore" />
*
* An empty cursor means the list is complete, and nothing renders.
*/
import { onBeforeUnmount, ref, watch } from "vue";
const props = defineProps<{ cursor: string; loading: boolean; failed?: boolean }>();
const emit = defineEmits<{ more: [] }>();
const sentinel = ref<HTMLElement | null>(null);
const visible = ref(false);
// One request per page. The observer reports every crossing, and a short last
// page leaves the sentinel on screen for good, so the cursor last asked for is
// remembered: a page is asked for once, and a failed one is not retried on
// every scroll, because a failure leaves the cursor where it was. Only
// Try again asks for it again.
let asked = "";
function pump() {
if (!visible.value || props.loading || props.failed || !props.cursor) return;
if (props.cursor === asked) return;
asked = props.cursor;
emit("more");
}
function retry() {
asked = props.cursor;
emit("more");
}
let observer: IntersectionObserver | undefined;
function observe(el: HTMLElement | null) {
observer?.disconnect();
observer = undefined;
if (!el) {
// Visibility and the cursor asked for go with the element. A list that
// starts over must not inherit "the end is on screen" from the one it
// replaced, and an offset cursor repeats from list to list, so "50" asked
// of the old list must not stop the new one asking for its own "50".
visible.value = false;
asked = "";
return;
}
observer = new IntersectionObserver(
(entries) => {
visible.value = entries.some((e) => e.isIntersecting);
pump();
},
// Extends the watched area a screen below the viewport, so the next page is
// asked for before the reader reaches the end rather than when they do.
{ rootMargin: "0px 0px 100% 0px" },
);
observer.observe(el);
}
// flush: "post": the template ref is only set once the DOM is patched.
watch(sentinel, observe, { flush: "post" });
// A page that lands without pushing the end off screen gives the observer
// nothing new to report, so carry on from the props, or the list stops early.
watch([() => props.cursor, () => props.loading, () => props.failed], pump);
onBeforeUnmount(() => observer?.disconnect());
</script>
<template>
<div v-if="cursor" class="sentinel">
<!-- The one case with a button: an observer re-firing on a failure would be
a request loop. -->
<p v-if="failed" class="sentinel-note sentinel-failed" role="alert">
Couldn’t load more.
<button type="button" class="btn-secondary btn-sm" @click="retry">Try again</button>
</p>
<template v-else>
<div ref="sentinel" class="sentinel-mark" aria-hidden="true"></div>
<p v-if="loading" class="sentinel-note" role="status">Loading more…</p>
</template>
</div>
</template>
<style scoped>
.sentinel { grid-column: 1 / -1; padding: 0.75rem 0; text-align: center; }
.sentinel-mark { height: 1px; }
.sentinel-note { margin: 0; font-size: 0.8rem; color: var(--fs-text-tertiary); }
.sentinel-failed { display: flex; align-items: center; justify-content: center; gap: 0.75rem; }
</style>
@@ -1,683 +0,0 @@
<script setup lang="ts">
import { ref, computed, watch, onMounted, onUnmounted, nextTick } from "vue";
import type { Editor } from "@tiptap/vue-3";
import { apiGet, apiPatch, apiPost, apiDelete } from "@/api/client";
import { useToastStore } from "@/stores/toast";
import { useAutoSave } from "@/composables/useAutoSave";
import { useTagSuggestions } from "@/composables/useTagSuggestions";
import { useNotesStore } from "@/stores/notes";
import TiptapEditor from "@/components/TiptapEditor.vue";
import TagInput from "@/components/TagInput.vue";
import MarkdownToolbar from "@/components/MarkdownToolbar.vue";
import WordCount from "@/components/WordCount.vue";
import { Trash2, X } from "lucide-vue-next";
import { relativeTimeOrDate } from "@/composables/useRelativeTime";
const props = defineProps<{
projectId: number;
activeNoteId: number | null;
}>();
const emit = defineEmits<{ "note-loaded": [id: number] }>();
interface NoteItem {
id: number;
title: string;
tags: string[];
updated_at: string;
}
const toast = useToastStore();
const notesStore = useNotesStore();
// List state
const projectNotes = ref<NoteItem[]>([]);
const listLoading = ref(false);
const deletingId = ref<number | null>(null);
const pendingDelete = ref<number | null>(null);
const searchQuery = ref("");
const filteredNotes = computed(() => {
const q = searchQuery.value.trim().toLowerCase();
if (!q) return projectNotes.value;
return projectNotes.value.filter(
(n) => n.title.toLowerCase().includes(q) || n.tags?.some((t) => t.toLowerCase().includes(q))
);
});
function matchedTags(note: NoteItem): string[] {
const q = searchQuery.value.trim().toLowerCase();
if (!q) return [];
return (note.tags ?? []).filter(
(t) => t.toLowerCase().includes(q) && !note.title.toLowerCase().includes(q)
);
}
// Editor state
const editingId = ref<number | null>(null);
const noteTitle = ref("");
const noteBody = ref("");
const noteTags = ref<string[]>([]);
const dirty = ref(false);
const saving = ref(false);
const tagSuggestions = useTagSuggestions(noteTitle, noteBody, noteTags, () => { dirty.value = true; });
// New note inline creation
const showNewNoteInput = ref(false);
const newNoteTitle = ref("");
const creatingNote = ref(false);
const newNoteTitleRef = ref<HTMLInputElement | null>(null);
function _noteKey(pid: number) { return `workspace_note_${pid}`; }
async function startNewNote() {
showNewNoteInput.value = true;
newNoteTitle.value = "";
await nextTick();
newNoteTitleRef.value?.focus();
}
function cancelNewNote() {
showNewNoteInput.value = false;
newNoteTitle.value = "";
}
async function createNote() {
const title = newNoteTitle.value.trim();
if (!title || creatingNote.value) return;
creatingNote.value = true;
try {
const note = await apiPost<{ id: number; title: string; body: string; tags: string[]; updated_at: string }>(
"/api/notes",
{ title, project_id: props.projectId, body: "", is_task: false }
);
showNewNoteInput.value = false;
newNoteTitle.value = "";
await loadProjectNotes();
await openNote(note.id);
} catch {
toast.show("Failed to create note", "error");
} finally {
creatingNote.value = false;
}
}
// TipTap editor ref
const titleRef = ref<HTMLInputElement | null>(null);
const editorRef = ref<InstanceType<typeof TiptapEditor> | null>(null);
const tiptapEditor = computed<Editor | null>(() =>
(editorRef.value?.editor as Editor | undefined) ?? null
);
// Link suggestions
interface LinkSuggestion { note_id: number; title: string; count: number; }
const linkSuggestions = ref<LinkSuggestion[]>([]);
let linkCheckTimer: ReturnType<typeof setTimeout> | null = null;
async function fetchLinkSuggestions() {
if (!editingId.value || !noteBody.value) { linkSuggestions.value = []; return; }
try {
const res = await apiPost<{ suggestions: LinkSuggestion[] }>('/api/notes/link-suggestions', {
body: noteBody.value,
project_id: props.projectId,
exclude_note_id: editingId.value,
});
linkSuggestions.value = res.suggestions;
} catch {
linkSuggestions.value = [];
}
}
function scheduleLinkCheck() {
if (linkCheckTimer) clearTimeout(linkCheckTimer);
linkCheckTimer = setTimeout(fetchLinkSuggestions, 2500);
}
function applyWikilinkToText(text: string, title: string): string {
const escaped = title.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const parts = text.split(/(\[\[[^\]]*\]\])/);
const pattern = new RegExp(`\\b${escaped}\\b`, 'gi');
return parts.map((part, i) =>
i % 2 === 0 ? part.replace(pattern, `[[${title}]]`) : part
).join('');
}
function applyLink(s: LinkSuggestion) {
noteBody.value = applyWikilinkToText(noteBody.value, s.title);
dirty.value = true;
linkSuggestions.value = linkSuggestions.value.filter(x => x.note_id !== s.note_id);
}
function applyAllLinks() {
let text = noteBody.value;
for (const s of linkSuggestions.value) text = applyWikilinkToText(text, s.title);
noteBody.value = text;
dirty.value = true;
linkSuggestions.value = [];
}
async function loadProjectNotes() {
listLoading.value = true;
try {
const data = await apiGet<{ notes: NoteItem[] }>(
`/api/projects/${props.projectId}/notes?type=note&limit=200`
);
projectNotes.value = data.notes ?? [];
} catch {
toast.show("Failed to load notes", "error");
} finally {
listLoading.value = false;
}
}
async function openNote(id: number) {
try {
const note = await apiGet<{ id: number; title: string; body: string; tags: string[]; updated_at: string }>(
`/api/notes/${id}`
);
editingId.value = note.id;
noteTitle.value = note.title;
noteBody.value = note.body ?? "";
noteTags.value = note.tags ?? [];
dirty.value = false;
tagSuggestions.dismissTagSuggestions();
linkSuggestions.value = [];
localStorage.setItem(_noteKey(props.projectId), String(id));
fetchLinkSuggestions();
emit("note-loaded", note.id);
} catch {
toast.show("Failed to load note", "error");
}
}
async function saveNote() {
if (!editingId.value) return;
saving.value = true;
try {
await apiPatch(`/api/notes/${editingId.value}`, {
title: noteTitle.value,
body: noteBody.value,
tags: noteTags.value,
});
dirty.value = false;
const idx = projectNotes.value.findIndex((n) => n.id === editingId.value);
if (idx !== -1) {
projectNotes.value[idx] = {
...projectNotes.value[idx],
title: noteTitle.value,
tags: noteTags.value,
updated_at: new Date().toISOString(),
};
projectNotes.value.sort((a, b) =>
new Date(b.updated_at).getTime() - new Date(a.updated_at).getTime()
);
}
} catch {
toast.show("Failed to save note", "error");
} finally {
saving.value = false;
}
}
function requestDelete(id: number, e: Event) {
e.stopPropagation();
if (deletingId.value === id) {
confirmDelete(id);
} else {
deletingId.value = id;
}
}
function cancelDelete(e: Event) {
e.stopPropagation();
deletingId.value = null;
}
async function confirmDelete(id: number) {
pendingDelete.value = id;
try {
await apiDelete(`/api/notes/${id}`);
projectNotes.value = projectNotes.value.filter((n) => n.id !== id);
if (editingId.value === id) {
editingId.value = null;
noteTitle.value = "";
noteBody.value = "";
localStorage.removeItem(_noteKey(props.projectId));
}
} catch {
toast.show("Failed to delete note", "error");
} finally {
pendingDelete.value = null;
deletingId.value = null;
}
}
watch(noteTitle, () => { dirty.value = true; });
watch(noteBody, () => { dirty.value = true; if (editingId.value) scheduleLinkCheck(); });
watch(noteTags, () => { dirty.value = true; });
watch(
() => props.activeNoteId,
async (id) => {
if (id) {
await openNote(id);
await loadProjectNotes();
}
}
);
useAutoSave(dirty, saving, saveNote, 60_000);
onMounted(async () => {
await loadProjectNotes();
const stored = localStorage.getItem(_noteKey(props.projectId));
if (stored) {
const id = Number(stored);
if (projectNotes.value.find((n) => n.id === id)) {
openNote(id).catch(() => {});
}
}
});
onUnmounted(() => { if (linkCheckTimer) clearTimeout(linkCheckTimer); });
defineExpose({ reload: loadProjectNotes });
</script>
<template>
<div class="ws-note-editor">
<!-- Left rail: always-visible note list -->
<div class="note-rail">
<div class="rail-header">
<template v-if="showNewNoteInput">
<input
ref="newNoteTitleRef"
v-model="newNoteTitle"
class="new-note-input"
placeholder="Note title..."
:disabled="creatingNote"
@keydown.enter="createNote"
@keydown.escape="cancelNewNote"
/>
<button class="btn-primary btn-inline" :disabled="creatingNote || !newNoteTitle.trim()" @click="createNote">
{{ creatingNote ? '…' : '+' }}
</button>
<button class="btn-text" aria-label="Cancel new note" @click="cancelNewNote"><X :size="16" /></button>
</template>
<template v-else>
<span class="rail-title">Notes</span>
<button class="btn-ghost btn-inline" @click="startNewNote" title="New note">+ New</button>
</template>
</div>
<div class="rail-search">
<input
v-model="searchQuery"
class="rail-search-input"
placeholder="Search…"
type="search"
aria-label="Search notes"
/>
<button v-if="searchQuery" class="btn-text btn-search-clear" aria-label="Clear search" @click="searchQuery = ''"><X :size="16" /></button>
</div>
<div v-if="listLoading" class="rail-state">Loading…</div>
<ul v-else class="note-list">
<li
v-for="note in filteredNotes"
:key="note.id"
:class="['note-row', { active: editingId === note.id }]"
@click="openNote(note.id)"
>
<div class="note-row-main">
<span class="note-row-title">{{ note.title || 'Untitled' }}</span>
<span class="note-row-age">{{ relativeTimeOrDate(note.updated_at) }}</span>
</div>
<div v-if="note.tags?.length" class="note-row-tags">
<span
v-for="tag in note.tags.slice(0, 2)"
:key="tag"
:class="['note-tag-pill', { 'tag-match': matchedTags(note).includes(tag) }]"
>#{{ tag }}</span>
<span v-if="note.tags.length > 2" class="note-tag-more">+{{ note.tags.length - 2 }}</span>
</div>
<div class="note-row-actions" @click.stop>
<template v-if="deletingId === note.id">
<button class="btn-danger-outline btn-inline" :disabled="pendingDelete === note.id" @click="requestDelete(note.id, $event)">
{{ pendingDelete === note.id ? '…' : 'Delete?' }}
</button>
<button class="btn-text" aria-label="Cancel delete" @click="cancelDelete($event)"><X :size="16" /></button>
</template>
<button v-else class="btn-text" title="Delete note" @click="requestDelete(note.id, $event)">
<Trash2 :size="16" />
</button>
</div>
</li>
<li v-if="filteredNotes.length === 0" class="rail-state">
{{ searchQuery.trim() ? 'No matches.' : 'No notes yet.' }}
</li>
</ul>
</div>
<!-- Right pane: editor -->
<div class="note-editor-pane">
<template v-if="editingId">
<div class="panel-header">
<div class="editor-header-right">
<WordCount :body="noteBody" />
<span v-if="dirty && !saving" class="unsaved">Unsaved</span>
<span v-if="saving" class="saving-txt">Saving…</span>
<button class="btn-primary btn-compact" :disabled="saving || !dirty" @click="saveNote">Save</button>
</div>
</div>
<div class="note-title-row">
<input
ref="titleRef"
v-model="noteTitle"
class="note-title-input"
placeholder="Note title"
@keydown.ctrl.s.prevent="saveNote"
@keydown.ctrl.e.prevent="editorRef?.editor?.commands.focus()"
/>
</div>
<div class="tag-row">
<TagInput v-model="noteTags" :fetchTags="(q: string) => notesStore.fetchAllTags(q)" />
<button
class="btn-ghost btn-compact"
:disabled="tagSuggestions.suggestingTags.value"
title="Auto-suggest tags from title and body"
@click="tagSuggestions.fetchTagSuggestions()"
>
{{ tagSuggestions.suggestingTags.value ? '…' : 'Suggest' }}
</button>
</div>
<div v-if="tagSuggestions.suggestedTags.value.length" class="tag-suggestions">
<span class="tag-suggestions-label">Suggested:</span>
<button
v-for="tag in tagSuggestions.suggestedTags.value"
:key="tag"
:class="['btn-tag-suggestion', { applied: tagSuggestions.appliedTags.value.has(tag) }]"
@click="tagSuggestions.applyTagSuggestion(tag)"
>#{{ tag }}{{ tagSuggestions.appliedTags.value.has(tag) ? ' ✓' : '' }}</button>
<button class="btn-text" aria-label="Dismiss tag suggestions" @click="tagSuggestions.dismissTagSuggestions()"><X :size="16" /></button>
</div>
<div class="toolbar-row">
<MarkdownToolbar :editor="tiptapEditor" />
</div>
<div v-if="linkSuggestions.length" class="link-suggest-strip">
<span class="link-suggest-label">Links:</span>
<span v-for="s in linkSuggestions" :key="s.note_id" class="link-suggest-chip" :title="`Appears ${s.count}× unlinked`">
<button class="btn-chip-link" @click="applyLink(s)">[[{{ s.title }}]]</button>
</span>
<button class="btn-ghost btn-inline" @click="applyAllLinks" title="Link all suggestions">All</button>
<button class="btn-text" aria-label="Dismiss link suggestions" @click="linkSuggestions = []"><X :size="16" /></button>
</div>
<div class="editor-area" @keydown.ctrl.s.prevent="saveNote" @keydown.ctrl.e.prevent="editorRef?.editor?.commands.focus()">
<TiptapEditor ref="editorRef" v-model="noteBody" @escape="titleRef?.focus()" />
</div>
</template>
<div v-else class="editor-empty-state">
<p>Select a note or create a new one</p>
</div>
</div>
</div>
</template>
<style scoped>
.ws-note-editor {
display: flex;
flex-direction: row;
height: 100%;
overflow: hidden;
background: var(--fs-surface-hover);
border-left: 1px solid var(--fs-border-color);
}
/* ── Left rail ── */
.note-rail {
width: 200px;
flex-shrink: 0;
display: flex;
flex-direction: column;
overflow: hidden;
background: var(--fs-surface-raised);
}
.rail-header {
display: flex;
align-items: center;
gap: 0.3rem;
padding: 0.5rem 0.6rem;
border-bottom: 1px solid var(--fs-border-color);
flex-shrink: 0;
}
.rail-title {
color: var(--fs-text-tertiary);
text-transform: uppercase;
letter-spacing: 0.04em;
font-size: 0.72rem;
font-weight: 500;
flex: 1;
}
.rail-search-input {
flex: 1;
background: transparent;
border: none;
font-size: 0.78rem;
color: var(--fs-text-primary);
min-width: 0;
padding: 0;
}
.rail-search-input:focus { outline: none; }
.rail-search-input::-webkit-search-cancel-button { display: none; }
/* Sits inside the search field: no padding, and must not flex. */
.btn-search-clear { padding: 0; flex-shrink: 0; }
.rail-state {
padding: 1rem 0.65rem;
font-size: 0.78rem;
color: var(--fs-text-tertiary);
}
/* Note list */
.note-list {
list-style: none;
margin: 0;
padding: 0;
overflow-y: auto;
flex: 1;
}
.note-row {
display: flex;
flex-direction: column;
padding: 0.4rem 0.6rem;
border-bottom: 1px solid color-mix(in srgb, var(--fs-border-color) 60%, transparent);
cursor: pointer;
gap: 0.15rem;
border-right: 2px solid transparent;
transition: background 0.12s;
}
.note-row:hover { background: color-mix(in srgb, var(--fs-accent) 5%, var(--fs-surface-hover)); }
.note-row.active {
background: color-mix(in srgb, var(--fs-accent) 8%, var(--fs-surface-hover));
border-right-color: var(--fs-accent);
}
.note-row-main {
display: flex;
align-items: baseline;
gap: 0.25rem;
}
.note-row-title {
flex: 1;
font-size: 0.78rem;
font-weight: 500;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: var(--fs-text-primary);
}
.note-row-age {
font-size: 0.62rem;
color: var(--fs-text-tertiary);
white-space: nowrap;
flex-shrink: 0;
}
.note-row-tags {
display: flex;
gap: 0.15rem;
overflow: hidden;
}
.note-tag-pill {
font-size: 0.58rem;
color: var(--fs-accent-fg);
background: color-mix(in srgb, var(--fs-accent) 10%, transparent);
border-radius: 999px;
padding: 0 0.3rem;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
max-width: 5rem;
}
.note-tag-pill.tag-match {
background: color-mix(in srgb, var(--fs-accent) 22%, transparent);
outline: 1px solid var(--fs-accent);
}
.note-tag-more {
font-size: 0.58rem;
color: var(--fs-text-tertiary);
white-space: nowrap;
}
.note-row-actions {
display: flex;
gap: 0.2rem;
align-items: center;
}
/* Editor UI */
.panel-header {
display: flex;
align-items: center;
padding: 0.5rem 0.75rem;
flex-shrink: 0;
gap: 0.5rem;
}
.editor-header-right {
display: flex;
align-items: center;
gap: 0.5rem;
margin-left: auto;
}
.unsaved { font-size: 0.72rem; color: var(--fs-text-tertiary); }
.saving-txt { font-size: 0.72rem; color: var(--fs-accent); }
/* Moss action-primary per Hybrid */
.note-title-input {
width: 100%;
background: transparent;
border: none;
font-size: 1.4rem;
font-weight: 500;
line-height: 1.25;
color: var(--fs-text-primary);
padding: 0;
font-family: 'Fraunces', serif;
letter-spacing: -0.01em;
}
.note-title-input:focus { outline: none; }
.note-title-input::placeholder {
color: var(--fs-text-tertiary);
}
.tag-row {
display: flex;
align-items: flex-start;
gap: 0.4rem;
padding: 0.4rem 0.6rem;
flex-shrink: 0;
}
.tag-row > :first-child { flex: 1; min-width: 0; }
.tag-suggestions {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 0.3rem;
padding: 0.35rem 0.6rem;
background: color-mix(in srgb, var(--fs-accent) 5%, var(--fs-surface-hover));
flex-shrink: 0;
}
.tag-suggestions-label { font-size: 0.72rem; color: var(--fs-text-tertiary); flex-shrink: 0; }
.btn-tag-suggestion {
background: none;
border: 1px solid var(--fs-border-color);
border-radius: 999px;
padding: 0.15rem 0.55rem;
font-size: 0.75rem;
color: var(--fs-text-primary);
cursor: pointer;
}
.btn-tag-suggestion:hover { border-color: var(--fs-accent); color: var(--fs-accent); }
.btn-tag-suggestion.applied {
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
border-color: var(--fs-accent);
color: var(--fs-accent-fg);
}
.link-suggest-strip {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 0.25rem;
padding: 0.3rem 0.6rem;
background: color-mix(in srgb, var(--fs-accent) 5%, var(--fs-surface-hover));
flex-shrink: 0;
}
.link-suggest-label {
font-size: 0.7rem;
color: var(--fs-text-tertiary);
flex-shrink: 0;
font-weight: 500;
text-transform: uppercase;
letter-spacing: 0.04em;
}
.link-suggest-chip { display: inline-flex; }
.btn-chip-link {
background: none;
border: 1px solid var(--fs-accent);
border-radius: 999px;
padding: 0.1rem 0.45rem;
font-size: 0.7rem;
color: var(--fs-accent);
cursor: pointer;
font-family: monospace;
white-space: nowrap;
}
.btn-chip-link:hover { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); }
</style>
@@ -1,618 +0,0 @@
<script setup lang="ts">
import { ref, computed, onMounted } from "vue";
import { RouterLink } from "vue-router";
import { apiGet, apiPatch, apiPost, apiDelete } from "@/api/client";
import { useToastStore } from "@/stores/toast";
import TaskLogSection from "@/components/TaskLogSection.vue";
import KindBadge from "@/components/KindBadge.vue";
import type { TaskKind } from "@/types/note";
import { renderMarkdown } from "@/utils/markdown";
import { Trash2, X } from "lucide-vue-next";
import { relativeTimeOrDate } from "@/composables/useRelativeTime";
const props = defineProps<{ projectId: number }>();
const toast = useToastStore();
interface Milestone {
id: number;
title: string;
status: string;
order_index: number;
}
interface Task {
id: number;
title: string;
status: string;
priority: string;
milestone_id: number | null;
due_date: string | null;
updated_at: string;
body?: string;
task_kind?: TaskKind;
}
const tasks = ref<Task[]>([]);
const milestones = ref<Milestone[]>([]);
const loading = ref(false);
// Detail slide-over
const activeTask = ref<Task | null>(null);
const taskBody = ref("");
const loadingBody = ref(false);
// Collapsed milestone groups (set of milestone IDs, null = "No Milestone" group)
const collapsedGroups = ref<Set<number | null>>(new Set());
// New task quick-add
const newTaskTitle = ref("");
const addingTask = ref(false);
const STATUS_CYCLE: Record<string, string> = {
todo: "in_progress",
in_progress: "done",
done: "todo",
};
const STATUS_ICON: Record<string, string> = {
todo: "○",
in_progress: "▶",
done: "✓",
};
const PRIORITY_CLASS: Record<string, string> = {
high: "priority-high",
medium: "priority-medium",
low: "priority-low",
};
function isRowOverdue(task: Task): boolean {
if (!task.due_date || task.status === "done") return false;
return task.due_date < new Date().toISOString().slice(0, 10);
}
// Group tasks by milestone, "No Milestone" first
const groupedTasks = computed(() => {
const noMs = tasks.value.filter((t) => t.milestone_id === null);
const msGroups = milestones.value.map((ms) => ({
milestone: ms,
tasks: tasks.value.filter((t) => t.milestone_id === ms.id),
}));
return { noMilestone: noMs, milestoneGroups: msGroups };
});
async function loadAll() {
loading.value = true;
try {
const [tasksRes, msRes] = await Promise.all([
apiGet<{ notes: Task[] }>(`/api/projects/${props.projectId}/notes?type=task&limit=200`),
apiGet<{ milestones: Milestone[] }>(`/api/projects/${props.projectId}/milestones`),
]);
tasks.value = tasksRes.notes ?? [];
milestones.value = msRes.milestones ?? [];
} catch {
toast.show("Failed to load tasks", "error");
} finally {
loading.value = false;
}
}
async function cycleStatus(task: Task, e: Event) {
e.stopPropagation();
const next = STATUS_CYCLE[task.status] ?? "todo";
try {
await apiPatch(`/api/tasks/${task.id}/status`, { status: next });
task.status = next;
if (activeTask.value?.id === task.id) activeTask.value.status = next;
} catch {
toast.show("Failed to update status", "error");
}
}
function toggleGroup(key: number | null) {
if (collapsedGroups.value.has(key)) {
collapsedGroups.value.delete(key);
} else {
collapsedGroups.value.add(key);
}
}
async function openTask(task: Task) {
activeTask.value = task;
taskBody.value = task.body ?? "";
loadingBody.value = true;
try {
const full = await apiGet<Task>(`/api/tasks/${task.id}`);
taskBody.value = full.body ?? "";
// Update cached task so re-opens don't re-fetch if unchanged
task.body = full.body;
} catch {
// Non-critical — body just won't show
} finally {
loadingBody.value = false;
}
}
function closeTask() {
activeTask.value = null;
taskBody.value = "";
deleteConfirmPending.value = false;
}
async function addTask() {
const title = newTaskTitle.value.trim();
if (!title) return;
addingTask.value = true;
try {
const task = await apiPost<Task>("/api/tasks", {
title,
project_id: props.projectId,
status: "todo",
});
tasks.value.unshift(task);
newTaskTitle.value = "";
} catch {
toast.show("Failed to create task", "error");
} finally {
addingTask.value = false;
}
}
const deletingTask = ref(false);
const deleteConfirmPending = ref(false);
const changingMilestone = ref(false);
async function setMilestone(milestoneId: number | null) {
if (!activeTask.value) return;
changingMilestone.value = true;
try {
await apiPatch(`/api/notes/${activeTask.value.id}`, { milestone_id: milestoneId });
activeTask.value.milestone_id = milestoneId;
// Update in tasks list too
const t = tasks.value.find((x) => x.id === activeTask.value!.id);
if (t) t.milestone_id = milestoneId;
} catch {
toast.show("Failed to update milestone", "error");
} finally {
changingMilestone.value = false;
}
}
async function deleteActiveTask() {
if (!activeTask.value) return;
if (!deleteConfirmPending.value) {
deleteConfirmPending.value = true;
return;
}
deletingTask.value = true;
try {
await apiDelete(`/api/tasks/${activeTask.value.id}`);
tasks.value = tasks.value.filter((t) => t.id !== activeTask.value!.id);
activeTask.value = null;
deleteConfirmPending.value = false;
} catch {
toast.show("Failed to delete task", "error");
} finally {
deletingTask.value = false;
}
}
function cancelDeleteTask() {
deleteConfirmPending.value = false;
}
onMounted(loadAll);
defineExpose({ reload: loadAll });
</script>
<template>
<div class="ws-task-panel">
<!-- Task list — always visible, shrinks when detail is open -->
<div :class="['task-list-view', { 'has-detail': !!activeTask }]">
<div class="panel-header">
<span class="panel-title">Tasks</span>
</div>
<div class="task-add">
<input
v-model="newTaskTitle"
class="task-add-input"
placeholder="New task..."
@keydown.enter="addTask"
/>
<button class="btn-primary btn-inline btn-add" :disabled="addingTask || !newTaskTitle.trim()" @click="addTask">+</button>
</div>
<div v-if="loading" class="state-msg">Loading...</div>
<div v-else class="groups-scroll">
<!-- No Milestone group (always first) -->
<div class="ms-group">
<button class="ms-group-header" @click="toggleGroup(null)">
<span class="ms-chevron">{{ collapsedGroups.has(null) ? '▶' : '▼' }}</span>
<span class="ms-name">No Milestone</span>
<span class="ms-count">{{ groupedTasks.noMilestone.length }}</span>
</button>
<ul v-show="!collapsedGroups.has(null)" class="task-items">
<li
v-for="task in groupedTasks.noMilestone"
:key="task.id"
:class="['task-row', { 'task-active': activeTask?.id === task.id }]"
@click="openTask(task)"
>
<button :class="['status-dot', `status-${task.status}`]" :title="`${task.status} — click to cycle`" @click="cycleStatus(task, $event)">{{ STATUS_ICON[task.status] ?? '○' }}</button>
<span v-if="task.priority && task.priority !== 'none'" :class="['priority-dot', PRIORITY_CLASS[task.priority] ?? '']"></span>
<span class="task-title" :class="{ done: task.status === 'done' }">{{ task.title }}</span>
<KindBadge :kind="task.task_kind" />
<span v-if="task.due_date" :class="['task-due', { overdue: isRowOverdue(task) }]">{{ task.due_date }}</span>
<span class="task-age">{{ relativeTimeOrDate(task.updated_at) }}</span>
</li>
<li v-if="groupedTasks.noMilestone.length === 0" class="empty-group">No tasks</li>
</ul>
</div>
<!-- Milestone groups -->
<div v-for="{ milestone, tasks: msTasks } in groupedTasks.milestoneGroups" :key="milestone.id" class="ms-group">
<button class="ms-group-header" @click="toggleGroup(milestone.id)">
<span class="ms-chevron">{{ collapsedGroups.has(milestone.id) ? '▶' : '▼' }}</span>
<span class="ms-name">{{ milestone.title }}</span>
<span :class="['ms-status', `ms-status-${milestone.status}`]">{{ milestone.status.replace('_',' ') }}</span>
<span class="ms-count">{{ msTasks.length }}</span>
</button>
<ul v-show="!collapsedGroups.has(milestone.id)" class="task-items">
<li
v-for="task in msTasks"
:key="task.id"
:class="['task-row', { 'task-active': activeTask?.id === task.id }]"
@click="openTask(task)"
>
<button :class="['status-dot', `status-${task.status}`]" :title="`${task.status} — click to cycle`" @click="cycleStatus(task, $event)">{{ STATUS_ICON[task.status] ?? '○' }}</button>
<span v-if="task.priority && task.priority !== 'none'" :class="['priority-dot', PRIORITY_CLASS[task.priority] ?? '']"></span>
<span class="task-title" :class="{ done: task.status === 'done' }">{{ task.title }}</span>
<KindBadge :kind="task.task_kind" />
<span v-if="task.due_date" :class="['task-due', { overdue: isRowOverdue(task) }]">{{ task.due_date }}</span>
<span class="task-age">{{ relativeTimeOrDate(task.updated_at) }}</span>
</li>
<li v-if="msTasks.length === 0" class="empty-group">No tasks</li>
</ul>
</div>
</div>
</div>
<!-- Detail pane — bottom split when a task is selected -->
<Transition name="detail-fade">
<div v-if="activeTask" class="task-detail">
<div class="detail-header">
<RouterLink :to="`/tasks/${activeTask.id}/edit`" target="_blank" class="btn-text btn-edit-task" title="Open full editor">Edit ↗</RouterLink>
<span :class="['status-cycler', `status-${activeTask.status}`]" @click="cycleStatus(activeTask, $event)" title="Click to cycle status">
{{ STATUS_ICON[activeTask.status] ?? "○" }} {{ activeTask.status.replace("_", " ") }}
</span>
<template v-if="deleteConfirmPending">
<button class="btn-danger-outline btn-inline btn-delete-confirm" :disabled="deletingTask" @click="deleteActiveTask">{{ deletingTask ? '...' : 'Delete?' }}</button>
<button class="btn-text" aria-label="Cancel delete" @click="cancelDeleteTask"><X :size="16" /></button>
</template>
<button v-else class="btn-text btn-delete-task" title="Delete task" @click="deleteActiveTask">
<Trash2 :size="16" />
</button>
<button class="btn-text btn-close-detail" @click="closeTask" aria-label="Close detail"><X :size="16" /></button>
</div>
<h3 class="detail-title">{{ activeTask.title }}</h3>
<div class="detail-meta">
<span v-if="activeTask.priority && activeTask.priority !== 'none'" class="meta-chip priority">{{ activeTask.priority }}</span>
<span v-if="activeTask.due_date" class="meta-chip due">Due {{ activeTask.due_date }}</span>
<select
class="milestone-select"
:value="activeTask.milestone_id ?? ''"
:disabled="changingMilestone"
@change="setMilestone(($event.target as HTMLSelectElement).value === '' ? null : Number(($event.target as HTMLSelectElement).value))"
>
<option value="">No Milestone</option>
<option v-for="ms in milestones" :key="ms.id" :value="ms.id">{{ ms.title }}</option>
</select>
</div>
<div v-if="taskBody || loadingBody" class="detail-body">
<div v-if="loadingBody" class="body-loading">…</div>
<div v-else class="prose" v-html="renderMarkdown(taskBody)" />
</div>
<div class="detail-log">
<TaskLogSection :task-id="activeTask.id" />
</div>
</div>
</Transition>
</div>
</template>
<style scoped>
.ws-task-panel {
position: relative;
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
background: var(--fs-surface-hover);
border-right: 1px solid var(--fs-border-color);
}
/* ── List view ── */
.task-list-view {
display: flex;
flex-direction: column;
flex: 1;
min-height: 0;
overflow: hidden;
}
.task-list-view.has-detail {
flex: 0 0 44%;
}
.task-active {
background: color-mix(in srgb, var(--fs-accent) 6%, var(--fs-surface-hover)) !important;
}
.panel-header {
padding: 0.6rem 0.75rem;
border-bottom: 1px solid var(--fs-border-color);
flex-shrink: 0;
}
.panel-title {
color: var(--fs-text-tertiary);
text-transform: uppercase;
letter-spacing: 0.04em;
font-size: 0.75rem;
font-weight: 500;
}
.task-add {
display: flex;
gap: 0.4rem;
padding: 0.45rem 0.6rem;
border-bottom: 1px solid var(--fs-border-color);
flex-shrink: 0;
}
.task-add-input {
flex: 1;
background: var(--fs-surface-page);
border: 1px solid var(--fs-border-color);
border-radius: 5px;
padding: 0.28rem 0.5rem;
font-size: 0.83rem;
color: var(--fs-text-primary);
}
.task-add-input:focus { outline: none; border-color: var(--fs-accent); }
.btn-add { font-size: 1rem; } /* a '+' glyph, not a label */
.groups-scroll {
flex: 1;
overflow-y: auto;
}
.ms-group {
border-bottom: 1px solid var(--fs-border-color);
}
.ms-group-header {
display: flex;
align-items: center;
gap: 0.4rem;
width: 100%;
padding: 0.4rem 0.65rem;
background: var(--fs-surface-raised);
border: none;
cursor: pointer;
text-align: left;
font-size: 0.8rem;
color: var(--fs-text-primary);
}
.ms-group-header:hover { background: color-mix(in srgb, var(--fs-accent) 8%, var(--fs-surface-hover)); }
.ms-chevron { font-size: 0.6rem; color: var(--fs-text-tertiary); width: 0.8rem; }
.ms-name { flex: 1; font-weight: 500; font-size: 0.8rem; }
.ms-count { font-size: 0.72rem; color: var(--fs-text-tertiary); background: var(--fs-surface-page); border-radius: 10px; padding: 0 0.4rem; }
.ms-status {
font-size: 0.68rem;
padding: 0.1rem 0.4rem;
border-radius: 10px;
text-transform: capitalize;
}
.ms-status-active { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent-fg); }
.ms-status-completed { background: color-mix(in srgb, var(--fs-success) 15%, transparent); color: var(--fs-success-fg); }
.task-items {
list-style: none;
margin: 0;
padding: 0;
}
.task-row {
display: flex;
align-items: center;
gap: 0.4rem;
padding: 0.35rem 0.65rem 0.35rem 1.4rem;
cursor: pointer;
border-bottom: 1px solid color-mix(in srgb, var(--fs-border-color) 50%, transparent);
}
.task-row:hover { background: color-mix(in srgb, var(--fs-accent) 5%, var(--fs-surface-hover)); }
.task-row:last-child { border-bottom: none; }
.status-dot {
flex-shrink: 0;
width: 1.35rem;
height: 1.35rem;
border-radius: 50%;
border: 1.5px solid var(--fs-border-color);
background: none;
cursor: pointer;
font-size: 0.62rem;
display: flex;
align-items: center;
justify-content: center;
}
.status-dot.status-in_progress { border-color: var(--fs-accent); color: var(--fs-accent); }
.status-dot.status-done { border-color: var(--fs-success); color: var(--fs-success); }
.task-title {
flex: 1;
font-size: 0.83rem;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: var(--fs-text-primary);
}
.task-title.done { text-decoration: line-through; color: var(--fs-text-tertiary); }
.task-age {
font-size: 0.68rem;
color: var(--fs-text-tertiary);
white-space: nowrap;
flex-shrink: 0;
}
.empty-group { padding: 0.4rem 1.4rem; font-size: 0.78rem; color: var(--fs-text-tertiary); }
.state-msg { padding: 1.5rem; text-align: center; font-size: 0.85rem; color: var(--fs-text-tertiary); }
/* ── Detail pane (bottom split) ── */
.task-detail {
flex: 1;
min-height: 0;
display: flex;
flex-direction: column;
overflow: hidden;
background: var(--fs-surface-hover);
border-top: 2px solid var(--fs-border-color);
}
.detail-header {
display: flex;
align-items: center;
gap: 0.6rem;
padding: 0.6rem 0.75rem;
border-bottom: 1px solid var(--fs-border-color);
flex-shrink: 0;
}
/* An interactive CYCLER, not a chip: it is clickable, outlined and
transparent. It shared a name with the task chip and was never the same
shape (#3132). */
.status-cycler {
padding: 0.2rem 0.55rem;
border-radius: 12px;
font-size: 0.75rem;
font-weight: 500;
cursor: pointer;
border: 1.5px solid var(--fs-border-color);
background: none;
text-transform: capitalize;
user-select: none;
margin-left: auto;
}
.status-cycler.status-in_progress { border-color: var(--fs-accent); color: var(--fs-accent-fg); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
.status-cycler.status-done { border-color: var(--fs-success); color: var(--fs-success-fg); background: color-mix(in srgb, var(--fs-success) 10%, transparent); }
.btn-edit-task { margin-left: 0.25rem; }
.btn-edit-task:hover { text-decoration: underline; }
.detail-body {
padding: 0.5rem 0.75rem 0.5rem;
border-bottom: 1px solid var(--fs-border-color);
flex-shrink: 0;
max-height: 40%;
overflow-y: auto;
}
.body-loading {
font-size: 0.8rem;
color: var(--fs-text-tertiary);
}
.detail-body .prose {
font-size: 0.83rem;
line-height: 1.5;
color: var(--fs-text-primary);
}
.btn-delete-task { margin-left: 0.25rem; }
.btn-delete-task:hover { color: var(--fs-action-destructive); }
.btn-delete-confirm { margin-left: 0.25rem; }
.detail-meta {
display: flex;
gap: 0.4rem;
padding: 0 0.75rem 0.5rem;
flex-shrink: 0;
}
.meta-chip {
font-size: 0.72rem;
padding: 0.15rem 0.5rem;
border-radius: 10px;
background: var(--fs-surface-page);
border: 1px solid var(--fs-border-color);
color: var(--fs-text-tertiary);
text-transform: capitalize;
}
.milestone-select {
font-size: 0.72rem;
padding: 0.15rem 0.4rem;
border-radius: 10px;
background: var(--fs-surface-page);
border: 1px solid var(--fs-border-color);
color: var(--fs-text-tertiary);
cursor: pointer;
max-width: 140px;
}
.milestone-select:disabled { opacity: 0.5; cursor: default; }
.milestone-select:focus { outline: none; border-color: var(--fs-accent); }
.detail-log {
flex: 1;
overflow-y: auto;
padding: 0 0.6rem 0.6rem;
border-top: 1px solid var(--fs-border-color);
}
/* Detail fade transition */
.detail-fade-enter-active,
.detail-fade-leave-active { transition: opacity 0.15s; }
.detail-fade-enter-from,
.detail-fade-leave-to { opacity: 0; }
/* Priority dots on task rows */
.priority-dot {
width: 6px;
height: 6px;
border-radius: 2px;
flex-shrink: 0;
}
.priority-high { background: #ef4444; }
.priority-medium { background: #f59e0b; }
.priority-low { background: #3b82f6; }
/* Due date on task rows */
.task-due {
font-size: 0.65rem;
color: var(--fs-text-tertiary);
white-space: nowrap;
flex-shrink: 0;
}
.task-due.overdue {
color: var(--fs-error);
font-weight: 500;
}
/* Close detail button */
.btn-close-detail { margin-left: 0.2rem; }
</style>
@@ -4,9 +4,9 @@
* full. Not on the list row: a history entry point there would compete with
* the row's actual job.
*
* A SIBLING OF HistoryPanel.vue, NOT A REUSE OF IT, and the reason is in its
* props: `noteId` + `currentBody`, a `NoteVersion` carrying tags and pin
* columns, a fetch of /api/notes/…, a `restore` emit, and pin/unpin buttons.
* A SIBLING OF VersionHistorySection.vue (a note's history), NOT A REUSE OF
* IT, and the reason is in its props: `noteId` + `currentBody`, a note
* version carrying tags, a fetch of /api/notes/…, and a `restore` emit.
* Every one of those is note-shaped. Rules have no tags, no pins, and
* deliberately no restore, and a rule's text is EIGHT fields rather than one
* body — which changes the central question from "what changed" to "which
+1 -1
View File
@@ -2,7 +2,7 @@
* Line diff — one copy, for every surface that shows what changed.
*
* WHY THIS FILE EXISTS. The one LCS diff walk, shared by `useAssist.ts`,
* `HistoryPanel.vue`, `VersionHistorySection.vue` and the rule edit history.
* `VersionHistorySection.vue` and the rule edit history.
* A copy per consumer means a fix has to be found in N places by someone who
* does not know N.
*/
+21 -7
View File
@@ -19,6 +19,7 @@ import {
type FamilyIdea, type IdeaSnapshot, type IdeaStatus,
} from "@/api/family";
import FamilyAdoptionMatrix from "@/components/FamilyAdoptionMatrix.vue";
import ScrollSentinel from "@/components/ScrollSentinel.vue";
import { useToastStore } from "@/stores/toast";
import { fmtStamp } from "@/utils/dateFormat";
import { recordHref } from "@/utils/recordHref";
@@ -36,6 +37,8 @@ const loading = ref(true);
const failed = ref(false);
const moreDecisions = ref(false);
const loadingMore = ref(false);
const moreFailed = ref(false);
const decisionOffset = ref(0);
// One pending reversal at a time: which row, which act, and its reason.
const pending = ref<{ kind: "undo" | "retire"; id: number } | null>(null);
@@ -63,7 +66,9 @@ async function load() {
criteria.value = ideaData.criteria;
conflictOrder.value = ideaData.conflict_order ?? [];
decisions.value = log;
decisionOffset.value = log.length;
moreDecisions.value = log.length === PAGE;
moreFailed.value = false;
} catch {
// Said out loud: a log that failed to load must not read as "nothing
// has been decided".
@@ -73,14 +78,20 @@ async function load() {
}
}
// Older decisions load as the log is scrolled. The log is offset-paged, so a
// decision recorded since the first page shifts the rest down by one; a row
// already shown is skipped, and the offset still counts what the server sent.
async function loadMore() {
loadingMore.value = true;
moreFailed.value = false;
try {
const next = await listFamilyDecisions(PAGE, decisions.value.length);
decisions.value.push(...next);
const next = await listFamilyDecisions(PAGE, decisionOffset.value);
const shown = new Set(decisions.value.map((d) => d.id));
decisions.value.push(...next.filter((d) => !shown.has(d.id)));
decisionOffset.value += next.length;
moreDecisions.value = next.length === PAGE;
} catch (e: unknown) {
toast.show(apiErrorMessage(e, "Could not load more decisions"), "error");
} catch {
moreFailed.value = true;
} finally {
loadingMore.value = false;
}
@@ -343,9 +354,12 @@ onMounted(load);
</form>
</li>
</ul>
<button v-if="moreDecisions" class="btn-secondary btn-sm" :disabled="loadingMore" @click="loadMore">
{{ loadingMore ? "Loading…" : "Older decisions" }}
</button>
<ScrollSentinel
:cursor="moreDecisions ? String(decisionOffset) : ''"
:loading="loadingMore"
:failed="moreFailed"
@more="loadMore"
/>
</section>
</template>
</main>
+50 -67
View File
@@ -1,5 +1,5 @@
<script setup lang="ts">
import { ref, computed, watch, onMounted, onUnmounted, nextTick } from "vue";
import { ref, computed, watch, onMounted, onUnmounted } from "vue";
import { useRouter } from "vue-router";
import { apiGet } from "@/api/client";
import type { TaskKind, TaskStatus, TaskPriority } from "@/types/note";
@@ -10,6 +10,7 @@ import UsageBadge from "@/components/UsageBadge.vue";
import NoteSweepPane from "@/components/NoteSweepPane.vue";
import StatusBadge from "@/components/StatusBadge.vue";
import PriorityBadge from "@/components/PriorityBadge.vue";
import ScrollSentinel from "@/components/ScrollSentinel.vue";
import GraphView from "@/views/GraphView.vue";
import {
FileText,
@@ -193,7 +194,7 @@ function onClickOutsideNewNote(e: MouseEvent) {
const ID_BATCH = 100; // IDs fetched per server round-trip
const INITIAL_CONTENT = 50; // items loaded on first render
const CONTENT_PAGE = 24; // items loaded per sentinel trigger
const CONTENT_PAGE = 24; // items loaded each time the end of the feed nears
const REFILL_THRESHOLD = 48; // fetch more IDs when queue drops below this
const items = ref<KnowledgeItem[]>([]);
@@ -214,11 +215,21 @@ const totalKnowledge = ref(0); // total matching items on server
const allIdsFetched = ref(false); // no more ID pages to fetch
const idsFetching = ref(false); // ID batch request in-flight
const contentFetching = ref(false); // content batch request in-flight
const sentinelVisible = ref(false); // updated by IntersectionObserver
const moreFailed = ref(false); // the last page asked for did not load
const loading = computed(() => idsFetching.value || contentFetching.value);
let fetchGen = 0; // incremented on each reset to invalidate stale responses
const feedGen = ref(0); // fetchGen, readable by the template
// What the next page is keyed on: the feed it belongs to and how many of its ids
// have been loaded. Ids, not items: a batch can return fewer records than it
// asked for, and the feed must still move on. Empty once every id is loaded.
const nextCursor = computed(() =>
idQueue.value.length > 0 || !allIdsFetched.value
? `${feedGen.value}:${idOffset.value - idQueue.value.length}`
: "",
);
function buildFilterParams(): URLSearchParams {
const p = new URLSearchParams();
@@ -244,7 +255,9 @@ async function fetchIdBatch(gen: number) {
idOffset.value += data.ids.length;
totalKnowledge.value = data.total;
if (!data.has_more) allIdsFetched.value = true;
} catch { /* silent */ }
} catch {
if (gen === fetchGen) moreFailed.value = true;
}
finally { if (gen === fetchGen) idsFetching.value = false; }
}
@@ -262,26 +275,38 @@ async function loadNextContent(gen: number, count: number) {
if (idQueue.value.length < REFILL_THRESHOLD && !allIdsFetched.value && !idsFetching.value) {
fetchIdBatch(gen);
}
} catch { /* silent */ }
finally {
if (gen !== fetchGen) return;
contentFetching.value = false;
// Sentinel may still be visible if new items didn't push it off screen
await nextTick();
if (sentinelVisible.value && idQueue.value.length > 0 && gen === fetchGen) {
loadNextContent(gen, CONTENT_PAGE);
} catch {
// The ids go back to the front of the queue, so Try again asks for the
// same records rather than skipping past them.
if (gen === fetchGen) {
idQueue.value.unshift(...toLoad);
moreFailed.value = true;
}
}
finally {
if (gen === fetchGen) contentFetching.value = false;
}
}
// The next page of the feed: more ids first when the queue has run dry.
async function loadMore() {
const gen = fetchGen;
moreFailed.value = false;
if (idQueue.value.length === 0) await fetchIdBatch(gen);
if (gen !== fetchGen) return;
await loadNextContent(gen, CONTENT_PAGE);
}
async function reset() {
fetchGen++;
feedGen.value = fetchGen;
const gen = fetchGen;
items.value = [];
idQueue.value = [];
idOffset.value = 0;
totalKnowledge.value = 0;
allIdsFetched.value = false;
moreFailed.value = false;
await fetchIdBatch(gen);
if (gen !== fetchGen) return;
await loadNextContent(gen, INITIAL_CONTENT);
@@ -294,23 +319,16 @@ async function fetchTags() {
} catch { /* silent */ }
}
async function resetAndReobserve() {
await reset();
await nextTick();
setupObserver();
}
function onSearchInput() {
if (searchDebounce) clearTimeout(searchDebounce);
searchDebounce = setTimeout(() => resetAndReobserve(), 380);
searchDebounce = setTimeout(() => reset(), 380);
}
watch([activeType, sortMode], () => resetAndReobserve());
// Closing the sweep remounts the feed, and with it the scroll sentinel — a
// fresh element the old observer is not watching. Without this the list loads
// its first page and then never loads another.
watch(sweepActive, (open) => { if (!open) resetAndReobserve(); });
watch(activeTag, () => { fetchCounts(); resetAndReobserve(); });
watch([activeType, sortMode], () => reset());
// Closing the sweep starts the feed over, so a note confirmed in the sweep is
// shown as it now stands.
watch(sweepActive, (open) => { if (!open) reset(); });
watch(activeTag, () => { fetchCounts(); reset(); });
// ─── Today bar ────────────────────────────────────────────────────────────────
@@ -377,24 +395,6 @@ function formatDate(iso: string): string {
return d.toLocaleDateString(undefined, { month: "short", day: "numeric", year: "numeric" });
}
// ─── Sentinel observer ────────────────────────────────────────────────────────
const sentinelEl = ref<HTMLElement | null>(null);
let observer: IntersectionObserver | null = null;
function setupObserver() {
observer?.disconnect();
if (!sentinelEl.value) return;
observer = new IntersectionObserver(
([entry]) => {
sentinelVisible.value = entry.isIntersecting;
if (entry.isIntersecting) loadNextContent(fetchGen, CONTENT_PAGE);
},
{ root: null, rootMargin: "200px" }
);
observer.observe(sentinelEl.value);
}
// ─── Lifecycle ────────────────────────────────────────────────────────────────
onMounted(async () => {
@@ -403,14 +403,11 @@ onMounted(async () => {
fetchTags();
fetchCounts();
fetchTodayBar();
await nextTick();
setupObserver();
});
onUnmounted(() => {
document.removeEventListener('click', onClickOutsideNewNote);
if (searchDebounce) clearTimeout(searchDebounce);
observer?.disconnect();
});
</script>
@@ -583,10 +580,7 @@ onUnmounted(() => {
</div>
<!-- The whole feed stands down while the sweep is open: two answers
to two different questions on one screen is neither. Wrapped
rather than given an extra v-if branch, because the scroll
sentinel lives inside the grid and the observer must not be left
holding a ref to something that never renders. -->
to two different questions on one screen is neither. -->
<template v-if="!sweepActive">
<!-- Loading / empty -->
<div v-if="loading && items.length === 0" class="knowledge-empty">Loading…</div>
@@ -675,10 +669,12 @@ onUnmounted(() => {
</div>
</div>
<!-- Sentinel — IntersectionObserver triggers next content batch -->
<div ref="sentinelEl" class="scroll-sentinel">
<span v-if="contentFetching" class="sentinel-loading">Loading…</span>
</div>
<ScrollSentinel
:cursor="nextCursor"
:loading="loading"
:failed="moreFailed"
@more="loadMore"
/>
</div>
</template>
</div>
@@ -1115,19 +1111,6 @@ onUnmounted(() => {
opacity: 0.85;
}
/* ── Sentinel ────────────────────────────────────────────── */
.scroll-sentinel {
grid-column: 1 / -1;
height: 40px;
display: flex;
align-items: center;
justify-content: center;
}
.sentinel-loading {
font-size: 0.8rem;
color: var(--fs-text-tertiary);
}
/* ── Graph panel ─────────────────────────────────────────── */
.graph-panel {
width: 500px;
+41 -140
View File
@@ -9,8 +9,10 @@ import { apiGet, apiPost, apiPut, apiDelete, listGroups, createGroup, deleteGrou
import type { User } from "@/types/auth";
import PaginationBar from "@/components/PaginationBar.vue";
import MomentsSettings from "@/components/MomentsSettings.vue";
import ReverseProxySettings from "@/components/ReverseProxySettings.vue";
import ReplyShapesSettings from "@/components/ReplyShapesSettings.vue";
import BackupScheduleSettings from "@/components/BackupScheduleSettings.vue";
import RestoreBackupDialog from "@/components/RestoreBackupDialog.vue";
import { formatBytes } from "@/utils/bytes";
import { fmtDate, fmtLogStamp } from "@/utils/dateFormat";
import { fetchVersion, type VersionPayload } from "@/api/version";
@@ -159,16 +161,10 @@ const kbWritePathEnabled = ref(true);
// code embeddings sit on a much higher similarity floor than prose, so 0.55
// lets unrelated code through. Shares top-k, not the threshold.
const kbWritePathThreshold = ref("0.68");
const kbRuleHintThreshold = ref("0.72");
// One bar for every rule hint — prompt, command and edit alike. They search
// the same rules, and what limits each is its budget, not the bar.
const kbRuleThreshold = ref("0.70");
const kbCheckpointThreshold = ref("0.80");
// The two ACT arms do not share a bar. A write-path query is a code payload; a
// pre-tool query is a shell command, often under a dozen words — less text,
// less signal, lower scores for the same relevance — so one shared bar makes
// them behave like different subsystems.
const kbToolRuleThreshold = ref("0.68");
// And the prompt boundary is a third query shape again — the operator's own
// prose rather than anything a tool produced.
const kbPromptRuleThreshold = ref("0.72");
// The reply arm: its bar is the bar at which a finished reply
// is HELD for one read, so it sits with the checkpoint, not with the hints.
const kbReplyRuleThreshold = ref("0.8");
@@ -367,14 +363,10 @@ async function saveKbInject() {
// Same `|| default` reasoning: falling back to 0 would surface every
// snippet in the corpus on every edit, which is the failure this knob fixes.
const wpT = asBar(kbWritePathThreshold.value, 0.68);
// Same `|| default` reasoning again, and it bites harder here: a rule hint
// fires on every write, so a fallback of 0 would attach a standing rule to
// every edit in the session.
const rhT = asBar(kbRuleHintThreshold.value, 0.72);
// Same `|| default` guard, and the same reason: this arm fires before every
// Bash call, so a fallback of 0 would put a rule in front of every command.
const trT = asBar(kbToolRuleThreshold.value, 0.68);
const prT = asBar(kbPromptRuleThreshold.value, 0.72);
// Same `|| default` reasoning again, and it bites harder here: rule hints
// fire on every edit and before every command, so a fallback of 0 would put
// a standing rule beside each of them.
const ruleT = asBar(kbRuleThreshold.value, 0.7);
// The checkpoint bar, and the `|| default` guard matters most here of all:
// this is the only number that can STOP a call, so a fallback of 0 would
// hold the first command of every session behind whatever ranked first.
@@ -409,10 +401,8 @@ async function saveKbInject() {
kbGateNoteOverlapFloor.value = String(gateOverlap);
kbPlanMatchThreshold.value = String(planT);
kbWritePathThreshold.value = String(wpT);
kbRuleHintThreshold.value = String(rhT);
kbRuleThreshold.value = String(ruleT);
kbCheckpointThreshold.value = String(cpT);
kbToolRuleThreshold.value = String(trT);
kbPromptRuleThreshold.value = String(prT);
kbReplyRuleThreshold.value = String(ryT);
savingKbInject.value = true;
kbInjectSaved.value = false;
@@ -426,20 +416,15 @@ async function saveKbInject() {
// measurements that split them.
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
kb_writepath_threshold: String(wpT),
// A THIRD corpus with a third bar — see RULEHINT_DEFAULT_THRESHOLD
// in services/plugin_context.py for why rules cannot share the
// code threshold any more than code could share the prose one.
kb_rulehint_threshold: String(rhT),
// A third corpus with a third bar — see RULE_FLOOR_DEFAULT in
// services/retrieval_pipeline.py for why every rule hint shares it.
kb_rule_threshold: String(ruleT),
// The one bar that stops a call rather than annotating it. Its own key,
// never derived from the two rule bars above: it answers a different
// never derived from the rule bar above: it answers a different
// question of the same scores — not "is this worth showing" but "is the
// corpus confident enough to be read first" — and a number that moves
// when another moves is a number nobody can reason about.
kb_checkpoint_threshold: String(cpT),
// A FOURTH and FIFTH bar, and they are separate keys on purpose: one
// number cannot serve arms whose queries are different shapes. Moving one must not move the others.
kb_toolrule_threshold: String(trT),
kb_promptrule_threshold: String(prT),
// The reply backstop's own bar: it HOLDS a reply, so like the
// checkpoint it is never derived from a hint bar.
kb_replyrule_threshold: String(ryT),
@@ -484,7 +469,7 @@ const confirmNewPassword = ref("");
const changingPassword = ref(false);
const invalidatingSessions = ref(false);
const exporting = ref(false);
const restoring = ref(false);
const showRestore = ref(false);
// Backup, export and restore walk the whole store, so they are slow BY DESIGN
// and the client's ordinary 30s default would cut them off mid-work. They are
// still bounded — every wait has a deadline, not necessarily a short one — and
@@ -526,7 +511,6 @@ async function copyCommit() {
commitCopied.value = true;
setTimeout(() => { commitCopied.value = false; }, 2000);
}
const restoreFileInput = ref<HTMLInputElement | null>(null);
// Migrate stored "admin" → "config"; unknown tabs fall back to "general"
const VALID_TABS = new Set(["general", "account", "notifications", "integrations", "data", "apikeys", "config", "users", "logs", "groups", "areas", "platforms"]);
@@ -864,14 +848,8 @@ onMounted(async () => {
if (allSettings.kb_checkpoint_threshold !== undefined) {
kbCheckpointThreshold.value = allSettings.kb_checkpoint_threshold;
}
if (allSettings.kb_rulehint_threshold !== undefined) {
kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
}
if (allSettings.kb_toolrule_threshold !== undefined) {
kbToolRuleThreshold.value = allSettings.kb_toolrule_threshold;
}
if (allSettings.kb_promptrule_threshold !== undefined) {
kbPromptRuleThreshold.value = allSettings.kb_promptrule_threshold;
if (allSettings.kb_rule_threshold !== undefined) {
kbRuleThreshold.value = allSettings.kb_rule_threshold;
}
if (allSettings.kb_replyrule_threshold !== undefined) {
kbReplyRuleThreshold.value = allSettings.kb_replyrule_threshold;
@@ -1102,10 +1080,6 @@ async function exportNotes(format: "markdown" | "json") {
}
}
function triggerRestoreUpload() {
restoreFileInput.value?.click();
}
async function saveNotifications() {
savingNotifications.value = true;
notificationsSaved.value = false;
@@ -1302,42 +1276,6 @@ async function runDbMaintenanceNow() {
}
}
async function handleRestoreFile(event: Event) {
const file = (event.target as HTMLInputElement).files?.[0];
if (!file) return;
restoring.value = true;
try {
// The scheduled backup's .tar goes up as it is; the JSON download is
// parsed first so a damaged file fails here, before the upload.
const isTar = file.name.toLowerCase().endsWith(".tar");
const res = await fetch("/api/admin/restore", {
method: "POST",
headers: { "Content-Type": isTar ? "application/x-tar" : "application/json" },
body: isTar ? file : JSON.stringify(JSON.parse(await file.text())),
signal: bulkDeadline(),
});
if (res.status === 413) {
throw new Error(
"the file is bigger than the web upload allows. Restore it with " +
"`python -m grimoire.restore <file>` inside the Grimoire container instead.",
);
}
if (!res.ok) {
const body = await res.json().catch(() => ({ error: `Error ${res.status}` }));
throw new Error((body as Record<string, string>).error || `Error ${res.status}`);
}
const result = await res.json();
toastStore.show(
`Restored ${result.stats?.users ?? 0} users, ${result.stats?.notes ?? 0} notes, ${result.stats?.conversations ?? 0} conversations`
);
} catch (e) {
toastStore.show("Restore failed: " + (e as Error).message, "error");
} finally {
restoring.value = false;
if (restoreFileInput.value) restoreFileInput.value.value = "";
}
}
async function testSearch() {
const q = searchQuery.value.trim();
if (!q) return;
@@ -1772,10 +1710,10 @@ async function deleteUser(userId: number) {
<p class="field-hint">How many snippets and issues one edit may be offered (1–10). Reach for this rather than the threshold when the hint feels long: lowering it keeps the best matches and drops the tail, while raising the bar above throws away good matches along with weak ones.</p>
</div>
<div class="field">
<label for="kb-rulehint-threshold">Standing-rule confidence threshold (0–1)</label>
<label for="kb-rule-threshold">Standing-rule confidence threshold (0–1)</label>
<input
id="kb-rulehint-threshold"
v-model="kbRuleHintThreshold"
id="kb-rule-threshold"
v-model="kbRuleThreshold"
type="number"
min="0"
max="1"
@@ -1784,14 +1722,14 @@ async function deleteUser(userId: number) {
style="max-width: 8rem"
/>
<p class="field-hint">
The same hint can mention a standing rule whose trigger resembles the
code being written. <em>Every</em> rule is eligible — nothing is
preloaded any more, so this is the only way a rule reaches a write.
Stricter than the threshold above, because there are far fewer rules
than snippets: with a small set something always ranks first, so the
bar has to carry more of the judgement. If rules arrive unread, or one
you needed never showed up, that is Claude's to notice and correct
from the telemetry — and the budget below is usually the better lever.
The bar for a standing rule or preference to be offered at all,
whether against the code being written, a command about to run, or
what you just said. <em>Every</em> rule is eligible — nothing is
preloaded, so this is how a rule reaches the work. One bar serves all
three: they search the same rules, and how many each may show is set
by its own budget below, which is usually the better lever. If rules
arrive unread, or one you needed never showed up, that is Claude's to
notice and correct from the telemetry.
</p>
</div>
<div class="field">
@@ -1834,26 +1772,6 @@ async function deleteUser(userId: number) {
only — file edits are never held.
</p>
</div>
<div class="field">
<label for="kb-toolrule-threshold">Command confidence threshold (0–1)</label>
<input
id="kb-toolrule-threshold"
v-model="kbToolRuleThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a rule surfacing before a <em>command</em> runs, where the
query is the command text rather than code. Lower than the one above
on purpose: a shell command is short, so it scores lower for the same
relevance — at a shared bar this arm spoke on 2% of calls against the
write path's 37%.
</p>
</div>
<div class="field">
<label for="kb-toolrule-topk">Max rules per command</label>
<input
@@ -1868,26 +1786,6 @@ async function deleteUser(userId: number) {
/>
<p class="field-hint">How many standing rules one command may be shown (1–10). This is the busiest arm there is — it fires before every command — so its budget is the one most worth keeping small.</p>
</div>
<div class="field">
<label for="kb-promptrule-threshold">Prompt confidence threshold (0–1)</label>
<input
id="kb-promptrule-threshold"
v-model="kbPromptRuleThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The bar for a rule or preference surfacing against <em>what you just
said</em>, before anything is done. It is the only moment that reaches
a rule about how to answer rather than how to act, so it runs once a
turn rather than once a tool call. A third query shape — your prose,
not a command or a file — which is why it carries its own number.
</p>
</div>
<div class="field">
<label for="kb-promptrule-topk">Max rules per prompt</label>
<input
@@ -2492,17 +2390,11 @@ async function deleteUser(userId: number) {
<button class="btn-secondary" @click="exportData('full')" :disabled="exporting">
{{ exporting ? "Exporting..." : "Full Backup" }}
</button>
<button class="btn-secondary btn-warn" @click="triggerRestoreUpload" :disabled="restoring">
{{ restoring ? "Restoring..." : "Restore from Backup" }}
<button class="btn-secondary btn-warn" @click="showRestore = true">
Restore from Backup
</button>
<input
ref="restoreFileInput"
type="file"
accept=".json,.tar"
class="hidden-file-input"
@change="handleRestoreFile"
/>
</div>
<RestoreBackupDialog v-if="showRestore" @close="showRestore = false" />
</section>
<section class="settings-section full-width">
<h2>Scheduled backups</h2>
@@ -2773,6 +2665,16 @@ async function deleteUser(userId: number) {
<div v-else class="empty-msg">Nothing asked yet.</div>
</section>
<section class="settings-section full-width">
<h2>Reverse proxy</h2>
<p class="section-desc">
How many proxies a request passes through before it reaches Grimoire. Sign-in rate
limits and the audit log key on the visitor's address, read through this count; if it
is too low, every visitor reads as the proxy and one run of bad sign-ins blocks everyone.
</p>
<ReverseProxySettings />
</section>
<section class="settings-section full-width">
<h2>Application URL</h2>
<p class="section-desc">
@@ -3660,7 +3562,6 @@ async function deleteUser(userId: number) {
flex-wrap: wrap;
gap: 0.5rem;
}
.hidden-file-input { display: none; }
/* Checkboxes */
.checkbox-field { margin-bottom: 1rem; }
+2 -1
View File
@@ -1,10 +1,11 @@
{
"name": "grimoire",
"description": "Grimoire for Claude Code: connects the grimoire MCP server, adds the hooks that deliver live project state and relevant records at the right moment, ships the shared client-neutral Grimoire skills (using-grimoire, writing-plans, reporting-back, systematic-debugging, verification, brainstorming, reusing-code, shape-accounting, family-canon), and syncs your saved Grimoire Processes as skills (/grimoire:sync).",
"version": "2026.10.21.2344",
"version": "2026.11.05.0301",
"author": {
"name": "Bryan Van Deusen"
},
"license": "AGPL-3.0-only",
"mcpServers": {
"grimoire": {
"type": "http",
+2 -1
View File
@@ -15,7 +15,7 @@ another one means adding files, not moving or rewriting any.
|---|---|---|
| **The skills** | `plugin/skills/*/SKILL.md` | Agent Skills (the open SKILL.md format). They state every Grimoire reflex in full and name no client. `tests/test_guidance_ownership.py` fails if a skill names a particular client, or references anything outside its own folder. Every client package ships this folder verbatim. |
| **The MCP server** | `<base URL>/mcp` | HTTP, `Authorization: Bearer <fmcp_ key>`. Its `_INSTRUCTIONS` is a client-neutral orientation — the workflow across tools (≤1,600 chars); each tool's description carries its contract; in-band responses (`placement`, `report_back`, `systems_hint`, the duplicate gate, the guessed-id refusal) fire in every client. |
| **The adapter API** | `<base URL>/api/plugin/*` | Endpoints any client's hooks can call with the same key. `GET`, so read scope is enough: `context` (live session state), `retrieve` (rules, preferences and notes for a message), `prior-art` (records and shape-ledger hints for code being written), `tool-rules` (rules for a command about to run), `moment-tools` (which tools reach a mounted moment), `shape-check` (definitions written this turn that nobody has judged), `claim-session` / `release-session` (task claims), `processes` (stored Processes to expose as skills). `POST`, so write scope: `moment` (rules mounted on the moments a tool call reaches), `rule-opened` (a rule was read), `reply-rules` (the finished reply, held for an unopened rule or — when the turn closed a task — for missing completion sections). |
| **The adapter API** | `<base URL>/api/plugin/*` | Endpoints any client's hooks can call with the same key. `GET`, so read scope is enough: `context` (live session state), `retrieve` (rules, preferences and notes for a message), `prior-art` (records and shape-ledger hints for code being written), `tool-rules` (rules for a command about to run), `moment-tools` (which tools reach a mounted moment), `shape-check` (definitions written this turn that nobody has judged), `claim-session` / `release-session` (task claims), `write-scope` (which project a pending write lands in, and whether this session may make it), `processes` (stored Processes to expose as skills). `POST`, so write scope: `moment` (rules mounted on the moments a tool call reaches), `rule-opened` (a rule was read), `reply-rules` (the finished reply, held for an unopened rule or — when the turn closed a task — for missing completion sections). |
| **The API key** | Grimoire → Settings → MCP Access | One `fmcp_` key per install. Write scope covers the MCP tools and every hook; a read key runs only the `GET` hooks. |
## Added by each client
@@ -39,6 +39,7 @@ another one means adding files, not moving or rewriting any.
| `hooks/grimoire_prior_art.sh` | PreToolUse on editor writes: `GET /api/plugin/prior-art`. |
| `hooks/grimoire_after_write.sh` | PostToolUse on shell commands: the same check for code written through the shell. |
| `hooks/grimoire_tool_rules.sh` | PreToolUse on shell commands: `GET /api/plugin/tool-rules`. |
| `hooks/grimoire_project_fence.sh` | PreToolUse on editor writes, shell commands and Grimoire's tools: `GET /api/plugin/write-scope` for an act aimed at another project, answered `ask` with the reason; PostToolUse on `enter_project` remembers the project a session without one entered. |
| `hooks/grimoire_moment.sh` | PreToolUse on every tool: the rules mounted on the moments the call reaches, `POST /api/plugin/moment`; skips tools `GET /api/plugin/moment-tools` says reach nothing mounted, and Grimoire's own (their responses carry `moment_rules`). |
| `hooks/grimoire_reply_check.sh` | Stop: sends the finished reply to `POST /api/plugin/reply-rules` — the rules mounted on the reply moments, and the reply against every rule's trigger, and — when the turn closed a task — the completion-section check; holds once per rule, with the reason the server returns, and never holds the rewrite. |
| `hooks/grimoire_shape_check.sh` | Stop: sends the definitions the turn wrote (the write hooks' `<sid>.written.ids` ledger) to `GET /api/plugin/shape-check`; blocks once, with the reason the server returns, so the agent judges what it built. |
+16 -2
View File
@@ -1,6 +1,6 @@
# Grimoire plugin for Claude Code
Turns a self-hosted [Grimoire](https://git.fabledsword.com/bvandeusen/FabledScribe)
Turns a self-hosted [Grimoire](https://git.fabledsword.com/bvandeusen/FabledGrimoire)
instance into a first-class Claude Code extension:
- **MCP tools** over your notes, tasks, projects, milestones, systems, and
@@ -41,7 +41,7 @@ The plugin ships inside the Grimoire app repo, so the marketplace *is* that repo
you always get the plugin version that matches your Grimoire instance.
```
/plugin marketplace add https://git.fabledsword.com/bvandeusen/FabledScribe.git
/plugin marketplace add https://git.fabledsword.com/bvandeusen/FabledGrimoire.git
/plugin install grimoire@grimoire-plugin
```
@@ -79,6 +79,20 @@ On install you'll be asked for:
message, as a short menu of titles and matched passages.
- `hooks/hooks.json` → PreToolUse hook on `Bash` (`hooks/grimoire_tool_rules.sh`)
→ `GET /api/plugin/tool-rules`: the rules that match the command about to run.
- `hooks/hooks.json` → PreToolUse hook on editor writes, `Bash` and Grimoire's
tools (`hooks/grimoire_project_fence.sh`) → `GET /api/plugin/write-scope`:
holds a session to its own project. A session belongs to the project its
starting directory resolves to (a `.grimoire` marker, else the git remote's
binding), or, started outside any, to the first project it enters. Editing
another project's files, a git commit, push, merge, rebase, pull, tag or
reset in its repo (from `git -C`, after `cd`, or in a worktree of it), and a
Grimoire write that lands in another project each stop on a permission
prompt naming that project, in every permission mode. Filing an issue there
(`create_task(kind="issue")`) and every read go ahead. An edit in the
session's own repo never calls the server. It can't see a non-git shell
write into another repo (`sed -i`, a redirect), but it fences that repo's
commit and push. When the instance doesn't answer, the act goes ahead and
the hook says so once per outage.
- `hooks/hooks.json` → PreToolUse hook on every tool (`hooks/grimoire_moment.sh`)
→ `POST /api/plugin/moment`: the rules mounted on the moments the call
reaches. It skips tools that `GET /api/plugin/moment-tools` says reach
+14 -8
View File
@@ -119,10 +119,11 @@ combined=""
n_files=0
while IFS= read -r rel_path; do
[ -n "${rel_path:-}" ] || continue
# A Bash call that rewrote many files is a refactor or a generator, not a
# shape being instantiated; four is enough to name what matters.
# Every changed file goes into the end-of-turn ledger, up to forty — a bound
# on this hook's run time, past which the call was a generator. The arms
# below speak for the first four only (see the check after the ledger).
n_files=$((n_files + 1))
[ "$n_files" -le 4 ] || break
[ "$n_files" -le 40 ] || break
file_path="$repo_root/$rel_path"
# The code just written: the ADDED lines of the uncommitted diff for a
@@ -136,14 +137,19 @@ while IFS= read -r rel_path; do
fresh="new"
fi
[ -n "$code" ] || continue
# `|| true` inside: an early-exiting `head` must not void its own output.
names=$(printf '%s' "$code" | grimoire_defs | sort -u | head -12 || true)
# What this write defined, for the end-of-turn question —
all_names=$(printf '%s' "$code" | grimoire_defs | sort -u || true)
# What this write defined, for the end-of-turn question: every definition,
# recorded before the arms below, which skip a write that defines nothing.
printf '%s\n' "$names" | grimoire_written_append "$safe_sid" "$rel_path" "$fresh" "$code"
printf '%s\n' "$all_names" | grimoire_written_append "$safe_sid" "$rel_path" "$fresh" "$code"
# Nothing DEFINED in what was written (prose, data, a call-site edit) →
# nothing to say; the arms are about shapes.
[ -n "$names" ] || continue
[ -n "$all_names" ] || continue
# A Bash call that rewrote many files is a refactor or a generator, not a
# shape being instantiated; four files and twelve names per file are enough
# for the arms to name what matters, and bound their git grep and URL.
[ "$n_files" -le 4 ] || continue
# `|| true` inside: an early-exiting `head` must not void its own output.
names=$(printf '%s\n' "$all_names" | head -12 || true)
local_lines=$(grimoire_local_dups "$repo_root" "$rel_path" <<< "$names") || local_lines=""
local_context=""
+1 -1
View File
@@ -1308,7 +1308,7 @@ grimoire_turn_fact() {
# Nothing Grimoire should be holding goes in it.
#
# {"instance": "https://grimoire.example.com", "project_id": 2,
# "project": "FabledScribe"}
# "project": "FabledGrimoire"}
#
# instance WHICH Grimoire the id belongs to, and the reason this file is
# not just a number. A project id means nothing on its own: id 2
+10 -3
View File
@@ -79,10 +79,16 @@ fi
# ARM 1 — BY NAME, LOCALLY: does a definition of this already exist
# in the repo? (grimoire_local_dups in grimoire_defs.sh carries the why.)
#
# `all_names` is every definition the write carries; it feeds the end-of-turn
# ledger, which must hear about all of them. `names` is the first twelve, and
# bounds the queries below — a git grep and a URL per name.
all_names=""
names=""
if [ -n "$code" ]; then
all_names=$(printf '%s' "$code" | grimoire_defs | sort -u || true)
# `|| true` inside: an early-exiting `head` must not void its own output.
names=$(printf '%s' "$code" | grimoire_defs | sort -u | head -12 || true)
names=$(printf '%s\n' "$all_names" | head -12 || true)
fi
local_lines=""
@@ -204,10 +210,11 @@ if [ -n "$session_id" ]; then
# Write onto a path that does not exist yet creates a file, which may be a
# unit in its own right (see grimoire_written_append). Written before any
# server call: the ask at the end of the turn must not depend on this
# write's hint having been answered.
# write's hint having been answered. Every definition, not the capped list
# the queries use; an Edit that names none records its enclosing one.
fresh=""
[ -e "$file_path" ] || fresh="new"
printf '%s\n' "$shapes" | grimoire_written_append "$safe_sid" "$rel_path" "$fresh" "$code"
printf '%s\n' "${all_names:-$shapes}" | grimoire_written_append "$safe_sid" "$rel_path" "$fresh" "$code"
idfile="$state_dir/${safe_sid}.ids"
syncfile="$state_dir/${safe_sid}.sync.ids"
derivefile="$state_dir/${safe_sid}.derive.ids"
+246
View File
@@ -0,0 +1,246 @@
#!/usr/bin/env bash
# Grimoire — hold a session to its own project.
#
# A session belongs to the project its starting directory resolves to
# (CLAUDE_PROJECT_DIR: a `.grimoire` marker, else the git remote's binding). A
# session that starts outside any project belongs to the first project it
# enters. Toward any other project a session may read anything and file an
# issue. Every other write stops on a permission prompt naming the other
# project, so the operator decides:
# - editing a file in another project's repo (Write, Edit, MultiEdit,
# NotebookEdit);
# - a git command that changes history or a remote there (commit, push,
# merge, rebase, pull, cherry-pick, revert, tag, am, reset, worktree add),
# wherever it runs from: `git -C DIR`, `cd DIR && git …`, or the shell's
# current directory;
# - a Grimoire write that lands in another project.
#
# `ask`, not `deny`. Which projects a session may change is the operator's
# call, and `ask` puts it to them each time, in every permission mode,
# bypassPermissions included.
#
# The server says where an act lands (GET /api/plugin/write-scope). Two cases
# never reach it:
# - a directory that resolves the same way as the session's own, which is
# nearly every edit;
# - a tool named like a read (READ_PREFIXES in services/write_scope.py, held
# equal to FENCE_READ_PREFIXES below by tests/test_write_scope.py).
#
# What it can't see: a non-git write from the shell (`sed -i`, a redirect)
# into another repo. That repo's commit and push are still fenced.
#
# EXIT 0, ALWAYS. When the server doesn't answer, the act goes ahead and the
# hook says once per outage that it couldn't check.
#
# Run as PostToolUse on enter_project, it records the project a session
# without one has entered, and says nothing.
set -uo pipefail
# shellcheck source=plugin/hooks/grimoire_defs.sh
. "$(dirname "${BASH_SOURCE[0]}")/grimoire_defs.sh"
FENCE_READ_PREFIXES="get_ list_ search find_ enter_project what_might_apply resolve_design_system"
FENCE_GIT_WRITES="commit push merge rebase pull cherry-pick revert tag am reset"
event=$(cat 2>/dev/null || true)
[ -n "$event" ] || exit 0
event_flat=$(printf '%s' "$event" | grimoire_json_flat)
[ -n "$event_flat" ] || exit 0
session_id=$(grimoire_json_pick "$event_flat" '.session_id')
tool_name=$(grimoire_json_pick "$event_flat" '.tool_name')
hook_event=$(grimoire_json_pick "$event_flat" '.hook_event_name')
cwd=$(grimoire_json_pick "$event_flat" '.cwd')
# Its own directory, which no compaction sweeps: the project a session
# entered stays its project for the whole session.
state_dir="${TMPDIR:-/tmp}/grimoire-fence"
safe_sid=$(printf '%s' "${session_id:-none}" | tr -c 'A-Za-z0-9._-' '_')
entered_file="$state_dir/${safe_sid}.entered"
# ── PostToolUse on enter_project: remember the first project entered ─────
if [ "$hook_event" = "PostToolUse" ]; then
case "$tool_name" in
*__enter_project)
pid=$(grimoire_json_pick "$event_flat" '.tool_input.project_id' | tr -cd '0-9')
if [ -n "$pid" ] && [ -n "$session_id" ] && [ ! -s "$entered_file" ]; then
mkdir -p "$state_dir" 2>/dev/null || true
printf '%s' "$pid" > "$entered_file" 2>/dev/null || true
fi
;;
esac
exit 0
fi
# The nearest directory at or above $1 that exists. A Write may create the
# directories its file goes in.
fence_existing_dir() {
local d="$1" depth=0
while [ -n "$d" ] && [ ! -d "$d" ] && [ "$depth" -lt 40 ]; do
d=$(dirname -- "$d" 2>/dev/null) || return 0
depth=$((depth + 1))
done
[ -d "$d" ] && printf '%s' "$d"
}
# "DIR<TAB>SUBCOMMAND" for each git command in the shell command $1 that
# changes history or a remote, with DIR resolved from $2: `cd` moves it for
# the segments after, `git -C` for that one command. Segments split on ; && ||
# | and newlines, and a quoted separator inside a message splits too; the
# words after it are read as a command only if they start with cd or git.
fence_git_targets() {
printf '%s\n' "$1" | awk -v start="$2" -v home="${HOME:-}" -v writes="$FENCE_GIT_WRITES" '
function unq(w) { gsub(/^["\047(]+|["\047)]+$/, "", w); return w }
function at(base, p) {
if (p ~ /^\//) return p
if (p == "~") return home
if (p ~ /^~\//) return home substr(p, 2)
return base "/" p
}
BEGIN { dir = start; n = split(writes, w, " "); for (i = 1; i <= n; i++) iswrite[w[i]] = 1 }
{
line = $0
gsub(/&&|\|\||;|\|/, "\n", line)
m = split(line, segs, "\n")
for (s = 1; s <= m; s++) {
k = split(segs[s], t, /[ \t]+/)
i = 1
while (i <= k && (t[i] == "" || t[i] ~ /^[A-Za-z_][A-Za-z0-9_]*=/)) i++
if (i > k) continue
cmd = unq(t[i])
if (cmd == "cd") {
if (i + 1 <= k && t[i + 1] != "") dir = at(dir, unq(t[i + 1]))
continue
}
if (cmd != "git" && cmd !~ /\/git$/) continue
gdir = dir; i++
while (i <= k && t[i] ~ /^-/) {
if (t[i] == "-C" && i + 1 <= k) { gdir = at(gdir, unq(t[i + 1])); i += 2; continue }
if (t[i] == "-c" || t[i] == "--git-dir" || t[i] == "--work-tree") { i += 2; continue }
i++
}
if (i > k) continue
sub_ = unq(t[i])
if (sub_ == "worktree") { if (i + 1 <= k && unq(t[i + 1]) == "add") print gdir "\tworktree add"; continue }
if (sub_ in iswrite) print gdir "\t" sub_
}
}' 2>/dev/null || true
}
# The short scalar arguments of the MCP call, as a JSON object of strings:
# ids, kind, status, a repo url, and arrays of ids. Long text (bodies, code) is
# left out; nothing that says where a write lands is long.
fence_tool_input() {
printf '%s\n' "$event_flat" | awk -F'\t' '
function add(k, v) { if (!(k in seen)) { seen[k] = 1; order[++n] = k } }
$2 ~ /^\.tool_input\.[A-Za-z_]+$/ && $3 != "null" && length($3) <= 200 {
k = substr($2, 13); add(k); val[k] = "\"" $3 "\""
}
$2 ~ /^\.tool_input\.[A-Za-z_]+\[[0-9]+\]$/ && $3 != "null" && length($3) <= 200 {
k = substr($2, 13); sub(/\[.*$/, "", k); add(k)
# Tested before assigning: mawk creates arr[k] as soon as it is the
# target, so the test inside the assignment would always be true.
item = "\"" $3 "\""
if (k in arr) item = arr[k] "," item
arr[k] = item
}
END {
printf "{"
for (i = 1; i <= n; i++) {
k = order[i]
printf "%s\"%s\":%s", (i > 1 ? "," : ""), k, ((k in arr) ? "[" arr[k] "]" : val[k])
}
printf "}"
}' 2>/dev/null || printf '{}'
}
grimoire_config || exit 0
command -v curl >/dev/null 2>&1 || exit 0
project_dir=${CLAUDE_PROJECT_DIR:-$PWD}
session_scope=$(grimoire_scope_query "$project_dir")
entered=$(cat "$entered_file" 2>/dev/null | tr -cd '0-9')
# No project to hold the session to: started outside any, and entered none.
[ -n "$session_scope" ] || [ -n "$entered" ] || exit 0
# Each check this act needs: "SCOPE<TAB>ACT" for a directory, or a single
# "tool" line for a Grimoire call.
checks=""
case "$tool_name" in
Write|Edit|MultiEdit|NotebookEdit)
path=$(grimoire_json_pick "$event_flat" '.tool_input.file_path')
[ -n "$path" ] || path=$(grimoire_json_pick "$event_flat" '.tool_input.notebook_path')
[ -n "$path" ] || exit 0
case "$path" in /*) ;; *) path="${cwd:-$PWD}/$path" ;; esac
dir=$(fence_existing_dir "$(dirname -- "$path")")
[ -n "$dir" ] || exit 0
scope=$(grimoire_scope_query "$dir")
[ -n "$scope" ] && [ "$scope" != "$session_scope" ] || exit 0
checks=$(printf '%s\t%s' "$scope" "Editing a file in")
;;
Bash)
command=$(grimoire_json_pick "$event_flat" '.tool_input.command')
case "$command" in *git*) ;; *) exit 0 ;; esac
while IFS=$'\t' read -r gdir gsub; do
[ -n "$gdir" ] || continue
[ -d "$gdir" ] || continue
scope=$(grimoire_scope_query "$gdir")
[ -n "$scope" ] && [ "$scope" != "$session_scope" ] || continue
checks="${checks}${checks:+$'\n'}${scope}"$'\t'"git ${gsub} in"
done <<EOF
$(fence_git_targets "$command" "${cwd:-$PWD}")
EOF
[ -n "$checks" ] || exit 0
;;
mcp__*)
short=${tool_name##*__}
for prefix in $FENCE_READ_PREFIXES; do
case "$short" in "$prefix"*) exit 0 ;; esac
done
checks="tool"
;;
*) exit 0 ;;
esac
base="${url%/}/api/plugin/write-scope?"
[ -n "$session_scope" ] && base="${base}${session_scope}&"
[ -n "$entered" ] && base="${base}fallback_project_id=${entered}&"
fence_unreached() {
local marker="$state_dir/${safe_sid}.fence.unreached" now last
mkdir -p "$state_dir" 2>/dev/null || true
now=$(date +%s 2>/dev/null) || now=0
if [ -f "$marker" ]; then
last=$(cat "$marker" 2>/dev/null) || last=0
case "$last" in ''|*[!0-9]*) last=0 ;; esac
[ $((now - last)) -lt 600 ] && return 0
fi
printf '%s' "$now" > "$marker" 2>/dev/null || true
grimoire_json_out "PreToolUse" "> Grimoire did not answer the project fence check, so this act went ahead without being checked against the session's project. If it changes another project's files or records, stop and file an issue there instead. Said once per outage."
}
while IFS= read -r check; do
[ -n "$check" ] || continue
if [ "$check" = "tool" ]; then
input=$(fence_tool_input | grimoire_urlenc)
q="tool=$(printf '%s' "$short" | grimoire_urlenc)&input=${input}"
else
scope=${check%%$'\t'*}
act=${check#*$'\t'}
q="target_${scope}&act=$(printf '%s' "$act" | grimoire_urlenc)"
fi
resp=$(curl -fsS --max-time 4 -H "Authorization: Bearer ${token}" "${base}${q}" 2>/dev/null) || {
fence_unreached
exit 0
}
rm -f "$state_dir/${safe_sid}.fence.unreached" 2>/dev/null || true
flat=$(printf '%s' "$resp" | grimoire_json_flat)
if [ "$(grimoire_json_pick "$flat" '.allowed')" = "false" ]; then
reason=$(grimoire_json_pick "$flat" '.reason')
esc=$(printf '%s' "$reason" | grimoire_json_escape) || exit 0
printf '{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"ask","permissionDecisionReason":"%s"}}\n' "$esc"
exit 0
fi
done <<EOF
$checks
EOF
exit 0
+17 -7
View File
@@ -72,19 +72,29 @@ printf '%s\n' "$rule_id" | grimoire_rules_append "$state_dir/${safe_sid}.opened.
# ── What came just before the open ───────────────────────────────────────
#
# grimoire_moment.sh writes every call to `grimoire-moment/<sid>.acts`. The ones
# from the last three minutes go to the server with this rule's id: a rule
# opened just after a moment fired is evidence it belongs on that moment, and
# once that repeats across sessions the server answers with one line asking
# the reader to offer the operator the mount. Nothing is mounted by this.
# grimoire_moment.sh writes every call to `grimoire-moment/<sid>.acts`. The
# last three, from the last ninety seconds, go to the server with this rule's
# id: a rule the agent went and opened right after a moment fired is evidence
# it belongs on that moment, and once that repeats across sessions the server
# answers with one line putting the question to the reader. Nothing is
# mounted by this.
#
# An open of a rule a hook has just NAMED is not reported. A ranked arm or a
# moment line put it in front of the reader, and that explains the open; the
# moment beside it did not, so counting it would propose mounts on whatever
# moment happened to fire while search was surfacing the rule. The window is
# short for the same reason: a few minutes of agent work reaches many moments.
command -v curl >/dev/null 2>&1 || exit 0
grimoire_config || exit 0
case ",$(grimoire_rules_live "$state_dir/${safe_sid}.rules.ids")," in
*",${rule_id},"*) exit 0 ;;
esac
acts_file="${TMPDIR:-/tmp}/grimoire-moment/${safe_sid}.acts"
[ -f "$acts_file" ] || exit 0
now=$(date +%s 2>/dev/null) || exit 0
acts=$(awk -F '\t' -v since=$((now - 180)) \
acts=$(awk -F '\t' -v since=$((now - 90)) \
'$1 + 0 >= since { sub(/^[^\t]*\t/, ""); print }' "$acts_file" 2>/dev/null \
| tail -n 30 | paste -sd ',' - 2>/dev/null)
| tail -n 3 | paste -sd ',' - 2>/dev/null)
[ -n "$acts" ] || exit 0
sid_esc=$(printf '%s' "$session_id" | grimoire_json_escape) || exit 0
+6 -4
View File
@@ -31,11 +31,13 @@ event_flat=$(printf '%s' "$event" | grimoire_json_flat)
session_id=$(grimoire_json_pick "$event_flat" '.session_id')
[ -n "$session_id" ] || exit 0
# The moment arm's per-session files (its tool cache and the acts ledger)
# describe a session that is over, on any exit — a clear
# included, since the next conversation is a new session id.
# The moment arm's per-session files (its tool cache and the acts ledger) and
# the fence's (the project the session entered, its outage marker) describe a
# session that is over, on any exit — a clear included, since the next
# conversation is a new session id.
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
rm -f "${TMPDIR:-/tmp}/grimoire-moment/${safe_sid}".* 2>/dev/null || true
rm -f "${TMPDIR:-/tmp}/grimoire-moment/${safe_sid}".* \
"${TMPDIR:-/tmp}/grimoire-fence/${safe_sid}".* 2>/dev/null || true
reason=$(grimoire_json_pick "$event_flat" '.reason')
[ "$reason" = "clear" ] && exit 0
+18
View File
@@ -25,6 +25,15 @@
}
],
"PreToolUse": [
{
"matcher": "Write|Edit|MultiEdit|NotebookEdit|Bash|mcp__.*grimoire.*",
"hooks": [
{
"type": "command",
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/grimoire_project_fence.sh\""
}
]
},
{
"matcher": "Write|Edit",
"hooks": [
@@ -89,6 +98,15 @@
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/grimoire_claim_session.sh\""
}
]
},
{
"matcher": "mcp__.*__enter_project",
"hooks": [
{
"type": "command",
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/grimoire_project_fence.sh\""
}
]
}
],
"PreCompact": [
+10 -2
View File
@@ -41,6 +41,14 @@ row carries a status:
nothing. Rows, never prose — a consumer list in a note or verification
detail cannot be sorted, queried, or diffed.
Rows are keyed by repo and path, so a move in the tree is a move in the
ledger. When a file or directory moves, run `move_path(project_id,
from_path, to_path)` before the next refresh; it carries the snippets
recorded there too. For a repo renamed on its
forge, run `bind_repo(new_url, project_id, moved_from=old_url)`. Otherwise
the refresh marks every judged row vanished and lists the same shapes again,
unjudged.
## The writer judges; audits check that it happened
The ledger is filled from your answers, given while you still know them. When
@@ -157,8 +165,8 @@ Three questions the ledger answers mechanically:
it), `variant` with the why if the departure is deliberate. The write-path
hint asks the same question in-band the moment such a shape is written.
- **History** — `shape_history(project_id, path, symbol?)`: the current rows
plus every `classified` / `vanished` / `reappeared` / `drifted` event with
its commit — "instance of #N from <date>, re-judged variant of #M because
plus every `classified` / `vanished` / `reappeared` / `drifted` / `moved`
event with its commit — "instance of #N from <date>, re-judged variant of #M because
R, vanished at C". Rows, not recollection.
- **Recheck** — `list_shapes(project_id, flag="recheck")`: judged
instances/variants whose body changed since judged. The judgment stands;
+6 -5
View File
@@ -277,11 +277,12 @@ bound — confine the session to it:
- **Pass that `project_id` to every read** (`search`, `list_tasks`,
`list_notes`). An unscoped read bleeds every other project's work into your
context.
- **Only reference or offer work on the in-scope project.** Don't surface,
suggest, or start work on other projects unless the operator explicitly widens
scope.
- If something clearly belongs to a *different* project, say so and **ask before
switching** — never silently operate cross-project.
- **Only reference or offer work on the in-scope project**, unless the
operator explicitly widens scope; work that belongs elsewhere, say so and ask.
- **Cross-project, the one change to make is filing an issue**:
`create_task(project_id=<theirs>, kind="issue")` with how you found it and the
evidence. Reading there is fine; editing its files, committing or pushing in
its repo, or changing its records is the operator's call, and the plugin asks.
## Read these when the moment comes
+10 -6
View File
@@ -73,12 +73,16 @@ than a mount is fixed through its trigger instead.
## When a line proposes a mount
A rule that keeps being opened just after the same moment, across several
sessions, probably belongs on that moment. A line says so, naming the rule,
the moment and how often. It is a question for the operator, not a change you
make: offer it in one line, and record their answer with
`judge_rule_moments` — `confirm` mounts the rule, `reject` with their reason
stops the question being asked again.
A rule that sessions keep going and opening just after the same moment —
without a hook having shown it to them first — may belong on that moment. A
line says so, naming the rule, the moment, the action that reached it and how
often. That is co-occurrence, not a recommendation, so judge it before it
reaches the operator: read the rule's trigger against what the moment means
(`list_moments`). Offer it only when the rule governs that point in the work
whatever the work is about, as a question with your reason; a rule about a
subject (a library, a file, a style) is reached by meaning, and the line is
left alone. Record their answer with `judge_rule_moments` — `confirm` mounts
the rule, `reject` with their reason stops the question being asked again.
The same tool answers proposals from a pass over the rules
(`rules_to_mount`, `propose_rule_moments`, `rule_moment_proposals`): a pass
+4 -2
View File
@@ -1,11 +1,13 @@
[build-system]
requires = ["setuptools>=68.0", "setuptools-scm>=8.0"]
# 77 is the first setuptools that reads `license` as an SPDX expression.
requires = ["setuptools>=77.0", "setuptools-scm>=8.0"]
build-backend = "setuptools.build_meta"
[project]
name = "grimoire"
version = "0.1.0"
description = "Self-hosted note-taking, project-management, and calendar app exposed to Claude via MCP"
description = "A self-hosted system of record for the work you do with Claude Code"
license = "AGPL-3.0-only"
requires-python = ">=3.14"
dependencies = [
"quart>=0.19",
+1 -1
View File
@@ -3,7 +3,7 @@
# requires-python = ">=3.10"
# dependencies = ["httpx>=0.27"]
# ///
"""Benchmark Ollama models for FabledScribe chat / curator workloads.
"""Benchmark Ollama models for FabledGrimoire chat / curator workloads.
Measures time-to-first-token, total wall time, and tokens/sec for each
(model, scenario) pair. Designed to be runnable against both local and
+17 -5
View File
@@ -395,6 +395,13 @@ SMOKE_EVENTS: dict[str, str] = {
{"session_id": "smoke", "cwd": ".", "hook_event_name": "SessionEnd",
"reason": "prompt_input_exit"}
),
# The project fence. A Grimoire write is the one act that always asks the
# server, so it is what reaches the network in the unreachable case: there
# the act goes ahead and the hook says once that it couldn't check.
"grimoire_project_fence.sh": json.dumps(
{"session_id": "smoke", "cwd": ".", "hook_event_name": "PreToolUse",
"tool_name": "mcp__grimoire__update_task", "tool_input": {"task_id": 1}}
),
# The shared library is sourced, never run; executed bare it defines
# functions and exits — silent by construction.
"grimoire_defs.sh": "",
@@ -415,9 +422,14 @@ STATIC_EMITTERS = {
# would look like a leak; it is the opposite.
"grimoire_precompact_preserve.sh": "preservation instructions present",
}
# The hooks that say so when a configured instance does not answer.
OUTAGE_SPEAKERS = {"grimoire_prior_art.sh", "grimoire_after_write.sh"}
OUTAGE_LINE = "> Grimoire did not answer the prior-art check"
# The hooks that say so when a configured instance does not answer, each with
# the line it says it in.
_PRIOR_ART_OUTAGE = "> Grimoire did not answer the prior-art check"
OUTAGE_LINES = {
"grimoire_prior_art.sh": _PRIOR_ART_OUTAGE,
"grimoire_after_write.sh": _PRIOR_ART_OUTAGE,
"grimoire_project_fence.sh": "> Grimoire did not answer the project fence check",
}
def _run_hook(script: Path, event: str, env_extra: dict[str, str]) -> subprocess.CompletedProcess:
@@ -464,13 +476,13 @@ def check_fail_open() -> None:
f"output must survive having no credentials ({what})")
else:
ok(f"{rel} [{label}]: exit 0, {what}")
elif out and label == "unreachable" and script.name in OUTAGE_SPEAKERS:
elif out and label == "unreachable" and script.name in OUTAGE_LINES:
# The only thing allowed here is the outage line itself.
try:
ctx = json.loads(out)["hookSpecificOutput"]["additionalContext"]
except (ValueError, KeyError, TypeError):
ctx = ""
if ctx.startswith(OUTAGE_LINE):
if ctx.startswith(OUTAGE_LINES[script.name]):
ok(f"{rel} [{label}]: exit 0, says the instance did not answer")
else:
fail(f"{rel} [{label}]: emitted output with no working instance "
+34 -9
View File
@@ -1,11 +1,13 @@
import logging
import os
import time
import traceback as tb_module
from pathlib import Path
from quart import Quart, g, jsonify, make_response, request, send_from_directory
from grimoire.config import Config
from grimoire.config import SECRET_KEY_FILENAME, Config
from grimoire.proxy import PerRequestSecureSession, apply_https_headers
from grimoire.routes.admin import admin_bp
from grimoire.routes.api import api
from grimoire.routes.auth import auth_bp
@@ -60,19 +62,33 @@ def create_app() -> Quart:
app.secret_key = Config.SECRET_KEY
app.config["SESSION_COOKIE_HTTPONLY"] = True
app.config["SESSION_COOKIE_SAMESITE"] = "Lax"
app.config["SESSION_COOKIE_SECURE"] = Config.SECURE_COOKIES
# Secure is decided per request, from the trusted proxy's X-Forwarded-Proto.
app.session_interface = PerRequestSecureSession()
if Config.SECRET_KEY == "dev-secret-change-me":
if os.environ.get("SECURE_COOKIES"):
logger.warning(
"SECRET_KEY is set to the default value — session cookies are insecure. "
"Set SECRET_KEY or SECRET_KEY_FILE for production use."
"SECURE_COOKIES is not used: the session cookie is Secure on every request "
"the trusted proxy reports as HTTPS. Remove it from the environment."
)
if Config.SECRET_KEY_SOURCE == "ephemeral":
logger.warning(
"No SECRET_KEY is set and %s does not exist, so this process is using a "
"temporary key. Sessions and saved credentials will not survive a restart.",
Config.DATA_DIR,
)
elif Config.SECRET_KEY_SOURCE == "data" and not os.path.ismount(Config.DATA_DIR):
logger.warning(
"%s is not a mounted volume. The instance's key (%s) and the model cache "
"are lost when the container is recreated; mount a volume there.",
Config.DATA_DIR, SECRET_KEY_FILENAME,
)
if not Config.TRUST_PROXY_HEADERS:
if os.environ.get("TRUST_PROXY_HEADERS") and not os.environ.get("TRUSTED_PROXY_HOPS"):
logger.warning(
"TRUST_PROXY_HEADERS is not set. If this instance is behind a reverse proxy "
"(nginx, Caddy, Traefik) set TRUST_PROXY_HEADERS=true so rate limiting uses "
"real client IPs rather than the proxy IP."
"TRUST_PROXY_HEADERS is not used (the starting proxy hop count is %d). "
"Remove it; set the count in Settings → Config, or TRUSTED_PROXY_HOPS "
"to change the starting value.",
Config.TRUSTED_PROXY_HOPS,
)
app.register_blueprint(admin_bp)
@@ -145,6 +161,7 @@ def create_app() -> Quart:
except Exception:
logger.debug("Failed to log usage", exc_info=True)
apply_https_headers(response)
response.headers.setdefault("X-Content-Type-Options", "nosniff")
response.headers.setdefault("X-Frame-Options", "DENY")
response.headers.setdefault("Referrer-Policy", "strict-origin-when-cross-origin")
@@ -185,6 +202,14 @@ def create_app() -> Quart:
start_notification_loop()
start_auth_token_retention_loop()
# The proxy hop count saved in Settings decides every request's client
# address, so it is loaded before serving, under a deadline (a failed
# or slow read leaves the environment's count in force), and re-read
# on an interval so a save in another worker reaches this one.
from grimoire.services import proxy_settings
await proxy_settings.load_at_startup()
proxy_settings.start_refresh_loop()
# Write down any shipped default that moved in this release.
#
# INLINE AND AWAITED, deliberately, against the instinct the block
+108 -7
View File
@@ -1,4 +1,13 @@
import os
import secrets
from pathlib import Path
# The file in the data directory that holds the instance's own SECRET_KEY.
SECRET_KEY_FILENAME = "secret_key"
# Placeholder keys that sample files and older compose files have shipped. A
# key anyone can read in this repository signs nothing safely.
PLACEHOLDER_SECRET_KEYS = frozenset({"dev-secret-change-me", "change-me-in-production"})
def _read_secret(env_var: str, secret_file_var: str, default: str) -> str:
@@ -16,14 +25,91 @@ def _read_secret(env_var: str, secret_file_var: str, default: str) -> str:
return default
def load_or_create_secret_key(data_dir: Path) -> str | None:
"""The instance's own SECRET_KEY, kept in `data_dir` and made on first use.
The first call writes a random key to `<data_dir>/secret_key`, readable by
its owner only; every later call, from this process or another, reads that
same key back. The file is published with a hard link, so two processes
starting together agree on one key and a crash never leaves an empty file.
Answers None when `data_dir` doesn't exist, which is a run outside the
container. An empty key file raises rather than being replaced, because a
new key would orphan everything sealed under the old one.
"""
if not data_dir.is_dir():
return None
path = data_dir / SECRET_KEY_FILENAME
if not path.exists():
tmp = data_dir / f".{SECRET_KEY_FILENAME}.{os.getpid()}"
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
try:
with os.fdopen(fd, "w") as f:
f.write(secrets.token_hex(32) + "\n")
f.flush()
os.fsync(f.fileno())
try:
os.link(tmp, path)
except FileExistsError:
pass
finally:
tmp.unlink(missing_ok=True)
key = path.read_text().strip()
if not key:
raise RuntimeError(
f"{path} is empty. Restore it from a copy, or delete it to start "
"with a new key (saved forge tokens and the backup secret are then "
"entered again)."
)
return key
def _secret_key() -> tuple[str, str]:
"""SECRET_KEY and where it came from: "env", "data" or "ephemeral".
An explicit key (SECRET_KEY or SECRET_KEY_FILE) wins and leaves the data
directory's key file alone, so unsetting it returns to that key. With no
explicit key the instance keeps its own in DATA_DIR. With neither, the
process gets a random key of its own that dies with it.
"""
explicit = _read_secret("SECRET_KEY", "SECRET_KEY_FILE", "")
if explicit:
return explicit, "env"
if os.environ.get("SECRET_KEY_FILE"):
# Falling through to another key would orphan what the named one sealed.
raise RuntimeError(
f"SECRET_KEY_FILE={os.environ['SECRET_KEY_FILE']} is missing or empty."
)
kept = load_or_create_secret_key(Path(os.environ.get("DATA_DIR", "/data")))
if kept:
return kept, "data"
return secrets.token_hex(32), "ephemeral"
def _trusted_proxy_hops() -> int:
"""TRUSTED_PROXY_HOPS, default 1. The older TRUST_PROXY_HEADERS switch is
still read when the hop count is unset: false means 0 hops. A value that
is not a whole number reads as -1, which validate() refuses."""
raw = os.environ.get("TRUSTED_PROXY_HOPS", "").strip()
if raw:
try:
return int(raw)
except ValueError:
return -1
if os.environ.get("TRUST_PROXY_HEADERS", "").strip().lower() in ("0", "false", "no"):
return 0
return 1
class Config:
DATABASE_URL: str = _read_secret(
"DATABASE_URL",
"DATABASE_URL_FILE",
"postgresql+asyncpg://grimoire:grimoire@localhost:5432/grimoire",
)
SECRET_KEY: str = _read_secret("SECRET_KEY", "SECRET_KEY_FILE", "dev-secret-change-me")
SECURE_COOKIES: bool = os.environ.get("SECURE_COOKIES", "").lower() in ("1", "true", "yes")
# Where the instance keeps its own SECRET_KEY. /data is a volume in every
# shipped compose file, beside the model cache and diagnostics.
DATA_DIR: str = os.environ.get("DATA_DIR", "/data")
SECRET_KEY, SECRET_KEY_SOURCE = _secret_key()
LOG_LEVEL: str = os.environ.get("LOG_LEVEL", "INFO")
LOG_RETENTION_DAYS: int = int(os.environ.get("LOG_RETENTION_DAYS", "90"))
@@ -36,7 +122,16 @@ class Config:
SMTP_FROM_NAME: str = os.environ.get("SMTP_FROM_NAME", "Fabled Grimoire")
SMTP_USE_TLS: bool = os.environ.get("SMTP_USE_TLS", "true").lower() in ("1", "true", "yes")
BASE_URL: str = os.environ.get("BASE_URL", "http://localhost:5000").rstrip("/")
TRUST_PROXY_HEADERS: bool = os.environ.get("TRUST_PROXY_HEADERS", "").lower() in ("1", "true", "yes")
# How many reverse proxies stand in front of the app; the client address
# and scheme are read that many entries in from the right of the
# forwarding headers (grimoire/proxy.py). 0 trusts no proxy. This is the
# install's starting value: a count saved in Settings → Config wins over
# it (services/proxy_settings.py).
TRUSTED_PROXY_HOPS: int = _trusted_proxy_hops()
TRUSTED_PROXY_HOPS_FROM_ENV: bool = bool(
os.environ.get("TRUSTED_PROXY_HOPS", "").strip()
or os.environ.get("TRUST_PROXY_HEADERS", "").strip()
)
# Git URL of the repo that ships this Grimoire plugin. There is one correct
# value per deployment (the repo serving the plugin), so it's a config
@@ -45,7 +140,7 @@ class Config:
# per-instance via the Plugin marketplace setting.
PLUGIN_MARKETPLACE_URL: str = os.environ.get(
"PLUGIN_MARKETPLACE_URL",
"https://git.fabledsword.com/bvandeusen/FabledScribe.git",
"https://git.fabledsword.com/bvandeusen/FabledGrimoire.git",
)
# OIDC / OAuth2 SSO (e.g. Authentik)
@@ -91,10 +186,16 @@ class Config:
errors.append(f"SMTP_PORT={cls.SMTP_PORT} must be between 1 and 65535")
if cls.oidc_enabled() and not cls.BASE_URL.startswith(("http://", "https://")):
errors.append(f"BASE_URL='{cls.BASE_URL}' must start with http:// or https:// when OIDC is enabled")
if cls.SECRET_KEY == "dev-secret-change-me" and cls.SECURE_COOKIES:
if cls.TRUSTED_PROXY_HOPS < 0:
errors.append(
"SECRET_KEY is set to the insecure default but SECURE_COOKIES=true indicates "
"a production deployment. Set SECRET_KEY or SECRET_KEY_FILE before starting."
f"TRUSTED_PROXY_HOPS={os.environ.get('TRUSTED_PROXY_HOPS')!r} must be a whole "
"number of proxies, 0 or more"
)
if cls.SECRET_KEY in PLACEHOLDER_SECRET_KEYS:
errors.append(
f"SECRET_KEY is set to the public placeholder {cls.SECRET_KEY!r}, so anyone "
"could forge a session. Remove SECRET_KEY to let the instance keep its own "
"key in DATA_DIR, or set a real one."
)
if errors:
raise ValueError("Configuration errors:\n" + "\n".join(f" - {e}" for e in errors))
+1 -1
View File
@@ -200,7 +200,7 @@ _WRITE_TOOLS = frozenset({
"merge_snippets", "unmerge_snippet",
"create_process", "update_process", "delete_process",
"classify_shapes", "classify_shapes_by_rule", "confirm_shape_proposals",
"refresh_pattern_coverage",
"refresh_pattern_coverage", "move_path",
# design systems
"create_design_system", "update_design_system", "delete_design_system",
"create_design_token", "update_design_token", "delete_design_token",
+43 -2
View File
@@ -11,9 +11,12 @@ from __future__ import annotations
from grimoire.mcp._context import current_user_id
from grimoire.services import projects as projects_svc
from grimoire.services import repo_bindings as repo_bindings_svc
from grimoire.services import snippets as snippets_svc
async def bind_repo(repo_url: str, project_id: int, ref: str = "") -> dict:
async def bind_repo(
repo_url: str, project_id: int, ref: str = "", moved_from: str = "",
) -> dict:
"""Bind a git repository to a Grimoire project for session-start context.
After this, any session started in that repo auto-loads the project's
@@ -27,21 +30,48 @@ async def bind_repo(repo_url: str, project_id: int, ref: str = "") -> dict:
the push, not the merge. Re-binding with ref="" keeps the standing ref;
pass ref="-" to clear it back to the default branch.
A repo renamed on its forge has a new remote, and its ledger is stored
under the old one. Pass the old remote as `moved_from` and the binding,
every shape judgment, and every snippet location recorded in the old repo
move to the new name together. Binding the new name without it starts an
empty ledger beside the old one. Running it again once the binding has
moved moves only what still names the old repo. A snippet shared with you
read-only keeps its location and is listed in `snippets_read_only`; its
owner moves it.
Args:
repo_url: the repo's git remote (e.g. the output of
`git remote get-url origin` — ssh or https form, both work).
project_id: the Grimoire project this repo represents.
ref: branch the ledger follows ("" = leave as is / default branch on
a new binding; "-" = clear to the default branch).
moved_from: the repo's remote before a rename on the forge. The
old name must be bound to this project, and the new one must be
neither bound nor holding a ledger of this project's.
"""
uid = current_user_id()
project = await projects_svc.get_project(uid, project_id)
if project is None:
raise ValueError(f"project {project_id} not found")
ref_arg = None if not ref else ("" if ref.strip() == "-" else ref)
moved = snippets = None
if moved_from:
already = (
await repo_bindings_svc.resolve_project(uid, moved_from) is None
and await repo_bindings_svc.resolve_project(uid, repo_url) == project_id
)
if already:
moved = 0
else:
_, moved = await repo_bindings_svc.move_binding(
uid, moved_from, repo_url, project_id,
)
snippets = await snippets_svc.move_location_repo(
uid, project_id, moved_from, repo_url,
)
binding = await repo_bindings_svc.set_binding(uid, repo_url, project_id, ref_arg)
follows = binding.ref or "the default branch"
return {
out = {
"repo_key": binding.repo_key,
"project_id": binding.project_id,
"project_title": project.title,
@@ -51,6 +81,17 @@ async def bind_repo(repo_url: str, project_id: int, ref: str = "") -> dict:
f"the ledger follows {follows}."
),
}
if moved is not None:
old_key = repo_bindings_svc.normalize_repo_key(moved_from)
out["moved_from"] = old_key
out["shapes_moved"] = moved
out["snippets_moved"] = snippets["moved"]
out["snippets_read_only"] = snippets["read_only"]
out["message"] += (
f" Moved from `{old_key}` with {moved} shape(s) and "
f"{len(snippets['moved'])} snippet(s)."
)
return out
async def list_repo_bindings() -> dict:
+5 -1
View File
@@ -42,7 +42,11 @@ async def menus_to_review(
Args:
source: the arm to review. Only arms whose search can be re-run
exactly are offered; today that is `auto_inject`.
exactly are offered: `auto_inject`, and the rule arms
`prompt_rule`, `pre_tool_rule` and `write_path_rule`. A rule
line's passage is the trigger the arm matched, and for the
command and write arms `within_budget` also needs the line to
sit inside the band of the top hit, as it did when shown.
n: how many calls to sample, 1–20. Random among the calls you have not
judged, so a review does not describe one afternoon.
days: how far back to sample from.
+30 -6
View File
@@ -10,13 +10,16 @@ preview-style warning.
"""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from grimoire.mcp._context import current_user_id
from grimoire.services import dedup as dedup_svc
from grimoire.services import moments as moments_svc
from grimoire.services import rulebooks as rulebooks_svc
from grimoire.services import trash as trash_svc
from grimoire.services.lesson_rules import confirmed_lesson_counts
from grimoire.services.rule_usage import (
record_rule_outcome, record_rule_pulled,
empty_rule_usage, record_rule_outcome, record_rule_pulled, usage_for_rules,
)
from grimoire.services import moment_delivery
@@ -204,6 +207,7 @@ def _rule_summary(r, moments: list[str] | None = None) -> dict:
async def list_rules(
rulebook_id: int = 0, topic_id: int = 0, project_id: int = 0,
with_usage: bool = False, days: int = 0,
) -> dict:
"""List rules — filter by rulebook, topic, and/or project.
@@ -213,9 +217,21 @@ async def list_rules(
project_id: 0 = no filter; positive = that project's OWN rules.
Global rules apply to every project, so they are listed by
rulebook or topic (or unfiltered), not under each project.
with_usage: add each rule's `usage` and `confirmed_lessons`. Off by
default: a listing is for applying rules, and this is for judging
them.
days: with `with_usage`, count only the last N days; 0 = everything
recorded.
rulebook_id and topic_id AND-combine; project_id lists a project's rules
on its own. Ownership-scoped.
`usage` is the rule's own telemetry: `surfaced_count` (a ranked surface
chose it), `ambient_count` (a bulk delivery carried it), `pull_count`,
`applied_count`, `departed_count` and the latest of each.
`confirmed_lessons` counts the lessons judged instances of the rule. Read
together they answer whether rules that situations keep proving are the
ones that surface and get opened.
"""
uid = current_user_id()
rows = await rulebooks_svc.list_rules(
@@ -224,12 +240,20 @@ async def list_rules(
topic_id=topic_id or None,
project_id=project_id or None,
)
ids = [r.id for r in rows]
# One batched read for the page, not one per row.
mounted = await rulebooks_svc.list_rule_moments([r.id for r in rows])
return {
"rules": [_rule_summary(r, mounted.get(r.id)) for r in rows],
"total": len(rows),
}
mounted = await rulebooks_svc.list_rule_moments(ids)
rules = [_rule_summary(r, mounted.get(r.id)) for r in rows]
if with_usage:
since = (
datetime.now(timezone.utc) - timedelta(days=days) if days > 0 else None
)
usage = await usage_for_rules(ids, since=since)
lessons = await confirmed_lesson_counts(uid, ids)
for row in rules:
row["usage"] = usage.get(row["id"], empty_rule_usage())
row["confirmed_lessons"] = lessons.get(row["id"], 0)
return {"rules": rules, "total": len(rows)}
async def get_rule(rule_id: int) -> dict:
+14 -2
View File
@@ -509,6 +509,15 @@ async def retrieval_telemetry(
Read it against `sources["write_path_rule"]`, which declines the large
majority of its calls like any other surface.
`rule_usage.by_source` is the per-surface split, the twin of
`usage.by_source`: for each surface that showed rules, `rules_surfaced`
(distinct rules), `rules_pulled` (how many of those an agent opened in the
window) and `pull_through`, null on an ambient surface. It is the readout
that says which arm — the prompt arm, the tool arm, a lesson pointing at
its rule — earns its lines. The same upper bound as the note split: a rule
two surfaces showed and an agent opened once counts for both.
`by_source_failed: true` means that one query failed.
`system_usage` — the RULINGS arm, from `system_usage_events`: how
often an area's rulings were shown because a command (`rulings_pre_tool`)
or an edit (`rulings_write_path`) touched its files, how often a System
@@ -590,8 +599,11 @@ async def retrieval_telemetry(
corpus. Not a verdict on them: a line carries its matched passage, so
an unopened record may have been unrelated, enough as shown, or already
in context. Judge a sample (`menus_to_review`) before acting on it.
For RULES there is no judged sample: a rule set aside again and again
is a trigger to fix (`update_rule(when_to_apply=...)`), not a floor.
For RULES, judge a sample of the rule arm that showed them
(`menus_to_review(source="prompt_rule")`, or `pre_tool_rule`,
`write_path_rule`); a rule set aside again and again where it does
not apply is a trigger to fix (`update_rule(when_to_apply=...)`), not
a floor.
- `read_and_unacted` — distinct rules OPENED in the window that recorded
no outcome, against the ones that did. The worse sibling of `surfaced_never_pulled` above: a
rule nobody opens is cheap, while a rule read and silently unchanged is
+44 -2
View File
@@ -12,6 +12,7 @@ from __future__ import annotations
from grimoire.mcp._context import current_user_id
from grimoire.services import coverage as coverage_svc
from grimoire.services import shape_ledger as shape_ledger_svc
from grimoire.services import snippets as snippets_svc
async def classify_shapes(
@@ -263,7 +264,8 @@ async def shape_history(
snippet_id, who, why — one per judgment, so a shape that was an instance
of #N and later a variant of #M shows both), `vanished`, `reappeared`,
`drifted` (the body moved under a judgment; see list_shapes flag=
"recheck"). Each event carries the commit the tree was read at.
"recheck"), `moved` (move_path carried the row; the event's `path`
is the one it left). Each event carries the commit the tree was read at.
Read it as a timeline: "instance of #N from <first classified at>,
re-judged variant of #M at <at> because <reason>, vanished at <commit>".
@@ -398,10 +400,50 @@ async def refresh_pattern_coverage(project_id: int) -> dict:
}
async def move_path(project_id: int, from_path: str, to_path: str) -> dict:
"""Carry what the record knows about a file or directory that moved in
the tree: its shape judgments and its snippets' recorded locations.
Both are keyed by path. After `git mv src/old src/new`, the next coverage
refresh would mark every row under src/old vanished and list the same
shapes again under src/new, unclassified, with every judgment left
behind. Each snippet recorded under src/old would also read `missing`.
Run this once the move is committed, before that refresh.
- Every ledger row under `from_path` is re-keyed to the same place under
`to_path`, in every repo bound to the project. It keeps its judgment,
uses edges and history, and gains a `moved` event (shape_history).
- Every snippet of the project recorded under `from_path` has its
location re-pathed.
A ledger row already at the new path is replaced when nobody judged it
(the refresh ran first, or the sync stamped it `scoped`). A judged one
stands; the old row stays where it is and is listed in `kept`. The
refresh that follows compares bodies as always, so a judged shape whose
body changed in the move comes back flagged `recheck`.
Args:
project_id: The project whose record moves. Requires write access.
from_path: The file or directory as it was recorded.
to_path: Where it lives now. Neither path may contain the other.
Returns {"moved": N, "replaced": N, "kept": [{path, symbol, kind}],
"snippets_moved": [ids], "snippets_read_only": [ids]}. A snippet shared
with you read-only stays where it is, and its owner moves it.
"""
uid = current_user_id()
out = await shape_ledger_svc.move_shapes(uid, project_id, from_path, to_path)
snippets = await snippets_svc.move_locations(uid, project_id, from_path, to_path)
out["snippets_moved"] = snippets["moved"]
out["snippets_read_only"] = snippets["read_only"]
if not (out["moved"] or out["kept"] or snippets["moved"] or snippets["read_only"]):
raise ValueError(f"nothing in project {project_id} is recorded under `{from_path}`")
return out
def register(mcp) -> None:
for fn in (
classify_shapes, classify_shapes_by_rule, list_shapes,
refresh_pattern_coverage, confirm_shape_proposals, shape_history,
stamps_to_review,
stamps_to_review, move_path,
):
mcp.tool(name=fn.__name__)(fn)
+1
View File
@@ -94,3 +94,4 @@ from grimoire.models.family import ( # noqa: E402, F401
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform,
FamilyIdeaReference, Platform, ProjectPlatform,
)
from grimoire.models.instance import InstanceIdentity # noqa: E402, F401
+4 -3
View File
@@ -311,8 +311,9 @@ class CodeShapeConsumer(Base):
# What a shape's history records. Not "appeared" — first_seen and
# created_at already say that on the row; history is for what CHANGED:
SHAPE_EVENTS = ("classified", "vanished", "reappeared", "drifted")
# created_at already say that on the row; history is for what CHANGED.
# A `moved` event carries the path the shape left; the row says where it went.
SHAPE_EVENTS = ("classified", "vanished", "reappeared", "drifted", "moved")
class CodeShapeEvent(Base):
@@ -321,7 +322,7 @@ class CodeShapeEvent(Base):
"We used #N here from <date>, #M replaced it at commit C, reason R" is a
question the ledger row alone cannot answer once it has moved on; this
table keeps each judgment (status, snippet, who, why, at which commit)
and each presence change (vanished / reappeared / drifted) as it
and each presence change (vanished / reappeared / drifted / moved) as it
happened. Denormalised path/symbol/kind so a directory's history reads
without joining; `snippet_id` is deliberately FK-free — history outlives
the snippet it names, which is the point.
+26
View File
@@ -0,0 +1,26 @@
from datetime import datetime
from sqlalchemy import CheckConstraint, DateTime, Integer, Text, func
from sqlalchemy.orm import Mapped, mapped_column
from grimoire.models import Base
class InstanceIdentity(Base):
"""The one row naming this database's id lineage.
Two stores with the same instance id number their rows the same way: one
is the other, or a restore of it made with the original ids kept. That is
what lets a backup roll a live instance back by id, and what refuses a
backup whose ids would land on unrelated rows. Made by the migration, so
an instance has it before anyone registers.
"""
__tablename__ = "instance_identity"
__table_args__ = (CheckConstraint("id = 1", name="ck_instance_identity_single_row"),)
id: Mapped[int] = mapped_column(Integer, primary_key=True, default=1)
instance_id: Mapped[str] = mapped_column(Text, nullable=False)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now()
)
+1 -1
View File
@@ -9,7 +9,7 @@ class RepoBinding(Base, TimestampMixin):
"""Maps a git repository to the Grimoire project it represents.
The SessionStart hook sends the working repo's normalized remote
(`repo_key`, e.g. ``git.fabledsword.com/bvandeusen/fabledscribe``) and the
(`repo_key`, e.g. ``git.fabledsword.com/bvandeusen/fabledgrimoire``) and the
server resolves it to a project — so the operator can work across many
repos/projects without pinning a project id in plugin config. The key is a
*stable identifier* (the git remote), not a dir-name heuristic.
+164
View File
@@ -0,0 +1,164 @@
"""What the trusted reverse proxy says about a request: who sent it, and
whether it arrived over HTTPS.
Grimoire serves plain HTTP; TLS ends at the operator's proxy. The proxy
reports the client's address in X-Forwarded-For and the scheme in
X-Forwarded-Proto, and each proxy on the road APPENDS its entry, so only the
right-hand end of either header was written by a proxy we trust. Anything to
the left of that is whatever the client chose to send.
The hop count says how many proxies stand in front of the app. An admin sets
it in Settings → Config; until they do, it is `TRUSTED_PROXY_HOPS` from the
environment (default 1). The entry read is that many in from the right. Zero
trusts no proxy: the headers are ignored and the socket peer is the client.
Never read X-Forwarded-For or X-Forwarded-Proto anywhere else. The client
address keys the rate limits and the audit log, and the scheme decides the
session cookie's Secure flag and HSTS. A second reader is a second trust rule.
"""
from __future__ import annotations
from quart import has_request_context, request
from quart.sessions import SecureCookieSessionInterface
from grimoire.config import Config
# HSTS for a year, sent only on a request the trusted proxy says is HTTPS.
# No includeSubDomains or preload: the app does not own the operator's domain.
HSTS_VALUE = "max-age=31536000"
def trusted_entry(header: str, hops: int) -> str | None:
"""The entry `hops` in from the right of a forwarding header.
A header shorter than that is read at its leftmost entry, the one the
first proxy wrote. A road with fewer proxies than the count (a LAN road
beside a Cloudflare one) still reads right that way. None when no proxy
is trusted or there is no header. Pure, so the trust boundary is
testable without a request."""
if hops <= 0:
return None
entries = [part.strip() for part in header.split(",") if part.strip()]
if not entries:
return None
return entries[-min(hops, len(entries))]
# The count saved in Settings, or None when none is saved. Held here because
# every request reads it and the read is synchronous; services/proxy_settings
# loads it at startup, refreshes it on an interval in every worker, and sets it
# at once in the worker that saved a new one.
_saved_hops: int | None = None
def set_saved_hops(hops: int | None) -> None:
global _saved_hops
_saved_hops = hops
def saved_hops_in_force() -> int | None:
"""The count saved in Settings that this worker holds, or None."""
return _saved_hops
def trusted_hops() -> int:
"""The hop count in force: the one saved in Settings, else the env's."""
return Config.TRUSTED_PROXY_HOPS if _saved_hops is None else _saved_hops
def _chain(name: str) -> str:
# Joined, because a header sent twice arrives as two values, and each
# proxy's append may land on either one.
return ",".join(request.headers.getlist(name))
def _forwarded(name: str) -> str | None:
return trusted_entry(_chain(name), trusted_hops())
def client_ip() -> str:
"""The caller's address, as the trusted proxy saw it."""
if not has_request_context():
return ""
return (_forwarded("X-Forwarded-For") or request.remote_addr or "")[:64]
def is_https() -> bool:
"""Whether this request arrived over HTTPS, by the trusted proxy's word."""
if not has_request_context():
return False
if request.scheme == "https":
return True
return (_forwarded("X-Forwarded-Proto") or "").lower() == "https"
class PerRequestSecureSession(SecureCookieSessionInterface):
"""The session cookie is Secure exactly when the request is HTTPS, so a
public deployment gets it and a plain-HTTP one keeps working."""
def get_cookie_secure(self, app) -> bool:
return is_https()
def apply_https_headers(response) -> None:
"""HSTS on an HTTPS request, and only then. The app never redirects to
HTTPS; plain HTTP is a supported way to run it."""
if is_https():
response.headers.setdefault("Strict-Transport-Security", HSTS_VALUE)
# Cloudflare writes the visitor's address here on every request it proxies.
# Read only to SUGGEST a hop count in the readout, never to decide one: a
# client that reaches the app without Cloudflare can send it too.
CLOUDFLARE_CLIENT_HEADER = "CF-Connecting-IP"
CLOUDFLARE_MARKERS = (CLOUDFLARE_CLIENT_HEADER, "CF-Ray")
READOUT_HOPS = range(0, 4)
MAX_HOPS = 5
def readout(peer: str, forwarded_for: str, forwarded_proto: str,
cloudflare_client: str | None, scheme: str = "http") -> dict:
"""What each hop count would make of one request, so an admin can pick
the count that shows their own address. Pure: the request's parts come in
as arguments.
`suggested` is the smallest count that reads the address Cloudflare
reports, when the request came through Cloudflare; otherwise None, since
only the admin knows which address is theirs.
"""
def read(hops: int) -> dict:
address = trusted_entry(forwarded_for, hops) or peer
proto = trusted_entry(forwarded_proto, hops)
https = scheme == "https" or (proto or "").lower() == "https"
return {"hops": hops, "client": address, "https": https}
suggested = None
if cloudflare_client:
suggested = next(
(h for h in range(1, MAX_HOPS + 1)
if trusted_entry(forwarded_for, h) == cloudflare_client),
None,
)
return {
"peer": peer,
"forwarded_for": [p.strip() for p in forwarded_for.split(",") if p.strip()],
"forwarded_proto": [p.strip() for p in forwarded_proto.split(",") if p.strip()],
"cloudflare": cloudflare_client is not None,
"cloudflare_client": cloudflare_client,
"rows": [read(h) for h in READOUT_HOPS],
"suggested": suggested,
}
def request_readout() -> dict:
"""`readout` for the current request."""
cloudflare = request.headers.get(CLOUDFLARE_CLIENT_HEADER)
if cloudflare is None and request.headers.get("CF-Ray"):
cloudflare = ""
return readout(
request.remote_addr or "",
_chain("X-Forwarded-For"),
_chain("X-Forwarded-Proto"),
cloudflare.strip() if cloudflare is not None else None,
request.scheme,
)
+11 -3
View File
@@ -2,15 +2,23 @@
IMPORTANT — deployment note:
Rate limit counters are stored in memory and are lost on process restart.
When deployed behind a reverse proxy (nginx, Caddy, Traefik) you MUST set
TRUST_PROXY_HEADERS=true so that the real client IP is used as the bucket key
rather than the proxy's IP (which would cause all users to share one bucket).
The bucket key is the client address from grimoire/proxy.py, read by the
proxy hop count (Settings → Config). A count that is too low keys every
caller on the proxy's address, so they all share one bucket.
"""
import asyncio
import time
from collections import defaultdict
# Said beside every "too many attempts" reply, because the one case a person
# can't diagnose on their own is everybody being blocked at once: the proxy
# count is too low, so every visitor reads as the proxy and shares one bucket.
SHARED_ADDRESS_HINT = (
"If everyone is being blocked, an admin should check the reverse proxy count "
"in Settings → Config: every visitor may be reading as the proxy's address."
)
_buckets: dict[str, list[float]] = defaultdict(list)
_lock = asyncio.Lock()
+80 -30
View File
@@ -1,20 +1,26 @@
"""Restore a Grimoire backup into an empty instance, from the command line.
"""Restore a Grimoire backup from the command line.
python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar
python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar --yes
This is how a fresh instance is brought back after losing the old one. Run it
inside the Grimoire container, which already has the database settings and has
already migrated the schema on start. It reads either the scheduled backup's
tar (services/backup_file.py) or the JSON the admin download produces.
Run it inside the Grimoire container, which already has the database settings
and has migrated the schema on start. It reads the scheduled backup's tar
(services/backup_file.py) or the JSON the admin download produces, and nothing
limits the file's size.
It refuses a database that already holds an account. A restore re-creates
every user, project and record from the file with new ids; into a live
instance it would add a second copy of everything rather than replace it.
(Settings → Backup & Restore still offers the upload for that case, on
purpose and with its own warning.)
It does what Settings → Restore does (services/backup.py replace_from_backup):
- **A new, empty instance** takes the backup as it is.
- **A live instance** rolls back to a backup of its own: everything goes back
to the moment the backup was taken, for every user.
- A backup from another instance, or one made before backups recorded their
instance, only goes into an empty instance.
Unlike the upload, nothing limits the file's size here, so this is also the
way in for a backup too big for the web upload.
Without --yes it only says what would change. With --yes it first saves the
current state as a backup (to the backup destination, or DATA_DIR/pre-restore),
then restores. Everyone is signed out afterwards.
A backup in the oldest (v1) format is copied into an empty instance with new
ids instead, as before.
Afterwards, restart Grimoire: search indexes are not part of a backup and are
rebuilt from the restored records at startup.
@@ -48,9 +54,39 @@ async def _account_count() -> int:
return (await session.execute(select(func.count()).select_from(User))).scalar_one()
async def run(path: Path) -> int:
from grimoire.models import engine
async def _restore_v1(payload: dict) -> int:
from grimoire.services.backup import restore_full_backup
accounts = await _account_count()
if accounts:
print(
f"This database already holds {accounts} account(s). A backup in the oldest "
"format is copied in with new ids rather than replacing anything, so it only "
"goes into an empty instance. Start Grimoire on an empty database and run "
"this again.",
file=sys.stderr,
)
return 3
stats = await restore_full_backup(payload)
print(json.dumps({"restored": stats}, indent=2, default=str))
return 0
def _print_preview(preview: dict) -> None:
taken = preview.get("exported_at") or "an unknown date"
if preview["mode"] == "rollback":
print(f"This rolls the whole instance back to {taken}, for every user. Anything "
"made since is removed, edits since are undone, records deleted since come back.")
else:
print(f"This instance has no records yet; it takes the backup from {taken} as it is.")
for count in preview["counts"]:
print(f" {count['kind']:<24} now {count['now']:>7} after {count['after']:>7}")
async def run(path: Path, *, yes: bool = False) -> int:
from grimoire.models import engine
from grimoire.services import backup_job
from grimoire.services.backup import RestoreRefused, replace_preview
from grimoire.services.backup_file import ArchiveError
try:
@@ -63,20 +99,29 @@ async def run(path: Path) -> int:
print(f"{path} isn't a backup this Grimoire can read: {exc}", file=sys.stderr)
return 2
accounts = await _account_count()
if accounts:
print(
f"This database already holds {accounts} account(s). A restore adds a second "
"copy of everything rather than replacing it, so it only runs into an empty "
"instance. Start Grimoire on an empty database and run this again.",
file=sys.stderr,
)
return 3
stats = await restore_full_backup(payload)
print(json.dumps({"restored": stats}, indent=2, default=str))
print("Restored. Restart Grimoire so the search indexes are rebuilt.", file=sys.stderr)
return 0
if (payload.get("version") or 1) < 2:
code = await _restore_v1(payload)
else:
try:
_print_preview(await replace_preview(payload))
if not yes:
print("Nothing was changed. Run it again with --yes to restore.",
file=sys.stderr)
return 4
result = await backup_job.rollback_to_backup(payload)
except RestoreRefused as exc:
print(str(exc), file=sys.stderr)
return 3
safety = result["safety_backup"]
print(json.dumps({"restored": result["restored"], "safety_backup": safety},
indent=2, default=str))
print(f"The state it replaced is saved as {safety['file']} in {safety['where']}.",
file=sys.stderr)
code = 0
if code == 0:
print("Restored. Restart Grimoire so the search indexes are rebuilt; "
"everyone signs in again.", file=sys.stderr)
return code
finally:
await engine.dispose()
@@ -84,11 +129,16 @@ async def run(path: Path) -> int:
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
prog="python -m grimoire.restore",
description="Restore a Grimoire backup (.tar or .json) into an empty instance.",
description=(
"Restore a Grimoire backup (.tar or .json): into an empty instance, or "
"roll this instance back to a backup of its own."
),
)
parser.add_argument("file", type=Path, help="the backup file")
parser.add_argument("--yes", action="store_true",
help="restore; without it, only say what would change")
args = parser.parse_args(argv)
return asyncio.run(run(args.file))
return asyncio.run(run(args.file, yes=args.yes))
if __name__ == "__main__":
+155 -35
View File
@@ -3,7 +3,11 @@ import json
from quart import Blueprint, Response, g, jsonify, request
from grimoire import proxy
from grimoire.auth import admin_required, login_required, get_current_user_id
from grimoire.config import Config
from grimoire.proxy import client_ip
from grimoire.services import proxy_settings
from grimoire.services.auth import (
create_invitation,
delete_user,
@@ -16,7 +20,6 @@ from grimoire.services.auth import (
from grimoire.services.backup import (
export_full_backup,
export_user_backup,
restore_full_backup,
)
from grimoire.services.email import SMTP_SETTING_KEYS, get_base_url, get_smtp_config, is_smtp_configured, send_test_email
from grimoire.services.logging import get_logs, get_log_stats, log_audit
@@ -46,7 +49,7 @@ async def backup():
else:
data = await export_user_backup(uid)
await log_audit("backup", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"scope": scope})
await log_audit("backup", user_id=uid, username=g.user.username, ip_address=client_ip(), details={"scope": scope})
return Response(
json.dumps(data, indent=2, default=str),
content_type="application/json",
@@ -56,31 +59,111 @@ async def backup():
)
async def _restore_payload() -> tuple[dict | None, tuple | None]:
"""The backup a restore request names: `?source=<file>` at the backup
destination, or the uploaded file (the JSON download as
application/json, or the tar as the raw body; held to the request-size
limit, which is why the destination is the way in for a large one).
Returns (payload, None) or (None, an error response)."""
import io
import tempfile
from pathlib import Path
from grimoire.services import backup_destination as dest
from grimoire.services.backup_file import ArchiveError, read_archive
source = request.args.get("source")
try:
if source:
config = await dest.load_config()
secret, _ = await dest.load_secret()
with tempfile.TemporaryDirectory(prefix="grimoire-restore-") as staging:
path = await dest.open_backup(config, secret, source, Path(staging))
def load() -> dict:
with open(path, "rb") as f:
return read_archive(f)
data = await asyncio.to_thread(load)
elif request.mimetype == "application/json":
data = await request.get_json()
else:
data = read_archive(io.BytesIO(await request.get_data()))
except dest.DestinationError as exc:
return None, (jsonify({"error": str(exc)}), 400)
except ArchiveError as exc:
return None, (jsonify({"error": f"That file isn't a Grimoire backup: {exc}"}), 400)
if not isinstance(data, dict) or not data:
return None, (jsonify({"error": "No backup data provided"}), 400)
return data, None
@admin_bp.route("/restore/sources", methods=["GET"])
@admin_required
async def restore_sources():
"""The backups at the backup destination, newest first, to restore from
without uploading one."""
from grimoire.services import backup_destination as dest
config = await dest.load_config()
secret, _ = await dest.load_secret()
try:
backups = await asyncio.wait_for(dest.list_backups(config, secret), _BACKUP_CHECK_TIMEOUT)
except dest.DestinationError as exc:
return jsonify({"backups": [], "error": str(exc)})
except (TimeoutError, asyncio.TimeoutError):
return jsonify({"backups": [], "error": (
f"The destination didn't answer within {int(_BACKUP_CHECK_TIMEOUT)} seconds."
)})
return jsonify({"backups": [{"name": b.name, "size_bytes": b.size_bytes} for b in backups]})
@admin_bp.route("/restore/preview", methods=["POST"])
@admin_required
async def restore_preview():
"""What restoring this backup would do, changing nothing: fresh or
rollback, when it was taken, and the counts now and after. 409 with the
reason when it can't be restored here."""
from grimoire.services.backup import RestoreRefused, replace_preview
data, error = await _restore_payload()
if error:
return error
try:
return jsonify(await replace_preview(data))
except RestoreRefused as exc:
return jsonify({"error": str(exc)}), 409
@admin_bp.route("/restore", methods=["POST"])
@admin_required
async def restore():
"""Restore from an uploaded backup: the JSON download (application/json)
or the scheduled backup's tar (sent as the raw body). The upload is held
to the app's request-size limit; `python -m grimoire.restore` has none, and
is the way in for a bigger file or a fresh instance."""
if request.mimetype == "application/json":
data = await request.get_json()
else:
import io
"""Put the whole instance back to a full backup (services/backup.py
replace_from_backup), after backing up what it replaces. `?confirm=rollback`
is required: this removes everything made since the backup, for every
user. Every session ends, the caller's included; search is rebuilt in the
background."""
from grimoire.services import backup_job as job
from grimoire.services.backup import RestoreRefused
from grimoire.services.backup_file import ArchiveError, read_archive
try:
data = read_archive(io.BytesIO(await request.get_data()))
except ArchiveError as exc:
return jsonify({"error": f"That file isn't a Grimoire backup: {exc}"}), 400
if not data:
return jsonify({"error": "No backup data provided"}), 400
stats = await restore_full_backup(data)
if request.args.get("confirm") != "rollback":
return jsonify({"error": "Restoring replaces the whole instance; confirm it first."}), 400
data, error = await _restore_payload()
if error:
return error
try:
result = await job.rollback_to_backup(data)
except RestoreRefused as exc:
return jsonify({"error": str(exc)}), 409
except job.BackupBusy:
return jsonify({"error": "A backup is running. Restore once it has finished."}), 409
job.start_reindex_after_restore()
uid = get_current_user_id()
await log_audit("restore", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"stats": stats})
return jsonify({"status": "ok", "stats": stats})
await log_audit(
"restore", user_id=uid, username=g.user.username, ip_address=client_ip(),
details={"mode": result["mode"], "exported_at": result.get("exported_at"),
"safety_backup": result["safety_backup"], "stats": result["restored"]},
)
return jsonify({"status": "ok", **result})
@admin_bp.route("/users", methods=["GET"])
@@ -100,7 +183,7 @@ async def remove_user(user_id: int):
deleted = await delete_user(user_id)
if not deleted:
return jsonify({"error": "User not found"}), 404
await log_audit("user_delete", user_id=current_uid, username=g.user.username, ip_address=request.remote_addr, details={"deleted_user_id": user_id})
await log_audit("user_delete", user_id=current_uid, username=g.user.username, ip_address=client_ip(), details={"deleted_user_id": user_id})
return jsonify({"status": "ok"})
@@ -121,7 +204,7 @@ async def toggle_registration():
uid = get_current_user_id()
await set_registration_open(uid, bool(open_val))
await log_audit("registration_toggle", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"open": bool(open_val)})
await log_audit("registration_toggle", user_id=uid, username=g.user.username, ip_address=client_ip(), details={"open": bool(open_val)})
return jsonify({"status": "ok", "open": bool(open_val)})
@@ -151,7 +234,7 @@ async def update_smtp():
if settings_to_save:
await set_settings_batch(uid, settings_to_save)
await log_audit("smtp_config", user_id=uid, username=g.user.username, ip_address=request.remote_addr)
await log_audit("smtp_config", user_id=uid, username=g.user.username, ip_address=client_ip())
return jsonify({"status": "ok"})
@@ -166,7 +249,7 @@ async def test_smtp():
uid = get_current_user_id()
try:
await send_test_email(recipient)
await log_audit("smtp_test", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"recipient": recipient})
await log_audit("smtp_test", user_id=uid, username=g.user.username, ip_address=client_ip(), details={"recipient": recipient})
return jsonify({"status": "ok"})
except Exception as e:
return jsonify({"error": str(e)}), 500
@@ -180,7 +263,6 @@ async def test_smtp():
@admin_bp.route("/forge-webhook", methods=["GET"])
@admin_required
async def get_forge_webhook_settings():
from grimoire.config import Config
from grimoire.routes.webhooks import FORGE_WEBHOOK_SECRET_KEY
webhook_secret = (
@@ -209,7 +291,7 @@ async def update_forge_webhook_settings():
# The secret is deliberately absent from the audit detail.
await log_audit(
"forge_webhook_config", user_id=uid, username=g.user.username,
ip_address=request.remote_addr, details={},
ip_address=client_ip(), details={},
)
return jsonify({"status": "ok"})
@@ -262,10 +344,48 @@ async def update_base_url():
return jsonify({"error": "Base URL must use http or https"}), 400
uid = get_current_user_id()
await set_setting(uid, "base_url", url)
await log_audit("base_url_config", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"base_url": url})
await log_audit("base_url_config", user_id=uid, username=g.user.username, ip_address=client_ip(), details={"base_url": url})
return jsonify({"status": "ok"})
def _proxy_payload() -> dict:
return {
"hops": proxy.trusted_hops(),
"source": proxy_settings.source(),
"env_hops": Config.TRUSTED_PROXY_HOPS,
"max_hops": proxy.MAX_HOPS,
"readout": proxy.request_readout(),
}
@admin_bp.route("/proxy", methods=["GET"])
@admin_required
async def get_proxy_setting():
"""The hop count in force, where it came from, and what each count would
make of the admin's own request."""
return jsonify(_proxy_payload())
@admin_bp.route("/proxy", methods=["PUT"])
@admin_required
async def update_proxy_setting():
"""Save the hop count, or `{"hops": null}` to go back to the environment's.
It applies to the next request."""
data = await request.get_json() or {}
raw = data.get("hops")
hops = None if raw is None else proxy_settings.parse_hops(raw)
if raw is not None and hops is None:
return jsonify({"error": f"The proxy count must be a whole number from 0 to {proxy.MAX_HOPS}."}), 400
before = proxy.trusted_hops()
await proxy_settings.save_hops(hops)
uid = get_current_user_id()
await log_audit(
"proxy_hops_config", user_id=uid, username=g.user.username, ip_address=client_ip(),
details={"before": before, "after": proxy.trusted_hops(), "saved": hops},
)
return jsonify(_proxy_payload())
@admin_bp.route("/db-maintenance", methods=["GET"])
@admin_required
async def get_db_maintenance():
@@ -310,7 +430,7 @@ async def update_db_maintenance():
uid = get_current_user_id()
await log_audit(
"db_maintenance_config", user_id=uid, username=g.user.username,
ip_address=request.remote_addr, details={"enabled": enabled, "hour": hour},
ip_address=client_ip(), details={"enabled": enabled, "hour": hour},
)
return jsonify({"status": "ok", "enabled": enabled, "hour": hour})
@@ -323,7 +443,7 @@ async def run_db_maintenance_now():
uid = get_current_user_id()
await log_audit(
"db_maintenance_run", user_id=uid, username=g.user.username,
ip_address=request.remote_addr,
ip_address=client_ip(),
)
summary = await run_maintenance()
return jsonify(summary)
@@ -389,7 +509,7 @@ async def update_backups():
uid = get_current_user_id()
await log_audit(
"backup_config", user_id=uid, username=g.user.username,
ip_address=request.remote_addr,
ip_address=client_ip(),
details={**config.to_dict(), "secret_changed": bool(
data.get("clear_s3_secret") or (isinstance(secret, str) and secret and secret != SECRET_MASK)
)},
@@ -439,7 +559,7 @@ async def run_backup_now():
return jsonify({"error": "A backup is already running."}), 409
uid = get_current_user_id()
await log_audit("backup_run", user_id=uid, username=g.user.username,
ip_address=request.remote_addr, details={})
ip_address=client_ip(), details={})
return jsonify({"status": "started"}), 202
@@ -463,7 +583,7 @@ async def create_invite():
"invitation_created",
user_id=uid,
username=g.user.username,
ip_address=request.remote_addr,
ip_address=client_ip(),
details={"invited_email": email},
)
return jsonify({"status": "ok"}), 201
@@ -497,7 +617,7 @@ async def delete_invitation(invitation_id: int):
"invitation_revoked",
user_id=uid,
username=g.user.username,
ip_address=request.remote_addr,
ip_address=client_ip(),
details={"invitation_id": invitation_id},
)
return jsonify({"status": "ok"})
+24 -32
View File
@@ -8,7 +8,8 @@ from quart import Blueprint, g, jsonify, redirect, request, session
from grimoire.auth import login_required, get_current_user_id
from grimoire.config import Config
from grimoire.rate_limit import is_rate_limited
from grimoire.proxy import client_ip
from grimoire.rate_limit import SHARED_ADDRESS_HINT, is_rate_limited
from grimoire.services.auth import (
authenticate,
change_password,
@@ -43,21 +44,12 @@ from grimoire.services.oauth import (
auth_bp = Blueprint("auth", __name__, url_prefix="/api/auth")
def _client_ip() -> str:
if Config.TRUST_PROXY_HEADERS:
for header in ("X-Forwarded-For", "X-Real-IP"):
val = request.headers.get(header, "").split(",")[0].strip()
if val:
return val
return request.remote_addr or ""
@auth_bp.route("/register", methods=["POST"])
async def register():
if not Config.LOCAL_AUTH_ENABLED:
return jsonify({"error": "Local authentication is disabled. Please use SSO."}), 403
if await is_rate_limited(f"register:{_client_ip()}", max_requests=5, window_seconds=300):
return jsonify({"error": "Too many registration attempts. Try again later."}), 429
if await is_rate_limited(f"register:{client_ip()}", max_requests=5, window_seconds=300):
return jsonify({"error": f"Too many registration attempts. Try again later. {SHARED_ADDRESS_HINT}"}), 429
if not await is_registration_open():
return jsonify({"error": "Registration is closed"}), 403
@@ -78,7 +70,7 @@ async def register():
session["user_id"] = user.id
session["session_version"] = user.session_version
await log_audit("register", user_id=user.id, username=user.username, ip_address=_client_ip())
await log_audit("register", user_id=user.id, username=user.username, ip_address=client_ip())
return jsonify(user.to_dict()), 201
@@ -86,8 +78,8 @@ async def register():
async def login():
if not Config.LOCAL_AUTH_ENABLED:
return jsonify({"error": "Local authentication is disabled. Please use SSO."}), 403
if await is_rate_limited(f"login:{_client_ip()}", max_requests=10, window_seconds=60):
return jsonify({"error": "Too many login attempts. Try again later."}), 429
if await is_rate_limited(f"login:{client_ip()}", max_requests=10, window_seconds=60):
return jsonify({"error": f"Too many login attempts. Try again later. {SHARED_ADDRESS_HINT}"}), 429
data = await request.get_json()
username = (data.get("username") or "").strip()
password = data.get("password") or ""
@@ -97,22 +89,22 @@ async def login():
user = await authenticate(username, password)
if not user:
await log_audit("login_failed", username=username, ip_address=_client_ip())
await log_audit("login_failed", username=username, ip_address=client_ip())
# Try to notify the actual user about the failed attempt
target_user = await get_user_by_username(username)
if target_user:
asyncio.create_task(notify_security_event(
target_user.id, "login_failed",
{"ip_address": _client_ip(), "username": username},
{"ip_address": client_ip(), "username": username},
))
return jsonify({"error": "Invalid username or password"}), 401
session["user_id"] = user.id
session["session_version"] = user.session_version
await log_audit("login", user_id=user.id, username=user.username, ip_address=_client_ip())
await log_audit("login", user_id=user.id, username=user.username, ip_address=client_ip())
asyncio.create_task(notify_security_event(
user.id, "login",
{"ip_address": _client_ip()},
{"ip_address": client_ip()},
))
return jsonify(user.to_dict())
@@ -122,10 +114,10 @@ async def logout():
uid = session.get("user_id")
if uid:
user = await get_user_by_id(uid)
await log_audit("logout", user_id=uid, username=user.username if user else None, ip_address=_client_ip())
await log_audit("logout", user_id=uid, username=user.username if user else None, ip_address=client_ip())
asyncio.create_task(notify_security_event(
uid, "logout",
{"ip_address": _client_ip()},
{"ip_address": client_ip()},
))
session.clear()
return jsonify({"status": "ok"})
@@ -159,10 +151,10 @@ async def update_password():
# Keep the current session alive with the new version (other sessions are now invalidated)
session["session_version"] = new_version
await log_audit("password_change", user_id=uid, username=g.user.username, ip_address=_client_ip())
await log_audit("password_change", user_id=uid, username=g.user.username, ip_address=client_ip())
asyncio.create_task(notify_security_event(
uid, "password_change",
{"ip_address": _client_ip()},
{"ip_address": client_ip()},
))
return jsonify({"status": "ok"})
@@ -178,7 +170,7 @@ async def invalidate_sessions_route():
uid = get_current_user_id()
new_version = await invalidate_other_sessions(uid)
session["session_version"] = new_version
await log_audit("sessions_invalidated", user_id=uid, username=g.user.username, ip_address=_client_ip())
await log_audit("sessions_invalidated", user_id=uid, username=g.user.username, ip_address=client_ip())
return jsonify({"status": "ok"})
@@ -208,14 +200,14 @@ async def update_email():
return jsonify({"error": "Email already in use by another account"}), 409
await update_user_email(uid, new_email)
await log_audit("email_change", user_id=uid, username=user.username, ip_address=_client_ip())
await log_audit("email_change", user_id=uid, username=user.username, ip_address=client_ip())
updated = await get_user_by_id(uid)
return jsonify(updated.to_dict())
@auth_bp.route("/forgot-password", methods=["POST"])
async def forgot_password():
if await is_rate_limited(f"forgot:{_client_ip()}", max_requests=5, window_seconds=300):
if await is_rate_limited(f"forgot:{client_ip()}", max_requests=5, window_seconds=300):
return jsonify({"status": "ok"})
data = await request.get_json()
email = (data.get("email") or "").strip().lower()
@@ -234,7 +226,7 @@ async def forgot_password():
"password_reset_requested",
user_id=user.id,
username=user.username,
ip_address=_client_ip(),
ip_address=client_ip(),
)
return jsonify({"status": "ok"})
@@ -242,8 +234,8 @@ async def forgot_password():
@auth_bp.route("/reset-password", methods=["POST"])
async def reset_password():
if await is_rate_limited(f"reset:{_client_ip()}", max_requests=10, window_seconds=60):
return jsonify({"error": "Too many attempts. Try again later."}), 429
if await is_rate_limited(f"reset:{client_ip()}", max_requests=10, window_seconds=60):
return jsonify({"error": f"Too many attempts. Try again later. {SHARED_ADDRESS_HINT}"}), 429
data = await request.get_json()
token = (data.get("token") or "").strip()
new_password = data.get("new_password") or ""
@@ -261,7 +253,7 @@ async def reset_password():
if user:
await log_audit(
"password_reset_completed",
user_id=user.id, username=user.username, ip_address=_client_ip(),
user_id=user.id, username=user.username, ip_address=client_ip(),
)
if user.email:
asyncio.create_task(send_password_reset_success_email(user.email))
@@ -304,7 +296,7 @@ async def register_with_invite():
"register_with_invite",
user_id=user.id,
username=user.username,
ip_address=_client_ip(),
ip_address=client_ip(),
)
return jsonify(user.to_dict()), 201
@@ -371,5 +363,5 @@ async def oauth_callback():
user = await find_or_create_oauth_user(sub, email, preferred_username)
session["user_id"] = user.id
session["session_version"] = user.session_version
await log_audit("oauth_login", user_id=user.id, username=user.username, ip_address=_client_ip())
await log_audit("oauth_login", user_id=user.id, username=user.username, ip_address=client_ip())
return redirect("/")
+62
View File
@@ -8,6 +8,8 @@ fmcp_<key>` API key (a read-scoped key suffices for this GET).
"""
from __future__ import annotations
import json
from quart import Blueprint, g, jsonify, request
from grimoire.auth import admin_required, get_current_user_id, login_required
@@ -21,6 +23,8 @@ from grimoire.services import report_check as report_check_svc
from grimoire.services import rule_moment_judgments as judgments_svc
from grimoire.services import shape_check as shape_check_svc
from grimoire.services import task_claims as task_claims_svc
from grimoire.services import write_scope as write_scope_svc
from grimoire.services.access import can_read_project
from grimoire.services.embeddings import memoized_query_embeddings
from grimoire.services.settings import get_admin_setting, set_setting
@@ -626,6 +630,64 @@ async def claim_session():
return jsonify({"claim": claim})
@plugin_bp.get("/write-scope")
@login_required
async def write_scope():
"""May this session do this act? Asked by `grimoire_project_fence.sh`
before a file edit, a git command or a Grimoire write.
A session belongs to the project its starting directory resolves to
(`repo` / `project_id`, as every plugin endpoint reads them), or when
that resolves to none, to `fallback_project_id`: the first project the
session entered. The act is
either a directory (`target_repo` / `target_project_id`) or an MCP call
(`tool`, with `input` as a JSON object of the call's short arguments: the
ids, kind and status that say where it writes). `act` words the act for the
reason, e.g. "git push in".
Returns `{allowed, session_project, targets, issue, reason}` from
services/write_scope.check_act. A GET, so a read-scoped key still fences
file edits and git commands.
"""
session_project, _repo, _unbound = await _project_scope()
if not session_project:
# A session that started outside any project belongs to the first one
# it entered; the hook remembers it.
try:
session_project = int(request.args.get("fallback_project_id") or 0)
except ValueError:
session_project = 0
if session_project and not await can_read_project(g.user.id, session_project):
session_project = 0
tool = (request.args.get("tool") or "").strip()
if tool.startswith("mcp__"):
tool = tool.rsplit("__", 1)[-1]
args: dict = {}
if tool:
try:
parsed = json.loads(request.args.get("input") or "{}")
except ValueError:
return jsonify({"error": "input must be a JSON object"}), 400
if not isinstance(parsed, dict):
return jsonify({"error": "input must be a JSON object"}), 400
args = parsed
target_project = 0
if not tool:
try:
target_project = int(request.args.get("target_project_id") or 0)
except ValueError:
target_project = 0
target_repo = (request.args.get("target_repo") or "").strip()
if not target_project and target_repo:
target_project = await repo_bindings_svc.resolve_project(g.user.id, target_repo) or 0
result = await write_scope_svc.check_act(
g.user.id, session_project, tool=tool, args=args,
target_project=target_project, act=(request.args.get("act") or "")[:80],
)
result["session_project"] = session_project or None
return jsonify(result)
@plugin_bp.get("/release-session")
@login_required
async def release_session():
File diff suppressed because it is too large Load Diff
+63 -3
View File
@@ -4,8 +4,9 @@ Two kinds of destination, both instance-wide and set by an admin:
- **folder**: a directory inside the container, usually a mounted share. A
backup service that watches the share (or a person) takes it from there.
- **s3**: any S3-compatible bucket, addressed by path. Grimoire only puts and
lists, so an append-only key is enough.
- **s3**: any S3-compatible bucket, addressed by path. Grimoire puts and
lists, and reads one back for a restore, so a key that can add and read
objects is enough; it never needs to delete.
Either way Grimoire ADDS one dated file per run and never removes one: keeping
and thinning the history is the destination's job (services/backup_file.py
@@ -28,7 +29,7 @@ from datetime import datetime
from pathlib import Path
from grimoire.services import backup_file
from grimoire.services.backup_s3 import S3Error, S3Target, list_objects, put_file
from grimoire.services.backup_s3 import S3Error, S3Target, get_file, list_objects, put_file
from grimoire.services.secrets import seal, unseal
from grimoire.services.settings import get_admin_setting, set_admin_setting
@@ -260,3 +261,62 @@ async def check_destination(
def _count_backups(folder: Path) -> int:
return sum(1 for name in os.listdir(folder) if backup_file.is_backup_name(name))
@dataclass(frozen=True)
class StoredBackup:
name: str
size_bytes: int
def _taken_order(name: str) -> str:
# The timestamp after the prefix, so grimoire- and scribe- names interleave
# by when they were taken.
return name.split("-", 1)[-1]
async def list_backups(config: BackupConfig, secret: str | None, limit: int = 50) -> list[StoredBackup]:
"""The backups at the destination, newest first."""
if config.destination == "folder":
folder = Path(config.folder_path)
if not folder.is_dir():
raise DestinationError(f"{folder} isn't there, or isn't a folder.")
def scan() -> list[StoredBackup]:
return [
StoredBackup(entry.name, entry.stat().st_size)
for entry in os.scandir(folder)
if entry.is_file() and backup_file.is_backup_name(entry.name)
]
found = await asyncio.to_thread(scan)
else:
problems = config.problems(has_secret=bool(secret))
if problems:
raise DestinationError(" ".join(problems))
try:
objects = await list_objects(s3_target(config, secret or ""))
except S3Error as exc:
raise DestinationError(str(exc)) from None
found = [StoredBackup(o.key, o.size) for o in objects if backup_file.is_backup_name(o.key)]
return sorted(found, key=lambda b: _taken_order(b.name), reverse=True)[:limit]
async def open_backup(config: BackupConfig, secret: str | None, name: str, staging: Path) -> Path:
"""A local path to the backup `name` at the destination: the file itself
in a folder, or a download into `staging` from a bucket. Only a name the
scheduled backup writes is accepted, so the request can't reach any other
file or object."""
if Path(name).name != name or not backup_file.is_backup_name(name):
raise DestinationError(f"{name!r} isn't the name of a Grimoire backup.")
if config.destination == "folder":
path = Path(config.folder_path) / name
if not path.is_file():
raise DestinationError(f"{name} isn't in {config.folder_path}.")
return path
target = s3_target(config, secret or "")
path = staging / name
try:
await get_file(target, name, path)
except S3Error as exc:
raise DestinationError(str(exc)) from None
return path
+66
View File
@@ -24,11 +24,14 @@ import os
import time
from datetime import datetime, timedelta, timezone
from pathlib import Path
from apscheduler.triggers.cron import CronTrigger
from grimoire.services.backup_destination import (
DestinationError, deliver, load_config, load_secret,
)
from grimoire.services.backup_file import write_archive
from grimoire.services.scheduler import ScheduledJob
from grimoire.services.settings import get_admin_setting, set_admin_setting
@@ -45,6 +48,9 @@ STALE_AFTER = timedelta(hours=26)
# has none, and a run that never returns holds the lock — every later backup
# would be skipped as "already running", silently, forever.
RUN_DEADLINE = timedelta(minutes=60)
# Under DATA_DIR: where a rollback keeps the backup of what it replaced when
# no backup destination takes it.
PRE_RESTORE_DIR = "pre-restore"
_LOCK = asyncio.Lock()
_TASKS: set[asyncio.Task] = set()
@@ -155,6 +161,66 @@ async def _export_and_deliver(config, secret, started):
)
async def save_state_before_restore() -> dict:
"""Back up the store as it is now, before a rollback replaces it, and say
where the file went: the backup destination when backups are on and it
takes the file, otherwise DATA_DIR/pre-restore. Raises when neither takes
it, so a rollback never runs without a way back."""
from grimoire.config import Config
from grimoire.services.backup import export_full_backup
at = datetime.now(timezone.utc)
app_version = os.environ.get("APP_VERSION", "dev")
payload = await export_full_backup()
note = None
config = await load_config()
if config.enabled:
secret, _ = await load_secret()
try:
delivered = await asyncio.wait_for(
deliver(payload, config, secret, at, app_version=app_version),
RUN_DEADLINE.total_seconds(),
)
return {"file": delivered.name, "size_bytes": delivered.size_bytes,
"where": delivered.where}
except (DestinationError, TimeoutError, asyncio.TimeoutError) as exc:
note = (f"The backup destination didn't take it ({exc or 'it timed out'}), "
"so it was kept on the instance instead.")
folder = Path(Config.DATA_DIR) / PRE_RESTORE_DIR
path = await asyncio.to_thread(write_archive, payload, folder, at, app_version=app_version)
saved = {"file": path.name, "size_bytes": path.stat().st_size, "where": str(folder)}
if note:
saved["note"] = note
return saved
async def rollback_to_backup(data: dict) -> dict:
"""Roll the store back to a full backup, after backing up what it
replaces. Refuses (RestoreRefused) before writing anything when the backup
can't be applied here, and raises BackupBusy while a backup is running:
a backup taken during a rollback would mix the two states."""
from grimoire.services.backup import check_replace, replace_from_backup
await check_replace(data)
if _LOCK.locked():
raise BackupBusy()
async with _LOCK:
safety = await save_state_before_restore()
result = await replace_from_backup(data)
result["safety_backup"] = safety
logger.info("rollback: the state it replaced is in %s/%s", safety["where"], safety["file"])
return result
def start_reindex_after_restore() -> None:
"""Rebuild the search vectors a rollback cleared, in the background."""
from grimoire.services.backup import reindex_after_restore
task = asyncio.get_running_loop().create_task(reindex_after_restore())
_TASKS.add(task)
task.add_done_callback(_TASKS.discard)
def start_backup_now() -> None:
"""Start a manual backup in the background (the request returns at once;
the page reads progress from `is_running` and the run list). Raises
+32 -3
View File
@@ -4,9 +4,10 @@ path-style addressing (endpoint/bucket/key), and the two calls a backup needs.
The signing is checked against AWS's published SigV4 examples in
tests/test_services_backup_s3.py.
Grimoire puts and lists; it never deletes. The bucket keeps the history and
thins it on its own policy, so a key that may only add objects is enough —
and is the safer key to hand a service, because a stolen one cannot wipe the
Grimoire puts and lists, and reads a backup back when an admin restores from
the bucket; it never deletes. The bucket keeps the history and thins it on
its own policy, so a key that may only add and read objects is enough — and
is the safer key to hand a service, because a stolen one cannot wipe the
history it writes into.
Written by hand rather than with boto3: two calls don't justify a large
@@ -120,6 +121,12 @@ def _says(response: httpx.Response) -> str:
"AccessDenied": "the key isn't allowed to do that",
}
said = hints.get(code) or message or response.reason_phrase
if not code and 400 <= response.status_code < 500:
# S3 refuses with an <Error><Code> body. A 4xx without one came from
# something else at that address — most often a web server on the
# default port, when the S3 service listens on a port of its own.
said = ("the address answered, but not as S3 — check the scheme and "
"the port (an S3 service usually listens on its own port)")
return f"HTTP {response.status_code}{f' {code}' if code else ''}: {said}."
@@ -165,6 +172,28 @@ async def put_file(target: S3Target, key: str, path: Path, *,
raise S3Error(f"Upload of {key} refused: {_says(response)}")
async def get_file(target: S3Target, key: str, path: Path, *,
transport: httpx.AsyncBaseTransport | None = None,
now: datetime | None = None) -> None:
"""Download one object to `path`, streamed to disk."""
url = target.url(key)
headers = sign("GET", url, {}, EMPTY_SHA256, target.access_key, target.secret_key,
target.region, now or datetime.now(timezone.utc))
try:
async with httpx.AsyncClient(verify=target.verify_tls,
timeout=UPLOAD_TIMEOUT_SECONDS,
transport=transport) as client:
async with client.stream("GET", url, headers=headers) as response:
if response.status_code != 200:
await response.aread()
raise S3Error(f"Reading {key} refused: {_says(response)}")
with open(path, "wb") as f:
async for chunk in response.aiter_bytes(CHUNK):
f.write(chunk)
except httpx.HTTPError as exc:
raise _reach_error(target, exc) from None
async def list_objects(target: S3Target, prefix: str = "", limit: int = 1000, *,
transport: httpx.AsyncBaseTransport | None = None,
now: datetime | None = None) -> list[StoredObject]:
+46
View File
@@ -0,0 +1,46 @@
"""This instance's id lineage (models/instance.py says what it means)."""
from __future__ import annotations
import uuid
from sqlalchemy import select
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.ext.asyncio import AsyncSession
from grimoire.models import async_session
from grimoire.models.instance import InstanceIdentity
async def _read_or_make(session: AsyncSession) -> str:
current = (await session.execute(select(InstanceIdentity.instance_id))).scalar_one_or_none()
if current:
return current
# The migration makes the row; this covers a schema built some other way.
await session.execute(
insert(InstanceIdentity)
.values(id=1, instance_id=str(uuid.uuid4()))
.on_conflict_do_nothing(index_elements=["id"])
)
return (await session.execute(select(InstanceIdentity.instance_id))).scalar_one()
async def get_instance_id(session: AsyncSession | None = None) -> str:
"""This instance's id. Inside `session` when given, so a caller's
transaction sees the same value it may be about to change."""
if session is not None:
return await _read_or_make(session)
async with async_session() as own:
value = await _read_or_make(own)
await own.commit()
return value
async def adopt_instance_id(session: AsyncSession, value: str) -> None:
"""Take a backup's id as this instance's own, inside the caller's
transaction: after a restore that kept the backup's original ids, this
store and the one the backup came from number their rows the same way."""
await session.execute(
insert(InstanceIdentity)
.values(id=1, instance_id=value)
.on_conflict_do_update(index_elements=["id"], set_={"instance_id": value})
)
+2 -2
View File
@@ -77,8 +77,8 @@ def location_matches(data: dict | None, parts: dict[str, str]) -> bool:
if all(
_path_matches((loc.get(key) or "").strip(), want)
if key == "path"
# Repo names are recorded free-form ("Grimoire" / "FabledScribe" /
# "fabledscribe") — case is never the distinguishing thing.
# Repo names are recorded free-form ("Grimoire" / "FabledGrimoire" /
# "fabledgrimoire") — case is never the distinguishing thing.
else (loc.get(key) or "").strip().lower() == want.lower()
if key == "repo"
else (loc.get(key) or "").strip() == want
+28
View File
@@ -324,6 +324,34 @@ async def lessons_for_rule(user_id: int, rule_id: int) -> list[dict]:
]
async def confirmed_lesson_counts(user_id: int, rule_ids) -> dict[int, int]:
"""{rule_id: confirmed lessons the caller can read}, for a list of rules.
`lessons_for_rule` in bulk, counted rather than listed: one GROUP BY, so a
rule list can say which rules are proven by situations without a query
per row. Rules with none read 0.
"""
from grimoire.services.access import readable_notes_clause
ids = [int(i) for i in rule_ids or []]
out = {i: 0 for i in ids}
if not ids:
return out
async with async_session() as session:
rows = (await session.execute(
select(LessonRuleLink.rule_id, func.count())
.join(Note, Note.id == LessonRuleLink.lesson_id)
.where(LessonRuleLink.rule_id.in_(ids))
.where(LessonRuleLink.state == CONFIRMED)
.where(Note.deleted_at.is_(None))
.where(readable_notes_clause(user_id))
.group_by(LessonRuleLink.rule_id)
)).all()
for rule_id, n in rows:
out[int(rule_id)] = int(n)
return out
async def judgments_for_lessons(lesson_ids) -> dict[int, dict]:
"""{lesson_id: {"rule_judgment": linked|no_rule|unjudged, "no_rule"?}}.
+189 -136
View File
@@ -143,8 +143,13 @@ WRITEPATH_ENABLED_KEY = "kb_writepath_enabled"
WRITEPATH_THRESHOLD_KEY = SURFACES["write_path"].floor_key
WRITEPATH_DEFAULT_ENABLED = True
WRITEPATH_DEFAULT_THRESHOLD = SURFACES["write_path"].floor_default
# How far past the write-path floor a snippet must resemble a NEW definition
# before the hint asks whether the code is a copy of it. A margin, not a second
# floor, so it moves when the operator tunes the write-path floor: at the
# default 0.68 the copy bar is 0.80.
WRITEPATH_COPY_MARGIN = 0.12
# The standing-rule arm gets its own bar, for a third corpus: 0.68 above is
# The standing-rule arms get their own bar, for a third corpus: 0.68 above is
# calibrated on code against note PROSE, not code against RULE TEXT.
#
# THE STRUCTURAL ARGUMENT, which holds on any install, including one with six
@@ -155,64 +160,27 @@ WRITEPATH_DEFAULT_THRESHOLD = SURFACES["write_path"].floor_default
# pool always returns something, so "the best match cleared the bar"
# drifts from "a good match exists" toward "N things were ranked". A bar
# calibrated for best-of-thousands is cleared by best-of-forty as
# arithmetic rather than relevance. (The arms search every tier, which
# makes the pool larger; the bar still errs toward silence on purpose.)
# arithmetic rather than relevance.
# 2. Rules are short imperative technical English — a far more HOMOGENEOUS
# corpus than note prose, and a more homogeneous corpus has a HIGHER
# noise floor than prose's 0.55-0.63.
#
# WHY 0.72 AND NOT A NUMBER OFF A HISTOGRAM. The exact offset between prose's
# floor and rule-text's is not derivable in general — it depends on how an
# install writes its rules — so the default errs deliberately toward SILENCE
# rather than toward recall, on an asymmetry that is itself structural: this
# hint fires on EVERY write. A missed rule is recoverable, because the rule is
# still in Grimoire and the agent can search it. A hint that cries wolf is not:
# it teaches the reader to skip the whole block, and the surface is lost along
# with the true positives it would have carried.
# ONE BAR FOR EVERY RULE ARM THAT ANNOTATES. The write path, the command arm
# and the prompt arm read the same key (`RULE_FLOOR_KEY` in retrieval_pipeline):
# they search the same rules, and what limits each one is its budget and band.
#
# TUNE IT FROM YOUR OWN INSTANCE: `retrieval_telemetry` reports
# `rule_usage.pull_through`. Raise this if rules arrive unread; lower it if
# rules you needed never arrived. What would RETIRE it: a cross-encoder rerank,
# which would make a similarity bar the wrong control entirely.
#
# This bar is the WRITE-PATH arm's. A code payload is long and rich, which is
# the case 0.72 suits; the command arm has its own bar below.
# `rule_usage.pull_through` and `near_miss_samples`. Raise it if rules arrive
# unread; lower it if rules you needed never arrived. What would RETIRE it: a
# cross-encoder rerank, which would make a similarity bar the wrong control
# entirely.
RULEHINT_THRESHOLD_KEY = SURFACES["write_path_rule"].floor_key
RULEHINT_DEFAULT_THRESHOLD = SURFACES["write_path_rule"].floor_default
# THE COMMAND ARM'S OWN BAR, AND WHY IT IS NOT THE WRITE PATH'S.
#
# A write-path query is a code payload, long and rich, while a pre-tool query
# is a shell command — often under a dozen words. Less text, less signal, lower
# scores for the same relevance. At the write path's 0.72 the command arm
# turns nearly everything away, with its refused mass piled just under the
# line, which is the shape a bar set too high leaves.
#
# WHAT 0.68 IS CALIBRATED AGAINST. Replayed queries, consequential acts
# against innocuous ones, scored against a rule corpus with symptom-bearing
# triggers:
#
# 0.7571 git push origin dev consequential
# 0.7245 cd ...; git fetch; git add -A consequential
# 0.7193 git pull --rebase origin dev consequential
# 0.6850 docker compose up -d consequential
# ---------------------------------------- 0.68
# 0.6735 wc -l src/*.py && date innocuous
# 0.6544 grep -rn useState src/ innocuous
# 0.6099 sed -n '120,160p' package.json innocuous
# 0.6056 ls -la && cat README.md innocuous
#
# At 0.72 three of the four consequential acts would retrieve nothing, though
# the right rules ranked first for each.
#
# THE SEPARATION IS NARROW (about 0.01), and that is a caveat, not a result.
# Eight probes set a direction; they do not settle a number.
# `near_miss_samples` on real traffic is what settles it, and this is the bar
# to re-read first.
#
# WHY A LOWER BAR IS SAFE HERE. The band does noise control downstream: a marginal hit that
# clears the bar still has to score within `_RULEHINT_BAND` of the top to be
# rendered. The bar's job shrank, so the bar can.
# THE COMMAND ARM READS THE SAME BAR. A shell command is short, so it scores
# lower than a code payload for the same relevance and the shared bar admits
# less here. The band does noise control downstream: a hit that clears the bar
# still has to score within `_RULEHINT_BAND` of the top to be rendered.
#
# The innocuous scores near the line are largely CROSS-PROJECT BLEED rather
# than bad ranking: another project's shell-command rule matching a shell
@@ -439,11 +407,13 @@ _CONCEPT_MIN_CHARS = 16
_AUTOINJECT_MAX_TOP_K = MAX_BUDGET
# THE CONVERSATION BESIDE THE PROMPT. The operator's message alone is a weak
# query mid-session: it is mostly a follow-up — "yes do that", "now fix the
# filter" — that names nothing a record could match. The hook sends the tail of the last assistant reply as `context`, and it is
# appended to a SHORT prompt only: a prompt that already says what it is about
# is the better query on its own, and diluting it is the one way this can make
# retrieval worse.
# query mid-session: it is mostly a follow-up — "now fix the filter" — that
# names too little for a record to match. The hook sends the tail of the last
# assistant reply as `context`, and it is appended to a SHORT prompt only: a
# prompt that already says what it is about is the better query on its own,
# and diluting it is the one way this can make retrieval worse. A prompt that
# names nothing at all ("yes do that") gets no notes menu (`_has_subject`);
# the prompt rule arm still searches it with the tail.
#
# Neither number costs a model token. The query is embedding input; what the
# session pays for is the menu, and the budget bounds that unchanged.
@@ -455,6 +425,42 @@ _AUTOINJECT_CONTEXT_PROMPT_MAX = 280
_AUTOINJECT_CONTEXT_MAX = 600
# WORDS THAT NAME NO SUBJECT. A prompt made only of these ("keep going", "yes
# please do", "1. yes 2. yes", "is it deployed?", "what is next?") is a turn of
# the conversation, not a question about anything. Its query would be the
# previous reply's tail alone, and the menu would hand back the records that
# reply was built from, which the session already has in context. One word
# outside this list ("now fix the filter") is a subject, and the prompt is
# searched as usual.
_NO_SUBJECT_WORDS = frozenset("""
a about above after again agreed ahead all already also am an and any
anything approved are as asked at awesome back be been before both build
but by can carry ci commit continue continuing could cool correct
deploy deployed described did do does done each else every everything
fine finish first fix for from get go going good got great green had has
have he here hold how i if im in instead into is it its just keep know
last left let lets like looks loose lgtm make may maybe me mentioned merge
milestone more move my next nice no not now of off ok okay on one open or
order other our out over pass passes passing perfect please previous
proceed project proposed push ready really recommended redeploy rest right
said same see she ship should so some sounds stack stacking start step
steps still stop suggested sure task tasks thank thanks that the them then
there these they thing things this those through to try up us waiting
want was we well were what whats when where which while who why will with
work would yeah yep yes yet you your
""".split())
def _has_subject(prompt: str) -> bool:
"""Does the operator's message name anything a record could be about?
Numbers are left out: a record named by number is looked up by id, not
matched, so it never needs the ranked menu to be found.
"""
words = re.findall(r"[a-z]+", prompt.lower().replace("'", ""))
return any(w not in _NO_SUBJECT_WORDS for w in words)
def _autoinject_query(prompt: str, context: str) -> str:
"""The prompt, with recent conversation appended when the prompt is thin.
@@ -478,17 +484,11 @@ def _autoinject_query(prompt: str, context: str) -> str:
# The operator's message is the only query that exists before one is composed,
# and this arm is what runs against it.
PROMPTRULE_THRESHOLD_KEY = SURFACES["prompt_rule"].floor_key
# INHERITED FROM THE ACT ARMS. 0.72 is tuned against
# code and shell commands. An operator's prose is a different query shape
# against the same documents, and nothing yet says the two distributions line
# up — triggers are written in the vocabulary of the MOMENT, which for most
# rules is act vocabulary, so prose may well score lower across the board.
#
# Starting at the act arms' number anyway is deliberate: it is the only value
# with evidence behind it, and guessing lower would put an unmeasured bar in
# front of a corpus that binds. Every call is logged under `prompt_rule`, so
# real traffic settles it — read `near_miss_samples` before moving this, not
# the percentile alone.
# THE SAME BAR AS THE ACT ARMS. An operator's prose is a different query shape
# against the same documents, and triggers are written in the vocabulary of
# the moment, which for most rules is act vocabulary. Every call is logged
# under `prompt_rule`, so read `near_miss_samples` before moving the shared
# bar, not the percentile alone.
PROMPTRULE_DEFAULT_THRESHOLD = SURFACES["prompt_rule"].floor_default
# ANCHORED ON THIS HOOK'S BUDGET rather than the act arms'. This arm fires
@@ -655,10 +655,6 @@ async def build_autoinject_hint(
# left to a ranking that can only match the words around them.
named = await _named_records(user_id, record_refs_svc.named_record_ids(q))
named_ids = [int(n.id) for n in named]
# Everything below searches, logs and fills its slots on the ENRICHED
# query, so the telemetry row records what was actually asked.
q = _autoinject_query(q, context)
# THE LEDGER IS A RENDERING FACT, NOT A RETRIEVAL ONE. Excluding seen ids
# from the search would make the second time a note is the best answer
# SILENCE, indistinguishable from "nothing matched", and would measure
@@ -672,15 +668,23 @@ async def build_autoinject_hint(
# lesson slots in that order, and the surfacing rows are written there.
# What is decided HERE is what makes this moment this moment — the query
# the operator's words became, and the records they named by number.
result = await rp.run_note_arm(
rp.AUTO_INJECT,
rp.NoteMoment(
user_id=user_id, query=q, project_id=project_id,
seen=frozenset(already), named=frozenset(named_ids),
),
floor=cfg["threshold"], budget=cfg["top_k"], io=_note_io(),
)
kept = result.menu
if not _has_subject(q):
# Nothing to search for: the menu would answer the previous reply,
# not the operator. Records named by number are still shown.
kept, chunks = [], {}
else:
# Everything below searches, logs and fills its slots on the ENRICHED
# query, so the telemetry row records what was actually asked.
q = _autoinject_query(q, context)
result = await rp.run_note_arm(
rp.AUTO_INJECT,
rp.NoteMoment(
user_id=user_id, query=q, project_id=project_id,
seen=frozenset(already), named=frozenset(named_ids),
),
floor=cfg["threshold"], budget=cfg["top_k"], io=_note_io(),
)
kept, chunks = result.menu, result.chunks
if not kept and not named:
return empty
@@ -782,7 +786,7 @@ async def build_autoinject_hint(
# a reason, and a reader cannot tell the two apart once indented alike.
passage = "" if nid in already else _menu_passage(
document_title(note.title, note.note_type, note.data, note.body),
(result.chunks.get(nid) or {}).get("text"), name,
(chunks.get(nid) or {}).get("text"), name,
)
who = _shared_by(note)
lines.extend(menu_entry(
@@ -1171,9 +1175,8 @@ async def get_writepath_config(user_id: int) -> dict:
independent pairs, resolved from the registry.
Each floor's reason is recorded where its default is: WRITEPATH (code
embeddings sit on a much higher similarity floor than prose), RULEHINT (a
third corpus again), TOOLRULE (a shell command is a different query shape
from a code payload and scores lower for the same relevance).
embeddings sit on a much higher similarity floor than prose) and RULEHINT
(a third corpus again, whose one bar both rule arms read).
THE BUDGET IS SPLIT TOO. "How many titles at once" does not mean the same
thing on both surfaces: this arm fires before every Write and Edit while
@@ -1446,6 +1449,9 @@ async def build_write_path_hint(
note.title, note.note_type, note.data, note.body,
),
"seen": int(note.id) in excluded,
# Read by the copy check, which asks for more
# resemblance than the menu does.
"score": float(score),
# Carried, not re-read off the rendered marker. The
# marker is prose assembled for a human and it already
# varies by kind, language and the `seen` flag — a
@@ -1500,16 +1506,35 @@ async def build_write_path_hint(
derive = [d for d in found if d.get("key") not in skip]
except Exception:
logger.warning("write-time derive check failed", exc_info=True)
# The COPY check: the derive check matches by name, so a renamed copy
# of a recorded snippet passes it and shows only as "similar" prior art.
# When this write introduces a definition and an unseen snippet on the
# menu resembles it well past the menu's own floor, say that a copy is
# the moment to extract. Only a recorded snippet can be seen this way —
# the ledger keeps no bodies to compare an unrecorded sibling against.
copies: list[dict] = []
copy_bar = cfg["threshold"] + WRITEPATH_COPY_MARGIN
strong = [
it for _m, it in menu
if it.get("kind") == "snippet" and not it.get("seen")
and it.get("score", 0.0) >= copy_bar
]
if strong and stamp_shapes and project_id:
named = {d["symbol"] for d in derive} | {s["symbol"] for s in suggested}
try:
fresh = await shape_ledger_svc.new_shapes_at(project_id, path, stamp_shapes)
copies = [
{"symbol": n, "kind": k, "snippet": strong[0]}
for k, n in fresh if n not in named
]
except Exception:
logger.warning("write-time copy check failed", exc_info=True)
staleness: list[str] = []
# The standing-rule arm further down is deliberately left on the far side
# of the guard below: it runs a SEMANTIC search, and moving it above would
# run an embedding query on every write in the session.
# The design arm is decided HERE, above the guard, so a UI write
# that matched no prior art still carries it — and it returns on its own
# rather than joining the guard's condition, because joining it would let
# a design line switch the standing-rule arm below on for writes where it
# has never run, moving that arm's call distribution under its floor.
# rather than joining the guard's condition: a design line alone is not a
# reason to render the prior-art menu's frame.
design_text, design_dedup = await _design_arm(
user_id, project_id, path, set(exclude_derive or []),
)
@@ -1520,7 +1545,62 @@ async def build_write_path_hint(
user_id, project_id, [path], seen=seen_ruling_systems,
source="rulings_write_path",
)
if not staleness and not synced and not menu and not suggested and not divergence and not derive:
# ── Standing rules that may apply here ───────────────────────────────
#
# A SUGGESTION, not a binding surface, and the distinction is the design:
# a rule BINDS by being tagged to an area the project works in,
# resolved deterministically at enter_project. This arm reaches for
# something weaker and still useful — a conditional rule whose trigger
# resembles what is being written, noticed at the moment it is relevant
# rather than by being resident in every session.
#
# EVERY RULE IS ELIGIBLE, including one already named this session: a
# rule the session was told about an hour ago is not a rule in front of
# the reader now.
#
# The arm itself is `retrieval_pipeline.run_rule_arm`: the band, the
# fresh/repeat split, the unconditional call row and the fresh-only
# surfacing rows are written there once for all three
# rule arms. What is decided HERE is only what makes this moment this
# moment — the query is the code being written (or the path, for an edit
# with no body), and a line says the rule may apply "here".
#
# EVERY WRITE WITH A PATH GETS THE CHECK, whether or not prior art matched:
# a rule about the file being written does not depend on a snippet
# resembling it. It runs before the notes guard below for that reason,
# at the price of one more search per write; its query, `code or path`,
# differs from the notes arm's concept query, so it is its own embedding.
#
# Fails open like every other arm: a rule hint must never break a write.
# The direct band, then a rule reached through a linked lesson,
# composed by `_rule_moment` exactly as the other two rule builders are.
rule_lines: list[str] = []
rule_ids: list[int] = []
shown_rule_ids: list[int] = []
checkpoint: dict = {}
try:
result = await _rule_moment(
rp.WRITE_PATH_RULE,
rp.RuleMoment(
user_id=user_id, query=code or path, project_id=project_id,
where="here", checkpoint_where="here",
exclude=frozenset(exclude_rule_ids or []),
held=frozenset(held_rule_ids or []),
),
floor=cfg["rule_threshold"], budget=cfg["rule_top_k"],
checkpoint_floor=cfg["checkpoint_threshold"],
)
rule_lines = result.lines
rule_ids.extend(result.rule_ids)
# Every rule line the band showed, repeats included — what the
# soft-link recorder pairs with this response's lessons.
shown_rule_ids = result.shown_rule_ids
checkpoint = result.checkpoint
except Exception:
logger.debug("write-path rule arm failed", exc_info=True)
if (not staleness and not synced and not menu and not suggested
and not divergence and not derive and not rule_lines):
if design_text or rulings["lines"]:
return {
**empty,
@@ -1632,6 +1712,8 @@ async def build_write_path_hint(
lines.append(_divergence_line(path, divergence))
if derive:
lines.append(_derive_line(path, derive))
if copies:
lines.append(_copy_line(copies))
# Split by arm, which is the whole reason this table exists. The place arm
# carries no score and so has no home in retrieval_logs, yet it fires on
@@ -1656,52 +1738,7 @@ async def build_write_path_hint(
user_id=user_id, note_ids=ids, source=arm, project_id=project_id,
)
# ── Standing rules that may apply here ───────────────────────────────
#
# A SUGGESTION, not a binding surface, and the distinction is the design:
# a rule BINDS by being tagged to an area the project works in,
# resolved deterministically at enter_project. This arm reaches for
# something weaker and still useful — a conditional rule whose trigger
# resembles what is being written, noticed at the moment it is relevant
# rather than by being resident in every session.
#
# EVERY RULE IS ELIGIBLE, including one already named this session: a
# rule the session was told about an hour ago is not a rule in front of
# the reader now.
#
# The arm itself is `retrieval_pipeline.run_rule_arm`: the band, the
# fresh/repeat split, the unconditional call row and the fresh-only
# surfacing rows are written there once for all three
# rule arms. What is decided HERE is only what makes this moment this
# moment — the query is the code being written (or the path, for an edit
# with no body), and a line says the rule may apply "here".
#
# Fails open like every other arm: a rule hint must never break a write.
# The direct band, then a rule reached through a linked lesson,
# composed by `_rule_moment` exactly as the other two rule builders are.
rule_ids: list[int] = []
shown_rule_ids: list[int] = []
checkpoint: dict = {}
try:
result = await _rule_moment(
rp.WRITE_PATH_RULE,
rp.RuleMoment(
user_id=user_id, query=code or path, project_id=project_id,
where="here", checkpoint_where="here",
exclude=frozenset(exclude_rule_ids or []),
held=frozenset(held_rule_ids or []),
),
floor=cfg["rule_threshold"], budget=cfg["rule_top_k"],
checkpoint_floor=cfg["checkpoint_threshold"],
)
lines.extend(result.lines)
rule_ids.extend(result.rule_ids)
# Every rule line the band showed, repeats included — what the
# soft-link recorder pairs with this response's lessons.
shown_rule_ids = result.shown_rule_ids
checkpoint = result.checkpoint
except Exception:
logger.debug("write-path rule arm failed", exc_info=True)
lines.extend(rule_lines)
# A lesson and a rule on this one response: the soft-link
# recorder counts the pair, keyed on the FILE — every edit to one file is
@@ -2004,6 +2041,22 @@ def _divergence_line(path: str, divergence: list[dict]) -> str:
)
def _copy_line(copies: list[dict]) -> str:
"""A new definition that closely resembles a recorded snippet under
another name: the derive check cannot see a renamed copy, so this says
it — as a question for the writer, since resemblance is not identity."""
snip = copies[0]["snippet"]
names = ", ".join(
f"`{'.' if c['kind'] == 'css' else ''}{c['symbol']}`" for c in copies
)
return (
f"> {names} {'is' if len(copies) == 1 else 'are'} new in this file and "
f"this code resembles snippet #{snip['id']} `{snip.get('name') or snip['title']}` "
f"(similar {snip['score']:.2f}). If it copies that shape, reuse #{snip['id']} "
"or extract the shared shape now, before another copy is written."
)
def _suggest_line(path: str, suggested: list[dict]) -> str:
"""One line naming the recorded shape this code looks like, as evidence
for the judgment the agent gives at the end of the turn — not a record of
+129
View File
@@ -0,0 +1,129 @@
"""The reverse-proxy hop count an admin saves in Settings → Config.
How many proxies stand in front of the app decides which address every rate
limit and audit line sees, and whether a request counts as HTTPS
(grimoire/proxy.py). Behind Cloudflare's proxied DNS the right count is 2; at
the default of 1 every visitor reads as a Cloudflare edge address and they
share one sign-in rate limit. So the count is set where an admin can find it
and check it against their own request, not only in the environment.
Precedence: the saved count, then `TRUSTED_PROXY_HOPS` from the environment,
then 1. Saving None removes the saved count, so the environment's applies
again.
It is one instance-wide value, kept on the first admin account like the other
instance settings. Every worker holds it in proxy.py, loads it at startup and
re-reads it every REFRESH_SECONDS, so a save in one worker reaches the others
within that interval; the worker that saved applies it at once.
"""
from __future__ import annotations
import asyncio
import logging
from sqlalchemy import delete, select
from grimoire import proxy
from grimoire.config import Config
from grimoire.models import async_session
from grimoire.models.setting import Setting
from grimoire.models.user import User
logger = logging.getLogger(__name__)
SETTING_KEY = "trusted_proxy_hops"
REFRESH_SECONDS = 30
# The load at startup runs inside the lifespan hook, and a hook that does not
# return is a worker that never serves (db_maintenance_scheduler explains the
# outage that taught this). One indexed row answers in milliseconds, so the
# deadline only fires when the database is already in trouble.
STARTUP_READ_TIMEOUT = 3.0
def parse_hops(raw) -> int | None:
"""A hop count from 0 to proxy.MAX_HOPS, or None for anything else. A
bool is refused, though Python counts it as an int."""
if isinstance(raw, bool):
return None
if isinstance(raw, str):
raw = raw.strip()
if not raw.isdigit():
return None
raw = int(raw)
if not isinstance(raw, int) or not 0 <= raw <= proxy.MAX_HOPS:
return None
return raw
def _admin_rows():
return (
select(Setting)
.join(User, Setting.user_id == User.id)
.where(User.role == "admin", Setting.key == SETTING_KEY)
)
async def saved_hops() -> int | None:
"""The count saved in Settings, or None when there is none (or what is
stored can't be read as one)."""
async with async_session() as session:
row = (await session.execute(_admin_rows().order_by(Setting.user_id))).scalars().first()
return parse_hops(row.value) if row else None
async def save_hops(hops: int | None) -> None:
"""Save the count, or remove it with None. Any copy on another admin's
account goes too, so exactly one value is ever read back."""
async with async_session() as session:
admin_ids = select(User.id).where(User.role == "admin")
await session.execute(
delete(Setting).where(Setting.key == SETTING_KEY, Setting.user_id.in_(admin_ids))
)
if hops is not None:
first_admin = (
await session.execute(select(User.id).where(User.role == "admin").order_by(User.id))
).scalars().first()
if first_admin is None:
raise ValueError("there is no admin account to keep the setting on")
session.add(Setting(user_id=first_admin, key=SETTING_KEY, value=str(hops)))
await session.commit()
proxy.set_saved_hops(hops)
async def refresh() -> None:
"""Load the saved count into this worker. A failed read keeps the value
it already had: an unreachable database must not switch the trust rule."""
try:
proxy.set_saved_hops(await saved_hops())
except Exception:
logger.debug("could not re-read the saved proxy hop count", exc_info=True)
async def load_at_startup() -> None:
"""`refresh`, bounded for the lifespan hook. On a timeout the environment's
count stays in force until the next refresh reaches the database."""
try:
await asyncio.wait_for(refresh(), timeout=STARTUP_READ_TIMEOUT)
except (TimeoutError, asyncio.TimeoutError):
logger.warning(
"reading the saved proxy hop count exceeded %.1fs; starting with %d "
"from the environment until the database answers",
STARTUP_READ_TIMEOUT, Config.TRUSTED_PROXY_HOPS,
)
_refresh_task = None
def start_refresh_loop() -> None:
global _refresh_task
if _refresh_task is None or _refresh_task.done():
from grimoire.services.background import start_periodic
_refresh_task = start_periodic(REFRESH_SECONDS, refresh, label="proxy_hops_refresh")
def source() -> str:
"""Where the count in force came from: "setting", "env" or "default"."""
if proxy.saved_hops_in_force() is not None:
return "setting"
return "env" if Config.TRUSTED_PROXY_HOPS_FROM_ENV else "default"
+58 -5
View File
@@ -9,9 +9,10 @@ from __future__ import annotations
import re
from sqlalchemy import select
from sqlalchemy import select, update
from grimoire.models import async_session
from grimoire.models.code_shape import CodeShape
from grimoire.models.repo_binding import RepoBinding
@@ -20,11 +21,11 @@ def normalize_repo_key(raw: str) -> str:
Collapses the equivalent clone URLs to one key, e.g.::
git@git.fabledsword.com:bvandeusen/FabledScribe.git
https://git.fabledsword.com/bvandeusen/fabledscribe.git
ssh://git@git.fabledsword.com:22/bvandeusen/fabledscribe
git@git.fabledsword.com:bvandeusen/FabledGrimoire.git
https://git.fabledsword.com/bvandeusen/fabledgrimoire.git
ssh://git@git.fabledsword.com:22/bvandeusen/fabledgrimoire
all normalize to ``git.fabledsword.com/bvandeusen/fabledscribe``.
all normalize to ``git.fabledsword.com/bvandeusen/fabledgrimoire``.
Returns "" for empty/garbage input so callers can treat it as "no repo".
"""
@@ -100,6 +101,58 @@ async def set_binding(
return binding
async def move_binding(
user_id: int, old_repo: str, new_repo: str, project_id: int,
) -> tuple[RepoBinding, int]:
"""Carry a renamed repo's binding, and its project's shape ledger, from the
old remote to the new one. Returns the binding and how many shapes moved.
Every ledger row is keyed by the repo it was found in, so a repo renamed on
its forge would otherwise start an empty ledger under the new name and
leave every judgment behind under the old one. Both keys change in one
transaction. It refuses rather than merge: the old key must be bound to
this project by the caller, and the new key must have neither a binding of
the caller's nor ledger rows of this project's.
"""
old_key = normalize_repo_key(old_repo)
new_key = normalize_repo_key(new_repo)
if not old_key or not new_key:
raise ValueError("both remotes must be parseable repo URLs")
if old_key == new_key:
raise ValueError(f"`{old_key}` is the same repo under both names")
async with async_session() as session:
mine = select(RepoBinding).where(RepoBinding.user_id == user_id)
binding = (
await session.execute(mine.where(RepoBinding.repo_key == old_key))
).scalar_one_or_none()
if binding is None or binding.project_id != project_id:
raise ValueError(f"`{old_key}` is not bound to project {project_id}")
if (
await session.execute(mine.where(RepoBinding.repo_key == new_key))
).scalar_one_or_none() is not None:
raise ValueError(
f"`{new_key}` is already bound; unbind it first, then move"
)
shapes_here = select(CodeShape.id).where(CodeShape.project_id == project_id)
if (
await session.execute(
shapes_here.where(CodeShape.repo_key == new_key).limit(1)
)
).first() is not None:
raise ValueError(
f"project {project_id} already has a ledger under `{new_key}`"
)
moved = await session.execute(
update(CodeShape)
.where(CodeShape.project_id == project_id, CodeShape.repo_key == old_key)
.values(repo_key=new_key)
)
binding.repo_key = new_key
await session.commit()
await session.refresh(binding)
return binding, moved.rowcount or 0
async def list_bindings(user_id: int) -> list[RepoBinding]:
async with async_session() as session:
rows = await session.execute(
+33 -11
View File
@@ -460,16 +460,31 @@ class RuleArm:
declared: Declared | None = None
# The arms differ on purpose where these flags differ. Whether the prompt arm
# should band, and whether the act arms should reserve a preference, are open
# questions to settle on telemetry, not by symmetry.
# ONE BAR FOR EVERY RULE ARM THAT ANNOTATES. The prompt, command and write
# arms search the same corpus, and what limits what they show is the budget
# and the band, not the floor. The floor only answers "is this worth ranking
# at all", and that question has one answer per corpus, not one per query
# shape — so the three share one settings key, and moving it moves all three.
# `set_dial` records the move on every surface that shares the key.
#
# Set low on purpose: a hit under it is never shown, so a bar set too high
# costs a rule that binds, while one set a little low costs a line the band
# usually drops. The reply backstop keeps its own number, because there the
# floor is a stop bar and not a hint bar.
RULE_FLOOR_KEY = "kb_rule_threshold"
RULE_FLOOR_DEFAULT = 0.70
# The arms differ on purpose where these flags differ: the prompt arm takes no
# band, so its budget alone decides how many hits it shows, and the act arms
# band and may hold the act. Every arm that annotates reserves a line for a
# preference.
PROMPT_RULE = RuleArm(
"prompt_rule", band=False, compact_tail=False, checkpoint=False,
preference_slot=True,
tuning=Surface(
name="prompt_rule",
floor_key="kb_promptrule_threshold",
floor_default=0.72,
floor_key=RULE_FLOOR_KEY,
floor_default=RULE_FLOOR_DEFAULT,
budget_key="kb_promptrule_top_k",
budget_default=3,
asks="the operator's message, against rule triggers",
@@ -480,11 +495,11 @@ PROMPT_RULE = RuleArm(
)
PRE_TOOL_RULE = RuleArm(
"pre_tool_rule", band=True, compact_tail=True, checkpoint=True,
preference_slot=False,
preference_slot=True,
tuning=Surface(
name="pre_tool_rule",
floor_key="kb_toolrule_threshold",
floor_default=0.68,
floor_key=RULE_FLOOR_KEY,
floor_default=RULE_FLOOR_DEFAULT,
budget_key="kb_toolrule_top_k",
budget_default=5,
asks="the command about to run, against rule triggers",
@@ -495,11 +510,11 @@ PRE_TOOL_RULE = RuleArm(
)
WRITE_PATH_RULE = RuleArm(
"write_path_rule", band=True, compact_tail=True, checkpoint=True,
preference_slot=False,
preference_slot=True,
tuning=Surface(
name="write_path_rule",
floor_key="kb_rulehint_threshold",
floor_default=0.72,
floor_key=RULE_FLOOR_KEY,
floor_default=RULE_FLOOR_DEFAULT,
budget_key="kb_rulehint_top_k",
budget_default=5,
asks="the code being written, against rule triggers",
@@ -689,6 +704,13 @@ async def _reserve_slot_for_preference(
same shape one corpus over, where snippets lose to project records that
merely resemble the query.
The act arms reserve the slot for the same reason one step later. A
preference about how to act ("let each action land before the next") is
ranked against every rule whose trigger names the same command, and the
act is its last chance: once the command has run there is nothing left
for it to shape. The slot's query reuses the arm's query, so it costs one
kind-filtered search and no extra embedding.
THE SLOT BUYS POSITION, NOT A LOWER BAR — same as `reuse_slot`, which also
reserves at `cfg["threshold"]`. A weak preference cannot buy the slot, so
silence stays the default and the reserved line is never worse than the
+74 -12
View File
@@ -46,13 +46,18 @@ from grimoire.models.base import iso
from grimoire.models.note_usage import PULLED, NoteUsageEvent
from grimoire.models.retrieval_judgment import VERDICTS, RetrievalJudgment
from grimoire.models.retrieval_log import RetrievalLog
from grimoire.models.rule_usage import PULLED as RULE_PULLED
from grimoire.models.rule_usage import RuleUsageEvent
logger = logging.getLogger(__name__)
# The arms a review can re-run FAITHFULLY. A re-run with different parameters
# from the arm's would judge a menu nobody was shown, so a surface joins this
# only with its own search spelled out in `_rerun`.
REVIEWABLE = ("auto_inject",)
# only with its own search spelled out in `_rerun`. The rule arms named here
# are the ones that annotate: each logs its query, floor, budget and project,
# which is everything its search takes.
RULE_REVIEWABLE = ("prompt_rule", "pre_tool_rule", "write_path_rule")
REVIEWABLE = ("auto_inject", *RULE_REVIEWABLE)
DEPTH_DEFAULT = 6
DEPTH_MAX = 10
@@ -94,7 +99,58 @@ def _after(stamp, call) -> bool:
async def _rerun(
user_id: int, row: RetrievalLog, depth: int, report: dict | None = None,
) -> list[dict]:
"""The logged call's query, searched again the way its arm searches.
"""The logged call's query, searched again the way its arm searches."""
if row.source in RULE_REVIEWABLE:
return await _rerun_rules(user_id, row, depth, report)
return await _rerun_notes(user_id, row, depth, report)
async def _rerun_rules(
user_id: int, row: RetrievalLog, depth: int, report: dict | None = None,
) -> list[dict]:
"""A rule arm's call, searched again: its query against rule triggers,
with the floor and project it logged.
`within_budget` is the arm's own cut: a line inside the budget, and, for
an arm that bands, inside the band of the top hit. The preference slot is
left out — it logs its own source. Rules created after the call are
dropped and named in `report["postdated"]`, as for notes.
"""
from grimoire.services import retrieval_pipeline as rp
from grimoire.services.embeddings import semantic_search_rules
arm = next(a for a in rp.RULE_ARMS if a.source == row.source)
hits = await semantic_search_rules(
user_id, row.query or "",
limit=depth + POSTDATED_SLACK,
threshold=row.threshold if row.threshold is not None else 0.0,
project_id=row.project_id or None,
)
postdated = [int(r.id) for _s, r in hits if _after(r.created_at, row.created_at)]
if report is not None:
report["postdated"] = postdated
hits = [h for h in hits if int(h[1].id) not in postdated][:depth]
budget = int(row.limit_n or 0)
cut = hits[:budget]
kept = {int(r.id) for _s, r in (rp._rule_band(cut) if arm.band else cut)}
return [{
"rank": rank,
"record_id": int(rule.id),
"score": round(float(score), 4),
"within_budget": int(rule.id) in kept,
"changed_since_call": _after(rule.updated_at, row.created_at),
"kind": rule.kind or "rule",
"name": rule.title,
# The trigger is what the arm matched against and what its line
# quotes, so it is the passage the reader saw.
"passage": rule.when_to_apply or rule.statement or None,
} for rank, (score, rule) in enumerate(hits, start=1)]
async def _rerun_notes(
user_id: int, row: RetrievalLog, depth: int, report: dict | None = None,
) -> list[dict]:
"""A notes arm's call, searched again.
`auto_inject`'s parameters, read from its spec (`retrieval_pipeline.
AUTO_INJECT`) rather than restated: which kinds, whose records, and the
@@ -151,19 +207,25 @@ async def _opened_after(user_id: int, row: RetrievalLog, ids: list[int]) -> set[
Agent pulls only — the `mcp_` prefix, per NoteUsageEvent's convention —
because the question is whether the line moved the session, not whether
the operator later browsed to the record.
the operator later browsed to the record. A rule arm's lines are read
from the rule table: a note and a rule can share an id.
"""
if not ids or row.created_at is None:
return set()
table, id_col, pulled = (
(RuleUsageEvent, RuleUsageEvent.rule_id, RULE_PULLED)
if row.source in RULE_REVIEWABLE
else (NoteUsageEvent, NoteUsageEvent.note_id, PULLED)
)
async with async_session() as session:
found = await session.execute(
select(NoteUsageEvent.note_id).distinct().where(
NoteUsageEvent.user_id == user_id,
NoteUsageEvent.event == PULLED,
NoteUsageEvent.source.startswith("mcp_", autoescape=True),
NoteUsageEvent.note_id.in_(ids),
NoteUsageEvent.created_at >= row.created_at,
NoteUsageEvent.created_at <= row.created_at + OPENED_WINDOW,
select(id_col).distinct().where(
table.user_id == user_id,
table.event == pulled,
table.source.startswith("mcp_", autoescape=True),
id_col.in_(ids),
table.created_at >= row.created_at,
table.created_at <= row.created_at + OPENED_WINDOW,
)
)
return {int(i) for (i,) in found.all()}
@@ -175,7 +237,7 @@ async def _logged_after(row: RetrievalLog, ids: list[int]) -> set[int]:
(a closed task takes no claim stamp), so it is read here."""
from grimoire.models.task_log import TaskLog
if not ids or row.created_at is None:
if not ids or row.created_at is None or row.source in RULE_REVIEWABLE:
return set()
async with async_session() as session:
found = await session.execute(
@@ -110,6 +110,9 @@ def dial_for_key(key: str) -> tuple[str, str] | None:
Derived rather than listed so a seventh surface is covered the moment it is
added here, which is the only way this stays true.
A floor key several surfaces share resolves to the first of them;
`set_dial` records the move on every surface in `sharing_floor`.
"""
for surface in SURFACES.values():
if key == surface.floor_key:
@@ -119,6 +122,16 @@ def dial_for_key(key: str) -> tuple[str, str] | None:
return None
def sharing_floor(name: str) -> list[str]:
"""Every surface that reads the same floor key as `name`, itself included.
The rule arms share one bar, so moving one surface's floor moves its
siblings, and each of them needs the event that says so.
"""
key = get_surface(name).floor_key
return [n for n, s in SURFACES.items() if s.floor_key == key]
async def floor_for(user_id: int, name: str) -> float:
"""This install's current floor for a surface, clamped to [0, 1]."""
s = get_surface(name)
+128 -24
View File
@@ -41,6 +41,7 @@ from grimoire.models.system_usage import SURFACED as SYSTEM_SURFACED
from grimoire.models.system_usage import SystemUsageEvent
from grimoire.services.rule_usage import is_ambient
from grimoire.models.retrieval_log import RetrievalLog
from grimoire.models.rulebook import Rule
from grimoire.services.retrieval_review import judged_block
from grimoire.services.retrieval_registry import (
POINTS, UNBIDDEN, get_point, is_registered, sources_expected_to_emit,
@@ -489,9 +490,10 @@ def _num(raw: str, fallback):
# to look next. ONE SENTENCE PER CORPUS because the two lines are built
# differently: a note line carries its matched passage, so an unopened
# note may have done its job, and the judged sample is the instrument that can
# tell. A rule line carries only its trigger and asks to be opened, and no
# judged sample covers the rule arms (`retrieval_review.REVIEWABLE`), so
# sending that reader to `menus_to_review` would send them to a refusal.
# tell. A rule line carries only its trigger and asks to be opened, so a rule
# set aside again and again is usually a trigger to fix; the judged sample
# (`retrieval_review.REVIEWABLE` names the rule arms it covers) says whether
# it was set aside rightly.
_NEVER_PULLED_READING = {
"notes": (
"That is not a verdict on them: a line carries its matched passage, "
@@ -504,11 +506,13 @@ _NEVER_PULLED_READING = {
"Not a verdict either, and read differently from notes: the count "
"includes rules that only arrived in a listing (the project "
"handshake, a planning read), and a rule line shows its trigger, so a "
"session can rightly set one aside on the trigger alone. There is no "
"judged sample for the rule arms. A rule set aside again and again "
"where it does not apply is a trigger to fix, not a floor to move — "
"`update_rule(when_to_apply=...)`, then `what_might_apply` with the "
"moment's own words to see where it ranks."
"session can rightly set one aside on the trigger alone. To tell a "
"rule set aside rightly from one missed, judge a sample of a rule "
"arm with `menus_to_review(source=\"prompt_rule\")` (or "
"`pre_tool_rule`, `write_path_rule`). A rule set aside again and "
"again where it does not apply is a trigger to fix, not a floor to "
"move — `update_rule(when_to_apply=...)`, then `what_might_apply` "
"with the moment's own words to see where it ranks."
),
}
@@ -803,6 +807,9 @@ def _silent_surfaces(sources: dict, usage: dict, rule_usage: dict,
return []
seen = set(sources) | set((usage.get("by_source") or {}))
seen |= set((rule_usage.get("by_source") or {}))
# A getter shows nothing, so it is seen only by its pulls.
seen |= set((usage.get("pulled_by_source") or {}))
seen |= set((rule_usage.get("pulled_by_source") or {}))
seen |= set(((system_usage or {}).get("by_source") or {}))
return [
{"source": s, "kind": POINTS[s].kind, "what": POINTS[s].what}
@@ -933,7 +940,9 @@ async def moment_usage(user_id: int | None, since) -> dict:
- `rules`: how many distinct rules did;
- `opened`: how many of those an AGENT then opened (`mcp_*`), at or after
their first delivery there in the window;
- `last_delivered_at`.
- `last_delivered_at`;
- `by_rule`: each rule delivered there, most delivered first, with its
`title`, `delivered` and whether it was `opened`.
`opened` is an UPPER BOUND, for by_source's reason: a pull records the
door, not the line that prompted it, so a rule delivered at two moments
@@ -980,15 +989,21 @@ async def moment_usage(user_id: int | None, since) -> dict:
pull.source.startswith("mcp_", autoescape=True),
pull.created_at >= per_rule.c.first_at,
)
# One row per (moment, rule); the moment's totals are summed from
# these below, so `by_rule` and the totals cannot disagree.
rows = (
await session.execute(
select(
per_rule.c.moment,
func.sum(per_rule.c.n).label("delivered"),
func.count().label("rules"),
func.count(case((opened, 1))).label("opened"),
func.max(per_rule.c.last_at).label("last_at"),
).group_by(per_rule.c.moment)
per_rule.c.rule_id,
Rule.title,
per_rule.c.n,
opened.label("opened"),
per_rule.c.last_at,
)
.select_from(per_rule)
.outerjoin(Rule, Rule.id == per_rule.c.rule_id)
.order_by(per_rule.c.moment, per_rule.c.n.desc(), per_rule.c.rule_id)
)
).all()
complete = await _complete_from(session, RuleUsageEvent, user_id)
@@ -997,15 +1012,26 @@ async def moment_usage(user_id: int | None, since) -> dict:
block["moment_usage_failed"] = True
return block
block["by_moment"] = {
moment: {
"delivered": int(delivered or 0),
"rules": int(rules or 0),
"opened": int(n_opened or 0),
"last_delivered_at": iso(last_at),
}
for moment, delivered, rules, n_opened, last_at in rows
}
by_moment: dict[str, dict] = {}
for moment, rule_id, title, n, was_opened, last_at in rows:
slot = by_moment.setdefault(moment, {
"delivered": 0, "rules": 0, "opened": 0,
"last_delivered_at": None, "by_rule": [],
})
slot["delivered"] += int(n or 0)
slot["rules"] += 1
slot["opened"] += 1 if was_opened else 0
stamp = iso(last_at)
if stamp and (slot["last_delivered_at"] or "") < stamp:
slot["last_delivered_at"] = stamp
# Which rule a shared moment delivered, and whether that rule was
# opened: the per-moment counts cannot say once two rules are
# mounted there. `title` is null for a rule deleted since.
slot["by_rule"].append({
"rule_id": int(rule_id), "title": title,
"delivered": int(n or 0), "opened": bool(was_opened),
})
block["by_moment"] = by_moment
block.update(_coverage(complete.get("*"), since))
return block
@@ -1109,7 +1135,7 @@ async def retrieval_summary(
# Assigned inside the try below; named here so the readout can tell
# "this query failed" from "this window has no rows".
by_source_rows = None
by_source_rows = rule_by_source_rows = None
rule_rows = None
distinct_rules_surfaced = distinct_rules_pulled = 0
# None means the coverage read did not happen — distinct from a table with
@@ -1426,6 +1452,53 @@ async def retrieval_summary(
rule_rows = None
distinct_rules_surfaced = distinct_rules_pulled = 0
distinct_rules_acted = 0
# The note block's per-source join, at the rule grain. The
# top-level rule ratio blends every arm — the prompt arm, the tool
# arm, a lesson pointing at its rule — and "which of them earns its
# lines" is the question each arm's bar is tuned against. Its own
# guard, for the reason the note join has one.
try:
rule_pulled_ids = (
select(RuleUsageEvent.rule_id)
.where(
RuleUsageEvent.created_at >= since,
RuleUsageEvent.user_id == user_id,
RuleUsageEvent.event == RULE_PULLED,
RuleUsageEvent.source.startswith("mcp_", autoescape=True),
)
.distinct()
.subquery()
)
rule_surfaced_pairs = (
select(RuleUsageEvent.source, RuleUsageEvent.rule_id)
.where(
RuleUsageEvent.created_at >= since,
RuleUsageEvent.user_id == user_id,
RuleUsageEvent.event == RULE_SURFACED,
)
.distinct()
.subquery()
)
rule_by_source_rows = (
await session.execute(
select(
rule_surfaced_pairs.c.source,
func.count().label("rules_surfaced"),
func.count(rule_pulled_ids.c.rule_id).label("rules_pulled"),
)
.select_from(
rule_surfaced_pairs.outerjoin(
rule_pulled_ids,
rule_pulled_ids.c.rule_id == rule_surfaced_pairs.c.rule_id,
)
)
.group_by(rule_surfaced_pairs.c.source)
)
).all()
except Exception:
logger.warning("per-source rule pull-through read failed", exc_info=True)
rule_by_source_rows = None
except Exception:
logger.warning("retrieval summary read failed", exc_info=True)
out["read_failed"] = True
@@ -1439,6 +1512,9 @@ async def retrieval_summary(
"distinct_notes_surfaced": int(distinct_surfaced or 0),
"distinct_notes_pulled": int(distinct_pulled or 0),
}
# Pulls per door. `by_source` is keyed by the surface that showed a record
# and leaves pulls out, so this is what says each getter is recording.
pulled_by_source: dict[str, int] = {}
for event, source, n in urows:
n = int(n)
if event == SURFACED:
@@ -1448,6 +1524,7 @@ async def retrieval_summary(
usage["surfaced"] += n
elif event == PULLED:
usage["pulled"] += n
pulled_by_source[source] = pulled_by_source.get(source, 0) + n
# The mcp_/rest_ split is load-bearing (see NoteUsageEvent's own
# comment). "Is
# this record dead weight?" is answered by ANY pull; "was that
@@ -1466,6 +1543,7 @@ async def retrieval_summary(
round(usage["pulled_by_agent"] / usage["surfaced"], 4)
if usage["surfaced"] else None
)
usage["pulled_by_source"] = pulled_by_source
# The same question, per surface — which is the one the top-level ratio
# cannot answer. A corpus average of 0.05 is compatible with one surface
@@ -1538,6 +1616,8 @@ async def retrieval_summary(
"applied": 0,
"departed": 0,
"distinct_rules_acted": int(distinct_rules_acted or 0),
# Pulls per door, as on the note block.
"pulled_by_source": {},
}
if rule_rows is None:
# The FLAG is added, the shape is kept — matching `by_source_failed`
@@ -1560,6 +1640,8 @@ async def retrieval_summary(
rule_usage["surfaced"] += n
elif event == RULE_PULLED:
rule_usage["pulled"] += n
pulled = rule_usage["pulled_by_source"]
pulled[source] = pulled.get(source, 0) + n
# Same split, and it carries MORE weight here than for notes.
# The arm's whole claim is "this rule may apply to what you are
# writing", and only an agent opening it says the claim landed.
@@ -1596,6 +1678,28 @@ async def retrieval_summary(
round(rule_usage["pulled_by_agent"] / rule_usage["surfaced"], 4)
if rule_usage["surfaced"] else None
)
# Per surface, the note block's shape and its caveat: a pull records the
# door, not the surface that led to it, so a rule two arms surfaced and an
# agent opened once counts as pulled for both. An upper bound — a surface
# reading near zero here is not being flattered.
if rule_by_source_rows is None:
rule_usage["by_source"] = {}
rule_usage["by_source_failed"] = True
else:
rule_usage["by_source"] = {
source: {
"rules_surfaced": int(n_surfaced or 0),
"rules_pulled": int(n_pulled or 0),
# None on an ambient surface, as for notes: nothing chose
# those rules, so the ratio would be misread.
"pull_through": (
None if is_ambient(source) or not n_surfaced
else round(int(n_pulled or 0) / int(n_surfaced), 4)
),
"ambient": is_ambient(source),
}
for source, n_surfaced, n_pulled in rule_by_source_rows
}
rule_usage.update(_coverage((rule_complete or {}).get("*"), since))
out["rule_usage"] = rule_usage
out["system_usage"] = await _system_usage(user_id, since)
+18 -6
View File
@@ -44,6 +44,7 @@ from grimoire.services.retrieval_surfaces import (
budget_for,
floor_for,
get_surface,
sharing_floor,
surface_names,
)
from grimoire.services.settings import set_setting
@@ -257,6 +258,13 @@ async def set_dial(
"reason": text, "actor": actor, "unchanged": True,
}
# A shared floor moves every surface that reads it, and each one's history
# has to show the move, or its telemetry changes with no event to explain it.
# The surface the caller named is recorded first, its siblings after it.
moved = [surface] + (
[n for n in sharing_floor(surface) if n != surface] if dial == "floor" else []
)
await set_setting(user_id, key, stored)
async with async_session() as session:
# Stamped with the space this number was chosen in. Read at
@@ -264,12 +272,13 @@ async def set_dial(
# the caller recording it are the same call, so there is no window in
# which they could disagree.
stamp = calibration_stamp()
session.add(RetrievalTuningEvent(
user_id=user_id, surface=surface, dial=dial,
old_value=old, new_value=applied, actor=actor, reason=text,
embedding_model=stamp["embedding_model"],
shape_version=stamp["shape_version"],
))
for name in moved:
session.add(RetrievalTuningEvent(
user_id=user_id, surface=name, dial=dial,
old_value=old, new_value=applied, actor=actor, reason=text,
embedding_model=stamp["embedding_model"],
shape_version=stamp["shape_version"],
))
await session.commit()
return {
@@ -286,6 +295,9 @@ async def set_dial(
# Always present, both ways round: a caller that has to test for the
# key's absence to learn the answer will eventually forget to.
"unchanged": False,
# The other surfaces this move carried with it: a shared floor is one
# number, and a caller tuning one rule arm has tuned them all.
"moved_with": [n for n in moved if n != surface],
}
+80 -17
View File
@@ -43,7 +43,7 @@ from __future__ import annotations
import logging
from datetime import datetime, timezone
from sqlalchemy import exists, func, not_, select
from sqlalchemy import exists, func, not_, or_, select
from sqlalchemy.exc import IntegrityError
from grimoire.models import async_session
@@ -274,6 +274,11 @@ async def pending(user_id: int, rule_id: int | None = None, limit: int = 200) ->
Each rule carries what it is mounted on now, so a proposal is read beside
the mounts it would add to.
A signal pair is a proposal only once it has crossed the bar and asked
(`proposed_at` in its evidence). Below the bar it is a row gathering
evidence, and listing it would bury the few pairs the signal stands
behind under every rule that was once opened after a moment.
"""
from grimoire.services.rulebooks import _owned_rules_clause
@@ -282,7 +287,13 @@ async def pending(user_id: int, rule_id: int | None = None, limit: int = 200) ->
stmt = (
select(RuleMomentJudgment, Rule)
.join(Rule, Rule.id == RuleMomentJudgment.rule_id)
.where(RuleMomentJudgment.state == SUGGESTED, _owned_rules_clause(user_id))
.where(
RuleMomentJudgment.state == SUGGESTED, _owned_rules_clause(user_id),
or_(
RuleMomentJudgment.source != "signal",
RuleMomentJudgment.evidence["proposed_at"].astext.isnot(None),
),
)
)
if rule_id:
stmt = stmt.where(Rule.id == int(rule_id))
@@ -483,17 +494,50 @@ def _keep(row: RuleMomentJudgment, note: str, now: datetime) -> None:
# ── The signal ───────────────────────────────────────────────────────────────
# How many distinct actions a signal pair remembers as having reached its
# moment. Enough to show the operator what the opens followed.
_SIGNAL_ACTIONS_CAP = 5
def _reached_phrase(evidence: dict) -> str:
reached = (evidence or {}).get("reached_by") or {}
if not isinstance(reached, dict) or not reached:
return ""
top = sorted(reached.items(), key=lambda kv: (-int(kv[1] or 0), kv[0]))[:3]
return " (reached by " + ", ".join(f"`{match}`" for match, _n in top) + ")"
def _signal_note(moment: str, evidence: dict) -> str:
"""The `why` a signal proposal shows the operator: what the opens followed."""
counts = lesson_rules.evidence_summary(evidence)
return (
f"Opened just after `{moment}`{_reached_phrase(evidence)} in "
f"{counts['situations']} distinct sessions, each time without a hook having "
f"shown the rule first. Co-occurrence only: confirm it if the rule governs "
f"that point in the work whatever the work is about."
)
def _proposal_line(rule: Rule, moment: str, evidence: dict) -> str:
"""The line an open past the bar returns to the session.
It says what the evidence is, and makes offering it conditional on the
reader's own reading: a session relays an unqualified "offer this" as a
recommendation, and co-occurrence is not one.
"""
counts = lesson_rules.evidence_summary(evidence)
kind = rule.kind or "rule"
return (
f"> {kind.capitalize()} #{rule.id} \"{rule.title}\" has been opened just after "
f"`{moment}` in {counts['situations']} distinct sessions — it may belong on "
f"that moment, where it would arrive without being searched for. Offer the "
f"operator the mount in one line; on their yes, `judge_rule_moments("
f"[{{\"rule_id\": {rule.id}, \"moment\": \"{moment}\", \"verdict\": \"confirm\", "
f"\"note\": \"why\"}}])` mounts it; on a no, `\"reject\"` with their why stops "
f"the asking."
f"> {kind.capitalize()} #{rule.id} \"{rule.title}\" was opened just after "
f"`{moment}`{_reached_phrase(evidence)} in {counts['situations']} distinct "
f"sessions, each time without a hook having shown it first. That is "
f"co-occurrence, not a recommendation. Read the {kind}'s trigger against what "
f"`{moment}` means (`list_moments`): only if the {kind} governs that point in "
f"the work whatever the work is about, ask the operator whether to mount it, "
f"with your reason; if it is about a subject, leave it. On their yes, "
f"`judge_rule_moments([{{\"rule_id\": {rule.id}, \"moment\": \"{moment}\", "
f"\"verdict\": \"confirm\", \"note\": \"why\"}}])` mounts it; on a no, "
f"`\"reject\"` with their why stops the asking."
)
@@ -510,24 +554,39 @@ def signal_moments(moments) -> list[str]:
async def co_occurred(
user_id: int, rule_id: int, moments, *, situation: str,
project_id: int | None = None,
project_id: int | None = None, reached: dict | None = None,
) -> str:
"""Record that a rule was opened shortly after these moments fired, and
return a proposal line once a (rule, moment) pair has earned one.
`situation` is what makes two opens distinct — the session, so a rule
opened after every push in one long session counts once. A pair already
mounted or judged gathers nothing. One proposal per open at most. Fails
open: this rides a hook, and bookkeeping must never break the read.
judged gathers nothing, and neither does a rule already mounted anywhere:
it arrives by lookup at its own moments, so an open of it is far likelier
to follow that delivery than to reveal a missing moment. `reached` maps a
moment to the action that reached it, kept as the proposal's evidence.
One proposal per open at most. Fails open: this rides a hook, and
bookkeeping must never break the read.
"""
try:
return await _co_occurred(user_id, rule_id, moments, situation, project_id)
return await _co_occurred(user_id, rule_id, moments, situation, project_id,
reached or {})
except Exception:
logger.warning("rule/moment co-occurrence could not be recorded", exc_info=True)
return ""
async def _co_occurred(user_id, rule_id, moments, situation, project_id) -> str:
def _add_reached(evidence: dict, match: str) -> dict:
"""`evidence` with one more open counted against the action that reached
the moment, keeping at most _SIGNAL_ACTIONS_CAP distinct actions."""
reached = dict(evidence.get("reached_by") or {})
match = (match or "").strip()[:80]
if match and (match in reached or len(reached) < _SIGNAL_ACTIONS_CAP):
reached[match] = int(reached.get(match) or 0) + 1
return {**evidence, "reached_by": reached}
async def _co_occurred(user_id, rule_id, moments, situation, project_id, reached) -> str:
wanted = signal_moments(moments)
key = lesson_rules.situation_key("s", situation)
rid = _int(rule_id)
@@ -540,11 +599,10 @@ async def _co_occurred(user_id, rule_id, moments, situation, project_id) -> str:
rule = owned.get(rid)
if rule is None:
return ""
mounted = (await _mounts(session, [rid])).get(rid, set())
if (await _mounts(session, [rid])).get(rid):
return ""
existing = await _rows(session, [rid])
for moment in wanted:
if moment in mounted:
continue
row = existing.get((rid, moment))
if row is not None and row.state != SUGGESTED:
continue
@@ -552,11 +610,13 @@ async def _co_occurred(user_id, rule_id, moments, situation, project_id) -> str:
row = RuleMomentJudgment(rule_id=rid, moment=moment, state=SUGGESTED, source="signal")
session.add(row)
ev = lesson_rules.add_evidence(row.evidence, key, project_id, now)
ev = _add_reached(ev, reached.get(moment, ""))
if proposal is None and lesson_rules.proposal_due(ev, now):
ev["proposed_at"] = now.isoformat()
ev["proposed_count"] = int(ev.get("proposed_count") or 0) + 1
proposal = (moment, ev)
row.evidence = ev
row.note = _signal_note(moment, ev)
try:
await session.commit()
except IntegrityError:
@@ -588,6 +648,7 @@ async def opened_after(
from grimoire.services.moment_actions import moments_for
reached: list[str] = []
reached_by: dict[str, str] = {}
for act in list(acts or [])[:_ACTS_CAP]:
if not isinstance(act, dict):
continue
@@ -604,11 +665,13 @@ async def opened_after(
name = hit.get("moment") if isinstance(hit, dict) else None
if name and name not in reached:
reached.append(name)
reached_by[name] = str(hit.get("match") or tool)
moments = signal_moments(reached)
context = ""
if moments and (session_id or "").strip():
context = await co_occurred(
user_id, rule_id, moments, situation=session_id.strip(), project_id=project_id,
reached=reached_by,
)
return {"moments": moments, "context": context}
+12 -2
View File
@@ -59,8 +59,9 @@ as short and as slow-moving as possible.
from __future__ import annotations
import logging
from datetime import datetime
from sqlalchemy import case, func, select
from sqlalchemy import case, func, select, true
from grimoire.models import async_session
from grimoire.models.base import iso
@@ -333,13 +334,18 @@ def empty_rule_usage() -> dict:
}
async def usage_for_rules(rule_ids: list[int]) -> dict[int, dict]:
async def usage_for_rules(
rule_ids: list[int], since: datetime | None = None,
) -> dict[int, dict]:
"""Aggregate usage for a set of rules: {rule_id: {counts + timestamps}}.
One GROUP BY for the whole page rather than a query per row — this feeds a
list view, so the per-row shape would be N+1 by construction. Rules with no
events come back with `empty_rule_usage()`, so the caller never has to tell
"no events" from "not in the result".
`since` limits the counts to events at or after it, so two windows can be
compared; without it they cover everything recorded.
"""
ids = [int(r) for r in rule_ids]
out: dict[int, dict] = {rid: empty_rule_usage() for rid in ids}
@@ -370,6 +376,10 @@ async def usage_for_rules(rule_ids: list[int]) -> dict[int, dict]:
ambient,
)
.where(RuleUsageEvent.rule_id.in_(ids))
.where(
RuleUsageEvent.created_at >= since
if since is not None else true()
)
.group_by(
RuleUsageEvent.rule_id,
RuleUsageEvent.event,
+288 -31
View File
@@ -23,9 +23,10 @@ from __future__ import annotations
import difflib
import logging
import posixpath
import re
from datetime import datetime, timedelta, timezone
from typing import Iterable, NamedTuple
from typing import Iterable, Mapping, NamedTuple
from sqlalchemy import select
@@ -170,6 +171,80 @@ async def sync_repo_shapes(
await session.commit()
async def move_shapes(
user_id: int, project_id: int, from_path: str, to_path: str,
) -> dict:
"""Carry the ledger rows under ``from_path`` (a file or a directory) to
the same place under ``to_path``, in every repo of the project.
Rows are keyed by path, so a directory moved in the tree would otherwise
vanish every judged row at the next sync and start the same shapes again,
unclassified, under the new path. A moved row is the same row — its id,
judgment, uses edges and history ride along — and a `moved` event
records the path it left. A row already at the target (the sync ran
first, or a provisional judgment landed there) is replaced when nobody
judged it; a judged one stands, and the old row is left where it is and
reported in ``kept``. Bodies are not compared here: the next sync does
that as always, and asks for a recheck where a judged body changed.
"""
from sqlalchemy import or_
from grimoire.services import access
from grimoire.services.snippets import _path_touches
old = (from_path or "").strip().strip("/")
new = (to_path or "").strip().strip("/")
if not old or not new:
raise ValueError("from_path and to_path are both required")
if _path_touches(old, new) or _path_touches(new, old):
raise ValueError(f"`{old}` and `{new}` overlap; move to a path outside the source")
if not await access.can_write_project(user_id, project_id):
raise ValueError(f"project {project_id} not found or no write access")
now = datetime.now(timezone.utc)
async with async_session() as session:
def beneath(prefix: str):
return select(CodeShape).where(
CodeShape.project_id == project_id,
or_(CodeShape.path == prefix, CodeShape.path.like(prefix + "/%")),
)
# LIKE reads `_` as a wildcard; the prefix test is the real filter.
sources = [
r for r in (await session.execute(beneath(old))).scalars().all()
if _path_touches(old, r.path)
]
targets = {
(r.repo_key, r.path, r.symbol, r.kind): r
for r in (await session.execute(beneath(new))).scalars().all()
}
moved, replaced, kept = 0, 0, []
for row in sources:
dest = new + row.path[len(old):]
there = targets.get((row.repo_key, dest, row.symbol, row.kind))
if there is not None:
if there.status not in _MECHANICAL_TODO:
kept.append({"path": row.path, "symbol": row.symbol, "kind": row.kind})
continue
# The target is the live sighting when the sync got there
# first; the moved row takes over its presence.
row.vanished_at = there.vanished_at
row.last_seen_commit = there.last_seen_commit or row.last_seen_commit
await session.delete(there)
await session.flush()
replaced += 1
session.add(CodeShapeEvent(
shape_id=row.id, project_id=project_id,
path=row.path, symbol=row.symbol, kind=row.kind,
event="moved", status=row.status, snippet_id=row.snippet_id,
classified_by=row.classified_by, reason=row.reason,
commit=row.last_seen_commit or "", at=now,
))
row.path = dest
moved += 1
await session.commit()
return {"moved": moved, "replaced": replaced, "kept": kept}
def _event(row: CodeShape, event: str, at: datetime, *, commit: str = "") -> CodeShapeEvent:
"""A history row for a state change on ``row`` — status/snippet/by/reason
are the row's CURRENT values, which for `classified` is the judgment
@@ -183,6 +258,46 @@ def _event(row: CodeShape, event: str, at: datetime, *, commit: str = "") -> Cod
)
async def provisional_row(
session, *, project_id: int, repo_key: str, path: str, symbol: str, kind: str,
) -> CodeShape:
"""The row for a shape a caller met before any live row for it existed:
a shape being written this turn, or one judged ahead of the sync.
The identity (project, repo, path, symbol, kind) is unique, so "no live
row" does not mean "no row". A VANISHED row under the identity is revived
and keeps its standing judgment, as a sync revives a shape that comes
back. The next sync then confirms it, or stamps it vanished again until
the code reaches the bound ref. With no row at all, a provisional one is
inserted inside a savepoint. If a concurrent request inserts the same
identity first, the savepoint rolls back and that request's row is used.
The caller's transaction survives either way.
"""
from sqlalchemy.exc import IntegrityError
def _same():
return select(CodeShape).where(
CodeShape.project_id == project_id, CodeShape.repo_key == repo_key,
CodeShape.path == path, CodeShape.symbol == symbol, CodeShape.kind == kind,
)
row = (await session.execute(_same())).scalars().first()
if row is None:
try:
async with session.begin_nested():
row = CodeShape(project_id=project_id, repo_key=repo_key,
path=path, symbol=symbol, kind=kind)
session.add(row)
await session.flush()
return row
except IntegrityError:
row = (await session.execute(_same())).scalars().one()
if row.vanished_at is not None:
row.vanished_at = None
session.add(_event(row, "reappeared", datetime.now(timezone.utc)))
return row
async def _judge(
session, row: CodeShape, *, status: str, snippet_id: int | None,
by: str | None, reason: str | None, at: datetime, reason_code: str | None = None,
@@ -246,6 +361,31 @@ async def record_uses(
return n
async def replace_reference_uses(
session, row: CodeShape, snippet_ids, *, evidence: str | None = None,
) -> int:
"""Make the row's mechanical `reference` edges exactly ``snippet_ids``.
`record_uses` only adds, so a by-name rule that tightens would otherwise
leave every edge the looser rule wrote. Only `reference` edges are
dropped: a judgment, or the write-path hook's evidence, outlives a
re-read of the body. Returns the number of edges written or refreshed."""
wanted = {int(x) for x in (snippet_ids or []) if x}
if row.id is not None:
stale = (
await session.execute(
select(CodeShapeUse).where(
CodeShapeUse.shape_id == row.id,
CodeShapeUse.basis == "reference",
)
)
).scalars().all()
for edge in stale:
if edge.snippet_id not in wanted:
await session.delete(edge)
return await record_uses(session, row, wanted, basis="reference", evidence=evidence)
async def uses_of(shape_ids) -> dict[int, list[CodeShapeUse]]:
"""{shape_id: [edges]} for a set of rows — the read side of record_uses."""
ids = [int(x) for x in shape_ids if x]
@@ -626,11 +766,10 @@ async def classify_shapes(
if not matches and repo_key and item.get("status") != "unclassified":
path = (item.get("path") or "").strip()
symbol = (item.get("symbol") or "").strip()
row = CodeShape(
project_id=project_id, repo_key=repo_key, path=path,
row = await provisional_row(
session, project_id=project_id, repo_key=repo_key, path=path,
symbol=symbol, kind=kind or snippet_kind(symbol, ""),
)
session.add(row)
by_key.setdefault((path, symbol), []).append(row)
matches = [row]
provisional += 1
@@ -1479,8 +1618,19 @@ async def suggest_write_path_instances(
continue
fields = snippet_fields(note)
symbol = fields.get("symbol") or ""
kind = snippet_kind(symbol, fields.get("language") or "")
if references_symbol(code, symbol, kind):
language = (fields.get("language") or "").strip().lower()
kind = snippet_kind(symbol, language)
canon = Canon(
int(sid), kind, symbol,
tuple(((loc.get("path") or ""), (loc.get("symbol") or ""))
for loc in fields.get("locations") or []),
"", "", int(note.project_id or 0), language,
)
# A redefinition of the pulled name in this payload is the copy the
# hint exists to catch, so the payload's own names are not "local".
if (kind != "sym" or same_family(path, language)) and names_canon(
kind, path, code, canon, project_id=project_id,
):
rank, why = 2, _REFERENCE_REASON.format(
sid=sid, symbol=_norm_symbol(symbol)
)
@@ -1544,13 +1694,13 @@ async def suggest_write_path_instances(
if row is None:
if not repo_key:
continue
row = CodeShape(
project_id=project_id, repo_key=repo_key,
# It may be a vanished row coming back, with its judgment.
row = await provisional_row(
session, project_id=project_id, repo_key=repo_key,
path=path, symbol=name, kind=kind,
)
session.add(row)
by_key[(name, kind)] = row
elif not (row.status in _MECHANICAL_TODO or row.classified_by == "hook"):
if not (row.status in _MECHANICAL_TODO or row.classified_by == "hook"):
continue # a judgment — or the canon itself — stands
# A proposal, never a status: the writer decides.
row.proposed_snippet_id = sid
@@ -1688,7 +1838,7 @@ def _semantic_priority(row) -> tuple:
# were examined under, so a tightened rule re-examines everything once. A
# change that does not alter what is proposed needs no bump. A stale "miss"
# basis on an old row is inert: a miss is not evidence (see _SEMANTIC_LIMIT).
_PROPOSER_VERSION = 6
_PROPOSER_VERSION = 9
# Signature resemblance floor, name blanked (difflib ratio) — and a length
# floor, because `def NAME():` resembles `def NAME(x):` at 0.95 while saying
# nothing; a family shape has parameters to resemble.
@@ -1731,6 +1881,13 @@ _FAMILY_BY_LANG = {
"css": "css", "scss": "css", "sass": "css", "less": "css",
"bash": "sh", "sh": "sh", "shell": "sh", "zsh": "sh",
"sql": "sql",
"go": "go", "golang": "go",
"rust": "rs", "rs": "rs",
"java": "jvm", "kotlin": "jvm", "kt": "jvm",
"swift": "swift", "dart": "dart",
"c": "c", "cpp": "c", "c++": "c", "h": "c",
"csharp": "cs", "c#": "cs", "cs": "cs",
"ruby": "rb", "rb": "rb", "php": "php",
}
_FAMILY_BY_EXT = {
".py": "py", ".pyi": "py",
@@ -1738,6 +1895,11 @@ _FAMILY_BY_EXT = {
".css": "css", ".scss": "css", ".sass": "css", ".less": "css",
".sh": "sh", ".bash": "sh", ".zsh": "sh",
".sql": "sql",
".go": "go", ".rs": "rs",
".java": "jvm", ".kt": "jvm", ".kts": "jvm",
".swift": "swift", ".dart": "dart",
".c": "c", ".h": "c", ".cc": "c", ".cpp": "c", ".hpp": "c",
".cs": "cs", ".rb": "rb", ".php": "php",
}
@@ -1803,10 +1965,79 @@ def text_contains(body: str, code: str) -> bool:
return a in b or b in a
def reference_canons(kind: str, path: str, symbol: str, body: str, canons: Iterable[Canon]) -> list[int]:
def _in_canon_home(canon: Canon, path: str) -> bool:
"""Is ``path`` in the canon's own package? A location that names a file
makes its directory home, exactly (a Go package is one directory); a
location that names a directory is home with everything under it."""
from grimoire.services.snippets import _path_touches
here = posixpath.dirname((path or "").strip("/"))
for lp, _ls in canon.locations:
lp = (lp or "").strip("/")
if not lp:
continue
if "." in lp.rsplit("/", 1)[-1]:
if posixpath.dirname(lp) == here:
return True
elif _path_touches(lp, path):
return True
return False
def names_canon(
kind: str, path: str, body: str, canon: Canon, *,
project_id: int = 0, defined_at: Mapping[str, frozenset[str]] | None = None,
) -> bool:
"""Does this body name THIS canon, rather than merely contain its word?
The by-name test behind the reference basis and every uses edge the
proposer writes. ``defined_at`` maps each name to the repo paths that
define it. A mention counts only when it can resolve to the canon:
- never across projects: one repo does not call a function defined in
another's, however alike the names are;
- never a name the file defines itself (a module's own `_run` helper),
unless the canon is recorded in this very file;
- when the repo also defines the name somewhere the canon is not
recorded (`sftpRun.file` beside the canon `importRun.file`), only from
the canon's own package: elsewhere the name says nothing about which
definition a call reaches;
- never a generic verb, with or without a leading underscore;
- a bare lowercase word (`file`, `seen`, `seal`) only where it is called
as `word(`, because as a plain word it is everyone's variable. No space
is allowed before the parenthesis: formatted code never has one, and
prose does ("a file (not a directory)").
Same kind and language family are checked by the callers' loops."""
sym = _norm_symbol(canon.symbol or "")
if not sym:
return False
if project_id and canon.project_id and canon.project_id != project_id:
return False
if sym.lstrip("_").lower() in _REFERENCE_STOPLIST:
return False
where = (defined_at or {}).get(sym, frozenset())
if where:
from grimoire.services.snippets import _path_touches
def at_canon(p: str) -> bool:
return any(lp and _path_touches(lp, p) for lp, _ls in canon.locations)
if path in where and not at_canon(path):
return False
if canon.locations and not all(at_canon(p) for p in where) \
and not _in_canon_home(canon, path):
return False
if kind == "sym" and re.fullmatch(r"[a-z]+", sym):
return re.search(rf"(?<![\w$]){re.escape(sym)}\(", body or "") is not None
return references_symbol(body, sym, kind)
def reference_canons(
kind: str, path: str, symbol: str, body: str, canons: Iterable[Canon], *,
project_id: int = 0, defined_at: Mapping[str, frozenset[str]] | None = None,
) -> list[int]:
"""Every canon this body NAMES — the uses edges the proposer can
write mechanically: same kind, same language family, symbol not in the
generic-verb stoplist, and not the shape's own name."""
write mechanically: same kind, same language family, not the shape's own
name, and named in the sense `names_canon` gives."""
norm_sym = _norm_symbol(symbol)
out: list[int] = []
for c in canons:
@@ -1816,9 +2047,7 @@ def reference_canons(kind: str, path: str, symbol: str, body: str, canons: Itera
continue
if _norm_symbol(c.symbol) == norm_sym:
continue
if _norm_symbol(c.symbol).lower() in _REFERENCE_STOPLIST:
continue
if references_symbol(body, c.symbol, kind):
if names_canon(kind, path, body, c, project_id=project_id, defined_at=defined_at):
out.append(c.snippet_id)
return out
@@ -1826,6 +2055,7 @@ def reference_canons(kind: str, path: str, symbol: str, body: str, canons: Itera
def match_canon(
kind: str, path: str, symbol: str, signature: str, body: str,
canons: Iterable[Canon], *, project_id: int = 0,
defined_at: Mapping[str, frozenset[str]] | None = None,
) -> tuple[int, str, float] | None:
"""The strongest (snippet_id, basis, score) a shape earns against the
canon catalog, by the non-semantic bases — or None. Strongest by
@@ -1850,11 +2080,7 @@ def match_canon(
if not any(location_covers(lp, ls, path, symbol) for lp, ls in c.locations):
offer("symbol", 1.0, c)
continue # its own location is canonical territory, not a proposal
if (
c.symbol
and _norm_symbol(c.symbol).lower() not in _REFERENCE_STOPLIST
and references_symbol(body, c.symbol, kind)
):
if names_canon(kind, path, body, c, project_id=project_id, defined_at=defined_at):
offer("reference", 0.9, c)
if c.code_norm and text_contains(body, c.code_norm):
offer("text", 0.95, c)
@@ -1990,6 +2216,14 @@ async def propose_for_repo(
if canons is None:
canons = await canon_catalog(user_id)
by_key = {(d[0], d[1], d[2]): d for d in definitions}
# Where the repo defines each name: a mention resolves to the definition
# in its own file, and a name defined in several packages only from the
# canon's own (see names_canon).
paths_by_name: dict[str, set[str]] = {}
for d in definitions:
if d[1] == "sym":
paths_by_name.setdefault(_norm_symbol(d[2]), set()).add(d[0])
defined_at = {n: frozenset(p) for n, p in paths_by_name.items()}
# The semantic arm is the widest net and, across projects, is noise: it
# is held to the shape's own project and language family. The precise
# bases (symbol/text) still reach family canon in other projects.
@@ -2033,10 +2267,10 @@ async def propose_for_repo(
# are a fact about the body, judged or not: the consumer map
# of a canon must include the call sites someone already
# classified. Mark it examined so the scan runs once per body.
used = reference_canons(row.kind, row.path, row.symbol, body, canons)
if used:
await record_uses(session, row, used, basis="reference",
evidence="proposer: body names the canon's symbol")
used = reference_canons(row.kind, row.path, row.symbol, body, canons,
project_id=project_id, defined_at=defined_at)
await replace_reference_uses(session, row, used,
evidence="proposer: body names the canon's symbol")
row.proposed_at = now
row.proposed_sha = examined_as
continue
@@ -2044,7 +2278,7 @@ async def propose_for_repo(
group = row.proposal_group # derive grouping is reassigned below
hit = match_canon(
row.kind, row.path, row.symbol, signature, body, canons,
project_id=project_id,
project_id=project_id, defined_at=defined_at,
)
_clear_proposal(row)
row.proposal_group = group
@@ -2052,10 +2286,10 @@ async def propose_for_repo(
row.proposed_sha = examined_as
# Consumption is recorded for every canon the body names,
# whatever the row is then judged to be.
used = reference_canons(row.kind, row.path, row.symbol, body, canons)
if used:
await record_uses(session, row, used, basis="reference",
evidence="proposer: body names the canon's symbol")
used = reference_canons(row.kind, row.path, row.symbol, body, canons,
project_id=project_id, defined_at=defined_at)
await replace_reference_uses(session, row, used,
evidence="proposer: body names the canon's symbol")
if hit:
row.proposed_snippet_id, row.proposal_basis, row.proposal_score = hit
row.proposal_group = None
@@ -2632,6 +2866,29 @@ async def write_time_derive(
return out
async def new_shapes_at(
project_id: int, path: str, shapes: list[tuple[str, str]]
) -> list[tuple[str, str]]:
"""The (kind, name) shapes the hook named at ``path`` that the ledger has
no live row for there — definitions this write introduces, as opposed to
ones it edits inside. Order follows ``shapes``."""
wanted = [(k, n) for k, n in shapes if n]
if not wanted:
return []
async with async_session() as session:
rows = (
await session.execute(
select(CodeShape.kind, CodeShape.symbol).where(
CodeShape.project_id == project_id,
CodeShape.path == path,
CodeShape.vanished_at.is_(None),
)
)
).all()
here = {(k, _norm_symbol(s)) for k, s in rows}
return [(k, n) for k, n in wanted if (k, _norm_symbol(n)) not in here]
async def flag_divergence(project_id: int, *, since: datetime | None) -> int:
"""Flag shapes created after ``since`` (the previous refresh) that sit
where a canon dominates and were not proposed as that canon. With no
+89
View File
@@ -1280,6 +1280,95 @@ def _path_touches(recorded: str, changed: str) -> bool:
return changed == recorded or changed.startswith(recorded + "/")
async def _rewrite_locations(user_id: int, project_id: int, rewrite) -> dict:
"""Apply ``rewrite`` to every recorded location of the project's live
snippets, and save each snippet a rewrite changed.
``rewrite(location)`` answers the new location, or None to leave it as it
is. Each snippet is saved through update_snippet, so its body and indexed
fields stay one record. A snippet the caller can only read is left as it
is and listed in ``read_only``.
"""
async with async_session() as session:
notes = (
await session.execute(
select(Note).where(
Note.project_id == project_id,
Note.note_type == SNIPPET_NOTE_TYPE,
Note.deleted_at.is_(None),
)
)
).scalars().all()
moved: list[int] = []
read_only: list[int] = []
for note in notes:
locations = snippet_fields(note).get("locations") or []
rewritten = [rewrite(loc) for loc in locations]
if all(new is None for new in rewritten):
continue
try:
await update_snippet(
user_id, int(note.id),
locations=[new or loc for new, loc in zip(rewritten, locations)],
)
except PermissionError:
read_only.append(int(note.id))
continue
moved.append(int(note.id))
return {"moved": moved, "read_only": read_only}
async def move_locations(
user_id: int, project_id: int, from_path: str, to_path: str,
) -> dict:
"""Re-path the recorded locations under ``from_path`` (a file or a
directory) to the same place under ``to_path``, on every live snippet of
the project the caller can edit.
The twin of the shape ledger's move: a snippet recorded at a path that
moved reads `missing` at its next check, and its canonical stamp no longer
covers the shape that moved. Returns {"moved": [ids], "read_only": [ids]}.
"""
old = (from_path or "").strip().strip("/")
new = (to_path or "").strip().strip("/")
def repath(loc: dict) -> dict | None:
if not _path_touches(old, loc.get("path") or ""):
return None
return {**loc, "path": new + loc["path"].strip("/")[len(old):]}
return await _rewrite_locations(user_id, project_id, repath)
async def move_location_repo(
user_id: int, project_id: int, old_repo: str, new_repo: str,
) -> dict:
"""Point the project's snippet locations recorded in ``old_repo`` at
``new_repo``, for a repo renamed on its forge.
The twin of the binding move: a location still naming the old repo
resolves to nothing on the forge. Repo names are recorded free-form, so a
location matches when it is the old remote in any form, or a trailing
part of it ("owner/Repo", "Repo"), compared without case. A match is
rewritten to the new normalized key. Returns {"moved": [ids],
"read_only": [ids]}.
"""
from grimoire.services.repo_bindings import normalize_repo_key
old_key = normalize_repo_key(old_repo)
new_key = normalize_repo_key(new_repo)
def rename(loc: dict) -> dict | None:
recorded = normalize_repo_key(loc.get("repo") or "")
if not recorded or recorded == new_key:
return None
if recorded != old_key and not old_key.endswith("/" + recorded):
return None
return {**loc, "repo": new_key}
return await _rewrite_locations(user_id, project_id, rename)
async def invalidate_for_push(
repo_key: str,
changed: list[str],
+338
View File
@@ -0,0 +1,338 @@
"""Which project a write lands in, so a session can be held to its own.
A Claude Code session belongs to one project: the one its starting directory
is bound to. Toward any other project it may read anything and file an issue,
and every other write waits for the operator. The plugin's PreToolUse hook
enforces that, because only the hook knows which session is acting: the MCP
endpoint is stateless and sees nothing but the caller's key. This module
answers the hook's one question: given the session's project and a pending
act, which projects would the act change, and may it go ahead?
An act is either a directory (a file edit or a git command; the hook resolves
it to a repo or a `.grimoire` marker) or an MCP tool call. For a tool call,
`WRITE_TARGETS` says, for every write tool, which of its arguments locate the
write. That means the record or project the write changes, never one it only
mentions: `arose_from_id` on a new task points at another task without
changing it. tests/test_write_scope.py holds the table to the registered tools,
so a new write tool can't be added without saying where it writes.
"""
from __future__ import annotations
from sqlalchemy import or_, select
# How an argument locates the project a write lands in.
PROJECT = "project" # a project id
NOTE = "note" # a task, note, lesson, snippet or process: its project
MILESTONE = "milestone" # a milestone: its project
SYSTEM = "system" # a System: its project
RULE = "rule" # a project rule: its project. A global rule: none
RELATION = "relation" # a rule relation: the projects of its two rules
TRASH = "trash" # a deleted batch: every project its rows belong to
REPO = "repo" # a repo remote: the project it is bound to
DECISION = "decision" # a family decision: its project, else its idea's home
LOCATORS = frozenset({PROJECT, NOTE, MILESTONE, SYSTEM, RULE, RELATION, TRASH, REPO, DECISION})
# Every write tool, and the arguments that locate its write. An empty entry is
# a write that lands in no project:
# - install-level settings (moment mappings, retrieval dials);
# - user-level records (processes, design systems, rulebooks, topics, global
# rules and preferences);
# - judgments the system asks a session for in any project (a rule's outcome,
# a lesson-to-rule link, moment proposals, a menu verdict). Those describe how
# the corpus is retrieved; they don't change another project's work.
WRITE_TARGETS: dict[str, dict[str, str]] = {
# notes, tasks, planning
"create_note": {"project_id": PROJECT},
"update_note": {"note_id": NOTE, "project_id": PROJECT},
"delete_note": {"note_id": NOTE},
"mark_note_verified": {"note_id": NOTE},
"create_lesson": {"project_id": PROJECT},
"update_lesson": {"lesson_id": NOTE},
"delete_lesson": {"lesson_id": NOTE},
"judge_lesson_link": {},
"create_task": {"project_id": PROJECT, "milestone_id": MILESTONE, "parent_id": NOTE},
"update_task": {"task_id": NOTE, "project_id": PROJECT, "milestone_id": MILESTONE},
"delete_task": {"task_id": NOTE},
"add_task_log": {"task_id": NOTE},
"create_records": {"project_id": PROJECT, "milestone_id": MILESTONE},
"start_planning": {"project_id": PROJECT},
"create_milestone": {"project_id": PROJECT},
"update_milestone": {"milestone_id": MILESTONE, "project_id": PROJECT},
"delete_milestone": {"milestone_id": MILESTONE},
# projects, Systems, repos
"create_project": {},
"update_project": {"project_id": PROJECT},
"delete_project": {"project_id": PROJECT},
"decide_project_inception": {"project_id": PROJECT},
"set_project_platforms": {"project_id": PROJECT},
"create_system": {"project_id": PROJECT},
"update_system": {"system_id": SYSTEM},
"delete_system": {"system_id": SYSTEM},
"map_system_to_canonical": {"system_id": SYSTEM},
# Binding a repo changes the project it's bound to and, when it was bound
# elsewhere, the project it leaves.
"bind_repo": {"project_id": PROJECT, "repo_url": REPO},
"unbind_repo": {"repo_url": REPO},
# family canon. An idea is a record in its home project; an adoption is a
# project's answer; a conflict is between the two projects it names.
"propose_family_idea": {"note_id": NOTE},
"promote_family_idea": {"note_id": NOTE},
"retire_family_idea": {"note_id": NOTE},
"revise_family_idea": {"note_id": NOTE},
"set_family_references": {"note_id": NOTE},
"set_family_topic": {"note_id": NOTE},
"undo_family_decision": {"decision_id": DECISION},
"assess_family_adoption": {"project_id": PROJECT},
"resolve_family_conflict": {"canon_project_id": PROJECT, "other_project_id": PROJECT},
# snippets, processes, the shape ledger
"create_snippet": {"project_id": PROJECT},
"update_snippet": {"snippet_id": NOTE, "project_id": PROJECT},
"delete_snippet": {"snippet_id": NOTE},
"verify_snippet": {"snippet_id": NOTE},
"merge_snippets": {"target_id": NOTE, "source_ids": NOTE},
"unmerge_snippet": {"survivor_id": NOTE, "source_id": NOTE},
"create_process": {},
"update_process": {"process_id": NOTE},
"delete_process": {"process_id": NOTE},
"classify_shapes": {"project_id": PROJECT},
"classify_shapes_by_rule": {"project_id": PROJECT},
"confirm_shape_proposals": {"project_id": PROJECT},
"refresh_pattern_coverage": {"project_id": PROJECT},
"move_path": {"project_id": PROJECT},
# design systems belong to their owner, not to a project; choosing one for
# a project is that project's write.
"create_design_system": {},
"update_design_system": {},
"delete_design_system": {},
"create_design_token": {},
"update_design_token": {},
"delete_design_token": {},
"set_project_design_system": {"project_id": PROJECT},
# rules
"create_rulebook": {},
"update_rulebook": {},
"delete_rulebook": {},
"create_topic": {},
"update_topic": {},
"delete_topic": {},
"create_rule": {},
"create_preference": {},
"create_project_rule": {"project_id": PROJECT},
"update_rule": {"rule_id": RULE},
"update_preference": {"rule_id": RULE},
"move_rule": {"rule_id": RULE, "project_id": PROJECT},
"delete_rule": {"rule_id": RULE},
"mark_rule_verified": {"rule_id": RULE},
"relate_rules": {"from_rule_id": RULE, "to_rule_id": RULE},
"unrelate_rules": {"relation_id": RELATION},
"rule_outcome": {},
# moments and retrieval: install-level, and judgments asked for anywhere
"tune_retrieval": {},
"migrate_retrieval_floor": {},
"map_action": {},
"unmap_action": {},
"propose_rule_moments": {},
"judge_rule_moments": {},
"rule_misfired": {},
"judge_menu": {},
# trash: a batch can hold rows from several projects
"restore": {"deleted_batch_id": TRASH},
"purge_trash": {"deleted_batch_id": TRASH},
}
# Argument names that would locate a write wherever they appear. The guard test
# requires every write tool parameter with one of these names to be declared
# above, or listed in REFERENCES below, so a tool that gains one can't write
# past the fence unnoticed.
LOCATOR_NAMES = frozenset({
"project_id", "task_id", "note_id", "lesson_id", "snippet_id", "process_id",
"milestone_id", "system_id", "rule_id", "deleted_batch_id",
})
# Arguments with a locating name that only point at a record, with why each one
# doesn't say where the write lands.
REFERENCES: dict[str, dict[str, str]] = {
"classify_shapes_by_rule": {
"snippet_id": "the canon the shapes are judged against; the rows written are project_id's",
},
"confirm_shape_proposals": {
"snippet_id": "narrows which proposals to confirm; the rows written are project_id's",
},
"judge_lesson_link": {
"lesson_id": "a judgment on a lesson-to-rule pairing the system asked for",
"rule_id": "a judgment on a lesson-to-rule pairing the system asked for",
},
"rule_outcome": {
"rule_id": "telemetry on what a rule changed in this session's own work",
},
"rule_misfired": {
"rule_id": "telemetry on a moment that fired a rule in this session",
"project_id": "the project the misfire happened in, recorded with the report",
},
}
# Tool-name prefixes the plugin's fence answers without asking the server: every
# tool named this way reads. tests/test_write_scope.py holds the hook's copy of
# this list equal to it, and holds every write tool to a name outside it.
READ_PREFIXES = (
"get_", "list_", "search", "find_", "enter_project", "what_might_apply",
"resolve_design_system",
)
# The one write a session may make in another project: filing an issue there,
# as a new open task outside any plan of theirs.
ISSUE_STATUSES = frozenset({"", "todo"})
def is_issue_filing(tool: str, args: dict) -> bool:
"""True for create_task(kind="issue") that opens a fresh, unplanned issue."""
if tool != "create_task":
return False
if str(args.get("kind") or "") != "issue":
return False
if str(args.get("status") or "") not in ISSUE_STATUSES:
return False
return not _positive_ints(args.get("milestone_id")) and not _positive_ints(args.get("parent_id"))
def _positive_ints(value) -> list[int]:
"""The ids an argument names. 0 and negative values mean "none" or "leave
unchanged" across the tools (update_task's project_id=-1 clears), so only
positive ids locate anything."""
values = value if isinstance(value, (list, tuple)) else [value]
out = []
for v in values:
try:
n = int(v)
except (TypeError, ValueError):
continue
if n > 0:
out.append(n)
return out
async def _projects_of(user_id: int, kind: str, value) -> set[int]:
"""The projects the argument `value` of locator `kind` lands in."""
from grimoire.models import async_session
from grimoire.models.family import FamilyDecision
from grimoire.models.milestone import Milestone
from grimoire.models.note import Note
from grimoire.models.rulebook import Rule, RuleRelation
from grimoire.models.system import System
from grimoire.services import repo_bindings
if kind == REPO:
found = await repo_bindings.resolve_project(user_id, str(value or ""))
return {found} if found else set()
if kind == TRASH:
batch = str(value or "").strip()
if not batch:
return set()
async with async_session() as session:
found: set[int] = set()
for model in (Note, Milestone, System, Rule):
rows = await session.execute(
select(model.project_id).where(model.deleted_batch_id == batch)
)
found.update(p for p in rows.scalars() if p)
return found
ids = _positive_ints(value)
if not ids:
return set()
if kind == PROJECT:
return set(ids)
async with async_session() as session:
if kind == RELATION:
rows = await session.execute(
select(Rule.project_id)
.join(RuleRelation, or_(RuleRelation.from_rule_id == Rule.id,
RuleRelation.to_rule_id == Rule.id))
.where(RuleRelation.id.in_(ids))
)
return {p for p in rows.scalars() if p}
if kind == DECISION:
rows = await session.execute(
select(FamilyDecision.project_id, Note.project_id)
.join(Note, Note.id == FamilyDecision.idea_id)
.where(FamilyDecision.id.in_(ids))
)
return {own or home for own, home in rows.all() if own or home}
model = {NOTE: Note, MILESTONE: Milestone, SYSTEM: System, RULE: Rule}[kind]
rows = await session.execute(select(model.project_id).where(model.id.in_(ids)))
return {p for p in rows.scalars() if p}
async def tool_targets(user_id: int, tool: str, args: dict) -> set[int]:
"""Every project the MCP call `tool(**args)` would change. Empty for a
read, a write that lands in no project, and a tool this server doesn't
have."""
found: set[int] = set()
for name, kind in WRITE_TARGETS.get(tool, {}).items():
if name in args:
found |= await _projects_of(user_id, kind, args[name])
return found
async def _titles(ids: list[int]) -> dict[int, str]:
from grimoire.models import async_session
from grimoire.models.project import Project
if not ids:
return {}
async with async_session() as session:
rows = await session.execute(select(Project.id, Project.title).where(Project.id.in_(ids)))
return {pid: title for pid, title in rows.all()}
def _named(project_id: int, titles: dict[int, str]) -> str:
title = titles.get(project_id)
return f"{title} (project {project_id})" if title else f"project {project_id}"
async def check_act(
user_id: int, session_project: int, *, tool: str = "", args: dict | None = None,
target_project: int = 0, act: str = "",
) -> dict:
"""May a session in `session_project` do this act?
The act is an MCP call (`tool` with `args`) or a directory already
resolved to `target_project`; `act` words it for the reason ("git push
in", "Editing a file in"). Returns:
- `allowed`: False when the act would change another project and isn't the
one cross-project write allowed;
- `targets`: the other projects, as {id, title};
- `issue`: the act is that allowed write, filing an issue;
- `reason`: what the operator reads in the prompt, and the session after.
A session with no project is held to nothing. A target the caller can't
read is left out: the write itself is refused there, so there is nothing
to ask about, and naming it would tell the caller where a record lives.
"""
from grimoire.services.access import can_read_project
args = args or {}
if tool:
landing = await tool_targets(user_id, tool, args)
else:
landing = {target_project} if target_project else set()
others = [
p for p in sorted(landing)
if p != session_project and await can_read_project(user_id, p)
]
if not session_project or not others:
return {"allowed": True, "targets": [], "issue": False, "reason": ""}
titles = await _titles([session_project, *others])
targets = [{"id": p, "title": titles.get(p, "")} for p in others]
if tool and is_issue_filing(tool, args):
return {"allowed": True, "targets": targets, "issue": True, "reason": ""}
there = ", ".join(_named(p, titles) for p in others)
what = act.strip() or (f"{tool} changes" if tool else "This changes")
first = others[0]
reason = (
f"This session belongs to {_named(session_project, titles)}. {what} {there}, "
f"a different project. From here, the one change to make there is filing an "
f"issue: create_task(project_id={first}, kind=\"issue\") with how it was found "
f"and the evidence. Approve only if this session should change {there} itself."
)
return {"allowed": False, "targets": targets, "issue": False, "reason": reason}
+17
View File
@@ -90,6 +90,23 @@ def test_after_write_names_what_bash_just_wrote_then_stays_quiet_until_the_next_
assert seen[0]["exclude_derive"] == ["dup:483a"]
assert set(seen[0]["shapes"][0].split(",")) == {"css:log-empty", "css:other"}
def test_after_write_records_every_definition_for_the_turn_but_queries_only_the_first_four_files(tmp_path):
"""The end-of-turn ledger hears every file and every definition a Bash call
wrote; the prior-art arms stay bounded to four files and twelve names."""
with http_sink(SINK_REPLY) as (port, seen):
env = _env(tmp_path, url=f"http://127.0.0.1:{port}")
repo = _repo(tmp_path, env)
for n in range(6):
body = "".join(f"def f{n}_{i:02d}():\n return {i}\n\n" for i in range(15))
(repo / f"gen{n}.py").write_text(body)
_run(repo, env, session="s-many")
ledger = (tmp_path / "grimoire-priorart" / "s-many.written.ids").read_text().splitlines()
assert len(ledger) == 6 * 15
assert "gen5.py\tsym\tf5_14" in ledger
assert len(seen) == 4
assert all(len(q["shapes"][0].split(",")) == 12 for q in seen)
def test_after_write_is_silent_where_it_has_nothing_to_say(tmp_path):
env = _env(tmp_path)
repo = _repo(tmp_path, env)
+45
View File
@@ -20,6 +20,7 @@ from grimoire.services.plugin_context import (
_AUTOINJECT_CONTEXT_MAX,
_AUTOINJECT_CONTEXT_PROMPT_MAX,
_autoinject_query,
_has_subject,
_menu_name,
_menu_passage,
)
@@ -56,6 +57,50 @@ def test_context_is_cut_from_the_end_of_the_reply():
assert tail.endswith("TAIL") and "HEAD" not in tail
# --- a prompt that names nothing ----------------------------------------------
@pytest.mark.parametrize("prompt", [
"keep going", "yes please do", "1. yes 2. yes 3. yes", "start step 6",
"is it deployed?", "awesome, what's next?", "go ahead",
"is there anything else open in this project?",
"please commit and let me know when it passes CI",
])
def test_a_turn_of_the_conversation_names_no_subject(prompt):
assert not _has_subject(prompt)
@pytest.mark.parametrize("prompt", [
"now fix the filter", "keep steward, it's our own app",
"continue into DRY 14", "I've added a maintenance plan",
"fix the Windows gate first, then step 2",
])
def test_one_word_of_its_own_is_a_subject(prompt):
assert _has_subject(prompt)
@pytest.mark.asyncio
async def test_a_prompt_with_no_subject_gets_no_menu_and_no_search():
"""Its query would be the previous reply's tail alone, and the menu would
hand back what that reply was built from."""
from grimoire.services import plugin_context as pc
search = AsyncMock(return_value=[(0.9, fake_note(id=11, title="the reply's own record", user_id=1))])
retrieval = MagicMock()
with patch.object(pc, "get_autoinject_config",
AsyncMock(return_value={"enabled": True, "threshold": 0.55, "top_k": 3})), \
patch.object(pc, "semantic_search_notes", search), \
patch.object(pc, "record_retrieval", retrieval), \
patch.object(pc, "record_surfaced", MagicMock()):
out = await pc.build_autoinject_hint(
1, "yes please do", project_id=2,
context="I'll move the library filters into the side column.",
)
assert out["context"] == "" and out["note_ids"] == []
search.assert_not_called()
retrieval.assert_not_called()
# --- what the hook extracts --------------------------------------------------
def _rec(role: str, blocks: list[dict], sidechain: bool = False) -> str:
+83
View File
@@ -0,0 +1,83 @@
"""The instance keeps its own SECRET_KEY in the data directory.
A deployment that never sets SECRET_KEY must get the same key on every start,
or everything sealed under the first one (forge tokens, the backup secret)
stops opening after the first container recreate.
"""
import stat
from unittest.mock import patch
import pytest
from grimoire import config
from grimoire.config import SECRET_KEY_FILENAME, load_or_create_secret_key
from grimoire.services import secrets
def test_the_first_start_writes_a_key_only_its_owner_can_read(tmp_path):
key = load_or_create_secret_key(tmp_path)
path = tmp_path / SECRET_KEY_FILENAME
assert key and len(key) == 64
assert path.read_text().strip() == key
assert stat.S_IMODE(path.stat().st_mode) == 0o600
assert [p.name for p in tmp_path.iterdir()] == [SECRET_KEY_FILENAME]
def test_every_later_start_reads_the_same_key_and_what_it_sealed_still_opens(tmp_path):
first = load_or_create_secret_key(tmp_path)
with patch.object(secrets.Config, "SECRET_KEY", first):
sealed = secrets.seal("forge-token")
second = load_or_create_secret_key(tmp_path)
assert second == first
with patch.object(secrets.Config, "SECRET_KEY", second):
assert secrets.unseal(sealed) == "forge-token"
def test_no_data_directory_means_no_kept_key(tmp_path):
assert load_or_create_secret_key(tmp_path / "absent") is None
def test_an_empty_key_file_is_refused_rather_than_replaced(tmp_path):
(tmp_path / SECRET_KEY_FILENAME).write_text("\n")
with pytest.raises(RuntimeError, match="is empty"):
load_or_create_secret_key(tmp_path)
assert (tmp_path / SECRET_KEY_FILENAME).read_text() == "\n"
def test_an_explicit_key_wins_and_leaves_the_kept_one_alone(tmp_path, monkeypatch):
monkeypatch.setenv("DATA_DIR", str(tmp_path))
monkeypatch.delenv("SECRET_KEY_FILE", raising=False)
monkeypatch.delenv("SECRET_KEY", raising=False)
kept, source = config._secret_key()
assert source == "data"
monkeypatch.setenv("SECRET_KEY", "an-operator-key")
assert config._secret_key() == ("an-operator-key", "env")
monkeypatch.delenv("SECRET_KEY")
assert config._secret_key() == (kept, "data")
def test_a_secret_key_file_that_is_missing_stops_the_start(tmp_path, monkeypatch):
monkeypatch.setenv("DATA_DIR", str(tmp_path))
monkeypatch.delenv("SECRET_KEY", raising=False)
monkeypatch.setenv("SECRET_KEY_FILE", str(tmp_path / "no-such-secret"))
with pytest.raises(RuntimeError, match="SECRET_KEY_FILE"):
config._secret_key()
assert not (tmp_path / SECRET_KEY_FILENAME).exists()
def test_with_nowhere_to_keep_a_key_the_process_gets_a_temporary_one(tmp_path, monkeypatch):
monkeypatch.setenv("DATA_DIR", str(tmp_path / "absent"))
monkeypatch.delenv("SECRET_KEY", raising=False)
monkeypatch.delenv("SECRET_KEY_FILE", raising=False)
key, source = config._secret_key()
assert source == "ephemeral"
assert key not in config.PLACEHOLDER_SECRET_KEYS and len(key) == 64
+1 -1
View File
@@ -65,7 +65,7 @@ def test_a_bare_integer_is_a_valid_marker(tmp_path):
def test_the_marker_is_found_from_a_subdirectory(tmp_path):
_marker(tmp_path, instance=INSTANCE, project_id=2, project="FabledScribe")
_marker(tmp_path, instance=INSTANCE, project_id=2, project="FabledGrimoire")
deep = tmp_path / "src" / "grimoire" / "services"
deep.mkdir(parents=True)
assert _sh(f'grimoire_scope_query "{deep}"', tmp_path) == "project_id=2"
@@ -5,7 +5,8 @@ the real `restore_full_backup`.
The unit lane already shows the tar hands back the dict it was given. What
only this lane shows is that the dict it hands back is one the restore
accepts — every id remapped to the restored rows, nothing dropped — and that
the command-line restore refuses a database that already has accounts.
the command-line restore refuses, on a live instance, a backup that doesn't
record this instance as its source.
"""
import pytest
import pytest_asyncio
@@ -97,10 +98,11 @@ async def test_the_tar_restores_with_every_id_remapped(written):
assert note.status == "todo"
async def test_the_command_refuses_a_database_with_accounts(written):
"""The source user exists, so this database is not empty: the command
must stop before restoring anything."""
assert await run(written) == 3
async def test_the_command_refuses_a_backup_from_no_known_instance(written):
"""This database is live and the backup names no instance it came from,
so it can't be rolled back to: the command stops before restoring
anything, even with --yes."""
assert await run(written, yes=True) == 3
async with async_session() as s:
restored = (await s.execute(select(User).where(User.username == RESTORED))).scalars().all()
assert restored == []
+214
View File
@@ -0,0 +1,214 @@
"""Real-Postgres proof that a full backup rolls this instance back by id.
The lane shares one database, so every test here takes its backup of the
whole store, changes its own rows, and rolls back to that backup: what it
puts back is exactly what was there when it started.
"""
from datetime import datetime, timezone
from unittest.mock import AsyncMock, patch
import pytest
import pytest_asyncio
from sqlalchemy import func, select
from grimoire.models import async_session
from grimoire.models.note import Note
from grimoire.models.project import Project
from grimoire.models.user import User
from grimoire.restore import run
from grimoire.services import backup, backup_job
from grimoire.services.api_keys import create_api_key, lookup_key
from grimoire.services.backup_file import write_archive
from grimoire.services.instance import adopt_instance_id, get_instance_id
from tests.helpers import ensure_user
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
OWNER = "rollback_owner"
@pytest_asyncio.fixture
async def seeded():
"""One user with a project and two tasks, committed, and a full backup
taken right after."""
async with async_session() as s:
user = await ensure_user(s, OWNER)
# A clean slate for this user: an earlier test's rows would otherwise
# be in this test's backup too.
for model in (Note, Project):
for row in (await s.execute(select(model).where(model.user_id == user.id))).scalars():
await s.delete(row)
await s.flush()
project = Project(user_id=user.id, title="Rollback drill")
s.add(project)
await s.flush()
kept = Note(user_id=user.id, project_id=project.id, title="before",
body="as backed up", note_type="note", task_kind="work", status="todo")
doomed = Note(user_id=user.id, project_id=project.id, title="deleted later",
body="", note_type="note", task_kind="work", status="todo")
s.add_all([kept, doomed])
await s.commit()
ids = {"user": user.id, "project": project.id, "kept": kept.id, "doomed": doomed.id}
key, _ = await create_api_key(ids["user"], "rollback drill", "read")
payload = await backup.export_full_backup()
return ids, payload, key
def _own(payload: dict, table: str, user_id: int) -> list[dict]:
return sorted(
(row for row in payload[table] if row.get("user_id") == user_id),
key=lambda row: row["id"],
)
async def _change_everything(ids: dict) -> int:
"""Edit one task, delete the other, add a task and a project. Returns the
new task's id."""
async with async_session() as s:
kept = await s.get(Note, ids["kept"])
kept.title = "after"
kept.body = "edited since the backup"
await s.delete(await s.get(Note, ids["doomed"]))
extra = Project(user_id=ids["user"], title="made after the backup")
s.add(extra)
await s.flush()
new = Note(user_id=ids["user"], project_id=extra.id, title="new since",
body="", note_type="note", task_kind="work", status="todo")
s.add(new)
await s.commit()
return new.id
async def test_a_rollback_puts_back_edits_and_deletions_and_removes_new_records(seeded):
ids, payload, _ = seeded
new_id = await _change_everything(ids)
result = await backup.replace_from_backup(payload)
assert result["mode"] == "rollback"
async with async_session() as s:
kept = await s.get(Note, ids["kept"])
assert (kept.title, kept.body) == ("before", "as backed up")
assert await s.get(Note, ids["doomed"]) is not None
assert await s.get(Note, new_id) is None
titles = (await s.execute(
select(Project.title).where(Project.user_id == ids["user"])
)).scalars().all()
assert titles == ["Rollback drill"]
after = await backup.export_full_backup()
for table in ("projects", "milestones", "notes"):
assert _own(after, table, ids["user"]) == _own(payload, table, ids["user"]), table
async def test_a_surviving_users_api_key_still_works_and_their_sessions_end(seeded):
ids, payload, key = seeded
async with async_session() as s:
version_before = (await s.get(User, ids["user"])).session_version
await _change_everything(ids)
await backup.replace_from_backup(payload)
found = await lookup_key(key)
assert found is not None and found.user_id == ids["user"]
async with async_session() as s:
assert (await s.get(User, ids["user"])).session_version == version_before + 1
async def test_new_records_after_a_rollback_take_fresh_ids(seeded):
ids, payload, _ = seeded
await _change_everything(ids)
await backup.replace_from_backup(payload)
async with async_session() as s:
highest = (await s.execute(select(func.max(Note.id)))).scalar_one()
note = Note(user_id=ids["user"], project_id=ids["project"], title="after rollback",
body="", note_type="note", task_kind="work", status="todo")
s.add(note)
await s.commit()
assert note.id > highest
await s.delete(note)
await s.commit()
@pytest.mark.parametrize("instance_id", ["another-instance", None])
async def test_a_backup_from_elsewhere_is_refused_on_a_live_instance(seeded, instance_id):
ids, payload, _ = seeded
foreign = {**payload, "instance_id": instance_id}
new_id = await _change_everything(ids)
with pytest.raises(backup.RestoreRefused):
await backup.replace_from_backup(foreign)
async with async_session() as s:
assert (await s.get(Note, ids["kept"])).title == "after"
assert await s.get(Note, new_id) is not None
await backup.replace_from_backup(payload)
async def test_a_users_own_export_is_refused(seeded):
_, payload, _ = seeded
with pytest.raises(backup.RestoreRefused):
await backup.replace_from_backup({**payload, "scope": "user"})
async def test_a_failure_part_way_changes_nothing(seeded):
ids, payload, _ = seeded
new_id = await _change_everything(ids)
with patch.object(backup, "_reset_sequences", AsyncMock(side_effect=RuntimeError("boom"))):
with pytest.raises(RuntimeError):
await backup.replace_from_backup(payload)
async with async_session() as s:
assert (await s.get(Note, ids["kept"])).title == "after"
assert await s.get(Note, ids["doomed"]) is None
assert await s.get(Note, new_id) is not None
await backup.replace_from_backup(payload)
async def test_a_fresh_instance_adopts_the_backups_instance_id(seeded):
_, payload, _ = seeded
own = await get_instance_id()
try:
with patch.object(backup, "_is_fresh", AsyncMock(return_value=True)):
result = await backup.replace_from_backup({**payload, "instance_id": "a-lineage"})
assert result["mode"] == "fresh"
assert await get_instance_id() == "a-lineage"
finally:
async with async_session() as s:
await adopt_instance_id(s, own)
await s.commit()
async def test_the_command_only_previews_without_yes(seeded, tmp_path):
ids, payload, _ = seeded
path = write_archive(payload, tmp_path, datetime.now(timezone.utc), app_version="test")
new_id = await _change_everything(ids)
assert await run(path) == 4
async with async_session() as s:
assert (await s.get(Note, ids["kept"])).title == "after"
assert await s.get(Note, new_id) is not None
await backup.replace_from_backup(payload)
async def test_the_command_rolls_back_from_the_tar_with_yes(seeded, tmp_path):
"""The scheduled backup's own file, read back through the command: the
whole store returns to it, after the safety backup is taken."""
ids, payload, _ = seeded
path = write_archive(payload, tmp_path, datetime.now(timezone.utc), app_version="test")
new_id = await _change_everything(ids)
saved = {"file": "grimoire-safety.tar", "size_bytes": 1, "where": str(tmp_path)}
with patch.object(backup_job, "save_state_before_restore",
AsyncMock(return_value=saved)) as save:
assert await run(path, yes=True) == 0
save.assert_awaited_once()
async with async_session() as s:
kept = await s.get(Note, ids["kept"])
assert (kept.title, kept.body) == ("before", "as backed up")
assert await s.get(Note, ids["doomed"]) is not None
assert await s.get(Note, new_id) is None

Some files were not shown because too many files have changed in this diff Show More