Restore rolls back to a backup; a session can only file issues in another project #220

Merged

bvandeusen merged 15 commits from dev into main 2026-11-05 09:29:55 -05:00

Conversation 0 Commits 15 Files Changed 52

+3853 -729

15 Commits

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5.5	49b5ceee41	docs(6856): the scope section says the cross-project rule in fewer lines, so using-grimoire fits its 350-line budget ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 22:01:22 -05:00
bvandeusenandClaude Opus 5.5	8f8263b347	docs(6856): the plugin's guidance and the docs say what a session may do in another project ... using-grimoire, which owns cross-project conduct, now says the one change to make toward another project is filing an issue with how it was found and the evidence, and that the plugin asks before anything more. The features page describes the fence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 21:52:09 -05:00
bvandeusenandClaude Opus 5.5	ddcac96542	fix(6855): the fence keeps its files where no compaction sweeps them, and builds id lists under mawk ... The project a session entered now lives in grimoire-fence, which the compaction clear never visits, and session end removes it. Arrays of ids in a tool's input were sent as [,"3","4"] under mawk, which creates an array element as soon as it is assigned to. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 21:51:34 -05:00
bvandeusenandClaude Opus 5.5	805fda5cb1	feat(6855): the plugin asks before a session writes to a project other than its own ... plugin/hooks/grimoire_project_fence.sh runs as PreToolUse on editor writes, Bash and Grimoire's tools. A session belongs to the project its starting directory resolves to (CLAUDE_PROJECT_DIR: a .grimoire marker, else the git remote's binding). A session that starts outside any project belongs to the first one it enters, which PostToolUse on enter_project records. These acts stop on permissionDecision "ask", with the server's reason naming the other project: - an edit in another project's repo or a worktree of it; - a git commit, push, merge, rebase, pull, cherry-pick, revert, tag, am, reset or worktree add there, found through git -C, a cd before it, or the shell's current directory; - a Grimoire write that lands in another project. A live probe showed "ask" holds in bypassPermissions. Filing an issue and every read go ahead. An edit or git command in the session's own repo, and any tool named like a read, never calls the server. Grimoire writes send only their short arguments, so the URL stays small. When the instance doesn't answer, the act goes ahead and the hook says once per outage that it couldn't check. check_plugin.py now holds one outage line per hook and has a smoke event for the fence. GET /api/plugin/write-scope takes fallback_project_id for the entered project. tests/test_project_fence.py drives the hook against real git repos and a stand-in server. tests/test_write_scope.py holds the hook's FENCE_READ_PREFIXES equal to services/write_scope.READ_PREFIXES. The plugin README and PACKAGING describe the fence, and the plugin is minted to 2026.11.05.0245. Milestone 583, step 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 21:45:37 -05:00
bvandeusenandClaude Opus 5.5	16b1d68ebd	fix(6854): the locating arguments that only point are declared as references, and the plugin version is minted ... The guard caught seven write-tool arguments with a locating name that are not where the write lands: - snippet_id on classify_shapes_by_rule and confirm_shape_proposals names the canon, not the rows written; - rule_outcome, rule_misfired and judge_lesson_link record judgments. REFERENCES lists each one with its reason, and the guard reads it. READ_PREFIXES names the tool prefixes the fence answers without the server, and a test holds every write tool to a name outside them. plugin.json is minted to 2026.11.05.0242 for the PACKAGING.md row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 21:42:14 -05:00
bvandeusenandClaude Opus 5.5	55f0f96016	feat(6854): the server says which project a pending act would change, and whether this session may do it ... GET /api/plugin/write-scope answers the plugin's fence. The session's project comes from repo / project_id, as every plugin endpoint reads it. The act is either: - a directory (target_repo / target_project_id), for a file edit or a git command; or - an MCP call (tool, plus input: the call's short arguments). It returns allowed, the other projects the act would change, and the reason the operator is asked. It is a GET so a read-scoped key still fences file edits and git. services/write_scope.WRITE_TARGETS lists every write tool with the arguments that locate its write: a project id, or a task, note, milestone, System, rule, relation, trash batch, repo or family decision, each traced to its project. A write that lands in no project (install settings, user-level records, the judgments a session is asked for anywhere) has an empty entry. The one cross-project write allowed is create_task(kind="issue") as a new, unplanned, open issue. A project the caller cannot read is never named. tests/test_write_scope.py: - requires an entry for every write tool; - requires every declared argument to exist on the tool; - refuses a locating argument (project_id, task_id, milestone_id and the rest) that a write tool takes without declaring it. The integration test traces each locator from real rows. Milestone 583, step 2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 21:39:03 -05:00
bvandeusenandClaude Opus 5.5	373f5bd454	feat(6839): the decision log and the knowledge feed load as you scroll, through one ScrollSentinel ... ScrollSentinel sits at the foot of a server-paged list and asks for the next page a screen before the reader reaches it. It asks once per cursor, forgets the cursor when its element goes, and shows Try again on a failed page instead of retrying in a loop. FamilyView's decision log loses its "Older decisions" button. A failed page shows in place rather than as a toast, and rows already shown are skipped when a new decision shifts the offset. KnowledgeView drops its inline IntersectionObserver for the component. The cursor is the ids consumed, not the items rendered, so a batch that returns fewer records than it asked for still moves the feed on. A failed batch puts its ids back and offers Try again, where it used to fail silently and stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 20:14:41 -05:00
bvandeusenandClaude Opus 5.5	846c87629f	test(6849): the restore command previews without --yes and rolls back from the tar with it ... Two integration tests in test_integration_backup_rollback.py drive grimoire.restore.run on the scheduled backup's tar, written from a full export of this instance: - without --yes it exits 4 and nothing changes; - with --yes it takes the safety backup, then puts back the edited and the deleted task and removes the new one. The file-restore test that expected a refusal "because the database has accounts" now names the reason it actually refuses: on a live instance, a backup that records no instance id can't be rolled back to. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 20:12:30 -05:00
bvandeusenandClaude Opus 5.5	37be3b0bff	feat(6849): the command-line restore uses the same rollback, and the docs say how restore works now ... python -m grimoire.restore <file> prints what the restore would change (the same preview as Settings) and stops. With --yes it runs backup_job.rollback_to_backup: the current state is saved first, then the whole instance goes back to the backup. An empty instance takes any full backup. A live one only takes a backup of its own, and otherwise exits 3 with the reason. A v1 backup keeps the copy-in path into an empty instance. docs/backups.md Restoring is rewritten: - what a restore puts back and removes; - the same-instance rule; - the safety backup; - what survives; - the web and command-line ways in. Its S3 key advice now asks for read as well as write when restoring from the bucket. README, features, architecture and the API reference follow. Milestone 582, step 5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 20:08:31 -05:00
bvandeusenandClaude Opus 5.5	1f5e963c35	feat(6848): Settings restore rolls back, from the destination or a file, after a preview ... The web restore no longer copies a backup in beside the live data. - GET /api/admin/restore/sources lists the backups at the backup destination, newest first. - POST /api/admin/restore/preview says what a restore would do, changing nothing: fresh or rollback, when the backup was taken, and counts now and after per kind. A backup that cannot apply here gets a 409 with the reason. - POST /api/admin/restore?confirm=rollback runs backup_job.rollback_to_backup: safety backup, replace, then search rebuilt in the background. Each takes ?source=<name> (a backup at the destination) or an upload. The destination path exists because uploads are held to the request-size limit, and a real instance's backup is far over it. backup_s3.get_file downloads from a bucket. backup_destination.list_backups/open_backup only accept names the scheduled backup writes, so a request cannot reach any other file. Settings -> Data -> Restore from Backup opens RestoreBackupDialog. The dialog: 1. Pick a backup at the destination or a file. 2. Read the preview. 3. Type "roll back". 4. Run it. The dialog then says where the replaced state was saved, and sends the user to sign in again. Closes #6840. Milestone 582, step 4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 20:06:07 -05:00
bvandeusenandClaude Opus 5.5	34f8d7060b	feat(6847): a rollback first backs up the state it replaces ... backup_job.rollback_to_backup: 1. Checks the backup can apply here (backup.check_replace, which shares _replace_mode with replace_from_backup), refusing before anything is written. 2. Takes the backup lock, so no scheduled backup runs during the rollback. 3. Saves the current store with save_state_before_restore. 4. Replaces the store. The safety file goes to the backup destination when backups are on and it takes it. Otherwise it goes to DATA_DIR/pre-restore, with a note when the destination refused. If neither takes it the rollback does not run. The result names where the file went. Milestone 582, step 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 20:02:26 -05:00
bvandeusenandClaude Opus 5.5	8806ab85b4	feat(6846): replace_from_backup rolls the store back to a full backup by id ... One transaction: - Refuse unless the backup is a full one and, on a live instance, carries this instance's id. A fresh instance (no records, at most the restoring account) takes any full backup and adopts its id. - Clear first: search vectors and the CSS consumer map (derived); every backed-up table nothing outside the backup points at, whole; the settings the backup carries; and the identity-table rows the backup lacks. A share granted by a removed user goes before the user. - Write the backup through the same _build_* helpers as the copy-in restore, now _load_v2(keep_ids=...). Identity tables (users, projects, milestones, notes, rulebooks, topics, rules, systems, design systems, code shapes, family decisions) go through _place. That keeps the backup's id and merges, so API keys, shares and forge connections of a surviving row stay. - Reset id sequences past the restored max, and sign every session out. reindex_after_restore runs the four embedding backfills afterwards. A unit guard checks that no row kept for outside references cascades from a table replaced whole. Integration tests cover edits, deletions and new rows put back, a surviving API key, sessions ending, fresh ids afterwards, foreign or id-less backups refused, a user export refused, a failure part-way changing nothing, and a fresh instance adopting the id. Milestone 582, step 2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 19:59:48 -05:00
bvandeusenandClaude Opus 5.5	6d8db0bb65	feat(6845): every instance has an id lineage, and every full backup carries it ... instance_identity is one row made by migration 0127 with a random id, so an instance has it before its first account. A full backup carries it in its header as instance_id (the tar manifest header and the JSON top level); BACKUP_VERSION is 28. The table rides in _NOT_INCLUDED: a restore decides from the header value rather than copying a row. Two stores with the same id number their rows the same way. That is what lets a later step roll a live instance back by original id, refuse a backup whose ids would land on unrelated rows, and have a fresh-instance restore adopt the backup's id (adopt_instance_id). Milestone 582, step 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 19:53:45 -05:00
bvandeusenandClaude Opus 5.5	3c01e17559	test(6838): give the test request its socket peer the way Quart reads it ... Quart takes remote_addr from a Remote-Addr header its ASGI layer writes from the scope (replacing any the client sent). A test request context does not read scope_base for it, so the two socket-peer tests saw an empty address. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 19:37:28 -05:00
bvandeusenandClaude Opus 5.5	dbd6d5b8b5	feat(6838): read the client address and HTTPS from the trusted proxy hop ... grimoire/proxy.py is the one reader of X-Forwarded-For and X-Forwarded-Proto. It takes the entry TRUSTED_PROXY_HOPS in from the right (default 1; a shorter header reads its leftmost entry; 0 trusts no proxy), so an entry a client puts left of the proxy's is never believed. Before this, with TRUST_PROXY_HEADERS on, the leftmost X-Forwarded-For entry keyed the login, register, forgot and reset rate limits and the audit log, and a client could choose it. The session cookie is Secure, and HSTS (max-age=31536000) is sent, exactly when the request is HTTPS by the trusted proxy's word. Plain HTTP is untouched and nothing redirects. SECURE_COOKIES is gone (a startup warning says so when it is still set); TRUST_PROXY_HEADERS=false still maps to 0 hops, with a warning to move to TRUSTED_PROXY_HOPS. Admin audit lines now use the same client address instead of the socket peer. The placeholder-key refusal keyed on SECURE_COOKIES, so it now refuses either placeholder key outright; the instance makes its own key when none is set. Family idea #5004 (IsHTTPS, trusted-hop X-Forwarded-Proto), canon v2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-04 19:32:20 -05:00