Restore rolls back to a backup; a session can only file issues in another project #220
@@ -96,7 +96,7 @@ docker compose up -d
|
||||
|
||||
Database migrations run when the app starts, and the key stays in its volume. Claude Code updates the plugin from its marketplace; run `/reload-plugins` after an update.
|
||||
|
||||
**Settings → Data → Scheduled backups** writes a daily backup of the database to a folder or an S3-compatible bucket, and `python -m grimoire.restore` brings a fresh instance back from one. See [Backups](docs/backups.md).
|
||||
**Settings → Data → Scheduled backups** writes a daily backup of the database to a folder or an S3-compatible bucket, and Settings → Data → Restore from Backup rolls the instance back to one, or brings a new instance back from it. See [Backups](docs/backups.md).
|
||||
|
||||
## Features
|
||||
|
||||
|
||||
@@ -272,7 +272,9 @@ These serve the plugin's hooks.
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/backup` | Export backup (format v4; `?scope=user` for own data) |
|
||||
| POST | `/api/admin/restore` | Restore from a backup |
|
||||
| GET | `/api/admin/restore/sources` | The backups at the backup destination, newest first |
|
||||
| POST | `/api/admin/restore/preview` | What restoring a backup (`?source=<name>` or an uploaded file) would change; 409 with the reason when it can't apply here |
|
||||
| POST | `/api/admin/restore?confirm=rollback` | Put the whole instance back to that backup, after saving the current state |
|
||||
| GET / DELETE | `/api/admin/users` · `/api/admin/users/:id` | List / delete users |
|
||||
| GET / PUT | `/api/admin/registration` | Get / toggle registration |
|
||||
| GET / PUT | `/api/admin/smtp` · POST `/api/admin/smtp/test` | SMTP config + test email |
|
||||
|
||||
@@ -87,7 +87,7 @@ Credentials Grimoire stores for outside services (forge tokens, the backup desti
|
||||
│ ├── routes/ REST blueprints
|
||||
│ ├── mcp/ MCP server and its tool modules
|
||||
│ ├── services/ shared logic
|
||||
│ └── restore.py `python -m grimoire.restore` for a fresh instance
|
||||
│ └── restore.py `python -m grimoire.restore`: the restore, from the command line
|
||||
├── frontend/src/ views, components, stores, composables, api client
|
||||
├── plugin/ Claude Code hooks, skills and commands
|
||||
├── scripts/ CI checks and the plugin version minter
|
||||
|
||||
+52
-21
@@ -88,9 +88,12 @@ You need four things:
|
||||
- **Access key.**
|
||||
- **Secret key.**
|
||||
|
||||
Grimoire uploads and lists; it never deletes. A key that may **only add
|
||||
objects** is therefore enough, and it is the safer key to hand out: a stolen
|
||||
one cannot wipe the history.
|
||||
Grimoire uploads and lists, and reads a backup back when you restore from the
|
||||
bucket; it never deletes. A key that may **add and read objects but not
|
||||
delete them** is therefore enough, and it is the safer key to hand out: a
|
||||
stolen one cannot wipe the history. A key that may only add objects still
|
||||
takes backups; restore then works by uploading a file or from the command
|
||||
line.
|
||||
|
||||
The secret is stored encrypted, under a key derived from `SECRET_KEY`. It is
|
||||
never shown again, and never included in a backup. If an instance's
|
||||
@@ -122,8 +125,10 @@ keeps service backups as "backup targets". Either destination works with it.
|
||||
1. On Archives' **Backups** page, make a target for Grimoire, named `grimoire`
|
||||
for example.
|
||||
2. Choose how Grimoire's files arrive:
|
||||
- **S3**: turn on the S3 way in for the target and make an **append-only**
|
||||
S3 key. In Grimoire, choose **An S3 bucket**:
|
||||
- **S3**: turn on the S3 way in for the target and make an S3 key that
|
||||
may add and read objects but not delete them (an **append-only** key
|
||||
takes backups too, but then Grimoire can't restore from the bucket). In
|
||||
Grimoire, choose **An S3 bucket**:
|
||||
- address: Archives' backup address (`http://<archives>:8081`);
|
||||
- bucket: the target's name;
|
||||
- access key and secret key: the key you just made.
|
||||
@@ -139,26 +144,52 @@ keeps service backups as "backup targets". Either destination works with it.
|
||||
|
||||
## Restoring
|
||||
|
||||
### Into a fresh instance
|
||||
A restore puts the **whole instance** back to the moment a backup was taken,
|
||||
for every user:
|
||||
|
||||
Start Grimoire on an empty database, copy the backup file into the container,
|
||||
and run:
|
||||
- records edited since go back to how they were;
|
||||
- records deleted since come back;
|
||||
- anything made since is removed.
|
||||
|
||||
It matches records by their ids, so a live instance only restores a backup **of
|
||||
its own**. Each instance has an id, and every backup records the one it came
|
||||
from. A backup from another instance, or one made before backups recorded it,
|
||||
only goes into a new, empty instance, which takes it as it is.
|
||||
|
||||
Before changing anything, the restore saves the current state as one more
|
||||
backup. It goes to the backup destination when scheduled backups are on, and
|
||||
otherwise to `/data/pre-restore`. The result says which file it is, so a
|
||||
restore can itself be undone.
|
||||
|
||||
What a backup doesn't carry stays as it is wherever its owner survives the
|
||||
restore: API keys, forge connections, sharing, logs, and the destination's S3
|
||||
secret. The trash isn't in backups, so anything in the trash when the backup
|
||||
was taken is gone afterwards. Everyone is signed out, and search is rebuilt.
|
||||
|
||||
### From the web
|
||||
|
||||
**Settings → Data → Restore from Backup** lists the backups at the backup
|
||||
destination. You can pick one of those, or upload a `.tar` or the JSON
|
||||
**Full Backup** download (uploads are held to the server's request-size
|
||||
limit). The restore then shows what would change: when the backup was taken,
|
||||
and how many accounts, projects, plans, notes and rules there are now and will
|
||||
be afterwards. Type `roll back` to run it. Then sign in again.
|
||||
|
||||
To read backups back from an S3 bucket, the key needs read access as well as
|
||||
write. Grimoire never deletes from the bucket.
|
||||
|
||||
### From the command line
|
||||
|
||||
For a file with no size limit, or a new instance with no backup destination
|
||||
yet, copy the file into the container and run:
|
||||
|
||||
```sh
|
||||
docker exec -it grimoire python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar
|
||||
```
|
||||
|
||||
- The command refuses a database that already has an account. A restore
|
||||
re-creates everything with new ids, so on a live instance it would add a
|
||||
second copy instead of replacing anything.
|
||||
- Nothing limits the file's size.
|
||||
- It also reads the JSON from the admin **Full Backup** download.
|
||||
It prints what would change and stops. Add `--yes` to restore. It reads the
|
||||
JSON download too. Afterwards, **restart Grimoire** so the search indexes are
|
||||
rebuilt.
|
||||
|
||||
Afterwards, **restart Grimoire** so the search indexes are rebuilt. Then issue
|
||||
new API keys, add forge connections again, and set up any sharing.
|
||||
|
||||
### From the web
|
||||
|
||||
**Settings → Data → Restore from Backup** takes the `.tar` as well as the
|
||||
JSON download. The upload is held to the server's request-size limit. For a
|
||||
bigger file, use the command above.
|
||||
After restoring into a new instance, issue new API keys, add forge connections
|
||||
again, and set up any sharing: none of them travel in a backup.
|
||||
|
||||
+2
-2
@@ -85,8 +85,8 @@ Share a project, note or task with users or groups, as viewer, editor or admin.
|
||||
## Export and backup
|
||||
|
||||
- **Personal export**: your notes and tasks as a Markdown ZIP (with YAML frontmatter) or JSON.
|
||||
- **Admin backup and restore**, with ids remapped so a backup restores onto another instance.
|
||||
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. `python -m grimoire.restore` brings a fresh instance back from one. See [Backups](backups.md).
|
||||
- **Admin backup and restore**: a restore rolls the whole instance back to one of its own backups, after saving the current state, or brings a new instance back from any backup.
|
||||
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. Restoring is in Settings, or `python -m grimoire.restore`. See [Backups](backups.md).
|
||||
|
||||
## Sign-in
|
||||
|
||||
|
||||
+80
-30
@@ -1,20 +1,26 @@
|
||||
"""Restore a Grimoire backup into an empty instance, from the command line.
|
||||
"""Restore a Grimoire backup from the command line.
|
||||
|
||||
python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar
|
||||
python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar --yes
|
||||
|
||||
This is how a fresh instance is brought back after losing the old one. Run it
|
||||
inside the Grimoire container, which already has the database settings and has
|
||||
already migrated the schema on start. It reads either the scheduled backup's
|
||||
tar (services/backup_file.py) or the JSON the admin download produces.
|
||||
Run it inside the Grimoire container, which already has the database settings
|
||||
and has migrated the schema on start. It reads the scheduled backup's tar
|
||||
(services/backup_file.py) or the JSON the admin download produces, and nothing
|
||||
limits the file's size.
|
||||
|
||||
It refuses a database that already holds an account. A restore re-creates
|
||||
every user, project and record from the file with new ids; into a live
|
||||
instance it would add a second copy of everything rather than replace it.
|
||||
(Settings → Backup & Restore still offers the upload for that case, on
|
||||
purpose and with its own warning.)
|
||||
It does what Settings → Restore does (services/backup.py replace_from_backup):
|
||||
- **A new, empty instance** takes the backup as it is.
|
||||
- **A live instance** rolls back to a backup of its own: everything goes back
|
||||
to the moment the backup was taken, for every user.
|
||||
- A backup from another instance, or one made before backups recorded their
|
||||
instance, only goes into an empty instance.
|
||||
|
||||
Unlike the upload, nothing limits the file's size here, so this is also the
|
||||
way in for a backup too big for the web upload.
|
||||
Without --yes it only says what would change. With --yes it first saves the
|
||||
current state as a backup (to the backup destination, or DATA_DIR/pre-restore),
|
||||
then restores. Everyone is signed out afterwards.
|
||||
|
||||
A backup in the oldest (v1) format is copied into an empty instance with new
|
||||
ids instead, as before.
|
||||
|
||||
Afterwards, restart Grimoire: search indexes are not part of a backup and are
|
||||
rebuilt from the restored records at startup.
|
||||
@@ -48,9 +54,39 @@ async def _account_count() -> int:
|
||||
return (await session.execute(select(func.count()).select_from(User))).scalar_one()
|
||||
|
||||
|
||||
async def run(path: Path) -> int:
|
||||
from grimoire.models import engine
|
||||
async def _restore_v1(payload: dict) -> int:
|
||||
from grimoire.services.backup import restore_full_backup
|
||||
|
||||
accounts = await _account_count()
|
||||
if accounts:
|
||||
print(
|
||||
f"This database already holds {accounts} account(s). A backup in the oldest "
|
||||
"format is copied in with new ids rather than replacing anything, so it only "
|
||||
"goes into an empty instance. Start Grimoire on an empty database and run "
|
||||
"this again.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 3
|
||||
stats = await restore_full_backup(payload)
|
||||
print(json.dumps({"restored": stats}, indent=2, default=str))
|
||||
return 0
|
||||
|
||||
|
||||
def _print_preview(preview: dict) -> None:
|
||||
taken = preview.get("exported_at") or "an unknown date"
|
||||
if preview["mode"] == "rollback":
|
||||
print(f"This rolls the whole instance back to {taken}, for every user. Anything "
|
||||
"made since is removed, edits since are undone, records deleted since come back.")
|
||||
else:
|
||||
print(f"This instance has no records yet; it takes the backup from {taken} as it is.")
|
||||
for count in preview["counts"]:
|
||||
print(f" {count['kind']:<24} now {count['now']:>7} after {count['after']:>7}")
|
||||
|
||||
|
||||
async def run(path: Path, *, yes: bool = False) -> int:
|
||||
from grimoire.models import engine
|
||||
from grimoire.services import backup_job
|
||||
from grimoire.services.backup import RestoreRefused, replace_preview
|
||||
from grimoire.services.backup_file import ArchiveError
|
||||
|
||||
try:
|
||||
@@ -63,20 +99,29 @@ async def run(path: Path) -> int:
|
||||
print(f"{path} isn't a backup this Grimoire can read: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
accounts = await _account_count()
|
||||
if accounts:
|
||||
print(
|
||||
f"This database already holds {accounts} account(s). A restore adds a second "
|
||||
"copy of everything rather than replacing it, so it only runs into an empty "
|
||||
"instance. Start Grimoire on an empty database and run this again.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 3
|
||||
|
||||
stats = await restore_full_backup(payload)
|
||||
print(json.dumps({"restored": stats}, indent=2, default=str))
|
||||
print("Restored. Restart Grimoire so the search indexes are rebuilt.", file=sys.stderr)
|
||||
return 0
|
||||
if (payload.get("version") or 1) < 2:
|
||||
code = await _restore_v1(payload)
|
||||
else:
|
||||
try:
|
||||
_print_preview(await replace_preview(payload))
|
||||
if not yes:
|
||||
print("Nothing was changed. Run it again with --yes to restore.",
|
||||
file=sys.stderr)
|
||||
return 4
|
||||
result = await backup_job.rollback_to_backup(payload)
|
||||
except RestoreRefused as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
return 3
|
||||
safety = result["safety_backup"]
|
||||
print(json.dumps({"restored": result["restored"], "safety_backup": safety},
|
||||
indent=2, default=str))
|
||||
print(f"The state it replaced is saved as {safety['file']} in {safety['where']}.",
|
||||
file=sys.stderr)
|
||||
code = 0
|
||||
if code == 0:
|
||||
print("Restored. Restart Grimoire so the search indexes are rebuilt; "
|
||||
"everyone signs in again.", file=sys.stderr)
|
||||
return code
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
@@ -84,11 +129,16 @@ async def run(path: Path) -> int:
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="python -m grimoire.restore",
|
||||
description="Restore a Grimoire backup (.tar or .json) into an empty instance.",
|
||||
description=(
|
||||
"Restore a Grimoire backup (.tar or .json): into an empty instance, or "
|
||||
"roll this instance back to a backup of its own."
|
||||
),
|
||||
)
|
||||
parser.add_argument("file", type=Path, help="the backup file")
|
||||
parser.add_argument("--yes", action="store_true",
|
||||
help="restore; without it, only say what would change")
|
||||
args = parser.parse_args(argv)
|
||||
return asyncio.run(run(args.file))
|
||||
return asyncio.run(run(args.file, yes=args.yes))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user