bvandeusen/FabledScribe
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 50 Wiki Activity

Add implementation plans: sharing/collaboration and backup rewrite

Browse Source 
- docs/plans/2026-03-11-sharing-collaboration.md
  7 chunks / 12 tasks covering DB migration, models, access control service,
  groups CRUD, sharing service, notifications, and all frontend components

- docs/plans/2026-03-11-backup-rewrite.md
  4 tasks covering v2 export (full + user), v2 restore with FK re-mapping,
  and round-trip verification; v1 restore preserved

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bryan Van Deusen
2026-03-11 11:46:54 -04:00
parent 5d3f8683a4
commit de4ab53363
2 changed files with 2561 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/2026-03-11-backup-rewrite.md
+538
View File
@@ -0,0 +1,538 @@
# Backup Service Rewrite Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Rewrite `services/backup.py` so that full and user backup/restore correctly includes every model added since the original implementation (projects, milestones, task logs, drafts, versions), with correct FK re-mapping on restore.
**Architecture:** Bump backup JSON format to version 2. Export is additive — all new tables exported. Restore builds an ID map for each table and patches FK references in the correct dependency order. V1 backups continue to restore via the existing code path.
**Tech Stack:** Python/SQLAlchemy 2.0 async, no new dependencies.
**Spec:** `docs/superpowers/specs/2026-03-11-backup-rewrite-design.md`
**Dependency note:** This plan can be split into Part A (pre-sharing models) and Part B (sharing models). Part A is independent and should be done first.
---
## Task 1: Extend export — full backup
**Files:**
- Modify: `src/fabledassistant/services/backup.py`
- [ ] **Step 1: Read the current `export_full_backup()` function**
Understand what is currently exported: users, notes (partial fields), conversations+messages, settings.
- [ ] **Step 2: Rewrite `export_full_backup()` to version 2**
Add all missing models to the import list at the top of `backup.py`:
```python
from fabledassistant.models.project import Project
from fabledassistant.models.milestone import Milestone
from fabledassistant.models.task_log import TaskLog
from fabledassistant.models.note_draft import NoteDraft
from fabledassistant.models.note_version import NoteVersion
from fabledassistant.models.push_subscription import PushSubscription
```
Rewrite `export_full_backup()`:
```python
async def export_full_backup() -> dict:
async with async_session() as session:
users = (await session.execute(select(User))).scalars().all()
projects = (await session.execute(select(Project))).scalars().all()
milestones = (await session.execute(select(Milestone))).scalars().all()
notes = (await session.execute(select(Note))).scalars().all()
task_logs = (await session.execute(select(TaskLog))).scalars().all()
note_drafts = (await session.execute(select(NoteDraft))).scalars().all()
note_versions = (await session.execute(select(NoteVersion).order_by(NoteVersion.note_id, NoteVersion.version_number))).scalars().all()
conversations = (await session.execute(
select(Conversation).options(selectinload(Conversation.messages))
)).scalars().all()
settings = (await session.execute(select(Setting))).scalars().all()
push_subs = (await session.execute(select(PushSubscription))).scalars().all()
return {
"version": 2,
"scope": "full",
"exported_at": datetime.now(timezone.utc).isoformat(),
"_security_notice": (
"This backup contains hashed passwords and push subscription keys. "
"Store it securely and restrict access."
),
"users": [
{
"id": u.id,
"username": u.username,
"email": u.email,
"password_hash": u.password_hash,
"oauth_sub": u.oauth_sub,
"role": u.role,
"session_version": u.session_version,
"created_at": u.created_at.isoformat(),
}
for u in users
],
"projects": [
{
"id": p.id,
"user_id": p.user_id,
"title": p.title,
"description": p.description,
"goal": p.goal,
"status": p.status,
"color": p.color,
"created_at": p.created_at.isoformat(),
"updated_at": p.updated_at.isoformat(),
}
for p in projects
],
"milestones": [
{
"id": m.id,
"user_id": m.user_id,
"project_id": m.project_id,
"title": m.title,
"description": m.description,
"status": m.status,
"order_index": m.order_index,
"created_at": m.created_at.isoformat(),
"updated_at": m.updated_at.isoformat(),
}
for m in milestones
],
"notes": [
{
"id": n.id,
"user_id": n.user_id,
"title": n.title,
"body": n.body,
"tags": n.tags or [],
"parent_id": n.parent_id,
"project_id": n.project_id,
"milestone_id": n.milestone_id,
"is_task": n.is_task,
"status": n.status,
"priority": n.priority,
"due_date": n.due_date.isoformat() if n.due_date else None,
"is_starred": getattr(n, "is_starred", False),
"is_pinned": getattr(n, "is_pinned", False),
"created_at": n.created_at.isoformat(),
"updated_at": n.updated_at.isoformat(),
}
for n in notes
],
"task_logs": [
{
"id": tl.id,
"user_id": tl.user_id,
"note_id": tl.note_id,
"description": tl.description,
"duration_minutes": tl.duration_minutes,
"logged_at": tl.logged_at.isoformat() if tl.logged_at else None,
"created_at": tl.created_at.isoformat(),
}
for tl in task_logs
],
"note_drafts": [
{
"id": nd.id,
"user_id": nd.user_id,
"note_id": nd.note_id,
"title": nd.title,
"body": nd.body,
"saved_at": nd.saved_at.isoformat() if nd.saved_at else None,
}
for nd in note_drafts
],
"note_versions": [
{
"id": nv.id,
"user_id": nv.user_id,
"note_id": nv.note_id,
"title": nv.title,
"body": nv.body,
"version_number": nv.version_number,
"created_at": nv.created_at.isoformat(),
}
for nv in note_versions
],
"conversations": [
{
"id": c.id,
"user_id": c.user_id,
"title": c.title,
"created_at": c.created_at.isoformat(),
"updated_at": c.updated_at.isoformat(),
"messages": [
{
"id": m.id,
"role": m.role,
"content": m.content,
"context_note_id": m.context_note_id,
"created_at": m.created_at.isoformat(),
}
for m in c.messages
],
}
for c in conversations
],
"settings": [
{"user_id": s.user_id, "key": s.key, "value": s.value}
for s in settings
],
"push_subscriptions": [
{
"user_id": ps.user_id,
"endpoint": ps.endpoint,
"p256dh": ps.p256dh,
"auth": ps.auth,
"created_at": ps.created_at.isoformat(),
}
for ps in push_subs
],
}
```
Check what exact field names exist on each model before assuming. Use `getattr(obj, "field", default)` for fields that may not exist on older DB versions.
- [ ] **Step 3: Commit**
```bash
git add src/fabledassistant/services/backup.py
git commit -m "feat(backup): v2 full export with projects, milestones, task_logs, drafts, versions"
```
---
## Task 2: Extend export — user backup
**Files:**
- Modify: `src/fabledassistant/services/backup.py`
- [ ] **Step 1: Rewrite `export_user_backup(user_id)`**
Same structure as full backup but filtered to `WHERE user_id = uid`. Apply same field additions. Omit sensitive fields (password_hash, oauth_sub) from user self-export.
```python
async def export_user_backup(user_id: int) -> dict:
async with async_session() as session:
user = await session.get(User, user_id)
projects = (await session.execute(select(Project).where(Project.user_id == user_id))).scalars().all()
milestones = (await session.execute(select(Milestone).where(Milestone.user_id == user_id))).scalars().all()
notes = (await session.execute(select(Note).where(Note.user_id == user_id))).scalars().all()
task_logs = (await session.execute(select(TaskLog).where(TaskLog.user_id == user_id))).scalars().all()
note_drafts = (await session.execute(select(NoteDraft).where(NoteDraft.user_id == user_id))).scalars().all()
note_versions = (await session.execute(
select(NoteVersion).where(NoteVersion.user_id == user_id)
.order_by(NoteVersion.note_id, NoteVersion.version_number)
)).scalars().all()
conversations = (await session.execute(
select(Conversation).options(selectinload(Conversation.messages))
.where(Conversation.user_id == user_id)
)).scalars().all()
settings = (await session.execute(select(Setting).where(Setting.user_id == user_id))).scalars().all()
return {
"version": 2,
"scope": "user",
"exported_at": datetime.now(timezone.utc).isoformat(),
"user": {
"id": user.id,
"username": user.username,
"email": user.email,
"role": user.role,
"created_at": user.created_at.isoformat(),
} if user else None,
"projects": [...], # same as full but user-filtered
"milestones": [...],
"notes": [...],
"task_logs": [...],
"note_drafts": [...],
"note_versions": [...],
"conversations": [...],
"settings": [...],
}
```
- [ ] **Step 2: Commit**
```bash
git add src/fabledassistant/services/backup.py
git commit -m "feat(backup): v2 user backup with all models"
```
---
## Task 3: Rewrite restore for v2
**Files:**
- Modify: `src/fabledassistant/services/backup.py`
- [ ] **Step 1: Add version dispatch to `restore_full_backup`**
```python
async def restore_full_backup(data: dict) -> dict:
version = data.get("version", 1)
if version == 1:
return await _restore_v1(data)
return await _restore_v2(data)
```
Move existing restore code into `_restore_v1(data)` with no changes.
- [ ] **Step 2: Implement `_restore_v2(data)` with full FK re-mapping**
Restore order (respects FK dependencies):
1. Users → build `user_id_map`
2. Projects (fk: user_id) → build `project_id_map`
3. Milestones (fk: user_id, project_id) → build `milestone_id_map`
4. Notes — first pass: insert without `parent_id` (fk: user_id, project_id, milestone_id) → build `note_id_map`
5. Notes — second pass: patch `parent_id` using `note_id_map`
6. TaskLogs (fk: user_id, note_id via note_id_map)
7. NoteDrafts (fk: user_id, note_id via note_id_map)
8. NoteVersions (fk: user_id, note_id via note_id_map) — export only, no restore by default (skip unless flag set)
9. Conversations (fk: user_id) → Messages (fk: conversation_id, context_note_id via note_id_map)
10. Settings (fk: user_id)
```python
async def _restore_v2(data: dict) -> dict:
from datetime import date, datetime, timezone
stats = {
"users": 0, "projects": 0, "milestones": 0, "notes": 0,
"task_logs": 0, "note_drafts": 0, "conversations": 0,
"messages": 0, "settings": 0
}
async with async_session() as session:
user_id_map: dict[int, int] = {}
project_id_map: dict[int, int] = {}
milestone_id_map: dict[int, int] = {}
note_id_map: dict[int, int] = {}
# 1. Users
for u_data in data.get("users", []):
old_id = u_data["id"]
user = User(
username=u_data["username"],
email=u_data.get("email"),
password_hash=u_data.get("password_hash"),
oauth_sub=u_data.get("oauth_sub"),
role=u_data.get("role", "user"),
session_version=u_data.get("session_version", 1),
created_at=datetime.fromisoformat(u_data["created_at"]) if u_data.get("created_at") else datetime.now(timezone.utc),
)
session.add(user)
await session.flush()
user_id_map[old_id] = user.id
stats["users"] += 1
# 2. Projects
for p_data in data.get("projects", []):
mapped_uid = user_id_map.get(p_data.get("user_id", 0))
if mapped_uid is None:
continue
proj = Project(
user_id=mapped_uid,
title=p_data.get("title", ""),
description=p_data.get("description"),
goal=p_data.get("goal"),
status=p_data.get("status", "active"),
color=p_data.get("color"),
created_at=datetime.fromisoformat(p_data["created_at"]) if p_data.get("created_at") else datetime.now(timezone.utc),
updated_at=datetime.fromisoformat(p_data["updated_at"]) if p_data.get("updated_at") else datetime.now(timezone.utc),
)
session.add(proj)
await session.flush()
project_id_map[p_data["id"]] = proj.id
stats["projects"] += 1
# 3. Milestones
for m_data in data.get("milestones", []):
mapped_uid = user_id_map.get(m_data.get("user_id", 0))
mapped_pid = project_id_map.get(m_data.get("project_id", 0))
if mapped_uid is None:
continue
ms = Milestone(
user_id=mapped_uid,
project_id=mapped_pid,
title=m_data.get("title", ""),
description=m_data.get("description"),
status=m_data.get("status", "open"),
order_index=m_data.get("order_index", 0),
created_at=datetime.fromisoformat(m_data["created_at"]) if m_data.get("created_at") else datetime.now(timezone.utc),
updated_at=datetime.fromisoformat(m_data["updated_at"]) if m_data.get("updated_at") else datetime.now(timezone.utc),
)
session.add(ms)
await session.flush()
milestone_id_map[m_data["id"]] = ms.id
stats["milestones"] += 1
# 4. Notes — first pass (no parent_id)
note_objects: list[tuple[int, int]] = [] # (old_parent_id, new_note_id)
for n_data in data.get("notes", []):
mapped_uid = user_id_map.get(n_data.get("user_id", 0))
if mapped_uid is None:
continue
due = None
if n_data.get("due_date"):
due = date.fromisoformat(n_data["due_date"])
note = Note(
user_id=mapped_uid,
title=n_data.get("title", ""),
body=n_data.get("body", ""),
tags=n_data.get("tags", []),
parent_id=None, # patched in second pass
project_id=project_id_map.get(n_data.get("project_id")) if n_data.get("project_id") else None,
milestone_id=milestone_id_map.get(n_data.get("milestone_id")) if n_data.get("milestone_id") else None,
is_task=n_data.get("is_task", False),
status=n_data.get("status"),
priority=n_data.get("priority"),
due_date=due,
created_at=datetime.fromisoformat(n_data["created_at"]) if n_data.get("created_at") else datetime.now(timezone.utc),
updated_at=datetime.fromisoformat(n_data["updated_at"]) if n_data.get("updated_at") else datetime.now(timezone.utc),
)
session.add(note)
await session.flush()
note_id_map[n_data["id"]] = note.id
if n_data.get("parent_id"):
note_objects.append((n_data["parent_id"], note.id))
stats["notes"] += 1
# 5. Notes — second pass: patch parent_id
for old_parent_id, new_note_id in note_objects:
new_parent_id = note_id_map.get(old_parent_id)
if new_parent_id:
note_row = await session.get(Note, new_note_id)
if note_row:
note_row.parent_id = new_parent_id
# 6. TaskLogs
for tl_data in data.get("task_logs", []):
mapped_uid = user_id_map.get(tl_data.get("user_id", 0))
mapped_nid = note_id_map.get(tl_data.get("note_id", 0))
if mapped_uid is None or mapped_nid is None:
continue
from fabledassistant.models.task_log import TaskLog
tl = TaskLog(
user_id=mapped_uid,
note_id=mapped_nid,
description=tl_data.get("description", ""),
duration_minutes=tl_data.get("duration_minutes"),
logged_at=datetime.fromisoformat(tl_data["logged_at"]) if tl_data.get("logged_at") else None,
created_at=datetime.fromisoformat(tl_data["created_at"]) if tl_data.get("created_at") else datetime.now(timezone.utc),
)
session.add(tl)
stats["task_logs"] += 1
# 7. NoteDrafts
for nd_data in data.get("note_drafts", []):
mapped_uid = user_id_map.get(nd_data.get("user_id", 0))
mapped_nid = note_id_map.get(nd_data.get("note_id", 0))
if mapped_uid is None or mapped_nid is None:
continue
from fabledassistant.models.note_draft import NoteDraft
nd = NoteDraft(
user_id=mapped_uid,
note_id=mapped_nid,
title=nd_data.get("title", ""),
body=nd_data.get("body", ""),
saved_at=datetime.fromisoformat(nd_data["saved_at"]) if nd_data.get("saved_at") else None,
)
session.add(nd)
stats["note_drafts"] += 1
# 8. Conversations + Messages
for c_data in data.get("conversations", []):
mapped_uid = user_id_map.get(c_data.get("user_id", 0))
if mapped_uid is None:
continue
conv = Conversation(
user_id=mapped_uid,
title=c_data.get("title", ""),
created_at=datetime.fromisoformat(c_data["created_at"]) if c_data.get("created_at") else datetime.now(timezone.utc),
updated_at=datetime.fromisoformat(c_data["updated_at"]) if c_data.get("updated_at") else datetime.now(timezone.utc),
)
session.add(conv)
await session.flush()
stats["conversations"] += 1
for m_data in c_data.get("messages", []):
msg = Message(
conversation_id=conv.id,
role=m_data["role"],
content=m_data.get("content", ""),
context_note_id=note_id_map.get(m_data["context_note_id"]) if m_data.get("context_note_id") else None,
created_at=datetime.fromisoformat(m_data["created_at"]) if m_data.get("created_at") else datetime.now(timezone.utc),
)
session.add(msg)
stats["messages"] += 1
# 9. Settings
for s_data in data.get("settings", []):
mapped_uid = user_id_map.get(s_data.get("user_id", 0))
if mapped_uid is None:
continue
setting = Setting(user_id=mapped_uid, key=s_data["key"], value=s_data.get("value", ""))
session.add(setting)
stats["settings"] += 1
await session.commit()
logger.info("Restored v2 backup: %s", stats)
return stats
```
- [ ] **Step 3: Add missing imports at top of file**
```python
from datetime import datetime, timezone
from fabledassistant.models.project import Project
from fabledassistant.models.milestone import Milestone
```
- [ ] **Step 4: Test restore round-trip**
```bash
# Export
curl -s -b cookies.txt "http://localhost:5000/api/admin/backup?scope=full" -o /tmp/backup_v2.json
# Inspect structure
python3 -c "import json; d=json.load(open('/tmp/backup_v2.json')); print(list(d.keys()))"
# Expected keys: version, scope, exported_at, users, projects, milestones, notes, task_logs, ...
```
- [ ] **Step 5: Typecheck**
```bash
docker compose exec app python -m py_compile src/fabledassistant/services/backup.py
```
Expected: No errors.
- [ ] **Step 6: Commit**
```bash
git add src/fabledassistant/services/backup.py
git commit -m "feat(backup): v2 restore with full FK re-mapping; v1 restore preserved"
```
---
## Task 4: Verification
- [ ] **Step 1: Full round-trip test**
1. Export full backup as admin
2. Verify JSON has `"version": 2` and all expected top-level keys
3. Verify notes have `project_id`, `milestone_id`, `is_task` fields
4. Verify `task_logs` array has entries (create a task log entry first if needed)
5. Restore backup to a clean test instance (or verify restore code path runs without error in dry-run)
- [ ] **Step 2: V1 backward compat test**
Take an old v1 backup JSON (or construct one without the `version` key) and run restore. Verify it takes the v1 path and doesn't error.
- [ ] **Step 3: Commit final verification note**
```bash
git commit --allow-empty -m "chore(backup): v2 backup rewrite verified"
```
docs/plans/2026-03-11-sharing-collaboration.md
+2023
View File
File diff suppressed because it is too large Load Diff