diff --git a/docs/sync.md b/docs/sync.md new file mode 100644 index 0000000..5d0e965 --- /dev/null +++ b/docs/sync.md @@ -0,0 +1,167 @@ +# ThoughtSync sync protocol + +The contract the local-first native clients (Tauri desktop, Android) implement +against. The server is the **sync hub**: each client keeps a full local store +(SQLite), works fully offline, and reconciles with the server when linked. The +web app does **not** use this API — it stays on the live REST API (`/api/notes`, +…) as an online-only stopgap. + +All sync endpoints live under `/api/sync`. Everything is **owner-scoped** and +**deterministic** (no AI). + +> **No Postgres CI lane.** Trigger/migration/sync behavior is verified by the +> operator on deploy, not in CI. Pure logic (LWW comparator, paging cursor, +> token hashing) is unit-tested. + +## Authentication — device bearer tokens + +Native clients authenticate with a long-lived **device token**, not a session +cookie. Only the token's SHA-256 hash is stored server-side; the plaintext is +shown once at creation. + +- **First link (no session):** `POST /api/auth/device-login` with + `{email, password, name}` → `{token, device, user}`. Store `token`; send it as + `Authorization: Bearer ` on every subsequent call. +- **From the web app (already signed in):** the user creates a token at + `/account` ("Linked devices"); `POST /api/auth/devices` `{name}` → `{token, + device}`. They paste it into the native app. +- **Manage:** `GET /api/auth/devices` (list), `DELETE /api/auth/devices/` + (revoke). A revoked token stops authenticating immediately. + +Every authenticated request (sync or otherwise) accepts the bearer token in +place of the session cookie. + +## The revision cursor + +Every syncable row carries a monotonic **`sync_revision`** (bigint), assigned by +a database trigger from a single shared sequence (`sync_revision_seq`) on every +insert/update. Because it's a shared sequence, one integer is a total-order +watermark across **all** of a user's notes and labels. It is not a timestamp — +it is immune to clock skew and is only ever compared with `>`. + +The client persists the highest cursor it has fully consumed and passes it back +as `?since=`. `since=0` (or absent) is a **full initial sync**. + +## Entities and what syncs + +- **Note** — the primary sync unit. It travels with its **checklist items, + label memberships, and attachment metadata inline** (same shape as the REST + serialization). A change to any child bumps the parent note's `sync_revision`, + so pulling the note re-syncs the whole thing. +- **Label** — the label *catalog* (name + color) syncs as its own entity so a + rename/recolor/delete propagates independently of notes. +- **Derived, NOT synced:** `[[wiki-links]]` and `#tags` are parsed from the note + body. Clients recompute them locally; the server recomputes them on push. They + never travel over the wire. +- **Attachment blobs** sync by id over the existing upload/download routes (see + Attachments below); only their metadata rides the delta feed. + +## Tombstones (deletes) + +Two levels, both propagate: + +- **Trash** — `deleted_at` is a normal field. A trashed note still syncs with its + content; the client shows it in its Trash. Restoring clears `deleted_at`. +- **Purge (permanent delete)** — becomes a **content-less tombstone**: `purged_at` + is set, title/body/items/labels/attachments are cleared/removed, and the row is + kept. A client seeing `purged_at != null` deletes the row from its local store. + Tombstones are retained indefinitely (cheap for a personal store); revisit if + they ever grow large. + +## Pull — `GET /api/sync/changes` + +Query: `?since=&limit=` (limit default 500, max 1000). + +Response: + +```json +{ + "notes": [ { "...full note...", "sync_revision": 42, "purged_at": null } ], + "labels": [ { "id": "...", "name": "...", "color": "...", + "sync_revision": 43, "purged_at": null, "created_at": "..." } ], + "cursor": 43, + "has_more": false +} +``` + +Returns **all** of the caller's notes + labels (any state — active, archived, +trash, purged) whose `sync_revision > since`, ascending by revision. Notes and +labels share the sequence, so paging merges the two streams: when either stream +fills a page, `cursor` advances only to the **smaller** of the two page +boundaries, so nothing between `cursor` and the next pull is skipped. Loop while +`has_more` is true, advancing `since = cursor` each time. + +Note attachment metadata carries `{id, url, mime, size, sha256}`. + +## Push — `POST /api/sync/push` + +Body: `{ "changes": [ ... ] }` (max 1000 per batch). Each change: + +```json +{ "entity": "note", "id": "", "op": "upsert", "edited_at": "", + "title": "...", "body": "...", "color": "blue", "kind": "text", + "pinned": false, "archived": false, "trashed": false, "remind_at": null, + "position": 0, "items": [ {"text": "...", "checked": false} ], + "label_ids": ["", ...], "created_at": "" } +``` + +- **Client-generated ids.** Notes/labels are UUIDs; the client mints the id when + it creates the row offline and sends it here. Create-if-absent, else update. +- **Whole-note semantics.** A note upsert carries the client's *full* current + state (not a partial patch) — the server overwrites all scalar fields, replaces + items, and sets manual label memberships from `label_ids` (tag-sourced labels + are re-derived from the body). `[[links]]`/`#tags` are recomputed server-side. +- **`op: "delete"`** purges (tombstones) the row. Trashing is just an upsert with + `trashed: true`. +- **Labels:** `{entity: "label", op: "upsert"|"delete", id, edited_at, name, + color}`. A per-owner name clash on a *different* id is rejected (fix locally + and retry). + +### Conflict resolution — last-write-wins + history + +On a clash the **most-recently-edited version wins**, by `edited_at` +(client wall-clock) compared against the server row's last-edit time +(`updated_at`). The client applies iff `client.edited_at >= server.updated_at`. +When a winning upsert overwrites an existing title/body, the server first +**snapshots the overwritten version into the note's version history** +(`note_revisions`) — so a "lost" edit is never truly lost; it's one Restore away. +A stale delete loses to a newer server edit. + +Response — per item, so the client can mark its local rows synced: + +```json +{ "results": [ + { "id": "...", "entity": "note", "status": "created", "sync_revision": 44 }, + { "id": "...", "entity": "note", "status": "kept", "sync_revision": 40 }, + { "id": "...", "entity": "label","status": "rejected","error": "name in use" } +] } +``` + +`status` ∈ `created | applied | kept | noop | rejected`. `kept` means the server +had a newer edit and the client should adopt the server version on its next pull. + +## Attachments (blobs) + +Metadata rides the delta feed (`id, url, mime, size, sha256`); the bytes move +over the existing routes: + +- **Upload:** `POST /api/notes//attachments` (multipart, field `file`; + optional field `id` to keep a client-minted attachment id). Re-uploading an id + the note already has is an idempotent no-op. Server stores + hashes the bytes. +- **Download:** `GET /api/notes//attachments/` (owner/shared scoped). +- The client uses `sha256` to skip blobs it already holds and to verify + integrity after download. (Currently image mimes only; broadening to any file + is tracked separately.) + +## A sync cycle + +1. **Push** local changes since the last sync (batched). Apply the per-item + results (mark synced, adopt server version where `kept`). +2. **Pull** from the stored `since` cursor until `has_more` is false. Upsert + notes/labels into the local store; delete rows whose `purged_at` is set; + download any attachment blobs referenced by a new/changed `sha256`. +3. Persist the new `cursor`. + +Initial sync is the same with `since=0`. Because everything is keyed by stable +ids and a monotonic cursor, the cycle is **idempotent and resumable** — a client +can crash mid-sync and simply resume from its last persisted cursor.