Sync 6: sync protocol doc (docs/sync.md)
The contract the Tauri/Android clients implement against: device-token auth, the shared-sequence revision cursor, note-as-sync-unit (+ derived links/tags not synced), trash vs purge tombstones, pull (GET /changes) + push (POST /push) request/response shapes, last-write-wins + history conflict policy, attachment blob sync by id + sha256, and the idempotent/resumable sync cycle (initial since=0 + resume). Docs only — CI paths exclude *.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRgehjoz7Yv8LkUfADxACm
This commit is contained in:
Claude Opus 4.8
+167
@@ -0,0 +1,167 @@
|
||||
# ThoughtSync sync protocol
|
||||
|
||||
The contract the local-first native clients (Tauri desktop, Android) implement
|
||||
against. The server is the **sync hub**: each client keeps a full local store
|
||||
(SQLite), works fully offline, and reconciles with the server when linked. The
|
||||
web app does **not** use this API — it stays on the live REST API (`/api/notes`,
|
||||
…) as an online-only stopgap.
|
||||
|
||||
All sync endpoints live under `/api/sync`. Everything is **owner-scoped** and
|
||||
**deterministic** (no AI).
|
||||
|
||||
> **No Postgres CI lane.** Trigger/migration/sync behavior is verified by the
|
||||
> operator on deploy, not in CI. Pure logic (LWW comparator, paging cursor,
|
||||
> token hashing) is unit-tested.
|
||||
|
||||
## Authentication — device bearer tokens
|
||||
|
||||
Native clients authenticate with a long-lived **device token**, not a session
|
||||
cookie. Only the token's SHA-256 hash is stored server-side; the plaintext is
|
||||
shown once at creation.
|
||||
|
||||
- **First link (no session):** `POST /api/auth/device-login` with
|
||||
`{email, password, name}` → `{token, device, user}`. Store `token`; send it as
|
||||
`Authorization: Bearer <token>` on every subsequent call.
|
||||
- **From the web app (already signed in):** the user creates a token at
|
||||
`/account` ("Linked devices"); `POST /api/auth/devices` `{name}` → `{token,
|
||||
device}`. They paste it into the native app.
|
||||
- **Manage:** `GET /api/auth/devices` (list), `DELETE /api/auth/devices/<id>`
|
||||
(revoke). A revoked token stops authenticating immediately.
|
||||
|
||||
Every authenticated request (sync or otherwise) accepts the bearer token in
|
||||
place of the session cookie.
|
||||
|
||||
## The revision cursor
|
||||
|
||||
Every syncable row carries a monotonic **`sync_revision`** (bigint), assigned by
|
||||
a database trigger from a single shared sequence (`sync_revision_seq`) on every
|
||||
insert/update. Because it's a shared sequence, one integer is a total-order
|
||||
watermark across **all** of a user's notes and labels. It is not a timestamp —
|
||||
it is immune to clock skew and is only ever compared with `>`.
|
||||
|
||||
The client persists the highest cursor it has fully consumed and passes it back
|
||||
as `?since=`. `since=0` (or absent) is a **full initial sync**.
|
||||
|
||||
## Entities and what syncs
|
||||
|
||||
- **Note** — the primary sync unit. It travels with its **checklist items,
|
||||
label memberships, and attachment metadata inline** (same shape as the REST
|
||||
serialization). A change to any child bumps the parent note's `sync_revision`,
|
||||
so pulling the note re-syncs the whole thing.
|
||||
- **Label** — the label *catalog* (name + color) syncs as its own entity so a
|
||||
rename/recolor/delete propagates independently of notes.
|
||||
- **Derived, NOT synced:** `[[wiki-links]]` and `#tags` are parsed from the note
|
||||
body. Clients recompute them locally; the server recomputes them on push. They
|
||||
never travel over the wire.
|
||||
- **Attachment blobs** sync by id over the existing upload/download routes (see
|
||||
Attachments below); only their metadata rides the delta feed.
|
||||
|
||||
## Tombstones (deletes)
|
||||
|
||||
Two levels, both propagate:
|
||||
|
||||
- **Trash** — `deleted_at` is a normal field. A trashed note still syncs with its
|
||||
content; the client shows it in its Trash. Restoring clears `deleted_at`.
|
||||
- **Purge (permanent delete)** — becomes a **content-less tombstone**: `purged_at`
|
||||
is set, title/body/items/labels/attachments are cleared/removed, and the row is
|
||||
kept. A client seeing `purged_at != null` deletes the row from its local store.
|
||||
Tombstones are retained indefinitely (cheap for a personal store); revisit if
|
||||
they ever grow large.
|
||||
|
||||
## Pull — `GET /api/sync/changes`
|
||||
|
||||
Query: `?since=<cursor>&limit=<n>` (limit default 500, max 1000).
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"notes": [ { "...full note...", "sync_revision": 42, "purged_at": null } ],
|
||||
"labels": [ { "id": "...", "name": "...", "color": "...",
|
||||
"sync_revision": 43, "purged_at": null, "created_at": "..." } ],
|
||||
"cursor": 43,
|
||||
"has_more": false
|
||||
}
|
||||
```
|
||||
|
||||
Returns **all** of the caller's notes + labels (any state — active, archived,
|
||||
trash, purged) whose `sync_revision > since`, ascending by revision. Notes and
|
||||
labels share the sequence, so paging merges the two streams: when either stream
|
||||
fills a page, `cursor` advances only to the **smaller** of the two page
|
||||
boundaries, so nothing between `cursor` and the next pull is skipped. Loop while
|
||||
`has_more` is true, advancing `since = cursor` each time.
|
||||
|
||||
Note attachment metadata carries `{id, url, mime, size, sha256}`.
|
||||
|
||||
## Push — `POST /api/sync/push`
|
||||
|
||||
Body: `{ "changes": [ ... ] }` (max 1000 per batch). Each change:
|
||||
|
||||
```json
|
||||
{ "entity": "note", "id": "<uuid>", "op": "upsert", "edited_at": "<iso8601>",
|
||||
"title": "...", "body": "...", "color": "blue", "kind": "text",
|
||||
"pinned": false, "archived": false, "trashed": false, "remind_at": null,
|
||||
"position": 0, "items": [ {"text": "...", "checked": false} ],
|
||||
"label_ids": ["<uuid>", ...], "created_at": "<iso8601, on create>" }
|
||||
```
|
||||
|
||||
- **Client-generated ids.** Notes/labels are UUIDs; the client mints the id when
|
||||
it creates the row offline and sends it here. Create-if-absent, else update.
|
||||
- **Whole-note semantics.** A note upsert carries the client's *full* current
|
||||
state (not a partial patch) — the server overwrites all scalar fields, replaces
|
||||
items, and sets manual label memberships from `label_ids` (tag-sourced labels
|
||||
are re-derived from the body). `[[links]]`/`#tags` are recomputed server-side.
|
||||
- **`op: "delete"`** purges (tombstones) the row. Trashing is just an upsert with
|
||||
`trashed: true`.
|
||||
- **Labels:** `{entity: "label", op: "upsert"|"delete", id, edited_at, name,
|
||||
color}`. A per-owner name clash on a *different* id is rejected (fix locally
|
||||
and retry).
|
||||
|
||||
### Conflict resolution — last-write-wins + history
|
||||
|
||||
On a clash the **most-recently-edited version wins**, by `edited_at`
|
||||
(client wall-clock) compared against the server row's last-edit time
|
||||
(`updated_at`). The client applies iff `client.edited_at >= server.updated_at`.
|
||||
When a winning upsert overwrites an existing title/body, the server first
|
||||
**snapshots the overwritten version into the note's version history**
|
||||
(`note_revisions`) — so a "lost" edit is never truly lost; it's one Restore away.
|
||||
A stale delete loses to a newer server edit.
|
||||
|
||||
Response — per item, so the client can mark its local rows synced:
|
||||
|
||||
```json
|
||||
{ "results": [
|
||||
{ "id": "...", "entity": "note", "status": "created", "sync_revision": 44 },
|
||||
{ "id": "...", "entity": "note", "status": "kept", "sync_revision": 40 },
|
||||
{ "id": "...", "entity": "label","status": "rejected","error": "name in use" }
|
||||
] }
|
||||
```
|
||||
|
||||
`status` ∈ `created | applied | kept | noop | rejected`. `kept` means the server
|
||||
had a newer edit and the client should adopt the server version on its next pull.
|
||||
|
||||
## Attachments (blobs)
|
||||
|
||||
Metadata rides the delta feed (`id, url, mime, size, sha256`); the bytes move
|
||||
over the existing routes:
|
||||
|
||||
- **Upload:** `POST /api/notes/<note_id>/attachments` (multipart, field `file`;
|
||||
optional field `id` to keep a client-minted attachment id). Re-uploading an id
|
||||
the note already has is an idempotent no-op. Server stores + hashes the bytes.
|
||||
- **Download:** `GET /api/notes/<note_id>/attachments/<id>` (owner/shared scoped).
|
||||
- The client uses `sha256` to skip blobs it already holds and to verify
|
||||
integrity after download. (Currently image mimes only; broadening to any file
|
||||
is tracked separately.)
|
||||
|
||||
## A sync cycle
|
||||
|
||||
1. **Push** local changes since the last sync (batched). Apply the per-item
|
||||
results (mark synced, adopt server version where `kept`).
|
||||
2. **Pull** from the stored `since` cursor until `has_more` is false. Upsert
|
||||
notes/labels into the local store; delete rows whose `purged_at` is set;
|
||||
download any attachment blobs referenced by a new/changed `sha256`.
|
||||
3. Persist the new `cursor`.
|
||||
|
||||
Initial sync is the same with `since=0`. Because everything is keyed by stable
|
||||
ids and a monotonic cursor, the cycle is **idempotent and resumable** — a client
|
||||
can crash mid-sync and simply resume from its last persisted cursor.
|
||||
Reference in New Issue
Block a user