thoughtsync/docs/sync.md

ThoughtSync sync protocol

The contract the local-first native clients (Tauri desktop, Android) implement against. The server is the sync hub: each client keeps a full local store (SQLite), works fully offline, and reconciles with the server when linked. The web app does not use this API — it stays on the live REST API (/api/notes, …) as an online-only stopgap.

All sync endpoints live under /api/sync. Everything is owner-scoped and deterministic (no AI).

No Postgres CI lane. Trigger/migration/sync behavior is verified by the operator on deploy, not in CI. Pure logic (LWW comparator, paging cursor, token hashing) is unit-tested.

Authentication — device bearer tokens

Native clients authenticate with a long-lived device token, not a session cookie. Only the token's SHA-256 hash is stored server-side; the plaintext is shown once at creation.

• First link (no session): POST /api/auth/device-login with {email, password, name} → {token, device, user}. Store token; send it as Authorization: Bearer <token> on every subsequent call.
• From the web app (already signed in): the user creates a token at /account ("Linked devices"); POST /api/auth/devices {name} → {token, device}. They paste it into the native app.
• Manage: GET /api/auth/devices (list), DELETE /api/auth/devices/<id> (revoke). A revoked token stops authenticating immediately.

Every authenticated request (sync or otherwise) accepts the bearer token in place of the session cookie.

The revision cursor

Every syncable row carries a monotonic sync_revision (bigint), assigned by a database trigger from a single shared sequence (sync_revision_seq) on every insert/update. Because it's a shared sequence, one integer is a total-order watermark across all of a user's notes and labels. It is not a timestamp — it is immune to clock skew and is only ever compared with >.

The client persists the highest cursor it has fully consumed and passes it back as ?since=. since=0 (or absent) is a full initial sync.

Entities and what syncs

• Note — the primary sync unit. It travels with its checklist items, label memberships, and attachment metadata inline (same shape as the REST serialization). A change to any child bumps the parent note's sync_revision, so pulling the note re-syncs the whole thing.
• Label — the label catalog (name + color) syncs as its own entity so a rename/recolor/delete propagates independently of notes.
• Derived, NOT synced: [[wiki-links]] and #tags are parsed from the note body. Clients recompute them locally; the server recomputes them on push. They never travel over the wire.
• Attachment blobs sync by id over the existing upload/download routes (see Attachments below); only their metadata rides the delta feed.

Tombstones (deletes)

Two levels, both propagate:

• Trash — deleted_at is a normal field. A trashed note still syncs with its content; the client shows it in its Trash. Restoring clears deleted_at.
• Purge (permanent delete) — becomes a content-less tombstone: purged_at is set, title/body/items/labels/attachments are cleared/removed, and the row is kept. A client seeing purged_at != null deletes the row from its local store. Tombstones are retained indefinitely (cheap for a personal store); revisit if they ever grow large.

Pull — GET /api/sync/changes

Query: ?since=<cursor>&limit=<n> (limit default 500, max 1000).

Response:

{
  "notes":  [ { "...full note...", "sync_revision": 42, "purged_at": null } ],
  "labels": [ { "id": "...", "name": "...", "color": "...",
               "sync_revision": 43, "purged_at": null, "created_at": "..." } ],
  "cursor": 43,
  "has_more": false
}

Returns all of the caller's notes + labels (any state — active, archived, trash, purged) whose sync_revision > since, ascending by revision. Notes and labels share the sequence, so paging merges the two streams: when either stream fills a page, cursor advances only to the smaller of the two page boundaries, so nothing between cursor and the next pull is skipped. Loop while has_more is true, advancing since = cursor each time.

Note attachment metadata carries {id, url, mime, size, sha256}.

Push — POST /api/sync/push

Body: { "changes": [ ... ] } (max 1000 per batch). Each change:

{ "entity": "note", "id": "<uuid>", "op": "upsert", "edited_at": "<iso8601>",
  "title": "...", "body": "...", "color": "blue", "kind": "text",
  "pinned": false, "archived": false, "trashed": false, "remind_at": null,
  "position": 0, "items": [ {"text": "...", "checked": false} ],
  "label_ids": ["<uuid>", ...], "created_at": "<iso8601, on create>" }

• Client-generated ids. Notes/labels are UUIDs; the client mints the id when it creates the row offline and sends it here. Create-if-absent, else update.
• Whole-note semantics. A note upsert carries the client's full current state (not a partial patch) — the server overwrites all scalar fields, replaces items, and sets manual label memberships from label_ids (tag-sourced labels are re-derived from the body). [[links]]/#tags are recomputed server-side.
• op: "delete" purges (tombstones) the row. Trashing is just an upsert with trashed: true.
• Labels: {entity: "label", op: "upsert"|"delete", id, edited_at, name, color}. A per-owner name clash on a different id is rejected (fix locally and retry).

Conflict resolution — last-write-wins + history

On a clash the most-recently-edited version wins, by edited_at (client wall-clock) compared against the server row's last-edit time (updated_at). The client applies iff client.edited_at >= server.updated_at. When a winning upsert overwrites an existing title/body, the server first snapshots the overwritten version into the note's version history (note_revisions) — so a "lost" edit is never truly lost; it's one Restore away. A stale delete loses to a newer server edit.

Response — per item, so the client can mark its local rows synced:

{ "results": [
  { "id": "...", "entity": "note", "status": "created", "sync_revision": 44 },
  { "id": "...", "entity": "note", "status": "kept",    "sync_revision": 40 },
  { "id": "...", "entity": "label","status": "rejected","error": "name in use" }
] }

status ∈ created | applied | kept | noop | rejected. kept means the server had a newer edit and the client should adopt the server version on its next pull.

Attachments (blobs)

Metadata rides the delta feed (id, url, mime, size, sha256); the bytes move over the existing routes:

• Upload: POST /api/notes/<note_id>/attachments (multipart, field file; optional field id to keep a client-minted attachment id). Re-uploading an id the note already has is an idempotent no-op. Server stores + hashes the bytes.
• Download: GET /api/notes/<note_id>/attachments/<id> (owner/shared scoped).
• The client uses sha256 to skip blobs it already holds and to verify integrity after download. (Currently image mimes only; broadening to any file is tracked separately.)

A sync cycle

1. Push local changes since the last sync (batched). Apply the per-item results (mark synced, adopt server version where kept).
2. Pull from the stored since cursor until has_more is false. Upsert notes/labels into the local store; delete rows whose purged_at is set; download any attachment blobs referenced by a new/changed sha256.
3. Persist the new cursor.

Initial sync is the same with since=0. Because everything is keyed by stable ids and a monotonic cursor, the cycle is idempotent and resumable — a client can crash mid-sync and simply resume from its last persisted cursor.