core: extract the store and sync engine into a shared crate (M12 step 1)
Desktop (Tauri) / Tauri desktop (Linux) (push) Failing after 48s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 1m50s
Desktop (Tauri) / Update manifest (push) Skipped

Android becomes a native Kotlin client over this same code (Scribe note 2730), so
the local store and sync engine stop being modules of the desktop app and become
`thoughtsync-core`, a crate with no UI framework in it at all.

This is a move, not a rewrite, and the measurement is why: every file in local/
and sync/ already carried ZERO Tauri references — 4,980 of 6,372 lines. The
coupling was 473 lines of command shim, which stays behind in the desktop crate
as src/commands/. Kept as git renames so history follows the files.

The desktop imports them under their old names (`use thoughtsync_core::{local,
sync}`) so every call site reads exactly as before. What moved is where they
live, not what they are.

Two things a workspace changes that are easy to miss, both caught before pushing:

[profile.release] now lives at the workspace ROOT. Cargo silently ignores
profiles declared by a non-root member — leaving it in the desktop crate would
have dropped lto/strip/opt-level from every release build with only a warning.

And a workspace shares ONE target dir, so the bundles moved from
desktop/src-tauri/target to target/. Thirteen references across publish-release,
debundle-graphics, verify.sh, package-prebuilt and the workflow now point there.
Pinning target-dir back would have been the smaller diff, but the Android lane
also produces Rust artifacts and they do not belong under desktop/.

Also retires the Tauri Android lane in the same push rather than leaving a path
that is being replaced: gen/android, android.yml and docs/android-dev.md are
gone, the mobile_entry_point attribute with them, and the lib drops to rlib —
staticlib/cdylib existed for Tauri mobile, and the .so Android loads will be
built from the core crate instead. Rule 22, no parallel path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-17 23:12:26 -04:00
co-authored by Claude Opus 5
parent c28f2bc00e
commit 0a7480cf9b
75 changed files with 246 additions and 1346 deletions
+318
View File
@@ -0,0 +1,318 @@
//! Local storage for attachment bytes (M10.7d).
//!
//! Content-addressed: a blob is filed under its own sha256, so the same image
//! attached to five notes is stored once and re-downloading it is free. The hash is
//! also the integrity check — bytes that don't hash to what the server advertised
//! are refused rather than filed under a name that lies about them.
//!
//! Attachment METADATA rides the delta feed; only the bytes come through here
//! (docs/sync.md).
use std::fs;
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// A sha256 in lowercase hex, and nothing else.
///
/// This is a **path-safety** check, not a formatting nicety: the hash is taken
/// straight from a server response and used as a filename. Without it, a hostile or
/// buggy server could send `../../…` and steer a write outside the blob directory.
fn is_hash(candidate: &str) -> bool {
candidate.len() == 64 && candidate.bytes().all(|b| b.is_ascii_hexdigit())
}
fn digest(bytes: &[u8]) -> String {
use sha2::{Digest, Sha256};
let mut hasher = Sha256::new();
hasher.update(bytes);
hasher
.finalize()
.iter()
.map(|b| format!("{b:02x}"))
.collect()
}
pub struct BlobStore {
root: PathBuf,
}
impl BlobStore {
/// Open (creating if needed) the blob directory.
pub fn new(root: PathBuf) -> std::io::Result<Self> {
fs::create_dir_all(&root)?;
Ok(Self { root })
}
pub fn root(&self) -> &Path {
&self.root
}
/// Where a blob lives, or `None` if the hash isn't one.
pub fn path(&self, sha256: &str) -> Option<PathBuf> {
let lower = sha256.to_ascii_lowercase();
is_hash(&lower).then(|| self.root.join(lower))
}
/// Whether we already hold these bytes. Drives the "don't download it twice"
/// skip, which is the entire point of keying by content.
pub fn has(&self, sha256: &str) -> bool {
self.path(sha256).is_some_and(|p| p.is_file())
}
/// File bytes under `expected`, refusing them if they don't hash to it.
///
/// Verifying on the way IN rather than on the way out means a corrupted transfer
/// can never be served later as if it were genuine — and the next sync simply
/// tries again, because the blob still counts as missing.
pub fn store(&self, expected: &str, bytes: &[u8]) -> Result<PathBuf, String> {
let path = self
.path(expected)
.ok_or_else(|| format!("refusing an attachment with a malformed hash: {expected}"))?;
let actual = digest(bytes);
if actual != expected.to_ascii_lowercase() {
return Err(format!(
"attachment failed its integrity check (expected {expected}, got {actual})"
));
}
fs::write(&path, bytes).map_err(|e| format!("couldn't save an attachment: {e}"))?;
Ok(path)
}
pub fn read(&self, sha256: &str) -> Option<Vec<u8>> {
fs::read(self.path(sha256)?).ok()
}
}
// --- Serving blobs to the webview (M10.7f) -----------------------------------
//
// A synced note's attachment `url` is the SERVER's relative path
// (`/api/notes/<id>/attachments/<aid>`). In the desktop webview that resolves
// against the app origin and 404s, and swapping in the absolute server URL wouldn't
// help either — that route needs a bearer token the webview won't send, and it would
// make an offline app fetch over the network to show a file it already has on disk.
//
// So the bytes are served locally, over a custom URI scheme, straight out of this
// store. The webview then caches and range-requests them like any other resource,
// which a `data:` URI would have thrown away.
/// The scheme the webview fetches attachment bytes over.
pub const BLOB_SCHEME: &str = "tsblob";
/// The blob directory, published once the app has resolved its data dir.
///
/// A `OnceLock` rather than Tauri's managed state because the scheme handler is
/// registered on the BUILDER, before `setup` has computed that path — and because
/// reading it this way keeps the handler independent of which Tauri 2.x minor
/// changed the handler's context argument.
static SERVE_ROOT: OnceLock<PathBuf> = OnceLock::new();
pub fn publish_root(root: PathBuf) {
let _ = SERVE_ROOT.set(root);
}
/// The URL an `<img>`/`<audio>`/`<a href>` should point at for these bytes.
///
/// **The two forms are not interchangeable.** A custom scheme is reachable as
/// `scheme://localhost/<path>` on Linux and macOS, but Windows and Android map it
/// onto `http://scheme.localhost/<path>`. Getting this wrong breaks exactly one
/// platform, silently, and CI cannot catch it — the runner is headless.
pub fn url_for(sha256: &str, mime: &str) -> String {
let query = urlencode(mime);
if cfg!(any(windows, target_os = "android")) {
format!("http://{BLOB_SCHEME}.localhost/{sha256}?mime={query}")
} else {
format!("{BLOB_SCHEME}://localhost/{sha256}?mime={query}")
}
}
/// Percent-encode the few characters a mime type can contain that don't belong in a
/// query value. Hand-rolled rather than adding a dependency for `/` and `+`.
fn urlencode(value: &str) -> String {
let mut out = String::with_capacity(value.len() + 8);
for b in value.bytes() {
match b {
b'a'..=b'z' | b'A'..=b'Z' | b'0'..=b'9' | b'-' | b'.' | b'_' | b'~' => {
out.push(b as char)
}
_ => out.push_str(&format!("%{b:02X}")),
}
}
out
}
fn urldecode(value: &str) -> String {
let bytes = value.as_bytes();
let mut out: Vec<u8> = Vec::with_capacity(bytes.len());
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'%' && i + 2 < bytes.len() {
let hex = std::str::from_utf8(&bytes[i + 1..i + 3]).unwrap_or("");
if let Ok(byte) = u8::from_str_radix(hex, 16) {
out.push(byte);
i += 3;
continue;
}
}
out.push(bytes[i]);
i += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
/// The Content-Type to serve for a claimed mime.
///
/// The mime rides in the URL and this scheme is an origin of its own, so echoing an
/// arbitrary type would let an attachment claiming `text/html` run as a document
/// there. Echoing is safe only because of the FAMILY check: nothing starting with
/// `image/` can name a scriptable type. Everything else is served as an opaque
/// download — the right treatment for an arbitrary file regardless.
fn content_type_for(mime: &str) -> String {
const RENDERABLE: &[&str] = &["image/", "audio/", "video/"];
let familiar = RENDERABLE.iter().any(|p| mime.starts_with(p)) || mime == "application/pdf";
// A header value can't carry control characters, and a mime type has no business
// being long — both would only arrive from a malformed or hostile feed.
let printable = mime.len() <= 100 && mime.bytes().all(|b| b.is_ascii_graphic());
if familiar && printable {
mime.to_string()
} else {
"application/octet-stream".to_string()
}
}
/// Serve one request from the blob store. `path` is the URI path, `query` its query.
pub fn serve(path: &str, query: Option<&str>) -> (u16, String, Vec<u8>) {
let requested = path.trim_start_matches('/');
let Some(root) = SERVE_ROOT.get() else {
// A request before the store was published — nothing to serve yet.
return (503, "text/plain".into(), Vec::new());
};
let store = BlobStore { root: root.clone() };
// `read` goes through `path`, which rejects anything that isn't a bare sha256 —
// so this handler inherits the traversal guard rather than re-implementing it.
let Some(bytes) = store.read(requested) else {
return (404, "text/plain".into(), Vec::new());
};
let claimed = query
.and_then(|q| q.split('&').find_map(|p| p.strip_prefix("mime=")))
.map(urldecode)
.unwrap_or_default();
(200, content_type_for(&claimed), bytes)
}
#[cfg(test)]
mod tests {
use super::*;
/// A blob store in a throwaway directory. No tempfile dependency for one test
/// fixture — the process id keeps concurrent runs apart.
fn store(tag: &str) -> BlobStore {
let dir = std::env::temp_dir().join(format!("ts-blobs-{}-{tag}", std::process::id()));
let _ = fs::remove_dir_all(&dir);
BlobStore::new(dir).expect("store")
}
/// sha256("hello") — a fixed vector, so a broken digest can't agree with itself.
const HELLO: &str = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824";
#[test]
fn digest_matches_a_known_vector() {
assert_eq!(digest(b"hello"), HELLO);
}
#[test]
fn stores_and_reads_back() {
let store = store("roundtrip");
assert!(!store.has(HELLO));
store.store(HELLO, b"hello").expect("store");
assert!(store.has(HELLO));
assert_eq!(store.read(HELLO).as_deref(), Some(&b"hello"[..]));
}
#[test]
fn refuses_bytes_that_dont_match_the_hash() {
// A corrupted or substituted transfer must never be filed under a name that
// claims it's genuine.
let store = store("mismatch");
let err = store.store(HELLO, b"goodbye").expect_err("must reject");
assert!(err.contains("integrity"), "got {err}");
assert!(!store.has(HELLO), "nothing should have been written");
}
#[test]
fn rejects_a_hash_that_could_escape_the_directory() {
// The hash arrives from a server response and becomes a filename.
let store = store("traversal");
assert!(store.path("../../etc/passwd").is_none());
assert!(store.store("../../etc/passwd", b"x").is_err());
assert!(store.path("").is_none());
assert!(store.path("nothex!!").is_none());
}
#[test]
fn accepts_an_uppercase_hash() {
// The wire format isn't guaranteed to be lowercase; the filename is.
let store = store("case");
store
.store(&HELLO.to_ascii_uppercase(), b"hello")
.expect("store");
assert!(store.has(HELLO), "should be found under the lowercase name");
}
#[test]
fn the_blob_url_carries_the_hash_and_the_mime() {
let url = url_for(HELLO, "image/png");
assert!(url.contains(HELLO), "the hash addresses the bytes: {url}");
assert!(url.contains("mime=image%2Fpng"), "mime encoded: {url}");
// The platform split is the whole risk of this feature, and CI is headless,
// so at least pin that the right branch was taken for THIS build.
if cfg!(any(windows, target_os = "android")) {
assert!(url.starts_with("http://tsblob.localhost/"), "{url}");
} else {
assert!(url.starts_with("tsblob://localhost/"), "{url}");
}
}
#[test]
fn url_encoding_round_trips_a_mime() {
assert_eq!(urldecode(&urlencode("image/svg+xml")), "image/svg+xml");
assert_eq!(urldecode(&urlencode("audio/mpeg")), "audio/mpeg");
// A malformed escape is left alone rather than eaten — the value still has to
// survive intact enough for `content_type_for` to reject it.
assert_eq!(urldecode("not-an-escape%ZZ"), "not-an-escape%ZZ");
}
#[test]
fn media_types_are_echoed_back() {
assert_eq!(content_type_for("image/png"), "image/png");
assert_eq!(content_type_for("audio/mpeg"), "audio/mpeg");
assert_eq!(content_type_for("application/pdf"), "application/pdf");
}
#[test]
fn a_scriptable_type_is_served_as_a_download() {
// This scheme is an origin of its own. An attachment claiming to be HTML
// must not be handed back as a document that can run there.
let opaque = "application/octet-stream";
assert_eq!(content_type_for("text/html"), opaque);
assert_eq!(content_type_for("application/javascript"), opaque);
assert_eq!(content_type_for(""), opaque);
// A control character can't reach a header value even under a safe family.
assert_eq!(content_type_for("image/png\r\nX-Evil: 1"), opaque);
}
#[test]
fn serving_refuses_a_path_that_isnt_a_hash() {
// Delegated to `path`, so the traversal guard is the same one `store` uses.
publish_root(std::env::temp_dir().join("ts-blobs-serve-guard"));
let (status, _, body) = serve("/../../etc/passwd", None);
assert_eq!(status, 404);
assert!(body.is_empty());
}
#[test]
fn missing_blob_reads_as_none() {
let store = store("missing");
assert!(store.read(HELLO).is_none());
assert!(!store.has(HELLO));
}
}
+433
View File
@@ -0,0 +1,433 @@
//! HTTP transport to a ThoughtSync server.
//!
//! Covers the compatibility handshake (M10.6) and device-token auth (M10.7a). The
//! engine that moves notes — push, pull, cursor — grows on top of the same client,
//! which is why the timeout, identity headers and error vocabulary live here rather
//! than inline at each call site.
//!
//! Nothing here runs unless the user has linked a server; the app is local-first and
//! fully usable with no network at all.
use std::time::Duration;
use reqwest::{RequestBuilder, StatusCode};
use serde::{Deserialize, Serialize};
use super::compat::{self, Compatibility, ServerInfo};
use super::wire;
/// Timeout for the short request/response calls in this module. Kept tight because a
/// user is watching a button while they run, and the most common mistake — a wrong
/// host on a LAN — fails by hanging rather than refusing, so an unbounded wait would
/// just look frozen. The sync engine's bulk transfers will need their own, longer one.
const REQUEST_TIMEOUT: Duration = Duration::from_secs(10);
/// Bulk transfers get much longer: a first full sync can be thousands of notes, and
/// failing one at ten seconds would make a large store impossible to ever pull.
const SYNC_TIMEOUT: Duration = Duration::from_secs(120);
/// Shared by every call that presents a token, so a revoked one reads the same way
/// wherever it surfaces.
const TOKEN_REJECTED: &str = "This server rejected the device token — it may have been \
revoked. Unlink and link again to issue a new one.";
/// What the link UI needs after a handshake: where we ended up (the normalized URL,
/// which may differ from what was typed), who answered, and whether we can work
/// with them.
#[derive(Debug, Serialize)]
pub struct ProbeResult {
pub base_url: String,
pub server: ServerInfo,
pub compatibility: Compatibility,
}
/// The account a device token belongs to. Surfaced after linking so the user can
/// confirm they linked the account they meant to — easy to get wrong on a server
/// hosting more than one.
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct Identity {
pub id: String,
pub email: String,
#[serde(default)]
pub display_name: String,
}
#[derive(Deserialize)]
struct DeviceLoginResponse {
token: String,
user: Identity,
}
/// What became of this device's token on the SERVER when unlinking.
///
/// Not a bool, and not an error: unlinking must never be blocked by the network —
/// wanting to stop syncing is a local decision — so the remote half reports back
/// instead of failing the call, and each outcome needs different advice.
///
/// Serialized tagged, like `Compatibility`, so the frontend can `switch` on `status`.
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
#[serde(tag = "status", rename_all = "snake_case")]
pub enum RevokeOutcome {
/// The server confirmed it: this token authenticates nothing now.
Revoked,
/// This server has no self-revoke route — it predates one. The token is still
/// live, and only the web app can retire it.
Unsupported,
/// We couldn't reach the server, or it refused. The token is still live.
Failed { reason: String },
/// Nothing to revoke; the app wasn't linked.
Skipped,
}
/// Retire the device token we authenticate with, server-side.
///
/// Identified by the token itself rather than a device id, because a token pasted
/// from the web app never carried one — a route keyed on the id would work for
/// exactly one of the two ways this app can be linked.
pub async fn revoke_self(base_url: &str, token: &str) -> RevokeOutcome {
let client = match http() {
Ok(client) => client,
Err(reason) => return RevokeOutcome::Failed { reason },
};
let request = prepare(client.delete(revoke_self_url(base_url)), Some(token));
let response = match request.send().await {
Ok(response) => response,
Err(e) => {
return RevokeOutcome::Failed {
reason: describe_transport_error(base_url, &e),
}
}
};
let status = response.status();
// 401 counts as revoked: the token already authenticates nothing — retired by
// another device, or purged server-side — which is the state we were asking for.
if status.is_success() || status == StatusCode::UNAUTHORIZED {
return RevokeOutcome::Revoked;
}
match status {
// No such route: a server older than self-revoke. Any other shape of 404
// (a proxy, a stale base URL) leaves the token live too, so the advice the
// user needs is the same either way.
StatusCode::NOT_FOUND | StatusCode::METHOD_NOT_ALLOWED => RevokeOutcome::Unsupported,
other => RevokeOutcome::Failed {
reason: unexpected_status(base_url, other),
},
}
}
fn http_with(timeout: Duration) -> Result<reqwest::Client, String> {
reqwest::Client::builder()
.timeout(timeout)
.build()
.map_err(|e| format!("Could not start the network client: {e}"))
}
fn http() -> Result<reqwest::Client, String> {
http_with(REQUEST_TIMEOUT)
}
/// Attach the client-identity headers every request carries, plus a bearer token
/// when we hold one.
fn prepare(builder: RequestBuilder, token: Option<&str>) -> RequestBuilder {
let mut builder = builder;
for (name, value) in compat::client_headers() {
builder = builder.header(name, value);
}
match token {
Some(t) => builder.bearer_auth(t),
None => builder,
}
}
fn unexpected_status(base_url: &str, status: StatusCode) -> String {
format!(
"{base_url} answered with HTTP {}. Check the address — a reverse proxy or a \
different site may be answering there.",
status.as_u16()
)
}
/// Ask a server who it is and whether we can sync with it.
///
/// `Err` means we never got a usable answer (bad address, unreachable, not a
/// ThoughtSync server). A server that answers but is *incompatible* comes back `Ok`
/// with a verdict — that distinction matters, because the two need very different
/// messages: one is "check what you typed", the other is "update something".
pub async fn probe(raw_url: &str) -> Result<ProbeResult, String> {
let base_url = compat::normalize_base_url(raw_url)
.ok_or("Enter a server address, like https://notes.example.com")?;
let request = prepare(http()?.get(config_url(&base_url)), None);
let response = request
.send()
.await
.map_err(|e| describe_transport_error(&base_url, &e))?;
let status = response.status();
if !status.is_success() {
return Err(unexpected_status(&base_url, status));
}
// Something answered 200 that isn't a ThoughtSync server (a router login page, a
// captive portal). Report the address, not the parse error, which would mean
// nothing to the person reading it.
let server: ServerInfo = response.json().await.map_err(|_| {
format!(
"{base_url} responded, but not with ThoughtSync's configuration. \
Is that the right address?"
)
})?;
let compatibility = compat::evaluate(&server);
Ok(ProbeResult {
base_url,
server,
compatibility,
})
}
/// Exchange email + password for a device bearer token.
///
/// The fresh-install path: it needs no existing session, which is what lets a brand
/// new desktop install link without visiting the web app first.
pub async fn device_login(
base_url: &str,
email: &str,
password: &str,
device_name: &str,
) -> Result<(String, Identity), String> {
let body = serde_json::json!({
"email": email,
"password": password,
"name": device_name,
});
let request = prepare(http()?.post(device_login_url(base_url)), None).json(&body);
let response = request
.send()
.await
.map_err(|e| describe_transport_error(base_url, &e))?;
let status = response.status();
if status == StatusCode::UNAUTHORIZED {
return Err("That email and password didn't match an account on this server.".to_string());
}
if !status.is_success() {
return Err(unexpected_status(base_url, status));
}
let parsed: DeviceLoginResponse = response
.json()
.await
.map_err(|_| format!("{base_url} signed us in but sent an unexpected reply."))?;
Ok((parsed.token, parsed.user))
}
/// Validate a token by asking whom it belongs to.
///
/// Used when the user pastes a token issued from the web app. Storing it unverified
/// would turn a copy/paste slip into a failure that only surfaces at the next sync,
/// far from the thing that caused it.
pub async fn fetch_identity(base_url: &str, token: &str) -> Result<Identity, String> {
let request = prepare(http()?.get(me_url(base_url)), Some(token));
let response = request
.send()
.await
.map_err(|e| describe_transport_error(base_url, &e))?;
let status = response.status();
if status == StatusCode::UNAUTHORIZED {
let message = "That token isn't valid on this server — it may have been revoked. \
Issue a new one from the web app under Account → Linked devices.";
return Err(message.to_string());
}
if !status.is_success() {
return Err(unexpected_status(base_url, status));
}
response
.json()
.await
.map_err(|_| format!("{base_url} accepted the token but sent an unexpected reply."))
}
/// Fetch one page of the change feed, starting after `since`.
///
/// The caller loops until `has_more` is false (see `pull::run`); paging lives there
/// rather than here so the transport stays a single request/response.
pub async fn fetch_changes(
base_url: &str,
token: &str,
since: i64,
) -> Result<wire::ChangesPage, String> {
let url = format!("{base_url}/api/sync/changes?since={since}");
let request = prepare(http_with(SYNC_TIMEOUT)?.get(url), Some(token));
let response = request
.send()
.await
.map_err(|e| describe_transport_error(base_url, &e))?;
let status = response.status();
if status == StatusCode::UNAUTHORIZED {
return Err(TOKEN_REJECTED.to_string());
}
if !status.is_success() {
return Err(unexpected_status(base_url, status));
}
response
.json()
.await
.map_err(|e| format!("Couldn't read the change feed from {base_url}: {e}"))
}
/// Download one attachment's bytes.
///
/// Metadata already arrived on the delta feed; this is only the payload, fetched
/// over the same route the web app uses (owner/shared scoped server-side).
pub async fn fetch_attachment(
base_url: &str,
token: &str,
note_id: &str,
attachment_id: &str,
) -> Result<Vec<u8>, String> {
let url = format!("{base_url}/api/notes/{note_id}/attachments/{attachment_id}");
let request = prepare(http_with(SYNC_TIMEOUT)?.get(url), Some(token));
let response = request
.send()
.await
.map_err(|e| describe_transport_error(base_url, &e))?;
let status = response.status();
if status == StatusCode::UNAUTHORIZED {
return Err(TOKEN_REJECTED.to_string());
}
if !status.is_success() {
return Err(unexpected_status(base_url, status));
}
response
.bytes()
.await
.map(|b| b.to_vec())
.map_err(|e| format!("Couldn't download an attachment from {base_url}: {e}"))
}
/// Send a batch of changes and hand back the raw reply.
///
/// Returns text rather than parsed results so this module stays pure transport —
/// `push::parse_results` owns the result shapes, and keeping them there is what lets
/// the parsing be unit-tested without a server.
pub async fn push_changes<T: Serialize>(
base_url: &str,
token: &str,
changes: &[T],
) -> Result<String, String> {
let body = serde_json::json!({ "changes": changes });
let url = format!("{base_url}/api/sync/push");
let request = prepare(http_with(SYNC_TIMEOUT)?.post(url), Some(token)).json(&body);
let response = request
.send()
.await
.map_err(|e| describe_transport_error(base_url, &e))?;
let status = response.status();
if status == StatusCode::UNAUTHORIZED {
return Err(TOKEN_REJECTED.to_string());
}
if !status.is_success() {
return Err(unexpected_status(base_url, status));
}
response
.text()
.await
.map_err(|e| format!("Couldn't read the push reply from {base_url}: {e}"))
}
/// The public, unauthenticated endpoint carrying the handshake.
fn config_url(base_url: &str) -> String {
format!("{base_url}/api/config")
}
fn device_login_url(base_url: &str) -> String {
format!("{base_url}/api/auth/device-login")
}
fn me_url(base_url: &str) -> String {
format!("{base_url}/api/auth/me")
}
/// `self` rather than a device id: see `revoke_self`.
fn revoke_self_url(base_url: &str) -> String {
format!("{base_url}/api/auth/devices/self")
}
/// Turn a transport failure into something a person can act on. reqwest's own
/// Display is accurate but reads like a stack trace.
fn describe_transport_error(base_url: &str, err: &reqwest::Error) -> String {
if err.is_timeout() {
// No specific duration here: these calls run under two different budgets
// (interactive vs bulk sync), and naming the wrong one is worse than naming
// none.
format!(
"{base_url} didn't respond in time. It may be offline, or unreachable \
from this network."
)
} else if err.is_connect() {
format!(
"Couldn't reach {base_url}. Check the address and that the server is \
running. If it uses plain HTTP, include http:// explicitly."
)
} else {
format!("Couldn't reach {base_url}: {err}")
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn urls_join_without_doubling_slashes() {
// normalize_base_url has already stripped any trailing slash, so plain
// concatenation is correct — this pins that assumption.
assert_eq!(
config_url("https://notes.example.com"),
"https://notes.example.com/api/config"
);
assert_eq!(
device_login_url("https://notes.example.com"),
"https://notes.example.com/api/auth/device-login"
);
assert_eq!(
me_url("https://notes.example.com"),
"https://notes.example.com/api/auth/me"
);
assert_eq!(
revoke_self_url("https://notes.example.com"),
"https://notes.example.com/api/auth/devices/self"
);
}
#[test]
fn revoke_outcome_serializes_tagged_for_the_frontend() {
// The UI decides between "signed out on the server" and "still valid, go
// revoke it" by reading this tag, so its shape is part of the contract.
let json = serde_json::to_string(&RevokeOutcome::Failed {
reason: "offline".into(),
})
.expect("outcome serializes");
assert!(json.contains("\"status\":\"failed\""), "got {json}");
let json = serde_json::to_string(&RevokeOutcome::Revoked).expect("outcome serializes");
assert!(json.contains("\"status\":\"revoked\""), "got {json}");
}
#[test]
fn urls_preserve_a_port_and_subpath() {
assert_eq!(
config_url("http://192.168.1.10:8000/thoughtsync"),
"http://192.168.1.10:8000/thoughtsync/api/config"
);
}
}
+393
View File
@@ -0,0 +1,393 @@
//! Client<->server compatibility handshake (M10.6).
//!
//! The desktop app is local-first: it never *needs* a server. When the user links
//! one, this module decides whether the two can actually talk — before a single
//! note moves. The sync engine (M10.7) consults it on link and on every sync.
//!
//! The contract is two integers per side, versioning the WIRE PROTOCOL separately
//! from either program's release version:
//!
//! | | this client | the server advertises |
//! |---|---|---|
//! | speaks | `CLIENT_PROTOCOL_VERSION` | `sync_protocol_version` |
//! | accepts down to | `MIN_SERVER_PROTOCOL_VERSION` | `min_client_protocol_version` |
//!
//! Each side declaring its own floor is what avoids app<->server lockstep: either
//! side can mark a change breaking without the other needing to ship in step. See
//! `docs/sync.md` for the policy that governs when those numbers move.
use serde::{Deserialize, Serialize};
/// The sync wire protocol this client speaks.
pub const CLIENT_PROTOCOL_VERSION: u32 = 1;
/// The oldest server protocol this client can drive — the symmetric half of the
/// server's `min_client_protocol_version`.
pub const MIN_SERVER_PROTOCOL_VERSION: u32 = 1;
/// Capabilities without which syncing is meaningless, so their absence BLOCKS the
/// link rather than degrading it.
pub const REQUIRED_FEATURES: &[&str] = &["notes", "labels"];
/// Capabilities whose absence costs a feature but not the link. Listing these
/// explicitly (rather than diffing against whatever the server happens to send) is
/// what lets the UI name exactly what the user will be missing.
pub const OPTIONAL_FEATURES: &[&str] = &["attachments", "tombstones", "revisions"];
/// The handshake fields of `GET /api/config`.
///
/// Every protocol field is optional because a server predating M10.6 simply won't
/// send them. That case has to read as "this server is too old to sync", not as a
/// parse failure — which would look to the user like they mistyped the URL.
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
pub struct ServerInfo {
#[serde(default)]
pub site_name: Option<String>,
/// The server's release version, for display only — never gate on it.
#[serde(default)]
pub version: Option<String>,
#[serde(default)]
pub sync_protocol_version: Option<u32>,
#[serde(default)]
pub min_client_protocol_version: Option<u32>,
#[serde(default)]
pub sync_features: Vec<String>,
/// How long the SERVER keeps a trashed note before purging it (0 = forever).
/// Once linked this is the window that actually applies, so the desktop's Trash
/// countdown has to come from here rather than from its own offline default.
#[serde(default)]
pub trash_retention_days: Option<u32>,
}
impl ServerInfo {
fn has_feature(&self, name: &str) -> bool {
self.sync_features.iter().any(|f| f.as_str() == name)
}
fn missing(&self, from: &[&str]) -> Vec<String> {
from.iter()
.copied()
.filter(|f| !self.has_feature(f))
.map(String::from)
.collect()
}
}
/// The verdict the link/settings UI renders and the sync engine obeys.
///
/// Serialized tagged so the frontend can `switch` on `status` directly.
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
#[serde(tag = "status", rename_all = "snake_case")]
pub enum Compatibility {
/// Full parity — sync everything.
Ok,
/// Safe to sync, but these named capabilities aren't available here.
Degraded { unavailable: Vec<String> },
/// Do not sync. `client_must_update` points the user at the side that can fix
/// it, so the message can be actionable instead of just "incompatible".
Incompatible {
reason: String,
client_must_update: bool,
},
}
fn incompatible(reason: &str, client_must_update: bool) -> Compatibility {
Compatibility::Incompatible {
reason: reason.to_string(),
client_must_update,
}
}
/// Decide whether this client can sync with the described server.
///
/// Pure: the transport fetches `ServerInfo`, this decides what it means. Keeping
/// the decision free of I/O is what makes every branch below unit-testable, which
/// matters because there is no Postgres/live-server lane in CI.
pub fn evaluate(info: &ServerInfo) -> Compatibility {
// Ordered most-fundamental first, so the user sees the root problem rather than
// a downstream symptom of it.
let Some(server_proto) = info.sync_protocol_version else {
return incompatible(
"This server doesn't support device sync — it predates the sync protocol. \
Update the server, then link again.",
false,
);
};
if server_proto < MIN_SERVER_PROTOCOL_VERSION {
return incompatible(
&format!(
"This server speaks sync protocol v{server_proto}, but this app needs \
at least v{MIN_SERVER_PROTOCOL_VERSION}. Update the server."
),
false,
);
}
// The server's floor is what hard-blocks an old client. Absent => no floor: a
// server that advertises a protocol but no minimum accepts anything.
let floor = info.min_client_protocol_version.unwrap_or(0);
if CLIENT_PROTOCOL_VERSION < floor {
return incompatible(
&format!(
"This server requires client protocol v{floor} or newer; this app \
speaks v{CLIENT_PROTOCOL_VERSION}. Update ThoughtSync."
),
true,
);
}
// A version match still isn't enough: a server can speak the protocol with a
// core capability compiled out or disabled.
let missing_required = info.missing(REQUIRED_FEATURES);
if !missing_required.is_empty() {
return incompatible(
&format!(
"This server is missing sync capabilities this app requires: {}.",
missing_required.join(", ")
),
false,
);
}
let unavailable = info.missing(OPTIONAL_FEATURES);
if unavailable.is_empty() {
Compatibility::Ok
} else {
Compatibility::Degraded { unavailable }
}
}
/// Headers this client puts on every request to a linked server, so the server can
/// log or gate on client identity without a separate handshake round-trip.
pub fn client_headers() -> [(&'static str, String); 2] {
let agent = format!("thoughtsync-desktop/{}", env!("CARGO_PKG_VERSION"));
[
("X-ThoughtSync-Client", agent),
(
"X-ThoughtSync-Protocol",
CLIENT_PROTOCOL_VERSION.to_string(),
),
]
}
/// Turn what a user typed into a base URL we can build request paths on, or `None`
/// if there's nothing usable in it.
///
/// A bare host gets **`https://`**, never `http://`. Silently downgrading would put
/// a long-lived device token on the wire in cleartext because someone omitted five
/// characters. Plain HTTP on a trusted LAN stays fully supported — the user just
/// has to type `http://` and thereby choose it.
pub fn normalize_base_url(raw: &str) -> Option<String> {
let trimmed = raw.trim();
if trimmed.is_empty() {
return None;
}
// Resolve the scheme BEFORE touching trailing slashes — stripping them first
// turns a bare "https://" into "https:", which then reads as a hostname.
let with_scheme = match trimmed.split_once("://") {
Some((scheme, rest)) => {
// Anything that isn't HTTP(S) (ftp://, file://, a stray "foo://") can't
// be a ThoughtSync server; reject rather than fail confusingly later.
let scheme = scheme.to_ascii_lowercase();
if scheme != "http" && scheme != "https" {
return None;
}
format!("{scheme}://{rest}")
}
None => format!("https://{trimmed}"),
};
let (scheme, rest) = with_scheme.split_once("://")?;
let rest = rest.trim_end_matches('/');
// Reject a scheme with no authority ("https://", "http:///path").
if rest.split(['/', '?', '#']).next().unwrap_or("").is_empty() {
return None;
}
Some(format!("{scheme}://{rest}"))
}
#[cfg(test)]
mod tests {
use super::*;
/// A server matching this client exactly, which each test then degrades.
fn current_server() -> ServerInfo {
ServerInfo {
site_name: Some("ThoughtSync".into()),
version: Some("0.1.0".into()),
sync_protocol_version: Some(CLIENT_PROTOCOL_VERSION),
min_client_protocol_version: Some(CLIENT_PROTOCOL_VERSION),
sync_features: REQUIRED_FEATURES
.iter()
.chain(OPTIONAL_FEATURES.iter())
.copied()
.map(String::from)
.collect(),
trash_retention_days: Some(30),
}
}
#[test]
fn current_server_is_fully_compatible() {
assert_eq!(evaluate(&current_server()), Compatibility::Ok);
}
#[test]
fn server_without_protocol_fields_is_too_old() {
// A pre-M10.6 server: /api/config parses, but carries no protocol block.
let info = ServerInfo {
site_name: Some("ThoughtSync".into()),
version: Some("0.0.9".into()),
..Default::default()
};
match evaluate(&info) {
Compatibility::Incompatible {
client_must_update, ..
} => assert!(!client_must_update, "the SERVER is the old side here"),
other => panic!("expected incompatible, got {other:?}"),
}
}
#[test]
fn client_older_than_the_servers_floor_must_update() {
let info = ServerInfo {
sync_protocol_version: Some(CLIENT_PROTOCOL_VERSION + 5),
min_client_protocol_version: Some(CLIENT_PROTOCOL_VERSION + 5),
..current_server()
};
match evaluate(&info) {
Compatibility::Incompatible {
client_must_update, ..
} => assert!(client_must_update),
other => panic!("expected incompatible, got {other:?}"),
}
}
#[test]
fn newer_server_within_our_floor_still_works() {
// The whole point of the two-number contract: a server can move ahead
// additively without locking out a client that predates the change.
let info = ServerInfo {
sync_protocol_version: Some(CLIENT_PROTOCOL_VERSION + 3),
min_client_protocol_version: Some(CLIENT_PROTOCOL_VERSION),
..current_server()
};
assert_eq!(evaluate(&info), Compatibility::Ok);
}
#[test]
fn server_with_no_declared_floor_accepts_us() {
let info = ServerInfo {
min_client_protocol_version: None,
..current_server()
};
assert_eq!(evaluate(&info), Compatibility::Ok);
}
#[test]
fn missing_optional_feature_degrades_rather_than_blocks() {
let info = ServerInfo {
sync_features: current_server()
.sync_features
.into_iter()
.filter(|f| f.as_str() != "attachments")
.collect(),
..current_server()
};
assert_eq!(
evaluate(&info),
Compatibility::Degraded {
unavailable: vec!["attachments".to_string()]
}
);
}
#[test]
fn missing_required_feature_blocks() {
let info = ServerInfo {
sync_features: vec!["labels".to_string()],
..current_server()
};
match evaluate(&info) {
Compatibility::Incompatible { reason, .. } => assert!(reason.contains("notes")),
other => panic!("expected incompatible, got {other:?}"),
}
}
#[test]
fn version_mismatch_outranks_a_missing_feature() {
// Both wrong → report the version, the root cause of the missing feature.
let info = ServerInfo {
sync_protocol_version: Some(CLIENT_PROTOCOL_VERSION + 2),
min_client_protocol_version: Some(CLIENT_PROTOCOL_VERSION + 2),
sync_features: vec![],
..current_server()
};
match evaluate(&info) {
Compatibility::Incompatible {
client_must_update, ..
} => assert!(client_must_update),
other => panic!("expected incompatible, got {other:?}"),
}
}
#[test]
fn verdict_serializes_tagged_for_the_frontend() {
let verdict = Compatibility::Degraded {
unavailable: vec!["attachments".into()],
};
let json = serde_json::to_string(&verdict).expect("verdict serializes");
assert!(json.contains("\"status\":\"degraded\""), "got {json}");
}
#[test]
fn server_info_tolerates_unknown_and_absent_fields() {
// Forward compatibility: a NEWER server sending fields we've never heard of
// must not break the handshake.
let info: ServerInfo = serde_json::from_str(
r#"{"site_name":"S","sync_protocol_version":1,
"min_client_protocol_version":1,
"sync_features":["notes","labels","attachments","tombstones","revisions"],
"some_future_field":{"nested":true}}"#,
)
.expect("unknown fields are ignored");
assert_eq!(evaluate(&info), Compatibility::Ok);
}
#[test]
fn client_headers_identify_app_and_protocol() {
let headers = client_headers();
assert_eq!(headers[0].0, "X-ThoughtSync-Client");
assert!(headers[0].1.starts_with("thoughtsync-desktop/"));
assert_eq!(headers[1].1, CLIENT_PROTOCOL_VERSION.to_string());
}
#[test]
fn base_url_defaults_to_https_and_trims() {
assert_eq!(
normalize_base_url(" notes.example.com/ "),
Some("https://notes.example.com".to_string())
);
assert_eq!(
normalize_base_url("https://notes.example.com///"),
Some("https://notes.example.com".to_string())
);
}
#[test]
fn base_url_keeps_an_explicit_http_choice() {
// Plain HTTP on a LAN is supported — the user just has to ask for it.
assert_eq!(
normalize_base_url("http://192.168.1.10:8000"),
Some("http://192.168.1.10:8000".to_string())
);
}
#[test]
fn base_url_rejects_junk() {
assert_eq!(normalize_base_url(""), None);
assert_eq!(normalize_base_url(" "), None);
assert_eq!(normalize_base_url("https://"), None);
assert_eq!(normalize_base_url("ftp://files.example.com"), None);
}
}
+77
View File
@@ -0,0 +1,77 @@
//! The sync cycle (M10.7c).
//!
//! Deliberately the ONLY way the UI can sync. Push and pull are each usable on their
//! own inside this crate, but exposing them separately would let a caller pull
//! without pushing, which quietly overwrites unsent local edits.
use chrono::{SecondsFormat, Utc};
use serde::Serialize;
use super::blobs::BlobStore;
use super::pull;
use super::push;
use super::state;
use crate::local::Db;
#[derive(Debug, Serialize)]
pub struct SyncOutcome {
pub push: push::PushSummary,
pub pull: pull::PullSummary,
/// The state after the cycle, so the UI updates from one round-trip instead of
/// following every sync with a status call.
pub status: state::Status,
}
/// Push, then pull — in that order, always.
///
/// Pull writes the server's version straight over the local row, so anything not yet
/// sent would be lost to it. Pushing first is what puts the local edit in front of
/// the server's last-write-wins comparison, and it's the reason
/// `PullSummary::clobbered_dirty` should be zero on every healthy cycle.
///
/// A failed push aborts before the pull. Pulling anyway would take the exact rows we
/// just failed to save and overwrite them — turning a recoverable network error into
/// lost work.
pub async fn run_cycle(
db: &Db,
blobs: &BlobStore,
base_url: &str,
token: &str,
) -> Result<SyncOutcome, String> {
let push = push::run(db, base_url, token).await?;
let pull = pull::run(db, blobs, base_url, token).await?;
if pull.clobbered_dirty > 0 {
// Push ran first and reported success, so nothing should still have been
// dirty. Reaching here means something wrote to the store mid-cycle, or a
// change never got collected — worth a loud line either way.
log::warn!(
"sync cycle overwrote {} locally-edited note(s) despite pushing first",
pull.clobbered_dirty
);
}
// While we're already talking to this server, re-read what it says about itself.
// Today that's the trash-retention window the Trash view counts down against, and
// it can change under us whenever an admin edits the setting. Best-effort on
// purpose: a config blip must not fail a cycle whose actual work already
// succeeded, and the stored value simply stays as it was.
let retention = super::client::probe(base_url)
.await
.ok()
.and_then(|p| p.server.trash_retention_days);
let status = {
let conn = db.0.lock().map_err(|e| e.to_string())?;
if let Some(days) = retention {
state::set_server_retention(&conn, days as i64).map_err(|e| e.to_string())?;
}
// Stamped only here, after BOTH halves succeeded. A timestamp written after a
// partial cycle would tell the user they're up to date when they aren't.
let now = Utc::now().to_rfc3339_opts(SecondsFormat::Millis, true);
state::mark_synced(&conn, &now).map_err(|e| e.to_string())?;
state::status(&conn).map_err(|e| e.to_string())?
};
Ok(SyncOutcome { push, pull, status })
}
+22
View File
@@ -0,0 +1,22 @@
//! Talking to a ThoughtSync server — entirely opt-in.
//!
//! The app is local-first: `local` is the source of truth and everything works
//! unlinked. Nothing in here runs until the user links a server.
//!
//! - `compat` — the version/capability handshake (M10.6): whether a given server can
//! be talked to at all. Pure decision logic, no I/O.
//! - `client` — HTTP transport: the handshake call and device-token auth.
//! - `state` — the persisted link record (server, token, change-feed cursor).
//! - `engine` — one full cycle: push local changes, then pull the server's.
//!
//! The UI surface that drives this lives in whichever client is wrapping the crate,
//! not here.
pub mod blobs;
pub mod client;
pub mod compat;
pub mod engine;
pub mod pull;
pub mod push;
pub mod state;
pub mod wire;
+840
View File
@@ -0,0 +1,840 @@
//! Pull: bring a server's changes into the local store (M10.7b).
//!
//! The feed is a single monotonic sequence shared by notes and labels, so one
//! integer cursor is a total-order watermark over both (docs/sync.md). We loop pages
//! until the server says there are no more, persisting the cursor **in the same
//! transaction** as the page it describes — a cursor committed ahead of its data
//! would silently skip those rows forever, which reads as a clean sync.
use chrono::{SecondsFormat, Utc};
use rusqlite::{params, Connection, OptionalExtension};
use serde::Serialize;
use super::blobs::BlobStore;
use super::client;
use super::state;
use super::wire;
use crate::local::Db;
/// Backstop against a server that never stops saying `has_more`. At the server's
/// 1000-row page cap this is 10M rows — far past any real store, so hitting it means
/// something is wrong, not that someone has a lot of notes.
const MAX_PAGES: usize = 10_000;
/// What a pull did — for the UI, and for the log when something looks off.
#[derive(Debug, Clone, Default, Serialize, PartialEq, Eq)]
pub struct PullSummary {
pub pages: usize,
pub notes_applied: usize,
pub notes_deleted: usize,
pub labels_applied: usize,
pub labels_deleted: usize,
pub cursor: i64,
/// Rows that still held unpushed local edits when the server's version landed on
/// top. Should be 0 in the normal cycle, because push runs first; anything higher
/// means local work was overwritten, which is worth saying out loud.
pub clobbered_dirty: usize,
pub blobs_downloaded: usize,
/// Attachments whose bytes couldn't be fetched or failed verification. Counted
/// rather than fatal — see `download_missing_blobs`.
pub blobs_failed: usize,
}
impl PullSummary {
fn absorb(&mut self, other: PullSummary) {
self.pages += other.pages;
self.notes_applied += other.notes_applied;
self.notes_deleted += other.notes_deleted;
self.labels_applied += other.labels_applied;
self.labels_deleted += other.labels_deleted;
self.clobbered_dirty += other.clobbered_dirty;
self.blobs_downloaded += other.blobs_downloaded;
self.blobs_failed += other.blobs_failed;
self.cursor = other.cursor;
}
}
/// `(note_id, attachment_id, sha256)` for every attachment that advertises a hash.
/// The caller filters against the blob store — which blobs we hold isn't a SQL
/// question.
pub fn hashed_attachments(conn: &Connection) -> rusqlite::Result<Vec<(String, String, String)>> {
let mut stmt = conn.prepare(
"SELECT note_id, id, sha256 FROM attachments
WHERE sha256 IS NOT NULL AND sha256 <> ''",
)?;
let rows = stmt.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)))?;
rows.collect()
}
/// Fetch the bytes for any attachment we have metadata for but no blob.
///
/// A failed attachment NEVER fails the sync. Notes are the primary data and they've
/// already landed; an image that didn't arrive is retried on the next cycle simply
/// because its blob still counts as missing. Aborting here would mean one unreachable
/// file could block every future sync.
async fn download_missing_blobs(
db: &Db,
blobs: &BlobStore,
base_url: &str,
token: &str,
) -> Result<(usize, usize), String> {
let wanted = {
let conn = db.0.lock().map_err(|e| e.to_string())?;
hashed_attachments(&conn).map_err(|e| e.to_string())?
};
let mut downloaded = 0;
let mut failed = 0;
for (note_id, attachment_id, sha256) in wanted {
// Content-addressed, so this skips blobs we already hold — including the same
// image attached to a different note.
if blobs.has(&sha256) {
continue;
}
match client::fetch_attachment(base_url, token, &note_id, &attachment_id).await {
Ok(bytes) => match blobs.store(&sha256, &bytes) {
Ok(_) => downloaded += 1,
Err(e) => {
log::warn!("attachment {attachment_id}: {e}");
failed += 1;
}
},
Err(e) => {
log::warn!("attachment {attachment_id}: {e}");
failed += 1;
}
}
}
Ok((downloaded, failed))
}
fn now() -> String {
Utc::now().to_rfc3339_opts(SecondsFormat::Millis, true)
}
/// Apply one page and advance the cursor, atomically.
///
/// Labels are applied before notes so a membership never references a label row that
/// doesn't exist yet.
pub fn apply_page(conn: &Connection, page: &wire::ChangesPage) -> rusqlite::Result<PullSummary> {
let tx = conn.unchecked_transaction()?;
let mut summary = PullSummary {
pages: 1,
cursor: page.cursor,
..Default::default()
};
for label in &page.labels {
if label.is_tombstone() {
tx.execute("DELETE FROM labels WHERE id = ?1", params![label.id])?;
summary.labels_deleted += 1;
} else {
upsert_label(&tx, label)?;
summary.labels_applied += 1;
}
}
for note in &page.notes {
if note.is_tombstone() {
// A purge tombstone carries no content — its only job is to say "delete
// your copy". Children go with it via ON DELETE CASCADE.
tx.execute("DELETE FROM notes WHERE id = ?1", params![note.id])?;
summary.notes_deleted += 1;
continue;
}
if is_dirty(&tx, &note.id)? {
summary.clobbered_dirty += 1;
}
upsert_note(&tx, note)?;
summary.notes_applied += 1;
}
state::set_cursor(&tx, page.cursor)?;
tx.commit()?;
Ok(summary)
}
fn is_dirty(conn: &Connection, note_id: &str) -> rusqlite::Result<bool> {
let dirty: Option<i64> = conn
.query_row(
"SELECT dirty FROM notes WHERE id = ?1",
params![note_id],
|r| r.get(0),
)
.optional()?;
Ok(dirty == Some(1))
}
fn upsert_label(conn: &Connection, label: &wire::Label) -> rusqlite::Result<()> {
// One label per name is enforced on both sides (locally a UNIQUE index on
// lower(name); on the server, per owner). A label created offline can therefore
// collide with one the server already had under a different id — "work" typed on
// this machine and "work" that already existed.
//
// The server's row wins, but its MEMBERSHIPS have to survive the swap. Just
// deleting the local duplicate would cascade its note_labels away, stripping the
// label off notes that this pull never even mentions — silent loss that no later
// page would repair. So: free the name, insert the server's row, re-point the
// memberships onto it, then drop the husk.
let duplicates: Vec<String> = {
let mut stmt =
conn.prepare("SELECT id FROM labels WHERE lower(name) = lower(?1) AND id <> ?2")?;
let rows = stmt.query_map(params![label.name, label.id], |r| r.get::<_, String>(0))?;
rows.collect::<rusqlite::Result<Vec<String>>>()?
};
// Renaming first is what makes the insert possible at all — the unique index
// would otherwise reject the server's row before anything could be merged.
for old in &duplicates {
conn.execute(
"UPDATE labels SET name = name || ' (superseded ' || id || ')' WHERE id = ?1",
params![old],
)?;
}
let created = label.created_at.clone().unwrap_or_else(now);
conn.execute(
"INSERT INTO labels (id, name, color, created_at, updated_at, sync_revision, dirty)
VALUES (?1, ?2, ?3, ?4, ?4, ?5, 0)
ON CONFLICT(id) DO UPDATE SET
name = excluded.name,
color = excluded.color,
sync_revision = excluded.sync_revision,
dirty = 0",
params![
label.id,
label.name,
label.color,
created,
label.sync_revision
],
)?;
for old in &duplicates {
// OR IGNORE guards a (note_id, label_id) collision. Today the unique index on
// lower(name) makes that unreachable — two same-name labels can't coexist
// locally — so this is belt-and-braces against that index changing, not a
// case we've seen. Anything it skips cascades away with the husk below, which
// is correct: those are duplicates of a membership that now exists.
conn.execute(
"UPDATE OR IGNORE note_labels SET label_id = ?1 WHERE label_id = ?2",
params![label.id, old],
)?;
conn.execute("DELETE FROM labels WHERE id = ?1", params![old])?;
}
Ok(())
}
fn upsert_note(conn: &Connection, note: &wire::Note) -> rusqlite::Result<()> {
let created = note.created_at.clone().unwrap_or_else(now);
let updated = note.updated_at.clone().unwrap_or_else(|| created.clone());
// The server's `deleted_at` is the authority on trash AGE. Taking it from the feed
// rather than stamping "now" locally is what keeps a note trashed three weeks ago
// from looking brand-new to a device that only just heard about it — otherwise
// every fresh install would silently reset the whole retention clock. Falls back
// to the note's updated_at only if an older server omits the field.
let trashed_at = if note.trashed {
note.deleted_at.clone().or_else(|| Some(updated.clone()))
} else {
None
};
// `created_at` is deliberately absent from the UPDATE clause: a note's birth time
// never changes, and the server's copy is the same value anyway.
conn.execute(
"INSERT INTO notes (id, title, body, color, kind, position, pinned, archived,
trashed, remind_at, recurrence, created_at, updated_at,
sync_revision, trashed_at, dirty)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, 0)
ON CONFLICT(id) DO UPDATE SET
title = excluded.title,
body = excluded.body,
color = excluded.color,
kind = excluded.kind,
position = excluded.position,
pinned = excluded.pinned,
archived = excluded.archived,
trashed = excluded.trashed,
remind_at = excluded.remind_at,
recurrence = excluded.recurrence,
updated_at = excluded.updated_at,
sync_revision = excluded.sync_revision,
trashed_at = excluded.trashed_at,
dirty = 0",
params![
note.id,
note.title,
note.body,
note.color,
note.kind,
note.position,
note.pinned,
note.archived,
note.trashed,
note.remind_at,
note.recurrence,
created,
updated,
note.sync_revision,
trashed_at,
],
)?;
// Children are replaced wholesale: a delta carries the note's FULL current state,
// so "what the server sent" IS the complete set. Diffing would be more code and
// could leave behind a row the server no longer has.
replace_items(conn, note)?;
replace_attachments(conn, note)?;
replace_previews(conn, note)?;
replace_labels(conn, note)?;
Ok(())
}
fn replace_items(conn: &Connection, note: &wire::Note) -> rusqlite::Result<()> {
conn.execute(
"DELETE FROM checklist_items WHERE note_id = ?1",
params![note.id],
)?;
for (index, item) in note.items.iter().enumerate() {
conn.execute(
"INSERT INTO checklist_items (id, note_id, text, checked, position)
VALUES (?1, ?2, ?3, ?4, ?5)",
params![
item.id,
note.id,
item.text,
item.checked,
position_of(item.position, index)
],
)?;
}
Ok(())
}
fn replace_attachments(conn: &Connection, note: &wire::Note) -> rusqlite::Result<()> {
conn.execute(
"DELETE FROM attachments WHERE note_id = ?1",
params![note.id],
)?;
for (index, att) in note.attachments.iter().enumerate() {
// The feed carries no explicit position for attachments — they arrive in
// creation order, so the index preserves it.
conn.execute(
"INSERT INTO attachments (id, note_id, url, filename, mime, size, sha256, position)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)",
params![
att.id,
note.id,
att.url,
att.filename,
att.mime,
att.size,
att.sha256,
index as i64
],
)?;
}
Ok(())
}
fn replace_previews(conn: &Connection, note: &wire::Note) -> rusqlite::Result<()> {
conn.execute(
"DELETE FROM link_previews WHERE note_id = ?1",
params![note.id],
)?;
for (index, preview) in note.previews.iter().enumerate() {
conn.execute(
"INSERT INTO link_previews (id, note_id, url, title, description, image_url,
site_name, position)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)",
params![
preview.id,
note.id,
preview.url,
preview.title,
preview.description,
preview.image_url,
preview.site_name,
index as i64
],
)?;
}
Ok(())
}
fn replace_labels(conn: &Connection, note: &wire::Note) -> rusqlite::Result<()> {
conn.execute(
"DELETE FROM note_labels WHERE note_id = ?1",
params![note.id],
)?;
for label in &note.labels {
ensure_label_stub(conn, label)?;
// `via_tag` is applied verbatim rather than re-derived from the body. The
// server already reconciled tags when it saved the note, and re-deriving here
// would call the local find-or-create path, which marks new labels dirty and
// would push them straight back — sync churn out of nothing.
conn.execute(
"INSERT OR IGNORE INTO note_labels (note_id, label_id, via_tag)
VALUES (?1, ?2, ?3)",
params![note.id, label.id, label.via_tag],
)?;
}
Ok(())
}
/// Materialize a label referenced by a note, if we don't have it yet.
///
/// Notes and labels page from one shared sequence, so a note can reference a label
/// whose own delta landed in an earlier page — or, right at a page boundary, hasn't
/// landed. The note carries enough of the label to create it, so a membership never
/// fails on a missing row. `OR IGNORE` because the label's real delta (later in this
/// page or a future one) is the authority on its name and color.
fn ensure_label_stub(conn: &Connection, label: &wire::NoteLabel) -> rusqlite::Result<()> {
let ts = now();
conn.execute(
"INSERT OR IGNORE INTO labels (id, name, color, created_at, updated_at, dirty)
VALUES (?1, ?2, ?3, ?4, ?4, 0)",
params![label.id, label.name, label.color, ts],
)?;
Ok(())
}
/// Trust an explicit position; fall back to arrival order when the server sent 0 for
/// everything (which is what an unordered list looks like on the wire).
fn position_of(explicit: i64, index: usize) -> i64 {
if explicit > 0 {
explicit
} else {
index as i64
}
}
/// Loop the feed to exhaustion, starting from the persisted cursor.
///
/// NOTE ON ORDERING: the full cycle is push-then-pull (docs/sync.md). Running this
/// against a store with unpushed edits lets the server's version land on top of them
/// — counted as `clobbered_dirty` and logged, rather than hidden.
pub async fn run(
db: &Db,
blobs: &BlobStore,
base_url: &str,
token: &str,
) -> Result<PullSummary, String> {
let mut total = PullSummary::default();
loop {
let since = {
let conn = db.0.lock().map_err(|e| e.to_string())?;
state::read(&conn).map_err(|e| e.to_string())?.last_cursor
};
let page = client::fetch_changes(base_url, token, since).await?;
// Trust the data over the flag: a server that claims more pages without
// advancing the cursor would spin this loop forever.
if page.has_more && page.cursor <= since {
return Err(format!(
"The server reported more changes but its cursor didn't advance past \
{since}. Stopping rather than looping forever."
));
}
let has_more = page.has_more;
let applied = {
let conn = db.0.lock().map_err(|e| e.to_string())?;
apply_page(&conn, &page).map_err(|e| e.to_string())?
};
total.absorb(applied);
if !has_more {
break;
}
if total.pages >= MAX_PAGES {
return Err(format!(
"Stopped after {MAX_PAGES} pages without reaching the end of the \
server's changes. Something is wrong with the feed."
));
}
}
// Notes first, bytes after: the metadata is what makes the attachments knowable,
// and knowing one is missing is what lets the next cycle retry it.
let (downloaded, failed) = download_missing_blobs(db, blobs, base_url, token).await?;
total.blobs_downloaded = downloaded;
total.blobs_failed = failed;
if total.clobbered_dirty > 0 {
log::warn!(
"pull overwrote {} note(s) that still had unpushed local edits",
total.clobbered_dirty
);
}
if total.blobs_failed > 0 {
log::warn!(
"pull: {} attachment(s) couldn't be downloaded; will retry next sync",
total.blobs_failed
);
}
log::info!(
"pull complete: {} page(s), {} note(s) applied, {} deleted, {} label(s) applied, cursor {}",
total.pages,
total.notes_applied,
total.notes_deleted,
total.labels_applied,
total.cursor
);
Ok(total)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::local::schema;
fn db() -> Connection {
let conn = Connection::open_in_memory().expect("in-memory db");
schema::migrate(&conn).expect("migrate");
conn
}
fn note(id: &str, revision: i64) -> wire::Note {
wire::Note {
id: id.to_string(),
title: Some("Title".into()),
body: "Body".into(),
color: "default".into(),
kind: "text".into(),
position: 0,
pinned: false,
archived: false,
trashed: false,
deleted_at: None,
remind_at: None,
recurrence: None,
created_at: Some("2026-07-26T00:00:00.000Z".into()),
updated_at: Some("2026-07-26T00:00:00.000Z".into()),
sync_revision: revision,
purged_at: None,
labels: vec![],
items: vec![],
attachments: vec![],
previews: vec![],
}
}
fn page(notes: Vec<wire::Note>, labels: Vec<wire::Label>, cursor: i64) -> wire::ChangesPage {
wire::ChangesPage {
notes,
labels,
cursor,
has_more: false,
}
}
fn count(conn: &Connection, sql: &str) -> i64 {
conn.query_row(sql, [], |r| r.get(0)).expect("count")
}
fn trash_stamp(conn: &Connection, id: &str) -> Option<String> {
let sql = "SELECT trashed_at FROM notes WHERE id = ?1";
conn.query_row(sql, [id], |r| r.get(0)).expect("stamp")
}
#[test]
fn applies_a_note_and_advances_the_cursor() {
let conn = db();
let summary = apply_page(&conn, &page(vec![note("n1", 7)], vec![], 7)).expect("apply");
assert_eq!(summary.notes_applied, 1);
assert_eq!(count(&conn, "SELECT COUNT(*) FROM notes"), 1);
assert_eq!(state::read(&conn).expect("state").last_cursor, 7);
}
#[test]
fn pulled_rows_are_not_dirty() {
// They came FROM the server, so pushing them back would be pure churn.
let conn = db();
apply_page(&conn, &page(vec![note("n1", 1)], vec![], 1)).expect("apply");
assert_eq!(count(&conn, "SELECT dirty FROM notes WHERE id = 'n1'"), 0);
}
#[test]
fn tombstone_deletes_the_local_note() {
let conn = db();
apply_page(&conn, &page(vec![note("n1", 1)], vec![], 1)).expect("apply");
let mut dead = note("n1", 2);
dead.purged_at = Some("2026-07-26T01:00:00.000Z".into());
let summary = apply_page(&conn, &page(vec![dead], vec![], 2)).expect("apply");
assert_eq!(summary.notes_deleted, 1);
assert_eq!(count(&conn, "SELECT COUNT(*) FROM notes"), 0);
}
#[test]
fn trashed_is_not_a_tombstone() {
// `trashed` is ordinary state that keeps syncing; only `purged_at` deletes.
let conn = db();
let mut trashed = note("n1", 1);
trashed.trashed = true;
apply_page(&conn, &page(vec![trashed], vec![], 1)).expect("apply");
assert_eq!(count(&conn, "SELECT COUNT(*) FROM notes"), 1);
assert_eq!(count(&conn, "SELECT trashed FROM notes WHERE id = 'n1'"), 1);
}
#[test]
fn trash_age_comes_from_the_server_not_from_now() {
// The retention countdown runs off this timestamp. Stamping it locally would
// hand every note a fresh 30 days on any device that syncs it for the first
// time — a note trashed last month would never expire anywhere.
let conn = db();
let mut trashed = note("n1", 1);
trashed.trashed = true;
trashed.deleted_at = Some("2026-06-01T09:30:00+00:00".into());
apply_page(&conn, &page(vec![trashed], vec![], 1)).expect("apply");
let stamped = trash_stamp(&conn, "n1");
assert_eq!(stamped.as_deref(), Some("2026-06-01T09:30:00+00:00"));
}
#[test]
fn restoring_a_note_server_side_clears_its_trash_stamp() {
let conn = db();
let mut trashed = note("n1", 1);
trashed.trashed = true;
trashed.deleted_at = Some("2026-06-01T09:30:00+00:00".into());
apply_page(&conn, &page(vec![trashed], vec![], 1)).expect("apply");
apply_page(&conn, &page(vec![note("n1", 2)], vec![], 2)).expect("apply");
let stamped = trash_stamp(&conn, "n1");
assert_eq!(stamped, None, "an untrashed note keeps no trash stamp");
}
#[test]
fn an_older_server_without_deleted_at_still_ages_the_trash() {
// Falls back to updated_at rather than leaving the stamp null, which would
// make the note un-expirable and its countdown blank.
let conn = db();
let mut trashed = note("n1", 1);
trashed.trashed = true;
trashed.deleted_at = None;
apply_page(&conn, &page(vec![trashed], vec![], 1)).expect("apply");
let stamped = trash_stamp(&conn, "n1");
assert_eq!(stamped.as_deref(), Some("2026-07-26T00:00:00.000Z"));
}
#[test]
fn children_are_replaced_not_merged() {
let conn = db();
let mut first = note("n1", 1);
first.items = vec![
wire::Item {
id: "i1".into(),
text: "one".into(),
checked: false,
position: 0,
},
wire::Item {
id: "i2".into(),
text: "two".into(),
checked: false,
position: 1,
},
];
apply_page(&conn, &page(vec![first], vec![], 1)).expect("apply");
assert_eq!(count(&conn, "SELECT COUNT(*) FROM checklist_items"), 2);
// The server dropped an item; the local copy must drop it too.
let mut second = note("n1", 2);
second.items = vec![wire::Item {
id: "i1".into(),
text: "one".into(),
checked: true,
position: 0,
}];
apply_page(&conn, &page(vec![second], vec![], 2)).expect("apply");
assert_eq!(count(&conn, "SELECT COUNT(*) FROM checklist_items"), 1);
}
#[test]
fn note_label_membership_materializes_a_missing_label() {
// The label's own delta may have landed in an earlier page, or not yet.
let conn = db();
let mut n = note("n1", 1);
n.labels = vec![wire::NoteLabel {
id: "l1".into(),
name: "work".into(),
color: "blue".into(),
via_tag: true,
}];
apply_page(&conn, &page(vec![n], vec![], 1)).expect("apply");
assert_eq!(count(&conn, "SELECT COUNT(*) FROM labels"), 1);
assert_eq!(
count(
&conn,
"SELECT via_tag FROM note_labels WHERE note_id = 'n1'"
),
1,
"via_tag is applied verbatim, not re-derived"
);
}
#[test]
fn server_label_replaces_a_local_duplicate_by_name() {
let conn = db();
conn.execute(
"INSERT INTO labels (id, name, color, created_at, updated_at, dirty)
VALUES ('local-id', 'Work', 'default', '2026-01-01', '2026-01-01', 1)",
[],
)
.expect("seed local label");
let server = wire::Label {
id: "server-id".into(),
name: "work".into(),
color: "blue".into(),
sync_revision: 5,
purged_at: None,
created_at: Some("2026-07-26T00:00:00.000Z".into()),
};
apply_page(&conn, &page(vec![], vec![server], 5)).expect("apply");
assert_eq!(count(&conn, "SELECT COUNT(*) FROM labels"), 1);
let id: String = conn
.query_row("SELECT id FROM labels", [], |r| r.get(0))
.expect("label");
assert_eq!(id, "server-id", "the server's row wins on pull");
}
#[test]
fn merging_a_duplicate_label_keeps_its_note_memberships() {
// The notes carrying the local label may not be in this page at all, so a
// plain delete would strip the label off them with nothing to repair it.
let conn = db();
apply_page(&conn, &page(vec![note("n1", 1)], vec![], 1)).expect("seed note");
conn.execute(
"INSERT INTO labels (id, name, color, created_at, updated_at, dirty)
VALUES ('local-id', 'Work', 'default', '2026-01-01', '2026-01-01', 1)",
[],
)
.expect("seed local label");
conn.execute(
"INSERT INTO note_labels (note_id, label_id, via_tag)
VALUES ('n1', 'local-id', 0)",
[],
)
.expect("seed membership");
let server = wire::Label {
id: "server-id".into(),
name: "work".into(),
color: "blue".into(),
sync_revision: 5,
purged_at: None,
created_at: None,
};
apply_page(&conn, &page(vec![], vec![server], 5)).expect("apply");
assert_eq!(count(&conn, "SELECT COUNT(*) FROM labels"), 1);
let label_id: String = conn
.query_row(
"SELECT label_id FROM note_labels WHERE note_id = 'n1'",
[],
|r| r.get(0),
)
.expect("membership survived");
assert_eq!(label_id, "server-id", "membership re-pointed, not dropped");
}
#[test]
fn label_tombstone_deletes_and_cascades_memberships() {
let conn = db();
let mut n = note("n1", 1);
n.labels = vec![wire::NoteLabel {
id: "l1".into(),
name: "work".into(),
color: "blue".into(),
via_tag: false,
}];
apply_page(&conn, &page(vec![n], vec![], 1)).expect("apply");
assert_eq!(count(&conn, "SELECT COUNT(*) FROM note_labels"), 1);
let dead = wire::Label {
id: "l1".into(),
name: "work".into(),
color: "blue".into(),
sync_revision: 2,
purged_at: Some("2026-07-26T01:00:00.000Z".into()),
created_at: None,
};
apply_page(&conn, &page(vec![], vec![dead], 2)).expect("apply");
assert_eq!(count(&conn, "SELECT COUNT(*) FROM labels"), 0);
assert_eq!(
count(&conn, "SELECT COUNT(*) FROM note_labels"),
0,
"membership should cascade with the label"
);
}
#[test]
fn overwriting_a_dirty_note_is_counted() {
let conn = db();
conn.execute(
"INSERT INTO notes (id, body, created_at, updated_at, dirty)
VALUES ('n1', 'local edit', '2026-01-01', '2026-01-01', 1)",
[],
)
.expect("seed dirty note");
let summary = apply_page(&conn, &page(vec![note("n1", 9)], vec![], 9)).expect("apply");
assert_eq!(summary.clobbered_dirty, 1);
}
#[test]
fn applying_a_fresh_note_reports_no_clobber() {
let conn = db();
let summary = apply_page(&conn, &page(vec![note("n1", 1)], vec![], 1)).expect("apply");
assert_eq!(summary.clobbered_dirty, 0);
}
#[test]
fn empty_page_still_advances_the_cursor() {
// The server can page past rows that were trimmed to the shared watermark.
let conn = db();
apply_page(&conn, &page(vec![], vec![], 42)).expect("apply");
assert_eq!(state::read(&conn).expect("state").last_cursor, 42);
}
#[test]
fn note_upsert_preserves_the_original_created_at() {
let conn = db();
apply_page(&conn, &page(vec![note("n1", 1)], vec![], 1)).expect("apply");
let mut later = note("n1", 2);
later.created_at = Some("2099-01-01T00:00:00.000Z".into());
apply_page(&conn, &page(vec![later], vec![], 2)).expect("apply");
let created: String = conn
.query_row("SELECT created_at FROM notes WHERE id = 'n1'", [], |r| {
r.get(0)
})
.expect("created_at");
assert_eq!(created, "2026-07-26T00:00:00.000Z");
}
#[test]
fn a_page_that_fails_leaves_the_cursor_untouched() {
// Atomicity is the whole resumability story: a cursor committed ahead of its
// data would skip those rows forever. Force a failure with a duplicate
// checklist-item id inside one page.
let conn = db();
let mut n = note("n1", 3);
n.items = vec![
wire::Item {
id: "dup".into(),
text: "one".into(),
checked: false,
position: 0,
},
wire::Item {
id: "dup".into(),
text: "two".into(),
checked: false,
position: 1,
},
];
assert!(apply_page(&conn, &page(vec![n], vec![], 3)).is_err());
assert_eq!(state::read(&conn).expect("state").last_cursor, 0);
assert_eq!(count(&conn, "SELECT COUNT(*) FROM notes"), 0);
}
}
+751
View File
@@ -0,0 +1,751 @@
//! Push: send local changes to the server and apply what it says (M10.7c).
//!
//! Two sources feed a push: rows flagged `dirty` (created or edited locally) and rows
//! in `pending_deletes` (permanently deleted locally — see `local::schema` v2 for why
//! a delete needs its own record).
//!
//! Sync is **whole-note**: an upsert carries the client's full current state, not a
//! patch (docs/sync.md). The server resolves conflicts last-write-wins by the client's
//! `edited_at`, snapshotting anything it overwrites into the note's version history.
use rusqlite::{params, Connection, OptionalExtension};
use serde::{Deserialize, Serialize};
use super::client;
use super::state;
use crate::local::Db;
/// The server rejects a batch larger than this (`MAX_PUSH` in `sync.py`).
const BATCH: usize = 500;
/// Backstop: a batch whose results never clear `dirty` would loop forever.
const MAX_BATCHES: usize = 10_000;
#[derive(Debug, Clone, Default, Serialize, PartialEq, Eq)]
pub struct PushSummary {
pub batches: usize,
pub sent: usize,
pub created: usize,
pub applied: usize,
/// The server had a newer edit and kept it. Not a failure — the local row stops
/// being dirty and the following pull adopts the server's version.
pub kept: usize,
pub noop: usize,
/// Still dirty, and surfaced: these need a human (a duplicate label name is the
/// realistic case). Silently retrying forever would be the wrong shape.
pub rejected: usize,
pub errors: Vec<String>,
}
impl PushSummary {
fn absorb(&mut self, other: PushSummary) {
self.batches += other.batches;
self.sent += other.sent;
self.created += other.created;
self.applied += other.applied;
self.kept += other.kept;
self.noop += other.noop;
self.rejected += other.rejected;
self.errors.extend(other.errors);
}
}
// --- outgoing shapes ---------------------------------------------------------
/// One entry in the `changes` array. Notes and labels share the envelope; serde skips
/// the fields that don't apply, so the server sees exactly the shape docs/sync.md
/// describes for each entity.
#[derive(Debug, Serialize)]
pub struct Change {
pub entity: &'static str,
pub id: String,
pub op: &'static str,
pub edited_at: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub body: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub color: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub kind: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub pinned: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub archived: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub trashed: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub remind_at: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub recurrence: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub position: Option<i64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub items: Option<Vec<ItemOut>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub label_ids: Option<Vec<String>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub created_at: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
}
impl Change {
fn delete(entity: &'static str, id: String, edited_at: String) -> Self {
Change {
entity,
id,
op: "delete",
edited_at,
title: None,
body: None,
color: None,
kind: None,
pinned: None,
archived: None,
trashed: None,
remind_at: None,
recurrence: None,
position: None,
items: None,
label_ids: None,
created_at: None,
name: None,
}
}
}
#[derive(Debug, Serialize)]
pub struct ItemOut {
pub text: String,
pub checked: bool,
}
// --- incoming results --------------------------------------------------------
#[derive(Debug, Deserialize)]
struct PushResponse {
#[serde(default)]
results: Vec<PushResult>,
}
#[derive(Debug, Clone, Deserialize)]
pub struct PushResult {
#[serde(default)]
pub id: Option<String>,
#[serde(default)]
pub entity: Option<String>,
#[serde(default)]
pub status: String,
#[serde(default)]
pub sync_revision: Option<i64>,
#[serde(default)]
pub error: Option<String>,
}
// --- collecting --------------------------------------------------------------
/// Everything waiting to go up, oldest edit first so a truncated batch still makes
/// forward progress in a sensible order.
pub fn collect(conn: &Connection, limit: usize) -> rusqlite::Result<Vec<Change>> {
let mut out = Vec::new();
collect_deletes(conn, &mut out, limit)?;
if out.len() < limit {
collect_labels(conn, &mut out, limit)?;
}
if out.len() < limit {
collect_notes(conn, &mut out, limit)?;
}
Ok(out)
}
fn collect_deletes(conn: &Connection, out: &mut Vec<Change>, limit: usize) -> rusqlite::Result<()> {
let mut stmt = conn.prepare(
"SELECT entity, id, deleted_at FROM pending_deletes ORDER BY deleted_at LIMIT ?1",
)?;
let rows = stmt.query_map(params![limit as i64], |r| {
Ok((
r.get::<_, String>(0)?,
r.get::<_, String>(1)?,
r.get::<_, String>(2)?,
))
})?;
for row in rows {
let (entity, id, deleted_at) = row?;
// Only 'note' and 'label' exist on the wire; anything else is a bug in a
// writer, and shipping it would earn a blanket rejection for the batch.
let entity: &'static str = match entity.as_str() {
"note" => "note",
"label" => "label",
_ => continue,
};
out.push(Change::delete(entity, id, deleted_at));
}
Ok(())
}
fn collect_labels(conn: &Connection, out: &mut Vec<Change>, limit: usize) -> rusqlite::Result<()> {
let remaining = limit.saturating_sub(out.len());
let mut stmt = conn.prepare(
"SELECT id, name, color, updated_at FROM labels
WHERE dirty = 1 ORDER BY updated_at LIMIT ?1",
)?;
let rows = stmt.query_map(params![remaining as i64], |r| {
Ok(Change {
entity: "label",
id: r.get(0)?,
op: "upsert",
name: Some(r.get(1)?),
color: Some(r.get(2)?),
edited_at: r.get(3)?,
title: None,
body: None,
kind: None,
pinned: None,
archived: None,
trashed: None,
remind_at: None,
recurrence: None,
position: None,
items: None,
label_ids: None,
created_at: None,
})
})?;
for row in rows {
out.push(row?);
}
Ok(())
}
fn collect_notes(conn: &Connection, out: &mut Vec<Change>, limit: usize) -> rusqlite::Result<()> {
let remaining = limit.saturating_sub(out.len());
let ids: Vec<String> = {
let mut stmt =
conn.prepare("SELECT id FROM notes WHERE dirty = 1 ORDER BY updated_at LIMIT ?1")?;
let rows = stmt.query_map(params![remaining as i64], |r| r.get::<_, String>(0))?;
rows.collect::<rusqlite::Result<Vec<String>>>()?
};
for id in ids {
out.push(note_change(conn, &id)?);
}
Ok(())
}
/// The note's own columns. A named struct rather than a twelve-wide tuple so the
/// field-to-column mapping stays readable at the call site.
struct NoteRow {
title: Option<String>,
body: String,
color: String,
kind: String,
position: i64,
pinned: bool,
archived: bool,
trashed: bool,
remind_at: Option<String>,
recurrence: Option<String>,
created_at: String,
updated_at: String,
}
fn note_row(conn: &Connection, id: &str) -> rusqlite::Result<NoteRow> {
conn.query_row(
"SELECT title, body, color, kind, position, pinned, archived, trashed,
remind_at, recurrence, created_at, updated_at
FROM notes WHERE id = ?1",
params![id],
|r| {
Ok(NoteRow {
title: r.get(0)?,
body: r.get(1)?,
color: r.get(2)?,
kind: r.get(3)?,
position: r.get(4)?,
pinned: r.get::<_, i64>(5)? != 0,
archived: r.get::<_, i64>(6)? != 0,
trashed: r.get::<_, i64>(7)? != 0,
remind_at: r.get(8)?,
recurrence: r.get(9)?,
created_at: r.get(10)?,
updated_at: r.get(11)?,
})
},
)
}
fn note_change(conn: &Connection, id: &str) -> rusqlite::Result<Change> {
let row = note_row(conn, id)?;
let items = {
let mut stmt = conn.prepare(
"SELECT text, checked FROM checklist_items WHERE note_id = ?1 ORDER BY position",
)?;
let rows = stmt.query_map(params![id], |r| {
Ok(ItemOut {
text: r.get(0)?,
checked: r.get::<_, i64>(1)? != 0,
})
})?;
rows.collect::<rusqlite::Result<Vec<ItemOut>>>()?
};
// MANUAL memberships only. Tag-sourced ones (`via_tag = 1`) are re-derived by the
// server from the body; sending them as label_ids would convert them into manual
// assignments that no longer disappear when the #tag is removed from the text.
let label_ids = {
let mut stmt =
conn.prepare("SELECT label_id FROM note_labels WHERE note_id = ?1 AND via_tag = 0")?;
let rows = stmt.query_map(params![id], |r| r.get::<_, String>(0))?;
rows.collect::<rusqlite::Result<Vec<String>>>()?
};
Ok(Change {
entity: "note",
id: id.to_string(),
op: "upsert",
// The local `updated_at` IS the client's edit time, which is what the
// server's last-write-wins comparison runs against.
edited_at: row.updated_at,
title: row.title,
body: Some(row.body),
color: Some(row.color),
kind: Some(row.kind),
pinned: Some(row.pinned),
archived: Some(row.archived),
trashed: Some(row.trashed),
remind_at: row.remind_at,
recurrence: row.recurrence,
position: Some(row.position),
items: Some(items),
label_ids: Some(label_ids),
created_at: Some(row.created_at),
name: None,
})
}
// --- applying results --------------------------------------------------------
/// Fold one batch's results back into the local store, atomically.
pub fn apply_results(
conn: &Connection,
sent: &[Change],
results: &[PushResult],
) -> rusqlite::Result<PushSummary> {
let tx = conn.unchecked_transaction()?;
let mut summary = PushSummary {
batches: 1,
sent: sent.len(),
..Default::default()
};
// The server answers positionally, one result per change. Zip rather than trust
// the echoed id: a rejected malformed entry may carry no id at all.
let mut lowest_kept: Option<i64> = None;
for (change, result) in sent.iter().zip(results.iter()) {
match result.status.as_str() {
"created" | "applied" => {
clear_dirty(&tx, change, result.sync_revision)?;
if result.status == "created" {
summary.created += 1;
} else {
summary.applied += 1;
}
if change.op == "delete" {
forget_pending_delete(&tx, change)?;
}
}
"noop" => {
// The server had nothing to do — typically a delete for a row it
// never saw (created and deleted while offline).
clear_dirty(&tx, change, result.sync_revision)?;
forget_pending_delete(&tx, change)?;
summary.noop += 1;
}
"kept" => {
// The server's version is newer. Stop being dirty — re-pushing would
// lose to the same comparison forever — and let the next pull bring
// the server's copy down.
clear_dirty(&tx, change, None)?;
if change.op == "delete" {
// Our delete lost to a newer server edit; the note lives on, and
// the pull will restore it locally. Drop the tombstone so we
// don't keep trying to delete a note the user has since edited.
forget_pending_delete(&tx, change)?;
}
if let Some(revision) = result.sync_revision {
lowest_kept = Some(lowest_kept.map_or(revision, |c: i64| c.min(revision)));
}
summary.kept += 1;
}
_ => {
// "rejected" and anything unrecognized: leave the row dirty so it is
// retried, and surface the reason. A duplicate label name is the
// realistic case and only a human can resolve it.
summary.rejected += 1;
let reason = result
.error
.clone()
.unwrap_or_else(|| result.status.clone());
summary
.errors
.push(format!("{} {}: {reason}", change.entity, change.id));
}
}
}
// A `kept` result means the server holds a version we have not seen. Normally its
// revision is above our cursor and the next pull fetches it anyway. If it is NOT
// — which happens when a skewed clock makes a genuinely later local edit look
// older — rewind so that note is re-fetched. Without this the local edit is
// dropped from sync and the stale copy stays on screen with nothing marking it.
if let Some(revision) = lowest_kept {
let current = state::read(&tx)?.last_cursor;
if revision <= current {
state::set_cursor(&tx, (revision - 1).max(0))?;
}
}
tx.commit()?;
Ok(summary)
}
fn clear_dirty(conn: &Connection, change: &Change, revision: Option<i64>) -> rusqlite::Result<()> {
// A delete has no local row left to update.
if change.op == "delete" {
return Ok(());
}
let table = match change.entity {
"label" => "labels",
_ => "notes",
};
match revision {
Some(rev) => conn.execute(
&format!("UPDATE {table} SET dirty = 0, sync_revision = ?2 WHERE id = ?1"),
params![change.id, rev],
)?,
None => conn.execute(
&format!("UPDATE {table} SET dirty = 0 WHERE id = ?1"),
params![change.id],
)?,
};
Ok(())
}
fn forget_pending_delete(conn: &Connection, change: &Change) -> rusqlite::Result<()> {
if change.op != "delete" {
return Ok(());
}
conn.execute(
"DELETE FROM pending_deletes WHERE entity = ?1 AND id = ?2",
params![change.entity, change.id],
)?;
Ok(())
}
/// True when anything is waiting to go up. Cheap enough to call before a cycle.
pub fn has_pending(conn: &Connection) -> rusqlite::Result<bool> {
let pending: Option<i64> = conn
.query_row(
"SELECT 1 FROM notes WHERE dirty = 1
UNION ALL SELECT 1 FROM labels WHERE dirty = 1
UNION ALL SELECT 1 FROM pending_deletes LIMIT 1",
[],
|r| r.get(0),
)
.optional()?;
Ok(pending.is_some())
}
/// Send everything pending, in batches, applying each batch's results before the
/// next is collected.
pub async fn run(db: &Db, base_url: &str, token: &str) -> Result<PushSummary, String> {
let mut total = PushSummary::default();
loop {
let batch = {
let conn = db.0.lock().map_err(|e| e.to_string())?;
collect(&conn, BATCH).map_err(|e| e.to_string())?
};
if batch.is_empty() {
break;
}
let raw = client::push_changes(base_url, token, &batch).await?;
let results = parse_results(&raw)?;
let applied = {
let conn = db.0.lock().map_err(|e| e.to_string())?;
apply_results(&conn, &batch, &results).map_err(|e| e.to_string())?
};
// Everything rejected clears nothing, so the same batch would be collected
// again forever. Stop and report instead.
let progressed = applied.rejected < applied.sent;
total.absorb(applied);
if !progressed {
break;
}
if total.batches >= MAX_BATCHES {
return Err(format!(
"Stopped after {MAX_BATCHES} push batches without draining the queue."
));
}
}
if total.rejected > 0 {
log::warn!(
"push: {} change(s) rejected by the server: {}",
total.rejected,
total.errors.join("; ")
);
}
log::info!(
"push complete: {} sent ({} created, {} applied, {} kept, {} noop, {} rejected)",
total.sent,
total.created,
total.applied,
total.kept,
total.noop,
total.rejected
);
Ok(total)
}
/// Parse the server's reply. Kept next to the shapes it produces.
pub fn parse_results(raw: &str) -> Result<Vec<PushResult>, String> {
let parsed: PushResponse =
serde_json::from_str(raw).map_err(|e| format!("Couldn't read the push response: {e}"))?;
Ok(parsed.results)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::local::schema;
use crate::local::store;
fn db() -> Connection {
let conn = Connection::open_in_memory().expect("in-memory db");
schema::migrate(&conn).expect("migrate");
conn
}
fn seed_note(conn: &Connection, id: &str, dirty: i64) {
conn.execute(
"INSERT INTO notes (id, title, body, color, kind, position, pinned, archived,
trashed, created_at, updated_at, sync_revision, dirty)
VALUES (?1, 'T', 'B', 'default', 'text', 0, 0, 0, 0,
'2026-07-26T00:00:00.000Z', '2026-07-26T00:00:00.000Z', 3, ?2)",
params![id, dirty],
)
.expect("seed note");
}
fn ok(status: &str, revision: Option<i64>) -> PushResult {
PushResult {
id: None,
entity: None,
status: status.to_string(),
sync_revision: revision,
error: None,
}
}
fn dirty_count(conn: &Connection) -> i64 {
conn.query_row("SELECT COUNT(*) FROM notes WHERE dirty = 1", [], |r| {
r.get(0)
})
.expect("count")
}
#[test]
fn collects_only_dirty_notes() {
let conn = db();
seed_note(&conn, "clean", 0);
seed_note(&conn, "dirty", 1);
let batch = collect(&conn, 100).expect("collect");
assert_eq!(batch.len(), 1);
assert_eq!(batch[0].id, "dirty");
assert_eq!(batch[0].op, "upsert");
}
#[test]
fn sends_only_manual_label_memberships() {
// Tag-sourced labels are re-derived server-side. Sending them as label_ids
// would convert them to manual assignments that survive removing the #tag.
let conn = db();
seed_note(&conn, "n1", 1);
for (id, name, via_tag) in [("manual", "Manual", 0), ("tagged", "Tagged", 1)] {
conn.execute(
"INSERT INTO labels (id, name, color, created_at, updated_at, dirty)
VALUES (?1, ?2, 'default', '2026-01-01', '2026-01-01', 0)",
params![id, name],
)
.expect("seed label");
conn.execute(
"INSERT INTO note_labels (note_id, label_id, via_tag) VALUES ('n1', ?1, ?2)",
params![id, via_tag],
)
.expect("seed membership");
}
let batch = collect(&conn, 100).expect("collect");
let note = batch.iter().find(|c| c.entity == "note").expect("note");
assert_eq!(note.label_ids.as_deref(), Some(&["manual".to_string()][..]));
}
#[test]
fn a_local_delete_becomes_a_delete_change() {
let conn = db();
seed_note(&conn, "n1", 0);
store::delete_forever(&conn, "n1").expect("delete");
let batch = collect(&conn, 100).expect("collect");
assert_eq!(batch.len(), 1);
assert_eq!(batch[0].op, "delete");
assert_eq!(batch[0].entity, "note");
assert_eq!(batch[0].id, "n1");
}
#[test]
fn applied_clears_dirty_and_records_the_revision() {
let conn = db();
seed_note(&conn, "n1", 1);
let batch = collect(&conn, 100).expect("collect");
apply_results(&conn, &batch, &[ok("applied", Some(42))]).expect("apply");
assert_eq!(dirty_count(&conn), 0);
let rev: i64 = conn
.query_row("SELECT sync_revision FROM notes WHERE id = 'n1'", [], |r| {
r.get(0)
})
.expect("revision");
assert_eq!(rev, 42);
}
#[test]
fn kept_clears_dirty_so_it_is_not_pushed_forever() {
// The server has a newer edit. Re-pushing would lose the same comparison
// every time; the following pull adopts the server's version instead.
let conn = db();
seed_note(&conn, "n1", 1);
let batch = collect(&conn, 100).expect("collect");
let summary = apply_results(&conn, &batch, &[ok("kept", Some(99))]).expect("apply");
assert_eq!(summary.kept, 1);
assert_eq!(dirty_count(&conn), 0);
}
#[test]
fn kept_rewinds_the_cursor_when_the_server_version_is_already_behind_it() {
// Clock skew: a genuinely later local edit can look older, so the server
// keeps its copy at a revision we have ALREADY consumed. Without a rewind the
// next pull skips it and the stale local copy stays on screen silently.
let conn = db();
seed_note(&conn, "n1", 1);
state::set_cursor(&conn, 100).expect("cursor");
let batch = collect(&conn, 100).expect("collect");
apply_results(&conn, &batch, &[ok("kept", Some(40))]).expect("apply");
assert_eq!(state::read(&conn).expect("state").last_cursor, 39);
}
#[test]
fn kept_leaves_the_cursor_alone_when_the_server_version_is_ahead() {
let conn = db();
seed_note(&conn, "n1", 1);
state::set_cursor(&conn, 10).expect("cursor");
let batch = collect(&conn, 100).expect("collect");
apply_results(&conn, &batch, &[ok("kept", Some(40))]).expect("apply");
assert_eq!(
state::read(&conn).expect("state").last_cursor,
10,
"the pending pull already covers it"
);
}
#[test]
fn rejected_stays_dirty_and_is_reported() {
let conn = db();
seed_note(&conn, "n1", 1);
let batch = collect(&conn, 100).expect("collect");
let mut bad = ok("rejected", None);
bad.error = Some("name in use".into());
let summary = apply_results(&conn, &batch, &[bad]).expect("apply");
assert_eq!(summary.rejected, 1);
assert_eq!(dirty_count(&conn), 1, "a rejected change must be retried");
assert!(summary.errors[0].contains("name in use"));
}
#[test]
fn an_acknowledged_delete_drops_its_tombstone() {
let conn = db();
seed_note(&conn, "n1", 0);
store::delete_forever(&conn, "n1").expect("delete");
let batch = collect(&conn, 100).expect("collect");
apply_results(&conn, &batch, &[ok("applied", Some(7))]).expect("apply");
assert!(!has_pending(&conn).expect("pending"));
}
#[test]
fn a_noop_delete_also_drops_its_tombstone() {
// Created and deleted entirely offline: the server never saw it.
let conn = db();
seed_note(&conn, "n1", 1);
store::delete_forever(&conn, "n1").expect("delete");
let batch = collect(&conn, 100).expect("collect");
apply_results(&conn, &batch, &[ok("noop", None)]).expect("apply");
assert!(!has_pending(&conn).expect("pending"));
}
#[test]
fn merging_labels_marks_the_affected_notes_dirty() {
// The membership change only reaches the server through the note itself.
let conn = db();
seed_note(&conn, "n1", 0);
for (id, name) in [("src", "Source"), ("dst", "Target")] {
conn.execute(
"INSERT INTO labels (id, name, color, created_at, updated_at, dirty)
VALUES (?1, ?2, 'default', '2026-01-01', '2026-01-01', 0)",
params![id, name],
)
.expect("seed label");
}
conn.execute(
"INSERT INTO note_labels (note_id, label_id, via_tag) VALUES ('n1', 'src', 0)",
[],
)
.expect("seed membership");
store::merge_labels(&conn, "src", "dst").expect("merge");
assert_eq!(dirty_count(&conn), 1, "the note's label set changed");
}
#[test]
fn has_pending_is_false_on_a_clean_store() {
let conn = db();
seed_note(&conn, "n1", 0);
assert!(!has_pending(&conn).expect("pending"));
}
#[test]
fn parse_results_reads_the_documented_shape() {
let results = parse_results(
r#"{"results":[{"id":"a","entity":"note","status":"created","sync_revision":44},
{"id":"b","entity":"label","status":"rejected","error":"name in use"}]}"#,
)
.expect("parse");
assert_eq!(results.len(), 2);
assert_eq!(results[0].status, "created");
assert_eq!(results[1].error.as_deref(), Some("name in use"));
}
#[test]
fn a_delete_change_serializes_without_note_fields() {
let change = Change::delete("note", "n1".into(), "2026-07-26T00:00:00.000Z".into());
let json = serde_json::to_string(&change).expect("serialize");
assert!(json.contains("\"op\":\"delete\""), "got {json}");
assert!(
!json.contains("body"),
"a delete carries no content: {json}"
);
}
}
+342
View File
@@ -0,0 +1,342 @@
//! The link record: which server this app is paired with, the device token that
//! authenticates to it, and how far it has consumed that server's change feed.
//!
//! One row, enforced by `CHECK (id = 1)` and seeded during migration, so every
//! operation here is an UPDATE — there is no create-or-missing case to handle.
//!
//! The token lives in the app-data SQLite file rather than an OS keyring on purpose:
//! the `keyring` crate needs libsecret/DBus on Linux, which adds a C dependency to a
//! binary that has to cross-compile, and fails outright on headless or minimal-WM
//! setups. Protecting the database file is the portable trade.
use rusqlite::{params, Connection};
use serde::Serialize;
/// The full link record, token included. Internal to the Rust side.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct SyncState {
pub server_url: Option<String>,
pub device_token: Option<String>,
pub last_cursor: i64,
pub last_sync_at: Option<String>,
/// The linked server's trash-retention window, as it last advertised it. `None`
/// until a probe or sync has learned it.
pub server_retention_days: Option<i64>,
}
impl SyncState {
/// Linked means BOTH a server and a credential for it. Either one alone is a
/// half-written link that nothing can act on, so it must not read as linked.
pub fn is_linked(&self) -> bool {
self.server_url.is_some() && self.device_token.is_some()
}
}
/// What the UI is allowed to see.
///
/// Deliberately has no `device_token` field: this crosses into the webview, and a
/// long-lived bearer token has no business being reachable from page scripts.
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct Status {
pub linked: bool,
pub server_url: Option<String>,
pub last_cursor: i64,
pub last_sync_at: Option<String>,
}
impl From<&SyncState> for Status {
fn from(s: &SyncState) -> Self {
Status {
linked: s.is_linked(),
server_url: s.server_url.clone(),
last_cursor: s.last_cursor,
last_sync_at: s.last_sync_at.clone(),
}
}
}
/// Treat a blank string as absent, so a half-cleared row can't masquerade as linked.
fn present(value: Option<String>) -> Option<String> {
value.filter(|s| !s.trim().is_empty())
}
pub fn read(conn: &Connection) -> rusqlite::Result<SyncState> {
conn.query_row(
"SELECT server_url, device_token, last_cursor, last_sync_at, server_retention_days
FROM sync_state WHERE id = 1",
[],
|row| {
let cursor: Option<String> = row.get(2)?;
Ok(SyncState {
last_sync_at: present(row.get(3)?),
server_retention_days: row.get(4)?,
server_url: present(row.get(0)?),
device_token: present(row.get(1)?),
// Stored TEXT (schema) but used as an integer watermark. Absent or
// unparseable means "start from the beginning" — always the safe
// reading, because a redundant full sync costs time, never data,
// whereas a too-high cursor silently skips changes.
last_cursor: cursor.and_then(|c| c.trim().parse().ok()).unwrap_or(0),
})
},
)
}
/// Record a link.
///
/// Resets the change-feed cursor whenever the server differs from the one previously
/// linked. A cursor is only meaningful against the server that issued it; carrying
/// one across would silently skip every change on the new server below that
/// watermark — data loss that looks like a successful sync. Re-linking the SAME
/// server (after a token refresh, say) keeps the cursor, so a routine re-auth doesn't
/// force a full re-download.
pub fn set_link(conn: &Connection, server_url: &str, device_token: &str) -> rusqlite::Result<()> {
let keep_cursor = read(conn)?.server_url.as_deref() == Some(server_url);
conn.execute(
"UPDATE sync_state
SET server_url = ?1,
device_token = ?2,
last_cursor = CASE WHEN ?3 THEN last_cursor ELSE NULL END
WHERE id = 1",
params![server_url, device_token, keep_cursor],
)?;
Ok(())
}
/// Forget the server entirely.
///
/// Clears the cursor as well as the credentials: a cursor left behind would, on the
/// next link, be interpreted against a server that never issued it.
pub fn clear_link(conn: &Connection) -> rusqlite::Result<()> {
conn.execute(
"UPDATE sync_state
SET server_url = NULL, device_token = NULL, last_cursor = NULL,
last_sync_at = NULL, server_retention_days = NULL
WHERE id = 1",
[],
)?;
Ok(())
}
/// Remember the linked server's trash-retention window (0 = it never purges).
///
/// Refreshed on every sync rather than only at link time, so changing the setting on
/// the server reaches the desktop's Trash countdown on the next cycle instead of
/// waiting for someone to re-link.
pub fn set_server_retention(conn: &Connection, days: i64) -> rusqlite::Result<()> {
conn.execute(
"UPDATE sync_state SET server_retention_days = ?1 WHERE id = 1",
params![days],
)?;
Ok(())
}
/// The retention window in force on THIS device: the linked server's if we know it,
/// otherwise the caller's offline default. A linked device must never enforce or
/// advertise its own window over the server's.
pub fn effective_retention_days(conn: &Connection, offline_default: i64) -> rusqlite::Result<i64> {
let state = read(conn)?;
if !state.is_linked() {
return Ok(offline_default);
}
// Linked but the server hasn't told us yet (linked by an older build, or no sync
// has completed). Fall back to the default rather than claiming "kept forever".
Ok(state.server_retention_days.unwrap_or(offline_default))
}
/// Stamp a completed sync. The cursor can't stand in for this: it's a revision
/// watermark, and it doesn't move at all when a sync correctly finds nothing new —
/// so "synced a moment ago, no changes" would be indistinguishable from "never
/// synced" without it.
pub fn mark_synced(conn: &Connection, when: &str) -> rusqlite::Result<()> {
conn.execute(
"UPDATE sync_state SET last_sync_at = ?1 WHERE id = 1",
params![when],
)?;
Ok(())
}
/// Advance the consumed-change watermark. Called by the pull loop (M10.7b) only
/// after a page has been fully applied.
pub fn set_cursor(conn: &Connection, cursor: i64) -> rusqlite::Result<()> {
conn.execute(
"UPDATE sync_state SET last_cursor = ?1 WHERE id = 1",
params![cursor.to_string()],
)?;
Ok(())
}
pub fn status(conn: &Connection) -> rusqlite::Result<Status> {
Ok(Status::from(&read(conn)?))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::local::schema;
fn db() -> Connection {
let conn = Connection::open_in_memory().expect("in-memory db");
schema::migrate(&conn).expect("migrate");
conn
}
#[test]
fn fresh_store_is_unlinked() {
let conn = db();
let state = read(&conn).expect("read");
assert_eq!(state, SyncState::default());
assert!(!state.is_linked());
assert_eq!(state.last_cursor, 0);
}
#[test]
fn link_round_trips() {
let conn = db();
set_link(&conn, "https://notes.example.com", "tok-1").expect("link");
let state = read(&conn).expect("read");
assert!(state.is_linked());
assert_eq!(
state.server_url.as_deref(),
Some("https://notes.example.com")
);
assert_eq!(state.device_token.as_deref(), Some("tok-1"));
}
#[test]
fn an_unlinked_device_uses_its_own_retention_window() {
let conn = db();
assert_eq!(effective_retention_days(&conn, 30).expect("read"), 30);
}
#[test]
fn a_linked_device_adopts_the_servers_window() {
// Including 0 — a server that keeps trash forever must not have this device
// showing a 30-day countdown that will never fire.
let conn = db();
set_link(&conn, "https://notes.example.com", "tok-1").expect("link");
set_server_retention(&conn, 0).expect("retention");
assert_eq!(effective_retention_days(&conn, 30).expect("read"), 0);
set_server_retention(&conn, 90).expect("retention");
assert_eq!(effective_retention_days(&conn, 30).expect("read"), 90);
}
#[test]
fn a_linked_device_that_hasnt_heard_yet_falls_back() {
// Linked by an older build, or no cycle has completed. The default is a
// safer guess than "forever", which would promise a note is being kept.
let conn = db();
set_link(&conn, "https://notes.example.com", "tok-1").expect("link");
assert_eq!(effective_retention_days(&conn, 30).expect("read"), 30);
}
#[test]
fn unlinking_forgets_the_servers_window() {
let conn = db();
set_link(&conn, "https://notes.example.com", "tok-1").expect("link");
set_server_retention(&conn, 90).expect("retention");
clear_link(&conn).expect("unlink");
assert_eq!(read(&conn).expect("read").server_retention_days, None);
assert_eq!(effective_retention_days(&conn, 30).expect("read"), 30);
}
#[test]
fn relinking_the_same_server_keeps_the_cursor() {
let conn = db();
set_link(&conn, "https://a.example.com", "tok-1").expect("link");
set_cursor(&conn, 4242).expect("cursor");
// e.g. the token was revoked and the user re-authenticated.
set_link(&conn, "https://a.example.com", "tok-2").expect("relink");
let state = read(&conn).expect("read");
assert_eq!(
state.last_cursor, 4242,
"a re-auth shouldn't force a full re-sync"
);
assert_eq!(state.device_token.as_deref(), Some("tok-2"));
}
#[test]
fn linking_a_different_server_resets_the_cursor() {
let conn = db();
set_link(&conn, "https://a.example.com", "tok-1").expect("link");
set_cursor(&conn, 4242).expect("cursor");
set_link(&conn, "https://b.example.com", "tok-2").expect("relink");
assert_eq!(
read(&conn).expect("read").last_cursor,
0,
"a cursor from another server would skip everything below it"
);
}
#[test]
fn unlink_clears_the_cursor_too() {
let conn = db();
set_link(&conn, "https://a.example.com", "tok-1").expect("link");
set_cursor(&conn, 99).expect("cursor");
clear_link(&conn).expect("unlink");
let state = read(&conn).expect("read");
assert!(!state.is_linked());
assert_eq!(state.last_cursor, 0);
assert!(state.server_url.is_none());
assert!(state.device_token.is_none());
}
#[test]
fn unlink_clears_the_last_sync_stamp() {
// Otherwise a freshly-linked server would claim it synced at a time that
// belonged to a different one.
let conn = db();
set_link(&conn, "https://a.example.com", "tok-1").expect("link");
mark_synced(&conn, "2026-07-26T04:00:00.000Z").expect("stamp");
assert!(read(&conn).expect("read").last_sync_at.is_some());
clear_link(&conn).expect("unlink");
assert!(read(&conn).expect("read").last_sync_at.is_none());
}
#[test]
fn half_written_link_is_not_linked() {
let conn = db();
conn.execute(
"UPDATE sync_state SET server_url = 'https://a.example.com' WHERE id = 1",
[],
)
.expect("partial write");
assert!(!read(&conn).expect("read").is_linked());
}
#[test]
fn blank_strings_count_as_absent() {
let conn = db();
conn.execute(
"UPDATE sync_state SET server_url = ' ', device_token = '' WHERE id = 1",
[],
)
.expect("blank write");
let state = read(&conn).expect("read");
assert!(!state.is_linked());
assert!(state.server_url.is_none());
}
#[test]
fn unparseable_cursor_falls_back_to_a_full_sync() {
let conn = db();
conn.execute(
"UPDATE sync_state SET last_cursor = 'garbage' WHERE id = 1",
[],
)
.expect("bad cursor");
assert_eq!(read(&conn).expect("read").last_cursor, 0);
}
#[test]
fn status_never_carries_the_token() {
let conn = db();
set_link(&conn, "https://a.example.com", "super-secret").expect("link");
let json = serde_json::to_string(&status(&conn).expect("status")).expect("serialize");
assert!(
!json.contains("super-secret"),
"token leaked to the webview: {json}"
);
assert!(json.contains("\"linked\":true"), "got {json}");
}
}
+166
View File
@@ -0,0 +1,166 @@
//! The delta-feed JSON shapes, exactly as `GET /api/sync/changes` sends them.
//!
//! Mirrors the server's serializers (`notes/serialize.py` + `serialize.py`) — see
//! `docs/sync.md` for the contract. Every field is `#[serde(default)]` or `Option`
//! so a NEWER server adding fields, or an older one omitting one, degrades to a
//! partial note rather than failing the whole page. Losing one attribute is
//! recoverable; refusing a page stalls sync permanently at that cursor.
use serde::Deserialize;
#[derive(Debug, Clone, Deserialize, Default)]
pub struct ChangesPage {
#[serde(default)]
pub notes: Vec<Note>,
#[serde(default)]
pub labels: Vec<Label>,
#[serde(default)]
pub cursor: i64,
#[serde(default)]
pub has_more: bool,
}
#[derive(Debug, Clone, Deserialize)]
pub struct Note {
pub id: String,
#[serde(default)]
pub title: Option<String>,
#[serde(default)]
pub body: String,
#[serde(default = "default_color")]
pub color: String,
#[serde(default = "default_kind")]
pub kind: String,
#[serde(default)]
pub position: i64,
#[serde(default)]
pub pinned: bool,
#[serde(default)]
pub archived: bool,
/// The server derives this from `deleted_at` — trash, NOT a tombstone.
#[serde(default)]
pub trashed: bool,
/// WHEN it was trashed. The trash-retention clock runs from here, so it has to be
/// the server's timestamp rather than anything this device invents. Absent from an
/// older server, which is why it's optional rather than required.
#[serde(default)]
pub deleted_at: Option<String>,
#[serde(default)]
pub remind_at: Option<String>,
#[serde(default)]
pub recurrence: Option<String>,
#[serde(default)]
pub created_at: Option<String>,
#[serde(default)]
pub updated_at: Option<String>,
#[serde(default)]
pub sync_revision: i64,
/// Set means the row was permanently purged: a content-less tombstone whose only
/// job is to tell clients to delete their copy.
#[serde(default)]
pub purged_at: Option<String>,
#[serde(default)]
pub labels: Vec<NoteLabel>,
#[serde(default)]
pub items: Vec<Item>,
#[serde(default)]
pub attachments: Vec<Attachment>,
#[serde(default)]
pub previews: Vec<Preview>,
}
impl Note {
pub fn is_tombstone(&self) -> bool {
self.purged_at.is_some()
}
}
/// A label as it appears attached to a note. Carries enough to materialize the label
/// row itself, which is what lets a membership be applied even if the label's own
/// delta hasn't arrived (see `pull::apply_page`).
#[derive(Debug, Clone, Deserialize)]
pub struct NoteLabel {
pub id: String,
#[serde(default)]
pub name: String,
#[serde(default = "default_color")]
pub color: String,
/// True when the membership came from a `#tag` in the body rather than a manual
/// assignment. Applied verbatim rather than re-derived — see `pull::apply_page`.
#[serde(default)]
pub via_tag: bool,
}
#[derive(Debug, Clone, Deserialize)]
pub struct Item {
pub id: String,
#[serde(default)]
pub text: String,
#[serde(default)]
pub checked: bool,
#[serde(default)]
pub position: i64,
}
#[derive(Debug, Clone, Deserialize)]
pub struct Attachment {
pub id: String,
#[serde(default)]
pub url: String,
#[serde(default)]
pub filename: Option<String>,
#[serde(default = "default_mime")]
pub mime: String,
#[serde(default)]
pub size: Option<i64>,
#[serde(default)]
pub sha256: Option<String>,
}
#[derive(Debug, Clone, Deserialize)]
pub struct Preview {
pub id: String,
#[serde(default)]
pub url: String,
#[serde(default)]
pub title: Option<String>,
#[serde(default)]
pub description: Option<String>,
#[serde(default)]
pub image_url: Option<String>,
#[serde(default)]
pub site_name: Option<String>,
}
#[derive(Debug, Clone, Deserialize)]
pub struct Label {
pub id: String,
#[serde(default)]
pub name: String,
#[serde(default = "default_color")]
pub color: String,
#[serde(default)]
pub sync_revision: i64,
#[serde(default)]
pub purged_at: Option<String>,
#[serde(default)]
pub created_at: Option<String>,
}
impl Label {
pub fn is_tombstone(&self) -> bool {
self.purged_at.is_some()
}
}
fn default_color() -> String {
"default".to_string()
}
fn default_kind() -> String {
"text".to_string()
}
fn default_mime() -> String {
"application/octet-stream".to_string()
}