core: extract the store and sync engine into a shared crate (M12 step 1)
Android becomes a native Kotlin client over this same code (Scribe note 2730), so
the local store and sync engine stop being modules of the desktop app and become
`thoughtsync-core`, a crate with no UI framework in it at all.
This is a move, not a rewrite, and the measurement is why: every file in local/
and sync/ already carried ZERO Tauri references — 4,980 of 6,372 lines. The
coupling was 473 lines of command shim, which stays behind in the desktop crate
as src/commands/. Kept as git renames so history follows the files.
The desktop imports them under their old names (`use thoughtsync_core::{local,
sync}`) so every call site reads exactly as before. What moved is where they
live, not what they are.
Two things a workspace changes that are easy to miss, both caught before pushing:
[profile.release] now lives at the workspace ROOT. Cargo silently ignores
profiles declared by a non-root member — leaving it in the desktop crate would
have dropped lto/strip/opt-level from every release build with only a warning.
And a workspace shares ONE target dir, so the bundles moved from
desktop/src-tauri/target to target/. Thirteen references across publish-release,
debundle-graphics, verify.sh, package-prebuilt and the workflow now point there.
Pinning target-dir back would have been the smaller diff, but the Android lane
also produces Rust artifacts and they do not belong under desktop/.
Also retires the Tauri Android lane in the same push rather than leaving a path
that is being replaced: gen/android, android.yml and docs/android-dev.md are
gone, the mobile_entry_point attribute with them, and the lib drops to rlib —
staticlib/cdylib existed for Tauri mobile, and the .so Android loads will be
built from the core crate instead. Rule 22, no parallel path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,318 @@
|
||||
//! Local storage for attachment bytes (M10.7d).
|
||||
//!
|
||||
//! Content-addressed: a blob is filed under its own sha256, so the same image
|
||||
//! attached to five notes is stored once and re-downloading it is free. The hash is
|
||||
//! also the integrity check — bytes that don't hash to what the server advertised
|
||||
//! are refused rather than filed under a name that lies about them.
|
||||
//!
|
||||
//! Attachment METADATA rides the delta feed; only the bytes come through here
|
||||
//! (docs/sync.md).
|
||||
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::OnceLock;
|
||||
|
||||
/// A sha256 in lowercase hex, and nothing else.
|
||||
///
|
||||
/// This is a **path-safety** check, not a formatting nicety: the hash is taken
|
||||
/// straight from a server response and used as a filename. Without it, a hostile or
|
||||
/// buggy server could send `../../…` and steer a write outside the blob directory.
|
||||
fn is_hash(candidate: &str) -> bool {
|
||||
candidate.len() == 64 && candidate.bytes().all(|b| b.is_ascii_hexdigit())
|
||||
}
|
||||
|
||||
fn digest(bytes: &[u8]) -> String {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(bytes);
|
||||
hasher
|
||||
.finalize()
|
||||
.iter()
|
||||
.map(|b| format!("{b:02x}"))
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub struct BlobStore {
|
||||
root: PathBuf,
|
||||
}
|
||||
|
||||
impl BlobStore {
|
||||
/// Open (creating if needed) the blob directory.
|
||||
pub fn new(root: PathBuf) -> std::io::Result<Self> {
|
||||
fs::create_dir_all(&root)?;
|
||||
Ok(Self { root })
|
||||
}
|
||||
|
||||
pub fn root(&self) -> &Path {
|
||||
&self.root
|
||||
}
|
||||
|
||||
/// Where a blob lives, or `None` if the hash isn't one.
|
||||
pub fn path(&self, sha256: &str) -> Option<PathBuf> {
|
||||
let lower = sha256.to_ascii_lowercase();
|
||||
is_hash(&lower).then(|| self.root.join(lower))
|
||||
}
|
||||
|
||||
/// Whether we already hold these bytes. Drives the "don't download it twice"
|
||||
/// skip, which is the entire point of keying by content.
|
||||
pub fn has(&self, sha256: &str) -> bool {
|
||||
self.path(sha256).is_some_and(|p| p.is_file())
|
||||
}
|
||||
|
||||
/// File bytes under `expected`, refusing them if they don't hash to it.
|
||||
///
|
||||
/// Verifying on the way IN rather than on the way out means a corrupted transfer
|
||||
/// can never be served later as if it were genuine — and the next sync simply
|
||||
/// tries again, because the blob still counts as missing.
|
||||
pub fn store(&self, expected: &str, bytes: &[u8]) -> Result<PathBuf, String> {
|
||||
let path = self
|
||||
.path(expected)
|
||||
.ok_or_else(|| format!("refusing an attachment with a malformed hash: {expected}"))?;
|
||||
let actual = digest(bytes);
|
||||
if actual != expected.to_ascii_lowercase() {
|
||||
return Err(format!(
|
||||
"attachment failed its integrity check (expected {expected}, got {actual})"
|
||||
));
|
||||
}
|
||||
fs::write(&path, bytes).map_err(|e| format!("couldn't save an attachment: {e}"))?;
|
||||
Ok(path)
|
||||
}
|
||||
|
||||
pub fn read(&self, sha256: &str) -> Option<Vec<u8>> {
|
||||
fs::read(self.path(sha256)?).ok()
|
||||
}
|
||||
}
|
||||
|
||||
// --- Serving blobs to the webview (M10.7f) -----------------------------------
|
||||
//
|
||||
// A synced note's attachment `url` is the SERVER's relative path
|
||||
// (`/api/notes/<id>/attachments/<aid>`). In the desktop webview that resolves
|
||||
// against the app origin and 404s, and swapping in the absolute server URL wouldn't
|
||||
// help either — that route needs a bearer token the webview won't send, and it would
|
||||
// make an offline app fetch over the network to show a file it already has on disk.
|
||||
//
|
||||
// So the bytes are served locally, over a custom URI scheme, straight out of this
|
||||
// store. The webview then caches and range-requests them like any other resource,
|
||||
// which a `data:` URI would have thrown away.
|
||||
|
||||
/// The scheme the webview fetches attachment bytes over.
|
||||
pub const BLOB_SCHEME: &str = "tsblob";
|
||||
|
||||
/// The blob directory, published once the app has resolved its data dir.
|
||||
///
|
||||
/// A `OnceLock` rather than Tauri's managed state because the scheme handler is
|
||||
/// registered on the BUILDER, before `setup` has computed that path — and because
|
||||
/// reading it this way keeps the handler independent of which Tauri 2.x minor
|
||||
/// changed the handler's context argument.
|
||||
static SERVE_ROOT: OnceLock<PathBuf> = OnceLock::new();
|
||||
|
||||
pub fn publish_root(root: PathBuf) {
|
||||
let _ = SERVE_ROOT.set(root);
|
||||
}
|
||||
|
||||
/// The URL an `<img>`/`<audio>`/`<a href>` should point at for these bytes.
|
||||
///
|
||||
/// **The two forms are not interchangeable.** A custom scheme is reachable as
|
||||
/// `scheme://localhost/<path>` on Linux and macOS, but Windows and Android map it
|
||||
/// onto `http://scheme.localhost/<path>`. Getting this wrong breaks exactly one
|
||||
/// platform, silently, and CI cannot catch it — the runner is headless.
|
||||
pub fn url_for(sha256: &str, mime: &str) -> String {
|
||||
let query = urlencode(mime);
|
||||
if cfg!(any(windows, target_os = "android")) {
|
||||
format!("http://{BLOB_SCHEME}.localhost/{sha256}?mime={query}")
|
||||
} else {
|
||||
format!("{BLOB_SCHEME}://localhost/{sha256}?mime={query}")
|
||||
}
|
||||
}
|
||||
|
||||
/// Percent-encode the few characters a mime type can contain that don't belong in a
|
||||
/// query value. Hand-rolled rather than adding a dependency for `/` and `+`.
|
||||
fn urlencode(value: &str) -> String {
|
||||
let mut out = String::with_capacity(value.len() + 8);
|
||||
for b in value.bytes() {
|
||||
match b {
|
||||
b'a'..=b'z' | b'A'..=b'Z' | b'0'..=b'9' | b'-' | b'.' | b'_' | b'~' => {
|
||||
out.push(b as char)
|
||||
}
|
||||
_ => out.push_str(&format!("%{b:02X}")),
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn urldecode(value: &str) -> String {
|
||||
let bytes = value.as_bytes();
|
||||
let mut out: Vec<u8> = Vec::with_capacity(bytes.len());
|
||||
let mut i = 0;
|
||||
while i < bytes.len() {
|
||||
if bytes[i] == b'%' && i + 2 < bytes.len() {
|
||||
let hex = std::str::from_utf8(&bytes[i + 1..i + 3]).unwrap_or("");
|
||||
if let Ok(byte) = u8::from_str_radix(hex, 16) {
|
||||
out.push(byte);
|
||||
i += 3;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
out.push(bytes[i]);
|
||||
i += 1;
|
||||
}
|
||||
String::from_utf8_lossy(&out).into_owned()
|
||||
}
|
||||
|
||||
/// The Content-Type to serve for a claimed mime.
|
||||
///
|
||||
/// The mime rides in the URL and this scheme is an origin of its own, so echoing an
|
||||
/// arbitrary type would let an attachment claiming `text/html` run as a document
|
||||
/// there. Echoing is safe only because of the FAMILY check: nothing starting with
|
||||
/// `image/` can name a scriptable type. Everything else is served as an opaque
|
||||
/// download — the right treatment for an arbitrary file regardless.
|
||||
fn content_type_for(mime: &str) -> String {
|
||||
const RENDERABLE: &[&str] = &["image/", "audio/", "video/"];
|
||||
let familiar = RENDERABLE.iter().any(|p| mime.starts_with(p)) || mime == "application/pdf";
|
||||
// A header value can't carry control characters, and a mime type has no business
|
||||
// being long — both would only arrive from a malformed or hostile feed.
|
||||
let printable = mime.len() <= 100 && mime.bytes().all(|b| b.is_ascii_graphic());
|
||||
if familiar && printable {
|
||||
mime.to_string()
|
||||
} else {
|
||||
"application/octet-stream".to_string()
|
||||
}
|
||||
}
|
||||
|
||||
/// Serve one request from the blob store. `path` is the URI path, `query` its query.
|
||||
pub fn serve(path: &str, query: Option<&str>) -> (u16, String, Vec<u8>) {
|
||||
let requested = path.trim_start_matches('/');
|
||||
let Some(root) = SERVE_ROOT.get() else {
|
||||
// A request before the store was published — nothing to serve yet.
|
||||
return (503, "text/plain".into(), Vec::new());
|
||||
};
|
||||
let store = BlobStore { root: root.clone() };
|
||||
// `read` goes through `path`, which rejects anything that isn't a bare sha256 —
|
||||
// so this handler inherits the traversal guard rather than re-implementing it.
|
||||
let Some(bytes) = store.read(requested) else {
|
||||
return (404, "text/plain".into(), Vec::new());
|
||||
};
|
||||
let claimed = query
|
||||
.and_then(|q| q.split('&').find_map(|p| p.strip_prefix("mime=")))
|
||||
.map(urldecode)
|
||||
.unwrap_or_default();
|
||||
(200, content_type_for(&claimed), bytes)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// A blob store in a throwaway directory. No tempfile dependency for one test
|
||||
/// fixture — the process id keeps concurrent runs apart.
|
||||
fn store(tag: &str) -> BlobStore {
|
||||
let dir = std::env::temp_dir().join(format!("ts-blobs-{}-{tag}", std::process::id()));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
BlobStore::new(dir).expect("store")
|
||||
}
|
||||
|
||||
/// sha256("hello") — a fixed vector, so a broken digest can't agree with itself.
|
||||
const HELLO: &str = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824";
|
||||
|
||||
#[test]
|
||||
fn digest_matches_a_known_vector() {
|
||||
assert_eq!(digest(b"hello"), HELLO);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stores_and_reads_back() {
|
||||
let store = store("roundtrip");
|
||||
assert!(!store.has(HELLO));
|
||||
store.store(HELLO, b"hello").expect("store");
|
||||
assert!(store.has(HELLO));
|
||||
assert_eq!(store.read(HELLO).as_deref(), Some(&b"hello"[..]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn refuses_bytes_that_dont_match_the_hash() {
|
||||
// A corrupted or substituted transfer must never be filed under a name that
|
||||
// claims it's genuine.
|
||||
let store = store("mismatch");
|
||||
let err = store.store(HELLO, b"goodbye").expect_err("must reject");
|
||||
assert!(err.contains("integrity"), "got {err}");
|
||||
assert!(!store.has(HELLO), "nothing should have been written");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_a_hash_that_could_escape_the_directory() {
|
||||
// The hash arrives from a server response and becomes a filename.
|
||||
let store = store("traversal");
|
||||
assert!(store.path("../../etc/passwd").is_none());
|
||||
assert!(store.store("../../etc/passwd", b"x").is_err());
|
||||
assert!(store.path("").is_none());
|
||||
assert!(store.path("nothex!!").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_an_uppercase_hash() {
|
||||
// The wire format isn't guaranteed to be lowercase; the filename is.
|
||||
let store = store("case");
|
||||
store
|
||||
.store(&HELLO.to_ascii_uppercase(), b"hello")
|
||||
.expect("store");
|
||||
assert!(store.has(HELLO), "should be found under the lowercase name");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_blob_url_carries_the_hash_and_the_mime() {
|
||||
let url = url_for(HELLO, "image/png");
|
||||
assert!(url.contains(HELLO), "the hash addresses the bytes: {url}");
|
||||
assert!(url.contains("mime=image%2Fpng"), "mime encoded: {url}");
|
||||
// The platform split is the whole risk of this feature, and CI is headless,
|
||||
// so at least pin that the right branch was taken for THIS build.
|
||||
if cfg!(any(windows, target_os = "android")) {
|
||||
assert!(url.starts_with("http://tsblob.localhost/"), "{url}");
|
||||
} else {
|
||||
assert!(url.starts_with("tsblob://localhost/"), "{url}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn url_encoding_round_trips_a_mime() {
|
||||
assert_eq!(urldecode(&urlencode("image/svg+xml")), "image/svg+xml");
|
||||
assert_eq!(urldecode(&urlencode("audio/mpeg")), "audio/mpeg");
|
||||
// A malformed escape is left alone rather than eaten — the value still has to
|
||||
// survive intact enough for `content_type_for` to reject it.
|
||||
assert_eq!(urldecode("not-an-escape%ZZ"), "not-an-escape%ZZ");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn media_types_are_echoed_back() {
|
||||
assert_eq!(content_type_for("image/png"), "image/png");
|
||||
assert_eq!(content_type_for("audio/mpeg"), "audio/mpeg");
|
||||
assert_eq!(content_type_for("application/pdf"), "application/pdf");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_scriptable_type_is_served_as_a_download() {
|
||||
// This scheme is an origin of its own. An attachment claiming to be HTML
|
||||
// must not be handed back as a document that can run there.
|
||||
let opaque = "application/octet-stream";
|
||||
assert_eq!(content_type_for("text/html"), opaque);
|
||||
assert_eq!(content_type_for("application/javascript"), opaque);
|
||||
assert_eq!(content_type_for(""), opaque);
|
||||
// A control character can't reach a header value even under a safe family.
|
||||
assert_eq!(content_type_for("image/png\r\nX-Evil: 1"), opaque);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serving_refuses_a_path_that_isnt_a_hash() {
|
||||
// Delegated to `path`, so the traversal guard is the same one `store` uses.
|
||||
publish_root(std::env::temp_dir().join("ts-blobs-serve-guard"));
|
||||
let (status, _, body) = serve("/../../etc/passwd", None);
|
||||
assert_eq!(status, 404);
|
||||
assert!(body.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_blob_reads_as_none() {
|
||||
let store = store("missing");
|
||||
assert!(store.read(HELLO).is_none());
|
||||
assert!(!store.has(HELLO));
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user