M10.7d: download attachment bytes into a content-addressed store (task 2107)
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 5s
CI & Build / Python tests (push) Successful in 8s
CI & Build / Build & push image (push) Successful in 35s
Desktop (Tauri) / Tauri desktop (Linux) (push) Failing after 1m29s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 1m47s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 5s
CI & Build / Python tests (push) Successful in 8s
CI & Build / Build & push image (push) Successful in 35s
Desktop (Tauri) / Tauri desktop (Linux) (push) Failing after 1m29s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 1m47s
The client half of task 1942's server work. Metadata already rides the delta feed; this fetches the payload so a synced image exists on the device. Blobs are filed under their own sha256, so the same image attached to five notes is stored once and re-downloading it is free — the dedupe the task asks for falls out of content addressing rather than needing bookkeeping. The hash is also the integrity check, applied on the way IN. Bytes that don't hash to what the server advertised are refused rather than filed under a name that lies about them — and because the blob then still counts as missing, the next sync simply tries again. SECURITY: the hash arrives in a server response and becomes a FILENAME, so it is validated as 64 hex characters before touching the filesystem. Without that, a hostile or buggy server could send "../../..." and steer a write outside the blob directory. Tested. A failed attachment never fails the sync. Notes are the primary data and have already landed; aborting here would let one unreachable file block every future sync. Counted, logged, surfaced in the UI as "they'll retry on the next sync", and retried because the blob is still absent. sha2 is pure Rust, so the Windows cross-compile lane pays nothing for it — the constraint recorded in ci-requirements.md. SPLIT, deliberately: this stores the bytes but does NOT yet render them in the webview. That half needs a custom URI scheme or the asset protocol, whose URL form differs by platform (Windows uses http://scheme.localhost/, others scheme://localhost/) — and CI cannot verify webview rendering at all, being headless with no webview. Guessing at it here would ship an unverifiable change on the most fragile lane. Follow-up filed; synced images will show as broken until it lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SreJkbxB4gx8pPsu8QbLPi
This commit is contained in:
@@ -0,0 +1,148 @@
|
||||
//! Local storage for attachment bytes (M10.7d).
|
||||
//!
|
||||
//! Content-addressed: a blob is filed under its own sha256, so the same image
|
||||
//! attached to five notes is stored once and re-downloading it is free. The hash is
|
||||
//! also the integrity check — bytes that don't hash to what the server advertised
|
||||
//! are refused rather than filed under a name that lies about them.
|
||||
//!
|
||||
//! Attachment METADATA rides the delta feed; only the bytes come through here
|
||||
//! (docs/sync.md).
|
||||
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// A sha256 in lowercase hex, and nothing else.
|
||||
///
|
||||
/// This is a **path-safety** check, not a formatting nicety: the hash is taken
|
||||
/// straight from a server response and used as a filename. Without it, a hostile or
|
||||
/// buggy server could send `../../…` and steer a write outside the blob directory.
|
||||
fn is_hash(candidate: &str) -> bool {
|
||||
candidate.len() == 64 && candidate.bytes().all(|b| b.is_ascii_hexdigit())
|
||||
}
|
||||
|
||||
fn digest(bytes: &[u8]) -> String {
|
||||
use sha2::{Digest, Sha256};
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(bytes);
|
||||
hasher
|
||||
.finalize()
|
||||
.iter()
|
||||
.map(|b| format!("{b:02x}"))
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub struct BlobStore {
|
||||
root: PathBuf,
|
||||
}
|
||||
|
||||
impl BlobStore {
|
||||
/// Open (creating if needed) the blob directory.
|
||||
pub fn new(root: PathBuf) -> std::io::Result<Self> {
|
||||
fs::create_dir_all(&root)?;
|
||||
Ok(Self { root })
|
||||
}
|
||||
|
||||
pub fn root(&self) -> &Path {
|
||||
&self.root
|
||||
}
|
||||
|
||||
/// Where a blob lives, or `None` if the hash isn't one.
|
||||
pub fn path(&self, sha256: &str) -> Option<PathBuf> {
|
||||
let lower = sha256.to_ascii_lowercase();
|
||||
is_hash(&lower).then(|| self.root.join(lower))
|
||||
}
|
||||
|
||||
/// Whether we already hold these bytes. Drives the "don't download it twice"
|
||||
/// skip, which is the entire point of keying by content.
|
||||
pub fn has(&self, sha256: &str) -> bool {
|
||||
self.path(sha256).is_some_and(|p| p.is_file())
|
||||
}
|
||||
|
||||
/// File bytes under `expected`, refusing them if they don't hash to it.
|
||||
///
|
||||
/// Verifying on the way IN rather than on the way out means a corrupted transfer
|
||||
/// can never be served later as if it were genuine — and the next sync simply
|
||||
/// tries again, because the blob still counts as missing.
|
||||
pub fn store(&self, expected: &str, bytes: &[u8]) -> Result<PathBuf, String> {
|
||||
let path = self
|
||||
.path(expected)
|
||||
.ok_or_else(|| format!("refusing an attachment with a malformed hash: {expected}"))?;
|
||||
let actual = digest(bytes);
|
||||
if actual != expected.to_ascii_lowercase() {
|
||||
return Err(format!(
|
||||
"attachment failed its integrity check (expected {expected}, got {actual})"
|
||||
));
|
||||
}
|
||||
fs::write(&path, bytes).map_err(|e| format!("couldn't save an attachment: {e}"))?;
|
||||
Ok(path)
|
||||
}
|
||||
|
||||
pub fn read(&self, sha256: &str) -> Option<Vec<u8>> {
|
||||
fs::read(self.path(sha256)?).ok()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// A blob store in a throwaway directory. No tempfile dependency for one test
|
||||
/// fixture — the process id keeps concurrent runs apart.
|
||||
fn store(tag: &str) -> BlobStore {
|
||||
let dir = std::env::temp_dir().join(format!("ts-blobs-{}-{tag}", std::process::id()));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
BlobStore::new(dir).expect("store")
|
||||
}
|
||||
|
||||
/// sha256("hello") — a fixed vector, so a broken digest can't agree with itself.
|
||||
const HELLO: &str = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824";
|
||||
|
||||
#[test]
|
||||
fn digest_matches_a_known_vector() {
|
||||
assert_eq!(digest(b"hello"), HELLO);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stores_and_reads_back() {
|
||||
let store = store("roundtrip");
|
||||
assert!(!store.has(HELLO));
|
||||
store.store(HELLO, b"hello").expect("store");
|
||||
assert!(store.has(HELLO));
|
||||
assert_eq!(store.read(HELLO).as_deref(), Some(&b"hello"[..]));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn refuses_bytes_that_dont_match_the_hash() {
|
||||
// A corrupted or substituted transfer must never be filed under a name that
|
||||
// claims it's genuine.
|
||||
let store = store("mismatch");
|
||||
let err = store.store(HELLO, b"goodbye").expect_err("must reject");
|
||||
assert!(err.contains("integrity"), "got {err}");
|
||||
assert!(!store.has(HELLO), "nothing should have been written");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_a_hash_that_could_escape_the_directory() {
|
||||
// The hash arrives from a server response and becomes a filename.
|
||||
let store = store("traversal");
|
||||
assert!(store.path("../../etc/passwd").is_none());
|
||||
assert!(store.store("../../etc/passwd", b"x").is_err());
|
||||
assert!(store.path("").is_none());
|
||||
assert!(store.path("nothex!!").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_an_uppercase_hash() {
|
||||
// The wire format isn't guaranteed to be lowercase; the filename is.
|
||||
let store = store("case");
|
||||
store.store(&HELLO.to_ascii_uppercase(), b"hello").expect("store");
|
||||
assert!(store.has(HELLO), "should be found under the lowercase name");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn missing_blob_reads_as_none() {
|
||||
let store = store("missing");
|
||||
assert!(store.read(HELLO).is_none());
|
||||
assert!(!store.has(HELLO));
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user