12 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	fa89da1fab	notes: color leaves the model, the wire and all three surfaces ... Step 3 of M315, and the destructive half. Steps 1 and 2 stopped every read of this field: a card is one neutral surface per theme, and the only coloured thing on a board is a tag. What was left was a column written by a picker and read by nothing. Rule 22 — the old path comes out completely. No flag, no fallback, no "override if set". Server: the column, the `?color=` facet, the create/update/serialise paths, the sync assignment, the front-matter line, and Keep's colour map. Alembic 0029 drops it and sweeps `"color"` out of stored saved-filter params — a view that silently filtered on a field the app no longer has would return nothing and never say why. That sweep is Python, not `params::jsonb - 'color'`, because Postgres has no try-cast and one malformed blob would abort a migration that is running over somebody's saved views. `NOTE_COLORS` moves from `models/note.py` to `colors.py`. A palette defined on the model that lost one is an invitation to put the column back; labels still name a colour, so the vocabulary belongs where the normalizer already is. Core: the field, the facet, the `NoteCreateInput`, and every read and write in store/push/pull. Local schema v9 drops the column and does the same saved-filter sweep, guarded on `json_valid` so a corrupt blob loses a key rather than becoming NULL. The uniffi layer drops `NoteEdit::Color` and `NoteDraft.color` with it. Web: `ColorPicker.vue`, the per-card swatch popover and its stylesheet rule, the FilterBar colour row, the facet in the query round-trip, and the colour half of the editor's baseline-and-save. Android: the `ColorSheet`, the `Picker.COLOR` case, the toolbar's swatch dot, `EditorAction.SetColor`. ## The protocol: v4, and the floor deliberately stays at 3 Checked against `compat.rs` and the push handler rather than trusting the `#[serde(default)]` annotation, because the v2 precedent points the other way: v2 dropped `kind` and `title` and DID raise both floors, on the rule that dropping a field a client sends and expects back is breaking. `color` fails the second half of that test. A v3 client reading a v4 note gets `"default"` from its own serde default and draws the colour it derives locally — the board it drew yesterday. A v3 client pushing `color` has the key ignored, since `_assign_note_fields` reads its payload key by key and never validates the shape. Neither direction errors and neither shows anything wrong. `title` was the note's NAME; this is a field that no longer renders. So `SYNC_PROTOCOL_VERSION` and `CLIENT_PROTOCOL_VERSION` go to 4, and both floors stay at 3. `docs/sync.md` carries the reasoning and the per-version history, and its push example is brought back in line — it still listed `title`, `kind` and `items`, all gone before this. Import stays tolerant: a pre-M315 export or a Keep takeout carrying `color:` imports fine, the key simply read past. Old exports must still import. #3041 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-28 14:07:03 -04:00
bvandeusenandClaude Opus 5	ad48d30c68	M311 step 1 — lift a tag that is standing on its own ... A tag was shown twice: once as the `#todo` you typed and once as a chip. The chip moved to the top of the card in 23fd2da; now the text goes — but only when the tag was the whole line. THE RULE: a line containing nothing but tags and whitespace is removed. Anything else is untouched. That is the conservative reading of "standalone" and it is the operator's: "only lift standalone tags, leave mid-sentence ones alone". The looser reading, also stripping a trailing tag off a prose line, is rejected because the text does not say which kind it is — `buy milk #grocery` is filing, `remember to call #mom` is the sentence's object, and lifting the second leaves "remember to call". Mangling a sentence to save a duplicate chip is a bad trade. Two guards. A line inside a ``` fence is never touched: a `#tag` there is a shell comment in somebody's snippet, and deleting it would eat a line of their example. And a note that is NOTHING but tags keeps its text rather than being blanked — a duplicated chip beats an empty card. WHY THIS IS NOT JUST A TEXT EDIT. `via_tag` labels are DERIVED from the body: reconcile detaches any row no longer backed by a `#tag`, and the picker only manages `via_tag=False` rows. So a naive lift deletes every tag on the next save, and leaves them unremovable until then. Resolved by giving `via_tag` a sharper meaning — backed by text still in the body — rather than deleting it: standalone lifted, attached as an ORDINARY label. Nothing derives it any more because nothing is left to derive it from. inline left in place, still derived, still detached when its text goes. Which costs nothing elsewhere, because both editors already gate their remove button on `!via_tag` (NoteEditor.vue:618, EditorChrome.kt:349). A lifted tag gets its × for free — and needs it, since deleting the text is no longer a way to remove one. No wire change, no column drop, no UI change. A tag that GRADUATES from inline to standalone is the sharp edge: its row has to be flipped before the detach pass, or the same row is dropped for no longer being in the body. That is the bug, and there is a test on it. The lift and the display_title re-derivation both live inside the function, which is renamed to admit it mutates the body. All seven call sites derive display_title BEFORE calling, so anywhere else and every note would be named after a line that had just been deleted. Spreading a derived-value update across seven write paths is the failure #2965 named: "easy to miss, and it is the common one". Existing notes lift lazily, on their next save. The migration that does the rest is step 2, and the core's own copy of the rule is step 3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-26 19:44:43 -04:00
bvandeusen	761c3b5e82	server: the body is the checklist here too, and note_items is dropped ... M304 steps 3 and the server half of 4. The client half landed in 668f7fa; these belong in one deploy, and the protocol floor below is what enforces that. notes/checklist.py is the Python half of a grammar that now exists three times — here, core/src/local/derive.rs, and (next) frontend/src/notes/markdown.ts. That triplication is the deliberate cost: the alternative is a round trip to the server before a phone can draw a checkbox. Each copy names the other two, and each is tested against the same table of cases, including the near-misses that must stay prose: `-[ ] x`, `- []`, `- [ ]x`, a `[ ]` mid-sentence. Routes: add/update/delete items stop touching rows and rewrite note.body, all through one _rewrite_body that runs the same sequence the PATCH route runs for a body change — because it IS a body change. Revisions, #tag reconciliation, the name, and link unfurls therefore happen in one place rather than three routes each remembering to. The reorder route is gone (rule 22). Reordering a checklist is moving a line, and no client ever called it — the only reference in the tree was a test asserting the route existed. The API still returns `items`, DERIVED from the body on the way out. That is not a second source of truth and it cannot disagree with the body it came from; it keeps the web client working across the rest of this milestone and saves any consumer that only wants to draw checkboxes from carrying a parser. Export drops its separate items block, in both formats. The body already ends with those exact lines, so writing them again would double every checklist in an export and then double it again on re-import. Import still ACCEPTS items, because a Keep takeout has a list and not a blob; it folds them in before the Note is built, so display_title and _reconcile_tags both see the finished text. Protocol 3 on both sides now. A v2 client is refused rather than half-served — which matters more than I first said: _apply_note_items returned early on an absent `items` key, so an un-bumped v3 client against a v2 server would not have LOST the rows, it would have kept them and then had the migration fold them a second time. Duplicated lists rather than missing ones. The floor prevents both. Migration 0027 folds every existing row into its note's body and drops the table. It inlines its own copy of the fold on purpose — a migration has to keep producing what it produced the day it ran — and a test pins that copy against the app's until they are allowed to diverge. updated_at is deliberately untouched: a client holding an unpushed edit keeps the newer timestamp, so last-write-wins keeps its work instead of the migration silently winning. The downgrade is honest rather than faithful. It recreates an empty note_items and leaves the bodies alone, because once items are lines nothing distinguishes one this migration wrote from one somebody typed, and a downgrade that guessed would eat hand-written lists. Recreating the table is still necessary: 0015's downgrade drops a trigger ON note_items, and IF EXISTS covers the trigger, not the table.	2026-08-24 08:03:37 -04:00
bvandeusenandClaude Opus 5	2707054563	A write should not cost a revision ... Every body change snapshotted into history — core/src/local/store.rs and notes/__init__.py both — so a write was expensive, and the clients compensated by writing as rarely as they could. BoardViewModel says it outright: "Saved on close rather than per keystroke, so a session of typing costs one write and one revision snapshot." That is durability paying for version history. An app kill mid-session lost everything typed, so that the revision list would stay tidy. The safety property is worth more than the feature it was subsidising, and no comparable product makes this trade: Keep and Apple Notes write continuously with no history, Docs and Notion write continuously and coalesce history behind the scenes, Obsidian debounces and snapshots on an interval. Save-on-close is the outlier, and this coupling is why we had it. A body change now earns a snapshot only if it is the first of an editing session — the body actually differs, and the note carries no revision from the last ten minutes. Session granularity falls out of the window rather than being declared. A snapshot stores the body as it was BEFORE the edit, so the first write of a sitting captures the note as you found it and every write after it inside the window adds nothing. One revision per sitting, with no commit flag for a client to send and no wire surface to carry it. That is why it is a time rule and not a protocol one. sync.py applies pushed bodies through the same check, so a client autosaving every second cannot make the server snapshot every second either — which a client-declared commit point could not have guaranteed without a protocol bump. Restoring a revision still snapshots unconditionally: a considered act, not a keystroke, and it stays undoable. Unblocks idle-debounced autosave, an honest updated_at, and the "Edited just now" line the editor is getting. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-23 21:47:01 -04:00
bvandeusen	7033995975	search is a facet on the board, not a place you go ... Operator (note 2930): tags exist so you can *"filter during a search"*. The server has always been able to do that — `GET /api/notes` composes `?q=` with `?label=` and the rest into one AND-ed query. The frontend never reached it. The header search box navigated to `/search`, and that view called a DIFFERENT endpoint — `GET /api/notes/search?q=`, full text only, no facets at all. So the one screen you landed on when you searched was the one screen where you could not narrow by tag. Tag filtering lived on the board's FilterBar, which is where you weren't searching. Two search boxes, two endpoints, and only the hidden one did what tags are for. Now the header box writes `?q=` into the board's URL beside whatever labels are already there, and stays on the lens you're in — searching while looking at Trash searches Trash. The box READS from the URL rather than holding its own copy, so it stays in step with the Filters panel's Clear and with a saved view opened from the sidebar. Deleted: `SearchView.vue`, its route, `GET /api/notes/search`, `repo.notes.search` and both adapter implementations, and the `notes_search` Tauri command whose only caller was the adapter entry. FilterBar loses its own "Search text…" input — it was the same facet, hidden behind a collapsed panel, duplicating a box that is always on screen. Filters now does what its name says: narrowing. The header does searching. `core::store::search` STAYS. Android calls it through the FFI (`search_notes`) and has its own search surface — which has the same no-tag-filter gap the web just lost, and deserves the same fix on its own terms rather than as a rider here.	2026-08-23 10:58:23 -04:00
bvandeusen	de72d27bd4	URLs unfurl on their own, and a lone link becomes the note ... Operator: *"I'd like for URLs to unfurl. To be the whole note when the note is a single URL, and to be a compact slot on the bottom of the note when the URL is inline. We also need to support multiple URLs in a single note."* Less new machinery than it sounds: `unfurl.py` already fetched and parsed OG tags, SSRF-hardened, and `note_link_previews` was already `UNIQUE(note_id, url)` — so several URLs per note has worked at the storage layer all along. What was missing was that it needed a button, had one size, and drew that size in the wrong place. **Automatic, and never in the way.** New `unfurl_queue.py` detects a body's URLs and fetches them on a background task AFTER the note is committed. Capture speed is the product: an unfurl is a five-second timeout against a host nobody controls, and a note has to persist the instant someone stops typing. Scheduled from create, from a body edit, and from a synced push — so a linked desktop or Android client gets previews too, on its next pull. An unlinked one has no server to ask and simply has none, which is the honest consequence of being offline. Safe to call on every save: it re-reads what's cached and does nothing when nothing is new. Capped at five URLs per note, silent on every failure (a link that won't fetch isn't an error the person needs — the note is fine, the link is still there), and it re-checks before storing, so a slow fetch can't resurrect a preview for a URL that was deleted while it was in flight. **Two presentations.** A note whose body is nothing but a URL renders as its preview and nothing else — printing the raw URL under a card that already says where it goes is saying the same thing twice, badly. Until the fetch lands, or if it never does, the URL stands in, so the card is never blank. Anything else gets a compact strip. **And the strip moved.** Previews were rendered ABOVE the body, which put a stranger's headline where the note's own first line should be — worse now that the first line IS the note's name. They sit at the foot of the card now, under the note's own words. The editor's "Preview example.com" button is gone with the manual path; removing an unwanted preview stays, and stays editor-only. Nine tests: three on detection (order, dedupe, sentence-punctuation trimming, non-http rejection) in the unit lane, and three in the integration lane for what only a real database shows — the upsert landing on the right row, a second pass fetching nothing, and a preview NOT being stored for a URL that left the body.	2026-08-23 01:05:37 -04:00
bvandeusen	95aa10c2c3	Remove the title field — a note is named by its first line ... Operator (note 2897): "notes shouldn't have a title field." The concept of a NAME stays — search results, export filenames and the command palette all need one — but nothing is typed into it any more. `display_title` is now the first non-empty line of the body, falling back to the first checklist item. That fallback is what step 2 bought, and the reason this could not go first: a checklist had no body to be named from, so the title was its only name. Now every note has a body, and a note that is only a checklist is named by its first item. Gone everywhere: the column and note_revisions.title (0026), the field on the core's Note/NoteCreateInput/NoteRevision and its SQLite columns (user_version 7), `normalize_title`, the wire field, the FFI record and `NoteEdit::Title` / `ClearTitle`, the web editor's "Title (optional)" input and the card's <h3>, and the Android title field in both the compose sheet and the editor. **The search vector had to be rebuilt, not just left alone.** `notes.search_vector` is a STORED GENERATED column whose expression names `title` — Postgres refuses to drop a column another generated column depends on. It is dropped and recreated over `display_title` at weight A, which keeps the original intent: a note's NAME ranks above the rest of its body. **An imported title becomes the note's first body line.** Keep notes carry one, and so does any ThoughtSync export taken before this. Dropping it would silently lose text someone wrote; folding it in puts it exactly where a name now lives, so the note arrives named as it was. Skipped when the body already opens with that line, so re-importing an export this code produced doesn't stack duplicates. Two smaller things fell out. The Android editor loses its bold first field — one weight throughout, because the first line is the note's name but not a different KIND of text, which is most of step 4 arriving early. And `ClearTitle`'s justification comment moved to `ClearRemindAt`, which is now the surviving example of why NoteEdit is a list rather than a struct of options. Protocol note corrected to say what actually shipped: v2 is "no kind, no title", one bump for the pair. Verified with the local Rust gate this time, not by CI: fmt, clippy and 116 tests all green before pushing. It caught four things — orphaned serde attributes where fields were removed, a `wire::Preview.title` I deleted by mistake (a link preview still has one), nine retention fixtures inserting a dropped column, and four rustfmt diffs.	2026-08-22 19:33:57 -04:00
bvandeusen	c46a4a7709	A checklist is something a note has, not something a note is ... `kind` was never a type. A plain TEXT column with no enum and no CHECK behind it, compared against a hardcoded ("text", "list") tuple in six places; `note_items` was always an ordinary child table keyed by note_id; serialization already emitted `items` whatever the kind; and the Android editor already toggled between the two losslessly, saying so in a comment. The storage has modelled "a body plus optional checkable items" the whole time. This deletes the gates that forbade it. Every surface: the create/PATCH gates, the ?kind= filter and its saved-filter facet, the three import/export branches, the column (alembic 0025); the core's `kind` field, its SQLite column (user_version 6), the sync wire, push and pull; the FFI records and `NoteEdit::Kind`; and on Android `NoteKind.kt`, `DraftKind`, the compose sheet's Note/List switch, and the branches in the card, the editor and the chrome. The editor's note⇄list toggle becomes "Add a checklist" — on both the web and Android. It is not a conversion any more: nothing moves, nothing is swapped, the body stays exactly where it is and the note gains somewhere to put items. The card renders both, in order. Two things that fell out of the merge rather than being aimed at: - The Keep importer was DISCARDING `textContent` whenever a note also had `listContent`, because the target could only hold one. Both survive now, and the test says so. - Markdown export wrote the body OR the checklist. It writes both. Protocol goes to v2, floor included: dropping a field a v1 client sends and expects back is breaking. `title` leaves in step 3 and lands in the same generation, so it needs no further bump. This is the change that will make the 0.1.227 build on the operator's phone refuse to sync — the in-app updater is independent of the handshake and remains the recovery path. The V1 SQLite schema deliberately KEEPS the kind column. V1 is the historical schema and every later block alters it, so removing it there would make a fresh database run V1 without the column and then v6's DROP COLUMN against a column that never existed — "no such column: kind" on every new install.	2026-08-22 12:53:53 -04:00
bvandeusen	bc22f8e249	Remove [[wiki-links]], backlinks and the graph ... Operator, 2026-08-22 (note 2897): ThoughtSync is an intermediary surface. You write here because it's easy — a notebook in your pocket — and later you recall the thing and go finish it somewhere else. Recall is the product; organization is secondary. A linking system is organization, and it isn't what this is for. So: `[[wiki-links]]`, backlinks, the `[[` autocomplete, the note_links table, `/api/notes/link-search`, `/api/notes/<id>/backlinks`, the whole graph blueprint and GraphView. Rust core loses `extract_links`, `backlinks`, `link_search` and `create_titled`; the desktop loses the three Tauri commands that exposed them. This subsumes 982d24c rather than reverting it. That commit bound links to a note id so a rename would stop rewriting other notes' bodies — real infra, but infra for a feature that is now gone, and nothing it added survives. Alembic 0023 stays in the chain anyway: it shipped in an image and may already be applied, and deleting an applied revision strands a database's version pointer. 0024 drops the table and takes the column with it. The history stays honest about the fact that it existed for a day. Two things deliberately kept, because they were serving recall and only incidentally serving links: - `/api/notes/titles` and the titles store. The command palette lists them so you can jump to a note by name. `resolve()` — the name→note lookup that only linking needed — is gone. - `display_title`. Every note still has a name for search results and export filenames. What that name is FOR changed; that it exists did not. `notes/links.py` is now `notes/tags.py`, holding the #tag→label reconciliation it always also owned. A file called links.py with no links in it would have been exactly the drift this removal is meant to end. Also swept out on the way: `_escape_like`, whose only caller was link-search, and the `graph` icon. Nothing lost that a person typed — note_links was always derived, and the `[[text]]` is still sitting in every body it was written in.	2026-08-22 12:00:57 -04:00
bvandeusen	982d24c83b	links: bind a [[link]] to a note, not to a string ... A wiki-link was stored only as normalized TEXT, so a note's NAME was the edge. Renaming it broke every inbound link — and the fix that shipped for that (task 1848, option b) was `_rename_inbound_links`: rewrite the `[[Old Name]]` text inside the body of every note that linked to the renamed one. That works while an explicit title exists to hold still. It stops being defensible the moment a note's name is just its first body line, which is where M13 is going: fixing a typo in your opening sentence would silently edit other notes' words, with nothing to opt out to. So this lands first, before the title comes out, and that window never ships. `note_links` gains `target_id`, bound when the link is written. `target_norm` stays and is what an UNRESOLVED link carries — linking to a note that doesn't exist yet is a supported way to create one, so a link has to be able to name a target that isn't there. Resolution reads the id, falling back to the name only where nothing was bound, which is what lets a forward link connect the moment its target appears. `_claim_unresolved_links` then binds it, so the fallback is a transitional state rather than a permanent one. `_rename_inbound_links` and `rewrite_link_title` are gone. What replaced them touches link rows only: a note's text is never modified by something happening to a different note. The client can no longer resolve links for itself, and that is the point. It used to look `[[text]]` up in a client-side name index, which only held together BECAUSE renaming rewrote the text everywhere. Now the written text can name something the target is no longer called, and only the server holds the binding — so each note serializes its resolved links (`norm`, `id`, and the target's name as it stands NOW). A renamed note reads correctly everywhere it is linked from, without a single body having been edited. Unresolved links are simply absent and fall through to the create-on-click affordance that already existed; so does the offline desktop store, which derives links at query time and has no binding to send. The name-fallback join is owner-scoped everywhere it appears. Bound ids were resolved owner-scoped when written, but matching on display_title alone would have let two users who each have a note called "Groceries" see the other's id and name through an unresolved link (rule 47). The new behaviour is all SQL and this suite runs without a database, so the dead helpers' tests are removed rather than replaced. This repo has no integration lane to hold that ground — noted, not papered over.	2026-08-22 11:02:39 -04:00
bvandeusenandClaude Opus 5	e64d67e904	Expire trash after 30 days, and make the deadline something you can see ... Trash had no end. A note sat in /trash until someone emptied it by hand, and its attachment BYTES sat on disk the whole time — the pile-up the operator asked about. Nothing purged; there was no scheduler at all. Retention is server-owned: `trash_retention_days` (default 30, 0 = keep forever) in the settings registry, so it lands in admin Settings with no migration and takes effect without a restart. A background sweep started in before_serving does the work. Clients learn about a purge the way they learn about any deletion — as a tombstone on the delta feed. An auto-purge nobody can see coming is data loss on a timer, so the window is now visible: /api/config publishes it, notes carry `deleted_at`, Trash leads with the policy, and each card counts down. The countdown rounds DOWN — saying "1 day left" for a note with ten minutes on the clock is the one error here that actually costs someone a note. Three things this turned up on the way: - `DELETE /api/notes/<id>` hard-deleted the row, leaving no tombstone at all. A permanent delete in the web UI never reached a linked device, which would keep its copy forever and push it back on the next edit. It now purges through the same path as everything else. - The purge left `note_revisions` and `note_link_previews` behind. A revision holds the full body, so the text of a "permanently deleted" note was still sitting in the database. - `deleted_at` now SURVIVES a purge instead of being cleared. It's still true, and it means every query that says "not trashed" excludes tombstones for free — without it a content-less row reads as a perfectly normal active note and shows up on the board as a blank card. Desktop keeps its own clock only when there's nobody else to keep one: the sweep runs at startup on an UNLINKED device and refuses otherwise. A linked client that expired notes on its own schedule could destroy something the server was deliberately keeping, then push that delete upstream. Local policy must never outrank the server's — so it also adopts the server's window for the countdown rather than showing its offline default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SreJkbxB4gx8pPsu8QbLPi	2026-07-26 16:20:13 -04:00
bvandeusenandClaude Opus 4.8	36b8f65dc6	M9 S1d: split the notes.py monolith into a cohesive package ... The 1574-line notes.py becomes a `notes/` package. The heavy shared logic moves into focused modules; the route handlers + blueprint registration stay together in __init__ so registration is trivially correct (most routes have no CI auth-test that would otherwise catch a route silently dropping out): - notes/_bp.py — the Blueprint (isolated so route modules could import it without a cycle; also the seam for a later route split). - notes/serialize.py — note (+labels/items/attachments/previews) serialization. - notes/links.py — [[wiki-link]] + #tag parsing and reconciliation. - notes/recurrence.py — recurring-reminder next-occurrence math. - notes/helpers.py — display-title/empty/filter/owner-fetch + filename/slug utils. - notes/import_export.py — export markdown + Keep/native import specs + zip budget. - notes/__init__.py — the `/api/notes` routes + re-exports the external surface (app.py imports `bp`; sync.py + tests import helpers). Pure reorganization — no behavior change (routes/helpers moved verbatim). Callers (app.py, sync.py, test_notes.py) are unchanged: `from thoughtsync.notes import X` resolves via the package __init__ (rule 22 — the package replaces the module). No import cycle (nothing in the package's dep chain imports notes; only app.py + sync.py consume it). New test_all_note_routes_registered asserts all 29 route endpoints are attached, so CI catches any module that fails to register. Runtime DB behavior operator-verified on deploy (no Postgres CI lane). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRgehjoz7Yv8LkUfADxACm	2026-07-23 23:50:48 -04:00