server: the body is the checklist here too, and note_items is dropped
CI & Build / Build now, or wait for Android? (push) Successful in 2s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 6s
CI & Build / Python tests (push) Failing after 8s
CI & Build / integration (push) Successful in 20s
CI & Build / Build & push image (push) Skipped

M304 steps 3 and the server half of 4. The client half landed in 668f7fa; these
belong in one deploy, and the protocol floor below is what enforces that.

notes/checklist.py is the Python half of a grammar that now exists three times —
here, core/src/local/derive.rs, and (next) frontend/src/notes/markdown.ts. That
triplication is the deliberate cost: the alternative is a round trip to the server
before a phone can draw a checkbox. Each copy names the other two, and each is
tested against the same table of cases, including the near-misses that must stay
prose: `-[ ] x`, `- []`, `- [ ]x`, a `[ ]` mid-sentence.

Routes: add/update/delete items stop touching rows and rewrite note.body, all
through one _rewrite_body that runs the same sequence the PATCH route runs for a
body change — because it IS a body change. Revisions, #tag reconciliation, the
name, and link unfurls therefore happen in one place rather than three routes each
remembering to.

The reorder route is gone (rule 22). Reordering a checklist is moving a line, and
no client ever called it — the only reference in the tree was a test asserting the
route existed.

The API still returns `items`, DERIVED from the body on the way out. That is not a
second source of truth and it cannot disagree with the body it came from; it keeps
the web client working across the rest of this milestone and saves any consumer
that only wants to draw checkboxes from carrying a parser.

Export drops its separate items block, in both formats. The body already ends with
those exact lines, so writing them again would double every checklist in an export
and then double it again on re-import. Import still ACCEPTS items, because a Keep
takeout has a list and not a blob; it folds them in before the Note is built, so
display_title and _reconcile_tags both see the finished text.

Protocol 3 on both sides now. A v2 client is refused rather than half-served —
which matters more than I first said: _apply_note_items returned early on an absent
`items` key, so an un-bumped v3 client against a v2 server would not have LOST the
rows, it would have kept them and then had the migration fold them a second time.
Duplicated lists rather than missing ones. The floor prevents both.

Migration 0027 folds every existing row into its note's body and drops the table.
It inlines its own copy of the fold on purpose — a migration has to keep producing
what it produced the day it ran — and a test pins that copy against the app's until
they are allowed to diverge. updated_at is deliberately untouched: a client holding
an unpushed edit keeps the newer timestamp, so last-write-wins keeps its work
instead of the migration silently winning.

The downgrade is honest rather than faithful. It recreates an empty note_items and
leaves the bodies alone, because once items are lines nothing distinguishes one this
migration wrote from one somebody typed, and a downgrade that guessed would eat
hand-written lists. Recreating the table is still necessary: 0015's downgrade drops
a trigger ON note_items, and IF EXISTS covers the trigger, not the table.
This commit is contained in:
2026-08-24 08:03:37 -04:00
parent 32dafca148
commit 761c3b5e82
13 changed files with 566 additions and 254 deletions
+150
View File
@@ -0,0 +1,150 @@
"""Checklist items — a note's body IS its checklist (M304).
A `- [ ] milk` line is the item. There is no `note_items` table beside the body any
more, which is what lets a list sit BETWEEN two paragraphs: rows had a position in a
table and no position in the text, so a separate list could only ever render after
the prose no matter how it was styled.
The same shape as `tags.py`, one strength further along. Tags are derived from the
body too, but they MATERIALISE into `note_labels` rows because the board queries by
label. Items materialise into nothing, because nothing queries them — their only
readers are the card, the editor and `display_title`. So `parse_items` is the whole
storage layer for a checklist, and the rewriters below are how one is edited.
THE GRAMMAR IS SHARED. Three implementations exist and they have to agree, because a
difference between any two of them is a checklist that changes shape when it syncs:
core/src/local/derive.rs the native clients (desktop + Android)
src/thoughtsync/notes/checklist.py this file, the server
frontend/src/notes/markdown.ts the browser
optional indent, `-` or `*`, one-or-more spaces, `[ ]`/`[x]`/`[X]`,
then either end-of-line or one-or-more spaces and the text.
`*` is accepted because markdown.ts already takes it for a plain bullet, and a rule
that allowed `* item` but not `* [ ] item` would be one nobody could guess. `- [ ]`
with nothing after it IS an item with empty text — that is what pressing Enter on a
list leaves behind, and refusing to parse it would make a half-typed list stop being
a list. `- [X]` parses as checked and renders back lowercase, so one canonical form
survives a round trip.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
# Anchored at both ends: a `[ ]` mid-sentence is prose, and `- [ ]x` (no space after
# the brackets) is a sentence that happens to start with brackets, not a marker.
_TASK_RE = re.compile(r"^(?P<indent>\s*)(?P<bullet>[-*]) +\[(?P<mark>[ xX])\](?: +(?P<text>.*))?$")
@dataclass(frozen=True)
class Item:
"""One checklist item. Its position in the parsed list is its identity — the same
thing `position` meant when these were rows, and all the wire ever carried."""
text: str
checked: bool
def parse_items(body: str | None) -> list[Item]:
"""Every checklist item in `body`, in the order they appear."""
out: list[Item] = []
for line in (body or "").split("\n"):
match = _TASK_RE.match(line)
if match:
out.append(Item(text=match.group("text") or "", checked=match.group("mark") in "xX"))
return out
def render_item(text: str, checked: bool, indent: str = "", bullet: str = "-") -> str:
"""One item as the line that stores it.
Always lowercase `x`, whatever was parsed: one canonical output is what makes a
round trip stable, so `- [X]` normalises the first time it is touched and never
again.
"""
mark = "x" if checked else " "
if not text:
return f"{indent}{bullet} [{mark}]"
return f"{indent}{bullet} [{mark}] {text}"
def strip_marker(line: str) -> str:
"""The text of a line with its task marker removed, or the line as it was.
For naming a note: a list-only note is named by its first item, and calling one
"- [ ] milk" would be showing someone the storage instead of the note.
"""
match = _TASK_RE.match(line)
return (match.group("text") or "") if match else line
def _rewrite(body: str, index: int, replace) -> str:
"""Rewrite the `index`-th task line with `replace`, or drop it when `replace`
returns None.
A body with fewer task lines than that is returned UNCHANGED rather than raising:
the index comes from a client that may be a moment behind the server, and a stale
request should do nothing rather than 500.
"""
lines = body.split("\n")
target = None
seen = 0
for n, line in enumerate(lines):
if _TASK_RE.match(line):
if seen == index:
target = n
break
seen += 1
if target is None:
return body
match = _TASK_RE.match(lines[target])
replacement = replace(match)
if replacement is None:
del lines[target]
else:
lines[target] = replacement
return "\n".join(lines)
def set_item_checked(body: str, index: int, checked: bool) -> str:
"""Tick or untick the `index`-th item, keeping its text, indent and bullet."""
return _rewrite(
body,
index,
lambda m: render_item(m.group("text") or "", checked, m.group("indent"), m.group("bullet")),
)
def set_item_text(body: str, index: int, text: str) -> str:
"""Replace the text of the `index`-th item, keeping its state and its bullet."""
return _rewrite(
body,
index,
lambda m: render_item(text.strip(), m.group("mark") in "xX", m.group("indent"), m.group("bullet")),
)
def remove_item(body: str, index: int) -> str:
"""Delete the `index`-th item, line and all."""
return _rewrite(body, index, lambda _m: None)
def append_item(body: str, text: str, checked: bool = False) -> str:
"""Add an item at the end of the body.
A blank line between prose and the list, nothing between consecutive items —
the layout `import_export._note_markdown` has always used when writing a checklist
out. That is not cosmetic: it is what the Alembic migration folds existing
`note_items` rows into AND what `derive::append_item` produces on every client, so
all three land on identical bodies. An export taken before the migration and one
taken after therefore differ in nothing.
"""
line = render_item(text.strip(), checked)
trimmed = body.rstrip("\n")
if not trimmed.strip():
return line
follows_a_list = bool(_TASK_RE.match(trimmed.split("\n")[-1]))
return f"{trimmed}\n{line}" if follows_a_list else f"{trimmed}\n\n{line}"