server: the body is the checklist here too, and note_items is dropped
CI & Build / Build now, or wait for Android? (push) Successful in 2s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 6s
CI & Build / Python tests (push) Failing after 8s
CI & Build / integration (push) Successful in 20s
CI & Build / Build & push image (push) Skipped
CI & Build / Build now, or wait for Android? (push) Successful in 2s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 6s
CI & Build / Python tests (push) Failing after 8s
CI & Build / integration (push) Successful in 20s
CI & Build / Build & push image (push) Skipped
M304 steps 3 and the server half of 4. The client half landed in 668f7fa; these belong in one deploy, and the protocol floor below is what enforces that. notes/checklist.py is the Python half of a grammar that now exists three times — here, core/src/local/derive.rs, and (next) frontend/src/notes/markdown.ts. That triplication is the deliberate cost: the alternative is a round trip to the server before a phone can draw a checkbox. Each copy names the other two, and each is tested against the same table of cases, including the near-misses that must stay prose: `-[ ] x`, `- []`, `- [ ]x`, a `[ ]` mid-sentence. Routes: add/update/delete items stop touching rows and rewrite note.body, all through one _rewrite_body that runs the same sequence the PATCH route runs for a body change — because it IS a body change. Revisions, #tag reconciliation, the name, and link unfurls therefore happen in one place rather than three routes each remembering to. The reorder route is gone (rule 22). Reordering a checklist is moving a line, and no client ever called it — the only reference in the tree was a test asserting the route existed. The API still returns `items`, DERIVED from the body on the way out. That is not a second source of truth and it cannot disagree with the body it came from; it keeps the web client working across the rest of this milestone and saves any consumer that only wants to draw checkboxes from carrying a parser. Export drops its separate items block, in both formats. The body already ends with those exact lines, so writing them again would double every checklist in an export and then double it again on re-import. Import still ACCEPTS items, because a Keep takeout has a list and not a blob; it folds them in before the Note is built, so display_title and _reconcile_tags both see the finished text. Protocol 3 on both sides now. A v2 client is refused rather than half-served — which matters more than I first said: _apply_note_items returned early on an absent `items` key, so an un-bumped v3 client against a v2 server would not have LOST the rows, it would have kept them and then had the migration fold them a second time. Duplicated lists rather than missing ones. The floor prevents both. Migration 0027 folds every existing row into its note's body and drops the table. It inlines its own copy of the fold on purpose — a migration has to keep producing what it produced the day it ran — and a test pins that copy against the app's until they are allowed to diverge. updated_at is deliberately untouched: a client holding an unpushed edit keeps the newer timestamp, so last-write-wins keeps its work instead of the migration silently winning. The downgrade is honest rather than faithful. It recreates an empty note_items and leaves the bodies alone, because once items are lines nothing distinguishes one this migration wrote from one somebody typed, and a downgrade that guessed would eat hand-written lists. Recreating the table is still necessary: 0015's downgrade drops a trigger ON note_items, and IF EXISTS covers the trigger, not the table.
This commit is contained in:
@@ -9,7 +9,6 @@ from . import ( # noqa: F401
|
||||
label,
|
||||
note,
|
||||
note_attachment,
|
||||
note_item,
|
||||
note_link_preview,
|
||||
note_revision,
|
||||
saved_filter,
|
||||
|
||||
@@ -46,7 +46,6 @@ class Note(Base):
|
||||
display_title: Mapped[str] = mapped_column(Text(), nullable=False, server_default="")
|
||||
body: Mapped[str] = mapped_column(Text(), nullable=False, server_default="")
|
||||
color: Mapped[str] = mapped_column(Text(), nullable=False, server_default="default")
|
||||
# 'text' (freeform body) or 'list' (a checklist of note_items).
|
||||
# Manual drag order (higher = earlier); 0 until the user reorders.
|
||||
position: Mapped[int] = mapped_column(Integer(), nullable=False, server_default="0")
|
||||
pinned: Mapped[bool] = mapped_column(Boolean(), nullable=False, server_default=func.false())
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, Text, func
|
||||
from sqlalchemy.dialects.postgresql import UUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from . import Base
|
||||
|
||||
|
||||
class NoteItem(Base):
|
||||
"""A single checklist item on a note.
|
||||
|
||||
Any note can have them. There is no note "kind" gating this — a checklist is
|
||||
something a note HAS, not something a note IS (M13 step 2).
|
||||
"""
|
||||
|
||||
__tablename__ = "note_items"
|
||||
|
||||
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
|
||||
note_id: Mapped[uuid.UUID] = mapped_column(
|
||||
UUID(as_uuid=True), ForeignKey("notes.id", ondelete="CASCADE"), nullable=False
|
||||
)
|
||||
text: Mapped[str] = mapped_column(Text(), nullable=False)
|
||||
checked: Mapped[bool] = mapped_column(Boolean(), nullable=False, server_default=func.false())
|
||||
position: Mapped[int] = mapped_column(Integer(), nullable=False, server_default="0")
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
|
||||
@@ -31,10 +31,16 @@ from ..labeling import reconcile_manual_labels, resolve_owned_label_ids
|
||||
from ..models.label import Label, NoteLabel
|
||||
from ..models.note import Note
|
||||
from ..models.note_attachment import NoteAttachment
|
||||
from ..models.note_item import NoteItem
|
||||
from ..models.note_link_preview import NoteLinkPreview
|
||||
from ..models.note_revision import NoteRevision
|
||||
from ..revisions import should_snapshot
|
||||
from .checklist import (
|
||||
append_item,
|
||||
parse_items,
|
||||
remove_item,
|
||||
set_item_checked,
|
||||
set_item_text,
|
||||
)
|
||||
from ..responses import json_error, not_found, parse_uuid
|
||||
from ..retention import purge_note
|
||||
from ..settings import get_setting
|
||||
@@ -70,7 +76,7 @@ from .tags import (
|
||||
parse_tags,
|
||||
)
|
||||
from .recurrence import REMINDER_RECURRENCES, next_occurrence, normalize_recurrence
|
||||
from .serialize import _items_for_notes, _labels_for_notes, _serialize_note, _serialize_notes
|
||||
from .serialize import _labels_for_notes, _serialize_note, _serialize_notes
|
||||
|
||||
__all__ = [
|
||||
"bp",
|
||||
@@ -225,7 +231,6 @@ async def export_notes():
|
||||
).all()
|
||||
ids = [n.id for n in notes_list]
|
||||
labels_map = await _labels_for_notes(db, ids)
|
||||
items_map = await _items_for_notes(db, ids)
|
||||
att_rows = (
|
||||
(await db.scalars(select(NoteAttachment).where(NoteAttachment.note_id.in_(ids)))).all() if ids else []
|
||||
)
|
||||
@@ -247,7 +252,6 @@ async def export_notes():
|
||||
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
for n in notes_list:
|
||||
labels = labels_map.get(n.id, [])
|
||||
items = items_map.get(n.id, [])
|
||||
atts = att_by_note.get(n.id, [])
|
||||
short = str(n.id)[:8]
|
||||
payload["notes"].append(
|
||||
@@ -263,13 +267,12 @@ async def export_notes():
|
||||
"created_at": n.created_at.isoformat() if n.created_at else None,
|
||||
"updated_at": n.updated_at.isoformat() if n.updated_at else None,
|
||||
"labels": [lb["name"] for lb in labels],
|
||||
"items": [{"text": it["text"], "checked": it["checked"]} for it in items],
|
||||
"attachments": [
|
||||
{"file": f"attachments/{short}/{os.path.basename(a.path)}", "mime": a.mime} for a in atts
|
||||
],
|
||||
}
|
||||
)
|
||||
zf.writestr(f"notes/{_slugify(n.display_title)}-{short}.md", _note_markdown(n, labels, items))
|
||||
zf.writestr(f"notes/{_slugify(n.display_title)}-{short}.md", _note_markdown(n, labels))
|
||||
for a in atts:
|
||||
src = Config.media_root() / a.path
|
||||
if src.is_file():
|
||||
@@ -385,24 +388,6 @@ async def reorder_notes():
|
||||
return jsonify({"ok": True})
|
||||
|
||||
|
||||
async def _name_for(db, note: Note, item_texts: list[str] | None = None) -> str:
|
||||
"""The note's display name, consulting its checklist only when the body is silent.
|
||||
|
||||
`item_texts` short-circuits the query for callers that already hold the items
|
||||
(create, import). Everyone else pays one narrow SELECT, and only when the body
|
||||
produced nothing — which is the uncommon case.
|
||||
"""
|
||||
name = derive_display_title(note.body)
|
||||
if name:
|
||||
return name
|
||||
if item_texts is not None:
|
||||
return derive_display_title("", item_texts[0] if item_texts else None)
|
||||
first = await db.scalar(
|
||||
select(NoteItem.text).where(NoteItem.note_id == note.id).order_by(NoteItem.position).limit(1)
|
||||
)
|
||||
return derive_display_title("", first)
|
||||
|
||||
|
||||
@bp.post("")
|
||||
@login_required
|
||||
async def create_note():
|
||||
@@ -419,17 +404,20 @@ async def create_note():
|
||||
Note.owner_id == g.user_id, Note.deleted_at.is_(None)
|
||||
)
|
||||
)
|
||||
# Items still arrive separately — a client holds a list, not a blob — but they
|
||||
# are folded into the body, which is where a checklist lives now (M304).
|
||||
for text in item_texts:
|
||||
body = append_item(body, text)
|
||||
note = Note(
|
||||
owner_id=g.user_id,
|
||||
display_title=derive_display_title(body, item_texts[0] if item_texts else None),
|
||||
display_title=derive_display_title(body),
|
||||
body=body,
|
||||
color=normalize_color(data.get("color")),
|
||||
position=int(max_pos) + 1,
|
||||
)
|
||||
db.add(note)
|
||||
await db.flush() # assign note.id before writing items/links
|
||||
for pos, text in enumerate(item_texts):
|
||||
db.add(NoteItem(note_id=note.id, text=text, position=pos))
|
||||
await db.flush() # assign note.id before writing links
|
||||
# The FOLDED body: an item can carry a #tag too.
|
||||
await _reconcile_tags(db, note)
|
||||
await db.commit()
|
||||
await db.refresh(note)
|
||||
@@ -484,7 +472,7 @@ async def update_note(note_id: str):
|
||||
if "recurrence" in data:
|
||||
note.recurrence = normalize_recurrence(data["recurrence"])
|
||||
if "body" in data:
|
||||
note.display_title = await _name_for(db, note)
|
||||
note.display_title = derive_display_title(note.body)
|
||||
await _reconcile_tags(db, note)
|
||||
# Version history: snapshot the PRE-edit body, once per editing session
|
||||
# rather than once per write — see revisions.should_snapshot. Writing often
|
||||
@@ -543,7 +531,7 @@ async def restore_revision(note_id: str, rev_id: str):
|
||||
# the revision — with the same body ripple as a normal edit.
|
||||
db.add(NoteRevision(note_id=note.id, body=note.body))
|
||||
note.body = rev.body
|
||||
note.display_title = await _name_for(db, note)
|
||||
note.display_title = derive_display_title(note.body)
|
||||
await _reconcile_tags(db, note)
|
||||
await db.commit()
|
||||
await db.refresh(note)
|
||||
@@ -575,11 +563,35 @@ async def set_note_labels(note_id: str):
|
||||
return jsonify(await _serialize_note(db, note))
|
||||
|
||||
|
||||
async def _get_item(db, note: Note, item_id: str) -> NoteItem | None:
|
||||
iid = parse_uuid(item_id)
|
||||
if iid is None:
|
||||
def _item_index(item_id: str) -> int | None:
|
||||
"""An item's id is its ordinal (see serialize.items_of). Anything else is a stale
|
||||
id from a client that has not reloaded, and the answer to those is 404."""
|
||||
try:
|
||||
index = int(item_id)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return await db.scalar(select(NoteItem).where(NoteItem.id == iid, NoteItem.note_id == note.id))
|
||||
return index if index >= 0 else None
|
||||
|
||||
|
||||
async def _rewrite_body(db, note: Note, body: str):
|
||||
"""Every item mutation is a body edit, so all of them land here.
|
||||
|
||||
One place means one place that snapshots a revision, re-derives `#tags`, recomputes
|
||||
the name and queues link unfurls — rather than three routes each remembering to.
|
||||
Deliberately the same sequence the PATCH route runs for a body change, because it
|
||||
IS a body change.
|
||||
"""
|
||||
old_body = note.body
|
||||
if await should_snapshot(db, note.id, old_body, body):
|
||||
db.add(NoteRevision(note_id=note.id, body=old_body))
|
||||
note.body = body
|
||||
note.display_title = derive_display_title(body)
|
||||
await _reconcile_tags(db, note)
|
||||
await db.commit()
|
||||
await db.refresh(note)
|
||||
if note.body != old_body:
|
||||
schedule_unfurls(note.id, note.body)
|
||||
return jsonify(await _serialize_note(db, note))
|
||||
|
||||
|
||||
@bp.post("/<note_id>/items")
|
||||
@@ -591,70 +603,49 @@ async def add_item(note_id: str):
|
||||
note = await _get_owned(db, note_id)
|
||||
if note is None:
|
||||
return not_found()
|
||||
max_pos = await db.scalar(
|
||||
select(func.coalesce(func.max(NoteItem.position), -1)).where(NoteItem.note_id == note.id)
|
||||
)
|
||||
db.add(NoteItem(note_id=note.id, text=text, position=int(max_pos) + 1))
|
||||
await db.commit()
|
||||
return jsonify(await _serialize_note(db, note)), 201
|
||||
response = await _rewrite_body(db, note, append_item(note.body, text))
|
||||
return response, 201
|
||||
|
||||
|
||||
@bp.patch("/<note_id>/items/<item_id>")
|
||||
@login_required
|
||||
async def update_item(note_id: str, item_id: str):
|
||||
data = await request.get_json(silent=True) or {}
|
||||
index = _item_index(item_id)
|
||||
if index is None:
|
||||
return not_found()
|
||||
async with session_scope() as db:
|
||||
note = await _get_owned(db, note_id)
|
||||
if note is None:
|
||||
return not_found()
|
||||
item = await _get_item(db, note, item_id)
|
||||
if item is None:
|
||||
if index >= len(parse_items(note.body)):
|
||||
return not_found()
|
||||
body = note.body
|
||||
if "text" in data and isinstance(data["text"], str):
|
||||
item.text = data["text"]
|
||||
body = set_item_text(body, index, data["text"])
|
||||
if "checked" in data:
|
||||
item.checked = bool(data["checked"])
|
||||
await db.commit()
|
||||
return jsonify(await _serialize_note(db, note))
|
||||
body = set_item_checked(body, index, bool(data["checked"]))
|
||||
return await _rewrite_body(db, note, body)
|
||||
|
||||
|
||||
@bp.delete("/<note_id>/items/<item_id>")
|
||||
@login_required
|
||||
async def delete_item(note_id: str, item_id: str):
|
||||
index = _item_index(item_id)
|
||||
if index is None:
|
||||
return not_found()
|
||||
async with session_scope() as db:
|
||||
note = await _get_owned(db, note_id)
|
||||
if note is None:
|
||||
return not_found()
|
||||
item = await _get_item(db, note, item_id)
|
||||
if item is None:
|
||||
if index >= len(parse_items(note.body)):
|
||||
return not_found()
|
||||
await db.delete(item)
|
||||
await db.commit()
|
||||
return jsonify(await _serialize_note(db, note))
|
||||
return await _rewrite_body(db, note, remove_item(note.body, index))
|
||||
|
||||
|
||||
@bp.post("/<note_id>/items/reorder")
|
||||
@login_required
|
||||
async def reorder_items(note_id: str):
|
||||
data = await request.get_json(silent=True) or {}
|
||||
order = data.get("item_ids")
|
||||
if not isinstance(order, list):
|
||||
return json_error("item_ids must be a list", 400)
|
||||
async with session_scope() as db:
|
||||
note = await _get_owned(db, note_id)
|
||||
if note is None:
|
||||
return not_found()
|
||||
existing = {
|
||||
str(i.id): i for i in (await db.scalars(select(NoteItem).where(NoteItem.note_id == note.id))).all()
|
||||
}
|
||||
pos = 0
|
||||
for iid in order:
|
||||
item = existing.get(str(iid))
|
||||
if item is not None:
|
||||
item.position = pos
|
||||
pos += 1
|
||||
await db.commit()
|
||||
return jsonify(await _serialize_note(db, note))
|
||||
# The reorder route is gone with M304. Reordering a checklist is moving a line, which
|
||||
# is something a text editor already does and no client ever called this for — the
|
||||
# only reference to it in the tree was a test asserting the route existed.
|
||||
|
||||
|
||||
@bp.post("/<note_id>/attachments")
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
"""Checklist items — a note's body IS its checklist (M304).
|
||||
|
||||
A `- [ ] milk` line is the item. There is no `note_items` table beside the body any
|
||||
more, which is what lets a list sit BETWEEN two paragraphs: rows had a position in a
|
||||
table and no position in the text, so a separate list could only ever render after
|
||||
the prose no matter how it was styled.
|
||||
|
||||
The same shape as `tags.py`, one strength further along. Tags are derived from the
|
||||
body too, but they MATERIALISE into `note_labels` rows because the board queries by
|
||||
label. Items materialise into nothing, because nothing queries them — their only
|
||||
readers are the card, the editor and `display_title`. So `parse_items` is the whole
|
||||
storage layer for a checklist, and the rewriters below are how one is edited.
|
||||
|
||||
THE GRAMMAR IS SHARED. Three implementations exist and they have to agree, because a
|
||||
difference between any two of them is a checklist that changes shape when it syncs:
|
||||
|
||||
core/src/local/derive.rs the native clients (desktop + Android)
|
||||
src/thoughtsync/notes/checklist.py this file, the server
|
||||
frontend/src/notes/markdown.ts the browser
|
||||
|
||||
optional indent, `-` or `*`, one-or-more spaces, `[ ]`/`[x]`/`[X]`,
|
||||
then either end-of-line or one-or-more spaces and the text.
|
||||
|
||||
`*` is accepted because markdown.ts already takes it for a plain bullet, and a rule
|
||||
that allowed `* item` but not `* [ ] item` would be one nobody could guess. `- [ ]`
|
||||
with nothing after it IS an item with empty text — that is what pressing Enter on a
|
||||
list leaves behind, and refusing to parse it would make a half-typed list stop being
|
||||
a list. `- [X]` parses as checked and renders back lowercase, so one canonical form
|
||||
survives a round trip.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
# Anchored at both ends: a `[ ]` mid-sentence is prose, and `- [ ]x` (no space after
|
||||
# the brackets) is a sentence that happens to start with brackets, not a marker.
|
||||
_TASK_RE = re.compile(r"^(?P<indent>\s*)(?P<bullet>[-*]) +\[(?P<mark>[ xX])\](?: +(?P<text>.*))?$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Item:
|
||||
"""One checklist item. Its position in the parsed list is its identity — the same
|
||||
thing `position` meant when these were rows, and all the wire ever carried."""
|
||||
|
||||
text: str
|
||||
checked: bool
|
||||
|
||||
|
||||
def parse_items(body: str | None) -> list[Item]:
|
||||
"""Every checklist item in `body`, in the order they appear."""
|
||||
out: list[Item] = []
|
||||
for line in (body or "").split("\n"):
|
||||
match = _TASK_RE.match(line)
|
||||
if match:
|
||||
out.append(Item(text=match.group("text") or "", checked=match.group("mark") in "xX"))
|
||||
return out
|
||||
|
||||
|
||||
def render_item(text: str, checked: bool, indent: str = "", bullet: str = "-") -> str:
|
||||
"""One item as the line that stores it.
|
||||
|
||||
Always lowercase `x`, whatever was parsed: one canonical output is what makes a
|
||||
round trip stable, so `- [X]` normalises the first time it is touched and never
|
||||
again.
|
||||
"""
|
||||
mark = "x" if checked else " "
|
||||
if not text:
|
||||
return f"{indent}{bullet} [{mark}]"
|
||||
return f"{indent}{bullet} [{mark}] {text}"
|
||||
|
||||
|
||||
def strip_marker(line: str) -> str:
|
||||
"""The text of a line with its task marker removed, or the line as it was.
|
||||
|
||||
For naming a note: a list-only note is named by its first item, and calling one
|
||||
"- [ ] milk" would be showing someone the storage instead of the note.
|
||||
"""
|
||||
match = _TASK_RE.match(line)
|
||||
return (match.group("text") or "") if match else line
|
||||
|
||||
|
||||
def _rewrite(body: str, index: int, replace) -> str:
|
||||
"""Rewrite the `index`-th task line with `replace`, or drop it when `replace`
|
||||
returns None.
|
||||
|
||||
A body with fewer task lines than that is returned UNCHANGED rather than raising:
|
||||
the index comes from a client that may be a moment behind the server, and a stale
|
||||
request should do nothing rather than 500.
|
||||
"""
|
||||
lines = body.split("\n")
|
||||
target = None
|
||||
seen = 0
|
||||
for n, line in enumerate(lines):
|
||||
if _TASK_RE.match(line):
|
||||
if seen == index:
|
||||
target = n
|
||||
break
|
||||
seen += 1
|
||||
if target is None:
|
||||
return body
|
||||
|
||||
match = _TASK_RE.match(lines[target])
|
||||
replacement = replace(match)
|
||||
if replacement is None:
|
||||
del lines[target]
|
||||
else:
|
||||
lines[target] = replacement
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def set_item_checked(body: str, index: int, checked: bool) -> str:
|
||||
"""Tick or untick the `index`-th item, keeping its text, indent and bullet."""
|
||||
return _rewrite(
|
||||
body,
|
||||
index,
|
||||
lambda m: render_item(m.group("text") or "", checked, m.group("indent"), m.group("bullet")),
|
||||
)
|
||||
|
||||
|
||||
def set_item_text(body: str, index: int, text: str) -> str:
|
||||
"""Replace the text of the `index`-th item, keeping its state and its bullet."""
|
||||
return _rewrite(
|
||||
body,
|
||||
index,
|
||||
lambda m: render_item(text.strip(), m.group("mark") in "xX", m.group("indent"), m.group("bullet")),
|
||||
)
|
||||
|
||||
|
||||
def remove_item(body: str, index: int) -> str:
|
||||
"""Delete the `index`-th item, line and all."""
|
||||
return _rewrite(body, index, lambda _m: None)
|
||||
|
||||
|
||||
def append_item(body: str, text: str, checked: bool = False) -> str:
|
||||
"""Add an item at the end of the body.
|
||||
|
||||
A blank line between prose and the list, nothing between consecutive items —
|
||||
the layout `import_export._note_markdown` has always used when writing a checklist
|
||||
out. That is not cosmetic: it is what the Alembic migration folds existing
|
||||
`note_items` rows into AND what `derive::append_item` produces on every client, so
|
||||
all three land on identical bodies. An export taken before the migration and one
|
||||
taken after therefore differ in nothing.
|
||||
"""
|
||||
line = render_item(text.strip(), checked)
|
||||
trimmed = body.rstrip("\n")
|
||||
if not trimmed.strip():
|
||||
return line
|
||||
follows_a_list = bool(_TASK_RE.match(trimmed.split("\n")[-1]))
|
||||
return f"{trimmed}\n{line}" if follows_a_list else f"{trimmed}\n\n{line}"
|
||||
@@ -3,6 +3,8 @@ board-filter narrowing, the owner-scoped fetch, and filename/slug sanitizers use
|
||||
both the attachment routes and the importer."""
|
||||
from __future__ import annotations
|
||||
|
||||
from .checklist import strip_marker
|
||||
|
||||
import os
|
||||
import re
|
||||
|
||||
@@ -19,29 +21,36 @@ VALID_FILTERS = {"active", "archived", "trash"}
|
||||
DISPLAY_TITLE_CAP = 200
|
||||
|
||||
|
||||
def derive_display_title(body: str | None, first_item: str | None = None) -> str:
|
||||
"""The note's display NAME: the first non-empty line of the body, else the first
|
||||
checklist item's text (both trimmed and length-capped).
|
||||
def derive_display_title(body: str | None) -> str:
|
||||
"""The note's display NAME: the first line of the body that says anything.
|
||||
|
||||
There is no explicit title to prefer any more (M13 step 3) — a note is a body plus
|
||||
optional items, and its name is simply the first thing written in it. Persisted as
|
||||
notes.display_title so search results and export filenames have something to say.
|
||||
There is no explicit title to prefer any more (M13 step 3) — a note is a body, and
|
||||
its name is simply the first thing written in it. Persisted as notes.display_title
|
||||
so search results and export filenames have something to say.
|
||||
|
||||
The item fallback is what step 2 bought: a note that is only a checklist would
|
||||
otherwise have no name at all, which is exactly the hole that made removing the
|
||||
title unsafe before checklists stopped being their own kind of thing.
|
||||
The old `first_item` fallback is gone with M304: items ARE body lines now, so a
|
||||
list-only note is named by its first item without anyone having to arrange it. What
|
||||
replaced the fallback is stripping the task marker — calling that note "- [ ] milk"
|
||||
would show someone the storage instead of the note — and skipping an EMPTY item, so
|
||||
a half-typed list does not leave a note with no name.
|
||||
|
||||
Deterministic — a literal first line, never generated.
|
||||
Mirrors `display_title` in core/src/local/store.rs. Deterministic — a literal first
|
||||
line, never generated.
|
||||
"""
|
||||
for line in (body or "").splitlines():
|
||||
stripped = line.strip()
|
||||
stripped = strip_marker(line.strip()).strip()
|
||||
if stripped:
|
||||
return stripped[:DISPLAY_TITLE_CAP]
|
||||
return (first_item or "").strip()[:DISPLAY_TITLE_CAP]
|
||||
return ""
|
||||
|
||||
|
||||
def is_empty_note(body: str | None, items: list | None = None) -> bool:
|
||||
"""Nothing worth keeping: no body text and no checklist items."""
|
||||
"""Nothing worth keeping: no body text and no checklist items.
|
||||
|
||||
`items` is still a separate argument because create still accepts them separately —
|
||||
the importer holds a list, not a blob — and they are folded into the body only
|
||||
after this check has decided the note is worth making at all.
|
||||
"""
|
||||
return not (body or "").strip() and not items
|
||||
|
||||
|
||||
|
||||
@@ -20,7 +20,6 @@ from ..config import Config
|
||||
from ..models.label import NoteLabel
|
||||
from ..models.note import Note
|
||||
from ..models.note_attachment import NoteAttachment
|
||||
from ..models.note_item import NoteItem
|
||||
from .helpers import (
|
||||
ALLOWED_IMAGE_MIMES,
|
||||
_attachment_ext,
|
||||
@@ -28,11 +27,12 @@ from .helpers import (
|
||||
derive_display_title,
|
||||
is_empty_note,
|
||||
)
|
||||
from .checklist import append_item
|
||||
from .tags import _find_or_create_label, _reconcile_tags
|
||||
from .recurrence import normalize_recurrence
|
||||
|
||||
|
||||
def _note_markdown(note: Note, labels: list, items: list) -> str:
|
||||
def _note_markdown(note: Note, labels: list) -> str:
|
||||
"""One note as a human-readable Markdown file with a small frontmatter block.
|
||||
The authoritative machine format is notes.json; this is for reading/portability."""
|
||||
fm = ["---"]
|
||||
@@ -54,11 +54,9 @@ def _note_markdown(note: Note, labels: list, items: list) -> str:
|
||||
# are written, body first, with a blank line between them when there is.
|
||||
if note.body:
|
||||
fm.append(note.body)
|
||||
if items:
|
||||
if note.body:
|
||||
fm.append("")
|
||||
for it in items:
|
||||
fm.append(f"- [{'x' if it['checked'] else ' '}] {it['text']}")
|
||||
# No separate items block any more. The body already ends with those exact lines
|
||||
# (M304) — this function is where their layout was decided, and appending them a
|
||||
# second time would double every checklist in an export.
|
||||
return "\n".join(fm) + "\n"
|
||||
|
||||
|
||||
@@ -294,9 +292,17 @@ async def _create_imported_note(
|
||||
if is_empty_note(body, item_texts):
|
||||
return False
|
||||
|
||||
# Items fold into the body, which is where a checklist lives now (M304). Done
|
||||
# before the Note is built so display_title and _reconcile_tags both see the
|
||||
# finished text — an imported item can carry a #tag like any other line.
|
||||
for it in items:
|
||||
text = (it.get("text") or "").strip()
|
||||
if text:
|
||||
body = append_item(body, text, bool(it.get("checked")))
|
||||
|
||||
note = Note(
|
||||
owner_id=owner_id,
|
||||
display_title=derive_display_title(body, item_texts[0] if item_texts else None),
|
||||
display_title=derive_display_title(body),
|
||||
body=body,
|
||||
color=normalize_color(spec.get("color")),
|
||||
pinned=bool(spec.get("pinned")),
|
||||
@@ -316,12 +322,7 @@ async def _create_imported_note(
|
||||
if spec.get("updated_at"):
|
||||
note.updated_at = spec["updated_at"]
|
||||
db.add(note)
|
||||
await db.flush() # assign note.id before items/labels/attachments/links
|
||||
|
||||
for pos, it in enumerate(items):
|
||||
text = (it.get("text") or "").strip()
|
||||
if text:
|
||||
db.add(NoteItem(note_id=note.id, text=text, checked=bool(it.get("checked")), position=pos))
|
||||
await db.flush() # assign note.id before labels/attachments/links
|
||||
|
||||
# Explicit (picker-style) labels are manual — via_tag=False. Inline #tags in the
|
||||
# body are handled by _reconcile_tags below, same as a normal create.
|
||||
|
||||
@@ -8,8 +8,8 @@ from sqlalchemy import select
|
||||
from ..models.label import Label, NoteLabel
|
||||
from ..models.note import Note
|
||||
from ..models.note_attachment import NoteAttachment
|
||||
from ..models.note_item import NoteItem
|
||||
from ..models.note_link_preview import NoteLinkPreview
|
||||
from .checklist import parse_items
|
||||
|
||||
|
||||
async def _labels_for_notes(db, note_ids: list) -> dict:
|
||||
@@ -30,23 +30,23 @@ async def _labels_for_notes(db, note_ids: list) -> dict:
|
||||
return result
|
||||
|
||||
|
||||
def _serialize_item(item: NoteItem) -> dict:
|
||||
return {"id": str(item.id), "text": item.text, "checked": item.checked, "position": item.position}
|
||||
def items_of(body: str | None) -> list[dict]:
|
||||
"""The note's checklist, read out of its body. No query, because there is no table.
|
||||
|
||||
Still emitted in the payload after M304, and that is not a second source of truth:
|
||||
it is DERIVED on the way out, so it cannot disagree with the body it came from. It
|
||||
saves every consumer that only wants to draw checkboxes from carrying a parser, and
|
||||
the ones that do carry one (the native clients, the browser) are free to ignore it
|
||||
and read the body.
|
||||
|
||||
async def _items_for_notes(db, note_ids: list) -> dict:
|
||||
"""Map note_id -> [checklist items] in one query, ordered by position."""
|
||||
result: dict = {}
|
||||
if not note_ids:
|
||||
return result
|
||||
items = (
|
||||
await db.scalars(
|
||||
select(NoteItem).where(NoteItem.note_id.in_(note_ids)).order_by(NoteItem.position, NoteItem.created_at)
|
||||
)
|
||||
).all()
|
||||
for item in items:
|
||||
result.setdefault(item.note_id, []).append(_serialize_item(item))
|
||||
return result
|
||||
The id is the item's ORDINAL, which is what the rewriters in `checklist.py` take,
|
||||
so a client holding one can act on it directly. It also shifts when an item is
|
||||
removed — every mutation returns the reloaded note for exactly that reason.
|
||||
"""
|
||||
return [
|
||||
{"id": str(i), "text": item.text, "checked": item.checked, "position": i}
|
||||
for i, item in enumerate(parse_items(body))
|
||||
]
|
||||
|
||||
|
||||
def _attachment_url(note_id, att_id) -> str:
|
||||
@@ -108,8 +108,7 @@ async def _serialize_note(db, note: Note) -> dict:
|
||||
data = note.serialize()
|
||||
labels = await _labels_for_notes(db, [note.id])
|
||||
data["labels"] = labels.get(note.id, [])
|
||||
items = await _items_for_notes(db, [note.id])
|
||||
data["items"] = items.get(note.id, [])
|
||||
data["items"] = items_of(note.body)
|
||||
attachments = await _attachments_for_notes(db, [note.id])
|
||||
data["attachments"] = attachments.get(note.id, [])
|
||||
previews = await _previews_for_notes(db, [note.id])
|
||||
@@ -120,14 +119,14 @@ async def _serialize_note(db, note: Note) -> dict:
|
||||
async def _serialize_notes(db, notes: list) -> list:
|
||||
ids = [n.id for n in notes]
|
||||
labels_map = await _labels_for_notes(db, ids)
|
||||
items_map = await _items_for_notes(db, ids)
|
||||
|
||||
attach_map = await _attachments_for_notes(db, ids)
|
||||
preview_map = await _previews_for_notes(db, ids)
|
||||
out = []
|
||||
for n in notes:
|
||||
data = n.serialize()
|
||||
data["labels"] = labels_map.get(n.id, [])
|
||||
data["items"] = items_map.get(n.id, [])
|
||||
data["items"] = items_of(n.body)
|
||||
data["attachments"] = attach_map.get(n.id, [])
|
||||
data["previews"] = preview_map.get(n.id, [])
|
||||
out.append(data)
|
||||
|
||||
@@ -32,7 +32,6 @@ from .db import session_scope
|
||||
from .models.label import NoteLabel
|
||||
from .models.note import Note
|
||||
from .models.note_attachment import NoteAttachment
|
||||
from .models.note_item import NoteItem
|
||||
from .models.note_link_preview import NoteLinkPreview
|
||||
from .models.note_revision import NoteRevision
|
||||
from .settings import get_setting
|
||||
@@ -85,7 +84,6 @@ async def purge_note(db, note: Note, edited_at: datetime | None = None) -> None:
|
||||
# place would make the note reappear whole on the next sweep.
|
||||
logger.warning("couldn't remove attachment file %s during purge", a.path, exc_info=True)
|
||||
await db.execute(sa_delete(NoteAttachment).where(NoteAttachment.note_id == note.id))
|
||||
await db.execute(sa_delete(NoteItem).where(NoteItem.note_id == note.id))
|
||||
await db.execute(sa_delete(NoteLabel).where(NoteLabel.note_id == note.id))
|
||||
await db.execute(sa_delete(NoteLinkPreview).where(NoteLinkPreview.note_id == note.id))
|
||||
await db.execute(sa_delete(NoteRevision).where(NoteRevision.note_id == note.id))
|
||||
|
||||
+9
-48
@@ -24,7 +24,6 @@ from .db import session_scope
|
||||
from .labeling import reconcile_manual_labels, resolve_owned_label_ids
|
||||
from .models.label import Label, NoteLabel
|
||||
from .models.note import Note
|
||||
from .models.note_item import NoteItem
|
||||
from .models.note_revision import NoteRevision
|
||||
from .revisions import should_snapshot
|
||||
from .notes import (
|
||||
@@ -63,8 +62,8 @@ MAX_PUSH = 1000 # per-batch change cap
|
||||
#
|
||||
# One bump for the pair: they landed in the same protocol generation, and nothing ever
|
||||
# ran against a half-applied v2.
|
||||
SYNC_PROTOCOL_VERSION = 2
|
||||
MIN_CLIENT_PROTOCOL_VERSION = 2
|
||||
SYNC_PROTOCOL_VERSION = 3
|
||||
MIN_CLIENT_PROTOCOL_VERSION = 3
|
||||
|
||||
# Named capabilities beyond the base protocol. An ADDITIVE change earns a name
|
||||
# here rather than a min-version bump, so a newer client meeting an older server
|
||||
@@ -213,49 +212,12 @@ def _assign_note_fields(note: Note, ch: dict) -> None:
|
||||
note.position = ch["position"]
|
||||
|
||||
|
||||
def _first_item_text(ch: dict) -> str:
|
||||
"""The first non-blank checklist item in a pushed change, or "".
|
||||
|
||||
Read straight from the payload rather than the database because the note's name is
|
||||
computed BEFORE `_apply_note_items` has written anything — and a note whose body is
|
||||
empty is named by its first item (M13 step 3).
|
||||
"""
|
||||
items = ch.get("items")
|
||||
if not isinstance(items, list):
|
||||
return ""
|
||||
for it in items:
|
||||
if isinstance(it, dict):
|
||||
text = (it.get("text") or "").strip()
|
||||
if text:
|
||||
return text
|
||||
return ""
|
||||
|
||||
|
||||
async def _apply_note_items(db, note: Note, ch: dict) -> None:
|
||||
"""Replace the note's checklist items with the client's (items sync inline).
|
||||
|
||||
Applies to ANY note. This used to delete every item when the note wasn't
|
||||
`kind == "list"`, which was survivable only because nothing could produce a note
|
||||
holding both a body and items. M13 makes that the normal shape — a checklist is
|
||||
something a note HAS, not something a note IS — and against that shape the old
|
||||
guard was a data-loss path: the first sync after adding a checklist to a note
|
||||
would have wiped it.
|
||||
|
||||
Removed ahead of the UI that can create the state, deliberately, so there is no
|
||||
window in which the two disagree.
|
||||
"""
|
||||
items = ch.get("items")
|
||||
if not isinstance(items, list):
|
||||
# Absent means "not telling us", not "empty". A client that omits the key
|
||||
# leaves what the server has; only an explicit [] clears it.
|
||||
return
|
||||
await db.execute(sa_delete(NoteItem).where(NoteItem.note_id == note.id))
|
||||
for pos, it in enumerate(items):
|
||||
if not isinstance(it, dict):
|
||||
continue
|
||||
text = (it.get("text") or "").strip()
|
||||
if text:
|
||||
db.add(NoteItem(note_id=note.id, text=text, checked=bool(it.get("checked")), position=pos))
|
||||
# `_first_item_text` and `_apply_note_items` lived here until M304. Both existed for
|
||||
# one reason — a checklist was a table beside the body — and both are gone with it. A
|
||||
# pushed change carries its items as `- [ ] ` lines inside `body`, so applying them is
|
||||
# applying the body, and naming the note is reading its first line. A client that still
|
||||
# sends an `items` array is a v2 client, and the version floor below turns it away
|
||||
# before any of this runs.
|
||||
|
||||
|
||||
async def _apply_note_manual_labels(db, note: Note, ch: dict) -> None:
|
||||
@@ -315,7 +277,7 @@ async def _apply_note(db, ch: dict) -> dict:
|
||||
|
||||
old_body = note.body
|
||||
_assign_note_fields(note, ch)
|
||||
note.display_title = derive_display_title(note.body, _first_item_text(ch))
|
||||
note.display_title = derive_display_title(note.body)
|
||||
if edited_at is not None:
|
||||
note.updated_at = edited_at
|
||||
# Non-destructive LWW: snapshot the overwritten server body into history —
|
||||
@@ -327,7 +289,6 @@ async def _apply_note(db, ch: dict) -> dict:
|
||||
if not creating and await should_snapshot(db, note.id, old_body, note.body):
|
||||
db.add(NoteRevision(note_id=note.id, body=old_body))
|
||||
await db.flush() # assign note.id before items/labels/links
|
||||
await _apply_note_items(db, note, ch)
|
||||
await _reconcile_tags(db, note)
|
||||
await _apply_note_manual_labels(db, note, ch)
|
||||
await db.flush()
|
||||
|
||||
Reference in New Issue
Block a user