rename(547): the server is grimoire — package, image, DB defaults, backups

Step 2 of the Scribe -> Grimoire rename.

- src/scribe -> src/grimoire, and every import, patch target, module path,
  hypercorn entry point and `python -m grimoire.restore` with it; pyproject,
  uv.lock and the frontend package are named grimoire.
- Image git.fabledsword.com/bvandeusen/fabledgrimoire; CI's Postgres service
  runs as grimoire / grimoire_test.
- Compose files default the DB user, database and (prod) volume to grimoire,
  each overridable from .env, so an install created as Scribe keeps its data
  by setting POSTGRES_USER, POSTGRES_DB and PGDATA_VOLUME.
- Backups are written as grimoire-<ts>.tar with format grimoire-backup-tar.
  Archives named scribe-* with the scribe-backup-tar manifest still restore
  and are still counted at the destination (folder and S3), with tests.
- Exports, the MCP auth realm, the ASGI scope key and the persistent logger
  carry the new name.

Kept on purpose: the secret-sealing key label stays "scribe.secrets.v1",
because every stored secret was derived under it.

Plugin identity (plugin name, hooks, env vars, .scribe marker, skills) is
step 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

src/grimoire/services/lessons.py

@@ -0,0 +1,651 @@
+"""Lesson service — a transferable insight, retrievable by situation.
+
+A *lesson* is a Note with ``note_type='lesson'``: a better way to think about a
+problem, or a solution that transfers, recorded so a later session meets it at
+the moment it applies — and **without binding the reader**.
+
+WHY THE KIND EXISTS
+
+Without it, rules are the only surface that is global AND situation-keyed, so
+an agent holding a transferable insight has one door, and that door binds —
+and sessions propose rules for things that should not be rules.
+
+The gap is a document-shape fact, not a threshold:
+
+  - a note is embedded as ``title\\nbody`` and is findable by **what it is
+    about**;
+  - a rule is embedded as ``{title} — {trigger}`` with ``When to apply:``
+    repeated at the head of the body, so the trigger appears twice in a short
+    document and dominates the vector — findable by **when it applies**.
+
+No tuning reaches across that: the field the query would match on is simply not
+in a note's document. So a lesson carries a trigger and is embedded like a rule,
+while staying a note in every other respect.
+
+WHERE THE TRIGGER LIVES
+
+In ``notes.data`` under ``when_to_apply``, written through a named parameter and
+mirrored into the head of the body — the shape snippets use for
+``when_to_use``. Not a column on ``notes``. It is NOT in the stored title: the
+title is the lesson's subject, and the trigger joins it only in the embedded
+document (``embeddings.document_title``).
+
+A named structured field rather than a column: snippets show such a field gets
+filled reliably without a guard, and a column would decide enforceability for
+every note kind at once.
+
+The trigger in the document is what makes the vector sharp. It reaches it
+twice — in the embedded title, joined at embed time, and in the body's first
+line.
+
+WHAT A LESSON INHERITS, AND THE CELLS LEFT EMPTY ON PURPOSE
+
+A new kind inherits the note machinery wholesale. Which parts it should NOT get
+is stated here, so that an empty cell is a decision rather than an oversight.
+
+Inherited, all deliberately:
+
+  - **versions** — a lesson is reworded as understanding improves, and what it
+    used to say is worth as much as any note's history.
+  - **supersession** — the event this most needs. A lesson replaced by a better
+    lesson is precisely what ``note_supersessions`` models, and the demotion
+    penalty already exists.
+  - **trash**, **the share ACL**, **tags**, **project and System tagging**,
+    **chunked embeddings**, **the near-duplicate gate**.
+
+NOT inherited, and each for a stated reason:
+
+  - **status / task_kind / milestone_id** — a lesson is not work. ``is_task`` is
+    ``status is not None``, so a lesson that acquired a status would become a
+    task and appear in open-work listings. This is the one cell where filling it
+    in by accident silently changes what the record IS.
+  - **recurrence** — task-only, and a lesson does not recur.
+  - **verify_with / expires_when** — available, because they are generic note
+    fields, but not part of a lesson's contract and not asked for on create.
+    Those mark a record that asserts a FACT about someone else's software and
+    can go false unwatched. A lesson is closer
+    to a norm — "a better way to think about this" has no truth value that rots
+    on its own. A lesson that does assert such a fact can still carry them.
+
+WHY A KIND NEEDS NO MIGRATION
+
+``note_type`` carries **no CHECK constraint** — only ``task_kind`` does
+(``notes_task_kind_check``). It is plain ``Text`` with a server default, so a
+value the database refuses on an un-migrated instance cannot arise for this
+column.
+
+The real vocabulary is ``services.knowledge._FACETS``, which is where a kind
+becomes reachable on the browse surface and validated at the door. That is one
+table feeding both dialects of the type filter, so adding a kind there is a
+single edit.
+"""
+from __future__ import annotations
+
+import re
+
+from grimoire.models import async_session
+from grimoire.models.note import Note
+
+LESSON_NOTE_TYPE = "lesson"
+
+# The key in `notes.data`. Named for the field it mirrors on `rules`, because it
+# answers the same question and a reader who knows one should not have to learn
+# a second word for it.
+TRIGGER_KEY = "when_to_apply"
+
+# What taught this lesson: the ids of the issues, tasks or notes it was drawn
+# from. A LIST, and that is the whole decision — see `normalize_sources`.
+SOURCES_KEY = "taught_by"
+
+# The body's trigger line, and the pattern that reads it back. The body is the
+# readable form and the thing that gets embedded; `data` is the queryable
+# mirror. Reads prefer the mirror and fall back to this, which is the discipline
+# `snippet_fields` follows and the reason a row written before the mirror
+# existed is still readable.
+_BODY_TRIGGER_RE = re.compile(r"^\*\*When to apply:\*\*\s*(.+?)\s*$", re.M)
+
+# The readable mirror of `data[SOURCES_KEY]`, and the pattern that reads it
+# back — the shape snippets use for `**Merged from:** #ids`, for the same
+# reason: the body is what a human sees and what survives a row with no `data`.
+_BODY_SOURCES_RE = re.compile(r"^\*\*Learned from:\*\*\s*(.+?)\s*$", re.M)
+_ID_RE = re.compile(r"#(\d+)")
+
+
+def lesson_trigger(note) -> str:
+    """When this lesson applies, or "" — the mirror first, then the body.
+
+    Prefers `data` for the same reason every snippet read does: it is indexed,
+    and parsing a body to answer a question the database can answer is how a
+    hot path ends up regexing markdown. The fallback is not dead code — it is
+    what makes a lesson readable if the mirror is ever absent, and an absent
+    mirror must degrade to the right answer rather than to silence.
+    """
+    data = getattr(note, "data", None) or {}
+    from_mirror = (data.get(TRIGGER_KEY) or "").strip() if isinstance(data, dict) else ""
+    if from_mirror:
+        return from_mirror
+    match = _BODY_TRIGGER_RE.search(getattr(note, "body", None) or "")
+    return match.group(1).strip() if match else ""
+
+
+def normalize_sources(entries: list | None) -> list[int]:
+    """The ids that taught this lesson — ints, de-duplicated, in the order given.
+
+    THE CARDINALITY DECISION, and why it is a list.
+
+    `arose_from_id` already exists and holds ONE id, which is the obvious first
+    answer and the wrong one. A good lesson often generalises several
+    incidents — a badge collision, an un-backfilled column, a duplicated const
+    — into one claim about a failure class. That is the shape a good lesson
+    HAS, not an edge case. A single
+    id would keep the first and silently drop the rest, and a record that drops
+    two of its three sources is worse than one that names none, because it reads
+    as complete.
+
+    It lives in `notes.data` rather than a join table for the reason the
+    trigger lives there: a join table would settle, for every note kind at
+    once, whether provenance is multi-valued. `data` is JSONB with a GIN
+    index, so the list is
+    queryable today and a table can be migrated to later if the need is shown.
+
+    Order is history, not sorting: the incidents stay in the sequence the writer
+    named them, which is the order they were learned in.
+    """
+    out: list[int] = []
+    seen: set[int] = set()
+    for raw in entries or []:
+        ident = raw.get("id") if isinstance(raw, dict) else raw
+        try:
+            i = int(ident)
+        except (TypeError, ValueError):
+            continue
+        if i > 0 and i not in seen:
+            seen.add(i)
+            out.append(i)
+    return out
+
+
+def lesson_sources(note) -> list[int]:
+    """What taught this lesson — the mirror first, then the body, then
+    `arose_from_id`.
+
+    Three fallbacks rather than two, because the third is what keeps this
+    honest on a record written before the kind existed: a note carrying only
+    `arose_from_id` has exactly one source and this returns it, so a caller
+    never has to ask which field to read.
+    """
+    data = getattr(note, "data", None) or {}
+    if isinstance(data, dict):
+        from_mirror = normalize_sources(data.get(SOURCES_KEY))
+        if from_mirror:
+            return from_mirror
+    match = _BODY_SOURCES_RE.search(getattr(note, "body", None) or "")
+    if match:
+        found = normalize_sources(_ID_RE.findall(match.group(1)))
+        if found:
+            return found
+    single = getattr(note, "arose_from_id", None)
+    return normalize_sources([single]) if single else []
+
+
+def sole_source(sources: list[int] | None) -> int | None:
+    """`arose_from_id` for this lesson: the id when there is exactly ONE.
+
+    Left NULL for a lesson drawn from several, deliberately. Every existing
+    surface that renders provenance reads `arose_from_id` and renders it as
+    THE origin; handing it one of three would make those surfaces state
+    something false. Showing nothing there is accurate — there is no single
+    origin — and `data[SOURCES_KEY]` carries all of them for the surfaces that
+    know to ask.
+    """
+    ids = normalize_sources(sources)
+    return ids[0] if len(ids) == 1 else None
+
+
+def compose_body(
+    insight: str, when_to_apply: str = "", learned_from: list[int] | None = None,
+) -> str:
+    """The lesson body — the trigger line first, the insight after.
+
+    The trigger's home in the text of the document, and half of what makes a
+    lesson findable: `chunk_document` joins `{embedded title}\\n{body}`, and
+    the embedded title is `what — when it applies` (`embeddings.document_title`),
+    so the document states WHEN IT APPLIES in the title and again in the first
+    line of the body. That twice-in-a-short-document shape is what measures
+    sharp in the corpus.
+
+    `**When to apply:**` rather than plain text: the body is the READABLE
+    form, `data` is the queryable mirror, and `_BODY_TRIGGER_RE` reads this
+    line back when the mirror is missing. Its markdown must therefore match
+    what that pattern expects, which is why neither is written by hand
+    anywhere else.
+
+    The insight goes in the body rather than being held out of the document.
+    `rule_document` excludes a rule's `why` because long narrative in ONE
+    vector lands on the centroid of "development" — but a body over the
+    budget is split into several chunks, EACH prefixed with the title, so a
+    lesson's story does not average itself into its trigger: it occupies its
+    own vectors, and every
+    one of them still carries the trigger in its prefix. Holding it out would
+    cost the reader the only part that explains the insight and would buy a
+    sharpness the chunker already provides.
+    """
+    lines = []
+    trigger = (when_to_apply or "").strip()
+    if trigger:
+        lines.append(f"**When to apply:** {trigger}")
+    insight = (insight or "").strip()
+    if insight:
+        lines.append(insight)
+    sources = normalize_sources(learned_from)
+    if sources:
+        # LAST, not beside the trigger. The first line has to be what this
+        # lesson is FOR; a provenance line above the insight would push the
+        # thing the reader came for below a list of ids, and would put
+        # numbers where the trigger's second appearance does its work.
+        lines.append("**Learned from:** " + ", ".join(f"#{i}" for i in sources))
+    return "\n\n".join(lines)
+
+
+# A lesson's NAME is its claim, and the claim is one line. `what` is the title
+# every listing, menu and search row prints, so a story written there instead
+# of in `insight` turns each of those lines into a document, with the claim it
+# exists to transfer somewhere in the middle. It also ranks badly: the embedded title
+# is `what — when_to_apply`, so a narrative crowds out the trigger.
+#
+# 240 is a long sentence. The bound is on what a NAME can be, not on what a
+# lesson can say — `insight` is the body, unbounded, chunked like any record,
+# and costs the ranking nothing.
+WHAT_MAX_CHARS = 240
+
+
+def require_claim(what: str | None) -> None:
+    """Refuse a `what` that is not one line — before anything is written.
+
+    A refusal rather than a cut, because a cut name reads as complete: the
+    writer would never learn the story went to the wrong field, and the
+    reader would get its first 240 characters as though they were the claim.
+    """
+    text = (what or "").strip()
+    multiline = "\n" in text
+    if not multiline and len(text) <= WHAT_MAX_CHARS:
+        return
+    shape = "runs over several lines" if multiline else f"is {len(text)} characters"
+    raise ValueError(
+        f"`what` {shape}; it is the lesson's NAME — one claim, one line, at "
+        f"most {WHAT_MAX_CHARS} characters — and every listing and menu prints "
+        "it whole. Nothing was written. Put the claim in `what`, said as the "
+        "sentence someone should come away with, and the incident that taught "
+        "it in `insight`, which is the body and is not bounded."
+    )
+
+
+def claim_line(what: str | None) -> str:
+    """A stored name as one menu line, whatever was stored.
+
+    The display half of `require_claim`, for names written before it existed:
+    a guard at the door does not undo a value already stored, and on an
+    install nobody has repaired, those rows would still print their whole
+    story on every menu. Over the bound, the name's first sentence is shown
+    and marked as cut — the record itself is unchanged, and opening it shows
+    the rest.
+    """
+    text = " ".join((what or "").split())
+    if len(text) <= WHAT_MAX_CHARS:
+        return text
+    first = re.split(r"(?<=[.!?])\s", text, maxsplit=1)[0]
+    if len(first) > WHAT_MAX_CHARS:
+        first = first[:WHAT_MAX_CHARS].rsplit(" ", 1)[0]
+    return first + " …"
+
+
+def lesson_document(
+    what: str, when_to_apply: str = "", insight: str = "",
+    learned_from: list[int] | None = None,
+) -> tuple[str, str]:
+    """The (title, body) a lesson is STORED as.
+
+    One call so the two halves cannot be composed apart. The body's first line
+    carries the trigger; a lesson without it (and without the `data` mirror)
+    would embed as an ordinary note wearing a label, and nothing would report
+    it: the record would look right in every listing and simply never be
+    retrieved at the moment it applies.
+
+    The title is the SUBJECT alone. A stored title carrying the trigger would
+    make every listing, menu and search row show a title that runs to
+    kilobytes. The trigger joins the title at embed time
+    (`embeddings.document_title`, reading `data`).
+    """
+    return (
+        (what or "").strip(),
+        compose_body(insight, when_to_apply, learned_from),
+    )
+
+
+def compose_data(
+    what: str, when_to_apply: str = "", learned_from: list[int] | None = None,
+) -> dict:
+    """The indexed mirror of the same fields the body renders (0070).
+
+    Written together with the body by the one caller that composes both, so the
+    two can never describe different things — the discipline `compose_data`
+    follows for snippets, and the reason `lesson_trigger` can prefer `data`
+    without checking whether it agrees with the prose.
+
+    Empty values are omitted so the column stays sparse: a lesson with no
+    sources has no `taught_by` key rather than an empty list, which keeps a
+    `?` containment query honest.
+    """
+    data: dict = {"what": what.strip()} if what and what.strip() else {}
+    trigger = (when_to_apply or "").strip()
+    if trigger:
+        data[TRIGGER_KEY] = trigger
+    sources = normalize_sources(learned_from)
+    if sources:
+        data[SOURCES_KEY] = sources
+    return data
+
+
+def recompose_data(note) -> dict:
+    """Rebuild a lesson's `data` mirror from its own title and body.
+
+    For the GENERIC note door. `update_lesson` composes the mirror itself from
+    the merged field set and never needs this; a plain `update_note(body=...)`
+    has no idea the mirror exists and would leave it behind.
+
+    THE COST OF LEAVING IT BEHIND IS HIGHER HERE THAN FOR A SNIPPET. A stale
+    snippet mirror reports the wrong path. A stale lesson mirror reports the
+    wrong TRIGGER — and `lesson_trigger` prefers the mirror, so the lesson goes
+    on being retrieved for the situation it used to name while displaying the
+    one it now names. The trigger is the entire retrieval story, so
+    that is not a degraded record; it is a record that fires at the wrong
+    moment and looks right when it does.
+
+    The body is the authority and the mirror is derived — already this file's
+    rule. This is its enforcement on the path that bypasses `update_lesson`.
+
+    The subject comes back out of the title through `untrigger_title`, the
+    inverse of the join that composed it, rather than by splitting on a
+    separator spelled a second time here.
+    """
+    from grimoire.services.embeddings import untrigger_title
+
+    body = getattr(note, "body", None) or ""
+    trigger_match = _BODY_TRIGGER_RE.search(body)
+    trigger = trigger_match.group(1).strip() if trigger_match else ""
+    what = untrigger_title(getattr(note, "title", None), trigger)
+    # Sources through the normal read, which already falls back body →
+    # arose_from_id. A body edit that drops the provenance line should drop
+    # the mirror's copy too: the body is the authority, and carrying a value
+    # the reader just deleted is the failure this function exists to prevent.
+    sources_match = _BODY_SOURCES_RE.search(body)
+    sources = (
+        normalize_sources(_ID_RE.findall(sources_match.group(1)))
+        if sources_match else []
+    )
+    return compose_data(what, trigger, sources)
+
+
+async def create_lesson(
+    user_id: int,
+    *,
+    what: str,
+    when_to_apply: str = "",
+    insight: str = "",
+    learned_from: list[int] | None = None,
+    tags: list[str] | None = None,
+    project_id: int | None = None,
+):
+    """Create a lesson note. Returns the created Note.
+
+    The title, the body and the `data` mirror are composed HERE from named
+    parameters rather than asked of the caller. That is the design, not a
+    convenience: a named structured field that a service composes is filled
+    reliably, where a convention an agent must type correctly is not.
+
+    `project_id` is accepted and kept, even though a lesson is reachable from
+    every project. Where it was learned is a fact worth keeping; it
+    simply stops being the limit of where it can be found.
+    """
+    from grimoire.services import notes as notes_svc
+
+    sources = normalize_sources(learned_from)
+    title, body = lesson_document(what, when_to_apply, insight, sources)
+    return await notes_svc.create_note(
+        user_id,
+        title=title,
+        body=body,
+        note_type=LESSON_NOTE_TYPE,
+        tags=tags,
+        project_id=project_id,
+        # NULL unless there is exactly one source — see `sole_source`.
+        arose_from_id=sole_source(sources),
+        data=compose_data(what, when_to_apply, sources),
+    )
+
+
+async def get_lesson(user_id: int, lesson_id: int):
+    """Fetch a lesson by id, or None if it isn't one / isn't readable.
+
+    Share-aware (services/access.py): a fetch by id is an explicit act, so it
+    resolves the caller's full read scope rather than ownership alone —
+    without this, a lesson a search legitimately surfaced could not then be
+    opened.
+    """
+    from grimoire.services import notes as notes_svc
+
+    result = await notes_svc.get_note_for_user(user_id, lesson_id)
+    if result is None:
+        return None
+    note, _permission = result
+    if note.note_type != LESSON_NOTE_TYPE or note.deleted_at is not None:
+        return None
+    return note
+
+
+def lesson_to_dict(note) -> dict:
+    """A lesson as either door returns it — the composed fields read back out,
+    not the raw row, so a caller sees the same vocabulary it wrote with.
+
+    In the SERVICE rather than in each door, on the `snippet_to_dict`
+    precedent: the REST route feeds the web UI and the MCP tools feed an
+    agent, and a shape spelled once per door is a shape that answers the two
+    of them differently the first time a field is added.
+    """
+    return {
+        "id": note.id,
+        "title": note.title,
+        "body": note.body,
+        # The composed vocabulary, not the storage: a caller that wrote
+        # `when_to_apply` reads `when_to_apply` back.
+        "what": (note.data or {}).get("what", "") if isinstance(note.data, dict) else "",
+        "when_to_apply": lesson_trigger(note),
+        "learned_from": lesson_sources(note),
+        "insight": lesson_insight(note),
+        "tags": list(note.tags or []),
+        "project_id": note.project_id,
+        "note_type": note.note_type,
+        "created_at": note.created_at.isoformat() if note.created_at else None,
+        "updated_at": note.updated_at.isoformat() if note.updated_at else None,
+    }
+
+
+async def source_records(user_id: int, ids: list[int]) -> list[dict]:
+    """The records that taught a lesson, resolved to something linkable.
+
+    `learned_from` is a list of bare ids, which is right for storage and
+    useless on a page: "#4181" tells a reader nothing about whether it is
+    worth opening. This resolves each to its title and kind so the UI can
+    label the link, and so a source that has been deleted simply drops out
+    rather than rendering a link to nothing.
+
+    One query for the whole list, not one per id — a lesson with six sources
+    would otherwise be six round trips to draw one panel.
+
+    Share-aware (services/access.py). Order follows `ids`, because that order is the
+    writer's: the first source is the one they reached for first.
+    """
+    from sqlalchemy import select
+
+    from grimoire.services.access import readable_notes_clause
+
+    wanted = normalize_sources(ids)
+    if not wanted:
+        return []
+    async with async_session() as session:
+        result = await session.execute(
+            select(Note)
+            .where(Note.id.in_(wanted))
+            .where(Note.deleted_at.is_(None))
+            .where(readable_notes_clause(user_id))
+        )
+        found = {n.id: n for n in result.scalars().all()}
+    return [
+        {
+            "id": n.id,
+            "title": n.title,
+            "note_type": n.note_type,
+            "is_task": n.status is not None,
+            "task_kind": n.task_kind,
+            "status": n.status,
+        }
+        for i in wanted
+        if (n := found.get(i)) is not None
+    ]
+
+
+async def lessons_taught_by(user_id: int, record_id: int, limit: int = 20):
+    """The lessons drawn FROM one record — the reverse of `learned_from`.
+
+    THE DIRECTION THAT GETS FORGOTTEN, and arguably the more useful one: a
+    reader opening an old issue wants to know what was learned from it, and
+    without this the relation is only navigable from the lesson's side. A
+    record that taught something should say so on its own page.
+
+    Queried through `data[SOURCES_KEY]` rather than by scanning bodies: the
+    mirror is JSONB with a GIN index, which is the whole reason the list
+    lives there. `path_exists` is the same dialect the snippet location
+    lookup uses, so both reverse lookups read the index the same way.
+
+    Share-aware via `readable_notes_clause`: this renders beside a
+    record the caller can already see, and a lesson someone shared with them
+    belongs in that list exactly as their own does.
+    """
+    from sqlalchemy import select
+
+    from grimoire.services.access import readable_notes_clause
+
+    try:
+        wanted = int(record_id)
+    except (TypeError, ValueError):
+        return []
+    if wanted <= 0:
+        return []
+
+    # The id is an int we just validated, never caller text, so it cannot
+    # break out of the expression — the same guarantee `location_jsonpath`
+    # gets from JSON-quoting its values.
+    jsonpath = f"$.{SOURCES_KEY}[*] ? (@ == {wanted})"
+    async with async_session() as session:
+        result = await session.execute(
+            select(Note)
+            .where(Note.note_type == LESSON_NOTE_TYPE)
+            .where(Note.deleted_at.is_(None))
+            .where(Note.data.path_exists(jsonpath))
+            .where(readable_notes_clause(user_id))
+            .order_by(Note.updated_at.desc())
+            .limit(max(1, min(limit, 100)))
+        )
+        return list(result.scalars().all())
+
+
+async def update_lesson(
+    user_id: int,
+    lesson_id: int,
+    *,
+    what: str | None = None,
+    when_to_apply: str | None = None,
+    insight: str | None = None,
+    learned_from: list[int] | None = None,
+    tags: list[str] | None = None,
+):
+    """Update a lesson, re-composing title, body and mirror from the merged
+    fields. Returns the updated Note, or None if it isn't a readable lesson.
+
+    READ-MODIFY-WRITE over the whole record rather than patching one half.
+    The three fields are not independent: the trigger appears in the title AND
+    at the head of the body, so editing it in place would need two edits that
+    a caller could do one of. Re-composing from the merged values means a
+    partial update cannot leave the halves disagreeing — which, because the
+    document is what ranks, would be a lesson that still reads correctly and
+    quietly stops being retrievable.
+    """
+    from grimoire.services import notes as notes_svc
+
+    note = await get_lesson(user_id, lesson_id)
+    if note is None:
+        return None
+
+    current = note.data if isinstance(note.data, dict) else {}
+    merged_what = current.get("what") or "" if what is None else what
+    merged_trigger = lesson_trigger(note) if when_to_apply is None else when_to_apply
+    merged_sources = (
+        lesson_sources(note) if learned_from is None
+        else normalize_sources(learned_from)
+    )
+    if insight is None:
+        insight = lesson_insight(note)
+
+    title, body = lesson_document(
+        merged_what, merged_trigger, insight, merged_sources,
+    )
+    fields: dict = {
+        "title": title,
+        "body": body,
+        "arose_from_id": sole_source(merged_sources),
+        "data": compose_data(merged_what, merged_trigger, merged_sources),
+    }
+    if tags is not None:
+        fields["tags"] = tags
+    return await notes_svc.update_note(user_id, lesson_id, **fields)
+
+
+def lesson_insight(note) -> str:
+    """The authored insight alone — the body with what `compose_body` added removed.
+
+    An update that keeps the insight hands it back to `compose_body`, which
+    re-adds the trigger and provenance. So this has to remove EXACTLY what was
+    added; anything it leaves behind is re-prepended and the body grows by that
+    much on every edit.
+
+    TAKES THE NOTE, NOT THE BODY. Filtering out lines matching
+    `_BODY_TRIGGER_RE` is not enough: that pattern is `re.M` and matches ONE
+    LINE, while `when_to_apply` is free text and routinely several paragraphs
+    — `compose_body` writes all of it after the marker. Paragraphs 2..n would
+    come back as part of "the insight", the full trigger would be prepended
+    again, and the record would gain (n-1) paragraphs per update. Silent, and
+    compounding.
+
+    What makes an exact strip possible is that the trigger is stored: the
+    mirror in `data` holds its full text, so `composed` below is
+    byte-for-byte what `compose_body` put at the head, and removing it is
+    subtraction rather than pattern-matching.
+
+    The fallback is not dead code. A row whose mirror is missing degrades to
+    line-wise removal, which is right for a one-paragraph trigger and is the
+    best that can be done without knowing where the trigger ends — there is
+    no marker for its last line.
+
+    The provenance line needs no such care: `compose_body` builds it by
+    joining ids with ", ", so it cannot contain a newline.
+    """
+    body = getattr(note, "body", None) or ""
+    trigger = lesson_trigger(note).strip()
+    head = body.lstrip("\n")
+    composed = f"**When to apply:** {trigger}" if trigger else ""
+    if composed and head.startswith(composed):
+        lines = head[len(composed):].splitlines()
+    else:
+        lines = [ln for ln in body.splitlines() if not _BODY_TRIGGER_RE.match(ln)]
+    kept = [ln for ln in lines if not _BODY_SOURCES_RE.match(ln)]
+    return "\n".join(kept).strip()