feat(lessons): the lesson kind, and one join for every trigger title (#3729)

Milestone 385 step 2, implementing decision #4157 from the step-1 spike.

The kind: `note_type='lesson'`, a note findable by WHEN IT APPLIES rather
than by what it is about. The trigger lives in `notes.data.when_to_apply`,
mirrored into the title and the head of the body — the shape snippets
already use, and the reason nothing re-embeds: chunk_document is untouched,
so CHUNKER_VERSION does not move.

NO MIGRATION, and the step assumed there would be one. `note_type` carries
no CHECK — only `task_kind` does (0056, 0065). Migration 0036 added it as
plain Text with a server default and nothing has gated it since, so rule 36
has no whitelist to expand and #3128's failure mode (a value the database
refuses) cannot arise for this column. The vocabulary that actually decides
what a reader can reach is services.knowledge._FACETS, which since #3161 is
one table feeding the door's validation, the counts and both dialects of
the type filter — so the kind lands there in a single edit.

ONE JOIN, not a fourth copy. `{subject} — {trigger}` had three
implementations: rule_document, snippets.compose_title, and this step
needed another. #3207 records what that costs, so the join moves to
embeddings.trigger_title beside embedding_text and all three delegate.
Behaviour is unchanged for rules and snippets; the guard calls each through
its own public name, so a re-implementation fails it.

The #3163 bill is stated in the service docstring rather than left to be
inferred: versions, supersession, trash, the share ACL, tags, project and
System tagging, chunked embeddings and the duplicate gate are all
inherited; status/task_kind/milestone_id and recurrence are not, and
verify_with/expires_when are available but outside the kind's contract.
The status cell is the one that matters — `is_task` IS `status is not
None`, so a lesson that acquired one would become a task.

The integration guard asserts the WRITE rather than the constraint: it
holds whether or not note_type is ever gated, and goes red only if it is
gated without this value.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

src/scribe/models/note.py

@@ -76,8 +76,27 @@ class Note(Base, TimestampMixin, SoftDeleteMixin):
     recurrence_next_spawn_at: Mapped[datetime | None] = mapped_column(
         DateTime(timezone=True), nullable=True
     )
-    # Note type — 'note' (default) or 'process' (a stored process). Task-ness is
-    # tracked by `status`, not here. (person/place/list entity types removed 2026-07.)
+    # WHAT KIND of record this is, on the note/entity axis. Task-ness is tracked
+    # by `status`, not here (person/place/list entity types removed 2026-07):
+    #   note    (default) — authored prose, findable by what it is ABOUT
+    #   process — a stored procedure, synced to a client as a skill
+    #   snippet — a reusable shape, with its structured fields mirrored in
+    #             `data` and its trigger composed into the title (0070)
+    #   lesson  — a transferable insight, findable by WHEN IT APPLIES rather
+    #             than by topic (milestone 385). Same mirror discipline as a
+    #             snippet: the trigger lives in `data.when_to_apply` and is
+    #             composed into the title and the head of the body, which is
+    #             what puts it in the embedded document. A lesson is never a
+    #             task — see `status` — because a lesson that acquired one
+    #             would start appearing in open-work listings.
+    #
+    # DELIBERATELY UNGATED. Unlike `task_kind` there is no CHECK on this column:
+    # migration 0036 added it as plain Text with a server default and nothing
+    # has constrained it since, so rule 36 has no whitelist to expand when a
+    # kind is added. The vocabulary that actually decides what a reader can
+    # reach is `services.knowledge._FACETS`; an unrecognised value there
+    # resolves to a filter matching nothing, which is the intended answer to a
+    # typo.
     note_type: Mapped[str] = mapped_column(Text, default="note", server_default="note")
     # Task sub-kind — what KIND of work this is, not how it is going:
     #   work   (default) — ships a change

src/scribe/services/embeddings.py

@@ -203,6 +203,33 @@ def embedding_text(title: str | None, body: str | None) -> str:
     return f"{title}\n{body}".strip() if body else title
 
 
+def trigger_title(subject: str | None, trigger: str | None) -> str:
+    """`{subject} — {trigger}` — the title half of a situation-keyed document.
+
+    ONE definition, because this join had three. `rule_document` built it for
+    rules, `snippets.compose_title` for snippets, and milestone 385 needed a
+    fourth for lessons — the shape #3207 records, where a fix or an improvement
+    then has to be found in N places by someone who does not know N.
+
+    WHY THE JOIN MATTERS AT ALL, measured in note #2485: the snippet was the
+    only sharp record in the corpus — a 0.153 top-to-second gap against
+    0.010–0.023 for everything else — and the cause was this title plus the
+    same trigger repeated in the body, so purpose appears twice in a short
+    document and dominates the vector. Every kind that must be findable by WHEN
+    IT APPLIES rather than what it is about is built on this line.
+
+    Either side alone is returned as-is: a record with no trigger yet degrades
+    to its subject and still embeds, just less sharply — which is an argument
+    for backfilling triggers, not for padding the title with whatever text is
+    to hand.
+    """
+    subject = (subject or "").strip()
+    trigger = (trigger or "").strip()
+    if subject and trigger:
+        return f"{subject} — {trigger}"
+    return subject or trigger
+
+
 # --- chunking (#280): the document shape ------------------------------------
 #
 # bge-small reads at most 512 tokens and fastembed silently truncates the rest,
@@ -785,7 +812,7 @@ def rule_document(
     if not trigger:
         return name or None, body or None
     return (
-        f"{name} — {trigger}" if name else trigger,
+        trigger_title(name, trigger),
         f"When to apply: {trigger}\n\n{body}" if body else f"When to apply: {trigger}",
     )
 

src/scribe/services/knowledge.py

@@ -292,6 +292,13 @@ _FACETS: dict[str, tuple[bool, str | None]] = {
     "note": (False, "note"),
     "process": (False, "process"),
     "snippet": (False, "snippet"),
+    # A transferable insight, keyed by the situation it applies to rather than
+    # by its topic (milestone 385). It reaches the browse surface through this
+    # one entry: the door's validation, the counts and both dialects of the
+    # type filter are all generated from this table, which is the property
+    # #3161 asked for so that adding a kind is a single edit rather than four
+    # that must agree.
+    "lesson": (False, "lesson"),
 }
 
 # The non-task record types, for the counts query. Derived so it cannot drift

src/scribe/services/lessons.py

@@ -0,0 +1,138 @@
+"""Lesson service — a transferable insight, retrievable by situation.
+
+A *lesson* is a Note with ``note_type='lesson'``: a better way to think about a
+problem, or a solution that transfers, recorded so a later session meets it at
+the moment it applies — and **without binding the reader**.
+
+WHY THE KIND EXISTS (milestone 385, from note #3727)
+
+Rules were the only surface that is global AND situation-keyed, so an agent
+holding a transferable insight had one door, and that door binds. The observed
+symptom was sessions offering rule proposals for things that should not be
+rules.
+
+The gap is a document-shape fact, not a threshold:
+
+  - a note is embedded as ``title\\nbody`` and is findable by **what it is
+    about**;
+  - a rule is embedded as ``{title} — {trigger}`` with ``When to apply:``
+    repeated at the head of the body, so the trigger appears twice in a short
+    document and dominates the vector — findable by **when it applies**.
+
+No tuning reaches across that: the field the query would match on is simply not
+in a note's document. So a lesson carries a trigger and is embedded like a rule,
+while staying a note in every other respect.
+
+WHERE THE TRIGGER LIVES (decision #4157, milestone 385 step 1)
+
+In ``notes.data`` under ``when_to_apply``, written through a named parameter and
+mirrored into the title and the head of the body — the shape snippets already
+use for ``when_to_use``. Not a column on ``notes``.
+
+That decision was measured rather than assumed. The whole snippet corpus —
+164 of 164 — carries a ``when_to_use`` with **no guard anywhere**, which refutes
+the premise that an unenforced field gets skipped. What it does NOT show is that
+an agent types a title convention correctly: ``compose_title`` builds the title
+from the parameter, so what is at 100% is a named structured field. A column
+would have bought enforceability at the price of deciding, for every note kind
+at once, a question nothing had measured.
+
+The mirror is what makes the vector sharp, and it is why nothing re-embeds:
+``chunk_document`` is untouched, so ``CHUNKER_VERSION`` does not move. The
+trigger reaches the document by being in the text, exactly as a snippet's is.
+
+WHAT A LESSON INHERITS, AND THE CELLS LEFT EMPTY ON PURPOSE (#3163)
+
+A new kind inherits the note machinery wholesale, and #3163 asks which parts it
+should NOT get — so that an empty cell is a decision rather than an oversight.
+
+Inherited, all deliberately:
+
+  - **versions** — a lesson is reworded as understanding improves, and what it
+    used to say is worth as much as any note's history.
+  - **supersession** — the event this most needs. A lesson replaced by a better
+    lesson is precisely what ``note_supersessions`` models, and the demotion
+    penalty already exists.
+  - **trash**, **the share ACL**, **tags**, **project and System tagging**,
+    **chunked embeddings**, **the near-duplicate gate**.
+
+NOT inherited, and each for a stated reason:
+
+  - **status / task_kind / milestone_id** — a lesson is not work. ``is_task`` is
+    ``status is not None``, so a lesson that acquired a status would become a
+    task and appear in open-work listings. This is the one cell where filling it
+    in by accident silently changes what the record IS.
+  - **recurrence** — task-only, and a lesson does not recur.
+  - **verify_with / expires_when** — available, because they are generic note
+    fields, but not part of a lesson's contract and not asked for on create. The
+    milestone-312 distinction is why: those mark a record that asserts a FACT
+    about someone else's software and can go false unwatched. A lesson is closer
+    to a norm — "a better way to think about this" has no truth value that rots
+    on its own. A lesson that does assert such a fact can still carry them.
+
+WHY THERE IS NO MIGRATION
+
+``note_type`` carries **no CHECK constraint** — only ``task_kind`` does
+(``notes_task_kind_check``, migrations 0056 / 0065). Migration 0036 added
+``note_type`` as plain ``Text`` with a server default and nothing has gated it
+since. So rule 36 has nothing to expand here, and the failure it guards against
+— a value the database refuses on an instance predating its migration — cannot
+arise for this column.
+
+The real vocabulary is ``services.knowledge._FACETS``, which is where a kind
+becomes reachable on the browse surface and validated at the door. That is one
+table feeding both dialects of the type filter, so adding a kind there is a
+single edit — a property #3161 recommended and that landed before this.
+"""
+from __future__ import annotations
+
+import re
+
+LESSON_NOTE_TYPE = "lesson"
+
+# The key in `notes.data`. Named for the field it mirrors on `rules`, because it
+# answers the same question and a reader who knows one should not have to learn
+# a second word for it.
+TRIGGER_KEY = "when_to_apply"
+
+# The body's trigger line, and the pattern that reads it back. The body is the
+# readable form and the thing that gets embedded; `data` is the queryable
+# mirror. Reads prefer the mirror and fall back to this, which is the discipline
+# `snippet_fields` follows and the reason a row written before the mirror
+# existed is still readable.
+_BODY_TRIGGER_RE = re.compile(r"^\*\*When to apply:\*\*\s*(.+?)\s*$", re.M)
+
+
+def lesson_trigger(note) -> str:
+    """When this lesson applies, or "" — the mirror first, then the body.
+
+    Prefers `data` for the same reason every snippet read does: it is indexed,
+    and parsing a body to answer a question the database can answer is how a
+    hot path ends up regexing markdown. The fallback is not dead code — it is
+    what makes a lesson readable if the mirror is ever absent, and an absent
+    mirror must degrade to the right answer rather than to silence.
+    """
+    data = getattr(note, "data", None) or {}
+    from_mirror = (data.get(TRIGGER_KEY) or "").strip() if isinstance(data, dict) else ""
+    if from_mirror:
+        return from_mirror
+    match = _BODY_TRIGGER_RE.search(getattr(note, "body", None) or "")
+    return match.group(1).strip() if match else ""
+
+
+def compose_title(what: str, when_to_apply: str = "") -> str:
+    """`{what} — {when it applies}`, the half of the document that ranks.
+
+    Built HERE rather than asked of the caller, and that distinction is the
+    whole evidence base for this design: the snippet corpus is at 100% on its
+    trigger because a service composes the title from a named parameter, not
+    because agents type separators reliably. A caller made to spell the
+    convention is the option milestone 385 step 1 rejected.
+
+    The join is `embeddings.trigger_title` — shared with rules and snippets, so
+    the three kinds that rank on a trigger cannot drift apart in how they say
+    so.
+    """
+    from scribe.services.embeddings import trigger_title
+
+    return trigger_title(what, when_to_apply)

src/scribe/services/snippets.py

@@ -55,10 +55,15 @@ UNSET: object = object()
 # --- serialize: structured fields -> note (title/body/tags) ------------------
 
 def compose_title(name: str, when_to_use: str = "") -> str:
-    """`name — when to use` (or just `name` when no usage note is given)."""
-    name = (name or "").strip()
-    when = (when_to_use or "").strip()
-    return f"{name} — {when}" if when else name
+    """`name — when to use` (or just `name` when no usage note is given).
+
+    The join itself lives in `embeddings.trigger_title`, which rules and
+    lessons build their titles from too. Kept as a named function here because
+    it is this module's public vocabulary and callers say `compose_title`.
+    """
+    from scribe.services.embeddings import trigger_title
+
+    return trigger_title(name, when_to_use)
 
 
 def compose_tags(language: str = "", tags: list[str] | None = None) -> list[str]: