feat(lessons): a lesson reaches the moment it applies, and says it binds nothing (#3732)

WHICH ARM — the two note arms, and no new one.

`write_path` filters kinds, so a lesson was not outranked there but
unreachable, which is #3702's shape: an arm that never had the candidate
reports a healthy bar. It now asks for lessons alongside snippets and
issues. The founding example of the kind is a lesson about a code shape,
and this is the arm that fires when code is written.

`auto_inject` does not filter kinds, so lessons were already candidates —
but scoped to the bound project, which for a kind whose whole claim is
that it transfers is the same silence. Both arms now pass
`include_global_kinds` (#3730).

WHOSE BUDGET — a reserved slot in the prompt menu, none on the write path.

The step's premise needs a correction: the notes menu and the rule hints
are separate functions with separate budgets, so a line reserved here
displaces a note, never a rule. (`RULEHINT_LIMIT` is also 5, not 1, since
#4102 made it a default rather than a cap.)

The trade taken: a note crowded out is a lost convenience and a rule
crowded out still fires at an act arm, but a lesson crowded out is the
feature failing — a lesson exists only to be met at the moment it
applies, so the arm IS its delivery and the loss is total and silent.
That is `preference_slot`'s argument, and the rarity is `reuse_slot`'s.
It buys position, never a lower bar, and it EXTENDS rather than evicting:
a displaced hit sits in the general search's own log row, and evicting it
would make two tables disagree about one call (#3668, #379).

No slot on the write path: that arm fires before every Write and Edit,
where a guaranteed extra line is a guaranteed extra interruption, and its
field is already just snippets, issues and lessons rather than the whole
corpus.

`lesson_slot` logs its own retrieval and its own surfacing from the first
deploy, and the general contest stays open to the kind — otherwise "the
slot earns its line" would be true by construction.

THE VOICE — "they don't always have to be followed".

The menu's register is already the non-binding one. What it lacked is
that a lesson reads as one more title in a list of material when it is
advice someone paid for. One clause, in the header, only when a lesson is
on the menu: weigh it, use your judgement, it is not a rule and binds
nothing. It deliberately does not borrow the rule arms' "before deciding
it does not apply", and a guard asserts that phrase never appears.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

src/scribe/services/plugin_context.py

@@ -28,6 +28,7 @@ from scribe.services import shape_ledger as shape_ledger_svc
 from scribe.services import snippets as snippets_svc
 from scribe.services.access import label_shared_items, owner_names_for
 from scribe.services.embeddings import semantic_search_notes, semantic_search_rules
+from scribe.services.lessons import LESSON_NOTE_TYPE
 from scribe.services.note_usage import record_surfaced
 from scribe.services.rule_usage import record_rule_surfaced
 from scribe.services.supersession import superseded_ids
@@ -759,6 +760,122 @@ async def _reserve_slot_for_reuse(
     return (kept + fresh)[:top_k]
 
 
+async def _reserve_slot_for_lesson(
+    user_id: int,
+    query: str,
+    kept: list,
+    cfg: dict,
+    *,
+    project_id: int | None,
+    already: set[int],
+) -> tuple[list, int | None]:
+    """Guarantee a lesson one slot, if one clears the bar (milestone 385 step 5).
+
+    WHY A SLOT, AND WHAT IT ACTUALLY DISPLACES
+
+    The step's own framing was that "a slot spent on a lesson is a slot not
+    spent on a rule that binds". That is not what happens here, and the
+    correction matters for judging the cost: the notes menu and the rule hints
+    are separate functions with separate budgets, composed by the caller
+    (`build_prompt_rule_hint` says why). A line reserved in THIS menu displaces
+    a note, a snippet or an issue — never a rule.
+
+    THE ASYMMETRY, which is `preference_slot`'s argument on a different corpus:
+
+      - a NOTE crowded out of this menu is a lost convenience. It stays
+        searchable, and the operator can ask for it.
+      - a RULE crowded out still fires at an act arm. The prompt hit is a
+        preview of a second chance.
+      - a LESSON crowded out is the feature failing. A lesson exists only to be
+        met at the moment it applies — nobody browses lessons looking for one —
+        so the arm that surfaces it IS its delivery, and the loss is total and
+        silent. Silent delivery failure is the exact shape #3727 recorded: an
+        insight with no home arrived as a rule proposal instead.
+
+    And the ratio only moves one way. Lessons are by design rare and hard-won
+    while project records grow with the work, which is the 200:1 problem
+    `reuse_slot` was built for (#2246), before it has had a chance to be
+    measured here.
+
+    THE SLOT BUYS POSITION, NOT A LOWER BAR. It reserves at the menu's own
+    threshold, so a weak lesson cannot buy the line and silence stays the
+    default — the discipline both existing slots keep.
+
+    IT EXTENDS, IT NEVER DISPLACES, siding with `preference_slot` over
+    `reuse_slot`. Two reasons, and the second is the one that would be hard to
+    recover later: a displaced hit was returned by the general search and sits
+    in that call's `retrieval_logs` row, so evicting it makes the two tables
+    disagree about the same call for a reason nothing in the data explains
+    (#3668, and milestone #379 is what that costs). The first is voice — a
+    lesson does not bind, and a record that does not bind should not be able
+    to throw a better-scoring one off the menu.
+
+    Returns the possibly-extended list and the id the slot spent. The caller
+    needs that id to keep each source's surfaced set matching its own log row:
+    this slot records its own surfacing under its own name, so counting it
+    again under `auto_inject` would double it.
+    """
+    if any(_record_kind(n) == LESSON_NOTE_TYPE for _s, n in kept):
+        return kept, None            # a lesson already placed on score
+
+    _t0 = time.perf_counter()
+    _rep: dict = {}
+    # KIND-FILTERED, so the slot can only ever be spent on what it is for —
+    # `preference_slot`'s reasoning: verifying the kind after an open search
+    # would let a stray note buy the line, and that line would be
+    # indistinguishable from one that earned its place.
+    #
+    # `include_global_kinds` is the half that makes a lesson reachable at all
+    # from a project it was not written on, which is this kind's whole claim
+    # (#3730). Without it the slot would be a guarantee that silently only
+    # applies to lessons learned here.
+    found = await semantic_search_notes(
+        user_id, query,
+        limit=1,
+        threshold=cfg["threshold"],
+        project_id=project_id,
+        exclude_ids={int(n.id) for _s, n in kept},
+        note_type=(LESSON_NOTE_TYPE,),
+        include_global_kinds=True,
+        scope="browse",
+        report=_rep,
+    )
+    fresh = [(s, n) for s, n in found if int(n.id) not in already]
+    # ITS OWN SOURCE, from the first deploy. This slot is a claim that a kind
+    # deserves a guaranteed line, and a claim like that has to be falsifiable:
+    # `best_available_id` (#3807) names the lesson a bar refused, and the
+    # result count says how often the guarantee was actually spent. Without
+    # this row the question "does the lesson slot earn its line?" would have no
+    # data behind it in either direction — which is #2463's finding, recorded
+    # about the slot that shipped without one.
+    record_retrieval(
+        user_id=user_id, source="lesson_slot", query=query,
+        threshold=cfg["threshold"], limit=1, project_id=project_id,
+        is_task=None, results=fresh,
+        best_available=_rep.get("best_available_score"),
+        best_available_id=_rep.get("best_available_id"),
+        searched=bool(_rep.get("searched", True)),
+        suppressed=len(found) - len(fresh),
+        duration_ms=(time.perf_counter() - _t0) * 1000.0,
+    )
+    kept_ids = {int(n.id) for _s, n in kept}
+    slot = [
+        (s, n) for s, n in found
+        if _record_kind(n) == LESSON_NOTE_TYPE and int(n.id) not in kept_ids
+    ][:1]
+    if not slot:
+        return kept, None
+
+    slot_id = int(slot[0][1].id)
+    # FRESH ONLY, matching the row above: a ledger repeat is rendered (#4101)
+    # but is not a new surfacing, so this source's two tables stay identical.
+    if slot_id not in already:
+        record_surfaced(
+            user_id=user_id, note_ids=[slot_id], source="lesson_slot",
+        )
+    return kept + slot, slot_id
+
+
 async def build_autoinject_hint(
     user_id: int,
     query: str,
@@ -811,6 +928,14 @@ async def build_autoinject_hint(
         limit=cfg["top_k"],
         threshold=cfg["threshold"],
         project_id=(project_id or None),
+        # LESSONS ARE PROJECT-INDEPENDENT (#3730), so they join this menu's
+        # candidate set from wherever they were learned. Widening it is what
+        # makes the reserved slot below falsifiable rather than decorative: if
+        # the slot were the only path a lesson had, the general contest would
+        # be permanently closed to the kind and "the slot earns its line" would
+        # be true by construction. The switch adds nothing else — it ORs in
+        # `GLOBAL_NOTE_TYPES` and no other kind is in it.
+        include_global_kinds=True,
         # Injection is the one retrieval nobody asked for, so it takes the BROWSE
         # scope: never a record shared one-to-one with the operator. What can
         # still appear is a collaborator's note inside a shared project — legible
@@ -850,6 +975,14 @@ async def build_autoinject_hint(
         user_id, q, kept, cfg, project_id=(project_id or None),
         already=already,
     )
+    # AFTER the reuse slot, because that one evicts the menu's weakest hit
+    # while this one extends: running them the other way round would let a
+    # reserved lesson be the line reuse throws off, and a slot that another
+    # slot can silently undo is not a guarantee.
+    kept, lesson_slot_id = await _reserve_slot_for_lesson(
+        user_id, q, kept, cfg, project_id=(project_id or None),
+        already=already,
+    )
 
     # A collaborator's note can reach this menu via a shared project, and the
     # operator never asked for it — so say whose it is. Unattributed, it reads as
@@ -867,10 +1000,34 @@ async def build_autoinject_hint(
     # once, rather than each repeated line having to explain itself.
     lines = [
         "> Possibly relevant from your Scribe records — open any in full with "
-        "`get_note(id)`, or `get_snippet` / `get_process` for those kinds "
-        "(titles only; a line marked `seen` was surfaced earlier this session "
-        "and may no longer be in context):",
+        "`get_note(id)`, or `get_snippet` / `get_process` / `get_lesson` for "
+        "those kinds (titles only; a line marked `seen` was surfaced earlier "
+        "this session and may no longer be in context):",
     ]
+    # THE REGISTER, SAID ONCE AND ONLY WHEN IT APPLIES (milestone 385 step 5).
+    #
+    # The operator's requirement for this kind was "they don't always have to
+    # be followed", and the risk is not that a reader mistakes a lesson for a
+    # rule — this menu's voice is already the non-binding one, deliberately
+    # (see the `seen` marker below). The risk is the opposite: read as one more
+    # title in a list of MATERIAL, a lesson looks like something to open if
+    # curious, when it is advice someone paid for. The line has to say "weigh
+    # this" without acquiring the rule arms' "before deciding it does not
+    # apply", which binds.
+    #
+    # In the HEADER rather than on each line, for the reason the `seen` flag is
+    # a flag: the meaning is the same for every lesson on the menu, and a
+    # clause repeated per line would cost more than it says. Conditional
+    # because a menu with no lesson should not pay for the sentence, and
+    # because a header that explains an absent kind reads as boilerplate —
+    # which is how a reader learns to skip headers.
+    if any(_record_kind(n) == LESSON_NOTE_TYPE for _s, n in kept):
+        lines.append(
+            "> A line marked `lesson` is something an earlier session learned "
+            "the hard way, kept because it should transfer. Weigh it against "
+            "what you are doing and use your judgement — a lesson is not a "
+            "rule and binds nothing."
+        )
     # A superseded record is DEMOTED, not removed (#278) — so one can still reach
     # this menu, and when it does the reader has to be told. An agent handed
     # stale material with nothing marking it acts on it with full confidence,
@@ -906,9 +1063,18 @@ async def build_autoinject_hint(
     # make this table disagree with `retrieval_logs` about the same call —
     # #3668's identity, which is the cheapest true statement available about
     # this pair of tables and is not worth a marker's convenience.
+    # THE RESERVED LESSON IS NOT THIS ARM'S SURFACING. It was fetched by its own
+    # query and already recorded under `lesson_slot`, so counting it here would
+    # book one delivery twice and leave `lesson_slot`'s two tables describing
+    # different numbers of the same event — #3668's identity, which is the
+    # cheapest true statement available about this pair of tables. It stays in
+    # `note_ids`, which is the session LEDGER and must list every line rendered.
     record_surfaced(
         user_id=user_id,
-        note_ids=[i for i in note_ids if i not in already],
+        note_ids=[
+            i for i in note_ids
+            if i not in already and i != lesson_slot_id
+        ],
         source="auto_inject",
     )
 
@@ -1709,8 +1875,28 @@ async def build_write_path_hint(
             # written and answers nothing; an ISSUE is corrective work with a
             # root cause in it, and a non-task note is durable knowledge. Both
             # earned their place; a todo did not.
-            note_type=("snippet", "note"),
+            #
+            # AND LESSONS (milestone 385 step 5). This arm is kind-FILTERED, so
+            # a kind absent from this tuple is not merely outranked here — it
+            # is unreachable, and nothing reports an arm that never had the
+            # candidate (#3702). The founding example of the kind is a lesson
+            # about a code shape ("two absolutely-positioned siblings"), which
+            # is the moment this arm fires and no other.
+            #
+            # NO RESERVED SLOT HERE, unlike the prompt menu. This arm fires
+            # before EVERY Write and Edit, where a guaranteed extra line is a
+            # guaranteed extra interruption per keystroke-batch — the same
+            # argument that keeps the act arms' budgets tight. And the contest
+            # is already fair: the field is snippets, issues and lessons rather
+            # than the whole corpus, so the 200:1 dilution a slot answers is
+            # not what happens here. Lessons surfaced by this arm are
+            # identifiable in the telemetry by their kind.
+            note_type=("snippet", "note", LESSON_NOTE_TYPE),
             task_kind="issue",
+            # Project-independent, for the reason the prompt menu passes it:
+            # a lesson's claim is that it transfers, and an arm scoped to the
+            # project it was written on cannot test that claim (#3730).
+            include_global_kinds=True,
             # Same reasoning as auto-inject: nobody asked for this, so it takes
             # the browse scope and never surfaces a one-to-one direct share.
             scope="browse",
@@ -1803,6 +1989,13 @@ async def build_write_path_hint(
                     marker,
                     {
                         "id": int(note.id), "title": note.title, "user_id": note.user_id,
+                        # Carried, not re-read off the rendered marker. The
+                        # marker is prose assembled for a human and it already
+                        # varies by kind, language and the `seen` flag — a
+                        # header that decided what to say by matching substrings
+                        # in it would break the next time a marker is reworded,
+                        # silently and in the direction of saying nothing.
+                        "kind": kind,
                         # Carried so the line can disclose a cross-language hit
                         # (#2244). The semantic arm is where these actually arise —
                         # a snippet recorded at the path you're editing is almost
@@ -1924,11 +2117,23 @@ async def build_write_path_hint(
         lines.append(
             f"> Prior art already recorded in Scribe for `{path}` — open one with "
             "`get_snippet(id)` for a snippet, `get_task(id)` for an issue, "
-            "`get_note(id)` otherwise. Reuse a snippet rather than writing a fresh "
-            "one-off; read an issue before repeating what it records "
+            "`get_lesson(id)` for a lesson, `get_note(id)` otherwise. Reuse a "
+            "snippet rather than writing a fresh one-off; read an issue before "
+            "repeating what it records "
             "(titles only; a line marked `seen` was surfaced earlier this "
             "session and may no longer be in context):"
         )
+    # The same clause the prompt menu carries, on the same condition and for
+    # the same reason: this menu's three other kinds are all things that WERE
+    # done here, and a lesson is the one line that is advice. Said once, only
+    # when one is on the menu.
+    if any(i.get("kind") == LESSON_NOTE_TYPE for i, _m, _o, _l in rendered):
+        lines.append(
+            "> A line marked `lesson` is something an earlier session learned "
+            "the hard way, kept because it should transfer. Weigh it against "
+            "the code you are about to write and use your judgement — a lesson "
+            "is not a rule and binds nothing."
+        )
     # Say what a language tag MEANS, and only when one is actually on the menu.
     # Without this the reader has to infer why "· python" is attached to a hit on
     # a .ts file, and the two ways of guessing wrong are both bad: dismiss it as

src/scribe/services/retrieval_surfaces.py

@@ -209,12 +209,20 @@ SURFACES: dict[str, Surface] = {
     ),
 }
 
-# Reserved slots are deliberately absent. `preference_slot` and `reuse_slot`
-# borrow their parent arm's floor and are hard-limited to one hit each, because
-# their entire purpose is to guarantee a single line to a kind of record that
-# keeps losing a general score contest (#2246, #3894). A budget of "1" is the
-# feature; exposing it as tunable would invite setting it to 0 and silently
-# removing the guarantee.
+# Reserved slots are deliberately absent. `preference_slot`, `reuse_slot` and
+# `lesson_slot` borrow their parent arm's floor and are hard-limited to one hit
+# each, because their entire purpose is to guarantee a single line to a kind of
+# record that keeps losing a general score contest (#2246, #3894) — or, for
+# `lesson_slot`, to a kind whose loss is total rather than merely a lost
+# convenience, since a lesson has no act arm to fall back on and nobody browses
+# lessons looking for one (milestone 385). A budget of "1" is the feature;
+# exposing it as tunable would invite setting it to 0 and silently removing the
+# guarantee.
+#
+# They are still logged under their own `source`, which is what keeps them
+# judgeable without being tunable: `best_available_id` names the record each
+# bar refused, so a slot that never places, or one that places weak hits, shows
+# up as evidence rather than as an argument.
 
 
 def surface_names() -> list[str]: