refactor(notes): a snippet's and lesson's stored title is its name; the trigger joins it only in the embedded document (milestone 427)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / integration (push) Successful in 52s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m35s
CI & Build / Build & push image (push) Successful in 32s

The title was `subject — trigger` because the stored title WAS the
embedded one, and the join is what makes these kinds rank on the
situation they apply to (#2485). Every surface that shows a title then
showed the trigger too -- menus, lists and search rows ran to kilobytes.

- embeddings.document_title(title, note_type, data, body) joins the
  trigger from `data` (body fallback) at embed time. Idempotent: an
  un-migrated composed title comes out the same, never doubled. The
  embed path, the startup backfill and the dedup gate's semantic signal
  all use it, so the embedded text -- and every vector -- is unchanged.
- Writers store the subject: snippet create/update (service, REST, MCP)
  and lesson_document. Both compose_title helpers are removed.
- Readers: dedup takes `data`; the menus strip the embedded title from a
  passage; list rows project `when_to_use`, which SnippetListView reads.
- 0108 rewrites existing rows on an exact `' — ' || <own trigger>`
  suffix with raw SQL, leaving updated_at alone so the backfill does not
  re-embed the corpus for identical vectors. Downgrade recomposes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-23 16:48:28 -04:00
co-authored by Claude Opus 5.5
parent bb632c4196
commit 66e21a6c60
24 changed files with 358 additions and 189 deletions
+28 -44
View File
@@ -26,20 +26,23 @@ while staying a note in every other respect.
WHERE THE TRIGGER LIVES (decision #4157, milestone 385 step 1)
In ``notes.data`` under ``when_to_apply``, written through a named parameter and
mirrored into the title and the head of the body — the shape snippets already
use for ``when_to_use``. Not a column on ``notes``.
mirrored into the head of the body — the shape snippets already use for
``when_to_use``. Not a column on ``notes``. Since milestone 427 it is NOT in the
stored title: the title is the lesson's subject, and the trigger joins it only
in the embedded document (``embeddings.document_title``).
That decision was measured rather than assumed. The whole snippet corpus —
164 of 164 — carries a ``when_to_use`` with **no guard anywhere**, which refutes
the premise that an unenforced field gets skipped. What it does NOT show is that
an agent types a title convention correctly: ``compose_title`` builds the title
an agent types a title convention correctly: the service composed the title
from the parameter, so what is at 100% is a named structured field. A column
would have bought enforceability at the price of deciding, for every note kind
at once, a question nothing had measured.
The mirror is what makes the vector sharp, and it is why nothing re-embeds:
``chunk_document`` is untouched, so ``CHUNKER_VERSION`` does not move. The
trigger reaches the document by being in the text, exactly as a snippet's is.
The trigger in the document is what makes the vector sharp. It reaches it
twice — in the embedded title, joined at embed time, and in the body's first
line — which is the text these kinds were always embedded as, so nothing
re-embeds and ``CHUNKER_VERSION`` does not move.
WHAT A LESSON INHERITS, AND THE CELLS LEFT EMPTY ON PURPOSE (#3163)
@@ -207,36 +210,17 @@ def sole_source(sources: list[int] | None) -> int | None:
return ids[0] if len(ids) == 1 else None
def compose_title(what: str, when_to_apply: str = "") -> str:
"""`{what} — {when it applies}`, the half of the document that ranks.
Built HERE rather than asked of the caller, and that distinction is the
whole evidence base for this design: the snippet corpus is at 100% on its
trigger because a service composes the title from a named parameter, not
because agents type separators reliably. A caller made to spell the
convention is the option milestone 385 step 1 rejected.
The join is `embeddings.trigger_title` — shared with rules and snippets, so
the three kinds that rank on a trigger cannot drift apart in how they say
so.
"""
from scribe.services.embeddings import trigger_title
return trigger_title(what, when_to_apply)
def compose_body(
insight: str, when_to_apply: str = "", learned_from: list[int] | None = None,
) -> str:
"""The lesson body — the trigger line first, the insight after.
The mirror of `compose_title` on the other half of the document, and the
reason the pair is what makes a lesson findable: `chunk_document` joins
them as `{title}\\n{body}`, so a lesson composed here states WHEN IT
APPLIES in the title and again in the first line of the body. That is the
twice-in-a-short-document shape note #2485 measured as the only sharp one
in the corpus, reached the way a snippet reaches it — by being in the text
— rather than by a second document builder at embed time.
The trigger's home in the text of the document, and half of what makes a
lesson findable: `chunk_document` joins `{embedded title}\\n{body}`, and
the embedded title is `what — when it applies` (`embeddings.document_title`,
milestone 427), so the document states WHEN IT APPLIES in the title and
again in the first line of the body. That is the twice-in-a-short-document
shape note #2485 measured as the only sharp one in the corpus.
`**When to apply:**` rather than plain text: the body is the READABLE
form, `data` is the queryable mirror, and `_BODY_TRIGGER_RE` reads this
@@ -275,23 +259,23 @@ def lesson_document(
what: str, when_to_apply: str = "", insight: str = "",
learned_from: list[int] | None = None,
) -> tuple[str, str]:
"""The (title, body) a lesson is STORED — and therefore embedded — as.
"""The (title, body) a lesson is STORED as.
One call so the two halves cannot be composed apart. A lesson whose title
carried the trigger and whose body did not would embed as an ordinary
note wearing a label, and nothing would report it: the record would look
right in every listing and simply never be retrieved at the moment it
applies.
One call so the two halves cannot be composed apart. The body's first line
carries the trigger; a lesson without it (and without the `data` mirror)
would embed as an ordinary note wearing a label, and nothing would report
it: the record would look right in every listing and simply never be
retrieved at the moment it applies.
Deliberately returns what is STORED, not a separate embed-time shape.
Rules need `rule_document` because a rule keeps its trigger in a column
and its title is a plain name, so the sharp document has to be synthesised
for the ranker and exists nowhere else. A lesson follows the snippet
instead — the stored record IS the sharp document — which is why nothing
re-embeds and `CHUNKER_VERSION` does not move.
The title is the SUBJECT alone (milestone 427). It used to carry the
trigger too, so the stored record was itself the sharp document — and every
listing, menu and search row then showed a title that ran to kilobytes.
The trigger now joins the title at embed time (`embeddings.document_title`,
reading `data`), producing the same text as before, so nothing re-embeds
and `CHUNKER_VERSION` does not move.
"""
return (
compose_title(what, when_to_apply),
(what or "").strip(),
compose_body(insight, when_to_apply, learned_from),
)