refactor(notes): a snippet's and lesson's stored title is its name; the trigger joins it only in the embedded document (milestone 427)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / integration (push) Successful in 52s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m35s
CI & Build / Build & push image (push) Successful in 32s

The title was `subject — trigger` because the stored title WAS the
embedded one, and the join is what makes these kinds rank on the
situation they apply to (#2485). Every surface that shows a title then
showed the trigger too -- menus, lists and search rows ran to kilobytes.

- embeddings.document_title(title, note_type, data, body) joins the
  trigger from `data` (body fallback) at embed time. Idempotent: an
  un-migrated composed title comes out the same, never doubled. The
  embed path, the startup backfill and the dedup gate's semantic signal
  all use it, so the embedded text -- and every vector -- is unchanged.
- Writers store the subject: snippet create/update (service, REST, MCP)
  and lesson_document. Both compose_title helpers are removed.
- Readers: dedup takes `data`; the menus strip the embedded title from a
  passage; list rows project `when_to_use`, which SnippetListView reads.
- 0108 rewrites existing rows on an exact `' — ' || <own trigger>`
  suffix with raw SQL, leaving updated_at alone so the backfill does not
  re-embed the corpus for identical vectors. Downgrade recomposes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-23 16:48:28 -04:00
co-authored by Claude Opus 5.5
parent bb632c4196
commit 66e21a6c60
24 changed files with 358 additions and 189 deletions
+57 -6
View File
@@ -214,10 +214,12 @@ TRIGGER_SEP = " — "
def trigger_title(subject: str | None, trigger: str | None) -> str:
"""`{subject}{trigger}` — the title half of a situation-keyed document.
ONE definition, because this join had three. `rule_document` built it for
rules, `snippets.compose_title` for snippets, and milestone 385 needed a
fourth for lessons — the shape #3207 records, where a fix or an improvement
then has to be found in N places by someone who does not know N.
ONE definition, because this join had threerules, snippets, and a
fourth for lessons (milestone 385) — the shape #3207 records, where a fix
or an improvement then has to be found in N places by someone who does not
know N. Since milestone 427 it builds EMBEDDED titles only: `rule_document`
for rules and `document_title` for snippets and lessons. No stored title
carries it.
WHY THE JOIN MATTERS AT ALL, measured in note #2485: the snippet was the
only sharp record in the corpus — a 0.153 top-to-second gap against
@@ -265,6 +267,51 @@ def untrigger_title(title: str | None, trigger: str | None) -> str:
return title
# The `data` key each trigger-keyed note kind mirrors its trigger under. Rules
# are not here: they keep the trigger in a column and `rule_document` builds
# their document from it.
_TRIGGER_DATA_KEYS = {"snippet": "when_to_use", "lesson": "when_to_apply"}
def document_title(
title: str | None, note_type: str | None, data: dict | None = None,
body: str | None = None,
) -> str | None:
"""The title a note is EMBEDDED under — its stored title, plus its trigger.
Milestone 427. A snippet's or lesson's STORED title is its subject alone;
the trigger lives in `data` (decision #4157). It still has to reach the
vector — the `subject — trigger` join is what makes these kinds rank on
the situation they apply to (#2485) — so it is joined HERE, at embed time,
rather than being carried in a title every listing then has to show.
IDEMPOTENT, and that is what makes the migration safe: a title that is
already composed (a row not yet migrated, an old backup restored) is
untriggered first, so it comes out the same and never doubled. The text is
byte-identical to what these kinds were embedded as before, so no vector
moves and the floors tuned against them stay calibrated.
`body` is the fallback when the mirror is missing, read by the kind's own
parser — the same degrade-to-the-body each kind's reader already has.
Every other kind, and a record with no trigger, keeps its title as-is.
"""
key = _TRIGGER_DATA_KEYS.get(note_type or "")
if key is None:
return title
trigger = ((data or {}).get(key) or "").strip() if isinstance(data, dict) else ""
if not trigger and body:
from types import SimpleNamespace
if note_type == "lesson":
from scribe.services.lessons import lesson_trigger
trigger = lesson_trigger(SimpleNamespace(data=None, body=body))
else:
from scribe.services.snippets import parse_snippet_fields
trigger = parse_snippet_fields(title or "", body).get("when_to_use", "")
if not trigger:
return title
return trigger_title(untrigger_title(title, trigger), trigger)
# --- chunking (#280): the document shape ------------------------------------
#
# bge-small reads at most 512 tokens and fastembed silently truncates the rest,
@@ -1081,10 +1128,14 @@ async def backfill_note_embeddings() -> None:
)
success = 0
for note_id in notes_to_embed:
row = await _current_row((Note.user_id, Note.title, Note.body), Note.id, note_id)
row = await _current_row(
(Note.user_id, Note.title, Note.body, Note.note_type, Note.data), Note.id, note_id,
)
if row is None:
continue # deleted between the scan and here
user_id, title, body = row
user_id, title, body, note_type, data = row
# The EMBEDDED title, as the write path builds it (milestone 427).
title = document_title(title, note_type, data, body)
if not chunk_document(title, body):
continue
await upsert_note_embedding(note_id, user_id, title, body)