docs(6206): embedding model docstrings state the sibling-table reason in place
The rule-embedding docstring carries the own-table test itself instead of pointing at a record; the later siblings refer back to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -10,12 +10,12 @@ from scribe.models import Base
|
||||
# native pgvector `vector(384)` (see migration 0067) so similarity is ranked in
|
||||
# Postgres by `<=>` rather than by a full-table Python cosine scan. The semantic
|
||||
# searches rank the CALLER'S in-scope chunks exactly rather than through the
|
||||
# HNSW index, which ranks before it filters (#4961; embeddings._rank_scoped).
|
||||
# HNSW index, which ranks before it filters (see embeddings._rank_scoped).
|
||||
EMBEDDING_DIM = 384
|
||||
|
||||
|
||||
class NoteEmbedding(Base):
|
||||
"""One embedding vector per CHUNK of a note (#280, migration 0077).
|
||||
"""One embedding vector per CHUNK of a note (migration 0077).
|
||||
|
||||
The model reads at most 512 tokens, so a single whole-document vector
|
||||
permanently lost everything past ~400 words. A note now stores one row per
|
||||
@@ -42,8 +42,8 @@ class NoteEmbedding(Base):
|
||||
# any note whose rows carry a stale version — shape changes become a
|
||||
# version bump instead of a table wipe.
|
||||
chunker_version: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
# embeddings.EMBEDDING_MODEL at write time — the SPACE the vector lives in
|
||||
# (#4132). The column is `vector(384)`, a width and not an identity, so a
|
||||
# embeddings.EMBEDDING_MODEL at write time — the SPACE the vector lives in.
|
||||
# The column is `vector(384)`, a width and not an identity, so a
|
||||
# swap to another 384-dim model writes a second geometry beside the first
|
||||
# with no error, and cosine across the two is a number that means nothing.
|
||||
# The version above says what text was embedded; this says in whose
|
||||
@@ -57,14 +57,11 @@ class NoteEmbedding(Base):
|
||||
|
||||
|
||||
class RuleEmbedding(Base):
|
||||
"""One embedding vector per CHUNK of a rule (milestone 307, note 3026).
|
||||
"""One embedding vector per CHUNK of a rule.
|
||||
|
||||
A SIBLING of NoteEmbedding rather than a generalisation of it, decided
|
||||
deliberately. The reasoning below turned out to be the only written
|
||||
statement of a rule Scribe applies to every record type, so it is now also
|
||||
NOTE 3163 — "When a record type earns its own table" — with the worked
|
||||
cases and the cost of leaving `notes`. Read that before splitting a record
|
||||
type off; this stays here because it is where the decision was made.
|
||||
deliberately. The reasoning below is the general test for when a record
|
||||
type earns its own embedding table; the later siblings apply it too.
|
||||
|
||||
- The embedding ROW could have been made polymorphic. The SEARCH could not.
|
||||
`semantic_search_notes` is a long function of Note-specific scoping —
|
||||
@@ -96,8 +93,7 @@ class RuleEmbedding(Base):
|
||||
embedding: Mapped[list] = mapped_column(Vector(EMBEDDING_DIM), nullable=False)
|
||||
# Exactly what this vector encodes — inspectable when a ranking surprises.
|
||||
# For a rule this is the trigger-first document, NOT the rule's `why`:
|
||||
# `why` is dated incident narrative and would drag every rule toward one
|
||||
# centroid (measured in note 2485).
|
||||
# `why` is incident narrative and drags every rule toward one centroid.
|
||||
chunk_text: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
chunker_version: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
embedding_model: Mapped[str] = mapped_column(Text, nullable=False) # see NoteEmbedding
|
||||
@@ -108,15 +104,15 @@ class RuleEmbedding(Base):
|
||||
|
||||
|
||||
class MilestoneEmbedding(Base):
|
||||
"""One embedding vector per CHUNK of a milestone (milestone 415).
|
||||
"""One embedding vector per CHUNK of a milestone.
|
||||
|
||||
The third sibling, for note 3163's reason: the row could be shared, the
|
||||
The third sibling, for RuleEmbedding's reason: the row could be shared, the
|
||||
search cannot. A milestone is scoped by its project, has no share of its
|
||||
own, and is searched to answer one question — "is there already a plan for
|
||||
this?" — which no note or rule search can answer, because a plan is not a
|
||||
note. Before this, a roadmap written as milestones was invisible to recall,
|
||||
and every later plan was opened as a new milestone beside the one that
|
||||
already described it.
|
||||
note. Without it, a roadmap written as milestones is invisible to recall,
|
||||
and a later plan gets opened as a new milestone beside the one that
|
||||
already describes it.
|
||||
|
||||
The document is the title, the one-line description and the plan body, the
|
||||
parts a reader uses to recognise a plan. Derived data: the startup backfill
|
||||
@@ -142,15 +138,15 @@ class MilestoneEmbedding(Base):
|
||||
|
||||
|
||||
class SystemEmbedding(Base):
|
||||
"""One embedding vector per CHUNK of a System's charter (#4251).
|
||||
"""One embedding vector per CHUNK of a System's charter.
|
||||
|
||||
The fourth sibling, for the reason note 3163 gives about the third: the row
|
||||
could be shared, the search cannot. A System's `description` is a charter —
|
||||
The fourth sibling, for the same reason as the others: the row could be
|
||||
shared, the search cannot. A System's `description` is a charter —
|
||||
several hundred words saying what belongs in that area and what does not —
|
||||
and it is the answer to "which part of this codebase does X live in".
|
||||
Before this there was no semantic path to one: `list_systems` enumerates,
|
||||
and `search(system_id=…)` uses a System as a FILTER over notes. So a System
|
||||
could narrow a search and could never be the answer to one.
|
||||
Without it there is no semantic path to one: `list_systems` enumerates,
|
||||
and `search(system_id=…)` uses a System as a FILTER over notes, so a System
|
||||
could narrow a search and never be the answer to one.
|
||||
|
||||
"Where does this belong?" is a different question from "what prior art is
|
||||
there?", which is why this is its own search rather than a note_type: a
|
||||
|
||||
Reference in New Issue
Block a user