feat(embeddings): every vector records the model whose space it lives in (#4132)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m1s
CI & Build / Python tests (push) Successful in 1m40s
CI & Build / Build & push image (push) Canceled after 27s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m1s
CI & Build / Python tests (push) Successful in 1m40s
CI & Build / Build & push image (push) Canceled after 27s
The four embedding tables stamped chunker_version but not the model, and vector(384) is a width, not an identity: a same-width model swap would write a second geometry beside the first with no error. - embedding_model on note/rule/milestone/system embeddings (0109; existing rows stamped with the only model any install has ever run). - Every write stamps EMBEDDING_MODEL; every backfill's "current" test is is_current_stamp(), both halves of calibration_stamp(). - migrate_floor refuses while any row its surface searches is off the live model, before sampling: re-embed, then migrate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -41,6 +41,14 @@ class NoteEmbedding(Base):
|
||||
# any note whose rows carry a stale version — shape changes become a
|
||||
# version bump instead of a table wipe.
|
||||
chunker_version: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
# embeddings.EMBEDDING_MODEL at write time — the SPACE the vector lives in
|
||||
# (#4132). The column is `vector(384)`, a width and not an identity, so a
|
||||
# swap to another 384-dim model writes a second geometry beside the first
|
||||
# with no error, and cosine across the two is a number that means nothing.
|
||||
# The version above says what text was embedded; this says in whose
|
||||
# geometry. Either one moving makes the row stale, and the backfill
|
||||
# re-embeds on both.
|
||||
embedding_model: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
default=lambda: datetime.now(timezone.utc),
|
||||
@@ -91,6 +99,7 @@ class RuleEmbedding(Base):
|
||||
# centroid (measured in note 2485).
|
||||
chunk_text: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
chunker_version: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
embedding_model: Mapped[str] = mapped_column(Text, nullable=False) # see NoteEmbedding
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
default=lambda: datetime.now(timezone.utc),
|
||||
@@ -124,6 +133,7 @@ class MilestoneEmbedding(Base):
|
||||
embedding: Mapped[list] = mapped_column(Vector(EMBEDDING_DIM), nullable=False)
|
||||
chunk_text: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
chunker_version: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
embedding_model: Mapped[str] = mapped_column(Text, nullable=False) # see NoteEmbedding
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
default=lambda: datetime.now(timezone.utc),
|
||||
@@ -163,6 +173,7 @@ class SystemEmbedding(Base):
|
||||
embedding: Mapped[list] = mapped_column(Vector(EMBEDDING_DIM), nullable=False)
|
||||
chunk_text: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
chunker_version: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
embedding_model: Mapped[str] = mapped_column(Text, nullable=False) # see NoteEmbedding
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
default=lambda: datetime.now(timezone.utc),
|
||||
|
||||
Reference in New Issue
Block a user