bvandeusenandClaude Opus 5
1fca8c2808
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 58s
CI & Build / Python tests (push) Successful in 1m38s
CI & Build / Build & push image (push) Successful in 27s
Step 2 of #4251. A System's `description` is a charter — several hundred words saying what belongs in that area and what does not — and it is the answer to "which part of this codebase does X live in". There was no semantic path to one: `list_systems` enumerates, and `search(system_id=…)` uses a System as a FILTER over notes. So a System could narrow a search and could never be the answer to one, and an agent asking where a record belonged had to read every charter or guess. ITS OWN SEARCH, not a `content_type` over notes, for the reason note 3163 gives about milestones: the row could be shared, the search cannot. A charter competing with the whole note corpus for one top-k is outranked by the records filed under it — the right answer crowded out by its own contents — and "where does this belong?" is a different question from "what prior art is there?", which a caller asking one should not have to read past answers to. So `system_embeddings` (0107) joins note_, rule_ and milestone_embeddings as the fourth sibling, with `system_document`, `upsert_system_embedding`, `semantic_search_systems`, a startup backfill and `search(content_type= "system")`. Scoped like milestones: with a project_id, that project's Systems if the caller can read the project (rule 78); without one, the caller's own. Archived Systems are excluded — an archived area is one the operator has said is no longer where things go, which is exactly the question being asked. `system_document` is the plainest of the four shapes on purpose. A charter is already written as the thing this search has to match, in the words someone asking would use — so there is no trigger to synthesise as `rule_document` must, and no second record to gather as `task_document` must. The stored charter IS the sharp document, the way a snippet's is. `color`, `status` and `order_index` stay out: presentation and bookkeeping, and a vector carrying them would be answering a question nobody asks of a charter. The search publishes `report["best_chunk"]` from the start rather than being retrofitted, which is what #4251 asked of any fourth search. It matters more here than anywhere: a charter runs long and a result shows its NAME, so a match on the paragraph that actually decides where a record belongs would otherwise be previewed by two words that cannot say. The id that comes back is the one `system_id`, `system_ids` and `list_system_records` already take, so the answer to "where does this belong?" is directly usable as "show me what is there" and as "file it here". `embed_system` sits beside `notes.embed_note` at the service for #2056's reason — every door gets it by construction. Not called on delete: that is a soft delete and the search joins through `System`, so the vectors are already unreachable, and leaving them means a restore is findable again immediately. `system_embeddings` is declared in backup's `_NOT_INCLUDED` as derived, beside its three siblings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
68 lines
2.4 KiB
Python
68 lines
2.4 KiB
Python
"""system_embeddings — a charter becomes an ANSWER, not just a filter (#4251)
|
|
|
|
Revision ID: 0107
|
|
Revises: 0106
|
|
Create Date: 2026-09-21
|
|
|
|
A System's `description` is a charter: several hundred words saying what
|
|
belongs in that area and what does not. It is the answer to "which part of
|
|
this codebase does X live in", and there was no semantic path to it.
|
|
`list_systems` enumerates and `search(system_id=…)` uses a System as a FILTER
|
|
over notes — so a System could narrow a search and could never be the answer
|
|
to one, and an agent asking where a record belonged had to read every charter
|
|
or guess.
|
|
|
|
The fourth sibling of note_embeddings (0067), rule_embeddings (0089) and
|
|
milestone_embeddings (0102), and for the same reason note 3163 gives about the
|
|
third: the row could be shared, the search cannot. "Where does this belong?"
|
|
is a different question from "what prior art is there?", and no note or rule
|
|
search can answer it, because a charter is not a note.
|
|
|
|
The vectors are DERIVED: nothing is backfilled here, the startup backfill
|
|
writes them.
|
|
"""
|
|
import sqlalchemy as sa
|
|
from alembic import op
|
|
|
|
revision = "0107"
|
|
down_revision = "0106"
|
|
branch_labels = None
|
|
depends_on = None
|
|
|
|
# Matches its three siblings — bge-small-en-v1.5, 384-dim.
|
|
_EMBEDDING_DIM = 384
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.create_table(
|
|
"system_embeddings",
|
|
sa.Column(
|
|
"system_id", sa.Integer(),
|
|
sa.ForeignKey("systems.id", ondelete="CASCADE"), primary_key=True,
|
|
),
|
|
sa.Column("chunk_index", sa.Integer(), primary_key=True),
|
|
sa.Column("chunk_text", sa.Text(), nullable=False),
|
|
sa.Column("chunker_version", sa.Integer(), nullable=False),
|
|
sa.Column(
|
|
"updated_at", sa.DateTime(timezone=True), nullable=False,
|
|
server_default=sa.text("now()"),
|
|
),
|
|
)
|
|
# Raw DDL for the vector column, as 0067, 0089 and 0102 do: the type comes
|
|
# from the pgvector extension, not SQLAlchemy's type system.
|
|
op.execute(
|
|
f"ALTER TABLE system_embeddings ADD COLUMN embedding vector({_EMBEDDING_DIM}) NOT NULL"
|
|
)
|
|
op.execute(
|
|
"""
|
|
CREATE INDEX ix_system_embeddings_embedding_hnsw
|
|
ON system_embeddings
|
|
USING hnsw (embedding vector_cosine_ops)
|
|
"""
|
|
)
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.execute("DROP INDEX IF EXISTS ix_system_embeddings_embedding_hnsw")
|
|
op.drop_table("system_embeddings")
|