8 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5.5	f4e9cd429b	feat(embeddings): every vector records the model whose space it lives in (#4132) ... The four embedding tables stamped chunker_version but not the model, and vector(384) is a width, not an identity: a same-width model swap would write a second geometry beside the first with no error. - embedding_model on note/rule/milestone/system embeddings (0109; existing rows stamped with the only model any install has ever run). - Every write stamps EMBEDDING_MODEL; every backfill's "current" test is is_current_stamp(), both halves of calibration_stamp(). - migrate_floor refuses while any row its surface searches is off the live model, before sampling: re-embed, then migrate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-23 19:02:30 -04:00
bvandeusenandClaude Opus 5	1fca8c2808	feat(retrieval): a System's charter becomes an answer, not just a filter (#4251) ... Step 2 of #4251. A System's `description` is a charter — several hundred words saying what belongs in that area and what does not — and it is the answer to "which part of this codebase does X live in". There was no semantic path to one: `list_systems` enumerates, and `search(system_id=…)` uses a System as a FILTER over notes. So a System could narrow a search and could never be the answer to one, and an agent asking where a record belonged had to read every charter or guess. ITS OWN SEARCH, not a `content_type` over notes, for the reason note 3163 gives about milestones: the row could be shared, the search cannot. A charter competing with the whole note corpus for one top-k is outranked by the records filed under it — the right answer crowded out by its own contents — and "where does this belong?" is a different question from "what prior art is there?", which a caller asking one should not have to read past answers to. So `system_embeddings` (0107) joins note_, rule_ and milestone_embeddings as the fourth sibling, with `system_document`, `upsert_system_embedding`, `semantic_search_systems`, a startup backfill and `search(content_type= "system")`. Scoped like milestones: with a project_id, that project's Systems if the caller can read the project (rule 78); without one, the caller's own. Archived Systems are excluded — an archived area is one the operator has said is no longer where things go, which is exactly the question being asked. `system_document` is the plainest of the four shapes on purpose. A charter is already written as the thing this search has to match, in the words someone asking would use — so there is no trigger to synthesise as `rule_document` must, and no second record to gather as `task_document` must. The stored charter IS the sharp document, the way a snippet's is. `color`, `status` and `order_index` stay out: presentation and bookkeeping, and a vector carrying them would be answering a question nobody asks of a charter. The search publishes `report["best_chunk"]` from the start rather than being retrofitted, which is what #4251 asked of any fourth search. It matters more here than anywhere: a charter runs long and a result shows its NAME, so a match on the paragraph that actually decides where a record belongs would otherwise be previewed by two words that cannot say. The id that comes back is the one `system_id`, `system_ids` and `list_system_records` already take, so the answer to "where does this belong?" is directly usable as "show me what is there" and as "file it here". `embed_system` sits beside `notes.embed_note` at the service for #2056's reason — every door gets it by construction. Not called on delete: that is a soft delete and the search joins through `System`, so the vectors are already unreachable, and leaving them means a restore is findable again immediately. `system_embeddings` is declared in backup's `_NOT_INCLUDED` as derived, beside its three siblings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 11:25:53 -04:00
bvandeusenandClaude Opus 5	3a501c2cac	feat(search): milestones are searchable by meaning — "is there already a plan for this?" (#4078) ... `search` covered notes, tasks and rules, and a milestone — the record a plan lives in — could not be found. A project whose roadmap was written as milestones had every later plan opened beside the one that already described it, because nothing could have told the session it existed. - milestone_embeddings (migration 0102): the third sibling of note_ and rule_embeddings, for note 3163's reason — the search is milestone-specific. The document is title — description, then description and the plan body, so a roadmap milestone with no description is still found by its design. - Written on create, on a title/description/body update, and for a plan made through start_planning / create_records, fire-and-forget with the parent-row claim (#3262); a startup backfill covers every existing milestone. Derived, so it joins _NOT_INCLUDED beside the other embeddings. - semantic_search_milestones: a project's milestones when the caller can read it (access.can_read_project), otherwise the caller's own; optional status. - search(content_type="milestone"): id, title, description, status, project and progress. Its own shape, and not part of "all", whose results are note-shaped. The docstring says what it is for: ask before start_planning. - Integration test on real Postgres: found in its project and not another, status narrows, an unreadable project returns nothing. Milestone 415 step 3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 13:34:03 -04:00
bvandeusen	454c617ca0	docs(models): the record-splitting rule gets a findable home — note 3163 (#3128 rec 9) ... Spike #3128's fourth question was whether a "when does a record type earn its own table" rule was worth writing down. It turned out to already exist, in full, in the RuleEmbedding docstring — the only written statement of a rule Scribe applies to every record type, sitting where nobody would look for it. Promoted to note 3163, with the three grounds (scoping different in kind, machine-written at volume, edge-or-event-not-document), the worked cases across the whole schema, and the bill: what `rules` had to re-import after leaving `notes`, including the two cells it left empty on purpose. The docstring stays put — it is where the decision was made — and now points at the note.	2026-08-28 13:11:02 -04:00
bvandeusenandClaude Opus 5	95a37318fc	feat(rules): rules become findable by meaning (#3030, milestone 307 step 4) ... Rules were the only major record type with no vector, so `search` could never return one and a rule could arrive only by being preloaded. That single fact is what made every rule compete for one always-on budget. THE DECISION THE TASK ASKED FOR, made explicitly: a sibling rule_embeddings table, not a polymorphic embedding row. The ROW could have been generalised; the SEARCH could not. semantic_search_notes is Note-specific scoping end to end — the visibility clause, the supersession penalty, note_type/task_kind/system filters — and a rule shares none of it, scoping instead by rulebook ownership or project. Generalising the row while still needing two searches is the worst of both: a key with referential integrity to neither table, on the path every session start runs, to share four columns. What is genuinely common is BEHAVIOUR — get_embedding, chunk_document, embedding_text, CHUNKER_VERSION — and those are reused as-is. Sharing them is the DRY win; sharing the table would have been the DRY costume. The document shape is measured, not chosen (note 2485). That pass found the snippet was the only discriminative record in the corpus — a 0.153 top-to-second gap against 0.010-0.023 — and that the cause was its SHAPE: purpose stated twice in a short single-topic document. rule_document reproduces it: the trigger in the title AND as the body's first line. And it excludes `why`, which matters more than any of it. `why` is dated incident narrative — rule 46's runs to 4,300 characters — and long multi-topic prose is exactly what made sixteen dev-logs mutually indistinguishable. Adding it would not give the vector more to work with; it would give every rule the SAME thing to work with. rule_document takes no `why` parameter at all, so a well-meaning caller cannot pass one. A rule with no trigger degrades to title + statement — findable, less sharp. That is an argument for backfilling triggers (step 6), not for padding the document with whatever text is nearby. search(content_type="rule") returns the rule WITH its why and how_to_apply: they are its operational half, the session payload never carries them, and a caller who went looking should not have to re-fetch. Writes re-index fire-and-forget like notes; startup backfills in its own try block so neither backfill can skip the other. rule_embeddings is derived, so it joins note_embeddings in the backup's explicitly-NOT-included list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 15:00:49 -04:00
bvandeusenandClaude Fable 5	0e70a3896b	feat(embeddings): per-chunk rows — schema, write path, version-aware backfill (#280 steps 2+3) ... note_embeddings becomes one row per chunk: PK (note_id, chunk_index), plus chunk_text (what this vector actually encodes) and chunker_version. Migration 0077 clears the table — embeddings are derived (0067 precedent) and the old whole-document rows are indistinguishable from single-chunk notes, so the startup backfill regenerates the corpus at the new shape. The backfill is now version-aware: a future shape change is a CHUNKER_VERSION bump that re-embeds exactly the stale notes, not another wipe. upsert_note_embedding takes (title, body) and chunks internally — one path for the write path, the recurrence spawn and the backfill. The recurrence spawn's own embed call is deleted outright: create_note already embeds via embed_note (#2056), so the spawn was a second copy of the rule. An emptied record now CLEARS its stale vectors instead of leaving them findable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:47:34 -04:00
bvandeusenandClaude Opus 4.8	513019786e	feat(search): pgvector substrate — vector(384) + HNSW for semantic search ... Move semantic_search_notes off the full-table Python cosine scan onto a native pgvector column: indexed ORDER BY embedding <=> :q LIMIT k (HNSW, cosine). Migration 0067 enables the extension, converts the JSONB embedding column to vector(384) (stale-dim rows dropped and regenerated by the startup backfill), and builds the HNSW cosine index. Postgres image moves postgres:16-alpine -> pgvector/pgvector:pg17 across prod, quickstart, and CI. Scribe: project 2, milestone 93, task 1031. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E	2026-06-22 20:10:15 -04:00
bvandeusenandClaude Opus 4.8	b255a0f90e	refactor: rename package fabledassistant -> scribe (code-only) ... Renames src/fabledassistant -> src/scribe and all imports, plus the default DB name and DB user/password (fabled -> scribe) in config + compose. 952 refs / 154 files. Reverses the old 'internal name stays fabledassistant' convention. Code-only: live databases are still physically named 'fabledassistant'. Deployed environments must set POSTGRES_DB / POSTGRES_USER (or rename the DB) since the defaults now resolve to 'scribe'. Repo (FabledScribe), git host (fabledsword), MCP (fabled-git) and the image name (fabledscribe) are intentionally unchanged. ruff check src/ clean locally; CI (typecheck + pytest) is the gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 15:48:35 -04:00