KB injection tuning: pgvector substrate + retrieval telemetry + title-first auto-inject #74

Merged

bvandeusen merged 4 commits from dev into main 2026-06-22 20:56:15 -04:00

Conversation 0 Commits 4 Files Changed 22

+1037 -32

bvandeusen commented 2026-06-22 20:56:07 -04:00

Owner

Copy Link

Copy Source

Ships the KB-injection-tuning effort (Scribe project 2, milestone 93) so stored knowledge reaches an agent's context proactively, without context bloat. Principle: push cheap awareness (titles), pull expensive content (bodies).

Step 1 — pgvector substrate (5130197)

• semantic_search_notes now ranks/filters/limits in Postgres via Vector.cosine_distance (<=>) on an HNSW cosine index, replacing the full-table Python cosine scan.
• Migration 0067: CREATE EXTENSION vector, JSONB→vector(384) (stale-dim rows dropped + regenerated by backfill), HNSW index. Reversible.
• Postgres image postgres:16-alpine → pgvector/pgvector:pg17 across prod, quickstart, and CI.

Step 2 — retrieval telemetry (807f478)

• New retrieval_logs table (migration 0068) + services/retrieval_telemetry.record_retrieval() (fire-and-forget), wired into the MCP search tool (source=mcp_search) and REST route (source=rest_search). Captures query, params, and per-result score distribution to tune thresholds from data.

Step 3 — title-first auto-inject, Path A (8126db3)

• GET /api/plugin/retrieve + build_autoinject_hint() with four anti-bloat gates: confidence threshold (stricter than pull), margin gate, per-session dedup (exclude_ids), top-k ceiling. Title-first payload (never bodies); logs source=auto_inject.
• New UserPromptSubmit plugin hook (scribe_autoinject.sh) with per-session dedup state.
• Per-user Settings UI card (enable / threshold / top-k) via /api/settings. Defaults: enabled, 0.55, top-3.

Step 4 — pull-path recall reflex (eec241d)

• Sharpened the recall-before-acting trigger in the plugin static context. Cross-encoder rerank deferred (Scribe task 1038) until auto_inject telemetry shows precision is the bottleneck.

Verification: CI green on dev for the code-bearing commits (incl. the real-PG17 integration lane applying migrations 0067 + 0068 and exercising the pgvector + telemetry paths).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E

Ships the KB-injection-tuning effort (Scribe project 2, milestone 93) so stored knowledge reaches an agent's context proactively, without context bloat. Principle: **push cheap awareness (titles), pull expensive content (bodies).** ### Step 1 — pgvector substrate (`5130197`) - `semantic_search_notes` now ranks/filters/limits in Postgres via `Vector.cosine_distance` (`<=>`) on an HNSW cosine index, replacing the full-table Python cosine scan. - Migration `0067`: `CREATE EXTENSION vector`, JSONB→`vector(384)` (stale-dim rows dropped + regenerated by backfill), HNSW index. Reversible. - Postgres image `postgres:16-alpine` → `pgvector/pgvector:pg17` across prod, quickstart, and CI. ### Step 2 — retrieval telemetry (`807f478`) - New `retrieval_logs` table (migration `0068`) + `services/retrieval_telemetry.record_retrieval()` (fire-and-forget), wired into the MCP search tool (`source=mcp_search`) and REST route (`source=rest_search`). Captures query, params, and per-result score distribution to tune thresholds from data. ### Step 3 — title-first auto-inject, Path A (`8126db3`) - `GET /api/plugin/retrieve` + `build_autoinject_hint()` with four anti-bloat gates: confidence threshold (stricter than pull), margin gate, per-session dedup (`exclude_ids`), top-k ceiling. Title-first payload (never bodies); logs `source=auto_inject`. - New `UserPromptSubmit` plugin hook (`scribe_autoinject.sh`) with per-session dedup state. - Per-user Settings UI card (enable / threshold / top-k) via `/api/settings`. Defaults: enabled, 0.55, top-3. ### Step 4 — pull-path recall reflex (`eec241d`) - Sharpened the recall-before-acting trigger in the plugin static context. Cross-encoder rerank deferred (Scribe task 1038) until `auto_inject` telemetry shows precision is the bottleneck. **Verification:** CI green on `dev` for the code-bearing commits (incl. the real-PG17 integration lane applying migrations 0067 + 0068 and exercising the pgvector + telemetry paths). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E

bvandeusen added 4 commits 2026-06-22 20:56:07 -04:00

feat(search): pgvector substrate — vector(384) + HNSW for semantic search ... 513019786e

Move semantic_search_notes off the full-table Python cosine scan onto a native
pgvector column: indexed ORDER BY embedding <=> :q LIMIT k (HNSW, cosine).
Migration 0067 enables the extension, converts the JSONB embedding column to
vector(384) (stale-dim rows dropped and regenerated by the startup backfill),
and builds the HNSW cosine index. Postgres image moves postgres:16-alpine ->
pgvector/pgvector:pg17 across prod, quickstart, and CI.

Scribe: project 2, milestone 93, task 1031.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E

feat(search): retrieval telemetry — log every semantic retrieval ... 807f478cac

Add retrieval_logs (migration 0068) + services/retrieval_telemetry with a
fire-and-forget record_retrieval(), wired into the MCP search tool
(source=mcp_search) and the REST search route (source=rest_search). Captures
query, effective params, and the per-result score distribution so KB-injection
thresholds can be tuned from data rather than guessed.

Scribe: project 2, milestone 93, task 1032.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E

feat(plugin): knowledge auto-inject (Path A) — title-first per-turn awareness ... 8126db3203

New UserPromptSubmit hook (scribe_autoinject.sh) + GET /api/plugin/retrieve that
surface the TITLES (never bodies) of the few notes clearing four anti-bloat
gates: a per-user confidence threshold (stricter than pull search), a margin
gate, per-session dedup (exclude_ids), and a top-k ceiling. Each retrieval is
logged to retrieval_logs as source=auto_inject so the threshold can be tuned
from data. Per-user config (enable / threshold / top-k) is DB-backed via
/api/settings with a Settings UI card; defaults enabled, threshold 0.55,
top-k 3 (conservative — tune once auto_inject telemetry accrues).

Scribe: project 2, milestone 93, task 1033.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E

feat(plugin): sharpen the recall-before-acting reflex in static context ... eec241d3c0

Turn the SessionStart static guidance into a concrete recall trigger — search
Scribe before answering about the operator projects/people/places/decisions or
starting a task, and pass the active project id to scope results — instead of a
vague "search for related work". Step 4 (pull-path sharpening); the
cross-encoder rerank half is deferred until auto_inject telemetry shows
precision is the bottleneck.

Scribe: project 2, milestone 93, task 1034.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E

bvandeusen merged commit 058e8794af into main 2026-06-22 20:56:15 -04:00

bvandeusen referenced this issue from a commit 2026-06-22 22:07:21 -04:00

fix(plugin): bump version 0.1.9 → 0.1.10 to ship auto-inject hook

bvandeusen referenced this pull request 2026-06-22 22:07:28 -04:00

fix(plugin): bump to 0.1.10 to ship the auto-inject hook #75

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate-bot bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#74