KB injection tuning: pgvector substrate + retrieval telemetry + title-first auto-inject #74

Merged
bvandeusen merged 4 commits from dev into main 2026-06-22 20:56:15 -04:00

4 Commits

Author SHA1 Message Date
bvandeusen eec241d3c0 feat(plugin): sharpen the recall-before-acting reflex in static context
Turn the SessionStart static guidance into a concrete recall trigger — search
Scribe before answering about the operator projects/people/places/decisions or
starting a task, and pass the active project id to scope results — instead of a
vague "search for related work". Step 4 (pull-path sharpening); the
cross-encoder rerank half is deferred until auto_inject telemetry shows
precision is the bottleneck.

Scribe: project 2, milestone 93, task 1034.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E
2026-06-22 20:39:54 -04:00
bvandeusen 8126db3203 feat(plugin): knowledge auto-inject (Path A) — title-first per-turn awareness
CI & Build / Python lint (push) Successful in 2s
CI & Build / integration (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 21s
CI & Build / Python tests (push) Successful in 43s
CI & Build / Build & push image (push) Successful in 55s
New UserPromptSubmit hook (scribe_autoinject.sh) + GET /api/plugin/retrieve that
surface the TITLES (never bodies) of the few notes clearing four anti-bloat
gates: a per-user confidence threshold (stricter than pull search), a margin
gate, per-session dedup (exclude_ids), and a top-k ceiling. Each retrieval is
logged to retrieval_logs as source=auto_inject so the threshold can be tuned
from data. Per-user config (enable / threshold / top-k) is DB-backed via
/api/settings with a Settings UI card; defaults enabled, threshold 0.55,
top-k 3 (conservative — tune once auto_inject telemetry accrues).

Scribe: project 2, milestone 93, task 1033.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E
2026-06-22 20:31:07 -04:00
bvandeusen 807f478cac feat(search): retrieval telemetry — log every semantic retrieval
CI & Build / Python lint (push) Successful in 2s
CI & Build / integration (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 44s
CI & Build / Build & push image (push) Successful in 58s
Add retrieval_logs (migration 0068) + services/retrieval_telemetry with a
fire-and-forget record_retrieval(), wired into the MCP search tool
(source=mcp_search) and the REST search route (source=rest_search). Captures
query, effective params, and the per-result score distribution so KB-injection
thresholds can be tuned from data rather than guessed.

Scribe: project 2, milestone 93, task 1032.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E
2026-06-22 20:10:15 -04:00
bvandeusen 513019786e feat(search): pgvector substrate — vector(384) + HNSW for semantic search
Move semantic_search_notes off the full-table Python cosine scan onto a native
pgvector column: indexed ORDER BY embedding <=> :q LIMIT k (HNSW, cosine).
Migration 0067 enables the extension, converts the JSONB embedding column to
vector(384) (stale-dim rows dropped and regenerated by the startup backfill),
and builds the HNSW cosine index. Postgres image moves postgres:16-alpine ->
pgvector/pgvector:pg17 across prod, quickstart, and CI.

Scribe: project 2, milestone 93, task 1031.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E
2026-06-22 20:10:15 -04:00