feat(rules): rules become findable by meaning (#3030, milestone 307 step 4)
CI & Build / Python lint (push) Failing after 9s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 44s
CI & Build / integration (push) Successful in 45s
CI & Build / Python tests (push) Successful in 1m26s
CI & Build / Build & push image (push) Skipped

Rules were the only major record type with no vector, so `search` could never
return one and a rule could arrive only by being preloaded. That single fact is
what made every rule compete for one always-on budget.

THE DECISION THE TASK ASKED FOR, made explicitly: a sibling rule_embeddings
table, not a polymorphic embedding row. The ROW could have been generalised;
the SEARCH could not. semantic_search_notes is Note-specific scoping end to end
— the visibility clause, the supersession penalty, note_type/task_kind/system
filters — and a rule shares none of it, scoping instead by rulebook ownership
or project. Generalising the row while still needing two searches is the worst
of both: a key with referential integrity to neither table, on the path every
session start runs, to share four columns. What is genuinely common is
BEHAVIOUR — get_embedding, chunk_document, embedding_text, CHUNKER_VERSION —
and those are reused as-is. Sharing them is the DRY win; sharing the table
would have been the DRY costume.

The document shape is measured, not chosen (note 2485). That pass found the
snippet was the only discriminative record in the corpus — a 0.153
top-to-second gap against 0.010-0.023 — and that the cause was its SHAPE:
purpose stated twice in a short single-topic document. rule_document
reproduces it: the trigger in the title AND as the body's first line.

And it excludes `why`, which matters more than any of it. `why` is dated
incident narrative — rule 46's runs to 4,300 characters — and long multi-topic
prose is exactly what made sixteen dev-logs mutually indistinguishable. Adding
it would not give the vector more to work with; it would give every rule the
SAME thing to work with. rule_document takes no `why` parameter at all, so a
well-meaning caller cannot pass one.

A rule with no trigger degrades to title + statement — findable, less sharp.
That is an argument for backfilling triggers (step 6), not for padding the
document with whatever text is nearby.

search(content_type="rule") returns the rule WITH its why and how_to_apply:
they are its operational half, the session payload never carries them, and a
caller who went looking should not have to re-fetch. Writes re-index
fire-and-forget like notes; startup backfills in its own try block so neither
backfill can skip the other. rule_embeddings is derived, so it joins
note_embeddings in the backup's explicitly-NOT-included list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-26 15:00:49 -04:00
co-authored by Claude Opus 5
parent 682bea5257
commit 95a37318fc
9 changed files with 455 additions and 8 deletions
+44 -2
View File
@@ -11,10 +11,44 @@ import time
from scribe.mcp._context import current_user_id
from scribe.services.access import owner_names_for
from scribe.services.embeddings import DEFAULT_SIMILARITY_THRESHOLD, semantic_search_notes
from scribe.services.embeddings import (
DEFAULT_SIMILARITY_THRESHOLD, semantic_search_notes, semantic_search_rules,
)
from scribe.services.retrieval_telemetry import record_retrieval, retrieval_summary
async def _search_rules(uid: int, q: str, limit: int) -> dict:
"""Rules by meaning — a separate result shape because a rule IS different.
A rule hit carries `why` and `how_to_apply`: they are the operational half
of a rule and the session-start payload never includes them, so a caller
who went looking should get the whole thing rather than a summary they then
have to re-fetch.
Rules are not project-scoped the way notes are (a family rule belongs to no
project), so `project_id` and `system_id` do not apply here.
"""
raw = await semantic_search_rules(uid, q, limit=limit)
return {
"results": [
{
"id": rule.id,
"title": rule.title,
"statement": rule.statement,
"when_to_apply": rule.when_to_apply or "",
"tier": rule.tier,
"why": rule.why or "",
"how_to_apply": rule.how_to_apply or "",
"topic_id": rule.topic_id,
"project_id": rule.project_id,
"similarity": float(score),
}
for score, rule in raw
],
"total": len(raw),
}
async def search(
q: str,
content_type: str = "all",
@@ -33,7 +67,13 @@ async def search(
Args:
q: search query string.
content_type: 'all' (default), 'note' (notes only), or 'task' (tasks only).
content_type: 'all' (default), 'note' (notes only), 'task' (tasks
only), or 'rule' (RULES only — the operator's standing
instructions, searchable by meaning since milestone 307).
Reach for 'rule' when you want to know whether a standing
instruction covers something: "is there a rule about release
tagging?". A hit carries the rule's `why` and `how_to_apply`,
which the session-start payload does not.
limit: maximum number of results (1-50).
project_id: Scope results to one project. PASS THE ACTIVE PROJECT'S ID
whenever a project is in scope (the one you entered with
@@ -56,6 +96,8 @@ async def search(
"""
uid = current_user_id()
limit = max(1, min(limit, 50))
if content_type == "rule":
return await _search_rules(uid, q, limit)
is_task = {"note": False, "task": True}.get(content_type) # None => any
t0 = time.perf_counter()
raw = await semantic_search_notes(