fix(search): show the passage that matched, not the opening of the body (#4243)
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m5s
CI & Build / Build & push image (push) Skipped
CI & Build / integration (push) Successful in 46s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m5s
CI & Build / Build & push image (push) Skipped
CI & Build / integration (push) Successful in 46s
Raised by the operator: are we limiting what comes back by character count,
and how do we verify the pertinent part is the part displayed?
We were not. mcp/tools/search.py sent (note.body or "")[:240] — a head cut,
with no marker that anything had been removed, so a 240-character preview of
a 4000-character record was indistinguishable from a complete short one.
The opening is the wrong span. The match is semantic and per chunk, and
semantic_search_notes collapses to best-chunk-per-note — its own comment at
the collapse says "the first appearance of a note is its best chunk". So the
system identified the passage that earned the hit and then discarded it:
select(Note, distance) kept no chunk column. A record could rank first on its
sixth paragraph, be previewed by its first, and be judged irrelevant on a
span the search had already scored lower. That biases against long records,
and it is self-concealing — the caller who does not open it never learns the
preview was misleading.
- embeddings: chunk_index/chunk_text ride along in the select, and the
collapse records the winner in report["best_chunk"]. Carried in `report`,
NOT by widening the return tuple: ten callers unpack (score, note) at
~18 sites and nothing would catch the misses (lesson #4207). `report` is
the side-channel this function already uses for best_available_score.
- search(): excerpt / excerpt_is / body_length, and read_full when there is
more. A caller that cannot tell a matched passage from a document opening
cannot judge whether to look deeper, which is the only decision the field
supports.
elide() moves to services/text.py so both callers share one copy, and it
keeps BOTH ends with a stated gap — it is the fallback for when nothing
identifies a better span than "all of it", not the goal.
Also fixes a guard that produced a false failure on the previous commit:
test_pull_telemetry checked `"project_id: int = 0" in body.split("\n")[0]`,
which sees only the first line, so wrapping get_task's signature over four
lines made it report a function that does take the project as one that does
not. Parsed with ast now, and proven to still reject an absent or
wrongly-typed parameter rather than being appeased by reflowing the code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
@@ -45,6 +45,7 @@ from scribe.services import task_logs as task_logs_svc
|
||||
from scribe.services import trash as trash_svc
|
||||
from scribe.services.note_usage import record_pulled
|
||||
from scribe.services.record_refs import refuse_guessed_ids
|
||||
from scribe.services.text import elide
|
||||
|
||||
|
||||
# A work log entry is prose, often long — the discipline asks for what was
|
||||
@@ -64,31 +65,6 @@ _WORK_LOG_ADVICE = (
|
||||
)
|
||||
|
||||
|
||||
def elide(text: str, budget: int) -> tuple[str, bool]:
|
||||
"""Cut to `budget` characters from the MIDDLE, keeping both ends.
|
||||
|
||||
A head-only cut — `text[:800]` — decides what a reader sees by character
|
||||
position, which is uncorrelated with what matters. Prose does not put its
|
||||
conclusion first: a log entry that opens with what was tried and closes
|
||||
with "so this shipped in 04775c3" loses exactly the sentence that answers
|
||||
the question, and the reader cannot tell, because a truncation marker says
|
||||
that something was removed and never whether it mattered.
|
||||
|
||||
So keep the opening (what this entry is about) AND the closing (where it
|
||||
landed), and say in between how much went. Two thirds to the head because
|
||||
that is where the subject is established; the tail needs less to carry a
|
||||
conclusion. `budget` of 0 means no cut.
|
||||
"""
|
||||
if budget <= 0 or len(text) <= budget:
|
||||
return text, False
|
||||
head_len = max(1, budget * 2 // 3)
|
||||
tail_len = max(1, budget - head_len)
|
||||
omitted = len(text) - head_len - tail_len
|
||||
head = text[:head_len].rstrip()
|
||||
tail = text[-tail_len:].lstrip()
|
||||
return f"{head}\n\n[… {omitted} characters omitted …]\n\n{tail}", True
|
||||
|
||||
|
||||
def work_log_payload(
|
||||
logs: list, total: int, chars: int, latest_chars: int = _WORK_LOG_LATEST_CHARS
|
||||
) -> dict:
|
||||
|
||||
Reference in New Issue
Block a user