Commit Graph
3 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 57781770c3 feat(write-path): query snippets by concept, not by raw code
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 20s
CI & Build / Python tests (push) Successful in 42s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Build & push image (push) Successful in 26s
Implements #2242, from the operator's question: would it make more sense to
search by the concept of the snippet than by the code itself?

It would, measurably. A snippet's embedded text is f"{title}\n{body}", and a
snippet's body is composed markdown — When to use / Signature / Location, then
the fenced code — so `when_to_use` appears TWICE in the vector and the document
is prose-forward. The semantic arm was interrogating it with raw code carrying
no prose at all. Measured on the deployed instance against snippet #2222:

  query built from            score   best unrelated   separation
  raw code body               0.743   0.630            0.11
  name + docstring            0.823   0.602            0.22
  hand-written concept prose  0.835   0.583            0.25

A 12-word description beats a near-verbatim reimplementation of the function,
and code-as-query RAISES the noise floor. It's also the cleanest explanation for
the fragment miss recorded on #2223: a short excerpt carries almost no prose to
match a document that is mostly prose.

So build the query from what the code says it's FOR — declarations plus the
first docstring / JSDoc / leading comment block — shaped as "name(params) — what
it does", mirroring a snippet's own title, which is the form that measured 0.823.

Server-side rather than in the hook: no manifest bump, so installed 0.1.20
plugins get this immediately; multi-language parsing in bash would be miserable;
and it's unit-testable here.

Two rules worth calling out, both measured rather than chosen:

- NO DOC, NO REWRITE. A bare identifier is not a concept and scored 0.671 vs the
  code body's 0.743. Separation from noise is identical either way (0.113), but
  the absolute drops under the 0.68 bar, so preferring a bare name would convert
  a comfortable hit into a miss. Undocumented code keeps the raw payload.
- The 48-char floor still judges the RAW payload, before the rewrite. A concept
  query is allowed to be shorter than the floor — that is the point, the best
  queries are short — but a sub-floor edit stays silent even with a docstring.
  Applying the floor after extraction would discard the best queries.

Regex, not a parser: this is on a PreToolUse critical path and an Edit's
new_string is rarely a valid module, so a miss must cost only a fallback. Every
unrecognised language (Vue SFC, config files) degrades to exactly the previous
behaviour.

Telemetry now logs the concept query rather than the code, since retrieval_logs
is what the threshold gets tuned from and the two aren't comparable.

0.68 is left alone: signal rises to 0.82 while noise FALLS to 0.58, so the bar
sits mid-gap instead of near the edge. To be re-measured against the deployed
instance rather than assumed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-07-30 09:58:06 -04:00
bvandeusenandClaude Opus 5 1fb883b72f fix(write-path): give the semantic arm its own threshold + a payload floor
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 6s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / integration (push) Successful in 20s
CI & Build / Python tests (push) Successful in 41s
CI & Build / Build & push image (push) Successful in 49s
Closes #2223. The write-path prior-art trigger's semantic arm shared
auto-inject's 0.55 threshold, which was tuned on prose. Code embeddings
sit on a much higher similarity floor — any two Python-shaped payloads
share keywords, indentation and structure — so 0.55 landed INSIDE the
noise band. Measured against the live instance:

  near-duplicate of a recorded helper   0.73-0.74   true positive
  unrelated colour math / Vue SFC / CSS 0.55-0.63   false positive
  `x = 1`                               0.58        false positive

6 of 8 probe payloads produced a nudge; 4 were noise. The margin gate
couldn't help — _AUTOINJECT_BAND is relative to the top hit, so with a
single hit it never engages.

Two gates are now the write-path arm's own:

- kb_writepath_threshold, default 0.68 — above every measured false
  positive, still 0.05 below both true positives. Auto-inject keeps
  0.55; it was tuned on prose and is not implicated. The comment this
  replaces explicitly reserved the split for when telemetry showed the
  surfaces wanted different values, so this is the change it described,
  not a reversal of it.
- WRITEPATH_MIN_CODE_CHARS = 48 non-whitespace chars, below which the
  semantic arm doesn't run at all. Whitespace is excluded so a deeply
  indented one-liner can't pass on padding. 48 sits under the smallest
  plausible reusable helper (~60) and well over a degenerate edit, so it
  errs toward keeping recall — precision is the threshold's job. This is
  the cheap half of the operator's #89 idea; the full length<->threshold
  curve stays open there, since they asked to brainstorm it rather than
  have a scale invented for them.

top_k stays shared — "how many titles at once" means the same thing on
both surfaces.

The existing tests were passing `code="x"` / `code="def f(): ..."` into
the semantic arm, i.e. exactly the payloads the floor now drops, so the
gate tests were never exercising a realistic payload. They now use a
REAL_CODE fixture, plus new coverage for the floor (trivial payload,
padding, place-arm unaffected, real helper passes) and a guard on the
constant itself.

Settings UI carries the new knob with the reasoning in its hint, and the
write-path checkbox no longer claims it shares the threshold.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-07-29 23:26:59 -04:00
bvandeusen e0328f2b1c feat(plugin): write-path trigger — offer prior art before code is rewritten
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 27s
CI & Build / Python tests (push) Successful in 47s
CI & Build / Build & push image (push) Successful in 1m2s
The milestone headline. Auto-inject fires on the operator's prompt; the
moment reuse is actually lost is later, when the agent decides mid-task to
write a helper. A PreToolUse hook on Write|Edit now fires there.

Channel: `additionalContext` with NO permissionDecision, so the note reaches
Claude beside the tool result and the write is never blocked — a recall aid
must not be able to stop the operator's work. Plain stdout would have been
invisible to the model, and deny/ask would have made a nudge into a gate.

Two arms, different in kind:
- BY PLACE — a snippet recorded at this path (or its directory) is prior art
  by definition, not resemblance, so it is neither scored nor thresholded.
  This is what #2083's reverse lookup was built to answer.
- BY MEANING — semantic search restricted to snippets (new `note_type` filter
  on semantic_search_notes) over the code about to be written.
Place ranks first; the top-k cap spans both arms.

Gates carried over from milestone 93 verbatim: threshold, margin, session
dedup, titles-never-bodies. Own `source='write_path'` in retrieval_logs so
precision is tunable separately — the docstring records that the place arm
is unlogged and hands that to #2085.

Its own on/off in Settings but the SAME threshold/top-k: one "how loud may
Scribe be" knob is easier to reason about than two that drift, and splitting
them later is then a data-backed change rather than a guess.

Details worth keeping: the hook sends a REPO-RELATIVE path because that is
how locations are recorded; the git remote resolves to a project and is never
used as the location `repo` filter (different namespaces, would silently
match nothing); the endpoint stays a GET because a read-scoped API key cannot
POST and every other hook depends on that.

plugin.json 0.1.17 -> 0.1.18. Refs #2082, milestone #232.
2026-07-28 09:08:17 -04:00