feat(write-path): query snippets by concept, not by raw code

Implements #2242, from the operator's question: would it make more sense to
search by the concept of the snippet than by the code itself?

It would, measurably. A snippet's embedded text is f"{title}\n{body}", and a
snippet's body is composed markdown — When to use / Signature / Location, then
the fenced code — so `when_to_use` appears TWICE in the vector and the document
is prose-forward. The semantic arm was interrogating it with raw code carrying
no prose at all. Measured on the deployed instance against snippet #2222:

  query built from            score   best unrelated   separation
  raw code body               0.743   0.630            0.11
  name + docstring            0.823   0.602            0.22
  hand-written concept prose  0.835   0.583            0.25

A 12-word description beats a near-verbatim reimplementation of the function,
and code-as-query RAISES the noise floor. It's also the cleanest explanation for
the fragment miss recorded on #2223: a short excerpt carries almost no prose to
match a document that is mostly prose.

So build the query from what the code says it's FOR — declarations plus the
first docstring / JSDoc / leading comment block — shaped as "name(params) — what
it does", mirroring a snippet's own title, which is the form that measured 0.823.

Server-side rather than in the hook: no manifest bump, so installed 0.1.20
plugins get this immediately; multi-language parsing in bash would be miserable;
and it's unit-testable here.

Two rules worth calling out, both measured rather than chosen:

- NO DOC, NO REWRITE. A bare identifier is not a concept and scored 0.671 vs the
  code body's 0.743. Separation from noise is identical either way (0.113), but
  the absolute drops under the 0.68 bar, so preferring a bare name would convert
  a comfortable hit into a miss. Undocumented code keeps the raw payload.
- The 48-char floor still judges the RAW payload, before the rewrite. A concept
  query is allowed to be shorter than the floor — that is the point, the best
  queries are short — but a sub-floor edit stays silent even with a docstring.
  Applying the floor after extraction would discard the best queries.

Regex, not a parser: this is on a PreToolUse critical path and an Edit's
new_string is rarely a valid module, so a miss must cost only a fallback. Every
unrecognised language (Vue SFC, config files) degrades to exactly the previous
behaviour.

Telemetry now logs the concept query rather than the code, since retrieval_logs
is what the threshold gets tuned from and the two aren't comparable.

0.68 is left alone: signal rises to 0.82 while noise FALLS to 0.58, so the bar
sits mid-gap instead of near the edge. To be re-measured against the deployed
instance rather than assumed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs

src/scribe/services/plugin_context.py

@@ -98,6 +98,69 @@ WRITEPATH_DEFAULT_THRESHOLD = 0.68
 # scale. It also saves a pointless embedding round-trip on trivial edits.
 WRITEPATH_MIN_CODE_CHARS = 48
 
+# --- concept extraction for the semantic arm's query (#2242) ------------------
+# A snippet's embedded text is f"{title}\n{body}", and for a snippet that body is
+# composed markdown: **When to use:**, **Signature:**, **Location:**, then the
+# fenced code. So `when_to_use` — the description of what the thing is FOR —
+# appears twice in the vector, and the document is prose-forward.
+#
+# The arm used to query it with raw code and no prose at all. Measured on the
+# deployed instance against snippet #2222, same corpus:
+#   query built from            score   best unrelated   separation
+#   raw code body               0.743   0.630            0.11
+#   name + docstring            0.823   0.602            0.22
+#   hand-written concept prose  0.835   0.583            0.25
+# A 12-word description beats a near-verbatim reimplementation of the function,
+# and code-as-query RAISES the noise floor. It is also the cleanest explanation
+# for the fragment miss recorded on #2223: a short code excerpt has almost no
+# prose to match against a document that is mostly prose.
+#
+# So we send the concept instead — and shape it like a snippet's own title,
+# "{name} — {when_to_use}", because that is the form the 0.823 measurement used.
+# Undocumented code yields little, and a Vue SFC or a config file yields nothing;
+# those fall back to the raw payload and behave exactly as before. This raises
+# the ceiling for documented helpers rather than fixing every case.
+
+# Declaration forms, one pattern per shape, every pattern exposing (name, params)
+# so composition doesn't have to care which matched. Deliberately regex and not a
+# real parser: this runs on a PreToolUse hook's critical path, the payload is
+# frequently a FRAGMENT that no parser would accept (an Edit's new_string is
+# rarely a valid module), and a miss costs only a fallback to today's behaviour.
+_CONCEPT_DECL_PATTERNS = (
+    # python: def / async def, and class with optional bases
+    re.compile(r"^[ \t]*(?:async[ \t]+)?def[ \t]+([A-Za-z_]\w*)[ \t]*(\([^)]*\))", re.M),
+    re.compile(r"^[ \t]*class[ \t]+([A-Za-z_]\w*)[ \t]*(\([^)]*\))?", re.M),
+    # js/ts: function decl, and the const-arrow form that dominates modern code
+    re.compile(r"^[ \t]*(?:export[ \t]+)?(?:default[ \t]+)?(?:async[ \t]+)?function[ \t]+([A-Za-z_$][\w$]*)[ \t]*(\([^)]*\))", re.M),
+    re.compile(r"^[ \t]*(?:export[ \t]+)?(?:const|let|var)[ \t]+([A-Za-z_$][\w$]*)[ \t]*=[ \t]*(?:async[ \t]*)?(\([^)]*\))[ \t]*=>", re.M),
+    # rust / go
+    re.compile(r"^[ \t]*(?:pub[ \t]+)?fn[ \t]+([A-Za-z_]\w*)[ \t]*(\([^)]*\))", re.M),
+    re.compile(r"^[ \t]*func[ \t]+(?:\([^)]*\)[ \t]*)?([A-Za-z_]\w*)[ \t]*(\([^)]*\))", re.M),
+    # posix shell: name() {
+    re.compile(r"^[ \t]*([A-Za-z_]\w*)[ \t]*(\(\))[ \t]*\{", re.M),
+)
+
+# Doc forms, tried in order. The Python pattern also matches a triple-quoted
+# string that isn't a docstring — accepted: a stray literal is still text about
+# what the code does far more often than it's misleading, and the cost is a
+# slightly worse query rather than a wrong answer.
+_CONCEPT_PY_DOC = re.compile(r'("""|\'\'\')(.*?)\1', re.S)
+_CONCEPT_JSDOC = re.compile(r"/\*\*(.*?)\*/", re.S)
+_CONCEPT_LEADING_COMMENT = re.compile(r"\A(?:[ \t]*(?://|#)[^\n]*\n?)+")
+# A shebang is a comment to the regex above but says nothing about what the code
+# DOES, and it would otherwise open the doc with "/usr/bin/env bash".
+_CONCEPT_SHEBANG = re.compile(r"\A#![^\n]*\n")
+_CONCEPT_COMMENT_MARKER = re.compile(r"^[ \t]*(?://+|#+!?)[ \t]?", re.M)
+_CONCEPT_JSDOC_STAR = re.compile(r"^[ \t]*\*+[ \t]?", re.M)
+
+# Cap the doc so a long module docstring can't drown out the declaration, and cap
+# declarations so a 40-function Write doesn't turn into a wall of signatures.
+_CONCEPT_MAX_DOC_CHARS = 400
+_CONCEPT_MAX_DECLS = 4
+# Below this much substance the "concept" is too thin to be a better query than
+# the code itself (e.g. all we found was `f()`), so we keep the raw payload.
+_CONCEPT_MIN_CHARS = 16
+
 # Margin gate: drop any hit more than this far below the top hit's score, so a
 # single strong match doesn't drag in a wall of barely-passing neighbours.
 _AUTOINJECT_BAND = 0.10
@@ -337,6 +400,77 @@ def _prior_art_line(item: dict, marker: str, owner: str | None) -> str:
     return line
 
 
+def _concept_doc(code: str) -> str:
+    """The first doc-ish prose in `code`: docstring, else JSDoc, else leading comments."""
+    m = _CONCEPT_PY_DOC.search(code)
+    if m:
+        return _collapse(m.group(2))
+
+    m = _CONCEPT_JSDOC.search(code)
+    if m:
+        return _collapse(_CONCEPT_JSDOC_STAR.sub("", m.group(1)))
+
+    # Only a comment block at the very TOP counts. A comment further down is
+    # usually about one line of the implementation, not about the whole thing.
+    m = _CONCEPT_LEADING_COMMENT.match(_CONCEPT_SHEBANG.sub("", code))
+    if m:
+        return _collapse(_CONCEPT_COMMENT_MARKER.sub("", m.group(0)))
+
+    return ""
+
+
+def _collapse(text: str) -> str:
+    """One line, single-spaced, length-capped — embedder input, not display text."""
+    return " ".join((text or "").split())[:_CONCEPT_MAX_DOC_CHARS].strip()
+
+
+def concept_query(code: str) -> str:
+    """Rewrite a write payload as a CONCEPT query, or "" to keep the raw payload.
+
+    Returns something shaped like a snippet's own title — "name(params) — what it
+    does" — because that is the form that measured best against the prose-forward
+    snippet documents (#2242; see the table at _CONCEPT_DECL_PATTERNS).
+
+    Returns "" rather than raising or guessing whenever there's nothing worth
+    sending: no declarations and no doc, or a result too thin to beat the code it
+    would replace. The caller treats "" as "use the payload as-is", so every
+    unhandled language degrades to exactly the previous behaviour.
+    """
+    if not code or not code.strip():
+        return ""
+
+    decls: list[str] = []
+    for pattern in _CONCEPT_DECL_PATTERNS:
+        for match in pattern.finditer(code):
+            name, params = match.group(1), match.group(2) or ""
+            label = f"{name}{params}".strip()
+            if label and label not in decls:
+                decls.append(label)
+            if len(decls) >= _CONCEPT_MAX_DECLS:
+                break
+        if len(decls) >= _CONCEPT_MAX_DECLS:
+            break
+
+    doc = _concept_doc(code)
+
+    # NO DOC, NO REWRITE. An identifier alone is not a concept, and it measured
+    # WORSE than the code it would replace: `collapse_into_clusters(edges)` scored
+    # 0.671 against #2222 where the full code body scored 0.743. Separation from
+    # the noise floor is identical (0.113 either way), but the absolute value
+    # drops below the 0.68 bar — so preferring a bare name would convert a
+    # comfortable hit into a miss. Undocumented code keeps the raw payload.
+    if not doc:
+        return ""
+
+    head = ", ".join(decls)
+    query = f"{head} — {doc}" if head else doc
+
+    # Guard against a doc so terse it says nothing ("# TODO", "/** x */").
+    if len("".join(query.split())) < _CONCEPT_MIN_CHARS:
+        return ""
+    return query
+
+
 async def get_writepath_config(user_id: int) -> dict:
     """Write-path trigger settings: its own `enabled` and `threshold`, auto-inject's top_k.
 
@@ -451,6 +585,14 @@ async def build_write_path_hint(
     # let a deeply-nested one-liner through on padding alone.
     if len("".join(query.split())) < WRITEPATH_MIN_CODE_CHARS:
         query = ""
+    # ORDER MATTERS: the floor above judges the RAW payload, this rewrites it.
+    # Snippet documents are prose-forward, so a concept query out-scores the code
+    # itself by a wide margin (#2242). The rewritten query is allowed to be
+    # short — "slugify(t) — turn text into a url slug" is a fine query at 38
+    # chars, and it only exists because the raw payload already cleared the
+    # floor. Applying the floor after this would throw away the best queries.
+    if query:
+        query = concept_query(query) or query
     if remaining > 0 and query:
         t0 = time.perf_counter()
         hits = await semantic_search_notes(