feat(write-path): query snippets by concept, not by raw code

Implements #2242, from the operator's question: would it make more sense to
search by the concept of the snippet than by the code itself?

It would, measurably. A snippet's embedded text is f"{title}\n{body}", and a
snippet's body is composed markdown — When to use / Signature / Location, then
the fenced code — so `when_to_use` appears TWICE in the vector and the document
is prose-forward. The semantic arm was interrogating it with raw code carrying
no prose at all. Measured on the deployed instance against snippet #2222:

  query built from            score   best unrelated   separation
  raw code body               0.743   0.630            0.11
  name + docstring            0.823   0.602            0.22
  hand-written concept prose  0.835   0.583            0.25

A 12-word description beats a near-verbatim reimplementation of the function,
and code-as-query RAISES the noise floor. It's also the cleanest explanation for
the fragment miss recorded on #2223: a short excerpt carries almost no prose to
match a document that is mostly prose.

So build the query from what the code says it's FOR — declarations plus the
first docstring / JSDoc / leading comment block — shaped as "name(params) — what
it does", mirroring a snippet's own title, which is the form that measured 0.823.

Server-side rather than in the hook: no manifest bump, so installed 0.1.20
plugins get this immediately; multi-language parsing in bash would be miserable;
and it's unit-testable here.

Two rules worth calling out, both measured rather than chosen:

- NO DOC, NO REWRITE. A bare identifier is not a concept and scored 0.671 vs the
  code body's 0.743. Separation from noise is identical either way (0.113), but
  the absolute drops under the 0.68 bar, so preferring a bare name would convert
  a comfortable hit into a miss. Undocumented code keeps the raw payload.
- The 48-char floor still judges the RAW payload, before the rewrite. A concept
  query is allowed to be shorter than the floor — that is the point, the best
  queries are short — but a sub-floor edit stays silent even with a docstring.
  Applying the floor after extraction would discard the best queries.

Regex, not a parser: this is on a PreToolUse critical path and an Edit's
new_string is rarely a valid module, so a miss must cost only a fallback. Every
unrecognised language (Vue SFC, config files) degrades to exactly the previous
behaviour.

Telemetry now logs the concept query rather than the code, since retrieval_logs
is what the threshold gets tuned from and the two aren't comparable.

0.68 is left alone: signal rises to 0.82 while noise FALLS to 0.58, so the bar
sits mid-gap instead of near the edge. To be re-measured against the deployed
instance rather than assumed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs

tests/test_write_path_trigger.py

@@ -423,6 +423,168 @@ def test_the_floor_stays_below_the_smallest_plausible_helper():
     assert pc.WRITEPATH_MIN_CODE_CHARS > len("".join("x = 1".split()))
 
 
+# --- concept extraction: query by what the code IS FOR, not by the code (#2242) ---
+
+def test_concept_query_prefers_the_name_and_docstring():
+    """Snippet documents are prose-forward — `when_to_use` appears in both the
+    title and the body — so a concept query out-scores the code itself (0.823 vs
+    0.743 measured against #2222). The shape mirrors a snippet's own title."""
+    from scribe.services.plugin_context import concept_query
+    q = concept_query(
+        'def collapse_into_clusters(edges):\n'
+        '    """Collapse similar-pairs into connected components."""\n'
+        '    parent = {}\n'
+        '    return parent\n'
+    )
+    assert q == "collapse_into_clusters(edges) — Collapse similar-pairs into connected components."
+
+
+def test_concept_query_reads_jsdoc_and_the_const_arrow_form():
+    """The dominant shape in this repo's frontend isn't `function foo()`."""
+    from scribe.services.plugin_context import concept_query
+    q = concept_query(
+        "/**\n"
+        " * Rate-limit a callback so it fires once after the last call.\n"
+        " */\n"
+        "export const debounce = (fn: Fn, wait = 250) => {\n"
+        "  let t: number | undefined;\n"
+        "};\n"
+    )
+    assert q == "debounce(fn: Fn, wait = 250) — Rate-limit a callback so it fires once after the last call."
+
+
+def test_concept_query_does_not_let_a_shebang_become_the_description():
+    """`#!` matches the leading-comment pattern but says nothing about the code."""
+    from scribe.services.plugin_context import concept_query
+    q = concept_query(
+        "#!/usr/bin/env bash\n"
+        "# Resolve the repo remote and url-encode it for the plugin API.\n"
+        "encode_remote() {\n"
+        "  git remote get-url origin\n"
+        "}\n"
+    )
+    assert q == "encode_remote() — Resolve the repo remote and url-encode it for the plugin API."
+    assert "usr/bin/env" not in q
+
+
+def test_concept_query_falls_back_when_there_is_no_doc():
+    """MEASURED RULE, not a taste call. An identifier alone scored 0.671 against
+    #2222 where the full code body scored 0.743 — same separation from the noise
+    floor (0.113), but below the 0.68 bar, so preferring the bare name would turn
+    a comfortable hit into a miss. Undocumented code keeps the raw payload."""
+    from scribe.services.plugin_context import concept_query
+    assert concept_query(
+        "def collapse_into_clusters(edges):\n"
+        "    parent = {}\n"
+        "    return parent\n"
+    ) == ""
+
+
+def test_concept_query_falls_back_on_a_doc_that_says_nothing():
+    from scribe.services.plugin_context import concept_query
+    assert concept_query('def f(x):\n    """TODO"""\n    return x\n') == ""
+
+
+def test_concept_query_falls_back_when_nothing_parses():
+    """A Vue SFC has neither a recognised declaration nor a doc block. Every
+    unhandled language must degrade to exactly the pre-#2242 behaviour."""
+    from scribe.services.plugin_context import concept_query
+    assert concept_query(
+        '<script setup lang="ts">\n'
+        "const props = defineProps<{ modelValue: string }>()\n"
+        "</script>\n"
+    ) == ""
+
+
+def test_concept_query_accepts_a_doc_without_a_declaration():
+    """A module docstring is already the concept; it needs no signature."""
+    from scribe.services.plugin_context import concept_query
+    q = concept_query('"""Turn a raw git remote into the project it belongs to."""\nMAPPING = {}\n')
+    assert q == "Turn a raw git remote into the project it belongs to."
+
+
+def test_concept_query_caps_what_it_collects():
+    """A 40-function Write must not become a wall of signatures, and a long module
+    docstring must not drown out the declaration."""
+    from scribe.services import plugin_context as pc
+    many = '"""Utilities."""\n' + "".join(
+        f"def helper_{i}(a, b):\n    return a\n" for i in range(12)
+    )
+    q = pc.concept_query(many)
+    assert q.count("helper_") == pc._CONCEPT_MAX_DECLS
+
+    long_doc = '"""' + ("word " * 400) + '"""\ndef f(a):\n    return a\n'
+    assert len(pc.concept_query(long_doc)) <= pc._CONCEPT_MAX_DOC_CHARS + 64
+
+
+@pytest.mark.asyncio
+async def test_semantic_arm_queries_the_concept_not_the_raw_payload():
+    from scribe.services import plugin_context as pc
+    search = AsyncMock(return_value=[])
+    with patch.object(pc, "get_writepath_config", AsyncMock(return_value=_cfg())), \
+         patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
+         patch.object(pc, "semantic_search_notes", search), \
+         patch.object(pc, "record_retrieval", MagicMock()):
+        await pc.build_write_path_hint(1, "src/x.py", code=REAL_CODE)
+    sent = search.await_args.args[1] if len(search.await_args.args) > 1 else search.await_args.kwargs["query"]
+    assert sent.startswith("debounce(fn, wait=0.25)")
+    assert "Rate-limit a callback so it fires once after the last call." in sent
+    assert "nonlocal timer" not in sent   # the implementation body is gone
+
+
+@pytest.mark.asyncio
+async def test_telemetry_records_the_query_actually_sent():
+    """retrieval_logs is what the threshold gets tuned from, so it has to hold the
+    concept query — logging the raw code would make the scores uninterpretable."""
+    from scribe.services import plugin_context as pc
+    rec = MagicMock()
+    with patch.object(pc, "get_writepath_config", AsyncMock(return_value=_cfg())), \
+         patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
+         patch.object(pc, "semantic_search_notes", AsyncMock(return_value=[])), \
+         patch.object(pc, "record_retrieval", rec):
+        await pc.build_write_path_hint(1, "src/x.py", code=REAL_CODE)
+    logged = rec.call_args.kwargs["query"]
+    assert logged.startswith("debounce(fn, wait=0.25)")
+    assert "nonlocal timer" not in logged
+
+
+@pytest.mark.asyncio
+async def test_the_floor_judges_the_raw_payload_not_the_concept():
+    """ORDER OF OPERATIONS. The floor asks "is this a helper being written?" of the
+    RAW payload; the rewrite happens after. A concept query is allowed to be
+    shorter than the floor — that's the whole point, since the best queries are
+    short — but a sub-floor payload must stay silent even when it has a docstring."""
+    from scribe.services import plugin_context as pc
+
+    # Clears the floor, and its concept is deliberately SHORTER than the floor.
+    verbose = (
+        'def slugify(text):\n'
+        '    """Turn text into a url slug."""\n'
+        '    out = re.sub(r"[^a-z0-9]+", "-", text.lower())\n'
+        '    return out.strip("-")\n'
+    )
+    search = AsyncMock(return_value=[])
+    with patch.object(pc, "get_writepath_config", AsyncMock(return_value=_cfg())), \
+         patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
+         patch.object(pc, "semantic_search_notes", search), \
+         patch.object(pc, "record_retrieval", MagicMock()):
+        await pc.build_write_path_hint(1, "src/x.py", code=verbose)
+    concept = pc.concept_query(verbose)
+    assert len("".join(concept.split())) < pc.WRITEPATH_MIN_CODE_CHARS
+    search.assert_called_once()   # ...and it still searched
+
+    # Under the floor, docstring notwithstanding.
+    tiny = 'def f(x):\n    """Slug it."""\n    return x\n'
+    search2 = AsyncMock(return_value=[])
+    with patch.object(pc, "get_writepath_config", AsyncMock(return_value=_cfg())), \
+         patch.object(pc.snippets_svc, "list_snippets", AsyncMock(return_value=([], 0))), \
+         patch.object(pc, "semantic_search_notes", search2), \
+         patch.object(pc, "record_retrieval", MagicMock()):
+        await pc.build_write_path_hint(1, "src/x.py", code=tiny)
+    assert len("".join(tiny.split())) < pc.WRITEPATH_MIN_CODE_CHARS
+    search2.assert_not_called()
+
+
 # --- the route between them --------------------------------------------------
 
 def test_route_reads_every_arg_the_hook_sends():