fix(telemetry): a logged query never carries a credential (#3925)

pre_tool_rule retrieves against the RAW COMMAND TEXT and write_path_rule
against the code being written, so whatever was on the command line or in
the buffer is what record_retrieval stored in retrieval_logs.query. A
command that exported a token stored the token.

Storing it was not the worst of it. near_miss_samples is the readout the
threshold docs tell you to open before moving a bar, so the value came back
OUT into an agent's context on the next tuning pass — which is exactly how
this was found, mid-way through #3853's threshold spike.

SCRUBBED ON WRITE, at _build_payload — the single seam every source reaches
the column through. A read-side filter would leave the secret in the table
where a backup or a debug query still reaches it, and a per-caller scrub
would be three places for one to be forgotten by whoever adds the fourth
arm.

REDACTED VISIBLY. `[redacted:<kind>]` rather than a silent deletion: a
reader who cannot tell a scrubbed query from a short one is being lied to
by the readout itself.

DELIBERATELY CONSERVATIVE — vendor-prefixed credentials, values assigned to
secret-NAMED variables, auth headers, PEM blocks. Things that are secrets by
construction. Entropy heuristics and long-opaque-string detection start
eating real queries, and a query is evidence: missing an exotic secret costs
one redaction nobody made, while eating a query costs the ability to tune
the bar at all.

The guard pins BOTH directions, and the second half is the one that matters.
A scrubber that eats evidence fails silently — it keeps looking like it
works while turning the one instrument for tuning a threshold into
unreadable stubs, which is the #2663 shape in a new place. So nine REAL
queries from this install's near-miss samples must survive byte for byte. If
a future pattern touches one, the pattern is too greedy.

Verified against the real shapes before commit: six credential formats
redacted (fabricated values), nine real queries unchanged, and the payload
seam confirmed to store "export API_TOKEN=[redacted:assigned] && git push".

This does NOT scrub rows already written. Purging those is separate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ

src/scribe/services/retrieval_telemetry.py

@@ -17,6 +17,7 @@ from __future__ import annotations
 
 import asyncio
 import logging
+import re
 from typing import Any
 
 from datetime import datetime, timedelta, timezone
@@ -44,6 +45,81 @@ _pending: set[asyncio.Task] = set()
 _reported = False
 
 
+
+# ── secrets never reach the query column (#3925) ───────────────────────
+#
+# `pre_tool_rule` retrieves against the RAW COMMAND TEXT and `write_path_rule`
+# against the code being written, so whatever was on the command line or in the
+# buffer is what gets logged. A command that exports a token therefore stored
+# the token — and worse than stored it: `near_miss_samples` is the readout the
+# threshold docs tell you to open before moving a bar, so the value came back
+# out into an agent's context on the next tuning pass. That is how this was
+# found.
+#
+# SCRUBBED ON WRITE, NOT ON READ. A read-side filter leaves the secret in the
+# table, where a backup, a debug query or a future readout still reaches it.
+# The value must never land.
+#
+# REDACTED VISIBLY, AND THIS IS THE PART THAT KEEPS THE READOUT HONEST. The
+# whole worth of a near-miss sample is reading the query that was actually
+# refused; a scrubber that silently deleted spans would turn the one instrument
+# for tuning a bar into unreadable stubs — the #2663 shape, where a surface
+# looks fine and has quietly stopped saying anything. A `[redacted:<kind>]`
+# marker keeps the sentence readable, keeps its shape and length roughly
+# intact for the ranker's reader, and says plainly that something was removed.
+#
+# DELIBERATELY CONSERVATIVE. These patterns match things that are secrets by
+# CONSTRUCTION — a vendor-prefixed credential, a value assigned to a
+# secret-named variable, an auth header, a PEM header. Anything cleverer
+# (entropy heuristics, long-opaque-string detection) starts eating real
+# queries, and a query is evidence. Missing an exotic secret costs one
+# redaction nobody made; eating a query costs the ability to tune the bar.
+_SECRET_PATTERNS: tuple[tuple[str, "re.Pattern[str]"], ...] = (
+    # Vendor-prefixed credentials. The prefix IS the tell, so no entropy
+    # guessing is needed — `fmcp_` is Scribe's own API key format.
+    ("token", re.compile(
+        r"\b(?:fmcp_|flt_|ghp_|gho_|ghs_|ghu_|github_pat_|glpat-|gitlab-ci-token:"
+        r"|xox[abprs]-|sk-[A-Za-z0-9]*-?|AKIA|ASIA)[A-Za-z0-9_\-]{12,}"
+    )),
+    # A value handed to a secret-NAMED variable, in shell, env files, YAML,
+    # JSON or a query string. The name is what identifies it, so the value can
+    # be anything.
+    ("assigned", re.compile(
+        r"(?i)\b([A-Za-z0-9_]*"
+        r"(?:token|secret|password|passwd|api[_-]?key|access[_-]?key|auth)"
+        r"[A-Za-z0-9_]*)"
+        r"(\s*[:=]\s*[\"']?)"
+        r"([^\s\"'&]{8,})"
+    )),
+    ("auth-header", re.compile(
+        r"(?i)(authorization\s*:\s*(?:bearer|basic|token)\s+)(\S+)"
+    )),
+    ("private-key", re.compile(
+        r"-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----"
+    )),
+)
+
+
+def scrub_secrets(text: str | None) -> str | None:
+    """Redact credential-shaped spans from a query before it is stored.
+
+    Pure and synchronous, so it is unit-testable and safe to run inline on the
+    write path. Returns the input unchanged when nothing matches, which is the
+    overwhelmingly common case and the one the patterns are tuned to protect.
+    """
+    if not text:
+        return text
+    for kind, pattern in _SECRET_PATTERNS:
+        if kind == "assigned":
+            text = pattern.sub(
+                lambda m: f"{m.group(1)}{m.group(2)}[redacted:{kind}]", text)
+        elif kind == "auth-header":
+            text = pattern.sub(lambda m: f"{m.group(1)}[redacted:{kind}]", text)
+        else:
+            text = pattern.sub(f"[redacted:{kind}]", text)
+    return text
+
+
 def _build_payload(
     *,
     user_id: int | None,
@@ -85,7 +161,10 @@ def _build_payload(
     return {
         "user_id": user_id,
         "source": source,
-        "query": query,
+        # Scrubbed HERE rather than at each caller: this is the only path to
+        # the column, and a per-caller scrub is three places for one of them
+        # to be forgotten by whoever adds the fourth arm.
+        "query": scrub_secrets(query),
         "threshold": threshold,
         "limit_n": limit,
         "project_id": project_id,