fix(telemetry): record a pull on get_task, closing the surfaced→pulled loop
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / TypeScript typecheck (push) Successful in 10s
CI & Build / integration (push) Successful in 12s
CI & Build / Python tests (push) Successful in 43s
CI & Build / Build & push image (push) Successful in 39s

Closes #2245. note_usage_events recorded `surfaced` for every auto-inject menu
line regardless of kind, but `pulled` only from get_note, get_snippet and the
REST snippet route. get_task recorded nothing.

Auto-inject ranks kind-blind over a corpus that is overwhelmingly tasks and
issues, so tasks are most of what it surfaces. Measured live, "write a function
to debounce a callback in the frontend" returned three tasks and zero snippets —
all three written as surfaced, none able to record a pull.

surfaced and pulled only mean anything as a PAIR; the rate between them is what
#1038 and #2085 gate on. So the gap sat exactly where the volume is, and the
metric would have said "auto-inject surfaces things nobody opens" for its own
dominant kind — an artifact of the instrumentation, not a fact about the feature,
and one that pointed at a plausible-sounding wrong conclusion.

get_note already carried a comment stating this was meant to cover ANY note kind
precisely so tasks wouldn't look like dead weight. get_task is a separate tool in
a separate module and never got the call — sibling drift, invisible because a
missing side effect changes no return value.

Guarded by a rule-#33 contract test that asserts, by source inspection, that
every getter reachable from an auto-inject menu calls record_pulled. Source
inspection because no behavioural test can see a call that isn't there.

Not fixed here: the REST note/task detail routes still record nothing while the
REST snippet route records `rest_snippet`. That asymmetry is real, but a human
reading a note in a browser is arguably not the same event as an agent recalling
one, and collapsing them could skew the signal the other way. Raised as a
question for the retrieval survey instead of decided in passing.

Pre-fix rows under-count task pulls, one-sidedly by kind — treat them as unknown
rather than zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
2026-07-30 10:35:23 -04:00
co-authored by Claude Opus 5
parent 390846a3d5
commit 6ca215d2b6
2 changed files with 43 additions and 0 deletions
+9
View File
@@ -27,6 +27,7 @@ from scribe.services import rulebooks as rulebooks_svc
from scribe.services import systems as systems_svc
from scribe.services import task_logs as task_logs_svc
from scribe.services import trash as trash_svc
from scribe.services.note_usage import record_pulled
async def list_tasks(
@@ -99,6 +100,14 @@ async def get_task(task_id: int) -> dict:
data["suppressed_rules"] = applicable.get("suppressed_rules", [])
data["suppressed_topics"] = applicable.get("suppressed_topics", [])
data.update(await access_svc.describe_provenance(uid, note))
# Same reasoning as get_note's record_pulled, and this is the tool where it
# matters MOST: auto-inject ranks kind-blind over a corpus that is
# overwhelmingly tasks and issues, so tasks dominate what it surfaces. Without
# this the surfaced→pulled loop was open exactly where the volume is — every
# surfaced task counted as never-pulled because the tool that opens one didn't
# say so, driving auto-inject's measured pull-through toward zero for its own
# dominant kind. #1038 and #2085 are explicitly gated on that number (#2245).
record_pulled(user_id=uid, note_id=int(note.id), source="mcp_get_task")
return data
+34
View File
@@ -186,3 +186,37 @@ async def test_auto_inject_records_what_survived_the_margin_gate():
assert surfaced.call_args.kwargs["note_ids"] == [1]
assert surfaced.call_args.kwargs["source"] == "auto_inject"
def test_every_getter_that_can_be_surfaced_also_records_a_pull():
"""Rule #33 contract check — and the one that would have caught #2245.
`surfaced` and `pulled` only mean something as a PAIR: the rate between them
is what #1038 and #2085 gate on. That pair is only closed if the tool which
OPENS a record reports it. `get_note` did, `get_snippet` did, `get_task` did
NOT — and auto-inject ranks kind-blind over a corpus that is overwhelmingly
tasks and issues, so the gap sat exactly where the volume is: every surfaced
task counted as never-pulled, dragging measured pull-through toward zero for
the menu's own dominant kind.
Asserted by source inspection rather than by calling the tools, because the
failure is a MISSING call — which no behavioural test of the tool's return
value can see.
"""
import inspect
from scribe.mcp.tools import notes as notes_tools
from scribe.mcp.tools import snippets as snippet_tools
from scribe.mcp.tools import tasks as task_tools
getters = (
(notes_tools, "get_note"),
(task_tools, "get_task"),
(snippet_tools, "get_snippet"),
)
for module, name in getters:
src = inspect.getsource(getattr(module, name))
assert "record_pulled(" in src, (
f"{name} can be surfaced in an auto-inject menu but records no pull — "
"its pull-through rate will read as zero regardless of real usage"
)