feat(telemetry): a usage event records which project the reader was in (#4196, #3735)

`RetrievalLog` has carried `project_id` since it existed, so "this record
was SURFACED on project B" was always answerable. `note_usage_events`
had none, so "this record was OPENED on project B" was not — and the two
cannot be joined to recover it, because there is deliberately no session
identity server-side. NoteUsageEvent's own docstring rules that out.

That gap sat exactly on the question milestone 385 exists to answer. A
lesson's whole claim is that it reaches a session on a project it was not
written on, and step 8's acceptance is "retrieved on a different project
AND opened". Each half was answerable; the conjunction was not.

WHICH project, because the name is ambiguous and the wrong reading makes
the column useless: it is the project the READER was in, never the one
the record belongs to. The record's own project is already on the note;
copying it here would answer a question nobody asked while looking like
it answered this one.

The surfacing half is free — every arm already holds the scope it just
searched, so auto_inject, lesson_slot, the write-path arms and
enter_project now record it. process_skill_sync does not and should not:
it installs every Process the operator can reach, which is not a
project-scoped question, so a project there would be a fiction.

The pull half needs the caller, since a getter knows only what it was
handed. The five single-record getters take `project_id: int = 0` and
pass it through, following the convention `search` and `create_*`
already set. Null stays an ordinary answer meaning "not reported" — a
pull with no project is still a pull and still counts toward dead
weight; it simply cannot speak to transfer. The four REST detail views
report none for now: a human opening a record in a browser is a
different event from an agent recalling one, and #2245 left that
asymmetry deliberately undecided.

Guarded the way #2245 and #2476 taught: by source inspection, because a
parameter that was never threaded through changes no return value and
shows up only as a column that is mysteriously always null. Three
guards — the signature, the pass-through, and the arms — plus the
can-fail test rule 167 asks for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

src/scribe/mcp/tools/lessons.py

@@ -177,7 +177,7 @@ async def create_lesson(
     return data
 
 
-async def get_lesson(lesson_id: int) -> dict:
+async def get_lesson(lesson_id: int, project_id: int = 0) -> dict:
     """Fetch one lesson by id, with its trigger and sources read back out.
 
     IF THIS LESSON JUST PROVED ITSELF, IT IS WORTH MORE THAN IT SAYS. You are
@@ -188,6 +188,10 @@ async def get_lesson(lesson_id: int) -> dict:
     re-keyed trigger — and the trigger is the edit that pays most, because a
     lesson keyed to a situation nobody is in looks exactly like one nobody
     needed.
+
+    `project_id` is the project you are WORKING IN, not this record's own.
+    Passing the active project is what makes "opened away from where it was
+    written" answerable; 0 leaves it unreported and the pull still counts.
     """
     uid = current_user_id()
     note = await lessons_svc.get_lesson(uid, lesson_id)
@@ -199,7 +203,10 @@ async def get_lesson(lesson_id: int) -> dict:
     # retrieval as any other note, so a getter that records nothing would leave
     # the kind permanently at zero pulls — reading as dead weight beside kinds
     # that merely had a counter (#2476, the repeat of #2245).
-    record_pulled(user_id=uid, note_id=int(note.id), source="mcp_get_lesson")
+    record_pulled(
+        user_id=uid, note_id=int(note.id),
+        source="mcp_get_lesson", project_id=project_id,
+    )
     return out
 
 

src/scribe/mcp/tools/notes.py

@@ -67,7 +67,7 @@ async def list_notes(
 
 
 
-async def get_note(note_id: int) -> dict:
+async def get_note(note_id: int, project_id: int = 0) -> dict:
     """Fetch the full content of a single Scribe note by its ID.
 
     Returns id, title, body (markdown), tags, project_id, created_at,
@@ -80,6 +80,10 @@ async def get_note(note_id: int) -> dict:
     this one up to date. It is still here and still readable — supersession
     demotes, it never hides — but read it as what was true when written, and
     open the newer note before acting on it.
+
+    `project_id` is the project you are WORKING IN, not this record's own.
+    Passing the active project is what makes "opened away from where it was
+    written" answerable; 0 leaves it unreported and the pull still counts.
     """
     uid = current_user_id()
     loaded = await notes_svc.get_note_for_user(uid, note_id)
@@ -92,7 +96,10 @@ async def get_note(note_id: int) -> dict:
     # menu surfaces notes, tasks and processes too, so restricting this to
     # snippets would leave those permanently at zero pulls and make them look
     # like dead weight next to snippets that merely had a counter (#2085).
-    record_pulled(user_id=uid, note_id=int(note.id), source="mcp_get_note")
+    record_pulled(
+        user_id=uid, note_id=int(note.id),
+        source="mcp_get_note", project_id=project_id,
+    )
     await supersession_svc.attach_relations(uid, note_id, out, hint=True)
     await systems_tools.attach_systems(
         uid, getattr(note, "user_id", uid) or uid, out, note.id, note.project_id

src/scribe/mcp/tools/processes.py

@@ -102,7 +102,7 @@ async def create_process(
     return note.to_dict()
 
 
-async def get_process(name_or_id: str) -> dict:
+async def get_process(name_or_id: str, project_id: int = 0) -> dict:
     """Fetch a stored process by name or id and return its full prompt — the
     fire mechanism. The operator says "run the <name> process"; call this and
     follow the returned body (including any 'clarify first' steps it contains).
@@ -127,6 +127,10 @@ async def get_process(name_or_id: str) -> dict:
     follow-it-as-written contract above applies only to the operator's own
     processes — a shared one is a proposal, and running it unasked would put
     someone else's judgement in charge of this session.
+
+    `project_id` is the project you are WORKING IN, not this record's own.
+    Passing the active project is what makes "opened away from where it was
+    written" answerable; 0 leaves it unreported and the pull still counts.
     """
     uid = current_user_id()
     note, candidates = await notes_svc.resolve_process(uid, name_or_id)
@@ -141,7 +145,10 @@ async def get_process(name_or_id: str) -> dict:
     # this, the getter the product points at is the one getter that records
     # nothing, and every process sits permanently at zero pulls looking like dead
     # weight beside kinds that merely had a counter (#2476, the repeat of #2245).
-    record_pulled(user_id=uid, note_id=int(note.id), source="mcp_get_process")
+    record_pulled(
+        user_id=uid, note_id=int(note.id),
+        source="mcp_get_process", project_id=project_id,
+    )
     return out
 
 

src/scribe/mcp/tools/projects.py

@@ -214,6 +214,7 @@ async def enter_project(project_id: int) -> dict:
         user_id=uid,
         note_ids=[int(t.id) for t in open_tasks],
         source="enter_project",
+        project_id=project_id,
     )
     # A project need not have one, and most installs won't — null is ordinary
     # here, not a missing prerequisite. Summary only: the guidance is ~9k of

src/scribe/mcp/tools/snippets.py

@@ -209,7 +209,7 @@ async def create_snippet(
     return data
 
 
-async def get_snippet(snippet_id: int) -> dict:
+async def get_snippet(snippet_id: int, project_id: int = 0) -> dict:
     """Fetch a snippet by id — the full record: code, signature, location, and a
     parsed `snippet` field of its structured parts.
 
@@ -236,6 +236,10 @@ async def get_snippet(snippet_id: int) -> dict:
     `owner` and your `permission`. Read that as ONE PERSON'S SUGGESTION, not as
     established practice here: judge it on its merits, say whose it is when you
     reference it, and don't adopt it as the house pattern without checking.
+
+    `project_id` is the project you are WORKING IN, not this record's own.
+    Passing the active project is what makes "opened away from where it was
+    written" answerable; 0 leaves it unreported and the pull still counts.
     """
     uid = current_user_id()
     note = await snippets_svc.get_snippet(uid, snippet_id)
@@ -250,7 +254,10 @@ async def get_snippet(snippet_id: int) -> dict:
     # snippets_svc.get_snippet — the service is also reached by update/merge
     # paths, and counting those would inflate exactly the number that is
     # supposed to mean "someone chose to look at this" (#2085).
-    record_pulled(user_id=uid, note_id=int(note.id), source="mcp_get_snippet")
+    record_pulled(
+        user_id=uid, note_id=int(note.id),
+        source="mcp_get_snippet", project_id=project_id,
+    )
     await systems_tools.attach_systems(
         uid, note.user_id, data, note.id, note.project_id
     )

src/scribe/mcp/tools/tasks.py

@@ -78,7 +78,7 @@ async def list_tasks(
     return {"tasks": [notes_svc.brief_row(n, titles) for n in rows], "total": total}
 
 
-async def get_task(task_id: int) -> dict:
+async def get_task(task_id: int, project_id: int = 0) -> dict:
     """Fetch a single Scribe task by ID.
 
     Returns id, title, body, status, priority, tags, project_id, milestone_id,
@@ -91,6 +91,10 @@ async def get_task(task_id: int) -> dict:
 
     A task another user shared with you also carries `shared`, `owner` and
     `permission` — it's their work item, not one you took on.
+
+    `project_id` is the project you are WORKING IN, not this record's own.
+    Passing the active project is what makes "opened away from where it was
+    written" answerable; 0 leaves it unreported and the pull still counts.
     """
     uid = current_user_id()
     loaded = await notes_svc.get_note_for_user(uid, task_id)
@@ -123,7 +127,10 @@ async def get_task(task_id: int) -> dict:
     await systems_tools.attach_systems(
         uid, getattr(note, "user_id", uid) or uid, data, note.id, note.project_id
     )
-    record_pulled(user_id=uid, note_id=int(note.id), source="mcp_get_task")
+    record_pulled(
+        user_id=uid, note_id=int(note.id),
+        source="mcp_get_task", project_id=project_id,
+    )
     return data
 
 

src/scribe/models/note_usage.py

@@ -66,12 +66,29 @@ class NoteUsageEvent(Base, CreatedAtMixin):
     # so never aggregate across the prefix without saying why (#1038, #2085).
     source: Mapped[str] = mapped_column(Text, nullable=False)
 
+    # The project the READER was in when this happened — not the project the
+    # record belongs to, which is already on `notes.project_id`. The whole
+    # point is the comparison between the two: a record opened somewhere other
+    # than where it was written is the evidence that it TRANSFERRED, which is
+    # the claim the lesson kind rests on (milestone 385) and the one thing
+    # RetrievalLog could half-answer and this table could not answer at all.
+    #
+    # Nullable, and null is ordinary rather than historical. A surfacing arm
+    # always knows the project it searched; a getter knows only what its caller
+    # passed, and a caller that passed nothing is reporting "unknown", not
+    # "none". Treat a null as unreported: it still counts toward "is this dead
+    # weight", and it cannot speak to "was it opened away from home".
+    project_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
+
     __table_args__ = (
         # The readout is always "these note ids, split by event" — a covering
         # composite beats separate single-column indexes for it.
         Index("ix_note_usage_note_event", "note_id", "event"),
         Index("ix_note_usage_created_at", "created_at"),
         Index("ix_note_usage_user_id", "user_id"),
+        # note_id first, like ix_note_usage_note_event: every readout starts
+        # from a set of note ids and narrows, never from a project.
+        Index("ix_note_usage_note_project", "note_id", "project_id"),
     )
 
     def to_dict(self) -> dict:

src/scribe/services/note_usage.py

@@ -85,13 +85,40 @@ def _schedule(rows: list[dict]) -> None:
     task.add_done_callback(_pending.discard)
 
 
+def _project_or_none(project_id: int | None) -> int | None:
+    """0 and None both mean "no project reported" — store one of them.
+
+    Callers reach this from two conventions at once: the MCP tools spell "no
+    project" as `0` (it is an int parameter with an int default), while the
+    column is nullable. Folding them here keeps every call site from having to
+    remember which one this function wants, and stops a row claiming it was
+    read on project #0.
+    """
+    try:
+        pid = int(project_id or 0)
+    except (TypeError, ValueError):
+        return None
+    return pid or None
+
+
 def record_surfaced(
-    *, user_id: int | None, note_ids: list[int] | set[int], source: str
+    *,
+    user_id: int | None,
+    note_ids: list[int] | set[int],
+    source: str,
+    project_id: int | None = None,
 ) -> None:
     """Fire-and-forget: record that these notes were shown to the agent.
 
     Takes the whole menu at once — one insert per surfacing event, not per note
     — because a menu is a single decision and its rows should land together.
+
+    `project_id` is where the READER was, not where the record lives. Every
+    surfacing arm knows it — it is the scope it just searched — so pass it;
+    it is what makes "surfaced away from home" answerable without joining
+    RetrievalLog. 0 is normalised to None: a project id of zero means "no
+    project" everywhere else in this codebase, and storing it would read as
+    project #0.
     """
     try:
         rows = [
@@ -100,6 +127,7 @@ def record_surfaced(
                 "note_id": int(nid),
                 "event": SURFACED,
                 "source": source,
+                "project_id": _project_or_none(project_id),
             }
             for nid in note_ids
         ]
@@ -109,8 +137,25 @@ def record_surfaced(
     _schedule(rows)
 
 
-def record_pulled(*, user_id: int | None, note_id: int, source: str) -> None:
-    """Fire-and-forget: record that a note was opened in full."""
+def record_pulled(
+    *,
+    user_id: int | None,
+    note_id: int,
+    source: str,
+    project_id: int | None = None,
+) -> None:
+    """Fire-and-forget: record that a note was opened in full.
+
+    `project_id` is where the READER was — the caller's active project, never
+    the record's own. Compared against the record's `project_id`, it answers
+    whether this was opened somewhere other than where it was written, which
+    is the evidence that a record TRANSFERRED (milestone 385, #3735).
+
+    Unlike a surfacing arm, a getter only knows what it was handed, so this
+    stays optional and null is an ordinary answer meaning "not reported". A
+    pull with no project is still a pull: it counts toward dead-weight
+    detection and simply cannot speak to transfer.
+    """
     try:
         rows = [
             {
@@ -118,6 +163,7 @@ def record_pulled(*, user_id: int | None, note_id: int, source: str) -> None:
                 "note_id": int(note_id),
                 "event": PULLED,
                 "source": source,
+                "project_id": _project_or_none(project_id),
             }
         ]
     except Exception:

src/scribe/services/plugin_context.py

@@ -872,6 +872,7 @@ async def _reserve_slot_for_lesson(
     if slot_id not in already:
         record_surfaced(
             user_id=user_id, note_ids=[slot_id], source="lesson_slot",
+            project_id=project_id,
         )
     return kept + slot, slot_id
 
@@ -1076,6 +1077,7 @@ async def build_autoinject_hint(
             if i not in already and i != lesson_slot_id
         ],
         source="auto_inject",
+        project_id=project_id,
     )
 
     return {"context": "\n".join(lines), "note_ids": note_ids, "config": cfg}
@@ -2178,7 +2180,9 @@ async def build_write_path_hint(
                else "write_path_semantic")
         by_arm.setdefault(arm, []).append(int(item["id"]))
     for arm, ids in by_arm.items():
-        record_surfaced(user_id=user_id, note_ids=ids, source=arm)
+        record_surfaced(
+            user_id=user_id, note_ids=ids, source=arm, project_id=project_id,
+        )
 
     # ── Standing rules that may apply here (milestone 307) ──────────────
     #