FabledGrimoire/src/scribe/models/note_usage.py

from sqlalchemy import Index, Integer, Text
from sqlalchemy.orm import Mapped, mapped_column

from scribe.models import Base
from scribe.models.base import CreatedAtMixin, iso

SURFACED = "surfaced"
PULLED = "pulled"


class NoteUsageEvent(Base, CreatedAtMixin):
    """One row per time a note was SURFACED to the agent, or PULLED in full.

    Answers the question RetrievalLog cannot: not "what did the ranker return
    and with what scores" but "did anyone ever actually open this?" A snippet
    nobody opens is not neutral — it competes for the injection budget on every
    future turn and dilutes the menu — so the surfaced:pulled ratio is what
    makes dead weight visible and prunable.

    WHY A SEPARATE TABLE FROM RetrievalLog. RetrievalLog is one row per *call*,
    keyed on the score distribution it exists to capture; folding un-scored
    events into it would corrupt exactly the distribution threshold tuning reads
    (see the KNOWN GAP note this closes in plugin_context.build_write_path_hint).
    This table is one row per *note per event*, which is the grain the usage
    readout needs and the grain RetrievalLog's JSONB `result_ids` can't be
    indexed at. The two are complements: RetrievalLog tunes the threshold, this
    tunes the corpus.

    WHY NOT IN-SESSION CORRELATION. The original framing was "correlate
    result_ids against a later get_note in the same session". There is no
    session identity server-side — the MCP endpoint is stateless and hooks send
    no session id — and introducing one would mean threading an opaque,
    client-supplied token through every read path for a signal that does not
    need it. Two independent counters answer the question without it: a note
    surfaced 40 times and never pulled is dead weight regardless of which
    sessions those events fell in.

    Deliberately FK-free on user_id and note_id (mirrors RetrievalLog/AppLog):
    telemetry outlives the row it describes, and deleting a note should not
    erase the evidence that it was surfaced 40 times and never once opened.
    """

    __tablename__ = "note_usage_events"

    id: Mapped[int] = mapped_column(primary_key=True)
    user_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
    note_id: Mapped[int] = mapped_column(Integer, nullable=False)
    # 'surfaced' | 'pulled'
    event: Mapped[str] = mapped_column(Text, nullable=False)
    # Which surface produced it. Kept granular so the place arm and the
    # semantic arm can be compared — that comparison is the whole reason the
    # place arm needed logging at all.
    #
    # A CONVENTION, not a fixed vocabulary: `mcp_<tool>` for an agent call,
    # `rest_<kind>` for a human opening a detail view, and a bare name for a
    # hook or background surface ('auto_inject', 'write_path_place',
    # 'write_path_semantic'). This comment deliberately no longer lists the
    # members — the previous list had gone stale, naming 'rest_note' that
    # nothing wrote while omitting sources that existed, and a half-true
    # enumeration reads as authoritative in exactly the way that misleads
    # (#2476). `grep -rn record_pulled\\\|record_surfaced src/` is the
    # authoritative list, and unlike a comment it cannot drift.
    #
    # The mcp_/rest_ split is load-bearing. "Is this dead weight?" is served by
    # any pull; "was that injected line useful?" is served by AGENT pulls only,
    # so never aggregate across the prefix without saying why (#1038, #2085).
    source: Mapped[str] = mapped_column(Text, nullable=False)

    # The project the READER was in when this happened — not the project the
    # record belongs to, which is already on `notes.project_id`. The whole
    # point is the comparison between the two: a record opened somewhere other
    # than where it was written is the evidence that it TRANSFERRED, which is
    # the claim the lesson kind rests on (milestone 385) and the one thing
    # RetrievalLog could half-answer and this table could not answer at all.
    #
    # Nullable, and null is ordinary rather than historical. A surfacing arm
    # always knows the project it searched; a getter knows only what its caller
    # passed, and a caller that passed nothing is reporting "unknown", not
    # "none". Treat a null as unreported: it still counts toward "is this dead
    # weight", and it cannot speak to "was it opened away from home".
    project_id: Mapped[int | None] = mapped_column(Integer, nullable=True)

    __table_args__ = (
        # The readout is always "these note ids, split by event" — a covering
        # composite beats separate single-column indexes for it.
        Index("ix_note_usage_note_event", "note_id", "event"),
        Index("ix_note_usage_created_at", "created_at"),
        Index("ix_note_usage_user_id", "user_id"),
        # note_id first, like ix_note_usage_note_event: every readout starts
        # from a set of note ids and narrows, never from a project.
        Index("ix_note_usage_note_project", "note_id", "project_id"),
    )

    def to_dict(self) -> dict:
        return {
            "id": self.id,
            "created_at": iso(self.created_at),
            "user_id": self.user_id,
            "note_id": self.note_id,
            "event": self.event,
            "source": self.source,
        }