fix(knowledge): the browse vocabulary catches up three kinds, and a snippet's mirror survives the generic door (#3128 recs 2-6)

Spike #3128 found the storage sound and the retrieval vocabulary frozen
before `issue` shipped (0065). Five things, in the order they had to land.

**The mirror (rec 5, the data-integrity one).** `notes.data` is DERIVED from
a snippet's body, but only `update_snippet` knew that. `update_note` is a
hasattr loop with no snippet awareness, and both doors reach it — so PATCH
/api/notes/<snippet_id> {body} rewrote the body and left the mirror behind.
`snippet_fields` PREFERS the mirror, so the row went on reporting its old
repo/path/symbol to the location reverse lookup and to prior-art recall while
displaying its new body: surfaced with full authority, and wrong.
`snippets.recompose_data` rebuilds it from the body, carrying `verification`
and `provenance` (neither is in the body to parse). An explicit `data` still
wins, so every snippet-service write is untouched.

**One facet table (rec 3), before adding any facet.** The type predicate was
written three times — SQL, Python over semantic candidates, and a ternary
computing the `is_task` pre-filter — and agreed only by luck. Adding `issue`
to the SQL arm alone would have set the pre-filter to is_task=False, handed
the Python arm a candidate set with no tasks in it, and returned an empty
semantic half for the Issues facet forever with nothing red. `_FACETS` now
generates all three. The Python arm also regains the `status IS NULL` half its
SQL twin always had.

**Issue and spike become facets (rec 2).** 435 issues — 17% of every task —
were filterable nowhere on the human surface, while retired `plan` (90 rows)
had a chip of its own. `_VALID_TYPES` was a hand-kept copy and is now derived.
`plan` stays a valid facet for its legacy rows; it loses its chip.

**Snippets stop being half-present in the feed (rec 4).** All 90 were in the
All list, in no count, wearing an empty badge, and opening in the note editor.
Counts now group by task_kind — every kind for the same two round-trips, which
is why `issue` had no number — and total includes snippets, so the All chip
matches the list it labels. Snippet cards route to /snippets/:id.

**The prose that excused it (rec 6).** `snippet_fields` and the `data` column
both still said pre-0070 rows were "never backfilled". True when 0070 landed,
false since `backfill_snippet_data` shipped, and it read as licence for a
stale mirror.

Tests: the pre-filter can never exclude a row its own facet accepts (the
regression, parameterised over every facet); both dialects select exactly
their own rows; an unknown facet matches nothing; the mirror follows a body or
title write, carries the verdict, and yields to an explicit `data`.
`compiled_sql` moves to tests/helpers rather than becoming a third copy.

Write-up: note #3161.

src/scribe/models/note.py

@@ -94,8 +94,11 @@ class Note(Base, TimestampMixin, SoftDeleteMixin):
     # name/language/signature/locations live here so they can be INDEXED. The
     # body keeps the same facts in readable markdown and remains what gets
     # embedded; this is a mirror for querying, not the source of truth for
-    # display. NULL on every row written before migration 0070, so readers fall
-    # back to parsing the body (see services/snippets.snippet_fields).
+    # display — and it is DERIVED, so every path that writes a snippet's body
+    # rewrites it too (services/snippets.recompose_data, called from
+    # notes.update_note). 0070 left it NULL on existing rows and
+    # snippets.backfill_snippet_data filled them at startup; readers still fall
+    # back to parsing the body when it is absent (snippet_fields).
     data: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
 
     __table_args__ = (

src/scribe/routes/knowledge.py

@@ -1,4 +1,4 @@
-"""Unified Knowledge endpoint — notes, tasks, plans, and processes in one queryable feed."""
+"""Unified Knowledge endpoint — every record kind in one queryable feed."""
 import logging
 
 from quart import Blueprint, jsonify, request
@@ -6,12 +6,18 @@ from quart import Blueprint, jsonify, request
 from scribe.auth import get_current_user_id, login_required
 from scribe.routes.utils import parse_pagination
 from scribe.services.access import label_shared_items
+from scribe.services.knowledge import FACET_TYPES
 
 logger = logging.getLogger(__name__)
 
 knowledge_bp = Blueprint("knowledge", __name__, url_prefix="/api/knowledge")
 
-_VALID_TYPES = {"note", "task", "plan", "process"}
+# Derived from the service's facet table, never re-listed here. This set was a
+# hand-kept copy and had drifted three kinds behind it: it admitted `plan`
+# (retired in 0066) and rejected `issue` (shipped in 0065, 435 rows) and
+# `snippet` — so the browse surface could not filter to the kinds it was
+# already rendering badges for (#3128).
+_VALID_TYPES = FACET_TYPES
 _VALID_SORTS = {"modified", "created", "alpha", "type"}
 
 
@@ -21,7 +27,9 @@ async def list_knowledge():
     """Return paginated knowledge objects with optional filtering.
 
     Query params:
-      type        — one of note|task|plan|process (omit for all)
+      type        — a facet from services.knowledge._FACETS: a record type
+                    (note|process|snippet) or a task kind (task for any,
+                    else work|issue|spike|plan). Omit for all.
       tags        — comma-separated tag filter (AND logic)
       sort        — modified|created|alpha|type (default: modified)
       q           — search query (semantic when provided, keyword fallback)
@@ -127,7 +135,7 @@ async def get_knowledge_batch():
 @knowledge_bp.route("/tags", methods=["GET"])
 @login_required
 async def list_knowledge_tags():
-    """Return all tags used across knowledge objects (excludes tasks)."""
+    """Return all tags used across knowledge objects, narrowed to one facet."""
     uid = get_current_user_id()
     note_type = request.args.get("type", "").strip().lower() or None
 

src/scribe/services/knowledge.py

@@ -1,4 +1,4 @@
-"""Knowledge service — unified query across notes, tasks, plans, and processes.
+"""Knowledge service — one query across every record kind Scribe holds.
 
 ACL (rules #47/#78, decision note 2094): these queries were owner-only until
 2026-07-25, which meant a record shared with you could be opened by id but never
@@ -265,22 +265,94 @@ def _note_to_item(note: Note) -> dict:
     return item
 
 
-def _apply_type_filter(stmt, note_type: str | None):
-    """Apply the type facet to a Note select.
+# What each type facet MEANS, once, for every arm that has to know.
+#
+# The vocabulary spans BOTH typing axes — `note_type` for non-task records and
+# `task_kind` for tasks — so a facet cannot be a filter on one column, which is
+# why this is a table rather than a chain of ifs. Each entry is
+# (is_task, the value pinned on that axis); None pins nothing, i.e. every task.
+#
+# It is a table because the alternative had already gone wrong. The predicate
+# was written three times — a SQL if-chain, a Python if-chain over semantic
+# candidates, and a ternary computing the `is_task` pre-filter — and the three
+# only agreed by luck. Adding `issue` to the SQL arm alone (the obvious edit,
+# and the one #3128 was about to make) would have set the pre-filter to
+# is_task=False, handed the Python arm a candidate set containing no tasks at
+# all, and returned an empty semantic half for the Issues facet forever, with
+# nothing red anywhere. A new facet is now one row here.
+#
+# `plan` is retired (0066) but kept: 90 legacy plan-tasks exist and a facet
+# they answer to costs one line. It simply has no chip in the UI any more.
+_FACETS: dict[str, tuple[bool, str | None]] = {
+    "task": (True, None),
+    "work": (True, "work"),
+    "issue": (True, "issue"),
+    "spike": (True, "spike"),
+    "plan": (True, "plan"),
+    "note": (False, "note"),
+    "process": (False, "process"),
+    "snippet": (False, "snippet"),
+}
 
-    'task' = any task (status not null); 'plan' = a task with task_kind='plan';
-    any other non-empty type = a non-task note of that note_type; None = all.
+# The non-task record types, for the counts query. Derived so it cannot drift
+# from the table above.
+NON_TASK_FACETS = tuple(
+    value for _is_task, value in _FACETS.values() if not _is_task and value
+)
 
-    Trashed rows (deleted_at set) are always excluded.
+# The whole vocabulary, for the door's request validation — public so the route
+# validates against the same table the query reads instead of a hand-kept copy.
+FACET_TYPES = frozenset(_FACETS)
+
+
+# An unrecognised facet resolves to "a non-task note whose note_type is that
+# string" — which matches nothing, since no row stores an unknown type. That is
+# the behaviour the old if-chain had by falling through, and it is the right
+# one: a typo should return an empty list, never the whole corpus.
+def _facet(note_type: str) -> tuple[bool, str | None]:
+    return _FACETS.get(note_type, (False, note_type))
+
+
+def facet_is_task(note_type: str | None) -> bool | None:
+    """The `is_task` pre-filter a facet implies — None when it spans both.
+
+    Used to narrow the semantic candidate set before it is fetched. Reads the
+    same table `_apply_type_filter` and `matches_facet` read, so the pre-filter
+    can no longer disagree with the predicate it is meant to anticipate.
     """
+    if not note_type:
+        return None
+    return _facet(note_type)[0]
+
+
+def matches_facet(note, note_type: str | None) -> bool:
+    """The Python dialect of `_apply_type_filter`, for candidates the vector
+    search has already fetched — there is no query left to narrow.
+
+    Generated from the same table, so this is a translation rather than a
+    second implementation. Note the `not note.is_task` arm: the hand-written
+    version omitted it and was saved only by the upstream pre-filter.
+    """
+    if not note_type:
+        return True
+    is_task, value = _facet(note_type)
+    if is_task:
+        return note.is_task and (value is None or note.task_kind == value)
+    return not note.is_task and note.note_type == value
+
+
+def _apply_type_filter(stmt, note_type: str | None):
+    """Apply the type facet to a Note select. Trashed rows are always excluded."""
     stmt = stmt.where(Note.deleted_at.is_(None))
-    if note_type == "task":
-        return stmt.where(Note.status.isnot(None))
-    if note_type == "plan":
-        return stmt.where(Note.status.isnot(None)).where(Note.task_kind == "plan")
-    if note_type:
-        return stmt.where(Note.note_type == note_type).where(Note.status.is_(None))
-    return stmt
+    if not note_type:
+        return stmt
+    is_task, value = _facet(note_type)
+    if is_task:
+        stmt = stmt.where(Note.status.isnot(None))
+        if value is not None:
+            stmt = stmt.where(Note.task_kind == value)
+        return stmt
+    return stmt.where(Note.status.is_(None)).where(Note.note_type == value)
 
 
 async def query_knowledge(
@@ -295,7 +367,7 @@ async def query_knowledge(
     locations: dict[str, str] | None = None,
     verification: str = "",
 ) -> tuple[list[dict], int]:
-    """Query knowledge objects (non-task notes) with filters.
+    """Query knowledge objects with filters.
 
     `project_id` narrows to one project (None = every project).
 
@@ -424,7 +496,7 @@ async def _semantic_knowledge_search(
             INTERACTIVE_SEARCH_THRESHOLD,
             semantic_search_notes,
         )
-        is_task_filter = True if note_type in ("task", "plan") else (False if note_type else None)
+        is_task_filter = facet_is_task(note_type)
         import time as _time
         _t0 = _time.perf_counter()
         candidates = await semantic_search_notes(
@@ -454,11 +526,7 @@ async def _semantic_knowledge_search(
         for _score, note in candidates:
             if note.deleted_at is not None:
                 continue
-            if note_type == "task" and not note.is_task:
-                continue
-            elif note_type == "plan" and (not note.is_task or note.task_kind != "plan"):
-                continue
-            elif note_type and note_type not in ("task", "plan") and note.note_type != note_type:
+            if not matches_facet(note, note_type):
                 continue
             if tags and not all(t in (note.tags or []) for t in tags):
                 continue
@@ -514,51 +582,40 @@ async def get_knowledge_counts(user_id: int, tags: list[str] | None = None) -> d
     search would surface."""
     visible = browsable_notes_clause(user_id)
     async with async_session() as session:
-        # Count non-task types
-        stmt = (
-            select(Note.note_type, func.count(Note.id))
-            .where(visible)
-            .where(Note.status.is_(None))
-            .where(Note.deleted_at.is_(None))
-            .where(Note.note_type.in_(["note", "process"]))
-            .group_by(Note.note_type)
-        )
-        if tags:
-            for tag in tags:
+        def _scoped(stmt):
+            stmt = stmt.where(visible).where(Note.deleted_at.is_(None))
+            for tag in tags or []:
                 stmt = stmt.where(Note.tags.contains([tag]))
-        rows = list((await session.execute(stmt)).all())
-        counts = {row[0]: row[1] for row in rows}
+            return stmt
 
-        # Count tasks separately (is_task = status IS NOT NULL)
-        task_stmt = (
-            select(func.count(Note.id))
-            .where(visible)
+        # One grouped query per typing axis. The task half used to be a count
+        # for 'task' plus a second count for 'plan', which is why 'issue' —
+        # 17% of every task here — had no number to show: each kind needed its
+        # own query and nobody added one. Grouping by task_kind counts every
+        # kind, including ones added later, for the same two round-trips.
+        non_task = _scoped(
+            select(Note.note_type, func.count(Note.id))
+            .where(Note.status.is_(None))
+            .where(Note.note_type.in_(NON_TASK_FACETS))
+        ).group_by(Note.note_type)
+        counts = {t: n for t, n in (await session.execute(non_task)).all()}
+
+        by_kind = _scoped(
+            select(Note.task_kind, func.count(Note.id))
             .where(Note.status.isnot(None))
-            .where(Note.deleted_at.is_(None))
-        )
-        if tags:
-            for tag in tags:
-                task_stmt = task_stmt.where(Note.tags.contains([tag]))
-        task_count: int = (await session.execute(task_stmt)).scalar_one()
-        counts["task"] = task_count
+        ).group_by(Note.task_kind)
+        kind_counts = {k: n for k, n in (await session.execute(by_kind)).all()}
 
-        # Plans are a subset of tasks (task_kind='plan'); counted for the facet
-        # but NOT added to total to avoid double-counting against "task".
-        plan_stmt = (
-            select(func.count(Note.id))
-            .where(visible)
-            .where(Note.status.isnot(None))
-            .where(Note.task_kind == "plan")
-            .where(Note.deleted_at.is_(None))
-        )
-        if tags:
-            for tag in tags:
-                plan_stmt = plan_stmt.where(Note.tags.contains([tag]))
-        counts["plan"] = (await session.execute(plan_stmt)).scalar_one()
+    # Kinds are SUBSETS of 'task' and are deliberately left out of the total —
+    # adding them would count every task twice.
+    counts["task"] = sum(kind_counts.values())
+    for kind, value in _FACETS.items():
+        if value[0] and value[1] is not None:
+            counts[kind] = kind_counts.get(kind, 0)
 
-    for t in ("note", "task", "plan", "process"):
+    for t in NON_TASK_FACETS:
         counts.setdefault(t, 0)
-    counts["total"] = sum(counts[t] for t in ("note", "task", "process"))
+    counts["total"] = counts["task"] + sum(counts[t] for t in NON_TASK_FACETS)
     return counts
 
 

src/scribe/services/notes.py

@@ -9,6 +9,13 @@ from scribe.models.note import Note, TaskKind, TaskPriority, TaskStatus
 
 logger = logging.getLogger(__name__)
 
+# The fields `snippets.parse_snippet_fields` reads. Writing any of them can
+# change what a snippet's derived `data` mirror should say, so update_note
+# recomposes the mirror when one moves. Kept here as a set of NAMES rather
+# than imported, because it describes update_note's own `fields` dict, not the
+# parser's signature.
+_PARSED_FROM_BODY = frozenset({"title", "body", "tags"})
+
 
 def embed_note(note) -> None:
     """Refresh a note's embedding, fire-and-forget.
@@ -439,6 +446,23 @@ async def update_note(user_id: int, note_id: int, **fields: object) -> Note | No
             elif key == "tags" and isinstance(value, list):
                 value = _normalize_tags(value)
             setattr(note, key, value)
+        # A snippet's `data` is DERIVED from its body — so a write that moves
+        # the body through this generic door must move the mirror with it
+        # (#3128). Without this, PATCH /api/notes/<snippet_id> {body} left the
+        # mirror behind, and snippet_fields PREFERS the mirror: the row went on
+        # reporting its old repo/path/symbol to prior-art recall while showing
+        # its new body. `update_snippet` composes the mirror itself and passes
+        # it explicitly, so an explicit `data` always wins — the caller that
+        # knows the field set beats the one that can only re-read the body.
+        if "data" not in fields and not _PARSED_FROM_BODY.isdisjoint(fields):
+            # Imported here, not at module scope: services/snippets.py calls
+            # back into this module (update_snippet -> update_note), so a
+            # top-level import is a cycle.
+            from scribe.services.snippets import (
+                SNIPPET_NOTE_TYPE, recompose_data,
+            )
+            if note.note_type == SNIPPET_NOTE_TYPE:
+                note.data = recompose_data(note)
         # Auto-set lifecycle timestamps on status transitions
         if "status" in fields:
             _now = datetime.now(timezone.utc)

src/scribe/services/snippets.py

@@ -540,14 +540,60 @@ def compose_data(
     return out
 
 
+def recompose_data(note) -> dict:
+    """Rebuild a snippet's `data` mirror from its own body, title and tags.
+
+    For the GENERIC note door. `update_snippet` composes the mirror itself from
+    the field set it just merged and never needs this; a plain
+    `update_note(body=...)` — which the Knowledge feed's editor issues, because
+    a snippet card there routes to /notes/:id — has no idea the mirror exists,
+    and left it stale. `snippet_fields` then PREFERS the stale mirror, so the
+    row reported its old repo/path/symbol to prior-art recall while displaying
+    its new body: confidently wrong, which is worse than no record (#3128).
+
+    The body is the authority; the mirror is derived. That is already the rule
+    this file states — it just had no enforcement on the path that bypasses
+    `update_snippet`.
+
+    `verification` and `provenance` are CARRIED, not recomposed, because
+    neither is in the body to parse — the same carry `compose_data` does for
+    the snippet service's own writes. Note that a verdict does not need
+    invalidating here: `code_sha` is recomputed from the new code, so a stale
+    verdict expires itself on read exactly as it does after any other edit.
+    """
+    parsed = parse_snippet_fields(note.title, note.body, note.tags)
+    prior = note.data or {}
+    return compose_data(
+        name=parsed["name"],
+        when_to_use=parsed["when_to_use"],
+        signature=parsed["signature"],
+        language=parsed["language"],
+        code=parsed["code"],
+        locations=parsed["locations"],
+        merged_from=parsed["merged_from"],
+        verification=prior.get("verification"),
+        provenance=prior.get("provenance"),
+    )
+
+
 def snippet_fields(note) -> dict:
     """Structured fields for a snippet, preferring the indexed `data` column and
     falling back to parsing the body.
 
-    Both paths must agree, because rows written before migration 0070 have no
-    `data` and are never backfilled — a hand-edited body is the authority for
-    those, and there is no deadline by which they must be converted. `code` only
-    ever comes from the body, since `data` doesn't carry it.
+    THE BODY IS THE AUTHORITY; `data` is a mirror derived from it. Every writer
+    keeps them in step — the snippet service composes the mirror from the field
+    set it just merged, `update_note` recomposes it when a body reaches the
+    generic door (#3128), and `backfill_snippet_data` filled the pre-0070 rows
+    at startup. The fallback below is therefore a belt to that braces, not a
+    second source of truth: it is what a row looks like before the backfill has
+    run, and it must keep agreeing with the mirror.
+
+    (This docstring used to say those rows were "never backfilled". That was
+    true when 0070 landed and stopped being true when the backfill shipped; it
+    is corrected here because the sentence read as licence for a stale mirror,
+    which is exactly the bug #3128 found.)
+
+    `code` only ever comes from the body, since `data` doesn't carry it.
     """
     parsed = parse_snippet_fields(note.title, note.body, note.tags)
     stored = getattr(note, "data", None)