fix(knowledge): the browse vocabulary catches up three kinds, and a snippet's mirror survives the generic door (#3128 recs 2-6)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / integration (push) Successful in 27s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 33s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / integration (push) Successful in 27s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 33s
Spike #3128 found the storage sound and the retrieval vocabulary frozen before `issue` shipped (0065). Five things, in the order they had to land. **The mirror (rec 5, the data-integrity one).** `notes.data` is DERIVED from a snippet's body, but only `update_snippet` knew that. `update_note` is a hasattr loop with no snippet awareness, and both doors reach it — so PATCH /api/notes/<snippet_id> {body} rewrote the body and left the mirror behind. `snippet_fields` PREFERS the mirror, so the row went on reporting its old repo/path/symbol to the location reverse lookup and to prior-art recall while displaying its new body: surfaced with full authority, and wrong. `snippets.recompose_data` rebuilds it from the body, carrying `verification` and `provenance` (neither is in the body to parse). An explicit `data` still wins, so every snippet-service write is untouched. **One facet table (rec 3), before adding any facet.** The type predicate was written three times — SQL, Python over semantic candidates, and a ternary computing the `is_task` pre-filter — and agreed only by luck. Adding `issue` to the SQL arm alone would have set the pre-filter to is_task=False, handed the Python arm a candidate set with no tasks in it, and returned an empty semantic half for the Issues facet forever with nothing red. `_FACETS` now generates all three. The Python arm also regains the `status IS NULL` half its SQL twin always had. **Issue and spike become facets (rec 2).** 435 issues — 17% of every task — were filterable nowhere on the human surface, while retired `plan` (90 rows) had a chip of its own. `_VALID_TYPES` was a hand-kept copy and is now derived. `plan` stays a valid facet for its legacy rows; it loses its chip. **Snippets stop being half-present in the feed (rec 4).** All 90 were in the All list, in no count, wearing an empty badge, and opening in the note editor. Counts now group by task_kind — every kind for the same two round-trips, which is why `issue` had no number — and total includes snippets, so the All chip matches the list it labels. Snippet cards route to /snippets/:id. **The prose that excused it (rec 6).** `snippet_fields` and the `data` column both still said pre-0070 rows were "never backfilled". True when 0070 landed, false since `backfill_snippet_data` shipped, and it read as licence for a stale mirror. Tests: the pre-filter can never exclude a row its own facet accepts (the regression, parameterised over every facet); both dialects select exactly their own rows; an unknown facet matches nothing; the mirror follows a body or title write, carries the verdict, and yields to an explicit `data`. `compiled_sql` moves to tests/helpers rather than becoming a third copy. Write-up: note #3161.
This commit is contained in:
@@ -24,7 +24,7 @@ const router = useRouter();
|
|||||||
|
|
||||||
interface KnowledgeItem {
|
interface KnowledgeItem {
|
||||||
id: number;
|
id: number;
|
||||||
note_type: "note" | "task" | "process";
|
note_type: "note" | "task" | "process" | "snippet";
|
||||||
title: string;
|
title: string;
|
||||||
snippet: string;
|
snippet: string;
|
||||||
tags: string[];
|
tags: string[];
|
||||||
@@ -42,9 +42,34 @@ interface KnowledgeItem {
|
|||||||
task_kind?: TaskKind;
|
task_kind?: TaskKind;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ─── The facet vocabulary ─────────────────────────────────────────────────────
|
||||||
|
// Mirrors services/knowledge._FACETS, which is where it is defined for real.
|
||||||
|
// A facet spans BOTH typing axes — a record TYPE (note / process / snippet) or
|
||||||
|
// a task KIND (`task` for any, else issue / spike) — because that is what this
|
||||||
|
// feed actually holds.
|
||||||
|
//
|
||||||
|
// `plan` is still a valid facet at the API, for the 90 legacy plan-tasks, but
|
||||||
|
// it has no chip: retired in 0066, it kept a chip of its own for longer than
|
||||||
|
// `issue` — 17% of every task here — went without one (#3128). Those rows are
|
||||||
|
// still reachable under Tasks, wearing a Plan badge.
|
||||||
|
type Facet = "" | "note" | "task" | "issue" | "spike" | "snippet" | "process";
|
||||||
|
|
||||||
|
// The facets that select TASKS. Kinds are subsets of `task`, so any of them
|
||||||
|
// means the duplicate report should be comparing tasks.
|
||||||
|
const TASK_FACETS = new Set<Facet>(["task", "issue", "spike"]);
|
||||||
|
|
||||||
|
const FACET_CHIPS: [Exclude<Facet, "">, string][] = [
|
||||||
|
["note", "Notes"],
|
||||||
|
["task", "Tasks"],
|
||||||
|
["issue", "Issues"],
|
||||||
|
["spike", "Spikes"],
|
||||||
|
["snippet", "Snippets"],
|
||||||
|
["process", "Processes"],
|
||||||
|
];
|
||||||
|
|
||||||
// ─── Filter state ─────────────────────────────────────────────────────────────
|
// ─── Filter state ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
const activeType = ref<"" | "note" | "task" | "plan" | "process">("");
|
const activeType = ref<Facet>("");
|
||||||
const activeTag = ref("");
|
const activeTag = ref("");
|
||||||
const sortMode = ref<"modified" | "created" | "alpha" | "type">("modified");
|
const sortMode = ref<"modified" | "created" | "alpha" | "type">("modified");
|
||||||
const searchQuery = ref("");
|
const searchQuery = ref("");
|
||||||
@@ -70,9 +95,10 @@ const dupGroups = ref<DupGroup[]>([]);
|
|||||||
const dupSuggestion = ref("");
|
const dupSuggestion = ref("");
|
||||||
const dupLoading = ref(false);
|
const dupLoading = ref(false);
|
||||||
const dupChecked = ref(false);
|
const dupChecked = ref(false);
|
||||||
// The report follows the type filter: viewing tasks checks tasks. Anything
|
// The report follows the type filter: viewing tasks — under ANY task facet,
|
||||||
// else (all / plan / process) checks notes — the kind with the most to find.
|
// including a single kind — checks tasks. Everything else checks notes, the
|
||||||
const dupKind = computed(() => (activeType.value === "task" ? "task" : "note"));
|
// kind with the most to find.
|
||||||
|
const dupKind = computed(() => (TASK_FACETS.has(activeType.value) ? "task" : "note"));
|
||||||
|
|
||||||
async function loadDuplicates() {
|
async function loadDuplicates() {
|
||||||
dupLoading.value = true;
|
dupLoading.value = true;
|
||||||
@@ -96,8 +122,11 @@ watch(dupKind, () => { dupChecked.value = false; dupGroups.value = []; });
|
|||||||
|
|
||||||
// ─── Type counts ──────────────────────────────────────────────────────────────
|
// ─── Type counts ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
interface KnowledgeCounts { note: number; task: number; plan: number; process: number; total: number }
|
// One number per facet, plus the grand total. Partial because the server sends
|
||||||
const typeCounts = ref<KnowledgeCounts>({ note: 0, task: 0, plan: 0, process: 0, total: 0 });
|
// a key only for a facet it has rows for. Kinds are subsets of `task` and are
|
||||||
|
// deliberately absent from `total` — including them would count an issue twice.
|
||||||
|
type KnowledgeCounts = Partial<Record<Exclude<Facet, "">, number>> & { total: number };
|
||||||
|
const typeCounts = ref<KnowledgeCounts>({ total: 0 });
|
||||||
|
|
||||||
async function fetchCounts() {
|
async function fetchCounts() {
|
||||||
try {
|
try {
|
||||||
@@ -274,9 +303,18 @@ function isOverdue(item: KnowledgeItem): boolean {
|
|||||||
return new Date(item.due_date) < new Date(new Date().toDateString());
|
return new Date(item.due_date) < new Date(new Date().toDateString());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Each record kind opens in ITS OWN editor. A snippet used to fall through to
|
||||||
|
// /notes/:id, whose save is a plain PATCH of the body — which left the snippet's
|
||||||
|
// derived `data` mirror describing the previous version (#3128). The service now
|
||||||
|
// recomposes the mirror either way, so this is no longer the guard; it is simply
|
||||||
|
// that the note editor cannot edit a snippet's signature, language or locations,
|
||||||
|
// and offering it as the way in was always wrong. Processes stay here on
|
||||||
|
// purpose: they have no editor of their own and the note editor knows the type.
|
||||||
function openItem(item: KnowledgeItem) {
|
function openItem(item: KnowledgeItem) {
|
||||||
if (item.note_type === 'task') {
|
if (item.note_type === 'task') {
|
||||||
router.push(`/tasks/${item.id}`);
|
router.push(`/tasks/${item.id}`);
|
||||||
|
} else if (item.note_type === 'snippet') {
|
||||||
|
router.push(`/snippets/${item.id}`);
|
||||||
} else {
|
} else {
|
||||||
router.push(`/notes/${item.id}`);
|
router.push(`/notes/${item.id}`);
|
||||||
}
|
}
|
||||||
@@ -380,14 +418,14 @@ onUnmounted(() => {
|
|||||||
<span v-if="typeCounts.total > 1" class="filter-count">{{ typeCounts.total }}</span>
|
<span v-if="typeCounts.total > 1" class="filter-count">{{ typeCounts.total }}</span>
|
||||||
</button>
|
</button>
|
||||||
<button
|
<button
|
||||||
v-for="[val, label, key] in ([['note','Notes','note'],['task','Tasks','task'],['plan','Plans','plan'],['process','Processes','process']] as [string,string,string][])"
|
v-for="[val, label] in FACET_CHIPS"
|
||||||
:key="val"
|
:key="val"
|
||||||
class="filter-btn"
|
class="filter-btn"
|
||||||
:class="{ active: activeType === val }"
|
:class="{ active: activeType === val }"
|
||||||
@click="activeType = (val as '' | 'note' | 'task' | 'plan' | 'process')"
|
@click="activeType = val"
|
||||||
>
|
>
|
||||||
<span class="filter-btn-label">{{ label }}</span>
|
<span class="filter-btn-label">{{ label }}</span>
|
||||||
<span v-if="typeCounts[key as keyof KnowledgeCounts] > 1" class="filter-count">{{ typeCounts[key as keyof KnowledgeCounts] }}</span>
|
<span v-if="(typeCounts[val] ?? 0) > 1" class="filter-count">{{ typeCounts[val] }}</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -501,6 +539,7 @@ onUnmounted(() => {
|
|||||||
<span v-if="item.note_type === 'note'">Note</span>
|
<span v-if="item.note_type === 'note'">Note</span>
|
||||||
<span v-else-if="item.note_type === 'task'">{{ item.task_kind === 'plan' ? 'Plan' : 'Task' }}</span>
|
<span v-else-if="item.note_type === 'task'">{{ item.task_kind === 'plan' ? 'Plan' : 'Task' }}</span>
|
||||||
<span v-else-if="item.note_type === 'process'">Process</span>
|
<span v-else-if="item.note_type === 'process'">Process</span>
|
||||||
|
<span v-else-if="item.note_type === 'snippet'">Snippet</span>
|
||||||
</span>
|
</span>
|
||||||
<!-- Kind sits BESIDE the type badge, not inside it: the type badge
|
<!-- Kind sits BESIDE the type badge, not inside it: the type badge
|
||||||
speaks the vocabulary of this view's type filter (note / task /
|
speaks the vocabulary of this view's type filter (note / task /
|
||||||
@@ -881,6 +920,14 @@ onUnmounted(() => {
|
|||||||
.badge--note { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: #7A6DA8; }
|
.badge--note { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: #7A6DA8; }
|
||||||
.badge--task { background: rgba(212,160,23,0.15); color: #fbbf24; }
|
.badge--task { background: rgba(212,160,23,0.15); color: #fbbf24; }
|
||||||
.badge--plan { background: rgba(99,102,241,0.18); color: #818cf8; }
|
.badge--plan { background: rgba(99,102,241,0.18); color: #818cf8; }
|
||||||
|
/* Snippet and process are NEUTRAL on purpose. Both were unstyled — and the
|
||||||
|
snippet had no label either, so all 90 of them rendered an empty chip in
|
||||||
|
this feed (#3128). Giving them hues would put a third and fourth colour
|
||||||
|
beside KindBadge's warm/cool pair on the same card; a record type that
|
||||||
|
isn't an alarm reads better as plain. Standard body pair, so the contrast
|
||||||
|
is the one the palette already guarantees. */
|
||||||
|
.badge--snippet,
|
||||||
|
.badge--process { background: var(--fs-surface-raised); color: var(--fs-text-secondary); }
|
||||||
|
|
||||||
.k-card-body { flex: 1; padding-right: 40px; }
|
.k-card-body { flex: 1; padding-right: 40px; }
|
||||||
.k-card-title {
|
.k-card-title {
|
||||||
|
|||||||
@@ -94,8 +94,11 @@ class Note(Base, TimestampMixin, SoftDeleteMixin):
|
|||||||
# name/language/signature/locations live here so they can be INDEXED. The
|
# name/language/signature/locations live here so they can be INDEXED. The
|
||||||
# body keeps the same facts in readable markdown and remains what gets
|
# body keeps the same facts in readable markdown and remains what gets
|
||||||
# embedded; this is a mirror for querying, not the source of truth for
|
# embedded; this is a mirror for querying, not the source of truth for
|
||||||
# display. NULL on every row written before migration 0070, so readers fall
|
# display — and it is DERIVED, so every path that writes a snippet's body
|
||||||
# back to parsing the body (see services/snippets.snippet_fields).
|
# rewrites it too (services/snippets.recompose_data, called from
|
||||||
|
# notes.update_note). 0070 left it NULL on existing rows and
|
||||||
|
# snippets.backfill_snippet_data filled them at startup; readers still fall
|
||||||
|
# back to parsing the body when it is absent (snippet_fields).
|
||||||
data: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
data: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
||||||
|
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Unified Knowledge endpoint — notes, tasks, plans, and processes in one queryable feed."""
|
"""Unified Knowledge endpoint — every record kind in one queryable feed."""
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from quart import Blueprint, jsonify, request
|
from quart import Blueprint, jsonify, request
|
||||||
@@ -6,12 +6,18 @@ from quart import Blueprint, jsonify, request
|
|||||||
from scribe.auth import get_current_user_id, login_required
|
from scribe.auth import get_current_user_id, login_required
|
||||||
from scribe.routes.utils import parse_pagination
|
from scribe.routes.utils import parse_pagination
|
||||||
from scribe.services.access import label_shared_items
|
from scribe.services.access import label_shared_items
|
||||||
|
from scribe.services.knowledge import FACET_TYPES
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
knowledge_bp = Blueprint("knowledge", __name__, url_prefix="/api/knowledge")
|
knowledge_bp = Blueprint("knowledge", __name__, url_prefix="/api/knowledge")
|
||||||
|
|
||||||
_VALID_TYPES = {"note", "task", "plan", "process"}
|
# Derived from the service's facet table, never re-listed here. This set was a
|
||||||
|
# hand-kept copy and had drifted three kinds behind it: it admitted `plan`
|
||||||
|
# (retired in 0066) and rejected `issue` (shipped in 0065, 435 rows) and
|
||||||
|
# `snippet` — so the browse surface could not filter to the kinds it was
|
||||||
|
# already rendering badges for (#3128).
|
||||||
|
_VALID_TYPES = FACET_TYPES
|
||||||
_VALID_SORTS = {"modified", "created", "alpha", "type"}
|
_VALID_SORTS = {"modified", "created", "alpha", "type"}
|
||||||
|
|
||||||
|
|
||||||
@@ -21,7 +27,9 @@ async def list_knowledge():
|
|||||||
"""Return paginated knowledge objects with optional filtering.
|
"""Return paginated knowledge objects with optional filtering.
|
||||||
|
|
||||||
Query params:
|
Query params:
|
||||||
type — one of note|task|plan|process (omit for all)
|
type — a facet from services.knowledge._FACETS: a record type
|
||||||
|
(note|process|snippet) or a task kind (task for any,
|
||||||
|
else work|issue|spike|plan). Omit for all.
|
||||||
tags — comma-separated tag filter (AND logic)
|
tags — comma-separated tag filter (AND logic)
|
||||||
sort — modified|created|alpha|type (default: modified)
|
sort — modified|created|alpha|type (default: modified)
|
||||||
q — search query (semantic when provided, keyword fallback)
|
q — search query (semantic when provided, keyword fallback)
|
||||||
@@ -127,7 +135,7 @@ async def get_knowledge_batch():
|
|||||||
@knowledge_bp.route("/tags", methods=["GET"])
|
@knowledge_bp.route("/tags", methods=["GET"])
|
||||||
@login_required
|
@login_required
|
||||||
async def list_knowledge_tags():
|
async def list_knowledge_tags():
|
||||||
"""Return all tags used across knowledge objects (excludes tasks)."""
|
"""Return all tags used across knowledge objects, narrowed to one facet."""
|
||||||
uid = get_current_user_id()
|
uid = get_current_user_id()
|
||||||
note_type = request.args.get("type", "").strip().lower() or None
|
note_type = request.args.get("type", "").strip().lower() or None
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Knowledge service — unified query across notes, tasks, plans, and processes.
|
"""Knowledge service — one query across every record kind Scribe holds.
|
||||||
|
|
||||||
ACL (rules #47/#78, decision note 2094): these queries were owner-only until
|
ACL (rules #47/#78, decision note 2094): these queries were owner-only until
|
||||||
2026-07-25, which meant a record shared with you could be opened by id but never
|
2026-07-25, which meant a record shared with you could be opened by id but never
|
||||||
@@ -265,22 +265,94 @@ def _note_to_item(note: Note) -> dict:
|
|||||||
return item
|
return item
|
||||||
|
|
||||||
|
|
||||||
def _apply_type_filter(stmt, note_type: str | None):
|
# What each type facet MEANS, once, for every arm that has to know.
|
||||||
"""Apply the type facet to a Note select.
|
#
|
||||||
|
# The vocabulary spans BOTH typing axes — `note_type` for non-task records and
|
||||||
|
# `task_kind` for tasks — so a facet cannot be a filter on one column, which is
|
||||||
|
# why this is a table rather than a chain of ifs. Each entry is
|
||||||
|
# (is_task, the value pinned on that axis); None pins nothing, i.e. every task.
|
||||||
|
#
|
||||||
|
# It is a table because the alternative had already gone wrong. The predicate
|
||||||
|
# was written three times — a SQL if-chain, a Python if-chain over semantic
|
||||||
|
# candidates, and a ternary computing the `is_task` pre-filter — and the three
|
||||||
|
# only agreed by luck. Adding `issue` to the SQL arm alone (the obvious edit,
|
||||||
|
# and the one #3128 was about to make) would have set the pre-filter to
|
||||||
|
# is_task=False, handed the Python arm a candidate set containing no tasks at
|
||||||
|
# all, and returned an empty semantic half for the Issues facet forever, with
|
||||||
|
# nothing red anywhere. A new facet is now one row here.
|
||||||
|
#
|
||||||
|
# `plan` is retired (0066) but kept: 90 legacy plan-tasks exist and a facet
|
||||||
|
# they answer to costs one line. It simply has no chip in the UI any more.
|
||||||
|
_FACETS: dict[str, tuple[bool, str | None]] = {
|
||||||
|
"task": (True, None),
|
||||||
|
"work": (True, "work"),
|
||||||
|
"issue": (True, "issue"),
|
||||||
|
"spike": (True, "spike"),
|
||||||
|
"plan": (True, "plan"),
|
||||||
|
"note": (False, "note"),
|
||||||
|
"process": (False, "process"),
|
||||||
|
"snippet": (False, "snippet"),
|
||||||
|
}
|
||||||
|
|
||||||
'task' = any task (status not null); 'plan' = a task with task_kind='plan';
|
# The non-task record types, for the counts query. Derived so it cannot drift
|
||||||
any other non-empty type = a non-task note of that note_type; None = all.
|
# from the table above.
|
||||||
|
NON_TASK_FACETS = tuple(
|
||||||
|
value for _is_task, value in _FACETS.values() if not _is_task and value
|
||||||
|
)
|
||||||
|
|
||||||
Trashed rows (deleted_at set) are always excluded.
|
# The whole vocabulary, for the door's request validation — public so the route
|
||||||
|
# validates against the same table the query reads instead of a hand-kept copy.
|
||||||
|
FACET_TYPES = frozenset(_FACETS)
|
||||||
|
|
||||||
|
|
||||||
|
# An unrecognised facet resolves to "a non-task note whose note_type is that
|
||||||
|
# string" — which matches nothing, since no row stores an unknown type. That is
|
||||||
|
# the behaviour the old if-chain had by falling through, and it is the right
|
||||||
|
# one: a typo should return an empty list, never the whole corpus.
|
||||||
|
def _facet(note_type: str) -> tuple[bool, str | None]:
|
||||||
|
return _FACETS.get(note_type, (False, note_type))
|
||||||
|
|
||||||
|
|
||||||
|
def facet_is_task(note_type: str | None) -> bool | None:
|
||||||
|
"""The `is_task` pre-filter a facet implies — None when it spans both.
|
||||||
|
|
||||||
|
Used to narrow the semantic candidate set before it is fetched. Reads the
|
||||||
|
same table `_apply_type_filter` and `matches_facet` read, so the pre-filter
|
||||||
|
can no longer disagree with the predicate it is meant to anticipate.
|
||||||
"""
|
"""
|
||||||
|
if not note_type:
|
||||||
|
return None
|
||||||
|
return _facet(note_type)[0]
|
||||||
|
|
||||||
|
|
||||||
|
def matches_facet(note, note_type: str | None) -> bool:
|
||||||
|
"""The Python dialect of `_apply_type_filter`, for candidates the vector
|
||||||
|
search has already fetched — there is no query left to narrow.
|
||||||
|
|
||||||
|
Generated from the same table, so this is a translation rather than a
|
||||||
|
second implementation. Note the `not note.is_task` arm: the hand-written
|
||||||
|
version omitted it and was saved only by the upstream pre-filter.
|
||||||
|
"""
|
||||||
|
if not note_type:
|
||||||
|
return True
|
||||||
|
is_task, value = _facet(note_type)
|
||||||
|
if is_task:
|
||||||
|
return note.is_task and (value is None or note.task_kind == value)
|
||||||
|
return not note.is_task and note.note_type == value
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_type_filter(stmt, note_type: str | None):
|
||||||
|
"""Apply the type facet to a Note select. Trashed rows are always excluded."""
|
||||||
stmt = stmt.where(Note.deleted_at.is_(None))
|
stmt = stmt.where(Note.deleted_at.is_(None))
|
||||||
if note_type == "task":
|
if not note_type:
|
||||||
return stmt.where(Note.status.isnot(None))
|
return stmt
|
||||||
if note_type == "plan":
|
is_task, value = _facet(note_type)
|
||||||
return stmt.where(Note.status.isnot(None)).where(Note.task_kind == "plan")
|
if is_task:
|
||||||
if note_type:
|
stmt = stmt.where(Note.status.isnot(None))
|
||||||
return stmt.where(Note.note_type == note_type).where(Note.status.is_(None))
|
if value is not None:
|
||||||
return stmt
|
stmt = stmt.where(Note.task_kind == value)
|
||||||
|
return stmt
|
||||||
|
return stmt.where(Note.status.is_(None)).where(Note.note_type == value)
|
||||||
|
|
||||||
|
|
||||||
async def query_knowledge(
|
async def query_knowledge(
|
||||||
@@ -295,7 +367,7 @@ async def query_knowledge(
|
|||||||
locations: dict[str, str] | None = None,
|
locations: dict[str, str] | None = None,
|
||||||
verification: str = "",
|
verification: str = "",
|
||||||
) -> tuple[list[dict], int]:
|
) -> tuple[list[dict], int]:
|
||||||
"""Query knowledge objects (non-task notes) with filters.
|
"""Query knowledge objects with filters.
|
||||||
|
|
||||||
`project_id` narrows to one project (None = every project).
|
`project_id` narrows to one project (None = every project).
|
||||||
|
|
||||||
@@ -424,7 +496,7 @@ async def _semantic_knowledge_search(
|
|||||||
INTERACTIVE_SEARCH_THRESHOLD,
|
INTERACTIVE_SEARCH_THRESHOLD,
|
||||||
semantic_search_notes,
|
semantic_search_notes,
|
||||||
)
|
)
|
||||||
is_task_filter = True if note_type in ("task", "plan") else (False if note_type else None)
|
is_task_filter = facet_is_task(note_type)
|
||||||
import time as _time
|
import time as _time
|
||||||
_t0 = _time.perf_counter()
|
_t0 = _time.perf_counter()
|
||||||
candidates = await semantic_search_notes(
|
candidates = await semantic_search_notes(
|
||||||
@@ -454,11 +526,7 @@ async def _semantic_knowledge_search(
|
|||||||
for _score, note in candidates:
|
for _score, note in candidates:
|
||||||
if note.deleted_at is not None:
|
if note.deleted_at is not None:
|
||||||
continue
|
continue
|
||||||
if note_type == "task" and not note.is_task:
|
if not matches_facet(note, note_type):
|
||||||
continue
|
|
||||||
elif note_type == "plan" and (not note.is_task or note.task_kind != "plan"):
|
|
||||||
continue
|
|
||||||
elif note_type and note_type not in ("task", "plan") and note.note_type != note_type:
|
|
||||||
continue
|
continue
|
||||||
if tags and not all(t in (note.tags or []) for t in tags):
|
if tags and not all(t in (note.tags or []) for t in tags):
|
||||||
continue
|
continue
|
||||||
@@ -514,51 +582,40 @@ async def get_knowledge_counts(user_id: int, tags: list[str] | None = None) -> d
|
|||||||
search would surface."""
|
search would surface."""
|
||||||
visible = browsable_notes_clause(user_id)
|
visible = browsable_notes_clause(user_id)
|
||||||
async with async_session() as session:
|
async with async_session() as session:
|
||||||
# Count non-task types
|
def _scoped(stmt):
|
||||||
stmt = (
|
stmt = stmt.where(visible).where(Note.deleted_at.is_(None))
|
||||||
select(Note.note_type, func.count(Note.id))
|
for tag in tags or []:
|
||||||
.where(visible)
|
|
||||||
.where(Note.status.is_(None))
|
|
||||||
.where(Note.deleted_at.is_(None))
|
|
||||||
.where(Note.note_type.in_(["note", "process"]))
|
|
||||||
.group_by(Note.note_type)
|
|
||||||
)
|
|
||||||
if tags:
|
|
||||||
for tag in tags:
|
|
||||||
stmt = stmt.where(Note.tags.contains([tag]))
|
stmt = stmt.where(Note.tags.contains([tag]))
|
||||||
rows = list((await session.execute(stmt)).all())
|
return stmt
|
||||||
counts = {row[0]: row[1] for row in rows}
|
|
||||||
|
|
||||||
# Count tasks separately (is_task = status IS NOT NULL)
|
# One grouped query per typing axis. The task half used to be a count
|
||||||
task_stmt = (
|
# for 'task' plus a second count for 'plan', which is why 'issue' —
|
||||||
select(func.count(Note.id))
|
# 17% of every task here — had no number to show: each kind needed its
|
||||||
.where(visible)
|
# own query and nobody added one. Grouping by task_kind counts every
|
||||||
|
# kind, including ones added later, for the same two round-trips.
|
||||||
|
non_task = _scoped(
|
||||||
|
select(Note.note_type, func.count(Note.id))
|
||||||
|
.where(Note.status.is_(None))
|
||||||
|
.where(Note.note_type.in_(NON_TASK_FACETS))
|
||||||
|
).group_by(Note.note_type)
|
||||||
|
counts = {t: n for t, n in (await session.execute(non_task)).all()}
|
||||||
|
|
||||||
|
by_kind = _scoped(
|
||||||
|
select(Note.task_kind, func.count(Note.id))
|
||||||
.where(Note.status.isnot(None))
|
.where(Note.status.isnot(None))
|
||||||
.where(Note.deleted_at.is_(None))
|
).group_by(Note.task_kind)
|
||||||
)
|
kind_counts = {k: n for k, n in (await session.execute(by_kind)).all()}
|
||||||
if tags:
|
|
||||||
for tag in tags:
|
|
||||||
task_stmt = task_stmt.where(Note.tags.contains([tag]))
|
|
||||||
task_count: int = (await session.execute(task_stmt)).scalar_one()
|
|
||||||
counts["task"] = task_count
|
|
||||||
|
|
||||||
# Plans are a subset of tasks (task_kind='plan'); counted for the facet
|
# Kinds are SUBSETS of 'task' and are deliberately left out of the total —
|
||||||
# but NOT added to total to avoid double-counting against "task".
|
# adding them would count every task twice.
|
||||||
plan_stmt = (
|
counts["task"] = sum(kind_counts.values())
|
||||||
select(func.count(Note.id))
|
for kind, value in _FACETS.items():
|
||||||
.where(visible)
|
if value[0] and value[1] is not None:
|
||||||
.where(Note.status.isnot(None))
|
counts[kind] = kind_counts.get(kind, 0)
|
||||||
.where(Note.task_kind == "plan")
|
|
||||||
.where(Note.deleted_at.is_(None))
|
|
||||||
)
|
|
||||||
if tags:
|
|
||||||
for tag in tags:
|
|
||||||
plan_stmt = plan_stmt.where(Note.tags.contains([tag]))
|
|
||||||
counts["plan"] = (await session.execute(plan_stmt)).scalar_one()
|
|
||||||
|
|
||||||
for t in ("note", "task", "plan", "process"):
|
for t in NON_TASK_FACETS:
|
||||||
counts.setdefault(t, 0)
|
counts.setdefault(t, 0)
|
||||||
counts["total"] = sum(counts[t] for t in ("note", "task", "process"))
|
counts["total"] = counts["task"] + sum(counts[t] for t in NON_TASK_FACETS)
|
||||||
return counts
|
return counts
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -9,6 +9,13 @@ from scribe.models.note import Note, TaskKind, TaskPriority, TaskStatus
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# The fields `snippets.parse_snippet_fields` reads. Writing any of them can
|
||||||
|
# change what a snippet's derived `data` mirror should say, so update_note
|
||||||
|
# recomposes the mirror when one moves. Kept here as a set of NAMES rather
|
||||||
|
# than imported, because it describes update_note's own `fields` dict, not the
|
||||||
|
# parser's signature.
|
||||||
|
_PARSED_FROM_BODY = frozenset({"title", "body", "tags"})
|
||||||
|
|
||||||
|
|
||||||
def embed_note(note) -> None:
|
def embed_note(note) -> None:
|
||||||
"""Refresh a note's embedding, fire-and-forget.
|
"""Refresh a note's embedding, fire-and-forget.
|
||||||
@@ -439,6 +446,23 @@ async def update_note(user_id: int, note_id: int, **fields: object) -> Note | No
|
|||||||
elif key == "tags" and isinstance(value, list):
|
elif key == "tags" and isinstance(value, list):
|
||||||
value = _normalize_tags(value)
|
value = _normalize_tags(value)
|
||||||
setattr(note, key, value)
|
setattr(note, key, value)
|
||||||
|
# A snippet's `data` is DERIVED from its body — so a write that moves
|
||||||
|
# the body through this generic door must move the mirror with it
|
||||||
|
# (#3128). Without this, PATCH /api/notes/<snippet_id> {body} left the
|
||||||
|
# mirror behind, and snippet_fields PREFERS the mirror: the row went on
|
||||||
|
# reporting its old repo/path/symbol to prior-art recall while showing
|
||||||
|
# its new body. `update_snippet` composes the mirror itself and passes
|
||||||
|
# it explicitly, so an explicit `data` always wins — the caller that
|
||||||
|
# knows the field set beats the one that can only re-read the body.
|
||||||
|
if "data" not in fields and not _PARSED_FROM_BODY.isdisjoint(fields):
|
||||||
|
# Imported here, not at module scope: services/snippets.py calls
|
||||||
|
# back into this module (update_snippet -> update_note), so a
|
||||||
|
# top-level import is a cycle.
|
||||||
|
from scribe.services.snippets import (
|
||||||
|
SNIPPET_NOTE_TYPE, recompose_data,
|
||||||
|
)
|
||||||
|
if note.note_type == SNIPPET_NOTE_TYPE:
|
||||||
|
note.data = recompose_data(note)
|
||||||
# Auto-set lifecycle timestamps on status transitions
|
# Auto-set lifecycle timestamps on status transitions
|
||||||
if "status" in fields:
|
if "status" in fields:
|
||||||
_now = datetime.now(timezone.utc)
|
_now = datetime.now(timezone.utc)
|
||||||
|
|||||||
@@ -540,14 +540,60 @@ def compose_data(
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def recompose_data(note) -> dict:
|
||||||
|
"""Rebuild a snippet's `data` mirror from its own body, title and tags.
|
||||||
|
|
||||||
|
For the GENERIC note door. `update_snippet` composes the mirror itself from
|
||||||
|
the field set it just merged and never needs this; a plain
|
||||||
|
`update_note(body=...)` — which the Knowledge feed's editor issues, because
|
||||||
|
a snippet card there routes to /notes/:id — has no idea the mirror exists,
|
||||||
|
and left it stale. `snippet_fields` then PREFERS the stale mirror, so the
|
||||||
|
row reported its old repo/path/symbol to prior-art recall while displaying
|
||||||
|
its new body: confidently wrong, which is worse than no record (#3128).
|
||||||
|
|
||||||
|
The body is the authority; the mirror is derived. That is already the rule
|
||||||
|
this file states — it just had no enforcement on the path that bypasses
|
||||||
|
`update_snippet`.
|
||||||
|
|
||||||
|
`verification` and `provenance` are CARRIED, not recomposed, because
|
||||||
|
neither is in the body to parse — the same carry `compose_data` does for
|
||||||
|
the snippet service's own writes. Note that a verdict does not need
|
||||||
|
invalidating here: `code_sha` is recomputed from the new code, so a stale
|
||||||
|
verdict expires itself on read exactly as it does after any other edit.
|
||||||
|
"""
|
||||||
|
parsed = parse_snippet_fields(note.title, note.body, note.tags)
|
||||||
|
prior = note.data or {}
|
||||||
|
return compose_data(
|
||||||
|
name=parsed["name"],
|
||||||
|
when_to_use=parsed["when_to_use"],
|
||||||
|
signature=parsed["signature"],
|
||||||
|
language=parsed["language"],
|
||||||
|
code=parsed["code"],
|
||||||
|
locations=parsed["locations"],
|
||||||
|
merged_from=parsed["merged_from"],
|
||||||
|
verification=prior.get("verification"),
|
||||||
|
provenance=prior.get("provenance"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def snippet_fields(note) -> dict:
|
def snippet_fields(note) -> dict:
|
||||||
"""Structured fields for a snippet, preferring the indexed `data` column and
|
"""Structured fields for a snippet, preferring the indexed `data` column and
|
||||||
falling back to parsing the body.
|
falling back to parsing the body.
|
||||||
|
|
||||||
Both paths must agree, because rows written before migration 0070 have no
|
THE BODY IS THE AUTHORITY; `data` is a mirror derived from it. Every writer
|
||||||
`data` and are never backfilled — a hand-edited body is the authority for
|
keeps them in step — the snippet service composes the mirror from the field
|
||||||
those, and there is no deadline by which they must be converted. `code` only
|
set it just merged, `update_note` recomposes it when a body reaches the
|
||||||
ever comes from the body, since `data` doesn't carry it.
|
generic door (#3128), and `backfill_snippet_data` filled the pre-0070 rows
|
||||||
|
at startup. The fallback below is therefore a belt to that braces, not a
|
||||||
|
second source of truth: it is what a row looks like before the backfill has
|
||||||
|
run, and it must keep agreeing with the mirror.
|
||||||
|
|
||||||
|
(This docstring used to say those rows were "never backfilled". That was
|
||||||
|
true when 0070 landed and stopped being true when the backfill shipped; it
|
||||||
|
is corrected here because the sentence read as licence for a stale mirror,
|
||||||
|
which is exactly the bug #3128 found.)
|
||||||
|
|
||||||
|
`code` only ever comes from the body, since `data` doesn't carry it.
|
||||||
"""
|
"""
|
||||||
parsed = parse_snippet_fields(note.title, note.body, note.tags)
|
parsed = parse_snippet_fields(note.title, note.body, note.tags)
|
||||||
stored = getattr(note, "data", None)
|
stored = getattr(note, "data", None)
|
||||||
|
|||||||
@@ -12,6 +12,16 @@ from types import SimpleNamespace
|
|||||||
from unittest.mock import AsyncMock, MagicMock
|
from unittest.mock import AsyncMock, MagicMock
|
||||||
|
|
||||||
|
|
||||||
|
def compiled_sql(element) -> str:
|
||||||
|
"""A SQLAlchemy clause or statement rendered as literal SQL text.
|
||||||
|
|
||||||
|
For asserting on the shape of a predicate without a database — which is how
|
||||||
|
the visibility clauses and the knowledge facets are both tested. Was a
|
||||||
|
private copy in each of those modules before #3128 needed a third.
|
||||||
|
"""
|
||||||
|
return str(element.compile(compile_kwargs={"literal_binds": True}))
|
||||||
|
|
||||||
|
|
||||||
def make_mock_session() -> AsyncMock:
|
def make_mock_session() -> AsyncMock:
|
||||||
"""A stand-in for ``async_session()`` — usable as ``async with``, with the
|
"""A stand-in for ``async_session()`` — usable as ``async with``, with the
|
||||||
commit/refresh/add surface a service touches.
|
commit/refresh/add surface a service touches.
|
||||||
|
|||||||
@@ -0,0 +1,120 @@
|
|||||||
|
"""The type facet, in both of its dialects.
|
||||||
|
|
||||||
|
The facet predicate is written twice by necessity — as SQL for rows the
|
||||||
|
database hands back, and as Python for candidates the vector search has
|
||||||
|
already fetched — plus a third time as the `is_task` pre-filter that narrows
|
||||||
|
the candidate set before it exists. Those three used to be hand-written and
|
||||||
|
agreed only by luck: adding `issue` to the SQL arm alone would have set the
|
||||||
|
pre-filter to is_task=False, handed the Python arm a candidate set with no
|
||||||
|
tasks in it, and returned an empty semantic half for the Issues facet forever
|
||||||
|
with nothing red anywhere (#3128).
|
||||||
|
|
||||||
|
They are now generated from one table. These tests pin the property that made
|
||||||
|
the trap possible, so it cannot come back by a different route.
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from scribe.services.knowledge import (
|
||||||
|
_FACETS,
|
||||||
|
FACET_TYPES,
|
||||||
|
NON_TASK_FACETS,
|
||||||
|
_apply_type_filter,
|
||||||
|
facet_is_task,
|
||||||
|
matches_facet,
|
||||||
|
)
|
||||||
|
from tests.helpers import compiled_sql, fake_note, fake_snippet, fake_task
|
||||||
|
|
||||||
|
|
||||||
|
def _sql(note_type):
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from scribe.models.note import Note
|
||||||
|
|
||||||
|
return compiled_sql(_apply_type_filter(select(Note.id), note_type))
|
||||||
|
|
||||||
|
|
||||||
|
# One representative row per shape the corpus actually holds.
|
||||||
|
ROWS = {
|
||||||
|
"plain note": fake_note(note_type="note"),
|
||||||
|
"process": fake_note(note_type="process"),
|
||||||
|
"snippet": fake_snippet(),
|
||||||
|
"work task": fake_task(task_kind="work", note_type="note"),
|
||||||
|
"issue": fake_task(task_kind="issue", note_type="note"),
|
||||||
|
"spike": fake_task(task_kind="spike", note_type="note"),
|
||||||
|
"legacy plan": fake_task(task_kind="plan", note_type="note"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("facet", sorted(FACET_TYPES))
|
||||||
|
def test_pre_filter_never_excludes_a_row_the_facet_wants(facet):
|
||||||
|
"""THE regression. `facet_is_task` narrows the semantic candidate set before
|
||||||
|
`matches_facet` ever sees it, so a pre-filter that disagrees with the
|
||||||
|
predicate doesn't return wrong rows — it returns NO rows, silently, on one
|
||||||
|
half of a hybrid search."""
|
||||||
|
want = facet_is_task(facet)
|
||||||
|
for label, row in ROWS.items():
|
||||||
|
if matches_facet(row, facet):
|
||||||
|
assert want is None or want == row.is_task, (
|
||||||
|
f"facet {facet!r} accepts the {label} row, but its pre-filter "
|
||||||
|
f"asks for is_task={want} while the row has is_task={row.is_task} "
|
||||||
|
f"— the semantic arm would never be handed this row"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"facet,expected",
|
||||||
|
[
|
||||||
|
("task", {"work task", "issue", "spike", "legacy plan"}),
|
||||||
|
("issue", {"issue"}),
|
||||||
|
("spike", {"spike"}),
|
||||||
|
("work", {"work task"}),
|
||||||
|
("plan", {"legacy plan"}),
|
||||||
|
("note", {"plain note"}),
|
||||||
|
("process", {"process"}),
|
||||||
|
("snippet", {"snippet"}),
|
||||||
|
("", set(ROWS)),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_each_facet_selects_exactly_its_own_rows(facet, expected):
|
||||||
|
assert {k for k, row in ROWS.items() if matches_facet(row, facet)} == expected
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_plain_note_is_not_selected_by_its_own_type_when_it_is_a_task():
|
||||||
|
"""A task's `note_type` is 'note' — that column says nothing about task-ness.
|
||||||
|
The hand-written Python arm omitted the `status IS NULL` half its SQL twin
|
||||||
|
carried, so it only avoided returning every task under the Notes facet
|
||||||
|
because the pre-filter had already dropped them."""
|
||||||
|
assert matches_facet(fake_task(note_type="note"), "note") is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_unknown_facet_matches_nothing_rather_than_everything():
|
||||||
|
"""A typo must return an empty list, never the whole corpus."""
|
||||||
|
assert all(not matches_facet(row, "wrok") for row in ROWS.values())
|
||||||
|
assert "notes.status IS NULL" in _sql("wrok")
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_live_task_kinds_are_all_facets():
|
||||||
|
"""`issue` shipped in 0065 and `spike` in 0091; the browse vocabulary went
|
||||||
|
three kinds without noticing either."""
|
||||||
|
for kind in ("work", "issue", "spike"):
|
||||||
|
assert kind in FACET_TYPES and _FACETS[kind][0] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_task_facets_are_the_note_types_and_only_those():
|
||||||
|
assert set(NON_TASK_FACETS) == {"note", "process", "snippet"}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("facet", sorted(FACET_TYPES))
|
||||||
|
def test_sql_arm_constrains_the_axis_the_facet_lives_on(facet):
|
||||||
|
"""The SQL dialect of the same table. A task facet must pin `status` (and,
|
||||||
|
for a single kind, `task_kind`); a record-type facet must pin `note_type`
|
||||||
|
AND exclude tasks."""
|
||||||
|
sql = _sql(facet)
|
||||||
|
is_task, value = _FACETS[facet]
|
||||||
|
assert "notes.deleted_at IS NULL" in sql
|
||||||
|
if is_task:
|
||||||
|
assert "notes.status IS NOT NULL" in sql
|
||||||
|
assert (f"notes.task_kind = '{value}'" in sql) is (value is not None)
|
||||||
|
else:
|
||||||
|
assert "notes.status IS NULL" in sql
|
||||||
|
assert f"notes.note_type = '{value}'" in sql
|
||||||
@@ -20,10 +20,9 @@ from scribe.services.access import (
|
|||||||
notes_visibility_clause,
|
notes_visibility_clause,
|
||||||
readable_notes_clause,
|
readable_notes_clause,
|
||||||
)
|
)
|
||||||
|
from tests.helpers import compiled_sql
|
||||||
|
|
||||||
|
_sql = compiled_sql
|
||||||
def _sql(clause) -> str:
|
|
||||||
return str(clause.compile(compile_kwargs={"literal_binds": True}))
|
|
||||||
|
|
||||||
|
|
||||||
def _read(user_id: int = 7) -> str:
|
def _read(user_id: int = 7) -> str:
|
||||||
|
|||||||
@@ -1,4 +1,12 @@
|
|||||||
"""get_knowledge_counts includes the 'process' type and counts it in total."""
|
"""get_knowledge_counts — one number per facet, and an honest total.
|
||||||
|
|
||||||
|
Two grouped queries, one per typing axis. It used to be three: a grouped count
|
||||||
|
over note_type restricted to ("note", "process"), a scalar count of tasks, and
|
||||||
|
a second scalar just for plans. That shape is why `issue` had no number —
|
||||||
|
every kind needed a query of its own and nobody added one — and why the "All"
|
||||||
|
chip sat ~90 below the list it labelled, since snippets were in the feed but
|
||||||
|
in no count (#3128).
|
||||||
|
"""
|
||||||
from unittest.mock import AsyncMock, MagicMock, patch
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -11,29 +19,68 @@ def _grouped(rows):
|
|||||||
return r
|
return r
|
||||||
|
|
||||||
|
|
||||||
def _scalar(n):
|
async def _counts(non_task_rows, kind_rows, **kwargs):
|
||||||
r = MagicMock()
|
|
||||||
r.scalar_one.return_value = n
|
|
||||||
return r
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_counts_include_process_in_facet_and_total():
|
|
||||||
session = make_mock_session()
|
session = make_mock_session()
|
||||||
# 1) grouped non-task counts, 2) task count, 3) plan count
|
# 1) non-task rows grouped by note_type, 2) task rows grouped by task_kind
|
||||||
session.execute = AsyncMock(side_effect=[
|
session.execute = AsyncMock(
|
||||||
_grouped([("note", 3), ("process", 2)]),
|
side_effect=[_grouped(non_task_rows), _grouped(kind_rows)]
|
||||||
_scalar(1), # tasks
|
)
|
||||||
_scalar(0), # plans
|
|
||||||
])
|
|
||||||
with patch("scribe.services.knowledge.async_session") as cls:
|
with patch("scribe.services.knowledge.async_session") as cls:
|
||||||
cls.return_value = session
|
cls.return_value = session
|
||||||
from scribe.services.knowledge import get_knowledge_counts
|
from scribe.services.knowledge import get_knowledge_counts
|
||||||
counts = await get_knowledge_counts(user_id=1)
|
return await get_knowledge_counts(user_id=1, **kwargs), session
|
||||||
|
|
||||||
assert counts["process"] == 2
|
|
||||||
# facet keys all present (setdefault)
|
@pytest.mark.asyncio
|
||||||
for key in ("note", "task", "plan", "process"):
|
async def test_every_facet_gets_a_number_including_the_kinds():
|
||||||
assert key in counts
|
counts, _ = await _counts(
|
||||||
# total = note(3) + task(1) + process(2)
|
[("note", 395), ("process", 3), ("snippet", 90)],
|
||||||
assert counts["total"] == 6
|
[("work", 2104), ("issue", 435), ("spike", 1), ("plan", 90)],
|
||||||
|
)
|
||||||
|
assert counts["note"] == 395
|
||||||
|
assert counts["process"] == 3
|
||||||
|
assert counts["snippet"] == 90
|
||||||
|
assert counts["issue"] == 435
|
||||||
|
assert counts["spike"] == 1
|
||||||
|
assert counts["plan"] == 90
|
||||||
|
assert counts["work"] == 2104
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_task_is_the_sum_of_its_kinds():
|
||||||
|
"""`task` is not counted separately any more — it is what the kinds add up
|
||||||
|
to, so the two can't disagree."""
|
||||||
|
counts, _ = await _counts([], [("work", 2104), ("issue", 435), ("spike", 1)])
|
||||||
|
assert counts["task"] == 2540
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_total_counts_snippets_and_counts_no_task_twice():
|
||||||
|
"""The All chip labels a feed that contains every kind, so it has to count
|
||||||
|
every kind — and exactly once. Kinds are subsets of `task`; adding them
|
||||||
|
would count each issue a second time."""
|
||||||
|
counts, _ = await _counts(
|
||||||
|
[("note", 10), ("process", 2), ("snippet", 5)],
|
||||||
|
[("work", 20), ("issue", 4)],
|
||||||
|
)
|
||||||
|
assert counts["task"] == 24
|
||||||
|
assert counts["total"] == 10 + 2 + 5 + 24
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_absent_facets_report_zero_rather_than_missing():
|
||||||
|
counts, _ = await _counts([("note", 1)], [])
|
||||||
|
assert counts["note"] == 1
|
||||||
|
for key in ("process", "snippet", "task", "work", "issue", "spike", "plan"):
|
||||||
|
assert counts[key] == 0, key
|
||||||
|
assert counts["total"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_a_tag_filter_narrows_both_axes():
|
||||||
|
"""A tag has to reach both queries, or the chips would disagree with each
|
||||||
|
other under a filter — tasks narrowed, notes not."""
|
||||||
|
_, session = await _counts([], [], tags=["python"])
|
||||||
|
assert session.execute.await_count == 2
|
||||||
|
for call in session.execute.await_args_list:
|
||||||
|
assert "notes.tags" in str(call.args[0])
|
||||||
|
|||||||
@@ -0,0 +1,111 @@
|
|||||||
|
"""A snippet's `data` mirror survives the GENERIC note door.
|
||||||
|
|
||||||
|
`notes.data` is derived from the body. The snippet service always composed it
|
||||||
|
from the field set it had just merged, so `update_snippet` was never the
|
||||||
|
problem — the problem was every other way a snippet's body could be written.
|
||||||
|
`update_note` is a `hasattr` loop with no snippet awareness, and both doors
|
||||||
|
reach it: PATCH /api/notes/<id> and the MCP update_note tool. The Knowledge
|
||||||
|
feed handed you that path, because a snippet card there routed to /notes/:id.
|
||||||
|
|
||||||
|
The failure was silent and the wrong way round: `snippet_fields` PREFERS the
|
||||||
|
mirror, so the row went on reporting its old repo/path/symbol to the location
|
||||||
|
reverse lookup and to prior-art recall while displaying its new body — a record
|
||||||
|
surfaced with full authority and wrong, which the drift-check docstring calls
|
||||||
|
worse than having no record at all (#3128).
|
||||||
|
"""
|
||||||
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from tests.helpers import fake_note, fake_snippet, make_mock_session
|
||||||
|
|
||||||
|
OLD_MIRROR = {
|
||||||
|
"name": "debounce",
|
||||||
|
"language": "javascript",
|
||||||
|
"locations": [{"repo": "Scribe", "path": "old/place.js", "symbol": "debounce"}],
|
||||||
|
"verification": {"status": "ok", "code_sha": "abc", "checked_at": "2026-01-01"},
|
||||||
|
"provenance": {"commit_sha": "deadbeef"},
|
||||||
|
}
|
||||||
|
|
||||||
|
MOVED_BODY = (
|
||||||
|
"**Locations:**\n"
|
||||||
|
"- `Scribe` · `new/place.ts` · `debounce`\n\n"
|
||||||
|
"```typescript\nexport const debounce = 1;\n```\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _update(note, **fields):
|
||||||
|
session = make_mock_session()
|
||||||
|
result = MagicMock()
|
||||||
|
result.scalars.return_value.first.return_value = note
|
||||||
|
session.execute = AsyncMock(return_value=result)
|
||||||
|
with patch("scribe.services.notes.async_session") as cls, \
|
||||||
|
patch("scribe.services.notes.embed_note", MagicMock()), \
|
||||||
|
patch("scribe.services.notes._maybe_reactivate_project", AsyncMock()), \
|
||||||
|
patch("scribe.services.note_versions.create_version", AsyncMock()):
|
||||||
|
cls.return_value = session
|
||||||
|
from scribe.services.notes import update_note
|
||||||
|
await update_note(user_id=7, note_id=note.id, **fields)
|
||||||
|
return note
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_a_body_write_moves_the_mirror_with_it():
|
||||||
|
note = fake_snippet(data=dict(OLD_MIRROR), project_id=None)
|
||||||
|
await _update(note, body=MOVED_BODY)
|
||||||
|
assert note.data["locations"] == [
|
||||||
|
{"repo": "Scribe", "path": "new/place.ts", "symbol": "debounce"}
|
||||||
|
], "the mirror still describes where the snippet used to live"
|
||||||
|
assert note.data["language"] == "typescript"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_the_verdict_and_provenance_are_carried_not_dropped():
|
||||||
|
"""Neither is in the body to parse, so recomposing must carry them. An
|
||||||
|
ordinary edit must not erase the last drift check — and it needs no
|
||||||
|
invalidation branch either: `code_sha` is recomputed from the new code, so
|
||||||
|
a verdict stamped against the old code expires itself on read."""
|
||||||
|
note = fake_snippet(data=dict(OLD_MIRROR), project_id=None)
|
||||||
|
await _update(note, body=MOVED_BODY)
|
||||||
|
assert note.data["verification"] == OLD_MIRROR["verification"]
|
||||||
|
assert note.data["provenance"] == OLD_MIRROR["provenance"]
|
||||||
|
assert note.data["code_sha"] != OLD_MIRROR["verification"]["code_sha"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_an_explicit_data_wins_over_recomposition():
|
||||||
|
"""`update_snippet` composes the mirror from the merged field set it holds
|
||||||
|
and passes it here. That caller knows things the body cannot be re-read for
|
||||||
|
— which locations were replaced, whether provenance survives the edit — so
|
||||||
|
an explicit mirror must not be recomputed out from under it."""
|
||||||
|
note = fake_snippet(data=dict(OLD_MIRROR), project_id=None)
|
||||||
|
authoritative = {"name": "from the service", "locations": []}
|
||||||
|
await _update(note, body=MOVED_BODY, data=authoritative)
|
||||||
|
assert note.data == authoritative
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_a_plain_note_is_left_alone():
|
||||||
|
"""Only snippets carry a mirror; a note's `data` must not be invented."""
|
||||||
|
note = fake_note(note_type="note", data=None, project_id=None)
|
||||||
|
await _update(note, body="just some prose")
|
||||||
|
assert note.data is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_a_write_that_cannot_change_the_parse_does_not_touch_the_mirror():
|
||||||
|
"""Status, priority, project — none of them is an input to the body parser,
|
||||||
|
so recomposing on them would be work for nothing and would rebuild a mirror
|
||||||
|
from a body nobody claimed to have changed."""
|
||||||
|
note = fake_snippet(data=dict(OLD_MIRROR), project_id=None)
|
||||||
|
await _update(note, project_id=4)
|
||||||
|
assert note.data == OLD_MIRROR
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_a_title_change_reaches_the_mirror_too():
|
||||||
|
"""A snippet's NAME lives in its title, not its body — `parse_snippet_fields`
|
||||||
|
reads both, so both are triggers."""
|
||||||
|
note = fake_snippet(data=dict(OLD_MIRROR), project_id=None)
|
||||||
|
await _update(note, title="throttle — cap a callback's rate")
|
||||||
|
assert note.data["name"] == "throttle"
|
||||||
|
assert note.data["when_to_use"] == "cap a callback's rate"
|
||||||
Reference in New Issue
Block a user