Compare commits

...
3 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 5605b863b2 docs(6205): lesson_rules comments state the reason, not the instance that taught it
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / Python tests (push) Canceled after 1m48s
Issue, milestone, snippet and rule-number citations are removed; the scope
docstring states the leak it prevents without naming the milestone that
closed it for the direct arms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:23:11 -04:00
bvandeusenandClaude Opus 5.5 21a409eac1 docs(6205): knowledge comments state the reason, not the instance that taught it
The ACL header states the share behaviour without its date and decision ids;
the facet table, door map, floor constant and facet-count comments state why
they are single sources instead of retelling how copies drifted; this
instance's counts (legacy plan-tasks, issue share) are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:22:42 -04:00
bvandeusenandClaude Opus 5.5 d1b442c572 docs(6205): lessons comments state the reason, not the instance that taught it
The module docstring keeps why the kind exists, where its trigger lives, what
it inherits and why it needs no migration, without milestone, decision and
note ids or this instance's snippet counts. The body-strip docstring states
why taking the note matters instead of retelling the defect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:21:57 -04:00
3 changed files with 142 additions and 164 deletions
+42 -52
View File
@@ -1,8 +1,7 @@
"""Knowledge service — one query across every record kind Scribe holds.
ACL (rules #47/#78, decision note 2094): these queries were owner-only until
2026-07-25, which meant a record shared with you could be opened by id but never
*found*. They now honour shares — at two different widths:
ACL (services/access.py): these queries honour shares, so a record shared with
you can be *found* as well as opened by id — at two different widths:
- **searching** (a `q` the caller typed) uses the full read scope, so a record
shared directly with you is findable when you go looking for it. BOTH halves
@@ -32,7 +31,7 @@ logger = logging.getLogger(__name__)
_SNIPPET_LEN = 200
# --- the location filter (reverse lookup, #2083) ------------------------------
# --- the location filter (reverse lookup) ------------------------------------
#
# ONE predicate in two dialects: "this record carries a location matching every
# part asked for." The SQL form runs in the browse arm and the keyword arm,
@@ -79,7 +78,7 @@ def location_matches(data: dict | None, parts: dict[str, str]) -> bool:
_path_matches((loc.get(key) or "").strip(), want)
if key == "path"
# Repo names are recorded free-form ("Scribe" / "FabledScribe" /
# "fabledscribe") — case is never the distinguishing thing (#2874).
# "fabledscribe") — case is never the distinguishing thing.
else (loc.get(key) or "").strip().lower() == want.lower()
if key == "repo"
else (loc.get(key) or "").strip() == want
@@ -106,7 +105,7 @@ def location_jsonpath(parts: dict[str, str]) -> str:
prefix = json.dumps(want.rstrip("/") + "/")
filters.append(f"(@.path == {literal} || @.path starts with {prefix})")
elif key == "repo":
# Case-insensitive, anchored, regex-escaped (#2874) — mirrors the
# Case-insensitive, anchored, regex-escaped — mirrors the
# Python dialect's .lower() compare.
pattern = json.dumps("^" + re.escape(want) + "$")
filters.append(f'(@.repo like_regex {pattern} flag "i")')
@@ -120,7 +119,7 @@ def _location_clause(parts: dict[str, str]):
return Note.data.path_exists(location_jsonpath(parts))
# --- drift-check filter (#2086) ----------------------------------------------
# --- drift-check filter ------------------------------------------------------
# `verification` selects on the drift-check verdict stored in `data.verification`
# (see services/snippets.py). Statuses are the service's own constants; the two
# composite values are what the operator actually asks for.
@@ -154,7 +153,7 @@ def verification_matches(data: dict | None, value: str) -> bool:
if not status:
return want == "unverified"
expired = verdict.get("code_sha") != (data or {}).get("code_sha")
# A push touched the recorded location since the verdict (#2691): the repo
# A push touched the recorded location since the verdict: the repo
# moved under it, so it needs a look even though it hasn't failed.
invalidated = bool(verdict.get("invalidated_by"))
if want == "unverified":
@@ -171,7 +170,7 @@ def verification_matches(data: dict | None, value: str) -> bool:
_VERIFY_DRIFTED_JSONPATH = '$.verification ? (@.status != "ok")'
_VERIFY_EXPIRED_JSONPATH = "$ ? (@.verification.code_sha != @.code_sha)"
_VERIFY_ANY_JSONPATH = "$.verification"
# A push touched the recorded location since the verdict (#2691).
# A push touched the recorded location since the verdict.
_VERIFY_INVALIDATED_JSONPATH = "$.verification.invalidated_by"
@@ -189,7 +188,7 @@ def _verification_clause(value: str):
return Note.data.path_exists(_VERIFY_DRIFTED_JSONPATH)
if want == "attention":
# Everything worth looking at: a failing verdict, an expired one, OR
# one whose recorded location a push has since touched (#2691).
# one whose recorded location a push has since touched.
return or_(
Note.data.path_exists(_VERIFY_DRIFTED_JSONPATH),
and_(has_verdict, Note.data.path_exists(_VERIFY_EXPIRED_JSONPATH)),
@@ -221,10 +220,9 @@ def _note_to_item(note: Note, chunks: dict[int, dict] | None = None) -> dict:
any other — it shows the record's opening. `snippet_is` says which, on
every row, so the two never have to be told apart by guessing.
This was `(note.body or "")[:_SNIPPET_LEN]` on every row alike: the head of
the document, on the human's main search surface, with nothing marking the
cut. A record matched on its sixth paragraph was shown its first, which the
search had already scored lower (#4243).
The head of the body on every row alike (`body[:_SNIPPET_LEN]`) would show
a record matched on its sixth paragraph its first, which the search had
already scored lower, with nothing marking the cut.
"""
item: dict = {
"id": note.id,
@@ -244,7 +242,7 @@ def _note_to_item(note: Note, chunks: dict[int, dict] | None = None) -> dict:
"created_at": iso(note.created_at),
"updated_at": iso(note.updated_at),
}
# Drift verdict (#2086), when one has been recorded. Included here rather
# Drift verdict, when one has been recorded. Included here rather
# than decorated on by the snippet layer because `current` is derivable from
# `data` alone — the verdict's code_sha against the row's — so this needs no
# body parsing and stays a plain projection of the column. Omitted entirely
@@ -253,7 +251,7 @@ def _note_to_item(note: Note, chunks: dict[int, dict] | None = None) -> dict:
# Snippet language, same reasoning as the verdict below: a plain projection of
# the `data` mirror, no body parsing. Needed because a prior-art hit in a
# DIFFERENT language than the file being written is useful as the shape of a
# solution but must not be mistaken for code to paste (#2244) — and the caller
# solution but must not be mistaken for code to paste — and the caller
# can only say "different" if the language is on the item.
language = (note.data or {}).get("language") if note.data else None
if language:
@@ -267,8 +265,8 @@ def _note_to_item(note: Note, chunks: dict[int, dict] | None = None) -> dict:
trigger = (note.data or {}).get("when_to_apply") if note.data else None
if trigger:
item["when_to_apply"] = trigger
# A snippet's, for the same reason — and since milestone 427 the title no
# longer carries it, so without this a list shows names with no situation.
# A snippet's, for the same reason — the stored title does not carry it,
# so without this a list shows names with no situation.
usage = (note.data or {}).get("when_to_use") if note.data else None
if usage:
item["when_to_use"] = usage
@@ -283,7 +281,7 @@ def _note_to_item(note: Note, chunks: dict[int, dict] | None = None) -> dict:
"path": verdict.get("path"),
}
# Present only when a push has touched the recorded location since the
# verdict (#2691) — the "recheck me" marker, cleared by re-verifying.
# verdict — the "recheck me" marker, cleared by re-verifying.
if verdict.get("invalidated_by"):
item["verification"]["invalidated_by"] = verdict["invalidated_by"]
@@ -305,17 +303,15 @@ def _note_to_item(note: Note, chunks: dict[int, dict] | None = None) -> dict:
# why this is a table rather than a chain of ifs. Each entry is
# (is_task, the value pinned on that axis); None pins nothing, i.e. every task.
#
# It is a table because the alternative had already gone wrong. The predicate
# was written three times — a SQL if-chain, a Python if-chain over semantic
# candidates, and a ternary computing the `is_task` pre-filter — and the three
# only agreed by luck. Adding `issue` to the SQL arm alone (the obvious edit,
# and the one #3128 was about to make) would have set the pre-filter to
# is_task=False, handed the Python arm a candidate set containing no tasks at
# all, and returned an empty semantic half for the Issues facet forever, with
# nothing red anywhere. A new facet is now one row here.
# It is a table because the predicate has three dialects — SQL, a Python test
# over semantic candidates, and the `is_task` pre-filter — and three hand-kept
# copies agree only by luck. Adding `issue` to the SQL arm alone would set the
# pre-filter to is_task=False, hand the Python arm a candidate set containing
# no tasks at all, and return an empty semantic half for the Issues facet
# forever, with nothing red anywhere. A new facet is one row here.
#
# `plan` is retired (0066) but kept: 90 legacy plan-tasks exist and a facet
# they answer to costs one line. It simply has no chip in the UI any more.
# `plan` is retired but kept: legacy plan-tasks may exist, and a facet they
# answer to costs one line. It has no chip in the UI.
_FACETS: dict[str, tuple[bool, str | None]] = {
"task": (True, None),
"work": (True, "work"),
@@ -326,11 +322,10 @@ _FACETS: dict[str, tuple[bool, str | None]] = {
"process": (False, "process"),
"snippet": (False, "snippet"),
# A transferable insight, keyed by the situation it applies to rather than
# by its topic (milestone 385). It reaches the browse surface through this
# one entry: the door's validation, the counts and both dialects of the
# type filter are all generated from this table, which is the property
# #3161 asked for so that adding a kind is a single edit rather than four
# that must agree.
# by its topic. It reaches the browse surface through this one entry: the
# door's validation, the counts and both dialects of the type filter are
# all generated from this table, so adding a kind is a single edit rather
# than four that must agree.
"lesson": (False, "lesson"),
}
@@ -358,7 +353,7 @@ def facet_is_task(note_type: str | None) -> bool | None:
Used to narrow the semantic candidate set before it is fetched. Reads the
same table `_apply_type_filter` and `matches_facet` read, so the pre-filter
can no longer disagree with the predicate it is meant to anticipate.
cannot disagree with the predicate it is meant to anticipate.
"""
if not note_type:
return None
@@ -388,7 +383,7 @@ def search_filters_for(facet: str) -> dict:
narrow and no fetched row to test — it is passing filters INTO
`semantic_search_notes`. `_apply_type_filter` is the SQL dialect and
`matches_facet` the Python one; all three read `_FACETS`, which is what
keeps "adding a kind" a single edit (#3161).
keeps "adding a kind" a single edit.
Returns kwargs rather than a tuple so a caller splats it and cannot pair
`task_kind` with `is_task=False` by writing the positions out of order.
@@ -421,10 +416,9 @@ _CONTENT_TYPE_BROAD = {
def content_type_filters(content_type: str, extra: tuple = ()) -> dict:
"""The search kwargs a door's `content_type` parameter implies.
Lives here, beside `_FACETS`, because a door that keeps its own map is how
this went wrong: the agent's search offered two kinds and browse offered
nine, and a third copy in `/api/search` offered two more quietly still
(#4250). Derived, so adding a kind stays one edit (#3161).
Lives here, beside `_FACETS`, because a door that keeps its own map drifts:
one door offers two kinds while another offers nine. Derived, so adding a
kind stays one edit.
Raises on anything unrecognised. `_facet` alone would fall back to reading
an unknown string as a note_type, which matches no row — so a typo comes
@@ -611,18 +605,15 @@ async def _semantic_knowledge_search(
query=q,
report=_semantic_report,
limit=min(200, limit * 4),
# The shared interactive floor — this was a bare `0.3` while
# routes/search.py had the same number as a commented constant, the
# exact pair where one moves and the other doesn't (#2463).
# The shared interactive floor, one constant, so no copy of the
# number can move without the others.
threshold=INTERACTIVE_SEARCH_THRESHOLD,
is_task=is_task_filter,
project_id=project_id,
)
# The human's MAIN search surface, and it logged nothing — so
# retrieval_logs claimed the web's search was /api/search, a narrower
# path with (measured) zero frontend consumers. Distinct source, so
# The human's MAIN search surface, logged under its own source so
# threshold tuning can include or exclude human queries deliberately
# rather than by accident (#2463).
# rather than by accident.
from scribe.services.retrieval_telemetry import record_retrieval
record_retrieval(
user_id=user_id, source="browse_search", query=q,
@@ -698,11 +689,10 @@ async def get_knowledge_counts(user_id: int, tags: list[str] | None = None) -> d
stmt = stmt.where(Note.tags.contains([tag]))
return stmt
# One grouped query per typing axis. The task half used to be a count
# for 'task' plus a second count for 'plan', which is why 'issue' —
# 17% of every task here — had no number to show: each kind needed its
# own query and nobody added one. Grouping by task_kind counts every
# kind, including ones added later, for the same two round-trips.
# One grouped query per typing axis. A count per kind needs a new
# query for every kind and leaves a kind nobody added with no number;
# grouping by task_kind counts every kind, including ones added later,
# in two round-trips.
non_task = _scoped(
select(Note.note_type, func.count(Note.id))
.where(Note.status.is_(None))
+29 -28
View File
@@ -1,4 +1,4 @@
"""Lessons point at rules (milestone 440, #4196).
"""Lessons point at rules.
A lesson is a non-binding record of one situation; a rule is the binding
choice for a class of them. This service owns the link between the two — which
@@ -7,17 +7,18 @@ rule a lesson is an instance of — and both directions of reading it.
THE STATES (models/lesson_rule_link.py says why each exists):
`suggested` while evidence accumulates, `confirmed` or `rejected` once a
judgment is made. Every write here is a JUDGMENT, so every write lands as
confirmed or rejected; `suggested` rows come from the co-surfacing recorder
(#4637), never from a caller naming a rule.
confirmed or rejected; `suggested` rows come from the co-surfacing recorder,
never from a caller naming a rule.
THE THIRD ANSWER (#4631). "Which rule is this an instance of?" has three
THE THIRD ANSWER. "Which rule is this an instance of?" has three
answers, and only two of them are links: a rule named (confirmed), or NO RULE
FITS — a `lesson_no_rule` row carrying the reason. A lesson with neither is
UNJUDGED, and `list_unjudged` lists those. The three are kept apart because a
lesson that stands alone is the raw material for a rule nobody has written yet
(#4634), and that is unreadable if "nobody looked" means the same thing.
(see Convergence below), and that is unreadable if "nobody looked" means the
same thing.
THE SOFT LINK (#4637). When one hook request puts a lesson and a rule in
THE SOFT LINK. When one hook request puts a lesson and a rule in
front of the reader together, `co_surfaced` records the pair as evidence on a
SUGGESTED link, and once that evidence crosses a bar the next co-arrival asks
the reader to judge it. The evidence counts distinct SITUATIONS, not
@@ -25,7 +26,7 @@ arrivals: two records that resemble each other arrive together on every
similar prompt, and counting each time would measure their similarity over
and over rather than any relation between them.
ACL (rule 78). Linking changes what a lesson says about itself, so it needs
ACL (services/access.py). Linking changes what a lesson says about itself, so it needs
WRITE on the lesson (`access.can_write_note`, share-aware). It names a rule, so
it needs the rule to be one the caller may read — rules are owner-scoped, and
`rulebooks._fetch_owned_rule` / `_owned_rules_clause` are that check's two
@@ -67,8 +68,8 @@ _NO_RULE_NOTE = "no rule fits: "
# What `rule_judgment` reads on a lesson payload — the three answers.
LINKED, NO_RULE, UNJUDGED = "linked", "no_rule", "unjudged"
# THE SOFT-LINK EVIDENCE BAR (#4637) — defaults, stated as defaults (rules 32,
# 115): nothing about this install chose them.
# THE SOFT-LINK EVIDENCE BAR — defaults, stated as defaults: nothing about an
# install chose them.
#
# Three distinct situations before a pair is put to the reader. One is
# coincidence and two is a pattern only by courtesy; three different prompts
@@ -361,7 +362,7 @@ async def attach_lesson_rules(user_id: int, rows: list[dict], *, key: str = "id"
"""Add `rules` and `rule_judgment` (plus `no_rule` when that is the answer)
to each lesson payload row, in place — a fixed number of queries per page.
Fail-open (snippet #4286): this decorates a lesson the caller already has,
Fail-open: this decorates a lesson the caller already has,
so a failed lookup leaves the keys off and logs, rather than refusing the
lesson. An absent key reads as "not attached", never as "no rule".
"""
@@ -470,7 +471,7 @@ async def rule_candidates(
async def attach_rule_lessons(user_id: int, data: dict, rule_id: int) -> None:
"""Add `lessons` to a rule payload, in place, when there are any.
Present-only, as a rule's `systems` and `relations` are (#2483), and
Present-only, as a rule's `systems` and `relations` are, and
fail-open for the reason `attach_lesson_rules` gives.
"""
try:
@@ -487,8 +488,8 @@ def situation_key(arm: str, text: str) -> str:
"""A fingerprint of one situation, so a repeat counts once.
`arm` is part of the key — "p" for a prompt, "w" for a file being written,
"s" for a session (the rule↔moment signal, milestone 458 step 7), "d"
for a day (the misfire signal, step 7b, whose door carries no session) —
"s" for a session (the rule↔moment signal), "d" for a day (the misfire
signal, whose door carries no session) —
because each names a different kind of situation and must not collide.
On the prompt arm the text is the prompt: lowercased, split into word
tokens of _MIN_TOKEN or more, de-duplicated and sorted, so the same ask
@@ -657,7 +658,7 @@ async def _co_surfaced(user_id, lesson_ids, rule_ids, arm, situation, project_id
return _proposal_line(lid, lesson_title, rid, rule.title, rule.kind or "rule", ev)
# ── Reaching a rule through its lessons (#4633) ──────────────────────────────
# ── Reaching a rule through its lessons ──────────────────────────────────────
async def confirmed_lessons(user_id: int) -> set[int]:
@@ -666,8 +667,8 @@ async def confirmed_lessons(user_id: int) -> set[int]:
CONFIRMED ONLY, and the reason is the soft link's: a suggested link that
carried its rule would put the two side by side on every prompt that
found the lesson, which is exactly the co-arrival #4637 counts as
evidence — the suggestion would manufacture its own proof.
found the lesson, which is exactly the co-arrival the soft link counts
as evidence — the suggestion would manufacture its own proof.
Asked first and cheaply, so an arm on an install with no confirmed links
— every install, until someone judges one — never pays for a search.
@@ -695,11 +696,11 @@ async def confirmed_rules_in_scope(
) -> dict[int, list]:
"""{lesson_id: [Rule]} — each lesson's CONFIRMED rules that may surface here.
"May surface here" is the rule arms' own scope (milestone 414): a global
rule, or a rule of THIS project. A lesson is project-independent and
surfaces everywhere, so without this a lesson linked to project A's rule
would carry that rule into project B's sessions — the leak milestone 414
closed for the direct arms, reopened through a side door.
"May surface here" is the rule arms' own scope: a global rule, or a rule
of THIS project. A lesson is project-independent and surfaces everywhere,
so without this a lesson linked to project A's rule would carry that rule
into project B's sessions — a leak the direct arms' scope prevents,
reopened through a side door.
"""
from scribe.services.rulebooks import _owned_rules_clause
@@ -727,16 +728,16 @@ async def confirmed_rules_in_scope(
return out
# ── Convergence: lessons with no rule that keep landing in one place (#4634) ─
# ── Convergence: lessons with no rule that keep landing in one place ────────
#
# A lesson answered "no rule fits" is a situation nothing binds. One is a
# lesson. Several that resemble each other are what a missing rule looks like
# from the outside — the same moment met again and again, each time written
# down as advice. This is noticed at the WRITE, when the newest of them is
# answered, and never by a sweep or a timer (#4183): the reader is in the
# answered, and never by a sweep or a timer: the reader is in the
# situation then, and a nudge arriving anywhere else is one nobody acts on.
#
# WHAT "RESEMBLE" MEANS (#5193). Not a fixed similarity. Lessons are written
# WHAT "RESEMBLE" MEANS. Not a fixed similarity. Lessons are written
# in one register — a sentence of advice and the moment it applies — so an
# embedder scores any two of them well above two unrelated texts, and a
# constant bar sits inside that band: every no-rule lesson then "resembles"
@@ -750,10 +751,10 @@ async def confirmed_rules_in_scope(
#
# And resemblance must be MUTUAL, among every member, not just to the lesson
# being answered. A lesson broad enough to stand near many others is a hub,
# and hub-and-spoke is how three unrelated lessons used to make a "group"
# through one general one. A clique is the shape of one situation met again.
# and hub-and-spoke lets three unrelated lessons make a "group" through one
# general one. A clique is the shape of one situation met again.
#
# DEFAULTS, stated as defaults (rules 32, 115). Three lessons — the new one
# DEFAULTS, stated as defaults. Three lessons — the new one
# and two it resembles — is the smallest group that is a pattern rather than
# a pair. One spread above the background is a modest bar for one direction
# alone; the strictness comes from requiring it of every pair in both
@@ -834,7 +835,7 @@ async def convergence_for(user_id: int, lesson_id: int) -> dict | None:
"""The convergence a newly answered no-rule lesson completes, or None.
Fail-open: this decorates a write that already succeeded, so a failed
search leaves the response as it was (snippet #4286's reasoning).
search leaves the response as it was.
"""
try:
return await _convergence_for(user_id, lesson_id)
+71 -84
View File
@@ -4,12 +4,11 @@ A *lesson* is a Note with ``note_type='lesson'``: a better way to think about a
problem, or a solution that transfers, recorded so a later session meets it at
the moment it applies — and **without binding the reader**.
WHY THE KIND EXISTS (milestone 385, from note #3727)
WHY THE KIND EXISTS
Rules were the only surface that is global AND situation-keyed, so an agent
holding a transferable insight had one door, and that door binds. The observed
symptom was sessions offering rule proposals for things that should not be
rules.
Without it, rules are the only surface that is global AND situation-keyed, so
an agent holding a transferable insight has one door, and that door binds —
and sessions propose rules for things that should not be rules.
The gap is a document-shape fact, not a threshold:
@@ -23,31 +22,26 @@ No tuning reaches across that: the field the query would match on is simply not
in a note's document. So a lesson carries a trigger and is embedded like a rule,
while staying a note in every other respect.
WHERE THE TRIGGER LIVES (decision #4157, milestone 385 step 1)
WHERE THE TRIGGER LIVES
In ``notes.data`` under ``when_to_apply``, written through a named parameter and
mirrored into the head of the body — the shape snippets already use for
``when_to_use``. Not a column on ``notes``. Since milestone 427 it is NOT in the
stored title: the title is the lesson's subject, and the trigger joins it only
in the embedded document (``embeddings.document_title``).
mirrored into the head of the body — the shape snippets use for
``when_to_use``. Not a column on ``notes``. It is NOT in the stored title: the
title is the lesson's subject, and the trigger joins it only in the embedded
document (``embeddings.document_title``).
That decision was measured rather than assumed. The whole snippet corpus —
164 of 164 — carries a ``when_to_use`` with **no guard anywhere**, which refutes
the premise that an unenforced field gets skipped. What it does NOT show is that
an agent types a title convention correctly: the service composed the title
from the parameter, so what is at 100% is a named structured field. A column
would have bought enforceability at the price of deciding, for every note kind
at once, a question nothing had measured.
A named structured field rather than a column: snippets show such a field gets
filled reliably without a guard, and a column would decide enforceability for
every note kind at once.
The trigger in the document is what makes the vector sharp. It reaches it
twice — in the embedded title, joined at embed time, and in the body's first
line — which is the text these kinds were always embedded as, so nothing
re-embeds and ``CHUNKER_VERSION`` does not move.
line.
WHAT A LESSON INHERITS, AND THE CELLS LEFT EMPTY ON PURPOSE (#3163)
WHAT A LESSON INHERITS, AND THE CELLS LEFT EMPTY ON PURPOSE
A new kind inherits the note machinery wholesale, and #3163 asks which parts it
should NOT get — so that an empty cell is a decision rather than an oversight.
A new kind inherits the note machinery wholesale. Which parts it should NOT get
is stated here, so that an empty cell is a decision rather than an oversight.
Inherited, all deliberately:
@@ -67,25 +61,23 @@ NOT inherited, and each for a stated reason:
in by accident silently changes what the record IS.
- **recurrence** — task-only, and a lesson does not recur.
- **verify_with / expires_when** — available, because they are generic note
fields, but not part of a lesson's contract and not asked for on create. The
milestone-312 distinction is why: those mark a record that asserts a FACT
about someone else's software and can go false unwatched. A lesson is closer
fields, but not part of a lesson's contract and not asked for on create.
Those mark a record that asserts a FACT about someone else's software and
can go false unwatched. A lesson is closer
to a norm — "a better way to think about this" has no truth value that rots
on its own. A lesson that does assert such a fact can still carry them.
WHY THERE IS NO MIGRATION
WHY A KIND NEEDS NO MIGRATION
``note_type`` carries **no CHECK constraint** — only ``task_kind`` does
(``notes_task_kind_check``, migrations 0056 / 0065). Migration 0036 added
``note_type`` as plain ``Text`` with a server default and nothing has gated it
since. So rule 36 has nothing to expand here, and the failure it guards against
— a value the database refuses on an instance predating its migration — cannot
arise for this column.
(``notes_task_kind_check``). It is plain ``Text`` with a server default, so a
value the database refuses on an un-migrated instance cannot arise for this
column.
The real vocabulary is ``services.knowledge._FACETS``, which is where a kind
becomes reachable on the browse surface and validated at the door. That is one
table feeding both dialects of the type filter, so adding a kind there is a
single edit — a property #3161 recommended and that landed before this.
single edit.
"""
from __future__ import annotations
@@ -142,18 +134,18 @@ def normalize_sources(entries: list | None) -> list[int]:
THE CARDINALITY DECISION, and why it is a list.
`arose_from_id` already exists and holds ONE id, which is the obvious first
answer and the wrong one. The lesson that started this milestone generalised
THREE incidents — a badge collision, an un-backfilled column, a duplicated
const — into one claim about failure classes no CI lane can see. Generalising
across incidents is the shape a good lesson HAS, not an edge case. A single
answer and the wrong one. A good lesson often generalises several
incidents — a badge collision, an un-backfilled column, a duplicated const
— into one claim about a failure class. That is the shape a good lesson
HAS, not an edge case. A single
id would keep the first and silently drop the rest, and a record that drops
two of its three sources is worse than one that names none, because it reads
as complete.
It lives in `notes.data` rather than a join table for exactly the reason
decision #4157 put the trigger there: a join table would settle, for every
note kind at once, whether provenance is multi-valued — a question nothing
has measured. `data` is JSONB with a GIN index (0070), so the list is
It lives in `notes.data` rather than a join table for the reason the
trigger lives there: a join table would settle, for every note kind at
once, whether provenance is multi-valued. `data` is JSONB with a GIN
index, so the list is
queryable today and a table can be migrated to later if the need is shown.
Order is history, not sorting: the incidents stay in the sequence the writer
@@ -217,10 +209,10 @@ def compose_body(
The trigger's home in the text of the document, and half of what makes a
lesson findable: `chunk_document` joins `{embedded title}\\n{body}`, and
the embedded title is `what — when it applies` (`embeddings.document_title`,
milestone 427), so the document states WHEN IT APPLIES in the title and
again in the first line of the body. That is the twice-in-a-short-document
shape note #2485 measured as the only sharp one in the corpus.
the embedded title is `what — when it applies` (`embeddings.document_title`),
so the document states WHEN IT APPLIES in the title and again in the first
line of the body. That twice-in-a-short-document shape is what measures
sharp in the corpus.
`**When to apply:**` rather than plain text: the body is the READABLE
form, `data` is the queryable mirror, and `_BODY_TRIGGER_RE` reads this
@@ -229,11 +221,11 @@ def compose_body(
anywhere else.
The insight goes in the body rather than being held out of the document.
`rule_document` excludes a rule's `why` because long dated narrative made
sixteen dev-logs land on the centroid of "development" — but that finding
predates chunking (#280). A body over the budget is now split into several
chunks, EACH prefixed with the title, so a lesson's story no longer
averages itself into its trigger: it occupies its own vectors, and every
`rule_document` excludes a rule's `why` because long narrative in ONE
vector lands on the centroid of "development" — but a body over the
budget is split into several chunks, EACH prefixed with the title, so a
lesson's story does not average itself into its trigger: it occupies its
own vectors, and every
one of them still carries the trigger in its prefix. Holding it out would
cost the reader the only part that explains the insight and would buy a
sharpness the chunker already provides.
@@ -255,11 +247,10 @@ def compose_body(
return "\n\n".join(lines)
# A lesson's NAME is its claim, and the claim is one line (#4797). `what` is
# the title every listing, menu and search row prints, so a story written there
# instead of in `insight` turns each of those lines into a document — the menu
# line for one such lesson ran to 1,416 characters, and the claim it existed to
# transfer sat somewhere in the middle. It also ranks badly: the embedded title
# A lesson's NAME is its claim, and the claim is one line. `what` is the title
# every listing, menu and search row prints, so a story written there instead
# of in `insight` turns each of those lines into a document, with the claim it
# exists to transfer somewhere in the middle. It also ranks badly: the embedded title
# is `what — when_to_apply`, so a narrative crowds out the trigger.
#
# 240 is a long sentence. The bound is on what a NAME can be, not on what a
@@ -320,12 +311,10 @@ def lesson_document(
it: the record would look right in every listing and simply never be
retrieved at the moment it applies.
The title is the SUBJECT alone (milestone 427). It used to carry the
trigger too, so the stored record was itself the sharp document — and every
listing, menu and search row then showed a title that ran to kilobytes.
The trigger now joins the title at embed time (`embeddings.document_title`,
reading `data`), producing the same text as before, so nothing re-embeds
and `CHUNKER_VERSION` does not move.
The title is the SUBJECT alone. A stored title carrying the trigger would
make every listing, menu and search row show a title that runs to
kilobytes. The trigger joins the title at embed time
(`embeddings.document_title`, reading `data`).
"""
return (
(what or "").strip(),
@@ -368,7 +357,7 @@ def recompose_data(note) -> dict:
snippet mirror reports the wrong path. A stale lesson mirror reports the
wrong TRIGGER — and `lesson_trigger` prefers the mirror, so the lesson goes
on being retrieved for the situation it used to name while displaying the
one it now names. The trigger is the entire retrieval story (step 3), so
one it now names. The trigger is the entire retrieval story, so
that is not a degraded record; it is a record that fires at the wrong
moment and looks right when it does.
@@ -410,14 +399,12 @@ async def create_lesson(
"""Create a lesson note. Returns the created Note.
The title, the body and the `data` mirror are composed HERE from named
parameters rather than asked of the caller. That is the whole evidence base
for this design and not a convenience: the snippet corpus carries a trigger
on 164 of 164 records with no guard anywhere, because a service builds the
title from a parameter — what is at 100% is a named structured field, not an
agent typing a convention correctly.
parameters rather than asked of the caller. That is the design, not a
convenience: a named structured field that a service composes is filled
reliably, where a convention an agent must type correctly is not.
`project_id` is accepted and kept, even though a lesson is reachable from
every project (step 3). Where it was learned is a fact worth keeping; it
every project. Where it was learned is a fact worth keeping; it
simply stops being the limit of where it can be found.
"""
from scribe.services import notes as notes_svc
@@ -440,9 +427,10 @@ async def create_lesson(
async def get_lesson(user_id: int, lesson_id: int):
"""Fetch a lesson by id, or None if it isn't one / isn't readable.
Share-aware (rule 78): a fetch by id is an explicit act, so it resolves the
caller's full read scope rather than ownership alone — without this, a
lesson a search legitimately surfaced could not then be opened (#2093).
Share-aware (services/access.py): a fetch by id is an explicit act, so it
resolves the caller's full read scope rather than ownership alone —
without this, a lesson a search legitimately surfaced could not then be
opened.
"""
from scribe.services import notes as notes_svc
@@ -494,7 +482,7 @@ async def source_records(user_id: int, ids: list[int]) -> list[dict]:
One query for the whole list, not one per id — a lesson with six sources
would otherwise be six round trips to draw one panel.
Share-aware (rule 78). Order follows `ids`, because that order is the
Share-aware (services/access.py). Order follows `ids`, because that order is the
writer's: the first source is the one they reached for first.
"""
from sqlalchemy import select
@@ -535,11 +523,11 @@ async def lessons_taught_by(user_id: int, record_id: int, limit: int = 20):
record that taught something should say so on its own page.
Queried through `data[SOURCES_KEY]` rather than by scanning bodies: the
mirror is JSONB with a GIN index (0070), which is the whole reason step 4
put the list there. `path_exists` is the same dialect the snippet location
mirror is JSONB with a GIN index, which is the whole reason the list
lives there. `path_exists` is the same dialect the snippet location
lookup uses, so both reverse lookups read the index the same way.
Share-aware (rule 78) via `readable_notes_clause`: this renders beside a
Share-aware via `readable_notes_clause`: this renders beside a
record the caller can already see, and a lesson someone shared with them
belongs in that list exactly as their own does.
"""
@@ -630,13 +618,12 @@ def lesson_insight(note) -> str:
added; anything it leaves behind is re-prepended and the body grows by that
much on every edit.
TAKES THE NOTE, NOT THE BODY, and that is the whole fix (#4272). The
previous shape filtered out lines matching `_BODY_TRIGGER_RE`, which is
`re.M` and therefore matches ONE LINE. `when_to_apply` is free text and is
routinely several paragraphs — `compose_body` writes all of it after the
marker, so paragraphs 2..n were ordinary lines the filter never touched.
They came back as part of "the insight", the full trigger was prepended
again, and the record gained (n-1) paragraphs per update. Silent, and
TAKES THE NOTE, NOT THE BODY. Filtering out lines matching
`_BODY_TRIGGER_RE` is not enough: that pattern is `re.M` and matches ONE
LINE, while `when_to_apply` is free text and routinely several paragraphs
— `compose_body` writes all of it after the marker. Paragraphs 2..n would
come back as part of "the insight", the full trigger would be prepended
again, and the record would gain (n-1) paragraphs per update. Silent, and
compounding.
What makes an exact strip possible is that the trigger is stored: the
@@ -645,9 +632,9 @@ def lesson_insight(note) -> str:
subtraction rather than pattern-matching.
The fallback is not dead code. A row whose mirror is missing degrades to
the old line-wise removal, which is right for a one-paragraph trigger and
is the best that can be done without knowing where the trigger ends —
there is no marker for its last line, which is why guessing was wrong.
line-wise removal, which is right for a one-paragraph trigger and is the
best that can be done without knowing where the trigger ends — there is
no marker for its last line.
The provenance line needs no such care: `compose_body` builds it by
joining ids with ", ", so it cannot contain a newline.