feat(lessons): judged when written — a new lesson is offered its rules, and "no rule fits" is an answer (milestone 440 step 2, #4631)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m15s
CI & Build / Build & push image (push) Skipped

"Which rule is this lesson an instance of?" now has three recorded answers:
a rule named (a confirmed link, #4630), no rule fits (new), or unjudged.

- Model + migration 0112: lesson_no_rule (lesson_id PK, CASCADE from the
  note; why; judged_at). A table rather than a key in notes.data, because
  that mirror is re-composed from the body on every edit and would erase it.
- Service (lesson_rules): set_no_rule rejects any confirmed link with the
  reason; a confirmation (set_lesson_rules or judge_link) deletes the answer;
  require_one_answer refuses both answers in one call before any write;
  judgments_for_lessons + attach_lesson_rules add rule_judgment (and no_rule)
  to every lesson payload; list_unjudged lists the open ones; rule_candidates
  searches rules with the lesson's claim + trigger at the explicit-search bar,
  None when the search could not run.
- MCP: create_lesson/update_lesson take no_rule; an unanswered create returns
  rule_candidates, rule_judgment and a rule_hint; list_lessons(unjudged=true).
- REST: the same on POST/PATCH /api/lessons and GET ?unjudged=1; create
  returns rule_candidates.
- Backup v19: a lesson_no_rule section, export (full and per-user) and import.
- Guidance: create_lesson docstring, writing-records.md in using-scribe (owner,
  pinned in test_guidance_ownership), create_rule docstring on linking the
  lessons a new rule governs. Plugin version minted.
- Tests: door units, integration for the three states, the rejection reason,
  scoping, cascade; backup registries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-01 13:25:27 -04:00
co-authored by Claude Opus 5.5
parent c8393975c3
commit f7d8dc2e55
15 changed files with 696 additions and 37 deletions
+206 -4
View File
@@ -10,6 +10,13 @@ judgment is made. Every write here is a JUDGMENT, so every write lands as
confirmed or rejected; `suggested` rows come from the co-surfacing recorder
(#4637), never from a caller naming a rule.
THE THIRD ANSWER (#4631). "Which rule is this an instance of?" has three
answers, and only two of them are links: a rule named (confirmed), or NO RULE
FITS — a `lesson_no_rule` row carrying the reason. A lesson with neither is
UNJUDGED, and `list_unjudged` lists those. The three are kept apart because a
lesson that stands alone is the raw material for a rule nobody has written yet
(#4634), and that is unreadable if "nobody looked" means the same thing.
ACL (rule 78). Linking changes what a lesson says about itself, so it needs
WRITE on the lesson (`access.can_write_note`, share-aware). It names a rule, so
it needs the rule to be one the caller may read — rules are owner-scoped, and
@@ -22,10 +29,12 @@ from __future__ import annotations
import logging
from datetime import datetime, timezone
from sqlalchemy import select
from sqlalchemy import and_, delete, exists, func, not_, select
from scribe.models import async_session
from scribe.models.lesson_rule_link import CONFIRMED, REJECTED, SUGGESTED, LessonRuleLink
from scribe.models.lesson_rule_link import (
CONFIRMED, REJECTED, SUGGESTED, LessonNoRule, LessonRuleLink,
)
from scribe.models.note import Note
from scribe.models.rulebook import Rule
@@ -39,6 +48,18 @@ VERDICTS = {"confirm": CONFIRMED, "reject": REJECTED}
# row can tell an explicit "not this rule" from a link dropped by a rewrite.
_REMOVED_NOTE = "removed from the lesson's rules by an update"
# Recorded on a confirmed link that a "no rule fits" answer overturned, ahead
# of that answer's reason.
_NO_RULE_NOTE = "no rule fits: "
# What `rule_judgment` reads on a lesson payload — the three answers.
LINKED, NO_RULE, UNJUDGED = "linked", "no_rule", "unjudged"
# How many rules a new lesson is offered to judge against. A handful, not the
# fifty `what_might_apply` returns: this is read in the create response, at
# the moment of writing, and a long tail there buries the one that fits.
CANDIDATE_LIMIT = 5
def _ids(values) -> list[int]:
"""Positive ints, de-duplicated, in the order given."""
@@ -83,6 +104,16 @@ async def require_rules(user_id: int, rule_ids) -> list[int]:
return wanted
def require_one_answer(linked, no_rule: str) -> None:
"""Refuse "these rules fit" and "no rule fits" together, before any write
— they contradict, and either order of applying them loses one."""
if linked and (no_rule or "").strip():
raise ValueError(
"rule_ids and no_rule are the two answers to \"which rule is this "
"an instance of?\" — give the one that holds. Nothing was written."
)
async def _upsert(session, lesson_id: int, rule_id: int, state: str, note: str) -> None:
now = datetime.now(timezone.utc)
row = (await session.execute(
@@ -104,6 +135,11 @@ async def _upsert(session, lesson_id: int, rule_id: int, state: str, note: str)
row.judged_at = now
async def _clear_no_rule(session, lesson_id: int) -> None:
"""A confirmed rule overturns "no rule fits" — both cannot be the answer."""
await session.execute(delete(LessonNoRule).where(LessonNoRule.lesson_id == lesson_id))
async def set_lesson_rules(
user_id: int, lesson_id: int, rule_ids, *, note: str = "",
) -> None:
@@ -134,9 +170,48 @@ async def set_lesson_rules(
row.judged_at = datetime.now(timezone.utc)
for rid in wanted:
await _upsert(session, lesson_id, rid, CONFIRMED, note)
if wanted:
await _clear_no_rule(session, lesson_id)
await session.commit()
async def set_no_rule(user_id: int, lesson_id: int, why: str) -> dict:
"""Record that no rule governs this lesson's situation, and why.
A confirmed link the lesson still carries becomes `rejected`, with the
reason: saying none fits is saying the linked one does not either. A
second answer replaces the first, so the reason on file is the latest.
"""
why = (why or "").strip()
if not why:
raise ValueError(
"no_rule takes the reason no rule fits, in a line — it is what "
"lets the answer be re-judged when a rule is later written for "
"this situation."
)
await _require_lesson_writable(user_id, lesson_id)
now = datetime.now(timezone.utc)
async with async_session() as session:
confirmed = (await session.execute(
select(LessonRuleLink).where(
LessonRuleLink.lesson_id == lesson_id,
LessonRuleLink.state == CONFIRMED,
)
)).scalars().all()
for row in confirmed:
row.state = REJECTED
row.note = _NO_RULE_NOTE + why
row.judged_at = now
answer = await session.get(LessonNoRule, lesson_id)
if answer is None:
session.add(LessonNoRule(lesson_id=lesson_id, why=why, judged_at=now))
else:
answer.why = why
answer.judged_at = now
await session.commit()
return {"why": why, "judged_at": now.isoformat()}
async def judge_link(
user_id: int, lesson_id: int, rule_id: int, verdict: str, note: str = "",
) -> dict:
@@ -148,6 +223,8 @@ async def judge_link(
[rid] = await require_rules(user_id, [rule_id])
async with async_session() as session:
await _upsert(session, lesson_id, rid, state, note)
if state == CONFIRMED:
await _clear_no_rule(session, lesson_id)
await session.commit()
row = (await session.execute(
select(LessonRuleLink).where(
@@ -210,22 +287,147 @@ async def lessons_for_rule(user_id: int, rule_id: int) -> list[dict]:
]
async def judgments_for_lessons(lesson_ids) -> dict[int, dict]:
"""{lesson_id: {"rule_judgment": linked|no_rule|unjudged, "no_rule"?}}.
Read from the link and answer tables directly, NOT from the reader's view
of the rules: a lesson shared with someone who cannot see its owner's rule
is still a judged lesson, and calling it unjudged would invite them to
judge it again.
"""
ids = [int(i) for i in lesson_ids or []]
if not ids:
return {}
async with async_session() as session:
linked = set((await session.execute(
select(LessonRuleLink.lesson_id).where(
LessonRuleLink.lesson_id.in_(ids),
LessonRuleLink.state == CONFIRMED,
)
)).scalars().all())
answers = {
a.lesson_id: a for a in (await session.execute(
select(LessonNoRule).where(LessonNoRule.lesson_id.in_(ids))
)).scalars().all()
}
out: dict[int, dict] = {}
for i in ids:
if i in linked:
out[i] = {"rule_judgment": LINKED}
elif i in answers:
out[i] = {"rule_judgment": NO_RULE, "no_rule": answers[i].to_dict()}
else:
out[i] = {"rule_judgment": UNJUDGED}
return out
async def attach_lesson_rules(user_id: int, rows: list[dict], *, key: str = "id") -> None:
"""Add `rules` to each lesson payload row, in place — one query per page.
"""Add `rules` and `rule_judgment` (plus `no_rule` when that is the answer)
to each lesson payload row, in place — a fixed number of queries per page.
Fail-open (snippet #4286): this decorates a lesson the caller already has,
so a failed lookup leaves the key off and logs, rather than refusing the
so a failed lookup leaves the keys off and logs, rather than refusing the
lesson. An absent key reads as "not attached", never as "no rule".
"""
ids = [int(r[key]) for r in rows if isinstance(r.get(key), int)]
try:
found = await rules_for_lessons(user_id, ids)
judged = await judgments_for_lessons(ids)
except Exception:
logger.warning("lesson→rule links could not be read", exc_info=True)
return
for r in rows:
if isinstance(r.get(key), int):
r["rules"] = found.get(r[key], [])
r.update(judged.get(r[key], {}))
def unjudged_clause():
"""SQL: a lesson (a `notes` row) with no confirmed rule and no "no rule
fits" answer. A rejected link alone leaves it unjudged — "not that rule"
does not say whether another one fits."""
from scribe.models.note import Note
return and_(
not_(exists().where(and_(
LessonRuleLink.lesson_id == Note.id,
LessonRuleLink.state == CONFIRMED,
))),
not_(exists().where(LessonNoRule.lesson_id == Note.id)),
)
async def list_unjudged(
user_id: int, *, tag: str = "", project_id: int | None = None,
limit: int = 50, offset: int = 0,
) -> tuple[list[dict], int]:
"""The lessons nobody has answered "which rule?" for, newest edit first.
The browse scope (`browsable_notes_clause`), the list it sits beside uses:
a lesson shared directly with the caller is search-only and stays out of
an ambient list. Rows come back in the knowledge list's item shape.
"""
from scribe.models.note import Note
from scribe.services.access import browsable_notes_clause
from scribe.services.knowledge import _note_to_item
from scribe.services.lessons import LESSON_NOTE_TYPE
base = (
select(Note)
.where(browsable_notes_clause(user_id))
.where(Note.note_type == LESSON_NOTE_TYPE)
.where(Note.deleted_at.is_(None))
.where(unjudged_clause())
)
if tag:
base = base.where(Note.tags.contains([tag]))
if project_id is not None:
base = base.where(Note.project_id == project_id)
async with async_session() as session:
total = (await session.execute(
select(func.count()).select_from(base.subquery())
)).scalar_one()
rows = (await session.execute(
base.order_by(Note.updated_at.desc()).limit(limit).offset(offset)
)).scalars().all()
return [_note_to_item(n) for n in rows], int(total)
async def rule_candidates(
user_id: int, what: str, when_to_apply: str, project_id: int | None = None,
) -> list[dict] | None:
"""The rules a lesson most resembles, for the writer to judge against.
Searched with the lesson's claim and its trigger — the same two halves a
rule's own document leads with (`{title} — {trigger}`), so like is
compared with like; the insight is the story and would dilute the vector.
Scoped as any project read is: global rules plus the lesson's project's
own. The bar is the explicit rule search's, since this is an explicit
question rather than an injection.
None when the search could not run (no embedder, a failed query), so a
caller can say "candidates unavailable" rather than "nothing resembles".
"""
from scribe.services.embeddings import (
DEFAULT_SIMILARITY_THRESHOLD, semantic_search_rules,
)
query = " — ".join(p for p in ((what or "").strip(), (when_to_apply or "").strip()) if p)
report: dict = {}
found = await semantic_search_rules(
user_id, query, limit=CANDIDATE_LIMIT,
threshold=DEFAULT_SIMILARITY_THRESHOLD, report=report,
project_id=project_id or None,
)
if not report.get("searched"):
return None
return [
{
"id": rule.id, "title": rule.title, "kind": rule.kind,
"when_to_apply": rule.when_to_apply or "", "score": round(score, 3),
}
for score, rule in found
]
async def attach_rule_lessons(user_id: int, data: dict, rule_id: int) -> None: