bvandeusenandClaude Opus 5.5
07169e1274
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 1m9s
CI & Build / integration (push) Successful in 1m21s
CI & Build / Python tests (push) Successful in 2m2s
CI & Build / Build & push image (push) Successful in 1m29s
Step 1 added family_ideas.topic_id but nothing could set it, so the note<->topic link the milestone promised existed only as a column. set_family_topic links or unlinks (0) a topic in a rulebook the caller owns, one idea per topic; the version does not move. get_family_idea and get_family_adoption list the topic's rules; list_family_ideas and the Family page name the topic. The model docstrings no longer claim the topic's rules are platform-scoped in retrieval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1037 lines
46 KiB
Python
1037 lines
46 KiB
Python
"""Family canon's promotion engine (milestone 463 step 3).
|
|
|
|
A family idea moves between three states — candidate, canon, retired — and
|
|
every move is a decision with a reason, written to `family_decisions`. No
|
|
person approves a promotion: the agent decides against the written criteria
|
|
below, and the log is what keeps one decision consistent with the last similar
|
|
one, and what lets a person read or undo any of them afterwards.
|
|
|
|
WHO DECIDES WHAT
|
|
|
|
- TRIGGERS (`citation_trigger`, `repeat_trigger`, `milestone_trigger`) only
|
|
ever OPEN an evaluation. The first two record the source record as a
|
|
`candidate` (decided_via "system") and hand the writer an in-band hint; a
|
|
closed milestone is not a record an idea can hang on, so it only hints. None
|
|
of them promotes.
|
|
- THE AGENT evaluates a candidate against the three criteria and either
|
|
promotes it or leaves it a candidate with the reason. A criterion with no
|
|
support vetoes on its own; the veto is logged too, because a held candidate
|
|
is precedent for the next one like it.
|
|
- A PERSON reads the log, and may retire an idea or undo a decision from the
|
|
web door. Those are the only operator acts, and neither is required.
|
|
|
|
PRECEDENT
|
|
|
|
Every promotion records the earlier decisions it was consistent with. The
|
|
engine finds them itself — the decisions on the ideas nearest this one by
|
|
meaning — and stores them alongside any the caller names, so "which precedents
|
|
were consulted" is a fact about the call, not a sentence a session might skip.
|
|
|
|
LEDGER ROWS AND UNDO (settled here, as the milestone asked)
|
|
|
|
Promotion opens an `unassessed` row for every project that is a member of one
|
|
of the idea's platforms and that the promoter can write. Leaving canon —
|
|
by retirement, or by undoing the promotion — deletes the `unassessed` rows,
|
|
because nobody judged them and they would only be noise. Rows somebody DID
|
|
judge (adopted, variant, exempt, owed) are kept: each is a decision with its
|
|
reason, and if the idea is promoted again its version moves, so every kept
|
|
row reads as needing a recheck rather than as still agreeing.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import re
|
|
from datetime import datetime, timezone
|
|
|
|
from sqlalchemy import delete, func, select
|
|
|
|
from scribe.models import async_session
|
|
from scribe.models.family import (
|
|
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform, Platform,
|
|
ProjectPlatform,
|
|
)
|
|
from scribe.models.note import Note
|
|
from scribe.models.project import Project
|
|
from scribe.models.rulebook import Rule
|
|
from scribe.services import access
|
|
from scribe.services import rulebooks as rulebooks_svc
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
# --- the criteria, as product text --------------------------------------------
|
|
#
|
|
# These are what an evaluation is judged against, on every install. They live
|
|
# here — and in the tool docstrings that quote them — rather than in any
|
|
# instance's rulebook (rules 115, 119).
|
|
|
|
CRITERIA = (
|
|
{
|
|
"key": "platform_terms",
|
|
"title": "Stated in platform terms",
|
|
"test": (
|
|
"Its 'when it applies' is stated in terms of a platform — what the "
|
|
"project is built on or ships as — not one app's domain. Any project "
|
|
"on the platform could read it and know whether it applies."
|
|
),
|
|
},
|
|
{
|
|
"key": "platform_problem",
|
|
"title": "Answers a platform problem or a cross-project stance",
|
|
"test": (
|
|
"It answers a problem the platform itself causes, or a stance the "
|
|
"operator holds across projects. One app's preference does not "
|
|
"qualify."
|
|
),
|
|
},
|
|
{
|
|
"key": "proven",
|
|
"title": "Proven at least once",
|
|
"test": (
|
|
"It has worked for real at least once — CI green, verified on a "
|
|
"device, or shipped — and the evidence is named. An unproven idea "
|
|
"stays a candidate."
|
|
),
|
|
},
|
|
)
|
|
CRITERIA_KEYS = tuple(c["key"] for c in CRITERIA)
|
|
|
|
TRIGGERS = ("citation", "repeat", "milestone")
|
|
|
|
# The repeat trigger's similarity floor. MEASURED, not guessed (2026-10-06,
|
|
# #4989): against a description of one pattern known to have been built four
|
|
# times in four projects, the records that implement it scored 0.79-0.82, and
|
|
# the best match in a project that never built it scored 0.65. A trigger only
|
|
# opens an evaluation, so a false positive costs one judgment, not a wrong
|
|
# promotion. Step 5 measures the cross-language threshold against known pairs
|
|
# and supersedes this.
|
|
REPEAT_THRESHOLD = 0.80
|
|
|
|
# Words that say a citation is the SOURCE of a pattern rather than background.
|
|
# "see #12" is a pointer; "matching #12" and "ported from #12" are lineage.
|
|
_LINEAGE = re.compile(
|
|
r"\b(match(?:es|ing)?|mirror(?:s|ing|ed)?|same (?:shape|pattern|approach|design) as|"
|
|
r"ported from|copied from|borrowed from|lifted from|taken from|based on|"
|
|
r"modell?ed on|follow(?:s|ing)? the (?:shape|pattern|approach) of|as (?:built|done) in|"
|
|
r"reuses?|re-?implement(?:s|ing)?|like)\b",
|
|
re.I,
|
|
)
|
|
# How far before a `#N` the lineage word may sit: the same clause, roughly.
|
|
_LINEAGE_WINDOW = 80
|
|
|
|
# Note kinds the repeat trigger compares: recorded knowledge and shapes, not
|
|
# the to-do list.
|
|
_REPEAT_KINDS = ("snippet", "note")
|
|
|
|
MEMBER_STATES = ("declared", "detected")
|
|
_IDEA_ACTIONS = ("propose", "promote", "revise", "retire")
|
|
|
|
|
|
def _now() -> datetime:
|
|
return datetime.now(timezone.utc)
|
|
|
|
|
|
# --- reads --------------------------------------------------------------------
|
|
|
|
async def _platform_slugs(session, note_id: int) -> list[str]:
|
|
rows = await session.execute(
|
|
select(Platform.slug)
|
|
.join(FamilyIdeaPlatform, FamilyIdeaPlatform.platform_id == Platform.id)
|
|
.where(FamilyIdeaPlatform.note_id == note_id)
|
|
.order_by(Platform.order_index.asc(), Platform.slug.asc())
|
|
)
|
|
return list(rows.scalars().all())
|
|
|
|
|
|
async def _topic(session, user_id: int, topic_id: int | None, *, rules: bool) -> dict | None:
|
|
"""The rule topic holding an idea's binding norms, for a reader. Rulebooks
|
|
belong to their owner, so a topic the reader does not own reads as no
|
|
topic."""
|
|
if not topic_id:
|
|
return None
|
|
topic = await rulebooks_svc.get_topic(topic_id, user_id)
|
|
if topic is None:
|
|
return None
|
|
out = {"id": topic.id, "title": topic.title, "rulebook_id": topic.rulebook_id}
|
|
if rules:
|
|
out["rules"] = [
|
|
{"id": r.id, "title": r.title} for r in (await session.execute(
|
|
select(Rule).where(Rule.topic_id == topic.id, Rule.deleted_at.is_(None))
|
|
.order_by(Rule.order_index.asc(), Rule.id.asc())
|
|
)).scalars().all()
|
|
]
|
|
return out
|
|
|
|
|
|
async def _snapshot(session, idea: FamilyIdea | None) -> dict | None:
|
|
"""The idea's state, as a decision's before/after. Slugs, never ids: an id
|
|
inside JSON cannot be remapped by a restore (the #3182 trap)."""
|
|
if idea is None:
|
|
return None
|
|
return {
|
|
"status": idea.status,
|
|
"applies_when": idea.applies_when or "",
|
|
"canon_version": idea.canon_version,
|
|
"platforms": await _platform_slugs(session, idea.note_id),
|
|
}
|
|
|
|
|
|
def _decision_dict(d: FamilyDecision, title: str | None = None) -> dict:
|
|
out = d.to_dict()
|
|
if title is not None:
|
|
out["idea_title"] = title
|
|
return out
|
|
|
|
|
|
async def get_idea(user_id: int, note_id: int) -> dict | None:
|
|
"""One idea with everything an evaluation reads: its state, platforms,
|
|
decision history (newest first), its ledger counts and the criteria.
|
|
None when the caller cannot read the note or it is not an idea."""
|
|
if not await access.can_read_note(user_id, note_id):
|
|
return None
|
|
async with async_session() as session:
|
|
idea = await session.get(FamilyIdea, note_id)
|
|
note = await session.get(Note, note_id)
|
|
if idea is None or note is None:
|
|
return None
|
|
decisions = (await session.execute(
|
|
select(FamilyDecision).where(FamilyDecision.idea_id == note_id)
|
|
.order_by(FamilyDecision.id.desc())
|
|
)).scalars().all()
|
|
counts = dict((await session.execute(
|
|
select(FamilyAdoption.status, func.count())
|
|
.where(FamilyAdoption.idea_id == note_id)
|
|
.group_by(FamilyAdoption.status)
|
|
)).all())
|
|
out = idea.to_dict()
|
|
out.update({
|
|
"title": note.title,
|
|
"note_type": note.note_type or "note",
|
|
"is_task": note.is_task,
|
|
"project_id": note.project_id,
|
|
"platforms": await _platform_slugs(session, note_id),
|
|
"topic": await _topic(session, user_id, idea.topic_id, rules=True),
|
|
"adoptions": counts,
|
|
"decisions": [_decision_dict(d) for d in decisions],
|
|
"undoable_decision_id": _undoable_id(decisions),
|
|
})
|
|
out["criteria"] = list(CRITERIA)
|
|
return out
|
|
|
|
|
|
async def list_ideas(
|
|
user_id: int, *, status: str | None = None, platform: str | None = None,
|
|
limit: int = 50, offset: int = 0,
|
|
) -> list[dict]:
|
|
"""Ideas whose note the caller can read, newest change first."""
|
|
async with async_session() as session:
|
|
query = (
|
|
select(FamilyIdea, Note.title, Note.note_type, Note.project_id, Note.status)
|
|
.join(Note, Note.id == FamilyIdea.note_id)
|
|
.where(access.readable_notes_clause(user_id), Note.deleted_at.is_(None))
|
|
)
|
|
if status:
|
|
query = query.where(FamilyIdea.status == status)
|
|
if platform:
|
|
query = query.where(FamilyIdea.note_id.in_(
|
|
select(FamilyIdeaPlatform.note_id)
|
|
.join(Platform, Platform.id == FamilyIdeaPlatform.platform_id)
|
|
.where(Platform.slug == platform)
|
|
))
|
|
rows = (await session.execute(
|
|
query.order_by(FamilyIdea.updated_at.desc()).limit(limit).offset(offset)
|
|
)).all()
|
|
out = []
|
|
for idea, title, note_type, project_id, note_status in rows:
|
|
d = idea.to_dict()
|
|
d.update({
|
|
"title": title,
|
|
"note_type": note_type or "note",
|
|
"is_task": note_status is not None,
|
|
"project_id": project_id,
|
|
"platforms": await _platform_slugs(session, idea.note_id),
|
|
"topic": await _topic(session, user_id, idea.topic_id, rules=False),
|
|
})
|
|
out.append(d)
|
|
return out
|
|
|
|
|
|
async def list_decisions(
|
|
user_id: int, *, idea_id: int | None = None, project_id: int | None = None,
|
|
limit: int = 50, offset: int = 0,
|
|
) -> list[dict]:
|
|
"""The decision log, newest first, over ideas the caller can read. Each
|
|
row says whether it is the one an undo would reverse; a project's
|
|
assessment also carries the project's title."""
|
|
async with async_session() as session:
|
|
query = (
|
|
select(FamilyDecision, Note.title, Project.title)
|
|
.join(Note, Note.id == FamilyDecision.idea_id)
|
|
.outerjoin(Project, Project.id == FamilyDecision.project_id)
|
|
.where(access.readable_notes_clause(user_id))
|
|
)
|
|
if idea_id:
|
|
query = query.where(FamilyDecision.idea_id == idea_id)
|
|
if project_id:
|
|
query = query.where(FamilyDecision.project_id == project_id)
|
|
rows = (await session.execute(
|
|
query.order_by(FamilyDecision.id.desc()).limit(limit).offset(offset)
|
|
)).all()
|
|
latest = await _latest_idea_decisions(
|
|
session, {d.idea_id for d, _, _ in rows if d.project_id is None})
|
|
latest_pair = await _latest_pair_decisions(
|
|
session, {(d.idea_id, d.project_id) for d, _, _ in rows if d.project_id is not None})
|
|
out = []
|
|
for d, title, project_title in rows:
|
|
item = _decision_dict(d, title)
|
|
if d.project_id is None:
|
|
item["undoable"] = latest.get(d.idea_id) == d.id and _can_undo(d)
|
|
else:
|
|
item["project_title"] = project_title
|
|
item["undoable"] = latest_pair.get((d.idea_id, d.project_id)) == d.id and _can_undo(d)
|
|
out.append(item)
|
|
return out
|
|
|
|
|
|
def _can_undo(d: FamilyDecision) -> bool:
|
|
"""A decision that changed something: an idea-level one, or one
|
|
project's assessment. A veto (a `propose` whose before and after agree)
|
|
changed nothing, and an undo is undone by deciding again, not by undoing
|
|
the undo."""
|
|
if d.before == d.after:
|
|
return False
|
|
if d.project_id is None:
|
|
return d.action in _IDEA_ACTIONS
|
|
return d.action == "assess"
|
|
|
|
|
|
def _undoable_id(decisions_newest_first) -> int | None:
|
|
for d in decisions_newest_first:
|
|
if d.project_id is None:
|
|
return d.id if _can_undo(d) else None
|
|
return None
|
|
|
|
|
|
async def _latest_pair_decisions(session, pairs: set[tuple[int, int]]) -> dict[tuple[int, int], int]:
|
|
"""The latest decision about each (idea, project) answer — the only one of
|
|
a project's decisions on an idea that an undo may reverse."""
|
|
if not pairs:
|
|
return {}
|
|
rows = await session.execute(
|
|
select(FamilyDecision.idea_id, FamilyDecision.project_id, func.max(FamilyDecision.id))
|
|
.where(
|
|
FamilyDecision.idea_id.in_({i for i, _ in pairs}),
|
|
FamilyDecision.project_id.in_({p for _, p in pairs}),
|
|
)
|
|
.group_by(FamilyDecision.idea_id, FamilyDecision.project_id)
|
|
)
|
|
return {(i, p): d for i, p, d in rows.all() if (i, p) in pairs}
|
|
|
|
|
|
async def _latest_idea_decisions(session, idea_ids: set[int]) -> dict[int, int]:
|
|
if not idea_ids:
|
|
return {}
|
|
rows = await session.execute(
|
|
select(FamilyDecision.idea_id, func.max(FamilyDecision.id))
|
|
.where(FamilyDecision.idea_id.in_(idea_ids), FamilyDecision.project_id.is_(None))
|
|
.group_by(FamilyDecision.idea_id)
|
|
)
|
|
return dict(rows.all())
|
|
|
|
|
|
async def nearest_ideas(user_id: int, note_id: int, limit: int = 5) -> list[tuple[float, Note]]:
|
|
"""The family ideas nearest this record by meaning, best first, as
|
|
(score, note). Empty when there are no other ideas or the embedder is
|
|
unavailable — precedent is a help to a decision, never a gate on it."""
|
|
from scribe.services.embeddings import embedding_text, semantic_search_notes
|
|
|
|
async with async_session() as session:
|
|
note = await session.get(Note, note_id)
|
|
if note is None:
|
|
return []
|
|
idea_ids = set((await session.execute(select(FamilyIdea.note_id))).scalars().all())
|
|
idea_ids.discard(note_id)
|
|
if not idea_ids:
|
|
return []
|
|
try:
|
|
hits = await semantic_search_notes(
|
|
user_id, embedding_text(note.title, note.body), exclude_ids={note_id},
|
|
limit=40, threshold=0.0, scope="read", include_global_kinds=True,
|
|
demote_superseded=False,
|
|
)
|
|
except Exception:
|
|
logger.warning("precedent search failed for idea %s", note_id, exc_info=True)
|
|
return []
|
|
return [(score, n) for score, n in hits if n.id in idea_ids][:limit]
|
|
|
|
|
|
async def precedents(user_id: int, note_id: int, limit: int = 5) -> list[dict]:
|
|
"""The decisions on the ideas nearest this one by meaning — what a new
|
|
decision about it should be consistent with. Each idea contributes its
|
|
latest idea-level decision. Empty when nothing similar has been decided,
|
|
or when the embedder is unavailable."""
|
|
ranked = await nearest_ideas(user_id, note_id, limit)
|
|
if not ranked:
|
|
return []
|
|
async with async_session() as session:
|
|
latest = await _latest_idea_decisions(session, {n.id for _, n in ranked})
|
|
decisions = {
|
|
d.id: d for d in (await session.execute(
|
|
select(FamilyDecision).where(FamilyDecision.id.in_(list(latest.values())))
|
|
)).scalars().all()
|
|
}
|
|
out = []
|
|
for score, n in ranked:
|
|
d = decisions.get(latest.get(n.id))
|
|
if d is not None:
|
|
item = _decision_dict(d, n.title)
|
|
item["similarity"] = round(float(score), 3)
|
|
out.append(item)
|
|
return out
|
|
|
|
|
|
# --- writes ---------------------------------------------------------------------
|
|
|
|
def _log(session, *, idea_id: int, action: str, reason: str, before, after,
|
|
evidence: dict | None, precedent_ids: list[int] | None, decided_via: str,
|
|
user_id: int | None, project_id: int | None = None) -> FamilyDecision:
|
|
row = FamilyDecision(
|
|
idea_id=idea_id, project_id=project_id, action=action, reason=reason.strip(),
|
|
before=before, after=after, evidence=evidence or {},
|
|
precedent_ids=list(dict.fromkeys(precedent_ids or [])),
|
|
decided_via=decided_via, user_id=user_id,
|
|
)
|
|
session.add(row)
|
|
return row
|
|
|
|
|
|
async def _resolve_platforms(session, slugs: list[str]) -> dict[str, int]:
|
|
wanted = list(dict.fromkeys(s.strip() for s in slugs if s and s.strip()))
|
|
if not wanted:
|
|
return {}
|
|
rows = (await session.execute(
|
|
select(Platform.slug, Platform.id)
|
|
.where(Platform.slug.in_(wanted), Platform.deleted_at.is_(None))
|
|
)).all()
|
|
known = dict(rows)
|
|
unknown = [s for s in wanted if s not in known]
|
|
if unknown:
|
|
raise ValueError(f"unknown platform(s): {', '.join(unknown)} (list_platforms)")
|
|
return known
|
|
|
|
|
|
async def _set_platforms(session, note_id: int, platform_ids) -> None:
|
|
await session.execute(delete(FamilyIdeaPlatform).where(FamilyIdeaPlatform.note_id == note_id))
|
|
for pid in dict.fromkeys(platform_ids):
|
|
session.add(FamilyIdeaPlatform(note_id=note_id, platform_id=pid))
|
|
|
|
|
|
async def _open_ledger(session, user_id: int, note_id: int) -> int:
|
|
"""An `unassessed` row for every member project of the idea's platforms
|
|
that the promoter can write and that has no row yet. Returns how many."""
|
|
project_ids = set((await session.execute(
|
|
select(ProjectPlatform.project_id)
|
|
.join(FamilyIdeaPlatform, FamilyIdeaPlatform.platform_id == ProjectPlatform.platform_id)
|
|
.join(Project, Project.id == ProjectPlatform.project_id)
|
|
.where(
|
|
FamilyIdeaPlatform.note_id == note_id,
|
|
ProjectPlatform.state.in_(MEMBER_STATES),
|
|
Project.deleted_at.is_(None),
|
|
)
|
|
)).scalars().all())
|
|
answered = set((await session.execute(
|
|
select(FamilyAdoption.project_id).where(FamilyAdoption.idea_id == note_id)
|
|
)).scalars().all())
|
|
opened = 0
|
|
for pid in sorted(project_ids - answered):
|
|
# Rule 78: a promotion reaches only the projects its promoter could
|
|
# have written an answer into themselves.
|
|
if await access.can_write_project(user_id, pid):
|
|
session.add(FamilyAdoption(project_id=pid, idea_id=note_id, status="unassessed"))
|
|
opened += 1
|
|
return opened
|
|
|
|
|
|
async def _close_unassessed(session, note_id: int) -> int:
|
|
result = await session.execute(
|
|
delete(FamilyAdoption).where(
|
|
FamilyAdoption.idea_id == note_id, FamilyAdoption.status == "unassessed",
|
|
)
|
|
)
|
|
return result.rowcount or 0
|
|
|
|
|
|
async def propose(
|
|
user_id: int, note_id: int, *, reason: str, trigger: str = "agent",
|
|
evidence: dict | None = None, applies_when: str | None = None,
|
|
decided_via: str = "agent",
|
|
) -> tuple[dict, bool]:
|
|
"""Record a note as a family-idea CANDIDATE. Returns (idea, created).
|
|
|
|
Idempotent: an existing idea comes back unchanged and nothing is logged —
|
|
a trigger firing on every edit of a record must not grow the log. Write-
|
|
gated on the note: an idea is state on someone's record (rule 78).
|
|
"""
|
|
if not (reason or "").strip():
|
|
raise ValueError("a proposal needs a reason — what makes this a family idea?")
|
|
if not await access.can_write_note(user_id, note_id):
|
|
raise ValueError(f"note {note_id} not found or no write access")
|
|
async with async_session() as session:
|
|
idea = await session.get(FamilyIdea, note_id)
|
|
if idea is not None:
|
|
return idea.to_dict(), False
|
|
idea = FamilyIdea(note_id=note_id, status="candidate",
|
|
applies_when=(applies_when or "").strip() or None)
|
|
session.add(idea)
|
|
await session.flush()
|
|
_log(
|
|
session, idea_id=note_id, action="propose", reason=reason, before=None,
|
|
after=await _snapshot(session, idea),
|
|
evidence={"trigger": trigger, **(evidence or {})},
|
|
precedent_ids=None, decided_via=decided_via, user_id=user_id,
|
|
)
|
|
await session.commit()
|
|
return idea.to_dict(), True
|
|
|
|
|
|
def vetoes(*, applies_when: str, platforms: list[str], criteria: dict, evidence: list) -> list[str]:
|
|
"""The criteria a promotion fails, by key. Pure.
|
|
|
|
Each fails ON ITS OWN when its support is missing:
|
|
- platform_terms — no reasoning, no `applies_when`, or no platform scope;
|
|
- platform_problem — no reasoning;
|
|
- proven — no reasoning, or no named evidence.
|
|
"""
|
|
def said(key: str) -> bool:
|
|
return bool(str(criteria.get(key) or "").strip())
|
|
|
|
failed = []
|
|
if not said("platform_terms") or not (applies_when or "").strip() or not platforms:
|
|
failed.append("platform_terms")
|
|
if not said("platform_problem"):
|
|
failed.append("platform_problem")
|
|
if not said("proven") or not [e for e in evidence or [] if str(e).strip()]:
|
|
failed.append("proven")
|
|
return failed
|
|
|
|
|
|
async def promote(
|
|
user_id: int, note_id: int, *, applies_when: str, platforms: list[str],
|
|
criteria: dict, evidence: list[str], reason: str,
|
|
precedent_ids: list[int] | None = None, decided_via: str = "agent",
|
|
) -> dict:
|
|
"""Evaluate a record against the three criteria and promote it to canon.
|
|
|
|
A record nobody proposed may be promoted directly — the candidate row is
|
|
created on the way. Any criterion without support vetoes the promotion:
|
|
the idea stays (or becomes) a candidate and the veto is logged as a
|
|
`propose` decision naming what failed, so it is precedent too.
|
|
|
|
On promotion: status canon, `applies_when` and the platform scope set, the
|
|
version bumped if this idea has been promoted before (so rows judged
|
|
against the earlier canon read as needing a recheck), an `unassessed`
|
|
ledger row opened per member project, and a decision logged with the
|
|
criteria reasoning, the evidence and the precedents consulted.
|
|
|
|
Raises ValueError on a malformed call (no reason, unknown platform, no
|
|
write access, already canon) — before anything is written.
|
|
"""
|
|
if not (reason or "").strip():
|
|
raise ValueError("a promotion needs a reason")
|
|
if not await access.can_write_note(user_id, note_id):
|
|
raise ValueError(f"note {note_id} not found or no write access")
|
|
criteria = {k: str((criteria or {}).get(k) or "").strip() for k in CRITERIA_KEYS}
|
|
evidence = [str(e).strip() for e in evidence or [] if str(e).strip()]
|
|
consulted = await precedents(user_id, note_id)
|
|
named = await _existing_decision_ids(precedent_ids or [])
|
|
precedent_list = list(dict.fromkeys(named + [p["id"] for p in consulted]))
|
|
|
|
async with async_session() as session:
|
|
known = await _resolve_platforms(session, platforms or [])
|
|
idea = await session.get(FamilyIdea, note_id)
|
|
if idea is not None and idea.status == "canon":
|
|
raise ValueError(
|
|
f"#{note_id} is already canon (version {idea.canon_version}); "
|
|
"retire it first, or record a revision"
|
|
)
|
|
created_now = idea is None
|
|
if created_now:
|
|
idea = FamilyIdea(note_id=note_id, status="candidate")
|
|
session.add(idea)
|
|
await session.flush()
|
|
# A row made by this call had no prior state: `before` says so, which
|
|
# also makes a veto that created a candidate undoable (it changed
|
|
# something — a record became a candidate).
|
|
before = None if created_now else await _snapshot(session, idea)
|
|
failed = vetoes(
|
|
applies_when=applies_when, platforms=list(known), criteria=criteria,
|
|
evidence=evidence,
|
|
)
|
|
record = {"criteria": criteria, "evidence": evidence}
|
|
if failed:
|
|
decision = _log(
|
|
session, idea_id=note_id, action="propose",
|
|
reason=f"held as a candidate — fails {', '.join(failed)}: {reason.strip()}",
|
|
before=before, after=await _snapshot(session, idea),
|
|
evidence={**record, "vetoed_by": failed},
|
|
precedent_ids=precedent_list, decided_via=decided_via, user_id=user_id,
|
|
)
|
|
await session.commit()
|
|
await session.refresh(decision)
|
|
return {
|
|
"promoted": False, "vetoed_by": failed, "idea": idea.to_dict(),
|
|
"decision": decision.to_dict(), "precedents": consulted,
|
|
}
|
|
|
|
# Re-promotion moves the version PAST every version this idea has ever
|
|
# held — including one an undo rolled back from — so no row judged
|
|
# against an earlier canon can read as agreeing with this one.
|
|
if await _promoted_before(session, note_id):
|
|
idea.canon_version = await next_version(session, idea)
|
|
idea.status = "canon"
|
|
idea.applies_when = applies_when.strip()
|
|
idea.updated_at = _now()
|
|
await _set_platforms(session, note_id, known.values())
|
|
await session.flush()
|
|
opened = await _open_ledger(session, user_id, note_id)
|
|
decision = _log(
|
|
session, idea_id=note_id, action="promote", reason=reason,
|
|
before=before, after=await _snapshot(session, idea),
|
|
evidence={**record, "ledger_rows_opened": opened},
|
|
precedent_ids=precedent_list, decided_via=decided_via, user_id=user_id,
|
|
)
|
|
await session.commit()
|
|
await session.refresh(decision)
|
|
return {
|
|
"promoted": True, "idea": idea.to_dict(), "ledger_rows_opened": opened,
|
|
"decision": decision.to_dict(), "precedents": consulted,
|
|
}
|
|
|
|
|
|
async def _promoted_before(session, note_id: int) -> bool:
|
|
return bool(await session.scalar(
|
|
select(func.count()).select_from(FamilyDecision)
|
|
.where(FamilyDecision.idea_id == note_id, FamilyDecision.action == "promote")
|
|
))
|
|
|
|
|
|
async def next_version(session, idea: FamilyIdea) -> int:
|
|
"""One past every canon version this idea has ever held — including one
|
|
an undo rolled back from — so no answer given against an earlier canon
|
|
can read as agreeing with the new one. Only idea-level snapshots carry an
|
|
idea version; an assessment's `after` holds the version it was judged
|
|
against, which is never ahead of the idea's."""
|
|
rows = (await session.execute(
|
|
select(FamilyDecision.after).where(
|
|
FamilyDecision.idea_id == idea.note_id, FamilyDecision.project_id.is_(None),
|
|
)
|
|
)).scalars().all()
|
|
seen = [idea.canon_version or 1] + [int(a.get("canon_version") or 1) for a in rows if a]
|
|
return max(seen) + 1
|
|
|
|
|
|
async def _close_unreached(session, note_id: int) -> int:
|
|
"""Delete the `unassessed` rows of projects no longer on any of the
|
|
idea's platforms — nobody judged them, and the idea no longer reaches
|
|
them. Judged rows stay, as history."""
|
|
members = (
|
|
select(ProjectPlatform.project_id)
|
|
.join(FamilyIdeaPlatform, FamilyIdeaPlatform.platform_id == ProjectPlatform.platform_id)
|
|
.where(FamilyIdeaPlatform.note_id == note_id, ProjectPlatform.state.in_(MEMBER_STATES))
|
|
)
|
|
result = await session.execute(
|
|
delete(FamilyAdoption).where(
|
|
FamilyAdoption.idea_id == note_id, FamilyAdoption.status == "unassessed",
|
|
FamilyAdoption.project_id.not_in(members),
|
|
)
|
|
)
|
|
return result.rowcount or 0
|
|
|
|
|
|
async def revise(
|
|
user_id: int, note_id: int, *, reason: str, applies_when: str | None = None,
|
|
platforms: list[str] | None = None, evidence: list[str] | None = None,
|
|
decided_via: str = "agent",
|
|
) -> dict:
|
|
"""Record that a canon idea's SUBSTANCE changed — its note was rewritten,
|
|
its applicability narrowed or widened, or its platforms changed.
|
|
|
|
The version moves, so every project's answer given against the earlier
|
|
version reads as needing a recheck (derived, never a stored flag). A
|
|
project newly in scope gets an `unassessed` row; an `unassessed` row of a
|
|
project no longer in scope is closed. Undoable like any idea-level
|
|
decision.
|
|
|
|
`applies_when` and `platforms` are left alone when None; given, they
|
|
replace the old ones and may not be empty (canon is always scoped).
|
|
"""
|
|
if not (reason or "").strip():
|
|
raise ValueError("a revision needs a reason — what changed in the idea?")
|
|
if not await access.can_write_note(user_id, note_id):
|
|
raise ValueError(f"note {note_id} not found or no write access")
|
|
if applies_when is not None and not applies_when.strip():
|
|
raise ValueError("canon needs an 'applies when' — leave it out to keep the current one")
|
|
if platforms is not None and not [p for p in platforms if (p or "").strip()]:
|
|
raise ValueError("canon needs at least one platform — leave it out to keep the current ones")
|
|
evidence = [str(e).strip() for e in evidence or [] if str(e).strip()]
|
|
async with async_session() as session:
|
|
idea = await session.get(FamilyIdea, note_id)
|
|
if idea is None or idea.status != "canon":
|
|
raise ValueError(f"#{note_id} is not family canon — only canon is revised")
|
|
known = await _resolve_platforms(session, platforms) if platforms is not None else None
|
|
before = await _snapshot(session, idea)
|
|
idea.canon_version = await next_version(session, idea)
|
|
if applies_when is not None:
|
|
idea.applies_when = applies_when.strip()
|
|
if known is not None:
|
|
await _set_platforms(session, note_id, known.values())
|
|
idea.updated_at = _now()
|
|
await session.flush()
|
|
opened = await _open_ledger(session, user_id, note_id)
|
|
closed = await _close_unreached(session, note_id)
|
|
decision = _log(
|
|
session, idea_id=note_id, action="revise", reason=reason, before=before,
|
|
after=await _snapshot(session, idea),
|
|
evidence={"evidence": evidence, "ledger_rows_opened": opened,
|
|
"ledger_rows_closed": closed},
|
|
precedent_ids=None, decided_via=decided_via, user_id=user_id,
|
|
)
|
|
await session.commit()
|
|
await session.refresh(decision)
|
|
return {"idea": idea.to_dict(), "decision": decision.to_dict(),
|
|
"ledger_rows_opened": opened, "ledger_rows_closed": closed}
|
|
|
|
|
|
async def set_topic(user_id: int, note_id: int, topic_id: int) -> dict:
|
|
"""Link an idea to the rule topic holding its binding norms, or unlink it
|
|
with topic_id 0. The note carries the idea; the topic carries the rules a
|
|
project on its platforms must follow. One idea per topic, since a topic's
|
|
norms belong to one standard.
|
|
|
|
Not a change of substance, so the version does not move: the norms were
|
|
rules before the link and are the same rules after it. Re-linking is the
|
|
undo, as with the references."""
|
|
if not await access.can_write_note(user_id, note_id):
|
|
raise ValueError(f"note {note_id} not found or no write access")
|
|
async with async_session() as session:
|
|
idea = await session.get(FamilyIdea, note_id)
|
|
if idea is None:
|
|
raise ValueError(f"#{note_id} is not a family idea")
|
|
if topic_id:
|
|
if await rulebooks_svc.get_topic(topic_id, user_id) is None:
|
|
raise ValueError(f"topic {topic_id} not found in a rulebook you own")
|
|
holder = (await session.execute(
|
|
select(FamilyIdea.note_id, Note.title)
|
|
.join(Note, Note.id == FamilyIdea.note_id)
|
|
.where(FamilyIdea.topic_id == topic_id, FamilyIdea.note_id != note_id)
|
|
)).first()
|
|
if holder is not None:
|
|
raise ValueError(
|
|
f"topic {topic_id} already carries the norms of #{holder[0]} "
|
|
f"“{holder[1]}” — unlink it there first, or merge the two ideas"
|
|
)
|
|
idea.topic_id = topic_id or None
|
|
idea.updated_at = _now()
|
|
await session.commit()
|
|
out = idea.to_dict()
|
|
out["topic"] = await _topic(session, user_id, idea.topic_id, rules=True)
|
|
return out
|
|
|
|
|
|
async def _existing_decision_ids(ids: list[int]) -> list[int]:
|
|
ids = [int(i) for i in ids if i]
|
|
if not ids:
|
|
return []
|
|
async with async_session() as session:
|
|
found = set((await session.execute(
|
|
select(FamilyDecision.id).where(FamilyDecision.id.in_(ids))
|
|
)).scalars().all())
|
|
missing = [i for i in ids if i not in found]
|
|
if missing:
|
|
raise ValueError(f"no such family decision(s): {', '.join(map(str, missing))}")
|
|
return ids
|
|
|
|
|
|
async def retire(user_id: int, note_id: int, *, reason: str, decided_via: str = "agent") -> dict:
|
|
"""Demote an idea. Its history and its judged ledger rows are kept; the
|
|
rows nobody judged are closed. Undoable."""
|
|
if not (reason or "").strip():
|
|
raise ValueError("retiring an idea needs a reason")
|
|
if not await access.can_write_note(user_id, note_id):
|
|
raise ValueError(f"note {note_id} not found or no write access")
|
|
async with async_session() as session:
|
|
idea = await session.get(FamilyIdea, note_id)
|
|
if idea is None:
|
|
raise ValueError(f"#{note_id} is not a family idea")
|
|
if idea.status == "retired":
|
|
raise ValueError(f"#{note_id} is already retired")
|
|
before = await _snapshot(session, idea)
|
|
idea.status = "retired"
|
|
idea.updated_at = _now()
|
|
closed = await _close_unassessed(session, note_id)
|
|
decision = _log(
|
|
session, idea_id=note_id, action="retire", reason=reason, before=before,
|
|
after=await _snapshot(session, idea), evidence={"ledger_rows_closed": closed},
|
|
precedent_ids=None, decided_via=decided_via, user_id=user_id,
|
|
)
|
|
await session.commit()
|
|
await session.refresh(decision)
|
|
return {"idea": idea.to_dict(), "decision": decision.to_dict()}
|
|
|
|
|
|
async def undo(user_id: int, decision_id: int, *, reason: str, decided_via: str = "agent") -> dict:
|
|
"""Reverse a decision, restoring the state it recorded as `before`. A
|
|
project's assessment is handed to the adoption ledger
|
|
(family_adoption.undo_assessment); the rest of this is idea-level.
|
|
|
|
Only the LATEST idea-level decision on an idea can be undone —
|
|
undoing an older one would rewrite a state later decisions were built on.
|
|
The undo is itself a decision, naming the one it reverses as its
|
|
precedent, so the history keeps both.
|
|
|
|
Undoing the proposal that created an idea retires it rather than deleting
|
|
it: deleting the idea would take its decision log with it.
|
|
"""
|
|
if not (reason or "").strip():
|
|
raise ValueError("an undo needs a reason")
|
|
async with async_session() as session:
|
|
target = await session.get(FamilyDecision, decision_id)
|
|
if target is None:
|
|
raise ValueError(f"no such family decision: {decision_id}")
|
|
if target.project_id is not None:
|
|
# One project's answer: the adoption ledger restores the row.
|
|
from scribe.services import family_adoption
|
|
return await family_adoption.undo_assessment(
|
|
user_id, target, reason=reason, decided_via=decided_via)
|
|
if not await access.can_write_note(user_id, target.idea_id):
|
|
raise ValueError(f"decision {decision_id} not found or no write access")
|
|
async with async_session() as session:
|
|
latest_id = (await _latest_idea_decisions(session, {target.idea_id})).get(target.idea_id)
|
|
if latest_id != target.id or not _can_undo(target):
|
|
if not _can_undo(target):
|
|
why = ("it changed nothing" if target.before == target.after
|
|
else f"a '{target.action}' decision is not undone this way")
|
|
else:
|
|
why = f"decision {latest_id} came after it — undo that one first"
|
|
raise ValueError(f"decision {decision_id} cannot be undone: {why}")
|
|
idea = await session.get(FamilyIdea, target.idea_id)
|
|
before = await _snapshot(session, idea)
|
|
# No recorded prior state means the decision CREATED the idea: before
|
|
# it, the record had no applicability test and no scope. It comes back
|
|
# as retired, with neither, rather than being deleted with its log.
|
|
prior = target.before or {
|
|
"status": "retired", "applies_when": "",
|
|
"canon_version": idea.canon_version, "platforms": [],
|
|
}
|
|
if prior["status"] == "canon" and not (prior.get("applies_when") or "").strip():
|
|
raise ValueError("the recorded prior state is canon with no 'applies when'")
|
|
known = await _resolve_platforms(session, prior.get("platforms") or [])
|
|
idea.status = prior["status"]
|
|
idea.applies_when = (prior.get("applies_when") or "").strip() or None
|
|
idea.canon_version = prior.get("canon_version") or idea.canon_version
|
|
idea.updated_at = _now()
|
|
await _set_platforms(session, idea.note_id, known.values())
|
|
await session.flush()
|
|
ledger: dict = {}
|
|
if idea.status == "canon":
|
|
ledger["ledger_rows_opened"] = await _open_ledger(session, user_id, idea.note_id)
|
|
else:
|
|
ledger["ledger_rows_closed"] = await _close_unassessed(session, idea.note_id)
|
|
decision = _log(
|
|
session, idea_id=idea.note_id, action="undo", reason=reason,
|
|
before=before, after=await _snapshot(session, idea),
|
|
evidence={"undid_action": target.action, **ledger},
|
|
precedent_ids=[target.id], decided_via=decided_via, user_id=user_id,
|
|
)
|
|
await session.commit()
|
|
await session.refresh(decision)
|
|
return {"idea": idea.to_dict(), "decision": decision.to_dict()}
|
|
|
|
|
|
# --- triggers -------------------------------------------------------------------
|
|
|
|
def lineage_citations(text: str | None) -> list[int]:
|
|
"""The `#N`s in a text that are cited as the SOURCE of a pattern — a
|
|
lineage word ("matching", "ported from", "same shape as", …) in the same
|
|
clause just before the reference. Pure, in order, de-duplicated."""
|
|
from scribe.services.record_refs import REF_RE
|
|
|
|
found: list[int] = []
|
|
for m in REF_RE.finditer(text or ""):
|
|
start = max(0, m.start() - _LINEAGE_WINDOW)
|
|
window = text[start:m.start()]
|
|
window = re.split(r"[\n.;]", window)[-1]
|
|
if _LINEAGE.search(window):
|
|
n = int(m.group(1))
|
|
if n not in found:
|
|
found.append(n)
|
|
return found
|
|
|
|
|
|
async def _member_platform_ids(session, project_id: int) -> set[int]:
|
|
return set((await session.execute(
|
|
select(ProjectPlatform.platform_id).where(
|
|
ProjectPlatform.project_id == project_id,
|
|
ProjectPlatform.state.in_(MEMBER_STATES),
|
|
)
|
|
)).scalars().all())
|
|
|
|
|
|
def _evaluate_line(note_id: int) -> str:
|
|
return (
|
|
f"Evaluate it now: get_family_idea({note_id}) shows the three criteria and "
|
|
"the nearest precedents; then promote_family_idea if all three hold, or "
|
|
"leave it a candidate (promote_family_idea records which criterion "
|
|
"failed). No one approves this — the criteria decide."
|
|
)
|
|
|
|
|
|
async def citation_trigger(user_id: int, note) -> str | None:
|
|
"""A record that cites ANOTHER project's record as the source of its
|
|
pattern opens an evaluation of that source as a family idea. Silent on a
|
|
citation without lineage words, and on a citation within one project."""
|
|
if not getattr(note, "project_id", None):
|
|
return None
|
|
cited = [n for n in lineage_citations(note.body) if n != note.id]
|
|
if not cited:
|
|
return None
|
|
async with async_session() as session:
|
|
rows = (await session.execute(
|
|
select(Note.id, Note.title, Note.project_id, Project.title)
|
|
.join(Project, Project.id == Note.project_id)
|
|
.where(
|
|
Note.id.in_(cited), Note.deleted_at.is_(None),
|
|
Note.project_id.is_not(None), Note.project_id != note.project_id,
|
|
)
|
|
)).all()
|
|
for source_id, source_title, _pid, project_title in sorted(rows, key=lambda r: cited.index(r[0])):
|
|
if not await access.can_write_note(user_id, source_id):
|
|
continue
|
|
idea, created = await propose(
|
|
user_id, source_id, trigger="citation", decided_via="system",
|
|
reason=f"#{note.id} cites it as the source of its pattern, across projects",
|
|
evidence={"cited_by": {"id": note.id, "title": note.title}},
|
|
)
|
|
if idea["status"] == "canon":
|
|
return (
|
|
f"This record follows #{source_id} \"{source_title}\" ({project_title}), "
|
|
"which is already family canon. This project answers it through "
|
|
"its adoption ledger rather than by copying it."
|
|
)
|
|
lead = "is now a family-idea candidate" if created else "is already a candidate"
|
|
return (
|
|
f"This record cites #{source_id} \"{source_title}\" ({project_title}) as "
|
|
f"the source of its pattern — an idea carried between projects, which "
|
|
f"{lead}. {_evaluate_line(source_id)}"
|
|
)
|
|
return None
|
|
|
|
|
|
async def repeat_trigger(user_id: int, note) -> str | None:
|
|
"""A new record whose meaning repeats a record in ANOTHER project that
|
|
shares a platform with this one opens an evaluation of the earlier record.
|
|
Silent when the projects share no platform, below the threshold, or when
|
|
the project has no platforms yet."""
|
|
from scribe.services.embeddings import embedding_text, semantic_search_notes
|
|
|
|
if not getattr(note, "project_id", None) or note.is_task:
|
|
return None
|
|
async with async_session() as session:
|
|
mine = await _member_platform_ids(session, note.project_id)
|
|
if not mine:
|
|
return None
|
|
hits = await semantic_search_notes(
|
|
user_id, embedding_text(note.title, note.body), exclude_ids={note.id},
|
|
limit=8, threshold=REPEAT_THRESHOLD, scope="read", note_type=_REPEAT_KINDS,
|
|
is_task=False, demote_superseded=False,
|
|
)
|
|
for score, other in hits:
|
|
if not other.project_id or other.project_id == note.project_id:
|
|
continue
|
|
async with async_session() as session:
|
|
shared = mine & await _member_platform_ids(session, other.project_id)
|
|
if not shared or not await access.can_write_note(user_id, other.id):
|
|
continue
|
|
idea, created = await propose(
|
|
user_id, other.id, trigger="repeat", decided_via="system",
|
|
reason=(f"#{note.id} repeats it in another project on a shared platform "
|
|
f"(similarity {score:.2f})"),
|
|
evidence={"repeated_by": {"id": note.id, "title": note.title},
|
|
"similarity": round(float(score), 3)},
|
|
)
|
|
if idea["status"] == "canon":
|
|
return (
|
|
f"This record repeats #{other.id} \"{other.title}\", which is already "
|
|
"family canon on a platform this project shares. Build from it."
|
|
)
|
|
lead = "is now a family-idea candidate" if created else "is already a candidate"
|
|
return (
|
|
f"This record repeats #{other.id} \"{other.title}\" from another project "
|
|
f"on a platform this one shares (similarity {score:.2f}). Two projects "
|
|
f"building the same idea is what family canon is for; #{other.id} {lead}. "
|
|
f"{_evaluate_line(other.id)}"
|
|
)
|
|
return None
|
|
|
|
|
|
async def milestone_is_open(user_id: int, milestone_id: int) -> bool:
|
|
"""Whether a milestone is open right now — read before a status write, so
|
|
the trigger fires on the transition into done and not on every re-save of
|
|
a closed one. Fail-open to False: a hint must never break the write."""
|
|
from scribe.services import milestones as milestones_svc
|
|
|
|
try:
|
|
milestone = await milestones_svc.get_milestone(user_id, milestone_id)
|
|
except Exception:
|
|
logger.warning("milestone %s status read failed", milestone_id, exc_info=True)
|
|
return False
|
|
return milestone is not None and milestone.status != "done"
|
|
|
|
|
|
async def milestone_trigger(user_id: int, milestone) -> str | None:
|
|
"""A milestone closing in a project that is on a platform: the moment to
|
|
ask whether what it built is something every project on that platform
|
|
will face. Records nothing — a milestone is not a record an idea can hang
|
|
on — so the hint asks for the note that would carry the idea."""
|
|
project_id = getattr(milestone, "project_id", None)
|
|
try:
|
|
if not project_id or not await access.can_read_project(user_id, project_id):
|
|
return None
|
|
async with async_session() as session:
|
|
names = (await session.execute(
|
|
select(Platform.name)
|
|
.join(ProjectPlatform, ProjectPlatform.platform_id == Platform.id)
|
|
.where(ProjectPlatform.project_id == project_id,
|
|
ProjectPlatform.state.in_(MEMBER_STATES),
|
|
Platform.deleted_at.is_(None))
|
|
.order_by(Platform.order_index.asc())
|
|
)).scalars().all()
|
|
except Exception:
|
|
# Fail-open: the milestone is already closed; the hint is decoration.
|
|
logger.warning("milestone trigger failed for %s", getattr(milestone, "id", None),
|
|
exc_info=True)
|
|
return None
|
|
if not names:
|
|
return None
|
|
return (
|
|
f"This milestone closed on {', '.join(names)}. If it solved something every "
|
|
"project on those platforms will face — a problem the platform causes, or a "
|
|
"stance held across projects — the idea belongs in the family: write it up "
|
|
"as a note (when it applies, the traps, what proved it) and "
|
|
"propose_family_idea it, or promote_family_idea if it already meets the "
|
|
"three criteria. If what it built is this project's alone, nothing to do."
|
|
)
|
|
|
|
|
|
async def attach_family_hint(user_id: int, data: dict, note, *, created: bool) -> None:
|
|
"""Ride the trigger hints on a write's response — fail-open: a hint must
|
|
never break the write it rides on. Citation first (an explicit claim of
|
|
lineage), then, on a create, the repeat check."""
|
|
try:
|
|
hint = await citation_trigger(user_id, note)
|
|
if hint is None and created:
|
|
hint = await repeat_trigger(user_id, note)
|
|
if hint:
|
|
data["family_hint"] = hint
|
|
except Exception:
|
|
logger.warning("family trigger failed for note %s", getattr(note, "id", None), exc_info=True)
|