feat(family): platforms, family ideas, the adoption ledger and its decision log (milestone 463 step 1, #4987)
CI & Build / TypeScript typecheck (push) Successful in 1m11s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 25s
CI & Build / integration (push) Successful in 1m50s
CI & Build / Python tests (push) Successful in 2m41s
CI & Build / Build & push image (push) Successful in 1m8s

When one project solves something every project on the same platform will
meet, that solution becomes family canon and every other project on the
platform answers it. This is the storage for that.

- platforms: a global catalog in the canonical_systems shape, seeded with
  generic technology names and the file markers step 2's detection reads.
- project_platforms: declared / detected / rejected. A rejected row is kept
  so detection cannot re-add what a person said no to.
- family_ideas: a note's family state. No new record type; any note, snippet
  or lesson becomes an idea. A canon idea must state when it applies (CHECK).
- family_idea_platforms: the only scope source. A linked rule topic takes its
  scope from the idea, so the two cannot disagree.
- family_idea_references: reference implementations, explicit not inferred.
- family_adoptions: one answer per (project, idea). Variant and exempt require
  a reason (CHECK). Recheck is derived from the two canon versions, never
  stored.
- family_decisions: the append-only log, with a required reason and the
  earlier decisions each one followed. The agent decides with no approval
  step, so precedent is what keeps its calls consistent.

Backup v25 carries all seven: platforms by slug, precedent ids remapped
through the decision map. Both column guards cover the new tables, and a
real-Postgres test exercises the CHECKs and the restore remaps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 09:33:27 -04:00
co-authored by Claude Opus 5.5
parent ccbccb025c
commit e68ccc5884
7 changed files with 1474 additions and 4 deletions
+6
View File
@@ -89,3 +89,9 @@ from scribe.models.forge_connection import ForgeConnection # noqa: E402, F401
from scribe.models.code_shape import CodeShape, CodeShapeConsumer, CodeShapeEvent, CodeShapeUse # noqa: E402, F401
from scribe.models.system import System, RecordSystem # noqa: E402, F401
from scribe.models.design_system import DesignSystem, DesignToken # noqa: E402, F401
# After notes, projects and rulebook topics: family canon foreign-keys all
# three (milestone 463).
from scribe.models.family import ( # noqa: E402, F401
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform,
FamilyIdeaReference, Platform, ProjectPlatform,
)
+357
View File
@@ -0,0 +1,357 @@
"""Family canon (milestone 463): ideas that every project on a platform shares,
and each project's recorded answer to them.
A FAMILY IDEA is not a new kind of record. It is an existing `notes` row — a
note, a snippet, a lesson — that has been given a platform scope. The note
carries the idea (when it applies, the traps, the checklist, the incidents
behind them); an optional rulebook topic carries the norms that bind; snippets
are the reference implementations, one per language. A new record type would
be one more thing reached for interchangeably with rules, processes, snippets
and design systems, and nothing here needs one.
What is shared is the IDEA, not the code. Each project implements it in its own
language; identical code is a by-product and never the test.
The tables, and the one job each does:
- `platforms` — the global catalog of what a project can be built on or ship
as. Same shape and reasoning as `canonical_systems`: no owner, so the same
word means the same thing in every project on the install.
- `project_platforms` — which platforms a project is. Membership is how
"is this in family?" stops being a judgment made per idea and becomes a
lookup.
- `family_ideas` — a note's family state: candidate, canon or retired, its
applicability test, its canon version, its linked rule topic.
- `family_idea_platforms` — the platforms an idea is for. The ONLY scope
source: a linked topic takes its scope from here rather than carrying its
own, so the two can never disagree.
- `family_idea_references` — the reference implementations.
- `family_adoptions` — one row per (project, idea): the project's answer.
- `family_decisions` — the append-only log of every promotion and every
ledger change, with its reason and the earlier decisions it followed. The
agent makes these calls with no approval step, so the log is how they stay
consistent (precedent) and how a person reviews or undoes one.
"""
from datetime import datetime
from sqlalchemy import (
BigInteger, DateTime, ForeignKey, Index, Integer, Text, text,
)
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from scribe.models import Base
from scribe.models.base import CreatedAtMixin, SoftDeleteMixin, TimestampMixin, iso
# Each tuple is the whitelist its CHECK constraint enforces (migration 0120).
# Rule 36: a new value means DROP + ADD CONSTRAINT in the same migration.
MEMBERSHIP_STATES = ("declared", "detected", "rejected")
IDEA_STATUSES = ("candidate", "canon", "retired")
ADOPTION_STATUSES = ("unassessed", "adopted", "variant", "exempt", "owed")
# The two adoption answers that are a departure, and so must say why.
REASONED_STATUSES = ("variant", "exempt")
DECISION_ACTIONS = ("propose", "promote", "revise", "retire", "assess", "undo")
DECIDERS = ("agent", "operator", "system")
class Platform(Base, TimestampMixin, SoftDeleteMixin):
"""What a project is built on or ships as — a runtime, a delivery channel
or a toolchain whose own behaviour causes the problems a family idea
answers.
FLAT, deliberately. A project declares several, and an idea is for
several, so "Android app and container image" needs no hierarchy to
express. Applicability narrower than a platform (an idea that only matters
to a sideloaded APK, not a store-distributed one) belongs in the idea's
`applies_when`, where a project it does not fit answers `exempt` with the
reason. Splitting platforms to carry that would grow the catalog every
time an idea got more specific.
GLOBAL, like `canonical_systems`: no owner, so a shared project inherits
the vocabulary rather than re-earning it. `slug` is the match key and the
form a backup carries, since ids are per-install.
`markers` are glob patterns over a bound repo's paths that suggest a
project is this platform (step 2's detection). A pattern without a slash
matches a file's basename anywhere in the tree; one with a slash matches
the repo-relative path. Empty means declare-only: some platforms leave no
reliable file behind.
"""
__tablename__ = "platforms"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
name: Mapped[str] = mapped_column(Text, nullable=False)
slug: Mapped[str] = mapped_column(Text, nullable=False)
description: Mapped[str | None] = mapped_column(Text, nullable=True)
markers: Mapped[list] = mapped_column(
JSONB, default=list, server_default=text("'[]'::jsonb"),
)
order_index: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
__table_args__ = (
# Unique among LIVE rows, so a retired entry doesn't block recreating
# the same platform (the canonical_systems convention).
Index(
"uq_platforms_slug", "slug",
unique=True, postgresql_where=text("deleted_at IS NULL"),
),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"name": self.name,
"slug": self.slug,
"description": self.description,
"markers": list(self.markers or []),
"order_index": self.order_index,
"created_at": iso(self.created_at),
"updated_at": iso(self.updated_at),
}
class ProjectPlatform(Base, CreatedAtMixin):
"""A project's answer to "are you this platform?".
Three states, because detection is automatic and must not be able to
override a person:
- ``declared`` — someone said so (at inception or in settings).
- ``detected`` — a marker in a bound repo said so.
- ``rejected`` — someone said NO. Kept as a row rather than deleted, so the
next refresh does not detect it straight back. Not membership.
Only `declared` and `detected` make a project a member of the platform's
family.
"""
__tablename__ = "project_platforms"
project_id: Mapped[int] = mapped_column(
Integer, ForeignKey("projects.id", ondelete="CASCADE"), primary_key=True,
)
platform_id: Mapped[int] = mapped_column(
Integer, ForeignKey("platforms.id", ondelete="CASCADE"), primary_key=True,
index=True,
)
# CHECK ck_project_platforms_state (migration 0120).
state: Mapped[str] = mapped_column(Text, default="declared", server_default="declared")
def to_dict(self) -> dict:
return {
"project_id": self.project_id,
"platform_id": self.platform_id,
"state": self.state,
"created_at": iso(self.created_at),
}
class FamilyIdea(Base, TimestampMixin):
"""A note's family state. Its presence is what makes the note a family
idea; the note itself is unchanged.
- ``candidate`` — recorded, not yet proven or not yet stated in platform
terms. Reaches nobody's ledger.
- ``canon`` — promoted. Every project on its platforms owes it an answer.
- ``retired`` — demoted. Kept, with its ledger, so the history reads.
`applies_when` is the applicability test every assessment starts from —
stated in platform terms, which is the first promotion criterion. A CHECK
refuses canon without one, so that criterion is the schema's to enforce
and not a sentence a session might skip.
`canon_version` moves whenever the idea's substance changes. An adoption
assessed against an older version needs rechecking — which is DERIVED by
comparing the two numbers, never stored as a flag that could go stale.
`topic_id` is the rule topic holding the idea's binding norms, if it has
any (the note↔topic link #3236 asked for). The topic takes its scope from
this idea; it never carries platforms of its own.
"""
__tablename__ = "family_ideas"
note_id: Mapped[int] = mapped_column(
Integer, ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True,
)
# CHECK ck_family_ideas_status (migration 0120).
status: Mapped[str] = mapped_column(Text, default="candidate", server_default="candidate")
# CHECK ck_family_ideas_canon_applies (migration 0120).
applies_when: Mapped[str | None] = mapped_column(Text, nullable=True)
canon_version: Mapped[int] = mapped_column(Integer, default=1, server_default="1")
topic_id: Mapped[int | None] = mapped_column(
BigInteger, ForeignKey("rulebook_topics.id", ondelete="SET NULL"), nullable=True,
)
__table_args__ = (
# One idea per topic: a topic's norms belong to one standard.
Index(
"uq_family_ideas_topic", "topic_id",
unique=True, postgresql_where=text("topic_id IS NOT NULL"),
),
Index("ix_family_ideas_status", "status"),
)
def to_dict(self) -> dict:
return {
"note_id": self.note_id,
"status": self.status,
"applies_when": self.applies_when or "",
"canon_version": self.canon_version,
"topic_id": self.topic_id,
"created_at": iso(self.created_at),
"updated_at": iso(self.updated_at),
}
class FamilyIdeaPlatform(Base, CreatedAtMixin):
"""A platform an idea is for. A join table with a model so the backup's
column guard can see it, and so the reverse lookup — every idea for a
platform — has an index."""
__tablename__ = "family_idea_platforms"
note_id: Mapped[int] = mapped_column(
Integer, ForeignKey("family_ideas.note_id", ondelete="CASCADE"), primary_key=True,
)
platform_id: Mapped[int] = mapped_column(
Integer, ForeignKey("platforms.id", ondelete="CASCADE"), primary_key=True,
index=True,
)
class FamilyIdeaReference(Base, CreatedAtMixin):
"""A reference implementation of an idea: a snippet a project can start
from. Explicit rather than inferred, because an owed task names "the
reference for your language", and a guess there sends a project to copy
the wrong thing. The snippet's language is read from the snippet."""
__tablename__ = "family_idea_references"
idea_id: Mapped[int] = mapped_column(
Integer, ForeignKey("family_ideas.note_id", ondelete="CASCADE"), primary_key=True,
)
snippet_id: Mapped[int] = mapped_column(
Integer, ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True,
index=True,
)
class FamilyAdoption(Base, TimestampMixin):
"""One project's answer to one family idea — the adoption ledger.
- ``unassessed`` — the idea reached the project and nobody has judged it.
- ``adopted`` — the project does it.
- ``variant`` — the project departs, for a reason that names a fact about
itself the canon did not account for. A preference is not a reason.
- ``exempt`` — the idea's `applies_when` is false for this project.
- ``owed`` — it applies, there is no reason to depart, and it is not done
yet. `owed_task_id` is the task filed in THIS project to do it.
`reason` is required for variant and exempt (CHECK), because those two are
the departures and the reason is the whole record. `canon_version` is the
idea version this answer was given against; NULL while unassessed.
The row is the CURRENT answer. How it got there — the reasons at each step
and the precedents each followed — is in `family_decisions`.
"""
__tablename__ = "family_adoptions"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
project_id: Mapped[int] = mapped_column(
Integer, ForeignKey("projects.id", ondelete="CASCADE"), index=True,
)
idea_id: Mapped[int] = mapped_column(
Integer, ForeignKey("family_ideas.note_id", ondelete="CASCADE"), index=True,
)
# CHECK ck_family_adoptions_status and ck_family_adoptions_reason (0120).
status: Mapped[str] = mapped_column(Text, default="unassessed", server_default="unassessed")
reason: Mapped[str | None] = mapped_column(Text, nullable=True)
canon_version: Mapped[int | None] = mapped_column(Integer, nullable=True)
assessed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# CHECK ck_family_adoptions_decided_via (0120). NULL while unassessed.
decided_via: Mapped[str | None] = mapped_column(Text, nullable=True)
owed_task_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("notes.id", ondelete="SET NULL"), nullable=True,
)
__table_args__ = (
Index("uq_family_adoptions_pair", "project_id", "idea_id", unique=True),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"project_id": self.project_id,
"idea_id": self.idea_id,
"status": self.status,
"reason": self.reason or "",
"canon_version": self.canon_version,
"assessed_at": iso(self.assessed_at),
"decided_via": self.decided_via,
"owed_task_id": self.owed_task_id,
"created_at": iso(self.created_at),
"updated_at": iso(self.updated_at),
}
class FamilyDecision(Base, CreatedAtMixin):
"""One decision about family canon — append-only.
`project_id` is NULL for a decision about the idea itself (propose,
promote, revise, retire) and set for a decision about one project's
answer (assess). `undo` reverses an earlier decision and names it in
`evidence`.
`before` and `after` are STATE snapshots — status, version, reason — and
deliberately hold no foreign ids: an id inside JSON cannot be remapped by a
restore, and would come back pointing at whatever took that number (the
#3182 trap). The ids this row needs are columns.
`precedent_ids` are the earlier decisions this one followed. They are what
keeps a call consistent with the last similar one when no person approves
either, so they are a list of ids into this same table, remapped at
restore through the decision map.
"""
__tablename__ = "family_decisions"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
idea_id: Mapped[int] = mapped_column(
Integer, ForeignKey("family_ideas.note_id", ondelete="CASCADE"), index=True,
)
project_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("projects.id", ondelete="CASCADE"), nullable=True, index=True,
)
# CHECK ck_family_decisions_action (0120).
action: Mapped[str] = mapped_column(Text)
reason: Mapped[str] = mapped_column(Text)
before: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
after: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
evidence: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
precedent_ids: Mapped[list] = mapped_column(
JSONB, default=list, server_default=text("'[]'::jsonb"),
)
# CHECK ck_family_decisions_decided_via (0120).
decided_via: Mapped[str] = mapped_column(Text, default="agent", server_default="agent")
# Who was acting — the session's user. The decider KIND is decided_via.
user_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("users.id", ondelete="SET NULL"), nullable=True,
)
def to_dict(self) -> dict:
return {
"id": self.id,
"idea_id": self.idea_id,
"project_id": self.project_id,
"action": self.action,
"reason": self.reason,
"before": self.before,
"after": self.after,
"evidence": self.evidence or {},
"precedent_ids": list(self.precedent_ids or []),
"decided_via": self.decided_via,
"user_id": self.user_id,
"created_at": iso(self.created_at),
}
+404 -1
View File
@@ -17,6 +17,10 @@ from scribe.models.system_usage import SystemUsageEvent
from scribe.models.moment_mapping import MomentMapping
from scribe.models.retrieval_tuning import RetrievalTuningEvent
from scribe.models.canonical_system import CanonicalSystem
from scribe.models.family import (
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform,
FamilyIdeaReference, Platform, ProjectPlatform,
)
from scribe.models.rulebook import (
RuleRelation, rule_moments as rule_moments_t, rule_systems as rule_systems_t,
)
@@ -112,8 +116,14 @@ logger = logging.getLogger(__name__)
# v24 (2026-10) added rule_moment_judgments.misfire (milestone 458 step 7b):
# the reports that a mounted rule arrived where it did not apply, and the
# operator's "keep it" that stops them being proposed as an unmount again.
# v25 (2026-10) added family canon (milestone 463): the platform catalog, each
# project's platforms, family ideas with their platforms and reference
# implementations, the adoption ledger and the decision log. The ledger is
# every project's recorded answer to a shared idea, and the log is the
# precedent each later answer follows. Lose either and every assessment is
# owed again, made fresh, with nothing to keep it consistent with the last.
# Bump when the serialized schema changes.
BACKUP_VERSION = 24
BACKUP_VERSION = 25
# Every table this backup carries, by its REAL name. Paired with _NOT_INCLUDED
# below, these two lists must together account for the entire schema — which is
@@ -164,6 +174,12 @@ _BACKED_UP = [
"rule_moments",
# v23 (2026-10): proposals and judgments about those mounts (step 7).
"rule_moment_judgments",
# v25 (2026-10): family canon (milestone 463). `platforms` is global like
# canonical_systems and rides every export for the same reason: the rows
# below name platforms by slug, and a partial catalog restores partial
# memberships.
"platforms", "project_platforms", "family_ideas", "family_idea_platforms",
"family_idea_references", "family_adoptions", "family_decisions",
]
# Tables intentionally NOT in the backup, surfaced in the payload so the gap is
@@ -298,6 +314,19 @@ _COLUMN_EXCLUSIONS: dict[str, set[str]] = {
},
"code_shape_events": set(),
"code_shape_uses": set(),
# Matched on SLUG at restore, like canonical_systems: a target install
# already seeded the standard platforms from its migrations.
"platforms": {"deleted_at", "deleted_batch_id", "id", "created_at", "updated_at"},
# The platform travels as `platform_slug` — ids are per-install.
"project_platforms": {"platform_id"},
"family_idea_platforms": {"platform_id"},
# Every column travels: the note and topic ids are SOURCE ids, remapped.
"family_ideas": set(),
"family_idea_references": set(),
"family_adoptions": {"id"},
# The id travels, unlike the other surrogate keys: precedent_ids point at
# it, so the restore needs the source id to build the decision map.
"family_decisions": set(),
}
@@ -381,6 +410,15 @@ _IMPORT_COLUMN_EXCLUSIONS: dict[str, set[str]] = {
},
"code_shape_events": {"id"},
"code_shape_uses": {"id"},
"platforms": {"id", "deleted_at", "deleted_batch_id", "created_at", "updated_at"},
# `platform_id` IS set, from the exported slug.
"project_platforms": set(),
"family_idea_platforms": set(),
"family_ideas": set(),
"family_idea_references": set(),
"family_adoptions": {"id"},
# Reason 2: re-issued. The source id only builds the precedent map.
"family_decisions": {"id"},
}
@@ -829,6 +867,99 @@ def _lesson_no_rule_rows(rows) -> list[dict]:
]
def _platform_rows(rows) -> list[dict]:
"""The global platform catalog (v25). Carried WITHOUT ids, matched on slug
at restore — the canonical_systems reasoning."""
return [
{
"name": r.name, "slug": r.slug, "description": r.description,
"markers": list(r.markers or []), "order_index": r.order_index,
}
for r in rows
]
def _project_platform_rows(rows, platform_slugs: dict[int, str]) -> list[dict]:
"""A project's platforms, by SLUG. A `rejected` row travels too: it is
someone's "no", and without it the first refresh detects it straight back."""
return [
{
"project_id": r.project_id,
"platform_slug": platform_slugs.get(r.platform_id or 0),
"state": r.state,
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _family_idea_rows(rows) -> list[dict]:
"""A note's family state (v25). note_id and topic_id are SOURCE ids."""
return [
{
"note_id": r.note_id, "status": r.status,
"applies_when": r.applies_when, "canon_version": r.canon_version,
"topic_id": r.topic_id,
"created_at": r.created_at.isoformat() if r.created_at else None,
"updated_at": r.updated_at.isoformat() if r.updated_at else None,
}
for r in rows
]
def _family_idea_platform_rows(rows, platform_slugs: dict[int, str]) -> list[dict]:
return [
{
"note_id": r.note_id,
"platform_slug": platform_slugs.get(r.platform_id or 0),
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _family_idea_reference_rows(rows) -> list[dict]:
return [
{
"idea_id": r.idea_id, "snippet_id": r.snippet_id,
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _family_adoption_rows(rows) -> list[dict]:
"""The adoption ledger (v25). Project, idea and owed task are SOURCE ids."""
return [
{
"project_id": r.project_id, "idea_id": r.idea_id,
"status": r.status, "reason": r.reason,
"canon_version": r.canon_version,
"assessed_at": r.assessed_at.isoformat() if r.assessed_at else None,
"decided_via": r.decided_via, "owed_task_id": r.owed_task_id,
"created_at": r.created_at.isoformat() if r.created_at else None,
"updated_at": r.updated_at.isoformat() if r.updated_at else None,
}
for r in rows
]
def _family_decision_rows(rows) -> list[dict]:
"""The decision log (v25), oldest first. The source `id` travels because
`precedent_ids` point at it; the restore rebuilds that map as it goes."""
return [
{
"id": r.id, "idea_id": r.idea_id, "project_id": r.project_id,
"action": r.action, "reason": r.reason,
"before": r.before, "after": r.after, "evidence": r.evidence,
"precedent_ids": list(r.precedent_ids or []),
"decided_via": r.decided_via, "user_id": r.user_id,
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _rule_rows(rows) -> list[dict]:
return [
{
@@ -923,6 +1054,24 @@ async def export_full_backup() -> dict:
rulebooks = (await session.execute(select(Rulebook))).scalars().all()
topics = (await session.execute(select(RulebookTopic))).scalars().all()
rules = (await session.execute(select(Rule))).scalars().all()
platforms = (await session.execute(
select(Platform).where(Platform.deleted_at.is_(None))
.order_by(Platform.order_index)
)).scalars().all()
project_platforms = (await session.execute(select(ProjectPlatform))).scalars().all()
family_ideas = (await session.execute(select(FamilyIdea))).scalars().all()
family_idea_platforms = (await session.execute(
select(FamilyIdeaPlatform)
)).scalars().all()
family_idea_references = (await session.execute(
select(FamilyIdeaReference)
)).scalars().all()
family_adoptions = (await session.execute(select(FamilyAdoption))).scalars().all()
# Oldest first: a precedent is always an earlier decision, so a restore
# in this order has every precedent mapped before anything cites it.
family_decisions = (await session.execute(
select(FamilyDecision).order_by(FamilyDecision.id)
)).scalars().all()
return {
"version": BACKUP_VERSION,
@@ -970,6 +1119,28 @@ async def export_full_backup() -> dict:
"code_shapes": _code_shape_rows(code_shapes),
"code_shape_events": _code_shape_event_rows(code_shape_events),
"code_shape_uses": _code_shape_use_rows(code_shape_uses),
**_family_sections(
platforms, project_platforms, family_ideas, family_idea_platforms,
family_idea_references, family_adoptions, family_decisions,
),
}
def _family_sections(
platforms, project_platforms, ideas, idea_platforms, references,
adoptions, decisions,
) -> dict:
"""The v25 payload sections, shared by both export scopes so the two
cannot serialise family canon differently."""
slugs = {p.id: p.slug for p in platforms}
return {
"platforms": _platform_rows(platforms),
"project_platforms": _project_platform_rows(project_platforms, slugs),
"family_ideas": _family_idea_rows(ideas),
"family_idea_platforms": _family_idea_platform_rows(idea_platforms, slugs),
"family_idea_references": _family_idea_reference_rows(references),
"family_adoptions": _family_adoption_rows(adoptions),
"family_decisions": _family_decision_rows(decisions),
}
@@ -1145,6 +1316,47 @@ async def export_user_backup(user_id: int) -> dict:
lesson_no_rule = (await session.execute(
select(LessonNoRule).where(LessonNoRule.lesson_id.in_(note_ids))
)).scalars().all() if note_ids else []
# Family canon (v25). The catalog is global and taken whole, for the
# canonical_systems reason. Everything else is scoped so that BOTH
# ends of every row restore: an idea is this user's note, a ledger row
# needs this user's project AND idea, a reference needs both notes.
platforms = (await session.execute(
select(Platform).where(Platform.deleted_at.is_(None))
.order_by(Platform.order_index)
)).scalars().all()
project_platforms = (await session.execute(
select(ProjectPlatform).where(ProjectPlatform.project_id.in_(project_ids))
)).scalars().all() if project_ids else []
family_ideas = (await session.execute(
select(FamilyIdea).where(FamilyIdea.note_id.in_(note_ids))
)).scalars().all() if note_ids else []
idea_ids = [i.note_id for i in family_ideas]
family_idea_platforms = (await session.execute(
select(FamilyIdeaPlatform).where(FamilyIdeaPlatform.note_id.in_(idea_ids))
)).scalars().all() if idea_ids else []
family_idea_references = (await session.execute(
select(FamilyIdeaReference).where(
FamilyIdeaReference.idea_id.in_(idea_ids),
FamilyIdeaReference.snippet_id.in_(note_ids),
)
)).scalars().all() if idea_ids else []
family_adoptions = (await session.execute(
select(FamilyAdoption).where(
FamilyAdoption.idea_id.in_(idea_ids),
FamilyAdoption.project_id.in_(project_ids),
)
)).scalars().all() if (idea_ids and project_ids) else []
# A decision about the idea itself has no project; one about a ledger
# row needs that row's project in this export too.
family_decisions = (await session.execute(
select(FamilyDecision).where(
FamilyDecision.idea_id.in_(idea_ids),
or_(
FamilyDecision.project_id.is_(None),
FamilyDecision.project_id.in_(project_ids or [0]),
),
).order_by(FamilyDecision.id)
)).scalars().all() if idea_ids else []
return {
"version": BACKUP_VERSION,
@@ -1194,6 +1406,10 @@ async def export_user_backup(user_id: int) -> dict:
"code_shapes": _code_shape_rows(code_shapes),
"code_shape_events": _code_shape_event_rows(code_shape_events),
"code_shape_uses": _code_shape_use_rows(code_shape_uses),
**_family_sections(
platforms, project_platforms, family_ideas, family_idea_platforms,
family_idea_references, family_adoptions, family_decisions,
),
}
@@ -1240,6 +1456,7 @@ class _Maps:
__slots__ = (
"users", "projects", "milestones", "notes", "rulebooks", "topics",
"rules", "systems", "design_systems", "shapes", "canonical_by_slug",
"platform_by_slug", "decisions",
)
def __init__(self) -> None:
@@ -1254,6 +1471,8 @@ class _Maps:
self.design_systems: dict[int, int] = {}
self.shapes: dict[int, int] = {}
self.canonical_by_slug: dict[str, int] = {}
self.platform_by_slug: dict[str, int] = {}
self.decisions: dict[int, int] = {}
def _build_user(row: dict, maps: _Maps) -> User:
@@ -1592,6 +1811,127 @@ def _build_rule_moment_judgment(row: dict, maps: _Maps) -> RuleMomentJudgment |
)
def _build_platform(row: dict, maps: _Maps) -> Platform | None:
"""Matched on SLUG, like a canonical system: this install seeded the
standard platforms from its migrations, so the common case creates nothing
and only an entry added on the source instance is built."""
slug = row.get("slug") or ""
if not slug or slug in maps.platform_by_slug:
return None
return Platform(
name=row.get("name", ""),
slug=slug,
description=row.get("description"),
markers=list(row.get("markers") or []),
order_index=row.get("order_index", 0),
)
def _build_project_platform(row: dict, maps: _Maps) -> ProjectPlatform | None:
"""Skipped unless both the project and the platform resolve."""
project = maps.projects.get(row.get("project_id", 0))
platform = maps.platform_by_slug.get(row.get("platform_slug") or "")
if project is None or platform is None:
return None
return ProjectPlatform(
project_id=project,
platform_id=platform,
state=row.get("state") or "declared",
created_at=_dt(row.get("created_at")),
)
def _build_family_idea(row: dict, maps: _Maps) -> FamilyIdea | None:
"""Skipped when its note did not restore — the state is that note's. The
topic DEGRADES to None: a standard whose rule topic did not come across is
still a standard, only without its binding half."""
note = maps.notes.get(row.get("note_id", 0))
if note is None:
return None
return FamilyIdea(
note_id=note,
status=row.get("status") or "candidate",
applies_when=row.get("applies_when"),
canon_version=row.get("canon_version") or 1,
topic_id=maps.topics.get(row.get("topic_id") or 0),
created_at=_dt(row.get("created_at")),
updated_at=_dt(row.get("updated_at")),
)
def _build_family_idea_platform(row: dict, maps: _Maps) -> FamilyIdeaPlatform | None:
note = maps.notes.get(row.get("note_id", 0))
platform = maps.platform_by_slug.get(row.get("platform_slug") or "")
if note is None or platform is None:
return None
return FamilyIdeaPlatform(
note_id=note, platform_id=platform, created_at=_dt(row.get("created_at")),
)
def _build_family_idea_reference(row: dict, maps: _Maps) -> FamilyIdeaReference | None:
idea = maps.notes.get(row.get("idea_id", 0))
snippet = maps.notes.get(row.get("snippet_id", 0))
if idea is None or snippet is None:
return None
return FamilyIdeaReference(
idea_id=idea, snippet_id=snippet, created_at=_dt(row.get("created_at")),
)
def _build_family_adoption(row: dict, maps: _Maps) -> FamilyAdoption | None:
"""Both the project and the idea must map — the row IS that pair. The owed
task degrades to None: the answer still stands without its task."""
project = maps.projects.get(row.get("project_id", 0))
idea = maps.notes.get(row.get("idea_id", 0))
if project is None or idea is None:
return None
return FamilyAdoption(
project_id=project,
idea_id=idea,
status=row.get("status") or "unassessed",
reason=row.get("reason"),
canon_version=row.get("canon_version"),
assessed_at=_dt_or_none(row.get("assessed_at")),
decided_via=row.get("decided_via"),
owed_task_id=maps.notes.get(row.get("owed_task_id") or 0),
created_at=_dt(row.get("created_at")),
updated_at=_dt(row.get("updated_at")),
)
def _build_family_decision(row: dict, maps: _Maps) -> FamilyDecision | None:
"""Skipped when its idea did not restore, or when it is about a project
that did not — an assessment without its project says nothing. The acting
user degrades to None. Precedents are remapped through the decision map
and a precedent that did not restore is dropped from the list rather than
left pointing at whatever took its number."""
idea = maps.notes.get(row.get("idea_id", 0))
if idea is None:
return None
project = None
if row.get("project_id") is not None:
project = maps.projects.get(row["project_id"])
if project is None:
return None
return FamilyDecision(
idea_id=idea,
project_id=project,
action=row.get("action") or "assess",
reason=row.get("reason") or "",
before=row.get("before"),
after=row.get("after"),
evidence=row.get("evidence"),
precedent_ids=[
maps.decisions[p] for p in (row.get("precedent_ids") or [])
if p in maps.decisions
],
decided_via=row.get("decided_via") or "agent",
user_id=maps.users.get(row.get("user_id") or 0),
created_at=_dt(row.get("created_at")),
)
def _build_rule_version(row: dict, maps: _Maps) -> RuleVersion | None:
rid = maps.rules.get(row.get("rule_id", 0))
if rid is None:
@@ -2010,6 +2350,9 @@ async def _restore_v2(data: dict) -> dict:
"rule_relations": 0, "rule_versions": 0,
"retrieval_tuning_events": 0, "lesson_rule_links": 0,
"lesson_no_rule": 0, "moment_mappings": 0,
"platforms": 0, "project_platforms": 0, "family_ideas": 0,
"family_idea_platforms": 0, "family_idea_references": 0,
"family_adoptions": 0, "family_decisions": 0,
}
async with async_session() as session:
@@ -2250,6 +2593,66 @@ async def _restore_v2(data: dict) -> dict:
session.add(answer)
stats["lesson_no_rule"] += 1
# Family canon (v25). Here because it needs projects, notes and topics
# all mapped. The platform catalog first, matched on slug and seeded
# from what this install already has — the 14c shape.
existing_platforms = (await session.execute(
select(Platform).where(Platform.deleted_at.is_(None))
)).scalars().all()
for platform in existing_platforms:
maps.platform_by_slug[platform.slug] = platform.id
for p_data in data.get("platforms", []):
platform = _build_platform(p_data, maps)
if platform is None:
continue
session.add(platform)
await session.flush()
maps.platform_by_slug[platform.slug] = platform.id
stats["platforms"] += 1
for pp in data.get("project_platforms", []):
membership = _build_project_platform(pp, maps)
if membership is None:
continue
session.add(membership)
stats["project_platforms"] += 1
for fi in data.get("family_ideas", []):
idea = _build_family_idea(fi, maps)
if idea is None:
continue
session.add(idea)
stats["family_ideas"] += 1
# The ideas must exist before anything foreign-keys them.
await session.flush()
for fp in data.get("family_idea_platforms", []):
scope = _build_family_idea_platform(fp, maps)
if scope is None:
continue
session.add(scope)
stats["family_idea_platforms"] += 1
for fr in data.get("family_idea_references", []):
ref = _build_family_idea_reference(fr, maps)
if ref is None:
continue
session.add(ref)
stats["family_idea_references"] += 1
for fa in data.get("family_adoptions", []):
adoption = _build_family_adoption(fa, maps)
if adoption is None:
continue
session.add(adoption)
stats["family_adoptions"] += 1
# Oldest first, flushed one at a time: each decision's new id goes into
# the map before a later decision can name it as a precedent.
for fd in sorted(data.get("family_decisions", []), key=lambda r: r.get("id") or 0):
decision = _build_family_decision(fd, maps)
if decision is None:
continue
session.add(decision)
await session.flush()
if fd.get("id"):
maps.decisions[int(fd["id"])] = decision.id
stats["family_decisions"] += 1
# A rule's edit history (milestone 323). Must come after the rules
# themselves — the rule map is only populated above — and both ids are
# ids in the SOURCE database, which is #3182's arose_from_id trap.