feat(family): platforms, family ideas, the adoption ledger and its decision log (milestone 463 step 1, #4987)
CI & Build / TypeScript typecheck (push) Successful in 1m11s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 25s
CI & Build / integration (push) Successful in 1m50s
CI & Build / Python tests (push) Successful in 2m41s
CI & Build / Build & push image (push) Successful in 1m8s

When one project solves something every project on the same platform will
meet, that solution becomes family canon and every other project on the
platform answers it. This is the storage for that.

- platforms: a global catalog in the canonical_systems shape, seeded with
  generic technology names and the file markers step 2's detection reads.
- project_platforms: declared / detected / rejected. A rejected row is kept
  so detection cannot re-add what a person said no to.
- family_ideas: a note's family state. No new record type; any note, snippet
  or lesson becomes an idea. A canon idea must state when it applies (CHECK).
- family_idea_platforms: the only scope source. A linked rule topic takes its
  scope from the idea, so the two cannot disagree.
- family_idea_references: reference implementations, explicit not inferred.
- family_adoptions: one answer per (project, idea). Variant and exempt require
  a reason (CHECK). Recheck is derived from the two canon versions, never
  stored.
- family_decisions: the append-only log, with a required reason and the
  earlier decisions each one followed. The agent decides with no approval
  step, so precedent is what keeps its calls consistent.

Backup v25 carries all seven: platforms by slug, precedent ids remapped
through the decision map. Both column guards cover the new tables, and a
real-Postgres test exercises the CHECKs and the restore remaps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-06 09:33:27 -04:00
co-authored by Claude Opus 5.5
parent ccbccb025c
commit e68ccc5884
7 changed files with 1474 additions and 4 deletions
+288
View File
@@ -0,0 +1,288 @@
"""family canon — platforms, family ideas, the adoption ledger and its decision
log (milestone 463 step 1)
Revision ID: 0120
Revises: 0119
Create Date: 2026-10-06
When one project solves something every project on the same platform will
meet, that solution becomes FAMILY CANON, and every other project on the
platform answers it: adopted, a variant (with a reason), exempt (with a
reason), or owed. These tables hold that.
- `platforms` — a GLOBAL catalog, the canonical_systems shape: no owner, slug
is the match key. Seeded with generic technology names only. Nothing here
names an app, a repo or a house convention: the catalog ships to every
install (rule 115).
- `project_platforms` — which platforms a project is (declared / detected /
rejected). Membership is what makes "is this in family?" a lookup.
- `family_ideas` — a NOTE's family state. No new record type: any notes row
(a note, a snippet, a lesson) becomes an idea by gaining one of these.
- `family_idea_platforms` — the platforms an idea is for; the only scope
source. A linked rule topic takes its scope from the idea.
- `family_idea_references` — reference implementations (snippets).
- `family_adoptions` — one row per (project, idea): the project's answer.
- `family_decisions` — append-only log of every promotion and ledger change.
No backfill. No project has declared a platform and no note is an idea yet;
inventing either would assert a judgment nobody made. The seed rows are
written here verbatim rather than imported, so the migration keeps running
unchanged after any service-side list moves on.
"""
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
from alembic import op
revision = "0120"
down_revision = "0119"
branch_labels = None
depends_on = None
# One place per whitelist, so each CHECK and the model's tuple cannot drift
# (rule 36: a new value later means DROP + ADD CONSTRAINT in one migration).
_MEMBERSHIP = ("declared", "detected", "rejected")
_IDEA_STATUSES = ("candidate", "canon", "retired")
_ADOPTION_STATUSES = ("unassessed", "adopted", "variant", "exempt", "owed")
_REASONED = ("variant", "exempt")
_ACTIONS = ("propose", "promote", "revise", "retire", "assess", "undo")
_DECIDERS = ("agent", "operator", "system")
def _in(column: str, values: tuple[str, ...]) -> str:
return f"{column} IN (" + ", ".join(f"'{v}'" for v in values) + ")"
# (name, slug, description, markers). A marker with no slash matches a file's
# basename anywhere in a repo; one with a slash matches the repo-relative
# path. Empty markers = declare-only, for a platform that leaves no reliable
# file behind. Generic on purpose — an install adds its own.
_SEED = (
("Android app", "android-app",
"A native Android client: an APK or app bundle installed on phones and tablets.",
["AndroidManifest.xml"]),
("iOS app", "ios-app",
"A native iOS or iPadOS client built with Xcode.",
["project.pbxproj"]),
("Web frontend", "web-frontend",
"A browser client built with a frontend toolchain: single-page apps and server-rendered frontends.",
["vite.config.*", "svelte.config.*", "vue.config.*", "next.config.*",
"nuxt.config.*", "angular.json"]),
("Browser extension", "browser-extension",
"An extension installed into a web browser and distributed through its add-on store or by file.",
[]),
("Desktop app", "desktop-app",
"A packaged desktop application for Windows, macOS or Linux.",
["tauri.conf.json", "electron-builder.*"]),
("Container image", "container-image",
"Ships as an OCI/Docker image that a host pulls and runs.",
["Dockerfile", "Containerfile", "*.Dockerfile"]),
("Go", "go",
"Written in Go.",
["go.mod"]),
("Python", "python",
"Written in Python.",
["pyproject.toml", "setup.py", "requirements.txt"]),
("Rust", "rust",
"Written in Rust.",
["Cargo.toml"]),
("PostgreSQL", "postgresql",
"Keeps its data in PostgreSQL.",
[]),
("GitHub Actions", "github-actions",
"Verified and released by GitHub Actions workflows.",
[".github/workflows/*"]),
("Gitea / Forgejo Actions", "gitea-actions",
"Verified and released by Gitea or Forgejo Actions workflows.",
[".gitea/workflows/*", ".forgejo/workflows/*"]),
)
def upgrade() -> None:
platforms = op.create_table(
"platforms",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("name", sa.Text(), nullable=False),
sa.Column("slug", sa.Text(), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("markers", postgresql.JSONB(), nullable=False,
server_default=sa.text("'[]'::jsonb")),
sa.Column("order_index", sa.Integer(), nullable=False, server_default="0"),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
sa.Column("deleted_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("deleted_batch_id", sa.Text(), nullable=True),
)
# Unique among LIVE rows only (the canonical_systems convention).
op.create_index(
"uq_platforms_slug", "platforms", ["slug"],
unique=True, postgresql_where=sa.text("deleted_at IS NULL"),
)
op.bulk_insert(
platforms,
[
{"name": name, "slug": slug, "description": description,
"markers": markers, "order_index": index}
for index, (name, slug, description, markers) in enumerate(_SEED)
],
)
op.create_table(
"project_platforms",
sa.Column("project_id", sa.Integer(),
sa.ForeignKey("projects.id", ondelete="CASCADE"), primary_key=True),
sa.Column("platform_id", sa.Integer(),
sa.ForeignKey("platforms.id", ondelete="CASCADE"), primary_key=True),
sa.Column("state", sa.Text(), nullable=False, server_default="declared"),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
)
op.create_check_constraint(
"ck_project_platforms_state", "project_platforms", _in("state", _MEMBERSHIP),
)
op.create_index("ix_project_platforms_platform_id", "project_platforms", ["platform_id"])
op.create_table(
"family_ideas",
sa.Column("note_id", sa.Integer(),
sa.ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True),
sa.Column("status", sa.Text(), nullable=False, server_default="candidate"),
sa.Column("applies_when", sa.Text(), nullable=True),
sa.Column("canon_version", sa.Integer(), nullable=False, server_default="1"),
# SET NULL: deleting a topic must not delete the standard it served.
sa.Column("topic_id", sa.BigInteger(),
sa.ForeignKey("rulebook_topics.id", ondelete="SET NULL"), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
)
op.create_check_constraint(
"ck_family_ideas_status", "family_ideas", _in("status", _IDEA_STATUSES),
)
# The first promotion criterion, held by the schema: an idea is not canon
# until it says, in platform terms, when it applies.
op.create_check_constraint(
"ck_family_ideas_canon_applies", "family_ideas",
"status <> 'canon' OR length(btrim(coalesce(applies_when, ''))) > 0",
)
op.create_check_constraint(
"ck_family_ideas_canon_version", "family_ideas", "canon_version >= 1",
)
op.create_index(
"uq_family_ideas_topic", "family_ideas", ["topic_id"],
unique=True, postgresql_where=sa.text("topic_id IS NOT NULL"),
)
op.create_index("ix_family_ideas_status", "family_ideas", ["status"])
op.create_table(
"family_idea_platforms",
sa.Column("note_id", sa.Integer(),
sa.ForeignKey("family_ideas.note_id", ondelete="CASCADE"), primary_key=True),
sa.Column("platform_id", sa.Integer(),
sa.ForeignKey("platforms.id", ondelete="CASCADE"), primary_key=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
)
op.create_index("ix_family_idea_platforms_platform_id", "family_idea_platforms", ["platform_id"])
op.create_table(
"family_idea_references",
sa.Column("idea_id", sa.Integer(),
sa.ForeignKey("family_ideas.note_id", ondelete="CASCADE"), primary_key=True),
sa.Column("snippet_id", sa.Integer(),
sa.ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
)
op.create_index("ix_family_idea_references_snippet_id", "family_idea_references", ["snippet_id"])
op.create_table(
"family_adoptions",
sa.Column("id", sa.BigInteger(), primary_key=True),
sa.Column("project_id", sa.Integer(),
sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=False),
sa.Column("idea_id", sa.Integer(),
sa.ForeignKey("family_ideas.note_id", ondelete="CASCADE"), nullable=False),
sa.Column("status", sa.Text(), nullable=False, server_default="unassessed"),
sa.Column("reason", sa.Text(), nullable=True),
sa.Column("canon_version", sa.Integer(), nullable=True),
sa.Column("assessed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("decided_via", sa.Text(), nullable=True),
sa.Column("owed_task_id", sa.Integer(),
sa.ForeignKey("notes.id", ondelete="SET NULL"), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
)
op.create_check_constraint(
"ck_family_adoptions_status", "family_adoptions", _in("status", _ADOPTION_STATUSES),
)
# A departure is only a record if it says why.
op.create_check_constraint(
"ck_family_adoptions_reason", "family_adoptions",
"NOT (" + _in("status", _REASONED) + ") "
"OR length(btrim(coalesce(reason, ''))) > 0",
)
op.create_check_constraint(
"ck_family_adoptions_decided_via", "family_adoptions",
"decided_via IS NULL OR " + _in("decided_via", _DECIDERS),
)
op.create_index(
"uq_family_adoptions_pair", "family_adoptions", ["project_id", "idea_id"], unique=True,
)
op.create_index("ix_family_adoptions_project_id", "family_adoptions", ["project_id"])
op.create_index("ix_family_adoptions_idea_id", "family_adoptions", ["idea_id"])
op.create_table(
"family_decisions",
sa.Column("id", sa.BigInteger(), primary_key=True),
sa.Column("idea_id", sa.Integer(),
sa.ForeignKey("family_ideas.note_id", ondelete="CASCADE"), nullable=False),
sa.Column("project_id", sa.Integer(),
sa.ForeignKey("projects.id", ondelete="CASCADE"), nullable=True),
sa.Column("action", sa.Text(), nullable=False),
sa.Column("reason", sa.Text(), nullable=False),
sa.Column("before", postgresql.JSONB(), nullable=True),
sa.Column("after", postgresql.JSONB(), nullable=True),
sa.Column("evidence", postgresql.JSONB(), nullable=True),
sa.Column("precedent_ids", postgresql.JSONB(), nullable=False,
server_default=sa.text("'[]'::jsonb")),
sa.Column("decided_via", sa.Text(), nullable=False, server_default="agent"),
sa.Column("user_id", sa.Integer(),
sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
)
op.create_check_constraint(
"ck_family_decisions_action", "family_decisions", _in("action", _ACTIONS),
)
op.create_check_constraint(
"ck_family_decisions_decided_via", "family_decisions", _in("decided_via", _DECIDERS),
)
# The reason IS the record — the agent decides with no approval step, so
# a decision that cannot say why is not one anybody can review or follow.
op.create_check_constraint(
"ck_family_decisions_reason", "family_decisions", "length(btrim(reason)) > 0",
)
op.create_index("ix_family_decisions_idea_id", "family_decisions", ["idea_id"])
op.create_index("ix_family_decisions_project_id", "family_decisions", ["project_id"])
def downgrade() -> None:
op.drop_index("ix_family_decisions_project_id", table_name="family_decisions")
op.drop_index("ix_family_decisions_idea_id", table_name="family_decisions")
op.drop_table("family_decisions")
op.drop_index("ix_family_adoptions_idea_id", table_name="family_adoptions")
op.drop_index("ix_family_adoptions_project_id", table_name="family_adoptions")
op.drop_index("uq_family_adoptions_pair", table_name="family_adoptions")
op.drop_table("family_adoptions")
op.drop_index("ix_family_idea_references_snippet_id", table_name="family_idea_references")
op.drop_table("family_idea_references")
op.drop_index("ix_family_idea_platforms_platform_id", table_name="family_idea_platforms")
op.drop_table("family_idea_platforms")
op.drop_index("ix_family_ideas_status", table_name="family_ideas")
op.drop_index("uq_family_ideas_topic", table_name="family_ideas")
op.drop_table("family_ideas")
op.drop_index("ix_project_platforms_platform_id", table_name="project_platforms")
op.drop_table("project_platforms")
op.drop_index("uq_platforms_slug", table_name="platforms")
op.drop_table("platforms")
+6
View File
@@ -89,3 +89,9 @@ from scribe.models.forge_connection import ForgeConnection # noqa: E402, F401
from scribe.models.code_shape import CodeShape, CodeShapeConsumer, CodeShapeEvent, CodeShapeUse # noqa: E402, F401
from scribe.models.system import System, RecordSystem # noqa: E402, F401
from scribe.models.design_system import DesignSystem, DesignToken # noqa: E402, F401
# After notes, projects and rulebook topics: family canon foreign-keys all
# three (milestone 463).
from scribe.models.family import ( # noqa: E402, F401
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform,
FamilyIdeaReference, Platform, ProjectPlatform,
)
+357
View File
@@ -0,0 +1,357 @@
"""Family canon (milestone 463): ideas that every project on a platform shares,
and each project's recorded answer to them.
A FAMILY IDEA is not a new kind of record. It is an existing `notes` row — a
note, a snippet, a lesson — that has been given a platform scope. The note
carries the idea (when it applies, the traps, the checklist, the incidents
behind them); an optional rulebook topic carries the norms that bind; snippets
are the reference implementations, one per language. A new record type would
be one more thing reached for interchangeably with rules, processes, snippets
and design systems, and nothing here needs one.
What is shared is the IDEA, not the code. Each project implements it in its own
language; identical code is a by-product and never the test.
The tables, and the one job each does:
- `platforms` — the global catalog of what a project can be built on or ship
as. Same shape and reasoning as `canonical_systems`: no owner, so the same
word means the same thing in every project on the install.
- `project_platforms` — which platforms a project is. Membership is how
"is this in family?" stops being a judgment made per idea and becomes a
lookup.
- `family_ideas` — a note's family state: candidate, canon or retired, its
applicability test, its canon version, its linked rule topic.
- `family_idea_platforms` — the platforms an idea is for. The ONLY scope
source: a linked topic takes its scope from here rather than carrying its
own, so the two can never disagree.
- `family_idea_references` — the reference implementations.
- `family_adoptions` — one row per (project, idea): the project's answer.
- `family_decisions` — the append-only log of every promotion and every
ledger change, with its reason and the earlier decisions it followed. The
agent makes these calls with no approval step, so the log is how they stay
consistent (precedent) and how a person reviews or undoes one.
"""
from datetime import datetime
from sqlalchemy import (
BigInteger, DateTime, ForeignKey, Index, Integer, Text, text,
)
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from scribe.models import Base
from scribe.models.base import CreatedAtMixin, SoftDeleteMixin, TimestampMixin, iso
# Each tuple is the whitelist its CHECK constraint enforces (migration 0120).
# Rule 36: a new value means DROP + ADD CONSTRAINT in the same migration.
MEMBERSHIP_STATES = ("declared", "detected", "rejected")
IDEA_STATUSES = ("candidate", "canon", "retired")
ADOPTION_STATUSES = ("unassessed", "adopted", "variant", "exempt", "owed")
# The two adoption answers that are a departure, and so must say why.
REASONED_STATUSES = ("variant", "exempt")
DECISION_ACTIONS = ("propose", "promote", "revise", "retire", "assess", "undo")
DECIDERS = ("agent", "operator", "system")
class Platform(Base, TimestampMixin, SoftDeleteMixin):
"""What a project is built on or ships as — a runtime, a delivery channel
or a toolchain whose own behaviour causes the problems a family idea
answers.
FLAT, deliberately. A project declares several, and an idea is for
several, so "Android app and container image" needs no hierarchy to
express. Applicability narrower than a platform (an idea that only matters
to a sideloaded APK, not a store-distributed one) belongs in the idea's
`applies_when`, where a project it does not fit answers `exempt` with the
reason. Splitting platforms to carry that would grow the catalog every
time an idea got more specific.
GLOBAL, like `canonical_systems`: no owner, so a shared project inherits
the vocabulary rather than re-earning it. `slug` is the match key and the
form a backup carries, since ids are per-install.
`markers` are glob patterns over a bound repo's paths that suggest a
project is this platform (step 2's detection). A pattern without a slash
matches a file's basename anywhere in the tree; one with a slash matches
the repo-relative path. Empty means declare-only: some platforms leave no
reliable file behind.
"""
__tablename__ = "platforms"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
name: Mapped[str] = mapped_column(Text, nullable=False)
slug: Mapped[str] = mapped_column(Text, nullable=False)
description: Mapped[str | None] = mapped_column(Text, nullable=True)
markers: Mapped[list] = mapped_column(
JSONB, default=list, server_default=text("'[]'::jsonb"),
)
order_index: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
__table_args__ = (
# Unique among LIVE rows, so a retired entry doesn't block recreating
# the same platform (the canonical_systems convention).
Index(
"uq_platforms_slug", "slug",
unique=True, postgresql_where=text("deleted_at IS NULL"),
),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"name": self.name,
"slug": self.slug,
"description": self.description,
"markers": list(self.markers or []),
"order_index": self.order_index,
"created_at": iso(self.created_at),
"updated_at": iso(self.updated_at),
}
class ProjectPlatform(Base, CreatedAtMixin):
"""A project's answer to "are you this platform?".
Three states, because detection is automatic and must not be able to
override a person:
- ``declared`` — someone said so (at inception or in settings).
- ``detected`` — a marker in a bound repo said so.
- ``rejected`` — someone said NO. Kept as a row rather than deleted, so the
next refresh does not detect it straight back. Not membership.
Only `declared` and `detected` make a project a member of the platform's
family.
"""
__tablename__ = "project_platforms"
project_id: Mapped[int] = mapped_column(
Integer, ForeignKey("projects.id", ondelete="CASCADE"), primary_key=True,
)
platform_id: Mapped[int] = mapped_column(
Integer, ForeignKey("platforms.id", ondelete="CASCADE"), primary_key=True,
index=True,
)
# CHECK ck_project_platforms_state (migration 0120).
state: Mapped[str] = mapped_column(Text, default="declared", server_default="declared")
def to_dict(self) -> dict:
return {
"project_id": self.project_id,
"platform_id": self.platform_id,
"state": self.state,
"created_at": iso(self.created_at),
}
class FamilyIdea(Base, TimestampMixin):
"""A note's family state. Its presence is what makes the note a family
idea; the note itself is unchanged.
- ``candidate`` — recorded, not yet proven or not yet stated in platform
terms. Reaches nobody's ledger.
- ``canon`` — promoted. Every project on its platforms owes it an answer.
- ``retired`` — demoted. Kept, with its ledger, so the history reads.
`applies_when` is the applicability test every assessment starts from —
stated in platform terms, which is the first promotion criterion. A CHECK
refuses canon without one, so that criterion is the schema's to enforce
and not a sentence a session might skip.
`canon_version` moves whenever the idea's substance changes. An adoption
assessed against an older version needs rechecking — which is DERIVED by
comparing the two numbers, never stored as a flag that could go stale.
`topic_id` is the rule topic holding the idea's binding norms, if it has
any (the note↔topic link #3236 asked for). The topic takes its scope from
this idea; it never carries platforms of its own.
"""
__tablename__ = "family_ideas"
note_id: Mapped[int] = mapped_column(
Integer, ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True,
)
# CHECK ck_family_ideas_status (migration 0120).
status: Mapped[str] = mapped_column(Text, default="candidate", server_default="candidate")
# CHECK ck_family_ideas_canon_applies (migration 0120).
applies_when: Mapped[str | None] = mapped_column(Text, nullable=True)
canon_version: Mapped[int] = mapped_column(Integer, default=1, server_default="1")
topic_id: Mapped[int | None] = mapped_column(
BigInteger, ForeignKey("rulebook_topics.id", ondelete="SET NULL"), nullable=True,
)
__table_args__ = (
# One idea per topic: a topic's norms belong to one standard.
Index(
"uq_family_ideas_topic", "topic_id",
unique=True, postgresql_where=text("topic_id IS NOT NULL"),
),
Index("ix_family_ideas_status", "status"),
)
def to_dict(self) -> dict:
return {
"note_id": self.note_id,
"status": self.status,
"applies_when": self.applies_when or "",
"canon_version": self.canon_version,
"topic_id": self.topic_id,
"created_at": iso(self.created_at),
"updated_at": iso(self.updated_at),
}
class FamilyIdeaPlatform(Base, CreatedAtMixin):
"""A platform an idea is for. A join table with a model so the backup's
column guard can see it, and so the reverse lookup — every idea for a
platform — has an index."""
__tablename__ = "family_idea_platforms"
note_id: Mapped[int] = mapped_column(
Integer, ForeignKey("family_ideas.note_id", ondelete="CASCADE"), primary_key=True,
)
platform_id: Mapped[int] = mapped_column(
Integer, ForeignKey("platforms.id", ondelete="CASCADE"), primary_key=True,
index=True,
)
class FamilyIdeaReference(Base, CreatedAtMixin):
"""A reference implementation of an idea: a snippet a project can start
from. Explicit rather than inferred, because an owed task names "the
reference for your language", and a guess there sends a project to copy
the wrong thing. The snippet's language is read from the snippet."""
__tablename__ = "family_idea_references"
idea_id: Mapped[int] = mapped_column(
Integer, ForeignKey("family_ideas.note_id", ondelete="CASCADE"), primary_key=True,
)
snippet_id: Mapped[int] = mapped_column(
Integer, ForeignKey("notes.id", ondelete="CASCADE"), primary_key=True,
index=True,
)
class FamilyAdoption(Base, TimestampMixin):
"""One project's answer to one family idea — the adoption ledger.
- ``unassessed`` — the idea reached the project and nobody has judged it.
- ``adopted`` — the project does it.
- ``variant`` — the project departs, for a reason that names a fact about
itself the canon did not account for. A preference is not a reason.
- ``exempt`` — the idea's `applies_when` is false for this project.
- ``owed`` — it applies, there is no reason to depart, and it is not done
yet. `owed_task_id` is the task filed in THIS project to do it.
`reason` is required for variant and exempt (CHECK), because those two are
the departures and the reason is the whole record. `canon_version` is the
idea version this answer was given against; NULL while unassessed.
The row is the CURRENT answer. How it got there — the reasons at each step
and the precedents each followed — is in `family_decisions`.
"""
__tablename__ = "family_adoptions"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
project_id: Mapped[int] = mapped_column(
Integer, ForeignKey("projects.id", ondelete="CASCADE"), index=True,
)
idea_id: Mapped[int] = mapped_column(
Integer, ForeignKey("family_ideas.note_id", ondelete="CASCADE"), index=True,
)
# CHECK ck_family_adoptions_status and ck_family_adoptions_reason (0120).
status: Mapped[str] = mapped_column(Text, default="unassessed", server_default="unassessed")
reason: Mapped[str | None] = mapped_column(Text, nullable=True)
canon_version: Mapped[int | None] = mapped_column(Integer, nullable=True)
assessed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# CHECK ck_family_adoptions_decided_via (0120). NULL while unassessed.
decided_via: Mapped[str | None] = mapped_column(Text, nullable=True)
owed_task_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("notes.id", ondelete="SET NULL"), nullable=True,
)
__table_args__ = (
Index("uq_family_adoptions_pair", "project_id", "idea_id", unique=True),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"project_id": self.project_id,
"idea_id": self.idea_id,
"status": self.status,
"reason": self.reason or "",
"canon_version": self.canon_version,
"assessed_at": iso(self.assessed_at),
"decided_via": self.decided_via,
"owed_task_id": self.owed_task_id,
"created_at": iso(self.created_at),
"updated_at": iso(self.updated_at),
}
class FamilyDecision(Base, CreatedAtMixin):
"""One decision about family canon — append-only.
`project_id` is NULL for a decision about the idea itself (propose,
promote, revise, retire) and set for a decision about one project's
answer (assess). `undo` reverses an earlier decision and names it in
`evidence`.
`before` and `after` are STATE snapshots — status, version, reason — and
deliberately hold no foreign ids: an id inside JSON cannot be remapped by a
restore, and would come back pointing at whatever took that number (the
#3182 trap). The ids this row needs are columns.
`precedent_ids` are the earlier decisions this one followed. They are what
keeps a call consistent with the last similar one when no person approves
either, so they are a list of ids into this same table, remapped at
restore through the decision map.
"""
__tablename__ = "family_decisions"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
idea_id: Mapped[int] = mapped_column(
Integer, ForeignKey("family_ideas.note_id", ondelete="CASCADE"), index=True,
)
project_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("projects.id", ondelete="CASCADE"), nullable=True, index=True,
)
# CHECK ck_family_decisions_action (0120).
action: Mapped[str] = mapped_column(Text)
reason: Mapped[str] = mapped_column(Text)
before: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
after: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
evidence: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
precedent_ids: Mapped[list] = mapped_column(
JSONB, default=list, server_default=text("'[]'::jsonb"),
)
# CHECK ck_family_decisions_decided_via (0120).
decided_via: Mapped[str] = mapped_column(Text, default="agent", server_default="agent")
# Who was acting — the session's user. The decider KIND is decided_via.
user_id: Mapped[int | None] = mapped_column(
Integer, ForeignKey("users.id", ondelete="SET NULL"), nullable=True,
)
def to_dict(self) -> dict:
return {
"id": self.id,
"idea_id": self.idea_id,
"project_id": self.project_id,
"action": self.action,
"reason": self.reason,
"before": self.before,
"after": self.after,
"evidence": self.evidence or {},
"precedent_ids": list(self.precedent_ids or []),
"decided_via": self.decided_via,
"user_id": self.user_id,
"created_at": iso(self.created_at),
}
+404 -1
View File
@@ -17,6 +17,10 @@ from scribe.models.system_usage import SystemUsageEvent
from scribe.models.moment_mapping import MomentMapping
from scribe.models.retrieval_tuning import RetrievalTuningEvent
from scribe.models.canonical_system import CanonicalSystem
from scribe.models.family import (
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform,
FamilyIdeaReference, Platform, ProjectPlatform,
)
from scribe.models.rulebook import (
RuleRelation, rule_moments as rule_moments_t, rule_systems as rule_systems_t,
)
@@ -112,8 +116,14 @@ logger = logging.getLogger(__name__)
# v24 (2026-10) added rule_moment_judgments.misfire (milestone 458 step 7b):
# the reports that a mounted rule arrived where it did not apply, and the
# operator's "keep it" that stops them being proposed as an unmount again.
# v25 (2026-10) added family canon (milestone 463): the platform catalog, each
# project's platforms, family ideas with their platforms and reference
# implementations, the adoption ledger and the decision log. The ledger is
# every project's recorded answer to a shared idea, and the log is the
# precedent each later answer follows. Lose either and every assessment is
# owed again, made fresh, with nothing to keep it consistent with the last.
# Bump when the serialized schema changes.
BACKUP_VERSION = 24
BACKUP_VERSION = 25
# Every table this backup carries, by its REAL name. Paired with _NOT_INCLUDED
# below, these two lists must together account for the entire schema — which is
@@ -164,6 +174,12 @@ _BACKED_UP = [
"rule_moments",
# v23 (2026-10): proposals and judgments about those mounts (step 7).
"rule_moment_judgments",
# v25 (2026-10): family canon (milestone 463). `platforms` is global like
# canonical_systems and rides every export for the same reason: the rows
# below name platforms by slug, and a partial catalog restores partial
# memberships.
"platforms", "project_platforms", "family_ideas", "family_idea_platforms",
"family_idea_references", "family_adoptions", "family_decisions",
]
# Tables intentionally NOT in the backup, surfaced in the payload so the gap is
@@ -298,6 +314,19 @@ _COLUMN_EXCLUSIONS: dict[str, set[str]] = {
},
"code_shape_events": set(),
"code_shape_uses": set(),
# Matched on SLUG at restore, like canonical_systems: a target install
# already seeded the standard platforms from its migrations.
"platforms": {"deleted_at", "deleted_batch_id", "id", "created_at", "updated_at"},
# The platform travels as `platform_slug` — ids are per-install.
"project_platforms": {"platform_id"},
"family_idea_platforms": {"platform_id"},
# Every column travels: the note and topic ids are SOURCE ids, remapped.
"family_ideas": set(),
"family_idea_references": set(),
"family_adoptions": {"id"},
# The id travels, unlike the other surrogate keys: precedent_ids point at
# it, so the restore needs the source id to build the decision map.
"family_decisions": set(),
}
@@ -381,6 +410,15 @@ _IMPORT_COLUMN_EXCLUSIONS: dict[str, set[str]] = {
},
"code_shape_events": {"id"},
"code_shape_uses": {"id"},
"platforms": {"id", "deleted_at", "deleted_batch_id", "created_at", "updated_at"},
# `platform_id` IS set, from the exported slug.
"project_platforms": set(),
"family_idea_platforms": set(),
"family_ideas": set(),
"family_idea_references": set(),
"family_adoptions": {"id"},
# Reason 2: re-issued. The source id only builds the precedent map.
"family_decisions": {"id"},
}
@@ -829,6 +867,99 @@ def _lesson_no_rule_rows(rows) -> list[dict]:
]
def _platform_rows(rows) -> list[dict]:
"""The global platform catalog (v25). Carried WITHOUT ids, matched on slug
at restore — the canonical_systems reasoning."""
return [
{
"name": r.name, "slug": r.slug, "description": r.description,
"markers": list(r.markers or []), "order_index": r.order_index,
}
for r in rows
]
def _project_platform_rows(rows, platform_slugs: dict[int, str]) -> list[dict]:
"""A project's platforms, by SLUG. A `rejected` row travels too: it is
someone's "no", and without it the first refresh detects it straight back."""
return [
{
"project_id": r.project_id,
"platform_slug": platform_slugs.get(r.platform_id or 0),
"state": r.state,
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _family_idea_rows(rows) -> list[dict]:
"""A note's family state (v25). note_id and topic_id are SOURCE ids."""
return [
{
"note_id": r.note_id, "status": r.status,
"applies_when": r.applies_when, "canon_version": r.canon_version,
"topic_id": r.topic_id,
"created_at": r.created_at.isoformat() if r.created_at else None,
"updated_at": r.updated_at.isoformat() if r.updated_at else None,
}
for r in rows
]
def _family_idea_platform_rows(rows, platform_slugs: dict[int, str]) -> list[dict]:
return [
{
"note_id": r.note_id,
"platform_slug": platform_slugs.get(r.platform_id or 0),
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _family_idea_reference_rows(rows) -> list[dict]:
return [
{
"idea_id": r.idea_id, "snippet_id": r.snippet_id,
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _family_adoption_rows(rows) -> list[dict]:
"""The adoption ledger (v25). Project, idea and owed task are SOURCE ids."""
return [
{
"project_id": r.project_id, "idea_id": r.idea_id,
"status": r.status, "reason": r.reason,
"canon_version": r.canon_version,
"assessed_at": r.assessed_at.isoformat() if r.assessed_at else None,
"decided_via": r.decided_via, "owed_task_id": r.owed_task_id,
"created_at": r.created_at.isoformat() if r.created_at else None,
"updated_at": r.updated_at.isoformat() if r.updated_at else None,
}
for r in rows
]
def _family_decision_rows(rows) -> list[dict]:
"""The decision log (v25), oldest first. The source `id` travels because
`precedent_ids` point at it; the restore rebuilds that map as it goes."""
return [
{
"id": r.id, "idea_id": r.idea_id, "project_id": r.project_id,
"action": r.action, "reason": r.reason,
"before": r.before, "after": r.after, "evidence": r.evidence,
"precedent_ids": list(r.precedent_ids or []),
"decided_via": r.decided_via, "user_id": r.user_id,
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _rule_rows(rows) -> list[dict]:
return [
{
@@ -923,6 +1054,24 @@ async def export_full_backup() -> dict:
rulebooks = (await session.execute(select(Rulebook))).scalars().all()
topics = (await session.execute(select(RulebookTopic))).scalars().all()
rules = (await session.execute(select(Rule))).scalars().all()
platforms = (await session.execute(
select(Platform).where(Platform.deleted_at.is_(None))
.order_by(Platform.order_index)
)).scalars().all()
project_platforms = (await session.execute(select(ProjectPlatform))).scalars().all()
family_ideas = (await session.execute(select(FamilyIdea))).scalars().all()
family_idea_platforms = (await session.execute(
select(FamilyIdeaPlatform)
)).scalars().all()
family_idea_references = (await session.execute(
select(FamilyIdeaReference)
)).scalars().all()
family_adoptions = (await session.execute(select(FamilyAdoption))).scalars().all()
# Oldest first: a precedent is always an earlier decision, so a restore
# in this order has every precedent mapped before anything cites it.
family_decisions = (await session.execute(
select(FamilyDecision).order_by(FamilyDecision.id)
)).scalars().all()
return {
"version": BACKUP_VERSION,
@@ -970,6 +1119,28 @@ async def export_full_backup() -> dict:
"code_shapes": _code_shape_rows(code_shapes),
"code_shape_events": _code_shape_event_rows(code_shape_events),
"code_shape_uses": _code_shape_use_rows(code_shape_uses),
**_family_sections(
platforms, project_platforms, family_ideas, family_idea_platforms,
family_idea_references, family_adoptions, family_decisions,
),
}
def _family_sections(
platforms, project_platforms, ideas, idea_platforms, references,
adoptions, decisions,
) -> dict:
"""The v25 payload sections, shared by both export scopes so the two
cannot serialise family canon differently."""
slugs = {p.id: p.slug for p in platforms}
return {
"platforms": _platform_rows(platforms),
"project_platforms": _project_platform_rows(project_platforms, slugs),
"family_ideas": _family_idea_rows(ideas),
"family_idea_platforms": _family_idea_platform_rows(idea_platforms, slugs),
"family_idea_references": _family_idea_reference_rows(references),
"family_adoptions": _family_adoption_rows(adoptions),
"family_decisions": _family_decision_rows(decisions),
}
@@ -1145,6 +1316,47 @@ async def export_user_backup(user_id: int) -> dict:
lesson_no_rule = (await session.execute(
select(LessonNoRule).where(LessonNoRule.lesson_id.in_(note_ids))
)).scalars().all() if note_ids else []
# Family canon (v25). The catalog is global and taken whole, for the
# canonical_systems reason. Everything else is scoped so that BOTH
# ends of every row restore: an idea is this user's note, a ledger row
# needs this user's project AND idea, a reference needs both notes.
platforms = (await session.execute(
select(Platform).where(Platform.deleted_at.is_(None))
.order_by(Platform.order_index)
)).scalars().all()
project_platforms = (await session.execute(
select(ProjectPlatform).where(ProjectPlatform.project_id.in_(project_ids))
)).scalars().all() if project_ids else []
family_ideas = (await session.execute(
select(FamilyIdea).where(FamilyIdea.note_id.in_(note_ids))
)).scalars().all() if note_ids else []
idea_ids = [i.note_id for i in family_ideas]
family_idea_platforms = (await session.execute(
select(FamilyIdeaPlatform).where(FamilyIdeaPlatform.note_id.in_(idea_ids))
)).scalars().all() if idea_ids else []
family_idea_references = (await session.execute(
select(FamilyIdeaReference).where(
FamilyIdeaReference.idea_id.in_(idea_ids),
FamilyIdeaReference.snippet_id.in_(note_ids),
)
)).scalars().all() if idea_ids else []
family_adoptions = (await session.execute(
select(FamilyAdoption).where(
FamilyAdoption.idea_id.in_(idea_ids),
FamilyAdoption.project_id.in_(project_ids),
)
)).scalars().all() if (idea_ids and project_ids) else []
# A decision about the idea itself has no project; one about a ledger
# row needs that row's project in this export too.
family_decisions = (await session.execute(
select(FamilyDecision).where(
FamilyDecision.idea_id.in_(idea_ids),
or_(
FamilyDecision.project_id.is_(None),
FamilyDecision.project_id.in_(project_ids or [0]),
),
).order_by(FamilyDecision.id)
)).scalars().all() if idea_ids else []
return {
"version": BACKUP_VERSION,
@@ -1194,6 +1406,10 @@ async def export_user_backup(user_id: int) -> dict:
"code_shapes": _code_shape_rows(code_shapes),
"code_shape_events": _code_shape_event_rows(code_shape_events),
"code_shape_uses": _code_shape_use_rows(code_shape_uses),
**_family_sections(
platforms, project_platforms, family_ideas, family_idea_platforms,
family_idea_references, family_adoptions, family_decisions,
),
}
@@ -1240,6 +1456,7 @@ class _Maps:
__slots__ = (
"users", "projects", "milestones", "notes", "rulebooks", "topics",
"rules", "systems", "design_systems", "shapes", "canonical_by_slug",
"platform_by_slug", "decisions",
)
def __init__(self) -> None:
@@ -1254,6 +1471,8 @@ class _Maps:
self.design_systems: dict[int, int] = {}
self.shapes: dict[int, int] = {}
self.canonical_by_slug: dict[str, int] = {}
self.platform_by_slug: dict[str, int] = {}
self.decisions: dict[int, int] = {}
def _build_user(row: dict, maps: _Maps) -> User:
@@ -1592,6 +1811,127 @@ def _build_rule_moment_judgment(row: dict, maps: _Maps) -> RuleMomentJudgment |
)
def _build_platform(row: dict, maps: _Maps) -> Platform | None:
"""Matched on SLUG, like a canonical system: this install seeded the
standard platforms from its migrations, so the common case creates nothing
and only an entry added on the source instance is built."""
slug = row.get("slug") or ""
if not slug or slug in maps.platform_by_slug:
return None
return Platform(
name=row.get("name", ""),
slug=slug,
description=row.get("description"),
markers=list(row.get("markers") or []),
order_index=row.get("order_index", 0),
)
def _build_project_platform(row: dict, maps: _Maps) -> ProjectPlatform | None:
"""Skipped unless both the project and the platform resolve."""
project = maps.projects.get(row.get("project_id", 0))
platform = maps.platform_by_slug.get(row.get("platform_slug") or "")
if project is None or platform is None:
return None
return ProjectPlatform(
project_id=project,
platform_id=platform,
state=row.get("state") or "declared",
created_at=_dt(row.get("created_at")),
)
def _build_family_idea(row: dict, maps: _Maps) -> FamilyIdea | None:
"""Skipped when its note did not restore — the state is that note's. The
topic DEGRADES to None: a standard whose rule topic did not come across is
still a standard, only without its binding half."""
note = maps.notes.get(row.get("note_id", 0))
if note is None:
return None
return FamilyIdea(
note_id=note,
status=row.get("status") or "candidate",
applies_when=row.get("applies_when"),
canon_version=row.get("canon_version") or 1,
topic_id=maps.topics.get(row.get("topic_id") or 0),
created_at=_dt(row.get("created_at")),
updated_at=_dt(row.get("updated_at")),
)
def _build_family_idea_platform(row: dict, maps: _Maps) -> FamilyIdeaPlatform | None:
note = maps.notes.get(row.get("note_id", 0))
platform = maps.platform_by_slug.get(row.get("platform_slug") or "")
if note is None or platform is None:
return None
return FamilyIdeaPlatform(
note_id=note, platform_id=platform, created_at=_dt(row.get("created_at")),
)
def _build_family_idea_reference(row: dict, maps: _Maps) -> FamilyIdeaReference | None:
idea = maps.notes.get(row.get("idea_id", 0))
snippet = maps.notes.get(row.get("snippet_id", 0))
if idea is None or snippet is None:
return None
return FamilyIdeaReference(
idea_id=idea, snippet_id=snippet, created_at=_dt(row.get("created_at")),
)
def _build_family_adoption(row: dict, maps: _Maps) -> FamilyAdoption | None:
"""Both the project and the idea must map — the row IS that pair. The owed
task degrades to None: the answer still stands without its task."""
project = maps.projects.get(row.get("project_id", 0))
idea = maps.notes.get(row.get("idea_id", 0))
if project is None or idea is None:
return None
return FamilyAdoption(
project_id=project,
idea_id=idea,
status=row.get("status") or "unassessed",
reason=row.get("reason"),
canon_version=row.get("canon_version"),
assessed_at=_dt_or_none(row.get("assessed_at")),
decided_via=row.get("decided_via"),
owed_task_id=maps.notes.get(row.get("owed_task_id") or 0),
created_at=_dt(row.get("created_at")),
updated_at=_dt(row.get("updated_at")),
)
def _build_family_decision(row: dict, maps: _Maps) -> FamilyDecision | None:
"""Skipped when its idea did not restore, or when it is about a project
that did not — an assessment without its project says nothing. The acting
user degrades to None. Precedents are remapped through the decision map
and a precedent that did not restore is dropped from the list rather than
left pointing at whatever took its number."""
idea = maps.notes.get(row.get("idea_id", 0))
if idea is None:
return None
project = None
if row.get("project_id") is not None:
project = maps.projects.get(row["project_id"])
if project is None:
return None
return FamilyDecision(
idea_id=idea,
project_id=project,
action=row.get("action") or "assess",
reason=row.get("reason") or "",
before=row.get("before"),
after=row.get("after"),
evidence=row.get("evidence"),
precedent_ids=[
maps.decisions[p] for p in (row.get("precedent_ids") or [])
if p in maps.decisions
],
decided_via=row.get("decided_via") or "agent",
user_id=maps.users.get(row.get("user_id") or 0),
created_at=_dt(row.get("created_at")),
)
def _build_rule_version(row: dict, maps: _Maps) -> RuleVersion | None:
rid = maps.rules.get(row.get("rule_id", 0))
if rid is None:
@@ -2010,6 +2350,9 @@ async def _restore_v2(data: dict) -> dict:
"rule_relations": 0, "rule_versions": 0,
"retrieval_tuning_events": 0, "lesson_rule_links": 0,
"lesson_no_rule": 0, "moment_mappings": 0,
"platforms": 0, "project_platforms": 0, "family_ideas": 0,
"family_idea_platforms": 0, "family_idea_references": 0,
"family_adoptions": 0, "family_decisions": 0,
}
async with async_session() as session:
@@ -2250,6 +2593,66 @@ async def _restore_v2(data: dict) -> dict:
session.add(answer)
stats["lesson_no_rule"] += 1
# Family canon (v25). Here because it needs projects, notes and topics
# all mapped. The platform catalog first, matched on slug and seeded
# from what this install already has — the 14c shape.
existing_platforms = (await session.execute(
select(Platform).where(Platform.deleted_at.is_(None))
)).scalars().all()
for platform in existing_platforms:
maps.platform_by_slug[platform.slug] = platform.id
for p_data in data.get("platforms", []):
platform = _build_platform(p_data, maps)
if platform is None:
continue
session.add(platform)
await session.flush()
maps.platform_by_slug[platform.slug] = platform.id
stats["platforms"] += 1
for pp in data.get("project_platforms", []):
membership = _build_project_platform(pp, maps)
if membership is None:
continue
session.add(membership)
stats["project_platforms"] += 1
for fi in data.get("family_ideas", []):
idea = _build_family_idea(fi, maps)
if idea is None:
continue
session.add(idea)
stats["family_ideas"] += 1
# The ideas must exist before anything foreign-keys them.
await session.flush()
for fp in data.get("family_idea_platforms", []):
scope = _build_family_idea_platform(fp, maps)
if scope is None:
continue
session.add(scope)
stats["family_idea_platforms"] += 1
for fr in data.get("family_idea_references", []):
ref = _build_family_idea_reference(fr, maps)
if ref is None:
continue
session.add(ref)
stats["family_idea_references"] += 1
for fa in data.get("family_adoptions", []):
adoption = _build_family_adoption(fa, maps)
if adoption is None:
continue
session.add(adoption)
stats["family_adoptions"] += 1
# Oldest first, flushed one at a time: each decision's new id goes into
# the map before a later decision can name it as a precedent.
for fd in sorted(data.get("family_decisions", []), key=lambda r: r.get("id") or 0):
decision = _build_family_decision(fd, maps)
if decision is None:
continue
session.add(decision)
await session.flush()
if fd.get("id"):
maps.decisions[int(fd["id"])] = decision.id
stats["family_decisions"] += 1
# A rule's edit history (milestone 323). Must come after the rules
# themselves — the rule map is only populated above — and both ids are
# ids in the SOURCE database, which is #3182's arose_from_id trap.
+69
View File
@@ -0,0 +1,69 @@
"""Family canon's schema — the parts that need no database (milestone 463 step 1).
The constraints themselves, against real Postgres, are in
tests/test_integration_family_canon.py. These pin that the migration and the
model agree on every whitelist (rule 36: a value one side accepts and the
other refuses only fails at INSERT time, which CI's unit lane never reaches),
and that the seed ships nothing an install other than this one would find
foreign (rule 115).
"""
from __future__ import annotations
import importlib.util
from pathlib import Path
from scribe.models import family
ROOT = Path(__file__).resolve().parents[1]
def _migration():
path = ROOT / "alembic" / "versions" / "0120_family_canon.py"
spec = importlib.util.spec_from_file_location("m0120", path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def test_the_migration_checks_and_the_model_agree():
m = _migration()
assert tuple(m._MEMBERSHIP) == family.MEMBERSHIP_STATES
assert tuple(m._IDEA_STATUSES) == family.IDEA_STATUSES
assert tuple(m._ADOPTION_STATUSES) == family.ADOPTION_STATUSES
assert tuple(m._REASONED) == family.REASONED_STATUSES
assert tuple(m._ACTIONS) == family.DECISION_ACTIONS
assert tuple(m._DECIDERS) == family.DECIDERS
def test_the_departures_that_need_a_reason_are_adoption_states():
assert set(family.REASONED_STATUSES) <= set(family.ADOPTION_STATUSES)
def test_the_seed_is_generic_and_unique():
seed = _migration()._SEED
slugs = [slug for _, slug, _, _ in seed]
assert len(slugs) == len(set(slugs)), "a duplicate slug breaks the live-unique index"
for name, slug, description, markers in seed:
assert name and slug and description
assert slug == slug.lower() and " " not in slug
# A list, possibly empty — never None. Detection reads it directly.
assert isinstance(markers, list)
def test_the_column_defaults_match_the_first_state_of_each_lifecycle():
"""A row created with no state given starts where its lifecycle starts.
An idea is a candidate until promoted; an answer is unassessed until
someone assesses it — never silently `adopted`."""
assert family.FamilyIdea.__table__.c.status.server_default.arg == "candidate"
assert family.FamilyAdoption.__table__.c.status.server_default.arg == "unassessed"
assert family.ProjectPlatform.__table__.c.state.server_default.arg == "declared"
def test_a_canon_ideas_scope_lives_on_the_idea_and_nowhere_else():
"""One scope source. A rule topic is linked FROM the idea and carries no
platforms of its own, so the two cannot disagree about who a standard
reaches (#4221's two-sources-of-truth shape)."""
from scribe.models.rulebook import RulebookTopic
assert "topic_id" in family.FamilyIdea.__table__.c
assert not {c.name for c in RulebookTopic.__table__.c} & {"platform_id", "platform_ids"}
+304
View File
@@ -0,0 +1,304 @@
"""Real-Postgres checks for family canon's schema (milestone 463 step 1).
Two kinds of claim live here, and both need a database:
1. THE CONSTRAINTS. The agent assesses and promotes with no approval step, so
the few things that must always hold are held by the schema rather than by
prose a session might skip: a canon idea states when it applies, a variant
or an exemption says why, a decision has a reason. Each is shown refusing
the bad row, and each beside a good row so the refusal is not a fixture
that fails for some other reason.
2. THE RESTORE REMAPS. Two seams can come back plausible and wrong:
- platforms travel by SLUG and must land on the destination's own seeded
rows, not create duplicates of them;
- `precedent_ids` is a list of ids INSIDE JSON. A restore that copied it
raw would leave each decision pointing at whatever took the old number —
populated, plausible, and about the wrong decision.
"""
import pytest
import pytest_asyncio
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from scribe.models import async_session
from scribe.models.family import (
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform,
FamilyIdeaReference, Platform, ProjectPlatform,
)
from scribe.models.note import Note
from scribe.models.project import Project
from scribe.models.user import User
from scribe.services import backup
from tests.helpers import ensure_user
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
OWNER_USERNAME = "family_canon_owner"
RESTORED_USERNAME = "family_canon_restored"
async def _purge(username: str) -> None:
"""user -> project / note is ON DELETE CASCADE, and every family table
cascades from a project or a note, so dropping the user's rows clears
everything this file made. Platforms are global and never created here
beyond the migration's seed, so there is nothing of theirs to clear."""
async with async_session() as s:
for user in (await s.execute(
select(User).where(User.username == username)
)).scalars().all():
for note in (await s.execute(
select(Note).where(Note.user_id == user.id)
)).scalars().all():
await s.delete(note)
for project in (await s.execute(
select(Project).where(Project.user_id == user.id)
)).scalars().all():
await s.delete(project)
if username == RESTORED_USERNAME:
await s.delete(user)
await s.commit()
@pytest_asyncio.fixture(autouse=True)
async def _no_leftovers():
"""SETUP ONLY — a database call after a `yield` in an autouse fixture
orphans a pooled connection (see the backup round-trip siblings)."""
await _purge(RESTORED_USERNAME)
await _purge(OWNER_USERNAME)
async def _platform(slug: str) -> Platform:
async with async_session() as s:
return (await s.execute(
select(Platform).where(Platform.slug == slug, Platform.deleted_at.is_(None))
)).scalars().one()
async def _owner_project_and_note() -> tuple[int, int, int]:
async with async_session() as s:
owner = await ensure_user(s, OWNER_USERNAME)
project = Project(user_id=owner.id, title="an app on a platform")
note = Note(user_id=owner.id, title="an idea", body="the idea")
s.add_all([project, note])
await s.commit()
return owner.id, project.id, note.id
async def _refused(*rows) -> bool:
async with async_session() as s:
s.add_all(list(rows))
try:
await s.commit()
except IntegrityError:
await s.rollback()
return True
return False
# --- the seed ---------------------------------------------------------------
async def test_the_migration_seeds_generic_platforms_with_their_markers():
android = await _platform("android-app")
assert "AndroidManifest.xml" in android.markers
# Declare-only platforms carry an empty list, never NULL: detection reads
# the list and must not have to special-case a missing one.
assert (await _platform("postgresql")).markers == []
# --- the constraints ----------------------------------------------------------
async def test_a_canon_idea_must_say_when_it_applies():
_, _, note_id = await _owner_project_and_note()
assert await _refused(FamilyIdea(note_id=note_id, status="canon", applies_when=" "))
# A candidate may be vague — that is what a candidate is.
assert not await _refused(FamilyIdea(note_id=note_id, status="candidate"))
async def test_a_variant_or_an_exemption_must_say_why():
_, project_id, note_id = await _owner_project_and_note()
assert not await _refused(FamilyIdea(
note_id=note_id, status="canon",
applies_when="any app that installs its own updates",
))
for status in ("variant", "exempt"):
assert await _refused(FamilyAdoption(
project_id=project_id, idea_id=note_id, status=status, reason=" ",
)), f"{status} with a blank reason was accepted"
# Owed and adopted need no reason: neither is a departure.
assert not await _refused(FamilyAdoption(
project_id=project_id, idea_id=note_id, status="owed", decided_via="agent",
))
async def test_an_unknown_state_is_refused():
_, project_id, note_id = await _owner_project_and_note()
assert not await _refused(FamilyIdea(note_id=note_id))
android = await _platform("android-app")
assert await _refused(ProjectPlatform(
project_id=project_id, platform_id=android.id, state="maybe",
))
assert await _refused(FamilyAdoption(
project_id=project_id, idea_id=note_id, status="declined",
))
async def test_a_decision_must_carry_a_reason():
_, _, note_id = await _owner_project_and_note()
assert not await _refused(FamilyIdea(note_id=note_id))
assert await _refused(FamilyDecision(idea_id=note_id, action="propose", reason=" "))
assert not await _refused(FamilyDecision(
idea_id=note_id, action="propose", reason="built twice, in two projects",
))
# --- the restore ------------------------------------------------------------
@pytest_asyncio.fixture
async def source():
"""An idea with everything family canon can hang off it: a platform scope,
a reference snippet, a project that declares one platform and rejects
another, a variant answer, and two decisions where the second cites the
first as its precedent.
`decoy` exists so the target database has an ADDITIONAL decision whose id
could collide with a source precedent id — without it, a raw-copied
precedent list could happen to point at nothing and read as dropped,
rather than as pointing at the wrong decision.
"""
owner_id, project_id, idea_id = await _owner_project_and_note()
android = await _platform("android-app")
container = await _platform("container-image")
async with async_session() as s:
snippet = Note(user_id=owner_id, title="the reference", body="code",
note_type="snippet")
s.add(snippet)
s.add(FamilyIdea(note_id=idea_id, status="canon", canon_version=2,
applies_when="any app that installs its own updates"))
await s.flush()
s.add_all([
FamilyIdeaPlatform(note_id=idea_id, platform_id=android.id),
FamilyIdeaReference(idea_id=idea_id, snippet_id=snippet.id),
ProjectPlatform(project_id=project_id, platform_id=android.id, state="declared"),
ProjectPlatform(project_id=project_id, platform_id=container.id, state="rejected"),
FamilyAdoption(project_id=project_id, idea_id=idea_id, status="variant",
reason="updates arrive through the store, not in-app",
canon_version=1, decided_via="agent"),
])
first = FamilyDecision(idea_id=idea_id, action="promote",
reason="proven in CI, stated in platform terms",
after={"status": "canon"})
s.add(first)
await s.flush()
second = FamilyDecision(idea_id=idea_id, project_id=project_id, action="assess",
reason="store-distributed, as in the promotion's terms",
precedent_ids=[first.id], decided_via="agent")
s.add(second)
await s.commit()
snippet_id = snippet.id
async with async_session() as s:
notes = (await s.execute(
select(Note).where(Note.id.in_([idea_id, snippet_id])).order_by(Note.id)
)).scalars().all()
payload = {
"version": backup.BACKUP_VERSION,
"users": backup._user_rows([await s.get(User, owner_id)]),
"projects": backup._project_rows([await s.get(Project, project_id)]),
"notes": backup._note_rows(notes),
**backup._family_sections(
(await s.execute(select(Platform))).scalars().all(),
(await s.execute(select(ProjectPlatform).where(
ProjectPlatform.project_id == project_id))).scalars().all(),
[await s.get(FamilyIdea, idea_id)],
(await s.execute(select(FamilyIdeaPlatform).where(
FamilyIdeaPlatform.note_id == idea_id))).scalars().all(),
(await s.execute(select(FamilyIdeaReference).where(
FamilyIdeaReference.idea_id == idea_id))).scalars().all(),
(await s.execute(select(FamilyAdoption).where(
FamilyAdoption.idea_id == idea_id))).scalars().all(),
(await s.execute(select(FamilyDecision).where(
FamilyDecision.idea_id == idea_id).order_by(FamilyDecision.id)
)).scalars().all(),
),
}
payload["users"][0]["username"] = RESTORED_USERNAME
return {"payload": payload}
@pytest_asyncio.fixture
async def restored(source):
async with async_session() as s:
before = len((await s.execute(select(Platform))).scalars().all())
stats = await backup.restore_full_backup(source["payload"])
async with async_session() as s:
user = (await s.execute(
select(User).where(User.username == RESTORED_USERNAME)
)).scalars().one()
project = (await s.execute(
select(Project).where(Project.user_id == user.id)
)).scalars().one()
notes = (await s.execute(
select(Note).where(Note.user_id == user.id)
)).scalars().all()
note_ids = [n.id for n in notes]
return {
"stats": stats,
"platforms_before": before,
"platforms_after": len((await s.execute(select(Platform))).scalars().all()),
"project": project,
"notes": {n.note_type: n for n in notes},
"idea": (await s.execute(
select(FamilyIdea).where(FamilyIdea.note_id.in_(note_ids))
)).scalars().one(),
"memberships": (await s.execute(
select(ProjectPlatform).where(ProjectPlatform.project_id == project.id)
)).scalars().all(),
"adoption": (await s.execute(
select(FamilyAdoption).where(FamilyAdoption.project_id == project.id)
)).scalars().one(),
"decisions": (await s.execute(
select(FamilyDecision).where(FamilyDecision.idea_id.in_(note_ids))
.order_by(FamilyDecision.id)
)).scalars().all(),
"references": (await s.execute(
select(FamilyIdeaReference).where(FamilyIdeaReference.idea_id.in_(note_ids))
)).scalars().all(),
}
async def test_platforms_match_by_slug_and_create_nothing(restored):
assert restored["stats"]["platforms"] == 0
assert restored["platforms_after"] == restored["platforms_before"]
async def test_a_rejected_platform_survives_so_detection_cannot_re_add_it(restored):
states = {m.state for m in restored["memberships"]}
assert states == {"declared", "rejected"}
async def test_the_idea_and_its_answer_come_back_whole(restored):
idea = restored["idea"]
assert idea.status == "canon"
assert idea.canon_version == 2
adoption = restored["adoption"]
assert adoption.status == "variant"
assert adoption.reason == "updates arrive through the store, not in-app"
# Assessed against version 1 of a version-2 idea: the derived recheck
# signal must survive the trip, which needs both numbers intact.
assert adoption.canon_version < idea.canon_version
async def test_the_reference_points_at_the_RESTORED_snippet(restored):
[ref] = restored["references"]
assert ref.snippet_id == restored["notes"]["snippet"].id
async def test_a_precedent_is_remapped_to_the_RESTORED_decision(restored):
"""The trap: precedent_ids is a list of ids inside JSON. Copied raw, it
would name the SOURCE decision's id — which in a shared database is a
real row, the wrong one."""
first, second = restored["decisions"]
assert first.action == "promote" and second.action == "assess"
assert second.precedent_ids == [first.id]
assert second.project_id == restored["project"].id
+46 -3
View File
@@ -27,7 +27,7 @@ def test_backup_version_is_current():
(Named for the number it asserted until v10, which is exactly the drift a
name-carrying-a-value invites; it now says what it checks.)"""
assert backup.BACKUP_VERSION == 24
assert backup.BACKUP_VERSION == 25
def _exportable_note(**over):
@@ -125,11 +125,25 @@ def test_a_repo_binding_carries_the_branch_its_ledger_follows():
assert row["ref"] == "dev"
class _AnySlug(dict):
"""A slug map in which every platform id resolves. The stand-in's
platform_id is an arbitrary integer, and the guard is about which columns
travel, not about what a missing platform does — that would make the
import builder skip a row the guard needs it to build."""
def get(self, key, default=None):
return "platform-x"
# The table -> (model, row helper) registry the column guard walks. Kept here
# rather than in the service because it exists only to be introspected: the
# product code already knows these pairings by calling them.
def _column_guard_targets():
from scribe.models.canonical_system import CanonicalSystem
from scribe.models.family import (
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform,
FamilyIdeaReference, Platform, ProjectPlatform,
)
from scribe.models.code_shape import CodeShape, CodeShapeEvent, CodeShapeUse
from scribe.models.lesson_rule_link import LessonNoRule, LessonRuleLink
from scribe.models.rule_moment_judgment import RuleMomentJudgment
@@ -187,6 +201,18 @@ def _column_guard_targets():
"code_shapes": (CodeShape, backup._code_shape_rows),
"code_shape_events": (CodeShapeEvent, backup._code_shape_event_rows),
"code_shape_uses": (CodeShapeUse, backup._code_shape_use_rows),
"platforms": (Platform, backup._platform_rows),
"project_platforms": (
ProjectPlatform, lambda rows: backup._project_platform_rows(rows, _AnySlug()),
),
"family_ideas": (FamilyIdea, backup._family_idea_rows),
"family_idea_platforms": (
FamilyIdeaPlatform,
lambda rows: backup._family_idea_platform_rows(rows, _AnySlug()),
),
"family_idea_references": (FamilyIdeaReference, backup._family_idea_reference_rows),
"family_adoptions": (FamilyAdoption, backup._family_adoption_rows),
"family_decisions": (FamilyDecision, backup._family_decision_rows),
}
@@ -306,6 +332,13 @@ def _import_guard_targets():
"code_shapes": backup._build_code_shape,
"code_shape_events": backup._build_code_shape_event,
"code_shape_uses": backup._build_code_shape_use,
"platforms": backup._build_platform,
"project_platforms": backup._build_project_platform,
"family_ideas": backup._build_family_idea,
"family_idea_platforms": backup._build_family_idea_platform,
"family_idea_references": backup._build_family_idea_reference,
"family_adoptions": backup._build_family_adoption,
"family_decisions": backup._build_family_decision,
}
return {
table: (model, helper, builders[table])
@@ -324,7 +357,8 @@ def _everything_maps(row: dict) -> "backup._Maps":
maps = backup._Maps()
ids = {v for v in row.values() if isinstance(v, int)} | {0, 1}
for name in ("users", "projects", "milestones", "notes", "rulebooks",
"topics", "rules", "systems", "design_systems", "shapes"):
"topics", "rules", "systems", "design_systems", "shapes",
"decisions"):
getattr(maps, name).update({i: i + 1000 for i in ids})
# `canonical_slug` only, never `slug`: a canonical_systems row is built
# exactly when its slug is NOT already known to the destination, so
@@ -332,6 +366,11 @@ def _everything_maps(row: dict) -> "backup._Maps":
slug = row.get("canonical_slug")
if slug:
maps.canonical_by_slug[slug] = 7
# `platform_slug` only, never `slug`, for the same reason: a platforms row
# is built exactly when its slug is unknown to the destination.
platform_slug = row.get("platform_slug")
if platform_slug:
maps.platform_by_slug[platform_slug] = 8
return maps
@@ -556,7 +595,11 @@ async def test_export_full_backup_contains_every_declared_section():
# v20: whether an area's rulings were read once shown.
"system_usage_events",
# v23: proposals and judgments about a rule's moments.
"rule_moment_judgments"):
"rule_moment_judgments",
# v25: family canon (milestone 463).
"platforms", "project_platforms", "family_ideas",
"family_idea_platforms", "family_idea_references",
"family_adoptions", "family_decisions"):
assert key in out, f"missing export section: {key}"
assert out[key] == []