feat(design): the panel now asks whether the app agrees with its own sheet
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 58s
CI & Build / Build & push image (push) Successful in 43s

Retiring rulebook #2 left the /design drift panel with no data source, and
because its empty state was well-written the feature read as working while it
could only ever render "nothing designated" (#2419). The original question is
genuinely gone: theme.css is generated from design system 2, so checking the
system against a sheet derived from it would be a tautology.

The question that survives is the one no server can answer. A generated sheet
still has to be LOADED and APPLIED, and nothing checked that it was:

  absent      the record declares a token the app doesn't have — the sheet was
              never regenerated after the record changed, or never loaded
  differs     the app has it with another value — a stale sheet, or a later
              rule that overrode it
  unrecorded  the app declares a token in the record's own family that the
              record has never heard of

Both sides go through the same engine so the comparison is honest: declared
values are set on an offscreen probe and read back, which performs the same
var() substitution the browser already did to the live values. Comparing raw
strings would mark every derived token as drift.

The designation moved with the feature — design_rulebook_id becomes
ui_design_system_id, with a migration deleting the retired key rather than
leaving an inert row. The prose extractor it fed goes too (#2288 said its
runtime role ended when the import landed).

Three orphans of the same shape, found alongside and fixed here:

- darkOverriddenNames hardcoded [data-theme="dark"]. The sheet went dark-first
  months ago, so it matched nothing and the "mode-aware" flag silently left the
  gallery. Now matches the SHAPE of a mode selector, which also holds for an
  install whose modes aren't light and dark.
- groupFor's prefix table never heard of --fs-, so 110 tokens sat under
  "other". Groups now come from the record where there is one; the table can
  only know families that shipped with the product (rule #115).
- The type scale was a hand-written table of nine sizes marked "no token",
  true when written and false since the scale was recorded. Now rendered from
  whatever size tokens the sheet declares, so it can't go stale twice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
This commit is contained in:
2026-08-03 20:50:03 -04:00
co-authored by Claude Opus 5
parent 841506b10c
commit 5b824c1626
12 changed files with 704 additions and 671 deletions
+31 -15
View File
@@ -1,29 +1,45 @@
"""Design-system surface — what the rulebook expects of the stylesheet.
"""This install's UI surface — which design system it claims to be built from.
The client owns the other half of the comparison: it reads live token values from
the browser (see `utils/designTokens.ts`), which is the only place they exist
resolved. This endpoint supplies the claims to check them against.
Kept separate from the design-systems CRUD blueprint on purpose. That one is
the RECORD: create a system, move a token, read the cascade. This one answers a
question about the RUNNING APP, and it exists because those are not the same
question. A design system can be a perfect record of a stylesheet the app never
loaded.
The client owns the other half. `utils/designTokens.ts` reads what the browser
actually resolved, which is the one thing no server can report, and compares it
to what this endpoint's system declares. So the comparison is
"does the app agree with its own sheet?" rather than "is the record
self-consistent?", which would be a tautology — the sheet is generated from the
record (#2419).
"""
from quart import Blueprint, jsonify
from scribe.auth import get_current_user_id, login_required
from scribe.services import design_rulebook_import as design_svc
from scribe.services import design_systems as ds_svc
design_bp = Blueprint("design", __name__, url_prefix="/api/design")
@design_bp.get("/expectations")
@design_bp.get("/ui-system")
@login_required
async def get_expectations():
"""Checkable claims from the rulebook this install designated as its design system.
async def get_ui_system():
"""The design system this install designated as the source of its own UI.
Returns `{"rulebook_id": int|null, "expectations": [...]}`.
Returns `{"design_system_id": int|null, "title": str|null}`.
`rulebook_id: null` is the NORMAL case, not an error — an install that has
not designated a design rulebook has nothing to compare against, and the
client shows an explanatory empty state (rule #115). Distinguishing it from
"designated but empty" is why the id is returned alongside the list.
Both nulls is the NORMAL case, not an error — an install that has not
designated one has nothing to check the running app against, and the client
shows an explanatory empty state (rule #115).
An id with a null title is the third case and the reason the id is returned
separately: designated, but deleted or not readable by this caller. Folding
that into "none designated" is precisely how a feature comes to render a
reassuring empty state forever.
"""
uid = get_current_user_id()
result = await design_svc.design_expectations(uid)
return jsonify(result.as_dict())
system_id, system = await ds_svc.ui_design_system(uid)
return jsonify({
"design_system_id": system_id,
"title": system.title if system else None,
})
@@ -1,232 +0,0 @@
"""Design-system expectations — turning rulebook prose into checkable claims.
Milestone #251 step 2. The drift panel compares what the design rulebook SAYS
against what the stylesheet and components actually DO. This module owns the
first half: reading a rulebook's rules and extracting the claims that can be
mechanically checked.
WHY THIS LIVES SERVER-SIDE. The frontend has no test runner — `vue-tsc --noEmit`
is the entire check — and this is the one genuinely fiddly piece of the feature.
Extraction happens here where pytest can assert on it; the comparison itself is
set arithmetic and stays in the browser, where the live token values are.
WHY NOT NLP. Rule statements are prose written for humans, and they should stay
that way — they are read by people far more often than they are parsed. So this
extracts only what is unambiguous in ANY prose: the hex colours and CSS custom
property names a rule mentions. Everything subtler (padding scales, type ramps)
needs a rule author to opt into a structured form, which is deliberately left for
when someone wants it rather than invented up front.
RULE #115. Nothing here assumes a design rulebook exists, or that it is this
operator's. An install designates one; an install that hasn't gets an empty
result and a panel that explains itself.
"""
from __future__ import annotations
import logging
import re
from dataclasses import dataclass, field
from scribe.models.rulebook import Rule
from scribe.services.settings import get_setting
logger = logging.getLogger(__name__)
# Which rulebook describes this install's design system. A plain setting rather
# than a column: no migration, discoverable in the Settings UI (rule #25), and
# honest about being a per-install choice rather than a property of the rulebook.
DESIGN_RULEBOOK_SETTING = "design_rulebook_id"
# `#abc` and `#aabbcc`, plus the 4/8-digit alpha forms.
_HEX = re.compile(r"#([0-9a-fA-F]{3,8})\b")
# A custom-property name as written in prose, including the slash shorthand the
# rulebook uses: `--fs-radius-sm/md/lg/xl`, `--fs-obsidian/iron/slate/pewter`.
_TOKEN = re.compile(r"(--[a-zA-Z][\w-]*(?:/[\w-]+)*)")
# Sentence-ish split. Rules use semicolons as hard breaks as often as periods.
_SENTENCE_SPLIT = re.compile(r"(?<=[.;])\s+|\n+")
# Negation markers. Checked PER SENTENCE, which is the whole trick — see
# _extract_from_sentence.
_NEGATIONS = ("never", "not ", "no ", "avoid", "don't", "must not", "excluded")
@dataclass
class Expectation:
"""One mechanically-checkable claim a rule makes."""
kind: str # "token" | "color" | "prohibited_color"
value: str # "--fs-obsidian" | "#14171a"
rule_id: int
rule_title: str
context: str # the sentence it came from, for showing your work
def as_dict(self) -> dict:
return {
"kind": self.kind,
"value": self.value,
"rule_id": self.rule_id,
"rule_title": self.rule_title,
"context": self.context,
}
@dataclass
class ExpectationSet:
rulebook_id: int | None = None
expectations: list[Expectation] = field(default_factory=list)
def as_dict(self) -> dict:
return {
"rulebook_id": self.rulebook_id,
"expectations": [e.as_dict() for e in self.expectations],
}
def normalize_hex(value: str) -> str | None:
"""Fold a hex colour to a comparable form, or None if it isn't one.
Load-bearing for the whole comparison: the rulebook writes `#FFFFFF` and the
code writes `#fff`, and those must compare equal or the single largest drift
finding (#2275) reads as zero. Expands 3-digit shorthand and lowercases.
Alpha forms (4 and 8 digit) keep their alpha — `#fff` and `#ffff` are not the
same colour, and silently dropping the alpha would invent equality.
"""
match = _HEX.fullmatch(value.strip()) or _HEX.match(value.strip())
if not match:
return None
digits = match.group(1).lower()
if len(digits) in (3, 4):
digits = "".join(c * 2 for c in digits)
if len(digits) not in (6, 8):
return None
return f"#{digits}"
def expand_token_shorthand(raw: str) -> list[str]:
"""`--fs-radius-sm/md/lg/xl` -> the four names it stands for.
The rulebook writes token families in a slash shorthand, and both forms it
uses expand correctly under one rule: take everything up to and including the
LAST hyphen of the first segment as the prefix, then append each alternative.
--fs-radius-sm/md/lg/xl prefix `--fs-radius-` -> sm, md, lg, xl
--fs-obsidian/iron/slate prefix `--fs-` -> obsidian, iron, slate
--fs-dur-fast/base/slow prefix `--fs-dur-` -> fast, base, slow
A name with no slash is returned as-is.
"""
if "/" not in raw:
return [raw]
head, *rest = raw.split("/")
cut = head.rfind("-")
if cut <= 1: # no hyphen beyond the leading `--`
return [head, *rest]
prefix = head[: cut + 1]
return [head, *[f"{prefix}{part}" for part in rest if part]]
def _is_negated(sentence: str) -> bool:
return any(marker in sentence.lower() for marker in _NEGATIONS)
def _extract_from_sentence(sentence: str, rule: Rule) -> list[Expectation]:
"""Claims in ONE sentence, with negation scoped to that sentence.
Sentence scope is what makes the prohibition detection usable. Rule 52 reads:
"Text tokens: Parchment #E8E4D8 …, Vellum #C2BFB4 …, Ash #9C9A92 ….
Pure white #FFFFFF is NEVER used as text color."
Three colours the palette REQUIRES and one it FORBIDS, in one statement.
Detecting negation across the whole statement would mark all four as
forbidden; detecting it per sentence gets all four right.
"""
out: list[Expectation] = []
negated = _is_negated(sentence)
for match in _HEX.finditer(sentence):
value = normalize_hex(match.group(0))
if not value:
continue
out.append(Expectation(
kind="prohibited_color" if negated else "color",
value=value,
rule_id=int(rule.id),
rule_title=rule.title,
context=sentence.strip(),
))
# Token names are not negated in practice — a rule says which tokens should
# exist, never which must not — so they are recorded as expectations
# regardless. If that ever changes, it needs its own kind rather than
# borrowing the colour one.
for match in _TOKEN.finditer(sentence):
for name in expand_token_shorthand(match.group(1)):
out.append(Expectation(
kind="token",
value=name,
rule_id=int(rule.id),
rule_title=rule.title,
context=sentence.strip(),
))
return out
def extract_expectations(rules: list[Rule]) -> list[Expectation]:
"""Every checkable claim across a set of rules, deduped on (kind, value).
First occurrence wins so the reported rule is the one that introduced the
claim, which is usually the most specific place to send a reader.
"""
seen: set[tuple[str, str]] = set()
out: list[Expectation] = []
for rule in rules:
text = " ".join(filter(None, [rule.statement or "", rule.how_to_apply or ""]))
for sentence in _SENTENCE_SPLIT.split(text):
if not sentence.strip():
continue
for expectation in _extract_from_sentence(sentence, rule):
key = (expectation.kind, expectation.value)
if key in seen:
continue
seen.add(key)
out.append(expectation)
return out
async def get_design_rulebook_id(user_id: int) -> int | None:
"""The rulebook this install designated as its design system, if any."""
raw = (await get_setting(user_id, DESIGN_RULEBOOK_SETTING, "")).strip()
if not raw:
return None
try:
value = int(raw)
except (TypeError, ValueError):
return None
return value if value > 0 else None
async def design_expectations(user_id: int) -> ExpectationSet:
"""Checkable claims from the designated design rulebook.
Returns an empty set when no rulebook is designated — the normal case for
any install but the one that set it up (rule #115). The caller shows an
explanatory empty state rather than treating this as an error.
"""
rulebook_id = await get_design_rulebook_id(user_id)
if rulebook_id is None:
return ExpectationSet()
from scribe.services import rulebooks as rulebooks_svc
try:
rules = await rulebooks_svc.list_rules(user_id, rulebook_id=rulebook_id)
except Exception:
logger.warning("Design rulebook %s could not be read", rulebook_id, exc_info=True)
return ExpectationSet(rulebook_id=rulebook_id)
return ExpectationSet(rulebook_id=rulebook_id, expectations=extract_expectations(rules))
+35
View File
@@ -37,6 +37,7 @@ from scribe.services.design_cascade import (
resolve_tokens,
would_cycle,
)
from scribe.services.settings import get_setting
logger = logging.getLogger(__name__)
@@ -134,6 +135,40 @@ async def get_design_system(user_id: int, design_system_id: int) -> DesignSystem
return system
# Which design system this install's own UI is built from. A plain setting
# rather than a column: no migration, discoverable in the Settings UI (rule
# #25), and honest about being a per-install claim rather than a property of the
# system — the same system can be the record for an app that never loads it.
UI_DESIGN_SYSTEM_SETTING = "ui_design_system_id"
async def ui_design_system(user_id: int) -> tuple[int | None, DesignSystem | None]:
"""The design system this install says its UI is built from.
Returns `(id, system)`. Three outcomes, deliberately distinguishable:
- `(None, None)` — nothing designated. The NORMAL state for any install but
the one that set it up (rule #115), not an error.
- `(id, None)` — designated, but gone or not readable by this caller. A
misconfiguration worth naming rather than silently degrading to "none",
which is exactly the failure that orphaned the panel this feeds (#2419).
- `(id, system)` — designated and readable.
A non-numeric setting value reads as nothing designated: the value is only
ever written by a `<select>` of real ids, so garbage here means hand-edited
or stale, and refusing to guess is better than raising on a page load.
"""
raw = (await get_setting(user_id, UI_DESIGN_SYSTEM_SETTING, "")).strip()
if not raw:
return None, None
try:
system_id = int(raw)
except ValueError:
logger.warning("Ignoring non-numeric %s: %r", UI_DESIGN_SYSTEM_SETTING, raw)
return None, None
return system_id, await get_design_system(user_id, system_id)
async def list_design_systems(user_id: int) -> list[DesignSystem]:
"""The caller's own systems, ordered by title. Empty is normal."""
async with async_session() as session: