feat(design-systems): check the components against the sheet they claim to use
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 43s
CI & Build / Build & push image (push) Successful in 35s

"The snippets use the tags from the sheet" was a relation nobody could verify.
Now it is three checks, and all three currently fail SILENTLY in this codebase:

  unknown             `var(--x)` where the system declares no `--x`. Renders as
                      nothing at all — no error, no failing test, no visual clue
                      beyond the element quietly not being styled.
  superseded literals a value the sheet said to stop writing, paired with the
                      token to write instead. Only possible because `supersedes`
                      is declared rather than inferred.
  local definitions   custom properties a snippet mints for itself instead of
                      reusing the sheet's — the bloat a shared sheet exists to
                      prevent, where a value stops being reused and starts being
                      restated per component.

The first is not hypothetical. Writing DesignSystemsView.vue earlier in this
same session I used `--color-accent` throughout; it does not exist, and nothing
in the toolchain noticed. This check is the thing that would have.

A token that is both defined and read locally is reported ONCE, as an unknown
reference — "--btn-bg does not exist in the sheet" is the more precise statement
of the same problem, and reporting both would double-count one fact.

Literal matching is boundary-aware and case-insensitive: `#fff` must not fire
inside `#ffffff` (different colours, and a finding on the wrong one sends
someone to change correct code), while `#FFFFFF` in a rulebook has to match
`#ffffff` in a stylesheet — the same trap `normalize_hex` exists for.

Snippets with nothing to report are omitted entirely. A list of everything that
is fine is a list nobody reads twice — the same principle the auto-inject menu
and the drift panel are both built on.

Two integration mistakes fixed while wiring it: `list_snippets` returns
`(rows, total)` and caps its limit at 100, and `get_snippet` returns a Note
model rather than a dict. The list rows carry a preview, not the code, so the
check reads each full body — checking the preview would have reported on a
truncation.
This commit is contained in:
2026-07-30 21:47:47 -04:00
parent b0a7d9e89b
commit 46d88f9e7e
8 changed files with 448 additions and 2 deletions
+91
View File
@@ -8,6 +8,7 @@ declaration.
from types import SimpleNamespace
from scribe.services.design_stylesheet import (
check_code_against_tokens,
duplicate_values,
is_valid_token_name,
render_stylesheet,
@@ -228,3 +229,93 @@ def test_valueless_tokens_never_count_as_duplicates_of_each_other():
"""Otherwise every unfilled token would collide with every other one and the
report would be nothing but noise on a fresh import."""
assert duplicate_values([_token("--fs-a", {}), _token("--fs-b", {})]) == {}
# --- reading the sheet from the other side ----------------------------------
#
# "The snippets use the tags from the sheet" made checkable. All three findings
# below are currently SILENT in this codebase — no error, no failing test.
def _tok(name, base=None, supersedes=()):
return SimpleNamespace(
name=name,
value_by_mode={"base": base} if base else {},
supersedes=list(supersedes),
group_name=None, purpose=None,
)
SHEET = [
_tok("--fs-obsidian", "#14171a"),
_tok("--fs-parchment", "#e8e4d8", supersedes=["#fff", "#ffffff"]),
]
def test_a_reference_to_a_token_that_does_not_exist_is_reported():
"""THE bug this exists for. `var(--color-accent)` where no such token exists
renders as nothing at all — invisible focus rings, unstyled elements, no
error and no failing test. It happened in this codebase this session."""
report = check_code_against_tokens("outline: 2px solid var(--color-accent);", SHEET)
assert report["unknown"] == ["--color-accent"]
assert report["used"] == []
def test_a_reference_that_resolves_is_reported_as_used_not_as_a_finding():
report = check_code_against_tokens("background: var(--fs-obsidian);", SHEET)
assert report["used"] == ["--fs-obsidian"]
assert report["unknown"] == []
def test_a_superseded_literal_is_found_and_names_its_replacement():
"""The reframe paying off end to end: the finding says what to WRITE, not
merely what is wrong. That is only possible because the token declared it."""
report = check_code_against_tokens("color: #fff;", SHEET)
assert report["superseded_literals"] == [
{"literal": "#fff", "use_instead": "--fs-parchment"}
]
def test_a_shorter_hex_does_not_match_inside_a_longer_one():
"""`#fff` must not fire on `#ffffff` — different colours, and a finding on
the wrong one sends someone to change code that was already correct."""
report = check_code_against_tokens("color: #ffffffee;", SHEET)
assert [f["literal"] for f in report["superseded_literals"]] == []
def test_the_literal_match_is_case_insensitive():
"""Rulebooks write `#FFFFFF` and code writes `#ffffff`. A case-sensitive
check would silently find nothing — the same trap normalize_hex exists for."""
report = check_code_against_tokens("color: #FFFFFF;", SHEET)
assert report["superseded_literals"] == [
{"literal": "#ffffff", "use_instead": "--fs-parchment"}
]
def test_a_token_defined_and_read_locally_is_reported_once_not_twice():
"""A snippet minting its own custom property is the bloat a shared sheet
exists to prevent — but when it also reads it back, that is ONE fact. The
unknown-reference check owns it, because "--btn-bg does not exist in the
sheet" is the more precise statement of the same problem."""
report = check_code_against_tokens(".btn { --btn-bg: #333; background: var(--btn-bg); }", SHEET)
assert report["unknown"] == ["--btn-bg"]
assert report["local_definitions"] == []
def test_a_local_definition_never_read_back_is_still_reported():
report = check_code_against_tokens(".btn { --btn-bg: #333; }", SHEET)
assert report["local_definitions"] == ["--btn-bg"]
def test_a_declaration_that_is_not_a_custom_property_is_not_mistaken_for_one():
report = check_code_against_tokens(".btn { color: red; background: blue; }", SHEET)
assert report["local_definitions"] == []
def test_clean_code_reports_nothing_to_act_on():
report = check_code_against_tokens(
".btn { background: var(--fs-obsidian); color: var(--fs-parchment); }", SHEET
)
assert report["unknown"] == []
assert report["superseded_literals"] == []
assert report["local_definitions"] == []
assert report["used"] == ["--fs-obsidian", "--fs-parchment"]
+5 -1
View File
@@ -25,6 +25,7 @@ def test_route_handlers_callable():
"list_design_systems", "create_design_system", "get_design_system",
"update_design_system", "delete_design_system", "resolve_design_system",
"import_design_system", "get_design_system_stylesheet",
"check_snippets_against_system",
"list_design_tokens", "create_design_token",
"update_design_token", "delete_design_token", "set_project_design_system",
):
@@ -47,6 +48,7 @@ def test_every_endpoint_is_reachable_on_the_app():
"/api/design-systems/<int:design_system_id>/resolved",
"/api/design-systems/<int:design_system_id>/import",
"/api/design-systems/<int:design_system_id>/stylesheet",
"/api/design-systems/<int:design_system_id>/snippet-check",
"/api/design-systems/<int:design_system_id>/tokens",
"/api/design-tokens/<int:token_id>",
"/api/projects/<int:project_id>/design-system",
@@ -61,7 +63,7 @@ def test_service_functions_take_user_id():
"update_design_system", "delete_design_system", "resolve_design_system",
"create_token", "list_tokens", "update_token", "delete_token",
"set_project_design_system", "import_from_rulebook",
"stylesheet_for_system",
"stylesheet_for_system", "check_snippets_against_system",
):
fn = getattr(svc, fn_name)
assert callable(fn)
@@ -94,6 +96,8 @@ def test_agent_and_web_surfaces_stay_at_parity():
# so the loop above can't pair it. It still has to exist on both.
assert callable(tools.import_design_system_from_rulebook)
assert callable(routes.import_design_system)
assert callable(tools.check_snippets_against_design_system)
assert callable(routes.check_snippets_against_system)
def test_every_mcp_tool_in_the_module_is_registered():