Files
FabledScribe/tests/test_mcp_tool_projects.py
T
bvandeusenandClaude Opus 5 0e10f6bb8a
CI & Build / Python lint (push) Failing after 3s
CI & Build / Plugin hooks (push) Failing after 12s
CI & Build / integration (push) Failing after 27s
CI & Build / TypeScript typecheck (push) Failing after 35s
CI & Build / Python tests (push) Failing after 37s
CI & Build / Build & push image (push) Skipped
feat(rules)!: retire the always-on tier — every rule arrives by retrieval (#394)
Milestone 394, steps 5-8. Operator: "remove the always on rule functionality
as the goal was to not have it at all since it didn't seem to work as
expected."

Unconditional preload had three failures the retrieval arms do not. It could
not be MEASURED — a resident rule is in the context whether or not it
mattered, so nothing distinguished "this governed the act" from "this was
scenery", and it was the one surface structurally exempt from the scoreboard
judging every other. It was SUMMARISED AWAY by compaction while the session
went on believing it held the rules. And it CROWDED OUT the few rules that
applied with the thirty that did not.

WHAT GOES

Schema (0100): rules.tier + ck_rules_tier, rule_versions.tier,
rulebooks.always_on, and project_rulebook_exclusions — a table recording a
project's opt-out of something that no longer binds it unasked.

Tools: list_always_on_rules, exclude_always_on_rulebook,
include_always_on_rulebook. Service: the same three plus rules_etag_for,
_valid_tier and the whole etag family. The SessionStart preload and the
write-path staleness arm go with them: nothing is resident, so nothing can
have drifted since a session loaded it.

THREE CALLS WORTH REVIEWING

enter_project got NARROWER, not wider. Its filter was `always_on OR
area-tagged`; dropping the tier arm leaves the deterministic half, so a
project with no canonical-tagged Systems gets no bulk rules and reaches them
by retrieval instead. Dropping the whole clause would have made that payload
bigger than the preload this milestone deletes.

Backups import tolerantly. A pre-394 archive carries tier, always_on and the
retired inception choice; none is read, and the exclusion key is DROPPED
rather than remapped, because restoring it would write data that
validate_inception now rejects as unknown.

The migration is irreversible in the way that matters and says so: downgrade
recreates the columns at their defaults and cannot restore which rules were
always-on. A value invented to fill a hole is not a measurement.

THE INSTRUCTION SURFACES SAY THE HARDER THING

Deleting "call list_always_on_rules()" is easy; replacing it is not, because
the new model asks a session to trust something it cannot see. All three
surfaces now say a session holds nothing, that rules arrive when work matches
them, and — the half that got dangerous — that "no rule arrived" means
"nothing matched", never "there is no rule". Under residency an empty session
was rare and suspicious; it is now the ordinary state of most turns, so
reading it as permission is wrong on nearly every turn rather than
occasionally. That is #3720's defect at session scale.

test_instruction_surfaces_agree is repointed rather than retired: its two
halves collapsed into one instruction, and it gains a guard that every
surface states what absence means. _INSTRUCTIONS is back at 1999/2000 —
the inception clause paid for the longer HOW line.

UI (rule 27, and the opportunity step 8 named)

The tier selector is gone, and what replaces it is the point: `when_to_apply`
is now the field that decides whether a rule is ever seen, so the editor
marks it required, warns while it is empty, and both rule lists badge a
trigger-less rule "never surfaces". A rule without one is not quiet, it is
unreachable.

TESTS

Two files deleted outright — test_rules_etag.py and test_inception_rules.py
tested subsystems that no longer exist. Elsewhere obsolete cases were removed
and the rest repointed. One deserves naming: the wiring test asserted the act
arms pass no `tier`, which had become an assertion that could not fail. It is
repointed onto `kind`, which does still exist and where the same claim is
live — a preference must reach a write exactly as a rule does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 16:22:17 -04:00

472 lines
20 KiB
Python

"""Tests for fable_*_project tools."""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from scribe.mcp.tools.projects import (
list_projects, get_project, create_project,
update_project, enter_project,
)
from tests.helpers import FakeMCP, fake_project
pytestmark = pytest.mark.usefixtures("_bind_user")
@pytest.fixture(autouse=True)
def _no_systems():
"""enter_project now surfaces the project's Systems as the tagging
vocabulary (#2546). These are tool-layer unit tests with no database, so
the lookup is stubbed to the common case — a project with none. The
populated shape is asserted in its own test below.
"""
with patch("scribe.mcp.tools.projects.systems_svc.list_systems",
AsyncMock(return_value=[])):
yield
@pytest.fixture(autouse=True)
def _no_coverage():
"""enter_project also reads the pattern-coverage cache (#2692) — same
deal: no database here, stub the common case (nothing computed). The
populated line is asserted in tests/test_pattern_coverage.py.
"""
with patch("scribe.mcp.tools.projects.coverage_svc.cached_coverage",
AsyncMock(return_value=None)):
yield
@pytest.fixture(autouse=True)
def _no_background_seed():
"""enter_project now fire-and-forgets a coverage self-seed (#2802). These
are no-database unit tests, so the spawn is stubbed out; the firing shape
has its own test below."""
with patch("scribe.mcp.tools.projects.spawn") as mock:
yield mock
@pytest.fixture(autouse=True)
def _no_bootstrap():
"""With _no_systems stubbing an empty vocabulary, every test here reaches
the zero-Systems branch, whose bootstrap ask (#2683) counts the project's
records — a database read. Stub the common case (young project, no ask);
the firing shape has its own test below.
"""
with patch("scribe.mcp.tools.projects.systems_tools.bootstrap_systems_ask",
AsyncMock(return_value=None)) as mock:
yield mock
@pytest.mark.asyncio
async def test_list_projects_wraps_in_dict():
rows = [fake_project(id=1), fake_project(id=2)]
with patch(
"scribe.mcp.tools.projects.projects_svc.list_projects",
AsyncMock(return_value=rows),
):
out = await list_projects()
assert len(out["projects"]) == 2
@pytest.mark.asyncio
async def test_get_project_enriches_with_milestone_summary():
p = fake_project(id=5, title="found")
milestone_summary = [{"id": 10, "title": "MS", "task_count": 3}]
applicable_payload = {
"rules": [], "truncated": False, "subscribed_rulebooks": [],
}
with patch(
"scribe.mcp.tools.projects.projects_svc.get_project",
AsyncMock(return_value=p),
), patch(
"scribe.mcp.tools.projects.milestones_svc.get_project_milestone_summary",
AsyncMock(return_value=milestone_summary),
), patch(
"scribe.mcp.tools.projects.rulebooks_svc.get_applicable_rules",
AsyncMock(return_value=applicable_payload),
):
out = await get_project(project_id=5)
assert out["id"] == 5
assert out["milestone_summary"] == milestone_summary
@pytest.mark.asyncio
async def test_get_project_includes_applicable_rules_and_subscribed_rulebooks():
"""The augmented get_project response includes applicable_rules and
subscribed_rulebooks pulled from services/rulebooks.get_applicable_rules.
"""
p = fake_project(id=3, title="Fabled Assistant")
milestone_summary = []
applicable_payload = {
"rules": [
{"id": 1, "title": "dev is home",
"statement": "Work directly on dev",
"topic_title": "git-workflow",
"rulebook_title": "FabledSword family"},
],
"truncated": False,
"subscribed_rulebooks": [{"id": 1, "title": "FabledSword family"}],
}
with patch(
"scribe.mcp.tools.projects.projects_svc.get_project",
AsyncMock(return_value=p),
), patch(
"scribe.mcp.tools.projects.milestones_svc.get_project_milestone_summary",
AsyncMock(return_value=milestone_summary),
), patch(
"scribe.mcp.tools.projects.rulebooks_svc.get_applicable_rules",
AsyncMock(return_value=applicable_payload),
):
out = await get_project(project_id=3)
assert out["applicable_rules"][0]["title"] == "dev is home"
assert out["subscribed_rulebooks"] == [{"id": 1, "title": "FabledSword family"}]
assert out["applicable_rules_truncated"] is False
@pytest.mark.asyncio
async def test_get_project_raises_when_not_found():
with patch(
"scribe.mcp.tools.projects.projects_svc.get_project",
AsyncMock(return_value=None),
):
with pytest.raises(ValueError, match="project 999 not found"):
await get_project(project_id=999)
@pytest.mark.asyncio
async def test_create_project_passes_color_empty_as_none():
p = fake_project()
mock = AsyncMock(return_value=p)
with patch("scribe.mcp.tools.projects.projects_svc.create_project", mock):
await create_project(title="P", color="")
assert mock.call_args.kwargs["color"] is None
@pytest.mark.asyncio
async def test_update_project_only_sends_non_default_fields():
p = fake_project()
mock = AsyncMock(return_value=p)
with patch("scribe.mcp.tools.projects.projects_svc.update_project", mock):
await update_project(project_id=1, status="archived")
args, kwargs = mock.call_args
assert args == (7, 1)
assert kwargs == {"status": "archived"}
@pytest.mark.asyncio
async def test_update_project_raises_when_not_found():
with patch(
"scribe.mcp.tools.projects.projects_svc.update_project",
AsyncMock(return_value=None),
):
with pytest.raises(ValueError, match="project 999 not found"):
await update_project(project_id=999, title="x")
@pytest.mark.asyncio
async def test_enter_project_composes_full_context():
"""enter_project pulls project + rules + milestone summary + open tasks +
recent notes in one composed call."""
p = fake_project(id=5, title="P")
applicable_payload = {
"rules": [{"id": 1, "title": "r1", "statement": "s",
"topic_title": "t", "rulebook_title": "rb"}],
"project_rules": [{"id": 99, "title": "pr1", "statement": "ps"}],
"truncated": False,
"subscribed_rulebooks": [{"id": 2, "title": "rb"}],
}
milestone_summary = [{"id": 10, "title": "MS", "task_count": 3}]
task1 = MagicMock()
task1.id = 100; task1.title = "T1"; task1.status = "in_progress"
task1.priority = "high"; task1.task_kind = "work"; task1.milestone_id = 10
note1 = MagicMock()
note1.id = 200; note1.title = "N1"
note1.updated_at = None # avoids datetime mocking
with patch(
"scribe.mcp.tools.projects.projects_svc.get_project",
AsyncMock(return_value=p),
), patch(
"scribe.mcp.tools.projects.rulebooks_svc.get_applicable_rules",
AsyncMock(return_value=applicable_payload),
), patch(
"scribe.mcp.tools.projects.milestones_svc.get_project_milestone_summary",
AsyncMock(return_value=milestone_summary),
), patch(
"scribe.mcp.tools.projects.notes_svc.list_notes",
AsyncMock(side_effect=[([task1], 1), ([note1], 1)]),
):
out = await enter_project(project_id=5)
assert out["project"]["id"] == 5
assert out["milestone_summary"] == milestone_summary
assert out["applicable_rules"][0]["title"] == "r1"
assert out["project_rules"][0]["id"] == 99
assert out["subscribed_rulebooks"] == [{"id": 2, "title": "rb"}]
assert out["open_tasks"][0]["id"] == 100
assert out["open_tasks"][0]["status"] == "in_progress"
assert out["recent_notes"][0]["id"] == 200
# No design system on this project -> the key is present and null, not
# absent. A caller that has to distinguish "no key" from "no system" will
# eventually get it wrong.
assert out["design_system"] is None
# No Systems -> present-and-empty, NOT absent: this key is the tagging
# vocabulary, and "this project has no named areas yet" is information the
# create-the-System instruction acts on.
assert out["systems"] == []
# Young project, no bootstrap ask -> the key is ABSENT, not null: it exists
# to be acted on, and a key that usually says null gets skipped (#2483).
assert "systems_bootstrap" not in out
@pytest.mark.asyncio
async def test_enter_project_surfaces_the_systems_vocabulary():
"""The tagging instruction is only executable if the vocabulary is in
front of the agent when it writes. It never was, and tagging stopped three
days after the feature landed — one System, nothing tagged since July 28
(#2546's audit). Trimmed to id/name/first-line: it rides on every session
start, and the full charter is get_system's job."""
p = fake_project(id=5)
sys1 = MagicMock()
sys1.id = 3
sys1.name = "retrieval"
sys1.description = "Embeddings, ranking, auto-inject.\nLong detail below."
with patch(
"scribe.mcp.tools.projects.projects_svc.get_project",
AsyncMock(return_value=p),
), patch(
"scribe.mcp.tools.projects.rulebooks_svc.get_applicable_rules",
AsyncMock(return_value={"rules": [], "truncated": False,
"subscribed_rulebooks": []}),
), patch(
"scribe.mcp.tools.projects.milestones_svc.get_project_milestone_summary",
AsyncMock(return_value=[]),
), patch(
"scribe.mcp.tools.projects.notes_svc.list_notes",
AsyncMock(side_effect=[([], 0), ([], 0)]),
), patch(
"scribe.mcp.tools.projects.systems_svc.list_systems",
AsyncMock(return_value=[sys1]),
):
out = await enter_project(project_id=5)
assert out["systems"] == [
{"id": 3, "name": "retrieval",
"description": "Embeddings, ranking, auto-inject."}
]
def _enter_project_stubs(p):
"""The four patches every enter_project test repeats, as one context list."""
return [
patch("scribe.mcp.tools.projects.projects_svc.get_project",
AsyncMock(return_value=p)),
patch("scribe.mcp.tools.projects.rulebooks_svc.get_applicable_rules",
AsyncMock(return_value={"rules": [], "truncated": False,
"subscribed_rulebooks": []})),
patch("scribe.mcp.tools.projects.milestones_svc.get_project_milestone_summary",
AsyncMock(return_value=[])),
patch("scribe.mcp.tools.projects.notes_svc.list_notes",
AsyncMock(side_effect=[([], 0), ([], 0)])),
]
@pytest.mark.asyncio
async def test_enter_project_carries_the_bootstrap_ask_when_it_fires():
"""The arrival-moment half of #2683: a mature zero-Systems project greets
the session with the concrete bootstrap ask, before it is deep in a task —
the moment minting a starter vocabulary is cheapest."""
import contextlib
ask = "This project has 282 records and NO Systems modelled — ..."
with contextlib.ExitStack() as stack:
for cm in _enter_project_stubs(fake_project(id=5)):
stack.enter_context(cm)
stack.enter_context(patch(
"scribe.mcp.tools.projects.systems_tools.bootstrap_systems_ask",
AsyncMock(return_value=ask),
))
out = await enter_project(project_id=5)
assert out["systems_bootstrap"] == ask
@pytest.mark.asyncio
async def test_enter_project_never_asks_bootstrap_once_a_vocabulary_exists(
_no_bootstrap,
):
"""The ask is self-retiring: the first System ends it — enter_project must
not even evaluate it once the vocabulary is non-empty."""
import contextlib
sys1 = MagicMock()
sys1.id = 3; sys1.name = "retrieval"; sys1.description = ""
with contextlib.ExitStack() as stack:
for cm in _enter_project_stubs(fake_project(id=5)):
stack.enter_context(cm)
stack.enter_context(patch(
"scribe.mcp.tools.projects.systems_svc.list_systems",
AsyncMock(return_value=[sys1]),
))
out = await enter_project(project_id=5)
assert "systems_bootstrap" not in out
_no_bootstrap.assert_not_awaited()
@pytest.mark.asyncio
async def test_enter_project_fires_the_coverage_seed_on_the_owner(
_no_background_seed,
):
"""The arrival self-seed (#2802): entering spawns refresh_if_stale on the
project OWNER's id with the cache read the enter already did — fire and
forget, so a missing ledger seeds itself without the UI button, and the
enter stays fast."""
import contextlib
project = fake_project(id=5)
project.user_id = 42 # explicit: the OWNER, not the caller (ctx uid=7)
with contextlib.ExitStack() as stack:
for cm in _enter_project_stubs(project):
stack.enter_context(cm)
seed = stack.enter_context(patch(
"scribe.mcp.tools.projects.coverage_svc.refresh_if_stale",
MagicMock(return_value=object()),
))
await enter_project(project_id=5)
seed.assert_called_once_with(42, 5, cached=None)
_no_background_seed.assert_called_once()
assert _no_background_seed.call_args.kwargs["site"] == "enter_project.coverage_seed"
@pytest.mark.asyncio
async def test_enter_project_hands_back_the_design_system_when_the_project_has_one():
"""The handshake is where an agent learns what binds it, and a design
system binds the same way a rule does. Before this it was reachable only by
an agent that already knew to call resolve_design_system — so the standards
were present in the store and absent from the work."""
p = fake_project(id=5, design_system_id=9)
design = {"id": 9, "title": "App kit", "guidance": [{"title": "House"}],
"token_count": 95, "token_groups": ["surface"],
"inherits_from": ["House"], "description": ""}
with patch(
"scribe.mcp.tools.projects.projects_svc.get_project",
AsyncMock(return_value=p),
), patch(
"scribe.mcp.tools.projects.rulebooks_svc.get_applicable_rules",
AsyncMock(return_value={"rules": [], "truncated": False,
"subscribed_rulebooks": []}),
), patch(
"scribe.mcp.tools.projects.milestones_svc.get_project_milestone_summary",
AsyncMock(return_value=[]),
), patch(
"scribe.mcp.tools.projects.notes_svc.list_notes",
AsyncMock(side_effect=[([], 0), ([], 0)]),
), patch(
"scribe.mcp.tools.projects.design_systems_svc.design_context",
AsyncMock(return_value=design),
) as ctx:
out = await enter_project(project_id=5)
assert out["design_system"]["token_count"] == 95
assert out["design_system"]["inherits_from"] == ["House"]
assert ctx.await_args.args == (7, 9) # caller's id, the project's system
@pytest.mark.asyncio
async def test_enter_project_raises_when_project_not_found():
with patch(
"scribe.mcp.tools.projects.projects_svc.get_project",
AsyncMock(return_value=None),
):
with pytest.raises(ValueError, match="project 999 not found"):
await enter_project(project_id=999)
def test_enter_project_registered_in_register():
"""register(mcp) registers enter_project alongside the existing tools."""
from scribe.mcp.tools.projects import register
mcp = FakeMCP()
register(mcp)
assert "enter_project" in mcp.names
# --- milestone 297: the inception doors ---------------------------------------
@pytest.mark.asyncio
async def test_create_project_without_inception_args_stays_undecided():
p = fake_project(id=5, title="P", inception=None)
with patch("scribe.mcp.tools.projects.projects_svc.create_project", AsyncMock(return_value=p)), \
patch("scribe.mcp.tools.projects.inception_svc.decide", AsyncMock()) as decide:
out = await create_project(title="P")
decide.assert_not_awaited()
assert "inception_hint" in out and "inception_effects" not in out
@pytest.mark.asyncio
async def test_create_project_with_inception_args_decides_via_mcp():
p = fake_project(id=5, title="P", inception=None)
decided = {"inception": {"via": "mcp", "choices": {}}, "effects": {"systems_seeded": []}}
with patch("scribe.mcp.tools.projects.projects_svc.create_project", AsyncMock(return_value=p)), \
patch("scribe.mcp.tools.projects.inception_svc.decide", AsyncMock(return_value=decided)) as decide:
out = await create_project(title="P", subscribe_rulebooks=[1], design_system_id=-1, seed_systems=True)
kw = decide.await_args.kwargs
assert decide.await_args.args[1] == 5 and kw["via"] == "mcp"
assert kw["choices"] == {"subscribe_rulebooks": [1],
"design_system_id": None, "seed_systems": True}
assert out["inception"]["via"] == "mcp" and "inception_effects" in out
@pytest.mark.asyncio
async def test_decide_project_inception_tool_records_an_inherit_all_decision_when_given_nothing():
from scribe.mcp.tools.projects import decide_project_inception
decided = {"inception": {"via": "mcp"}, "effects": {}}
with patch("scribe.mcp.tools.projects.inception_svc.decide", AsyncMock(return_value=decided)) as decide:
out = await decide_project_inception(project_id=5)
assert decide.await_args.kwargs["choices"] == {}
assert out["project_id"] == 5 and out["inception"]["via"] == "mcp"
@pytest.mark.asyncio
async def test_enter_project_carries_the_inception_ask_only_for_an_undecided_own_project():
applicable = {"rules": [], "project_rules": [], "truncated": False,
"subscribed_rulebooks": []}
ask = {"defaults": {}, "ask": "decide", "call": "decide_project_inception(...)"}
async def run(project):
with patch("scribe.mcp.tools.projects.projects_svc.get_project", AsyncMock(return_value=project)), \
patch("scribe.mcp.tools.projects.rulebooks_svc.get_applicable_rules", AsyncMock(return_value=applicable)), \
patch("scribe.mcp.tools.projects.milestones_svc.get_project_milestone_summary", AsyncMock(return_value=[])), \
patch("scribe.mcp.tools.projects.notes_svc.list_notes", AsyncMock(side_effect=[([], 0), ([], 0)])), \
patch("scribe.mcp.tools.projects.systems_svc.list_systems", AsyncMock(return_value=[])), \
patch("scribe.mcp.tools.projects.systems_tools.bootstrap_systems_ask", AsyncMock(return_value=None)), \
patch("scribe.mcp.tools.projects.inception_svc.inception_ask", AsyncMock(return_value=ask)) as asked:
return await enter_project(project_id=5), asked
# Own + undecided → the ask rides along.
out, asked = await run(fake_project(id=5, title="P", user_id=7, inception=None))
assert out["inception"] == ask and asked.await_count == 1
# Decided → absent, and the ask is not even built.
out, asked = await run(fake_project(id=5, title="P", user_id=7, inception={"via": "legacy"}))
assert "inception" not in out and asked.await_count == 0
# Someone else's (shared) project, undecided → not this caller's to decide.
out, asked = await run(fake_project(id=5, title="P", user_id=8, inception=None))
assert "inception" not in out and asked.await_count == 0
def test_inception_routes_and_tool_are_registered():
from scribe.app import create_app
from scribe.mcp.server import build_mcp_server
rules = {r.rule for r in create_app().url_map.iter_rules()}
assert "/api/projects/<int:project_id>/inception" in rules
assert "/api/projects/<int:project_id>/inception/defaults" in rules
mcp = build_mcp_server()
assert mcp._tool_manager.get_tool("decide_project_inception") is not None
tool = mcp._tool_manager.get_tool("create_project")
for name in ("subscribe_rulebooks", "design_system_id", "seed_systems"):
assert name in tool.parameters.get("properties", {}), name