CI & Build / Python lint (push) Failing after 3s
CI & Build / Plugin hooks (push) Failing after 12s
CI & Build / integration (push) Failing after 27s
CI & Build / TypeScript typecheck (push) Failing after 35s
CI & Build / Python tests (push) Failing after 37s
CI & Build / Build & push image (push) Skipped
Milestone 394, steps 5-8. Operator: "remove the always on rule functionality as the goal was to not have it at all since it didn't seem to work as expected." Unconditional preload had three failures the retrieval arms do not. It could not be MEASURED — a resident rule is in the context whether or not it mattered, so nothing distinguished "this governed the act" from "this was scenery", and it was the one surface structurally exempt from the scoreboard judging every other. It was SUMMARISED AWAY by compaction while the session went on believing it held the rules. And it CROWDED OUT the few rules that applied with the thirty that did not. WHAT GOES Schema (0100): rules.tier + ck_rules_tier, rule_versions.tier, rulebooks.always_on, and project_rulebook_exclusions — a table recording a project's opt-out of something that no longer binds it unasked. Tools: list_always_on_rules, exclude_always_on_rulebook, include_always_on_rulebook. Service: the same three plus rules_etag_for, _valid_tier and the whole etag family. The SessionStart preload and the write-path staleness arm go with them: nothing is resident, so nothing can have drifted since a session loaded it. THREE CALLS WORTH REVIEWING enter_project got NARROWER, not wider. Its filter was `always_on OR area-tagged`; dropping the tier arm leaves the deterministic half, so a project with no canonical-tagged Systems gets no bulk rules and reaches them by retrieval instead. Dropping the whole clause would have made that payload bigger than the preload this milestone deletes. Backups import tolerantly. A pre-394 archive carries tier, always_on and the retired inception choice; none is read, and the exclusion key is DROPPED rather than remapped, because restoring it would write data that validate_inception now rejects as unknown. The migration is irreversible in the way that matters and says so: downgrade recreates the columns at their defaults and cannot restore which rules were always-on. A value invented to fill a hole is not a measurement. THE INSTRUCTION SURFACES SAY THE HARDER THING Deleting "call list_always_on_rules()" is easy; replacing it is not, because the new model asks a session to trust something it cannot see. All three surfaces now say a session holds nothing, that rules arrive when work matches them, and — the half that got dangerous — that "no rule arrived" means "nothing matched", never "there is no rule". Under residency an empty session was rare and suspicious; it is now the ordinary state of most turns, so reading it as permission is wrong on nearly every turn rather than occasionally. That is #3720's defect at session scale. test_instruction_surfaces_agree is repointed rather than retired: its two halves collapsed into one instruction, and it gains a guard that every surface states what absence means. _INSTRUCTIONS is back at 1999/2000 — the inception clause paid for the longer HOW line. UI (rule 27, and the opportunity step 8 named) The tier selector is gone, and what replaces it is the point: `when_to_apply` is now the field that decides whether a rule is ever seen, so the editor marks it required, warns while it is empty, and both rule lists badge a trigger-less rule "never surfaces". A rule without one is not quiet, it is unreachable. TESTS Two files deleted outright — test_rules_etag.py and test_inception_rules.py tested subsystems that no longer exist. Elsewhere obsolete cases were removed and the rest repointed. One deserves naming: the wiring test asserted the act arms pass no `tier`, which had become an assertion that could not fail. It is repointed onto `kind`, which does still exist and where the same claim is live — a preference must reach a write exactly as a rule does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
325 lines
13 KiB
Python
325 lines
13 KiB
Python
"""Shared test helpers — the plain functions tests call, as opposed to the
|
|
fixtures in conftest.py.
|
|
|
|
Each of these was copied into several test modules before #2825 consolidated
|
|
them; a module imports what it needs with ``from tests.helpers import ...``.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
from contextlib import contextmanager
|
|
from datetime import datetime, timezone
|
|
from types import SimpleNamespace
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
|
|
async def drive_update_note(note, **kwargs):
|
|
"""Run `services/notes.update_note` against a stand-in row.
|
|
|
|
The patch stack is the point: update_note reaches for a version snapshot,
|
|
an embedding refresh and a project reactivation on its way out, none of
|
|
which a unit test has. Written twice — once for the snippet mirror
|
|
(#3128) and once for the verification fields (#3182/317) — before being
|
|
consolidated here.
|
|
|
|
Returns whatever update_note returned; assert on the `note` you passed in.
|
|
"""
|
|
from unittest.mock import AsyncMock as _AsyncMock
|
|
|
|
session = make_mock_session()
|
|
result = MagicMock()
|
|
result.scalars.return_value.first.return_value = note
|
|
session.execute = _AsyncMock(return_value=result)
|
|
with patch("scribe.services.notes.async_session") as cls, \
|
|
patch("scribe.services.notes.embed_note", MagicMock()), \
|
|
patch("scribe.services.notes._maybe_reactivate_project", _AsyncMock()), \
|
|
patch("scribe.services.note_versions.create_version", _AsyncMock()):
|
|
cls.return_value = session
|
|
from scribe.services.notes import update_note
|
|
return await update_note(user_id=7, note_id=note.id, **kwargs)
|
|
|
|
|
|
def tool_doc(module: str, name: str) -> str:
|
|
"""An MCP tool's docstring, whitespace-flattened.
|
|
|
|
Flattened because these are hard-wrapped at ~76 characters, so any phrase
|
|
worth asserting on is liable to straddle a line break — a property of the
|
|
formatter, not of the guidance. The disambiguator guard (#3123) learned
|
|
that on its own first run, matching raw text and reporting a phrase absent
|
|
that was plainly there.
|
|
|
|
Used by every test that pins the docstring CONTRACT rather than its
|
|
wording. The tool docstring is the agent-facing contract (rule 119), so
|
|
these guards exist to catch it being tidied down to a parameter list.
|
|
"""
|
|
import importlib
|
|
import re as _re
|
|
|
|
fn = getattr(importlib.import_module(module), name)
|
|
assert fn.__doc__, f"{name} has no docstring at all"
|
|
return _re.sub(r"\s+", " ", fn.__doc__)
|
|
|
|
|
|
def compiled_sql(element, dialect=None) -> str:
|
|
"""A SQLAlchemy clause or statement rendered as literal SQL text.
|
|
|
|
For asserting on the shape of a predicate without a database — which is how
|
|
the visibility clauses and the knowledge facets are both tested. Was a
|
|
private copy in each of those modules before #3128 needed a third.
|
|
|
|
Pass `dialect` when the assertion is about something only one backend
|
|
renders — a Postgres row-lock mode, say. The generic dialect is enough for
|
|
a predicate's shape and would quietly drop the rest.
|
|
"""
|
|
return str(element.compile(dialect=dialect, compile_kwargs={"literal_binds": True}))
|
|
|
|
|
|
def make_mock_session() -> AsyncMock:
|
|
"""A stand-in for ``async_session()`` — usable as ``async with``, with the
|
|
commit/refresh/add surface a service touches.
|
|
|
|
``add`` is a MagicMock because the real ``Session.add`` is synchronous;
|
|
an AsyncMock there would hand the service an un-awaited coroutine.
|
|
"""
|
|
s = AsyncMock()
|
|
s.__aenter__ = AsyncMock(return_value=s)
|
|
s.__aexit__ = AsyncMock(return_value=False)
|
|
s.add = MagicMock()
|
|
s.commit = AsyncMock()
|
|
s.refresh = AsyncMock()
|
|
return s
|
|
|
|
|
|
async def ensure_user(session, username: str, role: str = "user"):
|
|
"""Get-or-create a User by username inside an open session (flushed, not
|
|
committed).
|
|
|
|
Integration tests share one database for the whole lane run, so a second
|
|
test re-creating the same username dies on the unique constraint —
|
|
every integration seed goes through this instead of ``User(...)`` + add.
|
|
"""
|
|
from sqlalchemy import select
|
|
|
|
from scribe.models.user import User
|
|
|
|
existing = (
|
|
await session.execute(select(User).where(User.username == username))
|
|
).scalar_one_or_none()
|
|
if existing is not None:
|
|
return existing
|
|
user = User(username=username, role=role)
|
|
session.add(user)
|
|
await session.flush()
|
|
return user
|
|
|
|
|
|
def fake_record(**attrs) -> MagicMock:
|
|
"""A MagicMock record with REAL values on the attributes named, and a
|
|
``to_dict()`` that mirrors them.
|
|
|
|
The hazard this exists for (note 2109): an auto-created MagicMock attribute
|
|
is truthy and has a repr — so a bare MagicMock handed to the product reads
|
|
as trashed, shared, a task, and owned by a MagicMock. Name every attribute
|
|
the code under test will read; the per-model ``fake_*`` builders below
|
|
carry the ordinary defaults so a call site states only what the test is
|
|
about. ``created_at`` / ``updated_at`` are set as attributes but kept out
|
|
of ``to_dict()`` (no test serialises them, and the real models isoformat
|
|
them).
|
|
"""
|
|
n = MagicMock()
|
|
for key, value in attrs.items():
|
|
setattr(n, key, value)
|
|
n.to_dict.return_value = {
|
|
k: v for k, v in attrs.items() if k not in ("created_at", "updated_at")
|
|
}
|
|
return n
|
|
|
|
|
|
def _with_defaults(defaults: dict, attrs: dict) -> MagicMock:
|
|
values = dict(defaults)
|
|
values.update(attrs)
|
|
return fake_record(**values)
|
|
|
|
|
|
def _now():
|
|
return datetime.now(timezone.utc)
|
|
|
|
|
|
def fake_note(**attrs) -> MagicMock:
|
|
"""A stand-in Note: own (user_id=7, the caller `_bind_user` binds), live,
|
|
not a task, no structured data. The injected menu reads is_task /
|
|
task_kind / note_type for its kind marker, user_id for the "shared by …"
|
|
attribution, data for a snippet's language, deleted_at for trash."""
|
|
return _with_defaults({
|
|
"id": 1, "title": "t", "body": "", "tags": [], "user_id": 7,
|
|
"note_type": "note", "is_task": False, "task_kind": "work",
|
|
"data": None, "deleted_at": None,
|
|
# Milestone 317: a truthy mock here reads as "this note carries a
|
|
# check", which trips the guard on records that may not have one.
|
|
"verify_with": None, "expires_when": None, "verified_at": None,
|
|
}, attrs)
|
|
|
|
|
|
def fake_task(**attrs) -> MagicMock:
|
|
"""A stand-in task note — get_task reads parent_id, deleted_at, user_id."""
|
|
return _with_defaults({
|
|
"id": 1, "title": "t", "body": "", "status": "todo", "priority": "none",
|
|
"tags": [], "parent_id": None, "project_id": None, "is_task": True,
|
|
"task_kind": "work", "user_id": 7, "deleted_at": None,
|
|
"verify_with": None, "expires_when": None, "verified_at": None,
|
|
}, attrs)
|
|
|
|
|
|
def fake_snippet(**attrs) -> MagicMock:
|
|
"""A stand-in snippet note. ``data`` is explicitly None: snippet_fields
|
|
prefers `data` when truthy, and a MagicMock is truthy."""
|
|
return _with_defaults({
|
|
"id": 1, "title": "debounce — rate-limit a callback",
|
|
"body": "```js\nreturn 1\n```\n", "tags": ["js", "snippet"],
|
|
"note_type": "snippet", "is_task": False, "task_kind": "work",
|
|
"user_id": 7, "data": None, "deleted_at": None,
|
|
"status": None,
|
|
"verify_with": None, "expires_when": None, "verified_at": None,
|
|
}, attrs)
|
|
|
|
|
|
def fake_project(**attrs) -> MagicMock:
|
|
"""design_system_id is explicit: a truthy auto-attribute would route every
|
|
project through the design-system branch and out to a real database."""
|
|
return _with_defaults({
|
|
"id": 1, "title": "P", "description": "", "goal": "", "status": "active",
|
|
"color": None, "design_system_id": None, "user_id": 7,
|
|
}, attrs)
|
|
|
|
|
|
def fake_milestone(**attrs) -> MagicMock:
|
|
return _with_defaults({
|
|
"id": 1, "project_id": 1, "title": "MS", "description": None,
|
|
"status": "active", "order_index": 0,
|
|
}, attrs)
|
|
|
|
|
|
def fake_system(**attrs) -> MagicMock:
|
|
return _with_defaults(
|
|
{"id": 1, "name": "Reader", "project_id": 5, "canonical_id": None}, attrs,
|
|
)
|
|
|
|
|
|
def fake_rulebook(**attrs) -> MagicMock:
|
|
return _with_defaults({
|
|
"id": 1, "owner_user_id": 7, "title": "FabledSword family",
|
|
"description": "", "created_at": _now(), "updated_at": _now(),
|
|
}, attrs)
|
|
|
|
|
|
def fake_topic(**attrs) -> MagicMock:
|
|
return _with_defaults({
|
|
"id": 10, "rulebook_id": 1, "title": "git-workflow", "description": "",
|
|
"order_index": 0, "created_at": _now(), "updated_at": _now(),
|
|
}, attrs)
|
|
|
|
|
|
def fake_rule(**attrs) -> MagicMock:
|
|
return _with_defaults({
|
|
"id": 1, "topic_id": 10, "project_id": None, "title": "dev is home",
|
|
"statement": "Work directly on dev", "why": "", "how_to_apply": "",
|
|
# Named for the note-2109 reason the whole helper exists: unnamed,
|
|
# `when_to_apply` and `arose_from_id` would be truthy MagicMocks and
|
|
# rule_brief would attach both keys on every stand-in.
|
|
"when_to_apply": None, "arose_from_id": None,
|
|
# Named for the same reason one line up, and it bites harder here.
|
|
# `rule_brief` and `to_dict` both emit `kind or "rule"`, and a
|
|
# MagicMock is truthy — so an unnamed `kind` would put a MagicMock
|
|
# where every payload promises a force, and every stand-in rule would
|
|
# read as neither a rule nor a preference.
|
|
"kind": "rule",
|
|
# Same reason, and the same trap one field further on: an unnamed
|
|
# `verify_with` is a truthy MagicMock, so every stand-in rule would
|
|
# claim to carry a check and rule_brief would stamp a MagicMock date
|
|
# onto all of them. Most rules have none — that is the default here.
|
|
"verify_with": None, "expires_when": None, "verified_at": None,
|
|
"order_index": 0, "created_at": _now(), "updated_at": _now(),
|
|
}, attrs)
|
|
|
|
|
|
def plain_rule_detail():
|
|
"""Stub `rulebooks_svc.rule_detail` down to the record's own dict.
|
|
|
|
Every rule-tool unit test needs it and none of them wants it: the real
|
|
`rule_detail` reads the rule's Systems and its typed edges from the
|
|
database, which a unit test has none of. What these tests assert is that
|
|
the TOOL forwarded the right arguments, so the seam is stubbed the same
|
|
way the create/update calls themselves already are.
|
|
|
|
Consolidated here on its second copy, per this module's own reason for
|
|
existing (#2825) — two stubs for one seam drift apart quietly, and a test
|
|
stubbing the seam slightly differently is a test asserting something
|
|
slightly different than it appears to.
|
|
"""
|
|
async def _detail(_uid, rule, _system_ids=None):
|
|
return rule.to_dict()
|
|
return patch("scribe.mcp.tools.rulebooks.rulebooks_svc.rule_detail", _detail)
|
|
|
|
|
|
class FakeMCP:
|
|
"""Stand-in for the FastMCP server a tool module's ``register(mcp)`` is
|
|
handed: records the ``name=`` of every ``@mcp.tool(...)`` registration in
|
|
``names`` and leaves the function untouched, so a test can assert which
|
|
tools a module exposes."""
|
|
|
|
def __init__(self) -> None:
|
|
self.names: list[str] = []
|
|
|
|
def tool(self, name=None):
|
|
self.names.append(name)
|
|
return lambda fn: fn
|
|
|
|
|
|
def loc(path: str = "", repo: str = "", symbol: str = "") -> dict:
|
|
"""One snippet location, in the shape the record stores."""
|
|
return {"repo": repo, "path": path, "symbol": symbol}
|
|
|
|
|
|
def design_token_stub(name, value_by_mode, group_name=None, purpose=None,
|
|
order_index=0, supersedes=None) -> SimpleNamespace:
|
|
"""A design-token row as the cascade / stylesheet code reads it."""
|
|
return SimpleNamespace(
|
|
name=name, value_by_mode=value_by_mode, group_name=group_name,
|
|
purpose=purpose, order_index=order_index, supersedes=supersedes or [],
|
|
)
|
|
|
|
|
|
@contextmanager
|
|
def http_sink(reply: bytes = b'{"context":"","note_ids":[]}'):
|
|
"""A throwaway local HTTP listener for hook end-to-end tests: yields
|
|
``(port, seen)`` where ``seen`` collects every GET's parsed query string
|
|
(one dict per request, in order). Lets the shell be tested end to end —
|
|
the extraction, the encoding, the URL — without a Scribe instance.
|
|
|
|
Three test modules each carried their own ``_Sink`` handler before #2904
|
|
consolidated them here; pass ``reply`` for the body the hook should see.
|
|
"""
|
|
import http.server
|
|
import threading
|
|
import urllib.parse
|
|
|
|
seen: list[dict] = []
|
|
|
|
class _Sink(http.server.BaseHTTPRequestHandler):
|
|
def do_GET(self):
|
|
seen.append(urllib.parse.parse_qs(urllib.parse.urlparse(self.path).query))
|
|
self.send_response(200)
|
|
self.send_header("Content-Type", "application/json")
|
|
self.end_headers()
|
|
self.wfile.write(reply)
|
|
|
|
def log_message(self, *a):
|
|
pass
|
|
|
|
server = http.server.HTTPServer(("127.0.0.1", 0), _Sink)
|
|
threading.Thread(target=server.serve_forever, daemon=True).start()
|
|
try:
|
|
yield server.server_port, seen
|
|
finally:
|
|
server.shutdown()
|
|
server.server_close()
|