FabledScribe/tests/test_trigger_shape_is_taught.py

"""Every rule-write surface SHOWS what a trigger looks like.

WHY THIS EXISTS

`when_to_apply` is not documentation. `rule_document()` uses it twice — as
the embedded title's second half and again above the body — so it dominates
the vector, and a rule whose trigger names a CATEGORY rather than a moment
collapses toward its title and never arrives.

Bad triggers come from an uneven contract rather than from careless authors.
Triggers written where the guidance exists name a moment; triggers written
without it — "during hard debugging", "when reading any request from the
operator" — name categories. The guidance has to exist on the `update_*`
surfaces too, which is where every
RETROFITTED trigger is written, and retrofitting is most of the work: a
trigger that already reads fine as English is the one nobody rewrites.

WHAT THIS PINS

Two properties, neither of them wording:

  1. A tool accepting `when_to_apply` mentions it — a parameter the
     docstring never names is one nobody fills well.
  2. It carries a worked CONTRAST: one `RETRIEVES:` example and one
     `COLLAPSES:` example, quoted, and different from each other.

WHY A CONVENTION RATHER THAN A PROSE HEURISTIC

The second property guards the softer regression — guidance kept but
abstracted back to "name the moment in session vocabulary", which is advice
about being concrete that is not itself concrete.

Two ways to detect that in free prose fail, and the reason generalises:

  - Counting quoted multi-word phrases anywhere in the docstring measured
    ambient quotation, not demonstrated triggers. It PASSES the abstracted
    version by scoring unrelated prose, and text with an odd number of
    quote characters produced matches spanning the gap BETWEEN two
    unrelated phrases.
  - Scoping that count to a window after each "trigger"/"when_to_apply"
    mention FAILS `create_preference` in its correct state, because
    its examples sit further from the first mention than any defensible
    window reaches.

Both are proxies trying to infer demonstration from prose. The answer is to
stop inferring: a two-line labelled contrast is unambiguous to parse, free
in its wording, and a better teaching form than the sentences it replaced —
`RETRIEVES:`/`COLLAPSES:` names the mechanism, so the label does work for
the reader instead of only for the test. Where a property cannot be
measured, changing the shape of the thing is cheaper than a cleverer
measurement.

What it still cannot do is judge whether a GOOD example is good. It catches
the likely failure: a write surface shipping with the contrast absent,
half-present, or tidied back into abstract advice.

THE SURFACE LIST IS DERIVED, NEVER HAND-KEPT. It comes from what
`rulebooks.register()` actually hands the server, filtered to signatures
taking `when_to_apply`, so a rule-write tool added later is in scope on the
day it is added. A hand-kept list has to be remembered by the person least
likely to know it exists — the argument `RANKED_SOURCES` makes in
services/rule_usage.py.
"""
import inspect
import re

import pytest

from tests.helpers import tool_doc as _doc

_MODULE = "scribe.mcp.tools.rulebooks"

# The worked contrast. Quoted so the example's own punctuation stays inside
# it, labelled so no proximity guess is needed to find it.
_RETRIEVES = re.compile(r'RETRIEVES:\s*"([^"]+)"')
_COLLAPSES = re.compile(r'COLLAPSES:\s*"([^"]+)"')


def _registered_tools() -> list:
    """Every function `rulebooks.register()` actually hands to the server.

    Collected by handing `register` a stand-in that records what it is given,
    rather than by reading the source or repeating the tuple here. The point
    is that this cannot drift from what ships.
    """
    from scribe.mcp.tools import rulebooks

    collected: list = []

    class _Collector:
        def tool(self, name=None):
            def register_one(fn):
                collected.append(fn)
                return fn
            return register_one

    rulebooks.register(_Collector())
    return collected


def _trigger_writers() -> list[str]:
    """The registered tools that can write a rule's trigger."""
    names = [
        fn.__name__
        for fn in _registered_tools()
        if "when_to_apply" in inspect.signature(fn).parameters
    ]
    # An empty list would make every case below vacuous and the guard would
    # pass by having nothing to check — absence read as non-existence. Fail
    # loudly instead.
    assert names, "no rule-write surface takes when_to_apply; the guard is blind"
    return sorted(names)


@pytest.mark.parametrize("name", _trigger_writers())
def test_a_trigger_writing_surface_documents_the_field(name):
    """A tool that can write a trigger must say what the field is for."""
    doc = _doc(_MODULE, name)
    assert "when_to_apply" in doc, (
        f"{name} accepts when_to_apply but never mentions it in its "
        "docstring. The tool docstring is the agent-facing contract, and a "
        "trigger written against no contract names a category instead of a "
        "moment — and a category is not a thing any session ever types."
    )


@pytest.mark.parametrize("name", _trigger_writers())
def test_a_trigger_writing_surface_shows_the_contrast(name):
    """Advice about being concrete has to be concrete itself."""
    doc = _doc(_MODULE, name)
    retrieves, collapses = _RETRIEVES.findall(doc), _COLLAPSES.findall(doc)

    assert retrieves and collapses, (
        f"{name} is missing its worked contrast — found "
        f"{len(retrieves)} RETRIEVES and {len(collapses)} COLLAPSES example(s), "
        "and one of each is required. Telling an author to 'name the moment "
        "in session vocabulary' without showing one is the guidance that was "
        "already on file while the corpus filled with categories. Add two "
        'quoted lines: RETRIEVES: "<a moment a session actually produces — a '
        'command, an error, a half-formed ask>" and COLLAPSES: "<the same '
        'thing named as a category>".'
    )

    same = set(retrieves) & set(collapses)
    assert not same, (
        f"{name} shows the same text as both RETRIEVES and COLLAPSES "
        f"({next(iter(same))!r}). The pair teaches by DIFFERING — one moment "
        "written the way a session would produce it, and the same moment "
        "written as a category. Identical halves demonstrate nothing."
    )