Files
FabledScribe/tests/test_plugin_version_mint.py
T
bvandeusenandClaude Opus 5 a49e7ed2af
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 49s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / Python tests (push) Failing after 1m7s
CI & Build / Build & push image (push) Skipped
fix(plugin): the hooks need no jq and no tac (#4107)
Every hook opened `command -v jq >/dev/null 2>&1 || exit 0`, so on a machine
without jq the operator got no session context, no rules, no prior art and no
process sync — and not one word saying why, because `exit 0` is
indistinguishable from "ran fine, nothing to say". jq is absent by default on
macOS, on the Debian/Ubuntu slim images, on Alpine and in most CI containers.
That is not a prerequisite to document; it is the plugin handing its own
packaging problem to whoever installs it.

`tac` was worse: GNU-only, so the prior-art hook's enclosing-definition arm
did nothing at all on every Mac, silently, from the day it shipped. It is not
replaced but removed — scribe_defs judges each line independently, so
extracting forward and taking `tail -1` is the same answer as reversing and
taking the head, and it drops the early-exit `head` that #4042 was filed for.

No server contract changed, so a lagging plugin cache keeps working.

  scribe_json.awk   JSON -> IDX<TAB>PATH<TAB>VALUE. Two modes: `whole` for an
                    event or a response body, `lines` for a transcript, where
                    an unparseable record is dropped and the rest still read —
                    the `map(try fromjson catch empty)` the jq program opened
                    with. Arrays also report their LENGTH at `[#]`, which is
                    what keeps "zero notes" distinct from "no answer" (#2932).
  scribe_turn.awk   the turn-bounding program, replacing the thirty lines of
                    jq in the Stop hook.
  scribe_defs.sh    scribe_json_flat / _pick / _list / _len / _list_minus read,
                    scribe_json_out writes the envelope (five copies of one
                    shape, gone), scribe_urlenc replaces `jq -sRr '@uri'`.

Percent-encoding goes through `od -tu1` rather than an awk character loop on
purpose: awk's idea of a character follows the locale, so gawk reads an
accented letter as one and mawk as two, and an encoder built on substr() would
emit a different URL depending on which awk is installed. Encoding is defined
on bytes. Verified byte-identical to `jq -sRr '@uri'`.

Measured, not assumed. The per-event path costs 8ms against jq's 3ms. The
transcript path was 70x slower until two fixes: the Stop hook now finds where
the turn starts with a fixed-string grep before parsing (a needle carrying
unescaped quotes cannot occur inside a JSON string, so it matches only at a
record's top level — checked against a full JSON parse of a 27MB transcript:
152 prompt records, 152 matches, no misses, no extras), and the parser reads
each token out of a 1024-byte window instead of copying the rest of the buffer
per token, which was quadratic in line length on the 400KB tool results a
transcript carries.

Differential-tested against the jq program it replaces over 724 windows cut
from three real transcripts — 724 identical, 0 mismatched, 45 of them
exercising a real task close and a real reply. That sweep is what caught
`scribe_turn.awk` never setting FS, which truncated every multi-word reply at
its first space and was invisible to a test whose replies were all empty.

check_plugin.py's `jq -R` lint becomes a guard against either binary coming
back, and three smoke checks lose their `shutil.which("jq")` skip. jq is not
in `ci-python` either, so those three announced a skip on every CI run and had
never once run there: removing the dependency from the product also closed a
permanent hole in its verification. They pass now across all ten hooks.

tests/test_hook_json_reader.py is a differential against Python's `json` over
nested objects, arrays, unicode, escapes, control characters, empty cases and
a value longer than the token window, plus the envelope, the encoder and the
turn analyzer. 139 cases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
2026-09-20 12:20:45 -04:00

293 lines
13 KiB
Python

"""The plugin's version is MINTED, and CI is the control that it moved.
WHAT THIS IS ABOUT (milestone 334 step 3). `plugin/` ships straight from this
git repo — no build step, so no moment at which CI could stamp a version in.
The value is therefore minted by a script before the commit, and CI's job is
not to produce it but to prove it moved when it had to.
THE DIVERGENCE THESE GUARD. Two artifacts in one repo derive their versions
from different clocks, on purpose:
server image name from COMMIT time, ordering key from BUILD time
plugin one value, from MINT time
#3127 §2 prescribes commit time so two lanes building one source report one
string. The plugin has one lane and no build, so that reason does not reach
it. "Let's make these consistent" is the obvious tidy-up and it breaks
whichever artifact loses — which is why the difference is pinned here rather
than only explained in a comment.
The trade mint time makes — you cannot recompute the value from history, only
verify it moved — is acceptable ONLY because #3325 established that the
installer's refresh test is `===` with no ordering anywhere. Where a
comparator orders, an unreproducible version would be unverifiable too.
"""
import ast
import json
import pathlib
import re
from datetime import datetime, timedelta, timezone
import pytest
from scripts import check_plugin
from scripts.mint_plugin_version import VERSION_RE, mint, rewrite
MINT_SRC = pathlib.Path(check_plugin.ROOT) / "scripts" / "mint_plugin_version.py"
@pytest.fixture(autouse=True)
def _reset_failures():
"""`check_plugin.fail` appends to a module global; without this a failing
assertion in one test would be visible from the next."""
check_plugin.failures.clear()
yield
check_plugin.failures.clear()
def fake_manifest(version: str = "2026.09.01.2252") -> str:
return json.dumps(
{"name": "scribe", "description": "d", "version": version,
"userConfig": {"api_endpoint": {"type": "string"}}},
indent=2,
)
# ── The mint ───────────────────────────────────────────────────────────────
@pytest.mark.parametrize("when,expected", [
# THE midnight case, which #3127 checklist 10 names by hand. An unpadded
# `%-H%M` renders this hour as `0` and silently shortens the string.
(datetime(2026, 1, 5, 0, 0, tzinfo=timezone.utc), "2026.01.05.0000"),
(datetime(2026, 1, 5, 0, 9, tzinfo=timezone.utc), "2026.01.05.0009"),
(datetime(2026, 12, 31, 23, 59, tzinfo=timezone.utc), "2026.12.31.2359"),
(datetime(2026, 9, 1, 22, 52, tzinfo=timezone.utc), "2026.09.01.2252"),
])
def test_the_mint_zero_pads_every_field(when, expected):
assert mint(when) == expected
assert VERSION_RE.match(mint(when))
def test_the_mint_is_UTC_not_local():
"""A local-time mint would make the value depend on who ran it — two people
minting the same minute would disagree, and the string is the artifact's
identity."""
utc = datetime(2026, 9, 1, 22, 52, tzinfo=timezone.utc)
east = utc.astimezone(timezone(timedelta(hours=9)))
assert mint(east) == mint(utc) == "2026.09.01.2252"
def test_the_mint_reads_a_CLOCK_and_never_git():
"""The clock divergence from the server image, asserted structurally.
Mint time is only meaningful if nothing consults history — the moment this
script shells out to git it has quietly become a commit-time deriver, and
the two artifacts' clocks have been "made consistent" without anyone
deciding to. That change would pass every other test in this file.
Asserted over the AST rather than the text, because the module docstring
discusses git at length explaining why it is absent. This looks for USE,
not mention.
"""
tree = ast.parse(MINT_SRC.read_text())
imported = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
imported |= {a.name.split(".")[0] for a in node.names}
elif isinstance(node, ast.ImportFrom) and node.module:
imported.add(node.module.split(".")[0])
assert "subprocess" not in imported, (
"the mint script imports subprocess — a mint that can read history is "
"a commit-time deriver wearing the wrong name"
)
called = {ast.unparse(n.func) for n in ast.walk(tree) if isinstance(n, ast.Call)}
assert "datetime.now" in called, "the mint script no longer reads a clock"
# ── The rewrite ────────────────────────────────────────────────────────────
def test_the_rewrite_touches_exactly_one_line():
"""Surgical, not a JSON round-trip. The manifest's formatting and key order
are not this script's to decide, and a whole-file reformat would make every
mint an unreadable diff."""
before = fake_manifest("0.1.48")
after = rewrite(before, "2026.09.01.2252")
b, a = before.splitlines(), after.splitlines()
assert len(b) == len(a)
differing = [i for i, (x, y) in enumerate(zip(b, a)) if x != y]
assert len(differing) == 1
assert '"version": "2026.09.01.2252"' in a[differing[0]]
def test_the_rewrite_preserves_indentation_and_key_order():
weird = '{\n\t"name": "scribe",\n\t"version": "0.1.48",\n\t"z": 1\n}\n'
out = rewrite(weird, "2026.09.01.2252")
assert out == '{\n\t"name": "scribe",\n\t"version": "2026.09.01.2252",\n\t"z": 1\n}\n'
def test_the_rewrite_refuses_a_manifest_it_cannot_match():
"""Raises rather than falling back to a JSON round-trip: a manifest this
cannot match is one whose shape changed, and quietly reformatting the file
to cope would be a far larger edit than the caller asked for."""
with pytest.raises(ValueError):
rewrite('{"name": "scribe"}', "2026.09.01.2252")
def test_the_rewrite_refuses_TWO_version_lines():
"""A capped `subn` would report one replacement and look clean while the
second `version` — possibly the real one — kept its old value."""
two = '{\n "version": "0.1.48",\n "nested": {\n "version": "9.9.9"\n }\n}\n'
with pytest.raises(ValueError):
rewrite(two, "2026.09.01.2252")
# ── The version-relevant set includes its own deriver ──────────────────────
def test_the_mint_script_is_version_relevant():
"""#3127 §3's asymmetry. A change to how the version is COMPUTED is
compared against nothing — leave the deriver out of the set and a format
change never forces a re-mint, so the manifest keeps a value in the old
format indefinitely and nothing says so."""
paths = check_plugin.version_relevant_paths()
assert "scripts/mint_plugin_version.py" in paths
for shipped in check_plugin.SHIPPED_PATHS:
assert shipped in paths
def test_the_checker_is_NOT_version_relevant():
"""The inverse, and it is the easy mistake. A checker decides whether the
lane goes red, not what the artifact reports — so its absence here is a
decision, not an oversight."""
assert "scripts/check_plugin.py" not in check_plugin.version_relevant_paths()
# ── The check ──────────────────────────────────────────────────────────────
def run_check(here: str, there: str | None, changed_paths: list[str], monkeypatch):
"""Drive `check_version_is_minted` with git stubbed. Returns the failures."""
monkeypatch.setattr(
check_plugin, "_git",
lambda *a: (0, "\n".join(changed_paths)) if a[0] == "diff" else (0, ""),
)
monkeypatch.setattr(
check_plugin, "manifest_version",
lambda ref=None: here if ref is None else there,
)
monkeypatch.setattr(
check_plugin, "manifest_text",
lambda ref=None: fake_manifest(here if ref is None else (there or "0.0.0.0000")),
)
check_plugin.check_version_is_minted("origin/main")
return list(check_plugin.failures)
def test_content_changed_and_the_version_did_not_FAILS(monkeypatch):
"""#2209, exactly. The headline, and the only reason the check exists."""
failures = run_check(
"2026.09.01.2252", "2026.09.01.2252",
["plugin/hooks/scribe_session_context.sh"], monkeypatch,
)
assert len(failures) == 1
assert "still 2026.09.01.2252" in failures[0]
assert "scribe_session_context.sh" in failures[0]
def test_content_changed_and_the_version_moved_PASSES(monkeypatch):
assert run_check(
"2026.09.01.2252", "2026.08.30.1200",
["plugin/hooks/scribe_session_context.sh"], monkeypatch,
) == []
def test_a_version_that_is_not_the_canonical_shape_FAILS(monkeypatch):
"""`K4` returns the manifest string verbatim, so a malformed value is not
rejected by the installer — it either sorts as an ordinary string or, when
unreadable, forces a reinstall every session. Neither is loud (#3325)."""
failures = run_check("0.1.48", "0.1.47", [], monkeypatch)
assert len(failures) == 1
assert "not YYYY.MM.DD.HHMM" in failures[0]
@pytest.mark.parametrize("bad", ["2026.9.1.2252", "2026.09.01.252", "2026.09.01"])
def test_an_UNPADDED_or_short_version_FAILS(bad, monkeypatch):
"""The padding is the contract, not cosmetics — one shape for every version
in the family (#3127 checklist 10)."""
assert run_check(bad, "2026.08.30.1200", [], monkeypatch) != []
def test_a_version_in_the_future_FAILS(monkeypatch):
ahead = (datetime.now(timezone.utc) + timedelta(days=400)).strftime("%Y.%m.%d.%H%M")
failures = run_check(ahead, "2026.08.30.1200", [], monkeypatch)
assert len(failures) == 1
assert "in the future" in failures[0]
def test_a_version_minted_minutes_ago_is_NOT_in_the_future(monkeypatch):
"""The guard has to tolerate ordinary skew: the mint happens on a
workstation and the lane runs later, on another machine's clock."""
now = datetime.now(timezone.utc).strftime("%Y.%m.%d.%H%M")
assert run_check(now, "2026.08.30.1200", [], monkeypatch) == []
def test_nothing_changed_and_nothing_minted_PASSES(monkeypatch):
assert run_check("2026.09.01.2252", "2026.09.01.2252", [], monkeypatch) == []
def test_a_version_that_moved_with_no_content_change_is_NOT_a_failure(monkeypatch):
"""Deliberately a pass. A needless re-mint costs one cache refresh; failing
the lane over a harmless act is how a check earns a `--no-version` in
somebody's muscle memory and stops running at all. The implication that
matters is one-directional: content changed IMPLIES version moved."""
assert run_check("2026.09.01.2252", "2026.08.30.1200", [], monkeypatch) == []
def test_a_failed_diff_FAILS_rather_than_passing_quietly(monkeypatch):
"""A check that cannot run must not report the same thing as a check that
passed — #2663's lesson, and the reason this file's siblings exist.
`rev-parse` is stubbed to SUCCEED so only the diff fails. Failing every git
call would trip the base-branch guard first and this would pass while
proving nothing about the diff arm.
"""
monkeypatch.setattr(
check_plugin, "_git",
lambda *a: (128, "fatal: bad object") if a[0] == "diff" else (0, ""),
)
monkeypatch.setattr(check_plugin, "manifest_version",
lambda ref=None: "2026.09.01.2252")
check_plugin.check_version_is_minted("origin/main")
assert len(check_plugin.failures) == 1
assert "git diff" in check_plugin.failures[0]
# ── The real manifest ──────────────────────────────────────────────────────
def test_the_shipped_manifest_carries_a_minted_version():
"""The end of the hand-bumped scheme, asserted on the real file. `0.1.48`
was the last of 48 numbers a person typed."""
version = json.loads(check_plugin.MANIFEST.read_text())["version"]
assert VERSION_RE.match(version), (
f"the shipped manifest says {version!r}, which is not a minted version"
)
def test_the_session_context_hook_still_reads_the_version_field():
"""The marker #2220 asked for. The value's SHAPE changed, not the field or
its reader — if this had to move, the derivation went somewhere it should
not have.
The READER changed once, in #4107: `jq -r '.version'` became
`scribe_json_pick ... '.version'` when the hooks stopped depending on jq.
What this pins is unchanged — that the hook still reads `.version` out of
the manifest rather than deriving the string some other way."""
hook = (check_plugin.HOOKS_DIR / "scribe_session_context.sh").read_text()
assert re.search(r"scribe_json_pick[^\n]*'\.version'", hook)