rename(547): the server is grimoire — package, image, DB defaults, backups
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 56s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 44s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 56s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 44s
Step 2 of the Scribe -> Grimoire rename. - src/scribe -> src/grimoire, and every import, patch target, module path, hypercorn entry point and `python -m grimoire.restore` with it; pyproject, uv.lock and the frontend package are named grimoire. - Image git.fabledsword.com/bvandeusen/fabledgrimoire; CI's Postgres service runs as grimoire / grimoire_test. - Compose files default the DB user, database and (prod) volume to grimoire, each overridable from .env, so an install created as Scribe keeps its data by setting POSTGRES_USER, POSTGRES_DB and PGDATA_VOLUME. - Backups are written as grimoire-<ts>.tar with format grimoire-backup-tar. Archives named scribe-* with the scribe-backup-tar manifest still restore and are still counted at the destination (folder and S3), with tests. - Exports, the MCP auth realm, the ASGI scope key and the persistent logger carry the new name. Kept on purpose: the secret-sealing key label stays "scribe.secrets.v1", because every stored secret was derived under it. Plugin identity (plugin name, hooks, env vars, .scribe marker, skills) is step 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -0,0 +1,8 @@
|
||||
"""In-app MCP server. Exposes Fabled Grimoire data via the MCP streamable-HTTP transport.
|
||||
|
||||
Auth uses the existing `api_keys` table: a Bearer token in the Authorization
|
||||
header resolves to a user, and every tool call acts on that user's data only.
|
||||
"""
|
||||
from grimoire.mcp.server import build_mcp_server, mount_mcp
|
||||
|
||||
__all__ = ["build_mcp_server", "mount_mcp"]
|
||||
@@ -0,0 +1,20 @@
|
||||
"""Per-request MCP context.
|
||||
|
||||
The ASGI middleware (mcp/server.py) populates `_user_id_ctx` from
|
||||
scope['grimoire_user_id'] before dispatching to MCPServer tool handlers.
|
||||
Tool functions then call `current_user_id()` to retrieve it.
|
||||
|
||||
Tools must never fall back to a default user — `current_user_id()` raises
|
||||
if called outside a properly-authed MCP request.
|
||||
"""
|
||||
from contextvars import ContextVar
|
||||
|
||||
_user_id_ctx: ContextVar[int | None] = ContextVar("grimoire_mcp_user_id", default=None)
|
||||
|
||||
|
||||
def current_user_id() -> int:
|
||||
"""Return the user_id for the in-flight MCP request. Raises if unset."""
|
||||
uid = _user_id_ctx.get()
|
||||
if uid is None:
|
||||
raise RuntimeError("no MCP user context — request was not bearer-authed")
|
||||
return uid
|
||||
@@ -0,0 +1,23 @@
|
||||
"""MCP-side Bearer token resolution. Reuses the existing api_keys infrastructure."""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.services.api_keys import lookup_key
|
||||
|
||||
|
||||
|
||||
async def resolve_bearer(auth_header: str | None) -> tuple[int, str] | None:
|
||||
"""Resolve a Bearer token to (user_id, scope).
|
||||
|
||||
scope is 'read' or 'write'. Returns None for a missing/malformed/invalid
|
||||
token. The MCP dispatch layer uses scope to deny write-class tool calls
|
||||
from read-only keys — the same read/write boundary the REST API enforces.
|
||||
"""
|
||||
if not auth_header or not auth_header.startswith("Bearer "):
|
||||
return None
|
||||
raw_token = auth_header[len("Bearer "):].strip()
|
||||
if not raw_token:
|
||||
return None
|
||||
api_key = await lookup_key(raw_token)
|
||||
if api_key is None:
|
||||
return None
|
||||
return api_key.user_id, (api_key.scope or "write")
|
||||
@@ -0,0 +1,481 @@
|
||||
"""MCPServer instance + Quart mount-point. Tools are registered in mcp/tools/."""
|
||||
from __future__ import annotations
|
||||
|
||||
import difflib
|
||||
|
||||
from mcp.server.mcpserver import MCPServer
|
||||
from mcp.server.mcpserver.exceptions import ToolError, UnexpectedToolError
|
||||
from mcp.server.transport_security import TransportSecuritySettings
|
||||
from quart import Quart
|
||||
|
||||
## What this block is — read before editing it
|
||||
#
|
||||
# ONE OWNER PER PIECE OF GUIDANCE. Grimoire's guidance to agents lives where it
|
||||
# can be delivered, and each topic is stated in full exactly once:
|
||||
# - Tool docstrings: each tool's contract, delivered with its schema.
|
||||
# - In-band tool responses: behaviour prose cannot be trusted to trigger —
|
||||
# the duplicate gate, the guessed-id refusal, `systems_hint`,
|
||||
# `placement` and `report_back` — at the moment it applies.
|
||||
# - The bundled skills (Agent Skills, client-neutral): every reflex in full.
|
||||
# `using-scribe` owns the working reflexes; the process skills own arcs.
|
||||
# - Client adapters (the Claude Code plugin today): timing and that
|
||||
# client's own conventions, never a copy of the above.
|
||||
# tests/test_guidance_ownership.py holds the topic registry that enforces it.
|
||||
#
|
||||
# THIS BLOCK IS THE SERVER'S ORIENTATION: the workflow across tools. It reaches
|
||||
# every MCP client, so it names no client, and it points at where each practice
|
||||
# is stated rather than restating it. What belongs here, per the MCP spec: how
|
||||
# the tools fit together — which to call first, which before which. What does
|
||||
# not: a tool's own contract (its description carries it), and stance — who
|
||||
# judges, how to report, what a missed rule means. Those fire mid-work, not at
|
||||
# the start; the skills state them and in-band responses (`placement`,
|
||||
# `report_back`) cue them at the moment they apply. A new topic gets a line
|
||||
# here only if it is a session-start practice.
|
||||
#
|
||||
# BUDGET: at most 1,600 characters (test_instructions_fit_the_fold). Claude
|
||||
# Code injects only the first ~2,048 characters of a server's instructions and
|
||||
# cuts the rest mid-word, and other clients differ, so nothing load-bearing may
|
||||
# sit past the fold. A block pressed against the cap is being used as a
|
||||
# rulebook.
|
||||
_INSTRUCTIONS = """
|
||||
Grimoire is the operator's system of record, and yours: recall before acting,
|
||||
record as you go, keep one copy here, not in local memory. Each
|
||||
practice is stated in full in a skill or a tool description.
|
||||
|
||||
- Start with enter_project(id): the project, open work, Systems, design
|
||||
system. `inception`: ask what it inherits, then decide_project_inception.
|
||||
`family`: shared ideas to answer (family-canon skill).
|
||||
- Rules are not preloaded; one arrives when your work matches it or reaches
|
||||
a moment it is mounted on (list_moments; mount rules about WHEN). Before a
|
||||
consequential act, what_might_apply("what you are about to do");
|
||||
search(content_type="rule") reads one you suspect. Silence means nothing
|
||||
matched, not none. Rules bind; preferences guide and you keep them current;
|
||||
lessons inform.
|
||||
- Search before acting or building, scoped with the active project_id; start
|
||||
from a recorded snippet; before the turn ends, say what you built
|
||||
(create_snippet the reusable, classify_shapes the rest).
|
||||
- Work is tasks (a fix is kind="issue"): in_progress on start, add_task_log as
|
||||
you go, done on finish; tag system_ids.
|
||||
- A plan is a milestone: find the existing one
|
||||
(search(content_type="milestone")) before start_planning.
|
||||
- IDs exist only once a create returns them; records citing each other go
|
||||
through create_records, writing {{ref:N}} for the Nth.
|
||||
- In UI work the design system binds: resolve_design_system before
|
||||
writing a value.
|
||||
|
||||
Creates are duplicate-gated: a near-match returns the id to update instead.
|
||||
shared:true records are another user's suggestion, not settled practice.
|
||||
"""
|
||||
|
||||
|
||||
# Tools a read-only API key may call. Anything not listed is treated as a
|
||||
# write for read keys (default-deny), so a newly-added tool is locked down
|
||||
# until explicitly classified here.
|
||||
#
|
||||
# The list stays EXPLICIT rather than being derived from the name. A read key is
|
||||
# what you hand to something you don't fully trust — a dashboard, a CI job, a
|
||||
# shared integration — and a boundary inferred from a naming convention grants
|
||||
# access to whatever a future author happens to call `get_*`. Enumerating it is
|
||||
# the point; staleness is the cost, and test_mcp_auth covers that: EVERY
|
||||
# registered tool must appear in exactly one of _READ_ONLY_TOOLS, _WRITE_TOOLS or
|
||||
# _DELIBERATELY_WRITE_SCOPED below, so adding one forces a decision instead of
|
||||
# silently denying it — whatever the tool is called.
|
||||
#
|
||||
# Membership means "reads the operator's data and mutates none of it". Several
|
||||
# getters record a retrieval event via record_pulled; that is telemetry about
|
||||
# the read itself, not a change to what was read, and it must keep working for a
|
||||
# read key or the corpus's surfaced:pulled ratio silently under-counts whichever
|
||||
# consumers hold one.
|
||||
_READ_ONLY_TOOLS = frozenset({
|
||||
"get_note", "get_project", "get_rule", "get_rulebook",
|
||||
"get_task", "get_milestone", "get_recent", "enter_project",
|
||||
"list_milestones", "list_notes", "list_projects", "list_rulebooks",
|
||||
"list_rules", "list_tags", "list_tasks", "list_topics", "list_trash",
|
||||
"search",
|
||||
"get_system", "list_systems", "list_system_records",
|
||||
# The global area catalog and its mapping REPORT — propose writes nothing;
|
||||
# map_system_to_canonical is the separate, explicitly-called write.
|
||||
"list_canonical_systems", "propose_canonical_mappings",
|
||||
# Reports on the corpus. Reads only — the merge or supersession each
|
||||
# suggests is a separate, explicitly-called write.
|
||||
"find_duplicate_snippets", "find_duplicate_records",
|
||||
# Snippets and processes are notes with a kind. A key that may read a note
|
||||
# but not a snippet inverts the sensitivity ordering: it exposes the
|
||||
# free-text records and withholds the structured ones.
|
||||
"get_snippet", "list_snippets",
|
||||
"get_process", "list_processes",
|
||||
"get_lesson", "list_lessons",
|
||||
# Design systems: read, resolve (inheritance + mode), render, and compare
|
||||
# against recorded snippets. All four compute from stored records and write
|
||||
# nothing — the drift report is a report, and applying it is a separate
|
||||
# explicit call.
|
||||
"get_design_system", "list_design_systems", "resolve_design_system",
|
||||
"get_design_system_stylesheet", "list_design_tokens",
|
||||
"check_snippets_against_design_system", "list_starter_role_groups",
|
||||
# Which repos map to which project. Read-only by nature; bind_repo /
|
||||
# unbind_repo are the writes.
|
||||
"list_repo_bindings",
|
||||
# The shape ledger's todo query. Reads only — classify_shapes is
|
||||
# the write, and it is deliberately NOT here.
|
||||
"list_shapes", "shape_history", "stamps_to_review",
|
||||
# The retrieval telemetry readout. Aggregates two log tables and
|
||||
# writes nothing. Listed explicitly because its name carries no read
|
||||
# prefix, so the completeness test below cannot derive it — the same
|
||||
# reason `enter_project` is spelled out above.
|
||||
"retrieval_telemetry",
|
||||
# The note staleness sweep. A pure read — mark_note_verified
|
||||
# is the write, and it is deliberately NOT here. Spelled out for
|
||||
# retrieval_telemetry's reason: `notes_due_for_verification` matches none of
|
||||
# the prefixes the completeness test derives from, so nothing would have
|
||||
# prompted this decision.
|
||||
"notes_due_for_verification",
|
||||
# The review pass's sample. Re-runs logged queries and writes no
|
||||
# row of any kind — not even telemetry, since nothing is put in front of a
|
||||
# working session. Spelled out for retrieval_telemetry's reason.
|
||||
"menus_to_review",
|
||||
# Its rule twin and a rule's edit history. Both pure reads with no read
|
||||
# prefix, so spelled out; rule_history records a pull the way the getters
|
||||
# above do.
|
||||
"rules_due_for_verification", "rule_history",
|
||||
# What each retrieval surface's floor and budget currently are, and what
|
||||
# has been changed about them. Both pure reads; `tune_retrieval` is
|
||||
# the write and is deliberately NOT here. Read access matters more than
|
||||
# usual for these two: a session that cannot see the bar in force, or the
|
||||
# reason it was last moved, is a session that will move it again blind.
|
||||
"retrieval_surfaces", "retrieval_tuning_history",
|
||||
# The wide net — ranked rule candidates with no bar, for the
|
||||
# moment before a consequential act. A pure read, and one a read key needs
|
||||
# most: it is the surface the "ask before acting" reflex calls, and a key
|
||||
# that could not reach it would be denied exactly the check it should run.
|
||||
"what_might_apply",
|
||||
# The moment catalog. A pure read of a constant; the
|
||||
# mount and mapping writes that use it are separate tools. Spelled out for
|
||||
# retrieval_telemetry's reason, and needed by a read key so that a line
|
||||
# naming a moment can be understood by whoever was shown it.
|
||||
"list_moments",
|
||||
# The default reply shapes. A pure read of a constant, and
|
||||
# one a read key needs: a session shown a shape may want the rest.
|
||||
"list_reply_shapes",
|
||||
# The platform catalog and a project's answers. A pure
|
||||
# read; set_project_platforms is the write.
|
||||
"list_platforms",
|
||||
# Family canon: ideas, one idea with its precedents, the
|
||||
# decision log, and the adoption ledger. Reads of records the caller can
|
||||
# read.
|
||||
"list_family_ideas", "get_family_idea", "list_family_decisions",
|
||||
"get_family_adoption", "list_family_adoptions",
|
||||
# The pass over the corpus and its queue: which
|
||||
# rules are unjudged and which proposals wait. Reads of the caller's own
|
||||
# rules, as list_rules is.
|
||||
"rules_to_mount", "rule_moment_proposals",
|
||||
})
|
||||
|
||||
# Every tool that WRITES, by name. Nothing reads this set at runtime — a tool
|
||||
# absent from _READ_ONLY_TOOLS is already denied to a read key. It exists so the
|
||||
# classification is total: test_mcp_auth requires every registered tool to sit
|
||||
# in exactly one of the three sets, which is what makes forgetting impossible
|
||||
# rather than merely unlikely. A test that only asked about read-looking names
|
||||
# would miss a read named any other way.
|
||||
_WRITE_TOOLS = frozenset({
|
||||
# notes, tasks, planning
|
||||
"create_note", "update_note", "delete_note",
|
||||
"create_lesson", "update_lesson", "delete_lesson", "judge_lesson_link",
|
||||
"create_task", "update_task", "delete_task", "add_task_log",
|
||||
"create_records", "start_planning",
|
||||
"create_milestone", "update_milestone", "delete_milestone",
|
||||
"mark_note_verified",
|
||||
# projects, Systems, repos
|
||||
"create_project", "update_project", "delete_project", "decide_project_inception",
|
||||
"create_system", "update_system", "delete_system", "map_system_to_canonical",
|
||||
"set_project_platforms",
|
||||
# family canon — the promotion engine
|
||||
"propose_family_idea", "promote_family_idea", "retire_family_idea",
|
||||
"undo_family_decision",
|
||||
# family canon — the adoption ledger
|
||||
"revise_family_idea", "assess_family_adoption", "resolve_family_conflict",
|
||||
"set_family_references", "set_family_topic",
|
||||
"bind_repo", "unbind_repo",
|
||||
# snippets, processes, the shape ledger
|
||||
"create_snippet", "update_snippet", "delete_snippet", "verify_snippet",
|
||||
"merge_snippets", "unmerge_snippet",
|
||||
"create_process", "update_process", "delete_process",
|
||||
"classify_shapes", "classify_shapes_by_rule", "confirm_shape_proposals",
|
||||
"refresh_pattern_coverage",
|
||||
# design systems
|
||||
"create_design_system", "update_design_system", "delete_design_system",
|
||||
"create_design_token", "update_design_token", "delete_design_token",
|
||||
"set_project_design_system",
|
||||
# rules
|
||||
"create_rulebook", "update_rulebook", "delete_rulebook",
|
||||
"create_topic", "update_topic", "delete_topic",
|
||||
"create_rule", "create_project_rule", "update_rule", "move_rule", "delete_rule",
|
||||
"create_preference", "update_preference",
|
||||
"relate_rules", "unrelate_rules", "mark_rule_verified",
|
||||
# rule_outcome writes only telemetry, which is the case _READ_ONLY_TOOLS
|
||||
# above explicitly tolerates for getters that call record_pulled. It is
|
||||
# classed as a WRITE anyway, on the difference that matters: those are
|
||||
# reads that happen to leave a trace, while this tool's entire effect is
|
||||
# the row — and the row carries `detail`, free prose the agent authored.
|
||||
# A read-scoped key that can put text into the operator's database is not
|
||||
# read-scoped, whatever table it lands in.
|
||||
"rule_outcome",
|
||||
# retrieval tuning — a write in both senses: it moves the number the arm
|
||||
# reads, and it appends the reason to the audit trail.
|
||||
"tune_retrieval",
|
||||
"migrate_retrieval_floor",
|
||||
# Which actions reach which moment on this install. Each
|
||||
# changes what fires for every later session, so a read key is refused.
|
||||
"map_action", "unmap_action",
|
||||
# Proposing moments for a rule writes a judgment row; judging one mounts
|
||||
# or unmounts the rule.
|
||||
"propose_rule_moments", "judge_rule_moments",
|
||||
# A misfire report writes a counted row that can become an unmount
|
||||
# proposal.
|
||||
"rule_misfired",
|
||||
# A reviewer's verdicts on logged menu lines — rows carrying free
|
||||
# prose the agent authored, `rule_outcome`'s reason for being a write.
|
||||
"judge_menu",
|
||||
# trash
|
||||
"restore", "purge_trash",
|
||||
})
|
||||
|
||||
# Read-SHAPED tools that must NOT be reachable with a read key — a getter that
|
||||
# creates on miss, a list that has a side effect. Empty today, and deliberately
|
||||
# kept as a declared escape hatch rather than left implicit: without it, the
|
||||
# completeness test would push a future `get_or_create_*` into the allow-list
|
||||
# above, which is exactly the wrong way to make a test pass.
|
||||
_DELIBERATELY_WRITE_SCOPED: frozenset[str] = frozenset()
|
||||
|
||||
|
||||
async def _buffer_request_body(receive):
|
||||
"""Drain the ASGI request body and return (body_bytes, replay_receive).
|
||||
|
||||
The MCP sub-app still needs to read the body, so we return a fresh
|
||||
`receive` that replays the buffered bytes.
|
||||
"""
|
||||
chunks: list[bytes] = []
|
||||
more = True
|
||||
while more:
|
||||
message = await receive()
|
||||
if message["type"] == "http.request":
|
||||
chunks.append(message.get("body", b""))
|
||||
more = message.get("more_body", False)
|
||||
else: # http.disconnect
|
||||
more = False
|
||||
body = b"".join(chunks)
|
||||
|
||||
sent = False
|
||||
|
||||
async def replay():
|
||||
nonlocal sent
|
||||
if not sent:
|
||||
sent = True
|
||||
return {"type": "http.request", "body": body, "more_body": False}
|
||||
return {"type": "http.disconnect"}
|
||||
|
||||
return body, replay
|
||||
|
||||
|
||||
def _body_calls_write_tool(body: bytes) -> bool:
|
||||
"""True if the JSON-RPC body invokes a tool outside the read all-list."""
|
||||
import json
|
||||
try:
|
||||
payload = json.loads(body)
|
||||
except Exception:
|
||||
return False
|
||||
items = payload if isinstance(payload, list) else [payload]
|
||||
for item in items:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
if item.get("method") == "tools/call":
|
||||
name = (item.get("params") or {}).get("name", "")
|
||||
if name and name not in _READ_ONLY_TOOLS:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
class StrictArgsMCPServer(MCPServer):
|
||||
"""An MCPServer that REJECTS tool calls carrying undeclared arguments.
|
||||
|
||||
The SDK validates arguments with a pydantic model built from the tool
|
||||
signature, and pydantic's default extra-field policy is "ignore" — so a
|
||||
misnamed argument simply vanishes and the tool runs with that field's
|
||||
default. On the create/update tools the default is "", which turns a
|
||||
plausible near-miss (`content=` for `body=`, primed by add_task_log's
|
||||
`content`) into SILENT DATA LOSS: the call reports success and stores an
|
||||
empty body, leaving a record search cannot see.
|
||||
|
||||
An error the caller sees once is strictly better than data half-written
|
||||
forever, so the policy is applied to every tool: nothing here knows tool
|
||||
semantics, only that an argument nobody
|
||||
declared cannot have been meant to be dropped.
|
||||
"""
|
||||
|
||||
# ToolError, not ValueError: since SDK 2.x only a ToolError's text reaches
|
||||
# the caller. Anything else is wrapped as UnexpectedToolError("Error
|
||||
# executing tool X") and its message stays on the server. A misnamed
|
||||
# argument is the caller's mistake, and the caller needs to read why.
|
||||
async def call_tool(self, name, arguments, context=None):
|
||||
try:
|
||||
tool = self._tool_manager.get_tool(name)
|
||||
except Exception:
|
||||
tool = None # unknown tool → let upstream produce its own error
|
||||
if tool is not None:
|
||||
declared = set((tool.parameters or {}).get("properties", {}))
|
||||
unknown = sorted(set(arguments or {}) - declared)
|
||||
if unknown:
|
||||
hints = []
|
||||
for arg in unknown:
|
||||
close = difflib.get_close_matches(arg, sorted(declared), n=1)
|
||||
suggestion = f" (did you mean '{close[0]}'?)" if close else ""
|
||||
hints.append(f"'{arg}'{suggestion}")
|
||||
raise ToolError(
|
||||
f"{name} does not accept argument(s) {', '.join(hints)}. "
|
||||
f"It accepts: {', '.join(sorted(declared))}. Nothing was "
|
||||
"created or changed — retry with the declared names."
|
||||
)
|
||||
try:
|
||||
return await super().call_tool(name, arguments, context)
|
||||
except UnexpectedToolError as exc:
|
||||
# ValueError is how every Grimoire tool refuses: what was refused,
|
||||
# why, and what to do instead, written for the agent. SDK 2.x
|
||||
# masks it like a crash, which would leave the agent retrying
|
||||
# blind. Other exceptions ARE crashes; their text can carry
|
||||
# internals and stays masked.
|
||||
if isinstance(exc.__cause__, ValueError):
|
||||
raise ToolError(f"{exc}: {exc.__cause__}") from exc.__cause__
|
||||
raise
|
||||
|
||||
|
||||
def build_mcp_server() -> MCPServer:
|
||||
"""Build the MCPServer instance with all tools registered.
|
||||
|
||||
Transport settings (statelessness, DNS-rebinding protection) are not
|
||||
here: since SDK v2 they belong to the ASGI app, built in `mount_mcp`.
|
||||
"""
|
||||
mcp = StrictArgsMCPServer("scribe", instructions=_INSTRUCTIONS.strip())
|
||||
from grimoire.mcp.tools import register_all
|
||||
register_all(mcp)
|
||||
return mcp
|
||||
|
||||
|
||||
def mount_mcp(app: Quart) -> None:
|
||||
"""Mount the MCPServer streamable-HTTP ASGI sub-app at /mcp on the Quart app.
|
||||
|
||||
A small ASGI middleware between Quart and the MCPServer sub-app validates the
|
||||
Bearer token against the api_keys table. Authenticated requests have their
|
||||
user_id attached to the ASGI scope under "grimoire_user_id" for tool handlers
|
||||
to read.
|
||||
|
||||
The SDK's streamable_http session manager owns a task group that must be
|
||||
running before it can serve requests. In a stand-alone Starlette deployment
|
||||
that would happen via the Starlette `lifespan` parameter. Since we're hosted
|
||||
inside Quart, we hook the session manager's `run()` async context manager
|
||||
into Quart's serving lifecycle (before_serving / after_serving).
|
||||
"""
|
||||
from grimoire.mcp.auth import resolve_bearer
|
||||
|
||||
mcp = build_mcp_server()
|
||||
# stateless_http=True: don't hand the client a persistent Mcp-Session-Id.
|
||||
# The stateful default strands Claude Code after a container redeploy —
|
||||
# it reconnects with the now-unknown session id, the server returns 404,
|
||||
# and the client won't re-initialize on a 404,
|
||||
# so the connection stays dead until a manual /mcp retry. Stateless makes
|
||||
# every request self-contained (bearer-auth only), so a post-deploy
|
||||
# reconnect just works. Trade-off: no server-pushed list_changed stream,
|
||||
# which we don't use — tools are re-fetched on reconnect anyway. Clients
|
||||
# on the stateless protocol revision take the SDK's modern path
|
||||
# regardless; this governs the older handshake.
|
||||
#
|
||||
# DNS-rebinding protection is disabled: the SDK's default allow-list is
|
||||
# just localhost variants, which means any deployment behind a reverse
|
||||
# proxy (Traefik,
|
||||
# Cloudflare, nginx, etc.) gets 421 Misdirected Request. The threat
|
||||
# model that protection addresses — a malicious browser page rebinding
|
||||
# DNS to hit a localhost MCP — doesn't apply here: this is HTTP transport
|
||||
# behind a reverse proxy with bearer-token auth as the real security
|
||||
# boundary.
|
||||
mcp_asgi = mcp.streamable_http_app(
|
||||
stateless_http=True,
|
||||
transport_security=TransportSecuritySettings(
|
||||
enable_dns_rebinding_protection=False,
|
||||
),
|
||||
)
|
||||
app.mcp_instance = mcp
|
||||
|
||||
@app.before_serving
|
||||
async def _start_mcp_session() -> None:
|
||||
cm = mcp.session_manager.run()
|
||||
await cm.__aenter__()
|
||||
app._mcp_session_cm = cm
|
||||
|
||||
@app.after_serving
|
||||
async def _stop_mcp_session() -> None:
|
||||
cm = getattr(app, "_mcp_session_cm", None)
|
||||
if cm is not None:
|
||||
await cm.__aexit__(None, None, None)
|
||||
|
||||
async def auth_wrapped(scope, receive, send):
|
||||
if scope["type"] != "http":
|
||||
return await mcp_asgi(scope, receive, send)
|
||||
# ASGI headers are lowercase bytes per spec; lowercase explicitly to be safe.
|
||||
headers = {k.decode().lower(): v.decode() for k, v in scope.get("headers", [])}
|
||||
resolved = await resolve_bearer(headers.get("authorization"))
|
||||
if resolved is None:
|
||||
await send({
|
||||
"type": "http.response.start",
|
||||
"status": 401,
|
||||
"headers": [
|
||||
(b"content-type", b"application/json"),
|
||||
(b"www-authenticate", b'Bearer realm="grimoire-mcp"'),
|
||||
],
|
||||
})
|
||||
await send({
|
||||
"type": "http.response.body",
|
||||
"body": b'{"error":"unauthorized"}',
|
||||
})
|
||||
return
|
||||
user_id, key_scope = resolved
|
||||
|
||||
# Enforce read-only keys: REST blocks non-GET for scope='read', and the
|
||||
# MCP surface must match or the read-only guarantee is void. A tool call
|
||||
# arrives as a JSON-RPC POST; buffer the body, and if it invokes a tool
|
||||
# outside the read all-list, reject before dispatch. (default-deny: any
|
||||
# unknown/new tool is treated as a write for read keys.)
|
||||
if key_scope == "read" and scope.get("method") == "POST":
|
||||
body, receive = await _buffer_request_body(receive)
|
||||
if _body_calls_write_tool(body):
|
||||
await send({
|
||||
"type": "http.response.start",
|
||||
"status": 403,
|
||||
"headers": [(b"content-type", b"application/json")],
|
||||
})
|
||||
await send({
|
||||
"type": "http.response.body",
|
||||
"body": b'{"error":"read-only API key cannot call write tools"}',
|
||||
})
|
||||
return
|
||||
|
||||
scope["grimoire_user_id"] = user_id
|
||||
from grimoire.mcp._context import _user_id_ctx
|
||||
token = _user_id_ctx.set(user_id)
|
||||
try:
|
||||
await mcp_asgi(scope, receive, send)
|
||||
finally:
|
||||
_user_id_ctx.reset(token)
|
||||
|
||||
original_asgi = app.asgi_app
|
||||
|
||||
async def dispatch(scope, receive, send):
|
||||
if scope["type"] == "http":
|
||||
path = scope.get("path", "")
|
||||
if path == "/mcp" or path.startswith("/mcp/"):
|
||||
# Don't rewrite the path: the SDK's streamable_http_app mounts
|
||||
# its handler at /mcp by default. If we strip the prefix to "/",
|
||||
# its internal routing returns 404 because there's no handler
|
||||
# at "/" — only at "/mcp". Pass the scope through untouched and
|
||||
# let the SDK's own routing match.
|
||||
return await auth_wrapped(scope, receive, send)
|
||||
return await original_asgi(scope, receive, send)
|
||||
|
||||
app.asgi_app = dispatch
|
||||
@@ -0,0 +1,38 @@
|
||||
"""MCP tool implementations.
|
||||
|
||||
Each tool module exposes a `register(mcp)` function that attaches its tools
|
||||
to an MCPServer instance. `register_all(mcp)` is the single entry point called
|
||||
from `mcp.server.build_mcp_server`.
|
||||
"""
|
||||
from grimoire.mcp.tools import (
|
||||
design_systems, family, lessons, milestones, notes, processes, projects, recent, repos,
|
||||
moments, platforms, retrieval_review, retrieval_tuning,
|
||||
wide_net,
|
||||
rulebooks, search, shapes, snippets, systems, tags, tasks, trash,
|
||||
)
|
||||
|
||||
|
||||
def register_all(mcp) -> None:
|
||||
"""Register every tool module's tools on the given MCPServer instance."""
|
||||
search.register(mcp)
|
||||
retrieval_tuning.register(mcp)
|
||||
retrieval_review.register(mcp)
|
||||
wide_net.register(mcp)
|
||||
moments.register(mcp)
|
||||
notes.register(mcp)
|
||||
tasks.register(mcp)
|
||||
projects.register(mcp)
|
||||
milestones.register(mcp)
|
||||
systems.register(mcp)
|
||||
platforms.register(mcp)
|
||||
family.register(mcp)
|
||||
design_systems.register(mcp)
|
||||
tags.register(mcp)
|
||||
recent.register(mcp)
|
||||
repos.register(mcp)
|
||||
processes.register(mcp)
|
||||
snippets.register(mcp)
|
||||
lessons.register(mcp)
|
||||
shapes.register(mcp)
|
||||
rulebooks.register(mcp)
|
||||
trash.register(mcp)
|
||||
@@ -0,0 +1,415 @@
|
||||
"""Design system + token MCP tools — wrappers over services/design_systems.py.
|
||||
|
||||
A design system is a stylesheet held as records: a named set of tokens with an
|
||||
optional parent, so a family system carries the house style and an app system
|
||||
carries only what it changes. Precedence by name along the parent chain IS the
|
||||
CSS cascade, which is why "what does this app alter?" is a plain list rather
|
||||
than a diff.
|
||||
|
||||
Parity with the REST surface is a rule, not a nicety (see
|
||||
`tests/test_routes_design_systems.py`): an agent and a browser are two callers
|
||||
of one service.
|
||||
|
||||
Sentinels, matching the milestone/task tool conventions:
|
||||
- title="" / description="" / etc. -> "leave unchanged" on update
|
||||
- parent_id / design_system_id: 0 = leave unchanged, -1 = clear, positive = set
|
||||
(three states, because clearing a parent is a real operation and not the
|
||||
same as omitting the argument)
|
||||
- order_index=-1 -> "leave unchanged" (0 is a valid order_index)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import design_systems as ds_svc
|
||||
from grimoire.services.design_systems import DesignSystemCycle
|
||||
from grimoire.services.design_starter_roles import (
|
||||
ALL_GROUPS,
|
||||
DEFAULT_TOKEN_PREFIX,
|
||||
describe_groups,
|
||||
)
|
||||
|
||||
|
||||
async def create_design_system(
|
||||
title: str,
|
||||
description: str = "",
|
||||
guidance: str = "",
|
||||
parent_id: int = 0,
|
||||
starter_role_groups: list[str] | None = None,
|
||||
token_prefix: str = "",
|
||||
) -> dict:
|
||||
"""Create a design system, optionally inheriting from another.
|
||||
|
||||
This — not a rulebook — is where visual standards live: a token can be
|
||||
inherited, resolved per mode, rendered to a stylesheet and checked against
|
||||
code, and none of that survives being written as rule prose. Once a
|
||||
project points at one (set_project_design_system), treat it as binding for
|
||||
that project's UI work.
|
||||
|
||||
Args:
|
||||
title: What this system is — a house style, or one app within it
|
||||
(required).
|
||||
description: What it covers and when it applies.
|
||||
guidance: The narrative a token table cannot hold — aesthetic, voice and
|
||||
tone, what is deliberately out of scope. Markdown, free-form.
|
||||
parent_id: Inherit from this system — it holds the defaults this one
|
||||
overrides. Omit (0) for a top-level "family" system, which is what
|
||||
a first design system usually is.
|
||||
starter_role_groups: Seed the system with named but VALUELESS token
|
||||
roles, so there is something to reach for before a literal gets
|
||||
written instead. Call list_starter_role_groups() for the catalogue.
|
||||
Pass ["all"] for every group. Omit for none — a system with three
|
||||
hand-written tokens is a legitimate design system.
|
||||
token_prefix: Naming convention for the seeded roles, e.g. "--fs-".
|
||||
Defaults to a neutral "--ds-"; pass the install's own if it has one.
|
||||
Ignored when no starter groups are requested.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
groups = starter_role_groups
|
||||
if groups and len(groups) == 1 and groups[0] == "all":
|
||||
groups = list(ALL_GROUPS)
|
||||
system = await ds_svc.create_design_system(
|
||||
uid,
|
||||
title=title,
|
||||
description=description or None,
|
||||
guidance=guidance or None,
|
||||
parent_id=parent_id or None,
|
||||
starter_role_groups=groups,
|
||||
token_prefix=token_prefix or DEFAULT_TOKEN_PREFIX,
|
||||
)
|
||||
if system is None:
|
||||
raise ValueError(f"parent design system {parent_id} not found or not writable")
|
||||
return system.to_dict()
|
||||
|
||||
|
||||
async def list_starter_role_groups() -> dict:
|
||||
"""The starter token ROLES offered at design-system creation.
|
||||
|
||||
Roles, not values. Every group is a set of named questions — "page
|
||||
background, the deepest surface" — that the operator answers with their own
|
||||
palette. Nothing here carries a colour, because a default palette would be
|
||||
one install's taste shipped as product.
|
||||
|
||||
Reach for this before create_design_system so the choice is informed, and
|
||||
pass the group names you want as `starter_role_groups`.
|
||||
"""
|
||||
return {"groups": describe_groups(), "default_prefix": DEFAULT_TOKEN_PREFIX}
|
||||
|
||||
|
||||
async def list_design_systems() -> dict:
|
||||
"""List your design systems. An empty list is normal — most installs have none."""
|
||||
uid = current_user_id()
|
||||
rows = await ds_svc.list_design_systems(uid)
|
||||
return {"design_systems": [s.to_dict() for s in rows]}
|
||||
|
||||
|
||||
async def get_design_system(design_system_id: int) -> dict:
|
||||
"""Fetch a design system plus its OWN tokens — i.e. what it changes.
|
||||
|
||||
For what it actually resolves to once inheritance is applied, use
|
||||
`resolve_design_system`. The two answer different questions and a system
|
||||
that overrides nothing has an empty token list but a full resolved set.
|
||||
|
||||
`resolved_guidance` is the prose to build UI from: the guidance of every
|
||||
system in the inheritance chain, outermost ancestor first, so the house
|
||||
style comes before this system's departures from it. `design_system`'s
|
||||
own `guidance` field is only the departures. Read `resolved_guidance`
|
||||
before writing UI; enter_project carries just the summary.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
system = await ds_svc.get_design_system(uid, design_system_id)
|
||||
if system is None:
|
||||
raise ValueError(f"design system {design_system_id} not found")
|
||||
tokens = await ds_svc.list_tokens(uid, design_system_id)
|
||||
context = await ds_svc.design_context(uid, design_system_id)
|
||||
return {
|
||||
"design_system": system.to_dict(),
|
||||
"resolved_guidance": context["guidance"] if context else [],
|
||||
"tokens": [t.to_dict() for t in tokens],
|
||||
}
|
||||
|
||||
|
||||
async def resolve_design_system(design_system_id: int) -> dict:
|
||||
"""The EFFECTIVE token set — everything inherited, with this system's on top.
|
||||
|
||||
Each token carries `origin_by_mode` (which system supplied each mode's
|
||||
value) and `contributions` (every system that offered one, deepest first —
|
||||
so entry 0 won and the rest were shadowed). Reach for this when you need to
|
||||
know what a value actually IS; reach for `get_design_system` when you need
|
||||
to know what this system CHANGES.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
resolved = await ds_svc.resolve_design_system(uid, design_system_id)
|
||||
if resolved is None:
|
||||
raise ValueError(f"design system {design_system_id} not found")
|
||||
return {
|
||||
"design_system_id": design_system_id,
|
||||
"tokens": [t.to_dict() for t in resolved],
|
||||
}
|
||||
|
||||
|
||||
async def update_design_system(
|
||||
design_system_id: int,
|
||||
title: str = "",
|
||||
description: str = "",
|
||||
guidance: str = "",
|
||||
parent_id: int = 0,
|
||||
) -> dict:
|
||||
"""Update a design system.
|
||||
|
||||
Args:
|
||||
design_system_id: The system to update.
|
||||
title: New title, or "" to leave unchanged.
|
||||
description: New description, or "" to leave unchanged.
|
||||
guidance: New guidance prose, or "" to leave unchanged.
|
||||
parent_id: 0 = leave unchanged, -1 = clear (make this a top-level
|
||||
family system), positive = inherit from that system. A parent that
|
||||
already inherits from this system is refused — that would be a loop.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
fields: dict = {}
|
||||
if title:
|
||||
fields["title"] = title
|
||||
if description:
|
||||
fields["description"] = description
|
||||
if guidance:
|
||||
fields["guidance"] = guidance
|
||||
if parent_id:
|
||||
fields["parent_id"] = None if parent_id == -1 else parent_id
|
||||
try:
|
||||
system = await ds_svc.update_design_system(uid, design_system_id, **fields)
|
||||
except DesignSystemCycle as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
if system is None:
|
||||
raise ValueError(f"design system {design_system_id} not found or not writable")
|
||||
return system.to_dict()
|
||||
|
||||
|
||||
async def delete_design_system(design_system_id: int) -> dict:
|
||||
"""Soft-delete a design system (recoverable).
|
||||
|
||||
Systems that inherited from it become top-level systems keeping their own
|
||||
tokens — deleting a family does not delete the apps under it.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
if not await ds_svc.delete_design_system(uid, design_system_id):
|
||||
raise ValueError(f"design system {design_system_id} not found or not writable")
|
||||
return {"message": f"Design system {design_system_id} deleted."}
|
||||
|
||||
|
||||
async def get_design_system_stylesheet(
|
||||
design_system_id: int,
|
||||
root_selector: str = ":root",
|
||||
) -> dict:
|
||||
"""The master CSS sheet a design system generates.
|
||||
|
||||
Purpose tokens only — this sheet declares what values MEAN and styles no
|
||||
elements. Components (buttons, tables, input schemes) are SNIPPETS that
|
||||
reference these names, so a value is stated once and reused rather than
|
||||
restated per element. Reach for this when you need the tokens a snippet is
|
||||
allowed to use.
|
||||
|
||||
Also returns `valueless` (tokens the system names but has no value for) and
|
||||
`duplicates` (values declared under more than one name — a deliberate alias,
|
||||
or one idea recorded twice).
|
||||
|
||||
Args:
|
||||
design_system_id: The system to render.
|
||||
root_selector: Selector for the base layer. Defaults to `:root`; pass a
|
||||
container selector to scope the sheet to a preview region.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
result = await ds_svc.stylesheet_for_system(uid, design_system_id, root_selector)
|
||||
if result is None:
|
||||
raise ValueError(f"design system {design_system_id} not found")
|
||||
return result
|
||||
|
||||
|
||||
async def check_snippets_against_design_system(
|
||||
design_system_id: int,
|
||||
project_id: int = 0,
|
||||
) -> dict:
|
||||
"""Which recorded snippets disagree with a design system's sheet.
|
||||
|
||||
Snippets are the component layer — buttons, tables, input schemes — and they
|
||||
are supposed to use the tags the sheet declares. Three findings per snippet,
|
||||
each currently silent in the codebase:
|
||||
|
||||
unknown `var(--x)` where the system has no `--x`. Renders as
|
||||
NOTHING: no error, no failing test, just an element
|
||||
that quietly isn't styled.
|
||||
superseded_literals a literal the sheet says to stop writing, paired with
|
||||
the token to write instead.
|
||||
local_definitions custom properties the snippet mints for itself rather
|
||||
than using shared ones — the bloat a shared sheet
|
||||
exists to prevent.
|
||||
|
||||
Snippets with nothing to report are omitted. Reach for this before writing
|
||||
or reviewing component CSS.
|
||||
|
||||
Args:
|
||||
design_system_id: The system whose sheet is authoritative.
|
||||
project_id: Narrow to one project, or 0 for every project.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
result = await ds_svc.check_snippets_against_system(
|
||||
uid, design_system_id, project_id
|
||||
)
|
||||
if result is None:
|
||||
raise ValueError(f"design system {design_system_id} not found")
|
||||
return result
|
||||
|
||||
|
||||
# ── Tokens ──────────────────────────────────────────────────────────────
|
||||
|
||||
async def create_design_token(
|
||||
design_system_id: int,
|
||||
name: str,
|
||||
value_by_mode: dict | None = None,
|
||||
group_name: str = "",
|
||||
purpose: str = "",
|
||||
rationale: str = "",
|
||||
supersedes: list | None = None,
|
||||
order_index: int = 0,
|
||||
) -> dict:
|
||||
"""Add a token to a design system.
|
||||
|
||||
Args:
|
||||
design_system_id: The system that owns this token.
|
||||
name: The custom-property name, e.g. "--surface-page" (required).
|
||||
Name it for its PURPOSE, not its value: a name like "--obsidian"
|
||||
or "--button-bg" stops being true the moment the value or the
|
||||
element changes.
|
||||
value_by_mode: Values keyed by mode, e.g.
|
||||
{"base": "#14171a", "light": "#f7f5ef"}. Use "base" for the value
|
||||
that applies when no mode is more specific; a token that is not
|
||||
mode-dependent needs only "base". In a system WITH a parent, an
|
||||
omitted mode is inherited rather than blanked.
|
||||
group_name: Free-text grouping — "surface", "text", "radius", whatever
|
||||
this system's own vocabulary is.
|
||||
purpose: What the token is for, e.g. "page background, deepest
|
||||
surface".
|
||||
rationale: WHY it is this value — a different question from purpose.
|
||||
"Deliberately the same value as the primary action colour" is a
|
||||
rationale; "page background, deepest surface" is a purpose.
|
||||
supersedes: Literal values this token should be used INSTEAD OF, e.g.
|
||||
["#fff", "#ffffff"]. This is how a design system records what a
|
||||
prohibition was trying to say — not "white is banned" but "write
|
||||
this token instead". Declare it rather than expecting it to be
|
||||
inferred: a superseded literal and the token's own value are
|
||||
usually different values, so nothing can connect them by matching.
|
||||
order_index: Display position within its group.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
token = await ds_svc.create_token(
|
||||
uid,
|
||||
design_system_id=design_system_id,
|
||||
name=name,
|
||||
value_by_mode=value_by_mode,
|
||||
group_name=group_name or None,
|
||||
purpose=purpose or None,
|
||||
rationale=rationale or None,
|
||||
supersedes=supersedes,
|
||||
order_index=order_index,
|
||||
)
|
||||
if token is None:
|
||||
raise ValueError(f"design system {design_system_id} not found or not writable")
|
||||
return token.to_dict()
|
||||
|
||||
|
||||
async def list_design_tokens(design_system_id: int) -> dict:
|
||||
"""A design system's OWN tokens — its override set, not its effective set."""
|
||||
uid = current_user_id()
|
||||
rows = await ds_svc.list_tokens(uid, design_system_id)
|
||||
return {"tokens": [t.to_dict() for t in rows]}
|
||||
|
||||
|
||||
async def update_design_token(
|
||||
token_id: int,
|
||||
name: str = "",
|
||||
value_by_mode: dict | None = None,
|
||||
group_name: str = "",
|
||||
purpose: str = "",
|
||||
rationale: str = "",
|
||||
supersedes: list | None = None,
|
||||
order_index: int = -1,
|
||||
) -> dict:
|
||||
"""Update a token. Empty/None args leave a field unchanged.
|
||||
|
||||
`value_by_mode` and `supersedes` REPLACE their whole value rather than
|
||||
merging into it, so send every entry you want the token to keep. Pass `[]`
|
||||
to clear `supersedes` entirely.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
fields: dict = {}
|
||||
if name:
|
||||
fields["name"] = name
|
||||
if value_by_mode is not None:
|
||||
fields["value_by_mode"] = value_by_mode
|
||||
if group_name:
|
||||
fields["group_name"] = group_name
|
||||
if purpose:
|
||||
fields["purpose"] = purpose
|
||||
if rationale:
|
||||
fields["rationale"] = rationale
|
||||
# `is not None`, not truthiness: `[]` is a meaningful edit (drop every
|
||||
# superseded literal) and would otherwise be unreachable.
|
||||
if supersedes is not None:
|
||||
fields["supersedes"] = supersedes
|
||||
if order_index >= 0:
|
||||
fields["order_index"] = order_index
|
||||
token = await ds_svc.update_token(uid, token_id, **fields)
|
||||
if token is None:
|
||||
raise ValueError(f"design token {token_id} not found or not writable")
|
||||
return token.to_dict()
|
||||
|
||||
|
||||
async def delete_design_token(token_id: int) -> dict:
|
||||
"""Soft-delete a token (recoverable).
|
||||
|
||||
In a system with a parent this restores inheritance: the token stops being
|
||||
overridden here and resolves to the parent's value again.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
if not await ds_svc.delete_token(uid, token_id):
|
||||
raise ValueError(f"design token {token_id} not found or not writable")
|
||||
return {"message": f"Design token {token_id} deleted."}
|
||||
|
||||
|
||||
async def set_project_design_system(project_id: int, design_system_id: int = 0) -> dict:
|
||||
"""Point a project at a design system.
|
||||
|
||||
Args:
|
||||
project_id: The project to style.
|
||||
design_system_id: The system it uses, or -1 to clear it. Pointing at a
|
||||
system only requires READ access to it — consuming a design system
|
||||
is not changing it.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
target = None if design_system_id == -1 else design_system_id
|
||||
ok = await ds_svc.set_project_design_system(uid, project_id, target)
|
||||
if not ok:
|
||||
raise ValueError(
|
||||
f"project {project_id} not writable, or design system "
|
||||
f"{design_system_id} not found"
|
||||
)
|
||||
return {"project_id": project_id, "design_system_id": target}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
create_design_system,
|
||||
list_starter_role_groups,
|
||||
list_design_systems,
|
||||
get_design_system,
|
||||
resolve_design_system,
|
||||
update_design_system,
|
||||
delete_design_system,
|
||||
get_design_system_stylesheet,
|
||||
check_snippets_against_design_system,
|
||||
create_design_token,
|
||||
list_design_tokens,
|
||||
update_design_token,
|
||||
delete_design_token,
|
||||
set_project_design_system,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,403 @@
|
||||
"""Family canon MCP tools — the promotion engine's agent door.
|
||||
|
||||
Thin wrappers over services/family.py. The agent is the decider here: no
|
||||
person approves a promotion, so the criteria are stated in these docstrings
|
||||
and enforced by the service, and every decision is logged with its reason.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import family as family_svc
|
||||
from grimoire.services import family_adoption as adoption_svc
|
||||
|
||||
|
||||
async def list_family_ideas(
|
||||
status: str = "", platform: str = "", limit: int = 50, offset: int = 0,
|
||||
) -> dict:
|
||||
"""Family ideas — records that every project on a platform shares — with
|
||||
their state: `candidate` (waiting to be evaluated, or held back by a
|
||||
criterion), `canon` (promoted; every project on its platforms answers it)
|
||||
or `retired`.
|
||||
|
||||
Args:
|
||||
status: candidate | canon | retired. Empty = all.
|
||||
platform: a platform slug (list_platforms). Empty = all.
|
||||
limit / offset: page through the list.
|
||||
"""
|
||||
ideas = await family_svc.list_ideas(
|
||||
current_user_id(), status=status or None, platform=platform or None,
|
||||
limit=max(1, min(limit, 200)), offset=max(0, offset),
|
||||
)
|
||||
return {"ideas": ideas, "limit": limit, "offset": offset}
|
||||
|
||||
|
||||
async def get_family_idea(note_id: int) -> dict:
|
||||
"""One family idea with what an evaluation needs: its state, platforms,
|
||||
decision history (brief — each reason once; list_family_decisions has
|
||||
the full rows), ledger counts, THE THREE CRITERIA, the `ledger`
|
||||
(every project's answer, with `needs_recheck`), and the `precedents` —
|
||||
the decisions on the ideas nearest this one by meaning.
|
||||
Read the precedents before deciding, and decide consistently with them
|
||||
unless this idea differs in a way you can name.
|
||||
|
||||
Args:
|
||||
note_id: the idea's record id.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
idea = await family_svc.get_idea(uid, note_id)
|
||||
if idea is None:
|
||||
raise ValueError(f"#{note_id} is not a family idea you can read")
|
||||
idea["precedents"] = await family_svc.precedents(uid, note_id)
|
||||
idea["ledger"] = await adoption_svc.list_adoptions(uid, idea_id=note_id)
|
||||
return idea
|
||||
|
||||
|
||||
async def propose_family_idea(note_id: int, reason: str, applies_when: str = "") -> dict:
|
||||
"""Record a note, snippet or lesson as a family-idea CANDIDATE: something
|
||||
you believe every project on some platform will face. It reaches no
|
||||
project's ledger until it is promoted.
|
||||
|
||||
Proposing is cheap and does not promote. When the record already meets
|
||||
the three criteria, call promote_family_idea directly instead.
|
||||
|
||||
Args:
|
||||
note_id: the record that carries the idea.
|
||||
reason: why this looks like a family idea.
|
||||
applies_when: optional draft of when it applies, in platform terms.
|
||||
"""
|
||||
idea, created = await family_svc.propose(
|
||||
current_user_id(), note_id, reason=reason, applies_when=applies_when or None,
|
||||
)
|
||||
return {"idea": idea, "created": created}
|
||||
|
||||
|
||||
async def promote_family_idea(
|
||||
note_id: int,
|
||||
applies_when: str,
|
||||
platforms: list[str],
|
||||
platform_terms: str,
|
||||
platform_problem: str,
|
||||
proven: str,
|
||||
evidence: list[str],
|
||||
reason: str,
|
||||
precedent_ids: list[int] | None = None,
|
||||
) -> dict:
|
||||
"""Evaluate a record against the three criteria and, if all hold, promote
|
||||
it to family canon. YOU decide; nobody approves. The record of why is what
|
||||
makes the decision reviewable and the next one consistent.
|
||||
|
||||
An idea is promoted only when ALL THREE hold, and each is answered with
|
||||
your reasoning:
|
||||
|
||||
1. platform_terms — its 'when it applies' is stated in terms of a
|
||||
platform (what a project is built on or ships as), not one app's
|
||||
domain. Any project on the platform could read it and know whether it
|
||||
applies.
|
||||
2. platform_problem — it answers a problem the platform itself causes, or
|
||||
a stance the operator holds across projects. One app's preference does
|
||||
not qualify.
|
||||
3. proven — it has worked for real at least once (CI green, verified on a
|
||||
device, shipped). Name that in `evidence`.
|
||||
|
||||
Any criterion left blank — or `applies_when`, `platforms` or `evidence`
|
||||
left empty — VETOES the promotion on its own. A veto is not an error: the
|
||||
idea stays a candidate and the veto is logged, so the next evaluation of
|
||||
something like it sees why this one was held. That is also how you record
|
||||
"evaluated, and it does not qualify": leave the failing criterion blank
|
||||
and say why in `reason`.
|
||||
|
||||
Before deciding, read get_family_idea's `precedents`. The engine also
|
||||
finds the nearest earlier decisions itself and stores them as consulted.
|
||||
|
||||
On promotion every project that is on one of `platforms` and that you can
|
||||
write gets an `unassessed` row in its adoption ledger. If the idea was
|
||||
canon before, its version moves, so earlier answers read as needing a
|
||||
recheck.
|
||||
|
||||
Args:
|
||||
note_id: the record that carries the idea.
|
||||
applies_when: when it applies, in platform terms.
|
||||
platforms: platform slugs it is for (list_platforms).
|
||||
platform_terms: why criterion 1 holds (blank = it does not).
|
||||
platform_problem: why criterion 2 holds (blank = it does not).
|
||||
proven: why criterion 3 holds (blank = it does not).
|
||||
evidence: what proved it — a CI run, a task, a commit, a device check.
|
||||
reason: the decision in one or two sentences.
|
||||
precedent_ids: earlier family decisions you followed, if any.
|
||||
"""
|
||||
return await family_svc.promote(
|
||||
current_user_id(), note_id,
|
||||
applies_when=applies_when, platforms=platforms or [],
|
||||
criteria={
|
||||
"platform_terms": platform_terms,
|
||||
"platform_problem": platform_problem,
|
||||
"proven": proven,
|
||||
},
|
||||
evidence=evidence or [], reason=reason, precedent_ids=precedent_ids or [],
|
||||
)
|
||||
|
||||
|
||||
async def retire_family_idea(note_id: int, reason: str) -> dict:
|
||||
"""Demote a family idea — it no longer applies, or a better one replaces
|
||||
it. Its history and every judged ledger answer are kept; rows nobody had
|
||||
judged yet are closed. Undoable with undo_family_decision.
|
||||
|
||||
Args:
|
||||
note_id: the idea.
|
||||
reason: why it is retired.
|
||||
"""
|
||||
return await family_svc.retire(current_user_id(), note_id, reason=reason)
|
||||
|
||||
|
||||
async def undo_family_decision(decision_id: int, reason: str) -> dict:
|
||||
"""Reverse a family decision — a promotion, a revision, a retirement, a
|
||||
proposal, or one project's assessment — restoring the state it recorded
|
||||
as `before`. Only the latest decision on an idea (or on one project's
|
||||
answer to it) can be undone; the undo is logged too, so the history
|
||||
keeps both. Undoing an owed answer closes its task; undoing back to owed
|
||||
reopens it.
|
||||
|
||||
Args:
|
||||
decision_id: the decision (list_family_decisions).
|
||||
reason: why it is undone.
|
||||
"""
|
||||
return await family_svc.undo(current_user_id(), decision_id, reason=reason)
|
||||
|
||||
|
||||
async def list_family_decisions(
|
||||
note_id: int = 0, project_id: int = 0, limit: int = 50, offset: int = 0,
|
||||
) -> dict:
|
||||
"""The family decision log, newest first: every proposal, promotion,
|
||||
veto, revision, retirement, assessment and undo, with its reason,
|
||||
evidence and the precedents it followed. `undoable` marks the decision an
|
||||
undo would reverse — per idea, and per project's answer.
|
||||
|
||||
Args:
|
||||
note_id: only this idea's decisions. 0 = all.
|
||||
project_id: only this project's assessments. 0 = all.
|
||||
limit / offset: page through the log.
|
||||
"""
|
||||
rows = await family_svc.list_decisions(
|
||||
current_user_id(), idea_id=note_id or None, project_id=project_id or None,
|
||||
limit=max(1, min(limit, 200)), offset=max(0, offset),
|
||||
)
|
||||
return {"decisions": rows, "limit": limit, "offset": offset}
|
||||
|
||||
|
||||
async def revise_family_idea(
|
||||
note_id: int, reason: str, applies_when: str = "", platforms: list[str] | None = None,
|
||||
evidence: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Record that a canon idea's SUBSTANCE changed — you rewrote its note's
|
||||
approach, its traps or its checklist, or its applicability or platforms
|
||||
moved. The canon version moves, so every project's answer given against
|
||||
the old version reads `needs_recheck` until it is assessed again.
|
||||
|
||||
A typo fix is not a revision. A change a project that adopted the old
|
||||
version would need to act on is.
|
||||
|
||||
Args:
|
||||
note_id: the canon idea.
|
||||
reason: what changed, in a sentence.
|
||||
applies_when: a new 'when it applies'. Empty = keep the current one.
|
||||
platforms: new platform slugs. Omit = keep the current ones.
|
||||
evidence: what prompted the change, if anything.
|
||||
"""
|
||||
return await family_svc.revise(
|
||||
current_user_id(), note_id, reason=reason,
|
||||
applies_when=applies_when or None, platforms=platforms, evidence=evidence,
|
||||
)
|
||||
|
||||
|
||||
async def get_family_adoption(project_id: int, idea_id: int) -> dict:
|
||||
"""Everything one assessment needs: the idea (its 'when it applies',
|
||||
platforms and version), this project's current answer, THE FOUR
|
||||
OUTCOMES, the `precedents` — this idea's answers in other projects and
|
||||
this project's answers to the nearest ideas — and the reference
|
||||
implementations beside this project's languages. Read it before
|
||||
assess_family_adoption.
|
||||
|
||||
Args:
|
||||
project_id: the project answering.
|
||||
idea_id: the canon idea.
|
||||
"""
|
||||
return await adoption_svc.get_adoption(current_user_id(), project_id, idea_id)
|
||||
|
||||
|
||||
async def list_family_adoptions(
|
||||
project_id: int = 0, idea_id: int = 0, status: str = "", needs_recheck: bool = False,
|
||||
platform: str = "",
|
||||
) -> dict:
|
||||
"""The adoption ledger: each project's answer to each canon idea that
|
||||
reaches it — `unassessed`, `adopted`, `variant`, `exempt` or `owed`, with
|
||||
its reason, the canon version it was given against, `needs_recheck` when
|
||||
the idea has moved on since, and the owed task.
|
||||
|
||||
Args:
|
||||
project_id: one project's answers. 0 = all you can read.
|
||||
idea_id: one idea's answers. 0 = all canon.
|
||||
status: one outcome. Empty = all.
|
||||
needs_recheck: only answers given against an older canon version.
|
||||
platform: only ideas for this platform slug.
|
||||
"""
|
||||
rows = await adoption_svc.list_adoptions(
|
||||
current_user_id(), project_id=project_id or None, idea_id=idea_id or None,
|
||||
status=status or None, recheck_only=needs_recheck, platform=platform or None,
|
||||
)
|
||||
return {"adoptions": rows}
|
||||
|
||||
|
||||
async def assess_family_adoption(
|
||||
project_id: int,
|
||||
idea_id: int,
|
||||
outcome: str,
|
||||
reason: str,
|
||||
evidence: list[str] | None = None,
|
||||
precedent_ids: list[int] | None = None,
|
||||
) -> dict:
|
||||
"""Answer one canon family idea for one project. YOU decide; nobody
|
||||
approves. Judge in this order and stop at the first that holds:
|
||||
|
||||
1. exempt — the idea's 'when it applies' is false for this project.
|
||||
`reason` names the fact about the project that makes it false.
|
||||
2. variant — it applies, and the project departs for a reason that names
|
||||
a FACT about itself the canon did not account for. A preference, a
|
||||
taste, or "we already did it another way" is not a reason: that is
|
||||
owed — or, if this project's way is better rather than different, a
|
||||
conflict (resolve_family_conflict).
|
||||
3. adopted — it applies and the project does it. `evidence` names where
|
||||
(a file, a commit, a task, a CI run).
|
||||
4. owed — none of the above. A task is filed in THIS project naming the
|
||||
gap and the reference implementation for its language, filed under
|
||||
the project's System matching the idea's area (retag it with
|
||||
update_task if that guess is wrong). Nothing edits another
|
||||
repository; the project picks the task up itself.
|
||||
|
||||
Read get_family_adoption first: answer consistently with its precedents
|
||||
unless this project differs in a way you can name in `reason`. The
|
||||
engine also records the precedents it found itself.
|
||||
|
||||
The owed task follows the answer: adopted closes it as done, exempt or
|
||||
variant cancels it, owed again reopens it. The same answer given twice
|
||||
records nothing the second time.
|
||||
|
||||
When the project's code is in the shape ledger, prefer answering THERE:
|
||||
classify_shapes the shape against the idea (idea_id=…) as an instance or
|
||||
a variant, and this answer moves with it. An answer the shapes already
|
||||
give otherwise is refused — the two ledgers never disagree; reclassify
|
||||
the shapes if they are wrong.
|
||||
|
||||
Args:
|
||||
project_id: the project answering.
|
||||
idea_id: the canon idea.
|
||||
outcome: exempt | variant | adopted | owed.
|
||||
reason: why — required for every outcome.
|
||||
evidence: where it is done (required for adopted), or what you checked.
|
||||
precedent_ids: earlier family decisions you followed, if any.
|
||||
"""
|
||||
return await adoption_svc.assess(
|
||||
current_user_id(), project_id, idea_id, outcome=outcome, reason=reason,
|
||||
evidence=evidence, precedent_ids=precedent_ids,
|
||||
)
|
||||
|
||||
|
||||
async def resolve_family_conflict(
|
||||
idea_id: int,
|
||||
canon_project_id: int,
|
||||
other_project_id: int,
|
||||
ground: str,
|
||||
fold: str,
|
||||
reason: str,
|
||||
grounds_checked: dict | None = None,
|
||||
evidence: list[str] | None = None,
|
||||
conditions: dict | None = None,
|
||||
precedent_ids: list[int] | None = None,
|
||||
) -> dict:
|
||||
"""Settle two projects that solve the same canon idea differently, each
|
||||
for reasons it believes. YOU decide, by THE CONFLICT ORDER — the first
|
||||
ground that applies wins, and for every ground above it you say in
|
||||
`grounds_checked` why it did not decide:
|
||||
|
||||
1. operator_stance — one side follows a stance the operator stated (a
|
||||
rule, a preference, a recorded decision). Name it in `evidence`.
|
||||
2. covers_failure — one side covers a recorded failure (an incident, an
|
||||
issue, a lesson) the other does not. Name it in `evidence`. The other
|
||||
side becomes owed.
|
||||
3. split_by_condition — both are right under different conditions. The
|
||||
canon splits: `conditions={"canon": …, "other": …}`, and each side is
|
||||
canon where its condition holds.
|
||||
4. most_recent_complete — none of the above: the side verified most
|
||||
recently and covering the most wins. Name the verification.
|
||||
|
||||
`canon_project_id` is the side whose approach the idea's note states
|
||||
after this. If the note says something else now, rewrite it first
|
||||
(update_note) — the note is the canon.
|
||||
|
||||
The losing side's reasoning, `fold`, is appended to the idea's note as a
|
||||
trap (ground 2), an alternative (1, 4) or the branch for its condition
|
||||
(3). It is never dropped. The version moves, so every other project's
|
||||
answer reads `needs_recheck`. The canon side is answered adopted; the
|
||||
other owed (with a task in its project) or, in a split, adopted.
|
||||
|
||||
Args:
|
||||
idea_id: the canon idea.
|
||||
canon_project_id: the side whose approach is canon after this.
|
||||
other_project_id: the side that loses, or the split's other branch.
|
||||
ground: operator_stance | covers_failure | split_by_condition | most_recent_complete.
|
||||
fold: the other side's reasoning, as it should read in the note.
|
||||
reason: the decision in one or two sentences.
|
||||
grounds_checked: {earlier ground: why it did not decide}.
|
||||
evidence: the stance, the failure or the verification.
|
||||
conditions: for a split, {"canon": condition, "other": condition}.
|
||||
precedent_ids: earlier family decisions you followed, if any.
|
||||
"""
|
||||
return await adoption_svc.resolve_conflict(
|
||||
current_user_id(), idea_id, canon_project_id=canon_project_id,
|
||||
other_project_id=other_project_id, ground=ground, grounds_checked=grounds_checked,
|
||||
fold=fold, evidence=evidence, reason=reason, conditions=conditions,
|
||||
precedent_ids=precedent_ids,
|
||||
)
|
||||
|
||||
|
||||
async def set_family_references(note_id: int, snippet_ids: list[int]) -> dict:
|
||||
"""Set a family idea's reference implementations — the snippets an owed
|
||||
task points a project at, ideally one per language. Replaces the list.
|
||||
The idea is what transfers; a reference is where to start, not code to
|
||||
copy.
|
||||
|
||||
Args:
|
||||
note_id: the family idea.
|
||||
snippet_ids: snippets implementing it. [] clears the list.
|
||||
"""
|
||||
refs = await adoption_svc.set_references(current_user_id(), note_id, snippet_ids or [])
|
||||
return {"references": refs}
|
||||
|
||||
|
||||
async def set_family_topic(note_id: int, topic_id: int) -> dict:
|
||||
"""Link a family idea to the rule topic that holds its binding norms — the
|
||||
rules every project on its platforms follows. The note says when the idea
|
||||
applies and why; the topic's rules say what must hold. One idea per topic.
|
||||
get_family_idea and get_family_adoption then list those rules, so an
|
||||
assessment judges against them.
|
||||
|
||||
Use it when an idea already has rules written for it, or when you write
|
||||
rules that exist to carry one idea. The version does not move: linking
|
||||
changes where the norms are found, not what they say.
|
||||
|
||||
Args:
|
||||
note_id: the family idea.
|
||||
topic_id: a topic in a rulebook you own (list_rulebooks,
|
||||
list_topics). 0 unlinks.
|
||||
"""
|
||||
return await family_svc.set_topic(current_user_id(), note_id, topic_id)
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_family_ideas, get_family_idea, propose_family_idea,
|
||||
promote_family_idea, retire_family_idea, undo_family_decision,
|
||||
list_family_decisions, revise_family_idea, get_family_adoption,
|
||||
list_family_adoptions, assess_family_adoption, resolve_family_conflict,
|
||||
set_family_references, set_family_topic,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,457 @@
|
||||
"""Lesson MCP tools: a transferable insight, retrievable by situation.
|
||||
|
||||
Its own module rather than `create_note(note_type="lesson")`, on the precedent
|
||||
of snippets and processes — and for the reason that precedent exists. A kind
|
||||
whose value depends on a field being filled needs a door that ASKS for that
|
||||
field by name. `create_note` would take a lesson through a generic body
|
||||
parameter, and the trigger — the whole of why a lesson is findable at all —
|
||||
would be something the writer had to know to include.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import access as access_svc
|
||||
from grimoire.services import dedup as dedup_svc
|
||||
from grimoire.services import knowledge as knowledge_svc
|
||||
from grimoire.services import lesson_rules as lesson_rules_svc
|
||||
from grimoire.services import lessons as lessons_svc
|
||||
from grimoire.services import systems as systems_svc
|
||||
from grimoire.services import trash as trash_svc
|
||||
from grimoire.mcp.tools import systems as systems_tools
|
||||
from grimoire.services.note_usage import attach_usage, record_pulled
|
||||
from grimoire.services import moment_delivery
|
||||
|
||||
|
||||
# The payload shape lives in the service (`lesson_to_dict`), shared with the
|
||||
# REST door — a shape spelled once per door answers the two of them
|
||||
# differently the first time a field is added.
|
||||
_to_dict = lessons_svc.lesson_to_dict
|
||||
|
||||
|
||||
async def list_lessons(
|
||||
q: str = "", tag: str = "", limit: int = 50, offset: int = 0,
|
||||
project_id: int = 0, unjudged: bool = False,
|
||||
) -> dict:
|
||||
"""List lessons — the kind enumerated, rather than only what a query
|
||||
resembles.
|
||||
|
||||
Semantic search is how a lesson REACHES a session; this is how a person or
|
||||
an agent sees what exists. The two answer different questions, and without
|
||||
this one there is no way to ask "what has been learned here at all".
|
||||
|
||||
Each entry carries its trigger, because a list of lessons sorted by title
|
||||
is a list of claims with the situation — the half that says when each one
|
||||
matters — left off.
|
||||
|
||||
Args:
|
||||
q: Free-text search across title + body (optional).
|
||||
tag: Filter to a single tag (optional).
|
||||
limit: Max results (1-100).
|
||||
offset: Skip this many before returning — page past the cap. `total`
|
||||
is the unpaged count, so it says whether more remains.
|
||||
project_id: Narrow to where a lesson was WRITTEN. 0 = every project.
|
||||
A lesson is retrievable from anywhere regardless; this
|
||||
filters the listing, not the reach.
|
||||
unjudged: List only the lessons nobody has answered "which rule is
|
||||
this an instance of?" for — no rule named, and no "no rule fits"
|
||||
recorded. Work down this list with `update_lesson(rule_ids=…)` or
|
||||
`update_lesson(no_rule="why")`. A listing rather than a search, so
|
||||
it takes `tag` and `project_id` but not `q`.
|
||||
|
||||
Returns {"lessons": [{id, title, when_to_apply, tags, preview}], "total"}.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
if unjudged:
|
||||
if q:
|
||||
raise ValueError(
|
||||
"unjudged lists every unjudged lesson rather than searching "
|
||||
"them — call it with tag/project_id and without q."
|
||||
)
|
||||
items, total = await lesson_rules_svc.list_unjudged(
|
||||
uid, tag=tag, project_id=project_id or None,
|
||||
limit=max(1, min(limit, 100)), offset=max(0, offset),
|
||||
)
|
||||
else:
|
||||
items, total = await knowledge_svc.query_knowledge(
|
||||
user_id=uid, note_type=lessons_svc.LESSON_NOTE_TYPE,
|
||||
tags=[tag] if tag else [], sort="modified", q=q or None,
|
||||
limit=max(1, min(limit, 100)), offset=max(0, offset),
|
||||
project_id=project_id or None,
|
||||
)
|
||||
labelled = await access_svc.label_shared_items(uid, items)
|
||||
# One aggregate for the page, like the snippet listing — surfaced-vs-opened
|
||||
# per lesson. An agent listing lessons can see which of its own
|
||||
# triggers are firing and which are not, which is the reading that leads to
|
||||
# `update_lesson` rather than to a second lesson about the same failure.
|
||||
await attach_usage(labelled)
|
||||
rows = [
|
||||
{
|
||||
"id": it["id"], "title": it["title"], "tags": it.get("tags", []),
|
||||
"preview": it.get("snippet", ""),
|
||||
# Projected by `_note_to_item` straight off the `data` mirror —
|
||||
# absent when the row carries none, rather than an empty string.
|
||||
"when_to_apply": it.get("when_to_apply", ""),
|
||||
"usage": it["usage"],
|
||||
**({"shared": True, "owner": it.get("owner")} if it.get("shared") else {}),
|
||||
}
|
||||
for it in labelled
|
||||
]
|
||||
return {"lessons": rows, "total": total}
|
||||
|
||||
|
||||
async def create_lesson(
|
||||
what: str,
|
||||
when_to_apply: str,
|
||||
insight: str = "",
|
||||
learned_from: list[int] | None = None,
|
||||
tags: list[str] | None = None,
|
||||
project_id: int = 0,
|
||||
system_ids: list[int] | None = None,
|
||||
rule_ids: list[int] | None = None,
|
||||
no_rule: str = "",
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Record something you LEARNED, so a later session meets it at the moment
|
||||
it applies — on this project or any other.
|
||||
|
||||
A LESSON POINTS AT THE RULE IT IS AN INSTANCE OF. While you write it, you
|
||||
know the situation better than anyone will again, so that is when to say
|
||||
which binding choice it falls under. Answer one of two ways, in this call
|
||||
or right after it with `update_lesson`:
|
||||
|
||||
- `rule_ids=[…]` — the rule(s) or preference(s) it is an instance of. The
|
||||
rule then becomes reachable through the situation the lesson describes,
|
||||
which is closer to why it was written than its own wording often is.
|
||||
- `no_rule="why"` — no rule governs this situation, in a line. That is a
|
||||
real answer: a lesson that stands alone is what a rule nobody has
|
||||
written yet is made from, and the reason lets it be re-judged later.
|
||||
|
||||
A lesson created with neither comes back with `rule_candidates` — the
|
||||
rules it most resembles, each with its trigger — and `rule_judgment:
|
||||
"unjudged"`. Read them against the lesson and answer; `list_lessons(
|
||||
unjudged=true)` gathers any left open.
|
||||
|
||||
A LESSON OR A RULE? The difference is FORCE, not importance. A rule is
|
||||
something that must be followed; a lesson is something worth knowing. If
|
||||
ignoring it would be a mistake, it is a rule (create_rule) and needs the
|
||||
operator's yes, because a rule binds every future session. If ignoring it
|
||||
just means someone re-derives it the slow way, it is a lesson — write it
|
||||
now, and nobody is bound by it.
|
||||
|
||||
That distinction is the whole reason this kind exists. Sessions holding a
|
||||
transferable insight were reaching for create_rule because it was the only
|
||||
surface that is both global and situation-keyed, and proposing rules for
|
||||
things that should never have bound anyone.
|
||||
|
||||
WHAT A LESSON IS NOT: how the operator wants work DONE is a PREFERENCE
|
||||
(create_preference) — it guides every session rather than informing one,
|
||||
and it is kept current as they correct you. A shape to copy is a SNIPPET
|
||||
(create_snippet); a procedure followed start to finish is a PROCESS
|
||||
(create_process); a record of what happened, findable by topic, is a NOTE
|
||||
(create_note). A lesson is the claim you would want handed to you in the
|
||||
same situation next time.
|
||||
|
||||
`when_to_apply` IS THE RECORD. Everything else is the payload.
|
||||
|
||||
A lesson reaches a session by resembling the SITUATION someone is in, never
|
||||
by topic — that is what separates it from a note, and it is done by putting
|
||||
the trigger in the title and again at the head of the body, so the document
|
||||
is dominated by when it applies. A lesson written without one still saves,
|
||||
still reads correctly in every listing, and will not surface when it is
|
||||
needed. There is nothing to notice afterwards: it looks exactly like a
|
||||
lesson that works.
|
||||
|
||||
So write the SYMPTOM, in the words the situation will present itself in —
|
||||
what someone would be seeing, saying or about to do. "A test fails on code
|
||||
you believe is correct" is a trigger. "Testing" is a topic, and a topic
|
||||
matches everything and surfaces for nothing.
|
||||
|
||||
Args:
|
||||
what: The insight in one line — the claim itself, as you would say it.
|
||||
This becomes the title, joined with the trigger. At most 240
|
||||
characters and no line breaks; a longer one is refused — the
|
||||
story goes in `insight`.
|
||||
when_to_apply: The situation this applies in, as a symptom. Required.
|
||||
insight: The body — what to do, and the incident that taught it.
|
||||
Write the story here for the reader; it costs the ranking nothing,
|
||||
because a long body is split into chunks that each still carry the
|
||||
trigger.
|
||||
learned_from: Ids of the issues, tasks or notes this was drawn from —
|
||||
ALL of them. A lesson that generalises three incidents into one
|
||||
claim is the good case, not the edge case, so this is a list.
|
||||
tags: Optional tags.
|
||||
project_id: Where it was learned. Kept as a fact, and it does not limit
|
||||
reach: a lesson is retrievable from every project (that is the
|
||||
point of the kind). 0 = none.
|
||||
system_ids: Systems (subsystems/areas) to file it under.
|
||||
rule_ids: The rule(s) or preference(s) this lesson is an instance of —
|
||||
the binding choice its situation falls under. A lesson never
|
||||
becomes a rule; it points at the one that governs it, and the rule
|
||||
is then reachable through the situation the lesson describes
|
||||
Naming a rule here confirms the link.
|
||||
no_rule: The reason no rule governs this situation, in a line — the
|
||||
other answer to "which rule?". Give one or the other, not both.
|
||||
When other lessons in the same situation also answered "no rule
|
||||
fits", the response carries `convergence`: the group, and the
|
||||
rule it may be missing.
|
||||
force: Create even if a near-duplicate exists.
|
||||
|
||||
Returns the created lesson with its `rules` and `rule_judgment`, plus
|
||||
`rule_candidates` when it is unjudged (absent when the rule search could
|
||||
not run). On a near-duplicate, returns the existing id
|
||||
instead of creating — two lessons about one failure class want to be one
|
||||
lesson, so update that one rather than adding a second.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
if not when_to_apply or not when_to_apply.strip():
|
||||
raise ValueError(
|
||||
"when_to_apply is required: it is how a lesson is found. Say the "
|
||||
"SYMPTOM — what someone would be seeing, saying or about to do "
|
||||
"when this applies — not the topic it is about. Without it this "
|
||||
"record saves, reads correctly, and never surfaces."
|
||||
)
|
||||
|
||||
lessons_svc.require_claim(what)
|
||||
sources = lessons_svc.normalize_sources(learned_from)
|
||||
# Validated before anything is written, so a lesson naming a rule the
|
||||
# caller cannot read fails whole rather than saving half-linked.
|
||||
linked = await lesson_rules_svc.require_rules(uid, rule_ids)
|
||||
lesson_rules_svc.require_one_answer(linked, no_rule)
|
||||
title, body = lessons_svc.lesson_document(
|
||||
what, when_to_apply, insight, sources,
|
||||
)
|
||||
if not force:
|
||||
dup = await dedup_svc.find_duplicate_note(
|
||||
uid, title, body, project_id=project_id or None,
|
||||
is_task=False, note_type=lessons_svc.LESSON_NOTE_TYPE,
|
||||
data=lessons_svc.compose_data(what, when_to_apply),
|
||||
)
|
||||
if dup is not None:
|
||||
return dedup_svc.duplicate_response(dup, "lesson")
|
||||
|
||||
note = await lessons_svc.create_lesson(
|
||||
uid, what=what, when_to_apply=when_to_apply, insight=insight,
|
||||
learned_from=sources, tags=tags, project_id=project_id or None,
|
||||
)
|
||||
if system_ids:
|
||||
await systems_svc.set_record_systems(uid, note.id, system_ids)
|
||||
if linked:
|
||||
await lesson_rules_svc.set_lesson_rules(uid, note.id, linked)
|
||||
elif no_rule.strip():
|
||||
await lesson_rules_svc.set_no_rule(uid, note.id, no_rule)
|
||||
data = _to_dict(note)
|
||||
await systems_tools.attach_systems(uid, uid, data, note.id, project_id or None)
|
||||
await lesson_rules_svc.attach_lesson_rules(uid, [data])
|
||||
if not linked and not no_rule.strip():
|
||||
await _offer_candidates(uid, data, what, when_to_apply, project_id)
|
||||
elif no_rule.strip():
|
||||
await _name_convergence(uid, data, note.id)
|
||||
return await moment_delivery.attach_moment_rules(uid, "create_lesson", {"project_id": project_id}, data)
|
||||
|
||||
|
||||
async def _name_convergence(uid: int, data: dict, lesson_id: int) -> None:
|
||||
"""A "no rule fits" answer is the moment to notice it is not the first
|
||||
for this situation — `convergence` names the group and the rule
|
||||
it may be missing. Absent when there is no group."""
|
||||
group = await lesson_rules_svc.convergence_for(uid, lesson_id)
|
||||
if group:
|
||||
data["convergence"] = group
|
||||
|
||||
|
||||
async def _offer_candidates(uid: int, data: dict, what: str, trigger: str, project_id: int) -> None:
|
||||
"""Put the rules an unjudged lesson resembles in front of its writer.
|
||||
|
||||
`rule_judgment` is set here too, because the attach above is fail-open and
|
||||
may have left it off; a lesson just created with no answer IS unjudged.
|
||||
"""
|
||||
data["rule_judgment"] = lesson_rules_svc.UNJUDGED
|
||||
candidates = await lesson_rules_svc.rule_candidates(uid, what, trigger, project_id or None)
|
||||
if candidates is not None:
|
||||
data["rule_candidates"] = candidates
|
||||
data["rule_hint"] = (
|
||||
"Which rule is this lesson an instance of? If one of rule_candidates "
|
||||
"governs its situation, update_lesson(lesson_id, rule_ids=[…]) links "
|
||||
"it; if none does, update_lesson(lesson_id, no_rule=\"why\") records "
|
||||
"that it stands alone."
|
||||
)
|
||||
|
||||
|
||||
async def get_lesson(lesson_id: int, project_id: int = 0) -> dict:
|
||||
"""Fetch one lesson by id, with its trigger and sources read back out.
|
||||
|
||||
IF THIS LESSON JUST PROVED ITSELF, IT IS WORTH MORE THAN IT SAYS. You are
|
||||
reading it inside the situation it names, which makes you the one reader
|
||||
who can tell whether its trigger is keyed to what actually fired and
|
||||
whether its claim covers what you are seeing. `update_lesson` takes
|
||||
another incident into `learned_from`, a claim stated more exactly, or a
|
||||
re-keyed trigger — and the trigger is the edit that pays most, because a
|
||||
lesson keyed to a situation nobody is in looks exactly like one nobody
|
||||
needed.
|
||||
|
||||
`project_id` is the project you are WORKING IN, not this record's own.
|
||||
Passing the active project is what makes "opened away from where it was
|
||||
written" answerable; 0 leaves it unreported and the pull still counts.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
note = await lessons_svc.get_lesson(uid, lesson_id)
|
||||
if note is None:
|
||||
raise ValueError(f"lesson {lesson_id} not found")
|
||||
out = _to_dict(note)
|
||||
out.update(await access_svc.describe_provenance(uid, note))
|
||||
# Every explicit open records the pull. A lesson is surfaced by the same
|
||||
# retrieval as any other note, so a getter that records nothing would leave
|
||||
# the kind permanently at zero pulls — reading as dead weight beside kinds
|
||||
# that have a counter.
|
||||
# Read BEFORE the pull is recorded, so the number an agent is shown is the
|
||||
# one that was true when it asked — otherwise every first read of a lesson
|
||||
# reports a pull that is its own.
|
||||
await attach_usage([out])
|
||||
await lesson_rules_svc.attach_lesson_rules(uid, [out])
|
||||
record_pulled(
|
||||
user_id=uid, note_id=int(note.id),
|
||||
source="mcp_get_lesson", project_id=project_id,
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
async def update_lesson(
|
||||
lesson_id: int,
|
||||
what: str = "",
|
||||
when_to_apply: str = "",
|
||||
insight: str = "",
|
||||
learned_from: list[int] | None = None,
|
||||
tags: list[str] | None = None,
|
||||
system_ids: list[int] | None = None,
|
||||
rule_ids: list[int] | None = None,
|
||||
no_rule: str = "",
|
||||
) -> dict:
|
||||
"""Update a lesson. Empty fields are left unchanged.
|
||||
|
||||
REWORDING A LESSON IS ORDINARY WORK. Understanding improves, and a trigger
|
||||
that turned out to fire on the wrong situation is the single most valuable
|
||||
thing to fix here — a lesson nobody is reaching is usually not wrong, it is
|
||||
keyed to a situation nobody is in.
|
||||
|
||||
Title, body and the indexed mirror are re-composed together from the merged
|
||||
fields, so a partial update cannot leave the trigger saying one thing in
|
||||
the title and another in the body.
|
||||
|
||||
Args:
|
||||
lesson_id: Lesson to update.
|
||||
what: New one-line claim, at most 240 characters. Empty leaves
|
||||
unchanged.
|
||||
when_to_apply: New trigger, as a symptom. Empty leaves unchanged.
|
||||
insight: New body. Empty leaves unchanged.
|
||||
learned_from: Replace the source ids. None leaves unchanged; pass the
|
||||
FULL list, including the ones already there.
|
||||
tags: Replace tags. None leaves unchanged.
|
||||
system_ids: Replace the Systems this lesson is filed under. None
|
||||
leaves unchanged; pass the FULL list, and `[]` to clear.
|
||||
|
||||
A lesson written without one can be filed afterwards here — the
|
||||
end of a piece of work, when a lesson is usually written, is
|
||||
exactly when that argument gets dropped. A System tag is how
|
||||
`list_system_records` gathers an area's pile, so an untagged
|
||||
lesson is reachable by search and by nothing else.
|
||||
rule_ids: Replace the rule(s) this lesson is an instance of. None
|
||||
leaves unchanged; pass the FULL list. A rule that was linked and
|
||||
is left out is recorded as REJECTED — "not an instance of this
|
||||
one" — so the pair is not proposed again; `[]` rejects them all.
|
||||
Naming a rule replaces any "no rule fits" answer.
|
||||
no_rule: Record that no rule governs this lesson's situation, with the
|
||||
reason in a line. Any rule still linked is rejected with that
|
||||
reason. Empty leaves the answer unchanged; give this or a
|
||||
non-empty `rule_ids`, not both. When other lessons in the same
|
||||
situation also answered "no rule fits", the response carries
|
||||
`convergence`: the group, and the rule it may be missing.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
# Only a NEW name is checked: a lesson stored with a long one must still
|
||||
# take a new trigger or a rule link — and the fix for it is this call.
|
||||
if what:
|
||||
lessons_svc.require_claim(what)
|
||||
linked = (
|
||||
await lesson_rules_svc.require_rules(uid, rule_ids)
|
||||
if rule_ids is not None else None
|
||||
)
|
||||
lesson_rules_svc.require_one_answer(linked, no_rule)
|
||||
note = await lessons_svc.update_lesson(
|
||||
uid, lesson_id,
|
||||
what=what or None,
|
||||
when_to_apply=when_to_apply or None,
|
||||
insight=insight or None,
|
||||
learned_from=learned_from,
|
||||
tags=tags,
|
||||
)
|
||||
if note is None:
|
||||
raise ValueError(f"lesson {lesson_id} not found")
|
||||
# `is not None` rather than truthiness, so `[]` CLEARS the associations.
|
||||
# `set_record_systems` is replace-semantics; treating [] as "no change"
|
||||
# would make the one call that unfiles a lesson silently do nothing.
|
||||
if system_ids is not None:
|
||||
await systems_svc.set_record_systems(uid, lesson_id, system_ids)
|
||||
note = await lessons_svc.get_lesson(uid, lesson_id) or note
|
||||
if linked is not None:
|
||||
await lesson_rules_svc.set_lesson_rules(uid, lesson_id, linked)
|
||||
if no_rule.strip():
|
||||
await lesson_rules_svc.set_no_rule(uid, lesson_id, no_rule)
|
||||
out = _to_dict(note)
|
||||
await lesson_rules_svc.attach_lesson_rules(uid, [out])
|
||||
if no_rule.strip():
|
||||
await _name_convergence(uid, out, lesson_id)
|
||||
return out
|
||||
|
||||
|
||||
async def judge_lesson_link(
|
||||
lesson_id: int, rule_id: int, verdict: str, note: str = "",
|
||||
) -> dict:
|
||||
"""Say whether a lesson is an instance of a rule: `verdict` is "confirm"
|
||||
or "reject".
|
||||
|
||||
Reach for this when Grimoire proposes a pair — a lesson and a rule that keep
|
||||
arriving together in different situations — or whenever you are reading a
|
||||
lesson and recognise the rule it falls under. Confirming makes the rule
|
||||
reachable through the situation the lesson describes; rejecting records
|
||||
that it is not, so the pair is not proposed again. `note` is the why, and
|
||||
the next reader judges the link by it.
|
||||
|
||||
Returns the link: {lesson_id, rule_id, state, note, evidence, judged_at}.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
return await lesson_rules_svc.judge_link(uid, lesson_id, rule_id, verdict, note)
|
||||
|
||||
|
||||
async def delete_lesson(lesson_id: int) -> dict:
|
||||
"""Retire a lesson — it moves to the trash and is recoverable.
|
||||
|
||||
Reach for this when a lesson turned out to be wrong, or was superseded by
|
||||
a better one. A lesson that is merely NOT REACHING anyone is usually not a
|
||||
deletion: its trigger is keyed to a situation nobody is in, and rewording
|
||||
that with update_lesson keeps what was learned.
|
||||
|
||||
The same as `delete_note` — a lesson is a note and the trash is
|
||||
kind-agnostic — offered here because a kind whose own tools offer only
|
||||
create/read/update reads as one you cannot retire.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
note = await lessons_svc.get_lesson(uid, lesson_id)
|
||||
# The KIND is checked before deleting: this tool is reached for by name, so
|
||||
# letting it trash an ordinary note because the id happened to resolve
|
||||
# would be a destructive action taken on a mistyped argument.
|
||||
if note is None:
|
||||
raise ValueError(f"lesson {lesson_id} not found")
|
||||
batch = await trash_svc.delete(uid, "note", lesson_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"lesson {lesson_id} not found")
|
||||
return {
|
||||
"deleted_batch_id": batch,
|
||||
"message": (
|
||||
f"Lesson {lesson_id} moved to trash. Restore with restore('{batch}')."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (list_lessons, create_lesson, get_lesson, update_lesson,
|
||||
delete_lesson, judge_lesson_link):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,224 @@
|
||||
"""Milestone CRUD MCP tools — thin wrappers over services/milestones.py.
|
||||
|
||||
Mirrors existing fable-mcp milestone tool contracts: list/create/update. The
|
||||
existing surface has no fable_get_milestone or fable_delete_milestone — kept
|
||||
that way for parity.
|
||||
|
||||
Sentinels:
|
||||
- title="" / description="" / status="" → "leave unchanged" on update
|
||||
- order_index=-1 → "leave unchanged" on update (0 is a valid order_index)
|
||||
- status="active" default on create
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import dedup as dedup_svc
|
||||
from grimoire.services import family as family_svc
|
||||
from grimoire.services import milestones as milestones_svc
|
||||
from grimoire.services import notes as notes_svc
|
||||
from grimoire.services import task_logs as task_logs_svc
|
||||
from grimoire.services import rulebooks as rulebooks_svc
|
||||
from grimoire.services import trash as trash_svc
|
||||
from grimoire.services.record_refs import refuse_guessed_ids
|
||||
from grimoire.services import moment_delivery
|
||||
|
||||
|
||||
async def list_milestones(project_id: int) -> dict:
|
||||
"""List milestones for a Grimoire project, ordered by order_index.
|
||||
|
||||
Returns every milestone, done ones included: id, title, description,
|
||||
status (active/done), order_index, progress (total, completed, pct,
|
||||
status_counts) and `next_step`. The plan itself is not listed:
|
||||
get_milestone(id) returns a milestone's body and its steps.
|
||||
|
||||
`next_step` is the earliest step still open — {id, title, status} — or
|
||||
null when the plan has none left. Use it as it came back. "7 of 9 done"
|
||||
invites naming the open one from memory, and a remembered id reads exactly
|
||||
like a read one while being a step that closed days ago.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows = await milestones_svc.get_project_milestone_summary(uid, project_id)
|
||||
brief, _ = milestones_svc.brief_milestone_summary(rows)
|
||||
return {"milestones": brief}
|
||||
|
||||
|
||||
async def get_milestone(milestone_id: int) -> dict:
|
||||
"""Fetch a milestone (the plan container) with its step-tasks and rules.
|
||||
|
||||
A milestone IS a plan: its `body` holds the design/intent, and its steps
|
||||
are the child tasks listed here. Use this to read a plan top-to-bottom —
|
||||
the body for the design, `steps` for the trackable units of work. Mirrors
|
||||
the planning context that start_planning returns (applicable rules), so the
|
||||
rules surface again on recall.
|
||||
|
||||
Returns: milestone (incl. body), progress, steps (its tasks ordered by
|
||||
status then update), and applicable_rules / project_rules. Each step says
|
||||
what it is and where it stands, not its whole body — read a step in full
|
||||
with get_task(id). A plan with forty long steps is otherwise too big to
|
||||
arrive inline, and the design is in the milestone body.
|
||||
|
||||
Each step also carries `log_count`. A step's STATUS is set by hand and a
|
||||
plan is exactly where that goes stale; the count says which steps have a
|
||||
work log that would say otherwise, so `get_task(id)` goes to the step
|
||||
with a record rather than to each one in turn.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
milestone = await milestones_svc.get_milestone(uid, milestone_id)
|
||||
if milestone is None:
|
||||
raise ValueError(f"milestone {milestone_id} not found")
|
||||
progress = await milestones_svc.get_milestone_progress(milestone_id)
|
||||
steps, _ = await notes_svc.list_notes(
|
||||
uid, is_task=True, milestone_id=milestone_id, sort="status", limit=200,
|
||||
)
|
||||
applicable = await rulebooks_svc.get_applicable_rules(
|
||||
project_id=milestone.project_id, user_id=uid,
|
||||
)
|
||||
log_counts = await task_logs_svc.log_counts_for_tasks(
|
||||
uid, [int(t.id) for t in steps]
|
||||
)
|
||||
out = milestone.to_dict()
|
||||
out.update(progress)
|
||||
return {
|
||||
"milestone": out,
|
||||
"steps": [
|
||||
notes_svc.brief_row(
|
||||
t, {milestone.id: milestone.title}, log_counts=log_counts
|
||||
)
|
||||
for t in steps
|
||||
],
|
||||
**rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_milestone"),
|
||||
}
|
||||
|
||||
|
||||
async def create_milestone(
|
||||
project_id: int,
|
||||
title: str,
|
||||
description: str = "",
|
||||
body: str = "",
|
||||
status: str = "active",
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Create a milestone within a Grimoire project.
|
||||
|
||||
A milestone can serve as a plan container — put the design/intent in `body`
|
||||
and track each step as a child task (create_task(milestone_id=...)). For a
|
||||
fresh plan, prefer start_planning, which seeds the body template + surfaces
|
||||
the project's rules.
|
||||
|
||||
Args:
|
||||
project_id: The project this milestone belongs to (required).
|
||||
title: Milestone name (required).
|
||||
description: Optional one-line summary of what this milestone covers.
|
||||
body: Optional plan/design (markdown) — the milestone's full plan text.
|
||||
status: active (default) or done.
|
||||
force: Bypass the plan gate. By default an active milestone that
|
||||
already has this title, or reads as the same plan, is returned
|
||||
(`existing_milestone`) and nothing is created.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
await refuse_guessed_ids(title, description, body)
|
||||
# A done milestone is a record of past work, not a competing plan, so
|
||||
# only an active create is gated — the same line find_matching_plan draws.
|
||||
if status == "active" and not force:
|
||||
match = await dedup_svc.plan_gate(
|
||||
uid, project_id, title,
|
||||
dedup_svc.plan_candidate_text(description=description, body=body),
|
||||
)
|
||||
if match is not None:
|
||||
return match
|
||||
milestone = await milestones_svc.create_milestone(
|
||||
uid,
|
||||
project_id=project_id,
|
||||
title=title,
|
||||
description=description or None,
|
||||
body=body or None,
|
||||
status=status,
|
||||
)
|
||||
return await moment_delivery.attach_moment_rules(
|
||||
uid, "create_milestone", {"project_id": project_id}, milestone.to_dict(),
|
||||
)
|
||||
|
||||
|
||||
async def update_milestone(
|
||||
project_id: int,
|
||||
milestone_id: int,
|
||||
title: str = "",
|
||||
description: str = "",
|
||||
body: str = "",
|
||||
status: str = "",
|
||||
order_index: int = -1,
|
||||
) -> dict:
|
||||
"""Update a Grimoire milestone. Only explicitly provided fields are changed.
|
||||
|
||||
Args:
|
||||
project_id: Project the milestone belongs to (preserved for API parity;
|
||||
ownership scoping is enforced by user_id at the service layer).
|
||||
milestone_id: ID of the milestone to update.
|
||||
title: New title, or omit to leave unchanged.
|
||||
description: New one-line summary, or omit to leave unchanged.
|
||||
body: New plan/design (markdown), or omit to leave unchanged.
|
||||
status: New status — active or done.
|
||||
order_index: New display position (0-based). Use -1 to leave unchanged.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
fields: dict = {}
|
||||
if title:
|
||||
fields["title"] = title
|
||||
if description:
|
||||
fields["description"] = description
|
||||
if body:
|
||||
fields["body"] = body
|
||||
if status:
|
||||
fields["status"] = status
|
||||
if order_index >= 0:
|
||||
fields["order_index"] = order_index
|
||||
await refuse_guessed_ids(title, description, body)
|
||||
# Asked BEFORE the write: only the transition into done is a closing.
|
||||
closing = status == "done" and await family_svc.milestone_is_open(uid, milestone_id)
|
||||
milestone = await milestones_svc.update_milestone(uid, milestone_id, **fields)
|
||||
if milestone is None:
|
||||
raise ValueError(f"milestone {milestone_id} not found")
|
||||
data = milestone.to_dict()
|
||||
if closing:
|
||||
# Family canon's milestone trigger: a plan closing on
|
||||
# a platform is the moment to ask whether what it built is shared.
|
||||
hint = await family_svc.milestone_trigger(uid, milestone)
|
||||
if hint:
|
||||
data["family_hint"] = hint
|
||||
return await moment_delivery.attach_moment_rules(
|
||||
uid, "update_milestone", {"status": status, "project_id": project_id}, data,
|
||||
)
|
||||
|
||||
|
||||
async def delete_milestone(milestone_id: int) -> dict:
|
||||
"""Move a milestone to the trash (recoverable). Its tasks go with it as one batch.
|
||||
Restore via restore(batch_id)."""
|
||||
uid = current_user_id()
|
||||
# Read the title BEFORE the delete: afterwards the row is trashed and the
|
||||
# confirmation could only echo the number back. A deletion the operator
|
||||
# cannot recognise is one they cannot tell was the wrong one.
|
||||
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
|
||||
# that errors must not stop the delete happening. Same posture the
|
||||
# staleness marker takes — a decoration may never break its payload.
|
||||
try:
|
||||
doomed = await milestones_svc.get_milestone(uid, milestone_id)
|
||||
title = getattr(doomed, "title", "") if doomed else ""
|
||||
except Exception:
|
||||
title = ""
|
||||
batch = await trash_svc.delete(uid, "milestone", milestone_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"milestone {milestone_id} not found")
|
||||
return {"deleted": milestone_id, "title": title, "deleted_batch_id": batch,
|
||||
"message": f'Milestone {milestone_id} ("{title}") and its tasks '
|
||||
f"moved to trash. Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_milestones,
|
||||
get_milestone,
|
||||
create_milestone,
|
||||
update_milestone,
|
||||
delete_milestone,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,244 @@
|
||||
"""Moments as MCP tools: the catalog, and the in-session corrections to it.
|
||||
|
||||
A session needs the vocabulary in hand to mount a rule, to read a line that
|
||||
names a moment, and to fix a misfire — and the operator's ruling is that the
|
||||
fix happens in the session, not on a settings page:
|
||||
|
||||
"making the user leave the session to fix a misfire is not desirable and
|
||||
the llm session should be able to offer corrections"
|
||||
|
||||
So the mapping writes are tools, built to be offered mid-work and made on the
|
||||
operator's yes.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import moment_actions as actions_svc
|
||||
from grimoire.services import moments as moments_svc
|
||||
from grimoire.services import reply_shapes as shapes_svc
|
||||
from grimoire.services import rule_moment_judgments as judgments_svc
|
||||
|
||||
|
||||
async def list_moments() -> dict:
|
||||
"""The moments of work that rules mount on, and which actions reach each one here.
|
||||
|
||||
A rule mounted on a moment arrives whenever that moment happens, whatever
|
||||
the words of the work look like. That is how a rule reaches you when it is
|
||||
about WHEN something is done rather than WHAT it is about: a rule on
|
||||
finishing work belongs at `work.finish` and `reply.report`, where nothing
|
||||
said needs to resemble it.
|
||||
|
||||
Read this when you are about to mount a rule, when a line you were shown
|
||||
names a moment and you want its meaning, or when an action reached the
|
||||
wrong moment — or none — and you are about to correct it with
|
||||
`map_action` / `unmap_action`.
|
||||
|
||||
Names are `<area>.<verb>`; a named procedure (a skill or a stored process)
|
||||
is its own moment, `skill.<name>`, listed under `families`. Each moment
|
||||
says what is happening at it (`means`) and the kinds of action that
|
||||
typically reach it (`reached_by`).
|
||||
|
||||
`actions` maps each moment to the actions that reach it on this install:
|
||||
`via: "default"` ships with the product, `via: "install"` is this
|
||||
install's own mapping. `removed_defaults` are shipped defaults this install
|
||||
switched off.
|
||||
"""
|
||||
out = moments_svc.catalog()
|
||||
out.update(await actions_svc.actions_by_moment(current_user_id()))
|
||||
return out
|
||||
|
||||
|
||||
async def list_reply_shapes() -> dict:
|
||||
"""The default shapes of a reply, and the moment each one arrives at.
|
||||
|
||||
You do not need this to write a reply: the shapes come to you. The core
|
||||
arrives with the turn, and each kind's shape arrives at the moment before
|
||||
that kind of reply is written (closing a task, putting a question, opening
|
||||
a plan). Read this to see them all at once, to answer the operator about
|
||||
what the default is, or before writing a preference that changes one.
|
||||
|
||||
An operator's own adjustment to a shape is a `preference` mounted on that
|
||||
shape's `moment` (create_preference with `moments=[...]`); it arrives
|
||||
beside the default, and where the two differ the preference is what the
|
||||
operator asked for.
|
||||
|
||||
Each shape carries `key`, `title`, `moment` (and what the moment `means`),
|
||||
`delivered` (when it arrives) and `text`.
|
||||
"""
|
||||
return shapes_svc.catalog()
|
||||
|
||||
|
||||
async def map_action(tool: str, moment: str, match: str = "", reason: str = "") -> dict:
|
||||
"""Make an action reach a moment on this install — the in-session fix for a missed moment.
|
||||
|
||||
Use it when an action plainly happened at a moment and the moment did not
|
||||
fire: the operator ships with `make ship`, and nothing mounted on
|
||||
`work.deliver` arrived. Offer the mapping when you notice, in one line
|
||||
("that `make ship` was a deliver and nothing fired — map it?"), and make it
|
||||
on their yes. The correction lasts: every later session on this install
|
||||
gets it.
|
||||
|
||||
Args:
|
||||
tool: the tool as the harness names it — `Bash`, `Edit`,
|
||||
`update_task`. An MCP server prefix is ignored.
|
||||
moment: a moment from `list_moments`, or `skill.<name>`.
|
||||
match: which calls of the tool. Empty = every call. For a tool that
|
||||
runs a command, how the command starts (`make ship`, `./deploy.sh`)
|
||||
— it is checked against each part of a compound command line. For
|
||||
any other tool, its arguments as `field=value` pairs, comma-separated
|
||||
(`status=done`).
|
||||
reason: what misfired, or what this action is for here. Optional; it
|
||||
is shown beside the mapping when someone reviews it later.
|
||||
|
||||
Returns the change made and `now_reaches`: every moment that action
|
||||
reaches after the change, so you can confirm it in the same reply.
|
||||
Mapping a shipped default this install had switched off switches it back
|
||||
on.
|
||||
"""
|
||||
return await actions_svc.map_action(
|
||||
current_user_id(), tool, match, moment, reason=reason, actor="model",
|
||||
)
|
||||
|
||||
|
||||
async def unmap_action(tool: str, moment: str, match: str = "", reason: str = "") -> dict:
|
||||
"""Stop an action reaching a moment on this install — the fix for a moment that fires wrongly.
|
||||
|
||||
Use it when a moment fires on an action that is not that moment here — a
|
||||
`curl` to a local test server reaching `env.reach`, say — and the rules it
|
||||
brings are noise every time. Offer it when you notice, and make it on the
|
||||
operator's yes.
|
||||
|
||||
The arguments name the mapping exactly as `list_moments` shows it. This
|
||||
install's own mapping is removed; a shipped default is switched off for
|
||||
this install only, and stays off across upgrades. `map_action` with the
|
||||
same arguments switches a default back on.
|
||||
|
||||
Args:
|
||||
tool: the tool as `list_moments` names it.
|
||||
moment: the moment it should stop reaching.
|
||||
match: the mapping's match, as listed (empty for a whole-tool mapping).
|
||||
reason: why it misfires here — worth giving for a default, since it is
|
||||
the record of why this install differs from the product.
|
||||
|
||||
Returns the change made and `now_reaches` for that action.
|
||||
"""
|
||||
return await actions_svc.unmap_action(
|
||||
current_user_id(), tool, match, moment, reason=reason, actor="model",
|
||||
)
|
||||
|
||||
|
||||
async def rules_to_mount(limit: int = 25, offset: int = 0) -> dict:
|
||||
"""The rules nobody has decided the moments of — the pass over the corpus, a page at a time.
|
||||
|
||||
A rule written before moments existed is mounted on nothing and arrives
|
||||
only when its words resemble the work. Read each rule here, decide WHEN it
|
||||
applies, and record the answer with `propose_rule_moments`: the moments it
|
||||
belongs on, or that none fits because it is about WHAT is done rather than
|
||||
when. A rule leaves this list as soon as any answer is recorded, so the
|
||||
next call returns the next unread rules.
|
||||
|
||||
Returns `rules` (id, title, kind, statement, when_to_apply, home),
|
||||
`total_unjudged`, and the `catalog` to choose from.
|
||||
"""
|
||||
return await judgments_svc.unjudged_rules(current_user_id(), limit=limit, offset=offset)
|
||||
|
||||
|
||||
async def propose_rule_moments(proposals: list[dict]) -> dict:
|
||||
"""Propose the moments rules belong on — for the operator to confirm, never mounted by this call.
|
||||
|
||||
Each item is either
|
||||
`{"rule_id": N, "moments": ["work.finish", "reply.report"], "why": "…"}`
|
||||
or `{"rule_id": N, "none": "why no moment fits"}`.
|
||||
|
||||
Propose a moment when the rule governs that point in the work whatever
|
||||
the work is about — a rule about when work counts as done belongs where
|
||||
work is finished, delivered, verified and reported. Say none fits when
|
||||
the rule is about a subject (a library, a file, a style) and is best
|
||||
reached by meaning. `why` is shown to the operator beside the proposal;
|
||||
write it so they can say yes or no without opening the rule.
|
||||
|
||||
A moment proposal waits as `suggested` until judged; show the operator
|
||||
what you proposed (`rule_moment_proposals`) and let them decide. A pair
|
||||
already mounted or judged is skipped and listed under `skipped`; a bad
|
||||
item is refused alone under `refused`.
|
||||
"""
|
||||
return await judgments_svc.propose(current_user_id(), proposals)
|
||||
|
||||
|
||||
async def rule_moment_proposals(rule_id: int = 0) -> dict:
|
||||
"""The moment proposals waiting on the operator, grouped by rule.
|
||||
|
||||
Each rule carries what it is mounted on now and its proposals. Each
|
||||
proposal is a `mount` or an `unmount`, with the moment, where it came
|
||||
from (`pass` — read from the rule; `signal` — the rule kept being opened
|
||||
just after that moment fired; `misfire` — sessions reported the mount
|
||||
arriving where it did not apply, with their `reasons` and the actions
|
||||
that `reached_by` the moment), the reason, and the evidence counts.
|
||||
`rule_id` narrows to one rule.
|
||||
"""
|
||||
return await judgments_svc.pending(current_user_id(), rule_id=rule_id or None)
|
||||
|
||||
|
||||
async def judge_rule_moments(judgments: list[dict]) -> dict:
|
||||
"""Confirm or reject moment proposals — on the operator's word, since a confirm mounts the rule.
|
||||
|
||||
Each item is `{"rule_id": N, "moment": "work.finish", "verdict":
|
||||
"confirm" | "reject", "note": "why"}`. Confirm mounts the rule on that
|
||||
moment beside what it already has — on an `unmount` proposal, it keeps
|
||||
the mount and stops the misfire question; reject records that it does not
|
||||
belong there (and unmounts it if it was mounted), so neither the pass nor
|
||||
the signal proposes the pair again. Moment `""` judges a "no moment fits"
|
||||
answer. Put the operator's reason in `note`.
|
||||
"""
|
||||
return await judgments_svc.judge(current_user_id(), judgments)
|
||||
|
||||
|
||||
async def rule_misfired(rule_id: int, moment: str, why: str,
|
||||
reached_by: str = "", project_id: int = 0) -> dict:
|
||||
"""Report that a rule MOUNTED on a moment arrived there and did not apply.
|
||||
|
||||
A mount delivers its rule every time the moment fires, whatever the work
|
||||
is about — so a mount that is wrong is noise at every occurrence, and
|
||||
nothing else notices. Call this when a line said a rule arrived *at* a
|
||||
moment ("at work.verify, reached by `actions_run_read`"), you read it,
|
||||
and it does not govern what you were doing there. It costs one call and
|
||||
asks nothing of the operator; reports gather, counted once per day, and
|
||||
once a pair has them on three distinct days the response carries a line
|
||||
asking you to offer the operator the fix — take the rule off the moment,
|
||||
or unmap the action when it is the action that is wrong here.
|
||||
|
||||
A rule that applied, even one you were already following, is not a
|
||||
misfire. A rule that arrived by resemblance rather than a mount is
|
||||
refused with the fix for that (its trigger).
|
||||
|
||||
Args:
|
||||
rule_id: the rule the line named.
|
||||
moment: the moment it arrived at, as the line names it.
|
||||
why: what you were doing and why the rule did not bear on it. Required
|
||||
— it is what tells the operator whether the rule or the action
|
||||
is wrong.
|
||||
reached_by: the action the line says reached the moment (`git push`,
|
||||
`status=done`). Counted, so the operator can see which action
|
||||
keeps bringing it.
|
||||
project_id: the project you are working in (0 = none).
|
||||
|
||||
Returns `recorded`, `days` so far against the `bar`, and `context`: a line
|
||||
to act on once the bar is crossed, else "". A mount the operator chose to
|
||||
keep says so under `kept`.
|
||||
"""
|
||||
return await judgments_svc.misfired(
|
||||
current_user_id(), rule_id, moment, why=why, reached_by=reached_by,
|
||||
project_id=project_id or None,
|
||||
)
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
mcp.tool(name="list_moments")(list_moments)
|
||||
mcp.tool(name="list_reply_shapes")(list_reply_shapes)
|
||||
mcp.tool(name="map_action")(map_action)
|
||||
mcp.tool(name="unmap_action")(unmap_action)
|
||||
mcp.tool(name="rules_to_mount")(rules_to_mount)
|
||||
mcp.tool(name="propose_rule_moments")(propose_rule_moments)
|
||||
mcp.tool(name="rule_moment_proposals")(rule_moment_proposals)
|
||||
mcp.tool(name="judge_rule_moments")(judge_rule_moments)
|
||||
mcp.tool(name="rule_misfired")(rule_misfired)
|
||||
@@ -0,0 +1,501 @@
|
||||
"""Note CRUD MCP tools.
|
||||
|
||||
Thin wrappers over services/notes.py with is_task=False. Signatures mirror
|
||||
the existing fable-mcp contracts exactly so client behavior is preserved.
|
||||
|
||||
Sentinel conventions (inherited from existing fable-mcp tools):
|
||||
- `tag: str = ""` / `search_text: str = ""` — empty means "no filter"
|
||||
- `project_id: int = 0` — on create: orphan note (no project); on update:
|
||||
"leave unchanged" (there is no "remove from project" through this tool,
|
||||
which is a pre-existing limitation)
|
||||
- `title: str = ""` / `body: str = ""` on update — empty means "leave unchanged"
|
||||
- `tags: list[str] | None = None` — None means "leave unchanged"; [] clears
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.mcp.tools import systems as systems_tools
|
||||
from grimoire.services import access as access_svc
|
||||
from grimoire.services import dedup as dedup_svc
|
||||
from grimoire.services import family as family_svc
|
||||
from grimoire.services import notes as notes_svc
|
||||
from grimoire.services import supersession as supersession_svc
|
||||
from grimoire.services import systems as systems_svc
|
||||
from grimoire.services import trash as trash_svc
|
||||
from grimoire.services.note_usage import record_pulled
|
||||
from grimoire.services.record_refs import refuse_guessed_ids
|
||||
from grimoire.services import moment_delivery
|
||||
|
||||
|
||||
async def list_notes(
|
||||
limit: int = 20,
|
||||
offset: int = 0,
|
||||
tag: str = "",
|
||||
search_text: str = "",
|
||||
project_id: int = 0,
|
||||
) -> dict:
|
||||
"""List notes (non-task documents) stored in Grimoire.
|
||||
|
||||
Optionally filter by a single tag (plain string, no # prefix) or by
|
||||
`search_text`, which matches on MEANING — the same semantic match every
|
||||
search surface uses (keyword ILIKE is only the fallback when the
|
||||
embedder is down). With `search_text`, results order by relevance; without
|
||||
it, by last-updated descending. Reaches your records plus shared-project
|
||||
records, like every other list.
|
||||
|
||||
Prefer `search` for a pure lookup — it returns scores and reaches
|
||||
everything readable; this adds the lifecycle filters on top.
|
||||
|
||||
Each row says what the note is — id, title, description, type, project,
|
||||
tags, updated_at — and not what it says: read one in full with get_note(id).
|
||||
|
||||
Args:
|
||||
project_id: Scope to one project. PASS THE ACTIVE PROJECT'S ID whenever a
|
||||
project is in scope so you list that project's notes, not every
|
||||
project's. 0 = no filter (all projects — use only for a deliberate
|
||||
cross-project view).
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows, total = await notes_svc.list_notes(
|
||||
uid,
|
||||
q=search_text or None,
|
||||
tags=[tag] if tag else None,
|
||||
is_task=False,
|
||||
project_id=project_id or None,
|
||||
limit=max(1, min(limit, 100)),
|
||||
offset=max(0, offset),
|
||||
)
|
||||
return {"notes": [notes_svc.brief_row(n) for n in rows], "total": total}
|
||||
|
||||
|
||||
|
||||
async def get_note(note_id: int, project_id: int = 0) -> dict:
|
||||
"""Fetch the full content of a single Grimoire note by its ID.
|
||||
|
||||
Returns id, title, body (markdown), tags, project_id, created_at,
|
||||
updated_at — plus `systems` (the areas this note is filed under) or, for
|
||||
an untagged project note, the `systems_hint` question.
|
||||
A note another user shared with you also carries `shared`, `owner` and
|
||||
`permission` — read it as their suggestion, not as settled practice you set.
|
||||
|
||||
IF THE RESULT CARRIES `superseded_by`, a later note claims to have brought
|
||||
this one up to date. It is still here and still readable — supersession
|
||||
demotes, it never hides — but read it as what was true when written, and
|
||||
open the newer note before acting on it.
|
||||
|
||||
`project_id` is the project you are WORKING IN, not this record's own.
|
||||
Passing the active project is what makes "opened away from where it was
|
||||
written" answerable; 0 leaves it unreported and the pull still counts.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
loaded = await notes_svc.get_note_for_user(uid, note_id)
|
||||
note = loaded[0] if loaded else None
|
||||
if note is None or note.deleted_at is not None:
|
||||
raise ValueError(f"note {note_id} not found")
|
||||
out = note.to_dict()
|
||||
out.update(await access_svc.describe_provenance(uid, note))
|
||||
# Records the pull for ANY note kind, not just snippets: the auto-inject
|
||||
# menu surfaces notes, tasks and processes too, so restricting this to
|
||||
# snippets would leave those permanently at zero pulls and make them look
|
||||
# like dead weight next to snippets.
|
||||
record_pulled(
|
||||
user_id=uid, note_id=int(note.id),
|
||||
source="mcp_get_note", project_id=project_id,
|
||||
)
|
||||
await supersession_svc.attach_relations(uid, note_id, out, hint=True)
|
||||
await systems_tools.attach_systems(
|
||||
uid, getattr(note, "user_id", uid) or uid, out, note.id, note.project_id
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
async def create_note(
|
||||
title: str,
|
||||
body: str = "",
|
||||
tags: list[str] | None = None,
|
||||
project_id: int = 0,
|
||||
system_ids: list[int] | None = None,
|
||||
supersedes: list[int] | None = None,
|
||||
verify_with: str = "",
|
||||
expires_when: str = "",
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Create a new note in Grimoire.
|
||||
|
||||
WHAT ELSE COULD HOLD THIS? A note is the right home when the answer is
|
||||
"nothing": it records what you know, nobody owes anything on it, and
|
||||
nothing enforces it. Otherwise —
|
||||
- someone has to DO something -> create_task. A note titled "we should…"
|
||||
is a task nobody will ever see again.
|
||||
- future sessions must OBEY it -> create_rule. The test is whether
|
||||
ignoring it would be a mistake, not merely uninformed.
|
||||
- it is how the operator wants work DONE, and ignoring it costs
|
||||
consistency rather than correctness -> create_preference. No approval
|
||||
loop, and it stays yours to keep current as they correct you.
|
||||
- "merely uninformed" is the other half of the rule test, and it has its
|
||||
own home: something you learned that a later session should meet at the
|
||||
moment it applies -> create_lesson. A note is found by topic when
|
||||
someone goes looking; a lesson arrives on its own, keyed to the
|
||||
SITUATION. If you would want this handed to you next time you were in
|
||||
that spot, it is a lesson.
|
||||
- reusable code with a place in a repo -> create_snippet. The location is
|
||||
what lets it be found from the file someone is about to edit.
|
||||
- a procedure followed start to finish -> create_process.
|
||||
|
||||
Args:
|
||||
title: Note title (required).
|
||||
body: Markdown content. Supports [[wikilinks]] to other notes by title.
|
||||
tags: List of plain-string tags without # prefix, e.g. ["python", "ideas"].
|
||||
project_id: Associate with a project (use 0 for no project / orphan note).
|
||||
system_ids: Ids of the project's Systems to associate this note with
|
||||
(e.g. research about a subsystem). See list_systems / create_system.
|
||||
supersedes: Ids of EARLIER notes this one replaces or brings up to date.
|
||||
Reach for it whenever you write something that overtakes what an
|
||||
older note recorded — a re-measurement, a decision that reverses an
|
||||
earlier one, a dev-log covering ground a previous one covered.
|
||||
The older note stays readable and keeps its place in search; it
|
||||
simply stops competing with this one for the same question, and
|
||||
arrives labelled when it does surface. This records a CLAIM, not a
|
||||
verdict: it never says the older note was wrong, only that it is no
|
||||
longer the current answer.
|
||||
verify_with: HOW TO CHECK this note is still true. Leave empty for
|
||||
almost every note — that is the normal case, not an unfinished
|
||||
one.
|
||||
A note is a NORM or a CONSTRAINT. A norm is a decision ("we derive
|
||||
versions from commit time"): it has no truth value and changes only
|
||||
when its author changes it, which they know they did. A CONSTRAINT
|
||||
asserts a fact about someone else's software ("AMO refuses to
|
||||
re-sign a version", "this forge numbers CI runs per repository"),
|
||||
and it goes false with nobody watching. Only constraints get a
|
||||
check.
|
||||
The test, in one question: COULD THIS NOTE BECOME FALSE WITHOUT
|
||||
ANYONE EDITING IT? If no, leave this empty. Sharper still: is the
|
||||
thing this note describes YOURS TO CHANGE? If yes it is a
|
||||
decision. Measured against a real corpus, every note that earned a
|
||||
check was about somebody else's software.
|
||||
Three that look like candidates and are not: a resume pointer or
|
||||
"current state" note (goes stale fastest, but the cure is to
|
||||
update it, not to check it); a measurement of your own system (it
|
||||
goes false because you changed something, and you knew); and a
|
||||
decision that RESTS on someone else's behaviour (check the note
|
||||
asserting the fact, not the decision).
|
||||
A command, a path, a URL, a query. Prose is allowed; something
|
||||
runnable is better.
|
||||
expires_when: The STATE that ends it — deliberately not a date.
|
||||
"When Forgejo issues run numbers per workflow rather than per
|
||||
repository", not "in six months". Constraints expire when the
|
||||
ground moves, not on a schedule.
|
||||
force: Bypass the near-duplicate gate. By default, a note with the
|
||||
same title, or one that reads as a copy, in the same project BLOCKS
|
||||
the create and its id is returned so you update it instead. A note
|
||||
that reads CLOSE but not identical does not block: the note is
|
||||
created and the reply carries `overlaps` — open the top one and
|
||||
judge it. Same thing: fold into it and delete the new one. A
|
||||
sibling (the next dev-log, another part of one design): keep both.
|
||||
Set force true only when a blocked note is genuinely distinct.
|
||||
|
||||
AN ID EXISTS ONLY ONCE A CREATE RETURNS IT. A body citing a `#N` that has
|
||||
not been assigned yet is refused — every session and user draws from one
|
||||
sequence, so an expected id is taken by whoever creates next. Records that
|
||||
must cite each other: create_records, with {{ref:N}} placeholders.
|
||||
|
||||
Returns the created note object including its assigned id, OR — when a
|
||||
near-duplicate is found and force is false — {"duplicate": true,
|
||||
"existing_id": ..., "message": ...} and nothing is created. A tagged
|
||||
record shows its `systems`; created untagged in a project, the response
|
||||
carries the `systems_hint` question instead — answer it: tag the record,
|
||||
create the missing System, or deliberately leave it untagged. A
|
||||
`family_hint` means the note may carry an idea every project on a shared
|
||||
platform will need, and an evaluation was opened — the family-canon skill
|
||||
says how to judge it.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
await refuse_guessed_ids(title, body)
|
||||
overlaps: list = []
|
||||
if not force:
|
||||
dup = await dedup_svc.find_duplicate_note(
|
||||
uid, title, body, project_id=project_id or None,
|
||||
is_task=False, note_type="note", overlaps=overlaps,
|
||||
)
|
||||
if dup is not None:
|
||||
return dedup_svc.duplicate_response(dup, "note")
|
||||
note = await notes_svc.create_note(
|
||||
uid,
|
||||
title=title,
|
||||
body=body,
|
||||
tags=tags,
|
||||
project_id=project_id or None,
|
||||
verify_with=verify_with,
|
||||
expires_when=expires_when,
|
||||
)
|
||||
if system_ids:
|
||||
await systems_svc.set_record_systems(uid, note.id, system_ids)
|
||||
if supersedes:
|
||||
try:
|
||||
await supersession_svc.set_supersedes(uid, note.id, supersedes)
|
||||
except PermissionError as exc:
|
||||
# The note WAS created — surface the real reason rather than a
|
||||
# not-found, and leave the note rather than silently rolling it back.
|
||||
raise ValueError(str(exc)) from exc
|
||||
data = note.to_dict()
|
||||
await systems_tools.attach_systems(uid, uid, data, note.id, project_id or None)
|
||||
await supersession_svc.attach_relations(uid, note.id, data, hint=True)
|
||||
data.update(dedup_svc.note_overlap_response(overlaps, "note"))
|
||||
# Family canon's write-time triggers: a cited source of a
|
||||
# pattern, or the same idea in another project on a shared platform.
|
||||
await family_svc.attach_family_hint(uid, data, note, created=True)
|
||||
return await moment_delivery.attach_moment_rules(uid, "create_note", {"project_id": project_id}, data)
|
||||
|
||||
|
||||
async def update_note(
|
||||
note_id: int,
|
||||
title: str = "",
|
||||
body: str = "",
|
||||
tags: list[str] | None = None,
|
||||
project_id: int = 0,
|
||||
system_ids: list[int] | None = None,
|
||||
supersedes: list[int] | None = None,
|
||||
verify_with: str = "",
|
||||
expires_when: str = "",
|
||||
clear: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Update an existing Grimoire note. Only explicitly provided fields are changed.
|
||||
|
||||
Args:
|
||||
note_id: ID of the note to update.
|
||||
title: New title, or omit to leave unchanged.
|
||||
body: New markdown body, or omit to leave unchanged.
|
||||
tags: Replaces the full tag list. Pass [] to clear all tags. Omit to leave unchanged.
|
||||
project_id: New project association. Omit (or pass 0) to leave unchanged.
|
||||
system_ids: Replace this note's System associations with these ids
|
||||
(set-semantics). None = leave unchanged; [] = clear all.
|
||||
supersedes: Replace the ids of earlier notes this one replaces
|
||||
(set-semantics). None = leave unchanged; [] = clear all. See
|
||||
create_note for when to reach for it.
|
||||
verify_with: How to check this note is still true. Almost every note
|
||||
should leave this empty — that is the normal, finished state, not
|
||||
a gap: an empty check is the marker for "this is a decision, there
|
||||
is nothing to go and check". See create_note for the full
|
||||
norm-vs-constraint test; the short form is "could this become
|
||||
false without anyone editing it?".
|
||||
expires_when: The STATE that ends it, not a date.
|
||||
clear: Names of fields to UNSET — "verify_with", "expires_when".
|
||||
Needed because "" means "leave this alone" here, so there is no
|
||||
value that removes a field: an agent updating a body must not
|
||||
silently wipe a check it was not asked about. A note that stops
|
||||
being a constraint is cleared by naming the field, which cannot
|
||||
happen by accident.
|
||||
|
||||
Rewriting `verify_with` drops the note's verification stamp: a stamp
|
||||
certifies a particular check, and carrying it across a rewrite would vouch
|
||||
for something nobody has looked at.
|
||||
|
||||
A task and a snippet are both REFUSED a check, with a message saying where
|
||||
to go instead — a task's decay is its status, and a snippet has
|
||||
verify_snippet.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
fields: dict = {}
|
||||
if title:
|
||||
fields["title"] = title
|
||||
if body:
|
||||
fields["body"] = body
|
||||
if tags is not None:
|
||||
fields["tags"] = tags
|
||||
if project_id:
|
||||
fields["project_id"] = project_id
|
||||
if verify_with:
|
||||
fields["verify_with"] = verify_with
|
||||
if expires_when:
|
||||
fields["expires_when"] = expires_when
|
||||
await refuse_guessed_ids(title, body)
|
||||
note = await notes_svc.update_note(
|
||||
uid, note_id, clear=clear or (), **fields
|
||||
)
|
||||
if note is None:
|
||||
raise ValueError(f"note {note_id} not found")
|
||||
if system_ids is not None:
|
||||
await systems_svc.set_record_systems(uid, note_id, system_ids)
|
||||
if supersedes is not None:
|
||||
try:
|
||||
await supersession_svc.set_supersedes(uid, note_id, supersedes)
|
||||
except PermissionError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
data = note.to_dict()
|
||||
await systems_tools.attach_systems(
|
||||
uid, getattr(note, "user_id", uid) or uid, data, note_id, note.project_id
|
||||
)
|
||||
await supersession_svc.attach_relations(uid, note_id, data, hint=True)
|
||||
if body:
|
||||
await family_svc.attach_family_hint(uid, data, note, created=False)
|
||||
return data
|
||||
|
||||
|
||||
async def find_duplicate_records(kind: str = "note", threshold: float = 0.0) -> dict:
|
||||
"""Notes or tasks already recorded that closely resemble each other.
|
||||
|
||||
The create gate PREVENTS a duplicate arriving through an agent; the web UI
|
||||
deliberately has no gate (a human mid-thought must not be blocked by a 409),
|
||||
which makes YOU the corrections system — and a corrector has to be able to
|
||||
SEE what needs correcting. This is the finder. Run it when tidying a
|
||||
project's records, or when you suspect the same ground was covered twice.
|
||||
|
||||
Args:
|
||||
kind: "note" (documents), "task", or "lesson". Each kind is compared
|
||||
only against its own — a to-do and a write-up are not alternatives
|
||||
to each other. Snippets have their own report,
|
||||
find_duplicate_snippets, whose groups propose a lossless merge.
|
||||
LESSONS are the kind most worth running this on: their create gate
|
||||
is deliberately permissive (it sits above the band where two
|
||||
genuinely different lessons about one area would block each other),
|
||||
so this report is where that tolerance gets reviewed.
|
||||
threshold: Similarity floor, 0-1. 0 uses the configured setting.
|
||||
|
||||
READ THE `suggestion` FIELD BEFORE ACTING, because the right fix differs by
|
||||
what the records ARE, not how alike they score. For notes: a correction
|
||||
pair → declare `supersedes` on the newer; state smeared across dated
|
||||
records → extract it into the System's reference note and leave these as
|
||||
history; genuinely parallel records → leave them alone. NEVER merge notes.
|
||||
Each group carries `members` with dates and any `existing_supersessions`
|
||||
already declared inside it — a pair someone has ruled on is not an open
|
||||
question.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
# Validated against the service's own list rather than a second copy here,
|
||||
# so a kind added there reaches this door instead of being refused by a
|
||||
# literal nobody remembered to widen.
|
||||
allowed = tuple(k for k in dedup_svc._REPORT_KINDS if k != "snippet")
|
||||
if kind not in allowed:
|
||||
raise ValueError(
|
||||
f"kind must be one of {', '.join(allowed)} — snippets have their "
|
||||
"own report, find_duplicate_snippets"
|
||||
)
|
||||
return await dedup_svc.find_duplicate_records(
|
||||
uid, kind=kind, threshold=threshold if threshold > 0 else None,
|
||||
)
|
||||
|
||||
|
||||
async def delete_note(note_id: int) -> dict:
|
||||
"""Move a Grimoire note to the trash (recoverable). Restore via restore(batch_id)."""
|
||||
uid = current_user_id()
|
||||
# Read the title BEFORE the delete: afterwards the row is trashed and the
|
||||
# confirmation could only echo the number back. A deletion the operator
|
||||
# cannot recognise is one they cannot tell was the wrong one.
|
||||
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
|
||||
# that errors must not stop the delete happening. Same posture the
|
||||
# staleness marker takes — a decoration may never break its payload.
|
||||
try:
|
||||
loaded = await notes_svc.get_note_for_user(uid, note_id)
|
||||
title = getattr(loaded[0], "title", "") if loaded else ""
|
||||
except Exception:
|
||||
title = ""
|
||||
batch = await trash_svc.delete(uid, "note", note_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"note {note_id} not found")
|
||||
return {"deleted": note_id, "title": title, "deleted_batch_id": batch,
|
||||
"message": f'Note {note_id} ("{title}") moved to trash. '
|
||||
f"Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
async def notes_due_for_verification(
|
||||
older_than_days: int = 0, project_id: int = 0, never_only: bool = False,
|
||||
) -> dict:
|
||||
"""Which notes assert a FACT that nobody has confirmed lately.
|
||||
|
||||
A corpus of notes holds two kinds of thing. Most are DECISIONS or records
|
||||
of what happened — they have no truth value and cannot rot. A few assert a
|
||||
fact about someone else's software: what a signing service does on a
|
||||
duplicate upload, how a forge numbers its CI runs, what an updater
|
||||
compares. Those go false silently, with nobody present, and a
|
||||
cross-project reference note keeps being read as current by every project
|
||||
that cites it.
|
||||
|
||||
This lists the second kind, oldest verification first, NEVER-CHECKED AT
|
||||
THE TOP — a note nobody has ever confirmed is a claim with no evidence
|
||||
behind it at all. Each row carries `verify_with` in full, because you are
|
||||
about to go and run it, plus `expires_when` and `days_since_verified`.
|
||||
|
||||
Reach for it when curating, when a note's claim just contradicted what you
|
||||
observed, or periodically. Then, per row: run the check, and call
|
||||
mark_note_verified with what you found.
|
||||
|
||||
Notes with no `verify_with` never appear, and that is correct — they are
|
||||
decisions, and there is nothing to go and check. Do not "fix" their
|
||||
absence by giving them checks: this list is only worth reading while
|
||||
everything on it genuinely can go false.
|
||||
|
||||
Args:
|
||||
older_than_days: only notes last verified longer ago than this.
|
||||
Never-checked notes always qualify — they are the most overdue
|
||||
thing there is. 0 = no age filter.
|
||||
project_id: narrow to one project. 0 = every project. Unlike the rules
|
||||
sweep, this filter is safe: a note belongs to at most one project
|
||||
outright, where a project is bound by every GLOBAL rule as well as
|
||||
its own — so a project filter would UNDER-report rules.
|
||||
never_only: only notes nobody has ever verified.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
notes = await notes_svc.notes_due_for_verification(
|
||||
uid,
|
||||
older_than_days=older_than_days,
|
||||
project_id=project_id or None,
|
||||
never_only=never_only,
|
||||
)
|
||||
return {
|
||||
"notes": [notes_svc.verification_row(n) for n in notes],
|
||||
"total": len(notes),
|
||||
}
|
||||
|
||||
|
||||
async def mark_note_verified(note_id: int, still_true: bool = True) -> dict:
|
||||
"""Record that you ran a note's check — and what it said.
|
||||
|
||||
Call this AFTER actually running the note's `verify_with`, never on the
|
||||
strength of the claim sounding plausible. A stamp nobody earned is worse
|
||||
than no stamp: it moves the note to the bottom of the sweep and buys the
|
||||
claim another long silence.
|
||||
|
||||
`still_true=False` writes NOTHING. A note whose check failed is not in a
|
||||
special state to be recorded — it is WRONG, and the honest next moves are
|
||||
to correct it, supersede it, or find out why. So it stays at the top of
|
||||
the sweep until someone deals with it, and the response tells you what the
|
||||
note said would end it.
|
||||
|
||||
Args:
|
||||
note_id: the note whose check you ran.
|
||||
still_true: True if the check passed. False if the fact it asserts is
|
||||
no longer true — say so, that is the outcome worth having.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
note = await notes_svc.mark_note_verified(note_id, uid, still_true)
|
||||
if note is None:
|
||||
raise ValueError(
|
||||
f"note {note_id} not found, not writable by you, or carries no "
|
||||
f"verify_with (nothing to verify is not the same as verified)"
|
||||
)
|
||||
data = notes_svc.verification_row(note)
|
||||
data["verified"] = bool(still_true)
|
||||
if not still_true:
|
||||
data["next"] = (
|
||||
"This note is no longer true and is still being read as current "
|
||||
"by anything that cites it. Correct it with update_note, write "
|
||||
"the replacement with create_note(supersedes=[...]), or clear its "
|
||||
"check with update_note(clear=[\"verify_with\"]) if it has stopped "
|
||||
"asserting a fact at all. It stays at the top of "
|
||||
"notes_due_for_verification until one of those happens."
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_notes,
|
||||
get_note,
|
||||
create_note,
|
||||
update_note,
|
||||
find_duplicate_records,
|
||||
delete_note,
|
||||
notes_due_for_verification,
|
||||
mark_note_verified,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Platform MCP tools — what a project is built on or ships as.
|
||||
|
||||
Thin wrappers over services/platforms.py. The catalog is global; a project's
|
||||
platforms decide which family ideas reach it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import platforms as platforms_svc
|
||||
|
||||
|
||||
async def list_platforms(project_id: int = 0) -> dict:
|
||||
"""The GLOBAL platform catalog — runtimes, delivery channels and
|
||||
toolchains a project can be built on or ship as (Android app, container
|
||||
image, Go, …) — and, with a project_id, that project's answer for each.
|
||||
|
||||
A project's platforms decide which family ideas reach it: an idea is for
|
||||
some platforms, and every project that is one of them answers it. Pass
|
||||
slugs from here to set_project_platforms and decide_project_inception.
|
||||
|
||||
Each platform's `markers` are the file patterns that let the coverage
|
||||
refresh DETECT it in a bound repo. A project's `state` per platform is
|
||||
`declared` (a person said so), `detected` (a marker said so) or `rejected`
|
||||
(a person said no — kept so detection cannot add it back). Only declared
|
||||
and detected are membership.
|
||||
|
||||
Args:
|
||||
project_id: also return this project's answers. 0 = catalog only.
|
||||
"""
|
||||
catalog = await platforms_svc.list_platforms()
|
||||
out: dict = {"platforms": [p.to_dict() for p in catalog]}
|
||||
if project_id:
|
||||
rows = await platforms_svc.project_platforms(current_user_id(), project_id)
|
||||
if rows is None:
|
||||
raise ValueError(f"project {project_id} not found")
|
||||
out["project_platforms"] = rows
|
||||
return out
|
||||
|
||||
|
||||
async def set_project_platforms(
|
||||
project_id: int,
|
||||
declared: list[str] | None = None,
|
||||
rejected: list[str] | None = None,
|
||||
withdrawn: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Say which platforms a project is — or is not. Only the platforms named
|
||||
change; everything else is left exactly as it is.
|
||||
|
||||
Use it when the operator corrects what detection found, or when a project
|
||||
starts or stops shipping something (it gains an Android client; it drops
|
||||
its container image). At project creation, decide_project_inception's
|
||||
`platforms` is the place instead — it records the answer with the rest of
|
||||
what the project inherits.
|
||||
|
||||
Args:
|
||||
project_id: the project.
|
||||
declared: slugs the project IS (list_platforms).
|
||||
rejected: slugs it is NOT — recorded as a "no", so the coverage
|
||||
refresh never detects it back.
|
||||
withdrawn: slugs whose answer to drop entirely, so detection may
|
||||
decide again on the next refresh.
|
||||
"""
|
||||
updates: dict[str, str | None] = {}
|
||||
for slug in withdrawn or []:
|
||||
updates[slug] = None
|
||||
for slug in rejected or []:
|
||||
updates[slug] = "rejected"
|
||||
for slug in declared or []:
|
||||
updates[slug] = "declared"
|
||||
if not updates:
|
||||
raise ValueError("name at least one platform to declare, reject or withdraw")
|
||||
rows = await platforms_svc.set_project_platforms(current_user_id(), project_id, updates)
|
||||
return {"project_id": project_id, "project_platforms": rows}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (list_platforms, set_project_platforms):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,283 @@
|
||||
"""Stored-process MCP tools: reusable saved prompts (note_type='process').
|
||||
|
||||
A process is a Note whose body is a prompt the operator fires later
|
||||
("run the X process"). Mirrors entities.py — the tools wrap notes_svc directly.
|
||||
get_process is the fire mechanism: it returns the full prompt for Claude to run.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import access as access_svc
|
||||
from grimoire.services import dedup as dedup_svc
|
||||
from grimoire.services import knowledge as knowledge_svc
|
||||
from grimoire.services import moment_actions
|
||||
from grimoire.services import moments as moments_svc
|
||||
from grimoire.services import notes as notes_svc
|
||||
from grimoire.services import systems as systems_svc
|
||||
from grimoire.services import trash as trash_svc
|
||||
from grimoire.services.note_usage import record_pulled
|
||||
from grimoire.services import moment_delivery
|
||||
|
||||
|
||||
async def list_processes(
|
||||
q: str = "", tag: str = "", limit: int = 50, offset: int = 0,
|
||||
) -> dict:
|
||||
"""List stored processes (reusable saved prompts).
|
||||
|
||||
Args:
|
||||
q: Free-text search across title + body (optional).
|
||||
tag: Filter to a single tag (optional).
|
||||
limit: Max results (1-100).
|
||||
offset: Skip this many before returning — page past the cap.
|
||||
`total` is the unpaged count, so it says whether more remains.
|
||||
|
||||
Returns {"processes": [{id, title, tags, preview}], "total": int}. An entry
|
||||
marked `shared: true` with an `owner` is another person's procedure — treat
|
||||
it as a suggestion to raise with the operator, not as their own practice.
|
||||
|
||||
Searching (passing `q`) reaches processes shared directly with the operator;
|
||||
the plain list deliberately doesn't, so someone else's procedure never
|
||||
arrives unasked.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
items, total = await knowledge_svc.query_knowledge(
|
||||
user_id=uid, note_type="process", tags=[tag] if tag else [],
|
||||
sort="modified", q=q or None, limit=max(1, min(limit, 100)),
|
||||
offset=max(0, offset),
|
||||
)
|
||||
labelled = await access_svc.label_shared_items(uid, items)
|
||||
procs = [{"id": it["id"], "title": it["title"], "tags": it.get("tags", []),
|
||||
"preview": it.get("snippet", ""),
|
||||
**({"shared": True, "owner": it.get("owner")} if it.get("shared") else {})}
|
||||
for it in labelled]
|
||||
return {"processes": procs, "total": total}
|
||||
|
||||
|
||||
async def create_process(
|
||||
title: str, body: str, tags: list[str] | None = None,
|
||||
system_ids: list[int] | None = None, moments: list[str] | None = None,
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Create a stored process (a reusable saved prompt).
|
||||
|
||||
FOLLOWED, OR READ? A process is invoked deliberately and worked through
|
||||
start to finish. If it should apply whether or not anyone invokes it, it
|
||||
is a rule (create_rule) — that is the whole difference between a procedure
|
||||
and a standing instruction. If it is knowledge to consult rather than
|
||||
steps to execute, it is a note (create_note).
|
||||
|
||||
AUTHOR IT AS A SHAPE, NOT A SCRIPT. A process's value is the accumulated
|
||||
procedure — the steps, the taxonomy, the quality bar, the failure modes
|
||||
worth guarding. It must not force anything the invoking conversation
|
||||
didn't choose: no embedded approach mandates, no pre-granted approvals
|
||||
(a fan-out opt-in, a permission to act), no assumptions that overwrite
|
||||
the live request's intent. The conversation that invokes it supplies the
|
||||
parameters and always wins where they disagree; write clarify steps to
|
||||
seed from what the operator already said, not to re-ask it.
|
||||
|
||||
Args:
|
||||
title: Process name, e.g. "Drift Audit" (required).
|
||||
body: The full prompt to run later (markdown). Required.
|
||||
tags: Plain-string tags, no # prefix.
|
||||
system_ids: Systems (subsystems/areas) to file this process under, so
|
||||
an area-scoped read finds it.
|
||||
moments: The moments from list_moments this procedure is FOR — a
|
||||
release procedure is a deliver, a review procedure a verify.
|
||||
Loading the process then reaches each one, as well as its own
|
||||
`skill.scribe-proc-<slug>`, so a rule mounted on work.deliver
|
||||
arrives when the release procedure is loaded, before any of its
|
||||
steps run.
|
||||
force: Bypass the near-duplicate gate. By default, if a title- or
|
||||
meaning-similar process already exists, creation is BLOCKED and the
|
||||
existing one's id is returned so you update it instead. Set true
|
||||
only for a genuinely distinct procedure.
|
||||
|
||||
Returns the created process, OR — when a near-duplicate is found and force
|
||||
is false — {"duplicate": true, "existing_id": ..., "message": ...} (nothing
|
||||
created).
|
||||
|
||||
The gate matters more here than for other kinds: every process becomes a
|
||||
skill file that auto-surfaces on the operator's machine, so two near-identical
|
||||
procedures don't merely bloat the corpus — they compete to be followed, and
|
||||
which one wins is decided by a slug.
|
||||
"""
|
||||
if not (title or "").strip() or not (body or "").strip():
|
||||
raise ValueError("create_process requires a non-empty title and body")
|
||||
uid = current_user_id()
|
||||
if not force:
|
||||
dup = await dedup_svc.find_duplicate_note(
|
||||
uid, title, body, is_task=False, note_type="process",
|
||||
)
|
||||
if dup is not None:
|
||||
return dedup_svc.duplicate_response(dup, "process")
|
||||
declared = moments_svc.require_moments(moments)
|
||||
note = await notes_svc.create_note(
|
||||
uid, title=title.strip(), body=body, note_type="process", tags=tags,
|
||||
data=_with_moments(None, declared),
|
||||
)
|
||||
if system_ids:
|
||||
await systems_svc.set_record_systems(uid, note.id, system_ids)
|
||||
return await moment_delivery.attach_moment_rules(uid, "create_process", {}, _process_dict(note))
|
||||
|
||||
|
||||
def _with_moments(data: dict | None, declared: list[str] | None) -> dict | None:
|
||||
"""`data` with its moments replaced by `declared` (None leaves them)."""
|
||||
out = dict(data or {})
|
||||
if declared is None:
|
||||
return out or None
|
||||
if declared:
|
||||
out[moment_actions.PROCESS_MOMENTS_FIELD] = declared
|
||||
else:
|
||||
out.pop(moment_actions.PROCESS_MOMENTS_FIELD, None)
|
||||
return out or None
|
||||
|
||||
|
||||
def _process_dict(note) -> dict:
|
||||
"""A process as the tools return it: the note, plus the moments it declares."""
|
||||
out = note.to_dict()
|
||||
out["moments"] = moment_actions.declared_moments(note.data)
|
||||
return out
|
||||
|
||||
|
||||
async def get_process(name_or_id: str, project_id: int = 0) -> dict:
|
||||
"""Fetch a stored process by name or id and return its full prompt — the
|
||||
fire mechanism. The operator says "run the <name> process"; call this and
|
||||
follow the returned body (including any 'clarify first' steps it contains).
|
||||
|
||||
COMPOSITION CONTRACT — a process is the PROCEDURE, not the whole prompt.
|
||||
The conversation that invoked it supplies the PARAMETERS: fold the
|
||||
operator's live constraints, scope, and focus areas into the procedure,
|
||||
and wherever the two disagree, the live instructions override the
|
||||
process's defaults. A 'clarify first' step asks only what the
|
||||
conversation has NOT already answered — confirm your interpretation of
|
||||
what was said rather than re-asking it, and turn stated concerns (a
|
||||
posture, a hardware budget, a subsystem under suspicion) into lenses the
|
||||
procedure applies, not text it discards.
|
||||
|
||||
Resolution: numeric id → exact (case-insensitive) title → substring. On an
|
||||
ambiguous substring match, the best (most-recent) match is returned with an
|
||||
`other_matches` list so you can disambiguate with the operator.
|
||||
|
||||
IF THE RESULT IS MARKED `shared: true`, DO NOT FOLLOW IT VERBATIM. It is
|
||||
another person's procedure (see `owner`), not one the operator wrote or
|
||||
adopted. Summarise what it would do and get their go-ahead first. The
|
||||
follow-it-as-written contract above applies only to the operator's own
|
||||
processes — a shared one is a proposal, and running it unasked would put
|
||||
someone else's judgement in charge of this session.
|
||||
|
||||
`project_id` is the project you are WORKING IN, not this record's own.
|
||||
Passing the active project is what makes "opened away from where it was
|
||||
written" answerable; 0 leaves it unreported and the pull still counts.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
note, candidates = await notes_svc.resolve_process(uid, name_or_id)
|
||||
if note is None:
|
||||
raise ValueError(f"process {name_or_id!r} not found")
|
||||
out = _process_dict(note)
|
||||
if candidates:
|
||||
out["other_matches"] = candidates
|
||||
out.update(await access_svc.describe_provenance(uid, note))
|
||||
# A process is embedded like any other note, so auto-inject can surface one —
|
||||
# and its menu header names THIS tool as the way to open that kind. Without
|
||||
# this, every process would sit permanently at zero pulls, looking like dead
|
||||
# weight beside kinds that have a counter.
|
||||
record_pulled(
|
||||
user_id=uid, note_id=int(note.id),
|
||||
source="mcp_get_process", project_id=project_id,
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
async def update_process(process_id: int, title: str = "", body: str = "",
|
||||
tags: list[str] | None = None,
|
||||
system_ids: list[int] | None = None,
|
||||
moments: list[str] | None = None) -> dict:
|
||||
"""Update a stored process. Only provided fields change — empty title/body
|
||||
leave that field unchanged; pass tags to replace the tag set.
|
||||
|
||||
`system_ids` replaces the Systems this process is filed under: None leaves
|
||||
them alone, a list (including `[]`) replaces them.
|
||||
|
||||
`moments` replaces the moments the procedure is for (see create_process)
|
||||
the same way: None leaves them, [] clears them. The change reaches the
|
||||
next load of its skill — the skill file itself does not carry them.
|
||||
|
||||
Editing another user's process requires an editor or admin share from them; a
|
||||
read-only share is not enough and says so rather than claiming not-found.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
loaded = await notes_svc.get_note_for_user(uid, process_id)
|
||||
note = loaded[0] if loaded else None
|
||||
if note is None or note.note_type != "process" or note.deleted_at is not None:
|
||||
raise ValueError(f"process {process_id} not found")
|
||||
if not await access_svc.can_write_note(uid, process_id):
|
||||
raise ValueError(
|
||||
f"process {process_id} is shared with you read-only — ask its owner "
|
||||
f"for edit access, or save your own copy with create_process"
|
||||
)
|
||||
fields: dict = {}
|
||||
if title.strip():
|
||||
fields["title"] = title.strip()
|
||||
if body.strip():
|
||||
fields["body"] = body
|
||||
if tags is not None:
|
||||
fields["tags"] = tags
|
||||
declared = moments_svc.require_moments(moments)
|
||||
if declared is not None:
|
||||
fields["data"] = _with_moments(note.data, declared)
|
||||
# As the owner — update_note is owner-scoped and the write is authorised above.
|
||||
updated = await notes_svc.update_note(note.user_id, process_id, **fields)
|
||||
# The CALLER, not the owner. `update_note` above is owner-scoped because
|
||||
# the service demands it; `set_record_systems` is not — it runs its own
|
||||
# share-aware check and links only Systems the acting user can read, so
|
||||
# passing the owner would bypass the check and borrow their reach (#47).
|
||||
if system_ids is not None:
|
||||
await systems_svc.set_record_systems(uid, process_id, system_ids)
|
||||
if updated is None:
|
||||
raise ValueError(f"process {process_id} not found")
|
||||
out = _process_dict(updated)
|
||||
out.update(await access_svc.describe_provenance(uid, updated))
|
||||
return out
|
||||
|
||||
|
||||
async def delete_process(process_id: int) -> dict:
|
||||
"""Retire a stored process — it moves to the trash and is recoverable.
|
||||
|
||||
Reach for this when a procedure is wrong, superseded, or was never worth
|
||||
keeping. A stored process is installed as a skill file on the operator's
|
||||
machine and auto-surfaces there, so a bad one is followed rather than merely
|
||||
ignored — it costs more than a missing one.
|
||||
|
||||
The same as `delete_note` (a process is a note, and the trash is
|
||||
kind-agnostic), offered here because a kind whose own tools offer only
|
||||
create/read/update reads as one you cannot retire.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
loaded = await notes_svc.get_note_for_user(uid, process_id)
|
||||
note = loaded[0] if loaded else None
|
||||
# Check the KIND before deleting: this tool is reached for by name, and
|
||||
# letting it trash an ordinary note because the id happened to resolve would
|
||||
# be a destructive action taken on a mistyped argument.
|
||||
if note is None or note.note_type != "process" or note.deleted_at is not None:
|
||||
raise ValueError(f"process {process_id} not found")
|
||||
batch = await trash_svc.delete(uid, "note", process_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"process {process_id} not found")
|
||||
return {
|
||||
"deleted_batch_id": batch,
|
||||
"message": (
|
||||
f"Process {process_id} moved to trash. Restore with restore('{batch}'). "
|
||||
f"Its skill stub disappears on the operator's next process sync."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_processes,
|
||||
create_process,
|
||||
get_process,
|
||||
update_process,
|
||||
delete_process,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,514 @@
|
||||
"""Project CRUD MCP tools — thin wrappers over services/projects.py.
|
||||
|
||||
There is no delete tool: to stop working on a project, set its status to
|
||||
'archived'. create_project creates directly, with no similarity check or
|
||||
'confirmed' guard — the client is a capable model that can read the project
|
||||
list itself.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.mcp.tools import systems as systems_tools
|
||||
from grimoire.services import coverage as coverage_svc
|
||||
from grimoire.services import design_systems as design_systems_svc
|
||||
from grimoire.services import family_adoption as family_adoption_svc
|
||||
from grimoire.services import inception as inception_svc
|
||||
from grimoire.services import milestones as milestones_svc
|
||||
from grimoire.services import notes as notes_svc
|
||||
from grimoire.services import platforms as platforms_svc
|
||||
from grimoire.services import projects as projects_svc
|
||||
from grimoire.services import reply_shapes as reply_shapes_svc
|
||||
from grimoire.services import rulebooks as rulebooks_svc
|
||||
from grimoire.services import systems as systems_svc
|
||||
from grimoire.services import trash as trash_svc
|
||||
from grimoire.services.background import spawn
|
||||
from grimoire.services.note_usage import record_surfaced
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def list_projects() -> dict:
|
||||
"""List all Grimoire projects for the current user.
|
||||
|
||||
Returns id, title, description, goal, status (active/paused/completed/archived), color,
|
||||
and a short auto-generated summary for each project.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows = await projects_svc.list_projects(uid)
|
||||
return {"projects": [p.to_dict() for p in rows]}
|
||||
|
||||
|
||||
# How many of each kind the handshake lists. Enough to say what was worked on
|
||||
# lately and what's open; the rest is a list_milestones / list_tasks call away.
|
||||
# Every milestone's plan would push the handshake past what a client accepts
|
||||
# as a tool result.
|
||||
_HANDSHAKE_MILESTONES = 5
|
||||
_HANDSHAKE_OPEN_TASKS = 10
|
||||
_HANDSHAKE_UNPLANNED = 10
|
||||
|
||||
|
||||
async def enter_project(project_id: int) -> dict:
|
||||
"""Session-start handshake: a primer on the project before you work in it.
|
||||
|
||||
Call this FIRST whenever you're about to do project-scoped work
|
||||
(start_planning, create_task, update_*, anything that takes a project_id).
|
||||
One round-trip returns what the project is for, what was worked on
|
||||
lately, what's open, and the vocabulary to record against. It is kept
|
||||
small on purpose: each part names the call that has the rest.
|
||||
|
||||
No persistent server state: this is a read snapshot. Re-call if the
|
||||
session goes idle long enough that the data feels stale.
|
||||
|
||||
Args:
|
||||
project_id: The project to enter.
|
||||
|
||||
Returns a dict with keys: project, milestone_summary, open_tasks, systems,
|
||||
design_system, project_rules, pattern_coverage, reply_shape —
|
||||
plus unplanned_milestones, milestone_summary_omitted,
|
||||
unplanned_milestones_omitted, family, inception and systems_bootstrap,
|
||||
each present only when it applies (see below).
|
||||
|
||||
`reply_shape` is the default shape of every reply you write to the
|
||||
operator: conclusion first, the shortest reply that carries the answer,
|
||||
where the work stands, and the one thing they need to do. Write to it;
|
||||
an operator preference about replies that differs from it wins.
|
||||
|
||||
`project` is id, title, status and the full goal. get_project has the
|
||||
whole record.
|
||||
|
||||
`milestone_summary` is the 5 most recently touched milestones, any status,
|
||||
most recent first. Touched counts a step changing, not only the milestone
|
||||
itself. Each carries its description, its progress and its `next_step` —
|
||||
the earliest step still open, {id, title, status}, or null when none is —
|
||||
but NOT its plan: get_milestone(id) reads a plan and its steps.
|
||||
`milestone_summary_omitted` says how many others exist; list_milestones
|
||||
lists them all.
|
||||
|
||||
Name the next step from `next_step`, not from recall: progress alone says
|
||||
a plan has an open step and not which, and a step named from memory reads
|
||||
exactly like one that was read.
|
||||
|
||||
`unplanned_milestones` is the active milestones that have NO steps yet,
|
||||
in roadmap order (up to 10; `unplanned_milestones_omitted` counts the
|
||||
rest) — id, title and description. They are open work: a plan somebody
|
||||
wrote down and nobody has broken into steps. A milestone with no steps is
|
||||
never touched, so the recent list above can never show one. Before
|
||||
starting a new milestone for work, check whether one of these already
|
||||
describes it; if so, add steps to it with create_records(milestone_id=…)
|
||||
rather than opening a second plan for the same thing.
|
||||
|
||||
`open_tasks` is the 10 most recently touched todo / in-progress tasks,
|
||||
with or without a milestone. A work-log counts as touching its task. Each
|
||||
names its milestone. list_tasks has the rest.
|
||||
|
||||
`project_rules` lists the project's own rules by id and title. Global
|
||||
rules (the ones in rulebooks) apply here too and are not listed. Any rule
|
||||
reaches you in full when your work matches it — a global one or one of
|
||||
this project's, never another project's; get_rule(id) reads one, and
|
||||
search(content_type="rule", project_id=...) asks whether one covers what
|
||||
you are about to do.
|
||||
|
||||
`pattern_coverage` (usually null) is the shape-accounting line — how many
|
||||
of the bound repo's extracted shapes carry a classification against canon
|
||||
— e.g. "shape accounting: 3100/4573 shapes accounted for — 12 canonical ·
|
||||
2900 instance (estimate, computed <date>); 1473 unclassified, largest:
|
||||
internal/api". Unclassified IS the todo: as you
|
||||
touch code in those areas, classify the shapes you can (instances of
|
||||
recorded canon, deliberate variants, one-off exemptions) and record the
|
||||
canon that's missing with create_snippet. A null line on a forge-served
|
||||
project usually means the ledger is seeding in the background (entering
|
||||
triggers it); refresh_pattern_coverage computes it on the spot.
|
||||
|
||||
`systems` is the project's vocabulary of named subsystems/areas. It is
|
||||
returned here so you can TAG as you write: when creating or meaningfully
|
||||
updating a record, ask which of these areas it is about and pass their ids
|
||||
as `system_ids`. If the area a record describes is missing from this list,
|
||||
create it with create_system rather than leaving the area unmodelled. Read
|
||||
a subsystem's accumulated records with list_system_records. Each is id and name; get_system has the charter.
|
||||
|
||||
`platforms` is what the project is built on or ships as
|
||||
— Android app, container image, Go, … — each with how it is known
|
||||
(`declared` by a person, `detected` from a bound repo). They decide which
|
||||
family ideas reach this project. An empty list on a project that plainly
|
||||
ships something is worth correcting with set_project_platforms.
|
||||
|
||||
`family` appears ONLY when this project has family canon
|
||||
to answer: how many canon ideas on its platforms are `unassessed`, how
|
||||
many it `owed`s, and how many answers were given against an older canon
|
||||
version (`needs_recheck`) — each with the list_family_adoptions call that
|
||||
lists them. Answer an unassessed idea when your work reaches its area
|
||||
(get_family_adoption, then assess_family_adoption, or classify the code
|
||||
against it); the family-canon skill has the order.
|
||||
|
||||
`inception` appears ONLY when the project is yours and
|
||||
nobody has decided what it inherits: it carries the current defaults
|
||||
(design system, Systems, platforms), what to ask the
|
||||
operator — once — and the decide_project_inception call that answers it;
|
||||
it repeats on every enter until a decision is recorded.
|
||||
|
||||
`systems_bootstrap` appears ONLY when the project has many records and no
|
||||
Systems at all — act on it before starting other work: create_system a
|
||||
starter vocabulary from the areas the project's records name, directly
|
||||
and without asking permission, preferring the standard names the ask
|
||||
lists. It stops appearing the moment the first System exists.
|
||||
|
||||
`design_system` is null unless the project points at one. When present it
|
||||
is a summary (title, what it inherits, token count and groups) and
|
||||
`guidance_call`. The guidance binds any UI you write the way a rule does:
|
||||
before writing UI, read `resolved_guidance` from get_design_system (the
|
||||
house style AND this project's departures from it), and pull values with
|
||||
resolve_design_system or get_design_system_stylesheet before reaching for
|
||||
a literal.
|
||||
|
||||
Entering a project also SCOPES the session: reference and offer work on
|
||||
this project only, and pass its id to search / list_* so results stay
|
||||
inside it. If something clearly belongs to a different project, say so and
|
||||
ask before switching — never silently operate cross-project. The active
|
||||
project does not stick on the server (each call is self-contained);
|
||||
carrying its id forward is on you.
|
||||
|
||||
Don't wait to be told which project is in scope. When work clearly belongs
|
||||
to a project but none is entered, look for a match yourself (list_projects
|
||||
/ search on the repo or subject), propose it, and enter it once the
|
||||
operator confirms; if nothing matches, offer to create one — confirming
|
||||
name and goal first, never guessing a project into existence.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
project = await projects_svc.get_project(uid, project_id)
|
||||
if project is None:
|
||||
raise ValueError(f"project {project_id} not found")
|
||||
|
||||
applicable = await rulebooks_svc.get_applicable_rules(
|
||||
project_id=project_id, user_id=uid,
|
||||
)
|
||||
milestone_rows = await milestones_svc.get_project_milestone_summary(
|
||||
uid, project_id,
|
||||
)
|
||||
milestone_summary, omitted = milestones_svc.brief_milestone_summary(
|
||||
milestone_rows, limit=_HANDSHAKE_MILESTONES,
|
||||
)
|
||||
milestone_titles = {m["id"]: m.get("title") for m in milestone_rows}
|
||||
unplanned, unplanned_omitted = milestones_svc.unplanned_milestones(
|
||||
milestone_rows,
|
||||
exclude_ids={m["id"] for m in milestone_summary},
|
||||
limit=_HANDSHAKE_UNPLANNED,
|
||||
)
|
||||
open_tasks, _ = await notes_svc.list_notes(
|
||||
uid, is_task=True, project_id=project_id,
|
||||
status=["todo", "in_progress"], sort="touched", limit=_HANDSHAKE_OPEN_TASKS,
|
||||
)
|
||||
# The tagging vocabulary. Surfaced HERE because an instruction to "tag
|
||||
# records to Systems" is only executable if the list is in front of the
|
||||
# agent when it writes. Untagged writes also ask with the vocabulary
|
||||
# listed; this copy lets the first write tag.
|
||||
systems = await systems_svc.list_systems(uid, project_id)
|
||||
platforms = platforms_svc.members(
|
||||
await platforms_svc.project_platforms(uid, project_id) or []
|
||||
)
|
||||
|
||||
# The arrival-moment half of the bootstrap ask: session start is
|
||||
# when the agent has just read the project map and is not yet deep in a
|
||||
# task — the one moment minting a starter vocabulary is cheap. The
|
||||
# write-moment half rides untagged-record responses (attach_systems);
|
||||
# both retire the instant the first System exists.
|
||||
systems_bootstrap = None
|
||||
if not systems:
|
||||
systems_bootstrap = await systems_tools.bootstrap_systems_ask(
|
||||
uid, project_id
|
||||
)
|
||||
|
||||
# The inception ask: a project nobody has decided on inherits nothing,
|
||||
# silently — no design system, no Systems. Owner-only (deciding is the
|
||||
# owner's), and only until a decision is recorded; the key is ABSENT
|
||||
# otherwise.
|
||||
inception_ask = None
|
||||
if project.user_id == uid and not inception_svc.is_decided(project):
|
||||
inception_ask = await inception_svc.inception_ask(uid, project_id)
|
||||
|
||||
# An AMBIENT source: top-N-by-recency, not a ranked choice, and
|
||||
# the readout counts it apart so dead-weight detection isn't poisoned by
|
||||
# "recently updated in a project you opened".
|
||||
record_surfaced(
|
||||
user_id=uid,
|
||||
note_ids=[int(t.id) for t in open_tasks],
|
||||
source="enter_project",
|
||||
project_id=project_id,
|
||||
)
|
||||
# A project need not have one, and most installs won't — null is ordinary
|
||||
# here, not a missing prerequisite. Summary only: the guidance is ~9k of
|
||||
# prose most sessions never use, so it's one call away.
|
||||
design_system = None
|
||||
if project.design_system_id:
|
||||
design = await design_systems_svc.design_context(
|
||||
uid, project.design_system_id,
|
||||
)
|
||||
if design:
|
||||
design_system = {
|
||||
k: design[k]
|
||||
for k in ("id", "title", "inherits_from", "token_count", "token_groups")
|
||||
}
|
||||
design_system["guidance_call"] = (
|
||||
f"get_design_system({design['id']}) → resolved_guidance"
|
||||
)
|
||||
|
||||
# Cache read ONLY — computing coverage moves a repo tarball and never
|
||||
# belongs in this request path. Null is the ordinary state (no forge, or
|
||||
# never computed); the line appears exactly when there is evidence. Read
|
||||
# on the OWNER's id: bindings and the cache live with the project owner.
|
||||
coverage = await coverage_svc.cached_coverage(
|
||||
project.user_id or uid, project_id
|
||||
)
|
||||
# Arrival self-seed: a ledger that is absent or stale refreshes in
|
||||
# the BACKGROUND — entering is the moment the number is wanted, and the
|
||||
# UI button must not be the only path. This enter stays fast; the next
|
||||
# one carries the line. Forge-less projects exit the seed quietly.
|
||||
spawn(
|
||||
coverage_svc.refresh_if_stale(
|
||||
project.user_id or uid, project_id, cached=coverage
|
||||
),
|
||||
site="enter_project.coverage_seed",
|
||||
)
|
||||
|
||||
out = {
|
||||
"project": {
|
||||
"id": project.id, "title": project.title,
|
||||
"status": project.status, "goal": project.goal,
|
||||
},
|
||||
"pattern_coverage": coverage_svc.coverage_line(coverage) if coverage else None,
|
||||
"systems": [{"id": s.id, "name": s.name} for s in systems],
|
||||
"platforms": platforms,
|
||||
"design_system": design_system,
|
||||
"milestone_summary": milestone_summary,
|
||||
**rulebooks_svc.rules_payload(
|
||||
applicable, user_id=uid, source="enter_project", brief=True,
|
||||
),
|
||||
"open_tasks": [
|
||||
{
|
||||
"id": t.id, "title": t.title, "status": t.status,
|
||||
"milestone_id": t.milestone_id,
|
||||
"milestone_title": milestone_titles.get(t.milestone_id),
|
||||
}
|
||||
for t in open_tasks
|
||||
],
|
||||
}
|
||||
# Attached only when it applies — a key that usually says null trains
|
||||
# readers to skip it, and this one exists to be acted on.
|
||||
if omitted:
|
||||
out["milestone_summary_omitted"] = (
|
||||
f"{omitted} other milestone(s) not listed. "
|
||||
f"list_milestones({project_id}) lists every milestone."
|
||||
)
|
||||
if unplanned:
|
||||
out["unplanned_milestones"] = unplanned
|
||||
if unplanned_omitted:
|
||||
out["unplanned_milestones_omitted"] = (
|
||||
f"{unplanned_omitted} more active milestone(s) with no steps. "
|
||||
f"list_milestones({project_id}) lists every milestone."
|
||||
)
|
||||
if systems_bootstrap:
|
||||
out["systems_bootstrap"] = systems_bootstrap
|
||||
# Family canon: counts only, attached when one is
|
||||
# non-zero. A readout must never fail the handshake it rides on.
|
||||
try:
|
||||
family = await family_adoption_svc.family_readout(uid, project_id)
|
||||
except Exception:
|
||||
logger.warning("family readout failed for project %s", project_id, exc_info=True)
|
||||
family = None
|
||||
if family:
|
||||
out["family"] = family
|
||||
if inception_ask:
|
||||
out["inception"] = inception_ask
|
||||
# The core reply shape: for a client with no prompt hook,
|
||||
# entering the project is the one point every session passes before it
|
||||
# writes a reply. A plugin session gets it with its first turn as well;
|
||||
# one repeat per session is the price of reaching every client.
|
||||
out["reply_shape"] = reply_shapes_svc.render(reply_shapes_svc.core(), full=True)
|
||||
return out
|
||||
|
||||
|
||||
async def get_project(project_id: int) -> dict:
|
||||
"""Fetch a Grimoire project by ID.
|
||||
|
||||
Returns full project fields, a milestone_summary list (every milestone,
|
||||
with description, progress and `next_step` — the earliest still-open step,
|
||||
or null — but no plan body; get_milestone reads a plan), the project's own
|
||||
rules (project_rules), and applicable_rules: the
|
||||
global rules tagged to an area this project works in. Every other global
|
||||
rule applies too and arrives by retrieval when the work matches it.
|
||||
`platforms` is every platform the project has an answer for, with its
|
||||
state — rejected ones included, unlike enter_project's brief list.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
project = await projects_svc.get_project(uid, project_id)
|
||||
if project is None:
|
||||
raise ValueError(f"project {project_id} not found")
|
||||
data = project.to_dict()
|
||||
data["platforms"] = await platforms_svc.project_platforms(uid, project_id) or []
|
||||
rows = await milestones_svc.get_project_milestone_summary(uid, project_id)
|
||||
data["milestone_summary"], _ = milestones_svc.brief_milestone_summary(rows)
|
||||
applicable = await rulebooks_svc.get_applicable_rules(
|
||||
project_id=project_id, user_id=uid,
|
||||
)
|
||||
data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_project"))
|
||||
return data
|
||||
|
||||
|
||||
def _inception_choices(design_system_id, seed_systems, platforms=None) -> dict | None:
|
||||
"""The tool args → an inception choices object, or None when no inception
|
||||
arg was given at all (a bare create stays undecided and enter_project
|
||||
asks). design_system_id: 0 = not stated, -1 = explicitly none, n = that
|
||||
system. platforms: None = not stated (memberships untouched), a list =
|
||||
the whole answer."""
|
||||
if not design_system_id and seed_systems is None and platforms is None:
|
||||
return None
|
||||
choices = {
|
||||
"design_system_id": None if design_system_id in (0, -1) else design_system_id,
|
||||
"seed_systems": bool(seed_systems),
|
||||
}
|
||||
if platforms is not None:
|
||||
choices["platforms"] = list(platforms)
|
||||
return choices
|
||||
|
||||
|
||||
async def create_project(
|
||||
title: str,
|
||||
description: str = "",
|
||||
goal: str = "",
|
||||
status: str = "active",
|
||||
color: str = "",
|
||||
design_system_id: int = 0,
|
||||
seed_systems: bool | None = None,
|
||||
platforms: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Create a new project in Grimoire — and decide what it inherits.
|
||||
|
||||
A project's inheritance is a decision, not a default:
|
||||
before calling, ask the operator the three inception questions and pass
|
||||
the answers; a project created without either is UNDECIDED and
|
||||
enter_project will ask until decide_project_inception records it.
|
||||
Defaults if nobody decides: no design system, no Systems. Rules are not
|
||||
an inception question: global rules (in rulebooks) apply to every
|
||||
project, and a project's own rules are written with create_project_rule.
|
||||
|
||||
Args:
|
||||
title: Project name (required).
|
||||
description: Short summary of what the project is.
|
||||
goal: The desired outcome or definition of done for the project.
|
||||
status: one of active (default), paused, completed, archived.
|
||||
color: Optional hex colour for the project card (e.g. "#6366f1").
|
||||
design_system_id: the design system this project's UI is built from
|
||||
(list_design_systems); -1 = explicitly none; 0 = not stated.
|
||||
seed_systems: true mints the standard starter Systems (CI & Release,
|
||||
Auth & Access, …) so records can be tagged from day one.
|
||||
platforms: slugs of what the project is built on or ships as
|
||||
(list_platforms) — they decide which family ideas reach it. The
|
||||
list is the whole answer; omit it to leave the question open.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
project = await projects_svc.create_project(
|
||||
uid,
|
||||
title=title,
|
||||
description=description,
|
||||
goal=goal,
|
||||
status=status,
|
||||
color=color or None,
|
||||
)
|
||||
data = project.to_dict()
|
||||
choices = _inception_choices(design_system_id, seed_systems, platforms)
|
||||
if choices is not None:
|
||||
decided = await inception_svc.decide(uid, project.id, choices=choices, via="mcp")
|
||||
data["inception"] = decided["inception"]
|
||||
data["inception_effects"] = decided["effects"]
|
||||
else:
|
||||
data["inception_hint"] = (
|
||||
"Undecided: this project inherits its defaults until "
|
||||
"decide_project_inception records what it should inherit "
|
||||
"(enter_project will ask)."
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
async def decide_project_inception(
|
||||
project_id: int,
|
||||
design_system_id: int = 0,
|
||||
seed_systems: bool | None = None,
|
||||
platforms: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Record what a project inherits — answer enter_project's `inception` ask,
|
||||
or re-decide later.
|
||||
|
||||
Owner-only. Applies the effects through the ordinary tools' paths —
|
||||
set_project_design_system and the standard Systems seed — and writes the
|
||||
decision on the project last, so get_project/enter_project can say why
|
||||
the project has the design and Systems it has. Re-deciding replaces the
|
||||
design system and never re-seeds Systems a project already has.
|
||||
|
||||
Args: as create_project's inception args. Passing nothing records a
|
||||
decision to take nothing (no design system, no seed) — a valid answer,
|
||||
stated — and leaves the project's platforms as they are.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
choices = _inception_choices(design_system_id, seed_systems, platforms) or {}
|
||||
decided = await inception_svc.decide(uid, project_id, choices=choices, via="mcp")
|
||||
return {"project_id": project_id, **decided}
|
||||
|
||||
|
||||
async def update_project(
|
||||
project_id: int,
|
||||
title: str = "",
|
||||
description: str = "",
|
||||
goal: str = "",
|
||||
status: str = "",
|
||||
color: str = "",
|
||||
) -> dict:
|
||||
"""Update an existing Grimoire project. Only explicitly provided fields are changed.
|
||||
|
||||
Args:
|
||||
project_id: ID of the project to update.
|
||||
title: New title, or omit to leave unchanged.
|
||||
description: New description, or omit to leave unchanged.
|
||||
goal: New goal/definition-of-done, or omit to leave unchanged.
|
||||
status: New status — one of active, paused, completed, archived.
|
||||
color: New hex colour, or omit to leave unchanged.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
fields: dict = {}
|
||||
if title:
|
||||
fields["title"] = title
|
||||
if description:
|
||||
fields["description"] = description
|
||||
if goal:
|
||||
fields["goal"] = goal
|
||||
if status:
|
||||
fields["status"] = status
|
||||
if color:
|
||||
fields["color"] = color
|
||||
project = await projects_svc.update_project(uid, project_id, **fields)
|
||||
if project is None:
|
||||
raise ValueError(f"project {project_id} not found")
|
||||
return project.to_dict()
|
||||
|
||||
|
||||
async def delete_project(project_id: int) -> dict:
|
||||
"""Move a project to the trash (recoverable). Its milestones, tasks, and notes
|
||||
go with it as one batch. Restore via restore(batch_id)."""
|
||||
uid = current_user_id()
|
||||
batch = await trash_svc.delete(uid, "project", project_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"project {project_id} not found")
|
||||
return {"deleted_batch_id": batch,
|
||||
"message": f"Project {project_id} + its contents moved to trash. Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_projects,
|
||||
enter_project,
|
||||
get_project,
|
||||
create_project,
|
||||
update_project,
|
||||
delete_project, decide_project_inception,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,74 @@
|
||||
"""get_recent — cross-type recent-activity tool.
|
||||
|
||||
Returns the most-recently-touched notes, tasks, and projects for the user,
|
||||
ordered by updated_at descending. Useful for Claude to bootstrap context at
|
||||
the start of a conversation ("what was I working on?").
|
||||
|
||||
Aggregation is Python-side after two small per-table queries — simpler than
|
||||
a UNION ALL with type-discriminating columns, and fine for personal-scale data.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.note import Note
|
||||
from grimoire.models.project import Project
|
||||
|
||||
|
||||
async def get_recent(days: int = 7, limit: int = 25) -> dict:
|
||||
"""Return recently-touched items across notes, tasks, and projects.
|
||||
|
||||
Args:
|
||||
days: Look-back window in days (1-90).
|
||||
limit: Maximum number of items returned (1-100).
|
||||
|
||||
Returns:
|
||||
{"items": [{"id", "type", "title", "updated_at"}], "total": int}
|
||||
Sorted by updated_at descending.
|
||||
|
||||
Scope note: this spans ALL projects and takes no project filter. When a
|
||||
project is in scope, prefer list_tasks(project_id=...) /
|
||||
list_notes(project_id=...) so you don't surface other projects' activity.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
days = max(1, min(days, 90))
|
||||
limit = max(1, min(limit, 100))
|
||||
since = datetime.now(timezone.utc) - timedelta(days=days)
|
||||
items: list[dict] = []
|
||||
async with async_session() as session:
|
||||
notes = (await session.execute(
|
||||
select(Note).where(Note.user_id == uid, Note.updated_at >= since,
|
||||
Note.deleted_at.is_(None))
|
||||
.order_by(Note.updated_at.desc()).limit(limit)
|
||||
)).scalars().all()
|
||||
for n in notes:
|
||||
items.append({
|
||||
"id": n.id,
|
||||
"type": "task" if n.is_task else "note",
|
||||
"title": n.title,
|
||||
"updated_at": n.updated_at.isoformat(),
|
||||
})
|
||||
projects = (await session.execute(
|
||||
select(Project).where(Project.user_id == uid,
|
||||
Project.updated_at >= since,
|
||||
Project.deleted_at.is_(None))
|
||||
.order_by(Project.updated_at.desc()).limit(limit)
|
||||
)).scalars().all()
|
||||
for p in projects:
|
||||
items.append({
|
||||
"id": p.id,
|
||||
"type": "project",
|
||||
"title": p.title,
|
||||
"updated_at": p.updated_at.isoformat(),
|
||||
})
|
||||
items.sort(key=lambda r: r["updated_at"], reverse=True)
|
||||
items = items[:limit]
|
||||
return {"items": items, "total": len(items)}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
mcp.tool(name="get_recent")(get_recent)
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Repo -> project binding MCP tools.
|
||||
|
||||
Let the operator map the working git repository to a Grimoire project so the
|
||||
SessionStart hook auto-loads that project's context — without pinning a project
|
||||
id in plugin config (which would force a single project for every repo). Run
|
||||
`bind_repo` once per repo; the binding is keyed on the normalized git remote, so
|
||||
it survives re-clones and ssh/https URL differences.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import projects as projects_svc
|
||||
from grimoire.services import repo_bindings as repo_bindings_svc
|
||||
|
||||
|
||||
async def bind_repo(repo_url: str, project_id: int, ref: str = "") -> dict:
|
||||
"""Bind a git repository to a Grimoire project for session-start context.
|
||||
|
||||
After this, any session started in that repo auto-loads the project's
|
||||
context (the SessionStart hook sends the repo's remote; the server resolves
|
||||
it here). Idempotent — re-binding the same repo updates the target project.
|
||||
|
||||
The binding is also what the shape ledger reads (refresh_pattern_coverage):
|
||||
`ref` names the branch it follows. Default (""): the repo's default branch
|
||||
— which means the ledger only sees work after a merge. A project that
|
||||
works on a dev branch should bind with ref="dev" so classification follows
|
||||
the push, not the merge. Re-binding with ref="" keeps the standing ref;
|
||||
pass ref="-" to clear it back to the default branch.
|
||||
|
||||
Args:
|
||||
repo_url: the repo's git remote (e.g. the output of
|
||||
`git remote get-url origin` — ssh or https form, both work).
|
||||
project_id: the Grimoire project this repo represents.
|
||||
ref: branch the ledger follows ("" = leave as is / default branch on
|
||||
a new binding; "-" = clear to the default branch).
|
||||
"""
|
||||
uid = current_user_id()
|
||||
project = await projects_svc.get_project(uid, project_id)
|
||||
if project is None:
|
||||
raise ValueError(f"project {project_id} not found")
|
||||
ref_arg = None if not ref else ("" if ref.strip() == "-" else ref)
|
||||
binding = await repo_bindings_svc.set_binding(uid, repo_url, project_id, ref_arg)
|
||||
follows = binding.ref or "the default branch"
|
||||
return {
|
||||
"repo_key": binding.repo_key,
|
||||
"project_id": binding.project_id,
|
||||
"project_title": project.title,
|
||||
"ref": binding.ref,
|
||||
"message": (
|
||||
f"Bound `{binding.repo_key}` -> {project.title} (id {project.id}); "
|
||||
f"the ledger follows {follows}."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
async def list_repo_bindings() -> dict:
|
||||
"""List every repo -> project binding for the current user."""
|
||||
uid = current_user_id()
|
||||
bindings = await repo_bindings_svc.list_bindings(uid)
|
||||
return {"bindings": [b.to_dict() for b in bindings]}
|
||||
|
||||
|
||||
async def unbind_repo(repo_url: str) -> dict:
|
||||
"""Remove a repo's binding. Sessions there fall back to standing rules only.
|
||||
|
||||
Args:
|
||||
repo_url: the repo's git remote (same form used to bind it).
|
||||
"""
|
||||
uid = current_user_id()
|
||||
removed = await repo_bindings_svc.delete_binding(uid, repo_url)
|
||||
key = repo_bindings_svc.normalize_repo_key(repo_url)
|
||||
return {
|
||||
"removed": removed,
|
||||
"repo_key": key,
|
||||
"message": (
|
||||
f"Unbound `{key}`." if removed else f"No binding found for `{key}`."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (bind_repo, list_repo_bindings, unbind_repo):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Judging what a retrieval arm offered.
|
||||
|
||||
`retrieval_telemetry` says what the reader did with each line — opened it or
|
||||
not. These two say whether the line RELATED, which is a reading only someone
|
||||
looking at the query beside the line can do.
|
||||
"""
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import retrieval_review as review_svc
|
||||
|
||||
|
||||
async def menus_to_review(
|
||||
source: str = "auto_inject", n: int = 5, days: int = 14, depth: int = 6,
|
||||
) -> dict:
|
||||
"""A sample of logged injection menus, re-run so you can judge each line.
|
||||
|
||||
Reach for this before moving a budget or a floor, and whenever an open rate
|
||||
is about to be read as a verdict on relevance. It is not one: every menu
|
||||
line carries the passage that matched, so a record left unopened may have
|
||||
been unrelated, enough as shown, or already in context, and the usage
|
||||
counters cannot tell those apart. A judged sample can.
|
||||
|
||||
Each menu is one logged call: the query the arm searched (enriched, as it
|
||||
was asked), its `budget`, and `lines` — the query re-run with the arm's own
|
||||
parameters to `depth`, each line with `rank`, `score`, `kind`, `name`, the
|
||||
`passage` that matched, `within_budget` (was it inside the cut) and
|
||||
`opened_after` (did your agent open it within an hour). Lines past the
|
||||
budget are there on purpose: they are what a larger budget would add.
|
||||
|
||||
`missing` names ids the call logged that the re-run no longer finds — the
|
||||
corpus has moved since. Judge what is in front of you.
|
||||
|
||||
The re-run is against today's corpus, so it guards what has been written
|
||||
since. Records created after the call are left out and named in
|
||||
`postdated`: the call could not have offered them, and the session that
|
||||
made it often wrote them, quoting it. A line marked `changed_since_call`
|
||||
was written to afterwards; if its passage is plainly the later text, leave
|
||||
it unjudged. `logged` says the line was shown fresh at the time.
|
||||
|
||||
JUDGE FROM THE LINE; DO NOT OPEN THE RECORD. Opening counts as a pull and
|
||||
would credit the arm with an open it never earned. Then record verdicts
|
||||
with `judge_menu`. `how_to_judge` in the response repeats the vocabulary.
|
||||
|
||||
Args:
|
||||
source: the arm to review. Only arms whose search can be re-run
|
||||
exactly are offered; today that is `auto_inject`.
|
||||
n: how many calls to sample, 1–20. Random among the calls you have not
|
||||
judged, so a review does not describe one afternoon.
|
||||
days: how far back to sample from.
|
||||
depth: how many candidates to re-run per call, 1–10. Past the budget,
|
||||
so the cut itself can be judged.
|
||||
"""
|
||||
return await review_svc.menus_to_review(
|
||||
current_user_id(), source=source, n=n, days=days, depth=depth,
|
||||
)
|
||||
|
||||
|
||||
async def judge_menu(log_id: int, verdicts: list[dict]) -> dict:
|
||||
"""Record your verdict on lines of one menu from `menus_to_review`.
|
||||
|
||||
`verdicts` is a list of `{record_id, verdict, reason}`:
|
||||
|
||||
"on_point" — someone in the query's situation should read this.
|
||||
"adjacent" — the same area, but it would not change what they do.
|
||||
"unrelated" — close in words only.
|
||||
|
||||
`reason` is REQUIRED for every verdict: what in the query and the passage
|
||||
decided it. A verdict nobody can re-read is a threshold, and this exists to
|
||||
replace one. Rank, score and which side of the budget the line sat are
|
||||
taken from a fresh re-run, not from you. Judging a line again replaces your
|
||||
earlier verdict on it.
|
||||
|
||||
The verdicts are read back as `retrieval_telemetry`'s `judged` block.
|
||||
"""
|
||||
return await review_svc.judge_menu(
|
||||
current_user_id(), log_id=log_id, verdicts=verdicts,
|
||||
)
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
mcp.tool(name="menus_to_review")(menus_to_review)
|
||||
mcp.tool(name="judge_menu")(judge_menu)
|
||||
@@ -0,0 +1,189 @@
|
||||
"""Reading and moving a retrieval surface's floor and budget.
|
||||
|
||||
The write half of the loop `retrieval_telemetry` opens. That tool says what a
|
||||
bar did; these say what the bar IS, and let it be changed with the argument
|
||||
attached.
|
||||
"""
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import retrieval_migration as migration_svc
|
||||
from grimoire.services import retrieval_tuning as tuning_svc
|
||||
|
||||
|
||||
async def retrieval_surfaces() -> dict:
|
||||
"""Every retrieval surface Grimoire pushes on, with its floor and its budget.
|
||||
|
||||
Read this before changing either, and read it beside
|
||||
`retrieval_telemetry(days=…, near_miss_samples=5)` — this tool says what is
|
||||
in force, that one says what it did.
|
||||
|
||||
Each surface carries what it ASKS, what corpus it asks OVER, and how often
|
||||
it FIRES, because a floor cannot be moved sensibly without all three. An arm
|
||||
firing before every Bash call is spending attention on a scale an arm firing
|
||||
once a turn is not, and the same number means different things to a query
|
||||
that is a shell command, a code payload, or an operator's sentence.
|
||||
|
||||
`floor` is a COST floor: is this worth ranking at all. It is not a relevance
|
||||
judgement — relevance is decided by the reader, which is the only
|
||||
participant that can read a record's trigger against the actual situation,
|
||||
and every injected line says so out loud ("read it before deciding it does
|
||||
not apply").
|
||||
|
||||
`budget` is what actually binds. It is how many lines this surface may spend
|
||||
on one injection, and it is the control to reach for when a surface feels
|
||||
noisy — lowering a budget removes the weakest candidates, while raising a
|
||||
floor removes whichever candidates happen to sit under a number.
|
||||
|
||||
`last_change` carries the reason last given for each dial, or is absent when
|
||||
the surface is still on its shipped starting point. Those starting points
|
||||
are starting points: they were measured against one corpus with one
|
||||
embedding model and cannot be right for another install by construction.
|
||||
That is why this tool exists rather than a better set of defaults.
|
||||
|
||||
`calibration` says what SPACE each number was chosen in, per dial, and
|
||||
whether that space has moved. A floor is a cosine distance in one embedding
|
||||
model's geometry over documents cut one particular way; swap the model and
|
||||
every score shifts at once, re-cut the documents and the same record embeds
|
||||
different text. Either way the number is a measurement of something that no
|
||||
longer exists, and no amount of telemetry will say so — the scores simply
|
||||
come out different and the bar keeps applying.
|
||||
|
||||
Read the three fields apart:
|
||||
|
||||
- `stale: true` with `model_changed` — the geometry is new. Every floor on
|
||||
this install needs re-measuring, not adjusting; a number that meant "quite
|
||||
similar" in the old space means nothing particular in this one.
|
||||
- `stale: true` with `shape_changed` — the documents are cut differently.
|
||||
The scale still holds, but what a record embeds has changed, so which
|
||||
records clear a bar has.
|
||||
- `stale: null`, `source: "unstamped"` — the dial was moved before Grimoire
|
||||
recorded this. It was measured under SOMETHING and there is no way to say
|
||||
what, which is a different answer from "it is fine". Treat it as worth
|
||||
re-measuring, and the next tuning call stamps it.
|
||||
|
||||
NOTHING IS RETUNED AUTOMATICALLY on the strength of this, here or anywhere.
|
||||
A stale stamp says a number is no longer a measurement; it does not say what
|
||||
the number should be. That judgement wants the same procedure as any other
|
||||
tuning change — `retrieval_telemetry(near_miss_samples=5)`, open the records
|
||||
it names, then `tune_retrieval` with what you read in the reason.
|
||||
"""
|
||||
return {"surfaces": await tuning_svc.current_settings(current_user_id())}
|
||||
|
||||
|
||||
async def tune_retrieval(
|
||||
surface: str, dial: str, value: float, reason: str, actor: str = "model",
|
||||
) -> dict:
|
||||
"""Move one surface's floor or budget, recording why.
|
||||
|
||||
DO NOT CALL THIS FROM A PERCENTILE ALONE. `retrieval_telemetry` gives
|
||||
`near_misses.p90` — the mass sitting just under the bar — and that number
|
||||
says nothing about whether the mass is RELEVANT. The two have been measured
|
||||
disagreeing: one surface logged 69 consecutive declines with the refused
|
||||
record 0.0006 under its bar, and every percentile said "lower it". The
|
||||
refused record turned out to be a rule about interpreting a REQUEST, matched
|
||||
against a query about report layout — a false positive. Lowering would have
|
||||
attached that rule to every completion report ever written. The statistic
|
||||
and the correct action pointed in opposite directions, and only opening the
|
||||
record could tell.
|
||||
|
||||
So the procedure is: `retrieval_telemetry(near_miss_samples=5)`, then
|
||||
`get_rule` / `get_note` the `record_id`s it names, and decide whether those
|
||||
records SHOULD have surfaced for those queries. Then move the dial, and say
|
||||
in `reason` what you read and what it showed.
|
||||
|
||||
`reason` is required and must be substantive. It is the guardrail, not
|
||||
bookkeeping: it is what lets the operator review a change they did not make,
|
||||
disagree with it, and revert it. A number that moved with no stated basis is
|
||||
one nobody can audit — including you, next week.
|
||||
|
||||
WHICH DIAL. Reach for `budget` when the complaint is volume, and `floor`
|
||||
when the complaint is quality. Raising a floor to quieten a surface throws
|
||||
away its best candidates along with its worst, because a floor cannot tell
|
||||
rank from relevance; lowering a budget keeps the top of the ranking and
|
||||
drops the tail, which is usually what was wanted.
|
||||
|
||||
Args:
|
||||
surface: the surface name from `retrieval_surfaces` — also its
|
||||
`retrieval_telemetry` source, so the two always name the same arm.
|
||||
dial: "floor" or "budget".
|
||||
value: the new value. A floor is clamped to [0, 1], a budget to a whole
|
||||
number in [1, 10]. The result says whether a clamp bit, which
|
||||
matters: believing you set a value the registry corrected means
|
||||
reading the next telemetry as evidence about a bar never in force.
|
||||
reason: what you read and what it showed. Required.
|
||||
actor: "model" (default) or "human" — pass "human" only when relaying a
|
||||
value the operator chose themselves, so the audit trail can tell
|
||||
a change they made from one made on their behalf.
|
||||
"""
|
||||
return await tuning_svc.set_dial(
|
||||
current_user_id(), surface, dial, value, reason=reason, actor=actor,
|
||||
)
|
||||
|
||||
|
||||
async def retrieval_tuning_history(surface: str = "", limit: int = 20) -> dict:
|
||||
"""What has been changed about retrieval on this install, newest first.
|
||||
|
||||
Reach for it before moving a dial that has been moved before — the previous
|
||||
reason is the argument the next change has to answer, and a surface that has
|
||||
been walked up and down repeatedly is evidence the floor is not the problem.
|
||||
|
||||
Args:
|
||||
surface: restrict to one surface; omit for everything.
|
||||
limit: how many events, default 20, capped at 200.
|
||||
"""
|
||||
return {
|
||||
"events": await tuning_svc.tuning_history(
|
||||
current_user_id(), surface=surface or None, limit=limit,
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
async def migrate_retrieval_floor(
|
||||
surface: str, sample: int = 200, apply: bool = False,
|
||||
) -> dict:
|
||||
"""Carry one surface's floor across an embedding-model or chunker change.
|
||||
|
||||
Reach for this when `retrieval_surfaces` reports a dial `stale` — and only
|
||||
then. A floor is a cosine similarity, so a new embedding model moves every
|
||||
score on the install at once and the stored number silently stops describing
|
||||
anything. Re-deriving six floors by reading telemetry is the work this
|
||||
avoids.
|
||||
|
||||
WHAT IT TRANSFERS. Not the number — the SELECTIVITY. A floor's real content
|
||||
is a decision about how much of what an arm sees is worth spending attention
|
||||
on, and that decision survives a change of units. This measures what fraction
|
||||
of the surface's recent calls the old floor admitted, re-scores those same
|
||||
queries under the current model, and proposes the value that admits the same
|
||||
fraction.
|
||||
|
||||
DRY RUN BY DEFAULT. With `apply=False` it returns the arithmetic and writes
|
||||
nothing. Read it before applying: the sample sizes, both score ranges, and
|
||||
whether the shift looks like a change of scale or like a corpus that has not
|
||||
finished re-embedding. A model change is the moment every number is
|
||||
uncertain at once, which is the worst moment to let a statistic move dials
|
||||
unattended.
|
||||
|
||||
AND IT IS STILL A STARTING POINT. Percentile-preserving means the new floor
|
||||
is as good as the old one was, not better — if the old floor was wrong, this
|
||||
faithfully carries the wrongness onto the new scale. It is the number to
|
||||
begin from while the surface collects enough calls to judge properly, which
|
||||
is `retrieval_telemetry(near_miss_samples=5)` and `tune_retrieval` as usual.
|
||||
|
||||
Args:
|
||||
surface: the surface name from `retrieval_surfaces`.
|
||||
sample: how many recent logged calls to re-score, default 200. Each one
|
||||
is an embedding plus a scan, so this is real work; a surface with
|
||||
only a handful of logged calls gives a percentile made of noise.
|
||||
apply: False (default) returns the proposal. True writes it, as an
|
||||
ordinary tuning event with the arithmetic in its reason — so a
|
||||
migrated floor is reviewable and revertible like any other.
|
||||
"""
|
||||
return await migration_svc.migrate_floor(
|
||||
current_user_id(), surface, sample=sample, apply=apply,
|
||||
)
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
mcp.tool(name="retrieval_surfaces")(retrieval_surfaces)
|
||||
mcp.tool(name="migrate_retrieval_floor")(migrate_retrieval_floor)
|
||||
mcp.tool(name="tune_retrieval")(tune_retrieval)
|
||||
mcp.tool(name="retrieval_tuning_history")(retrieval_tuning_history)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,656 @@
|
||||
"""search — semantic search across the user's notes and tasks.
|
||||
|
||||
Mirrors the existing fable-mcp contract so Claude's prior usage pattern keeps
|
||||
working. Differences from fable-mcp:
|
||||
- calls services.embeddings.semantic_search_notes directly instead of HTTP
|
||||
- user_id comes from mcp.current_user_id() rather than a global API key
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services.access import owner_names_for
|
||||
from grimoire.services.knowledge import content_type_filters
|
||||
from grimoire.services.text import MATCHED_PASSAGE, excerpt_fields
|
||||
from grimoire.services.embeddings import (
|
||||
DEFAULT_SIMILARITY_THRESHOLD, semantic_search_milestones, semantic_search_notes,
|
||||
semantic_search_rules, semantic_search_systems,
|
||||
)
|
||||
from grimoire.services import rulebooks as rulebooks_svc
|
||||
from grimoire.services.retrieval_telemetry import record_retrieval, retrieval_summary
|
||||
|
||||
|
||||
# A matched chunk is at most _CHUNK_CHAR_BUDGET (1400) characters, and it is
|
||||
# the evidence the ranking was built on, so it gets most of that room. Elision
|
||||
# inside a chunk is far less lossy than a head cut of a whole record: the
|
||||
# region is already the right one.
|
||||
_EXCERPT_CHARS = 1000
|
||||
|
||||
|
||||
# The kinds `content_type` accepts are DERIVED from the facet table, not listed
|
||||
# again here — that table is where a kind is declared, and a second hand-kept
|
||||
# copy here would let the agent's door and browse offer different kinds.
|
||||
# `content_type_filters` carries
|
||||
# the mapping, the 'all'/'note' special cases and the refusal.
|
||||
#
|
||||
# These two do not reach `semantic_search_notes` at all: they have their own
|
||||
# search and their own result shape, so they are dispatched before the mapping
|
||||
# and passed in only so the refusal message lists everything THIS door takes.
|
||||
_OWN_SEARCH = ("rule", "milestone", "system")
|
||||
|
||||
|
||||
async def _search_rules(uid: int, q: str, limit: int, project_id: int) -> dict:
|
||||
"""Rules by meaning — a separate result shape because a rule IS different.
|
||||
|
||||
A rule hit carries `why` and `how_to_apply`: they are the operational half
|
||||
of a rule and the session-start payload never includes them, so a caller
|
||||
who went looking should get the whole thing rather than a summary they then
|
||||
have to re-fetch. It also carries the rule's check (`verify_with`,
|
||||
`expires_when`, `last_verified`) when it has one — a search hit is exactly
|
||||
the moment someone is about to act on a rule, and "this asserts a fact
|
||||
nobody has confirmed" is part of what the rule says.
|
||||
|
||||
`project_id` scopes the way it does for notes, with one difference: a
|
||||
GLOBAL rule (one in a rulebook) belongs to no project and applies in every
|
||||
one, so a scoped search returns global rules plus that project's own.
|
||||
Without a project it asks the whole rulebook — every rule, whatever its
|
||||
home — because that is the question an unscoped "is there a rule about
|
||||
this" is asking. `system_id` does not apply to rules.
|
||||
"""
|
||||
if project_id:
|
||||
raw = await semantic_search_rules(uid, q, limit=limit, project_id=project_id)
|
||||
else:
|
||||
raw = await semantic_search_rules(uid, q, limit=limit, everywhere=True)
|
||||
return {
|
||||
"results": [
|
||||
{
|
||||
"id": rule.id,
|
||||
"title": rule.title,
|
||||
"statement": rule.statement,
|
||||
"when_to_apply": rule.when_to_apply or "",
|
||||
"why": rule.why or "",
|
||||
"how_to_apply": rule.how_to_apply or "",
|
||||
"verify_with": rule.verify_with or "",
|
||||
"expires_when": rule.expires_when or "",
|
||||
# Only on a rule that carries a check; its absence means the
|
||||
# rule is a decision, not that nobody has looked.
|
||||
**(
|
||||
{"last_verified": rulebooks_svc.last_verified_label(rule)}
|
||||
if rule.verify_with else {}
|
||||
),
|
||||
"topic_id": rule.topic_id,
|
||||
"project_id": rule.project_id,
|
||||
"similarity": float(score),
|
||||
}
|
||||
for score, rule in raw
|
||||
],
|
||||
"total": len(raw),
|
||||
}
|
||||
|
||||
|
||||
async def _search_milestones(uid: int, q: str, limit: int, project_id: int) -> dict:
|
||||
"""Milestones by meaning — "is there already a plan for this?".
|
||||
|
||||
Its own result shape, like rules: a milestone is a plan with progress, not
|
||||
a note with a body. The plan itself is left out — get_milestone reads it —
|
||||
because a search hit is for recognising a plan, and bodies run long. What
|
||||
does come along is `matched`: the one passage of the body the query
|
||||
actually hit, with `matched_is` saying whether it is that passage or
|
||||
merely the body's opening. Recognising a plan means recognising the part
|
||||
of it you were asking about, and a description written at the start need
|
||||
not mention that part.
|
||||
Not part of content_type="all", whose results are note-shaped.
|
||||
"""
|
||||
report: dict = {}
|
||||
raw = await semantic_search_milestones(
|
||||
uid, q, project_id=project_id or None, limit=limit, report=report,
|
||||
)
|
||||
chunks = report.get("best_chunk") or {}
|
||||
progress: dict[int, dict] = {}
|
||||
if raw:
|
||||
from grimoire.services import milestones as milestones_svc
|
||||
|
||||
for pid in {m.project_id for _s, m in raw}:
|
||||
for row in await milestones_svc.get_project_milestone_summary(uid, pid):
|
||||
progress[row["id"]] = row
|
||||
return {
|
||||
"results": [
|
||||
{
|
||||
"id": m.id,
|
||||
"title": m.title,
|
||||
"description": m.description or "",
|
||||
# The plan body stays out — get_milestone reads it — but the
|
||||
# passage that MATCHED comes along, because a plan is
|
||||
# recognised by the part of it the query was about and a
|
||||
# description need not mention that part at all.
|
||||
**excerpt_fields(
|
||||
m.body or "", chunks.get(int(m.id)), _EXCERPT_CHARS,
|
||||
key="matched",
|
||||
),
|
||||
"status": m.status,
|
||||
"project_id": m.project_id,
|
||||
"total": progress.get(m.id, {}).get("total", 0),
|
||||
"completed": progress.get(m.id, {}).get("completed", 0),
|
||||
"similarity": float(score),
|
||||
}
|
||||
for score, m in raw
|
||||
],
|
||||
"total": len(raw),
|
||||
}
|
||||
|
||||
|
||||
async def _search_systems(uid: int, q: str, limit: int, project_id: int) -> dict:
|
||||
"""Systems by meaning — "where does this belong?".
|
||||
|
||||
A System's `description` is a charter: several hundred words saying what
|
||||
belongs in that area and what does not. `list_systems` enumerates them and
|
||||
`system_id` filters by one; this is the search where a System is the
|
||||
answer, so an agent asking where a record belongs need not read every
|
||||
charter.
|
||||
|
||||
Its own result shape and its own search, not a `content_type` over notes,
|
||||
because the question is different: "where does this belong?" is not "what
|
||||
prior art is there?". A charter competing with the whole note corpus for
|
||||
one top-k would also be outranked by the records filed under it, and the
|
||||
right answer would be crowded out by its own contents.
|
||||
|
||||
The charter's `matched` passage comes along rather than the whole thing.
|
||||
A charter is long and the paragraph that decides where a record belongs is
|
||||
the one worth reading; the rest is get_system.
|
||||
|
||||
The id that comes back is the one `search(system_id=…)`,
|
||||
`list_system_records` and every `system_ids` argument take — so the answer
|
||||
to "where does this belong?" is directly usable as "show me what is there"
|
||||
and as "file it here".
|
||||
"""
|
||||
report: dict = {}
|
||||
raw = await semantic_search_systems(
|
||||
uid, q, project_id=project_id or None, limit=limit, report=report,
|
||||
)
|
||||
chunks = report.get("best_chunk") or {}
|
||||
return {
|
||||
"results": [
|
||||
{
|
||||
"id": sys_.id,
|
||||
"name": sys_.name,
|
||||
**excerpt_fields(
|
||||
sys_.description or "", chunks.get(int(sys_.id)),
|
||||
_EXCERPT_CHARS, key="matched",
|
||||
),
|
||||
"project_id": sys_.project_id,
|
||||
"similarity": float(score),
|
||||
}
|
||||
for score, sys_ in raw
|
||||
],
|
||||
"total": len(raw),
|
||||
}
|
||||
|
||||
|
||||
def result_excerpt(note, chunk: dict | None) -> dict:
|
||||
"""The part of a record a caller judges "should I open this?" on.
|
||||
|
||||
The document's opening is the wrong span. The match is semantic and
|
||||
per-chunk, and `semantic_search_notes` collapses to best-chunk-per-note, so
|
||||
the passage that earned the hit is known. A record can rank first on its
|
||||
sixth paragraph; previewing its first would show the caller the part the
|
||||
search judged less relevant. When the span is cut, the result says so.
|
||||
|
||||
The choice of span lives in services/text.py, shared with the web's
|
||||
knowledge search, so the two doors cannot drift on which text a reader is
|
||||
shown or on whether they are told what it is.
|
||||
"""
|
||||
out = excerpt_fields(note.body or "", chunk, _EXCERPT_CHARS)
|
||||
if out.get("excerpt_is") == MATCHED_PASSAGE and (chunk or {}).get("index") is not None:
|
||||
out["chunk_index"] = int(chunk["index"])
|
||||
return out
|
||||
|
||||
|
||||
async def search(
|
||||
q: str,
|
||||
content_type: str = "all",
|
||||
limit: int = 10,
|
||||
project_id: int = 0,
|
||||
system_id: int = 0,
|
||||
) -> dict:
|
||||
"""Semantic search over the user's existing notes and tasks — Grimoire's recall.
|
||||
|
||||
Reach for this BEFORE answering a question about the user's work or starting
|
||||
a task: the user's second-brain almost always already holds related prior
|
||||
art. Check for an existing ticket before opening a new one (search with
|
||||
content_type='task'), and for prior notes/decisions before re-deriving them.
|
||||
Treating Grimoire as the first place to look — not a place to only write — is
|
||||
the difference between it being a trustworthy record and a write-only log.
|
||||
|
||||
Args:
|
||||
q: search query string.
|
||||
content_type: which kind of record to search. 'all' (default) spans
|
||||
every note and task.
|
||||
|
||||
THE BROAD TWO: 'note' is any non-task record — it still includes
|
||||
snippets, lessons and processes, so it means "knowledge, not work
|
||||
items". 'task' is any task whatever its kind.
|
||||
|
||||
THE SPECIFIC KINDS, each narrowing to one: 'snippet' (recorded
|
||||
prior art — reach for this BEFORE writing a helper, rather than
|
||||
searching 'all' and reading past the issues), 'lesson' (a
|
||||
transferable insight, the kind that exists to be recalled by
|
||||
situation), 'process' (a stored procedure the operator saved),
|
||||
'issue' (corrective work — "has this already been reported?"),
|
||||
'spike' (a time-boxed investigation, whose output is an answer),
|
||||
'work', and 'plan' (retired; the ~90 legacy plan-tasks).
|
||||
|
||||
An unrecognised value is REFUSED with the list of valid ones
|
||||
rather than quietly returning nothing: an empty result set is a
|
||||
claim that the corpus holds nothing, and a typo must not be able
|
||||
to make that claim.
|
||||
|
||||
THREE KINDS WITH THEIR OWN SEARCH AND THEIR OWN RESULT SHAPE,
|
||||
because each answers a question no note search can:
|
||||
'rule' (the operator's standing instructions), 'milestone' ("is there already a
|
||||
plan for this?"), and 'system' ("where does this belong?" — a
|
||||
System's charter says what belongs in an area and what does not,
|
||||
and the id it returns is the one `system_id` and `system_ids`
|
||||
take).
|
||||
Reach for 'rule' when you want to know whether a standing
|
||||
instruction covers something: "is there a rule about release
|
||||
tagging?". A hit carries the rule's `why` and `how_to_apply`,
|
||||
which the session-start payload does not. With a project_id,
|
||||
rules come back as the global rules plus that project's own;
|
||||
with 0, every rule in the rulebook. Or 'milestone' (PLANS):
|
||||
reach for it before start_planning to ask whether a plan for
|
||||
this work already exists — a match is where new steps go
|
||||
(create_records(milestone_id=…)), not a reason to open a second
|
||||
milestone. Hits carry title, description, status and progress;
|
||||
get_milestone reads the plan. Not included in 'all'.
|
||||
limit: maximum number of results (1-50).
|
||||
project_id: Scope results to one project. PASS THE ACTIVE PROJECT'S ID
|
||||
whenever a project is in scope (the one you entered with
|
||||
enter_project) — otherwise this searches across ALL projects and
|
||||
bleeds unrelated work into the result set. 0 = search everything
|
||||
(use only when you genuinely want a cross-project sweep).
|
||||
A LESSON is the exception and arrives whatever the scope: the kind
|
||||
records an insight that transfers, so it is reachable from a
|
||||
project it was not written on.
|
||||
system_id: Narrow to records tagged to one System (a named
|
||||
subsystem/area — enter_project lists them). Use when investigating
|
||||
a specific subsystem: it cuts the candidates to records someone
|
||||
deliberately filed under that area. 0 = no system filter.
|
||||
list_system_records gives the same slice unranked.
|
||||
|
||||
Returns:
|
||||
{"results": [{"id", "title", "excerpt", "excerpt_is", "body_length",
|
||||
"is_task", "tags", "similarity"}],
|
||||
"total": int}
|
||||
|
||||
`excerpt` is a SPAN of the record, not the record. `excerpt_is` says
|
||||
which span: "matched_passage" is the chunk that actually earned the
|
||||
hit — the evidence the ranking was built on, and the right thing to
|
||||
judge relevance from. "body_opening" is a fallback for a record with
|
||||
no stored chunk, and is only the beginning of the text, which may say
|
||||
nothing about why it matched. `body_length` is the whole record's
|
||||
size, so a long record previewed by a short span is visible as one;
|
||||
`read_full` appears when there is more, and opening the id by
|
||||
get_note / get_task is how you get it. Judge from the passage, not
|
||||
from the fact that a preview looked thin.
|
||||
|
||||
A result marked `shared: true` with an `owner` belongs to another user —
|
||||
that person's suggestion, not the operator's own record or settled practice.
|
||||
Weigh it on its merits and say whose it is when you use it.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
limit = max(1, min(limit, 50))
|
||||
if content_type == "rule":
|
||||
return await _search_rules(uid, q, limit, project_id)
|
||||
if content_type == "milestone":
|
||||
return await _search_milestones(uid, q, limit, project_id)
|
||||
if content_type == "system":
|
||||
return await _search_systems(uid, q, limit, project_id)
|
||||
filters = content_type_filters(content_type, extra=_OWN_SEARCH)
|
||||
is_task = filters.get("is_task")
|
||||
t0 = time.perf_counter()
|
||||
report: dict = {}
|
||||
raw = await semantic_search_notes(
|
||||
uid, q, limit=limit, **filters,
|
||||
project_id=project_id or None,
|
||||
system_id=system_id or None,
|
||||
# A LESSON is reachable from any project. The kind
|
||||
# exists to carry an insight to the next project, so a project filter
|
||||
# that hid it would hide it precisely where it is worth having. Only
|
||||
# the project filter widens — everything else about the scoping holds,
|
||||
# and a caller narrowing by `content_type` still gets what it asked
|
||||
# for. This is the explicit search, where the operator asked; the
|
||||
# unasked-for arms decide their own budget separately.
|
||||
include_global_kinds=True,
|
||||
# An explicit search reaches everything the operator may read, including
|
||||
# records shared with them one-to-one.
|
||||
scope="read",
|
||||
report=report,
|
||||
)
|
||||
record_retrieval(
|
||||
user_id=uid, source="mcp_search", query=q,
|
||||
threshold=DEFAULT_SIMILARITY_THRESHOLD, limit=limit,
|
||||
project_id=project_id or None, is_task=is_task, results=raw,
|
||||
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
||||
best_available=report.get("best_available_score"),
|
||||
best_available_id=report.get("best_available_id"),
|
||||
searched=bool(report.get("searched", True)),
|
||||
)
|
||||
owners = await owner_names_for(
|
||||
{int(note.user_id) for _s, note in raw if note.user_id != uid}
|
||||
)
|
||||
chunks = report.get("best_chunk") or {}
|
||||
return {
|
||||
"results": [
|
||||
{
|
||||
"id": note.id,
|
||||
"title": note.title,
|
||||
**result_excerpt(note, chunks.get(int(note.id))),
|
||||
"is_task": bool(note.is_task),
|
||||
"tags": list(note.tags or []),
|
||||
"similarity": float(score),
|
||||
**(
|
||||
{"shared": True, "owner": owners.get(int(note.user_id))}
|
||||
if note.user_id != uid else {}
|
||||
),
|
||||
}
|
||||
for score, note in raw
|
||||
],
|
||||
"total": len(raw),
|
||||
}
|
||||
|
||||
|
||||
async def retrieval_telemetry(
|
||||
days: int = 30, near_miss_samples: int = 0,
|
||||
) -> dict:
|
||||
"""What the retrieval telemetry says about YOUR surfaces, over a window.
|
||||
|
||||
The read half of the tuning loop, whose write half is `tune_retrieval`.
|
||||
Reach for it before moving any floor or budget, and read the records it
|
||||
names rather than its percentiles alone: every percentile can say "lower
|
||||
the bar" while each refused record is a false positive, so
|
||||
`near_miss_samples=5` and opening the ids it returns is the step that
|
||||
separates a real miss from a bar doing its job.
|
||||
|
||||
Five readouts, from the five tables built for them:
|
||||
|
||||
`sources` — per retrieval surface (`auto_inject`, `write_path`,
|
||||
`mcp_search`, …), from `retrieval_logs`: `calls`, `zero_result_calls`,
|
||||
`near_misses`, the `top_score` spread (p10/p50/p90/min/max),
|
||||
`avg_result_count` and `p90_duration_ms`.
|
||||
|
||||
THE NUMBER TO READ FIRST IS `near_misses.p90`, AGAINST THE THRESHOLD IN
|
||||
FORCE FOR THAT SURFACE. It is measured on the calls the BAR turned away —
|
||||
zero-result calls, minus the ones whose zero was a repeat the reader had
|
||||
already been shown — using the best score the ranker reached before the bar
|
||||
rejected it. So it is the one figure here that says something the bar
|
||||
cannot make true by construction, and `max` is always below the threshold:
|
||||
an above-bar candidate nobody excluded would have been returned. A bar at 0.72 turning away a stream of 0.71s is set too
|
||||
high by a hair and the surface is losing hits it should have had. The same
|
||||
bar turning away 0.30s is working, and the corpus simply had nothing. Both
|
||||
render as a zero-result call, and nothing else in this readout tells them
|
||||
apart.
|
||||
|
||||
`near_miss_samples` (0-20, default 0) TURNS THE PERCENTILES INTO RECORDS
|
||||
YOU CAN READ. Each source then carries `near_miss_records`: its highest
|
||||
scoring declines, each with the `record_id` the bar refused and the `query`
|
||||
that asked. Reach for it whenever you are about to move a threshold.
|
||||
|
||||
THE PERCENTILE CANNOT SETTLE A BAR ON ITS OWN, and this is the whole reason
|
||||
the parameter exists. `near_misses.p90` says mass is sitting just under the
|
||||
line; it says nothing about whether that mass is RELEVANT, and those are
|
||||
different questions. Lowering a bar to where the mass is, without reading
|
||||
what is there, is choosing a firing rate rather than a quality. Pull-through
|
||||
cannot referee it either — the injected rule line already carries title and
|
||||
trigger, so a session can comply without ever calling `get_rule`, which
|
||||
makes rule pull-through understate usefulness by construction. Reading the
|
||||
rejected records is the method that actually answers it.
|
||||
|
||||
Off by default because it is a LISTING, not a statistic: it is for the
|
||||
moment you are making a decision, not for every readout.
|
||||
|
||||
`near_misses` is `null` when no declining call in the window measured it —
|
||||
older rows may not carry the measurement. That is "not measured", not
|
||||
"nothing came close"; a 0.0 there would be a claim about the corpus
|
||||
invented out of a caller's silence.
|
||||
|
||||
A search that returns nothing WITHOUT having run — an empty query, an
|
||||
unavailable embedder, a failed database query — writes no row: a row would
|
||||
report a call that never happened and drag the clear rate down with
|
||||
phantom declines. So a null is "searched, and nothing came close", and a
|
||||
broken search shows up as a WARNING in the application log rather than as
|
||||
a quiet zero in here.
|
||||
|
||||
There is no "cleared the threshold" count, because it would be a
|
||||
tautology: the search applies the bar before returning, so every returned
|
||||
result cleared it, and that count would always equal
|
||||
`calls - zero_result_calls`.
|
||||
|
||||
CHECK `suppression` BEFORE CONCLUDING ANYTHING FROM `zero_result_calls`. A
|
||||
zero-result call is two different events wearing one number: the ranker
|
||||
found nothing above the bar, or it found only what this session had already
|
||||
been shown. Just the first is evidence about the bar. `suppression` splits
|
||||
them where the surface can tell — `zero_because_already_shown` comes off
|
||||
`zero_result_calls` to leave the true ranker declines.
|
||||
|
||||
`suppression` is `null` when NO row in the window reported it, and that is
|
||||
"not measured here", NOT "none suppressed". Surfaces that pass their
|
||||
exclusions into the search never see what was dropped, so they cannot say.
|
||||
Read a null as "unknown", never as a zero: an artifact read as a
|
||||
measurement mis-scopes the surface.
|
||||
|
||||
`usage` — NOTES ONLY, from `note_usage_events`, at the per-note grain
|
||||
`retrieval_logs` cannot be indexed at: `surfaced` (ranked surfacings — a
|
||||
scored surface CHOSE the record), `ambient` (the rest), `pulled` split into
|
||||
`pulled_by_agent` / `pulled_by_human`, the distinct-note counts, and
|
||||
`pull_through`. That ratio is the corpus-side precision signal: records
|
||||
surfaced often and opened never are dead weight competing for the injection
|
||||
budget every turn.
|
||||
|
||||
`pull_through` is AGENT pulls over RANKED surfacings, and both halves of
|
||||
that matter. "Is this record dead weight?" is answered by any pull; "was
|
||||
that injected line useful?" — the question a threshold or a reranker is
|
||||
tuned against — only by a pull the agent made. Aggregating across the
|
||||
mcp_/rest_ prefix would silently answer the wrong one.
|
||||
|
||||
`usage["by_source"]` — THE number to tune a threshold against, because the
|
||||
top-level `pull_through` is a corpus average and averages the surfaces
|
||||
together. Per surface: `notes_surfaced`, `notes_pulled`, `pull_through`,
|
||||
and `ambient: true` on surfaces whose surfacings were not scored choices
|
||||
(their ratio is null — "surfaced often, opened never" is not a judgment
|
||||
about a record nothing chose). Read it as: of the distinct notes THIS
|
||||
surface put in front of the agent, how many did the agent then open?
|
||||
|
||||
It is an UPPER BOUND per surface: a pull records the door it came
|
||||
through, not the surface that led there, so a note surfaced by two surfaces
|
||||
and opened once counts for both — attribution would need a session
|
||||
identity the logs do not carry. `by_source_failed: true` means that one
|
||||
query failed while the rest of the readout stood.
|
||||
|
||||
`rule_usage` — the same question for RULES, from `rule_usage_events`:
|
||||
`surfaced` and `ambient`, `pulled` split into `pulled_by_agent` /
|
||||
`pulled_by_human`, the distinct-rule counts, and `pull_through` on the same
|
||||
definition (agent pulls over RANKED surfacings).
|
||||
|
||||
`applied`, `departed` AND `distinct_rules_acted` ARE WHAT HAPPENED AFTER
|
||||
THE RULE WAS OPENED. A pull says the rule was read; these say it
|
||||
changed something. `applied` counts rules followed, `departed` rules
|
||||
deliberately not followed — kept apart rather than summed, because a
|
||||
departure carries the reason the agent gave and is evidence about the
|
||||
RULE, while an application is evidence about the agent. There is
|
||||
deliberately no count of rules read and quietly ignored: that state is what
|
||||
is left over when a rule was pulled and neither outcome arrived, and
|
||||
`read_and_unacted` below is where it is reported. Asking an agent to
|
||||
declare it would be asking it to notice an omission it is defined by not
|
||||
noticing.
|
||||
|
||||
A SEPARATE BLOCK, not folded into `usage`, and reading it as one number
|
||||
with that is the mistake to avoid. The corpora differ by orders of
|
||||
magnitude — a few dozen eligible rules against thousands of notes — so a
|
||||
blended ratio would be the note ratio with noise on it and would hide the
|
||||
rule arm entirely.
|
||||
|
||||
`surfaced` VS `ambient` IS THE READING THAT MATTERS HERE. `surfaced` counts
|
||||
rules a ranker chose — today only the write-path arm — and those are claims
|
||||
a pull can settle. `ambient` counts BULK DELIVERIES: the `rules_payload`
|
||||
surfaces (`enter_project`, `get_project`, `get_milestone`,
|
||||
`start_planning`, `get_task`), which hand over the whole applicable set at
|
||||
once with nobody choosing anything. A large `ambient` says a bulk set
|
||||
is big and arrives often — never that it is useful, and never that it is
|
||||
read.
|
||||
|
||||
`pull_through` therefore divides by `surfaced` alone. Fold the preload in
|
||||
and growing a bulk set would depress the arm's measured precision while
|
||||
trimming it would flatter it, for reasons having nothing to do with the
|
||||
arm. To judge a BULK surface instead, compare `ambient` against pulls of
|
||||
those same rules over time: a set surfaced thousands of times and opened
|
||||
never is the dead-weight signal, one level up.
|
||||
|
||||
Read it against `sources["write_path_rule"]`, which declines the large
|
||||
majority of its calls like any other surface.
|
||||
|
||||
`system_usage` — the RULINGS arm, from `system_usage_events`: how
|
||||
often an area's rulings were shown because a command (`rulings_pre_tool`)
|
||||
or an edit (`rulings_write_path`) touched its files, how often a System
|
||||
was then opened (`mcp_get_system`), and `by_system` naming the areas, most
|
||||
shown first. A lookup, not a ranker, so it has no row in `sources` and no
|
||||
floor to tune. And NO `pull_through`, on purpose: the rulings travel in
|
||||
full in the line, so a session reads them without opening anything, and
|
||||
a ratio of opens would read near zero on an arm that is working.
|
||||
`system_usage_failed: true` means its read broke and the zeros mean nothing.
|
||||
|
||||
`judged` — whether the lines RELATED, from `retrieval_judgments`:
|
||||
per source, a reviewer's verdicts (`on_point` / `adjacent` / `unrelated`)
|
||||
on a sample of logged menus, `within_budget` against `beyond_budget` and
|
||||
`by_rank`. Every usage block above counts what the reader DID; a line
|
||||
carries its matched passage, so "not opened" is no verdict on relevance,
|
||||
and this is the block that holds one. `on_point_unopened` counts lines
|
||||
that related and were left closed. Read it before moving a budget: if the
|
||||
ranks past the cut are mostly `on_point`, the budget is costing hits; if
|
||||
they are mostly `unrelated`, it is doing its job. Empty until someone
|
||||
reviews — `menus_to_review` and `judge_menu` fill it. A sample is small by
|
||||
nature, so the counts are printed, not rates. `judged_failed: true` means
|
||||
the read broke.
|
||||
|
||||
EVERY COUNTER BLOCK CARRIES ITS OWN COVERAGE — `complete_from` and
|
||||
`covers_window`. `complete_from` is when the number became trustworthy:
|
||||
for one source, its first recorded row; for a section that sums several,
|
||||
the LATEST of theirs, because a total is complete only once every
|
||||
contributor was being written. `covers_window: false` means the window
|
||||
reaches back further than the recording does, so the count is a fraction
|
||||
of the period it appears to describe.
|
||||
|
||||
READ IT BEFORE COMPARING TWO NUMBERS, and especially before comparing
|
||||
across a deploy. A counter added last week, read over a 30-day window,
|
||||
reports a real count against an imagined denominator — and the result is
|
||||
a plausible fraction rather than an obvious zero, which is what makes it
|
||||
dangerous.
|
||||
|
||||
`covers_window` is null, never false, when nothing was ever recorded:
|
||||
"no measurement" is not "partial measurement", the same distinction
|
||||
`suppression`'s null carries a few paragraphs up.
|
||||
|
||||
A SOURCE SHOWING `calls: 0` WAS RECORDING AND MADE NO CALLS. `sources`
|
||||
lists every source the table has ever held, not only those active in the
|
||||
window, so a surface that stopped firing stays visible rather than
|
||||
disappearing — being absent is reserved for a source that has never
|
||||
recorded at all. Its score fields are null, not zero: the calls are a
|
||||
real observation, the distribution is not one.
|
||||
|
||||
`rule_usage_failed: true` means that read failed while the rest of the
|
||||
readout stood. The counts are still present so a caller can render, but
|
||||
they are zeros meaning "could not find out", not "nothing happened" — do
|
||||
not report a pull-through from a block carrying that flag.
|
||||
|
||||
`warnings` IS THE PART TO READ FIRST. Everything above is a
|
||||
distribution; this is a verdict, the checks a careful reader would
|
||||
otherwise redo by hand on every reading. An EMPTY LIST means checked and clean — it is always present, so
|
||||
its emptiness is an answer rather than a gap. Each entry carries the
|
||||
numbers that triggered it, so you can disagree with the rule instead of
|
||||
having to redo the arithmetic:
|
||||
|
||||
- `cannot_decline` — an arm that fires unasked answered every one of its
|
||||
calls. It cannot say nothing, which means it is not applying a floor.
|
||||
Only ever raised for unbidden arms known to log unconditionally: a
|
||||
search returning a list every time is doing its job, and an arm whose
|
||||
zeros were never written would flag a LOGGING bug while pointing you at
|
||||
a threshold.
|
||||
- `band_hugs_floor` — the weakest tenth of what an arm returns sits on
|
||||
its floor. The bar is doing the selecting and the score is not, so
|
||||
moving that floor changes how MUCH you get, not how good it is.
|
||||
- `floor_moved_mid_window` — that arm's floor CHANGED inside the window,
|
||||
by a release or by a dial turn, so its calls were made under two bars
|
||||
and the band check above is suspended for it rather than answered
|
||||
wrongly. Ask again with a `days` starting after the named date. The
|
||||
warning replaces `band_hugs_floor` for that arm; it never accompanies
|
||||
it.
|
||||
- `no_duration` — rows written without timings. A logging gap, not a slow
|
||||
arm, and it devalues every other number from that source.
|
||||
- `surfaced_never_pulled` — distinct records shown and never opened, per
|
||||
corpus. Not a verdict on them: a line carries its matched passage, so
|
||||
an unopened record may have been unrelated, enough as shown, or already
|
||||
in context. Judge a sample (`menus_to_review`) before acting on it.
|
||||
For RULES there is no judged sample: a rule set aside again and again
|
||||
is a trigger to fix (`update_rule(when_to_apply=...)`), not a floor.
|
||||
- `read_and_unacted` — distinct rules OPENED in the window that recorded
|
||||
no outcome, against the ones that did. The worse sibling of `surfaced_never_pulled` above: a
|
||||
rule nobody opens is cheap, while a rule read and silently unchanged is
|
||||
indistinguishable from one that worked. It does not say which of the two
|
||||
causes it is — a rule mis-triggering, arriving where it does not apply,
|
||||
or a rule being ignored — and those want opposite fixes, so read the
|
||||
rules before moving anything.
|
||||
- `outcomes_never_recorded` — rules were opened and NOT ONE outcome exists
|
||||
anywhere in the window. Deliberately a separate code, and not a
|
||||
`read_and_unacted` with a zero in it: a window with no outcomes at all
|
||||
cannot tell "every rule was ignored" from "nothing on this install calls
|
||||
`rule_outcome` yet", and reporting the first would manufacture a finding
|
||||
out of an unwired feature. Wire the outcome call before reading this as
|
||||
a fact about the corpus.
|
||||
- `unregistered_source` — rows under a source missing from
|
||||
`retrieval_registry`. Its numbers are real; no verdict could be
|
||||
computed, because nothing says whether it was asked or fired unbidden.
|
||||
|
||||
`silent_surfaces` IS THE HALF THE ROWS CANNOT SHOW YOU. Every check above
|
||||
reads rows, so an arm that produced none is invisible to all of them and
|
||||
looks exactly like an arm that does not exist. This list is driven by the
|
||||
declared registry instead: points expected to emit that emitted nothing.
|
||||
Points that are legitimately quiet — the web-UI-only sources on an install
|
||||
driven through MCP — are excluded by declaration rather than by silence,
|
||||
so a justified quiet never reads as a gap. The list stays EMPTY on a
|
||||
window with little traffic: on a fresh install every point is silent, and
|
||||
reporting all of them would be describing the emptiness.
|
||||
|
||||
WARNINGS ARE COMPUTED OVER THE BLOCKS ABOVE, not over a second query, so
|
||||
one can never disagree with the numbers printed beside it. A window whose
|
||||
read failed produces none at all — a verdict over rows that did not load
|
||||
would describe the outage while appearing to describe the system.
|
||||
|
||||
Two thresholds govern them, both settings so an install driven harder can
|
||||
say so: `retrieval_warn_min_calls` (default 30) is how much traffic a
|
||||
source needs before its silence means anything, and
|
||||
`retrieval_warn_floor_epsilon` (default 0.02) is how close to the bar
|
||||
counts as piled on it.
|
||||
|
||||
Scoped to your own telemetry — a retrieval log records what your agent
|
||||
asked for, query text included, and is not a shared record kind.
|
||||
|
||||
`read_failed: true` means the query itself failed — deliberately distinct
|
||||
from an empty window, which would otherwise read as every counter at
|
||||
zero.
|
||||
|
||||
Args:
|
||||
days: window size, default 30. Clamped to at least 1.
|
||||
near_miss_samples: 0-20, default 0. How many of each source's highest
|
||||
scoring DECLINES to list by record, with the query that asked.
|
||||
Pass it when you are about to move a threshold; leave it off
|
||||
otherwise. See the near-miss section above for why a percentile
|
||||
alone cannot settle a bar.
|
||||
"""
|
||||
return await retrieval_summary(
|
||||
current_user_id(), days=days, near_miss_samples=near_miss_samples,
|
||||
)
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
mcp.tool(name="search")(search)
|
||||
mcp.tool(name="retrieval_telemetry")(retrieval_telemetry)
|
||||
@@ -0,0 +1,407 @@
|
||||
"""Shape-ledger MCP tools — the classification write/read surface.
|
||||
|
||||
The accounting model: the snippet library records CANON (small);
|
||||
the ledger accounts for EVERY extracted shape (total). These tools are how
|
||||
agents move shapes out of `unclassified` — the todo state — and how they read
|
||||
what still needs judgment. The ledger rows themselves are fed by the coverage
|
||||
refresh; a judgment given with `repo` may also land on a shape written this
|
||||
turn that the sync has not seen yet.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import coverage as coverage_svc
|
||||
from grimoire.services import shape_ledger as shape_ledger_svc
|
||||
|
||||
|
||||
async def classify_shapes(
|
||||
project_id: int, classifications: list[dict], via: str = "agent", repo: str = ""
|
||||
) -> dict:
|
||||
"""Record judgments for a project's code shapes — in batch, as rows.
|
||||
|
||||
EVERY shape in a bound repo should end up classified:
|
||||
- `instance` of snippet N — it conforms to recorded canon (family-level
|
||||
canon in another project counts; that fully accounts for the shape).
|
||||
- `variant` of snippet N — a deliberate, named departure. `reason`
|
||||
(the why) is REQUIRED; it is the record.
|
||||
- `exempt` — judged genuinely one-off. `reason` REQUIRED.
|
||||
- `canonical` of snippet N — this row IS the snippet's reference
|
||||
(rarely set by hand; the coverage sync stamps these mechanically).
|
||||
- `unclassified` — withdraw a judgment; the shape rejoins the todo.
|
||||
|
||||
Consumer maps belong HERE, not in prose: when an audit enumerates call
|
||||
sites of a canonical helper, each call site's defining shape is an
|
||||
`instance` row — a sentence in a verification detail cannot be sorted,
|
||||
queried, or diffed.
|
||||
|
||||
Args:
|
||||
project_id: The project whose ledger is being judged.
|
||||
classifications: Objects of {path, symbol, status, kind?, snippet_id?,
|
||||
idea_id?, reason?, reason_code?}. path+symbol name the shape exactly as
|
||||
list_shapes shows it; kind ("sym"/"css") narrows when one file
|
||||
defines both. snippet_id is required for canonical/instance/
|
||||
variant — or idea_id, a family idea: the shape is then judged
|
||||
against that idea's reference implementation in the shape's
|
||||
language, or its first reference when none is in that language
|
||||
(an idea is shared across languages: a Python shape can be an
|
||||
instance of an idea whose reference is Go). reason is required
|
||||
for variant/exempt. reason_code is
|
||||
an OPTIONAL index beside the prose (one of: scoped-css,
|
||||
one-off-handler, test-helper, convention-plumbing, pure-helper,
|
||||
generated, script, typed-record) so the ledger can be filtered
|
||||
and aggregated by kind of one-off — the prose stays the record.
|
||||
uses is an OPTIONAL list of snippet ids this shape CALLS:
|
||||
conformance (status + snippet_id) says what shape it is, uses
|
||||
says which canonical helpers it consumes — a service function
|
||||
can be an instance of the service-function convention AND use
|
||||
hash_token. Consumer maps are uses edges; list_shapes(uses=N)
|
||||
and get_snippet's `uses` read them.
|
||||
via: Who is judging — "agent" (default), "audit" (a sweep), or
|
||||
"import" (carrying maps recorded elsewhere).
|
||||
repo: The repo the shapes live in (its remote URL or bound key), for
|
||||
judging what you wrote THIS turn — shapes the ledger has not
|
||||
synced yet. With it, a shape no row matches is recorded as a
|
||||
provisional row under that repo and judged; the next sync
|
||||
confirms it. Must be bound to the project. Omit it to judge only
|
||||
synced rows.
|
||||
|
||||
FAMILY CANON FOLLOWS. A shape judged against a canon family idea's
|
||||
reference IS the project's answer to that idea: an instance answers it
|
||||
`adopted`, a variant `variant` (with the shape's reason), and
|
||||
withdrawing the shapes that gave an answer returns it to `unassessed`.
|
||||
The adoption ledger is moved for you — `family` in the result lists the
|
||||
answers that moved — and assess_family_adoption refuses an answer the
|
||||
shapes contradict. The family-canon skill has when to answer an idea.
|
||||
|
||||
All-or-nothing: a structural error, a missing snippet target, an unbound
|
||||
repo, or no write access applies NOTHING. Returns {"classified": N,
|
||||
"unmatched": [...], "provisional": N?, "family": [...]?} — unmatched names
|
||||
shapes no live ledger row matches (the tree may have moved since you
|
||||
listed; pass `repo` for a shape you just wrote, or re-run the project's
|
||||
coverage refresh).
|
||||
"""
|
||||
uid = current_user_id()
|
||||
return await shape_ledger_svc.classify_shapes(
|
||||
uid, project_id, classifications, via=via, repo=repo
|
||||
)
|
||||
|
||||
|
||||
async def list_shapes(
|
||||
project_id: int,
|
||||
status: str = "",
|
||||
path: str = "",
|
||||
snippet_id: int = 0,
|
||||
include_vanished: bool = False,
|
||||
limit: int = 100,
|
||||
offset: int = 0,
|
||||
proposal: str = "",
|
||||
flag: str = "",
|
||||
compact: bool = False,
|
||||
uses: int = 0,
|
||||
) -> dict:
|
||||
"""Read a project's shape ledger — `status="unclassified"` IS the todo.
|
||||
|
||||
Every extracted definition in the project's bound repos has a row here
|
||||
(fed by the coverage refresh). Filters compose:
|
||||
|
||||
Args:
|
||||
status: canonical | instance | variant | exempt | scoped | unclassified.
|
||||
`scoped` is the sync's mechanical stamp on one-offs by
|
||||
construction (a Vue component's scoped <style> rules and its
|
||||
<script setup> functions): accounted for, not judged, still
|
||||
proposed against / grouped / flagged, and overridable by any
|
||||
classify_shapes judgment. The human todo is `unclassified`.
|
||||
path: exact file, or a directory — matches everything beneath it
|
||||
(the coverage line's "largest" dirs go straight in here).
|
||||
snippet_id: rows classified against this snippet (instance/variant
|
||||
of it — conformance).
|
||||
uses: rows that CALL this snippet — the consumer map proper,
|
||||
whatever shape each row is itself; edges come from judgments
|
||||
(classify_shapes uses=), the write-path hook, and the proposer's
|
||||
by-name reference hits.
|
||||
include_vanished: include shapes no longer in the tree (history).
|
||||
limit/offset: page through big ledgers (limit caps at 500).
|
||||
compact: rows as `path · symbol · kind · status · signature` plus
|
||||
snippet_id / by / proposal / diverges_from / recheck only when
|
||||
set — no commits, shas or timestamps. THE form for an audit:
|
||||
a full 500-row page fits the tool budget. The default rows carry
|
||||
everything (shape_history-grade bookkeeping).
|
||||
proposal: the proposer's queue — "any", "canon" (rows the
|
||||
machine thinks are an instance of a snippet: `proposal` carries
|
||||
snippet_id, basis, score), "derive" (rows that repeat with NO
|
||||
canon: `proposal.group` names the family), or one basis
|
||||
(symbol/text/reference/signature/semantic/family). "family" is
|
||||
a canon family idea's reference — another project's, often
|
||||
another language's — that the body means: the top match over
|
||||
every snippet you can read, on a platform the project shares.
|
||||
flag: the divergence readout — "divergence": shapes new
|
||||
since the previous refresh in a directory where one canon
|
||||
dominates the judged siblings and NOT proposed as that canon
|
||||
(`diverges_from` names it: button B where button A is canon —
|
||||
classify it: instance if it should use the canon, variant with
|
||||
the why if deliberate); "recheck": judged instances/variants
|
||||
whose body changed since judged (the judgment stands; confirm
|
||||
it again with classify_shapes, or re-judge); "unused-css":
|
||||
live css rules no file's markup names — a
|
||||
deletion candidate to look at, never auto-deleted. Transition
|
||||
classes and concatenated names are read, so the list is
|
||||
worth acting on; a name assembled in a script still is not.
|
||||
|
||||
Returns {"shapes": [...], "total": N} — total counts every match, not
|
||||
just this page. Every css row carries `used_by` {count, paths} — the
|
||||
files whose markup names its class (the CSS consumer
|
||||
map: a scoped rule is used by its own template; a shared recipe by
|
||||
many; a count of 0 is "no template names it"). Each row's
|
||||
`classified_by` says who judged: agent /
|
||||
audit / import are judgments; `mechanical` is the canonical stamp the
|
||||
sync applies; `hook` is write-path EVIDENCE — the session pulled
|
||||
a snippet and then wrote code referencing/resembling it, so the shape
|
||||
was stamped an instance with the evidence in `reason`. A hook row is
|
||||
overridable by any classify_shapes call; it never overrides yours.
|
||||
|
||||
THE FAST PATH through a big todo is the proposer's queue: every coverage
|
||||
refresh matches unclassified shapes against canon (strongest basis
|
||||
first: same symbol elsewhere → textual containment → body references
|
||||
the canon → signature resemblance → semantic, and a family idea's
|
||||
reference in any language) and attaches a
|
||||
`proposal` to each row it can speak for. Review `proposal="canon"` by
|
||||
snippet or directory, then confirm_shape_proposals the ones that hold —
|
||||
hundreds at a time — and classify_shapes the rest (variant/exempt, or
|
||||
instance of a different snippet). `proposal="derive"` lists the
|
||||
derive-first candidates: a repeating shape with NO recorded canon is
|
||||
never N loose classifications — consolidate onto a reference,
|
||||
create_snippet it, then classify the group against it.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows, total = await shape_ledger_svc.list_project_shapes(
|
||||
uid, project_id,
|
||||
status=status, path=path, snippet_id=snippet_id,
|
||||
include_vanished=include_vanished, limit=limit, offset=offset,
|
||||
proposal=proposal, flag=flag, uses=uses,
|
||||
)
|
||||
shapes = [r.to_compact() if compact else r.to_dict() for r in rows]
|
||||
used_by = await shape_ledger_svc.used_by_map(rows)
|
||||
for row, out in zip(rows, shapes):
|
||||
if row.id in used_by:
|
||||
out["used_by"] = used_by[row.id]
|
||||
return {"shapes": shapes, "total": total}
|
||||
|
||||
|
||||
async def classify_shapes_by_rule(
|
||||
project_id: int,
|
||||
path: str,
|
||||
status: str,
|
||||
pattern: str = "",
|
||||
kind: str = "",
|
||||
snippet_id: int = 0,
|
||||
reason: str = "",
|
||||
via: str = "agent",
|
||||
include_judged: bool = False,
|
||||
reason_code: str = "",
|
||||
uses: list[int] | None = None,
|
||||
) -> dict:
|
||||
"""The sweep form of classify_shapes: ONE judgment applied to every
|
||||
unclassified shape under a directory whose symbol matches a glob.
|
||||
|
||||
For the long tail an audit judges by family, not by row — "every scoped
|
||||
rule under frontend/src/views is exempt: styles one element of its view",
|
||||
"every `*_scheduler.py` symbol is an instance of ScheduledJob" — where
|
||||
listing 900 rows and sending them back is the whole cost. The row form
|
||||
stays the precise tool; reach for it when each row gets its own reason.
|
||||
|
||||
Args:
|
||||
project_id: The project whose ledger is being judged.
|
||||
path: A file, or a directory and everything beneath it. Required —
|
||||
a sweep names what it judges.
|
||||
status: instance | variant | exempt | unclassified (canonical is the
|
||||
sync's stamp, not a sweep's).
|
||||
pattern: Shell glob on the symbol (`*_rows`, `_*`, `modal-*`, `*`);
|
||||
"" = every symbol under path.
|
||||
kind: "sym" or "css" to narrow; "" = both.
|
||||
snippet_id: Required for instance/variant — the canon judged against.
|
||||
reason: Required for variant/exempt — the why, recorded on every row.
|
||||
via: "agent" (default) | "audit" | "import".
|
||||
reason_code: Optional catalogue code beside the reason (see
|
||||
classify_shapes) — a sweep is exactly where one applies.
|
||||
uses: Optional snippet ids every matched shape CALLS — e.g.
|
||||
"every *_scheduler.py symbol uses ScheduledJob".
|
||||
include_judged: By default only unjudged rows are touched —
|
||||
`unclassified` and the sync's mechanical `scoped` stamp — a
|
||||
sweep never silently overwrites a judgment. True re-judges every
|
||||
matching live row (use to re-confirm after a recheck, or to
|
||||
revise a family you judged earlier).
|
||||
|
||||
One transaction: applies whole or not at all. Returns
|
||||
{"classified": N, "sample": ["path::symbol", ...], "family": [...]?}
|
||||
(sample: first 12, sorted) so you can see what the rule reached; N = 0
|
||||
means the rule matched nothing live and unclassified — widen the pattern
|
||||
or refresh coverage. As with classify_shapes, a family idea's reference
|
||||
judged here moves the project's answer to that idea (`family`).
|
||||
"""
|
||||
uid = current_user_id()
|
||||
try:
|
||||
return await shape_ledger_svc.classify_shapes_where(
|
||||
uid, project_id, path=path, status=status, pattern=pattern,
|
||||
kind=kind, snippet_id=snippet_id or None, reason=reason or None,
|
||||
via=via, include_judged=include_judged,
|
||||
reason_code=reason_code or None, uses=uses or None,
|
||||
)
|
||||
except ValueError as exc:
|
||||
return {"error": str(exc)}
|
||||
|
||||
|
||||
async def shape_history(
|
||||
project_id: int, path: str, symbol: str = "", limit: int = 200
|
||||
) -> dict:
|
||||
"""What was used here, when, and why — a shape's (or a directory's)
|
||||
history from the ledger.
|
||||
|
||||
`shapes` are the current rows at `path` (a file, or a directory and
|
||||
everything beneath it; `symbol` narrows to one definition) with
|
||||
first/last-seen commits, vanished_at, and the standing judgment;
|
||||
`events` are the state changes, oldest first: `classified` (status,
|
||||
snippet_id, who, why — one per judgment, so a shape that was an instance
|
||||
of #N and later a variant of #M shows both), `vanished`, `reappeared`,
|
||||
`drifted` (the body moved under a judgment; see list_shapes flag=
|
||||
"recheck"). Each event carries the commit the tree was read at.
|
||||
|
||||
Read it as a timeline: "instance of #N from <first classified at>,
|
||||
re-judged variant of #M at <at> because <reason>, vanished at <commit>".
|
||||
Read-only; requires read access to the project.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
return await shape_ledger_svc.shape_history(
|
||||
uid, project_id, path, symbol=symbol, limit=limit
|
||||
)
|
||||
|
||||
|
||||
async def stamps_to_review(project_id: int, top: int = 10) -> dict:
|
||||
"""Judged rows whose evidence no longer meets the ledger's bar, and canons
|
||||
whose own rows no longer agree what they are. **Read-only — you decide.**
|
||||
|
||||
This tool deliberately cannot fix anything. Every row it lists was written
|
||||
unattended by the write-path hook on a similarity score, and what made
|
||||
that harmful was not one bad score but a machine recording a permanent
|
||||
classification with nobody reading it. Un-asserting them automatically
|
||||
would be the same mistake with a wider blast radius. So: read the
|
||||
evidence, judge, and record the judgment yourself with `classify_shapes`
|
||||
— under your own name, with a reason.
|
||||
|
||||
`weak` — rows the hook stamped on a resemblance below the current floor
|
||||
(`floor` in the response). The floor arrived after they did, and a guard
|
||||
at the point of classification never undoes what is already stored. Rows
|
||||
an agent or an audit judged are NOT listed at any age: a judgment is not
|
||||
weak evidence, it is a different kind of evidence. Each row carries its
|
||||
score, signature and derived form so you can judge rather than trust the
|
||||
threshold.
|
||||
|
||||
`incoherent` — canons whose membership no longer agrees what the canon
|
||||
is. A canon asserts that some shapes are the same sort of thing; when its
|
||||
members are a class, three getters and a dozen tests, that assertion has
|
||||
stopped being true and every base-rate reading built on it — the
|
||||
divergence prompt included — is reading noise.
|
||||
|
||||
Disagreement is judged by FAMILY (a sync helper and an async one are one
|
||||
family), and a method defined beside a member class counts as part of the
|
||||
shape rather than against it — so a model convention that covers both the
|
||||
class and its `to_dict` reads as coherent, which it is. Each
|
||||
entry carries `families` and `forms`, the majority `family`, `attached`
|
||||
(minority rows excused as methods of a member), `strangers` — THE ROWS
|
||||
THAT DO NOT FIT, which is what you judge — and `unattended` / `weak_rows`,
|
||||
the count written by the hook with nobody reading. An incoherence whose
|
||||
rows are all judged is likelier this check being strict than a bad
|
||||
ledger; one made of unattended rows is the real thing.
|
||||
|
||||
A row listed here is a question, not a verdict. Some will be correct.
|
||||
|
||||
Read-only; requires read access to the project.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows = await shape_ledger_svc.live_rows_for(uid, project_id)
|
||||
return shape_ledger_svc.stamps_to_review(rows, top=top)
|
||||
|
||||
|
||||
async def confirm_shape_proposals(
|
||||
project_id: int,
|
||||
snippet_id: int = 0,
|
||||
path: str = "",
|
||||
basis: str = "",
|
||||
min_score: float = 0.0,
|
||||
) -> dict:
|
||||
"""Confirm the proposer's canon proposals you have reviewed, in batch.
|
||||
|
||||
The machine proposes, judgment classifies: each matching row —
|
||||
live, unclassified, carrying a `proposal` with a snippet_id — becomes
|
||||
`instance` of that snippet, classified_by="agent", reason naming the
|
||||
basis and score. Narrow to what you actually looked at: at least one of
|
||||
snippet_id (confirm one canon's whole queue after reading its
|
||||
`list_shapes(proposal="canon", ...)` page), path (a directory you
|
||||
audited), or basis (e.g. "symbol" and "reference" are near-certain;
|
||||
"semantic" and "family" deserve a look first) is required — a bare
|
||||
confirm-all is not a judgment. min_score trims a basis's tail.
|
||||
|
||||
Proposals you do NOT confirm are judged with classify_shapes (variant,
|
||||
exempt, or instance of a different snippet) — any judgment retires the
|
||||
proposal. A confirmed family reference answers that idea `adopted` in
|
||||
the project (`family` lists the answers that moved). Requires write
|
||||
access. Returns {"confirmed": N, "family": [...]?}.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
try:
|
||||
return await shape_ledger_svc.confirm_proposals(
|
||||
uid, project_id, snippet_id=snippet_id, path=path, basis=basis,
|
||||
min_score=min_score,
|
||||
)
|
||||
except ValueError as exc:
|
||||
return {"error": str(exc)}
|
||||
|
||||
|
||||
async def refresh_pattern_coverage(project_id: int) -> dict:
|
||||
"""Seed or refresh the project's shape ledger NOW, and return the readout.
|
||||
|
||||
The synchronous form of the arrival-moment background seed:
|
||||
downloads the bound repos through the OWNER's forge connections, upserts
|
||||
every extracted shape into the ledger (new shapes arrive `unclassified`),
|
||||
re-stamps snippet reference locations as canonical, and recomputes the
|
||||
accounting. Reach for it when the ledger must be current before you act —
|
||||
a classification batch about to run, a coverage question asked directly —
|
||||
rather than waiting on the background seed an enter_project triggers.
|
||||
|
||||
Takes seconds, not milliseconds (it moves repo archives). Requires write
|
||||
access to the project. Errors name the fix: no forge connection → the
|
||||
owner adds one (Settings → Integrations → Git Forges); no served repo →
|
||||
bind_repo on a host a connection serves.
|
||||
|
||||
The refresh is also when the mechanical proposer runs: with the
|
||||
repo bodies in hand it matches every changed unclassified shape against
|
||||
canon and records proposals (see list_shapes proposal=), then regroups
|
||||
the derive-first candidates. Semantic matching is capped per refresh, so
|
||||
a large ledger's queue grows across refreshes rather than in one.
|
||||
|
||||
Returns the accounting payload — total, accounted, counts by status,
|
||||
unclassified, repos, largest_gaps, `proposed` (canon proposals awaiting
|
||||
confirmation), `derive_groups` (the biggest repeats-with-no-canon
|
||||
families, each css one with `consumers` — the files whose markup
|
||||
render it), `unused_css` (css rules no template names —
|
||||
counting a `<Transition name=>`'s generated classes and concatenated
|
||||
names as named; None where the map has no evidence of templates), `derive_new` (copies
|
||||
that joined a family since the previous
|
||||
refresh — the drift to act on now: derive the canon, don't queue an
|
||||
audit), `proposer` (what this refresh examined) — plus
|
||||
`pattern_coverage`, the same one-line summary enter_project carries.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
coverage = await coverage_svc.refresh_for_caller(uid, project_id)
|
||||
return {
|
||||
"pattern_coverage": coverage_svc.coverage_line(coverage),
|
||||
**coverage,
|
||||
}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
classify_shapes, classify_shapes_by_rule, list_shapes,
|
||||
refresh_pattern_coverage, confirm_shape_proposals, shape_history,
|
||||
stamps_to_review,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,607 @@
|
||||
"""Snippet MCP tools: record reusable functions/components for later recall.
|
||||
|
||||
A snippet is a Note with note_type='snippet' (see services/snippets.py). Because
|
||||
snippets are ordinary embedded notes, once recorded they surface through the same
|
||||
semantic search + title-first auto-inject as everything else — so a reusable
|
||||
thing recorded once can be recalled before it's re-written as a one-off.
|
||||
|
||||
The tools wrap services/snippets.py, mirroring the note/process tools (dedup gate
|
||||
on create, System association passthrough).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.mcp.tools import systems as systems_tools
|
||||
from grimoire.services import access as access_svc
|
||||
from grimoire.services import dedup as dedup_svc
|
||||
from grimoire.services import family as family_svc
|
||||
from grimoire.services import snippets as snippets_svc
|
||||
from grimoire.services.note_usage import attach_usage, record_pulled
|
||||
from grimoire.services import systems as systems_svc
|
||||
from grimoire.services import moment_delivery
|
||||
|
||||
|
||||
async def list_snippets(
|
||||
q: str = "", tag: str = "", limit: int = 50, offset: int = 0,
|
||||
project_id: int = 0,
|
||||
repo: str = "", path: str = "", symbol: str = "", verification: str = "",
|
||||
) -> dict:
|
||||
"""List recorded snippets — the project's pattern library.
|
||||
|
||||
Search here BEFORE building any shape (a component, control, route
|
||||
handler, service class, helper, scaffold): a recorded shape is the
|
||||
starting point for every later instance, and building without checking is
|
||||
how the same button ends up defined four diverging ways.
|
||||
|
||||
Two ways to ask, usable together: by MEANING (`q` — "what do I need this code
|
||||
to do?") and by PLACE (`repo`/`path`/`symbol` — "what canonical helpers
|
||||
already live in the file I'm about to edit?"). Reach for the place form
|
||||
before you write or change code in a file: it is the cheap way to find prior
|
||||
art you'd otherwise duplicate a few lines down.
|
||||
|
||||
Args:
|
||||
q: Free-text search across name + body (optional). Matches on meaning as
|
||||
well as wording, so describe what you need the code to DO.
|
||||
tag: Filter to a single tag, e.g. a language like "python" (optional).
|
||||
limit: Max results (1-100).
|
||||
offset: Skip this many before returning — page through a corpus
|
||||
larger than one call. `total` is the unpaged count, so
|
||||
offset+limit against it says whether more remains.
|
||||
project_id: Narrow to one project. 0 (default) searches every project —
|
||||
usually what you want, since a helper you need here may well have
|
||||
been written somewhere else.
|
||||
repo: Narrow to snippets recorded in this repo, matched exactly — the
|
||||
same repo string used when recording, e.g. "Grimoire".
|
||||
path: Narrow to snippets recorded at this path. Matches the exact file
|
||||
OR anything beneath it, so "frontend/src" finds
|
||||
"frontend/src/lib/x.ts" as well as itself.
|
||||
symbol: Narrow to snippets recorded under this symbol name, exactly.
|
||||
verification: Narrow on the drift check (see verify_snippet).
|
||||
"attention" is the one to reach for — everything whose recorded
|
||||
location or code no longer checks out, plus everything whose verdict
|
||||
expired because the snippet was edited after it was checked. Also
|
||||
accepts "ok", "unverified", "drifted", or a specific failure:
|
||||
"missing", "moved", "changed".
|
||||
|
||||
`repo`/`path`/`symbol` must all match the SAME recorded location, so a
|
||||
snippet that lives in repo A and, separately, at path B in another repo is
|
||||
not returned for repo=A + path=B.
|
||||
|
||||
Returns {"snippets": [{id, title, when_to_use, tags, preview, usage}],
|
||||
"total": int}. The title is the snippet's name and `when_to_use` says when
|
||||
to reach for it; open one in full with get_snippet(id).
|
||||
|
||||
`usage` is {surfaced_count, pull_count, last_surfaced_at, last_pulled_at}:
|
||||
how often the entry has been put in front of an agent versus actually
|
||||
opened. Treat a high surfaced_count with a zero pull_count as a prompt to
|
||||
fix the record — usually its "when to reach for it" doesn't say when — or to
|
||||
delete it. Such an entry is not harmless: it takes a slot in every future
|
||||
auto-inject menu and crowds out something useful.
|
||||
|
||||
An entry marked `shared: true` with an `owner` belongs to someone else — one
|
||||
person's suggestion, not settled practice here. Weigh it on its merits and
|
||||
attribute it when you use it. Searching (passing `q`) also reaches snippets
|
||||
shared directly with the operator; browsing without a query deliberately
|
||||
does not, so those stay out of ambient results until asked for.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
items, total = await snippets_svc.list_snippets(
|
||||
uid, q=q or None, tag=tag, limit=max(1, min(limit, 100)),
|
||||
offset=max(0, offset),
|
||||
project_id=project_id or None,
|
||||
repo=repo, path=path, symbol=symbol, verification=verification,
|
||||
)
|
||||
labeled = await access_svc.label_shared_items(uid, items)
|
||||
await attach_usage(labeled)
|
||||
return {"snippets": labeled, "total": total}
|
||||
|
||||
|
||||
async def create_snippet(
|
||||
name: str,
|
||||
code: str,
|
||||
language: str = "",
|
||||
signature: str = "",
|
||||
when_to_use: str = "",
|
||||
repo: str = "",
|
||||
path: str = "",
|
||||
symbol: str = "",
|
||||
locations: list[dict] | None = None,
|
||||
tags: list[str] | None = None,
|
||||
project_id: int = 0,
|
||||
system_ids: list[int] | None = None,
|
||||
force: bool = False,
|
||||
commit_sha: str = "",
|
||||
notes: str = "",
|
||||
) -> dict:
|
||||
"""Record a shape in the project's pattern library, so every later
|
||||
instance starts from it instead of re-deriving it.
|
||||
|
||||
IS THE SHAPE THE POINT, OR THE ADVICE? A snippet is code with a LOCATION —
|
||||
that is what lets it surface from the file someone is about to edit. If
|
||||
what wants recording is a standing instruction about how to work, it is a
|
||||
rule (create_rule); a procedure followed start to finish is a process
|
||||
(create_process); what you LEARNED rather than what to copy is a note
|
||||
(create_note).
|
||||
|
||||
Reach for this the FIRST time any shape is built — a component, a control,
|
||||
a route handler, a service class, a helper, a test scaffold — not only
|
||||
when something is judged "reusable": the builder of the first instance
|
||||
can't know what will recur, and a missed record is invisible until it
|
||||
resurfaces as an uninformed duplicate. Over-recording is safe (dead weight
|
||||
shows in the usage counters and can be pruned); under-recording has no
|
||||
signal. A deliberate departure from a recorded shape is recorded as its
|
||||
own named variant, not left as drift. Before building, search first — the
|
||||
shape may already be recorded.
|
||||
|
||||
Args:
|
||||
name: Short name of the function/component, e.g. "useDebouncedRef".
|
||||
code: The code itself (required).
|
||||
language: Language/format, e.g. "python", "vue", "sql". Becomes a tag and
|
||||
the code-fence language.
|
||||
signature: One-line signature/interface, e.g. "debounce(fn, ms) -> fn".
|
||||
when_to_use: The situation to reach for it in — a sentence or two, e.g.
|
||||
"Debouncing a reactive input before it triggers a fetch." This is
|
||||
what the snippet is RANKED on: it is joined onto the name in every
|
||||
vector the snippet is embedded as, so each extra paragraph blurs the
|
||||
one situation it should surface for. Several distinct situations
|
||||
are fine; the explanation is not — that goes in `notes`.
|
||||
notes: Everything worth saying that is not the situation: why it is
|
||||
shaped this way, what it replaced, caveats, history. Stored after
|
||||
the code under its own heading and shown with the snippet. When a
|
||||
later session learns something about the snippet, it goes here.
|
||||
repo/path/symbol: Canonical location of the reference implementation.
|
||||
locations: Several locations at once, as [{"repo","path","symbol"}, ...],
|
||||
when you already know the thing lives in more than one place. Takes
|
||||
precedence over the single repo/path/symbol shorthand.
|
||||
tags: Extra plain-string tags (language + "snippet" are added for you).
|
||||
project_id: Associate with a project (0 = no project). Snippets surface
|
||||
proactively within their project; search finds them across projects.
|
||||
system_ids: Ids of the project's Systems to associate this snippet with.
|
||||
force: Bypass the near-duplicate gate (see below).
|
||||
commit_sha: The commit the code was read at (`git rev-parse HEAD` — you
|
||||
have the repo, so it's free). The recorded location is the source
|
||||
of truth for the code and the stored body is a cache of it; this
|
||||
stamps what the cache is a cache OF, so staleness is judgeable
|
||||
later. Optional, but pass it whenever you're recording from a
|
||||
checkout.
|
||||
|
||||
When `when_to_use` reads like a write-up — long, several paragraphs, or
|
||||
headed — the response carries `trigger_advice`: move the explanation into
|
||||
`notes` with update_snippet.
|
||||
|
||||
Returns the created snippet (including a parsed `snippet` field), OR — when a
|
||||
duplicate already exists and force is false — {"duplicate": true,
|
||||
"existing_id": ..., "message": ...} and nothing is created. When that happens
|
||||
and it really is the same reusable thing found in another place, prefer
|
||||
merge_snippets(existing_id, [new...]) — or record then merge — to unify them
|
||||
into ONE canonical record (which then carries every call site as a location),
|
||||
rather than forcing a second copy with force=true. A tagged record shows
|
||||
its `systems`; created untagged in a project, the response carries the
|
||||
`systems_hint` question instead — answer it: tag, create the missing
|
||||
System, or deliberately skip. A `family_hint` means the shape may be one
|
||||
every project on a shared platform will need, and an evaluation was
|
||||
opened — the family-canon skill says how to judge it.
|
||||
|
||||
WHAT THE GATE MATCHES ON. Exact identity first — an existing snippet at the
|
||||
same repo · path · symbol, or holding byte-identical code. Those are certain,
|
||||
and force is almost never the right answer to them. Only then a semantic
|
||||
check, held to a high bar so that VARIANTS of one component are not refused:
|
||||
`.btn-primary` and `.btn-secondary` read alike and are two different things,
|
||||
so record both.
|
||||
"""
|
||||
if not (name or "").strip() or not (code or "").strip():
|
||||
raise ValueError("create_snippet requires a non-empty name and code")
|
||||
uid = current_user_id()
|
||||
|
||||
# The NAME is the title; the trigger rides in `data` and
|
||||
# joins the title only in the embedded document.
|
||||
title = name.strip()
|
||||
body = snippets_svc.compose_body(
|
||||
code=code, language=language, signature=signature,
|
||||
when_to_use=when_to_use, repo=repo, path=path, symbol=symbol,
|
||||
locations=locations, notes=notes,
|
||||
)
|
||||
if not force:
|
||||
dup = await dedup_svc.find_duplicate_note(
|
||||
uid, title, body, project_id=project_id or None,
|
||||
is_task=False, note_type=snippets_svc.SNIPPET_NOTE_TYPE,
|
||||
# The artefact itself, not just its description — the gate compares
|
||||
# location and code before it compares prose.
|
||||
code=code,
|
||||
locations=snippets_svc.resolve_locations(repo, path, symbol, locations),
|
||||
data=snippets_svc.compose_data(name=name, when_to_use=when_to_use),
|
||||
)
|
||||
if dup is not None:
|
||||
return dedup_svc.duplicate_response(dup, "snippet")
|
||||
|
||||
note = await snippets_svc.create_snippet(
|
||||
uid, name=name, code=code, language=language, signature=signature,
|
||||
when_to_use=when_to_use, repo=repo, path=path, symbol=symbol,
|
||||
locations=locations, tags=tags, project_id=project_id or None,
|
||||
commit_sha=commit_sha, notes=notes,
|
||||
)
|
||||
if system_ids:
|
||||
await systems_svc.set_record_systems(uid, note.id, system_ids)
|
||||
data = snippets_svc.snippet_to_dict(note)
|
||||
await systems_tools.attach_systems(uid, uid, data, note.id, project_id or None)
|
||||
advice = snippets_svc.trigger_advice(when_to_use)
|
||||
if advice:
|
||||
data["trigger_advice"] = advice
|
||||
# Family canon: the same shape recorded in another project
|
||||
# on a shared platform opens an evaluation of the earlier one.
|
||||
await family_svc.attach_family_hint(uid, data, note, created=True)
|
||||
return await moment_delivery.attach_moment_rules(uid, "create_snippet", {"project_id": project_id}, data)
|
||||
|
||||
|
||||
async def get_snippet(snippet_id: int, project_id: int = 0) -> dict:
|
||||
"""Fetch a snippet by id — the full record: code, signature, location, and a
|
||||
parsed `snippet` field of its structured parts.
|
||||
|
||||
On an instance with a forge configured, the response also carries
|
||||
`body_source` + `body_freshness`: "current" means the code was just
|
||||
confirmed against the recorded location; "diverged" or "missing" means
|
||||
the source moved on — trust the location over the cached body and
|
||||
consider verify_snippet after you look.
|
||||
|
||||
A record kept VERBATIM is confirmed by containment. A deliberately
|
||||
ANNOTATED one — commentary the source does not carry — cannot be, so it
|
||||
reads "current" on the authority of a standing `ok` verdict stamped at
|
||||
the very commit just fetched; `verification` in the same payload
|
||||
shows that basis. Edit the record, or let the file move past that commit,
|
||||
and it reads "diverged" again until someone re-runs verify_snippet.
|
||||
|
||||
When the shape ledger has judgments against this snippet, the response
|
||||
carries `instances` (shapes classified as conforming to it — the
|
||||
structured consumer map) and/or `variants` (named departures, each with
|
||||
its why). Consult them before changing the snippet's contract: they are
|
||||
the call sites your change lands on (classify_shapes maintains them).
|
||||
|
||||
If the record belongs to someone else it carries `shared: true` with the
|
||||
`owner` and your `permission`. Read that as ONE PERSON'S SUGGESTION, not as
|
||||
established practice here: judge it on its merits, say whose it is when you
|
||||
reference it, and don't adopt it as the house pattern without checking.
|
||||
|
||||
`project_id` is the project you are WORKING IN, not this record's own.
|
||||
Passing the active project is what makes "opened away from where it was
|
||||
written" answerable; 0 leaves it unreported and the pull still counts.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
note = await snippets_svc.get_snippet(uid, snippet_id)
|
||||
if note is None:
|
||||
raise ValueError(f"snippet {snippet_id} not found")
|
||||
data = snippets_svc.snippet_to_dict(note)
|
||||
# Forge-checked freshness: attaches body_source/body_freshness
|
||||
# when the instance has a forge; a no-forge instance sees no new fields.
|
||||
await snippets_svc.attach_live_body(note, data)
|
||||
data.update(await access_svc.describe_provenance(uid, note))
|
||||
# A "pull" is an explicit open, so it's recorded HERE rather than in
|
||||
# snippets_svc.get_snippet — the service is also reached by update/merge
|
||||
# paths, and counting those would inflate exactly the number that is
|
||||
# supposed to mean "someone chose to look at this".
|
||||
record_pulled(
|
||||
user_id=uid, note_id=int(note.id),
|
||||
source="mcp_get_snippet", project_id=project_id,
|
||||
)
|
||||
await systems_tools.attach_systems(
|
||||
uid, note.user_id, data, note.id, note.project_id
|
||||
)
|
||||
# The structured consumer map: ledger rows judged against this
|
||||
# snippet. Attached only when non-empty — and never for projects
|
||||
# the caller can't read.
|
||||
from grimoire.services import shape_ledger as shape_ledger_svc
|
||||
|
||||
consumers = await shape_ledger_svc.snippet_consumers(uid, int(note.id))
|
||||
if consumers["instances"]:
|
||||
data["instances"] = consumers["instances"]
|
||||
if consumers["variants"]:
|
||||
data["variants"] = consumers["variants"]
|
||||
if consumers.get("uses"):
|
||||
# The call sites: shapes that use this snippet, whatever
|
||||
# shape they are themselves.
|
||||
data["uses"] = consumers["uses"]
|
||||
return data
|
||||
|
||||
|
||||
async def unmerge_snippet(survivor_id: int, source_id: int) -> dict:
|
||||
"""Reverse ONE source out of a merged snippet — the inverse of merge_snippets.
|
||||
|
||||
Restores the source record and strips exactly what it contributed from the
|
||||
survivor: the locations and tags it ADDED at merge time, never the ones the
|
||||
survivor already had. Reach for it when a merge turns out to have unified two
|
||||
things that only looked alike.
|
||||
|
||||
Also the fix for a half-undone merge. Restoring a merged-in source from the
|
||||
trash by hand brings the record back but leaves the survivor still claiming
|
||||
its call sites, so both records claim the same places and the reverse lookup
|
||||
reads the duplicate claims as real. Running this on an already-restored
|
||||
source repairs that: it skips the restore and does the subtraction.
|
||||
|
||||
Args:
|
||||
survivor_id: The snippet that absorbed the other.
|
||||
source_id: The snippet to pull back out of it.
|
||||
|
||||
Returns {"survivor": {...}, "restored": {...}}.
|
||||
|
||||
Refuses, with the reason, when: the survivor has no record of absorbing that
|
||||
id; the source was purged from the trash; or the merge predates per-source
|
||||
provenance, in which case what it contributed isn't known and subtracting a
|
||||
guess could strip call sites the survivor genuinely owns — restore it from
|
||||
the trash and adjust both records by hand instead.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
try:
|
||||
result = await snippets_svc.unmerge_snippet(uid, survivor_id, source_id)
|
||||
except snippets_svc.UnmergeError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
if result is None:
|
||||
raise ValueError(f"snippet {survivor_id} not found")
|
||||
survivor, restored = result
|
||||
return {
|
||||
"survivor": snippets_svc.snippet_to_dict(survivor),
|
||||
"restored": snippets_svc.snippet_to_dict(restored) if restored else None,
|
||||
}
|
||||
|
||||
|
||||
async def find_duplicate_snippets(threshold: float = 0.0) -> dict:
|
||||
"""Find snippets already recorded that look like duplicates of each other.
|
||||
|
||||
The create gate PREVENTS a new duplicate and merge_snippets CURES one you
|
||||
point it at — this is the missing third piece: it FINDS the ones already in
|
||||
the record, so nobody has to notice them by hand.
|
||||
|
||||
Results are grouped into candidate merge SETS, not just pairs. Grouping is
|
||||
transitive: if A resembles B and B resembles C, all three land in one set
|
||||
even when A and C don't directly clear the bar. That mirrors what merge does
|
||||
(it folds every source into one survivor), but it means a chain of mild
|
||||
resemblances can rope in a member that isn't really alike — so read a set as
|
||||
a proposal and check the members before acting.
|
||||
|
||||
Reports only YOUR snippets. merge_snippets requires one owner across the
|
||||
whole set, so surfacing someone else's would propose a merge that can't be
|
||||
performed.
|
||||
|
||||
Acting on a group: pick the best record as the canonical target, then
|
||||
`merge_snippets(target_id, [other ids])`. Merge unions the fields and folds
|
||||
every source's location in, so the survivor is findable at all their call
|
||||
sites; the sources are trashed, recoverably. Prefer as target the one with
|
||||
the clearest "when to reach for it" — merge keeps the target's title.
|
||||
|
||||
Args:
|
||||
threshold: Similarity floor, 0-1. 0 (default) uses the configured
|
||||
setting. Raise it if the report is noisy, lower it to catch more.
|
||||
|
||||
Returns {"groups": [{"note_ids", "snippets", "top_score"}], "pairs",
|
||||
"threshold"}. An empty `groups` means nothing resembles anything else that
|
||||
closely — the common and desirable case.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
return await dedup_svc.find_duplicate_snippets(
|
||||
uid, threshold=threshold if threshold > 0 else None
|
||||
)
|
||||
|
||||
|
||||
async def verify_snippet(
|
||||
snippet_id: int, status: str, detail: str = "", path: str = "",
|
||||
commit_sha: str = "",
|
||||
) -> dict:
|
||||
"""Record whether a snippet's recorded location and code still match source.
|
||||
|
||||
YOU do the checking — Grimoire has no copy of the repo and deliberately never
|
||||
gets one. This tool only remembers your verdict so it becomes queryable and
|
||||
so the operator can see what has rotted.
|
||||
|
||||
The procedure, once per snippet you're checking:
|
||||
1. `get_snippet(id)` — read its `snippet.locations` and `snippet.code`.
|
||||
2. Does the recorded path still exist in the working tree? If not →
|
||||
status="missing".
|
||||
3. Does the recorded symbol still appear in that file? If not →
|
||||
status="moved" (the file is there, the thing isn't).
|
||||
4. Does the source still match the recorded code, allowing for formatting?
|
||||
Judge whether it still does the same thing — an added parameter or a
|
||||
changed branch is "changed"; a reindent is not. If it diverged →
|
||||
status="changed".
|
||||
5. All three hold → status="ok".
|
||||
|
||||
Put what you actually found in `detail` ("renamed to parse_location_str",
|
||||
"moved to services/knowledge.py"). It's what makes the record fixable later
|
||||
by someone who wasn't here, so write it for them, not as a status echo.
|
||||
|
||||
CONSUMERS you enumerate while checking ("all N call sites still route
|
||||
through it") are ledger rows, not detail prose: classify each consuming
|
||||
definition as an `instance` of this snippet with classify_shapes — prose
|
||||
cannot be sorted, queried, or diffed, and the rows
|
||||
are what make "what uses this?" answerable forever. `detail` keeps the
|
||||
WHY and what changed, nothing that belongs in a row.
|
||||
|
||||
A verdict expires automatically if the snippet is edited afterwards: it is
|
||||
stamped with a hash of the code it was checked against, so it can never go
|
||||
on vouching for code nobody checked. Re-verify after fixing a record.
|
||||
|
||||
Args:
|
||||
snippet_id: The snippet you checked.
|
||||
status: "ok" | "missing" | "moved" | "changed".
|
||||
detail: What you found — free text, shown to the operator.
|
||||
path: The path you actually checked, if it differs from the recorded
|
||||
one (e.g. you found the symbol at its new home). Defaults to the
|
||||
recorded path.
|
||||
commit_sha: The commit the working tree was at when you checked
|
||||
(`git rev-parse HEAD`). An "ok" verdict with it also refreshes the
|
||||
body's provenance — you just proved the cached code matches the
|
||||
source at that commit.
|
||||
|
||||
Requires write access: a verdict changes how the record is presented, so
|
||||
being able to read a snippet someone shared with you doesn't let you mark
|
||||
it broken.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
note = await snippets_svc.record_verification(
|
||||
uid, snippet_id, status=status, detail=detail, path=path,
|
||||
commit_sha=commit_sha,
|
||||
)
|
||||
if note is None:
|
||||
raise ValueError(
|
||||
f"snippet {snippet_id} not found, or you don't have write access to it"
|
||||
)
|
||||
return snippets_svc.snippet_to_dict(note)
|
||||
|
||||
|
||||
async def update_snippet(
|
||||
snippet_id: int,
|
||||
name: str | None = None,
|
||||
code: str | None = None,
|
||||
language: str | None = None,
|
||||
signature: str | None = None,
|
||||
when_to_use: str | None = None,
|
||||
repo: str | None = None,
|
||||
path: str | None = None,
|
||||
symbol: str | None = None,
|
||||
locations: list[dict] | None = None,
|
||||
tags: list[str] | None = None,
|
||||
project_id: int = 0,
|
||||
system_ids: list[int] | None = None,
|
||||
commit_sha: str = "",
|
||||
notes: str | None = None,
|
||||
) -> dict:
|
||||
"""Update a snippet. Only the fields you pass change.
|
||||
|
||||
An omitted field is left alone; an EMPTY STRING clears it — so a stale
|
||||
signature, a wrong "when to use", or an obsolete location can be removed, not
|
||||
just overwritten. A snippet that surfaces in recall with wrong details is
|
||||
worse than none, so correcting downward has to be possible.
|
||||
|
||||
Args:
|
||||
when_to_use: The situation to reach for it in — a sentence or two. It
|
||||
is what the snippet is ranked on, so correct it toward the
|
||||
situation; an explanation belongs in `notes`.
|
||||
notes: Replaces the free-text notes (why, history, caveats). Pass the
|
||||
whole text — read the current notes from get_snippet first when
|
||||
adding to them.
|
||||
locations: Replace the whole location set, as [{"repo","path","symbol"},
|
||||
...]. Pass [] to clear every location. The single repo/path/symbol
|
||||
args instead overlay onto the FIRST location, leaving the rest.
|
||||
tags: Replaces the extra-tag set (language + "snippet" are re-derived).
|
||||
project_id: 0 leaves it unchanged, -1 detaches it from its project, a
|
||||
positive id moves it.
|
||||
commit_sha: When you're updating the code from a checkout, the commit
|
||||
it was read at (`git rev-parse HEAD`). Restamps the body's
|
||||
provenance; changing the code WITHOUT it drops the old stamp,
|
||||
since the new body no longer comes from that commit.
|
||||
|
||||
Editing someone else's snippet requires an editor or admin share from them.
|
||||
A read-only share is refused with a message saying so — record your own
|
||||
version instead of trying to force it.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
if project_id == 0:
|
||||
project = snippets_svc.UNSET
|
||||
elif project_id < 0:
|
||||
project = None
|
||||
else:
|
||||
project = project_id
|
||||
|
||||
try:
|
||||
note = await snippets_svc.update_snippet(
|
||||
uid, snippet_id,
|
||||
name=name, code=code, language=language,
|
||||
signature=signature, when_to_use=when_to_use,
|
||||
repo=repo, path=path, symbol=symbol,
|
||||
locations=locations, tags=tags, project_id=project,
|
||||
commit_sha=commit_sha or None, notes=notes,
|
||||
)
|
||||
except PermissionError as exc:
|
||||
# Readable but not writable — surface the real reason, not "not found".
|
||||
raise ValueError(str(exc)) from exc
|
||||
if note is None:
|
||||
raise ValueError(f"snippet {snippet_id} not found")
|
||||
if system_ids is not None:
|
||||
# The CALLER, not the owner. `update_snippet` above already
|
||||
# raised PermissionError if this user may not write, so the tagging
|
||||
# runs with the actor's own System visibility rather than the owner's.
|
||||
await systems_svc.set_record_systems(uid, snippet_id, system_ids)
|
||||
data = snippets_svc.snippet_to_dict(note)
|
||||
data.update(await access_svc.describe_provenance(uid, note))
|
||||
await systems_tools.attach_systems(
|
||||
uid, note.user_id, data, snippet_id, note.project_id
|
||||
)
|
||||
# Advised on the trigger as it now STANDS, not only when this call set it:
|
||||
# an edit to anything else is the moment someone is already in the record.
|
||||
advice = snippets_svc.trigger_advice(data["snippet"].get("when_to_use"))
|
||||
if advice:
|
||||
data["trigger_advice"] = advice
|
||||
return data
|
||||
|
||||
|
||||
async def delete_snippet(snippet_id: int) -> dict:
|
||||
"""Retire a snippet you recorded — it moves to the trash and is recoverable.
|
||||
|
||||
Reach for this when a snippet is wrong, obsolete, or was never worth keeping.
|
||||
A recorded snippet is offered as prior art on every matching turn, so a bad
|
||||
one costs more than a missing one. If instead it's a duplicate of something
|
||||
that should survive, prefer merge_snippets — that keeps the call sites.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
# Read before deleting so the confirmation can NAME what went — an id
|
||||
# alone leaves the operator unable to tell which snippet this was.
|
||||
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
|
||||
# that errors must not stop the delete happening. Same posture the
|
||||
# staleness marker takes — a decoration may never break its payload.
|
||||
try:
|
||||
doomed = await snippets_svc.get_snippet(uid, snippet_id)
|
||||
title = getattr(doomed, "title", "") if doomed else ""
|
||||
except Exception:
|
||||
title = ""
|
||||
if not await snippets_svc.delete_snippet(uid, snippet_id):
|
||||
raise ValueError(f"snippet {snippet_id} not found")
|
||||
return {"deleted": True, "id": snippet_id, "title": title}
|
||||
|
||||
|
||||
async def merge_snippets(target_id: int, source_ids: list[int]) -> dict:
|
||||
"""Unify duplicate/variant snippets INTO one canonical record — the cure for
|
||||
the same reusable thing recorded as several one-offs.
|
||||
|
||||
Keeps the target as the canonical: its name, when-to-use, signature, language
|
||||
and code win. The sources' locations and extra tags are folded in — so the
|
||||
survivor ends up carrying EVERY call site as a location (which is itself the
|
||||
"this is duplicated N times" signal) — and the source records are moved to the
|
||||
trash (recoverable). The survivor is re-embedded so recall stops surfacing the
|
||||
now-merged duplicates.
|
||||
|
||||
The survivor records what it absorbed as `merged_from` (in its `snippet`
|
||||
fields and as a "Merged from: #ids" line in the body), so a variant that got
|
||||
folded in leaves a trace outside the trash. It accumulates across merges.
|
||||
|
||||
Reversible: each entry also records what that source contributed, so
|
||||
`unmerge_snippet(target_id, source_id)` can restore it and strip exactly
|
||||
those locations back off — never the ones the target already had.
|
||||
|
||||
Args:
|
||||
target_id: The snippet to keep (the canonical record).
|
||||
source_ids: Snippet ids to fold into the target and retire. Ids that
|
||||
aren't your snippets are skipped; target_id in the list is ignored.
|
||||
|
||||
Returns the merged canonical snippet (with a parsed `snippet` field) plus
|
||||
`merged_ids` — the source ids actually merged and trashed.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
ids = [s for s in (source_ids or []) if s != target_id]
|
||||
if not ids:
|
||||
raise ValueError("merge_snippets requires at least one other source_id")
|
||||
try:
|
||||
result = await snippets_svc.merge_snippets(uid, target_id, ids)
|
||||
except PermissionError as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
if result is None:
|
||||
raise ValueError(f"snippet {target_id} not found")
|
||||
note, merged_ids = result
|
||||
data = snippets_svc.snippet_to_dict(note)
|
||||
data["merged_ids"] = merged_ids
|
||||
return data
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_snippets, create_snippet, get_snippet, update_snippet,
|
||||
delete_snippet, merge_snippets, verify_snippet, find_duplicate_snippets,
|
||||
unmerge_snippet,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,480 @@
|
||||
"""System CRUD + record-association MCP tools — wrappers over services/systems.py.
|
||||
|
||||
A System is a per-project, reusable, self-describing subsystem/area that any
|
||||
record (note, task, or issue) can be associated with — so research, build-work,
|
||||
and corrective work line up under the same area and recurring problem-spots are
|
||||
visible. The service enforces the multi-user ACL (project permission); these
|
||||
tools are thin wrappers.
|
||||
|
||||
Sentinels (match the milestone/task tool conventions):
|
||||
- name="" / description="" / color="" / status="" → "leave unchanged" on update
|
||||
- order_index=-1 → "leave unchanged" on update (0 is a valid order_index)
|
||||
- path_patterns=None → "leave unchanged" on update ([] clears them)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import canonical_systems as canonical_systems_svc
|
||||
from grimoire.services import milestones as milestones_svc
|
||||
from grimoire.services import notes as notes_svc
|
||||
from grimoire.services import systems as systems_svc
|
||||
from grimoire.services import system_usage as system_usage_svc
|
||||
from grimoire.services.system_usage import record_system_pulled
|
||||
|
||||
# Below this, a project is young enough that the mild "which area is this
|
||||
# about?" question stays proportionate; at or above it, a zero-Systems project
|
||||
# has shown that the mild question does not get answered, and the ask
|
||||
# escalates.
|
||||
_BOOTSTRAP_MIN_RECORDS = 20
|
||||
_BOOTSTRAP_TITLES = 6
|
||||
|
||||
# The standard vocabulary: area names that recur across software
|
||||
# projects, offered so "CI & Release" means the same thing in every project
|
||||
# on the instance. Consistency comes from the shared names — NOT from asking
|
||||
# the operator to approve each System; agents mint directly. Generic by
|
||||
# design: archetypes any codebase could have, never one install's
|
||||
# subsystems. Mint freely beyond the list; the duplicate gate guards sprawl.
|
||||
# The vocabulary lives in the GLOBAL canonical catalog — the inception seed
|
||||
# mints it and this ask names it, one list for both, in a table so a rule can
|
||||
# reference an area by id.
|
||||
|
||||
|
||||
async def bootstrap_systems_ask(user_id: int, project_id: int) -> str | None:
|
||||
"""The escalated vocabulary-bootstrap ask for a mature zero-Systems project.
|
||||
|
||||
The generic zero-state question habituates: identical on every record,
|
||||
maximal in scope ("invent the taxonomy"), asked at wrap-up time — so
|
||||
organic sessions skip it forever and only audit-shaped sessions ever mint.
|
||||
What separates the nudges that convert from the prose that
|
||||
doesn't (the duplicate gate, the prior-art "already defined in 2 files")
|
||||
is the project's OWN evidence in the ask — so this one carries the record
|
||||
count and the recent titles, and asks for a concrete deliverable: create
|
||||
a starter set directly, preferring the standard cross-project names.
|
||||
Deliberately NOT an approval flow: the operator is not a
|
||||
permission gate for vocabulary; the standard names carry consistency.
|
||||
|
||||
Self-retiring by construction: callers only reach for it while the
|
||||
project has zero Systems, so the first create_system ends it everywhere.
|
||||
Returns None below the record threshold or on any failure (fail-open —
|
||||
a hint must never break the call it rides on).
|
||||
"""
|
||||
try:
|
||||
recent, total = await notes_svc.list_notes(
|
||||
user_id, project_id=project_id, sort="updated_at",
|
||||
limit=_BOOTSTRAP_TITLES,
|
||||
)
|
||||
except Exception:
|
||||
return None
|
||||
if total < _BOOTSTRAP_MIN_RECORDS:
|
||||
return None
|
||||
titles = "; ".join(
|
||||
'"' + " ".join((n.title or "").split())[:70] + '"' for n in recent
|
||||
)
|
||||
try:
|
||||
standard = ", ".join(name for name, _charter in await systems_svc.standard_systems())
|
||||
except Exception:
|
||||
standard = ""
|
||||
# An install whose catalog is empty still gets a usable ask — the standard
|
||||
# names are an aid, not the point of the question.
|
||||
standard_line = (
|
||||
"Where an area fits a standard name, use it verbatim so it means the "
|
||||
f"same thing in every project: {standard}. Mint freely beyond that "
|
||||
"list — the duplicate gate guards sprawl. "
|
||||
) if standard else "The duplicate gate guards sprawl. "
|
||||
return (
|
||||
f"This project has {total} records and NO Systems modelled — none of "
|
||||
"them can be tagged to an area, so recurring problem-spots stay "
|
||||
"invisible. Bootstrap the vocabulary now, in this session, without "
|
||||
"asking permission — creating Systems is your call, not an approval "
|
||||
f"flow. From the areas the records themselves name (recent: {titles}), "
|
||||
"create_system 3-6 Systems, each with a one-paragraph charter, then "
|
||||
f"tag this record (system_ids=[...]). {standard_line}"
|
||||
"This ask repeats until the first "
|
||||
"System exists; answering it once retires it for every future record."
|
||||
)
|
||||
|
||||
|
||||
async def untagged_systems_hint(user_id: int, project_id: int) -> str | None:
|
||||
"""The Systems question, for an untagged project record.
|
||||
|
||||
Not a tool — attach_systems() rides this on tool responses so the question
|
||||
arrives in-band at the exact moment a record is touched. It is ONE
|
||||
question regardless of vocabulary state ("which area is this about, and is
|
||||
it modelled?"); only the vocabulary listing varies, because an empty
|
||||
vocabulary is not an exemption — it is the question at its most urgent.
|
||||
Instruction prose alone does not fire at write time; in-band behavior
|
||||
(the duplicate gate) does.
|
||||
|
||||
In a MATURE zero-Systems project the question escalates to the bootstrap
|
||||
ask instead: the mild form does not get answered there, and
|
||||
an evidence-carrying, deliverable-shaped ask is the form that does.
|
||||
"""
|
||||
# Fail-open like the dedup gate: a hint must never break the call.
|
||||
try:
|
||||
systems = await systems_svc.list_systems(user_id, project_id)
|
||||
except Exception:
|
||||
return None
|
||||
if systems:
|
||||
vocab = "Existing Systems: " + ", ".join(
|
||||
f"#{s.id} {s.name}" for s in systems
|
||||
) + "."
|
||||
else:
|
||||
ask = await bootstrap_systems_ask(user_id, project_id)
|
||||
if ask:
|
||||
return ask
|
||||
vocab = "This project has no Systems yet."
|
||||
return (
|
||||
"This record is untagged — which area(s) of the project is it about? "
|
||||
f"{vocab} Tag it (system_ids=[...]) — cross-cutting records like "
|
||||
"audits take several; create_system any area it concerns that isn't "
|
||||
"modelled yet (a sweep that names areas is the moment to mint them, "
|
||||
"and the duplicate gate guards against sprawl). Leave it untagged "
|
||||
"only if it is genuinely about no particular area."
|
||||
)
|
||||
|
||||
|
||||
async def attach_systems(
|
||||
caller_id: int,
|
||||
owner_id: int,
|
||||
data: dict,
|
||||
note_id: int,
|
||||
project_id: int | None,
|
||||
) -> None:
|
||||
"""Attach a record's Systems to its payload — or the Systems question.
|
||||
|
||||
ONE seam for every tool that returns a project record, read or write: a
|
||||
tagged record shows its areas (the touching-a-System reflex needs the
|
||||
affiliation visible on read, not just settable on write), an untagged
|
||||
project record carries the question instead. Neither field is ever
|
||||
attached empty (same reasoning as supersession_svc.attach_relations — a
|
||||
field that always says nothing trains readers to skip fields). The hint
|
||||
goes only to the record's owner: tagging someone else's record in someone
|
||||
else's project is not the caller's call to make. Fail-open — decoration
|
||||
must never break the call it rides on.
|
||||
"""
|
||||
try:
|
||||
systems = await systems_svc.list_record_systems(owner_id, note_id)
|
||||
if systems:
|
||||
data["systems"] = [s.to_dict() for s in systems]
|
||||
elif project_id and caller_id == owner_id:
|
||||
hint = await untagged_systems_hint(owner_id, project_id)
|
||||
if hint:
|
||||
data["systems_hint"] = hint
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
async def create_system(
|
||||
project_id: int,
|
||||
name: str,
|
||||
description: str = "",
|
||||
color: str = "",
|
||||
path_patterns: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Create a System (a reusable, self-describing subsystem/area) in a project.
|
||||
|
||||
Associate records with it via the `system_ids` arg on create/update_task and
|
||||
create/update_note.
|
||||
|
||||
Create one the moment two records would share an area that has no System
|
||||
yet — the same two-or-more test snippets use. Don't wait to be asked to
|
||||
name an area that plainly exists in the code, and don't route the
|
||||
creation through operator approval — minting vocabulary is the agent's
|
||||
call; an unmodelled area means every record about it stays
|
||||
untaggable. An audit or sweep that walks the codebase is a DISCOVERY
|
||||
moment: mint the Systems it names as it names them — the duplicate gate
|
||||
below, plus reviewing the existing list, is what guards against sprawl,
|
||||
not holding back. Prefer the standard cross-project names where the area
|
||||
fits one (CI & Release, Auth & Access, Data Model & Storage, API Surface,
|
||||
UI & Design, Import & Export, Background Jobs, Observability) so the same
|
||||
word means the same thing in every project. Give each one a one-paragraph
|
||||
charter, not just a label: the description is what tells a later session
|
||||
whether a record belongs here.
|
||||
|
||||
The description is also where the operator's RULINGS for the area live:
|
||||
a `Rulings` section at its end, one line per decision the operator made
|
||||
about how this area must behave — the statement, who decided, when, and
|
||||
the record it came from. The description arrives with every record filed
|
||||
under the System, so a ruling there reaches each session working in the
|
||||
area; a quote in a work log reaches one only if a search matches it.
|
||||
|
||||
`path_patterns` name the files that ARE the area: globs relative to the
|
||||
repo root, `*` within one directory, `**` across any depth, and a plain
|
||||
directory covering everything under it (`src/billing`,
|
||||
`frontend/src/components/Billing*.vue`). The description says what the
|
||||
area is for; the patterns say where it lives, so work touching those files
|
||||
can be traced to the System without a search. Give them when the area's
|
||||
files are known; a System without them still works for tagging.
|
||||
|
||||
Args:
|
||||
project_id: The project this system belongs to (required).
|
||||
name: Short label (required).
|
||||
description: What the system is and how it's used — a name is rarely enough.
|
||||
color: Optional UI accent (hex), or empty.
|
||||
path_patterns: The area's files as repo-relative globs, or omit.
|
||||
|
||||
Duplicate-gated like the other creates: if a System with the same
|
||||
normalized name already exists in this project (archived included), the
|
||||
call returns {"duplicate": true, "existing_id": ...} instead of creating —
|
||||
tag records to that one, or update_system it if its charter needs work.
|
||||
|
||||
Also mapped against the GLOBAL area catalog, so the same word means the
|
||||
same thing in every project. A name that IS a catalog area
|
||||
up to spelling ("CI and Release" vs "CI & Release") is mapped for you and
|
||||
the response says so. A name that merely RESEMBLES one comes back with
|
||||
`canonical_suggestion` — an offer, not a decision: apply it with
|
||||
map_system_to_canonical if it really is that area, ignore it if this is a
|
||||
project-specific area. Either way the System is created; the catalog never
|
||||
blocks a name.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
assessment = await systems_svc.assess_system_name(uid, project_id, name)
|
||||
duplicate = assessment["duplicate"]
|
||||
if duplicate:
|
||||
return {
|
||||
"duplicate": True,
|
||||
"existing_id": duplicate["id"],
|
||||
"message": (
|
||||
f"System '{duplicate['name']}' (#{duplicate['id']}) already "
|
||||
"covers this area in this project. Tag records to it with "
|
||||
"system_ids, or update_system it if the charter needs "
|
||||
"revising — a second System with the same name would split "
|
||||
"the area's records across two piles."
|
||||
),
|
||||
}
|
||||
# An exact match is mechanical, so it is applied; an overlap is a judgment
|
||||
# call, so it is only offered (see services/canonical_systems).
|
||||
canonical = assessment["canonical"]
|
||||
applied = canonical["id"] if canonical and canonical["basis"] == "exact" else None
|
||||
system = await systems_svc.create_system(
|
||||
uid, project_id=project_id, name=name,
|
||||
description=description or None, color=color or None,
|
||||
canonical_id=applied, path_patterns=path_patterns,
|
||||
)
|
||||
if system is None:
|
||||
raise ValueError(f"cannot create system in project {project_id} (no write access)")
|
||||
out = system.to_dict()
|
||||
if applied:
|
||||
out["canonical_note"] = (
|
||||
f"Mapped to the global area '{canonical['name']}' — the same "
|
||||
"spelling-insensitive name. Your System keeps the name you gave it."
|
||||
)
|
||||
elif canonical:
|
||||
out["canonical_suggestion"] = {
|
||||
**canonical,
|
||||
"message": (
|
||||
f"The global catalog has '{canonical['name']}', which may be "
|
||||
f"this same area. If it is, map_system_to_canonical("
|
||||
f"{system.id}, {canonical['id']}) so records and rules about "
|
||||
"this area line up across projects. If this area is specific "
|
||||
"to this project, ignore it — unmapped is a valid state."
|
||||
),
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
async def list_systems(project_id: int, include_archived: bool = False) -> dict:
|
||||
"""List a project's systems (active by default), ordered by order_index.
|
||||
|
||||
Pass include_archived=True to include archived systems.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows = await systems_svc.list_systems(uid, project_id, include_archived=include_archived)
|
||||
return {"systems": [s.to_dict() for s in rows]}
|
||||
|
||||
|
||||
async def get_system(system_id: int) -> dict:
|
||||
"""Fetch a System plus the records associated with it.
|
||||
|
||||
Returns the system, plus its associated records split into `issues`,
|
||||
`tasks` (work/plan), and `notes` — each saying what the record is and where
|
||||
it sits, not what it says (open one with get_note / get_task) — and
|
||||
`usage`: how many times its rulings were shown to a session because its
|
||||
files were touched (`surfaced_count`), and how many times it was opened.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
system = await systems_svc.get_system(uid, system_id)
|
||||
if system is None:
|
||||
raise ValueError(f"system {system_id} not found")
|
||||
# The pull half of the rulings arm's measurement: a
|
||||
# System whose rulings were shown, then opened.
|
||||
record_system_pulled(user_id=uid, system_id=system_id, source="mcp_get_system")
|
||||
records = await systems_svc.list_records_for_system(uid, system_id)
|
||||
titles = await milestones_svc.titles_for({r.milestone_id for r in records})
|
||||
issues, tasks, notes = [], [], []
|
||||
for r in records:
|
||||
d = notes_svc.brief_row(r, titles)
|
||||
if r.status is None:
|
||||
notes.append(d)
|
||||
elif r.task_kind == "issue":
|
||||
issues.append(d)
|
||||
else:
|
||||
tasks.append(d)
|
||||
data = system.to_dict()
|
||||
data["issues"] = issues
|
||||
data["tasks"] = tasks
|
||||
data["notes"] = notes
|
||||
# How often this area's rulings were shown to a session and the System
|
||||
# opened. Here and not on list_systems: one lookup on a read that
|
||||
# asked for the System, rather than a column on every project handshake.
|
||||
data["usage"] = (await system_usage_svc.usage_for_systems([system.id]))[system.id]
|
||||
return data
|
||||
|
||||
|
||||
async def update_system(
|
||||
system_id: int,
|
||||
name: str = "",
|
||||
description: str = "",
|
||||
color: str = "",
|
||||
status: str = "",
|
||||
order_index: int = -1,
|
||||
path_patterns: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Update a System. Only explicitly provided fields change.
|
||||
|
||||
`description` REPLACES the whole text, so read it with get_system first
|
||||
and send it back entire. Recording a RULING — the operator's decision
|
||||
about how this area must behave — is an edit here: add or change its line
|
||||
in the `Rulings` section at the end, in the turn they decide, citing the
|
||||
record it came from. Overturned, the line changes rather than gaining a
|
||||
successor: the section holds what is in force, and the work log keeps the
|
||||
history. If the charter above it contradicts a ruling, fix that sentence
|
||||
in the same edit.
|
||||
|
||||
`path_patterns` also REPLACES the whole list, and `[]` clears it.
|
||||
|
||||
Args:
|
||||
status: 'active' or 'archived'. Archive a system to retire it without
|
||||
losing history; archived systems hide from default lists.
|
||||
order_index: display position (0-based); -1 = leave unchanged.
|
||||
path_patterns: The area's files as repo-relative globs (see
|
||||
create_system). Omit to leave unchanged; [] clears them.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
fields: dict = {}
|
||||
if name:
|
||||
fields["name"] = name
|
||||
if description:
|
||||
fields["description"] = description
|
||||
if color:
|
||||
fields["color"] = color
|
||||
if status:
|
||||
fields["status"] = status
|
||||
if order_index >= 0:
|
||||
fields["order_index"] = order_index
|
||||
if path_patterns is not None:
|
||||
fields["path_patterns"] = path_patterns
|
||||
system = await systems_svc.update_system(uid, system_id, **fields)
|
||||
if system is None:
|
||||
raise ValueError(f"system {system_id} not found or no write access")
|
||||
return system.to_dict()
|
||||
|
||||
|
||||
async def list_system_records(
|
||||
system_id: int, kind: str = "", open_only: bool = False
|
||||
) -> dict:
|
||||
"""Everything filed under one System — the way to READ a subsystem.
|
||||
|
||||
Reach for this when investigating a specific area: it returns the notes,
|
||||
tasks, issues and snippets someone deliberately tagged to it — the
|
||||
subsystem's accumulated record, unranked. Start with its reference note if
|
||||
one exists (titled "«System» — reference"); that is the living state, and
|
||||
the rest is history and open work around it. For a ranked cut of the same
|
||||
slice, search(system_id=...) filters semantic search to this association.
|
||||
|
||||
Args:
|
||||
kind: filter by task_kind — 'issue', 'work', 'spike' (or the retired
|
||||
'plan'). Omit for all.
|
||||
open_only: limit to tasks not done/cancelled (e.g. open issues only).
|
||||
|
||||
Each record says what it is and where it sits, not what it says — open one
|
||||
with get_note / get_task / get_snippet.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows = await systems_svc.list_records_for_system(
|
||||
uid, system_id, kind=kind or None, open_only=open_only,
|
||||
)
|
||||
titles = await milestones_svc.titles_for({r.milestone_id for r in rows})
|
||||
return {"records": [notes_svc.brief_row(r, titles) for r in rows]}
|
||||
|
||||
|
||||
async def delete_system(system_id: int) -> dict:
|
||||
"""Soft-delete a System (recoverable). Its record associations are removed."""
|
||||
uid = current_user_id()
|
||||
ok = await systems_svc.delete_system(uid, system_id)
|
||||
if not ok:
|
||||
raise ValueError(f"system {system_id} not found or no write access")
|
||||
return {"message": f"System {system_id} deleted."}
|
||||
|
||||
|
||||
async def list_canonical_systems() -> dict:
|
||||
"""The GLOBAL vocabulary of area names, shared by every project.
|
||||
|
||||
These are the standard names to prefer when creating a System, so the same
|
||||
word means the same thing in every project on the instance — and the ids
|
||||
a cross-project record can point at. A project's own
|
||||
System keeps whatever name the project calls the area; mapping it here is
|
||||
an association, never a rename.
|
||||
|
||||
Reach for it before create_system when the area is an ordinary one (CI,
|
||||
auth, storage, the API, the UI), and pass the matching `canonical_id`.
|
||||
"""
|
||||
entries = await canonical_systems_svc.list_canonical_systems()
|
||||
return {"canonical_systems": [e.to_dict() for e in entries]}
|
||||
|
||||
|
||||
async def propose_canonical_mappings(project_id: int) -> dict:
|
||||
"""Suggest a global area for each of this project's UNMAPPED Systems.
|
||||
|
||||
Returns PROPOSALS ONLY — nothing is written. Confirm the ones that are
|
||||
right with map_system_to_canonical(system_id, canonical_id); ignore the
|
||||
rest. Each carries a `basis`:
|
||||
|
||||
- `exact` — the names reduce to the same match key ("CI and Release" vs
|
||||
"CI & Release"). Safe to confirm without much thought.
|
||||
- `overlap` — they share a meaningful word ("CI & runners" vs "CI &
|
||||
Release"). A judgment call: confirm only if they really are the same
|
||||
area, since a wrong mapping surfaces cross-project records in the wrong
|
||||
place.
|
||||
|
||||
A System with no proposal is not a problem — unmapped is a valid resting
|
||||
state, and a genuinely project-specific area should stay that way.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
return {"proposals": await canonical_systems_svc.propose_mappings(uid, project_id)}
|
||||
|
||||
|
||||
async def map_system_to_canonical(system_id: int, canonical_id: int = 0) -> dict:
|
||||
"""Map one of a project's Systems onto a global area (or clear it).
|
||||
|
||||
Sets `canonical_id` and NOTHING else — the System's name, charter and every
|
||||
record tagged to it are untouched. Pass canonical_id=0 to unmap.
|
||||
|
||||
Args:
|
||||
canonical_id: id from list_canonical_systems; 0 clears the mapping.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
system = await canonical_systems_svc.set_system_canonical(
|
||||
uid, system_id, canonical_id or None,
|
||||
)
|
||||
if system is None:
|
||||
raise ValueError(
|
||||
f"system {system_id} not found, no write access, "
|
||||
f"or canonical_id {canonical_id} is not a live catalog entry"
|
||||
)
|
||||
return system.to_dict()
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
create_system,
|
||||
list_systems,
|
||||
get_system,
|
||||
update_system,
|
||||
list_system_records,
|
||||
delete_system,
|
||||
list_canonical_systems,
|
||||
propose_canonical_mappings,
|
||||
map_system_to_canonical,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,54 @@
|
||||
"""list_tags — return the user's tag vocabulary with usage counts.
|
||||
|
||||
Python-side aggregation rather than SQL UNNEST. Personal-scale (~thousands of
|
||||
notes) makes the perf cost negligible, and a one-pass dict counter is much
|
||||
easier to mock in tests than a GROUP BY-with-unnest expression.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.note import Note
|
||||
|
||||
|
||||
def _aggregate_tag_counts(tag_lists) -> dict[str, int]:
|
||||
"""Count occurrences across a sequence of (possibly-None) tag lists.
|
||||
|
||||
Extracted so the aggregation logic can be unit-tested without a DB.
|
||||
"""
|
||||
counts: dict[str, int] = {}
|
||||
for tags in tag_lists:
|
||||
for tag in (tags or []):
|
||||
counts[tag] = counts.get(tag, 0) + 1
|
||||
return counts
|
||||
|
||||
|
||||
async def list_tags(limit: int = 50) -> dict:
|
||||
"""Return the user's most-used tags with usage counts.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of tags to return (1-200).
|
||||
|
||||
Returns:
|
||||
{"tags": [{"tag": "ops", "count": 12}, ...], "total": int}
|
||||
Sorted by count descending.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
limit = max(1, min(limit, 200))
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(Note.tags).where(Note.user_id == uid, Note.deleted_at.is_(None))
|
||||
)
|
||||
tag_lists = [row[0] for row in result.all()]
|
||||
counts = _aggregate_tag_counts(tag_lists)
|
||||
top = sorted(counts.items(), key=lambda x: x[1], reverse=True)[:limit]
|
||||
return {
|
||||
"tags": [{"tag": t, "count": c} for t, c in top],
|
||||
"total": len(top),
|
||||
}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
mcp.tool(name="list_tags")(list_tags)
|
||||
@@ -0,0 +1,768 @@
|
||||
"""Task CRUD MCP tools.
|
||||
|
||||
Tasks are notes with a non-null `status` — same model, different filter.
|
||||
Wrappers call services/notes.py for CRUD with is_task=True and add the
|
||||
task-specific fields (status, priority, due_date, parent_id), plus
|
||||
services/task_logs.py for add_task_log AND for reading those logs back —
|
||||
`get_task` returns them and `list_tasks` counts them. The log is the record
|
||||
written during the work; a body is a claim written before it, so a session
|
||||
opening a task needs both.
|
||||
|
||||
Sentinels (preserved from existing fable-mcp):
|
||||
- status="" / priority="" / title="" / body="" → "leave unchanged" on update
|
||||
- status="todo" is the default on create (creates a task; non-null status is
|
||||
what makes a Note a Task)
|
||||
- priority="none" sets explicit no-priority; priority="" is "leave unchanged"
|
||||
- project_id=0 / milestone_id=0 / parent_id=0 → "no association" on create,
|
||||
"leave unchanged" on update; on update, -1 clears the FK (sets it NULL)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.mcp.tools import systems as systems_tools
|
||||
from grimoire.services import access as access_svc
|
||||
from grimoire.services import dedup as dedup_svc
|
||||
from grimoire.services import family as family_svc
|
||||
from grimoire.services import family_adoption as family_adoption_svc
|
||||
from grimoire.services import milestones as milestones_svc
|
||||
from grimoire.services import notes as notes_svc
|
||||
# Imported by NAME, not reached through notes_svc: minted_kind is pure
|
||||
# validation, not a service call, and a test that stubs the service module to
|
||||
# avoid the database would otherwise stub the validation too — turning a
|
||||
# guard into a MagicMock that approves anything.
|
||||
from grimoire.services.notes import minted_kind
|
||||
from grimoire.services import placement as placement_svc
|
||||
from grimoire.services import planning as planning_svc
|
||||
from grimoire.services import record_batch as batch_svc
|
||||
from grimoire.services import rulebooks as rulebooks_svc
|
||||
from grimoire.services import systems as systems_svc
|
||||
from grimoire.services import task_logs as task_logs_svc
|
||||
from grimoire.services import trash as trash_svc
|
||||
from grimoire.services.note_usage import record_pulled
|
||||
from grimoire.services.record_refs import refuse_guessed_ids
|
||||
from grimoire.services.text import elide
|
||||
from grimoire.services import moment_delivery
|
||||
|
||||
|
||||
# A work log entry is prose, often long — the discipline asks for what was
|
||||
# decided and why, not a line of status. Two caps, not one, because the entries
|
||||
# are not equally useful: the NEWEST answers "where does this actually stand",
|
||||
# which is the question the block exists for, so it arrives whole up to a
|
||||
# generous ceiling. Older entries are there to say what happened and when, and
|
||||
# a headline does that.
|
||||
_WORK_LOG_ENTRIES = 3
|
||||
_WORK_LOG_CHARS = 800
|
||||
_WORK_LOG_LATEST_CHARS = 4000
|
||||
|
||||
_WORK_LOG_ADVICE = (
|
||||
"The body is a CLAIM, written once before the work. These entries are the "
|
||||
"RECORD, written during and after it. Where the two disagree the log is "
|
||||
"later — read it before acting on what the body says the status is."
|
||||
)
|
||||
|
||||
|
||||
def work_log_payload(
|
||||
logs: list, total: int, chars: int, latest_chars: int = _WORK_LOG_LATEST_CHARS
|
||||
) -> dict:
|
||||
"""The `work_log` block: recent entries newest-first, plus what was elided.
|
||||
|
||||
`total` is the count of ALL entries, not of `logs` — a reader has to be
|
||||
able to tell "this task has no record" from "you were shown the last three
|
||||
of nine", and those are the same response if the count comes from the
|
||||
entries handed over.
|
||||
"""
|
||||
entries = []
|
||||
for i, log in enumerate(logs):
|
||||
row = log.to_dict() if hasattr(log, "to_dict") else dict(log)
|
||||
row.pop("updated_at", None)
|
||||
content = row.get("content") or ""
|
||||
# The first row IS the newest — logs_for_task orders descending.
|
||||
budget = latest_chars if i == 0 else chars
|
||||
if chars <= 0:
|
||||
budget = 0
|
||||
text, cut = elide(content, budget)
|
||||
row["content"] = text
|
||||
if cut:
|
||||
row["full_length"] = len(content)
|
||||
row["truncated"] = True
|
||||
entries.append(row)
|
||||
|
||||
out: dict = {"total": total, "entries": entries}
|
||||
if total == 0:
|
||||
return out
|
||||
|
||||
out["advice"] = _WORK_LOG_ADVICE
|
||||
not_shown = total - len(entries)
|
||||
if not_shown > 0:
|
||||
out["not_shown"] = not_shown
|
||||
if not_shown > 0 or any(e.get("truncated") for e in entries):
|
||||
out["read_all"] = (
|
||||
"Entries were shortened or omitted. "
|
||||
"get_task(task_id, log_limit=0, log_chars=0) returns every entry "
|
||||
"in full — reach for it rather than judging from what is here, "
|
||||
"which was selected by recency and length, not by relevance."
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
async def list_tasks(
|
||||
limit: int = 20,
|
||||
offset: int = 0,
|
||||
status: str = "",
|
||||
project_id: int = 0,
|
||||
kind: str = "",
|
||||
) -> dict:
|
||||
"""List tasks in Grimoire.
|
||||
|
||||
Args:
|
||||
status: Filter by status — one of: todo, in_progress, done, cancelled. Omit for all.
|
||||
project_id: Filter to a specific project. PASS THE ACTIVE PROJECT'S ID
|
||||
whenever a project is in scope so you list that project's tasks, not
|
||||
every project's. 0 = no filter (all projects — use only for a
|
||||
deliberate cross-project view).
|
||||
kind: Filter by task kind — 'work', 'issue', 'spike' (or the retired
|
||||
'plan'). Omit (empty) for all kinds.
|
||||
|
||||
Results are ordered by last-updated descending. Each row says what the task
|
||||
is and where it sits — id, title, status, kind, priority, milestone (id and
|
||||
title), tags, updated_at, plus description, parent and due date when set —
|
||||
and not what it says: read one in full with get_task(id).
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows, total = await notes_svc.list_notes(
|
||||
uid,
|
||||
is_task=True,
|
||||
status=status or None,
|
||||
project_id=project_id or None,
|
||||
task_kind=kind or None,
|
||||
limit=max(1, min(limit, 100)),
|
||||
offset=max(0, offset),
|
||||
)
|
||||
titles = await milestones_svc.titles_for({n.milestone_id for n in rows})
|
||||
# One aggregate for the page, zero-filled: a reader scanning a list needs
|
||||
# to know WHICH rows carry a record before choosing what to open, and a
|
||||
# missing key would read as "no logs" on every row rather than on the
|
||||
# rows that have none.
|
||||
log_counts = await task_logs_svc.log_counts_for_tasks(
|
||||
uid, [int(n.id) for n in rows]
|
||||
)
|
||||
return {
|
||||
"tasks": [
|
||||
notes_svc.brief_row(n, titles, log_counts=log_counts) for n in rows
|
||||
],
|
||||
"total": total,
|
||||
}
|
||||
|
||||
|
||||
async def get_task(
|
||||
task_id: int,
|
||||
project_id: int = 0,
|
||||
log_limit: int = _WORK_LOG_ENTRIES,
|
||||
log_chars: int = _WORK_LOG_CHARS,
|
||||
) -> dict:
|
||||
"""Fetch a single Grimoire task by ID.
|
||||
|
||||
Returns id, title, body, status, priority, tags, project_id, milestone_id,
|
||||
parent_id, parent_title, due_date, created_at, updated_at — plus `systems`
|
||||
(the areas this task is filed under; read a subsystem's whole pile with
|
||||
list_system_records) or, for an untagged project task, the `systems_hint`
|
||||
question. For legacy
|
||||
kind=plan tasks, the response also includes the project's applicable_rules
|
||||
and project_rules (new plans are milestones — use get_milestone for those).
|
||||
|
||||
AND `work_log` — the entries add_task_log wrote, newest first, with
|
||||
`total` for how many exist. READ IT BEFORE TRUSTING THE BODY. A body is
|
||||
written once, at the start, when the least is known; the log is written
|
||||
during the work and after it. A task whose body says "not started" and
|
||||
whose log records a partial ship is not a contradiction to resolve — the
|
||||
log is simply later; trusting the body alone is how finished work gets
|
||||
rebuilt.
|
||||
|
||||
The NEWEST entry arrives whole (to 4000 characters); older ones are
|
||||
shortened to a headline. Anything shortened is cut from the MIDDLE, so
|
||||
the opening and the closing both survive — a log entry's conclusion is at
|
||||
its end — and the gap states how many characters went. An entry that was
|
||||
cut says so and carries its `full_length`, and the block as a whole says
|
||||
when it is showing you less than the record holds.
|
||||
|
||||
Args:
|
||||
log_limit: How many of the most recent entries to include (default 3).
|
||||
0 returns every entry.
|
||||
log_chars: Budget for the OLDER entries (default 800). 0 returns
|
||||
every entry in full, the newest included.
|
||||
|
||||
A task another user shared with you also carries `shared`, `owner` and
|
||||
`permission` — it's their work item, not one you took on.
|
||||
|
||||
`project_id` is the project you are WORKING IN, not this record's own.
|
||||
Passing the active project is what makes "opened away from where it was
|
||||
written" answerable; 0 leaves it unreported and the pull still counts.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
loaded = await notes_svc.get_note_for_user(uid, task_id)
|
||||
note = loaded[0] if loaded else None
|
||||
if note is None or note.deleted_at is not None:
|
||||
raise ValueError(f"task {task_id} not found")
|
||||
data = note.to_dict()
|
||||
parent_title = None
|
||||
if note.parent_id:
|
||||
parent_loaded = await notes_svc.get_note_for_user(uid, note.parent_id)
|
||||
if parent_loaded is not None:
|
||||
parent_title = parent_loaded[0].title
|
||||
data["parent_title"] = parent_title
|
||||
|
||||
# Legacy kind=plan tasks predate milestone-as-plan; still surface their
|
||||
# project's rules on read so the historical plans stay useful.
|
||||
if data.get("task_kind") == "plan" and note.project_id:
|
||||
applicable = await rulebooks_svc.get_applicable_rules(
|
||||
project_id=note.project_id, user_id=uid,
|
||||
)
|
||||
data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_task"))
|
||||
data.update(await access_svc.describe_provenance(uid, note))
|
||||
# Same reasoning as get_note's record_pulled, and this is the tool where it
|
||||
# matters MOST: auto-inject ranks kind-blind over a corpus that is
|
||||
# overwhelmingly tasks and issues, so tasks dominate what it surfaces. Without
|
||||
# this every surfaced task would count as never-pulled, driving
|
||||
# auto-inject's measured pull-through toward zero for its own dominant
|
||||
# kind.
|
||||
await systems_tools.attach_systems(
|
||||
uid, getattr(note, "user_id", uid) or uid, data, note.id, note.project_id
|
||||
)
|
||||
# Counted separately rather than inferred from the rows handed back: with a
|
||||
# limit applied, len(entries) is the size of the window, not of the record,
|
||||
# and "3 entries" and "the last 3 of 9" have to read differently.
|
||||
log_rows = await task_logs_svc.logs_for_task(
|
||||
uid, int(note.id), limit=max(0, log_limit)
|
||||
)
|
||||
log_total = (
|
||||
len(log_rows) if log_limit <= 0
|
||||
else await task_logs_svc.count_logs_for_task(uid, int(note.id))
|
||||
)
|
||||
data["work_log"] = work_log_payload(log_rows, log_total, max(0, log_chars))
|
||||
record_pulled(
|
||||
user_id=uid, note_id=int(note.id),
|
||||
source="mcp_get_task", project_id=project_id,
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
async def create_task(
|
||||
title: str,
|
||||
body: str = "",
|
||||
status: str = "todo",
|
||||
priority: str = "",
|
||||
project_id: int = 0,
|
||||
milestone_id: int = 0,
|
||||
parent_id: int = 0,
|
||||
tags: list[str] | None = None,
|
||||
kind: str = "work",
|
||||
system_ids: list[int] | None = None,
|
||||
arose_from_id: int = 0,
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Create a new task in Grimoire.
|
||||
|
||||
IS ANYTHING ACTUALLY OWED? A task carries a status and someone is on the
|
||||
hook to move it. If nothing is owed — you are recording what you learned,
|
||||
decided or observed — that is a note (create_note), and filing it here
|
||||
leaves a to-do nobody will ever close. If the work is an ARC of several
|
||||
steps toward one goal, start_planning makes the milestone that holds
|
||||
them; a task is one step, not the plan.
|
||||
|
||||
Args:
|
||||
title: Task title (required).
|
||||
body: Markdown description / notes for the task.
|
||||
status: Initial status — one of: todo (default), in_progress, done, cancelled.
|
||||
priority: One of: low, medium, high, or 'none'. Omit (empty string) to leave unset.
|
||||
project_id: Associate with a project (0 = no project).
|
||||
milestone_id: Place within a project milestone (0 = no milestone).
|
||||
When the work belongs to an active plan — a milestone in
|
||||
enter_project's lists or found by search(content_type=
|
||||
"milestone") — pass its id, so the plan shows all of its work.
|
||||
parent_id: Make this a sub-task of another task (0 = top-level).
|
||||
tags: List of plain-string tags without # prefix.
|
||||
kind: 'work' (default), 'issue', or 'spike'.
|
||||
An ISSUE is corrective work — a problem you fixed or are fixing;
|
||||
record symptom → root cause → fix in the body.
|
||||
A SPIKE is time-boxed and its output is KNOWLEDGE rather than a
|
||||
change: "find out whether the runner can be given a bash shell",
|
||||
"work out why the index is not used". It succeeds by producing an
|
||||
answer, so nothing ships at the end of it — which is why filing
|
||||
one as `work` makes a finished investigation look like an
|
||||
abandoned change. Reach for it when the honest deliverable is a
|
||||
finding, and say in the body what would close the box: a time, or
|
||||
the question being answered well enough to act on.
|
||||
(Plans are milestones now — call start_planning to begin a plan;
|
||||
'plan' is not a valid kind here.)
|
||||
system_ids: Ids of the project's Systems (reusable subsystem/area
|
||||
objects; see list_systems / create_system) to associate this task with.
|
||||
arose_from_id: For an issue, the id of the task/feature it arose from;
|
||||
for a spike, the record that raised the question — including a
|
||||
standing rule whose check just failed. 0 = none.
|
||||
force: Bypass the near-duplicate gate. By default, a task with the
|
||||
same title, or one that reads as a copy, in the same project BLOCKS
|
||||
the create and its id is returned so you update it instead. A task
|
||||
that reads CLOSE but not identical does not block: the task is
|
||||
created and the reply carries `overlaps` — open the top one and
|
||||
judge whether it is the same work (fold in, delete the new one) or
|
||||
separate work (keep both). Set force true only when a blocked task
|
||||
is genuinely distinct.
|
||||
|
||||
AN ID EXISTS ONLY ONCE A CREATE RETURNS IT. Never write the id you expect
|
||||
a record to get: every session and user draws from one sequence, so the
|
||||
number is taken by whoever creates next. A body citing a `#N` that has not
|
||||
been assigned yet is refused. Creating several records that cite each
|
||||
other? Use create_records (or start_planning(steps=...)) and write
|
||||
{{ref:N}} — the real ids are filled in as they are created.
|
||||
|
||||
Returns the created task, OR — when a near-duplicate is found and force is
|
||||
false — {"duplicate": true, "existing_id": ..., "message": ...} (nothing
|
||||
created). A task in a project carries `placement` (see update_task). A tagged record shows its `systems`; created untagged in a
|
||||
project, the response carries the `systems_hint` question instead —
|
||||
answer it: tag the record, create the missing System, or deliberately
|
||||
leave it untagged.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
if kind == "plan":
|
||||
raise ValueError(
|
||||
"kind=plan is retired — a plan is now a milestone. Call "
|
||||
"start_planning(project_id, title) to begin a plan (it creates the "
|
||||
"milestone + seeds the design), then create each step as its own "
|
||||
"task with create_task(milestone_id=<that milestone>)."
|
||||
)
|
||||
await refuse_guessed_ids(title, body)
|
||||
overlaps: list = []
|
||||
if not force:
|
||||
dup = await dedup_svc.find_duplicate_note(
|
||||
uid, title, body, project_id=project_id or None,
|
||||
is_task=True, note_type="note", overlaps=overlaps,
|
||||
)
|
||||
if dup is not None:
|
||||
return dedup_svc.duplicate_response(dup, "task")
|
||||
note = await notes_svc.create_note(
|
||||
uid,
|
||||
title=title,
|
||||
body=body,
|
||||
status=status,
|
||||
priority=priority or None,
|
||||
project_id=project_id or None,
|
||||
milestone_id=milestone_id or None,
|
||||
parent_id=parent_id or None,
|
||||
tags=tags,
|
||||
task_kind=minted_kind(kind),
|
||||
arose_from_id=arose_from_id or None,
|
||||
)
|
||||
if system_ids:
|
||||
await systems_svc.set_record_systems(uid, note.id, system_ids)
|
||||
data = note.to_dict()
|
||||
await systems_tools.attach_systems(uid, uid, data, note.id, project_id or None)
|
||||
data.update(dedup_svc.note_overlap_response(overlaps, "task"))
|
||||
# A task that says it is "matching" another project's work names the
|
||||
# source of a pattern — family canon's citation trigger.
|
||||
await family_svc.attach_family_hint(uid, data, note, created=True)
|
||||
return await placement_svc.attach_placement(uid, data, note)
|
||||
|
||||
|
||||
async def update_task(
|
||||
task_id: int,
|
||||
title: str = "",
|
||||
body: str = "",
|
||||
status: str = "",
|
||||
priority: str = "",
|
||||
project_id: int = 0,
|
||||
milestone_id: int = 0,
|
||||
system_ids: list[int] | None = None,
|
||||
arose_from_id: int = 0,
|
||||
kind: str = "",
|
||||
) -> dict:
|
||||
"""Update an existing Grimoire task. Only explicitly provided fields are changed.
|
||||
|
||||
Args:
|
||||
task_id: ID of the task to update.
|
||||
title: New title, or omit to leave unchanged.
|
||||
body: New markdown body, or omit to leave unchanged.
|
||||
status: New status — one of: todo, in_progress, done, cancelled. Drive
|
||||
the lifecycle: set in_progress when you start, done when complete —
|
||||
don't leave finished work at todo.
|
||||
priority: New priority — one of: none, low, medium, high.
|
||||
project_id: New project. 0 = leave unchanged, -1 = clear (remove from
|
||||
its project; also clears the milestone), positive = set.
|
||||
milestone_id: New milestone. 0 = leave unchanged, -1 = clear (remove
|
||||
from its milestone), positive = set.
|
||||
system_ids: Replace this task's System associations with these ids
|
||||
(set-semantics). None = leave unchanged; [] = clear all.
|
||||
arose_from_id: Provenance (issue → originating task). 0 = leave unchanged,
|
||||
-1 = clear, positive = set.
|
||||
kind: Re-file this task as 'work', 'issue' or 'spike'. Omit (empty) to
|
||||
leave unchanged. Correcting a kind is ordinary — what a task turns
|
||||
out to BE is often clear only once it is under way, and a piece of
|
||||
work that becomes an investigation should say so. 'plan' is
|
||||
refused: plans are milestones (start_planning), and the value
|
||||
survives only so historical plan-tasks stay writable.
|
||||
|
||||
The response carries `placement` for a task in a project: its `project`,
|
||||
and for a task in a milestone its `milestone`, `position` ({step, of}),
|
||||
`progress` ({completed, total, pct}) and `next` (the next open step, or
|
||||
null). These are the facts to use when telling the operator where the
|
||||
work sits and what comes next — read them from here rather than
|
||||
reconstructing them, because a remembered milestone title or "next step"
|
||||
reads exactly like a real one when it is wrong.
|
||||
|
||||
Closing a task (done or cancelled) also returns `reply_shape`, the
|
||||
default shape of the completion report you are about to write, and
|
||||
`report_back`: a one-line reminder of what that reply should cover. The
|
||||
operator's own preferences for that report arrive beside it in
|
||||
`moment_rules` — the ones mounted on finishing work. Where one differs
|
||||
from the default shape, the preference is what the operator asked for.
|
||||
When family adoptions were answered `owed` while the task was
|
||||
open, they come back as `family_owed` ({idea_id, idea_title, project_id,
|
||||
project_title, owed_task_id}): name each in the report — it is work
|
||||
filed into that project.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
fields: dict = {}
|
||||
if title:
|
||||
fields["title"] = title
|
||||
if body:
|
||||
fields["body"] = body
|
||||
if status:
|
||||
fields["status"] = status
|
||||
if priority:
|
||||
fields["priority"] = priority
|
||||
# Optional FKs: 0 = leave unchanged, -1 = clear (set NULL), positive = set.
|
||||
if project_id == -1:
|
||||
fields["project_id"] = None
|
||||
fields["milestone_id"] = None # a milestone can't outlive its project
|
||||
elif project_id:
|
||||
fields["project_id"] = project_id
|
||||
if milestone_id == -1:
|
||||
fields["milestone_id"] = None
|
||||
elif milestone_id:
|
||||
fields["milestone_id"] = milestone_id
|
||||
if arose_from_id == -1:
|
||||
fields["arose_from_id"] = None
|
||||
elif arose_from_id:
|
||||
fields["arose_from_id"] = arose_from_id
|
||||
if kind:
|
||||
fields["task_kind"] = minted_kind(kind)
|
||||
await refuse_guessed_ids(title, body)
|
||||
note = await notes_svc.update_note(uid, task_id, **fields)
|
||||
if note is None:
|
||||
raise ValueError(f"task {task_id} not found")
|
||||
if system_ids is not None:
|
||||
await systems_svc.set_record_systems(uid, task_id, system_ids)
|
||||
data = note.to_dict()
|
||||
await systems_tools.attach_systems(
|
||||
uid, getattr(note, "user_id", uid) or uid, data, task_id, note.project_id
|
||||
)
|
||||
await placement_svc.attach_placement(uid, data, note)
|
||||
if body:
|
||||
await family_svc.attach_family_hint(uid, data, note, created=False)
|
||||
if status in _CLOSING_STATUSES:
|
||||
data["report_back"] = REPORT_BACK_CUE
|
||||
# Owed family adoptions filed while the task was open — work now
|
||||
# waiting in some project, which the report names.
|
||||
await family_adoption_svc.attach_owed_adoptions(uid, data, note)
|
||||
return await moment_delivery.attach_moment_rules(
|
||||
uid, "update_task", {"status": status, "project_id": project_id}, data,
|
||||
)
|
||||
|
||||
|
||||
async def add_task_log(task_id: int, content: str) -> dict:
|
||||
"""Append a timestamped progress log entry to a Grimoire task.
|
||||
|
||||
Use this to record work sessions, decisions, or status updates over time
|
||||
without overwriting the task's main body. Each entry is stored separately
|
||||
and shown chronologically in the task view.
|
||||
|
||||
What you write here comes back from `get_task` as `work_log`, newest
|
||||
first, and is counted on every row of `list_tasks` and every step of
|
||||
`get_milestone` — so write for the session that opens this task next, not
|
||||
for a reader who already knows what you were doing. What that reader
|
||||
cannot get from the body or the diff is what you tried, what you ruled
|
||||
out, and where it actually stands.
|
||||
|
||||
A log on an open task also marks it as being worked by this session — its
|
||||
`claim`. That is what brings the task and its newest
|
||||
entries back to you after a compaction; it lapses by itself once the
|
||||
session stops writing, and a status of done, cancelled or todo clears it.
|
||||
|
||||
The response shows the task's `systems` — or, if the task is an untagged
|
||||
project record, the `systems_hint` question: logging work IS working in
|
||||
some area, so answer it (update_task with system_ids, or create_system
|
||||
the missing area) rather than logging past it.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
log = await task_logs_svc.create_log(uid, task_id, content)
|
||||
data = log.to_dict() if hasattr(log, "to_dict") else {
|
||||
"id": log.id, "task_id": log.task_id, "content": log.content,
|
||||
"created_at": log.created_at.isoformat() if log.created_at else None,
|
||||
}
|
||||
# A work-log is work happening on the task NOW — the strongest moment to
|
||||
# ask which System's territory that work is in. Fail-open decoration.
|
||||
try:
|
||||
loaded = await notes_svc.get_note_for_user(uid, task_id)
|
||||
if loaded:
|
||||
task = loaded[0]
|
||||
await systems_tools.attach_systems(
|
||||
uid, getattr(task, "user_id", uid) or uid,
|
||||
data, task_id, task.project_id,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
return data
|
||||
|
||||
|
||||
# The in-band completion cue. The completion shape itself rides the closing
|
||||
# response as `reply_shape`; this is the one line
|
||||
# that says what the reply covers, for every MCP client, at the moment a piece
|
||||
# of work closes. One line on purpose: a template here would be read as the
|
||||
# reply itself.
|
||||
_CLOSING_STATUSES = ("done", "cancelled")
|
||||
REPORT_BACK_CUE = (
|
||||
"Reporting this to the operator? Say where it sits (from `placement`), "
|
||||
"what now works, what needs them, and what comes next."
|
||||
)
|
||||
|
||||
_ITEM_KEYS = {"title", "body", "type", "status", "priority", "kind", "tags", "system_ids"}
|
||||
|
||||
|
||||
def _batch_items(records: list[dict], *, what: str = "record") -> list[batch_svc.BatchItem]:
|
||||
"""Parse the door's plain dicts into BatchItems, refusing unknown keys.
|
||||
|
||||
Strict for the same reason StrictArgsMCPServer is: a misspelt key
|
||||
silently dropped — `milestone` for a per-record milestone, `desc` for body —
|
||||
creates a record that looks right and is missing what the caller sent.
|
||||
"""
|
||||
items: list[batch_svc.BatchItem] = []
|
||||
for i, raw in enumerate(records or [], start=1):
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError(f"{what} {i} must be an object with at least a title")
|
||||
unknown = set(raw) - _ITEM_KEYS
|
||||
if unknown:
|
||||
raise ValueError(
|
||||
f"{what} {i} has unknown field(s) {sorted(unknown)}; "
|
||||
f"allowed: {sorted(_ITEM_KEYS)}"
|
||||
)
|
||||
rtype = raw.get("type") or "task"
|
||||
if rtype not in ("task", "note"):
|
||||
raise ValueError(f"{what} {i}: type must be 'task' or 'note', got {rtype!r}")
|
||||
items.append(batch_svc.BatchItem(
|
||||
title=raw.get("title") or "",
|
||||
body=raw.get("body") or "",
|
||||
is_task=rtype == "task",
|
||||
status=raw.get("status") or "todo",
|
||||
priority=raw.get("priority") or None,
|
||||
task_kind=raw.get("kind") or "work",
|
||||
tags=list(raw.get("tags") or []),
|
||||
system_ids=list(raw.get("system_ids") or []),
|
||||
))
|
||||
return items
|
||||
|
||||
|
||||
async def _first_duplicate(
|
||||
uid: int, items: list, project_id: int | None,
|
||||
overlaps: dict | None = None,
|
||||
) -> dict | None:
|
||||
"""The duplicate gate over a whole batch — the first hit blocks all of it.
|
||||
|
||||
`overlaps`, when given, collects each record's near matches below the copy
|
||||
band by its 1-based position, for the reply once the batch is
|
||||
created."""
|
||||
for i, item in enumerate(items, start=1):
|
||||
found: list = []
|
||||
dup = await dedup_svc.find_duplicate_note(
|
||||
uid, item.title, item.body, project_id=project_id,
|
||||
is_task=item.is_task, note_type="note", overlaps=found,
|
||||
)
|
||||
if found and overlaps is not None:
|
||||
overlaps[i] = found
|
||||
if dup is not None:
|
||||
payload = dedup_svc.duplicate_response(dup, "task" if item.is_task else "note")
|
||||
payload["record"] = i
|
||||
payload["message"] = f"Record {i} of the batch: {payload['message']} Nothing in the batch was created."
|
||||
return payload
|
||||
return None
|
||||
|
||||
|
||||
async def create_records(
|
||||
records: list[dict],
|
||||
project_id: int = 0,
|
||||
milestone_id: int = 0,
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Create several tasks and/or notes in ONE call, so they can cite each other.
|
||||
|
||||
Reach for this whenever records you are about to create need to reference
|
||||
one another — a set of tasks that name their siblings, a reference note
|
||||
listing the tasks it indexes. Writing the ids you EXPECT them to get is
|
||||
always wrong: every session and user draws from one sequence, so another
|
||||
create takes those numbers, and a body citing an unassigned `#N` is refused.
|
||||
A plan (a milestone plus its steps) is start_planning(steps=...), which
|
||||
uses the same mechanism. One record with nothing to cite is create_task or
|
||||
create_note.
|
||||
|
||||
In any record's body write {{ref:N}} where the Nth record's id belongs
|
||||
(1-based, in the order listed). It is replaced with `#<id> "<title>"`
|
||||
once the ids exist. Everything is created in one transaction: a bad
|
||||
placeholder, an invalid field or a duplicate creates NOTHING.
|
||||
|
||||
Args:
|
||||
records: The records, in order. Each is an object with `title`
|
||||
(required) and optionally `body`, `type` ('task', the default, or
|
||||
'note'), and for tasks `status`, `priority` and `kind`
|
||||
('work' | 'issue' | 'spike'); plus `tags` and `system_ids`.
|
||||
project_id: The project every record belongs to (0 = none, or taken
|
||||
from milestone_id).
|
||||
milestone_id: File every record under this existing milestone (0 = none).
|
||||
This is how steps are added to a plan that already exists,
|
||||
including one start_planning handed back as `existing_milestone`.
|
||||
force: Bypass the near-duplicate gate for the whole batch. By default
|
||||
the first record that near-duplicates an existing one BLOCKS the
|
||||
batch, and its existing id comes back so you can update it instead.
|
||||
|
||||
Returns {"ids": [...], "records": [...]} in the order given — the ids
|
||||
cite-able from here on — OR a duplicate payload naming which `record`
|
||||
matched, with nothing created. The ids are the real ones and are not
|
||||
necessarily consecutive: other sessions keep creating meanwhile, and
|
||||
nothing here depends on the numbers being adjacent.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
items = _batch_items(records)
|
||||
await refuse_guessed_ids(*[t for item in items for t in (item.title, item.body)])
|
||||
overlaps: dict = {}
|
||||
if not force:
|
||||
dup = await _first_duplicate(uid, items, project_id or None, overlaps)
|
||||
if dup is not None:
|
||||
return dup
|
||||
_ms, notes = await batch_svc.create_batch(
|
||||
uid, items, project_id=project_id or None, milestone_id=milestone_id or None,
|
||||
)
|
||||
out = {"ids": [n.id for n in notes], "records": [n.to_dict() for n in notes]}
|
||||
out.update(dedup_svc.batch_overlap_response(overlaps))
|
||||
return out
|
||||
|
||||
|
||||
async def start_planning(
|
||||
project_id: int,
|
||||
title: str,
|
||||
body: str = "",
|
||||
steps: list[dict] | None = None,
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Begin a plan in Grimoire (the preferred home for plans — not a local .md file).
|
||||
|
||||
Reach for this when the work has an ARC — several steps toward one goal,
|
||||
worth tracking as a unit. Work without one (a fix, a one-file change, a
|
||||
question answered) is a task, not a plan: create_task, drive its status, and
|
||||
record progress with add_task_log. A design or decision you are RECORDING
|
||||
rather than executing is a note (create_note) — a plan nobody is going to
|
||||
work through is a document filed in the place reserved for open work. A milestone holding a single step is
|
||||
ceremony, and it leaves the project with a plan that never meant anything.
|
||||
|
||||
Creates a MILESTONE that IS the plan: its `body` is seeded with a design
|
||||
template (Goal/Approach/Verification) under the given project, and the call
|
||||
returns it together with the project's applicable Rulebook rules and brief
|
||||
context. The milestone is the plan container — the individual steps live as
|
||||
first-class child tasks under it, not as checkboxes in the body.
|
||||
|
||||
PASS THE STEPS HERE when you already know them. The milestone and every
|
||||
step are then created in one transaction, and the plan body and the steps
|
||||
can cite each other with placeholders that become real ids: {{ref:N}} is
|
||||
the Nth step (1-based) and {{ref:milestone}} is this milestone. Never write
|
||||
the ids you expect records to get — other sessions and users draw from the
|
||||
same sequence, and a body citing an unassigned `#N` is refused.
|
||||
|
||||
Without steps: edit the design afterwards with update_milestone(
|
||||
milestone_id, body=...), and add steps with create_records(milestone_id=
|
||||
<this id>, ...) — or create_task for a single one. Track each with status
|
||||
+ add_task_log. Do NOT put steps as checkboxes in the milestone body.
|
||||
|
||||
(kind=plan tasks are retired — use this instead. Existing historical
|
||||
plan-tasks remain readable but new planning goes through milestones.)
|
||||
|
||||
Args:
|
||||
project_id: The project this plan is for.
|
||||
title: A short title for the plan/milestone.
|
||||
body: The plan's design (markdown). Omit to seed the Goal/Approach/
|
||||
Verification template.
|
||||
steps: The plan's step-tasks, in order — each an object with `title`
|
||||
(required) and optionally `body`, `status`, `priority`, `kind`
|
||||
('work' | 'issue' | 'spike'), `tags`, `system_ids`.
|
||||
force: Bypass both duplicate gates. By default nothing is created —
|
||||
milestone included — when an ACTIVE plan in the project already
|
||||
has this title or reads as the same plan, or when a step
|
||||
near-duplicates an existing task. Pass it once you have read the
|
||||
match and know this is separate work.
|
||||
|
||||
Returns the milestone, the project's applicable rules and brief context,
|
||||
plus `steps` (the created tasks, in order) when steps were given — OR a
|
||||
duplicate payload with nothing created: `existing_milestone` when a plan
|
||||
already covers this (add your steps to it with create_records(
|
||||
milestone_id=...)), or `record` naming the step that matched a task.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
items = _batch_items(steps or [], what="step")
|
||||
await refuse_guessed_ids(body, *[t for item in items for t in (item.title, item.body)])
|
||||
if not force:
|
||||
# The plan before its steps: when a plan already covers this, the
|
||||
# answer is to add these steps THERE, and a step-level match would
|
||||
# only name one symptom of that.
|
||||
match = await dedup_svc.plan_gate(
|
||||
uid, project_id, title,
|
||||
dedup_svc.plan_candidate_text(
|
||||
body=body, steps=[(i.title, i.body) for i in items],
|
||||
),
|
||||
)
|
||||
if match is not None:
|
||||
return match
|
||||
overlaps: dict = {}
|
||||
if items and not force:
|
||||
dup = await _first_duplicate(uid, items, project_id or None, overlaps)
|
||||
if dup is not None:
|
||||
return dup
|
||||
result = await planning_svc.start_planning(
|
||||
user_id=uid, project_id=project_id, title=title,
|
||||
body=body or None, steps=items or None,
|
||||
)
|
||||
if isinstance(result, dict):
|
||||
result.update(dedup_svc.batch_overlap_response(overlaps))
|
||||
await moment_delivery.attach_moment_rules(uid, "start_planning", {"project_id": project_id}, result)
|
||||
return result
|
||||
|
||||
|
||||
async def delete_task(task_id: int) -> dict:
|
||||
"""Move a Grimoire task (or plan) to the trash (recoverable). Sub-tasks go with it.
|
||||
Restore via restore(batch_id)."""
|
||||
uid = current_user_id()
|
||||
# Read the title BEFORE the delete: afterwards the row is trashed and the
|
||||
# confirmation could only echo the number back. A deletion the operator
|
||||
# cannot recognise is one they cannot tell was the wrong one.
|
||||
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
|
||||
# that errors must not stop the delete happening. Same posture the
|
||||
# staleness marker takes — a decoration may never break its payload.
|
||||
try:
|
||||
loaded = await notes_svc.get_note_for_user(uid, task_id)
|
||||
title = getattr(loaded[0], "title", "") if loaded else ""
|
||||
except Exception:
|
||||
title = ""
|
||||
batch = await trash_svc.delete(uid, "task", task_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"task {task_id} not found")
|
||||
return {"deleted": task_id, "title": title, "deleted_batch_id": batch,
|
||||
"message": f'Task {task_id} ("{title}") moved to trash. '
|
||||
f"Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (
|
||||
list_tasks,
|
||||
get_task,
|
||||
create_task,
|
||||
create_records,
|
||||
update_task,
|
||||
add_task_log,
|
||||
start_planning,
|
||||
delete_task,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,48 @@
|
||||
"""Trash MCP tools — unified, batch-based recovery surface.
|
||||
|
||||
Deletes go through the per-type delete_* tools (which soft-delete via the trash
|
||||
service). These tools list / restore / permanently-purge trashed content by
|
||||
the deleted_batch_id that a delete operation returns.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import trash as trash_svc
|
||||
|
||||
|
||||
async def list_trash() -> dict:
|
||||
"""List trashed items, grouped by the deletion batch they were removed in.
|
||||
|
||||
Each batch has: batch_id, deleted_at, a summary (the lead item's title),
|
||||
count, and the member items (type + id + title). Restore a batch with
|
||||
restore(deleted_batch_id).
|
||||
"""
|
||||
return {"batches": await trash_svc.list_trash(current_user_id())}
|
||||
|
||||
|
||||
async def restore(deleted_batch_id: str) -> dict:
|
||||
"""Restore a whole deletion batch from the trash (un-delete)."""
|
||||
n = await trash_svc.restore(current_user_id(), deleted_batch_id)
|
||||
return {"restored": n, "batch_id": deleted_batch_id}
|
||||
|
||||
|
||||
async def purge_trash(deleted_batch_id: str, confirmed: bool = False) -> dict:
|
||||
"""Permanently delete a trashed batch. IRREVERSIBLE — pass confirmed=True.
|
||||
|
||||
This is the only true hard-delete tool; everything else is recoverable.
|
||||
"""
|
||||
if not confirmed:
|
||||
return {
|
||||
"warning": (
|
||||
f"Permanently delete batch {deleted_batch_id}? This cannot be "
|
||||
f"undone. Pass confirmed=True to proceed."
|
||||
),
|
||||
"confirmed_required": True,
|
||||
}
|
||||
n = await trash_svc.purge(current_user_id(), deleted_batch_id)
|
||||
return {"purged": n, "batch_id": deleted_batch_id}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (list_trash, restore, purge_trash):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
@@ -0,0 +1,209 @@
|
||||
"""The wide net — every candidate that might govern what you are about to do.
|
||||
|
||||
WHY IT IS A PULL
|
||||
|
||||
Up to fifty candidates pushed unasked would be a wall the reader skims, with
|
||||
the governing record indistinguishable from forty-nine others. Fifty in a PULL
|
||||
is a different object: it arrives only when a session asks, it crowds nothing
|
||||
out, and it is the honest home for "do not exclude potentially important data".
|
||||
|
||||
WHY IT IS NOT `search(content_type="rule")`
|
||||
|
||||
That tool is the DEEP pull and should stay that way — `_search_rules` returns
|
||||
`statement`, `why` and `how_to_apply` in full, on the reasoning that a caller
|
||||
who went looking deserves the whole record rather than a summary to re-fetch.
|
||||
Six results already run to thousands of tokens.
|
||||
|
||||
This is the SHALLOW pull: many candidates, each just enough to decide whether
|
||||
to open it. Opposite trade-off, so it is a second tool rather than a bigger
|
||||
`limit` on the first.
|
||||
|
||||
WHY THE TAIL IS TRUNCATED RATHER THAN FULL
|
||||
|
||||
Fifty is not free. A line runs to a hundred-odd tokens once its trigger is
|
||||
rendered, so fifty full lines is several thousand — cheap next to an arm that
|
||||
fires before every Bash call, but a tool that promises a free wide net gets
|
||||
reached for casually and then regretted.
|
||||
|
||||
So this uses the push arms' graduated shape: the top few carry their trigger
|
||||
in full, the rest carry a cut of it. TRUNCATED, never
|
||||
dropped — the trigger is what lets a reader judge relevance without opening
|
||||
the record, and a teaser without one is just an id.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import textwrap
|
||||
import time
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services.embeddings import semantic_search_rules
|
||||
from grimoire.services.retrieval_telemetry import record_retrieval
|
||||
|
||||
# The telemetry `source`, and a PULL — never added to AMBIENT_SOURCES (those
|
||||
# are deliveries nobody chose) and never confused with a push arm. Keeping it
|
||||
# separate keeps the push arms' near-miss distributions about the push arms: a
|
||||
# pull mixed into them would move the numbers their floors are tuned from.
|
||||
SOURCE = "wide_net"
|
||||
|
||||
# No bar. That is the point of the tool rather than an oversight — the caller
|
||||
# asked for the wide net precisely because they do not trust a bar to decide
|
||||
# for them here. Every row carries its score, so the reader sees where the
|
||||
# ranking falls off and judges it themselves.
|
||||
THRESHOLD = 0.0
|
||||
|
||||
MAX_LIMIT = 50
|
||||
DEFAULT_LIMIT = 25
|
||||
|
||||
# How many candidates carry their trigger in full before the rest are cut.
|
||||
DEFAULT_DETAIL = 5
|
||||
|
||||
# The cut length for the tail. Long enough to carry the first clause of a
|
||||
# trigger — which is where these state the act they are about — short enough
|
||||
# that forty-five of them stay affordable.
|
||||
_TEASER_CHARS = 140
|
||||
|
||||
|
||||
def _teaser(text: str) -> tuple[str, bool]:
|
||||
"""Shorten at a word break with a visible cut. Returns (text, was_cut).
|
||||
|
||||
The technique is `plugin_context._goal_line`'s, and the fallback is the
|
||||
part worth copying: `textwrap.shorten` returns a bare "…" when the string
|
||||
is one unbroken word longer than the cap, and a raw slice ends mid-word
|
||||
claiming to be the whole thing. Kept local rather than shared
|
||||
because the two callers wrap it in different sentences; if a third appears,
|
||||
that is the moment to extract it rather than now.
|
||||
"""
|
||||
flat = " ".join((text or "").split())
|
||||
if len(flat) <= _TEASER_CHARS:
|
||||
return flat, False
|
||||
short = textwrap.shorten(flat, width=_TEASER_CHARS, placeholder="…")
|
||||
if short == "…":
|
||||
short = flat[: _TEASER_CHARS - 1] + "…"
|
||||
return short, True
|
||||
|
||||
|
||||
async def what_might_apply(
|
||||
query: str,
|
||||
limit: int = DEFAULT_LIMIT,
|
||||
detail: int = DEFAULT_DETAIL,
|
||||
kind: str = "",
|
||||
project_id: int = 0,
|
||||
) -> dict:
|
||||
"""Every rule that might bear on what you are about to do, ranked, with no bar.
|
||||
|
||||
REACH FOR THIS BEFORE A CONSEQUENTIAL OR IRREVERSIBLE ACT — a push, a
|
||||
merge, a delete, a deploy, anything outward-facing — and before handing
|
||||
work back because you are unsure whether you are allowed to finish it.
|
||||
|
||||
It is the tool the "ask before a consequential act" reflex should call.
|
||||
A rule reaches a session by retrieval, and the arms that push rules at you
|
||||
have a small budget: they deliver the few highest-scoring candidates and
|
||||
say nothing about what sat just underneath. That is right for an arm that
|
||||
fires before every command and wrong for the one moment you actually want
|
||||
to be sure. This is that moment's tool.
|
||||
|
||||
WHAT IT RETURNS, AND WHY THE TAIL LOOKS LIKE NOISE
|
||||
|
||||
There is no threshold. You get the `limit` nearest candidates whatever they
|
||||
score, ordered, each with its score — so the tail IS expected to be
|
||||
irrelevant, and that is the design. You are not reading the list for its
|
||||
average quality; you are reading it for the one record you would otherwise
|
||||
have missed. Scan the triggers, open what looks live with `get_rule(id)`,
|
||||
and ignore the rest.
|
||||
|
||||
The first few carry their trigger in full; the rest carry a cut of it,
|
||||
marked `truncated`. A cut trigger is still enough to decide whether to
|
||||
open the record, which is the whole job of a teaser.
|
||||
|
||||
`kind` is on every row and is never something to infer: a **rule** must be
|
||||
followed, a **preference** records how the operator wants work done.
|
||||
Missing a rule is a mistake; missing a preference costs consistency.
|
||||
|
||||
A NOTE ON WHAT THIS CANNOT DO. It is a pull, so it only helps if you ask.
|
||||
The failure it was built for — a session withholding a routine action
|
||||
because the rule permitting it never arrived — produces no tool call of its
|
||||
own, so nothing will prompt you. Asking is the habit; this is where to put
|
||||
it.
|
||||
|
||||
Args:
|
||||
query: what you are about to do, in the words you would use to
|
||||
describe it — "push to dev after committing", "delete the staging
|
||||
database", "merge dev to main". A command string works; a sentence
|
||||
usually works better, because triggers are written as prose.
|
||||
limit: how many candidates, default 25, capped at 50.
|
||||
detail: how many carry their trigger in FULL before the rest are cut,
|
||||
default 5.
|
||||
kind: "rule" or "preference" to restrict; omit for both.
|
||||
project_id: scope to one project — its own rules plus every global
|
||||
one. Omit to ask the whole rulebook.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
limit = max(1, min(int(limit), MAX_LIMIT))
|
||||
detail = max(0, min(int(detail), limit))
|
||||
|
||||
report: dict = {}
|
||||
started = time.perf_counter()
|
||||
if project_id:
|
||||
raw = await semantic_search_rules(
|
||||
uid, query, limit=limit, threshold=THRESHOLD, kind=kind or None,
|
||||
report=report, project_id=project_id,
|
||||
)
|
||||
else:
|
||||
raw = await semantic_search_rules(
|
||||
uid, query, limit=limit, threshold=THRESHOLD, kind=kind or None,
|
||||
report=report, everywhere=True,
|
||||
)
|
||||
duration_ms = (time.perf_counter() - started) * 1000
|
||||
|
||||
candidates = []
|
||||
for rank, (score, rule) in enumerate(raw):
|
||||
full = rank < detail
|
||||
trigger, was_cut = (
|
||||
(" ".join((rule.when_to_apply or "").split()), False)
|
||||
if full else _teaser(rule.when_to_apply or "")
|
||||
)
|
||||
candidates.append({
|
||||
"id": rule.id,
|
||||
"title": rule.title,
|
||||
# Force, not topic. See the docstring — this is never inferred.
|
||||
"kind": rule.kind,
|
||||
# A rule in a rulebook topic is global; one on a project binds
|
||||
# there alone. Which it is changes how far a
|
||||
# reader should generalise from it.
|
||||
"scope": "project" if rule.project_id else "global",
|
||||
"when_to_apply": trigger,
|
||||
"truncated": was_cut,
|
||||
"score": round(float(score), 4),
|
||||
})
|
||||
|
||||
# Logged as a pull, with `searched` honoured: a search that never ran must
|
||||
# not be recorded as a ranker declining.
|
||||
record_retrieval(
|
||||
user_id=uid,
|
||||
source=SOURCE,
|
||||
query=query,
|
||||
threshold=THRESHOLD,
|
||||
limit=limit,
|
||||
project_id=project_id or None,
|
||||
is_task=None,
|
||||
results=raw,
|
||||
duration_ms=duration_ms,
|
||||
best_available=report.get("best_available_score"),
|
||||
best_available_id=report.get("best_available_id"),
|
||||
searched=report.get("searched", True),
|
||||
)
|
||||
|
||||
return {
|
||||
"candidates": candidates,
|
||||
"returned": len(candidates),
|
||||
"detailed": min(detail, len(candidates)),
|
||||
# Present so a caller can tell "the corpus offered nothing" from "the
|
||||
# search never ran" — an empty query, an unavailable embedder and a
|
||||
# failed query all return zero rows and mean different things.
|
||||
"searched": bool(report.get("searched", True)),
|
||||
"open_with": "get_rule(id)",
|
||||
}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
mcp.tool(name="what_might_apply")(what_might_apply)
|
||||
Reference in New Issue
Block a user