feat(mcp): connect card, MCP docs, and an end-to-end roles/audit test over HTTP

MCP-8 #4972 "Connect card, docs, and end-to-end integration test".

- AI access page: a "Connect a client" card with the /mcp URL (from
  Settings → General → Public base URL, else the request host) and recipes
  for Claude Code, Claude Desktop (mcp-remote) and other Streamable HTTP
  clients, plus the reverse-proxy note. The token-created page shows the
  same recipes with the new token filled in, ready to paste.
- docs/core/mcp.md: connecting, the reverse-proxy requirements, the
  security model (token ceiling, default deny, check-mode default, secrets,
  audit), tools by role, response conventions, adding tools.
  docs/architecture.md describes the /mcp dispatch outside the blueprint
  pipeline; docs/reference/code-map.md gains an MCP section.
- tests/integration/test_mcp.py: over HTTP, a viewer token lists no write
  tools and is refused one by name; an operator token reads get_overview
  and its maintenance-window create/end land in the audit log as
  "<user> via MCP" with the token id. (Revoked → 401 was already covered.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/architecture.md

@@ -44,6 +44,30 @@ All routes are Quart Blueprints registered in `app.py`:
 
 Plugin blueprints are mounted automatically by `load_plugins()` using the plugin directory name as the URL prefix.
 
+`/ai-access/` (`ai_access_bp`, `steward/ai_access/routes.py`) is where a signed-in user manages their API tokens.
+
+### `/mcp`: outside the blueprint pipeline
+
+`mount_mcp(app)` (`steward/mcp/server.py`) wraps `app.asgi_app`. A request to
+`/mcp` is handed to the MCP SDK's streamable-HTTP app before Quart sees it, so
+no blueprint, session cookie or `before_request` hook runs for it. Each
+request then goes through these steps:
+
+1. **Database gate.** If the database is unreachable, the request gets a 503
+   with a readable message.
+2. **Bearer resolution.** `api_tokens.resolve_token()` turns the token into a
+   `Caller` (user, effective role, token id). A missing, revoked or expired
+   token gets a 401.
+3. **Caller context.** The `Caller` and the app are set in ContextVars
+   (`steward/mcp/_context.py`). Tools read them through `current_caller()` and
+   `mcp_app()`, because Quart's `current_app` and `session` do not exist there.
+4. **Role gate.** `StewardMCPServer.refusal()` hides and refuses tools above
+   the caller's role, and plugin tools whose plugin is disabled.
+
+The SDK's session manager runs in Quart's `before_serving`/`after_serving`
+lifecycle. The server is stateless: no MCP session outlives a request. See
+[MCP server](core/mcp.md) for the security model and the tools.
+
 ---
 
 ## Scheduler

docs/core/mcp.md

@@ -0,0 +1,132 @@
+# MCP server
+
+Steward serves the [Model Context Protocol](https://modelcontextprotocol.io) at
+`/mcp`. An AI client connected to it can look into everything Steward watches
+(hosts, monitors, alerts, metrics, containers, playbook runs) and, with a
+high enough role, change it. A token's role sets that ceiling.
+
+**Source:** `steward/mcp/`. The tools live in `steward/mcp/tools/`, and a
+plugin's tools live in `plugins/<name>/mcp_tools.py`.
+**Transport:** streamable HTTP, stateless. Every request carries its own
+token, so a client survives a Steward restart without reconnecting.
+
+---
+
+## Connecting
+
+1. In Steward, open **AI access** and create a token. Pick the lowest role
+   that does the job.
+2. Copy the token (it is shown once) and use one of the recipes on that
+   page. They carry the real endpoint URL and the new token. For Claude Code:
+
+   ```sh
+   claude mcp add --transport http steward https://steward.example/mcp \
+     --header "Authorization: Bearer stw_…"
+   ```
+
+3. Ask the client to call `whoami`. It answers with the user, the effective
+   role and what that role allows.
+
+Claude Desktop connects through `mcp-remote`. Open WebUI and other clients
+need an MCP server with the **Streamable HTTP** transport and bearer
+authentication. The AI access page shows the recipes for both.
+
+### Behind a reverse proxy
+
+- The URL on the AI access page is **Settings → General → Public base URL**
+  plus `/mcp`. If that setting is empty, the URL is built from the request's
+  host instead.
+- The proxy must pass the `Authorization` header through unchanged.
+- It must not buffer responses. Replies may arrive as server-sent events
+  (`text/event-stream`). For nginx, set `proxy_buffering off;` on the
+  location.
+- Steward does no Host-header (DNS-rebinding) check on `/mcp`, because the
+  bearer token is the security boundary. Serve it over HTTPS.
+
+---
+
+## Security model
+
+| Rule | How it is enforced |
+|---|---|
+| Bearer tokens only, no OAuth | `steward/core/api_tokens.py`. Only a hash is stored, and the token is shown once. |
+| A token never acts above its user | Effective role = min(token role, user's **current** role). Lowering a user lowers every token they hold; disabling the user stops them all. |
+| Default deny | A tool is reachable only if it was registered with a role (`register_tool(fn, role=..., read_only=...)`). `tools/list` shows a caller only the tools their role allows. A call above the role is refused, and the refusal names the role it needs. |
+| Plugin tools follow the plugin | A disabled plugin's tools are hidden and refused. |
+| Dry run first | `run_playbook`, `create_schedule` and `set_alert_action` default to Ansible check mode (`--check --diff`). Changes are applied only when the caller passes `check=false`. A playbook its author flagged `# steward:confirm: true` also needs `confirmed=true`. `prune_docker` previews unless `check=false`. |
+| Secrets stay put | Playbook variables Steward detects as secret reach the run but are never stored, audited or returned. Schedules and alert actions refuse them, since they would have to store them. Inventory variables are listed by name only, and HTTP monitor header values are redacted. |
+| Every change is audited | Each write records an audit event as `<user> via MCP`, with the token id in its detail. `list_audit_events(user="<user>")` includes those events. |
+| Unknown arguments are refused | A misspelt argument is rejected with a "did you mean", rather than silently dropped. |
+
+Tokens never mint or revoke tokens; that happens only on the AI access
+page, in a browser session.
+
+---
+
+## Tools by role
+
+Each role includes the tools of the roles above it in this list.
+
+### Viewer: read everything
+
+| Area | Tools |
+|---|---|
+| Start here | `whoami`, `get_overview` (what is wrong now, ranked, each finding with the call that shows its evidence) |
+| Hosts and metrics | `list_hosts`, `get_host`, `get_host_metrics`, `query_metrics` |
+| Monitors | `list_monitors`, `get_monitor` |
+| Alerts | `list_alerts`, `get_alert_history`, `list_alert_rules`, `get_alert_rule_options`, `list_maintenance_windows` |
+| Automation | `list_playbooks`, `get_playbook`, `list_inventory`, `list_ansible_runs`, `get_ansible_run`, `wait_for_run`, `list_schedules` |
+| Audit | `list_audit_events` |
+| Docker plugin | `list_containers`, `get_container`, `list_swarm_services`, `search_container_logs`, `get_docker_disk_usage` |
+| Host agent plugin | `list_agents` |
+
+### Operator: run playbooks, change monitoring
+
+| Area | Tools |
+|---|---|
+| Automation | `run_playbook`, `cancel_ansible_run`, `create_schedule`, `update_schedule`, `delete_schedule` |
+| Monitors | `create_monitor`, `update_monitor`, `set_monitor_enabled` |
+| Alerts | `create_alert_rule`, `update_alert_rule`, `acknowledge_alert`, `create_maintenance_window`, `end_maintenance_window` |
+
+### Admin: destructive and remediation
+
+`set_alert_action` (the playbook a rule runs when it fires), `delete_monitor`,
+`delete_alert_rule`, `prune_docker`.
+
+The client sees each tool's full contract: arguments, what an empty value
+means, and what comes back. That contract is the tool's docstring, so read the
+docstrings in `steward/mcp/tools/` for the detail.
+
+---
+
+## Response conventions
+
+Every tool follows these conventions (`steward/mcp/tools/_shape.py`):
+
+- **Times** are ISO-8601 UTC with a `Z`. Anything with a "when" also carries
+  `age_s`, so the client never does clock arithmetic.
+- **Lists are bounded.** A list response carries `total` and `truncated`. When
+  it was cut, `more` names the argument that fetches the rest. Nothing is cut
+  silently.
+- **Time series** come back as a summary (min/avg/max/p95/last) plus a series
+  of at most 60 points, never raw rows.
+- **Lookup by name or id.** A miss suggests close matches and names the tool
+  that lists the valid choices.
+- **Refusals are readable.** Bad input becomes a tool error that says what was
+  wrong, why, and what to do instead, and nothing is changed. The tools share
+  their validation with the web forms (`monitors/service.py`,
+  `alerts/service.py`), so a value the UI refuses is refused here for the same
+  reason.
+- **Database outages** return a plain "Steward cannot reach its database"
+  rather than a stack trace.
+
+---
+
+## Adding tools
+
+Core tools: add an `async def` to a module in `steward/mcp/tools/` and list it
+in that module's `register()` with its minimum role and `read_only`. A plugin
+exposes `register_mcp_tools(register)`, and optionally `mcp_overview(db)` to
+contribute findings to `get_overview`. See
+[Writing a plugin](../plugins/writing-a-plugin.md). A tool that changes
+anything calls `audit_mcp(...)` from `steward/mcp/tools/_audit.py`.

docs/reference/code-map.md

@@ -136,3 +136,25 @@ Quick reference for where key functions, models, and entry points live in the co
 | `steward/migrations/versions/` | Core schema (hosts, users, monitors, alerts, app_settings) |
 | `plugins/traefik/migrations/versions/` | `traefik_metrics` table |
 | `alembic.ini` | Alembic config; `version_locations` lists all migration directories |
+
+---
+
+## MCP server and API tokens
+
+| What | File | Function / Class |
+|---|---|---|
+| Mount `/mcp` (ASGI dispatch, bearer auth, DB gate) | `steward/mcp/server.py` | `mount_mcp(app)` |
+| Server, role registry, refusals | `steward/mcp/server.py` | `StewardMCPServer`, `register_tool()`, `refusal()` |
+| Plugin tools and overview findings | `steward/mcp/server.py` | `register_plugin(name, module)` |
+| Caller and app inside a tool | `steward/mcp/_context.py` | `current_caller()`, `mcp_app()` |
+| Core tool modules | `steward/mcp/tools/` | `register_all()`; one module per area |
+| Response conventions | `steward/mcp/tools/_shape.py` | `iso`, `age_s`, `bounded`, `lookback`, `summarize`, `pick` |
+| Audit a write as "<user> via MCP" | `steward/mcp/tools/_audit.py` | `audit_mcp()` |
+| "What is wrong now" | `steward/mcp/tools/overview.py` | `get_overview()` |
+| Playbooks, runs, schedules, alert actions | `steward/mcp/tools/automation.py` | `run_playbook()`, `wait_for_run()`, … |
+| Monitor validation (forms and tools) | `steward/monitors/service.py` | `monitor_fields()`, `build_config()` |
+| Alert rule / window validation | `steward/alerts/service.py` | `rule_fields()`, `window_fields()`, `acknowledge()` |
+| Docker and host-agent tools | `plugins/docker/mcp_tools.py`, `plugins/host_agent/mcp_tools.py` | `register_tools()`, `mcp_overview()` |
+| Token model | `steward/models/api_tokens.py` | `ApiToken` |
+| Mint, resolve, revoke tokens | `steward/core/api_tokens.py` | `create_token()`, `resolve_token()`, `revoke_token()` |
+| Token management page and connect card | `steward/ai_access/routes.py`, `steward/templates/ai_access/` | `ai_access_bp` |

steward/ai_access/routes.py

@@ -14,6 +14,8 @@ from sqlalchemy import select
 from steward.auth.middleware import current_user_id, require_role
 from steward.core import api_tokens
 from steward.core.audit import log_audit
+from steward.core.settings import public_base_url
+from steward.mcp.server import MCP_PATH
 from steward.models.users import User, UserRole
 
 ai_access_bp = Blueprint("ai_access", __name__, url_prefix="/ai-access")
@@ -21,6 +23,11 @@ ai_access_bp = Blueprint("ai_access", __name__, url_prefix="/ai-access")
 _EXPIRY_CHOICES = [("", "Never"), ("30", "30 days"), ("90", "90 days"), ("365", "1 year")]
 
 
+def _mcp_url() -> str:
+    """The /mcp endpoint as a client outside this request reaches it."""
+    return public_base_url(request) + MCP_PATH
+
+
 async def _session_user(db) -> User:
     user_id = current_user_id()
     if not user_id:
@@ -42,6 +49,7 @@ async def _render_index(db, user: User, *, error: str | None = None, status: int
         grantable=api_tokens.grantable_roles(user.role),
         expiry_choices=_EXPIRY_CHOICES,
         error=error,
+        mcp_url=_mcp_url(),
     )
     return body, status
 
@@ -81,7 +89,8 @@ async def create():
     await log_audit(current_app, user.id, user.username, "api_token.create",
                     entity_type="api_token", entity_id=str(token.id),
                     detail={"name": token.name, "role": token.role.value})
-    return await render_template("ai_access/created.html", raw_token=raw, token=token)
+    return await render_template("ai_access/created.html", raw_token=raw, token=token,
+                                 mcp_url=_mcp_url())
 
 
 @ai_access_bp.post("/tokens/<int:token_id>/revoke")

steward/templates/ai_access/_connect.html

@@ -0,0 +1,56 @@
+{#- How to point an AI client at /mcp. `mcp_url` is the endpoint; `token` is the
+    raw token on the page that just created one, else a placeholder. -#}
+{% set tok = token or "stw_…your token…" %}
+{% set pre = "background:var(--bg);padding:0.75rem;border-radius:4px;overflow-x:auto;font-size:0.78rem;color:var(--text);margin:0.5rem 0 0;white-space:pre;" %}
+<div class="card" id="connect">
+  <div class="section-title" style="margin-bottom:0.5rem;">Connect a client</div>
+  <div style="font-size:0.85rem;color:var(--text-muted);line-height:1.55;margin-bottom:0.75rem;">
+    Steward's MCP endpoint. Clients send the token as
+    <code>Authorization: Bearer …</code> on every request.
+  </div>
+  <div style="display:flex;gap:0.5rem;align-items:center;margin-bottom:1rem;">
+    <input type="text" id="mcp-url" value="{{ mcp_url }}" readonly aria-label="MCP endpoint URL"
+           style="flex:1;font-family:ui-monospace,monospace;font-size:0.8rem;background:var(--bg);color:var(--text);">
+    <button type="button" class="btn btn-ghost btn-sm" style="white-space:nowrap;"
+            onclick="navigator.clipboard.writeText(document.getElementById('mcp-url').value).then(()=>this.textContent='Copied')">Copy</button>
+  </div>
+
+  <details open>
+    <summary style="cursor:pointer;font-weight:600;">Claude Code</summary>
+    <pre style="{{ pre }}">claude mcp add --transport http steward {{ mcp_url }} \
+  --header "Authorization: Bearer {{ tok }}"</pre>
+  </details>
+
+  <details style="margin-top:0.75rem;">
+    <summary style="cursor:pointer;font-weight:600;">Claude Desktop</summary>
+    <div style="font-size:0.8rem;color:var(--text-muted);margin-top:0.4rem;line-height:1.5;">
+      Add to <code>claude_desktop_config.json</code>. It bridges through
+      <code>mcp-remote</code>, which needs Node.js.
+    </div>
+    <pre style="{{ pre }}">{
+  "mcpServers": {
+    "steward": {
+      "command": "npx",
+      "args": ["mcp-remote", "{{ mcp_url }}",
+               "--header", "Authorization:${STEWARD_AUTH}"],
+      "env": { "STEWARD_AUTH": "Bearer {{ tok }}" }
+    }
+  }
+}</pre>
+  </details>
+
+  <details style="margin-top:0.75rem;">
+    <summary style="cursor:pointer;font-weight:600;">Open WebUI and other clients</summary>
+    <div style="font-size:0.8rem;color:var(--text-muted);margin-top:0.4rem;line-height:1.5;">
+      Add an MCP server using the <strong>Streamable HTTP</strong> transport, with the URL
+      above and bearer authentication set to the token. In Open WebUI this is an external
+      tool connection of type MCP; menu names vary between versions.
+    </div>
+  </details>
+
+  <div style="font-size:0.78rem;color:var(--text-muted);margin-top:1rem;line-height:1.5;">
+    Behind a reverse proxy, the URL above comes from <strong>Settings → General → Public
+    base URL</strong> when it is set. The proxy must pass the <code>Authorization</code>
+    header through and not buffer responses, since replies stream.
+  </div>
+</div>

steward/templates/ai_access/created.html

@@ -33,6 +33,8 @@
     If it leaks, revoke it from the AI access page.
   </p>
 
+  {% with token = raw_token %}{% include "ai_access/_connect.html" %}{% endwith %}
+
   <a href="/ai-access/" class="btn btn-ghost">Back to AI access</a>
 </div>
 {% endblock %}

steward/templates/ai_access/index.html

@@ -116,4 +116,9 @@
   </div>
 
 </div>
+
+<div style="max-width:720px;margin-top:1.5rem;">
+  {% set token = none %}
+  {% include "ai_access/_connect.html" %}
+</div>
 {% endblock %}

tests/integration/test_api_tokens.py

@@ -139,6 +139,8 @@ def test_routes_create_list_and_revoke(app):
         assert "You have no tokens yet" in page
         # An operator can't offer admin.
         assert 'value="admin"' not in page
+        # The connect card names the endpoint, with a placeholder for the token.
+        assert 'value="http://localhost/mcp"' in page and "claude mcp add" in page
 
         resp = await client.post("/ai-access/tokens", form={
             "name": "laptop", "role": "admin", "expires_in_days": ""})
@@ -148,7 +150,10 @@ def test_routes_create_list_and_revoke(app):
         resp = await client.post("/ai-access/tokens", form={
             "name": "laptop", "role": "operator", "expires_in_days": "30"})
         assert resp.status_code == 200
-        raw = re.search(r'value="(stw_[^"]+)"', await resp.get_data(as_text=True)).group(1)
+        created = await resp.get_data(as_text=True)
+        raw = re.search(r'value="(stw_[^"]+)"', created).group(1)
+        # The recipes on the created page carry the new token, ready to paste.
+        assert f"Bearer {raw}" in created and "mcp-remote" in created
         caller = await api_tokens.resolve_token(app, raw)
         assert caller is not None and caller.role == UserRole.operator
 

tests/integration/test_mcp.py

@@ -72,3 +72,74 @@ def test_mcp_over_http_with_a_minted_token(app):
             assert status == 401
 
     asyncio.run(_go())
+
+
+def _tool_result(body) -> dict:
+    return rpc_result(body)["result"]
+
+
+@_NEEDS_DB
+def test_roles_reads_and_audited_writes_over_http(app):
+    """Over HTTP: a viewer sees no write tools and is refused one by name; an
+    operator reads get_overview and makes an audited change."""
+    from sqlalchemy import select
+    from steward.core import api_tokens
+    from steward.models.audit import AuditEvent
+    from steward.models.users import User, UserRole
+
+    async def _go():
+        uid = str(uuid.uuid4())
+        username = f"mcp-op-{uid[:8]}"
+        async with app.db_sessionmaker() as s:
+            async with s.begin():
+                s.add(User(id=uid, username=username, email=f"{uid}@test",
+                           password_hash="x", role=UserRole.operator, is_active=True))
+        async with app.db_sessionmaker() as s:
+            viewer_raw, _ = await api_tokens.create_token(
+                s, user_id=uid, user_role=UserRole.operator, name="ro", role=UserRole.viewer)
+            op_raw, op_token = await api_tokens.create_token(
+                s, user_id=uid, user_role=UserRole.operator, name="rw", role=UserRole.operator)
+            await s.commit()
+        viewer = {**_HEADERS, "Authorization": f"Bearer {viewer_raw}"}
+        operator = {**_HEADERS, "Authorization": f"Bearer {op_raw}"}
+
+        async with app.mcp_server.session_manager.run():
+            _, _, body = await asgi_request(app, "POST", "/mcp", viewer, rpc("tools/list", id_=1))
+            names = {t["name"] for t in _tool_result(body)["tools"]}
+            assert "get_overview" in names and "list_playbooks" in names
+            assert not names & {"run_playbook", "create_monitor", "delete_alert_rule"}
+
+            _, _, body = await asgi_request(app, "POST", "/mcp", viewer, rpc(
+                "tools/call", {"name": "create_maintenance_window",
+                               "arguments": {"name": "x", "duration_minutes": 5}}, id_=2))
+            refused = _tool_result(body) if "result" in rpc_result(body) else rpc_result(body)["error"]
+            assert "needs the operator role" in json.dumps(refused)
+
+            _, _, body = await asgi_request(app, "POST", "/mcp", operator, rpc(
+                "tools/call", {"name": "get_overview", "arguments": {}}, id_=3))
+            result = _tool_result(body)
+            assert not result.get("isError"), result
+            overview = json.loads(result["content"][0]["text"])
+            assert {"generated_at", "all_clear", "counts", "findings"} <= set(overview)
+
+            name = f"it-window-{uid[:8]}"
+            _, _, body = await asgi_request(app, "POST", "/mcp", operator, rpc(
+                "tools/call", {"name": "create_maintenance_window",
+                               "arguments": {"name": name, "duration_minutes": 5}}, id_=4))
+            result = _tool_result(body)
+            assert not result.get("isError"), result
+            window_id = json.loads(result["content"][0]["text"])["window_id"]
+            _, _, body = await asgi_request(app, "POST", "/mcp", operator, rpc(
+                "tools/call", {"name": "end_maintenance_window",
+                               "arguments": {"window": window_id}}, id_=5))
+            assert not _tool_result(body).get("isError")
+
+        async with app.db_sessionmaker() as s:
+            rows = list((await s.execute(select(AuditEvent).where(
+                AuditEvent.user_id == uid, AuditEvent.entity_id == name))).scalars())
+        assert sorted(e.action for e in rows) == ["maintenance_window.created",
+                                                  "maintenance_window.ended"]
+        assert {e.username for e in rows} == {f"{username} via MCP"}
+        assert all(json.loads(e.detail_json)["token_id"] == op_token.id for e in rows)
+
+    asyncio.run(_go())