diff --git a/docs/core/mcp.md b/docs/core/mcp.md index 2ff2ed7..422a64a 100644 --- a/docs/core/mcp.md +++ b/docs/core/mcp.md @@ -78,7 +78,7 @@ Each role includes the tools of the roles above it in this list. | Metric anomalies | `list_anomalies` (metrics that left their normal for the hour, to review first), `get_anomaly` (one anomaly with each occurrence against what is usual at that hour, the metric around it, what else happened on its host — alerts, Ansible runs, container deaths and starts, maintenance, reboots, other anomalies — and how earlier ones on the same metric were judged), `get_metric_normal` (the learned normal slot by slot, and the level that would be flagged), `get_anomaly_settings` (the review window and each floor) | | Automation | `list_playbooks`, `get_playbook`, `list_inventory`, `list_ansible_runs`, `get_ansible_run`, `wait_for_run`, `list_schedules` | | Audit | `list_audit_events` | -| Timeline | `get_timeline` (everything that happened, newest first, in one list: alerts firing and clearing, playbook runs and what they changed, changes to Steward's settings; by host, area or kind, and kept after the records it came from age out) | +| Timeline | `get_timeline` (everything that happened, newest first, in one list: alerts firing and clearing, playbook runs and what they changed, maintenance, reboots, deploys and crash-loops, reviewed anomalies, changes recorded by hand, changes to Steward's settings; by host, area or kind, and kept after the records it came from age out) | | Docker plugin | `list_containers`, `get_container`, `list_swarm_services`, `search_container_logs`, `get_docker_disk_usage` | | Host agent (built in, always on) | `list_agents`, `list_maintenance_plans` (each plan's group, weekly slot, next run and latest run: how far it got, who it skipped and why), `get_maintenance_job` (a host's update jobs or one job: state, packages updated, whether it rebooted, why it failed), `get_host_senders` (who sent or received a host's network traffic over a window: its containers and NFS mounts ranked at the host's peak, and what none of them accounts for) | | Traefik plugin | `list_routers` (each router graded as the Traefik page grades it, with the status codes behind a problem, what they mean and the swarm service behind it), `get_router` (one router's history, codes, normal and anomalies), `list_certificates` (served certificates, days left, the routers they cover), `get_traffic` (the access log's clients, countries and routers, and Traefik's own figures) | @@ -95,6 +95,7 @@ Each role includes the tools of the roles above it in this list. | Monitors | `create_monitor`, `update_monitor`, `set_monitor_enabled` | | Alerts | `create_alert_rule`, `update_alert_rule`, `acknowledge_alert`, `send_test_alert`, `create_maintenance_window`, `end_maintenance_window` | | Metric anomalies | `review_anomaly` (mark one or several normal, so their hours are learned, or real, with a note saying why) | +| Timeline | `log_change` (record a change Steward can't see happen — a config edited by hand, a disk swapped — on the hosts it was done on, under an area; shown as by " via MCP") | ### Admin: destructive and remediation diff --git a/docs/core/timeline.md b/docs/core/timeline.md index c2cb7c0..c0fb63a 100644 --- a/docs/core/timeline.md +++ b/docs/core/timeline.md @@ -1,6 +1,6 @@ # Timeline -The timeline is one list of everything that happened to the infrastructure, newest first: alerts firing and clearing, playbook runs and what they changed, maintenance runs and reboots, deploys and crash-loops, anomalies once reviewed, and changes to Steward's own settings. Open it from **Timeline** in the sidebar, from a host's **History** card, or through the `get_timeline` MCP tool. +The timeline is one list of everything that happened to the infrastructure, newest first: alerts firing and clearing, playbook runs and what they changed, maintenance runs and reboots, deploys and crash-loops, anomalies once reviewed, changes recorded by hand, and changes to Steward's own settings. Open it from **Timeline** in the sidebar, from a host's **History** card, or through the `get_timeline` MCP tool. Each entry stands on its own. An alert event, an Ansible run, a maintenance job or an audit row is pruned after its own retention, but the entry for it stays. Its link then reads "Details no longer kept". @@ -22,24 +22,31 @@ Every source writes its entry in the same transaction as its own record, through | Maintenance jobs, when they end (and again if the host comes back late) | host agent: `maintenance`, `cadence` | `maintenance.done`, `maintenance.failed` | | A host away: a reboot, a crash, lost contact, the agent restarting; kept up to date until it ends | host agent: `downtime.record_events` | `host.rebooted`, `host.lost_contact`, `host.agent_restarted` | | Container snapshots: a new image, a recreated container, a restart or death | Docker: `ingest.persist_host_docker` → `timeline.from_snapshot` | `docker.deployed`, `docker.crash_loop`, `docker.restarted` | +| A change recorded by hand, from the Timeline page, a host's History card or MCP `log_change` | `timeline/changes.py` | `change.recorded` | A source whose record changes after it is written keeps one entry for it (`record.record_or_update`): a downtime gets its length when it ends, an anomaly changes type when it is reviewed. Docker's units are swarm services where a container has one, so a service's tasks, replaced on every deploy and crash, read as one subject. A swarm task moving to another host is not a deploy. A crash loop uses the same rule as the Docker pages: Settings → Thresholds' deaths within an hour. The death that reaches it opens one `docker.crash_loop` entry, and each later one extends it until the container has been quiet for an hour. The deaths before it stay as hidden `docker.restarted` entries. -Some audit actions are left to another writer, because the event itself is recorded there: starting a run, acknowledging an alert, and starting maintenance. A person's own preferences (time zone, email settings) are not recorded. `catalog.audit_type` holds that mapping. +Some audit actions are left to another writer, because the event itself is recorded there: starting a run, acknowledging an alert, starting maintenance, and recording a change by hand. A person's own preferences (time zone, email settings) are not recorded. `catalog.audit_type` holds that mapping. On its first start, Steward fills the timeline from the last 90 days of records that predate it (`service.backfill`): core's alert events, finished runs and audit rows, and each loaded plugin's own through its `timeline_backfill(session, since)` (the host agent's maintenance jobs and downtime). It fills from each source once and records which in `timeline.filled`, so a plugin loaded later fills on the next daily turn. An entry for a record that is already covered is skipped. Docker has no fill: its events can't tell a deploy from a restart after the fact. `get_anomaly`'s "what else was going on" reads the timeline for the host or resource, hidden types included, beside each occurrence. +## Recording a change by hand + +Steward can't see every change happen: a config edited on a host, a disk swapped, a service moved. Whoever made it records it, with **Record a change** on the Timeline page or a host's History card, or the `log_change` MCP tool. It says what was done, the area it is about, when, the hosts it was done on (none for a change not on any one host), optionally what it was done to (a container, a file) and a longer detail. + +An operator or admin can record one. Its author can edit or delete it from its **Edit** link, and so can an admin. One recorded through MCP is shown as by " via MCP" and is editable by that token's user. Recording, editing and deleting are audited (`timeline.change_recorded`, `…_edited`, `…_deleted`), and those audit rows stay off the timeline, which holds the change itself. + ## Areas and kinds An entry has a **kind**, which says how it came about: Change, Automation, Alert, Anomaly, Ticket, Observed (Steward saw it happen with no rule or person behind it: a reboot, a crash-loop) or Steward. Kinds are fixed in `catalog.KINDS`. An entry also has an **area**, which says what part of the infrastructure it is about. The areas start as Service, Host OS, Hardware, Storage, Network, Security and Monitoring. -Each entry is written with an area of its own. An alert's area comes from what its rule watches (`catalog.area_of_metric`): drive health is Hardware, a mount is Storage, an HTTP monitor is Service. Other entries use their type's area. +Each entry is written with an area of its own. An alert's area comes from what its rule watches (`catalog.area_of_metric`): drive health is Hardware, a mount is Storage, an HTTP monitor is Service. A change recorded by hand has the area its author picked. Other entries use their type's area. ## Settings → Timeline diff --git a/steward/audit/service.py b/steward/audit/service.py index 537fabd..a79fd48 100644 --- a/steward/audit/service.py +++ b/steward/audit/service.py @@ -94,6 +94,12 @@ async def entity_labels(db, items: list[dict]) -> dict[tuple[str, str], dict]: labels[(etype, r.id)] = { "label": run_title(r.playbook_path, scopes.get(r.inventory_scope, r.inventory_scope)), "url": f"/ansible/runs/{r.id}", "gone": False} + elif etype == "timeline_entry": + from steward.models.timeline import TimelineEntry + for row_id, summary in (await db.execute( + select(TimelineEntry.id, TimelineEntry.summary) + .where(TimelineEntry.id.in_(list(wanted))))).all(): + labels[(etype, row_id)] = {"label": summary, "url": None, "gone": False} elif etype in _SELF_NAMED: for i in wanted: labels[(etype, i)] = {"label": i, "url": _SELF_NAMED[etype], "gone": False} @@ -206,6 +212,9 @@ ACTIONS = { "syslog.pattern_added": "added syslog pattern", "syslog.pattern_updated": "changed syslog pattern", "syslog.pattern_removed": "removed syslog pattern", + "timeline.change_recorded": "recorded a change:", + "timeline.change_edited": "edited the recorded change", + "timeline.change_deleted": "deleted the recorded change", "backup.made": "sent a backup", "backup.failed": "tried to send a backup, which failed:", "backup.restored": "restored Steward from a backup", @@ -218,7 +227,7 @@ AREAS = (("auth.", "Sign-ins"), ("account.", "Accounts"), ("user.", "Users"), (" ("ansible.run", "Playbook runs"), ("ansible.", "Ansible (all)"), ("settings.", "Settings"), ("plugin.", "Plugins"), ("api_token.", "AI access"), ("docker.", "Docker"), ("snmp.", "SNMP"), ("prometheus.", "Prometheus"), - ("syslog.", "Syslog"), ("backup.", "Backups")) + ("syslog.", "Syslog"), ("backup.", "Backups"), ("timeline.", "Timeline")) def action_text(action: str) -> str: diff --git a/steward/mcp/tools/timeline.py b/steward/mcp/tools/timeline.py index cb264ad..b9b5fcf 100644 --- a/steward/mcp/tools/timeline.py +++ b/steward/mcp/tools/timeline.py @@ -1,13 +1,39 @@ -"""The timeline: everything that happened to the infrastructure, in one list.""" +"""The timeline: everything that happened to the infrastructure, in one +list, and changes recorded on it by hand.""" from __future__ import annotations +from datetime import datetime, timezone + +from steward.mcp._context import current_caller from steward.models.timeline import TimelineEntry from steward.models.users import UserRole -from steward.timeline import catalog, config as tl_config, service as tl +from steward.timeline import catalog, changes, config as tl_config, service as tl +from ._audit import audit_mcp, mcp_username from ._shape import age_s, bounded, clamp_limit, db_session, iso, lookback +def _entry(v: dict) -> dict: + e = v["entry"] + return { + "entry_id": e.id, + "at": iso(e.at), + "age_s": age_s(e.at), + "until": iso(e.until), + "type": e.type, + "kind": v["kind"], + "area": v["area"], + "summary": e.summary, + "note": v["note"], + "hosts": e.hosts, + "subject": e.subject, + "by": e.actor, + "url": v["url"], + "details_kept": not v["gone"], + "ref": {"type": e.ref_type, "id": e.ref_id} if e.ref_type else None, + } + + async def get_timeline(host: str = "", area: str = "", kind: str = "", type: str = "", range: str = "7d", include_hidden: bool = False, limit: int = 50) -> dict: """What happened to the infrastructure, newest first, from every source in @@ -48,25 +74,7 @@ async def get_timeline(host: str = "", area: str = "", kind: str = "", type: str rows = (await db.execute(stmt.order_by(TimelineEntry.at.desc(), TimelineEntry.id.desc()) .limit(limit + 1))).scalars().all() items = await tl.views(db, cfg, rows) - entries = [ - { - "at": iso(v["entry"].at), - "age_s": age_s(v["entry"].at), - "until": iso(v["entry"].until), - "type": v["entry"].type, - "kind": v["kind"], - "area": v["area"], - "summary": v["entry"].summary, - "note": v["note"], - "hosts": v["entry"].hosts, - "subject": v["entry"].subject, - "by": v["entry"].actor, - "url": v["url"], - "details_kept": not v["gone"], - "ref": {"type": v["entry"].ref_type, "id": v["entry"].ref_id} if v["entry"].ref_type else None, - } - for v in items - ] + entries = [_entry(v) for v in items] out = {"range": key, **bounded(entries, limit, key="entries"), "areas": dict(cfg.labels), "types": sorted(catalog.TYPES)} if out["truncated"]: @@ -76,5 +84,67 @@ async def get_timeline(host: str = "", area: str = "", kind: str = "", type: str return out +def _area(cfg, area: str) -> str: + """An area by its key or its label, as Settings → Timeline names it.""" + want = (area or "").strip().lower() + for key, label in cfg.areas: + if want in (key, label.lower()): + return key + raise ValueError(f"area {area!r} is not one of {', '.join(cfg.labels)} (get_timeline lists them).") + + +async def log_change(summary: str, area: str, hosts: list[str] | str | None = None, subject: str = "", + detail: str = "", at: str = "") -> dict: + """Record on the timeline a change Steward can't see happen: a config + edited by hand, a package installed, a disk swapped, a service moved, a + setting changed in another system. Record one whenever you, or the person + you work with, change the infrastructure outside Steward's own tools, so + "what changed on this host lately?" has the answer later. Playbook runs, + maintenance, deploys and Steward's own settings are recorded already. + + summary: what was done, one line (up to 300 characters), e.g. "Moved + Postgres data to the new SSD". + area: what part of the infrastructure it is about, by key or label: + service, host_os, hardware, storage, network, security, monitoring, as + Settings → Timeline names them (get_timeline's `areas`). + hosts: the host(s) it was done on, by exact name (list_hosts); none for a + change not on any one host. + subject: what it was done to, when that is not a host: a container, a + service, a file ("postgres", "/etc/fstab"). + detail: why and how, and anything worth knowing if it goes wrong (up to + 4000 characters). + at: when it was done, ISO-8601 (a time with no zone is UTC); default now. + + Shown as by " via MCP"; the person whose token this is can edit or + delete it from the Timeline page. Recorded in the audit log. + """ + if isinstance(hosts, str): + hosts = hosts.split(",") + when = None + if (at or "").strip(): + try: + when = datetime.fromisoformat(at.strip().replace("Z", "+00:00")) + except ValueError: + raise ValueError(f"at {at!r} is not an ISO-8601 time, e.g. 2026-10-06T22:00:00Z.") from None + when = when if when.tzinfo else when.replace(tzinfo=timezone.utc) + caller = current_caller() + async with db_session() as db: + async with db.begin(): + cfg = await tl_config.load(db) + try: + f = await changes.fields(db, cfg, summary=summary, area=_area(cfg, area), hosts=hosts or [], + subject=subject, note=detail, at=when) + except ValueError as exc: + if "No host is named" in str(exc): + raise ValueError(f"{exc} list_hosts gives the names.") from None + raise + entry = changes.log(db, f, author_id=caller.user_id, actor=mcp_username(caller.username)) + [v] = await tl.views(db, cfg, [entry]) + await audit_mcp("timeline.change_recorded", "timeline_entry", entry.id, + {"name": entry.summary, "hosts": entry.hosts, "area": entry.area}) + return _entry(v) + + def register(register_tool) -> None: register_tool(get_timeline, role=UserRole.viewer, read_only=True) + register_tool(log_change, role=UserRole.operator) diff --git a/steward/templates/timeline/_entry.html b/steward/templates/timeline/_entry.html index aa9f313..afc254e 100644 --- a/steward/templates/timeline/_entry.html +++ b/steward/templates/timeline/_entry.html @@ -1,16 +1,19 @@ {# One timeline entry's parts, shared by the Timeline page and a host's card. `item` is a timeline.service.view() row. #} -{# What happened: its kind, the sentence, a note, and the full record while it is kept. #} -{% macro entry_what(item) -%} +{# What happened: its kind, the sentence, a note, and the full record while it + is kept; a change the reader recorded (or any, for an admin) can be edited, + returning to `back`. #} +{% macro entry_what(item, back="/timeline/") -%} {%- set e = item.entry -%}
{%- if item.kind_label %}{{ item.kind_label }} · {% endif -%} {{ e.summary }}
-{%- if item.note %}
{{ item.note }}
{% endif %} +{%- if item.note %}
{{ item.note }}
{% endif %} {%- if item.url %}
Details
{%- elif item.gone %}
Details no longer kept
{% endif %} +{%- if item.editable %}
Edit
{% endif %} {%- endmacro %} {# Where: the hosts it happened on, each linked to its timeline, and what it happened to. #} diff --git a/steward/templates/timeline/_host_card.html b/steward/templates/timeline/_host_card.html index b2a347a..55e6e6a 100644 --- a/steward/templates/timeline/_host_card.html +++ b/steward/templates/timeline/_host_card.html @@ -5,7 +5,10 @@

History

- All of {{ host.name }}'s history + + {%- if session.user_role in ('operator', 'admin') %}Record a change · {% endif -%} + All of {{ host.name }}'s history +
{% if items %} @@ -13,7 +16,7 @@ {% for item in items %} - + {% endfor %} diff --git a/steward/templates/timeline/_rows.html b/steward/templates/timeline/_rows.html index 7783a77..6ba02a4 100644 --- a/steward/templates/timeline/_rows.html +++ b/steward/templates/timeline/_rows.html @@ -8,7 +8,7 @@ {% if item.opens_group %}{% endif %} - + diff --git a/steward/templates/timeline/change_form.html b/steward/templates/timeline/change_form.html new file mode 100644 index 0000000..78d6ab9 --- /dev/null +++ b/steward/templates/timeline/change_form.html @@ -0,0 +1,83 @@ +{# timeline/change_form.html — record a change by hand, or edit one (timeline.changes): + what was done, the area it is about, the hosts, when, and any detail. Saving + returns to `back`, the page the form was opened from. #} +{% extends "base.html" %} +{% from "_macros.html" import crumbs, page_header %} +{% set title = "Edit a recorded change" if entry else "Record a change" %} +{% block title %}{{ title }} — Timeline — Steward{% endblock %} +{% block breadcrumb %}{{ crumbs([("Timeline", "/timeline/"), (title, "")]) }}{% endblock %} +{% block content %} +{{ page_header(title, "Something Steward can't see happen: a config edited, a disk swapped, a service moved.") }} + +{% if error %}{% endif %} + +{{ csrf_field() }} + +
+

The change

+
+ + +
+
+ + + What part of the infrastructure it is about. +
+
+ + +
+
+ + +
+
+ + +
+
+ +
+

Hosts

+

The change shows in each ticked host's history. Tick none for a change not on any one host.

+ {% if hosts %} +
+ {% for h in hosts %} + + {% endfor %} +
+ {% else %} +

No hosts yet.

+ {% endif %} +
+ +
+ + Cancel +
+ + +{% if entry %} +
+
+

Delete this change

+

Recorded by {{ entry.actor }}. Deleting takes it off the timeline; the audit log keeps that it was.

+
+
{{ csrf_field() }} + + + +
+{% endif %} +{% endblock %} diff --git a/steward/templates/timeline/list.html b/steward/templates/timeline/list.html index 7360d54..94261e7 100644 --- a/steward/templates/timeline/list.html +++ b/steward/templates/timeline/list.html @@ -8,6 +8,7 @@ {% block title %}Timeline — Steward{% endblock %} {% block content %} {% call page_header("Timeline", "What happened, newest first: alerts, playbook runs and changes, on every host.") %} + {% if session.user_role in ('operator', 'admin') %}Record a change{% endif %} {% if session.user_role == 'admin' %}Areas and types{% endif %} {% endcall %} diff --git a/steward/timeline/catalog.py b/steward/timeline/catalog.py index 68f8a50..0202b84 100644 --- a/steward/timeline/catalog.py +++ b/steward/timeline/catalog.py @@ -75,6 +75,8 @@ for _t in ( "What an alert rule watches stopped reporting."), ("alert.resumed", "Readings resumed", "alert", "monitoring", True, "What an alert rule watches reported again after a silence."), + ("change.recorded", "Change recorded", "change", "service", True, + "A change someone recorded by hand, on the Timeline or a host's page, or through MCP."), ("ansible.changed", "Playbook changed something", "automation", "host_os", True, "A playbook run finished and changed at least one host."), ("ansible.unchanged", "Playbook changed nothing", "automation", "host_os", False, @@ -145,6 +147,8 @@ _AUDIT_TYPES = ( ("account.time_zone", None), ("account.email", None), ("host_agent.maintenance_plan", "steward.config"), ("host_agent.maintenance", None), ("docker.prune.", None), + # A change recorded by hand is on the timeline as itself (timeline.changes). + ("timeline.change", None), ("auth.login", "steward.signin"), ("auth.setup", "steward.config"), ("account.", "steward.access"), ("user.", "steward.access"), ("api_token.", "steward.access"), ("backup.", "steward.backup"), diff --git a/steward/timeline/changes.py b/steward/timeline/changes.py new file mode 100644 index 0000000..25d139c --- /dev/null +++ b/steward/timeline/changes.py @@ -0,0 +1,87 @@ +"""Changes a person or an agent records by hand: a config edited on a host, a +disk swapped, a service moved. Steward can't see these happen, so whoever +made the change writes it down, from the Timeline page, a host's page, or +the log_change MCP tool. + +Each is one entry of type change.recorded, sorted under the area its author +picked. Its author can edit or delete it, and so can an admin; the entry +keeps the author's user id in its detail for that. Writing, editing and +deleting are audited, but the audit rows stay off the timeline, which holds +the change itself (catalog.audit_type). +""" +from __future__ import annotations + +from datetime import datetime, timedelta, timezone + +from sqlalchemy import select + +from .config import Config +from .record import record +from steward.models.timeline import TimelineEntry + +TYPE = "change.recorded" +SUMMARY_MAX = 300 +NOTE_MAX = 4000 +SUBJECT_MAX = 255 +# A change is recorded once it is made; a clock a little ahead is allowed for. +_AHEAD = timedelta(minutes=5) + + +async def fields(db, cfg: Config, *, summary: str, area: str, hosts=(), subject: str = "", + note: str = "", at: datetime | None = None, now: datetime | None = None) -> dict: + """The entry's fields, checked: a summary, an area Settings → Timeline + lists, hosts Steward knows by name, and a time not in the future. Raises + ValueError saying what to fix.""" + now = now or datetime.now(timezone.utc) + summary = " ".join((summary or "").split()) + if not summary: + raise ValueError("Say what was done.") + if len(summary) > SUMMARY_MAX: + raise ValueError(f"Keep what was done to {SUMMARY_MAX} characters; put the rest in the detail.") + if area not in cfg.labels: + raise ValueError(f"Pick an area: {', '.join(cfg.labels.values())}.") + from steward.models.hosts import Host + wanted = sorted({h.strip() for h in hosts if h and h.strip()}) + known = set((await db.execute(select(Host.name).where(Host.name.in_(wanted)))).scalars()) if wanted else set() + unknown = [h for h in wanted if h not in known] + if unknown: + raise ValueError(f"No host is named {', '.join(unknown)}.") + note = (note or "").strip() + if len(note) > NOTE_MAX: + raise ValueError(f"Keep the detail to {NOTE_MAX} characters.") + at = at or now + if at > now + _AHEAD: + raise ValueError("A change is recorded once it is made; pick a time that has passed.") + return {"summary": summary, "area": area, "hosts": wanted, + "subject": (subject or "").strip()[:SUBJECT_MAX] or None, "note": note or None, "at": at} + + +def log(db, f: dict, *, author_id: str | None, actor: str) -> TimelineEntry: + """Add the entry `fields` checked, written by `actor` ("" or + " via MCP").""" + return record(db, type=TYPE, at=f["at"], summary=f["summary"], area=f["area"], hosts=f["hosts"], + subject=f["subject"], actor=actor, detail=_detail(f, author_id)) + + +def edit(entry: TimelineEntry, f: dict) -> None: + author_id = (entry.detail or {}).get("author_id") + entry.at, entry.summary, entry.area = f["at"], f["summary"], f["area"] + entry.hosts, entry.subject = f["hosts"], f["subject"] + entry.detail = _detail(f, author_id) + + +def _detail(f: dict, author_id: str | None) -> dict: + return {"note": f["note"], "author_id": author_id} + + +def can_edit(entry: TimelineEntry, user_id: str | None, role: str | None) -> bool: + """Whether this reader may edit or delete `entry`: a recorded change, + by its author or an admin.""" + if entry.type != TYPE or not user_id: + return False + return role == "admin" or (entry.detail or {}).get("author_id") == user_id + + +async def get(db, entry_id: str) -> TimelineEntry | None: + entry = await db.get(TimelineEntry, entry_id) + return entry if entry is not None and entry.type == TYPE else None diff --git a/steward/timeline/routes.py b/steward/timeline/routes.py index 376e077..9923ec2 100644 --- a/steward/timeline/routes.py +++ b/steward/timeline/routes.py @@ -1,17 +1,20 @@ from __future__ import annotations -from quart import Blueprint, current_app, redirect, render_template, request, session, url_for +from datetime import datetime, timezone +from urllib.parse import urlencode + +from quart import Blueprint, abort, current_app, redirect, render_template, request, session, url_for from sqlalchemy import select from steward.audit.service import mark_day_groups -from steward.auth.middleware import require_role +from steward.auth.middleware import require_role, safe_next_url from steward.core import formatting as fmt from steward.core.audit import audit_request from steward.core.paging import keyset, next_page from steward.models.hosts import Host from steward.models.timeline import TimelineEntry from steward.models.users import UserRole -from steward.timeline import catalog, config as tl_config, service as tl +from steward.timeline import catalog, changes, config as tl_config, service as tl timeline_bp = Blueprint("timeline", __name__) @@ -34,14 +37,15 @@ async def timeline(): include_hidden=f["hidden"]) stmt = keyset(stmt, TimelineEntry.at, TimelineEntry.id, request.args.get("before")) rows, after = next_page((await db.execute(stmt)).scalars().all(), lambda e: (e.at, e.id)) - items = await tl.views(db, cfg, rows) + items = _editable(await tl.views(db, cfg, rows)) hosts = [] if request.args.get("before") else \ list((await db.execute(select(Host.name).order_by(Host.name))).scalars()) last_group = mark_day_groups(items, fmt.zone(session.get("tz")), request.args.get("group", ""), at=lambda item: item["entry"].at) keep = {k: v for k, v in (("host", f["host"]), ("area", f["area"]), ("kind", f["kind"]), ("hidden", "1" if f["hidden"] else "")) if v} - ctx = {"items": items, "after": after, "last_group": last_group, "filters": f, "keep": keep, + back = "/timeline/" + ("?" + urlencode(keep) if keep else "") + ctx = {"items": items, "after": after, "last_group": last_group, "filters": f, "keep": keep, "back": back, "areas": cfg.areas, "kinds": catalog.KINDS, "hosts": hosts if not f["host"] or f["host"] in hosts else [f["host"], *hosts]} if request.args.get("before"): @@ -58,10 +62,149 @@ async def host_card(host_id: str): if host is None: return "", 404 cfg = await tl_config.load(db) - items = await tl.recent_for_host(db, cfg, host.name) + items = _editable(await tl.recent_for_host(db, cfg, host.name)) return await render_template("timeline/_host_card.html", host=host, items=items) +def _editable(items: list[dict]) -> list[dict]: + """Mark the entries this reader may edit: changes they recorded, or any as an admin.""" + for item in items: + item["editable"] = changes.can_edit(item["entry"], session.get("user_id"), session.get("user_role")) + return items + + +# ── Changes recorded by hand ──────────────────────────────────────────────── + +def _local(value: str | None) -> datetime | None: + """A datetime-local field, in the reader's own zone.""" + value = (value or "").strip() + if not value: + return None + try: + local = datetime.fromisoformat(value) + except ValueError: + raise ValueError(f"{value!r} is not a date and time.") from None + return local.replace(tzinfo=fmt.zone(session.get("tz"))).astimezone(timezone.utc) + + +def _as_local(when: datetime) -> str: + return when.astimezone(fmt.zone(session.get("tz"))).strftime("%Y-%m-%dT%H:%M") + + +async def _change_page(db, cfg, *, values: dict, entry=None, back: str, error: str | None = None, + status: int = 200): + from steward.models.hosts import Host + hosts = list((await db.execute(select(Host.name).order_by(Host.name))).scalars()) + # A host named by an entry and since removed stays ticked, so a save keeps it. + hosts += [h for h in values["hosts"] if h not in hosts] + return await render_template("timeline/change_form.html", values=values, entry=entry, back=back, + error=error, hosts=hosts, areas=cfg.areas), status + + +def _back(raw: str | None) -> str: + """Where saving returns: the page the form was opened from, else the timeline.""" + return safe_next_url(raw) or "/timeline/" + + +async def _checked(db, cfg) -> tuple[dict | None, dict, str, str | None]: + """(fields, values, back, error) from the posted form: the fields + changes.fields checked, or None and why not, with the form's own values + to show again.""" + form = await request.form + values = {"summary": form.get("summary", ""), "area": form.get("area", ""), + "hosts": form.getlist("hosts"), "subject": form.get("subject", ""), + "note": form.get("note", ""), "at": form.get("at", "")} + back = _back(form.get("back")) + try: + f = await changes.fields(db, cfg, summary=values["summary"], area=values["area"], + hosts=values["hosts"], subject=values["subject"], + note=values["note"], at=_local(values["at"])) + except ValueError as exc: + return None, values, back, str(exc) + return f, values, back, None + + +@timeline_bp.get("/timeline/changes/new") +@require_role(UserRole.operator) +async def new_change(): + """The form for recording a change; ?host= ticks a host, ?back= is where saving returns.""" + host = request.args.get("host", "").strip() + values = {"summary": "", "area": "", "hosts": [host] if host else [], "subject": "", "note": "", + "at": _as_local(datetime.now(timezone.utc))} + async with current_app.db_sessionmaker() as db: + cfg = await tl_config.load(db) + return await _change_page(db, cfg, values=values, back=_back(request.args.get("back"))) + + +@timeline_bp.post("/timeline/changes") +@require_role(UserRole.operator) +async def record_change(): + async with current_app.db_sessionmaker() as db: + async with db.begin(): + cfg = await tl_config.load(db) + f, values, back, error = await _checked(db, cfg) + if error: + return await _change_page(db, cfg, values=values, back=back, error=error, status=400) + entry = changes.log(db, f, author_id=session.get("user_id"), actor=session.get("username", "")) + await audit_request("timeline.change_recorded", entity_type="timeline_entry", entity_id=entry.id, + detail={"name": entry.summary, "hosts": entry.hosts, "area": entry.area}) + return redirect(back) + + +async def _own(db, entry_id: str): + """The recorded change `entry_id`, when this reader may edit it; else 404 or 403.""" + entry = await changes.get(db, entry_id) + if entry is None: + abort(404) + if not changes.can_edit(entry, session.get("user_id"), session.get("user_role")): + abort(403) + return entry + + +@timeline_bp.get("/timeline/changes//edit") +@require_role(UserRole.operator) +async def edit_change(entry_id: str): + async with current_app.db_sessionmaker() as db: + entry = await _own(db, entry_id) + cfg = await tl_config.load(db) + values = {"summary": entry.summary, "area": entry.area, "hosts": list(entry.hosts), + "subject": entry.subject or "", "note": (entry.detail or {}).get("note") or "", + "at": _as_local(entry.at)} + return await _change_page(db, cfg, values=values, entry=entry, + back=_back(request.args.get("back"))) + + +@timeline_bp.post("/timeline/changes/") +@require_role(UserRole.operator) +async def save_change(entry_id: str): + async with current_app.db_sessionmaker() as db: + async with db.begin(): + entry = await _own(db, entry_id) + cfg = await tl_config.load(db) + f, values, back, error = await _checked(db, cfg) + if error: + return await _change_page(db, cfg, values=values, entry=entry, back=back, + error=error, status=400) + changes.edit(entry, f) + await audit_request("timeline.change_edited", entity_type="timeline_entry", entity_id=entry_id, + detail={"name": f["summary"]}) + return redirect(back) + + +@timeline_bp.post("/timeline/changes//delete") +@require_role(UserRole.operator) +async def delete_change(entry_id: str): + form = await request.form + async with current_app.db_sessionmaker() as db: + async with db.begin(): + entry = await _own(db, entry_id) + summary = entry.summary + await db.delete(entry) + await audit_request("timeline.change_deleted", entity_type="timeline_entry", entity_id=entry_id, + detail={"name": summary}) + return redirect(_back(form.get("back"))) + + @timeline_bp.get("/settings/timeline/") @require_role(UserRole.admin) async def settings_page(): diff --git a/tests/core/test_timeline.py b/tests/core/test_timeline.py index 752d3a8..5ad5647 100644 --- a/tests/core/test_timeline.py +++ b/tests/core/test_timeline.py @@ -268,3 +268,84 @@ async def test_settings_lists_each_type_under_its_kind(app): assert 'name="type_shown_ansible.check" aria-label="Show Playbook dry run">' in page assert '' in page assert 'aria-current="page">Timeline' in page + + +# ── Changes recorded by hand ──────────────────────────────────────────────── + +def test_a_recorded_change_is_its_own_type_and_its_audit_rows_stay_off(): + t = catalog.TYPES["change.recorded"] + assert (t.kind, t.shown) == ("change", True) + for action in ("timeline.change_recorded", "timeline.change_edited", "timeline.change_deleted"): + assert catalog.audit_type(action) is None + + +async def test_a_change_is_checked_before_it_is_recorded(): + from datetime import timedelta + from steward.timeline import changes + cfg = _cfg() + f = await changes.fields(None, cfg, summary=" Swapped the PSU ", area="hardware", subject=" psu ", + note="", now=NOW) + assert f == {"summary": "Swapped the PSU", "area": "hardware", "hosts": [], "subject": "psu", + "note": None, "at": NOW} + for kw, said in ((dict(summary=" ", area="hardware"), "Say what was done"), + (dict(summary="x" * 301, area="hardware"), "300 characters"), + (dict(summary="x", area="plumbing"), "Pick an area: Service, Host OS"), + (dict(summary="x", area="hardware", note="n" * 4001), "4000 characters"), + (dict(summary="x", area="hardware", at=NOW + timedelta(minutes=10)), "has passed")): + with pytest.raises(ValueError, match=said): + await changes.fields(None, cfg, now=NOW, **kw) + # A clock a little ahead is allowed for. + assert (await changes.fields(None, cfg, summary="x", area="hardware", at=NOW + timedelta(minutes=2), + now=NOW))["at"] == NOW + timedelta(minutes=2) + + +def test_only_its_author_or_an_admin_edits_a_recorded_change(): + from steward.timeline import changes + mine = _entry(type_="change.recorded", detail={"author_id": "u1"}) + assert changes.can_edit(mine, "u1", "operator") + assert not changes.can_edit(mine, "u2", "operator") + assert changes.can_edit(mine, "u2", "admin") + assert not changes.can_edit(mine, None, "admin") # a share link + assert not changes.can_edit(_entry(), "u1", "admin") # only recorded changes + + +async def test_a_recorded_change_the_reader_may_edit_has_an_edit_link(app): + mine = _item(_entry(id="c1", type_="change.recorded", area="storage", hosts=["web-1"], subject=None, + summary="Moved Postgres to the SSD", detail={"note": "old disk\nis spare", + "author_id": "u1"}), + editable=True) + theirs = _item(_entry(id="c2", type_="change.recorded", summary="Theirs", detail={"author_id": "u2"}), + editable=False, opens_group=False) + async with app.test_request_context("/timeline/"): + rows = await render_template("timeline/_rows.html", items=[mine, theirs], last_group="Today", + after=None, keep={"host": "web-1"}, back="/timeline/?host=web-1") + assert 'Edit' in rows + assert "/timeline/changes/c2/edit" not in rows + assert "Change · " in rows and "old disk\nis spare" in rows + + +async def test_an_operator_records_a_change_from_a_host_s_card(app): + host = NS(id="h1", name="web-1") + async with app.test_request_context("/timeline/host/h1"): + from quart import session + session["user_role"] = "operator" + card = await render_template("timeline/_host_card.html", host=host, items=[]) + session["user_role"] = "viewer" + viewer = await render_template("timeline/_host_card.html", host=host, items=[]) + assert 'href="/timeline/changes/new?host=web-1&back=/hosts/h1">Record a change' in card + assert "Record a change" not in viewer + + +async def test_the_change_form_names_its_fields_and_offers_delete_only_when_editing(app): + values = {"summary": "", "area": "", "hosts": ["web-1"], "subject": "", "note": "", "at": "2026-10-24T08:00"} + entry = NS(id="c1", actor="bryan", summary="Swapped the PSU") + async with app.test_request_context("/timeline/changes/new"): + new = await render_template("timeline/change_form.html", values=values, entry=None, back="/hosts/h1", + error=None, hosts=["db-1", "web-1"], areas=AREAS) + edit = await render_template("timeline/change_form.html", values=values, entry=entry, + back="/timeline/", error="Say what was done.", hosts=["web-1"], areas=AREAS) + assert 'action="/timeline/changes"' in new and 'value="2026-10-24T08:00"' in new + assert 'value="web-1" checked' in new and 'value="db-1">' in new + assert 'Cancel' in new and 'id="chg-delete"' not in new + assert 'action="/timeline/changes/c1"' in edit and 'action="/timeline/changes/c1/delete"' in edit + assert "Say what was done." in edit diff --git a/tests/integration/test_timeline_changes.py b/tests/integration/test_timeline_changes.py new file mode 100644 index 0000000..146540d --- /dev/null +++ b/tests/integration/test_timeline_changes.py @@ -0,0 +1,165 @@ +"""Integration: a change recorded by hand (#6350), from the page and through +MCP log_change. The form reads its time in the reader's own zone; only the +author or an admin edits or deletes one; each step is audited and the audit +rows stay off the timeline. Requires STEWARD_DATABASE_URL. +""" +from __future__ import annotations + +import asyncio +import os +import time +import uuid +from datetime import datetime, timedelta, timezone + +import pytest + +pytestmark = pytest.mark.integration + + +@pytest.fixture +def app(): + if not os.environ.get("STEWARD_DATABASE_URL"): + pytest.skip("needs Postgres") + from steward.app import create_app + return create_app(testing=False) + + +async def _user(app, name: str, role: str, tz: str | None = None): + from steward.models.users import User, UserRole + uid = str(uuid.uuid4()) + async with app.db_sessionmaker() as s: + async with s.begin(): + s.add(User(id=uid, username=name, email=f"{name}@test", password_hash="x", + role=UserRole(role), is_active=True)) + client = app.test_client() + async with client.session_transaction() as sess: + sess.update(user_id=uid, user_role=role, username=name, _csrf="test-token", login_at=time.time(), + **({"tz": tz} if tz else {})) + return client, uid + + +async def _host(app, name: str): + from steward.models.hosts import Host + async with app.db_sessionmaker() as s: + async with s.begin(): + host = Host(name=name, address="10.8.8.8") + s.add(host) + return host + + +async def _changes(app, host: str): + from sqlalchemy import select + from steward.models.timeline import TimelineEntry + async with app.db_sessionmaker() as s: + return (await s.execute(select(TimelineEntry).where(TimelineEntry.hosts.contains([host])) + .order_by(TimelineEntry.at))).scalars().all() + + +async def _audited(app, entry_id: str) -> list[str]: + from sqlalchemy import select + from steward.models.audit import AuditEvent + async with app.db_sessionmaker() as s: + return list((await s.execute(select(AuditEvent.action).where(AuditEvent.entity_id == entry_id) + .order_by(AuditEvent.timestamp))).scalars()) + + +def test_a_change_recorded_on_the_page_is_edited_by_its_author_and_deleted_by_an_admin(app): + async def _go(): + tag = uuid.uuid4().hex[:8] + name = f"tl-ch-{tag}" + host = await _host(app, name) + author, author_id = await _user(app, f"ch-a-{tag}", "operator", tz="America/New_York") + other, _ = await _user(app, f"ch-o-{tag}", "operator") + admin, _ = await _user(app, f"ch-ad-{tag}", "admin") + viewer, _ = await _user(app, f"ch-v-{tag}", "viewer") + form = {"csrf_token": "test-token", "summary": "Moved Postgres data to the new SSD", "area": "storage", + "hosts": name, "subject": "postgres", "note": "Old disk kept as a spare.", + "at": "2026-10-20T08:00", "back": f"/hosts/{host.id}"} + + # The form opens from the host's card with the host ticked; a viewer can't open it. + page = await (await author.get(f"/timeline/changes/new?host={name}&back=/hosts/{host.id}")).get_data(as_text=True) + assert f'value="{name}" checked' in page and f'value="/hosts/{host.id}"' in page + assert (await viewer.get("/timeline/changes/new")).status_code == 403 + + # Refused, with the form kept: an unknown host writes nothing. + bad = await author.post("/timeline/changes", form={**form, "hosts": "no-such-host"}) + assert bad.status_code == 400 + text = await bad.get_data(as_text=True) + assert "No host is named no-such-host" in text and "Moved Postgres data" in text + + res = await author.post("/timeline/changes", form=form) + assert res.status_code == 302 and res.headers["Location"] == f"/hosts/{host.id}" + [entry] = await _changes(app, name) + assert (entry.type, entry.area, entry.subject, entry.actor) == ( + "change.recorded", "storage", "postgres", f"ch-a-{tag}") + assert entry.at == datetime(2026, 10, 20, 12, 0, tzinfo=timezone.utc) # 08:00 in New York + assert entry.detail == {"note": "Old disk kept as a spare.", "author_id": author_id} + + # Its author sees Edit on the timeline; another operator does not, and can't edit it. + mine = await (await author.get(f"/timeline/?host={name}")).get_data(as_text=True) + assert f"/timeline/changes/{entry.id}/edit" in mine + theirs = await (await other.get(f"/timeline/?host={name}")).get_data(as_text=True) + assert f"/timeline/changes/{entry.id}/edit" not in theirs + assert (await other.get(f"/timeline/changes/{entry.id}/edit")).status_code == 403 + assert (await other.post(f"/timeline/changes/{entry.id}", form=form)).status_code == 403 + + edit = await (await author.get(f"/timeline/changes/{entry.id}/edit")).get_data(as_text=True) + assert 'value="2026-10-20T08:00"' in edit # shown in their zone + await author.post(f"/timeline/changes/{entry.id}", + form={**form, "summary": "Moved Postgres data to the SSD", "area": "hardware"}) + [entry] = await _changes(app, name) + assert (entry.summary, entry.area) == ("Moved Postgres data to the SSD", "hardware") + assert entry.detail["author_id"] == author_id # still theirs + + res = await admin.post(f"/timeline/changes/{entry.id}/delete", + form={"csrf_token": "test-token", "back": "/timeline/"}) + assert res.status_code == 302 and await _changes(app, name) == [] + assert await _audited(app, entry.id) == [ + "timeline.change_recorded", "timeline.change_edited", "timeline.change_deleted"] + # The audit rows are not on the timeline: the change was, as itself. + from sqlalchemy import select + from steward.models.timeline import TimelineEntry + async with app.db_sessionmaker() as s: + leaked = (await s.execute(select(TimelineEntry).where( + TimelineEntry.summary.contains("Moved Postgres data"), + TimelineEntry.actor.in_([f"ch-a-{tag}", f"ch-ad-{tag}"])))).scalars().all() + assert leaked == [] + + asyncio.run(_go()) + + +def test_log_change_records_a_change_through_mcp(app): + from steward.core.api_tokens import Caller + from steward.mcp import _context + from steward.mcp.tools.timeline import get_timeline, log_change + from steward.models.users import UserRole + + async def _go(): + tag = uuid.uuid4().hex[:8] + name = f"tl-mc-{tag}" + await _host(app, name) + _, uid = await _user(app, f"mc-{tag}", "operator") + _context._app_ctx.set(app) + _context._caller_ctx.set(Caller(user_id=uid, username=f"mc-{tag}", role=UserRole.operator, token_id=7)) + + when = (datetime.now(timezone.utc) - timedelta(days=1)).replace(microsecond=0) + out = await log_change("Raised the swap to 8 GiB", "Host OS", hosts=name, subject="/swapfile", + detail="It ran out during the backup.", + at=when.isoformat().replace("+00:00", "Z")) + assert out["type"] == "change.recorded" and out["kind"] == "change" and out["area"] == "host_os" + assert out["by"] == f"mc-{tag} via MCP" and out["hosts"] == [name] + assert out["at"] == when.isoformat().replace("+00:00", "Z") and out["note"] == "It ran out during the backup." + [entry] = await _changes(app, name) + assert entry.detail["author_id"] == uid # the token's user can edit it + assert await _audited(app, entry.id) == ["timeline.change_recorded"] + seen = await get_timeline(host=name, range="30d") + assert [e["entry_id"] for e in seen["entries"]] == [entry.id] + + with pytest.raises(ValueError, match="list_hosts"): + await log_change("x", "storage", hosts=["nope"]) + with pytest.raises(ValueError, match="area 'plumbing'"): + await log_change("x", "plumbing") + with pytest.raises(ValueError, match="ISO-8601"): + await log_change("x", "storage", at="yesterday") + + asyncio.run(_go()) diff --git a/tests/mcp/test_anomaly_review.py b/tests/mcp/test_anomaly_review.py index 21d20d8..c08fbf6 100644 --- a/tests/mcp/test_anomaly_review.py +++ b/tests/mcp/test_anomaly_review.py @@ -152,3 +152,11 @@ def test_a_quick_restart_is_shown_though_it_never_broke_the_minute(): [ev] = _events.gaps_in([NOW, slow], NOW, NOW + timedelta(hours=1), [(slow, 300.0)]) assert "restarted" in ev["what"] assert _events.gaps_in(ticks, NOW + timedelta(minutes=2), NOW + timedelta(hours=1), [(restart, 35.0)]) == [] + + +def test_the_timeline_tools_are_registered_with_their_roles(app): + mcp = app.mcp_server + for name, role, read_only in (("get_timeline", UserRole.viewer, True), + ("log_change", UserRole.operator, False)): + assert mcp.tool_roles[name] is role, name + assert mcp._tool_manager.get_tool(name).annotations.read_only_hint is read_only, name
{{ item.entry.at|when }}{{ entry_what(item) }}{{ entry_what(item, '/hosts/' ~ host.id) }} {{ item.entry.actor }}
{{ item.group }}
{{ e.at|when }}{{ entry_what(item) }}{{ entry_what(item, back) }} {{ entry_where(e) }} {{ e.actor }} {{ item.area_label }}