Compare commits
49
Commits
db62985966
...
main
@@ -28,10 +28,10 @@ services:
|
||||
environment:
|
||||
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-grimoire}:${POSTGRES_PASSWORD:-grimoire}@db:5432/${POSTGRES_DB:-grimoire}"
|
||||
LOG_LEVEL: "${LOG_LEVEL:-INFO}"
|
||||
# Behind an HTTPS reverse proxy:
|
||||
# Port 5000 is published straight to clients, so no proxy is trusted.
|
||||
# Behind a reverse proxy, set 1 (one per proxy) and BASE_URL:
|
||||
TRUSTED_PROXY_HOPS: "0"
|
||||
# BASE_URL: "https://grimoire.example.com"
|
||||
# SECURE_COOKIES: "true"
|
||||
# TRUST_PROXY_HEADERS: "true"
|
||||
volumes:
|
||||
- app_data:/data
|
||||
depends_on:
|
||||
@@ -75,8 +75,7 @@ Everything is optional. Set these in the environment or in a `.env` file beside
|
||||
| Variable | When | What it does |
|
||||
|----------|------|--------------|
|
||||
| `BASE_URL` | Behind a proxy | The public URL, used in email links and SSO redirects |
|
||||
| `SECURE_COOKIES` | Over HTTPS | `true` when Grimoire is served over HTTPS |
|
||||
| `TRUST_PROXY_HEADERS` | Behind a proxy | `true` to read the client address from the proxy's headers |
|
||||
| `TRUSTED_PROXY_HOPS` | Rarely | The starting count of reverse proxies in front of Grimoire (default `1`; the compose file above sets `0` because it publishes the port directly). Check and change it in **Settings → Config → Reverse proxy**, which shows what each count reads for your own request; behind Cloudflare it is usually `2` |
|
||||
| `POSTGRES_PASSWORD` | Before the first start | The database password (defaults to `grimoire`; the database is reachable only inside the compose network). Postgres takes it once, when the volume is created |
|
||||
| `SECRET_KEY` | Rarely | A key you manage yourself, instead of the one Grimoire keeps. It wins over the kept key; remove it to go back |
|
||||
|
||||
@@ -97,7 +96,7 @@ docker compose up -d
|
||||
|
||||
Database migrations run when the app starts, and the key stays in its volume. Claude Code updates the plugin from its marketplace; run `/reload-plugins` after an update.
|
||||
|
||||
**Settings → Data → Scheduled backups** writes a daily backup of the database to a folder or an S3-compatible bucket, and `python -m grimoire.restore` brings a fresh instance back from one. See [Backups](docs/backups.md).
|
||||
**Settings → Data → Scheduled backups** writes a daily backup of the database to a folder or an S3-compatible bucket, and Settings → Data → Restore from Backup rolls the instance back to one, or brings a new instance back from it. See [Backups](docs/backups.md).
|
||||
|
||||
## Features
|
||||
|
||||
@@ -132,7 +131,7 @@ The full list is in [Features](docs/features.md).
|
||||
|
||||
| What you see | Why | Fix |
|
||||
|--------------|-----|-----|
|
||||
| `SECRET_KEY is set to the public placeholder` at startup | `SECURE_COOKIES=true` while `SECRET_KEY` is `dev-secret-change-me` | Remove `SECRET_KEY`, or set a real one |
|
||||
| `SECRET_KEY is set to the public placeholder` at startup | `SECRET_KEY` is `dev-secret-change-me` or `change-me-in-production` | Remove `SECRET_KEY`, or set a real one |
|
||||
| Settings asks for forge tokens or the backup secret again | The instance's key changed: the `app_data` volume was replaced, or `SECRET_KEY` was set or removed | Enter them again, or restore the previous key |
|
||||
| `Can't locate revision identified by '…'` at startup | An older image is running against a database a newer one has already migrated | Run the newer image; migrations only move forward |
|
||||
| The plugin's tools are missing in Claude Code | The plugin isn't installed, or Claude Code hasn't reloaded it | `/plugin` to install or update it, then `/reload-plugins` |
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
"""drop machine-written uses edges that cross projects
|
||||
|
||||
Revision ID: 0125
|
||||
Revises: 0124
|
||||
Create Date: 2026-10-26
|
||||
|
||||
A `uses` edge with basis `reference` is the proposer's by-name reading: the
|
||||
shape's body names a canon's symbol. A body never calls a function defined in
|
||||
another project's repository, so such an edge across projects is a common
|
||||
word matched, not a call. The proposer no longer writes them, and this removes
|
||||
the ones it wrote before. Edges a person or an audit judged keep their own
|
||||
basis and are left alone. Rows within one project are re-read on the next
|
||||
coverage refresh, which replaces their reference edges under the same rule.
|
||||
|
||||
Downgrade restores nothing: the removed edges were never facts.
|
||||
"""
|
||||
from alembic import op
|
||||
|
||||
revision = "0125"
|
||||
down_revision = "0124"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(
|
||||
"DELETE FROM code_shape_uses u USING code_shapes s, notes n "
|
||||
"WHERE u.shape_id = s.id AND u.snippet_id = n.id "
|
||||
"AND u.basis = 'reference' "
|
||||
"AND n.project_id IS NOT NULL AND n.project_id <> s.project_id"
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
pass
|
||||
@@ -0,0 +1,34 @@
|
||||
"""drop write-path uses edges that cross projects
|
||||
|
||||
Revision ID: 0126
|
||||
Revises: 0125
|
||||
Create Date: 2026-10-26
|
||||
|
||||
The write path records a `uses` edge with basis `hook` when a session pulled a
|
||||
snippet and then wrote code naming its symbol. That is by-name evidence, like
|
||||
the proposer's `reference` edges 0125 removed, and a body never calls a
|
||||
function defined in another project's repository. The write path now applies
|
||||
the same by-name test, so it writes no such edge; this removes the ones it
|
||||
wrote before. Judged edges keep their own basis and are left alone.
|
||||
|
||||
Downgrade restores nothing: the removed edges were never facts.
|
||||
"""
|
||||
from alembic import op
|
||||
|
||||
revision = "0126"
|
||||
down_revision = "0125"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(
|
||||
"DELETE FROM code_shape_uses u USING code_shapes s, notes n "
|
||||
"WHERE u.shape_id = s.id AND u.snippet_id = n.id "
|
||||
"AND u.basis = 'hook' "
|
||||
"AND n.project_id IS NOT NULL AND n.project_id <> s.project_id"
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
pass
|
||||
@@ -0,0 +1,39 @@
|
||||
"""instance identity — the id lineage a backup carries
|
||||
|
||||
Revision ID: 0127
|
||||
Revises: 0126
|
||||
Create Date: 2026-11-05
|
||||
|
||||
One row naming this database's id lineage. Every full backup carries it, a
|
||||
restore into a fresh instance adopts it (the restore keeps the original ids),
|
||||
and rolling a live instance back requires the backup's to match, so a backup's
|
||||
ids never land on unrelated rows. Made here with a random value, so every
|
||||
instance has one before its first account exists.
|
||||
"""
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0127"
|
||||
down_revision = "0126"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"instance_identity",
|
||||
sa.Column("id", sa.Integer(), primary_key=True),
|
||||
sa.Column("instance_id", sa.Text(), nullable=False),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), nullable=False,
|
||||
server_default=sa.func.now(),
|
||||
),
|
||||
sa.CheckConstraint("id = 1", name="ck_instance_identity_single_row"),
|
||||
)
|
||||
op.execute(
|
||||
"INSERT INTO instance_identity (id, instance_id) VALUES (1, gen_random_uuid()::text)"
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_table("instance_identity")
|
||||
@@ -8,8 +8,9 @@ services:
|
||||
# a Docker secret and sets SECRET_KEY_FILE to its path).
|
||||
SECRET_KEY: "${SECRET_KEY}"
|
||||
LOG_LEVEL: "${LOG_LEVEL:-INFO}"
|
||||
TRUST_PROXY_HEADERS: "true"
|
||||
SECURE_COOKIES: "true"
|
||||
# Proxies in front of the app (Traefik = 1). The client address and
|
||||
# HTTPS are read from that far in from the right of X-Forwarded-*.
|
||||
TRUSTED_PROXY_HOPS: "1"
|
||||
volumes:
|
||||
- app_data:/data
|
||||
networks:
|
||||
|
||||
+3
-3
@@ -13,10 +13,10 @@ services:
|
||||
environment:
|
||||
DATABASE_URL: "postgresql+asyncpg://${POSTGRES_USER:-grimoire}:${POSTGRES_PASSWORD:-grimoire}@db:5432/${POSTGRES_DB:-grimoire}"
|
||||
LOG_LEVEL: "${LOG_LEVEL:-INFO}"
|
||||
# Behind an HTTPS reverse proxy:
|
||||
# Port 5000 is published straight to clients, so no proxy is trusted.
|
||||
# Behind a reverse proxy, set 1 (one per proxy) and BASE_URL:
|
||||
TRUSTED_PROXY_HOPS: "0"
|
||||
# BASE_URL: "https://grimoire.example.com"
|
||||
# SECURE_COOKIES: "true"
|
||||
# TRUST_PROXY_HEADERS: "true"
|
||||
volumes:
|
||||
- app_data:/data
|
||||
depends_on:
|
||||
|
||||
@@ -251,6 +251,7 @@ These serve the plugin's hooks.
|
||||
| POST | `/api/plugin/rule-opened` | Records that a session opened a rule it was shown |
|
||||
| GET | `/api/plugin/shape-check` | The end-of-turn question about what this turn wrote |
|
||||
| GET | `/api/plugin/claim-session` · `/release-session` | Bind a session to its task claims / release them at session end |
|
||||
| GET | `/api/plugin/write-scope` | Which projects a pending act would change (a directory, or an MCP call's `tool` and `input`), and whether a session in its own project may do it. Filing an issue elsewhere is allowed; every other cross-project write is answered `allowed: false` with the reason the operator is asked |
|
||||
| GET | `/api/plugin/processes` | Stored Processes for skill-stub sync |
|
||||
| GET | `/api/plugin/prior-art` | Write-path hint for the plugin hooks (params: `path`, `code`, `repo`, `shapes`, `exclude_ids`, `exclude_sync_ids`, `exclude_derive`); returns `context`, `note_ids`, `sync_note_ids`, `stamped`, `divergence`, `derive`, `derive_keys` |
|
||||
| GET / POST | `/api/projects/<id>/coverage`, `…/coverage/refresh` | Shape-ledger accounting (`pattern_coverage` line, counts, `derive_groups` — css groups carry `consumers`, `derive_new`, `unused_css`, `divergence`, `recheck`) |
|
||||
@@ -272,12 +273,15 @@ These serve the plugin's hooks.
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/backup` | Export backup (format v4; `?scope=user` for own data) |
|
||||
| POST | `/api/admin/restore` | Restore from a backup |
|
||||
| GET | `/api/admin/restore/sources` | The backups at the backup destination, newest first |
|
||||
| POST | `/api/admin/restore/preview` | What restoring a backup (`?source=<name>` or an uploaded file) would change; 409 with the reason when it can't apply here |
|
||||
| POST | `/api/admin/restore?confirm=rollback` | Put the whole instance back to that backup, after saving the current state |
|
||||
| GET / DELETE | `/api/admin/users` · `/api/admin/users/:id` | List / delete users |
|
||||
| GET / PUT | `/api/admin/registration` | Get / toggle registration |
|
||||
| GET / PUT | `/api/admin/smtp` · POST `/api/admin/smtp/test` | SMTP config + test email |
|
||||
| GET | `/api/admin/logs` · `/api/admin/logs/stats` | Log entries + category counts |
|
||||
| GET / PUT | `/api/admin/base-url` | Get / set the public base URL |
|
||||
| GET / PUT | `/api/admin/proxy` | Get / set how many reverse proxies stand in front of Grimoire (`{"hops": n}`, or `null` for the environment's count), with what each count would read for this request |
|
||||
| GET / PUT | `/api/admin/db-maintenance` (+ `/health`, POST `/run`) | VACUUM schedule, health, manual run |
|
||||
| POST / GET / DELETE | `/api/admin/invitations` (+ `/:id`) | Create / list / revoke invite links |
|
||||
| GET / PUT | `/api/admin/backups` | Scheduled backups: destination, schedule and recent runs |
|
||||
|
||||
@@ -69,10 +69,10 @@ Every database wait on startup has a deadline. Migrations wait for the database
|
||||
|
||||
## Authentication
|
||||
|
||||
- **Sessions**: `HttpOnly`, `SameSite=Lax` cookies, `Secure` when `SECURE_COOKIES=true`. Changing a password bumps the user's session version and signs out their other sessions.
|
||||
- **Sessions**: `HttpOnly`, `SameSite=Lax` cookies, `Secure` on a request the trusted proxy reports as HTTPS, which also gets HSTS (`grimoire/proxy.py`). Changing a password bumps the user's session version and signs out their other sessions.
|
||||
- **API keys**: `Authorization: Bearer fmcp_...`, stored hashed. A `read` key may only call read endpoints and read MCP tools.
|
||||
- **OIDC** single sign-on with PKCE, optionally replacing local logins ([SSO / OAuth](sso-oauth.md)).
|
||||
- **Rate limits** on register, login and forgot-password, per client IP (`TRUST_PROXY_HEADERS` reads the IP from the proxy's headers).
|
||||
- **Rate limits** on register, login and forgot-password, per client IP, read from `X-Forwarded-For` by `TRUSTED_PROXY_HOPS`.
|
||||
|
||||
Credentials Grimoire stores for outside services (forge tokens, the backup destination's S3 secret) are sealed under a key derived from `SECRET_KEY`, which the instance makes on first start and keeps in `/data/secret_key` unless one is set.
|
||||
|
||||
@@ -87,7 +87,7 @@ Credentials Grimoire stores for outside services (forge tokens, the backup desti
|
||||
│ ├── routes/ REST blueprints
|
||||
│ ├── mcp/ MCP server and its tool modules
|
||||
│ ├── services/ shared logic
|
||||
│ └── restore.py `python -m grimoire.restore` for a fresh instance
|
||||
│ └── restore.py `python -m grimoire.restore`: the restore, from the command line
|
||||
├── frontend/src/ views, components, stores, composables, api client
|
||||
├── plugin/ Claude Code hooks, skills and commands
|
||||
├── scripts/ CI checks and the plugin version minter
|
||||
|
||||
+52
-21
@@ -88,9 +88,12 @@ You need four things:
|
||||
- **Access key.**
|
||||
- **Secret key.**
|
||||
|
||||
Grimoire uploads and lists; it never deletes. A key that may **only add
|
||||
objects** is therefore enough, and it is the safer key to hand out: a stolen
|
||||
one cannot wipe the history.
|
||||
Grimoire uploads and lists, and reads a backup back when you restore from the
|
||||
bucket; it never deletes. A key that may **add and read objects but not
|
||||
delete them** is therefore enough, and it is the safer key to hand out: a
|
||||
stolen one cannot wipe the history. A key that may only add objects still
|
||||
takes backups; restore then works by uploading a file or from the command
|
||||
line.
|
||||
|
||||
The secret is stored encrypted, under a key derived from `SECRET_KEY`. It is
|
||||
never shown again, and never included in a backup. If an instance's
|
||||
@@ -122,8 +125,10 @@ keeps service backups as "backup targets". Either destination works with it.
|
||||
1. On Archives' **Backups** page, make a target for Grimoire, named `grimoire`
|
||||
for example.
|
||||
2. Choose how Grimoire's files arrive:
|
||||
- **S3**: turn on the S3 way in for the target and make an **append-only**
|
||||
S3 key. In Grimoire, choose **An S3 bucket**:
|
||||
- **S3**: turn on the S3 way in for the target and make an S3 key that
|
||||
may add and read objects but not delete them (an **append-only** key
|
||||
takes backups too, but then Grimoire can't restore from the bucket). In
|
||||
Grimoire, choose **An S3 bucket**:
|
||||
- address: Archives' backup address (`http://<archives>:8081`);
|
||||
- bucket: the target's name;
|
||||
- access key and secret key: the key you just made.
|
||||
@@ -139,26 +144,52 @@ keeps service backups as "backup targets". Either destination works with it.
|
||||
|
||||
## Restoring
|
||||
|
||||
### Into a fresh instance
|
||||
A restore puts the **whole instance** back to the moment a backup was taken,
|
||||
for every user:
|
||||
|
||||
Start Grimoire on an empty database, copy the backup file into the container,
|
||||
and run:
|
||||
- records edited since go back to how they were;
|
||||
- records deleted since come back;
|
||||
- anything made since is removed.
|
||||
|
||||
It matches records by their ids, so a live instance only restores a backup **of
|
||||
its own**. Each instance has an id, and every backup records the one it came
|
||||
from. A backup from another instance, or one made before backups recorded it,
|
||||
only goes into a new, empty instance, which takes it as it is.
|
||||
|
||||
Before changing anything, the restore saves the current state as one more
|
||||
backup. It goes to the backup destination when scheduled backups are on, and
|
||||
otherwise to `/data/pre-restore`. The result says which file it is, so a
|
||||
restore can itself be undone.
|
||||
|
||||
What a backup doesn't carry stays as it is wherever its owner survives the
|
||||
restore: API keys, forge connections, sharing, logs, and the destination's S3
|
||||
secret. The trash isn't in backups, so anything in the trash when the backup
|
||||
was taken is gone afterwards. Everyone is signed out, and search is rebuilt.
|
||||
|
||||
### From the web
|
||||
|
||||
**Settings → Data → Restore from Backup** lists the backups at the backup
|
||||
destination. You can pick one of those, or upload a `.tar` or the JSON
|
||||
**Full Backup** download (uploads are held to the server's request-size
|
||||
limit). The restore then shows what would change: when the backup was taken,
|
||||
and how many accounts, projects, plans, notes and rules there are now and will
|
||||
be afterwards. Type `roll back` to run it. Then sign in again.
|
||||
|
||||
To read backups back from an S3 bucket, the key needs read access as well as
|
||||
write. Grimoire never deletes from the bucket.
|
||||
|
||||
### From the command line
|
||||
|
||||
For a file with no size limit, or a new instance with no backup destination
|
||||
yet, copy the file into the container and run:
|
||||
|
||||
```sh
|
||||
docker exec -it grimoire python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar
|
||||
```
|
||||
|
||||
- The command refuses a database that already has an account. A restore
|
||||
re-creates everything with new ids, so on a live instance it would add a
|
||||
second copy instead of replacing anything.
|
||||
- Nothing limits the file's size.
|
||||
- It also reads the JSON from the admin **Full Backup** download.
|
||||
It prints what would change and stops. Add `--yes` to restore. It reads the
|
||||
JSON download too. Afterwards, **restart Grimoire** so the search indexes are
|
||||
rebuilt.
|
||||
|
||||
Afterwards, **restart Grimoire** so the search indexes are rebuilt. Then issue
|
||||
new API keys, add forge connections again, and set up any sharing.
|
||||
|
||||
### From the web
|
||||
|
||||
**Settings → Data → Restore from Backup** takes the `.tar` as well as the
|
||||
JSON download. The upload is held to the server's request-size limit. For a
|
||||
bigger file, use the command above.
|
||||
After restoring into a new instance, issue new API keys, add forge connections
|
||||
again, and set up any sharing: none of them travel in a backup.
|
||||
|
||||
+14
-8
@@ -14,8 +14,7 @@ Any variable marked *secret* can also be read from a file: set `<NAME>_FILE` to
|
||||
| `SECRET_KEY` *(secret)* | kept in `DATA_DIR` | Signs sessions and seals stored credentials. Unset, the instance makes its own on first start and keeps it in `DATA_DIR/secret_key`. Set it only to manage the key yourself; it wins over the kept one (see below) |
|
||||
| `DATA_DIR` | `/data` | Where the instance keeps its own `SECRET_KEY`. Mount a volume here |
|
||||
| `BASE_URL` | `http://localhost:5000` | Public URL, used in email links and OIDC redirects |
|
||||
| `SECURE_COOKIES` | `false` | `true` when Grimoire is served over HTTPS |
|
||||
| `TRUST_PROXY_HEADERS` | `false` | `true` behind a reverse proxy, to read the client IP from `X-Forwarded-For` / `X-Real-IP` |
|
||||
| `TRUSTED_PROXY_HOPS` | `1` | The starting count of reverse proxies in front of Grimoire. **Settings → Config → Reverse proxy** is where the count is checked and set, and a count saved there wins over this. The client address and the scheme are read that many entries in from the right of `X-Forwarded-For` and `X-Forwarded-Proto`; `0` ignores both headers. See [Behind a reverse proxy](#behind-a-reverse-proxy) |
|
||||
| `LOG_LEVEL` | `INFO` | `DEBUG`, `INFO`, `WARNING` or `ERROR` |
|
||||
| `LOG_RETENTION_DAYS` | `90` | Days of app logs to keep |
|
||||
| `FASTEMBED_CACHE_DIR` | `/data/fastembed-cache` | Where the embedding model is stored after its first download |
|
||||
@@ -77,7 +76,7 @@ docker compose -f docker-compose.dev.yml up --build
|
||||
|
||||
### Docker Swarm
|
||||
|
||||
[`docker-compose.prod.yml`](../docker-compose.prod.yml) is a stack file: the app and database on a private overlay network, with `SECURE_COOKIES` and `TRUST_PROXY_HEADERS` on. A Swarm can move the app to another node, away from a node-local `/data` volume, so the stack sets its own key: provide `DB_PASSWORD` and `SECRET_KEY` in the environment (or mount a Docker secret and set `SECRET_KEY_FILE`):
|
||||
[`docker-compose.prod.yml`](../docker-compose.prod.yml) is a stack file: the app and database on a private overlay network, trusting one proxy (`TRUSTED_PROXY_HOPS: "1"`). A Swarm can move the app to another node, away from a node-local `/data` volume, so the stack sets its own key: provide `DB_PASSWORD` and `SECRET_KEY` in the environment (or mount a Docker secret and set `SECRET_KEY_FILE`):
|
||||
|
||||
```bash
|
||||
DB_PASSWORD=... SECRET_KEY=... docker stack deploy -c docker-compose.prod.yml grimoire
|
||||
@@ -96,13 +95,16 @@ Back up the database with **Settings → Data → Scheduled backups** ([Backups]
|
||||
|
||||
## Behind a reverse proxy
|
||||
|
||||
Grimoire doesn't terminate TLS. Put it behind Nginx, Traefik or Caddy, forward to port 5000, and keep port 5000 off the internet. Set `SECURE_COOKIES=true`, `TRUST_PROXY_HEADERS=true` and `BASE_URL`.
|
||||
Grimoire doesn't terminate TLS. Put it behind Nginx, Traefik or Caddy, forward to port 5000, and keep port 5000 off the internet. Set `BASE_URL`, then set the proxy count to the number of proxies a request passes through: one proxy is `1`, the default; Cloudflare's proxied DNS in front of your own proxy is `2`.
|
||||
|
||||
**Settings → Config → Reverse proxy** sets the count and checks it against your own request. It shows the address Grimoire reads you as, and what each count from 0 to 3 would read. Pick the count that shows your own address. When a request comes through Cloudflare, the section names the count that reads the address Cloudflare reports, with a button that sets it. `TRUSTED_PROXY_HOPS` in the environment is only the starting value: a count saved in Settings wins, and **Use the environment's** goes back to it.
|
||||
|
||||
A count that is too low makes every visitor read as the proxy, so they share one sign-in rate limit and a few bad sign-ins block everyone. The "too many attempts" message says so and points here. The block on sign-ins lifts after a minute; an admin who is already signed in can fix the count at once.
|
||||
|
||||
```nginx
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:5000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
@@ -112,12 +114,16 @@ location / {
|
||||
}
|
||||
```
|
||||
|
||||
With `TRUST_PROXY_HEADERS=true`, make sure the proxy replaces any `X-Forwarded-For` the client sends.
|
||||
Each proxy appends its entry to `X-Forwarded-For` and `X-Forwarded-Proto`, so Grimoire reads the entry that many in from the right: the one your own proxy wrote. Whatever a client puts to the left of it is ignored. A header with fewer entries than that (a LAN road with one proxy beside a Cloudflare road with two) is read at its leftmost entry, the one the first proxy wrote.
|
||||
|
||||
When that entry says `https`, the session cookie is marked `Secure` and the response carries `Strict-Transport-Security: max-age=31536000`. Over plain HTTP neither is sent, and Grimoire never redirects to HTTPS, so a LAN install keeps working.
|
||||
|
||||
A hop count higher than the real number of proxies lets a client choose its own address, so count them. With `0`, the headers are ignored and the socket peer is the client: use it when clients reach port 5000 directly.
|
||||
|
||||
## Security checklist
|
||||
|
||||
- **Keep the key.** Forge tokens and the backup destination's S3 secret are sealed under a key derived from `SECRET_KEY`, so a copy of the database alone can't read them. Unset, the instance makes a random key and keeps it in `/data/secret_key`, readable by its owner only; keep that volume. If the key changes (say, a restore onto an instance with a different key), Settings marks those credentials and asks for them again; nothing else breaks. The app warns at startup when `SECRET_KEY` is a public placeholder or `/data` is not a mounted volume, and refuses to start with `dev-secret-change-me` when `SECURE_COOKIES=true`.
|
||||
- **`SECURE_COOKIES=true`** whenever Grimoire is served over HTTPS.
|
||||
- **Keep the key.** Forge tokens and the backup destination's S3 secret are sealed under a key derived from `SECRET_KEY`, so a copy of the database alone can't read them. Unset, the instance makes a random key and keeps it in `/data/secret_key`, readable by its owner only; keep that volume. If the key changes (say, a restore onto an instance with a different key), Settings marks those credentials and asks for them again; nothing else breaks. The app warns at startup when `SECRET_KEY` is a public placeholder or `/data` is not a mounted volume, and refuses to start with a placeholder key (`dev-secret-change-me`, `change-me-in-production`).
|
||||
- **The proxy count** (Settings → Config → Reverse proxy) matches the number of proxies in front of Grimoire, so rate limits and the audit log see the real client and the cookie is `Secure` over HTTPS. The section shows whether it does.
|
||||
- **Registration** is open only for the first account. Invite people, or open it briefly from Settings.
|
||||
- **Signing everyone else out**: changing a password ends that user's other sessions; Settings → Account has a button for it too.
|
||||
- **API keys** are per user, shown once, and can be read-only. Revoke any you no longer use under Settings → MCP Access.
|
||||
|
||||
+4
-3
@@ -66,6 +66,7 @@ Saved procedures, such as a drift audit or a DRY pass. The plugin syncs each one
|
||||
The whole store is reachable through the MCP endpoint at `/mcp`, authenticated with an API key. The Claude Code plugin in this repo adds:
|
||||
|
||||
- hooks for session start, each prompt, before and after tool calls, the end of a turn, compaction and session end, all failing open if Grimoire is unreachable;
|
||||
- a project fence: a session belongs to the project its starting directory is bound to. Toward any other project it can read anything and file an issue. Editing that project's files, committing or pushing in its repo, or changing its records first asks you, in every permission mode;
|
||||
- skills for the working practices: using Grimoire, writing plans, systematic debugging, verification, brainstorming, reporting back, reusing code, shape accounting and family canon;
|
||||
- your stored processes as skills.
|
||||
|
||||
@@ -85,8 +86,8 @@ Share a project, note or task with users or groups, as viewer, editor or admin.
|
||||
## Export and backup
|
||||
|
||||
- **Personal export**: your notes and tasks as a Markdown ZIP (with YAML frontmatter) or JSON.
|
||||
- **Admin backup and restore**, with ids remapped so a backup restores onto another instance.
|
||||
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. `python -m grimoire.restore` brings a fresh instance back from one. See [Backups](backups.md).
|
||||
- **Admin backup and restore**: a restore rolls the whole instance back to one of its own backups, after saving the current state, or brings a new instance back from any backup.
|
||||
- **Scheduled backups**: a daily, dated, uncompressed tar to a folder or an S3-compatible bucket, laid out so a deduplicating tool such as restic stores only what changed. Restoring is in Settings, or `python -m grimoire.restore`. See [Backups](backups.md).
|
||||
|
||||
## Sign-in
|
||||
|
||||
@@ -102,7 +103,7 @@ Email and password, plus optional OIDC single sign-on (Authentik, Keycloak, ...)
|
||||
| Integrations | SearXNG, git forge connections |
|
||||
| Data | Export, backup and restore, scheduled backups (admin) |
|
||||
| MCP Access | API keys and the command to connect Claude |
|
||||
| Config (admin) | What's running, application URL, plugin marketplace, database maintenance, SMTP, forge webhook |
|
||||
| Config (admin) | What's running, reverse proxy count (checked against your own request), application URL, plugin marketplace, database maintenance, SMTP, forge webhook |
|
||||
| Shared areas (admin) | The canonical areas systems map to |
|
||||
| Platforms (admin) | The platform catalogue family canon is grouped by |
|
||||
| Users (admin) | Registration, invites, users |
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { apiGet, apiPost, apiPut } from "@/api/client";
|
||||
import { apiGet, apiPost, apiPostRaw, apiPut } from "@/api/client";
|
||||
|
||||
/**
|
||||
* Scheduled backups — `/api/admin/backups`. One dated,
|
||||
@@ -71,3 +71,72 @@ export function checkBackupDestination(
|
||||
export function runBackupNow(): Promise<{ status: string }> {
|
||||
return apiPost<{ status: string }>("/api/admin/backups/run", {});
|
||||
}
|
||||
|
||||
|
||||
// ── Restore: put the whole instance back to a full backup ─────────────────
|
||||
|
||||
export interface StoredBackup {
|
||||
name: string;
|
||||
size_bytes: number;
|
||||
}
|
||||
|
||||
export interface RestoreCount {
|
||||
kind: string;
|
||||
now: number;
|
||||
after: number;
|
||||
}
|
||||
|
||||
export interface RestorePreview {
|
||||
/** "fresh": an empty instance takes the backup. "rollback": this instance
|
||||
* goes back to its own earlier backup. */
|
||||
mode: "fresh" | "rollback";
|
||||
exported_at: string | null;
|
||||
counts: RestoreCount[];
|
||||
}
|
||||
|
||||
export interface RestoreResult extends RestorePreview {
|
||||
status: "ok";
|
||||
restored: Record<string, number>;
|
||||
safety_backup: { file: string; size_bytes: number; where: string; note?: string };
|
||||
}
|
||||
|
||||
/** A backup at the destination, by name, or a file picked from this computer. */
|
||||
export type RestoreFrom = { source: string } | { file: File };
|
||||
|
||||
// Reading, checking and writing the whole store is slow by design; bounded,
|
||||
// but in minutes.
|
||||
const RESTORE_TIMEOUT_MS = 10 * 60 * 1000;
|
||||
|
||||
export function listRestoreSources(): Promise<{ backups: StoredBackup[]; error?: string }> {
|
||||
return apiGet("/api/admin/restore/sources", { timeoutMs: 60_000 });
|
||||
}
|
||||
|
||||
async function restoreCall<T>(path: string, from: RestoreFrom, query: Record<string, string> = {}): Promise<T> {
|
||||
const params = new URLSearchParams(query);
|
||||
if ("source" in from) {
|
||||
params.set("source", from.source);
|
||||
return apiPostRaw<T>(`${path}?${params}`, null, null, { timeoutMs: RESTORE_TIMEOUT_MS });
|
||||
}
|
||||
const qs = params.toString() ? `?${params}` : "";
|
||||
const isTar = from.file.name.toLowerCase().endsWith(".tar");
|
||||
// The JSON download is parsed first, so a damaged file fails here, before
|
||||
// the upload.
|
||||
const body = isTar ? from.file : JSON.stringify(JSON.parse(await from.file.text()));
|
||||
return apiPostRaw<T>(`${path}${qs}`, body, isTar ? "application/x-tar" : "application/json",
|
||||
{ timeoutMs: RESTORE_TIMEOUT_MS });
|
||||
}
|
||||
|
||||
export function previewRestore(from: RestoreFrom): Promise<RestorePreview> {
|
||||
return restoreCall<RestorePreview>("/api/admin/restore/preview", from);
|
||||
}
|
||||
|
||||
export function restoreBackup(from: RestoreFrom): Promise<RestoreResult> {
|
||||
return restoreCall<RestoreResult>("/api/admin/restore", from, { confirm: "rollback" });
|
||||
}
|
||||
|
||||
/** When a backup named `grimoire-20261015T020000Z.tar` was taken. */
|
||||
export function backupTakenAt(name: string): Date | null {
|
||||
const m = /(\d{4})(\d{2})(\d{2})T(\d{2})(\d{2})(\d{2})Z/.exec(name);
|
||||
if (!m) return null;
|
||||
return new Date(Date.UTC(+m[1], +m[2] - 1, +m[3], +m[4], +m[5], +m[6]));
|
||||
}
|
||||
|
||||
@@ -156,6 +156,15 @@ export function apiPost<T>(path: string, body: unknown, opts?: RequestOpts): Pro
|
||||
return request<T>(path, { method: "POST", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
|
||||
}
|
||||
|
||||
/** A POST whose body is sent as it is (a file, or text already serialised),
|
||||
* under the same deadline and error handling as every other call. */
|
||||
export function apiPostRaw<T>(
|
||||
path: string, body: BodyInit | null, contentType: string | null, opts?: RequestOpts,
|
||||
): Promise<T> {
|
||||
const headers = contentType ? { "Content-Type": contentType } : undefined;
|
||||
return request<T>(path, { method: "POST", headers, body }, opts);
|
||||
}
|
||||
|
||||
export function apiPut<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
|
||||
return request<T>(path, { method: "PUT", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
|
||||
}
|
||||
|
||||
@@ -44,6 +44,14 @@ export interface RemovedDefault {
|
||||
created_at: string | null;
|
||||
}
|
||||
|
||||
export interface MomentRuleUsage {
|
||||
rule_id: number;
|
||||
/** Null for a rule deleted since it was delivered. */
|
||||
title: string | null;
|
||||
delivered: number;
|
||||
opened: boolean;
|
||||
}
|
||||
|
||||
export interface MomentUsage {
|
||||
/** How many times a mounted rule arrived at this moment. */
|
||||
delivered: number;
|
||||
@@ -53,6 +61,8 @@ export interface MomentUsage {
|
||||
* the door, not the line that prompted it. */
|
||||
opened: number;
|
||||
last_delivered_at: string | null;
|
||||
/** Each rule delivered here, most delivered first. */
|
||||
by_rule: MomentRuleUsage[];
|
||||
}
|
||||
|
||||
export interface MomentsPayload {
|
||||
|
||||
@@ -1,466 +0,0 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, computed, onMounted } from "vue";
|
||||
import { apiGet, pinNoteVersion, unpinNoteVersion } from "@/api/client";
|
||||
import DiffView from "@/components/DiffView.vue";
|
||||
import { computeDiff, type DiffLine } from "@/utils/diff";
|
||||
import { fmtStamp } from "@/utils/dateFormat";
|
||||
|
||||
interface NoteVersion {
|
||||
id: number;
|
||||
note_id: number;
|
||||
title: string;
|
||||
tags: string[];
|
||||
body?: string;
|
||||
pin_kind: "auto" | "manual" | null;
|
||||
pin_label: string | null;
|
||||
created_at: string;
|
||||
}
|
||||
|
||||
const props = defineProps<{
|
||||
noteId: number;
|
||||
currentBody: string;
|
||||
}>();
|
||||
|
||||
const emit = defineEmits<{
|
||||
(e: 'restore', body: string, tags: string[]): void;
|
||||
(e: 'close'): void;
|
||||
}>();
|
||||
|
||||
const versions = ref<NoteVersion[]>([]);
|
||||
const selectedVersion = ref<NoteVersion | null>(null);
|
||||
const loading = ref(false);
|
||||
const loadingDetail = ref(false);
|
||||
|
||||
const diff = computed<DiffLine[]>(() => {
|
||||
if (!selectedVersion.value?.body) return [];
|
||||
|
||||
return computeDiff(props.currentBody, selectedVersion.value.body);
|
||||
});
|
||||
|
||||
async function loadVersions() {
|
||||
loading.value = true;
|
||||
try {
|
||||
const data = await apiGet<{ versions: NoteVersion[] }>(
|
||||
`/api/notes/${props.noteId}/versions`
|
||||
);
|
||||
versions.value = data.versions;
|
||||
if (versions.value.length > 0) {
|
||||
await selectVersionItem(versions.value[0]);
|
||||
}
|
||||
} catch {
|
||||
// silent
|
||||
} finally {
|
||||
loading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function selectVersionItem(v: NoteVersion) {
|
||||
if (v.body !== undefined) {
|
||||
selectedVersion.value = v;
|
||||
return;
|
||||
}
|
||||
loadingDetail.value = true;
|
||||
try {
|
||||
const full = await apiGet<NoteVersion>(
|
||||
`/api/notes/${props.noteId}/versions/${v.id}`
|
||||
);
|
||||
// Cache body back onto list item
|
||||
const listItem = versions.value.find(x => x.id === v.id);
|
||||
if (listItem) listItem.body = full.body;
|
||||
selectedVersion.value = full;
|
||||
} catch {
|
||||
// silent
|
||||
} finally {
|
||||
loadingDetail.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
function restore() {
|
||||
if (!selectedVersion.value?.body) return;
|
||||
emit('restore', selectedVersion.value.body, selectedVersion.value.tags ?? []);
|
||||
}
|
||||
|
||||
// ── Pin / unpin / edit-label state ──────────────────────────────────────────
|
||||
|
||||
const editingLabel = ref(false);
|
||||
const draftLabel = ref("");
|
||||
const pinSaving = ref(false);
|
||||
|
||||
function startEditLabel() {
|
||||
if (!selectedVersion.value) return;
|
||||
draftLabel.value = selectedVersion.value.pin_label ?? "";
|
||||
editingLabel.value = true;
|
||||
}
|
||||
|
||||
function cancelEditLabel() {
|
||||
editingLabel.value = false;
|
||||
draftLabel.value = "";
|
||||
}
|
||||
|
||||
async function savePin() {
|
||||
if (!selectedVersion.value || pinSaving.value) return;
|
||||
pinSaving.value = true;
|
||||
try {
|
||||
const updated = await pinNoteVersion(
|
||||
props.noteId, selectedVersion.value.id, draftLabel.value || null,
|
||||
);
|
||||
// Reflect server state in the local list + selected version.
|
||||
const idx = versions.value.findIndex(v => v.id === updated.id);
|
||||
if (idx >= 0) {
|
||||
versions.value[idx].pin_kind = updated.pin_kind;
|
||||
versions.value[idx].pin_label = updated.pin_label;
|
||||
}
|
||||
if (selectedVersion.value) {
|
||||
selectedVersion.value.pin_kind = updated.pin_kind;
|
||||
selectedVersion.value.pin_label = updated.pin_label;
|
||||
}
|
||||
editingLabel.value = false;
|
||||
} catch {
|
||||
// silent — leaving the form open lets the user retry
|
||||
} finally {
|
||||
pinSaving.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function unpinSelected() {
|
||||
if (!selectedVersion.value || pinSaving.value) return;
|
||||
pinSaving.value = true;
|
||||
try {
|
||||
await unpinNoteVersion(props.noteId, selectedVersion.value.id);
|
||||
const idx = versions.value.findIndex(v => v.id === selectedVersion.value!.id);
|
||||
if (idx >= 0) {
|
||||
versions.value[idx].pin_kind = null;
|
||||
versions.value[idx].pin_label = null;
|
||||
}
|
||||
if (selectedVersion.value) {
|
||||
selectedVersion.value.pin_kind = null;
|
||||
selectedVersion.value.pin_label = null;
|
||||
}
|
||||
} catch {
|
||||
// silent
|
||||
} finally {
|
||||
pinSaving.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(loadVersions);
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<teleport to="body">
|
||||
<div class="modal-overlay" @click.self="emit('close')">
|
||||
<div class="modal-card history-modal">
|
||||
<div class="history-header">
|
||||
<h3 class="history-title">Version History</h3>
|
||||
<button class="history-close" aria-label="Close version history" @click="emit('close')">×</button>
|
||||
</div>
|
||||
|
||||
<div class="history-body">
|
||||
<!-- Left: version list -->
|
||||
<div class="history-list">
|
||||
<div v-if="loading" class="history-empty">Loading...</div>
|
||||
<div v-else-if="!versions.length" class="history-empty">No versions yet.</div>
|
||||
<div
|
||||
v-for="v in versions"
|
||||
:key="v.id"
|
||||
:class="['history-item', { selected: selectedVersion?.id === v.id }]"
|
||||
@click="selectVersionItem(v)"
|
||||
>
|
||||
<div class="history-item-title">
|
||||
<span
|
||||
v-if="v.pin_kind === 'manual'"
|
||||
class="pin-badge pin-badge-manual"
|
||||
:title="v.pin_label || 'Pinned'"
|
||||
aria-label="manually pinned"
|
||||
>●</span>
|
||||
<span
|
||||
v-else-if="v.pin_kind === 'auto'"
|
||||
class="pin-badge pin-badge-auto"
|
||||
:title="v.pin_label || 'Auto-pinned (stable)'"
|
||||
aria-label="auto-pinned"
|
||||
>◐</span>
|
||||
{{ v.title || 'Untitled' }}
|
||||
</div>
|
||||
<div
|
||||
v-if="v.pin_kind === 'manual' && v.pin_label"
|
||||
class="history-item-label"
|
||||
>{{ v.pin_label }}</div>
|
||||
<div class="history-item-date">{{ fmtStamp(v.created_at) }}</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Right: diff -->
|
||||
<div class="history-diff">
|
||||
<div v-if="loadingDetail" class="history-empty">Loading version...</div>
|
||||
<div v-else-if="!selectedVersion" class="history-empty">Select a version to compare.</div>
|
||||
<template v-else>
|
||||
<div class="version-pin-controls">
|
||||
<!-- Label editor -->
|
||||
<div v-if="editingLabel" class="pin-label-form">
|
||||
<input
|
||||
v-model="draftLabel"
|
||||
maxlength="500"
|
||||
type="text"
|
||||
class="pin-label-input"
|
||||
placeholder="Optional label (e.g. 'post-network-refresh runbook')"
|
||||
/>
|
||||
<button class="btn-pin-save" :disabled="pinSaving" @click="savePin">
|
||||
{{ pinSaving ? "Saving…" : "Save" }}
|
||||
</button>
|
||||
<button class="btn-pin-cancel" @click="cancelEditLabel">Cancel</button>
|
||||
</div>
|
||||
|
||||
<!-- Default: kind-aware action row -->
|
||||
<div v-else class="pin-actions">
|
||||
<span v-if="selectedVersion.pin_kind === 'manual'" class="pin-state">
|
||||
<span class="pin-badge pin-badge-manual">●</span>
|
||||
Manually pinned{{ selectedVersion.pin_label ? `: ${selectedVersion.pin_label}` : '' }}
|
||||
</span>
|
||||
<span v-else-if="selectedVersion.pin_kind === 'auto'" class="pin-state">
|
||||
<span class="pin-badge pin-badge-auto">◐</span>
|
||||
Auto-pinned{{ selectedVersion.pin_label ? `: ${selectedVersion.pin_label}` : '' }}
|
||||
</span>
|
||||
|
||||
<button
|
||||
v-if="selectedVersion.pin_kind === null"
|
||||
class="btn-pin"
|
||||
:disabled="pinSaving"
|
||||
@click="startEditLabel"
|
||||
>Pin version</button>
|
||||
<button
|
||||
v-else-if="selectedVersion.pin_kind === 'manual'"
|
||||
class="btn-pin-edit"
|
||||
:disabled="pinSaving"
|
||||
@click="startEditLabel"
|
||||
>Edit label</button>
|
||||
<button
|
||||
v-else
|
||||
class="btn-pin"
|
||||
:disabled="pinSaving"
|
||||
@click="startEditLabel"
|
||||
>Pin permanently</button>
|
||||
<button
|
||||
v-if="selectedVersion.pin_kind === 'manual'"
|
||||
class="btn-unpin"
|
||||
:disabled="pinSaving"
|
||||
@click="unpinSelected"
|
||||
>Unpin</button>
|
||||
</div>
|
||||
</div>
|
||||
<DiffView :diff="diff" />
|
||||
</template>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="history-footer">
|
||||
<button
|
||||
class="btn-primary"
|
||||
:disabled="!selectedVersion?.body"
|
||||
@click="restore"
|
||||
>Restore this version</button>
|
||||
<button class="modal-btn" @click="emit('close')">Close</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</teleport>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.history-modal {
|
||||
max-width: 900px;
|
||||
width: 95%;
|
||||
height: 80vh;
|
||||
max-height: 700px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.history-header {
|
||||
flex-shrink: 0;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: 0.9rem 1.25rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
}
|
||||
|
||||
.history-title {
|
||||
margin: 0;
|
||||
font-size: 1rem;
|
||||
}
|
||||
|
||||
.history-close {
|
||||
background: none;
|
||||
border: none;
|
||||
font-size: 1.25rem;
|
||||
cursor: pointer;
|
||||
color: var(--fs-text-tertiary);
|
||||
line-height: 1;
|
||||
padding: 0.1rem 0.3rem;
|
||||
}
|
||||
.history-close:hover { color: var(--fs-text-primary); }
|
||||
|
||||
.history-body {
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
display: flex;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.history-list {
|
||||
width: 220px;
|
||||
flex-shrink: 0;
|
||||
border-right: 1px solid var(--fs-border-color);
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.history-item {
|
||||
padding: 0.6rem 0.9rem;
|
||||
cursor: pointer;
|
||||
border-left: 3px solid transparent;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
}
|
||||
.history-item:hover { background: var(--fs-surface-raised); }
|
||||
.history-item.selected {
|
||||
border-left-color: var(--fs-accent);
|
||||
background: var(--fs-surface-raised);
|
||||
}
|
||||
|
||||
.history-item-title {
|
||||
font-size: 0.85rem;
|
||||
font-weight: 500;
|
||||
color: var(--fs-text-primary);
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.history-item-date {
|
||||
font-size: 0.75rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
margin-top: 0.15rem;
|
||||
}
|
||||
|
||||
.history-diff {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
padding: 0.5rem;
|
||||
}
|
||||
|
||||
.history-empty {
|
||||
padding: 1rem;
|
||||
font-size: 0.85rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
|
||||
.history-footer {
|
||||
flex-shrink: 0;
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
justify-content: flex-end;
|
||||
padding: 0.75rem 1.25rem;
|
||||
border-top: 1px solid var(--fs-border-color);
|
||||
}
|
||||
|
||||
|
||||
/* ── Pin badges + label rendering ───────────────────────────────────────── */
|
||||
.pin-badge {
|
||||
display: inline-block;
|
||||
width: 0.7rem;
|
||||
text-align: center;
|
||||
margin-right: 0.35rem;
|
||||
font-size: 0.85em;
|
||||
line-height: 1;
|
||||
}
|
||||
.pin-badge-manual { color: var(--fs-accent); }
|
||||
.pin-badge-auto { color: var(--fs-text-tertiary); }
|
||||
|
||||
.history-item-label {
|
||||
font-size: 0.72rem;
|
||||
color: var(--fs-accent);
|
||||
font-style: italic;
|
||||
margin-top: 0.15rem;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
/* ── Pin controls above the diff ────────────────────────────────────────── */
|
||||
.version-pin-controls {
|
||||
padding: 0.4rem 0.5rem 0.5rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
font-size: 0.82rem;
|
||||
}
|
||||
.pin-actions {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.pin-state {
|
||||
font-style: italic;
|
||||
color: var(--fs-text-tertiary);
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.btn-pin, .btn-pin-edit, .btn-unpin {
|
||||
padding: 0.25rem 0.7rem;
|
||||
font-size: 0.78rem;
|
||||
background: transparent;
|
||||
color: inherit;
|
||||
border: 1px solid var(--fs-border-color);
|
||||
border-radius: 999px;
|
||||
cursor: pointer;
|
||||
}
|
||||
.btn-pin:hover:not(:disabled), .btn-pin-edit:hover:not(:disabled) {
|
||||
background: rgba(99, 102, 241, 0.12);
|
||||
border-color: var(--fs-accent);
|
||||
}
|
||||
.btn-unpin:hover:not(:disabled) {
|
||||
background: rgba(239, 68, 68, 0.10);
|
||||
border-color: rgba(239, 68, 68, 0.5);
|
||||
}
|
||||
.pin-label-form {
|
||||
display: flex;
|
||||
gap: 0.4rem;
|
||||
align-items: center;
|
||||
}
|
||||
.pin-label-input {
|
||||
flex: 1;
|
||||
padding: 0.3rem 0.5rem;
|
||||
font-size: 0.85rem;
|
||||
background: var(--fs-surface-page);
|
||||
border: 1px solid var(--fs-border-color);
|
||||
border-radius: var(--fs-radius-sm);
|
||||
color: inherit;
|
||||
}
|
||||
.pin-label-input:focus {
|
||||
outline: none;
|
||||
border-color: var(--fs-accent);
|
||||
}
|
||||
.btn-pin-save, .btn-pin-cancel {
|
||||
padding: 0.3rem 0.7rem;
|
||||
font-size: 0.78rem;
|
||||
background: transparent;
|
||||
color: inherit;
|
||||
border: 1px solid var(--fs-border-color);
|
||||
border-radius: var(--fs-radius-sm);
|
||||
cursor: pointer;
|
||||
}
|
||||
.btn-pin-save:hover:not(:disabled) {
|
||||
background: rgba(99, 102, 241, 0.12);
|
||||
border-color: var(--fs-accent);
|
||||
}
|
||||
.btn-pin-save:disabled, .btn-pin-cancel:disabled,
|
||||
.btn-pin:disabled, .btn-pin-edit:disabled, .btn-unpin:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: progress;
|
||||
}
|
||||
</style>
|
||||
@@ -1,258 +0,0 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, computed } from "vue";
|
||||
import { renderMarkdown } from "@/utils/markdown";
|
||||
import type { DiffLine } from "@/composables/useAssist";
|
||||
|
||||
const props = defineProps<{
|
||||
phase: "streaming" | "review";
|
||||
label: string;
|
||||
streamingText: string;
|
||||
diff: DiffLine[];
|
||||
proposedText: string;
|
||||
}>();
|
||||
|
||||
const emit = defineEmits<{
|
||||
accept: [];
|
||||
reject: [];
|
||||
cancel: [];
|
||||
}>();
|
||||
|
||||
const showFullText = ref(false);
|
||||
|
||||
const renderedStreaming = computed(() => renderMarkdown(props.streamingText));
|
||||
const renderedProposed = computed(() => renderMarkdown(props.proposedText));
|
||||
|
||||
const markers: Record<DiffLine["type"], string> = {
|
||||
equal: " ",
|
||||
delete: "−",
|
||||
insert: "+",
|
||||
};
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<!-- ── Streaming ───────────────────────────────────────────────── -->
|
||||
<div v-if="phase === 'streaming'" class="iap iap-streaming">
|
||||
<div class="iap-header">
|
||||
<span class="iap-pulse"></span>
|
||||
<span class="iap-label">{{ label }}</span>
|
||||
<button class="iap-btn-cancel" @click="emit('cancel')">Cancel</button>
|
||||
</div>
|
||||
<div class="iap-stream-preview prose" v-html="renderedStreaming"></div>
|
||||
<div v-if="!streamingText" class="iap-waiting">Waiting for model…</div>
|
||||
</div>
|
||||
|
||||
<!-- ── Review ─────────────────────────────────────────────────── -->
|
||||
<div v-else-if="phase === 'review'" class="iap iap-review">
|
||||
<div class="iap-header">
|
||||
<span class="iap-review-title">Proposed changes</span>
|
||||
<button class="iap-btn-toggle" @click="showFullText = !showFullText">
|
||||
{{ showFullText ? "Show diff" : "Show full text" }}
|
||||
</button>
|
||||
<div class="iap-actions">
|
||||
<button class="iap-btn-accept" @click="emit('accept')">✓ Accept</button>
|
||||
<button class="iap-btn-reject" @click="emit('reject')">✗ Reject</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Diff view -->
|
||||
<div v-if="!showFullText" class="iap-diff">
|
||||
<div
|
||||
v-for="(line, i) in diff"
|
||||
:key="i"
|
||||
:class="['iap-diff-line', `iap-diff-${line.type}`]"
|
||||
>
|
||||
<span class="iap-diff-marker">{{ markers[line.type] }}</span>
|
||||
<span class="iap-diff-text">{{ line.text }}</span>
|
||||
</div>
|
||||
<div v-if="!diff.length" class="iap-diff-empty">No changes.</div>
|
||||
</div>
|
||||
|
||||
<!-- Full text view -->
|
||||
<div v-else class="iap-full-text prose" v-html="renderedProposed"></div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.iap {
|
||||
border-radius: var(--fs-radius-sm);
|
||||
margin-bottom: 0.75rem;
|
||||
overflow: hidden;
|
||||
border: 1px solid var(--fs-border-color);
|
||||
background: var(--fs-surface-page);
|
||||
}
|
||||
|
||||
/* ── Header ── */
|
||||
.iap-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
padding: 0.45rem 0.75rem;
|
||||
font-size: 0.85rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
background: var(--fs-surface-raised);
|
||||
}
|
||||
|
||||
/* ── Streaming ── */
|
||||
.iap-streaming {
|
||||
border-color: var(--fs-accent);
|
||||
}
|
||||
.iap-streaming .iap-header {
|
||||
background: color-mix(in srgb, var(--fs-accent) 8%, var(--fs-surface-raised));
|
||||
}
|
||||
|
||||
.iap-pulse {
|
||||
display: inline-block;
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
border-radius: 50%;
|
||||
background: var(--fs-accent);
|
||||
flex-shrink: 0;
|
||||
animation: iap-pulse 1.2s ease-in-out infinite;
|
||||
}
|
||||
@keyframes iap-pulse {
|
||||
0%, 100% { opacity: 1; transform: scale(1); }
|
||||
50% { opacity: 0.4; transform: scale(0.8); }
|
||||
}
|
||||
|
||||
.iap-label {
|
||||
flex: 1;
|
||||
font-weight: 500;
|
||||
color: var(--fs-text-primary);
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.iap-btn-cancel {
|
||||
background: none;
|
||||
border: 1px solid var(--fs-border-color);
|
||||
color: var(--fs-text-secondary);
|
||||
border-radius: var(--fs-radius-sm);
|
||||
padding: 0.15rem 0.5rem;
|
||||
cursor: pointer;
|
||||
font-size: 0.8rem;
|
||||
font-family: inherit;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.iap-btn-cancel:hover {
|
||||
border-color: var(--fs-error);
|
||||
color: var(--fs-error);
|
||||
}
|
||||
|
||||
.iap-stream-preview {
|
||||
padding: 0.75rem;
|
||||
font-size: 0.9rem;
|
||||
min-height: 2.5rem;
|
||||
max-height: 320px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.iap-waiting {
|
||||
padding: 0.75rem;
|
||||
font-size: 0.85rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
|
||||
/* ── Review ── */
|
||||
.iap-review-title {
|
||||
flex: 1;
|
||||
font-weight: 600;
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
|
||||
.iap-btn-toggle {
|
||||
background: none;
|
||||
border: 1px solid var(--fs-border-color);
|
||||
color: var(--fs-text-secondary);
|
||||
border-radius: var(--fs-radius-sm);
|
||||
padding: 0.15rem 0.5rem;
|
||||
cursor: pointer;
|
||||
font-size: 0.78rem;
|
||||
font-family: inherit;
|
||||
}
|
||||
.iap-btn-toggle:hover {
|
||||
border-color: var(--fs-accent);
|
||||
color: var(--fs-accent);
|
||||
}
|
||||
|
||||
.iap-actions {
|
||||
display: flex;
|
||||
gap: 0.35rem;
|
||||
}
|
||||
|
||||
.iap-btn-accept,
|
||||
.iap-btn-reject {
|
||||
border: none;
|
||||
border-radius: var(--fs-radius-sm);
|
||||
padding: 0.2rem 0.65rem;
|
||||
cursor: pointer;
|
||||
font-size: 0.8rem;
|
||||
font-family: inherit;
|
||||
font-weight: var(--fs-weight-medium);
|
||||
}
|
||||
.iap-btn-accept {
|
||||
background: var(--fs-success);
|
||||
color: var(--fs-text-on-action);
|
||||
}
|
||||
.iap-btn-accept:hover { opacity: 0.85; }
|
||||
|
||||
.iap-btn-reject {
|
||||
background: var(--fs-surface-raised);
|
||||
color: var(--fs-text-secondary);
|
||||
border: 1px solid var(--fs-border-color);
|
||||
}
|
||||
.iap-btn-reject:hover {
|
||||
border-color: var(--fs-error);
|
||||
color: var(--fs-error);
|
||||
}
|
||||
|
||||
/* ── Diff ── */
|
||||
.iap-diff {
|
||||
font-family: ui-monospace, "Cascadia Code", "Fira Mono", monospace;
|
||||
font-size: 0.82rem;
|
||||
line-height: 1.6;
|
||||
max-height: 440px;
|
||||
overflow-y: auto;
|
||||
padding: 0.4rem 0;
|
||||
}
|
||||
|
||||
.iap-diff-line {
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
padding: 0 0.75rem;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.iap-diff-equal { color: var(--fs-text-tertiary); }
|
||||
.iap-diff-delete {
|
||||
background: color-mix(in srgb, var(--fs-error) 10%, transparent);
|
||||
color: var(--fs-error-fg);
|
||||
}
|
||||
.iap-diff-insert {
|
||||
background: color-mix(in srgb, var(--fs-success) 10%, transparent);
|
||||
color: var(--fs-success-fg);
|
||||
}
|
||||
|
||||
.iap-diff-marker {
|
||||
width: 1ch;
|
||||
flex-shrink: 0;
|
||||
font-weight: 700;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.iap-diff-empty {
|
||||
padding: 0.75rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
font-size: 0.85rem;
|
||||
font-family: inherit;
|
||||
}
|
||||
|
||||
/* ── Full text ── */
|
||||
.iap-full-text {
|
||||
padding: 0.75rem;
|
||||
font-size: 0.9rem;
|
||||
max-height: 440px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
</style>
|
||||
@@ -0,0 +1,37 @@
|
||||
<script setup lang="ts">
|
||||
import type { MomentRuleUsage } from "@/api/moments";
|
||||
|
||||
/**
|
||||
* Which rules a moment delivered, and whether an agent went on to read each.
|
||||
* The moment's own counts cannot say this once two rules are mounted there.
|
||||
* Closed by default: the counts in the row above are what most visits want.
|
||||
*/
|
||||
defineProps<{ rules: MomentRuleUsage[] }>();
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<details v-if="rules.length" class="moment-rules">
|
||||
<summary>Rules delivered here</summary>
|
||||
<ul class="delivered-list">
|
||||
<li v-for="r in rules" :key="r.rule_id" class="delivered-row">
|
||||
<span class="delivered-title">{{ r.title ?? `Rule ${r.rule_id} (deleted)` }}</span>
|
||||
<span class="delivered-count">delivered {{ r.delivered }}</span>
|
||||
<span :class="r.opened ? 'delivered-opened' : 'delivered-unopened'">
|
||||
{{ r.opened ? "opened" : "not opened" }}
|
||||
</span>
|
||||
</li>
|
||||
</ul>
|
||||
</details>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.moment-rules { margin-top: var(--fs-space-2); font-size: var(--fs-size-tiny); }
|
||||
.moment-rules summary { cursor: pointer; color: var(--fs-text-tertiary); }
|
||||
.delivered-list { list-style: none; margin: var(--fs-space-1) 0 0; padding: 0; display: flex; flex-direction: column; gap: var(--fs-space-1); }
|
||||
.delivered-row { display: flex; flex-wrap: wrap; align-items: baseline; gap: var(--fs-space-2); font-variant-numeric: tabular-nums; }
|
||||
.delivered-title { color: var(--fs-text-primary); font-size: var(--fs-size-body-sm); }
|
||||
.delivered-count { color: var(--fs-text-secondary); }
|
||||
.delivered-opened { color: var(--fs-text-secondary); }
|
||||
/* Not read is a fact about this window, not a fault: quiet, like "nothing mounted". */
|
||||
.delivered-unopened { color: var(--fs-text-tertiary); font-style: italic; }
|
||||
</style>
|
||||
@@ -2,6 +2,7 @@
|
||||
import { computed, onMounted, ref } from "vue";
|
||||
import { useMomentsStore } from "@/stores/moments";
|
||||
import MomentProposals from "@/components/MomentProposals.vue";
|
||||
import MomentRuleUsage from "@/components/MomentRuleUsage.vue";
|
||||
import type { MomentAction, MomentUsage } from "@/api/moments";
|
||||
import { fmtDate } from "@/utils/dateFormat";
|
||||
|
||||
@@ -167,6 +168,7 @@ onMounted(() => store.load(true));
|
||||
>×</button>
|
||||
</span>
|
||||
</div>
|
||||
<MomentRuleUsage v-if="usage(m.name)" :rules="usage(m.name)!.by_rule" />
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
@@ -186,6 +188,7 @@ onMounted(() => store.load(true));
|
||||
</template>
|
||||
</span>
|
||||
</div>
|
||||
<MomentRuleUsage v-if="usage(name)" :rules="usage(name)!.by_rule" />
|
||||
</li>
|
||||
</ul>
|
||||
</template>
|
||||
|
||||
@@ -1,168 +0,0 @@
|
||||
<script setup lang="ts">
|
||||
import { useRouter } from "vue-router";
|
||||
import type { Note } from "@/types/note";
|
||||
import TagPill from "@/components/TagPill.vue";
|
||||
import { relativeTime } from "@/composables/useRelativeTime";
|
||||
import { renderPreview } from "@/utils/markdown";
|
||||
|
||||
const props = defineProps<{ note: Note; compact?: boolean }>();
|
||||
const emit = defineEmits<{ "tag-click": [tag: string] }>();
|
||||
const router = useRouter();
|
||||
|
||||
function onTagClick(tag: string) {
|
||||
emit("tag-click", tag);
|
||||
}
|
||||
|
||||
function goEdit() {
|
||||
router.push(`/notes/${props.note.id}/edit`);
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<router-link :to="`/notes/${note.id}`" :class="['note-card', { compact }]">
|
||||
|
||||
<!-- Compact: single row -->
|
||||
<template v-if="compact">
|
||||
<span class="note-title-compact">{{ note.title || "Untitled" }}</span>
|
||||
<div class="note-tags-compact">
|
||||
<TagPill
|
||||
v-for="tag in note.tags.slice(0, 3)"
|
||||
:key="tag"
|
||||
:tag="tag"
|
||||
@click.stop="onTagClick(tag)"
|
||||
/>
|
||||
</div>
|
||||
<span class="timestamp-compact">{{ relativeTime(note.updated_at) }}</span>
|
||||
</template>
|
||||
|
||||
<!-- Full: original layout -->
|
||||
<template v-else>
|
||||
<div class="note-top">
|
||||
<h3 class="note-title">{{ note.title || "Untitled" }}</h3>
|
||||
<button class="btn-edit" title="Edit" @click.prevent.stop="goEdit">Edit</button>
|
||||
</div>
|
||||
<div v-if="note.body" class="note-preview prose" v-html="renderPreview(note.body)"></div>
|
||||
<div class="note-meta">
|
||||
<TagPill
|
||||
v-for="tag in note.tags"
|
||||
:key="tag"
|
||||
:tag="tag"
|
||||
@click.stop="onTagClick(tag)"
|
||||
/>
|
||||
<span class="timestamp">{{ relativeTime(note.updated_at) }}</span>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
</router-link>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.note-card {
|
||||
display: block;
|
||||
padding: 1rem;
|
||||
border-radius: var(--fs-radius-lg);
|
||||
text-decoration: none;
|
||||
color: inherit;
|
||||
background: var(--fs-surface-raised);
|
||||
box-shadow: 0 1px 4px rgba(0, 0, 0, 0.06), 0 0 0 1px color-mix(in srgb, var(--fs-accent) 6%, transparent);
|
||||
transition: box-shadow 0.2s, transform 0.18s ease;
|
||||
}
|
||||
.note-card:hover {
|
||||
box-shadow: 0 4px 16px rgba(0, 0, 0, 0.10), 0 0 0 1px color-mix(in srgb, var(--fs-accent) 14.0%, transparent);
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
|
||||
/* Compact single-row layout */
|
||||
.note-card.compact {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.6rem;
|
||||
padding: 0.45rem 0.75rem;
|
||||
background: var(--fs-surface-raised);
|
||||
box-shadow: none;
|
||||
border-radius: 0;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
transform: none !important;
|
||||
}
|
||||
.note-card.compact:first-child {
|
||||
border-top: 1px solid var(--fs-border-color);
|
||||
}
|
||||
.note-card.compact:hover {
|
||||
box-shadow: none;
|
||||
background: color-mix(in srgb, var(--fs-accent) 4%, transparent);
|
||||
transform: none;
|
||||
}
|
||||
.note-title-compact {
|
||||
font-size: 0.9rem;
|
||||
font-weight: 500;
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.note-tags-compact {
|
||||
display: flex;
|
||||
gap: 0.3rem;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.timestamp-compact {
|
||||
font-size: 0.72rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
flex-shrink: 0;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
/* Full layout */
|
||||
.note-top {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 0.5rem;
|
||||
margin-bottom: 0.25rem;
|
||||
}
|
||||
.note-title {
|
||||
margin: 0;
|
||||
font-size: 1.1rem;
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.btn-edit {
|
||||
flex-shrink: 0;
|
||||
padding: 0.25rem 0.6rem;
|
||||
font-size: 0.8rem;
|
||||
background: var(--fs-surface-raised);
|
||||
color: var(--fs-text-secondary);
|
||||
border: 1px solid var(--fs-border-color);
|
||||
border-radius: var(--fs-radius-sm);
|
||||
cursor: pointer;
|
||||
transition: color 0.15s, border-color 0.15s;
|
||||
}
|
||||
.btn-edit:hover {
|
||||
color: var(--fs-accent);
|
||||
border-color: var(--fs-accent);
|
||||
}
|
||||
.note-preview {
|
||||
margin: 0 0 0.5rem;
|
||||
color: var(--fs-text-secondary);
|
||||
font-size: 0.9rem;
|
||||
max-height: 7.5em;
|
||||
overflow: hidden;
|
||||
position: relative;
|
||||
-webkit-mask-image: linear-gradient(to bottom, black 60%, transparent 100%);
|
||||
mask-image: linear-gradient(to bottom, black 60%, transparent 100%);
|
||||
}
|
||||
.note-meta {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.timestamp {
|
||||
margin-left: auto;
|
||||
font-size: 0.75rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
</style>
|
||||
@@ -0,0 +1,278 @@
|
||||
<script setup lang="ts">
|
||||
/**
|
||||
* Restore: put the whole instance back to a full backup.
|
||||
*
|
||||
* Choose a backup (one at the backup destination, or a file from this
|
||||
* computer), see what it would change, type the confirmation, run it. On a
|
||||
* live instance this is a rollback to the instance's own earlier backup; on a
|
||||
* new, empty one it takes the backup in. Either way the server backs up the
|
||||
* current state first and signs everyone out afterwards.
|
||||
*/
|
||||
import { computed, onMounted, ref } from "vue";
|
||||
import { apiErrorMessage } from "@/api/client";
|
||||
import {
|
||||
backupTakenAt, listRestoreSources, previewRestore, restoreBackup,
|
||||
type RestoreFrom, type RestorePreview, type RestoreResult, type StoredBackup,
|
||||
} from "@/api/backups";
|
||||
import { formatBytes } from "@/utils/bytes";
|
||||
|
||||
const emit = defineEmits<{ close: [] }>();
|
||||
|
||||
const CONFIRM_WORDS = "roll back";
|
||||
|
||||
type Step = "choose" | "checking" | "preview" | "running" | "done";
|
||||
const step = ref<Step>("choose");
|
||||
const sources = ref<StoredBackup[]>([]);
|
||||
const sourcesError = ref("");
|
||||
const sourcesLoading = ref(true);
|
||||
const chosen = ref<RestoreFrom | null>(null);
|
||||
const chosenLabel = ref("");
|
||||
const preview = ref<RestorePreview | null>(null);
|
||||
const result = ref<RestoreResult | null>(null);
|
||||
const error = ref("");
|
||||
const typed = ref("");
|
||||
const fileInput = ref<HTMLInputElement | null>(null);
|
||||
|
||||
const confirmed = computed(() => typed.value.trim().toLowerCase() === CONFIRM_WORDS);
|
||||
|
||||
function when(value: Date | string | null | undefined): string {
|
||||
if (!value) return "an unknown date";
|
||||
const date = typeof value === "string" ? new Date(value) : value;
|
||||
return Number.isNaN(date.getTime()) ? "an unknown date" : date.toLocaleString();
|
||||
}
|
||||
|
||||
onMounted(async () => {
|
||||
try {
|
||||
const answer = await listRestoreSources();
|
||||
sources.value = answer.backups;
|
||||
sourcesError.value = answer.error ?? "";
|
||||
} catch (e) {
|
||||
sourcesError.value = apiErrorMessage(e, "Could not list the backups at the destination.");
|
||||
} finally {
|
||||
sourcesLoading.value = false;
|
||||
}
|
||||
});
|
||||
|
||||
async function check(from: RestoreFrom, label: string) {
|
||||
chosen.value = from;
|
||||
chosenLabel.value = label;
|
||||
error.value = "";
|
||||
typed.value = "";
|
||||
step.value = "checking";
|
||||
try {
|
||||
preview.value = await previewRestore(from);
|
||||
step.value = "preview";
|
||||
} catch (e) {
|
||||
error.value = apiErrorMessage(e, "That backup could not be read.");
|
||||
step.value = "choose";
|
||||
}
|
||||
}
|
||||
|
||||
function pickFile(event: Event) {
|
||||
const file = (event.target as HTMLInputElement).files?.[0];
|
||||
if (fileInput.value) fileInput.value.value = "";
|
||||
if (file) check({ file }, file.name);
|
||||
}
|
||||
|
||||
async function run() {
|
||||
if (!chosen.value || !confirmed.value) return;
|
||||
error.value = "";
|
||||
step.value = "running";
|
||||
try {
|
||||
result.value = await restoreBackup(chosen.value);
|
||||
step.value = "done";
|
||||
} catch (e) {
|
||||
error.value = apiErrorMessage(e, "The restore failed. Nothing was changed.");
|
||||
step.value = "preview";
|
||||
}
|
||||
}
|
||||
|
||||
function finish() {
|
||||
// Every session ended with the restore, this one included.
|
||||
window.location.assign("/login");
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<teleport to="body">
|
||||
<div class="modal-overlay" @click="step === 'running' || step === 'done' ? null : emit('close')">
|
||||
<div class="modal-card restore-card" @click.stop>
|
||||
<template v-if="step === 'choose' || step === 'checking'">
|
||||
<h3 class="modal-title">Restore from a backup</h3>
|
||||
<p class="modal-message">
|
||||
Puts the whole instance back to the moment the backup was taken, for every user.
|
||||
You'll see what changes before anything happens.
|
||||
</p>
|
||||
<p v-if="error" class="restore-error" role="alert">{{ error }}</p>
|
||||
|
||||
<h4 class="restore-subhead">At the backup destination</h4>
|
||||
<p v-if="sourcesLoading" class="restore-muted">Looking for backups…</p>
|
||||
<p v-else-if="sourcesError" class="restore-muted">{{ sourcesError }}</p>
|
||||
<p v-else-if="!sources.length" class="restore-muted">No backups there yet.</p>
|
||||
<ul v-else class="restore-sources">
|
||||
<li v-for="b in sources" :key="b.name">
|
||||
<button
|
||||
class="restore-source"
|
||||
:disabled="step === 'checking'"
|
||||
@click="check({ source: b.name }, when(backupTakenAt(b.name)))"
|
||||
>
|
||||
<span>{{ when(backupTakenAt(b.name)) }}</span>
|
||||
<span class="restore-muted">{{ formatBytes(b.size_bytes) }}</span>
|
||||
</button>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h4 class="restore-subhead">From this computer</h4>
|
||||
<p class="restore-muted">
|
||||
A backup file (.tar) or the Full Backup download (.json). Uploads are size-limited;
|
||||
a large file restores from the destination above, or with
|
||||
<code>python -m grimoire.restore</code>.
|
||||
</p>
|
||||
<div class="modal-actions restore-actions-left">
|
||||
<button class="modal-btn" :disabled="step === 'checking'" @click="fileInput?.click()">
|
||||
Choose a file…
|
||||
</button>
|
||||
<input ref="fileInput" type="file" accept=".json,.tar" class="hidden-file-input" @change="pickFile" />
|
||||
</div>
|
||||
|
||||
<p v-if="step === 'checking'" class="restore-muted">Reading {{ chosenLabel }}…</p>
|
||||
<div class="modal-actions">
|
||||
<button class="modal-btn" @click="emit('close')">Cancel</button>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<template v-else-if="(step === 'preview' || step === 'running') && preview">
|
||||
<h3 class="modal-title">
|
||||
{{ preview.mode === "rollback" ? "Roll back to this backup?" : "Restore this backup?" }}
|
||||
</h3>
|
||||
<p class="modal-message">
|
||||
<template v-if="preview.mode === 'rollback'">
|
||||
Everything goes back to {{ when(preview.exported_at) }}, for every user. Anything made
|
||||
since is removed, edits since are undone, and records deleted since come back.
|
||||
</template>
|
||||
<template v-else>
|
||||
This instance has no records yet, so it takes the backup from {{ when(preview.exported_at) }} as it is.
|
||||
</template>
|
||||
The current state is backed up first, and everyone is signed out afterwards.
|
||||
</p>
|
||||
<table class="restore-counts">
|
||||
<thead><tr><th></th><th>Now</th><th>After</th></tr></thead>
|
||||
<tbody>
|
||||
<tr v-for="c in preview.counts" :key="c.kind">
|
||||
<td>{{ c.kind }}</td><td>{{ c.now }}</td><td>{{ c.after }}</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p v-if="error" class="restore-error" role="alert">{{ error }}</p>
|
||||
<label class="restore-confirm">
|
||||
Type <strong>{{ CONFIRM_WORDS }}</strong> to continue
|
||||
<input v-model="typed" class="fs-input input" :disabled="step === 'running'" autocomplete="off" />
|
||||
</label>
|
||||
<div class="modal-actions">
|
||||
<button class="modal-btn" :disabled="step === 'running'" @click="step = 'choose'">Back</button>
|
||||
<button
|
||||
class="modal-btn modal-btn-danger"
|
||||
:disabled="!confirmed || step === 'running'"
|
||||
@click="run"
|
||||
>{{ step === "running" ? "Restoring…" : preview.mode === "rollback" ? "Roll back" : "Restore" }}</button>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<template v-else-if="step === 'done' && result">
|
||||
<h3 class="modal-title">Restored</h3>
|
||||
<p class="modal-message">
|
||||
The instance is back to {{ when(result.exported_at) }}.
|
||||
The state it replaced was saved as <code>{{ result.safety_backup.file }}</code>
|
||||
in {{ result.safety_backup.where }}.
|
||||
<template v-if="result.safety_backup.note"> {{ result.safety_backup.note }}</template>
|
||||
Search is being rebuilt in the background. Everyone has been signed out, so sign in again.
|
||||
</p>
|
||||
<div class="modal-actions">
|
||||
<button class="modal-btn modal-btn-primary" @click="finish">Sign in again</button>
|
||||
</div>
|
||||
</template>
|
||||
</div>
|
||||
</div>
|
||||
</teleport>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.restore-card {
|
||||
max-width: 34rem;
|
||||
max-height: 85vh;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.restore-subhead {
|
||||
margin: 1rem 0 0.4rem;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
.restore-muted {
|
||||
font-size: 0.85rem;
|
||||
color: var(--fs-text-secondary);
|
||||
margin: 0 0 0.5rem;
|
||||
}
|
||||
.restore-error {
|
||||
font-size: 0.85rem;
|
||||
color: var(--fs-action-destructive);
|
||||
margin: 0 0 0.75rem;
|
||||
}
|
||||
.restore-sources {
|
||||
list-style: none;
|
||||
margin: 0 0 0.5rem;
|
||||
padding: 0;
|
||||
max-height: 14rem;
|
||||
overflow-y: auto;
|
||||
border: 1px solid var(--fs-border-color);
|
||||
border-radius: var(--fs-radius-sm);
|
||||
}
|
||||
.restore-source {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
width: 100%;
|
||||
padding: 0.45rem 0.7rem;
|
||||
border: none;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
background: none;
|
||||
color: var(--fs-text-primary);
|
||||
font: inherit;
|
||||
font-size: 0.875rem;
|
||||
text-align: left;
|
||||
cursor: pointer;
|
||||
}
|
||||
.restore-sources li:last-child .restore-source {
|
||||
border-bottom: none;
|
||||
}
|
||||
.restore-source:hover:not(:disabled) {
|
||||
background: var(--fs-surface-hover);
|
||||
}
|
||||
.restore-actions-left {
|
||||
justify-content: flex-start;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
.restore-counts {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
font-size: 0.875rem;
|
||||
margin: 0 0 1rem;
|
||||
}
|
||||
.restore-counts th,
|
||||
.restore-counts td {
|
||||
text-align: right;
|
||||
padding: 0.25rem 0.5rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
}
|
||||
.restore-counts th:first-child,
|
||||
.restore-counts td:first-child {
|
||||
text-align: left;
|
||||
}
|
||||
.hidden-file-input {
|
||||
display: none;
|
||||
}
|
||||
.restore-confirm {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.35rem;
|
||||
font-size: 0.875rem;
|
||||
margin: 0 0 1rem;
|
||||
}
|
||||
</style>
|
||||
@@ -0,0 +1,309 @@
|
||||
<script setup lang="ts">
|
||||
import { computed, onMounted, ref } from "vue";
|
||||
import { apiGet, apiPut, apiErrorMessage } from "@/api/client";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
|
||||
/**
|
||||
* How many reverse proxies stand in front of Grimoire, checked against the
|
||||
* admin's own request.
|
||||
*
|
||||
* Each proxy appends to X-Forwarded-For, and Grimoire reads the entry this many
|
||||
* in from the right as the visitor's address. Every sign-in rate limit and
|
||||
* audit line keys on that address, and the scheme read the same way decides
|
||||
* whether the session cookie is Secure. Too low a count makes every visitor
|
||||
* read as the proxy, so they share one rate limit and one bad sign-in run
|
||||
* blocks everyone. Behind Cloudflare's proxied DNS the usual count is 2.
|
||||
*/
|
||||
interface ReadoutRow { hops: number; client: string; https: boolean }
|
||||
interface Readout {
|
||||
peer: string;
|
||||
forwarded_for: string[];
|
||||
forwarded_proto: string[];
|
||||
cloudflare: boolean;
|
||||
cloudflare_client: string | null;
|
||||
rows: ReadoutRow[];
|
||||
suggested: number | null;
|
||||
}
|
||||
interface ProxyConfig {
|
||||
hops: number;
|
||||
source: "setting" | "env" | "default";
|
||||
env_hops: number;
|
||||
max_hops: number;
|
||||
readout: Readout;
|
||||
}
|
||||
|
||||
const toast = useToastStore();
|
||||
const config = ref<ProxyConfig | null>(null);
|
||||
const loadError = ref("");
|
||||
const draft = ref<number>(1);
|
||||
const saving = ref(false);
|
||||
const saved = ref(false);
|
||||
|
||||
async function load() {
|
||||
loadError.value = "";
|
||||
try {
|
||||
config.value = await apiGet<ProxyConfig>("/api/admin/proxy");
|
||||
draft.value = config.value.hops;
|
||||
} catch (e) {
|
||||
loadError.value = apiErrorMessage(e, "The proxy settings could not be loaded.");
|
||||
}
|
||||
}
|
||||
|
||||
async function save(hops: number | null) {
|
||||
saving.value = true;
|
||||
saved.value = false;
|
||||
try {
|
||||
config.value = await apiPut<ProxyConfig>("/api/admin/proxy", { hops });
|
||||
draft.value = config.value.hops;
|
||||
saved.value = true;
|
||||
setTimeout(() => (saved.value = false), 2000);
|
||||
} catch (e) {
|
||||
toast.show(apiErrorMessage(e, "Failed to save the proxy count"), "error");
|
||||
} finally {
|
||||
saving.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
const readout = computed(() => config.value?.readout ?? null);
|
||||
const inForce = computed(() => readout.value?.rows.find((r) => r.hops === config.value?.hops) ?? null);
|
||||
|
||||
// The verdict that leads the section. A Cloudflare request names the count to
|
||||
// set, since Cloudflare reports the visitor's address; any other road can only
|
||||
// show the address read and let the admin judge whether it is theirs.
|
||||
const verdict = computed<"set-suggested" | "cloudflare-unmatched" | "check" | null>(() => {
|
||||
const r = readout.value;
|
||||
if (!r || !config.value) return null;
|
||||
if (r.cloudflare && r.suggested !== null && r.suggested !== config.value.hops) return "set-suggested";
|
||||
if (r.cloudflare && r.suggested === null) return "cloudflare-unmatched";
|
||||
return "check";
|
||||
});
|
||||
|
||||
const sourceLabel = computed(() => {
|
||||
const c = config.value;
|
||||
if (!c) return "";
|
||||
if (c.source === "setting") return "Saved here.";
|
||||
if (c.source === "env") return `From TRUSTED_PROXY_HOPS in the environment (${c.env_hops}).`;
|
||||
return "The default.";
|
||||
});
|
||||
|
||||
const draftValid = computed(() =>
|
||||
Number.isInteger(draft.value) && draft.value >= 0 && draft.value <= (config.value?.max_hops ?? 5),
|
||||
);
|
||||
|
||||
onMounted(load);
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="proxy-settings">
|
||||
<p v-if="loadError" class="error-msg">
|
||||
{{ loadError }}
|
||||
<button type="button" class="btn-ghost btn-compact" @click="load">Try again</button>
|
||||
</p>
|
||||
<p v-else-if="!config" class="state-msg">Reading this request…</p>
|
||||
|
||||
<template v-else-if="readout">
|
||||
<div v-if="verdict === 'set-suggested'" class="proxy-warning" role="alert">
|
||||
<p class="proxy-warning-title">Set the proxy count to {{ readout.suggested }}</p>
|
||||
<p class="proxy-warning-body">
|
||||
This request came through Cloudflare, which reports your address as
|
||||
<code>{{ readout.cloudflare_client }}</code>. At {{ config.hops }}, Grimoire reads
|
||||
<code>{{ inForce?.client }}</code> instead, so visitors share Cloudflare's addresses
|
||||
and one sign-in limit.
|
||||
</p>
|
||||
<button type="button" class="btn-primary" :disabled="saving" @click="save(readout.suggested)">
|
||||
Use {{ readout.suggested }}
|
||||
</button>
|
||||
</div>
|
||||
<div v-else-if="verdict === 'cloudflare-unmatched'" class="proxy-warning" role="alert">
|
||||
<p class="proxy-warning-title">This request came through Cloudflare</p>
|
||||
<p class="proxy-warning-body">
|
||||
No count reads the address Cloudflare reports for you. Check the forwarding chain below,
|
||||
and that your own proxy passes Cloudflare's <code>X-Forwarded-For</code> on.
|
||||
</p>
|
||||
</div>
|
||||
<p v-else class="proxy-verdict">
|
||||
Grimoire reads this request as coming from <code>{{ inForce?.client || "an unknown address" }}</code>
|
||||
<template v-if="inForce?.https"> over HTTPS</template>. If that is your address, the count
|
||||
is right. If it is your proxy's, raise the count.
|
||||
</p>
|
||||
|
||||
<div class="field proxy-field">
|
||||
<label for="proxy-hops">Proxies in front of Grimoire</label>
|
||||
<div class="proxy-input-row">
|
||||
<input
|
||||
id="proxy-hops"
|
||||
v-model.number="draft"
|
||||
type="number"
|
||||
min="0"
|
||||
:max="config.max_hops"
|
||||
step="1"
|
||||
class="fs-input input proxy-hops-input"
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
class="btn-primary"
|
||||
:disabled="saving || !draftValid || draft === config.hops"
|
||||
@click="save(draft)"
|
||||
>
|
||||
{{ saving ? "Saving..." : "Save" }}
|
||||
</button>
|
||||
<button
|
||||
v-if="config.source === 'setting'"
|
||||
type="button"
|
||||
class="btn-ghost"
|
||||
:disabled="saving"
|
||||
@click="save(null)"
|
||||
>
|
||||
Use the environment's ({{ config.env_hops }})
|
||||
</button>
|
||||
<span v-if="saved" class="saved-msg">Saved. It applies to the next request.</span>
|
||||
</div>
|
||||
<p class="field-hint">
|
||||
{{ sourceLabel }} One proxy (Traefik, Nginx, Caddy) is 1. Cloudflare in front of your own
|
||||
proxy is 2. 0 ignores the forwarding headers and uses the connecting address.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<table class="proxy-table">
|
||||
<caption>What each count makes of this request</caption>
|
||||
<thead>
|
||||
<tr><th class="num">Count</th><th>Reads you as</th><th>HTTPS</th><th></th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr
|
||||
v-for="row in readout.rows"
|
||||
:key="row.hops"
|
||||
:class="{ 'proxy-row-current': row.hops === config.hops }"
|
||||
>
|
||||
<td class="num">{{ row.hops }}</td>
|
||||
<td><code>{{ row.client || "unknown" }}</code></td>
|
||||
<td>{{ row.https ? "yes" : "no" }}</td>
|
||||
<td class="proxy-row-tags">
|
||||
<span v-if="row.hops === config.hops" class="proxy-tag">in force</span>
|
||||
<span v-if="row.hops === readout.suggested && row.hops !== config.hops" class="proxy-tag proxy-tag-suggested">
|
||||
suggested
|
||||
</span>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<dl class="proxy-chain">
|
||||
<dt>Connecting address</dt>
|
||||
<dd><code>{{ readout.peer || "unknown" }}</code></dd>
|
||||
<dt>X-Forwarded-For</dt>
|
||||
<dd v-if="readout.forwarded_for.length"><code>{{ readout.forwarded_for.join(", ") }}</code></dd>
|
||||
<dd v-else class="proxy-absent">not sent</dd>
|
||||
<dt>X-Forwarded-Proto</dt>
|
||||
<dd v-if="readout.forwarded_proto.length"><code>{{ readout.forwarded_proto.join(", ") }}</code></dd>
|
||||
<dd v-else class="proxy-absent">not sent</dd>
|
||||
</dl>
|
||||
</template>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.proxy-warning {
|
||||
margin-bottom: var(--fs-space-4);
|
||||
padding: var(--fs-space-3);
|
||||
border: 1px solid var(--fs-warning);
|
||||
border-radius: var(--fs-radius-md);
|
||||
background: color-mix(in srgb, var(--fs-warning) 12%, var(--fs-surface-raised));
|
||||
}
|
||||
.proxy-warning-title {
|
||||
margin: 0 0 var(--fs-space-2);
|
||||
font-weight: 600;
|
||||
color: var(--fs-warning-fg);
|
||||
}
|
||||
.proxy-warning-body {
|
||||
margin: 0 0 var(--fs-space-3);
|
||||
font-size: var(--fs-size-body-sm);
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
.proxy-verdict {
|
||||
margin: 0 0 var(--fs-space-4);
|
||||
font-size: var(--fs-size-body-sm);
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
.proxy-field {
|
||||
margin-bottom: var(--fs-space-4);
|
||||
}
|
||||
.proxy-input-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: var(--fs-space-3);
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.proxy-hops-input {
|
||||
width: 5rem;
|
||||
}
|
||||
.proxy-table {
|
||||
width: 100%;
|
||||
max-width: 560px;
|
||||
border-collapse: collapse;
|
||||
font-size: var(--fs-size-body-sm);
|
||||
margin-bottom: var(--fs-space-4);
|
||||
}
|
||||
.proxy-table caption {
|
||||
text-align: left;
|
||||
font-size: var(--fs-size-label);
|
||||
color: var(--fs-text-secondary);
|
||||
margin-bottom: var(--fs-space-2);
|
||||
}
|
||||
.proxy-table th,
|
||||
.proxy-table td {
|
||||
text-align: left;
|
||||
padding: var(--fs-space-1) var(--fs-space-3);
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
}
|
||||
.proxy-table th {
|
||||
font-size: var(--fs-size-tiny);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
color: var(--fs-text-tertiary);
|
||||
font-weight: 500;
|
||||
}
|
||||
.proxy-table .num {
|
||||
text-align: right;
|
||||
font-variant-numeric: tabular-nums;
|
||||
width: 4rem;
|
||||
}
|
||||
.proxy-row-current td {
|
||||
background: var(--fs-surface-hover);
|
||||
font-weight: 600;
|
||||
}
|
||||
.proxy-row-tags {
|
||||
white-space: nowrap;
|
||||
}
|
||||
.proxy-tag {
|
||||
display: inline-block;
|
||||
padding: 0 var(--fs-space-2);
|
||||
border-radius: var(--fs-radius-pill);
|
||||
font-size: var(--fs-size-tiny);
|
||||
font-weight: 500;
|
||||
color: var(--fs-text-secondary);
|
||||
border: 1px solid var(--fs-border-active);
|
||||
}
|
||||
.proxy-tag-suggested {
|
||||
color: var(--fs-warning-fg);
|
||||
border-color: var(--fs-warning);
|
||||
}
|
||||
.proxy-chain {
|
||||
display: grid;
|
||||
grid-template-columns: max-content 1fr;
|
||||
gap: var(--fs-space-1) var(--fs-space-4);
|
||||
margin: 0;
|
||||
font-size: var(--fs-size-body-sm);
|
||||
}
|
||||
.proxy-chain dt {
|
||||
color: var(--fs-text-secondary);
|
||||
}
|
||||
.proxy-chain dd {
|
||||
margin: 0;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
.proxy-absent {
|
||||
font-style: italic;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
</style>
|
||||
@@ -0,0 +1,96 @@
|
||||
<script setup lang="ts">
|
||||
/**
|
||||
* The end of a server-paged list: asks for the next page as the reader nears
|
||||
* it, in place of a "Load more" button. The parent keeps the paging — what the
|
||||
* next page is keyed on, whether one is in flight, whether the last one
|
||||
* failed — and this owns the observing.
|
||||
*
|
||||
* <ScrollSentinel :cursor="more ? String(rows.length) : ''"
|
||||
* :loading="loadingMore" :failed="moreFailed" @more="loadMore" />
|
||||
*
|
||||
* An empty cursor means the list is complete, and nothing renders.
|
||||
*/
|
||||
import { onBeforeUnmount, ref, watch } from "vue";
|
||||
|
||||
const props = defineProps<{ cursor: string; loading: boolean; failed?: boolean }>();
|
||||
const emit = defineEmits<{ more: [] }>();
|
||||
|
||||
const sentinel = ref<HTMLElement | null>(null);
|
||||
const visible = ref(false);
|
||||
|
||||
// One request per page. The observer reports every crossing, and a short last
|
||||
// page leaves the sentinel on screen for good, so the cursor last asked for is
|
||||
// remembered: a page is asked for once, and a failed one is not retried on
|
||||
// every scroll, because a failure leaves the cursor where it was. Only
|
||||
// Try again asks for it again.
|
||||
let asked = "";
|
||||
|
||||
function pump() {
|
||||
if (!visible.value || props.loading || props.failed || !props.cursor) return;
|
||||
if (props.cursor === asked) return;
|
||||
asked = props.cursor;
|
||||
emit("more");
|
||||
}
|
||||
|
||||
function retry() {
|
||||
asked = props.cursor;
|
||||
emit("more");
|
||||
}
|
||||
|
||||
let observer: IntersectionObserver | undefined;
|
||||
|
||||
function observe(el: HTMLElement | null) {
|
||||
observer?.disconnect();
|
||||
observer = undefined;
|
||||
if (!el) {
|
||||
// Visibility and the cursor asked for go with the element. A list that
|
||||
// starts over must not inherit "the end is on screen" from the one it
|
||||
// replaced, and an offset cursor repeats from list to list, so "50" asked
|
||||
// of the old list must not stop the new one asking for its own "50".
|
||||
visible.value = false;
|
||||
asked = "";
|
||||
return;
|
||||
}
|
||||
observer = new IntersectionObserver(
|
||||
(entries) => {
|
||||
visible.value = entries.some((e) => e.isIntersecting);
|
||||
pump();
|
||||
},
|
||||
// Extends the watched area a screen below the viewport, so the next page is
|
||||
// asked for before the reader reaches the end rather than when they do.
|
||||
{ rootMargin: "0px 0px 100% 0px" },
|
||||
);
|
||||
observer.observe(el);
|
||||
}
|
||||
|
||||
// flush: "post": the template ref is only set once the DOM is patched.
|
||||
watch(sentinel, observe, { flush: "post" });
|
||||
|
||||
// A page that lands without pushing the end off screen gives the observer
|
||||
// nothing new to report, so carry on from the props, or the list stops early.
|
||||
watch([() => props.cursor, () => props.loading, () => props.failed], pump);
|
||||
|
||||
onBeforeUnmount(() => observer?.disconnect());
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div v-if="cursor" class="sentinel">
|
||||
<!-- The one case with a button: an observer re-firing on a failure would be
|
||||
a request loop. -->
|
||||
<p v-if="failed" class="sentinel-note sentinel-failed" role="alert">
|
||||
Couldn’t load more.
|
||||
<button type="button" class="btn-secondary btn-sm" @click="retry">Try again</button>
|
||||
</p>
|
||||
<template v-else>
|
||||
<div ref="sentinel" class="sentinel-mark" aria-hidden="true"></div>
|
||||
<p v-if="loading" class="sentinel-note" role="status">Loading more…</p>
|
||||
</template>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.sentinel { grid-column: 1 / -1; padding: 0.75rem 0; text-align: center; }
|
||||
.sentinel-mark { height: 1px; }
|
||||
.sentinel-note { margin: 0; font-size: 0.8rem; color: var(--fs-text-tertiary); }
|
||||
.sentinel-failed { display: flex; align-items: center; justify-content: center; gap: 0.75rem; }
|
||||
</style>
|
||||
@@ -1,683 +0,0 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, computed, watch, onMounted, onUnmounted, nextTick } from "vue";
|
||||
import type { Editor } from "@tiptap/vue-3";
|
||||
import { apiGet, apiPatch, apiPost, apiDelete } from "@/api/client";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
import { useAutoSave } from "@/composables/useAutoSave";
|
||||
import { useTagSuggestions } from "@/composables/useTagSuggestions";
|
||||
import { useNotesStore } from "@/stores/notes";
|
||||
import TiptapEditor from "@/components/TiptapEditor.vue";
|
||||
import TagInput from "@/components/TagInput.vue";
|
||||
import MarkdownToolbar from "@/components/MarkdownToolbar.vue";
|
||||
import WordCount from "@/components/WordCount.vue";
|
||||
import { Trash2, X } from "lucide-vue-next";
|
||||
import { relativeTimeOrDate } from "@/composables/useRelativeTime";
|
||||
|
||||
const props = defineProps<{
|
||||
projectId: number;
|
||||
activeNoteId: number | null;
|
||||
}>();
|
||||
|
||||
const emit = defineEmits<{ "note-loaded": [id: number] }>();
|
||||
|
||||
interface NoteItem {
|
||||
id: number;
|
||||
title: string;
|
||||
tags: string[];
|
||||
updated_at: string;
|
||||
}
|
||||
|
||||
const toast = useToastStore();
|
||||
const notesStore = useNotesStore();
|
||||
|
||||
// List state
|
||||
const projectNotes = ref<NoteItem[]>([]);
|
||||
const listLoading = ref(false);
|
||||
const deletingId = ref<number | null>(null);
|
||||
const pendingDelete = ref<number | null>(null);
|
||||
const searchQuery = ref("");
|
||||
|
||||
const filteredNotes = computed(() => {
|
||||
const q = searchQuery.value.trim().toLowerCase();
|
||||
if (!q) return projectNotes.value;
|
||||
return projectNotes.value.filter(
|
||||
(n) => n.title.toLowerCase().includes(q) || n.tags?.some((t) => t.toLowerCase().includes(q))
|
||||
);
|
||||
});
|
||||
|
||||
function matchedTags(note: NoteItem): string[] {
|
||||
const q = searchQuery.value.trim().toLowerCase();
|
||||
if (!q) return [];
|
||||
return (note.tags ?? []).filter(
|
||||
(t) => t.toLowerCase().includes(q) && !note.title.toLowerCase().includes(q)
|
||||
);
|
||||
}
|
||||
|
||||
// Editor state
|
||||
const editingId = ref<number | null>(null);
|
||||
const noteTitle = ref("");
|
||||
const noteBody = ref("");
|
||||
const noteTags = ref<string[]>([]);
|
||||
const dirty = ref(false);
|
||||
const saving = ref(false);
|
||||
|
||||
const tagSuggestions = useTagSuggestions(noteTitle, noteBody, noteTags, () => { dirty.value = true; });
|
||||
|
||||
// New note inline creation
|
||||
const showNewNoteInput = ref(false);
|
||||
const newNoteTitle = ref("");
|
||||
const creatingNote = ref(false);
|
||||
const newNoteTitleRef = ref<HTMLInputElement | null>(null);
|
||||
|
||||
function _noteKey(pid: number) { return `workspace_note_${pid}`; }
|
||||
|
||||
async function startNewNote() {
|
||||
showNewNoteInput.value = true;
|
||||
newNoteTitle.value = "";
|
||||
await nextTick();
|
||||
newNoteTitleRef.value?.focus();
|
||||
}
|
||||
|
||||
function cancelNewNote() {
|
||||
showNewNoteInput.value = false;
|
||||
newNoteTitle.value = "";
|
||||
}
|
||||
|
||||
async function createNote() {
|
||||
const title = newNoteTitle.value.trim();
|
||||
if (!title || creatingNote.value) return;
|
||||
creatingNote.value = true;
|
||||
try {
|
||||
const note = await apiPost<{ id: number; title: string; body: string; tags: string[]; updated_at: string }>(
|
||||
"/api/notes",
|
||||
{ title, project_id: props.projectId, body: "", is_task: false }
|
||||
);
|
||||
showNewNoteInput.value = false;
|
||||
newNoteTitle.value = "";
|
||||
await loadProjectNotes();
|
||||
await openNote(note.id);
|
||||
} catch {
|
||||
toast.show("Failed to create note", "error");
|
||||
} finally {
|
||||
creatingNote.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
// TipTap editor ref
|
||||
const titleRef = ref<HTMLInputElement | null>(null);
|
||||
const editorRef = ref<InstanceType<typeof TiptapEditor> | null>(null);
|
||||
const tiptapEditor = computed<Editor | null>(() =>
|
||||
(editorRef.value?.editor as Editor | undefined) ?? null
|
||||
);
|
||||
|
||||
// Link suggestions
|
||||
interface LinkSuggestion { note_id: number; title: string; count: number; }
|
||||
const linkSuggestions = ref<LinkSuggestion[]>([]);
|
||||
let linkCheckTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
|
||||
async function fetchLinkSuggestions() {
|
||||
if (!editingId.value || !noteBody.value) { linkSuggestions.value = []; return; }
|
||||
try {
|
||||
const res = await apiPost<{ suggestions: LinkSuggestion[] }>('/api/notes/link-suggestions', {
|
||||
body: noteBody.value,
|
||||
project_id: props.projectId,
|
||||
exclude_note_id: editingId.value,
|
||||
});
|
||||
linkSuggestions.value = res.suggestions;
|
||||
} catch {
|
||||
linkSuggestions.value = [];
|
||||
}
|
||||
}
|
||||
|
||||
function scheduleLinkCheck() {
|
||||
if (linkCheckTimer) clearTimeout(linkCheckTimer);
|
||||
linkCheckTimer = setTimeout(fetchLinkSuggestions, 2500);
|
||||
}
|
||||
|
||||
function applyWikilinkToText(text: string, title: string): string {
|
||||
const escaped = title.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
const parts = text.split(/(\[\[[^\]]*\]\])/);
|
||||
const pattern = new RegExp(`\\b${escaped}\\b`, 'gi');
|
||||
return parts.map((part, i) =>
|
||||
i % 2 === 0 ? part.replace(pattern, `[[${title}]]`) : part
|
||||
).join('');
|
||||
}
|
||||
|
||||
function applyLink(s: LinkSuggestion) {
|
||||
noteBody.value = applyWikilinkToText(noteBody.value, s.title);
|
||||
dirty.value = true;
|
||||
linkSuggestions.value = linkSuggestions.value.filter(x => x.note_id !== s.note_id);
|
||||
}
|
||||
|
||||
function applyAllLinks() {
|
||||
let text = noteBody.value;
|
||||
for (const s of linkSuggestions.value) text = applyWikilinkToText(text, s.title);
|
||||
noteBody.value = text;
|
||||
dirty.value = true;
|
||||
linkSuggestions.value = [];
|
||||
}
|
||||
|
||||
async function loadProjectNotes() {
|
||||
listLoading.value = true;
|
||||
try {
|
||||
const data = await apiGet<{ notes: NoteItem[] }>(
|
||||
`/api/projects/${props.projectId}/notes?type=note&limit=200`
|
||||
);
|
||||
projectNotes.value = data.notes ?? [];
|
||||
} catch {
|
||||
toast.show("Failed to load notes", "error");
|
||||
} finally {
|
||||
listLoading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function openNote(id: number) {
|
||||
try {
|
||||
const note = await apiGet<{ id: number; title: string; body: string; tags: string[]; updated_at: string }>(
|
||||
`/api/notes/${id}`
|
||||
);
|
||||
editingId.value = note.id;
|
||||
noteTitle.value = note.title;
|
||||
noteBody.value = note.body ?? "";
|
||||
noteTags.value = note.tags ?? [];
|
||||
dirty.value = false;
|
||||
tagSuggestions.dismissTagSuggestions();
|
||||
linkSuggestions.value = [];
|
||||
localStorage.setItem(_noteKey(props.projectId), String(id));
|
||||
fetchLinkSuggestions();
|
||||
emit("note-loaded", note.id);
|
||||
} catch {
|
||||
toast.show("Failed to load note", "error");
|
||||
}
|
||||
}
|
||||
|
||||
async function saveNote() {
|
||||
if (!editingId.value) return;
|
||||
saving.value = true;
|
||||
try {
|
||||
await apiPatch(`/api/notes/${editingId.value}`, {
|
||||
title: noteTitle.value,
|
||||
body: noteBody.value,
|
||||
tags: noteTags.value,
|
||||
});
|
||||
dirty.value = false;
|
||||
const idx = projectNotes.value.findIndex((n) => n.id === editingId.value);
|
||||
if (idx !== -1) {
|
||||
projectNotes.value[idx] = {
|
||||
...projectNotes.value[idx],
|
||||
title: noteTitle.value,
|
||||
tags: noteTags.value,
|
||||
updated_at: new Date().toISOString(),
|
||||
};
|
||||
projectNotes.value.sort((a, b) =>
|
||||
new Date(b.updated_at).getTime() - new Date(a.updated_at).getTime()
|
||||
);
|
||||
}
|
||||
} catch {
|
||||
toast.show("Failed to save note", "error");
|
||||
} finally {
|
||||
saving.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
function requestDelete(id: number, e: Event) {
|
||||
e.stopPropagation();
|
||||
if (deletingId.value === id) {
|
||||
confirmDelete(id);
|
||||
} else {
|
||||
deletingId.value = id;
|
||||
}
|
||||
}
|
||||
|
||||
function cancelDelete(e: Event) {
|
||||
e.stopPropagation();
|
||||
deletingId.value = null;
|
||||
}
|
||||
|
||||
async function confirmDelete(id: number) {
|
||||
pendingDelete.value = id;
|
||||
try {
|
||||
await apiDelete(`/api/notes/${id}`);
|
||||
projectNotes.value = projectNotes.value.filter((n) => n.id !== id);
|
||||
if (editingId.value === id) {
|
||||
editingId.value = null;
|
||||
noteTitle.value = "";
|
||||
noteBody.value = "";
|
||||
localStorage.removeItem(_noteKey(props.projectId));
|
||||
}
|
||||
} catch {
|
||||
toast.show("Failed to delete note", "error");
|
||||
} finally {
|
||||
pendingDelete.value = null;
|
||||
deletingId.value = null;
|
||||
}
|
||||
}
|
||||
|
||||
watch(noteTitle, () => { dirty.value = true; });
|
||||
watch(noteBody, () => { dirty.value = true; if (editingId.value) scheduleLinkCheck(); });
|
||||
watch(noteTags, () => { dirty.value = true; });
|
||||
|
||||
watch(
|
||||
() => props.activeNoteId,
|
||||
async (id) => {
|
||||
if (id) {
|
||||
await openNote(id);
|
||||
await loadProjectNotes();
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
useAutoSave(dirty, saving, saveNote, 60_000);
|
||||
|
||||
onMounted(async () => {
|
||||
await loadProjectNotes();
|
||||
const stored = localStorage.getItem(_noteKey(props.projectId));
|
||||
if (stored) {
|
||||
const id = Number(stored);
|
||||
if (projectNotes.value.find((n) => n.id === id)) {
|
||||
openNote(id).catch(() => {});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
onUnmounted(() => { if (linkCheckTimer) clearTimeout(linkCheckTimer); });
|
||||
|
||||
defineExpose({ reload: loadProjectNotes });
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="ws-note-editor">
|
||||
|
||||
<!-- Left rail: always-visible note list -->
|
||||
<div class="note-rail">
|
||||
<div class="rail-header">
|
||||
<template v-if="showNewNoteInput">
|
||||
<input
|
||||
ref="newNoteTitleRef"
|
||||
v-model="newNoteTitle"
|
||||
class="new-note-input"
|
||||
placeholder="Note title..."
|
||||
:disabled="creatingNote"
|
||||
@keydown.enter="createNote"
|
||||
@keydown.escape="cancelNewNote"
|
||||
/>
|
||||
<button class="btn-primary btn-inline" :disabled="creatingNote || !newNoteTitle.trim()" @click="createNote">
|
||||
{{ creatingNote ? '…' : '+' }}
|
||||
</button>
|
||||
<button class="btn-text" aria-label="Cancel new note" @click="cancelNewNote"><X :size="16" /></button>
|
||||
</template>
|
||||
<template v-else>
|
||||
<span class="rail-title">Notes</span>
|
||||
<button class="btn-ghost btn-inline" @click="startNewNote" title="New note">+ New</button>
|
||||
</template>
|
||||
</div>
|
||||
|
||||
<div class="rail-search">
|
||||
<input
|
||||
v-model="searchQuery"
|
||||
class="rail-search-input"
|
||||
placeholder="Search…"
|
||||
type="search"
|
||||
aria-label="Search notes"
|
||||
/>
|
||||
<button v-if="searchQuery" class="btn-text btn-search-clear" aria-label="Clear search" @click="searchQuery = ''"><X :size="16" /></button>
|
||||
</div>
|
||||
|
||||
<div v-if="listLoading" class="rail-state">Loading…</div>
|
||||
<ul v-else class="note-list">
|
||||
<li
|
||||
v-for="note in filteredNotes"
|
||||
:key="note.id"
|
||||
:class="['note-row', { active: editingId === note.id }]"
|
||||
@click="openNote(note.id)"
|
||||
>
|
||||
<div class="note-row-main">
|
||||
<span class="note-row-title">{{ note.title || 'Untitled' }}</span>
|
||||
<span class="note-row-age">{{ relativeTimeOrDate(note.updated_at) }}</span>
|
||||
</div>
|
||||
<div v-if="note.tags?.length" class="note-row-tags">
|
||||
<span
|
||||
v-for="tag in note.tags.slice(0, 2)"
|
||||
:key="tag"
|
||||
:class="['note-tag-pill', { 'tag-match': matchedTags(note).includes(tag) }]"
|
||||
>#{{ tag }}</span>
|
||||
<span v-if="note.tags.length > 2" class="note-tag-more">+{{ note.tags.length - 2 }}</span>
|
||||
</div>
|
||||
<div class="note-row-actions" @click.stop>
|
||||
<template v-if="deletingId === note.id">
|
||||
<button class="btn-danger-outline btn-inline" :disabled="pendingDelete === note.id" @click="requestDelete(note.id, $event)">
|
||||
{{ pendingDelete === note.id ? '…' : 'Delete?' }}
|
||||
</button>
|
||||
<button class="btn-text" aria-label="Cancel delete" @click="cancelDelete($event)"><X :size="16" /></button>
|
||||
</template>
|
||||
<button v-else class="btn-text" title="Delete note" @click="requestDelete(note.id, $event)">
|
||||
<Trash2 :size="16" />
|
||||
</button>
|
||||
</div>
|
||||
</li>
|
||||
<li v-if="filteredNotes.length === 0" class="rail-state">
|
||||
{{ searchQuery.trim() ? 'No matches.' : 'No notes yet.' }}
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<!-- Right pane: editor -->
|
||||
<div class="note-editor-pane">
|
||||
<template v-if="editingId">
|
||||
<div class="panel-header">
|
||||
<div class="editor-header-right">
|
||||
<WordCount :body="noteBody" />
|
||||
<span v-if="dirty && !saving" class="unsaved">Unsaved</span>
|
||||
<span v-if="saving" class="saving-txt">Saving…</span>
|
||||
<button class="btn-primary btn-compact" :disabled="saving || !dirty" @click="saveNote">Save</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="note-title-row">
|
||||
<input
|
||||
ref="titleRef"
|
||||
v-model="noteTitle"
|
||||
class="note-title-input"
|
||||
placeholder="Note title"
|
||||
@keydown.ctrl.s.prevent="saveNote"
|
||||
@keydown.ctrl.e.prevent="editorRef?.editor?.commands.focus()"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div class="tag-row">
|
||||
<TagInput v-model="noteTags" :fetchTags="(q: string) => notesStore.fetchAllTags(q)" />
|
||||
<button
|
||||
class="btn-ghost btn-compact"
|
||||
:disabled="tagSuggestions.suggestingTags.value"
|
||||
title="Auto-suggest tags from title and body"
|
||||
@click="tagSuggestions.fetchTagSuggestions()"
|
||||
>
|
||||
{{ tagSuggestions.suggestingTags.value ? '…' : 'Suggest' }}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div v-if="tagSuggestions.suggestedTags.value.length" class="tag-suggestions">
|
||||
<span class="tag-suggestions-label">Suggested:</span>
|
||||
<button
|
||||
v-for="tag in tagSuggestions.suggestedTags.value"
|
||||
:key="tag"
|
||||
:class="['btn-tag-suggestion', { applied: tagSuggestions.appliedTags.value.has(tag) }]"
|
||||
@click="tagSuggestions.applyTagSuggestion(tag)"
|
||||
>#{{ tag }}{{ tagSuggestions.appliedTags.value.has(tag) ? ' ✓' : '' }}</button>
|
||||
<button class="btn-text" aria-label="Dismiss tag suggestions" @click="tagSuggestions.dismissTagSuggestions()"><X :size="16" /></button>
|
||||
</div>
|
||||
|
||||
<div class="toolbar-row">
|
||||
<MarkdownToolbar :editor="tiptapEditor" />
|
||||
</div>
|
||||
|
||||
<div v-if="linkSuggestions.length" class="link-suggest-strip">
|
||||
<span class="link-suggest-label">Links:</span>
|
||||
<span v-for="s in linkSuggestions" :key="s.note_id" class="link-suggest-chip" :title="`Appears ${s.count}× unlinked`">
|
||||
<button class="btn-chip-link" @click="applyLink(s)">[[{{ s.title }}]]</button>
|
||||
</span>
|
||||
<button class="btn-ghost btn-inline" @click="applyAllLinks" title="Link all suggestions">All</button>
|
||||
<button class="btn-text" aria-label="Dismiss link suggestions" @click="linkSuggestions = []"><X :size="16" /></button>
|
||||
</div>
|
||||
|
||||
<div class="editor-area" @keydown.ctrl.s.prevent="saveNote" @keydown.ctrl.e.prevent="editorRef?.editor?.commands.focus()">
|
||||
<TiptapEditor ref="editorRef" v-model="noteBody" @escape="titleRef?.focus()" />
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<div v-else class="editor-empty-state">
|
||||
<p>Select a note or create a new one</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.ws-note-editor {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
height: 100%;
|
||||
overflow: hidden;
|
||||
background: var(--fs-surface-hover);
|
||||
border-left: 1px solid var(--fs-border-color);
|
||||
}
|
||||
|
||||
/* ── Left rail ── */
|
||||
.note-rail {
|
||||
width: 200px;
|
||||
flex-shrink: 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
overflow: hidden;
|
||||
background: var(--fs-surface-raised);
|
||||
}
|
||||
|
||||
.rail-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.3rem;
|
||||
padding: 0.5rem 0.6rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.rail-title {
|
||||
color: var(--fs-text-tertiary);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
font-size: 0.72rem;
|
||||
font-weight: 500;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.rail-search-input {
|
||||
flex: 1;
|
||||
background: transparent;
|
||||
border: none;
|
||||
font-size: 0.78rem;
|
||||
color: var(--fs-text-primary);
|
||||
min-width: 0;
|
||||
padding: 0;
|
||||
}
|
||||
.rail-search-input:focus { outline: none; }
|
||||
.rail-search-input::-webkit-search-cancel-button { display: none; }
|
||||
|
||||
/* Sits inside the search field: no padding, and must not flex. */
|
||||
.btn-search-clear { padding: 0; flex-shrink: 0; }
|
||||
|
||||
.rail-state {
|
||||
padding: 1rem 0.65rem;
|
||||
font-size: 0.78rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
|
||||
/* Note list */
|
||||
.note-list {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
overflow-y: auto;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.note-row {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
padding: 0.4rem 0.6rem;
|
||||
border-bottom: 1px solid color-mix(in srgb, var(--fs-border-color) 60%, transparent);
|
||||
cursor: pointer;
|
||||
gap: 0.15rem;
|
||||
border-right: 2px solid transparent;
|
||||
transition: background 0.12s;
|
||||
}
|
||||
.note-row:hover { background: color-mix(in srgb, var(--fs-accent) 5%, var(--fs-surface-hover)); }
|
||||
.note-row.active {
|
||||
background: color-mix(in srgb, var(--fs-accent) 8%, var(--fs-surface-hover));
|
||||
border-right-color: var(--fs-accent);
|
||||
}
|
||||
|
||||
.note-row-main {
|
||||
display: flex;
|
||||
align-items: baseline;
|
||||
gap: 0.25rem;
|
||||
}
|
||||
|
||||
.note-row-title {
|
||||
flex: 1;
|
||||
font-size: 0.78rem;
|
||||
font-weight: 500;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
|
||||
.note-row-age {
|
||||
font-size: 0.62rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
white-space: nowrap;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.note-row-tags {
|
||||
display: flex;
|
||||
gap: 0.15rem;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.note-tag-pill {
|
||||
font-size: 0.58rem;
|
||||
color: var(--fs-accent-fg);
|
||||
background: color-mix(in srgb, var(--fs-accent) 10%, transparent);
|
||||
border-radius: 999px;
|
||||
padding: 0 0.3rem;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
max-width: 5rem;
|
||||
}
|
||||
.note-tag-pill.tag-match {
|
||||
background: color-mix(in srgb, var(--fs-accent) 22%, transparent);
|
||||
outline: 1px solid var(--fs-accent);
|
||||
}
|
||||
|
||||
.note-tag-more {
|
||||
font-size: 0.58rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.note-row-actions {
|
||||
display: flex;
|
||||
gap: 0.2rem;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
/* Editor UI */
|
||||
.panel-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
padding: 0.5rem 0.75rem;
|
||||
flex-shrink: 0;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
.editor-header-right {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
margin-left: auto;
|
||||
}
|
||||
|
||||
.unsaved { font-size: 0.72rem; color: var(--fs-text-tertiary); }
|
||||
.saving-txt { font-size: 0.72rem; color: var(--fs-accent); }
|
||||
|
||||
/* Moss action-primary per Hybrid */
|
||||
|
||||
.note-title-input {
|
||||
width: 100%;
|
||||
background: transparent;
|
||||
border: none;
|
||||
font-size: 1.4rem;
|
||||
font-weight: 500;
|
||||
line-height: 1.25;
|
||||
color: var(--fs-text-primary);
|
||||
padding: 0;
|
||||
font-family: 'Fraunces', serif;
|
||||
letter-spacing: -0.01em;
|
||||
}
|
||||
.note-title-input:focus { outline: none; }
|
||||
.note-title-input::placeholder {
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
|
||||
.tag-row {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
gap: 0.4rem;
|
||||
padding: 0.4rem 0.6rem;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.tag-row > :first-child { flex: 1; min-width: 0; }
|
||||
|
||||
.tag-suggestions {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
gap: 0.3rem;
|
||||
padding: 0.35rem 0.6rem;
|
||||
background: color-mix(in srgb, var(--fs-accent) 5%, var(--fs-surface-hover));
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.tag-suggestions-label { font-size: 0.72rem; color: var(--fs-text-tertiary); flex-shrink: 0; }
|
||||
|
||||
.btn-tag-suggestion {
|
||||
background: none;
|
||||
border: 1px solid var(--fs-border-color);
|
||||
border-radius: 999px;
|
||||
padding: 0.15rem 0.55rem;
|
||||
font-size: 0.75rem;
|
||||
color: var(--fs-text-primary);
|
||||
cursor: pointer;
|
||||
}
|
||||
.btn-tag-suggestion:hover { border-color: var(--fs-accent); color: var(--fs-accent); }
|
||||
.btn-tag-suggestion.applied {
|
||||
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
|
||||
border-color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
}
|
||||
|
||||
.link-suggest-strip {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
flex-wrap: wrap;
|
||||
gap: 0.25rem;
|
||||
padding: 0.3rem 0.6rem;
|
||||
background: color-mix(in srgb, var(--fs-accent) 5%, var(--fs-surface-hover));
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.link-suggest-label {
|
||||
font-size: 0.7rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
flex-shrink: 0;
|
||||
font-weight: 500;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
}
|
||||
.link-suggest-chip { display: inline-flex; }
|
||||
|
||||
.btn-chip-link {
|
||||
background: none;
|
||||
border: 1px solid var(--fs-accent);
|
||||
border-radius: 999px;
|
||||
padding: 0.1rem 0.45rem;
|
||||
font-size: 0.7rem;
|
||||
color: var(--fs-accent);
|
||||
cursor: pointer;
|
||||
font-family: monospace;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.btn-chip-link:hover { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); }
|
||||
|
||||
</style>
|
||||
@@ -1,618 +0,0 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, computed, onMounted } from "vue";
|
||||
import { RouterLink } from "vue-router";
|
||||
import { apiGet, apiPatch, apiPost, apiDelete } from "@/api/client";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
import TaskLogSection from "@/components/TaskLogSection.vue";
|
||||
import KindBadge from "@/components/KindBadge.vue";
|
||||
import type { TaskKind } from "@/types/note";
|
||||
import { renderMarkdown } from "@/utils/markdown";
|
||||
import { Trash2, X } from "lucide-vue-next";
|
||||
import { relativeTimeOrDate } from "@/composables/useRelativeTime";
|
||||
|
||||
const props = defineProps<{ projectId: number }>();
|
||||
|
||||
const toast = useToastStore();
|
||||
|
||||
interface Milestone {
|
||||
id: number;
|
||||
title: string;
|
||||
status: string;
|
||||
order_index: number;
|
||||
}
|
||||
|
||||
interface Task {
|
||||
id: number;
|
||||
title: string;
|
||||
status: string;
|
||||
priority: string;
|
||||
milestone_id: number | null;
|
||||
due_date: string | null;
|
||||
updated_at: string;
|
||||
body?: string;
|
||||
task_kind?: TaskKind;
|
||||
}
|
||||
|
||||
const tasks = ref<Task[]>([]);
|
||||
const milestones = ref<Milestone[]>([]);
|
||||
const loading = ref(false);
|
||||
|
||||
// Detail slide-over
|
||||
const activeTask = ref<Task | null>(null);
|
||||
const taskBody = ref("");
|
||||
const loadingBody = ref(false);
|
||||
|
||||
// Collapsed milestone groups (set of milestone IDs, null = "No Milestone" group)
|
||||
const collapsedGroups = ref<Set<number | null>>(new Set());
|
||||
|
||||
// New task quick-add
|
||||
const newTaskTitle = ref("");
|
||||
const addingTask = ref(false);
|
||||
|
||||
const STATUS_CYCLE: Record<string, string> = {
|
||||
todo: "in_progress",
|
||||
in_progress: "done",
|
||||
done: "todo",
|
||||
};
|
||||
|
||||
const STATUS_ICON: Record<string, string> = {
|
||||
todo: "○",
|
||||
in_progress: "▶",
|
||||
done: "✓",
|
||||
};
|
||||
|
||||
const PRIORITY_CLASS: Record<string, string> = {
|
||||
high: "priority-high",
|
||||
medium: "priority-medium",
|
||||
low: "priority-low",
|
||||
};
|
||||
|
||||
function isRowOverdue(task: Task): boolean {
|
||||
if (!task.due_date || task.status === "done") return false;
|
||||
return task.due_date < new Date().toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
// Group tasks by milestone, "No Milestone" first
|
||||
const groupedTasks = computed(() => {
|
||||
const noMs = tasks.value.filter((t) => t.milestone_id === null);
|
||||
const msGroups = milestones.value.map((ms) => ({
|
||||
milestone: ms,
|
||||
tasks: tasks.value.filter((t) => t.milestone_id === ms.id),
|
||||
}));
|
||||
return { noMilestone: noMs, milestoneGroups: msGroups };
|
||||
});
|
||||
|
||||
async function loadAll() {
|
||||
loading.value = true;
|
||||
try {
|
||||
const [tasksRes, msRes] = await Promise.all([
|
||||
apiGet<{ notes: Task[] }>(`/api/projects/${props.projectId}/notes?type=task&limit=200`),
|
||||
apiGet<{ milestones: Milestone[] }>(`/api/projects/${props.projectId}/milestones`),
|
||||
]);
|
||||
tasks.value = tasksRes.notes ?? [];
|
||||
milestones.value = msRes.milestones ?? [];
|
||||
} catch {
|
||||
toast.show("Failed to load tasks", "error");
|
||||
} finally {
|
||||
loading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function cycleStatus(task: Task, e: Event) {
|
||||
e.stopPropagation();
|
||||
const next = STATUS_CYCLE[task.status] ?? "todo";
|
||||
try {
|
||||
await apiPatch(`/api/tasks/${task.id}/status`, { status: next });
|
||||
task.status = next;
|
||||
if (activeTask.value?.id === task.id) activeTask.value.status = next;
|
||||
} catch {
|
||||
toast.show("Failed to update status", "error");
|
||||
}
|
||||
}
|
||||
|
||||
function toggleGroup(key: number | null) {
|
||||
if (collapsedGroups.value.has(key)) {
|
||||
collapsedGroups.value.delete(key);
|
||||
} else {
|
||||
collapsedGroups.value.add(key);
|
||||
}
|
||||
}
|
||||
|
||||
async function openTask(task: Task) {
|
||||
activeTask.value = task;
|
||||
taskBody.value = task.body ?? "";
|
||||
loadingBody.value = true;
|
||||
try {
|
||||
const full = await apiGet<Task>(`/api/tasks/${task.id}`);
|
||||
taskBody.value = full.body ?? "";
|
||||
// Update cached task so re-opens don't re-fetch if unchanged
|
||||
task.body = full.body;
|
||||
} catch {
|
||||
// Non-critical — body just won't show
|
||||
} finally {
|
||||
loadingBody.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
function closeTask() {
|
||||
activeTask.value = null;
|
||||
taskBody.value = "";
|
||||
deleteConfirmPending.value = false;
|
||||
}
|
||||
|
||||
async function addTask() {
|
||||
const title = newTaskTitle.value.trim();
|
||||
if (!title) return;
|
||||
addingTask.value = true;
|
||||
try {
|
||||
const task = await apiPost<Task>("/api/tasks", {
|
||||
title,
|
||||
project_id: props.projectId,
|
||||
status: "todo",
|
||||
});
|
||||
tasks.value.unshift(task);
|
||||
newTaskTitle.value = "";
|
||||
} catch {
|
||||
toast.show("Failed to create task", "error");
|
||||
} finally {
|
||||
addingTask.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
const deletingTask = ref(false);
|
||||
const deleteConfirmPending = ref(false);
|
||||
const changingMilestone = ref(false);
|
||||
|
||||
async function setMilestone(milestoneId: number | null) {
|
||||
if (!activeTask.value) return;
|
||||
changingMilestone.value = true;
|
||||
try {
|
||||
await apiPatch(`/api/notes/${activeTask.value.id}`, { milestone_id: milestoneId });
|
||||
activeTask.value.milestone_id = milestoneId;
|
||||
// Update in tasks list too
|
||||
const t = tasks.value.find((x) => x.id === activeTask.value!.id);
|
||||
if (t) t.milestone_id = milestoneId;
|
||||
} catch {
|
||||
toast.show("Failed to update milestone", "error");
|
||||
} finally {
|
||||
changingMilestone.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function deleteActiveTask() {
|
||||
if (!activeTask.value) return;
|
||||
if (!deleteConfirmPending.value) {
|
||||
deleteConfirmPending.value = true;
|
||||
return;
|
||||
}
|
||||
deletingTask.value = true;
|
||||
try {
|
||||
await apiDelete(`/api/tasks/${activeTask.value.id}`);
|
||||
tasks.value = tasks.value.filter((t) => t.id !== activeTask.value!.id);
|
||||
activeTask.value = null;
|
||||
deleteConfirmPending.value = false;
|
||||
} catch {
|
||||
toast.show("Failed to delete task", "error");
|
||||
} finally {
|
||||
deletingTask.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
function cancelDeleteTask() {
|
||||
deleteConfirmPending.value = false;
|
||||
}
|
||||
|
||||
onMounted(loadAll);
|
||||
defineExpose({ reload: loadAll });
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="ws-task-panel">
|
||||
|
||||
<!-- Task list — always visible, shrinks when detail is open -->
|
||||
<div :class="['task-list-view', { 'has-detail': !!activeTask }]">
|
||||
<div class="panel-header">
|
||||
<span class="panel-title">Tasks</span>
|
||||
</div>
|
||||
|
||||
<div class="task-add">
|
||||
<input
|
||||
v-model="newTaskTitle"
|
||||
class="task-add-input"
|
||||
placeholder="New task..."
|
||||
@keydown.enter="addTask"
|
||||
/>
|
||||
<button class="btn-primary btn-inline btn-add" :disabled="addingTask || !newTaskTitle.trim()" @click="addTask">+</button>
|
||||
</div>
|
||||
|
||||
<div v-if="loading" class="state-msg">Loading...</div>
|
||||
|
||||
<div v-else class="groups-scroll">
|
||||
<!-- No Milestone group (always first) -->
|
||||
<div class="ms-group">
|
||||
<button class="ms-group-header" @click="toggleGroup(null)">
|
||||
<span class="ms-chevron">{{ collapsedGroups.has(null) ? '▶' : '▼' }}</span>
|
||||
<span class="ms-name">No Milestone</span>
|
||||
<span class="ms-count">{{ groupedTasks.noMilestone.length }}</span>
|
||||
</button>
|
||||
<ul v-show="!collapsedGroups.has(null)" class="task-items">
|
||||
<li
|
||||
v-for="task in groupedTasks.noMilestone"
|
||||
:key="task.id"
|
||||
:class="['task-row', { 'task-active': activeTask?.id === task.id }]"
|
||||
@click="openTask(task)"
|
||||
>
|
||||
<button :class="['status-dot', `status-${task.status}`]" :title="`${task.status} — click to cycle`" @click="cycleStatus(task, $event)">{{ STATUS_ICON[task.status] ?? '○' }}</button>
|
||||
<span v-if="task.priority && task.priority !== 'none'" :class="['priority-dot', PRIORITY_CLASS[task.priority] ?? '']"></span>
|
||||
<span class="task-title" :class="{ done: task.status === 'done' }">{{ task.title }}</span>
|
||||
<KindBadge :kind="task.task_kind" />
|
||||
<span v-if="task.due_date" :class="['task-due', { overdue: isRowOverdue(task) }]">{{ task.due_date }}</span>
|
||||
<span class="task-age">{{ relativeTimeOrDate(task.updated_at) }}</span>
|
||||
</li>
|
||||
<li v-if="groupedTasks.noMilestone.length === 0" class="empty-group">No tasks</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<!-- Milestone groups -->
|
||||
<div v-for="{ milestone, tasks: msTasks } in groupedTasks.milestoneGroups" :key="milestone.id" class="ms-group">
|
||||
<button class="ms-group-header" @click="toggleGroup(milestone.id)">
|
||||
<span class="ms-chevron">{{ collapsedGroups.has(milestone.id) ? '▶' : '▼' }}</span>
|
||||
<span class="ms-name">{{ milestone.title }}</span>
|
||||
<span :class="['ms-status', `ms-status-${milestone.status}`]">{{ milestone.status.replace('_',' ') }}</span>
|
||||
<span class="ms-count">{{ msTasks.length }}</span>
|
||||
</button>
|
||||
<ul v-show="!collapsedGroups.has(milestone.id)" class="task-items">
|
||||
<li
|
||||
v-for="task in msTasks"
|
||||
:key="task.id"
|
||||
:class="['task-row', { 'task-active': activeTask?.id === task.id }]"
|
||||
@click="openTask(task)"
|
||||
>
|
||||
<button :class="['status-dot', `status-${task.status}`]" :title="`${task.status} — click to cycle`" @click="cycleStatus(task, $event)">{{ STATUS_ICON[task.status] ?? '○' }}</button>
|
||||
<span v-if="task.priority && task.priority !== 'none'" :class="['priority-dot', PRIORITY_CLASS[task.priority] ?? '']"></span>
|
||||
<span class="task-title" :class="{ done: task.status === 'done' }">{{ task.title }}</span>
|
||||
<KindBadge :kind="task.task_kind" />
|
||||
<span v-if="task.due_date" :class="['task-due', { overdue: isRowOverdue(task) }]">{{ task.due_date }}</span>
|
||||
<span class="task-age">{{ relativeTimeOrDate(task.updated_at) }}</span>
|
||||
</li>
|
||||
<li v-if="msTasks.length === 0" class="empty-group">No tasks</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Detail pane — bottom split when a task is selected -->
|
||||
<Transition name="detail-fade">
|
||||
<div v-if="activeTask" class="task-detail">
|
||||
<div class="detail-header">
|
||||
<RouterLink :to="`/tasks/${activeTask.id}/edit`" target="_blank" class="btn-text btn-edit-task" title="Open full editor">Edit ↗</RouterLink>
|
||||
<span :class="['status-cycler', `status-${activeTask.status}`]" @click="cycleStatus(activeTask, $event)" title="Click to cycle status">
|
||||
{{ STATUS_ICON[activeTask.status] ?? "○" }} {{ activeTask.status.replace("_", " ") }}
|
||||
</span>
|
||||
<template v-if="deleteConfirmPending">
|
||||
<button class="btn-danger-outline btn-inline btn-delete-confirm" :disabled="deletingTask" @click="deleteActiveTask">{{ deletingTask ? '...' : 'Delete?' }}</button>
|
||||
<button class="btn-text" aria-label="Cancel delete" @click="cancelDeleteTask"><X :size="16" /></button>
|
||||
</template>
|
||||
<button v-else class="btn-text btn-delete-task" title="Delete task" @click="deleteActiveTask">
|
||||
<Trash2 :size="16" />
|
||||
</button>
|
||||
<button class="btn-text btn-close-detail" @click="closeTask" aria-label="Close detail"><X :size="16" /></button>
|
||||
</div>
|
||||
|
||||
<h3 class="detail-title">{{ activeTask.title }}</h3>
|
||||
|
||||
<div class="detail-meta">
|
||||
<span v-if="activeTask.priority && activeTask.priority !== 'none'" class="meta-chip priority">{{ activeTask.priority }}</span>
|
||||
<span v-if="activeTask.due_date" class="meta-chip due">Due {{ activeTask.due_date }}</span>
|
||||
<select
|
||||
class="milestone-select"
|
||||
:value="activeTask.milestone_id ?? ''"
|
||||
:disabled="changingMilestone"
|
||||
@change="setMilestone(($event.target as HTMLSelectElement).value === '' ? null : Number(($event.target as HTMLSelectElement).value))"
|
||||
>
|
||||
<option value="">No Milestone</option>
|
||||
<option v-for="ms in milestones" :key="ms.id" :value="ms.id">{{ ms.title }}</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<div v-if="taskBody || loadingBody" class="detail-body">
|
||||
<div v-if="loadingBody" class="body-loading">…</div>
|
||||
<div v-else class="prose" v-html="renderMarkdown(taskBody)" />
|
||||
</div>
|
||||
|
||||
<div class="detail-log">
|
||||
<TaskLogSection :task-id="activeTask.id" />
|
||||
</div>
|
||||
</div>
|
||||
</Transition>
|
||||
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.ws-task-panel {
|
||||
position: relative;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
height: 100%;
|
||||
overflow: hidden;
|
||||
background: var(--fs-surface-hover);
|
||||
border-right: 1px solid var(--fs-border-color);
|
||||
}
|
||||
|
||||
/* ── List view ── */
|
||||
.task-list-view {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
overflow: hidden;
|
||||
}
|
||||
.task-list-view.has-detail {
|
||||
flex: 0 0 44%;
|
||||
}
|
||||
.task-active {
|
||||
background: color-mix(in srgb, var(--fs-accent) 6%, var(--fs-surface-hover)) !important;
|
||||
}
|
||||
|
||||
.panel-header {
|
||||
padding: 0.6rem 0.75rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.panel-title {
|
||||
color: var(--fs-text-tertiary);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.task-add {
|
||||
display: flex;
|
||||
gap: 0.4rem;
|
||||
padding: 0.45rem 0.6rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.task-add-input {
|
||||
flex: 1;
|
||||
background: var(--fs-surface-page);
|
||||
border: 1px solid var(--fs-border-color);
|
||||
border-radius: 5px;
|
||||
padding: 0.28rem 0.5rem;
|
||||
font-size: 0.83rem;
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
.task-add-input:focus { outline: none; border-color: var(--fs-accent); }
|
||||
|
||||
.btn-add { font-size: 1rem; } /* a '+' glyph, not a label */
|
||||
|
||||
.groups-scroll {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.ms-group {
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
}
|
||||
|
||||
.ms-group-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.4rem;
|
||||
width: 100%;
|
||||
padding: 0.4rem 0.65rem;
|
||||
background: var(--fs-surface-raised);
|
||||
border: none;
|
||||
cursor: pointer;
|
||||
text-align: left;
|
||||
font-size: 0.8rem;
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
.ms-group-header:hover { background: color-mix(in srgb, var(--fs-accent) 8%, var(--fs-surface-hover)); }
|
||||
|
||||
.ms-chevron { font-size: 0.6rem; color: var(--fs-text-tertiary); width: 0.8rem; }
|
||||
.ms-name { flex: 1; font-weight: 500; font-size: 0.8rem; }
|
||||
.ms-count { font-size: 0.72rem; color: var(--fs-text-tertiary); background: var(--fs-surface-page); border-radius: 10px; padding: 0 0.4rem; }
|
||||
|
||||
.ms-status {
|
||||
font-size: 0.68rem;
|
||||
padding: 0.1rem 0.4rem;
|
||||
border-radius: 10px;
|
||||
text-transform: capitalize;
|
||||
}
|
||||
.ms-status-active { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent-fg); }
|
||||
.ms-status-completed { background: color-mix(in srgb, var(--fs-success) 15%, transparent); color: var(--fs-success-fg); }
|
||||
|
||||
.task-items {
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.task-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.4rem;
|
||||
padding: 0.35rem 0.65rem 0.35rem 1.4rem;
|
||||
cursor: pointer;
|
||||
border-bottom: 1px solid color-mix(in srgb, var(--fs-border-color) 50%, transparent);
|
||||
}
|
||||
.task-row:hover { background: color-mix(in srgb, var(--fs-accent) 5%, var(--fs-surface-hover)); }
|
||||
.task-row:last-child { border-bottom: none; }
|
||||
|
||||
.status-dot {
|
||||
flex-shrink: 0;
|
||||
width: 1.35rem;
|
||||
height: 1.35rem;
|
||||
border-radius: 50%;
|
||||
border: 1.5px solid var(--fs-border-color);
|
||||
background: none;
|
||||
cursor: pointer;
|
||||
font-size: 0.62rem;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
.status-dot.status-in_progress { border-color: var(--fs-accent); color: var(--fs-accent); }
|
||||
.status-dot.status-done { border-color: var(--fs-success); color: var(--fs-success); }
|
||||
|
||||
.task-title {
|
||||
flex: 1;
|
||||
font-size: 0.83rem;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
.task-title.done { text-decoration: line-through; color: var(--fs-text-tertiary); }
|
||||
|
||||
.task-age {
|
||||
font-size: 0.68rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
white-space: nowrap;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.empty-group { padding: 0.4rem 1.4rem; font-size: 0.78rem; color: var(--fs-text-tertiary); }
|
||||
|
||||
.state-msg { padding: 1.5rem; text-align: center; font-size: 0.85rem; color: var(--fs-text-tertiary); }
|
||||
|
||||
/* ── Detail pane (bottom split) ── */
|
||||
.task-detail {
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
overflow: hidden;
|
||||
background: var(--fs-surface-hover);
|
||||
border-top: 2px solid var(--fs-border-color);
|
||||
}
|
||||
|
||||
.detail-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.6rem;
|
||||
padding: 0.6rem 0.75rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
/* An interactive CYCLER, not a chip: it is clickable, outlined and
|
||||
transparent. It shared a name with the task chip and was never the same
|
||||
shape (#3132). */
|
||||
.status-cycler {
|
||||
padding: 0.2rem 0.55rem;
|
||||
border-radius: 12px;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 500;
|
||||
cursor: pointer;
|
||||
border: 1.5px solid var(--fs-border-color);
|
||||
background: none;
|
||||
text-transform: capitalize;
|
||||
user-select: none;
|
||||
margin-left: auto;
|
||||
}
|
||||
.status-cycler.status-in_progress { border-color: var(--fs-accent); color: var(--fs-accent-fg); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
|
||||
.status-cycler.status-done { border-color: var(--fs-success); color: var(--fs-success-fg); background: color-mix(in srgb, var(--fs-success) 10%, transparent); }
|
||||
|
||||
.btn-edit-task { margin-left: 0.25rem; }
|
||||
.btn-edit-task:hover { text-decoration: underline; }
|
||||
|
||||
.detail-body {
|
||||
padding: 0.5rem 0.75rem 0.5rem;
|
||||
border-bottom: 1px solid var(--fs-border-color);
|
||||
flex-shrink: 0;
|
||||
max-height: 40%;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.body-loading {
|
||||
font-size: 0.8rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
|
||||
.detail-body .prose {
|
||||
font-size: 0.83rem;
|
||||
line-height: 1.5;
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
|
||||
.btn-delete-task { margin-left: 0.25rem; }
|
||||
.btn-delete-task:hover { color: var(--fs-action-destructive); }
|
||||
|
||||
.btn-delete-confirm { margin-left: 0.25rem; }
|
||||
|
||||
|
||||
.detail-meta {
|
||||
display: flex;
|
||||
gap: 0.4rem;
|
||||
padding: 0 0.75rem 0.5rem;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.meta-chip {
|
||||
font-size: 0.72rem;
|
||||
padding: 0.15rem 0.5rem;
|
||||
border-radius: 10px;
|
||||
background: var(--fs-surface-page);
|
||||
border: 1px solid var(--fs-border-color);
|
||||
color: var(--fs-text-tertiary);
|
||||
text-transform: capitalize;
|
||||
}
|
||||
|
||||
.milestone-select {
|
||||
font-size: 0.72rem;
|
||||
padding: 0.15rem 0.4rem;
|
||||
border-radius: 10px;
|
||||
background: var(--fs-surface-page);
|
||||
border: 1px solid var(--fs-border-color);
|
||||
color: var(--fs-text-tertiary);
|
||||
cursor: pointer;
|
||||
max-width: 140px;
|
||||
}
|
||||
.milestone-select:disabled { opacity: 0.5; cursor: default; }
|
||||
.milestone-select:focus { outline: none; border-color: var(--fs-accent); }
|
||||
|
||||
.detail-log {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
padding: 0 0.6rem 0.6rem;
|
||||
border-top: 1px solid var(--fs-border-color);
|
||||
}
|
||||
|
||||
/* Detail fade transition */
|
||||
.detail-fade-enter-active,
|
||||
.detail-fade-leave-active { transition: opacity 0.15s; }
|
||||
.detail-fade-enter-from,
|
||||
.detail-fade-leave-to { opacity: 0; }
|
||||
|
||||
/* Priority dots on task rows */
|
||||
.priority-dot {
|
||||
width: 6px;
|
||||
height: 6px;
|
||||
border-radius: 2px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.priority-high { background: #ef4444; }
|
||||
.priority-medium { background: #f59e0b; }
|
||||
.priority-low { background: #3b82f6; }
|
||||
|
||||
/* Due date on task rows */
|
||||
.task-due {
|
||||
font-size: 0.65rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
white-space: nowrap;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.task-due.overdue {
|
||||
color: var(--fs-error);
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
/* Close detail button */
|
||||
.btn-close-detail { margin-left: 0.2rem; }
|
||||
</style>
|
||||
@@ -4,9 +4,9 @@
|
||||
* full. Not on the list row: a history entry point there would compete with
|
||||
* the row's actual job.
|
||||
*
|
||||
* A SIBLING OF HistoryPanel.vue, NOT A REUSE OF IT, and the reason is in its
|
||||
* props: `noteId` + `currentBody`, a `NoteVersion` carrying tags and pin
|
||||
* columns, a fetch of /api/notes/…, a `restore` emit, and pin/unpin buttons.
|
||||
* A SIBLING OF VersionHistorySection.vue (a note's history), NOT A REUSE OF
|
||||
* IT, and the reason is in its props: `noteId` + `currentBody`, a note
|
||||
* version carrying tags, a fetch of /api/notes/…, and a `restore` emit.
|
||||
* Every one of those is note-shaped. Rules have no tags, no pins, and
|
||||
* deliberately no restore, and a rule's text is EIGHT fields rather than one
|
||||
* body — which changes the central question from "what changed" to "which
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
* Line diff — one copy, for every surface that shows what changed.
|
||||
*
|
||||
* WHY THIS FILE EXISTS. The one LCS diff walk, shared by `useAssist.ts`,
|
||||
* `HistoryPanel.vue`, `VersionHistorySection.vue` and the rule edit history.
|
||||
* `VersionHistorySection.vue` and the rule edit history.
|
||||
* A copy per consumer means a fix has to be found in N places by someone who
|
||||
* does not know N.
|
||||
*/
|
||||
|
||||
@@ -19,6 +19,7 @@ import {
|
||||
type FamilyIdea, type IdeaSnapshot, type IdeaStatus,
|
||||
} from "@/api/family";
|
||||
import FamilyAdoptionMatrix from "@/components/FamilyAdoptionMatrix.vue";
|
||||
import ScrollSentinel from "@/components/ScrollSentinel.vue";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
import { fmtStamp } from "@/utils/dateFormat";
|
||||
import { recordHref } from "@/utils/recordHref";
|
||||
@@ -36,6 +37,8 @@ const loading = ref(true);
|
||||
const failed = ref(false);
|
||||
const moreDecisions = ref(false);
|
||||
const loadingMore = ref(false);
|
||||
const moreFailed = ref(false);
|
||||
const decisionOffset = ref(0);
|
||||
|
||||
// One pending reversal at a time: which row, which act, and its reason.
|
||||
const pending = ref<{ kind: "undo" | "retire"; id: number } | null>(null);
|
||||
@@ -63,7 +66,9 @@ async function load() {
|
||||
criteria.value = ideaData.criteria;
|
||||
conflictOrder.value = ideaData.conflict_order ?? [];
|
||||
decisions.value = log;
|
||||
decisionOffset.value = log.length;
|
||||
moreDecisions.value = log.length === PAGE;
|
||||
moreFailed.value = false;
|
||||
} catch {
|
||||
// Said out loud: a log that failed to load must not read as "nothing
|
||||
// has been decided".
|
||||
@@ -73,14 +78,20 @@ async function load() {
|
||||
}
|
||||
}
|
||||
|
||||
// Older decisions load as the log is scrolled. The log is offset-paged, so a
|
||||
// decision recorded since the first page shifts the rest down by one; a row
|
||||
// already shown is skipped, and the offset still counts what the server sent.
|
||||
async function loadMore() {
|
||||
loadingMore.value = true;
|
||||
moreFailed.value = false;
|
||||
try {
|
||||
const next = await listFamilyDecisions(PAGE, decisions.value.length);
|
||||
decisions.value.push(...next);
|
||||
const next = await listFamilyDecisions(PAGE, decisionOffset.value);
|
||||
const shown = new Set(decisions.value.map((d) => d.id));
|
||||
decisions.value.push(...next.filter((d) => !shown.has(d.id)));
|
||||
decisionOffset.value += next.length;
|
||||
moreDecisions.value = next.length === PAGE;
|
||||
} catch (e: unknown) {
|
||||
toast.show(apiErrorMessage(e, "Could not load more decisions"), "error");
|
||||
} catch {
|
||||
moreFailed.value = true;
|
||||
} finally {
|
||||
loadingMore.value = false;
|
||||
}
|
||||
@@ -343,9 +354,12 @@ onMounted(load);
|
||||
</form>
|
||||
</li>
|
||||
</ul>
|
||||
<button v-if="moreDecisions" class="btn-secondary btn-sm" :disabled="loadingMore" @click="loadMore">
|
||||
{{ loadingMore ? "Loading…" : "Older decisions" }}
|
||||
</button>
|
||||
<ScrollSentinel
|
||||
:cursor="moreDecisions ? String(decisionOffset) : ''"
|
||||
:loading="loadingMore"
|
||||
:failed="moreFailed"
|
||||
@more="loadMore"
|
||||
/>
|
||||
</section>
|
||||
</template>
|
||||
</main>
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, computed, watch, onMounted, onUnmounted, nextTick } from "vue";
|
||||
import { ref, computed, watch, onMounted, onUnmounted } from "vue";
|
||||
import { useRouter } from "vue-router";
|
||||
import { apiGet } from "@/api/client";
|
||||
import type { TaskKind, TaskStatus, TaskPriority } from "@/types/note";
|
||||
@@ -10,6 +10,7 @@ import UsageBadge from "@/components/UsageBadge.vue";
|
||||
import NoteSweepPane from "@/components/NoteSweepPane.vue";
|
||||
import StatusBadge from "@/components/StatusBadge.vue";
|
||||
import PriorityBadge from "@/components/PriorityBadge.vue";
|
||||
import ScrollSentinel from "@/components/ScrollSentinel.vue";
|
||||
import GraphView from "@/views/GraphView.vue";
|
||||
import {
|
||||
FileText,
|
||||
@@ -193,7 +194,7 @@ function onClickOutsideNewNote(e: MouseEvent) {
|
||||
|
||||
const ID_BATCH = 100; // IDs fetched per server round-trip
|
||||
const INITIAL_CONTENT = 50; // items loaded on first render
|
||||
const CONTENT_PAGE = 24; // items loaded per sentinel trigger
|
||||
const CONTENT_PAGE = 24; // items loaded each time the end of the feed nears
|
||||
const REFILL_THRESHOLD = 48; // fetch more IDs when queue drops below this
|
||||
|
||||
const items = ref<KnowledgeItem[]>([]);
|
||||
@@ -214,11 +215,21 @@ const totalKnowledge = ref(0); // total matching items on server
|
||||
const allIdsFetched = ref(false); // no more ID pages to fetch
|
||||
const idsFetching = ref(false); // ID batch request in-flight
|
||||
const contentFetching = ref(false); // content batch request in-flight
|
||||
const sentinelVisible = ref(false); // updated by IntersectionObserver
|
||||
const moreFailed = ref(false); // the last page asked for did not load
|
||||
|
||||
const loading = computed(() => idsFetching.value || contentFetching.value);
|
||||
|
||||
let fetchGen = 0; // incremented on each reset to invalidate stale responses
|
||||
const feedGen = ref(0); // fetchGen, readable by the template
|
||||
|
||||
// What the next page is keyed on: the feed it belongs to and how many of its ids
|
||||
// have been loaded. Ids, not items: a batch can return fewer records than it
|
||||
// asked for, and the feed must still move on. Empty once every id is loaded.
|
||||
const nextCursor = computed(() =>
|
||||
idQueue.value.length > 0 || !allIdsFetched.value
|
||||
? `${feedGen.value}:${idOffset.value - idQueue.value.length}`
|
||||
: "",
|
||||
);
|
||||
|
||||
function buildFilterParams(): URLSearchParams {
|
||||
const p = new URLSearchParams();
|
||||
@@ -244,7 +255,9 @@ async function fetchIdBatch(gen: number) {
|
||||
idOffset.value += data.ids.length;
|
||||
totalKnowledge.value = data.total;
|
||||
if (!data.has_more) allIdsFetched.value = true;
|
||||
} catch { /* silent */ }
|
||||
} catch {
|
||||
if (gen === fetchGen) moreFailed.value = true;
|
||||
}
|
||||
finally { if (gen === fetchGen) idsFetching.value = false; }
|
||||
}
|
||||
|
||||
@@ -262,26 +275,38 @@ async function loadNextContent(gen: number, count: number) {
|
||||
if (idQueue.value.length < REFILL_THRESHOLD && !allIdsFetched.value && !idsFetching.value) {
|
||||
fetchIdBatch(gen);
|
||||
}
|
||||
} catch { /* silent */ }
|
||||
finally {
|
||||
if (gen !== fetchGen) return;
|
||||
contentFetching.value = false;
|
||||
// Sentinel may still be visible if new items didn't push it off screen
|
||||
await nextTick();
|
||||
if (sentinelVisible.value && idQueue.value.length > 0 && gen === fetchGen) {
|
||||
loadNextContent(gen, CONTENT_PAGE);
|
||||
} catch {
|
||||
// The ids go back to the front of the queue, so Try again asks for the
|
||||
// same records rather than skipping past them.
|
||||
if (gen === fetchGen) {
|
||||
idQueue.value.unshift(...toLoad);
|
||||
moreFailed.value = true;
|
||||
}
|
||||
}
|
||||
finally {
|
||||
if (gen === fetchGen) contentFetching.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
// The next page of the feed: more ids first when the queue has run dry.
|
||||
async function loadMore() {
|
||||
const gen = fetchGen;
|
||||
moreFailed.value = false;
|
||||
if (idQueue.value.length === 0) await fetchIdBatch(gen);
|
||||
if (gen !== fetchGen) return;
|
||||
await loadNextContent(gen, CONTENT_PAGE);
|
||||
}
|
||||
|
||||
async function reset() {
|
||||
fetchGen++;
|
||||
feedGen.value = fetchGen;
|
||||
const gen = fetchGen;
|
||||
items.value = [];
|
||||
idQueue.value = [];
|
||||
idOffset.value = 0;
|
||||
totalKnowledge.value = 0;
|
||||
allIdsFetched.value = false;
|
||||
moreFailed.value = false;
|
||||
await fetchIdBatch(gen);
|
||||
if (gen !== fetchGen) return;
|
||||
await loadNextContent(gen, INITIAL_CONTENT);
|
||||
@@ -294,23 +319,16 @@ async function fetchTags() {
|
||||
} catch { /* silent */ }
|
||||
}
|
||||
|
||||
async function resetAndReobserve() {
|
||||
await reset();
|
||||
await nextTick();
|
||||
setupObserver();
|
||||
}
|
||||
|
||||
function onSearchInput() {
|
||||
if (searchDebounce) clearTimeout(searchDebounce);
|
||||
searchDebounce = setTimeout(() => resetAndReobserve(), 380);
|
||||
searchDebounce = setTimeout(() => reset(), 380);
|
||||
}
|
||||
|
||||
watch([activeType, sortMode], () => resetAndReobserve());
|
||||
// Closing the sweep remounts the feed, and with it the scroll sentinel — a
|
||||
// fresh element the old observer is not watching. Without this the list loads
|
||||
// its first page and then never loads another.
|
||||
watch(sweepActive, (open) => { if (!open) resetAndReobserve(); });
|
||||
watch(activeTag, () => { fetchCounts(); resetAndReobserve(); });
|
||||
watch([activeType, sortMode], () => reset());
|
||||
// Closing the sweep starts the feed over, so a note confirmed in the sweep is
|
||||
// shown as it now stands.
|
||||
watch(sweepActive, (open) => { if (!open) reset(); });
|
||||
watch(activeTag, () => { fetchCounts(); reset(); });
|
||||
|
||||
// ─── Today bar ────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -377,24 +395,6 @@ function formatDate(iso: string): string {
|
||||
return d.toLocaleDateString(undefined, { month: "short", day: "numeric", year: "numeric" });
|
||||
}
|
||||
|
||||
// ─── Sentinel observer ────────────────────────────────────────────────────────
|
||||
|
||||
const sentinelEl = ref<HTMLElement | null>(null);
|
||||
let observer: IntersectionObserver | null = null;
|
||||
|
||||
function setupObserver() {
|
||||
observer?.disconnect();
|
||||
if (!sentinelEl.value) return;
|
||||
observer = new IntersectionObserver(
|
||||
([entry]) => {
|
||||
sentinelVisible.value = entry.isIntersecting;
|
||||
if (entry.isIntersecting) loadNextContent(fetchGen, CONTENT_PAGE);
|
||||
},
|
||||
{ root: null, rootMargin: "200px" }
|
||||
);
|
||||
observer.observe(sentinelEl.value);
|
||||
}
|
||||
|
||||
// ─── Lifecycle ────────────────────────────────────────────────────────────────
|
||||
|
||||
onMounted(async () => {
|
||||
@@ -403,14 +403,11 @@ onMounted(async () => {
|
||||
fetchTags();
|
||||
fetchCounts();
|
||||
fetchTodayBar();
|
||||
await nextTick();
|
||||
setupObserver();
|
||||
});
|
||||
|
||||
onUnmounted(() => {
|
||||
document.removeEventListener('click', onClickOutsideNewNote);
|
||||
if (searchDebounce) clearTimeout(searchDebounce);
|
||||
observer?.disconnect();
|
||||
});
|
||||
</script>
|
||||
|
||||
@@ -583,10 +580,7 @@ onUnmounted(() => {
|
||||
</div>
|
||||
|
||||
<!-- The whole feed stands down while the sweep is open: two answers
|
||||
to two different questions on one screen is neither. Wrapped
|
||||
rather than given an extra v-if branch, because the scroll
|
||||
sentinel lives inside the grid and the observer must not be left
|
||||
holding a ref to something that never renders. -->
|
||||
to two different questions on one screen is neither. -->
|
||||
<template v-if="!sweepActive">
|
||||
<!-- Loading / empty -->
|
||||
<div v-if="loading && items.length === 0" class="knowledge-empty">Loading…</div>
|
||||
@@ -675,10 +669,12 @@ onUnmounted(() => {
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Sentinel — IntersectionObserver triggers next content batch -->
|
||||
<div ref="sentinelEl" class="scroll-sentinel">
|
||||
<span v-if="contentFetching" class="sentinel-loading">Loading…</span>
|
||||
</div>
|
||||
<ScrollSentinel
|
||||
:cursor="nextCursor"
|
||||
:loading="loading"
|
||||
:failed="moreFailed"
|
||||
@more="loadMore"
|
||||
/>
|
||||
</div>
|
||||
</template>
|
||||
</div>
|
||||
@@ -1115,19 +1111,6 @@ onUnmounted(() => {
|
||||
opacity: 0.85;
|
||||
}
|
||||
|
||||
/* ── Sentinel ────────────────────────────────────────────── */
|
||||
.scroll-sentinel {
|
||||
grid-column: 1 / -1;
|
||||
height: 40px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
.sentinel-loading {
|
||||
font-size: 0.8rem;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
|
||||
/* ── Graph panel ─────────────────────────────────────────── */
|
||||
.graph-panel {
|
||||
width: 500px;
|
||||
|
||||
@@ -9,8 +9,10 @@ import { apiGet, apiPost, apiPut, apiDelete, listGroups, createGroup, deleteGrou
|
||||
import type { User } from "@/types/auth";
|
||||
import PaginationBar from "@/components/PaginationBar.vue";
|
||||
import MomentsSettings from "@/components/MomentsSettings.vue";
|
||||
import ReverseProxySettings from "@/components/ReverseProxySettings.vue";
|
||||
import ReplyShapesSettings from "@/components/ReplyShapesSettings.vue";
|
||||
import BackupScheduleSettings from "@/components/BackupScheduleSettings.vue";
|
||||
import RestoreBackupDialog from "@/components/RestoreBackupDialog.vue";
|
||||
import { formatBytes } from "@/utils/bytes";
|
||||
import { fmtDate, fmtLogStamp } from "@/utils/dateFormat";
|
||||
import { fetchVersion, type VersionPayload } from "@/api/version";
|
||||
@@ -159,16 +161,10 @@ const kbWritePathEnabled = ref(true);
|
||||
// code embeddings sit on a much higher similarity floor than prose, so 0.55
|
||||
// lets unrelated code through. Shares top-k, not the threshold.
|
||||
const kbWritePathThreshold = ref("0.68");
|
||||
const kbRuleHintThreshold = ref("0.72");
|
||||
// One bar for every rule hint — prompt, command and edit alike. They search
|
||||
// the same rules, and what limits each is its budget, not the bar.
|
||||
const kbRuleThreshold = ref("0.70");
|
||||
const kbCheckpointThreshold = ref("0.80");
|
||||
// The two ACT arms do not share a bar. A write-path query is a code payload; a
|
||||
// pre-tool query is a shell command, often under a dozen words — less text,
|
||||
// less signal, lower scores for the same relevance — so one shared bar makes
|
||||
// them behave like different subsystems.
|
||||
const kbToolRuleThreshold = ref("0.68");
|
||||
// And the prompt boundary is a third query shape again — the operator's own
|
||||
// prose rather than anything a tool produced.
|
||||
const kbPromptRuleThreshold = ref("0.72");
|
||||
// The reply arm: its bar is the bar at which a finished reply
|
||||
// is HELD for one read, so it sits with the checkpoint, not with the hints.
|
||||
const kbReplyRuleThreshold = ref("0.8");
|
||||
@@ -367,14 +363,10 @@ async function saveKbInject() {
|
||||
// Same `|| default` reasoning: falling back to 0 would surface every
|
||||
// snippet in the corpus on every edit, which is the failure this knob fixes.
|
||||
const wpT = asBar(kbWritePathThreshold.value, 0.68);
|
||||
// Same `|| default` reasoning again, and it bites harder here: a rule hint
|
||||
// fires on every write, so a fallback of 0 would attach a standing rule to
|
||||
// every edit in the session.
|
||||
const rhT = asBar(kbRuleHintThreshold.value, 0.72);
|
||||
// Same `|| default` guard, and the same reason: this arm fires before every
|
||||
// Bash call, so a fallback of 0 would put a rule in front of every command.
|
||||
const trT = asBar(kbToolRuleThreshold.value, 0.68);
|
||||
const prT = asBar(kbPromptRuleThreshold.value, 0.72);
|
||||
// Same `|| default` reasoning again, and it bites harder here: rule hints
|
||||
// fire on every edit and before every command, so a fallback of 0 would put
|
||||
// a standing rule beside each of them.
|
||||
const ruleT = asBar(kbRuleThreshold.value, 0.7);
|
||||
// The checkpoint bar, and the `|| default` guard matters most here of all:
|
||||
// this is the only number that can STOP a call, so a fallback of 0 would
|
||||
// hold the first command of every session behind whatever ranked first.
|
||||
@@ -409,10 +401,8 @@ async function saveKbInject() {
|
||||
kbGateNoteOverlapFloor.value = String(gateOverlap);
|
||||
kbPlanMatchThreshold.value = String(planT);
|
||||
kbWritePathThreshold.value = String(wpT);
|
||||
kbRuleHintThreshold.value = String(rhT);
|
||||
kbRuleThreshold.value = String(ruleT);
|
||||
kbCheckpointThreshold.value = String(cpT);
|
||||
kbToolRuleThreshold.value = String(trT);
|
||||
kbPromptRuleThreshold.value = String(prT);
|
||||
kbReplyRuleThreshold.value = String(ryT);
|
||||
savingKbInject.value = true;
|
||||
kbInjectSaved.value = false;
|
||||
@@ -426,20 +416,15 @@ async function saveKbInject() {
|
||||
// measurements that split them.
|
||||
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
|
||||
kb_writepath_threshold: String(wpT),
|
||||
// A THIRD corpus with a third bar — see RULEHINT_DEFAULT_THRESHOLD
|
||||
// in services/plugin_context.py for why rules cannot share the
|
||||
// code threshold any more than code could share the prose one.
|
||||
kb_rulehint_threshold: String(rhT),
|
||||
// A third corpus with a third bar — see RULE_FLOOR_DEFAULT in
|
||||
// services/retrieval_pipeline.py for why every rule hint shares it.
|
||||
kb_rule_threshold: String(ruleT),
|
||||
// The one bar that stops a call rather than annotating it. Its own key,
|
||||
// never derived from the two rule bars above: it answers a different
|
||||
// never derived from the rule bar above: it answers a different
|
||||
// question of the same scores — not "is this worth showing" but "is the
|
||||
// corpus confident enough to be read first" — and a number that moves
|
||||
// when another moves is a number nobody can reason about.
|
||||
kb_checkpoint_threshold: String(cpT),
|
||||
// A FOURTH and FIFTH bar, and they are separate keys on purpose: one
|
||||
// number cannot serve arms whose queries are different shapes. Moving one must not move the others.
|
||||
kb_toolrule_threshold: String(trT),
|
||||
kb_promptrule_threshold: String(prT),
|
||||
// The reply backstop's own bar: it HOLDS a reply, so like the
|
||||
// checkpoint it is never derived from a hint bar.
|
||||
kb_replyrule_threshold: String(ryT),
|
||||
@@ -484,7 +469,7 @@ const confirmNewPassword = ref("");
|
||||
const changingPassword = ref(false);
|
||||
const invalidatingSessions = ref(false);
|
||||
const exporting = ref(false);
|
||||
const restoring = ref(false);
|
||||
const showRestore = ref(false);
|
||||
// Backup, export and restore walk the whole store, so they are slow BY DESIGN
|
||||
// and the client's ordinary 30s default would cut them off mid-work. They are
|
||||
// still bounded — every wait has a deadline, not necessarily a short one — and
|
||||
@@ -526,7 +511,6 @@ async function copyCommit() {
|
||||
commitCopied.value = true;
|
||||
setTimeout(() => { commitCopied.value = false; }, 2000);
|
||||
}
|
||||
const restoreFileInput = ref<HTMLInputElement | null>(null);
|
||||
|
||||
// Migrate stored "admin" → "config"; unknown tabs fall back to "general"
|
||||
const VALID_TABS = new Set(["general", "account", "notifications", "integrations", "data", "apikeys", "config", "users", "logs", "groups", "areas", "platforms"]);
|
||||
@@ -864,14 +848,8 @@ onMounted(async () => {
|
||||
if (allSettings.kb_checkpoint_threshold !== undefined) {
|
||||
kbCheckpointThreshold.value = allSettings.kb_checkpoint_threshold;
|
||||
}
|
||||
if (allSettings.kb_rulehint_threshold !== undefined) {
|
||||
kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
|
||||
}
|
||||
if (allSettings.kb_toolrule_threshold !== undefined) {
|
||||
kbToolRuleThreshold.value = allSettings.kb_toolrule_threshold;
|
||||
}
|
||||
if (allSettings.kb_promptrule_threshold !== undefined) {
|
||||
kbPromptRuleThreshold.value = allSettings.kb_promptrule_threshold;
|
||||
if (allSettings.kb_rule_threshold !== undefined) {
|
||||
kbRuleThreshold.value = allSettings.kb_rule_threshold;
|
||||
}
|
||||
if (allSettings.kb_replyrule_threshold !== undefined) {
|
||||
kbReplyRuleThreshold.value = allSettings.kb_replyrule_threshold;
|
||||
@@ -1102,10 +1080,6 @@ async function exportNotes(format: "markdown" | "json") {
|
||||
}
|
||||
}
|
||||
|
||||
function triggerRestoreUpload() {
|
||||
restoreFileInput.value?.click();
|
||||
}
|
||||
|
||||
async function saveNotifications() {
|
||||
savingNotifications.value = true;
|
||||
notificationsSaved.value = false;
|
||||
@@ -1302,42 +1276,6 @@ async function runDbMaintenanceNow() {
|
||||
}
|
||||
}
|
||||
|
||||
async function handleRestoreFile(event: Event) {
|
||||
const file = (event.target as HTMLInputElement).files?.[0];
|
||||
if (!file) return;
|
||||
restoring.value = true;
|
||||
try {
|
||||
// The scheduled backup's .tar goes up as it is; the JSON download is
|
||||
// parsed first so a damaged file fails here, before the upload.
|
||||
const isTar = file.name.toLowerCase().endsWith(".tar");
|
||||
const res = await fetch("/api/admin/restore", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": isTar ? "application/x-tar" : "application/json" },
|
||||
body: isTar ? file : JSON.stringify(JSON.parse(await file.text())),
|
||||
signal: bulkDeadline(),
|
||||
});
|
||||
if (res.status === 413) {
|
||||
throw new Error(
|
||||
"the file is bigger than the web upload allows. Restore it with " +
|
||||
"`python -m grimoire.restore <file>` inside the Grimoire container instead.",
|
||||
);
|
||||
}
|
||||
if (!res.ok) {
|
||||
const body = await res.json().catch(() => ({ error: `Error ${res.status}` }));
|
||||
throw new Error((body as Record<string, string>).error || `Error ${res.status}`);
|
||||
}
|
||||
const result = await res.json();
|
||||
toastStore.show(
|
||||
`Restored ${result.stats?.users ?? 0} users, ${result.stats?.notes ?? 0} notes, ${result.stats?.conversations ?? 0} conversations`
|
||||
);
|
||||
} catch (e) {
|
||||
toastStore.show("Restore failed: " + (e as Error).message, "error");
|
||||
} finally {
|
||||
restoring.value = false;
|
||||
if (restoreFileInput.value) restoreFileInput.value.value = "";
|
||||
}
|
||||
}
|
||||
|
||||
async function testSearch() {
|
||||
const q = searchQuery.value.trim();
|
||||
if (!q) return;
|
||||
@@ -1772,10 +1710,10 @@ async function deleteUser(userId: number) {
|
||||
<p class="field-hint">How many snippets and issues one edit may be offered (1–10). Reach for this rather than the threshold when the hint feels long: lowering it keeps the best matches and drops the tail, while raising the bar above throws away good matches along with weak ones.</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-rulehint-threshold">Standing-rule confidence threshold (0–1)</label>
|
||||
<label for="kb-rule-threshold">Standing-rule confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-rulehint-threshold"
|
||||
v-model="kbRuleHintThreshold"
|
||||
id="kb-rule-threshold"
|
||||
v-model="kbRuleThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
@@ -1784,14 +1722,14 @@ async function deleteUser(userId: number) {
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The same hint can mention a standing rule whose trigger resembles the
|
||||
code being written. <em>Every</em> rule is eligible — nothing is
|
||||
preloaded any more, so this is the only way a rule reaches a write.
|
||||
Stricter than the threshold above, because there are far fewer rules
|
||||
than snippets: with a small set something always ranks first, so the
|
||||
bar has to carry more of the judgement. If rules arrive unread, or one
|
||||
you needed never showed up, that is Claude's to notice and correct
|
||||
from the telemetry — and the budget below is usually the better lever.
|
||||
The bar for a standing rule or preference to be offered at all,
|
||||
whether against the code being written, a command about to run, or
|
||||
what you just said. <em>Every</em> rule is eligible — nothing is
|
||||
preloaded, so this is how a rule reaches the work. One bar serves all
|
||||
three: they search the same rules, and how many each may show is set
|
||||
by its own budget below, which is usually the better lever. If rules
|
||||
arrive unread, or one you needed never showed up, that is Claude's to
|
||||
notice and correct from the telemetry.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
@@ -1834,26 +1772,6 @@ async function deleteUser(userId: number) {
|
||||
only — file edits are never held.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-toolrule-threshold">Command confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-toolrule-threshold"
|
||||
v-model="kbToolRuleThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The bar for a rule surfacing before a <em>command</em> runs, where the
|
||||
query is the command text rather than code. Lower than the one above
|
||||
on purpose: a shell command is short, so it scores lower for the same
|
||||
relevance — at a shared bar this arm spoke on 2% of calls against the
|
||||
write path's 37%.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-toolrule-topk">Max rules per command</label>
|
||||
<input
|
||||
@@ -1868,26 +1786,6 @@ async function deleteUser(userId: number) {
|
||||
/>
|
||||
<p class="field-hint">How many standing rules one command may be shown (1–10). This is the busiest arm there is — it fires before every command — so its budget is the one most worth keeping small.</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-promptrule-threshold">Prompt confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-promptrule-threshold"
|
||||
v-model="kbPromptRuleThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The bar for a rule or preference surfacing against <em>what you just
|
||||
said</em>, before anything is done. It is the only moment that reaches
|
||||
a rule about how to answer rather than how to act, so it runs once a
|
||||
turn rather than once a tool call. A third query shape — your prose,
|
||||
not a command or a file — which is why it carries its own number.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-promptrule-topk">Max rules per prompt</label>
|
||||
<input
|
||||
@@ -2492,17 +2390,11 @@ async function deleteUser(userId: number) {
|
||||
<button class="btn-secondary" @click="exportData('full')" :disabled="exporting">
|
||||
{{ exporting ? "Exporting..." : "Full Backup" }}
|
||||
</button>
|
||||
<button class="btn-secondary btn-warn" @click="triggerRestoreUpload" :disabled="restoring">
|
||||
{{ restoring ? "Restoring..." : "Restore from Backup" }}
|
||||
<button class="btn-secondary btn-warn" @click="showRestore = true">
|
||||
Restore from Backup
|
||||
</button>
|
||||
<input
|
||||
ref="restoreFileInput"
|
||||
type="file"
|
||||
accept=".json,.tar"
|
||||
class="hidden-file-input"
|
||||
@change="handleRestoreFile"
|
||||
/>
|
||||
</div>
|
||||
<RestoreBackupDialog v-if="showRestore" @close="showRestore = false" />
|
||||
</section>
|
||||
<section class="settings-section full-width">
|
||||
<h2>Scheduled backups</h2>
|
||||
@@ -2773,6 +2665,16 @@ async function deleteUser(userId: number) {
|
||||
<div v-else class="empty-msg">Nothing asked yet.</div>
|
||||
</section>
|
||||
|
||||
<section class="settings-section full-width">
|
||||
<h2>Reverse proxy</h2>
|
||||
<p class="section-desc">
|
||||
How many proxies a request passes through before it reaches Grimoire. Sign-in rate
|
||||
limits and the audit log key on the visitor's address, read through this count; if it
|
||||
is too low, every visitor reads as the proxy and one run of bad sign-ins blocks everyone.
|
||||
</p>
|
||||
<ReverseProxySettings />
|
||||
</section>
|
||||
|
||||
<section class="settings-section full-width">
|
||||
<h2>Application URL</h2>
|
||||
<p class="section-desc">
|
||||
@@ -3660,7 +3562,6 @@ async function deleteUser(userId: number) {
|
||||
flex-wrap: wrap;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
.hidden-file-input { display: none; }
|
||||
|
||||
/* Checkboxes */
|
||||
.checkbox-field { margin-bottom: 1rem; }
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "grimoire",
|
||||
"description": "Grimoire for Claude Code: connects the grimoire MCP server, adds the hooks that deliver live project state and relevant records at the right moment, ships the shared client-neutral Grimoire skills (using-grimoire, writing-plans, reporting-back, systematic-debugging, verification, brainstorming, reusing-code, shape-accounting, family-canon), and syncs your saved Grimoire Processes as skills (/grimoire:sync).",
|
||||
"version": "2026.10.22.1945",
|
||||
"version": "2026.11.05.0301",
|
||||
"author": {
|
||||
"name": "Bryan Van Deusen"
|
||||
},
|
||||
|
||||
+2
-1
@@ -15,7 +15,7 @@ another one means adding files, not moving or rewriting any.
|
||||
|---|---|---|
|
||||
| **The skills** | `plugin/skills/*/SKILL.md` | Agent Skills (the open SKILL.md format). They state every Grimoire reflex in full and name no client. `tests/test_guidance_ownership.py` fails if a skill names a particular client, or references anything outside its own folder. Every client package ships this folder verbatim. |
|
||||
| **The MCP server** | `<base URL>/mcp` | HTTP, `Authorization: Bearer <fmcp_ key>`. Its `_INSTRUCTIONS` is a client-neutral orientation — the workflow across tools (≤1,600 chars); each tool's description carries its contract; in-band responses (`placement`, `report_back`, `systems_hint`, the duplicate gate, the guessed-id refusal) fire in every client. |
|
||||
| **The adapter API** | `<base URL>/api/plugin/*` | Endpoints any client's hooks can call with the same key. `GET`, so read scope is enough: `context` (live session state), `retrieve` (rules, preferences and notes for a message), `prior-art` (records and shape-ledger hints for code being written), `tool-rules` (rules for a command about to run), `moment-tools` (which tools reach a mounted moment), `shape-check` (definitions written this turn that nobody has judged), `claim-session` / `release-session` (task claims), `processes` (stored Processes to expose as skills). `POST`, so write scope: `moment` (rules mounted on the moments a tool call reaches), `rule-opened` (a rule was read), `reply-rules` (the finished reply, held for an unopened rule or — when the turn closed a task — for missing completion sections). |
|
||||
| **The adapter API** | `<base URL>/api/plugin/*` | Endpoints any client's hooks can call with the same key. `GET`, so read scope is enough: `context` (live session state), `retrieve` (rules, preferences and notes for a message), `prior-art` (records and shape-ledger hints for code being written), `tool-rules` (rules for a command about to run), `moment-tools` (which tools reach a mounted moment), `shape-check` (definitions written this turn that nobody has judged), `claim-session` / `release-session` (task claims), `write-scope` (which project a pending write lands in, and whether this session may make it), `processes` (stored Processes to expose as skills). `POST`, so write scope: `moment` (rules mounted on the moments a tool call reaches), `rule-opened` (a rule was read), `reply-rules` (the finished reply, held for an unopened rule or — when the turn closed a task — for missing completion sections). |
|
||||
| **The API key** | Grimoire → Settings → MCP Access | One `fmcp_` key per install. Write scope covers the MCP tools and every hook; a read key runs only the `GET` hooks. |
|
||||
|
||||
## Added by each client
|
||||
@@ -39,6 +39,7 @@ another one means adding files, not moving or rewriting any.
|
||||
| `hooks/grimoire_prior_art.sh` | PreToolUse on editor writes: `GET /api/plugin/prior-art`. |
|
||||
| `hooks/grimoire_after_write.sh` | PostToolUse on shell commands: the same check for code written through the shell. |
|
||||
| `hooks/grimoire_tool_rules.sh` | PreToolUse on shell commands: `GET /api/plugin/tool-rules`. |
|
||||
| `hooks/grimoire_project_fence.sh` | PreToolUse on editor writes, shell commands and Grimoire's tools: `GET /api/plugin/write-scope` for an act aimed at another project, answered `ask` with the reason; PostToolUse on `enter_project` remembers the project a session without one entered. |
|
||||
| `hooks/grimoire_moment.sh` | PreToolUse on every tool: the rules mounted on the moments the call reaches, `POST /api/plugin/moment`; skips tools `GET /api/plugin/moment-tools` says reach nothing mounted, and Grimoire's own (their responses carry `moment_rules`). |
|
||||
| `hooks/grimoire_reply_check.sh` | Stop: sends the finished reply to `POST /api/plugin/reply-rules` — the rules mounted on the reply moments, and the reply against every rule's trigger, and — when the turn closed a task — the completion-section check; holds once per rule, with the reason the server returns, and never holds the rewrite. |
|
||||
| `hooks/grimoire_shape_check.sh` | Stop: sends the definitions the turn wrote (the write hooks' `<sid>.written.ids` ledger) to `GET /api/plugin/shape-check`; blocks once, with the reason the server returns, so the agent judges what it built. |
|
||||
|
||||
@@ -79,6 +79,20 @@ On install you'll be asked for:
|
||||
message, as a short menu of titles and matched passages.
|
||||
- `hooks/hooks.json` → PreToolUse hook on `Bash` (`hooks/grimoire_tool_rules.sh`)
|
||||
→ `GET /api/plugin/tool-rules`: the rules that match the command about to run.
|
||||
- `hooks/hooks.json` → PreToolUse hook on editor writes, `Bash` and Grimoire's
|
||||
tools (`hooks/grimoire_project_fence.sh`) → `GET /api/plugin/write-scope`:
|
||||
holds a session to its own project. A session belongs to the project its
|
||||
starting directory resolves to (a `.grimoire` marker, else the git remote's
|
||||
binding), or, started outside any, to the first project it enters. Editing
|
||||
another project's files, a git commit, push, merge, rebase, pull, tag or
|
||||
reset in its repo (from `git -C`, after `cd`, or in a worktree of it), and a
|
||||
Grimoire write that lands in another project each stop on a permission
|
||||
prompt naming that project, in every permission mode. Filing an issue there
|
||||
(`create_task(kind="issue")`) and every read go ahead. An edit in the
|
||||
session's own repo never calls the server. It can't see a non-git shell
|
||||
write into another repo (`sed -i`, a redirect), but it fences that repo's
|
||||
commit and push. When the instance doesn't answer, the act goes ahead and
|
||||
the hook says so once per outage.
|
||||
- `hooks/hooks.json` → PreToolUse hook on every tool (`hooks/grimoire_moment.sh`)
|
||||
→ `POST /api/plugin/moment`: the rules mounted on the moments the call
|
||||
reaches. It skips tools that `GET /api/plugin/moment-tools` says reach
|
||||
|
||||
@@ -119,10 +119,11 @@ combined=""
|
||||
n_files=0
|
||||
while IFS= read -r rel_path; do
|
||||
[ -n "${rel_path:-}" ] || continue
|
||||
# A Bash call that rewrote many files is a refactor or a generator, not a
|
||||
# shape being instantiated; four is enough to name what matters.
|
||||
# Every changed file goes into the end-of-turn ledger, up to forty — a bound
|
||||
# on this hook's run time, past which the call was a generator. The arms
|
||||
# below speak for the first four only (see the check after the ledger).
|
||||
n_files=$((n_files + 1))
|
||||
[ "$n_files" -le 4 ] || break
|
||||
[ "$n_files" -le 40 ] || break
|
||||
file_path="$repo_root/$rel_path"
|
||||
|
||||
# The code just written: the ADDED lines of the uncommitted diff for a
|
||||
@@ -136,14 +137,19 @@ while IFS= read -r rel_path; do
|
||||
fresh="new"
|
||||
fi
|
||||
[ -n "$code" ] || continue
|
||||
# `|| true` inside: an early-exiting `head` must not void its own output.
|
||||
names=$(printf '%s' "$code" | grimoire_defs | sort -u | head -12 || true)
|
||||
# What this write defined, for the end-of-turn question —
|
||||
all_names=$(printf '%s' "$code" | grimoire_defs | sort -u || true)
|
||||
# What this write defined, for the end-of-turn question: every definition,
|
||||
# recorded before the arms below, which skip a write that defines nothing.
|
||||
printf '%s\n' "$names" | grimoire_written_append "$safe_sid" "$rel_path" "$fresh" "$code"
|
||||
printf '%s\n' "$all_names" | grimoire_written_append "$safe_sid" "$rel_path" "$fresh" "$code"
|
||||
# Nothing DEFINED in what was written (prose, data, a call-site edit) →
|
||||
# nothing to say; the arms are about shapes.
|
||||
[ -n "$names" ] || continue
|
||||
[ -n "$all_names" ] || continue
|
||||
# A Bash call that rewrote many files is a refactor or a generator, not a
|
||||
# shape being instantiated; four files and twelve names per file are enough
|
||||
# for the arms to name what matters, and bound their git grep and URL.
|
||||
[ "$n_files" -le 4 ] || continue
|
||||
# `|| true` inside: an early-exiting `head` must not void its own output.
|
||||
names=$(printf '%s\n' "$all_names" | head -12 || true)
|
||||
|
||||
local_lines=$(grimoire_local_dups "$repo_root" "$rel_path" <<< "$names") || local_lines=""
|
||||
local_context=""
|
||||
|
||||
@@ -79,10 +79,16 @@ fi
|
||||
|
||||
# ARM 1 — BY NAME, LOCALLY: does a definition of this already exist
|
||||
# in the repo? (grimoire_local_dups in grimoire_defs.sh carries the why.)
|
||||
#
|
||||
# `all_names` is every definition the write carries; it feeds the end-of-turn
|
||||
# ledger, which must hear about all of them. `names` is the first twelve, and
|
||||
# bounds the queries below — a git grep and a URL per name.
|
||||
all_names=""
|
||||
names=""
|
||||
if [ -n "$code" ]; then
|
||||
all_names=$(printf '%s' "$code" | grimoire_defs | sort -u || true)
|
||||
# `|| true` inside: an early-exiting `head` must not void its own output.
|
||||
names=$(printf '%s' "$code" | grimoire_defs | sort -u | head -12 || true)
|
||||
names=$(printf '%s\n' "$all_names" | head -12 || true)
|
||||
fi
|
||||
|
||||
local_lines=""
|
||||
@@ -204,10 +210,11 @@ if [ -n "$session_id" ]; then
|
||||
# Write onto a path that does not exist yet creates a file, which may be a
|
||||
# unit in its own right (see grimoire_written_append). Written before any
|
||||
# server call: the ask at the end of the turn must not depend on this
|
||||
# write's hint having been answered.
|
||||
# write's hint having been answered. Every definition, not the capped list
|
||||
# the queries use; an Edit that names none records its enclosing one.
|
||||
fresh=""
|
||||
[ -e "$file_path" ] || fresh="new"
|
||||
printf '%s\n' "$shapes" | grimoire_written_append "$safe_sid" "$rel_path" "$fresh" "$code"
|
||||
printf '%s\n' "${all_names:-$shapes}" | grimoire_written_append "$safe_sid" "$rel_path" "$fresh" "$code"
|
||||
idfile="$state_dir/${safe_sid}.ids"
|
||||
syncfile="$state_dir/${safe_sid}.sync.ids"
|
||||
derivefile="$state_dir/${safe_sid}.derive.ids"
|
||||
|
||||
@@ -0,0 +1,246 @@
|
||||
#!/usr/bin/env bash
|
||||
# Grimoire — hold a session to its own project.
|
||||
#
|
||||
# A session belongs to the project its starting directory resolves to
|
||||
# (CLAUDE_PROJECT_DIR: a `.grimoire` marker, else the git remote's binding). A
|
||||
# session that starts outside any project belongs to the first project it
|
||||
# enters. Toward any other project a session may read anything and file an
|
||||
# issue. Every other write stops on a permission prompt naming the other
|
||||
# project, so the operator decides:
|
||||
# - editing a file in another project's repo (Write, Edit, MultiEdit,
|
||||
# NotebookEdit);
|
||||
# - a git command that changes history or a remote there (commit, push,
|
||||
# merge, rebase, pull, cherry-pick, revert, tag, am, reset, worktree add),
|
||||
# wherever it runs from: `git -C DIR`, `cd DIR && git …`, or the shell's
|
||||
# current directory;
|
||||
# - a Grimoire write that lands in another project.
|
||||
#
|
||||
# `ask`, not `deny`. Which projects a session may change is the operator's
|
||||
# call, and `ask` puts it to them each time, in every permission mode,
|
||||
# bypassPermissions included.
|
||||
#
|
||||
# The server says where an act lands (GET /api/plugin/write-scope). Two cases
|
||||
# never reach it:
|
||||
# - a directory that resolves the same way as the session's own, which is
|
||||
# nearly every edit;
|
||||
# - a tool named like a read (READ_PREFIXES in services/write_scope.py, held
|
||||
# equal to FENCE_READ_PREFIXES below by tests/test_write_scope.py).
|
||||
#
|
||||
# What it can't see: a non-git write from the shell (`sed -i`, a redirect)
|
||||
# into another repo. That repo's commit and push are still fenced.
|
||||
#
|
||||
# EXIT 0, ALWAYS. When the server doesn't answer, the act goes ahead and the
|
||||
# hook says once per outage that it couldn't check.
|
||||
#
|
||||
# Run as PostToolUse on enter_project, it records the project a session
|
||||
# without one has entered, and says nothing.
|
||||
set -uo pipefail
|
||||
|
||||
# shellcheck source=plugin/hooks/grimoire_defs.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/grimoire_defs.sh"
|
||||
|
||||
FENCE_READ_PREFIXES="get_ list_ search find_ enter_project what_might_apply resolve_design_system"
|
||||
FENCE_GIT_WRITES="commit push merge rebase pull cherry-pick revert tag am reset"
|
||||
|
||||
event=$(cat 2>/dev/null || true)
|
||||
[ -n "$event" ] || exit 0
|
||||
event_flat=$(printf '%s' "$event" | grimoire_json_flat)
|
||||
[ -n "$event_flat" ] || exit 0
|
||||
|
||||
session_id=$(grimoire_json_pick "$event_flat" '.session_id')
|
||||
tool_name=$(grimoire_json_pick "$event_flat" '.tool_name')
|
||||
hook_event=$(grimoire_json_pick "$event_flat" '.hook_event_name')
|
||||
cwd=$(grimoire_json_pick "$event_flat" '.cwd')
|
||||
# Its own directory, which no compaction sweeps: the project a session
|
||||
# entered stays its project for the whole session.
|
||||
state_dir="${TMPDIR:-/tmp}/grimoire-fence"
|
||||
safe_sid=$(printf '%s' "${session_id:-none}" | tr -c 'A-Za-z0-9._-' '_')
|
||||
entered_file="$state_dir/${safe_sid}.entered"
|
||||
|
||||
# ── PostToolUse on enter_project: remember the first project entered ─────
|
||||
if [ "$hook_event" = "PostToolUse" ]; then
|
||||
case "$tool_name" in
|
||||
*__enter_project)
|
||||
pid=$(grimoire_json_pick "$event_flat" '.tool_input.project_id' | tr -cd '0-9')
|
||||
if [ -n "$pid" ] && [ -n "$session_id" ] && [ ! -s "$entered_file" ]; then
|
||||
mkdir -p "$state_dir" 2>/dev/null || true
|
||||
printf '%s' "$pid" > "$entered_file" 2>/dev/null || true
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# The nearest directory at or above $1 that exists. A Write may create the
|
||||
# directories its file goes in.
|
||||
fence_existing_dir() {
|
||||
local d="$1" depth=0
|
||||
while [ -n "$d" ] && [ ! -d "$d" ] && [ "$depth" -lt 40 ]; do
|
||||
d=$(dirname -- "$d" 2>/dev/null) || return 0
|
||||
depth=$((depth + 1))
|
||||
done
|
||||
[ -d "$d" ] && printf '%s' "$d"
|
||||
}
|
||||
|
||||
# "DIR<TAB>SUBCOMMAND" for each git command in the shell command $1 that
|
||||
# changes history or a remote, with DIR resolved from $2: `cd` moves it for
|
||||
# the segments after, `git -C` for that one command. Segments split on ; && ||
|
||||
# | and newlines, and a quoted separator inside a message splits too; the
|
||||
# words after it are read as a command only if they start with cd or git.
|
||||
fence_git_targets() {
|
||||
printf '%s\n' "$1" | awk -v start="$2" -v home="${HOME:-}" -v writes="$FENCE_GIT_WRITES" '
|
||||
function unq(w) { gsub(/^["\047(]+|["\047)]+$/, "", w); return w }
|
||||
function at(base, p) {
|
||||
if (p ~ /^\//) return p
|
||||
if (p == "~") return home
|
||||
if (p ~ /^~\//) return home substr(p, 2)
|
||||
return base "/" p
|
||||
}
|
||||
BEGIN { dir = start; n = split(writes, w, " "); for (i = 1; i <= n; i++) iswrite[w[i]] = 1 }
|
||||
{
|
||||
line = $0
|
||||
gsub(/&&|\|\||;|\|/, "\n", line)
|
||||
m = split(line, segs, "\n")
|
||||
for (s = 1; s <= m; s++) {
|
||||
k = split(segs[s], t, /[ \t]+/)
|
||||
i = 1
|
||||
while (i <= k && (t[i] == "" || t[i] ~ /^[A-Za-z_][A-Za-z0-9_]*=/)) i++
|
||||
if (i > k) continue
|
||||
cmd = unq(t[i])
|
||||
if (cmd == "cd") {
|
||||
if (i + 1 <= k && t[i + 1] != "") dir = at(dir, unq(t[i + 1]))
|
||||
continue
|
||||
}
|
||||
if (cmd != "git" && cmd !~ /\/git$/) continue
|
||||
gdir = dir; i++
|
||||
while (i <= k && t[i] ~ /^-/) {
|
||||
if (t[i] == "-C" && i + 1 <= k) { gdir = at(gdir, unq(t[i + 1])); i += 2; continue }
|
||||
if (t[i] == "-c" || t[i] == "--git-dir" || t[i] == "--work-tree") { i += 2; continue }
|
||||
i++
|
||||
}
|
||||
if (i > k) continue
|
||||
sub_ = unq(t[i])
|
||||
if (sub_ == "worktree") { if (i + 1 <= k && unq(t[i + 1]) == "add") print gdir "\tworktree add"; continue }
|
||||
if (sub_ in iswrite) print gdir "\t" sub_
|
||||
}
|
||||
}' 2>/dev/null || true
|
||||
}
|
||||
|
||||
# The short scalar arguments of the MCP call, as a JSON object of strings:
|
||||
# ids, kind, status, a repo url, and arrays of ids. Long text (bodies, code) is
|
||||
# left out; nothing that says where a write lands is long.
|
||||
fence_tool_input() {
|
||||
printf '%s\n' "$event_flat" | awk -F'\t' '
|
||||
function add(k, v) { if (!(k in seen)) { seen[k] = 1; order[++n] = k } }
|
||||
$2 ~ /^\.tool_input\.[A-Za-z_]+$/ && $3 != "null" && length($3) <= 200 {
|
||||
k = substr($2, 13); add(k); val[k] = "\"" $3 "\""
|
||||
}
|
||||
$2 ~ /^\.tool_input\.[A-Za-z_]+\[[0-9]+\]$/ && $3 != "null" && length($3) <= 200 {
|
||||
k = substr($2, 13); sub(/\[.*$/, "", k); add(k)
|
||||
# Tested before assigning: mawk creates arr[k] as soon as it is the
|
||||
# target, so the test inside the assignment would always be true.
|
||||
item = "\"" $3 "\""
|
||||
if (k in arr) item = arr[k] "," item
|
||||
arr[k] = item
|
||||
}
|
||||
END {
|
||||
printf "{"
|
||||
for (i = 1; i <= n; i++) {
|
||||
k = order[i]
|
||||
printf "%s\"%s\":%s", (i > 1 ? "," : ""), k, ((k in arr) ? "[" arr[k] "]" : val[k])
|
||||
}
|
||||
printf "}"
|
||||
}' 2>/dev/null || printf '{}'
|
||||
}
|
||||
|
||||
grimoire_config || exit 0
|
||||
command -v curl >/dev/null 2>&1 || exit 0
|
||||
|
||||
project_dir=${CLAUDE_PROJECT_DIR:-$PWD}
|
||||
session_scope=$(grimoire_scope_query "$project_dir")
|
||||
entered=$(cat "$entered_file" 2>/dev/null | tr -cd '0-9')
|
||||
# No project to hold the session to: started outside any, and entered none.
|
||||
[ -n "$session_scope" ] || [ -n "$entered" ] || exit 0
|
||||
|
||||
# Each check this act needs: "SCOPE<TAB>ACT" for a directory, or a single
|
||||
# "tool" line for a Grimoire call.
|
||||
checks=""
|
||||
case "$tool_name" in
|
||||
Write|Edit|MultiEdit|NotebookEdit)
|
||||
path=$(grimoire_json_pick "$event_flat" '.tool_input.file_path')
|
||||
[ -n "$path" ] || path=$(grimoire_json_pick "$event_flat" '.tool_input.notebook_path')
|
||||
[ -n "$path" ] || exit 0
|
||||
case "$path" in /*) ;; *) path="${cwd:-$PWD}/$path" ;; esac
|
||||
dir=$(fence_existing_dir "$(dirname -- "$path")")
|
||||
[ -n "$dir" ] || exit 0
|
||||
scope=$(grimoire_scope_query "$dir")
|
||||
[ -n "$scope" ] && [ "$scope" != "$session_scope" ] || exit 0
|
||||
checks=$(printf '%s\t%s' "$scope" "Editing a file in")
|
||||
;;
|
||||
Bash)
|
||||
command=$(grimoire_json_pick "$event_flat" '.tool_input.command')
|
||||
case "$command" in *git*) ;; *) exit 0 ;; esac
|
||||
while IFS=$'\t' read -r gdir gsub; do
|
||||
[ -n "$gdir" ] || continue
|
||||
[ -d "$gdir" ] || continue
|
||||
scope=$(grimoire_scope_query "$gdir")
|
||||
[ -n "$scope" ] && [ "$scope" != "$session_scope" ] || continue
|
||||
checks="${checks}${checks:+$'\n'}${scope}"$'\t'"git ${gsub} in"
|
||||
done <<EOF
|
||||
$(fence_git_targets "$command" "${cwd:-$PWD}")
|
||||
EOF
|
||||
[ -n "$checks" ] || exit 0
|
||||
;;
|
||||
mcp__*)
|
||||
short=${tool_name##*__}
|
||||
for prefix in $FENCE_READ_PREFIXES; do
|
||||
case "$short" in "$prefix"*) exit 0 ;; esac
|
||||
done
|
||||
checks="tool"
|
||||
;;
|
||||
*) exit 0 ;;
|
||||
esac
|
||||
|
||||
base="${url%/}/api/plugin/write-scope?"
|
||||
[ -n "$session_scope" ] && base="${base}${session_scope}&"
|
||||
[ -n "$entered" ] && base="${base}fallback_project_id=${entered}&"
|
||||
|
||||
fence_unreached() {
|
||||
local marker="$state_dir/${safe_sid}.fence.unreached" now last
|
||||
mkdir -p "$state_dir" 2>/dev/null || true
|
||||
now=$(date +%s 2>/dev/null) || now=0
|
||||
if [ -f "$marker" ]; then
|
||||
last=$(cat "$marker" 2>/dev/null) || last=0
|
||||
case "$last" in ''|*[!0-9]*) last=0 ;; esac
|
||||
[ $((now - last)) -lt 600 ] && return 0
|
||||
fi
|
||||
printf '%s' "$now" > "$marker" 2>/dev/null || true
|
||||
grimoire_json_out "PreToolUse" "> Grimoire did not answer the project fence check, so this act went ahead without being checked against the session's project. If it changes another project's files or records, stop and file an issue there instead. Said once per outage."
|
||||
}
|
||||
|
||||
while IFS= read -r check; do
|
||||
[ -n "$check" ] || continue
|
||||
if [ "$check" = "tool" ]; then
|
||||
input=$(fence_tool_input | grimoire_urlenc)
|
||||
q="tool=$(printf '%s' "$short" | grimoire_urlenc)&input=${input}"
|
||||
else
|
||||
scope=${check%%$'\t'*}
|
||||
act=${check#*$'\t'}
|
||||
q="target_${scope}&act=$(printf '%s' "$act" | grimoire_urlenc)"
|
||||
fi
|
||||
resp=$(curl -fsS --max-time 4 -H "Authorization: Bearer ${token}" "${base}${q}" 2>/dev/null) || {
|
||||
fence_unreached
|
||||
exit 0
|
||||
}
|
||||
rm -f "$state_dir/${safe_sid}.fence.unreached" 2>/dev/null || true
|
||||
flat=$(printf '%s' "$resp" | grimoire_json_flat)
|
||||
if [ "$(grimoire_json_pick "$flat" '.allowed')" = "false" ]; then
|
||||
reason=$(grimoire_json_pick "$flat" '.reason')
|
||||
esc=$(printf '%s' "$reason" | grimoire_json_escape) || exit 0
|
||||
printf '{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"ask","permissionDecisionReason":"%s"}}\n' "$esc"
|
||||
exit 0
|
||||
fi
|
||||
done <<EOF
|
||||
$checks
|
||||
EOF
|
||||
exit 0
|
||||
@@ -72,19 +72,29 @@ printf '%s\n' "$rule_id" | grimoire_rules_append "$state_dir/${safe_sid}.opened.
|
||||
|
||||
# ── What came just before the open ───────────────────────────────────────
|
||||
#
|
||||
# grimoire_moment.sh writes every call to `grimoire-moment/<sid>.acts`. The ones
|
||||
# from the last three minutes go to the server with this rule's id: a rule
|
||||
# opened just after a moment fired is evidence it belongs on that moment, and
|
||||
# once that repeats across sessions the server answers with one line asking
|
||||
# the reader to offer the operator the mount. Nothing is mounted by this.
|
||||
# grimoire_moment.sh writes every call to `grimoire-moment/<sid>.acts`. The
|
||||
# last three, from the last ninety seconds, go to the server with this rule's
|
||||
# id: a rule the agent went and opened right after a moment fired is evidence
|
||||
# it belongs on that moment, and once that repeats across sessions the server
|
||||
# answers with one line putting the question to the reader. Nothing is
|
||||
# mounted by this.
|
||||
#
|
||||
# An open of a rule a hook has just NAMED is not reported. A ranked arm or a
|
||||
# moment line put it in front of the reader, and that explains the open; the
|
||||
# moment beside it did not, so counting it would propose mounts on whatever
|
||||
# moment happened to fire while search was surfacing the rule. The window is
|
||||
# short for the same reason: a few minutes of agent work reaches many moments.
|
||||
command -v curl >/dev/null 2>&1 || exit 0
|
||||
grimoire_config || exit 0
|
||||
case ",$(grimoire_rules_live "$state_dir/${safe_sid}.rules.ids")," in
|
||||
*",${rule_id},"*) exit 0 ;;
|
||||
esac
|
||||
acts_file="${TMPDIR:-/tmp}/grimoire-moment/${safe_sid}.acts"
|
||||
[ -f "$acts_file" ] || exit 0
|
||||
now=$(date +%s 2>/dev/null) || exit 0
|
||||
acts=$(awk -F '\t' -v since=$((now - 180)) \
|
||||
acts=$(awk -F '\t' -v since=$((now - 90)) \
|
||||
'$1 + 0 >= since { sub(/^[^\t]*\t/, ""); print }' "$acts_file" 2>/dev/null \
|
||||
| tail -n 30 | paste -sd ',' - 2>/dev/null)
|
||||
| tail -n 3 | paste -sd ',' - 2>/dev/null)
|
||||
[ -n "$acts" ] || exit 0
|
||||
|
||||
sid_esc=$(printf '%s' "$session_id" | grimoire_json_escape) || exit 0
|
||||
|
||||
@@ -31,11 +31,13 @@ event_flat=$(printf '%s' "$event" | grimoire_json_flat)
|
||||
session_id=$(grimoire_json_pick "$event_flat" '.session_id')
|
||||
[ -n "$session_id" ] || exit 0
|
||||
|
||||
# The moment arm's per-session files (its tool cache and the acts ledger)
|
||||
# describe a session that is over, on any exit — a clear
|
||||
# included, since the next conversation is a new session id.
|
||||
# The moment arm's per-session files (its tool cache and the acts ledger) and
|
||||
# the fence's (the project the session entered, its outage marker) describe a
|
||||
# session that is over, on any exit — a clear included, since the next
|
||||
# conversation is a new session id.
|
||||
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
|
||||
rm -f "${TMPDIR:-/tmp}/grimoire-moment/${safe_sid}".* 2>/dev/null || true
|
||||
rm -f "${TMPDIR:-/tmp}/grimoire-moment/${safe_sid}".* \
|
||||
"${TMPDIR:-/tmp}/grimoire-fence/${safe_sid}".* 2>/dev/null || true
|
||||
|
||||
reason=$(grimoire_json_pick "$event_flat" '.reason')
|
||||
[ "$reason" = "clear" ] && exit 0
|
||||
|
||||
@@ -25,6 +25,15 @@
|
||||
}
|
||||
],
|
||||
"PreToolUse": [
|
||||
{
|
||||
"matcher": "Write|Edit|MultiEdit|NotebookEdit|Bash|mcp__.*grimoire.*",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/grimoire_project_fence.sh\""
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": "Write|Edit",
|
||||
"hooks": [
|
||||
@@ -89,6 +98,15 @@
|
||||
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/grimoire_claim_session.sh\""
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": "mcp__.*__enter_project",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/grimoire_project_fence.sh\""
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"PreCompact": [
|
||||
|
||||
@@ -277,11 +277,12 @@ bound — confine the session to it:
|
||||
- **Pass that `project_id` to every read** (`search`, `list_tasks`,
|
||||
`list_notes`). An unscoped read bleeds every other project's work into your
|
||||
context.
|
||||
- **Only reference or offer work on the in-scope project.** Don't surface,
|
||||
suggest, or start work on other projects unless the operator explicitly widens
|
||||
scope.
|
||||
- If something clearly belongs to a *different* project, say so and **ask before
|
||||
switching** — never silently operate cross-project.
|
||||
- **Only reference or offer work on the in-scope project**, unless the
|
||||
operator explicitly widens scope; work that belongs elsewhere, say so and ask.
|
||||
- **Cross-project, the one change to make is filing an issue**:
|
||||
`create_task(project_id=<theirs>, kind="issue")` with how you found it and the
|
||||
evidence. Reading there is fine; editing its files, committing or pushing in
|
||||
its repo, or changing its records is the operator's call, and the plugin asks.
|
||||
|
||||
## Read these when the moment comes
|
||||
|
||||
|
||||
@@ -73,12 +73,16 @@ than a mount is fixed through its trigger instead.
|
||||
|
||||
## When a line proposes a mount
|
||||
|
||||
A rule that keeps being opened just after the same moment, across several
|
||||
sessions, probably belongs on that moment. A line says so, naming the rule,
|
||||
the moment and how often. It is a question for the operator, not a change you
|
||||
make: offer it in one line, and record their answer with
|
||||
`judge_rule_moments` — `confirm` mounts the rule, `reject` with their reason
|
||||
stops the question being asked again.
|
||||
A rule that sessions keep going and opening just after the same moment —
|
||||
without a hook having shown it to them first — may belong on that moment. A
|
||||
line says so, naming the rule, the moment, the action that reached it and how
|
||||
often. That is co-occurrence, not a recommendation, so judge it before it
|
||||
reaches the operator: read the rule's trigger against what the moment means
|
||||
(`list_moments`). Offer it only when the rule governs that point in the work
|
||||
whatever the work is about, as a question with your reason; a rule about a
|
||||
subject (a library, a file, a style) is reached by meaning, and the line is
|
||||
left alone. Record their answer with `judge_rule_moments` — `confirm` mounts
|
||||
the rule, `reject` with their reason stops the question being asked again.
|
||||
|
||||
The same tool answers proposals from a pass over the rules
|
||||
(`rules_to_mount`, `propose_rule_moments`, `rule_moment_proposals`): a pass
|
||||
|
||||
+17
-5
@@ -395,6 +395,13 @@ SMOKE_EVENTS: dict[str, str] = {
|
||||
{"session_id": "smoke", "cwd": ".", "hook_event_name": "SessionEnd",
|
||||
"reason": "prompt_input_exit"}
|
||||
),
|
||||
# The project fence. A Grimoire write is the one act that always asks the
|
||||
# server, so it is what reaches the network in the unreachable case: there
|
||||
# the act goes ahead and the hook says once that it couldn't check.
|
||||
"grimoire_project_fence.sh": json.dumps(
|
||||
{"session_id": "smoke", "cwd": ".", "hook_event_name": "PreToolUse",
|
||||
"tool_name": "mcp__grimoire__update_task", "tool_input": {"task_id": 1}}
|
||||
),
|
||||
# The shared library is sourced, never run; executed bare it defines
|
||||
# functions and exits — silent by construction.
|
||||
"grimoire_defs.sh": "",
|
||||
@@ -415,9 +422,14 @@ STATIC_EMITTERS = {
|
||||
# would look like a leak; it is the opposite.
|
||||
"grimoire_precompact_preserve.sh": "preservation instructions present",
|
||||
}
|
||||
# The hooks that say so when a configured instance does not answer.
|
||||
OUTAGE_SPEAKERS = {"grimoire_prior_art.sh", "grimoire_after_write.sh"}
|
||||
OUTAGE_LINE = "> Grimoire did not answer the prior-art check"
|
||||
# The hooks that say so when a configured instance does not answer, each with
|
||||
# the line it says it in.
|
||||
_PRIOR_ART_OUTAGE = "> Grimoire did not answer the prior-art check"
|
||||
OUTAGE_LINES = {
|
||||
"grimoire_prior_art.sh": _PRIOR_ART_OUTAGE,
|
||||
"grimoire_after_write.sh": _PRIOR_ART_OUTAGE,
|
||||
"grimoire_project_fence.sh": "> Grimoire did not answer the project fence check",
|
||||
}
|
||||
|
||||
|
||||
def _run_hook(script: Path, event: str, env_extra: dict[str, str]) -> subprocess.CompletedProcess:
|
||||
@@ -464,13 +476,13 @@ def check_fail_open() -> None:
|
||||
f"output must survive having no credentials ({what})")
|
||||
else:
|
||||
ok(f"{rel} [{label}]: exit 0, {what}")
|
||||
elif out and label == "unreachable" and script.name in OUTAGE_SPEAKERS:
|
||||
elif out and label == "unreachable" and script.name in OUTAGE_LINES:
|
||||
# The only thing allowed here is the outage line itself.
|
||||
try:
|
||||
ctx = json.loads(out)["hookSpecificOutput"]["additionalContext"]
|
||||
except (ValueError, KeyError, TypeError):
|
||||
ctx = ""
|
||||
if ctx.startswith(OUTAGE_LINE):
|
||||
if ctx.startswith(OUTAGE_LINES[script.name]):
|
||||
ok(f"{rel} [{label}]: exit 0, says the instance did not answer")
|
||||
else:
|
||||
fail(f"{rel} [{label}]: emitted output with no working instance "
|
||||
|
||||
+21
-10
@@ -6,7 +6,8 @@ from pathlib import Path
|
||||
|
||||
from quart import Quart, g, jsonify, make_response, request, send_from_directory
|
||||
|
||||
from grimoire.config import PLACEHOLDER_SECRET_KEYS, SECRET_KEY_FILENAME, Config
|
||||
from grimoire.config import SECRET_KEY_FILENAME, Config
|
||||
from grimoire.proxy import PerRequestSecureSession, apply_https_headers
|
||||
from grimoire.routes.admin import admin_bp
|
||||
from grimoire.routes.api import api
|
||||
from grimoire.routes.auth import auth_bp
|
||||
@@ -61,13 +62,13 @@ def create_app() -> Quart:
|
||||
app.secret_key = Config.SECRET_KEY
|
||||
app.config["SESSION_COOKIE_HTTPONLY"] = True
|
||||
app.config["SESSION_COOKIE_SAMESITE"] = "Lax"
|
||||
app.config["SESSION_COOKIE_SECURE"] = Config.SECURE_COOKIES
|
||||
# Secure is decided per request, from the trusted proxy's X-Forwarded-Proto.
|
||||
app.session_interface = PerRequestSecureSession()
|
||||
|
||||
if Config.SECRET_KEY in PLACEHOLDER_SECRET_KEYS:
|
||||
if os.environ.get("SECURE_COOKIES"):
|
||||
logger.warning(
|
||||
"SECRET_KEY is set to a public placeholder, so session cookies are insecure. "
|
||||
"Remove it to let the instance keep its own key in %s, or set a real one.",
|
||||
Config.DATA_DIR,
|
||||
"SECURE_COOKIES is not used: the session cookie is Secure on every request "
|
||||
"the trusted proxy reports as HTTPS. Remove it from the environment."
|
||||
)
|
||||
if Config.SECRET_KEY_SOURCE == "ephemeral":
|
||||
logger.warning(
|
||||
@@ -82,11 +83,12 @@ def create_app() -> Quart:
|
||||
Config.DATA_DIR, SECRET_KEY_FILENAME,
|
||||
)
|
||||
|
||||
if not Config.TRUST_PROXY_HEADERS:
|
||||
if os.environ.get("TRUST_PROXY_HEADERS") and not os.environ.get("TRUSTED_PROXY_HOPS"):
|
||||
logger.warning(
|
||||
"TRUST_PROXY_HEADERS is not set. If this instance is behind a reverse proxy "
|
||||
"(nginx, Caddy, Traefik) set TRUST_PROXY_HEADERS=true so rate limiting uses "
|
||||
"real client IPs rather than the proxy IP."
|
||||
"TRUST_PROXY_HEADERS is not used (the starting proxy hop count is %d). "
|
||||
"Remove it; set the count in Settings → Config, or TRUSTED_PROXY_HOPS "
|
||||
"to change the starting value.",
|
||||
Config.TRUSTED_PROXY_HOPS,
|
||||
)
|
||||
|
||||
app.register_blueprint(admin_bp)
|
||||
@@ -159,6 +161,7 @@ def create_app() -> Quart:
|
||||
except Exception:
|
||||
logger.debug("Failed to log usage", exc_info=True)
|
||||
|
||||
apply_https_headers(response)
|
||||
response.headers.setdefault("X-Content-Type-Options", "nosniff")
|
||||
response.headers.setdefault("X-Frame-Options", "DENY")
|
||||
response.headers.setdefault("Referrer-Policy", "strict-origin-when-cross-origin")
|
||||
@@ -199,6 +202,14 @@ def create_app() -> Quart:
|
||||
start_notification_loop()
|
||||
start_auth_token_retention_loop()
|
||||
|
||||
# The proxy hop count saved in Settings decides every request's client
|
||||
# address, so it is loaded before serving, under a deadline (a failed
|
||||
# or slow read leaves the environment's count in force), and re-read
|
||||
# on an interval so a save in another worker reaches this one.
|
||||
from grimoire.services import proxy_settings
|
||||
await proxy_settings.load_at_startup()
|
||||
proxy_settings.start_refresh_loop()
|
||||
|
||||
# Write down any shipped default that moved in this release.
|
||||
#
|
||||
# INLINE AND AWAITED, deliberately, against the instinct the block
|
||||
|
||||
+34
-6
@@ -85,6 +85,21 @@ def _secret_key() -> tuple[str, str]:
|
||||
return secrets.token_hex(32), "ephemeral"
|
||||
|
||||
|
||||
def _trusted_proxy_hops() -> int:
|
||||
"""TRUSTED_PROXY_HOPS, default 1. The older TRUST_PROXY_HEADERS switch is
|
||||
still read when the hop count is unset: false means 0 hops. A value that
|
||||
is not a whole number reads as -1, which validate() refuses."""
|
||||
raw = os.environ.get("TRUSTED_PROXY_HOPS", "").strip()
|
||||
if raw:
|
||||
try:
|
||||
return int(raw)
|
||||
except ValueError:
|
||||
return -1
|
||||
if os.environ.get("TRUST_PROXY_HEADERS", "").strip().lower() in ("0", "false", "no"):
|
||||
return 0
|
||||
return 1
|
||||
|
||||
|
||||
class Config:
|
||||
DATABASE_URL: str = _read_secret(
|
||||
"DATABASE_URL",
|
||||
@@ -95,7 +110,6 @@ class Config:
|
||||
# shipped compose file, beside the model cache and diagnostics.
|
||||
DATA_DIR: str = os.environ.get("DATA_DIR", "/data")
|
||||
SECRET_KEY, SECRET_KEY_SOURCE = _secret_key()
|
||||
SECURE_COOKIES: bool = os.environ.get("SECURE_COOKIES", "").lower() in ("1", "true", "yes")
|
||||
LOG_LEVEL: str = os.environ.get("LOG_LEVEL", "INFO")
|
||||
LOG_RETENTION_DAYS: int = int(os.environ.get("LOG_RETENTION_DAYS", "90"))
|
||||
|
||||
@@ -108,7 +122,16 @@ class Config:
|
||||
SMTP_FROM_NAME: str = os.environ.get("SMTP_FROM_NAME", "Fabled Grimoire")
|
||||
SMTP_USE_TLS: bool = os.environ.get("SMTP_USE_TLS", "true").lower() in ("1", "true", "yes")
|
||||
BASE_URL: str = os.environ.get("BASE_URL", "http://localhost:5000").rstrip("/")
|
||||
TRUST_PROXY_HEADERS: bool = os.environ.get("TRUST_PROXY_HEADERS", "").lower() in ("1", "true", "yes")
|
||||
# How many reverse proxies stand in front of the app; the client address
|
||||
# and scheme are read that many entries in from the right of the
|
||||
# forwarding headers (grimoire/proxy.py). 0 trusts no proxy. This is the
|
||||
# install's starting value: a count saved in Settings → Config wins over
|
||||
# it (services/proxy_settings.py).
|
||||
TRUSTED_PROXY_HOPS: int = _trusted_proxy_hops()
|
||||
TRUSTED_PROXY_HOPS_FROM_ENV: bool = bool(
|
||||
os.environ.get("TRUSTED_PROXY_HOPS", "").strip()
|
||||
or os.environ.get("TRUST_PROXY_HEADERS", "").strip()
|
||||
)
|
||||
|
||||
# Git URL of the repo that ships this Grimoire plugin. There is one correct
|
||||
# value per deployment (the repo serving the plugin), so it's a config
|
||||
@@ -163,11 +186,16 @@ class Config:
|
||||
errors.append(f"SMTP_PORT={cls.SMTP_PORT} must be between 1 and 65535")
|
||||
if cls.oidc_enabled() and not cls.BASE_URL.startswith(("http://", "https://")):
|
||||
errors.append(f"BASE_URL='{cls.BASE_URL}' must start with http:// or https:// when OIDC is enabled")
|
||||
if cls.SECRET_KEY == "dev-secret-change-me" and cls.SECURE_COOKIES:
|
||||
if cls.TRUSTED_PROXY_HOPS < 0:
|
||||
errors.append(
|
||||
"SECRET_KEY is set to the public placeholder 'dev-secret-change-me' but "
|
||||
"SECURE_COOKIES=true indicates a production deployment. Remove SECRET_KEY "
|
||||
"to let the instance keep its own key in DATA_DIR, or set a real one."
|
||||
f"TRUSTED_PROXY_HOPS={os.environ.get('TRUSTED_PROXY_HOPS')!r} must be a whole "
|
||||
"number of proxies, 0 or more"
|
||||
)
|
||||
if cls.SECRET_KEY in PLACEHOLDER_SECRET_KEYS:
|
||||
errors.append(
|
||||
f"SECRET_KEY is set to the public placeholder {cls.SECRET_KEY!r}, so anyone "
|
||||
"could forge a session. Remove SECRET_KEY to let the instance keep its own "
|
||||
"key in DATA_DIR, or set a real one."
|
||||
)
|
||||
if errors:
|
||||
raise ValueError("Configuration errors:\n" + "\n".join(f" - {e}" for e in errors))
|
||||
|
||||
@@ -42,7 +42,11 @@ async def menus_to_review(
|
||||
|
||||
Args:
|
||||
source: the arm to review. Only arms whose search can be re-run
|
||||
exactly are offered; today that is `auto_inject`.
|
||||
exactly are offered: `auto_inject`, and the rule arms
|
||||
`prompt_rule`, `pre_tool_rule` and `write_path_rule`. A rule
|
||||
line's passage is the trigger the arm matched, and for the
|
||||
command and write arms `within_budget` also needs the line to
|
||||
sit inside the band of the top hit, as it did when shown.
|
||||
n: how many calls to sample, 1–20. Random among the calls you have not
|
||||
judged, so a review does not describe one afternoon.
|
||||
days: how far back to sample from.
|
||||
|
||||
@@ -10,13 +10,16 @@ preview-style warning.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from grimoire.mcp._context import current_user_id
|
||||
from grimoire.services import dedup as dedup_svc
|
||||
from grimoire.services import moments as moments_svc
|
||||
from grimoire.services import rulebooks as rulebooks_svc
|
||||
from grimoire.services import trash as trash_svc
|
||||
from grimoire.services.lesson_rules import confirmed_lesson_counts
|
||||
from grimoire.services.rule_usage import (
|
||||
record_rule_outcome, record_rule_pulled,
|
||||
empty_rule_usage, record_rule_outcome, record_rule_pulled, usage_for_rules,
|
||||
)
|
||||
from grimoire.services import moment_delivery
|
||||
|
||||
@@ -204,6 +207,7 @@ def _rule_summary(r, moments: list[str] | None = None) -> dict:
|
||||
|
||||
async def list_rules(
|
||||
rulebook_id: int = 0, topic_id: int = 0, project_id: int = 0,
|
||||
with_usage: bool = False, days: int = 0,
|
||||
) -> dict:
|
||||
"""List rules — filter by rulebook, topic, and/or project.
|
||||
|
||||
@@ -213,9 +217,21 @@ async def list_rules(
|
||||
project_id: 0 = no filter; positive = that project's OWN rules.
|
||||
Global rules apply to every project, so they are listed by
|
||||
rulebook or topic (or unfiltered), not under each project.
|
||||
with_usage: add each rule's `usage` and `confirmed_lessons`. Off by
|
||||
default: a listing is for applying rules, and this is for judging
|
||||
them.
|
||||
days: with `with_usage`, count only the last N days; 0 = everything
|
||||
recorded.
|
||||
|
||||
rulebook_id and topic_id AND-combine; project_id lists a project's rules
|
||||
on its own. Ownership-scoped.
|
||||
|
||||
`usage` is the rule's own telemetry: `surfaced_count` (a ranked surface
|
||||
chose it), `ambient_count` (a bulk delivery carried it), `pull_count`,
|
||||
`applied_count`, `departed_count` and the latest of each.
|
||||
`confirmed_lessons` counts the lessons judged instances of the rule. Read
|
||||
together they answer whether rules that situations keep proving are the
|
||||
ones that surface and get opened.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rows = await rulebooks_svc.list_rules(
|
||||
@@ -224,12 +240,20 @@ async def list_rules(
|
||||
topic_id=topic_id or None,
|
||||
project_id=project_id or None,
|
||||
)
|
||||
ids = [r.id for r in rows]
|
||||
# One batched read for the page, not one per row.
|
||||
mounted = await rulebooks_svc.list_rule_moments([r.id for r in rows])
|
||||
return {
|
||||
"rules": [_rule_summary(r, mounted.get(r.id)) for r in rows],
|
||||
"total": len(rows),
|
||||
}
|
||||
mounted = await rulebooks_svc.list_rule_moments(ids)
|
||||
rules = [_rule_summary(r, mounted.get(r.id)) for r in rows]
|
||||
if with_usage:
|
||||
since = (
|
||||
datetime.now(timezone.utc) - timedelta(days=days) if days > 0 else None
|
||||
)
|
||||
usage = await usage_for_rules(ids, since=since)
|
||||
lessons = await confirmed_lesson_counts(uid, ids)
|
||||
for row in rules:
|
||||
row["usage"] = usage.get(row["id"], empty_rule_usage())
|
||||
row["confirmed_lessons"] = lessons.get(row["id"], 0)
|
||||
return {"rules": rules, "total": len(rows)}
|
||||
|
||||
|
||||
async def get_rule(rule_id: int) -> dict:
|
||||
|
||||
@@ -509,6 +509,15 @@ async def retrieval_telemetry(
|
||||
Read it against `sources["write_path_rule"]`, which declines the large
|
||||
majority of its calls like any other surface.
|
||||
|
||||
`rule_usage.by_source` is the per-surface split, the twin of
|
||||
`usage.by_source`: for each surface that showed rules, `rules_surfaced`
|
||||
(distinct rules), `rules_pulled` (how many of those an agent opened in the
|
||||
window) and `pull_through`, null on an ambient surface. It is the readout
|
||||
that says which arm — the prompt arm, the tool arm, a lesson pointing at
|
||||
its rule — earns its lines. The same upper bound as the note split: a rule
|
||||
two surfaces showed and an agent opened once counts for both.
|
||||
`by_source_failed: true` means that one query failed.
|
||||
|
||||
`system_usage` — the RULINGS arm, from `system_usage_events`: how
|
||||
often an area's rulings were shown because a command (`rulings_pre_tool`)
|
||||
or an edit (`rulings_write_path`) touched its files, how often a System
|
||||
@@ -590,8 +599,11 @@ async def retrieval_telemetry(
|
||||
corpus. Not a verdict on them: a line carries its matched passage, so
|
||||
an unopened record may have been unrelated, enough as shown, or already
|
||||
in context. Judge a sample (`menus_to_review`) before acting on it.
|
||||
For RULES there is no judged sample: a rule set aside again and again
|
||||
is a trigger to fix (`update_rule(when_to_apply=...)`), not a floor.
|
||||
For RULES, judge a sample of the rule arm that showed them
|
||||
(`menus_to_review(source="prompt_rule")`, or `pre_tool_rule`,
|
||||
`write_path_rule`); a rule set aside again and again where it does
|
||||
not apply is a trigger to fix (`update_rule(when_to_apply=...)`), not
|
||||
a floor.
|
||||
- `read_and_unacted` — distinct rules OPENED in the window that recorded
|
||||
no outcome, against the ones that did. The worse sibling of `surfaced_never_pulled` above: a
|
||||
rule nobody opens is cheap, while a rule read and silently unchanged is
|
||||
|
||||
@@ -94,3 +94,4 @@ from grimoire.models.family import ( # noqa: E402, F401
|
||||
FamilyAdoption, FamilyDecision, FamilyIdea, FamilyIdeaPlatform,
|
||||
FamilyIdeaReference, Platform, ProjectPlatform,
|
||||
)
|
||||
from grimoire.models.instance import InstanceIdentity # noqa: E402, F401
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import CheckConstraint, DateTime, Integer, Text, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from grimoire.models import Base
|
||||
|
||||
|
||||
class InstanceIdentity(Base):
|
||||
"""The one row naming this database's id lineage.
|
||||
|
||||
Two stores with the same instance id number their rows the same way: one
|
||||
is the other, or a restore of it made with the original ids kept. That is
|
||||
what lets a backup roll a live instance back by id, and what refuses a
|
||||
backup whose ids would land on unrelated rows. Made by the migration, so
|
||||
an instance has it before anyone registers.
|
||||
"""
|
||||
|
||||
__tablename__ = "instance_identity"
|
||||
__table_args__ = (CheckConstraint("id = 1", name="ck_instance_identity_single_row"),)
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, default=1)
|
||||
instance_id: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||
)
|
||||
@@ -0,0 +1,164 @@
|
||||
"""What the trusted reverse proxy says about a request: who sent it, and
|
||||
whether it arrived over HTTPS.
|
||||
|
||||
Grimoire serves plain HTTP; TLS ends at the operator's proxy. The proxy
|
||||
reports the client's address in X-Forwarded-For and the scheme in
|
||||
X-Forwarded-Proto, and each proxy on the road APPENDS its entry, so only the
|
||||
right-hand end of either header was written by a proxy we trust. Anything to
|
||||
the left of that is whatever the client chose to send.
|
||||
|
||||
The hop count says how many proxies stand in front of the app. An admin sets
|
||||
it in Settings → Config; until they do, it is `TRUSTED_PROXY_HOPS` from the
|
||||
environment (default 1). The entry read is that many in from the right. Zero
|
||||
trusts no proxy: the headers are ignored and the socket peer is the client.
|
||||
|
||||
Never read X-Forwarded-For or X-Forwarded-Proto anywhere else. The client
|
||||
address keys the rate limits and the audit log, and the scheme decides the
|
||||
session cookie's Secure flag and HSTS. A second reader is a second trust rule.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from quart import has_request_context, request
|
||||
from quart.sessions import SecureCookieSessionInterface
|
||||
|
||||
from grimoire.config import Config
|
||||
|
||||
# HSTS for a year, sent only on a request the trusted proxy says is HTTPS.
|
||||
# No includeSubDomains or preload: the app does not own the operator's domain.
|
||||
HSTS_VALUE = "max-age=31536000"
|
||||
|
||||
|
||||
def trusted_entry(header: str, hops: int) -> str | None:
|
||||
"""The entry `hops` in from the right of a forwarding header.
|
||||
|
||||
A header shorter than that is read at its leftmost entry, the one the
|
||||
first proxy wrote. A road with fewer proxies than the count (a LAN road
|
||||
beside a Cloudflare one) still reads right that way. None when no proxy
|
||||
is trusted or there is no header. Pure, so the trust boundary is
|
||||
testable without a request."""
|
||||
if hops <= 0:
|
||||
return None
|
||||
entries = [part.strip() for part in header.split(",") if part.strip()]
|
||||
if not entries:
|
||||
return None
|
||||
return entries[-min(hops, len(entries))]
|
||||
|
||||
|
||||
# The count saved in Settings, or None when none is saved. Held here because
|
||||
# every request reads it and the read is synchronous; services/proxy_settings
|
||||
# loads it at startup, refreshes it on an interval in every worker, and sets it
|
||||
# at once in the worker that saved a new one.
|
||||
_saved_hops: int | None = None
|
||||
|
||||
|
||||
def set_saved_hops(hops: int | None) -> None:
|
||||
global _saved_hops
|
||||
_saved_hops = hops
|
||||
|
||||
|
||||
def saved_hops_in_force() -> int | None:
|
||||
"""The count saved in Settings that this worker holds, or None."""
|
||||
return _saved_hops
|
||||
|
||||
|
||||
def trusted_hops() -> int:
|
||||
"""The hop count in force: the one saved in Settings, else the env's."""
|
||||
return Config.TRUSTED_PROXY_HOPS if _saved_hops is None else _saved_hops
|
||||
|
||||
|
||||
def _chain(name: str) -> str:
|
||||
# Joined, because a header sent twice arrives as two values, and each
|
||||
# proxy's append may land on either one.
|
||||
return ",".join(request.headers.getlist(name))
|
||||
|
||||
|
||||
def _forwarded(name: str) -> str | None:
|
||||
return trusted_entry(_chain(name), trusted_hops())
|
||||
|
||||
|
||||
def client_ip() -> str:
|
||||
"""The caller's address, as the trusted proxy saw it."""
|
||||
if not has_request_context():
|
||||
return ""
|
||||
return (_forwarded("X-Forwarded-For") or request.remote_addr or "")[:64]
|
||||
|
||||
|
||||
def is_https() -> bool:
|
||||
"""Whether this request arrived over HTTPS, by the trusted proxy's word."""
|
||||
if not has_request_context():
|
||||
return False
|
||||
if request.scheme == "https":
|
||||
return True
|
||||
return (_forwarded("X-Forwarded-Proto") or "").lower() == "https"
|
||||
|
||||
|
||||
class PerRequestSecureSession(SecureCookieSessionInterface):
|
||||
"""The session cookie is Secure exactly when the request is HTTPS, so a
|
||||
public deployment gets it and a plain-HTTP one keeps working."""
|
||||
|
||||
def get_cookie_secure(self, app) -> bool:
|
||||
return is_https()
|
||||
|
||||
|
||||
def apply_https_headers(response) -> None:
|
||||
"""HSTS on an HTTPS request, and only then. The app never redirects to
|
||||
HTTPS; plain HTTP is a supported way to run it."""
|
||||
if is_https():
|
||||
response.headers.setdefault("Strict-Transport-Security", HSTS_VALUE)
|
||||
|
||||
|
||||
# Cloudflare writes the visitor's address here on every request it proxies.
|
||||
# Read only to SUGGEST a hop count in the readout, never to decide one: a
|
||||
# client that reaches the app without Cloudflare can send it too.
|
||||
CLOUDFLARE_CLIENT_HEADER = "CF-Connecting-IP"
|
||||
CLOUDFLARE_MARKERS = (CLOUDFLARE_CLIENT_HEADER, "CF-Ray")
|
||||
READOUT_HOPS = range(0, 4)
|
||||
MAX_HOPS = 5
|
||||
|
||||
|
||||
def readout(peer: str, forwarded_for: str, forwarded_proto: str,
|
||||
cloudflare_client: str | None, scheme: str = "http") -> dict:
|
||||
"""What each hop count would make of one request, so an admin can pick
|
||||
the count that shows their own address. Pure: the request's parts come in
|
||||
as arguments.
|
||||
|
||||
`suggested` is the smallest count that reads the address Cloudflare
|
||||
reports, when the request came through Cloudflare; otherwise None, since
|
||||
only the admin knows which address is theirs.
|
||||
"""
|
||||
def read(hops: int) -> dict:
|
||||
address = trusted_entry(forwarded_for, hops) or peer
|
||||
proto = trusted_entry(forwarded_proto, hops)
|
||||
https = scheme == "https" or (proto or "").lower() == "https"
|
||||
return {"hops": hops, "client": address, "https": https}
|
||||
|
||||
suggested = None
|
||||
if cloudflare_client:
|
||||
suggested = next(
|
||||
(h for h in range(1, MAX_HOPS + 1)
|
||||
if trusted_entry(forwarded_for, h) == cloudflare_client),
|
||||
None,
|
||||
)
|
||||
return {
|
||||
"peer": peer,
|
||||
"forwarded_for": [p.strip() for p in forwarded_for.split(",") if p.strip()],
|
||||
"forwarded_proto": [p.strip() for p in forwarded_proto.split(",") if p.strip()],
|
||||
"cloudflare": cloudflare_client is not None,
|
||||
"cloudflare_client": cloudflare_client,
|
||||
"rows": [read(h) for h in READOUT_HOPS],
|
||||
"suggested": suggested,
|
||||
}
|
||||
|
||||
|
||||
def request_readout() -> dict:
|
||||
"""`readout` for the current request."""
|
||||
cloudflare = request.headers.get(CLOUDFLARE_CLIENT_HEADER)
|
||||
if cloudflare is None and request.headers.get("CF-Ray"):
|
||||
cloudflare = ""
|
||||
return readout(
|
||||
request.remote_addr or "",
|
||||
_chain("X-Forwarded-For"),
|
||||
_chain("X-Forwarded-Proto"),
|
||||
cloudflare.strip() if cloudflare is not None else None,
|
||||
request.scheme,
|
||||
)
|
||||
@@ -2,15 +2,23 @@
|
||||
|
||||
IMPORTANT — deployment note:
|
||||
Rate limit counters are stored in memory and are lost on process restart.
|
||||
When deployed behind a reverse proxy (nginx, Caddy, Traefik) you MUST set
|
||||
TRUST_PROXY_HEADERS=true so that the real client IP is used as the bucket key
|
||||
rather than the proxy's IP (which would cause all users to share one bucket).
|
||||
The bucket key is the client address from grimoire/proxy.py, read by the
|
||||
proxy hop count (Settings → Config). A count that is too low keys every
|
||||
caller on the proxy's address, so they all share one bucket.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from collections import defaultdict
|
||||
|
||||
# Said beside every "too many attempts" reply, because the one case a person
|
||||
# can't diagnose on their own is everybody being blocked at once: the proxy
|
||||
# count is too low, so every visitor reads as the proxy and shares one bucket.
|
||||
SHARED_ADDRESS_HINT = (
|
||||
"If everyone is being blocked, an admin should check the reverse proxy count "
|
||||
"in Settings → Config: every visitor may be reading as the proxy's address."
|
||||
)
|
||||
|
||||
_buckets: dict[str, list[float]] = defaultdict(list)
|
||||
_lock = asyncio.Lock()
|
||||
|
||||
|
||||
+80
-30
@@ -1,20 +1,26 @@
|
||||
"""Restore a Grimoire backup into an empty instance, from the command line.
|
||||
"""Restore a Grimoire backup from the command line.
|
||||
|
||||
python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar
|
||||
python -m grimoire.restore /backups/grimoire-20261015T020000Z.tar --yes
|
||||
|
||||
This is how a fresh instance is brought back after losing the old one. Run it
|
||||
inside the Grimoire container, which already has the database settings and has
|
||||
already migrated the schema on start. It reads either the scheduled backup's
|
||||
tar (services/backup_file.py) or the JSON the admin download produces.
|
||||
Run it inside the Grimoire container, which already has the database settings
|
||||
and has migrated the schema on start. It reads the scheduled backup's tar
|
||||
(services/backup_file.py) or the JSON the admin download produces, and nothing
|
||||
limits the file's size.
|
||||
|
||||
It refuses a database that already holds an account. A restore re-creates
|
||||
every user, project and record from the file with new ids; into a live
|
||||
instance it would add a second copy of everything rather than replace it.
|
||||
(Settings → Backup & Restore still offers the upload for that case, on
|
||||
purpose and with its own warning.)
|
||||
It does what Settings → Restore does (services/backup.py replace_from_backup):
|
||||
- **A new, empty instance** takes the backup as it is.
|
||||
- **A live instance** rolls back to a backup of its own: everything goes back
|
||||
to the moment the backup was taken, for every user.
|
||||
- A backup from another instance, or one made before backups recorded their
|
||||
instance, only goes into an empty instance.
|
||||
|
||||
Unlike the upload, nothing limits the file's size here, so this is also the
|
||||
way in for a backup too big for the web upload.
|
||||
Without --yes it only says what would change. With --yes it first saves the
|
||||
current state as a backup (to the backup destination, or DATA_DIR/pre-restore),
|
||||
then restores. Everyone is signed out afterwards.
|
||||
|
||||
A backup in the oldest (v1) format is copied into an empty instance with new
|
||||
ids instead, as before.
|
||||
|
||||
Afterwards, restart Grimoire: search indexes are not part of a backup and are
|
||||
rebuilt from the restored records at startup.
|
||||
@@ -48,9 +54,39 @@ async def _account_count() -> int:
|
||||
return (await session.execute(select(func.count()).select_from(User))).scalar_one()
|
||||
|
||||
|
||||
async def run(path: Path) -> int:
|
||||
from grimoire.models import engine
|
||||
async def _restore_v1(payload: dict) -> int:
|
||||
from grimoire.services.backup import restore_full_backup
|
||||
|
||||
accounts = await _account_count()
|
||||
if accounts:
|
||||
print(
|
||||
f"This database already holds {accounts} account(s). A backup in the oldest "
|
||||
"format is copied in with new ids rather than replacing anything, so it only "
|
||||
"goes into an empty instance. Start Grimoire on an empty database and run "
|
||||
"this again.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 3
|
||||
stats = await restore_full_backup(payload)
|
||||
print(json.dumps({"restored": stats}, indent=2, default=str))
|
||||
return 0
|
||||
|
||||
|
||||
def _print_preview(preview: dict) -> None:
|
||||
taken = preview.get("exported_at") or "an unknown date"
|
||||
if preview["mode"] == "rollback":
|
||||
print(f"This rolls the whole instance back to {taken}, for every user. Anything "
|
||||
"made since is removed, edits since are undone, records deleted since come back.")
|
||||
else:
|
||||
print(f"This instance has no records yet; it takes the backup from {taken} as it is.")
|
||||
for count in preview["counts"]:
|
||||
print(f" {count['kind']:<24} now {count['now']:>7} after {count['after']:>7}")
|
||||
|
||||
|
||||
async def run(path: Path, *, yes: bool = False) -> int:
|
||||
from grimoire.models import engine
|
||||
from grimoire.services import backup_job
|
||||
from grimoire.services.backup import RestoreRefused, replace_preview
|
||||
from grimoire.services.backup_file import ArchiveError
|
||||
|
||||
try:
|
||||
@@ -63,20 +99,29 @@ async def run(path: Path) -> int:
|
||||
print(f"{path} isn't a backup this Grimoire can read: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
accounts = await _account_count()
|
||||
if accounts:
|
||||
print(
|
||||
f"This database already holds {accounts} account(s). A restore adds a second "
|
||||
"copy of everything rather than replacing it, so it only runs into an empty "
|
||||
"instance. Start Grimoire on an empty database and run this again.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 3
|
||||
|
||||
stats = await restore_full_backup(payload)
|
||||
print(json.dumps({"restored": stats}, indent=2, default=str))
|
||||
print("Restored. Restart Grimoire so the search indexes are rebuilt.", file=sys.stderr)
|
||||
return 0
|
||||
if (payload.get("version") or 1) < 2:
|
||||
code = await _restore_v1(payload)
|
||||
else:
|
||||
try:
|
||||
_print_preview(await replace_preview(payload))
|
||||
if not yes:
|
||||
print("Nothing was changed. Run it again with --yes to restore.",
|
||||
file=sys.stderr)
|
||||
return 4
|
||||
result = await backup_job.rollback_to_backup(payload)
|
||||
except RestoreRefused as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
return 3
|
||||
safety = result["safety_backup"]
|
||||
print(json.dumps({"restored": result["restored"], "safety_backup": safety},
|
||||
indent=2, default=str))
|
||||
print(f"The state it replaced is saved as {safety['file']} in {safety['where']}.",
|
||||
file=sys.stderr)
|
||||
code = 0
|
||||
if code == 0:
|
||||
print("Restored. Restart Grimoire so the search indexes are rebuilt; "
|
||||
"everyone signs in again.", file=sys.stderr)
|
||||
return code
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
@@ -84,11 +129,16 @@ async def run(path: Path) -> int:
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="python -m grimoire.restore",
|
||||
description="Restore a Grimoire backup (.tar or .json) into an empty instance.",
|
||||
description=(
|
||||
"Restore a Grimoire backup (.tar or .json): into an empty instance, or "
|
||||
"roll this instance back to a backup of its own."
|
||||
),
|
||||
)
|
||||
parser.add_argument("file", type=Path, help="the backup file")
|
||||
parser.add_argument("--yes", action="store_true",
|
||||
help="restore; without it, only say what would change")
|
||||
args = parser.parse_args(argv)
|
||||
return asyncio.run(run(args.file))
|
||||
return asyncio.run(run(args.file, yes=args.yes))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+155
-35
@@ -3,7 +3,11 @@ import json
|
||||
|
||||
from quart import Blueprint, Response, g, jsonify, request
|
||||
|
||||
from grimoire import proxy
|
||||
from grimoire.auth import admin_required, login_required, get_current_user_id
|
||||
from grimoire.config import Config
|
||||
from grimoire.proxy import client_ip
|
||||
from grimoire.services import proxy_settings
|
||||
from grimoire.services.auth import (
|
||||
create_invitation,
|
||||
delete_user,
|
||||
@@ -16,7 +20,6 @@ from grimoire.services.auth import (
|
||||
from grimoire.services.backup import (
|
||||
export_full_backup,
|
||||
export_user_backup,
|
||||
restore_full_backup,
|
||||
)
|
||||
from grimoire.services.email import SMTP_SETTING_KEYS, get_base_url, get_smtp_config, is_smtp_configured, send_test_email
|
||||
from grimoire.services.logging import get_logs, get_log_stats, log_audit
|
||||
@@ -46,7 +49,7 @@ async def backup():
|
||||
else:
|
||||
data = await export_user_backup(uid)
|
||||
|
||||
await log_audit("backup", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"scope": scope})
|
||||
await log_audit("backup", user_id=uid, username=g.user.username, ip_address=client_ip(), details={"scope": scope})
|
||||
return Response(
|
||||
json.dumps(data, indent=2, default=str),
|
||||
content_type="application/json",
|
||||
@@ -56,31 +59,111 @@ async def backup():
|
||||
)
|
||||
|
||||
|
||||
async def _restore_payload() -> tuple[dict | None, tuple | None]:
|
||||
"""The backup a restore request names: `?source=<file>` at the backup
|
||||
destination, or the uploaded file (the JSON download as
|
||||
application/json, or the tar as the raw body; held to the request-size
|
||||
limit, which is why the destination is the way in for a large one).
|
||||
Returns (payload, None) or (None, an error response)."""
|
||||
import io
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from grimoire.services import backup_destination as dest
|
||||
from grimoire.services.backup_file import ArchiveError, read_archive
|
||||
|
||||
source = request.args.get("source")
|
||||
try:
|
||||
if source:
|
||||
config = await dest.load_config()
|
||||
secret, _ = await dest.load_secret()
|
||||
with tempfile.TemporaryDirectory(prefix="grimoire-restore-") as staging:
|
||||
path = await dest.open_backup(config, secret, source, Path(staging))
|
||||
|
||||
def load() -> dict:
|
||||
with open(path, "rb") as f:
|
||||
return read_archive(f)
|
||||
data = await asyncio.to_thread(load)
|
||||
elif request.mimetype == "application/json":
|
||||
data = await request.get_json()
|
||||
else:
|
||||
data = read_archive(io.BytesIO(await request.get_data()))
|
||||
except dest.DestinationError as exc:
|
||||
return None, (jsonify({"error": str(exc)}), 400)
|
||||
except ArchiveError as exc:
|
||||
return None, (jsonify({"error": f"That file isn't a Grimoire backup: {exc}"}), 400)
|
||||
if not isinstance(data, dict) or not data:
|
||||
return None, (jsonify({"error": "No backup data provided"}), 400)
|
||||
return data, None
|
||||
|
||||
|
||||
@admin_bp.route("/restore/sources", methods=["GET"])
|
||||
@admin_required
|
||||
async def restore_sources():
|
||||
"""The backups at the backup destination, newest first, to restore from
|
||||
without uploading one."""
|
||||
from grimoire.services import backup_destination as dest
|
||||
|
||||
config = await dest.load_config()
|
||||
secret, _ = await dest.load_secret()
|
||||
try:
|
||||
backups = await asyncio.wait_for(dest.list_backups(config, secret), _BACKUP_CHECK_TIMEOUT)
|
||||
except dest.DestinationError as exc:
|
||||
return jsonify({"backups": [], "error": str(exc)})
|
||||
except (TimeoutError, asyncio.TimeoutError):
|
||||
return jsonify({"backups": [], "error": (
|
||||
f"The destination didn't answer within {int(_BACKUP_CHECK_TIMEOUT)} seconds."
|
||||
)})
|
||||
return jsonify({"backups": [{"name": b.name, "size_bytes": b.size_bytes} for b in backups]})
|
||||
|
||||
|
||||
@admin_bp.route("/restore/preview", methods=["POST"])
|
||||
@admin_required
|
||||
async def restore_preview():
|
||||
"""What restoring this backup would do, changing nothing: fresh or
|
||||
rollback, when it was taken, and the counts now and after. 409 with the
|
||||
reason when it can't be restored here."""
|
||||
from grimoire.services.backup import RestoreRefused, replace_preview
|
||||
|
||||
data, error = await _restore_payload()
|
||||
if error:
|
||||
return error
|
||||
try:
|
||||
return jsonify(await replace_preview(data))
|
||||
except RestoreRefused as exc:
|
||||
return jsonify({"error": str(exc)}), 409
|
||||
|
||||
|
||||
@admin_bp.route("/restore", methods=["POST"])
|
||||
@admin_required
|
||||
async def restore():
|
||||
"""Restore from an uploaded backup: the JSON download (application/json)
|
||||
or the scheduled backup's tar (sent as the raw body). The upload is held
|
||||
to the app's request-size limit; `python -m grimoire.restore` has none, and
|
||||
is the way in for a bigger file or a fresh instance."""
|
||||
if request.mimetype == "application/json":
|
||||
data = await request.get_json()
|
||||
else:
|
||||
import io
|
||||
"""Put the whole instance back to a full backup (services/backup.py
|
||||
replace_from_backup), after backing up what it replaces. `?confirm=rollback`
|
||||
is required: this removes everything made since the backup, for every
|
||||
user. Every session ends, the caller's included; search is rebuilt in the
|
||||
background."""
|
||||
from grimoire.services import backup_job as job
|
||||
from grimoire.services.backup import RestoreRefused
|
||||
|
||||
from grimoire.services.backup_file import ArchiveError, read_archive
|
||||
|
||||
try:
|
||||
data = read_archive(io.BytesIO(await request.get_data()))
|
||||
except ArchiveError as exc:
|
||||
return jsonify({"error": f"That file isn't a Grimoire backup: {exc}"}), 400
|
||||
if not data:
|
||||
return jsonify({"error": "No backup data provided"}), 400
|
||||
|
||||
stats = await restore_full_backup(data)
|
||||
if request.args.get("confirm") != "rollback":
|
||||
return jsonify({"error": "Restoring replaces the whole instance; confirm it first."}), 400
|
||||
data, error = await _restore_payload()
|
||||
if error:
|
||||
return error
|
||||
try:
|
||||
result = await job.rollback_to_backup(data)
|
||||
except RestoreRefused as exc:
|
||||
return jsonify({"error": str(exc)}), 409
|
||||
except job.BackupBusy:
|
||||
return jsonify({"error": "A backup is running. Restore once it has finished."}), 409
|
||||
job.start_reindex_after_restore()
|
||||
uid = get_current_user_id()
|
||||
await log_audit("restore", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"stats": stats})
|
||||
return jsonify({"status": "ok", "stats": stats})
|
||||
await log_audit(
|
||||
"restore", user_id=uid, username=g.user.username, ip_address=client_ip(),
|
||||
details={"mode": result["mode"], "exported_at": result.get("exported_at"),
|
||||
"safety_backup": result["safety_backup"], "stats": result["restored"]},
|
||||
)
|
||||
return jsonify({"status": "ok", **result})
|
||||
|
||||
|
||||
@admin_bp.route("/users", methods=["GET"])
|
||||
@@ -100,7 +183,7 @@ async def remove_user(user_id: int):
|
||||
deleted = await delete_user(user_id)
|
||||
if not deleted:
|
||||
return jsonify({"error": "User not found"}), 404
|
||||
await log_audit("user_delete", user_id=current_uid, username=g.user.username, ip_address=request.remote_addr, details={"deleted_user_id": user_id})
|
||||
await log_audit("user_delete", user_id=current_uid, username=g.user.username, ip_address=client_ip(), details={"deleted_user_id": user_id})
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
|
||||
@@ -121,7 +204,7 @@ async def toggle_registration():
|
||||
|
||||
uid = get_current_user_id()
|
||||
await set_registration_open(uid, bool(open_val))
|
||||
await log_audit("registration_toggle", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"open": bool(open_val)})
|
||||
await log_audit("registration_toggle", user_id=uid, username=g.user.username, ip_address=client_ip(), details={"open": bool(open_val)})
|
||||
return jsonify({"status": "ok", "open": bool(open_val)})
|
||||
|
||||
|
||||
@@ -151,7 +234,7 @@ async def update_smtp():
|
||||
|
||||
if settings_to_save:
|
||||
await set_settings_batch(uid, settings_to_save)
|
||||
await log_audit("smtp_config", user_id=uid, username=g.user.username, ip_address=request.remote_addr)
|
||||
await log_audit("smtp_config", user_id=uid, username=g.user.username, ip_address=client_ip())
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
|
||||
@@ -166,7 +249,7 @@ async def test_smtp():
|
||||
uid = get_current_user_id()
|
||||
try:
|
||||
await send_test_email(recipient)
|
||||
await log_audit("smtp_test", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"recipient": recipient})
|
||||
await log_audit("smtp_test", user_id=uid, username=g.user.username, ip_address=client_ip(), details={"recipient": recipient})
|
||||
return jsonify({"status": "ok"})
|
||||
except Exception as e:
|
||||
return jsonify({"error": str(e)}), 500
|
||||
@@ -180,7 +263,6 @@ async def test_smtp():
|
||||
@admin_bp.route("/forge-webhook", methods=["GET"])
|
||||
@admin_required
|
||||
async def get_forge_webhook_settings():
|
||||
from grimoire.config import Config
|
||||
from grimoire.routes.webhooks import FORGE_WEBHOOK_SECRET_KEY
|
||||
|
||||
webhook_secret = (
|
||||
@@ -209,7 +291,7 @@ async def update_forge_webhook_settings():
|
||||
# The secret is deliberately absent from the audit detail.
|
||||
await log_audit(
|
||||
"forge_webhook_config", user_id=uid, username=g.user.username,
|
||||
ip_address=request.remote_addr, details={},
|
||||
ip_address=client_ip(), details={},
|
||||
)
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
@@ -262,10 +344,48 @@ async def update_base_url():
|
||||
return jsonify({"error": "Base URL must use http or https"}), 400
|
||||
uid = get_current_user_id()
|
||||
await set_setting(uid, "base_url", url)
|
||||
await log_audit("base_url_config", user_id=uid, username=g.user.username, ip_address=request.remote_addr, details={"base_url": url})
|
||||
await log_audit("base_url_config", user_id=uid, username=g.user.username, ip_address=client_ip(), details={"base_url": url})
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
|
||||
def _proxy_payload() -> dict:
|
||||
return {
|
||||
"hops": proxy.trusted_hops(),
|
||||
"source": proxy_settings.source(),
|
||||
"env_hops": Config.TRUSTED_PROXY_HOPS,
|
||||
"max_hops": proxy.MAX_HOPS,
|
||||
"readout": proxy.request_readout(),
|
||||
}
|
||||
|
||||
|
||||
@admin_bp.route("/proxy", methods=["GET"])
|
||||
@admin_required
|
||||
async def get_proxy_setting():
|
||||
"""The hop count in force, where it came from, and what each count would
|
||||
make of the admin's own request."""
|
||||
return jsonify(_proxy_payload())
|
||||
|
||||
|
||||
@admin_bp.route("/proxy", methods=["PUT"])
|
||||
@admin_required
|
||||
async def update_proxy_setting():
|
||||
"""Save the hop count, or `{"hops": null}` to go back to the environment's.
|
||||
It applies to the next request."""
|
||||
data = await request.get_json() or {}
|
||||
raw = data.get("hops")
|
||||
hops = None if raw is None else proxy_settings.parse_hops(raw)
|
||||
if raw is not None and hops is None:
|
||||
return jsonify({"error": f"The proxy count must be a whole number from 0 to {proxy.MAX_HOPS}."}), 400
|
||||
before = proxy.trusted_hops()
|
||||
await proxy_settings.save_hops(hops)
|
||||
uid = get_current_user_id()
|
||||
await log_audit(
|
||||
"proxy_hops_config", user_id=uid, username=g.user.username, ip_address=client_ip(),
|
||||
details={"before": before, "after": proxy.trusted_hops(), "saved": hops},
|
||||
)
|
||||
return jsonify(_proxy_payload())
|
||||
|
||||
|
||||
@admin_bp.route("/db-maintenance", methods=["GET"])
|
||||
@admin_required
|
||||
async def get_db_maintenance():
|
||||
@@ -310,7 +430,7 @@ async def update_db_maintenance():
|
||||
uid = get_current_user_id()
|
||||
await log_audit(
|
||||
"db_maintenance_config", user_id=uid, username=g.user.username,
|
||||
ip_address=request.remote_addr, details={"enabled": enabled, "hour": hour},
|
||||
ip_address=client_ip(), details={"enabled": enabled, "hour": hour},
|
||||
)
|
||||
return jsonify({"status": "ok", "enabled": enabled, "hour": hour})
|
||||
|
||||
@@ -323,7 +443,7 @@ async def run_db_maintenance_now():
|
||||
uid = get_current_user_id()
|
||||
await log_audit(
|
||||
"db_maintenance_run", user_id=uid, username=g.user.username,
|
||||
ip_address=request.remote_addr,
|
||||
ip_address=client_ip(),
|
||||
)
|
||||
summary = await run_maintenance()
|
||||
return jsonify(summary)
|
||||
@@ -389,7 +509,7 @@ async def update_backups():
|
||||
uid = get_current_user_id()
|
||||
await log_audit(
|
||||
"backup_config", user_id=uid, username=g.user.username,
|
||||
ip_address=request.remote_addr,
|
||||
ip_address=client_ip(),
|
||||
details={**config.to_dict(), "secret_changed": bool(
|
||||
data.get("clear_s3_secret") or (isinstance(secret, str) and secret and secret != SECRET_MASK)
|
||||
)},
|
||||
@@ -439,7 +559,7 @@ async def run_backup_now():
|
||||
return jsonify({"error": "A backup is already running."}), 409
|
||||
uid = get_current_user_id()
|
||||
await log_audit("backup_run", user_id=uid, username=g.user.username,
|
||||
ip_address=request.remote_addr, details={})
|
||||
ip_address=client_ip(), details={})
|
||||
return jsonify({"status": "started"}), 202
|
||||
|
||||
|
||||
@@ -463,7 +583,7 @@ async def create_invite():
|
||||
"invitation_created",
|
||||
user_id=uid,
|
||||
username=g.user.username,
|
||||
ip_address=request.remote_addr,
|
||||
ip_address=client_ip(),
|
||||
details={"invited_email": email},
|
||||
)
|
||||
return jsonify({"status": "ok"}), 201
|
||||
@@ -497,7 +617,7 @@ async def delete_invitation(invitation_id: int):
|
||||
"invitation_revoked",
|
||||
user_id=uid,
|
||||
username=g.user.username,
|
||||
ip_address=request.remote_addr,
|
||||
ip_address=client_ip(),
|
||||
details={"invitation_id": invitation_id},
|
||||
)
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
+24
-32
@@ -8,7 +8,8 @@ from quart import Blueprint, g, jsonify, redirect, request, session
|
||||
|
||||
from grimoire.auth import login_required, get_current_user_id
|
||||
from grimoire.config import Config
|
||||
from grimoire.rate_limit import is_rate_limited
|
||||
from grimoire.proxy import client_ip
|
||||
from grimoire.rate_limit import SHARED_ADDRESS_HINT, is_rate_limited
|
||||
from grimoire.services.auth import (
|
||||
authenticate,
|
||||
change_password,
|
||||
@@ -43,21 +44,12 @@ from grimoire.services.oauth import (
|
||||
auth_bp = Blueprint("auth", __name__, url_prefix="/api/auth")
|
||||
|
||||
|
||||
def _client_ip() -> str:
|
||||
if Config.TRUST_PROXY_HEADERS:
|
||||
for header in ("X-Forwarded-For", "X-Real-IP"):
|
||||
val = request.headers.get(header, "").split(",")[0].strip()
|
||||
if val:
|
||||
return val
|
||||
return request.remote_addr or ""
|
||||
|
||||
|
||||
@auth_bp.route("/register", methods=["POST"])
|
||||
async def register():
|
||||
if not Config.LOCAL_AUTH_ENABLED:
|
||||
return jsonify({"error": "Local authentication is disabled. Please use SSO."}), 403
|
||||
if await is_rate_limited(f"register:{_client_ip()}", max_requests=5, window_seconds=300):
|
||||
return jsonify({"error": "Too many registration attempts. Try again later."}), 429
|
||||
if await is_rate_limited(f"register:{client_ip()}", max_requests=5, window_seconds=300):
|
||||
return jsonify({"error": f"Too many registration attempts. Try again later. {SHARED_ADDRESS_HINT}"}), 429
|
||||
if not await is_registration_open():
|
||||
return jsonify({"error": "Registration is closed"}), 403
|
||||
|
||||
@@ -78,7 +70,7 @@ async def register():
|
||||
|
||||
session["user_id"] = user.id
|
||||
session["session_version"] = user.session_version
|
||||
await log_audit("register", user_id=user.id, username=user.username, ip_address=_client_ip())
|
||||
await log_audit("register", user_id=user.id, username=user.username, ip_address=client_ip())
|
||||
return jsonify(user.to_dict()), 201
|
||||
|
||||
|
||||
@@ -86,8 +78,8 @@ async def register():
|
||||
async def login():
|
||||
if not Config.LOCAL_AUTH_ENABLED:
|
||||
return jsonify({"error": "Local authentication is disabled. Please use SSO."}), 403
|
||||
if await is_rate_limited(f"login:{_client_ip()}", max_requests=10, window_seconds=60):
|
||||
return jsonify({"error": "Too many login attempts. Try again later."}), 429
|
||||
if await is_rate_limited(f"login:{client_ip()}", max_requests=10, window_seconds=60):
|
||||
return jsonify({"error": f"Too many login attempts. Try again later. {SHARED_ADDRESS_HINT}"}), 429
|
||||
data = await request.get_json()
|
||||
username = (data.get("username") or "").strip()
|
||||
password = data.get("password") or ""
|
||||
@@ -97,22 +89,22 @@ async def login():
|
||||
|
||||
user = await authenticate(username, password)
|
||||
if not user:
|
||||
await log_audit("login_failed", username=username, ip_address=_client_ip())
|
||||
await log_audit("login_failed", username=username, ip_address=client_ip())
|
||||
# Try to notify the actual user about the failed attempt
|
||||
target_user = await get_user_by_username(username)
|
||||
if target_user:
|
||||
asyncio.create_task(notify_security_event(
|
||||
target_user.id, "login_failed",
|
||||
{"ip_address": _client_ip(), "username": username},
|
||||
{"ip_address": client_ip(), "username": username},
|
||||
))
|
||||
return jsonify({"error": "Invalid username or password"}), 401
|
||||
|
||||
session["user_id"] = user.id
|
||||
session["session_version"] = user.session_version
|
||||
await log_audit("login", user_id=user.id, username=user.username, ip_address=_client_ip())
|
||||
await log_audit("login", user_id=user.id, username=user.username, ip_address=client_ip())
|
||||
asyncio.create_task(notify_security_event(
|
||||
user.id, "login",
|
||||
{"ip_address": _client_ip()},
|
||||
{"ip_address": client_ip()},
|
||||
))
|
||||
return jsonify(user.to_dict())
|
||||
|
||||
@@ -122,10 +114,10 @@ async def logout():
|
||||
uid = session.get("user_id")
|
||||
if uid:
|
||||
user = await get_user_by_id(uid)
|
||||
await log_audit("logout", user_id=uid, username=user.username if user else None, ip_address=_client_ip())
|
||||
await log_audit("logout", user_id=uid, username=user.username if user else None, ip_address=client_ip())
|
||||
asyncio.create_task(notify_security_event(
|
||||
uid, "logout",
|
||||
{"ip_address": _client_ip()},
|
||||
{"ip_address": client_ip()},
|
||||
))
|
||||
session.clear()
|
||||
return jsonify({"status": "ok"})
|
||||
@@ -159,10 +151,10 @@ async def update_password():
|
||||
|
||||
# Keep the current session alive with the new version (other sessions are now invalidated)
|
||||
session["session_version"] = new_version
|
||||
await log_audit("password_change", user_id=uid, username=g.user.username, ip_address=_client_ip())
|
||||
await log_audit("password_change", user_id=uid, username=g.user.username, ip_address=client_ip())
|
||||
asyncio.create_task(notify_security_event(
|
||||
uid, "password_change",
|
||||
{"ip_address": _client_ip()},
|
||||
{"ip_address": client_ip()},
|
||||
))
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
@@ -178,7 +170,7 @@ async def invalidate_sessions_route():
|
||||
uid = get_current_user_id()
|
||||
new_version = await invalidate_other_sessions(uid)
|
||||
session["session_version"] = new_version
|
||||
await log_audit("sessions_invalidated", user_id=uid, username=g.user.username, ip_address=_client_ip())
|
||||
await log_audit("sessions_invalidated", user_id=uid, username=g.user.username, ip_address=client_ip())
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
|
||||
@@ -208,14 +200,14 @@ async def update_email():
|
||||
return jsonify({"error": "Email already in use by another account"}), 409
|
||||
|
||||
await update_user_email(uid, new_email)
|
||||
await log_audit("email_change", user_id=uid, username=user.username, ip_address=_client_ip())
|
||||
await log_audit("email_change", user_id=uid, username=user.username, ip_address=client_ip())
|
||||
updated = await get_user_by_id(uid)
|
||||
return jsonify(updated.to_dict())
|
||||
|
||||
|
||||
@auth_bp.route("/forgot-password", methods=["POST"])
|
||||
async def forgot_password():
|
||||
if await is_rate_limited(f"forgot:{_client_ip()}", max_requests=5, window_seconds=300):
|
||||
if await is_rate_limited(f"forgot:{client_ip()}", max_requests=5, window_seconds=300):
|
||||
return jsonify({"status": "ok"})
|
||||
data = await request.get_json()
|
||||
email = (data.get("email") or "").strip().lower()
|
||||
@@ -234,7 +226,7 @@ async def forgot_password():
|
||||
"password_reset_requested",
|
||||
user_id=user.id,
|
||||
username=user.username,
|
||||
ip_address=_client_ip(),
|
||||
ip_address=client_ip(),
|
||||
)
|
||||
|
||||
return jsonify({"status": "ok"})
|
||||
@@ -242,8 +234,8 @@ async def forgot_password():
|
||||
|
||||
@auth_bp.route("/reset-password", methods=["POST"])
|
||||
async def reset_password():
|
||||
if await is_rate_limited(f"reset:{_client_ip()}", max_requests=10, window_seconds=60):
|
||||
return jsonify({"error": "Too many attempts. Try again later."}), 429
|
||||
if await is_rate_limited(f"reset:{client_ip()}", max_requests=10, window_seconds=60):
|
||||
return jsonify({"error": f"Too many attempts. Try again later. {SHARED_ADDRESS_HINT}"}), 429
|
||||
data = await request.get_json()
|
||||
token = (data.get("token") or "").strip()
|
||||
new_password = data.get("new_password") or ""
|
||||
@@ -261,7 +253,7 @@ async def reset_password():
|
||||
if user:
|
||||
await log_audit(
|
||||
"password_reset_completed",
|
||||
user_id=user.id, username=user.username, ip_address=_client_ip(),
|
||||
user_id=user.id, username=user.username, ip_address=client_ip(),
|
||||
)
|
||||
if user.email:
|
||||
asyncio.create_task(send_password_reset_success_email(user.email))
|
||||
@@ -304,7 +296,7 @@ async def register_with_invite():
|
||||
"register_with_invite",
|
||||
user_id=user.id,
|
||||
username=user.username,
|
||||
ip_address=_client_ip(),
|
||||
ip_address=client_ip(),
|
||||
)
|
||||
return jsonify(user.to_dict()), 201
|
||||
|
||||
@@ -371,5 +363,5 @@ async def oauth_callback():
|
||||
user = await find_or_create_oauth_user(sub, email, preferred_username)
|
||||
session["user_id"] = user.id
|
||||
session["session_version"] = user.session_version
|
||||
await log_audit("oauth_login", user_id=user.id, username=user.username, ip_address=_client_ip())
|
||||
await log_audit("oauth_login", user_id=user.id, username=user.username, ip_address=client_ip())
|
||||
return redirect("/")
|
||||
|
||||
@@ -8,6 +8,8 @@ fmcp_<key>` API key (a read-scoped key suffices for this GET).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from quart import Blueprint, g, jsonify, request
|
||||
|
||||
from grimoire.auth import admin_required, get_current_user_id, login_required
|
||||
@@ -21,6 +23,8 @@ from grimoire.services import report_check as report_check_svc
|
||||
from grimoire.services import rule_moment_judgments as judgments_svc
|
||||
from grimoire.services import shape_check as shape_check_svc
|
||||
from grimoire.services import task_claims as task_claims_svc
|
||||
from grimoire.services import write_scope as write_scope_svc
|
||||
from grimoire.services.access import can_read_project
|
||||
from grimoire.services.embeddings import memoized_query_embeddings
|
||||
from grimoire.services.settings import get_admin_setting, set_setting
|
||||
|
||||
@@ -626,6 +630,64 @@ async def claim_session():
|
||||
return jsonify({"claim": claim})
|
||||
|
||||
|
||||
@plugin_bp.get("/write-scope")
|
||||
@login_required
|
||||
async def write_scope():
|
||||
"""May this session do this act? Asked by `grimoire_project_fence.sh`
|
||||
before a file edit, a git command or a Grimoire write.
|
||||
|
||||
A session belongs to the project its starting directory resolves to
|
||||
(`repo` / `project_id`, as every plugin endpoint reads them), or when
|
||||
that resolves to none, to `fallback_project_id`: the first project the
|
||||
session entered. The act is
|
||||
either a directory (`target_repo` / `target_project_id`) or an MCP call
|
||||
(`tool`, with `input` as a JSON object of the call's short arguments: the
|
||||
ids, kind and status that say where it writes). `act` words the act for the
|
||||
reason, e.g. "git push in".
|
||||
|
||||
Returns `{allowed, session_project, targets, issue, reason}` from
|
||||
services/write_scope.check_act. A GET, so a read-scoped key still fences
|
||||
file edits and git commands.
|
||||
"""
|
||||
session_project, _repo, _unbound = await _project_scope()
|
||||
if not session_project:
|
||||
# A session that started outside any project belongs to the first one
|
||||
# it entered; the hook remembers it.
|
||||
try:
|
||||
session_project = int(request.args.get("fallback_project_id") or 0)
|
||||
except ValueError:
|
||||
session_project = 0
|
||||
if session_project and not await can_read_project(g.user.id, session_project):
|
||||
session_project = 0
|
||||
tool = (request.args.get("tool") or "").strip()
|
||||
if tool.startswith("mcp__"):
|
||||
tool = tool.rsplit("__", 1)[-1]
|
||||
args: dict = {}
|
||||
if tool:
|
||||
try:
|
||||
parsed = json.loads(request.args.get("input") or "{}")
|
||||
except ValueError:
|
||||
return jsonify({"error": "input must be a JSON object"}), 400
|
||||
if not isinstance(parsed, dict):
|
||||
return jsonify({"error": "input must be a JSON object"}), 400
|
||||
args = parsed
|
||||
target_project = 0
|
||||
if not tool:
|
||||
try:
|
||||
target_project = int(request.args.get("target_project_id") or 0)
|
||||
except ValueError:
|
||||
target_project = 0
|
||||
target_repo = (request.args.get("target_repo") or "").strip()
|
||||
if not target_project and target_repo:
|
||||
target_project = await repo_bindings_svc.resolve_project(g.user.id, target_repo) or 0
|
||||
result = await write_scope_svc.check_act(
|
||||
g.user.id, session_project, tool=tool, args=args,
|
||||
target_project=target_project, act=(request.args.get("act") or "")[:80],
|
||||
)
|
||||
result["session_project"] = session_project or None
|
||||
return jsonify(result)
|
||||
|
||||
|
||||
@plugin_bp.get("/release-session")
|
||||
@login_required
|
||||
async def release_session():
|
||||
|
||||
+686
-414
File diff suppressed because it is too large
Load Diff
@@ -4,8 +4,9 @@ Two kinds of destination, both instance-wide and set by an admin:
|
||||
|
||||
- **folder**: a directory inside the container, usually a mounted share. A
|
||||
backup service that watches the share (or a person) takes it from there.
|
||||
- **s3**: any S3-compatible bucket, addressed by path. Grimoire only puts and
|
||||
lists, so an append-only key is enough.
|
||||
- **s3**: any S3-compatible bucket, addressed by path. Grimoire puts and
|
||||
lists, and reads one back for a restore, so a key that can add and read
|
||||
objects is enough; it never needs to delete.
|
||||
|
||||
Either way Grimoire ADDS one dated file per run and never removes one: keeping
|
||||
and thinning the history is the destination's job (services/backup_file.py
|
||||
@@ -28,7 +29,7 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from grimoire.services import backup_file
|
||||
from grimoire.services.backup_s3 import S3Error, S3Target, list_objects, put_file
|
||||
from grimoire.services.backup_s3 import S3Error, S3Target, get_file, list_objects, put_file
|
||||
from grimoire.services.secrets import seal, unseal
|
||||
from grimoire.services.settings import get_admin_setting, set_admin_setting
|
||||
|
||||
@@ -260,3 +261,62 @@ async def check_destination(
|
||||
|
||||
def _count_backups(folder: Path) -> int:
|
||||
return sum(1 for name in os.listdir(folder) if backup_file.is_backup_name(name))
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class StoredBackup:
|
||||
name: str
|
||||
size_bytes: int
|
||||
|
||||
|
||||
def _taken_order(name: str) -> str:
|
||||
# The timestamp after the prefix, so grimoire- and scribe- names interleave
|
||||
# by when they were taken.
|
||||
return name.split("-", 1)[-1]
|
||||
|
||||
|
||||
async def list_backups(config: BackupConfig, secret: str | None, limit: int = 50) -> list[StoredBackup]:
|
||||
"""The backups at the destination, newest first."""
|
||||
if config.destination == "folder":
|
||||
folder = Path(config.folder_path)
|
||||
if not folder.is_dir():
|
||||
raise DestinationError(f"{folder} isn't there, or isn't a folder.")
|
||||
|
||||
def scan() -> list[StoredBackup]:
|
||||
return [
|
||||
StoredBackup(entry.name, entry.stat().st_size)
|
||||
for entry in os.scandir(folder)
|
||||
if entry.is_file() and backup_file.is_backup_name(entry.name)
|
||||
]
|
||||
found = await asyncio.to_thread(scan)
|
||||
else:
|
||||
problems = config.problems(has_secret=bool(secret))
|
||||
if problems:
|
||||
raise DestinationError(" ".join(problems))
|
||||
try:
|
||||
objects = await list_objects(s3_target(config, secret or ""))
|
||||
except S3Error as exc:
|
||||
raise DestinationError(str(exc)) from None
|
||||
found = [StoredBackup(o.key, o.size) for o in objects if backup_file.is_backup_name(o.key)]
|
||||
return sorted(found, key=lambda b: _taken_order(b.name), reverse=True)[:limit]
|
||||
|
||||
|
||||
async def open_backup(config: BackupConfig, secret: str | None, name: str, staging: Path) -> Path:
|
||||
"""A local path to the backup `name` at the destination: the file itself
|
||||
in a folder, or a download into `staging` from a bucket. Only a name the
|
||||
scheduled backup writes is accepted, so the request can't reach any other
|
||||
file or object."""
|
||||
if Path(name).name != name or not backup_file.is_backup_name(name):
|
||||
raise DestinationError(f"{name!r} isn't the name of a Grimoire backup.")
|
||||
if config.destination == "folder":
|
||||
path = Path(config.folder_path) / name
|
||||
if not path.is_file():
|
||||
raise DestinationError(f"{name} isn't in {config.folder_path}.")
|
||||
return path
|
||||
target = s3_target(config, secret or "")
|
||||
path = staging / name
|
||||
try:
|
||||
await get_file(target, name, path)
|
||||
except S3Error as exc:
|
||||
raise DestinationError(str(exc)) from None
|
||||
return path
|
||||
|
||||
@@ -24,11 +24,14 @@ import os
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
|
||||
from grimoire.services.backup_destination import (
|
||||
DestinationError, deliver, load_config, load_secret,
|
||||
)
|
||||
from grimoire.services.backup_file import write_archive
|
||||
from grimoire.services.scheduler import ScheduledJob
|
||||
from grimoire.services.settings import get_admin_setting, set_admin_setting
|
||||
|
||||
@@ -45,6 +48,9 @@ STALE_AFTER = timedelta(hours=26)
|
||||
# has none, and a run that never returns holds the lock — every later backup
|
||||
# would be skipped as "already running", silently, forever.
|
||||
RUN_DEADLINE = timedelta(minutes=60)
|
||||
# Under DATA_DIR: where a rollback keeps the backup of what it replaced when
|
||||
# no backup destination takes it.
|
||||
PRE_RESTORE_DIR = "pre-restore"
|
||||
|
||||
_LOCK = asyncio.Lock()
|
||||
_TASKS: set[asyncio.Task] = set()
|
||||
@@ -155,6 +161,66 @@ async def _export_and_deliver(config, secret, started):
|
||||
)
|
||||
|
||||
|
||||
async def save_state_before_restore() -> dict:
|
||||
"""Back up the store as it is now, before a rollback replaces it, and say
|
||||
where the file went: the backup destination when backups are on and it
|
||||
takes the file, otherwise DATA_DIR/pre-restore. Raises when neither takes
|
||||
it, so a rollback never runs without a way back."""
|
||||
from grimoire.config import Config
|
||||
from grimoire.services.backup import export_full_backup
|
||||
|
||||
at = datetime.now(timezone.utc)
|
||||
app_version = os.environ.get("APP_VERSION", "dev")
|
||||
payload = await export_full_backup()
|
||||
note = None
|
||||
config = await load_config()
|
||||
if config.enabled:
|
||||
secret, _ = await load_secret()
|
||||
try:
|
||||
delivered = await asyncio.wait_for(
|
||||
deliver(payload, config, secret, at, app_version=app_version),
|
||||
RUN_DEADLINE.total_seconds(),
|
||||
)
|
||||
return {"file": delivered.name, "size_bytes": delivered.size_bytes,
|
||||
"where": delivered.where}
|
||||
except (DestinationError, TimeoutError, asyncio.TimeoutError) as exc:
|
||||
note = (f"The backup destination didn't take it ({exc or 'it timed out'}), "
|
||||
"so it was kept on the instance instead.")
|
||||
folder = Path(Config.DATA_DIR) / PRE_RESTORE_DIR
|
||||
path = await asyncio.to_thread(write_archive, payload, folder, at, app_version=app_version)
|
||||
saved = {"file": path.name, "size_bytes": path.stat().st_size, "where": str(folder)}
|
||||
if note:
|
||||
saved["note"] = note
|
||||
return saved
|
||||
|
||||
|
||||
async def rollback_to_backup(data: dict) -> dict:
|
||||
"""Roll the store back to a full backup, after backing up what it
|
||||
replaces. Refuses (RestoreRefused) before writing anything when the backup
|
||||
can't be applied here, and raises BackupBusy while a backup is running:
|
||||
a backup taken during a rollback would mix the two states."""
|
||||
from grimoire.services.backup import check_replace, replace_from_backup
|
||||
|
||||
await check_replace(data)
|
||||
if _LOCK.locked():
|
||||
raise BackupBusy()
|
||||
async with _LOCK:
|
||||
safety = await save_state_before_restore()
|
||||
result = await replace_from_backup(data)
|
||||
result["safety_backup"] = safety
|
||||
logger.info("rollback: the state it replaced is in %s/%s", safety["where"], safety["file"])
|
||||
return result
|
||||
|
||||
|
||||
def start_reindex_after_restore() -> None:
|
||||
"""Rebuild the search vectors a rollback cleared, in the background."""
|
||||
from grimoire.services.backup import reindex_after_restore
|
||||
|
||||
task = asyncio.get_running_loop().create_task(reindex_after_restore())
|
||||
_TASKS.add(task)
|
||||
task.add_done_callback(_TASKS.discard)
|
||||
|
||||
|
||||
def start_backup_now() -> None:
|
||||
"""Start a manual backup in the background (the request returns at once;
|
||||
the page reads progress from `is_running` and the run list). Raises
|
||||
|
||||
@@ -4,9 +4,10 @@ path-style addressing (endpoint/bucket/key), and the two calls a backup needs.
|
||||
The signing is checked against AWS's published SigV4 examples in
|
||||
tests/test_services_backup_s3.py.
|
||||
|
||||
Grimoire puts and lists; it never deletes. The bucket keeps the history and
|
||||
thins it on its own policy, so a key that may only add objects is enough —
|
||||
and is the safer key to hand a service, because a stolen one cannot wipe the
|
||||
Grimoire puts and lists, and reads a backup back when an admin restores from
|
||||
the bucket; it never deletes. The bucket keeps the history and thins it on
|
||||
its own policy, so a key that may only add and read objects is enough — and
|
||||
is the safer key to hand a service, because a stolen one cannot wipe the
|
||||
history it writes into.
|
||||
|
||||
Written by hand rather than with boto3: two calls don't justify a large
|
||||
@@ -120,6 +121,12 @@ def _says(response: httpx.Response) -> str:
|
||||
"AccessDenied": "the key isn't allowed to do that",
|
||||
}
|
||||
said = hints.get(code) or message or response.reason_phrase
|
||||
if not code and 400 <= response.status_code < 500:
|
||||
# S3 refuses with an <Error><Code> body. A 4xx without one came from
|
||||
# something else at that address — most often a web server on the
|
||||
# default port, when the S3 service listens on a port of its own.
|
||||
said = ("the address answered, but not as S3 — check the scheme and "
|
||||
"the port (an S3 service usually listens on its own port)")
|
||||
return f"HTTP {response.status_code}{f' {code}' if code else ''}: {said}."
|
||||
|
||||
|
||||
@@ -165,6 +172,28 @@ async def put_file(target: S3Target, key: str, path: Path, *,
|
||||
raise S3Error(f"Upload of {key} refused: {_says(response)}")
|
||||
|
||||
|
||||
async def get_file(target: S3Target, key: str, path: Path, *,
|
||||
transport: httpx.AsyncBaseTransport | None = None,
|
||||
now: datetime | None = None) -> None:
|
||||
"""Download one object to `path`, streamed to disk."""
|
||||
url = target.url(key)
|
||||
headers = sign("GET", url, {}, EMPTY_SHA256, target.access_key, target.secret_key,
|
||||
target.region, now or datetime.now(timezone.utc))
|
||||
try:
|
||||
async with httpx.AsyncClient(verify=target.verify_tls,
|
||||
timeout=UPLOAD_TIMEOUT_SECONDS,
|
||||
transport=transport) as client:
|
||||
async with client.stream("GET", url, headers=headers) as response:
|
||||
if response.status_code != 200:
|
||||
await response.aread()
|
||||
raise S3Error(f"Reading {key} refused: {_says(response)}")
|
||||
with open(path, "wb") as f:
|
||||
async for chunk in response.aiter_bytes(CHUNK):
|
||||
f.write(chunk)
|
||||
except httpx.HTTPError as exc:
|
||||
raise _reach_error(target, exc) from None
|
||||
|
||||
|
||||
async def list_objects(target: S3Target, prefix: str = "", limit: int = 1000, *,
|
||||
transport: httpx.AsyncBaseTransport | None = None,
|
||||
now: datetime | None = None) -> list[StoredObject]:
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
"""This instance's id lineage (models/instance.py says what it means)."""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.dialects.postgresql import insert
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.instance import InstanceIdentity
|
||||
|
||||
|
||||
async def _read_or_make(session: AsyncSession) -> str:
|
||||
current = (await session.execute(select(InstanceIdentity.instance_id))).scalar_one_or_none()
|
||||
if current:
|
||||
return current
|
||||
# The migration makes the row; this covers a schema built some other way.
|
||||
await session.execute(
|
||||
insert(InstanceIdentity)
|
||||
.values(id=1, instance_id=str(uuid.uuid4()))
|
||||
.on_conflict_do_nothing(index_elements=["id"])
|
||||
)
|
||||
return (await session.execute(select(InstanceIdentity.instance_id))).scalar_one()
|
||||
|
||||
|
||||
async def get_instance_id(session: AsyncSession | None = None) -> str:
|
||||
"""This instance's id. Inside `session` when given, so a caller's
|
||||
transaction sees the same value it may be about to change."""
|
||||
if session is not None:
|
||||
return await _read_or_make(session)
|
||||
async with async_session() as own:
|
||||
value = await _read_or_make(own)
|
||||
await own.commit()
|
||||
return value
|
||||
|
||||
|
||||
async def adopt_instance_id(session: AsyncSession, value: str) -> None:
|
||||
"""Take a backup's id as this instance's own, inside the caller's
|
||||
transaction: after a restore that kept the backup's original ids, this
|
||||
store and the one the backup came from number their rows the same way."""
|
||||
await session.execute(
|
||||
insert(InstanceIdentity)
|
||||
.values(id=1, instance_id=value)
|
||||
.on_conflict_do_update(index_elements=["id"], set_={"instance_id": value})
|
||||
)
|
||||
@@ -324,6 +324,34 @@ async def lessons_for_rule(user_id: int, rule_id: int) -> list[dict]:
|
||||
]
|
||||
|
||||
|
||||
async def confirmed_lesson_counts(user_id: int, rule_ids) -> dict[int, int]:
|
||||
"""{rule_id: confirmed lessons the caller can read}, for a list of rules.
|
||||
|
||||
`lessons_for_rule` in bulk, counted rather than listed: one GROUP BY, so a
|
||||
rule list can say which rules are proven by situations without a query
|
||||
per row. Rules with none read 0.
|
||||
"""
|
||||
from grimoire.services.access import readable_notes_clause
|
||||
|
||||
ids = [int(i) for i in rule_ids or []]
|
||||
out = {i: 0 for i in ids}
|
||||
if not ids:
|
||||
return out
|
||||
async with async_session() as session:
|
||||
rows = (await session.execute(
|
||||
select(LessonRuleLink.rule_id, func.count())
|
||||
.join(Note, Note.id == LessonRuleLink.lesson_id)
|
||||
.where(LessonRuleLink.rule_id.in_(ids))
|
||||
.where(LessonRuleLink.state == CONFIRMED)
|
||||
.where(Note.deleted_at.is_(None))
|
||||
.where(readable_notes_clause(user_id))
|
||||
.group_by(LessonRuleLink.rule_id)
|
||||
)).all()
|
||||
for rule_id, n in rows:
|
||||
out[int(rule_id)] = int(n)
|
||||
return out
|
||||
|
||||
|
||||
async def judgments_for_lessons(lesson_ids) -> dict[int, dict]:
|
||||
"""{lesson_id: {"rule_judgment": linked|no_rule|unjudged, "no_rule"?}}.
|
||||
|
||||
|
||||
@@ -143,8 +143,13 @@ WRITEPATH_ENABLED_KEY = "kb_writepath_enabled"
|
||||
WRITEPATH_THRESHOLD_KEY = SURFACES["write_path"].floor_key
|
||||
WRITEPATH_DEFAULT_ENABLED = True
|
||||
WRITEPATH_DEFAULT_THRESHOLD = SURFACES["write_path"].floor_default
|
||||
# How far past the write-path floor a snippet must resemble a NEW definition
|
||||
# before the hint asks whether the code is a copy of it. A margin, not a second
|
||||
# floor, so it moves when the operator tunes the write-path floor: at the
|
||||
# default 0.68 the copy bar is 0.80.
|
||||
WRITEPATH_COPY_MARGIN = 0.12
|
||||
|
||||
# The standing-rule arm gets its own bar, for a third corpus: 0.68 above is
|
||||
# The standing-rule arms get their own bar, for a third corpus: 0.68 above is
|
||||
# calibrated on code against note PROSE, not code against RULE TEXT.
|
||||
#
|
||||
# THE STRUCTURAL ARGUMENT, which holds on any install, including one with six
|
||||
@@ -155,64 +160,27 @@ WRITEPATH_DEFAULT_THRESHOLD = SURFACES["write_path"].floor_default
|
||||
# pool always returns something, so "the best match cleared the bar"
|
||||
# drifts from "a good match exists" toward "N things were ranked". A bar
|
||||
# calibrated for best-of-thousands is cleared by best-of-forty as
|
||||
# arithmetic rather than relevance. (The arms search every tier, which
|
||||
# makes the pool larger; the bar still errs toward silence on purpose.)
|
||||
# arithmetic rather than relevance.
|
||||
# 2. Rules are short imperative technical English — a far more HOMOGENEOUS
|
||||
# corpus than note prose, and a more homogeneous corpus has a HIGHER
|
||||
# noise floor than prose's 0.55-0.63.
|
||||
#
|
||||
# WHY 0.72 AND NOT A NUMBER OFF A HISTOGRAM. The exact offset between prose's
|
||||
# floor and rule-text's is not derivable in general — it depends on how an
|
||||
# install writes its rules — so the default errs deliberately toward SILENCE
|
||||
# rather than toward recall, on an asymmetry that is itself structural: this
|
||||
# hint fires on EVERY write. A missed rule is recoverable, because the rule is
|
||||
# still in Grimoire and the agent can search it. A hint that cries wolf is not:
|
||||
# it teaches the reader to skip the whole block, and the surface is lost along
|
||||
# with the true positives it would have carried.
|
||||
# ONE BAR FOR EVERY RULE ARM THAT ANNOTATES. The write path, the command arm
|
||||
# and the prompt arm read the same key (`RULE_FLOOR_KEY` in retrieval_pipeline):
|
||||
# they search the same rules, and what limits each one is its budget and band.
|
||||
#
|
||||
# TUNE IT FROM YOUR OWN INSTANCE: `retrieval_telemetry` reports
|
||||
# `rule_usage.pull_through`. Raise this if rules arrive unread; lower it if
|
||||
# rules you needed never arrived. What would RETIRE it: a cross-encoder rerank,
|
||||
# which would make a similarity bar the wrong control entirely.
|
||||
#
|
||||
# This bar is the WRITE-PATH arm's. A code payload is long and rich, which is
|
||||
# the case 0.72 suits; the command arm has its own bar below.
|
||||
# `rule_usage.pull_through` and `near_miss_samples`. Raise it if rules arrive
|
||||
# unread; lower it if rules you needed never arrived. What would RETIRE it: a
|
||||
# cross-encoder rerank, which would make a similarity bar the wrong control
|
||||
# entirely.
|
||||
RULEHINT_THRESHOLD_KEY = SURFACES["write_path_rule"].floor_key
|
||||
RULEHINT_DEFAULT_THRESHOLD = SURFACES["write_path_rule"].floor_default
|
||||
|
||||
# THE COMMAND ARM'S OWN BAR, AND WHY IT IS NOT THE WRITE PATH'S.
|
||||
#
|
||||
# A write-path query is a code payload, long and rich, while a pre-tool query
|
||||
# is a shell command — often under a dozen words. Less text, less signal, lower
|
||||
# scores for the same relevance. At the write path's 0.72 the command arm
|
||||
# turns nearly everything away, with its refused mass piled just under the
|
||||
# line, which is the shape a bar set too high leaves.
|
||||
#
|
||||
# WHAT 0.68 IS CALIBRATED AGAINST. Replayed queries, consequential acts
|
||||
# against innocuous ones, scored against a rule corpus with symptom-bearing
|
||||
# triggers:
|
||||
#
|
||||
# 0.7571 git push origin dev consequential
|
||||
# 0.7245 cd ...; git fetch; git add -A consequential
|
||||
# 0.7193 git pull --rebase origin dev consequential
|
||||
# 0.6850 docker compose up -d consequential
|
||||
# ---------------------------------------- 0.68
|
||||
# 0.6735 wc -l src/*.py && date innocuous
|
||||
# 0.6544 grep -rn useState src/ innocuous
|
||||
# 0.6099 sed -n '120,160p' package.json innocuous
|
||||
# 0.6056 ls -la && cat README.md innocuous
|
||||
#
|
||||
# At 0.72 three of the four consequential acts would retrieve nothing, though
|
||||
# the right rules ranked first for each.
|
||||
#
|
||||
# THE SEPARATION IS NARROW (about 0.01), and that is a caveat, not a result.
|
||||
# Eight probes set a direction; they do not settle a number.
|
||||
# `near_miss_samples` on real traffic is what settles it, and this is the bar
|
||||
# to re-read first.
|
||||
#
|
||||
# WHY A LOWER BAR IS SAFE HERE. The band does noise control downstream: a marginal hit that
|
||||
# clears the bar still has to score within `_RULEHINT_BAND` of the top to be
|
||||
# rendered. The bar's job shrank, so the bar can.
|
||||
# THE COMMAND ARM READS THE SAME BAR. A shell command is short, so it scores
|
||||
# lower than a code payload for the same relevance and the shared bar admits
|
||||
# less here. The band does noise control downstream: a hit that clears the bar
|
||||
# still has to score within `_RULEHINT_BAND` of the top to be rendered.
|
||||
#
|
||||
# The innocuous scores near the line are largely CROSS-PROJECT BLEED rather
|
||||
# than bad ranking: another project's shell-command rule matching a shell
|
||||
@@ -439,11 +407,13 @@ _CONCEPT_MIN_CHARS = 16
|
||||
_AUTOINJECT_MAX_TOP_K = MAX_BUDGET
|
||||
|
||||
# THE CONVERSATION BESIDE THE PROMPT. The operator's message alone is a weak
|
||||
# query mid-session: it is mostly a follow-up — "yes do that", "now fix the
|
||||
# filter" — that names nothing a record could match. The hook sends the tail of the last assistant reply as `context`, and it is
|
||||
# appended to a SHORT prompt only: a prompt that already says what it is about
|
||||
# is the better query on its own, and diluting it is the one way this can make
|
||||
# retrieval worse.
|
||||
# query mid-session: it is mostly a follow-up — "now fix the filter" — that
|
||||
# names too little for a record to match. The hook sends the tail of the last
|
||||
# assistant reply as `context`, and it is appended to a SHORT prompt only: a
|
||||
# prompt that already says what it is about is the better query on its own,
|
||||
# and diluting it is the one way this can make retrieval worse. A prompt that
|
||||
# names nothing at all ("yes do that") gets no notes menu (`_has_subject`);
|
||||
# the prompt rule arm still searches it with the tail.
|
||||
#
|
||||
# Neither number costs a model token. The query is embedding input; what the
|
||||
# session pays for is the menu, and the budget bounds that unchanged.
|
||||
@@ -455,6 +425,42 @@ _AUTOINJECT_CONTEXT_PROMPT_MAX = 280
|
||||
_AUTOINJECT_CONTEXT_MAX = 600
|
||||
|
||||
|
||||
# WORDS THAT NAME NO SUBJECT. A prompt made only of these ("keep going", "yes
|
||||
# please do", "1. yes 2. yes", "is it deployed?", "what is next?") is a turn of
|
||||
# the conversation, not a question about anything. Its query would be the
|
||||
# previous reply's tail alone, and the menu would hand back the records that
|
||||
# reply was built from, which the session already has in context. One word
|
||||
# outside this list ("now fix the filter") is a subject, and the prompt is
|
||||
# searched as usual.
|
||||
_NO_SUBJECT_WORDS = frozenset("""
|
||||
a about above after again agreed ahead all already also am an and any
|
||||
anything approved are as asked at awesome back be been before both build
|
||||
but by can carry ci commit continue continuing could cool correct
|
||||
deploy deployed described did do does done each else every everything
|
||||
fine finish first fix for from get go going good got great green had has
|
||||
have he here hold how i if im in instead into is it its just keep know
|
||||
last left let lets like looks loose lgtm make may maybe me mentioned merge
|
||||
milestone more move my next nice no not now of off ok okay on one open or
|
||||
order other our out over pass passes passing perfect please previous
|
||||
proceed project proposed push ready really recommended redeploy rest right
|
||||
said same see she ship should so some sounds stack stacking start step
|
||||
steps still stop suggested sure task tasks thank thanks that the them then
|
||||
there these they thing things this those through to try up us waiting
|
||||
want was we well were what whats when where which while who why will with
|
||||
work would yeah yep yes yet you your
|
||||
""".split())
|
||||
|
||||
|
||||
def _has_subject(prompt: str) -> bool:
|
||||
"""Does the operator's message name anything a record could be about?
|
||||
|
||||
Numbers are left out: a record named by number is looked up by id, not
|
||||
matched, so it never needs the ranked menu to be found.
|
||||
"""
|
||||
words = re.findall(r"[a-z]+", prompt.lower().replace("'", ""))
|
||||
return any(w not in _NO_SUBJECT_WORDS for w in words)
|
||||
|
||||
|
||||
def _autoinject_query(prompt: str, context: str) -> str:
|
||||
"""The prompt, with recent conversation appended when the prompt is thin.
|
||||
|
||||
@@ -478,17 +484,11 @@ def _autoinject_query(prompt: str, context: str) -> str:
|
||||
# The operator's message is the only query that exists before one is composed,
|
||||
# and this arm is what runs against it.
|
||||
PROMPTRULE_THRESHOLD_KEY = SURFACES["prompt_rule"].floor_key
|
||||
# INHERITED FROM THE ACT ARMS. 0.72 is tuned against
|
||||
# code and shell commands. An operator's prose is a different query shape
|
||||
# against the same documents, and nothing yet says the two distributions line
|
||||
# up — triggers are written in the vocabulary of the MOMENT, which for most
|
||||
# rules is act vocabulary, so prose may well score lower across the board.
|
||||
#
|
||||
# Starting at the act arms' number anyway is deliberate: it is the only value
|
||||
# with evidence behind it, and guessing lower would put an unmeasured bar in
|
||||
# front of a corpus that binds. Every call is logged under `prompt_rule`, so
|
||||
# real traffic settles it — read `near_miss_samples` before moving this, not
|
||||
# the percentile alone.
|
||||
# THE SAME BAR AS THE ACT ARMS. An operator's prose is a different query shape
|
||||
# against the same documents, and triggers are written in the vocabulary of
|
||||
# the moment, which for most rules is act vocabulary. Every call is logged
|
||||
# under `prompt_rule`, so read `near_miss_samples` before moving the shared
|
||||
# bar, not the percentile alone.
|
||||
PROMPTRULE_DEFAULT_THRESHOLD = SURFACES["prompt_rule"].floor_default
|
||||
|
||||
# ANCHORED ON THIS HOOK'S BUDGET rather than the act arms'. This arm fires
|
||||
@@ -655,10 +655,6 @@ async def build_autoinject_hint(
|
||||
# left to a ranking that can only match the words around them.
|
||||
named = await _named_records(user_id, record_refs_svc.named_record_ids(q))
|
||||
named_ids = [int(n.id) for n in named]
|
||||
# Everything below searches, logs and fills its slots on the ENRICHED
|
||||
# query, so the telemetry row records what was actually asked.
|
||||
q = _autoinject_query(q, context)
|
||||
|
||||
# THE LEDGER IS A RENDERING FACT, NOT A RETRIEVAL ONE. Excluding seen ids
|
||||
# from the search would make the second time a note is the best answer
|
||||
# SILENCE, indistinguishable from "nothing matched", and would measure
|
||||
@@ -672,15 +668,23 @@ async def build_autoinject_hint(
|
||||
# lesson slots in that order, and the surfacing rows are written there.
|
||||
# What is decided HERE is what makes this moment this moment — the query
|
||||
# the operator's words became, and the records they named by number.
|
||||
result = await rp.run_note_arm(
|
||||
rp.AUTO_INJECT,
|
||||
rp.NoteMoment(
|
||||
user_id=user_id, query=q, project_id=project_id,
|
||||
seen=frozenset(already), named=frozenset(named_ids),
|
||||
),
|
||||
floor=cfg["threshold"], budget=cfg["top_k"], io=_note_io(),
|
||||
)
|
||||
kept = result.menu
|
||||
if not _has_subject(q):
|
||||
# Nothing to search for: the menu would answer the previous reply,
|
||||
# not the operator. Records named by number are still shown.
|
||||
kept, chunks = [], {}
|
||||
else:
|
||||
# Everything below searches, logs and fills its slots on the ENRICHED
|
||||
# query, so the telemetry row records what was actually asked.
|
||||
q = _autoinject_query(q, context)
|
||||
result = await rp.run_note_arm(
|
||||
rp.AUTO_INJECT,
|
||||
rp.NoteMoment(
|
||||
user_id=user_id, query=q, project_id=project_id,
|
||||
seen=frozenset(already), named=frozenset(named_ids),
|
||||
),
|
||||
floor=cfg["threshold"], budget=cfg["top_k"], io=_note_io(),
|
||||
)
|
||||
kept, chunks = result.menu, result.chunks
|
||||
if not kept and not named:
|
||||
return empty
|
||||
|
||||
@@ -782,7 +786,7 @@ async def build_autoinject_hint(
|
||||
# a reason, and a reader cannot tell the two apart once indented alike.
|
||||
passage = "" if nid in already else _menu_passage(
|
||||
document_title(note.title, note.note_type, note.data, note.body),
|
||||
(result.chunks.get(nid) or {}).get("text"), name,
|
||||
(chunks.get(nid) or {}).get("text"), name,
|
||||
)
|
||||
who = _shared_by(note)
|
||||
lines.extend(menu_entry(
|
||||
@@ -1171,9 +1175,8 @@ async def get_writepath_config(user_id: int) -> dict:
|
||||
independent pairs, resolved from the registry.
|
||||
|
||||
Each floor's reason is recorded where its default is: WRITEPATH (code
|
||||
embeddings sit on a much higher similarity floor than prose), RULEHINT (a
|
||||
third corpus again), TOOLRULE (a shell command is a different query shape
|
||||
from a code payload and scores lower for the same relevance).
|
||||
embeddings sit on a much higher similarity floor than prose) and RULEHINT
|
||||
(a third corpus again, whose one bar both rule arms read).
|
||||
|
||||
THE BUDGET IS SPLIT TOO. "How many titles at once" does not mean the same
|
||||
thing on both surfaces: this arm fires before every Write and Edit while
|
||||
@@ -1446,6 +1449,9 @@ async def build_write_path_hint(
|
||||
note.title, note.note_type, note.data, note.body,
|
||||
),
|
||||
"seen": int(note.id) in excluded,
|
||||
# Read by the copy check, which asks for more
|
||||
# resemblance than the menu does.
|
||||
"score": float(score),
|
||||
# Carried, not re-read off the rendered marker. The
|
||||
# marker is prose assembled for a human and it already
|
||||
# varies by kind, language and the `seen` flag — a
|
||||
@@ -1500,16 +1506,35 @@ async def build_write_path_hint(
|
||||
derive = [d for d in found if d.get("key") not in skip]
|
||||
except Exception:
|
||||
logger.warning("write-time derive check failed", exc_info=True)
|
||||
# The COPY check: the derive check matches by name, so a renamed copy
|
||||
# of a recorded snippet passes it and shows only as "similar" prior art.
|
||||
# When this write introduces a definition and an unseen snippet on the
|
||||
# menu resembles it well past the menu's own floor, say that a copy is
|
||||
# the moment to extract. Only a recorded snippet can be seen this way —
|
||||
# the ledger keeps no bodies to compare an unrecorded sibling against.
|
||||
copies: list[dict] = []
|
||||
copy_bar = cfg["threshold"] + WRITEPATH_COPY_MARGIN
|
||||
strong = [
|
||||
it for _m, it in menu
|
||||
if it.get("kind") == "snippet" and not it.get("seen")
|
||||
and it.get("score", 0.0) >= copy_bar
|
||||
]
|
||||
if strong and stamp_shapes and project_id:
|
||||
named = {d["symbol"] for d in derive} | {s["symbol"] for s in suggested}
|
||||
try:
|
||||
fresh = await shape_ledger_svc.new_shapes_at(project_id, path, stamp_shapes)
|
||||
copies = [
|
||||
{"symbol": n, "kind": k, "snippet": strong[0]}
|
||||
for k, n in fresh if n not in named
|
||||
]
|
||||
except Exception:
|
||||
logger.warning("write-time copy check failed", exc_info=True)
|
||||
staleness: list[str] = []
|
||||
|
||||
# The standing-rule arm further down is deliberately left on the far side
|
||||
# of the guard below: it runs a SEMANTIC search, and moving it above would
|
||||
# run an embedding query on every write in the session.
|
||||
# The design arm is decided HERE, above the guard, so a UI write
|
||||
# that matched no prior art still carries it — and it returns on its own
|
||||
# rather than joining the guard's condition, because joining it would let
|
||||
# a design line switch the standing-rule arm below on for writes where it
|
||||
# has never run, moving that arm's call distribution under its floor.
|
||||
# rather than joining the guard's condition: a design line alone is not a
|
||||
# reason to render the prior-art menu's frame.
|
||||
design_text, design_dedup = await _design_arm(
|
||||
user_id, project_id, path, set(exclude_derive or []),
|
||||
)
|
||||
@@ -1520,7 +1545,62 @@ async def build_write_path_hint(
|
||||
user_id, project_id, [path], seen=seen_ruling_systems,
|
||||
source="rulings_write_path",
|
||||
)
|
||||
if not staleness and not synced and not menu and not suggested and not divergence and not derive:
|
||||
# ── Standing rules that may apply here ───────────────────────────────
|
||||
#
|
||||
# A SUGGESTION, not a binding surface, and the distinction is the design:
|
||||
# a rule BINDS by being tagged to an area the project works in,
|
||||
# resolved deterministically at enter_project. This arm reaches for
|
||||
# something weaker and still useful — a conditional rule whose trigger
|
||||
# resembles what is being written, noticed at the moment it is relevant
|
||||
# rather than by being resident in every session.
|
||||
#
|
||||
# EVERY RULE IS ELIGIBLE, including one already named this session: a
|
||||
# rule the session was told about an hour ago is not a rule in front of
|
||||
# the reader now.
|
||||
#
|
||||
# The arm itself is `retrieval_pipeline.run_rule_arm`: the band, the
|
||||
# fresh/repeat split, the unconditional call row and the fresh-only
|
||||
# surfacing rows are written there once for all three
|
||||
# rule arms. What is decided HERE is only what makes this moment this
|
||||
# moment — the query is the code being written (or the path, for an edit
|
||||
# with no body), and a line says the rule may apply "here".
|
||||
#
|
||||
# EVERY WRITE WITH A PATH GETS THE CHECK, whether or not prior art matched:
|
||||
# a rule about the file being written does not depend on a snippet
|
||||
# resembling it. It runs before the notes guard below for that reason,
|
||||
# at the price of one more search per write; its query, `code or path`,
|
||||
# differs from the notes arm's concept query, so it is its own embedding.
|
||||
#
|
||||
# Fails open like every other arm: a rule hint must never break a write.
|
||||
# The direct band, then a rule reached through a linked lesson,
|
||||
# composed by `_rule_moment` exactly as the other two rule builders are.
|
||||
rule_lines: list[str] = []
|
||||
rule_ids: list[int] = []
|
||||
shown_rule_ids: list[int] = []
|
||||
checkpoint: dict = {}
|
||||
try:
|
||||
result = await _rule_moment(
|
||||
rp.WRITE_PATH_RULE,
|
||||
rp.RuleMoment(
|
||||
user_id=user_id, query=code or path, project_id=project_id,
|
||||
where="here", checkpoint_where="here",
|
||||
exclude=frozenset(exclude_rule_ids or []),
|
||||
held=frozenset(held_rule_ids or []),
|
||||
),
|
||||
floor=cfg["rule_threshold"], budget=cfg["rule_top_k"],
|
||||
checkpoint_floor=cfg["checkpoint_threshold"],
|
||||
)
|
||||
rule_lines = result.lines
|
||||
rule_ids.extend(result.rule_ids)
|
||||
# Every rule line the band showed, repeats included — what the
|
||||
# soft-link recorder pairs with this response's lessons.
|
||||
shown_rule_ids = result.shown_rule_ids
|
||||
checkpoint = result.checkpoint
|
||||
except Exception:
|
||||
logger.debug("write-path rule arm failed", exc_info=True)
|
||||
|
||||
if (not staleness and not synced and not menu and not suggested
|
||||
and not divergence and not derive and not rule_lines):
|
||||
if design_text or rulings["lines"]:
|
||||
return {
|
||||
**empty,
|
||||
@@ -1632,6 +1712,8 @@ async def build_write_path_hint(
|
||||
lines.append(_divergence_line(path, divergence))
|
||||
if derive:
|
||||
lines.append(_derive_line(path, derive))
|
||||
if copies:
|
||||
lines.append(_copy_line(copies))
|
||||
|
||||
# Split by arm, which is the whole reason this table exists. The place arm
|
||||
# carries no score and so has no home in retrieval_logs, yet it fires on
|
||||
@@ -1656,52 +1738,7 @@ async def build_write_path_hint(
|
||||
user_id=user_id, note_ids=ids, source=arm, project_id=project_id,
|
||||
)
|
||||
|
||||
# ── Standing rules that may apply here ───────────────────────────────
|
||||
#
|
||||
# A SUGGESTION, not a binding surface, and the distinction is the design:
|
||||
# a rule BINDS by being tagged to an area the project works in,
|
||||
# resolved deterministically at enter_project. This arm reaches for
|
||||
# something weaker and still useful — a conditional rule whose trigger
|
||||
# resembles what is being written, noticed at the moment it is relevant
|
||||
# rather than by being resident in every session.
|
||||
#
|
||||
# EVERY RULE IS ELIGIBLE, including one already named this session: a
|
||||
# rule the session was told about an hour ago is not a rule in front of
|
||||
# the reader now.
|
||||
#
|
||||
# The arm itself is `retrieval_pipeline.run_rule_arm`: the band, the
|
||||
# fresh/repeat split, the unconditional call row and the fresh-only
|
||||
# surfacing rows are written there once for all three
|
||||
# rule arms. What is decided HERE is only what makes this moment this
|
||||
# moment — the query is the code being written (or the path, for an edit
|
||||
# with no body), and a line says the rule may apply "here".
|
||||
#
|
||||
# Fails open like every other arm: a rule hint must never break a write.
|
||||
# The direct band, then a rule reached through a linked lesson,
|
||||
# composed by `_rule_moment` exactly as the other two rule builders are.
|
||||
rule_ids: list[int] = []
|
||||
shown_rule_ids: list[int] = []
|
||||
checkpoint: dict = {}
|
||||
try:
|
||||
result = await _rule_moment(
|
||||
rp.WRITE_PATH_RULE,
|
||||
rp.RuleMoment(
|
||||
user_id=user_id, query=code or path, project_id=project_id,
|
||||
where="here", checkpoint_where="here",
|
||||
exclude=frozenset(exclude_rule_ids or []),
|
||||
held=frozenset(held_rule_ids or []),
|
||||
),
|
||||
floor=cfg["rule_threshold"], budget=cfg["rule_top_k"],
|
||||
checkpoint_floor=cfg["checkpoint_threshold"],
|
||||
)
|
||||
lines.extend(result.lines)
|
||||
rule_ids.extend(result.rule_ids)
|
||||
# Every rule line the band showed, repeats included — what the
|
||||
# soft-link recorder pairs with this response's lessons.
|
||||
shown_rule_ids = result.shown_rule_ids
|
||||
checkpoint = result.checkpoint
|
||||
except Exception:
|
||||
logger.debug("write-path rule arm failed", exc_info=True)
|
||||
lines.extend(rule_lines)
|
||||
|
||||
# A lesson and a rule on this one response: the soft-link
|
||||
# recorder counts the pair, keyed on the FILE — every edit to one file is
|
||||
@@ -2004,6 +2041,22 @@ def _divergence_line(path: str, divergence: list[dict]) -> str:
|
||||
)
|
||||
|
||||
|
||||
def _copy_line(copies: list[dict]) -> str:
|
||||
"""A new definition that closely resembles a recorded snippet under
|
||||
another name: the derive check cannot see a renamed copy, so this says
|
||||
it — as a question for the writer, since resemblance is not identity."""
|
||||
snip = copies[0]["snippet"]
|
||||
names = ", ".join(
|
||||
f"`{'.' if c['kind'] == 'css' else ''}{c['symbol']}`" for c in copies
|
||||
)
|
||||
return (
|
||||
f"> {names} {'is' if len(copies) == 1 else 'are'} new in this file and "
|
||||
f"this code resembles snippet #{snip['id']} `{snip.get('name') or snip['title']}` "
|
||||
f"(similar {snip['score']:.2f}). If it copies that shape, reuse #{snip['id']} "
|
||||
"or extract the shared shape now, before another copy is written."
|
||||
)
|
||||
|
||||
|
||||
def _suggest_line(path: str, suggested: list[dict]) -> str:
|
||||
"""One line naming the recorded shape this code looks like, as evidence
|
||||
for the judgment the agent gives at the end of the turn — not a record of
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
"""The reverse-proxy hop count an admin saves in Settings → Config.
|
||||
|
||||
How many proxies stand in front of the app decides which address every rate
|
||||
limit and audit line sees, and whether a request counts as HTTPS
|
||||
(grimoire/proxy.py). Behind Cloudflare's proxied DNS the right count is 2; at
|
||||
the default of 1 every visitor reads as a Cloudflare edge address and they
|
||||
share one sign-in rate limit. So the count is set where an admin can find it
|
||||
and check it against their own request, not only in the environment.
|
||||
|
||||
Precedence: the saved count, then `TRUSTED_PROXY_HOPS` from the environment,
|
||||
then 1. Saving None removes the saved count, so the environment's applies
|
||||
again.
|
||||
|
||||
It is one instance-wide value, kept on the first admin account like the other
|
||||
instance settings. Every worker holds it in proxy.py, loads it at startup and
|
||||
re-reads it every REFRESH_SECONDS, so a save in one worker reaches the others
|
||||
within that interval; the worker that saved applies it at once.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
from sqlalchemy import delete, select
|
||||
|
||||
from grimoire import proxy
|
||||
from grimoire.config import Config
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.setting import Setting
|
||||
from grimoire.models.user import User
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
SETTING_KEY = "trusted_proxy_hops"
|
||||
REFRESH_SECONDS = 30
|
||||
# The load at startup runs inside the lifespan hook, and a hook that does not
|
||||
# return is a worker that never serves (db_maintenance_scheduler explains the
|
||||
# outage that taught this). One indexed row answers in milliseconds, so the
|
||||
# deadline only fires when the database is already in trouble.
|
||||
STARTUP_READ_TIMEOUT = 3.0
|
||||
|
||||
|
||||
def parse_hops(raw) -> int | None:
|
||||
"""A hop count from 0 to proxy.MAX_HOPS, or None for anything else. A
|
||||
bool is refused, though Python counts it as an int."""
|
||||
if isinstance(raw, bool):
|
||||
return None
|
||||
if isinstance(raw, str):
|
||||
raw = raw.strip()
|
||||
if not raw.isdigit():
|
||||
return None
|
||||
raw = int(raw)
|
||||
if not isinstance(raw, int) or not 0 <= raw <= proxy.MAX_HOPS:
|
||||
return None
|
||||
return raw
|
||||
|
||||
|
||||
def _admin_rows():
|
||||
return (
|
||||
select(Setting)
|
||||
.join(User, Setting.user_id == User.id)
|
||||
.where(User.role == "admin", Setting.key == SETTING_KEY)
|
||||
)
|
||||
|
||||
|
||||
async def saved_hops() -> int | None:
|
||||
"""The count saved in Settings, or None when there is none (or what is
|
||||
stored can't be read as one)."""
|
||||
async with async_session() as session:
|
||||
row = (await session.execute(_admin_rows().order_by(Setting.user_id))).scalars().first()
|
||||
return parse_hops(row.value) if row else None
|
||||
|
||||
|
||||
async def save_hops(hops: int | None) -> None:
|
||||
"""Save the count, or remove it with None. Any copy on another admin's
|
||||
account goes too, so exactly one value is ever read back."""
|
||||
async with async_session() as session:
|
||||
admin_ids = select(User.id).where(User.role == "admin")
|
||||
await session.execute(
|
||||
delete(Setting).where(Setting.key == SETTING_KEY, Setting.user_id.in_(admin_ids))
|
||||
)
|
||||
if hops is not None:
|
||||
first_admin = (
|
||||
await session.execute(select(User.id).where(User.role == "admin").order_by(User.id))
|
||||
).scalars().first()
|
||||
if first_admin is None:
|
||||
raise ValueError("there is no admin account to keep the setting on")
|
||||
session.add(Setting(user_id=first_admin, key=SETTING_KEY, value=str(hops)))
|
||||
await session.commit()
|
||||
proxy.set_saved_hops(hops)
|
||||
|
||||
|
||||
async def refresh() -> None:
|
||||
"""Load the saved count into this worker. A failed read keeps the value
|
||||
it already had: an unreachable database must not switch the trust rule."""
|
||||
try:
|
||||
proxy.set_saved_hops(await saved_hops())
|
||||
except Exception:
|
||||
logger.debug("could not re-read the saved proxy hop count", exc_info=True)
|
||||
|
||||
|
||||
async def load_at_startup() -> None:
|
||||
"""`refresh`, bounded for the lifespan hook. On a timeout the environment's
|
||||
count stays in force until the next refresh reaches the database."""
|
||||
try:
|
||||
await asyncio.wait_for(refresh(), timeout=STARTUP_READ_TIMEOUT)
|
||||
except (TimeoutError, asyncio.TimeoutError):
|
||||
logger.warning(
|
||||
"reading the saved proxy hop count exceeded %.1fs; starting with %d "
|
||||
"from the environment until the database answers",
|
||||
STARTUP_READ_TIMEOUT, Config.TRUSTED_PROXY_HOPS,
|
||||
)
|
||||
|
||||
|
||||
_refresh_task = None
|
||||
|
||||
|
||||
def start_refresh_loop() -> None:
|
||||
global _refresh_task
|
||||
if _refresh_task is None or _refresh_task.done():
|
||||
from grimoire.services.background import start_periodic
|
||||
_refresh_task = start_periodic(REFRESH_SECONDS, refresh, label="proxy_hops_refresh")
|
||||
|
||||
|
||||
def source() -> str:
|
||||
"""Where the count in force came from: "setting", "env" or "default"."""
|
||||
if proxy.saved_hops_in_force() is not None:
|
||||
return "setting"
|
||||
return "env" if Config.TRUSTED_PROXY_HOPS_FROM_ENV else "default"
|
||||
@@ -460,16 +460,31 @@ class RuleArm:
|
||||
declared: Declared | None = None
|
||||
|
||||
|
||||
# The arms differ on purpose where these flags differ. Whether the prompt arm
|
||||
# should band, and whether the act arms should reserve a preference, are open
|
||||
# questions to settle on telemetry, not by symmetry.
|
||||
# ONE BAR FOR EVERY RULE ARM THAT ANNOTATES. The prompt, command and write
|
||||
# arms search the same corpus, and what limits what they show is the budget
|
||||
# and the band, not the floor. The floor only answers "is this worth ranking
|
||||
# at all", and that question has one answer per corpus, not one per query
|
||||
# shape — so the three share one settings key, and moving it moves all three.
|
||||
# `set_dial` records the move on every surface that shares the key.
|
||||
#
|
||||
# Set low on purpose: a hit under it is never shown, so a bar set too high
|
||||
# costs a rule that binds, while one set a little low costs a line the band
|
||||
# usually drops. The reply backstop keeps its own number, because there the
|
||||
# floor is a stop bar and not a hint bar.
|
||||
RULE_FLOOR_KEY = "kb_rule_threshold"
|
||||
RULE_FLOOR_DEFAULT = 0.70
|
||||
|
||||
# The arms differ on purpose where these flags differ: the prompt arm takes no
|
||||
# band, so its budget alone decides how many hits it shows, and the act arms
|
||||
# band and may hold the act. Every arm that annotates reserves a line for a
|
||||
# preference.
|
||||
PROMPT_RULE = RuleArm(
|
||||
"prompt_rule", band=False, compact_tail=False, checkpoint=False,
|
||||
preference_slot=True,
|
||||
tuning=Surface(
|
||||
name="prompt_rule",
|
||||
floor_key="kb_promptrule_threshold",
|
||||
floor_default=0.72,
|
||||
floor_key=RULE_FLOOR_KEY,
|
||||
floor_default=RULE_FLOOR_DEFAULT,
|
||||
budget_key="kb_promptrule_top_k",
|
||||
budget_default=3,
|
||||
asks="the operator's message, against rule triggers",
|
||||
@@ -480,11 +495,11 @@ PROMPT_RULE = RuleArm(
|
||||
)
|
||||
PRE_TOOL_RULE = RuleArm(
|
||||
"pre_tool_rule", band=True, compact_tail=True, checkpoint=True,
|
||||
preference_slot=False,
|
||||
preference_slot=True,
|
||||
tuning=Surface(
|
||||
name="pre_tool_rule",
|
||||
floor_key="kb_toolrule_threshold",
|
||||
floor_default=0.68,
|
||||
floor_key=RULE_FLOOR_KEY,
|
||||
floor_default=RULE_FLOOR_DEFAULT,
|
||||
budget_key="kb_toolrule_top_k",
|
||||
budget_default=5,
|
||||
asks="the command about to run, against rule triggers",
|
||||
@@ -495,11 +510,11 @@ PRE_TOOL_RULE = RuleArm(
|
||||
)
|
||||
WRITE_PATH_RULE = RuleArm(
|
||||
"write_path_rule", band=True, compact_tail=True, checkpoint=True,
|
||||
preference_slot=False,
|
||||
preference_slot=True,
|
||||
tuning=Surface(
|
||||
name="write_path_rule",
|
||||
floor_key="kb_rulehint_threshold",
|
||||
floor_default=0.72,
|
||||
floor_key=RULE_FLOOR_KEY,
|
||||
floor_default=RULE_FLOOR_DEFAULT,
|
||||
budget_key="kb_rulehint_top_k",
|
||||
budget_default=5,
|
||||
asks="the code being written, against rule triggers",
|
||||
@@ -689,6 +704,13 @@ async def _reserve_slot_for_preference(
|
||||
same shape one corpus over, where snippets lose to project records that
|
||||
merely resemble the query.
|
||||
|
||||
The act arms reserve the slot for the same reason one step later. A
|
||||
preference about how to act ("let each action land before the next") is
|
||||
ranked against every rule whose trigger names the same command, and the
|
||||
act is its last chance: once the command has run there is nothing left
|
||||
for it to shape. The slot's query reuses the arm's query, so it costs one
|
||||
kind-filtered search and no extra embedding.
|
||||
|
||||
THE SLOT BUYS POSITION, NOT A LOWER BAR — same as `reuse_slot`, which also
|
||||
reserves at `cfg["threshold"]`. A weak preference cannot buy the slot, so
|
||||
silence stays the default and the reserved line is never worse than the
|
||||
|
||||
@@ -46,13 +46,18 @@ from grimoire.models.base import iso
|
||||
from grimoire.models.note_usage import PULLED, NoteUsageEvent
|
||||
from grimoire.models.retrieval_judgment import VERDICTS, RetrievalJudgment
|
||||
from grimoire.models.retrieval_log import RetrievalLog
|
||||
from grimoire.models.rule_usage import PULLED as RULE_PULLED
|
||||
from grimoire.models.rule_usage import RuleUsageEvent
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# The arms a review can re-run FAITHFULLY. A re-run with different parameters
|
||||
# from the arm's would judge a menu nobody was shown, so a surface joins this
|
||||
# only with its own search spelled out in `_rerun`.
|
||||
REVIEWABLE = ("auto_inject",)
|
||||
# only with its own search spelled out in `_rerun`. The rule arms named here
|
||||
# are the ones that annotate: each logs its query, floor, budget and project,
|
||||
# which is everything its search takes.
|
||||
RULE_REVIEWABLE = ("prompt_rule", "pre_tool_rule", "write_path_rule")
|
||||
REVIEWABLE = ("auto_inject", *RULE_REVIEWABLE)
|
||||
|
||||
DEPTH_DEFAULT = 6
|
||||
DEPTH_MAX = 10
|
||||
@@ -94,7 +99,58 @@ def _after(stamp, call) -> bool:
|
||||
async def _rerun(
|
||||
user_id: int, row: RetrievalLog, depth: int, report: dict | None = None,
|
||||
) -> list[dict]:
|
||||
"""The logged call's query, searched again the way its arm searches.
|
||||
"""The logged call's query, searched again the way its arm searches."""
|
||||
if row.source in RULE_REVIEWABLE:
|
||||
return await _rerun_rules(user_id, row, depth, report)
|
||||
return await _rerun_notes(user_id, row, depth, report)
|
||||
|
||||
|
||||
async def _rerun_rules(
|
||||
user_id: int, row: RetrievalLog, depth: int, report: dict | None = None,
|
||||
) -> list[dict]:
|
||||
"""A rule arm's call, searched again: its query against rule triggers,
|
||||
with the floor and project it logged.
|
||||
|
||||
`within_budget` is the arm's own cut: a line inside the budget, and, for
|
||||
an arm that bands, inside the band of the top hit. The preference slot is
|
||||
left out — it logs its own source. Rules created after the call are
|
||||
dropped and named in `report["postdated"]`, as for notes.
|
||||
"""
|
||||
from grimoire.services import retrieval_pipeline as rp
|
||||
from grimoire.services.embeddings import semantic_search_rules
|
||||
|
||||
arm = next(a for a in rp.RULE_ARMS if a.source == row.source)
|
||||
hits = await semantic_search_rules(
|
||||
user_id, row.query or "",
|
||||
limit=depth + POSTDATED_SLACK,
|
||||
threshold=row.threshold if row.threshold is not None else 0.0,
|
||||
project_id=row.project_id or None,
|
||||
)
|
||||
postdated = [int(r.id) for _s, r in hits if _after(r.created_at, row.created_at)]
|
||||
if report is not None:
|
||||
report["postdated"] = postdated
|
||||
hits = [h for h in hits if int(h[1].id) not in postdated][:depth]
|
||||
budget = int(row.limit_n or 0)
|
||||
cut = hits[:budget]
|
||||
kept = {int(r.id) for _s, r in (rp._rule_band(cut) if arm.band else cut)}
|
||||
return [{
|
||||
"rank": rank,
|
||||
"record_id": int(rule.id),
|
||||
"score": round(float(score), 4),
|
||||
"within_budget": int(rule.id) in kept,
|
||||
"changed_since_call": _after(rule.updated_at, row.created_at),
|
||||
"kind": rule.kind or "rule",
|
||||
"name": rule.title,
|
||||
# The trigger is what the arm matched against and what its line
|
||||
# quotes, so it is the passage the reader saw.
|
||||
"passage": rule.when_to_apply or rule.statement or None,
|
||||
} for rank, (score, rule) in enumerate(hits, start=1)]
|
||||
|
||||
|
||||
async def _rerun_notes(
|
||||
user_id: int, row: RetrievalLog, depth: int, report: dict | None = None,
|
||||
) -> list[dict]:
|
||||
"""A notes arm's call, searched again.
|
||||
|
||||
`auto_inject`'s parameters, read from its spec (`retrieval_pipeline.
|
||||
AUTO_INJECT`) rather than restated: which kinds, whose records, and the
|
||||
@@ -151,19 +207,25 @@ async def _opened_after(user_id: int, row: RetrievalLog, ids: list[int]) -> set[
|
||||
|
||||
Agent pulls only — the `mcp_` prefix, per NoteUsageEvent's convention —
|
||||
because the question is whether the line moved the session, not whether
|
||||
the operator later browsed to the record.
|
||||
the operator later browsed to the record. A rule arm's lines are read
|
||||
from the rule table: a note and a rule can share an id.
|
||||
"""
|
||||
if not ids or row.created_at is None:
|
||||
return set()
|
||||
table, id_col, pulled = (
|
||||
(RuleUsageEvent, RuleUsageEvent.rule_id, RULE_PULLED)
|
||||
if row.source in RULE_REVIEWABLE
|
||||
else (NoteUsageEvent, NoteUsageEvent.note_id, PULLED)
|
||||
)
|
||||
async with async_session() as session:
|
||||
found = await session.execute(
|
||||
select(NoteUsageEvent.note_id).distinct().where(
|
||||
NoteUsageEvent.user_id == user_id,
|
||||
NoteUsageEvent.event == PULLED,
|
||||
NoteUsageEvent.source.startswith("mcp_", autoescape=True),
|
||||
NoteUsageEvent.note_id.in_(ids),
|
||||
NoteUsageEvent.created_at >= row.created_at,
|
||||
NoteUsageEvent.created_at <= row.created_at + OPENED_WINDOW,
|
||||
select(id_col).distinct().where(
|
||||
table.user_id == user_id,
|
||||
table.event == pulled,
|
||||
table.source.startswith("mcp_", autoescape=True),
|
||||
id_col.in_(ids),
|
||||
table.created_at >= row.created_at,
|
||||
table.created_at <= row.created_at + OPENED_WINDOW,
|
||||
)
|
||||
)
|
||||
return {int(i) for (i,) in found.all()}
|
||||
@@ -175,7 +237,7 @@ async def _logged_after(row: RetrievalLog, ids: list[int]) -> set[int]:
|
||||
(a closed task takes no claim stamp), so it is read here."""
|
||||
from grimoire.models.task_log import TaskLog
|
||||
|
||||
if not ids or row.created_at is None:
|
||||
if not ids or row.created_at is None or row.source in RULE_REVIEWABLE:
|
||||
return set()
|
||||
async with async_session() as session:
|
||||
found = await session.execute(
|
||||
|
||||
@@ -110,6 +110,9 @@ def dial_for_key(key: str) -> tuple[str, str] | None:
|
||||
|
||||
Derived rather than listed so a seventh surface is covered the moment it is
|
||||
added here, which is the only way this stays true.
|
||||
|
||||
A floor key several surfaces share resolves to the first of them;
|
||||
`set_dial` records the move on every surface in `sharing_floor`.
|
||||
"""
|
||||
for surface in SURFACES.values():
|
||||
if key == surface.floor_key:
|
||||
@@ -119,6 +122,16 @@ def dial_for_key(key: str) -> tuple[str, str] | None:
|
||||
return None
|
||||
|
||||
|
||||
def sharing_floor(name: str) -> list[str]:
|
||||
"""Every surface that reads the same floor key as `name`, itself included.
|
||||
|
||||
The rule arms share one bar, so moving one surface's floor moves its
|
||||
siblings, and each of them needs the event that says so.
|
||||
"""
|
||||
key = get_surface(name).floor_key
|
||||
return [n for n, s in SURFACES.items() if s.floor_key == key]
|
||||
|
||||
|
||||
async def floor_for(user_id: int, name: str) -> float:
|
||||
"""This install's current floor for a surface, clamped to [0, 1]."""
|
||||
s = get_surface(name)
|
||||
|
||||
@@ -41,6 +41,7 @@ from grimoire.models.system_usage import SURFACED as SYSTEM_SURFACED
|
||||
from grimoire.models.system_usage import SystemUsageEvent
|
||||
from grimoire.services.rule_usage import is_ambient
|
||||
from grimoire.models.retrieval_log import RetrievalLog
|
||||
from grimoire.models.rulebook import Rule
|
||||
from grimoire.services.retrieval_review import judged_block
|
||||
from grimoire.services.retrieval_registry import (
|
||||
POINTS, UNBIDDEN, get_point, is_registered, sources_expected_to_emit,
|
||||
@@ -489,9 +490,10 @@ def _num(raw: str, fallback):
|
||||
# to look next. ONE SENTENCE PER CORPUS because the two lines are built
|
||||
# differently: a note line carries its matched passage, so an unopened
|
||||
# note may have done its job, and the judged sample is the instrument that can
|
||||
# tell. A rule line carries only its trigger and asks to be opened, and no
|
||||
# judged sample covers the rule arms (`retrieval_review.REVIEWABLE`), so
|
||||
# sending that reader to `menus_to_review` would send them to a refusal.
|
||||
# tell. A rule line carries only its trigger and asks to be opened, so a rule
|
||||
# set aside again and again is usually a trigger to fix; the judged sample
|
||||
# (`retrieval_review.REVIEWABLE` names the rule arms it covers) says whether
|
||||
# it was set aside rightly.
|
||||
_NEVER_PULLED_READING = {
|
||||
"notes": (
|
||||
"That is not a verdict on them: a line carries its matched passage, "
|
||||
@@ -504,11 +506,13 @@ _NEVER_PULLED_READING = {
|
||||
"Not a verdict either, and read differently from notes: the count "
|
||||
"includes rules that only arrived in a listing (the project "
|
||||
"handshake, a planning read), and a rule line shows its trigger, so a "
|
||||
"session can rightly set one aside on the trigger alone. There is no "
|
||||
"judged sample for the rule arms. A rule set aside again and again "
|
||||
"where it does not apply is a trigger to fix, not a floor to move — "
|
||||
"`update_rule(when_to_apply=...)`, then `what_might_apply` with the "
|
||||
"moment's own words to see where it ranks."
|
||||
"session can rightly set one aside on the trigger alone. To tell a "
|
||||
"rule set aside rightly from one missed, judge a sample of a rule "
|
||||
"arm with `menus_to_review(source=\"prompt_rule\")` (or "
|
||||
"`pre_tool_rule`, `write_path_rule`). A rule set aside again and "
|
||||
"again where it does not apply is a trigger to fix, not a floor to "
|
||||
"move — `update_rule(when_to_apply=...)`, then `what_might_apply` "
|
||||
"with the moment's own words to see where it ranks."
|
||||
),
|
||||
}
|
||||
|
||||
@@ -803,6 +807,9 @@ def _silent_surfaces(sources: dict, usage: dict, rule_usage: dict,
|
||||
return []
|
||||
seen = set(sources) | set((usage.get("by_source") or {}))
|
||||
seen |= set((rule_usage.get("by_source") or {}))
|
||||
# A getter shows nothing, so it is seen only by its pulls.
|
||||
seen |= set((usage.get("pulled_by_source") or {}))
|
||||
seen |= set((rule_usage.get("pulled_by_source") or {}))
|
||||
seen |= set(((system_usage or {}).get("by_source") or {}))
|
||||
return [
|
||||
{"source": s, "kind": POINTS[s].kind, "what": POINTS[s].what}
|
||||
@@ -933,7 +940,9 @@ async def moment_usage(user_id: int | None, since) -> dict:
|
||||
- `rules`: how many distinct rules did;
|
||||
- `opened`: how many of those an AGENT then opened (`mcp_*`), at or after
|
||||
their first delivery there in the window;
|
||||
- `last_delivered_at`.
|
||||
- `last_delivered_at`;
|
||||
- `by_rule`: each rule delivered there, most delivered first, with its
|
||||
`title`, `delivered` and whether it was `opened`.
|
||||
|
||||
`opened` is an UPPER BOUND, for by_source's reason: a pull records the
|
||||
door, not the line that prompted it, so a rule delivered at two moments
|
||||
@@ -980,15 +989,21 @@ async def moment_usage(user_id: int | None, since) -> dict:
|
||||
pull.source.startswith("mcp_", autoescape=True),
|
||||
pull.created_at >= per_rule.c.first_at,
|
||||
)
|
||||
# One row per (moment, rule); the moment's totals are summed from
|
||||
# these below, so `by_rule` and the totals cannot disagree.
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(
|
||||
per_rule.c.moment,
|
||||
func.sum(per_rule.c.n).label("delivered"),
|
||||
func.count().label("rules"),
|
||||
func.count(case((opened, 1))).label("opened"),
|
||||
func.max(per_rule.c.last_at).label("last_at"),
|
||||
).group_by(per_rule.c.moment)
|
||||
per_rule.c.rule_id,
|
||||
Rule.title,
|
||||
per_rule.c.n,
|
||||
opened.label("opened"),
|
||||
per_rule.c.last_at,
|
||||
)
|
||||
.select_from(per_rule)
|
||||
.outerjoin(Rule, Rule.id == per_rule.c.rule_id)
|
||||
.order_by(per_rule.c.moment, per_rule.c.n.desc(), per_rule.c.rule_id)
|
||||
)
|
||||
).all()
|
||||
complete = await _complete_from(session, RuleUsageEvent, user_id)
|
||||
@@ -997,15 +1012,26 @@ async def moment_usage(user_id: int | None, since) -> dict:
|
||||
block["moment_usage_failed"] = True
|
||||
return block
|
||||
|
||||
block["by_moment"] = {
|
||||
moment: {
|
||||
"delivered": int(delivered or 0),
|
||||
"rules": int(rules or 0),
|
||||
"opened": int(n_opened or 0),
|
||||
"last_delivered_at": iso(last_at),
|
||||
}
|
||||
for moment, delivered, rules, n_opened, last_at in rows
|
||||
}
|
||||
by_moment: dict[str, dict] = {}
|
||||
for moment, rule_id, title, n, was_opened, last_at in rows:
|
||||
slot = by_moment.setdefault(moment, {
|
||||
"delivered": 0, "rules": 0, "opened": 0,
|
||||
"last_delivered_at": None, "by_rule": [],
|
||||
})
|
||||
slot["delivered"] += int(n or 0)
|
||||
slot["rules"] += 1
|
||||
slot["opened"] += 1 if was_opened else 0
|
||||
stamp = iso(last_at)
|
||||
if stamp and (slot["last_delivered_at"] or "") < stamp:
|
||||
slot["last_delivered_at"] = stamp
|
||||
# Which rule a shared moment delivered, and whether that rule was
|
||||
# opened: the per-moment counts cannot say once two rules are
|
||||
# mounted there. `title` is null for a rule deleted since.
|
||||
slot["by_rule"].append({
|
||||
"rule_id": int(rule_id), "title": title,
|
||||
"delivered": int(n or 0), "opened": bool(was_opened),
|
||||
})
|
||||
block["by_moment"] = by_moment
|
||||
block.update(_coverage(complete.get("*"), since))
|
||||
return block
|
||||
|
||||
@@ -1109,7 +1135,7 @@ async def retrieval_summary(
|
||||
|
||||
# Assigned inside the try below; named here so the readout can tell
|
||||
# "this query failed" from "this window has no rows".
|
||||
by_source_rows = None
|
||||
by_source_rows = rule_by_source_rows = None
|
||||
rule_rows = None
|
||||
distinct_rules_surfaced = distinct_rules_pulled = 0
|
||||
# None means the coverage read did not happen — distinct from a table with
|
||||
@@ -1426,6 +1452,53 @@ async def retrieval_summary(
|
||||
rule_rows = None
|
||||
distinct_rules_surfaced = distinct_rules_pulled = 0
|
||||
distinct_rules_acted = 0
|
||||
|
||||
# The note block's per-source join, at the rule grain. The
|
||||
# top-level rule ratio blends every arm — the prompt arm, the tool
|
||||
# arm, a lesson pointing at its rule — and "which of them earns its
|
||||
# lines" is the question each arm's bar is tuned against. Its own
|
||||
# guard, for the reason the note join has one.
|
||||
try:
|
||||
rule_pulled_ids = (
|
||||
select(RuleUsageEvent.rule_id)
|
||||
.where(
|
||||
RuleUsageEvent.created_at >= since,
|
||||
RuleUsageEvent.user_id == user_id,
|
||||
RuleUsageEvent.event == RULE_PULLED,
|
||||
RuleUsageEvent.source.startswith("mcp_", autoescape=True),
|
||||
)
|
||||
.distinct()
|
||||
.subquery()
|
||||
)
|
||||
rule_surfaced_pairs = (
|
||||
select(RuleUsageEvent.source, RuleUsageEvent.rule_id)
|
||||
.where(
|
||||
RuleUsageEvent.created_at >= since,
|
||||
RuleUsageEvent.user_id == user_id,
|
||||
RuleUsageEvent.event == RULE_SURFACED,
|
||||
)
|
||||
.distinct()
|
||||
.subquery()
|
||||
)
|
||||
rule_by_source_rows = (
|
||||
await session.execute(
|
||||
select(
|
||||
rule_surfaced_pairs.c.source,
|
||||
func.count().label("rules_surfaced"),
|
||||
func.count(rule_pulled_ids.c.rule_id).label("rules_pulled"),
|
||||
)
|
||||
.select_from(
|
||||
rule_surfaced_pairs.outerjoin(
|
||||
rule_pulled_ids,
|
||||
rule_pulled_ids.c.rule_id == rule_surfaced_pairs.c.rule_id,
|
||||
)
|
||||
)
|
||||
.group_by(rule_surfaced_pairs.c.source)
|
||||
)
|
||||
).all()
|
||||
except Exception:
|
||||
logger.warning("per-source rule pull-through read failed", exc_info=True)
|
||||
rule_by_source_rows = None
|
||||
except Exception:
|
||||
logger.warning("retrieval summary read failed", exc_info=True)
|
||||
out["read_failed"] = True
|
||||
@@ -1439,6 +1512,9 @@ async def retrieval_summary(
|
||||
"distinct_notes_surfaced": int(distinct_surfaced or 0),
|
||||
"distinct_notes_pulled": int(distinct_pulled or 0),
|
||||
}
|
||||
# Pulls per door. `by_source` is keyed by the surface that showed a record
|
||||
# and leaves pulls out, so this is what says each getter is recording.
|
||||
pulled_by_source: dict[str, int] = {}
|
||||
for event, source, n in urows:
|
||||
n = int(n)
|
||||
if event == SURFACED:
|
||||
@@ -1448,6 +1524,7 @@ async def retrieval_summary(
|
||||
usage["surfaced"] += n
|
||||
elif event == PULLED:
|
||||
usage["pulled"] += n
|
||||
pulled_by_source[source] = pulled_by_source.get(source, 0) + n
|
||||
# The mcp_/rest_ split is load-bearing (see NoteUsageEvent's own
|
||||
# comment). "Is
|
||||
# this record dead weight?" is answered by ANY pull; "was that
|
||||
@@ -1466,6 +1543,7 @@ async def retrieval_summary(
|
||||
round(usage["pulled_by_agent"] / usage["surfaced"], 4)
|
||||
if usage["surfaced"] else None
|
||||
)
|
||||
usage["pulled_by_source"] = pulled_by_source
|
||||
|
||||
# The same question, per surface — which is the one the top-level ratio
|
||||
# cannot answer. A corpus average of 0.05 is compatible with one surface
|
||||
@@ -1538,6 +1616,8 @@ async def retrieval_summary(
|
||||
"applied": 0,
|
||||
"departed": 0,
|
||||
"distinct_rules_acted": int(distinct_rules_acted or 0),
|
||||
# Pulls per door, as on the note block.
|
||||
"pulled_by_source": {},
|
||||
}
|
||||
if rule_rows is None:
|
||||
# The FLAG is added, the shape is kept — matching `by_source_failed`
|
||||
@@ -1560,6 +1640,8 @@ async def retrieval_summary(
|
||||
rule_usage["surfaced"] += n
|
||||
elif event == RULE_PULLED:
|
||||
rule_usage["pulled"] += n
|
||||
pulled = rule_usage["pulled_by_source"]
|
||||
pulled[source] = pulled.get(source, 0) + n
|
||||
# Same split, and it carries MORE weight here than for notes.
|
||||
# The arm's whole claim is "this rule may apply to what you are
|
||||
# writing", and only an agent opening it says the claim landed.
|
||||
@@ -1596,6 +1678,28 @@ async def retrieval_summary(
|
||||
round(rule_usage["pulled_by_agent"] / rule_usage["surfaced"], 4)
|
||||
if rule_usage["surfaced"] else None
|
||||
)
|
||||
# Per surface, the note block's shape and its caveat: a pull records the
|
||||
# door, not the surface that led to it, so a rule two arms surfaced and an
|
||||
# agent opened once counts as pulled for both. An upper bound — a surface
|
||||
# reading near zero here is not being flattered.
|
||||
if rule_by_source_rows is None:
|
||||
rule_usage["by_source"] = {}
|
||||
rule_usage["by_source_failed"] = True
|
||||
else:
|
||||
rule_usage["by_source"] = {
|
||||
source: {
|
||||
"rules_surfaced": int(n_surfaced or 0),
|
||||
"rules_pulled": int(n_pulled or 0),
|
||||
# None on an ambient surface, as for notes: nothing chose
|
||||
# those rules, so the ratio would be misread.
|
||||
"pull_through": (
|
||||
None if is_ambient(source) or not n_surfaced
|
||||
else round(int(n_pulled or 0) / int(n_surfaced), 4)
|
||||
),
|
||||
"ambient": is_ambient(source),
|
||||
}
|
||||
for source, n_surfaced, n_pulled in rule_by_source_rows
|
||||
}
|
||||
rule_usage.update(_coverage((rule_complete or {}).get("*"), since))
|
||||
out["rule_usage"] = rule_usage
|
||||
out["system_usage"] = await _system_usage(user_id, since)
|
||||
|
||||
@@ -44,6 +44,7 @@ from grimoire.services.retrieval_surfaces import (
|
||||
budget_for,
|
||||
floor_for,
|
||||
get_surface,
|
||||
sharing_floor,
|
||||
surface_names,
|
||||
)
|
||||
from grimoire.services.settings import set_setting
|
||||
@@ -257,6 +258,13 @@ async def set_dial(
|
||||
"reason": text, "actor": actor, "unchanged": True,
|
||||
}
|
||||
|
||||
# A shared floor moves every surface that reads it, and each one's history
|
||||
# has to show the move, or its telemetry changes with no event to explain it.
|
||||
# The surface the caller named is recorded first, its siblings after it.
|
||||
moved = [surface] + (
|
||||
[n for n in sharing_floor(surface) if n != surface] if dial == "floor" else []
|
||||
)
|
||||
|
||||
await set_setting(user_id, key, stored)
|
||||
async with async_session() as session:
|
||||
# Stamped with the space this number was chosen in. Read at
|
||||
@@ -264,12 +272,13 @@ async def set_dial(
|
||||
# the caller recording it are the same call, so there is no window in
|
||||
# which they could disagree.
|
||||
stamp = calibration_stamp()
|
||||
session.add(RetrievalTuningEvent(
|
||||
user_id=user_id, surface=surface, dial=dial,
|
||||
old_value=old, new_value=applied, actor=actor, reason=text,
|
||||
embedding_model=stamp["embedding_model"],
|
||||
shape_version=stamp["shape_version"],
|
||||
))
|
||||
for name in moved:
|
||||
session.add(RetrievalTuningEvent(
|
||||
user_id=user_id, surface=name, dial=dial,
|
||||
old_value=old, new_value=applied, actor=actor, reason=text,
|
||||
embedding_model=stamp["embedding_model"],
|
||||
shape_version=stamp["shape_version"],
|
||||
))
|
||||
await session.commit()
|
||||
|
||||
return {
|
||||
@@ -286,6 +295,9 @@ async def set_dial(
|
||||
# Always present, both ways round: a caller that has to test for the
|
||||
# key's absence to learn the answer will eventually forget to.
|
||||
"unchanged": False,
|
||||
# The other surfaces this move carried with it: a shared floor is one
|
||||
# number, and a caller tuning one rule arm has tuned them all.
|
||||
"moved_with": [n for n in moved if n != surface],
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -43,7 +43,7 @@ from __future__ import annotations
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import exists, func, not_, select
|
||||
from sqlalchemy import exists, func, not_, or_, select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
from grimoire.models import async_session
|
||||
@@ -274,6 +274,11 @@ async def pending(user_id: int, rule_id: int | None = None, limit: int = 200) ->
|
||||
|
||||
Each rule carries what it is mounted on now, so a proposal is read beside
|
||||
the mounts it would add to.
|
||||
|
||||
A signal pair is a proposal only once it has crossed the bar and asked
|
||||
(`proposed_at` in its evidence). Below the bar it is a row gathering
|
||||
evidence, and listing it would bury the few pairs the signal stands
|
||||
behind under every rule that was once opened after a moment.
|
||||
"""
|
||||
from grimoire.services.rulebooks import _owned_rules_clause
|
||||
|
||||
@@ -282,7 +287,13 @@ async def pending(user_id: int, rule_id: int | None = None, limit: int = 200) ->
|
||||
stmt = (
|
||||
select(RuleMomentJudgment, Rule)
|
||||
.join(Rule, Rule.id == RuleMomentJudgment.rule_id)
|
||||
.where(RuleMomentJudgment.state == SUGGESTED, _owned_rules_clause(user_id))
|
||||
.where(
|
||||
RuleMomentJudgment.state == SUGGESTED, _owned_rules_clause(user_id),
|
||||
or_(
|
||||
RuleMomentJudgment.source != "signal",
|
||||
RuleMomentJudgment.evidence["proposed_at"].astext.isnot(None),
|
||||
),
|
||||
)
|
||||
)
|
||||
if rule_id:
|
||||
stmt = stmt.where(Rule.id == int(rule_id))
|
||||
@@ -483,17 +494,50 @@ def _keep(row: RuleMomentJudgment, note: str, now: datetime) -> None:
|
||||
# ── The signal ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
# How many distinct actions a signal pair remembers as having reached its
|
||||
# moment. Enough to show the operator what the opens followed.
|
||||
_SIGNAL_ACTIONS_CAP = 5
|
||||
|
||||
|
||||
def _reached_phrase(evidence: dict) -> str:
|
||||
reached = (evidence or {}).get("reached_by") or {}
|
||||
if not isinstance(reached, dict) or not reached:
|
||||
return ""
|
||||
top = sorted(reached.items(), key=lambda kv: (-int(kv[1] or 0), kv[0]))[:3]
|
||||
return " (reached by " + ", ".join(f"`{match}`" for match, _n in top) + ")"
|
||||
|
||||
|
||||
def _signal_note(moment: str, evidence: dict) -> str:
|
||||
"""The `why` a signal proposal shows the operator: what the opens followed."""
|
||||
counts = lesson_rules.evidence_summary(evidence)
|
||||
return (
|
||||
f"Opened just after `{moment}`{_reached_phrase(evidence)} in "
|
||||
f"{counts['situations']} distinct sessions, each time without a hook having "
|
||||
f"shown the rule first. Co-occurrence only: confirm it if the rule governs "
|
||||
f"that point in the work whatever the work is about."
|
||||
)
|
||||
|
||||
|
||||
def _proposal_line(rule: Rule, moment: str, evidence: dict) -> str:
|
||||
"""The line an open past the bar returns to the session.
|
||||
|
||||
It says what the evidence is, and makes offering it conditional on the
|
||||
reader's own reading: a session relays an unqualified "offer this" as a
|
||||
recommendation, and co-occurrence is not one.
|
||||
"""
|
||||
counts = lesson_rules.evidence_summary(evidence)
|
||||
kind = rule.kind or "rule"
|
||||
return (
|
||||
f"> {kind.capitalize()} #{rule.id} \"{rule.title}\" has been opened just after "
|
||||
f"`{moment}` in {counts['situations']} distinct sessions — it may belong on "
|
||||
f"that moment, where it would arrive without being searched for. Offer the "
|
||||
f"operator the mount in one line; on their yes, `judge_rule_moments("
|
||||
f"[{{\"rule_id\": {rule.id}, \"moment\": \"{moment}\", \"verdict\": \"confirm\", "
|
||||
f"\"note\": \"why\"}}])` mounts it; on a no, `\"reject\"` with their why stops "
|
||||
f"the asking."
|
||||
f"> {kind.capitalize()} #{rule.id} \"{rule.title}\" was opened just after "
|
||||
f"`{moment}`{_reached_phrase(evidence)} in {counts['situations']} distinct "
|
||||
f"sessions, each time without a hook having shown it first. That is "
|
||||
f"co-occurrence, not a recommendation. Read the {kind}'s trigger against what "
|
||||
f"`{moment}` means (`list_moments`): only if the {kind} governs that point in "
|
||||
f"the work whatever the work is about, ask the operator whether to mount it, "
|
||||
f"with your reason; if it is about a subject, leave it. On their yes, "
|
||||
f"`judge_rule_moments([{{\"rule_id\": {rule.id}, \"moment\": \"{moment}\", "
|
||||
f"\"verdict\": \"confirm\", \"note\": \"why\"}}])` mounts it; on a no, "
|
||||
f"`\"reject\"` with their why stops the asking."
|
||||
)
|
||||
|
||||
|
||||
@@ -510,24 +554,39 @@ def signal_moments(moments) -> list[str]:
|
||||
|
||||
async def co_occurred(
|
||||
user_id: int, rule_id: int, moments, *, situation: str,
|
||||
project_id: int | None = None,
|
||||
project_id: int | None = None, reached: dict | None = None,
|
||||
) -> str:
|
||||
"""Record that a rule was opened shortly after these moments fired, and
|
||||
return a proposal line once a (rule, moment) pair has earned one.
|
||||
|
||||
`situation` is what makes two opens distinct — the session, so a rule
|
||||
opened after every push in one long session counts once. A pair already
|
||||
mounted or judged gathers nothing. One proposal per open at most. Fails
|
||||
open: this rides a hook, and bookkeeping must never break the read.
|
||||
judged gathers nothing, and neither does a rule already mounted anywhere:
|
||||
it arrives by lookup at its own moments, so an open of it is far likelier
|
||||
to follow that delivery than to reveal a missing moment. `reached` maps a
|
||||
moment to the action that reached it, kept as the proposal's evidence.
|
||||
One proposal per open at most. Fails open: this rides a hook, and
|
||||
bookkeeping must never break the read.
|
||||
"""
|
||||
try:
|
||||
return await _co_occurred(user_id, rule_id, moments, situation, project_id)
|
||||
return await _co_occurred(user_id, rule_id, moments, situation, project_id,
|
||||
reached or {})
|
||||
except Exception:
|
||||
logger.warning("rule/moment co-occurrence could not be recorded", exc_info=True)
|
||||
return ""
|
||||
|
||||
|
||||
async def _co_occurred(user_id, rule_id, moments, situation, project_id) -> str:
|
||||
def _add_reached(evidence: dict, match: str) -> dict:
|
||||
"""`evidence` with one more open counted against the action that reached
|
||||
the moment, keeping at most _SIGNAL_ACTIONS_CAP distinct actions."""
|
||||
reached = dict(evidence.get("reached_by") or {})
|
||||
match = (match or "").strip()[:80]
|
||||
if match and (match in reached or len(reached) < _SIGNAL_ACTIONS_CAP):
|
||||
reached[match] = int(reached.get(match) or 0) + 1
|
||||
return {**evidence, "reached_by": reached}
|
||||
|
||||
|
||||
async def _co_occurred(user_id, rule_id, moments, situation, project_id, reached) -> str:
|
||||
wanted = signal_moments(moments)
|
||||
key = lesson_rules.situation_key("s", situation)
|
||||
rid = _int(rule_id)
|
||||
@@ -540,11 +599,10 @@ async def _co_occurred(user_id, rule_id, moments, situation, project_id) -> str:
|
||||
rule = owned.get(rid)
|
||||
if rule is None:
|
||||
return ""
|
||||
mounted = (await _mounts(session, [rid])).get(rid, set())
|
||||
if (await _mounts(session, [rid])).get(rid):
|
||||
return ""
|
||||
existing = await _rows(session, [rid])
|
||||
for moment in wanted:
|
||||
if moment in mounted:
|
||||
continue
|
||||
row = existing.get((rid, moment))
|
||||
if row is not None and row.state != SUGGESTED:
|
||||
continue
|
||||
@@ -552,11 +610,13 @@ async def _co_occurred(user_id, rule_id, moments, situation, project_id) -> str:
|
||||
row = RuleMomentJudgment(rule_id=rid, moment=moment, state=SUGGESTED, source="signal")
|
||||
session.add(row)
|
||||
ev = lesson_rules.add_evidence(row.evidence, key, project_id, now)
|
||||
ev = _add_reached(ev, reached.get(moment, ""))
|
||||
if proposal is None and lesson_rules.proposal_due(ev, now):
|
||||
ev["proposed_at"] = now.isoformat()
|
||||
ev["proposed_count"] = int(ev.get("proposed_count") or 0) + 1
|
||||
proposal = (moment, ev)
|
||||
row.evidence = ev
|
||||
row.note = _signal_note(moment, ev)
|
||||
try:
|
||||
await session.commit()
|
||||
except IntegrityError:
|
||||
@@ -588,6 +648,7 @@ async def opened_after(
|
||||
from grimoire.services.moment_actions import moments_for
|
||||
|
||||
reached: list[str] = []
|
||||
reached_by: dict[str, str] = {}
|
||||
for act in list(acts or [])[:_ACTS_CAP]:
|
||||
if not isinstance(act, dict):
|
||||
continue
|
||||
@@ -604,11 +665,13 @@ async def opened_after(
|
||||
name = hit.get("moment") if isinstance(hit, dict) else None
|
||||
if name and name not in reached:
|
||||
reached.append(name)
|
||||
reached_by[name] = str(hit.get("match") or tool)
|
||||
moments = signal_moments(reached)
|
||||
context = ""
|
||||
if moments and (session_id or "").strip():
|
||||
context = await co_occurred(
|
||||
user_id, rule_id, moments, situation=session_id.strip(), project_id=project_id,
|
||||
reached=reached_by,
|
||||
)
|
||||
return {"moments": moments, "context": context}
|
||||
|
||||
|
||||
@@ -59,8 +59,9 @@ as short and as slow-moving as possible.
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import case, func, select
|
||||
from sqlalchemy import case, func, select, true
|
||||
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.base import iso
|
||||
@@ -333,13 +334,18 @@ def empty_rule_usage() -> dict:
|
||||
}
|
||||
|
||||
|
||||
async def usage_for_rules(rule_ids: list[int]) -> dict[int, dict]:
|
||||
async def usage_for_rules(
|
||||
rule_ids: list[int], since: datetime | None = None,
|
||||
) -> dict[int, dict]:
|
||||
"""Aggregate usage for a set of rules: {rule_id: {counts + timestamps}}.
|
||||
|
||||
One GROUP BY for the whole page rather than a query per row — this feeds a
|
||||
list view, so the per-row shape would be N+1 by construction. Rules with no
|
||||
events come back with `empty_rule_usage()`, so the caller never has to tell
|
||||
"no events" from "not in the result".
|
||||
|
||||
`since` limits the counts to events at or after it, so two windows can be
|
||||
compared; without it they cover everything recorded.
|
||||
"""
|
||||
ids = [int(r) for r in rule_ids]
|
||||
out: dict[int, dict] = {rid: empty_rule_usage() for rid in ids}
|
||||
@@ -370,6 +376,10 @@ async def usage_for_rules(rule_ids: list[int]) -> dict[int, dict]:
|
||||
ambient,
|
||||
)
|
||||
.where(RuleUsageEvent.rule_id.in_(ids))
|
||||
.where(
|
||||
RuleUsageEvent.created_at >= since
|
||||
if since is not None else true()
|
||||
)
|
||||
.group_by(
|
||||
RuleUsageEvent.rule_id,
|
||||
RuleUsageEvent.event,
|
||||
|
||||
@@ -23,9 +23,10 @@ from __future__ import annotations
|
||||
|
||||
import difflib
|
||||
import logging
|
||||
import posixpath
|
||||
import re
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Iterable, NamedTuple
|
||||
from typing import Iterable, Mapping, NamedTuple
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
@@ -257,6 +258,46 @@ def _event(row: CodeShape, event: str, at: datetime, *, commit: str = "") -> Cod
|
||||
)
|
||||
|
||||
|
||||
async def provisional_row(
|
||||
session, *, project_id: int, repo_key: str, path: str, symbol: str, kind: str,
|
||||
) -> CodeShape:
|
||||
"""The row for a shape a caller met before any live row for it existed:
|
||||
a shape being written this turn, or one judged ahead of the sync.
|
||||
|
||||
The identity (project, repo, path, symbol, kind) is unique, so "no live
|
||||
row" does not mean "no row". A VANISHED row under the identity is revived
|
||||
and keeps its standing judgment, as a sync revives a shape that comes
|
||||
back. The next sync then confirms it, or stamps it vanished again until
|
||||
the code reaches the bound ref. With no row at all, a provisional one is
|
||||
inserted inside a savepoint. If a concurrent request inserts the same
|
||||
identity first, the savepoint rolls back and that request's row is used.
|
||||
The caller's transaction survives either way.
|
||||
"""
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
def _same():
|
||||
return select(CodeShape).where(
|
||||
CodeShape.project_id == project_id, CodeShape.repo_key == repo_key,
|
||||
CodeShape.path == path, CodeShape.symbol == symbol, CodeShape.kind == kind,
|
||||
)
|
||||
|
||||
row = (await session.execute(_same())).scalars().first()
|
||||
if row is None:
|
||||
try:
|
||||
async with session.begin_nested():
|
||||
row = CodeShape(project_id=project_id, repo_key=repo_key,
|
||||
path=path, symbol=symbol, kind=kind)
|
||||
session.add(row)
|
||||
await session.flush()
|
||||
return row
|
||||
except IntegrityError:
|
||||
row = (await session.execute(_same())).scalars().one()
|
||||
if row.vanished_at is not None:
|
||||
row.vanished_at = None
|
||||
session.add(_event(row, "reappeared", datetime.now(timezone.utc)))
|
||||
return row
|
||||
|
||||
|
||||
async def _judge(
|
||||
session, row: CodeShape, *, status: str, snippet_id: int | None,
|
||||
by: str | None, reason: str | None, at: datetime, reason_code: str | None = None,
|
||||
@@ -320,6 +361,31 @@ async def record_uses(
|
||||
return n
|
||||
|
||||
|
||||
async def replace_reference_uses(
|
||||
session, row: CodeShape, snippet_ids, *, evidence: str | None = None,
|
||||
) -> int:
|
||||
"""Make the row's mechanical `reference` edges exactly ``snippet_ids``.
|
||||
|
||||
`record_uses` only adds, so a by-name rule that tightens would otherwise
|
||||
leave every edge the looser rule wrote. Only `reference` edges are
|
||||
dropped: a judgment, or the write-path hook's evidence, outlives a
|
||||
re-read of the body. Returns the number of edges written or refreshed."""
|
||||
wanted = {int(x) for x in (snippet_ids or []) if x}
|
||||
if row.id is not None:
|
||||
stale = (
|
||||
await session.execute(
|
||||
select(CodeShapeUse).where(
|
||||
CodeShapeUse.shape_id == row.id,
|
||||
CodeShapeUse.basis == "reference",
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
for edge in stale:
|
||||
if edge.snippet_id not in wanted:
|
||||
await session.delete(edge)
|
||||
return await record_uses(session, row, wanted, basis="reference", evidence=evidence)
|
||||
|
||||
|
||||
async def uses_of(shape_ids) -> dict[int, list[CodeShapeUse]]:
|
||||
"""{shape_id: [edges]} for a set of rows — the read side of record_uses."""
|
||||
ids = [int(x) for x in shape_ids if x]
|
||||
@@ -700,11 +766,10 @@ async def classify_shapes(
|
||||
if not matches and repo_key and item.get("status") != "unclassified":
|
||||
path = (item.get("path") or "").strip()
|
||||
symbol = (item.get("symbol") or "").strip()
|
||||
row = CodeShape(
|
||||
project_id=project_id, repo_key=repo_key, path=path,
|
||||
row = await provisional_row(
|
||||
session, project_id=project_id, repo_key=repo_key, path=path,
|
||||
symbol=symbol, kind=kind or snippet_kind(symbol, ""),
|
||||
)
|
||||
session.add(row)
|
||||
by_key.setdefault((path, symbol), []).append(row)
|
||||
matches = [row]
|
||||
provisional += 1
|
||||
@@ -1553,8 +1618,19 @@ async def suggest_write_path_instances(
|
||||
continue
|
||||
fields = snippet_fields(note)
|
||||
symbol = fields.get("symbol") or ""
|
||||
kind = snippet_kind(symbol, fields.get("language") or "")
|
||||
if references_symbol(code, symbol, kind):
|
||||
language = (fields.get("language") or "").strip().lower()
|
||||
kind = snippet_kind(symbol, language)
|
||||
canon = Canon(
|
||||
int(sid), kind, symbol,
|
||||
tuple(((loc.get("path") or ""), (loc.get("symbol") or ""))
|
||||
for loc in fields.get("locations") or []),
|
||||
"", "", int(note.project_id or 0), language,
|
||||
)
|
||||
# A redefinition of the pulled name in this payload is the copy the
|
||||
# hint exists to catch, so the payload's own names are not "local".
|
||||
if (kind != "sym" or same_family(path, language)) and names_canon(
|
||||
kind, path, code, canon, project_id=project_id,
|
||||
):
|
||||
rank, why = 2, _REFERENCE_REASON.format(
|
||||
sid=sid, symbol=_norm_symbol(symbol)
|
||||
)
|
||||
@@ -1618,13 +1694,13 @@ async def suggest_write_path_instances(
|
||||
if row is None:
|
||||
if not repo_key:
|
||||
continue
|
||||
row = CodeShape(
|
||||
project_id=project_id, repo_key=repo_key,
|
||||
# It may be a vanished row coming back, with its judgment.
|
||||
row = await provisional_row(
|
||||
session, project_id=project_id, repo_key=repo_key,
|
||||
path=path, symbol=name, kind=kind,
|
||||
)
|
||||
session.add(row)
|
||||
by_key[(name, kind)] = row
|
||||
elif not (row.status in _MECHANICAL_TODO or row.classified_by == "hook"):
|
||||
if not (row.status in _MECHANICAL_TODO or row.classified_by == "hook"):
|
||||
continue # a judgment — or the canon itself — stands
|
||||
# A proposal, never a status: the writer decides.
|
||||
row.proposed_snippet_id = sid
|
||||
@@ -1762,7 +1838,7 @@ def _semantic_priority(row) -> tuple:
|
||||
# were examined under, so a tightened rule re-examines everything once. A
|
||||
# change that does not alter what is proposed needs no bump. A stale "miss"
|
||||
# basis on an old row is inert: a miss is not evidence (see _SEMANTIC_LIMIT).
|
||||
_PROPOSER_VERSION = 6
|
||||
_PROPOSER_VERSION = 9
|
||||
# Signature resemblance floor, name blanked (difflib ratio) — and a length
|
||||
# floor, because `def NAME():` resembles `def NAME(x):` at 0.95 while saying
|
||||
# nothing; a family shape has parameters to resemble.
|
||||
@@ -1805,6 +1881,13 @@ _FAMILY_BY_LANG = {
|
||||
"css": "css", "scss": "css", "sass": "css", "less": "css",
|
||||
"bash": "sh", "sh": "sh", "shell": "sh", "zsh": "sh",
|
||||
"sql": "sql",
|
||||
"go": "go", "golang": "go",
|
||||
"rust": "rs", "rs": "rs",
|
||||
"java": "jvm", "kotlin": "jvm", "kt": "jvm",
|
||||
"swift": "swift", "dart": "dart",
|
||||
"c": "c", "cpp": "c", "c++": "c", "h": "c",
|
||||
"csharp": "cs", "c#": "cs", "cs": "cs",
|
||||
"ruby": "rb", "rb": "rb", "php": "php",
|
||||
}
|
||||
_FAMILY_BY_EXT = {
|
||||
".py": "py", ".pyi": "py",
|
||||
@@ -1812,6 +1895,11 @@ _FAMILY_BY_EXT = {
|
||||
".css": "css", ".scss": "css", ".sass": "css", ".less": "css",
|
||||
".sh": "sh", ".bash": "sh", ".zsh": "sh",
|
||||
".sql": "sql",
|
||||
".go": "go", ".rs": "rs",
|
||||
".java": "jvm", ".kt": "jvm", ".kts": "jvm",
|
||||
".swift": "swift", ".dart": "dart",
|
||||
".c": "c", ".h": "c", ".cc": "c", ".cpp": "c", ".hpp": "c",
|
||||
".cs": "cs", ".rb": "rb", ".php": "php",
|
||||
}
|
||||
|
||||
|
||||
@@ -1877,10 +1965,79 @@ def text_contains(body: str, code: str) -> bool:
|
||||
return a in b or b in a
|
||||
|
||||
|
||||
def reference_canons(kind: str, path: str, symbol: str, body: str, canons: Iterable[Canon]) -> list[int]:
|
||||
def _in_canon_home(canon: Canon, path: str) -> bool:
|
||||
"""Is ``path`` in the canon's own package? A location that names a file
|
||||
makes its directory home, exactly (a Go package is one directory); a
|
||||
location that names a directory is home with everything under it."""
|
||||
from grimoire.services.snippets import _path_touches
|
||||
|
||||
here = posixpath.dirname((path or "").strip("/"))
|
||||
for lp, _ls in canon.locations:
|
||||
lp = (lp or "").strip("/")
|
||||
if not lp:
|
||||
continue
|
||||
if "." in lp.rsplit("/", 1)[-1]:
|
||||
if posixpath.dirname(lp) == here:
|
||||
return True
|
||||
elif _path_touches(lp, path):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def names_canon(
|
||||
kind: str, path: str, body: str, canon: Canon, *,
|
||||
project_id: int = 0, defined_at: Mapping[str, frozenset[str]] | None = None,
|
||||
) -> bool:
|
||||
"""Does this body name THIS canon, rather than merely contain its word?
|
||||
|
||||
The by-name test behind the reference basis and every uses edge the
|
||||
proposer writes. ``defined_at`` maps each name to the repo paths that
|
||||
define it. A mention counts only when it can resolve to the canon:
|
||||
- never across projects: one repo does not call a function defined in
|
||||
another's, however alike the names are;
|
||||
- never a name the file defines itself (a module's own `_run` helper),
|
||||
unless the canon is recorded in this very file;
|
||||
- when the repo also defines the name somewhere the canon is not
|
||||
recorded (`sftpRun.file` beside the canon `importRun.file`), only from
|
||||
the canon's own package: elsewhere the name says nothing about which
|
||||
definition a call reaches;
|
||||
- never a generic verb, with or without a leading underscore;
|
||||
- a bare lowercase word (`file`, `seen`, `seal`) only where it is called
|
||||
as `word(`, because as a plain word it is everyone's variable. No space
|
||||
is allowed before the parenthesis: formatted code never has one, and
|
||||
prose does ("a file (not a directory)").
|
||||
Same kind and language family are checked by the callers' loops."""
|
||||
sym = _norm_symbol(canon.symbol or "")
|
||||
if not sym:
|
||||
return False
|
||||
if project_id and canon.project_id and canon.project_id != project_id:
|
||||
return False
|
||||
if sym.lstrip("_").lower() in _REFERENCE_STOPLIST:
|
||||
return False
|
||||
where = (defined_at or {}).get(sym, frozenset())
|
||||
if where:
|
||||
from grimoire.services.snippets import _path_touches
|
||||
|
||||
def at_canon(p: str) -> bool:
|
||||
return any(lp and _path_touches(lp, p) for lp, _ls in canon.locations)
|
||||
|
||||
if path in where and not at_canon(path):
|
||||
return False
|
||||
if canon.locations and not all(at_canon(p) for p in where) \
|
||||
and not _in_canon_home(canon, path):
|
||||
return False
|
||||
if kind == "sym" and re.fullmatch(r"[a-z]+", sym):
|
||||
return re.search(rf"(?<![\w$]){re.escape(sym)}\(", body or "") is not None
|
||||
return references_symbol(body, sym, kind)
|
||||
|
||||
|
||||
def reference_canons(
|
||||
kind: str, path: str, symbol: str, body: str, canons: Iterable[Canon], *,
|
||||
project_id: int = 0, defined_at: Mapping[str, frozenset[str]] | None = None,
|
||||
) -> list[int]:
|
||||
"""Every canon this body NAMES — the uses edges the proposer can
|
||||
write mechanically: same kind, same language family, symbol not in the
|
||||
generic-verb stoplist, and not the shape's own name."""
|
||||
write mechanically: same kind, same language family, not the shape's own
|
||||
name, and named in the sense `names_canon` gives."""
|
||||
norm_sym = _norm_symbol(symbol)
|
||||
out: list[int] = []
|
||||
for c in canons:
|
||||
@@ -1890,9 +2047,7 @@ def reference_canons(kind: str, path: str, symbol: str, body: str, canons: Itera
|
||||
continue
|
||||
if _norm_symbol(c.symbol) == norm_sym:
|
||||
continue
|
||||
if _norm_symbol(c.symbol).lower() in _REFERENCE_STOPLIST:
|
||||
continue
|
||||
if references_symbol(body, c.symbol, kind):
|
||||
if names_canon(kind, path, body, c, project_id=project_id, defined_at=defined_at):
|
||||
out.append(c.snippet_id)
|
||||
return out
|
||||
|
||||
@@ -1900,6 +2055,7 @@ def reference_canons(kind: str, path: str, symbol: str, body: str, canons: Itera
|
||||
def match_canon(
|
||||
kind: str, path: str, symbol: str, signature: str, body: str,
|
||||
canons: Iterable[Canon], *, project_id: int = 0,
|
||||
defined_at: Mapping[str, frozenset[str]] | None = None,
|
||||
) -> tuple[int, str, float] | None:
|
||||
"""The strongest (snippet_id, basis, score) a shape earns against the
|
||||
canon catalog, by the non-semantic bases — or None. Strongest by
|
||||
@@ -1924,11 +2080,7 @@ def match_canon(
|
||||
if not any(location_covers(lp, ls, path, symbol) for lp, ls in c.locations):
|
||||
offer("symbol", 1.0, c)
|
||||
continue # its own location is canonical territory, not a proposal
|
||||
if (
|
||||
c.symbol
|
||||
and _norm_symbol(c.symbol).lower() not in _REFERENCE_STOPLIST
|
||||
and references_symbol(body, c.symbol, kind)
|
||||
):
|
||||
if names_canon(kind, path, body, c, project_id=project_id, defined_at=defined_at):
|
||||
offer("reference", 0.9, c)
|
||||
if c.code_norm and text_contains(body, c.code_norm):
|
||||
offer("text", 0.95, c)
|
||||
@@ -2064,6 +2216,14 @@ async def propose_for_repo(
|
||||
if canons is None:
|
||||
canons = await canon_catalog(user_id)
|
||||
by_key = {(d[0], d[1], d[2]): d for d in definitions}
|
||||
# Where the repo defines each name: a mention resolves to the definition
|
||||
# in its own file, and a name defined in several packages only from the
|
||||
# canon's own (see names_canon).
|
||||
paths_by_name: dict[str, set[str]] = {}
|
||||
for d in definitions:
|
||||
if d[1] == "sym":
|
||||
paths_by_name.setdefault(_norm_symbol(d[2]), set()).add(d[0])
|
||||
defined_at = {n: frozenset(p) for n, p in paths_by_name.items()}
|
||||
# The semantic arm is the widest net and, across projects, is noise: it
|
||||
# is held to the shape's own project and language family. The precise
|
||||
# bases (symbol/text) still reach family canon in other projects.
|
||||
@@ -2107,10 +2267,10 @@ async def propose_for_repo(
|
||||
# are a fact about the body, judged or not: the consumer map
|
||||
# of a canon must include the call sites someone already
|
||||
# classified. Mark it examined so the scan runs once per body.
|
||||
used = reference_canons(row.kind, row.path, row.symbol, body, canons)
|
||||
if used:
|
||||
await record_uses(session, row, used, basis="reference",
|
||||
evidence="proposer: body names the canon's symbol")
|
||||
used = reference_canons(row.kind, row.path, row.symbol, body, canons,
|
||||
project_id=project_id, defined_at=defined_at)
|
||||
await replace_reference_uses(session, row, used,
|
||||
evidence="proposer: body names the canon's symbol")
|
||||
row.proposed_at = now
|
||||
row.proposed_sha = examined_as
|
||||
continue
|
||||
@@ -2118,7 +2278,7 @@ async def propose_for_repo(
|
||||
group = row.proposal_group # derive grouping is reassigned below
|
||||
hit = match_canon(
|
||||
row.kind, row.path, row.symbol, signature, body, canons,
|
||||
project_id=project_id,
|
||||
project_id=project_id, defined_at=defined_at,
|
||||
)
|
||||
_clear_proposal(row)
|
||||
row.proposal_group = group
|
||||
@@ -2126,10 +2286,10 @@ async def propose_for_repo(
|
||||
row.proposed_sha = examined_as
|
||||
# Consumption is recorded for every canon the body names,
|
||||
# whatever the row is then judged to be.
|
||||
used = reference_canons(row.kind, row.path, row.symbol, body, canons)
|
||||
if used:
|
||||
await record_uses(session, row, used, basis="reference",
|
||||
evidence="proposer: body names the canon's symbol")
|
||||
used = reference_canons(row.kind, row.path, row.symbol, body, canons,
|
||||
project_id=project_id, defined_at=defined_at)
|
||||
await replace_reference_uses(session, row, used,
|
||||
evidence="proposer: body names the canon's symbol")
|
||||
if hit:
|
||||
row.proposed_snippet_id, row.proposal_basis, row.proposal_score = hit
|
||||
row.proposal_group = None
|
||||
@@ -2706,6 +2866,29 @@ async def write_time_derive(
|
||||
return out
|
||||
|
||||
|
||||
async def new_shapes_at(
|
||||
project_id: int, path: str, shapes: list[tuple[str, str]]
|
||||
) -> list[tuple[str, str]]:
|
||||
"""The (kind, name) shapes the hook named at ``path`` that the ledger has
|
||||
no live row for there — definitions this write introduces, as opposed to
|
||||
ones it edits inside. Order follows ``shapes``."""
|
||||
wanted = [(k, n) for k, n in shapes if n]
|
||||
if not wanted:
|
||||
return []
|
||||
async with async_session() as session:
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(CodeShape.kind, CodeShape.symbol).where(
|
||||
CodeShape.project_id == project_id,
|
||||
CodeShape.path == path,
|
||||
CodeShape.vanished_at.is_(None),
|
||||
)
|
||||
)
|
||||
).all()
|
||||
here = {(k, _norm_symbol(s)) for k, s in rows}
|
||||
return [(k, n) for k, n in wanted if (k, _norm_symbol(n)) not in here]
|
||||
|
||||
|
||||
async def flag_divergence(project_id: int, *, since: datetime | None) -> int:
|
||||
"""Flag shapes created after ``since`` (the previous refresh) that sit
|
||||
where a canon dominates and were not proposed as that canon. With no
|
||||
|
||||
@@ -0,0 +1,338 @@
|
||||
"""Which project a write lands in, so a session can be held to its own.
|
||||
|
||||
A Claude Code session belongs to one project: the one its starting directory
|
||||
is bound to. Toward any other project it may read anything and file an issue,
|
||||
and every other write waits for the operator. The plugin's PreToolUse hook
|
||||
enforces that, because only the hook knows which session is acting: the MCP
|
||||
endpoint is stateless and sees nothing but the caller's key. This module
|
||||
answers the hook's one question: given the session's project and a pending
|
||||
act, which projects would the act change, and may it go ahead?
|
||||
|
||||
An act is either a directory (a file edit or a git command; the hook resolves
|
||||
it to a repo or a `.grimoire` marker) or an MCP tool call. For a tool call,
|
||||
`WRITE_TARGETS` says, for every write tool, which of its arguments locate the
|
||||
write. That means the record or project the write changes, never one it only
|
||||
mentions: `arose_from_id` on a new task points at another task without
|
||||
changing it. tests/test_write_scope.py holds the table to the registered tools,
|
||||
so a new write tool can't be added without saying where it writes.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from sqlalchemy import or_, select
|
||||
|
||||
# How an argument locates the project a write lands in.
|
||||
PROJECT = "project" # a project id
|
||||
NOTE = "note" # a task, note, lesson, snippet or process: its project
|
||||
MILESTONE = "milestone" # a milestone: its project
|
||||
SYSTEM = "system" # a System: its project
|
||||
RULE = "rule" # a project rule: its project. A global rule: none
|
||||
RELATION = "relation" # a rule relation: the projects of its two rules
|
||||
TRASH = "trash" # a deleted batch: every project its rows belong to
|
||||
REPO = "repo" # a repo remote: the project it is bound to
|
||||
DECISION = "decision" # a family decision: its project, else its idea's home
|
||||
|
||||
LOCATORS = frozenset({PROJECT, NOTE, MILESTONE, SYSTEM, RULE, RELATION, TRASH, REPO, DECISION})
|
||||
|
||||
# Every write tool, and the arguments that locate its write. An empty entry is
|
||||
# a write that lands in no project:
|
||||
# - install-level settings (moment mappings, retrieval dials);
|
||||
# - user-level records (processes, design systems, rulebooks, topics, global
|
||||
# rules and preferences);
|
||||
# - judgments the system asks a session for in any project (a rule's outcome,
|
||||
# a lesson-to-rule link, moment proposals, a menu verdict). Those describe how
|
||||
# the corpus is retrieved; they don't change another project's work.
|
||||
WRITE_TARGETS: dict[str, dict[str, str]] = {
|
||||
# notes, tasks, planning
|
||||
"create_note": {"project_id": PROJECT},
|
||||
"update_note": {"note_id": NOTE, "project_id": PROJECT},
|
||||
"delete_note": {"note_id": NOTE},
|
||||
"mark_note_verified": {"note_id": NOTE},
|
||||
"create_lesson": {"project_id": PROJECT},
|
||||
"update_lesson": {"lesson_id": NOTE},
|
||||
"delete_lesson": {"lesson_id": NOTE},
|
||||
"judge_lesson_link": {},
|
||||
"create_task": {"project_id": PROJECT, "milestone_id": MILESTONE, "parent_id": NOTE},
|
||||
"update_task": {"task_id": NOTE, "project_id": PROJECT, "milestone_id": MILESTONE},
|
||||
"delete_task": {"task_id": NOTE},
|
||||
"add_task_log": {"task_id": NOTE},
|
||||
"create_records": {"project_id": PROJECT, "milestone_id": MILESTONE},
|
||||
"start_planning": {"project_id": PROJECT},
|
||||
"create_milestone": {"project_id": PROJECT},
|
||||
"update_milestone": {"milestone_id": MILESTONE, "project_id": PROJECT},
|
||||
"delete_milestone": {"milestone_id": MILESTONE},
|
||||
# projects, Systems, repos
|
||||
"create_project": {},
|
||||
"update_project": {"project_id": PROJECT},
|
||||
"delete_project": {"project_id": PROJECT},
|
||||
"decide_project_inception": {"project_id": PROJECT},
|
||||
"set_project_platforms": {"project_id": PROJECT},
|
||||
"create_system": {"project_id": PROJECT},
|
||||
"update_system": {"system_id": SYSTEM},
|
||||
"delete_system": {"system_id": SYSTEM},
|
||||
"map_system_to_canonical": {"system_id": SYSTEM},
|
||||
# Binding a repo changes the project it's bound to and, when it was bound
|
||||
# elsewhere, the project it leaves.
|
||||
"bind_repo": {"project_id": PROJECT, "repo_url": REPO},
|
||||
"unbind_repo": {"repo_url": REPO},
|
||||
# family canon. An idea is a record in its home project; an adoption is a
|
||||
# project's answer; a conflict is between the two projects it names.
|
||||
"propose_family_idea": {"note_id": NOTE},
|
||||
"promote_family_idea": {"note_id": NOTE},
|
||||
"retire_family_idea": {"note_id": NOTE},
|
||||
"revise_family_idea": {"note_id": NOTE},
|
||||
"set_family_references": {"note_id": NOTE},
|
||||
"set_family_topic": {"note_id": NOTE},
|
||||
"undo_family_decision": {"decision_id": DECISION},
|
||||
"assess_family_adoption": {"project_id": PROJECT},
|
||||
"resolve_family_conflict": {"canon_project_id": PROJECT, "other_project_id": PROJECT},
|
||||
# snippets, processes, the shape ledger
|
||||
"create_snippet": {"project_id": PROJECT},
|
||||
"update_snippet": {"snippet_id": NOTE, "project_id": PROJECT},
|
||||
"delete_snippet": {"snippet_id": NOTE},
|
||||
"verify_snippet": {"snippet_id": NOTE},
|
||||
"merge_snippets": {"target_id": NOTE, "source_ids": NOTE},
|
||||
"unmerge_snippet": {"survivor_id": NOTE, "source_id": NOTE},
|
||||
"create_process": {},
|
||||
"update_process": {"process_id": NOTE},
|
||||
"delete_process": {"process_id": NOTE},
|
||||
"classify_shapes": {"project_id": PROJECT},
|
||||
"classify_shapes_by_rule": {"project_id": PROJECT},
|
||||
"confirm_shape_proposals": {"project_id": PROJECT},
|
||||
"refresh_pattern_coverage": {"project_id": PROJECT},
|
||||
"move_path": {"project_id": PROJECT},
|
||||
# design systems belong to their owner, not to a project; choosing one for
|
||||
# a project is that project's write.
|
||||
"create_design_system": {},
|
||||
"update_design_system": {},
|
||||
"delete_design_system": {},
|
||||
"create_design_token": {},
|
||||
"update_design_token": {},
|
||||
"delete_design_token": {},
|
||||
"set_project_design_system": {"project_id": PROJECT},
|
||||
# rules
|
||||
"create_rulebook": {},
|
||||
"update_rulebook": {},
|
||||
"delete_rulebook": {},
|
||||
"create_topic": {},
|
||||
"update_topic": {},
|
||||
"delete_topic": {},
|
||||
"create_rule": {},
|
||||
"create_preference": {},
|
||||
"create_project_rule": {"project_id": PROJECT},
|
||||
"update_rule": {"rule_id": RULE},
|
||||
"update_preference": {"rule_id": RULE},
|
||||
"move_rule": {"rule_id": RULE, "project_id": PROJECT},
|
||||
"delete_rule": {"rule_id": RULE},
|
||||
"mark_rule_verified": {"rule_id": RULE},
|
||||
"relate_rules": {"from_rule_id": RULE, "to_rule_id": RULE},
|
||||
"unrelate_rules": {"relation_id": RELATION},
|
||||
"rule_outcome": {},
|
||||
# moments and retrieval: install-level, and judgments asked for anywhere
|
||||
"tune_retrieval": {},
|
||||
"migrate_retrieval_floor": {},
|
||||
"map_action": {},
|
||||
"unmap_action": {},
|
||||
"propose_rule_moments": {},
|
||||
"judge_rule_moments": {},
|
||||
"rule_misfired": {},
|
||||
"judge_menu": {},
|
||||
# trash: a batch can hold rows from several projects
|
||||
"restore": {"deleted_batch_id": TRASH},
|
||||
"purge_trash": {"deleted_batch_id": TRASH},
|
||||
}
|
||||
|
||||
# Argument names that would locate a write wherever they appear. The guard test
|
||||
# requires every write tool parameter with one of these names to be declared
|
||||
# above, or listed in REFERENCES below, so a tool that gains one can't write
|
||||
# past the fence unnoticed.
|
||||
LOCATOR_NAMES = frozenset({
|
||||
"project_id", "task_id", "note_id", "lesson_id", "snippet_id", "process_id",
|
||||
"milestone_id", "system_id", "rule_id", "deleted_batch_id",
|
||||
})
|
||||
|
||||
# Arguments with a locating name that only point at a record, with why each one
|
||||
# doesn't say where the write lands.
|
||||
REFERENCES: dict[str, dict[str, str]] = {
|
||||
"classify_shapes_by_rule": {
|
||||
"snippet_id": "the canon the shapes are judged against; the rows written are project_id's",
|
||||
},
|
||||
"confirm_shape_proposals": {
|
||||
"snippet_id": "narrows which proposals to confirm; the rows written are project_id's",
|
||||
},
|
||||
"judge_lesson_link": {
|
||||
"lesson_id": "a judgment on a lesson-to-rule pairing the system asked for",
|
||||
"rule_id": "a judgment on a lesson-to-rule pairing the system asked for",
|
||||
},
|
||||
"rule_outcome": {
|
||||
"rule_id": "telemetry on what a rule changed in this session's own work",
|
||||
},
|
||||
"rule_misfired": {
|
||||
"rule_id": "telemetry on a moment that fired a rule in this session",
|
||||
"project_id": "the project the misfire happened in, recorded with the report",
|
||||
},
|
||||
}
|
||||
|
||||
# Tool-name prefixes the plugin's fence answers without asking the server: every
|
||||
# tool named this way reads. tests/test_write_scope.py holds the hook's copy of
|
||||
# this list equal to it, and holds every write tool to a name outside it.
|
||||
READ_PREFIXES = (
|
||||
"get_", "list_", "search", "find_", "enter_project", "what_might_apply",
|
||||
"resolve_design_system",
|
||||
)
|
||||
|
||||
# The one write a session may make in another project: filing an issue there,
|
||||
# as a new open task outside any plan of theirs.
|
||||
ISSUE_STATUSES = frozenset({"", "todo"})
|
||||
|
||||
|
||||
def is_issue_filing(tool: str, args: dict) -> bool:
|
||||
"""True for create_task(kind="issue") that opens a fresh, unplanned issue."""
|
||||
if tool != "create_task":
|
||||
return False
|
||||
if str(args.get("kind") or "") != "issue":
|
||||
return False
|
||||
if str(args.get("status") or "") not in ISSUE_STATUSES:
|
||||
return False
|
||||
return not _positive_ints(args.get("milestone_id")) and not _positive_ints(args.get("parent_id"))
|
||||
|
||||
|
||||
def _positive_ints(value) -> list[int]:
|
||||
"""The ids an argument names. 0 and negative values mean "none" or "leave
|
||||
unchanged" across the tools (update_task's project_id=-1 clears), so only
|
||||
positive ids locate anything."""
|
||||
values = value if isinstance(value, (list, tuple)) else [value]
|
||||
out = []
|
||||
for v in values:
|
||||
try:
|
||||
n = int(v)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if n > 0:
|
||||
out.append(n)
|
||||
return out
|
||||
|
||||
|
||||
async def _projects_of(user_id: int, kind: str, value) -> set[int]:
|
||||
"""The projects the argument `value` of locator `kind` lands in."""
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.family import FamilyDecision
|
||||
from grimoire.models.milestone import Milestone
|
||||
from grimoire.models.note import Note
|
||||
from grimoire.models.rulebook import Rule, RuleRelation
|
||||
from grimoire.models.system import System
|
||||
from grimoire.services import repo_bindings
|
||||
|
||||
if kind == REPO:
|
||||
found = await repo_bindings.resolve_project(user_id, str(value or ""))
|
||||
return {found} if found else set()
|
||||
if kind == TRASH:
|
||||
batch = str(value or "").strip()
|
||||
if not batch:
|
||||
return set()
|
||||
async with async_session() as session:
|
||||
found: set[int] = set()
|
||||
for model in (Note, Milestone, System, Rule):
|
||||
rows = await session.execute(
|
||||
select(model.project_id).where(model.deleted_batch_id == batch)
|
||||
)
|
||||
found.update(p for p in rows.scalars() if p)
|
||||
return found
|
||||
ids = _positive_ints(value)
|
||||
if not ids:
|
||||
return set()
|
||||
if kind == PROJECT:
|
||||
return set(ids)
|
||||
async with async_session() as session:
|
||||
if kind == RELATION:
|
||||
rows = await session.execute(
|
||||
select(Rule.project_id)
|
||||
.join(RuleRelation, or_(RuleRelation.from_rule_id == Rule.id,
|
||||
RuleRelation.to_rule_id == Rule.id))
|
||||
.where(RuleRelation.id.in_(ids))
|
||||
)
|
||||
return {p for p in rows.scalars() if p}
|
||||
if kind == DECISION:
|
||||
rows = await session.execute(
|
||||
select(FamilyDecision.project_id, Note.project_id)
|
||||
.join(Note, Note.id == FamilyDecision.idea_id)
|
||||
.where(FamilyDecision.id.in_(ids))
|
||||
)
|
||||
return {own or home for own, home in rows.all() if own or home}
|
||||
model = {NOTE: Note, MILESTONE: Milestone, SYSTEM: System, RULE: Rule}[kind]
|
||||
rows = await session.execute(select(model.project_id).where(model.id.in_(ids)))
|
||||
return {p for p in rows.scalars() if p}
|
||||
|
||||
|
||||
async def tool_targets(user_id: int, tool: str, args: dict) -> set[int]:
|
||||
"""Every project the MCP call `tool(**args)` would change. Empty for a
|
||||
read, a write that lands in no project, and a tool this server doesn't
|
||||
have."""
|
||||
found: set[int] = set()
|
||||
for name, kind in WRITE_TARGETS.get(tool, {}).items():
|
||||
if name in args:
|
||||
found |= await _projects_of(user_id, kind, args[name])
|
||||
return found
|
||||
|
||||
|
||||
async def _titles(ids: list[int]) -> dict[int, str]:
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.project import Project
|
||||
|
||||
if not ids:
|
||||
return {}
|
||||
async with async_session() as session:
|
||||
rows = await session.execute(select(Project.id, Project.title).where(Project.id.in_(ids)))
|
||||
return {pid: title for pid, title in rows.all()}
|
||||
|
||||
|
||||
def _named(project_id: int, titles: dict[int, str]) -> str:
|
||||
title = titles.get(project_id)
|
||||
return f"{title} (project {project_id})" if title else f"project {project_id}"
|
||||
|
||||
|
||||
async def check_act(
|
||||
user_id: int, session_project: int, *, tool: str = "", args: dict | None = None,
|
||||
target_project: int = 0, act: str = "",
|
||||
) -> dict:
|
||||
"""May a session in `session_project` do this act?
|
||||
|
||||
The act is an MCP call (`tool` with `args`) or a directory already
|
||||
resolved to `target_project`; `act` words it for the reason ("git push
|
||||
in", "Editing a file in"). Returns:
|
||||
- `allowed`: False when the act would change another project and isn't the
|
||||
one cross-project write allowed;
|
||||
- `targets`: the other projects, as {id, title};
|
||||
- `issue`: the act is that allowed write, filing an issue;
|
||||
- `reason`: what the operator reads in the prompt, and the session after.
|
||||
|
||||
A session with no project is held to nothing. A target the caller can't
|
||||
read is left out: the write itself is refused there, so there is nothing
|
||||
to ask about, and naming it would tell the caller where a record lives.
|
||||
"""
|
||||
from grimoire.services.access import can_read_project
|
||||
|
||||
args = args or {}
|
||||
if tool:
|
||||
landing = await tool_targets(user_id, tool, args)
|
||||
else:
|
||||
landing = {target_project} if target_project else set()
|
||||
others = [
|
||||
p for p in sorted(landing)
|
||||
if p != session_project and await can_read_project(user_id, p)
|
||||
]
|
||||
if not session_project or not others:
|
||||
return {"allowed": True, "targets": [], "issue": False, "reason": ""}
|
||||
titles = await _titles([session_project, *others])
|
||||
targets = [{"id": p, "title": titles.get(p, "")} for p in others]
|
||||
if tool and is_issue_filing(tool, args):
|
||||
return {"allowed": True, "targets": targets, "issue": True, "reason": ""}
|
||||
there = ", ".join(_named(p, titles) for p in others)
|
||||
what = act.strip() or (f"{tool} changes" if tool else "This changes")
|
||||
first = others[0]
|
||||
reason = (
|
||||
f"This session belongs to {_named(session_project, titles)}. {what} {there}, "
|
||||
f"a different project. From here, the one change to make there is filing an "
|
||||
f"issue: create_task(project_id={first}, kind=\"issue\") with how it was found "
|
||||
f"and the evidence. Approve only if this session should change {there} itself."
|
||||
)
|
||||
return {"allowed": False, "targets": targets, "issue": False, "reason": reason}
|
||||
@@ -90,6 +90,23 @@ def test_after_write_names_what_bash_just_wrote_then_stays_quiet_until_the_next_
|
||||
assert seen[0]["exclude_derive"] == ["dup:483a"]
|
||||
assert set(seen[0]["shapes"][0].split(",")) == {"css:log-empty", "css:other"}
|
||||
|
||||
def test_after_write_records_every_definition_for_the_turn_but_queries_only_the_first_four_files(tmp_path):
|
||||
"""The end-of-turn ledger hears every file and every definition a Bash call
|
||||
wrote; the prior-art arms stay bounded to four files and twelve names."""
|
||||
with http_sink(SINK_REPLY) as (port, seen):
|
||||
env = _env(tmp_path, url=f"http://127.0.0.1:{port}")
|
||||
repo = _repo(tmp_path, env)
|
||||
for n in range(6):
|
||||
body = "".join(f"def f{n}_{i:02d}():\n return {i}\n\n" for i in range(15))
|
||||
(repo / f"gen{n}.py").write_text(body)
|
||||
_run(repo, env, session="s-many")
|
||||
ledger = (tmp_path / "grimoire-priorart" / "s-many.written.ids").read_text().splitlines()
|
||||
assert len(ledger) == 6 * 15
|
||||
assert "gen5.py\tsym\tf5_14" in ledger
|
||||
assert len(seen) == 4
|
||||
assert all(len(q["shapes"][0].split(",")) == 12 for q in seen)
|
||||
|
||||
|
||||
def test_after_write_is_silent_where_it_has_nothing_to_say(tmp_path):
|
||||
env = _env(tmp_path)
|
||||
repo = _repo(tmp_path, env)
|
||||
|
||||
@@ -20,6 +20,7 @@ from grimoire.services.plugin_context import (
|
||||
_AUTOINJECT_CONTEXT_MAX,
|
||||
_AUTOINJECT_CONTEXT_PROMPT_MAX,
|
||||
_autoinject_query,
|
||||
_has_subject,
|
||||
_menu_name,
|
||||
_menu_passage,
|
||||
)
|
||||
@@ -56,6 +57,50 @@ def test_context_is_cut_from_the_end_of_the_reply():
|
||||
assert tail.endswith("TAIL") and "HEAD" not in tail
|
||||
|
||||
|
||||
# --- a prompt that names nothing ----------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("prompt", [
|
||||
"keep going", "yes please do", "1. yes 2. yes 3. yes", "start step 6",
|
||||
"is it deployed?", "awesome, what's next?", "go ahead",
|
||||
"is there anything else open in this project?",
|
||||
"please commit and let me know when it passes CI",
|
||||
])
|
||||
def test_a_turn_of_the_conversation_names_no_subject(prompt):
|
||||
assert not _has_subject(prompt)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("prompt", [
|
||||
"now fix the filter", "keep steward, it's our own app",
|
||||
"continue into DRY 14", "I've added a maintenance plan",
|
||||
"fix the Windows gate first, then step 2",
|
||||
])
|
||||
def test_one_word_of_its_own_is_a_subject(prompt):
|
||||
assert _has_subject(prompt)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_prompt_with_no_subject_gets_no_menu_and_no_search():
|
||||
"""Its query would be the previous reply's tail alone, and the menu would
|
||||
hand back what that reply was built from."""
|
||||
from grimoire.services import plugin_context as pc
|
||||
|
||||
search = AsyncMock(return_value=[(0.9, fake_note(id=11, title="the reply's own record", user_id=1))])
|
||||
retrieval = MagicMock()
|
||||
with patch.object(pc, "get_autoinject_config",
|
||||
AsyncMock(return_value={"enabled": True, "threshold": 0.55, "top_k": 3})), \
|
||||
patch.object(pc, "semantic_search_notes", search), \
|
||||
patch.object(pc, "record_retrieval", retrieval), \
|
||||
patch.object(pc, "record_surfaced", MagicMock()):
|
||||
out = await pc.build_autoinject_hint(
|
||||
1, "yes please do", project_id=2,
|
||||
context="I'll move the library filters into the side column.",
|
||||
)
|
||||
|
||||
assert out["context"] == "" and out["note_ids"] == []
|
||||
search.assert_not_called()
|
||||
retrieval.assert_not_called()
|
||||
|
||||
|
||||
# --- what the hook extracts --------------------------------------------------
|
||||
|
||||
def _rec(role: str, blocks: list[dict], sidechain: bool = False) -> str:
|
||||
|
||||
@@ -5,7 +5,8 @@ the real `restore_full_backup`.
|
||||
The unit lane already shows the tar hands back the dict it was given. What
|
||||
only this lane shows is that the dict it hands back is one the restore
|
||||
accepts — every id remapped to the restored rows, nothing dropped — and that
|
||||
the command-line restore refuses a database that already has accounts.
|
||||
the command-line restore refuses, on a live instance, a backup that doesn't
|
||||
record this instance as its source.
|
||||
"""
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
@@ -97,10 +98,11 @@ async def test_the_tar_restores_with_every_id_remapped(written):
|
||||
assert note.status == "todo"
|
||||
|
||||
|
||||
async def test_the_command_refuses_a_database_with_accounts(written):
|
||||
"""The source user exists, so this database is not empty: the command
|
||||
must stop before restoring anything."""
|
||||
assert await run(written) == 3
|
||||
async def test_the_command_refuses_a_backup_from_no_known_instance(written):
|
||||
"""This database is live and the backup names no instance it came from,
|
||||
so it can't be rolled back to: the command stops before restoring
|
||||
anything, even with --yes."""
|
||||
assert await run(written, yes=True) == 3
|
||||
async with async_session() as s:
|
||||
restored = (await s.execute(select(User).where(User.username == RESTORED))).scalars().all()
|
||||
assert restored == []
|
||||
|
||||
@@ -0,0 +1,214 @@
|
||||
"""Real-Postgres proof that a full backup rolls this instance back by id.
|
||||
|
||||
The lane shares one database, so every test here takes its backup of the
|
||||
whole store, changes its own rows, and rolls back to that backup: what it
|
||||
puts back is exactly what was there when it started.
|
||||
"""
|
||||
from datetime import datetime, timezone
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.note import Note
|
||||
from grimoire.models.project import Project
|
||||
from grimoire.models.user import User
|
||||
from grimoire.restore import run
|
||||
from grimoire.services import backup, backup_job
|
||||
from grimoire.services.api_keys import create_api_key, lookup_key
|
||||
from grimoire.services.backup_file import write_archive
|
||||
from grimoire.services.instance import adopt_instance_id, get_instance_id
|
||||
from tests.helpers import ensure_user
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
|
||||
|
||||
OWNER = "rollback_owner"
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def seeded():
|
||||
"""One user with a project and two tasks, committed, and a full backup
|
||||
taken right after."""
|
||||
async with async_session() as s:
|
||||
user = await ensure_user(s, OWNER)
|
||||
# A clean slate for this user: an earlier test's rows would otherwise
|
||||
# be in this test's backup too.
|
||||
for model in (Note, Project):
|
||||
for row in (await s.execute(select(model).where(model.user_id == user.id))).scalars():
|
||||
await s.delete(row)
|
||||
await s.flush()
|
||||
project = Project(user_id=user.id, title="Rollback drill")
|
||||
s.add(project)
|
||||
await s.flush()
|
||||
kept = Note(user_id=user.id, project_id=project.id, title="before",
|
||||
body="as backed up", note_type="note", task_kind="work", status="todo")
|
||||
doomed = Note(user_id=user.id, project_id=project.id, title="deleted later",
|
||||
body="", note_type="note", task_kind="work", status="todo")
|
||||
s.add_all([kept, doomed])
|
||||
await s.commit()
|
||||
ids = {"user": user.id, "project": project.id, "kept": kept.id, "doomed": doomed.id}
|
||||
key, _ = await create_api_key(ids["user"], "rollback drill", "read")
|
||||
payload = await backup.export_full_backup()
|
||||
return ids, payload, key
|
||||
|
||||
|
||||
def _own(payload: dict, table: str, user_id: int) -> list[dict]:
|
||||
return sorted(
|
||||
(row for row in payload[table] if row.get("user_id") == user_id),
|
||||
key=lambda row: row["id"],
|
||||
)
|
||||
|
||||
|
||||
async def _change_everything(ids: dict) -> int:
|
||||
"""Edit one task, delete the other, add a task and a project. Returns the
|
||||
new task's id."""
|
||||
async with async_session() as s:
|
||||
kept = await s.get(Note, ids["kept"])
|
||||
kept.title = "after"
|
||||
kept.body = "edited since the backup"
|
||||
await s.delete(await s.get(Note, ids["doomed"]))
|
||||
extra = Project(user_id=ids["user"], title="made after the backup")
|
||||
s.add(extra)
|
||||
await s.flush()
|
||||
new = Note(user_id=ids["user"], project_id=extra.id, title="new since",
|
||||
body="", note_type="note", task_kind="work", status="todo")
|
||||
s.add(new)
|
||||
await s.commit()
|
||||
return new.id
|
||||
|
||||
|
||||
async def test_a_rollback_puts_back_edits_and_deletions_and_removes_new_records(seeded):
|
||||
ids, payload, _ = seeded
|
||||
new_id = await _change_everything(ids)
|
||||
|
||||
result = await backup.replace_from_backup(payload)
|
||||
|
||||
assert result["mode"] == "rollback"
|
||||
async with async_session() as s:
|
||||
kept = await s.get(Note, ids["kept"])
|
||||
assert (kept.title, kept.body) == ("before", "as backed up")
|
||||
assert await s.get(Note, ids["doomed"]) is not None
|
||||
assert await s.get(Note, new_id) is None
|
||||
titles = (await s.execute(
|
||||
select(Project.title).where(Project.user_id == ids["user"])
|
||||
)).scalars().all()
|
||||
assert titles == ["Rollback drill"]
|
||||
|
||||
after = await backup.export_full_backup()
|
||||
for table in ("projects", "milestones", "notes"):
|
||||
assert _own(after, table, ids["user"]) == _own(payload, table, ids["user"]), table
|
||||
|
||||
|
||||
async def test_a_surviving_users_api_key_still_works_and_their_sessions_end(seeded):
|
||||
ids, payload, key = seeded
|
||||
async with async_session() as s:
|
||||
version_before = (await s.get(User, ids["user"])).session_version
|
||||
await _change_everything(ids)
|
||||
|
||||
await backup.replace_from_backup(payload)
|
||||
|
||||
found = await lookup_key(key)
|
||||
assert found is not None and found.user_id == ids["user"]
|
||||
async with async_session() as s:
|
||||
assert (await s.get(User, ids["user"])).session_version == version_before + 1
|
||||
|
||||
|
||||
async def test_new_records_after_a_rollback_take_fresh_ids(seeded):
|
||||
ids, payload, _ = seeded
|
||||
await _change_everything(ids)
|
||||
await backup.replace_from_backup(payload)
|
||||
|
||||
async with async_session() as s:
|
||||
highest = (await s.execute(select(func.max(Note.id)))).scalar_one()
|
||||
note = Note(user_id=ids["user"], project_id=ids["project"], title="after rollback",
|
||||
body="", note_type="note", task_kind="work", status="todo")
|
||||
s.add(note)
|
||||
await s.commit()
|
||||
assert note.id > highest
|
||||
await s.delete(note)
|
||||
await s.commit()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("instance_id", ["another-instance", None])
|
||||
async def test_a_backup_from_elsewhere_is_refused_on_a_live_instance(seeded, instance_id):
|
||||
ids, payload, _ = seeded
|
||||
foreign = {**payload, "instance_id": instance_id}
|
||||
new_id = await _change_everything(ids)
|
||||
|
||||
with pytest.raises(backup.RestoreRefused):
|
||||
await backup.replace_from_backup(foreign)
|
||||
|
||||
async with async_session() as s:
|
||||
assert (await s.get(Note, ids["kept"])).title == "after"
|
||||
assert await s.get(Note, new_id) is not None
|
||||
await backup.replace_from_backup(payload)
|
||||
|
||||
|
||||
async def test_a_users_own_export_is_refused(seeded):
|
||||
_, payload, _ = seeded
|
||||
with pytest.raises(backup.RestoreRefused):
|
||||
await backup.replace_from_backup({**payload, "scope": "user"})
|
||||
|
||||
|
||||
async def test_a_failure_part_way_changes_nothing(seeded):
|
||||
ids, payload, _ = seeded
|
||||
new_id = await _change_everything(ids)
|
||||
|
||||
with patch.object(backup, "_reset_sequences", AsyncMock(side_effect=RuntimeError("boom"))):
|
||||
with pytest.raises(RuntimeError):
|
||||
await backup.replace_from_backup(payload)
|
||||
|
||||
async with async_session() as s:
|
||||
assert (await s.get(Note, ids["kept"])).title == "after"
|
||||
assert await s.get(Note, ids["doomed"]) is None
|
||||
assert await s.get(Note, new_id) is not None
|
||||
await backup.replace_from_backup(payload)
|
||||
|
||||
|
||||
async def test_a_fresh_instance_adopts_the_backups_instance_id(seeded):
|
||||
_, payload, _ = seeded
|
||||
own = await get_instance_id()
|
||||
try:
|
||||
with patch.object(backup, "_is_fresh", AsyncMock(return_value=True)):
|
||||
result = await backup.replace_from_backup({**payload, "instance_id": "a-lineage"})
|
||||
assert result["mode"] == "fresh"
|
||||
assert await get_instance_id() == "a-lineage"
|
||||
finally:
|
||||
async with async_session() as s:
|
||||
await adopt_instance_id(s, own)
|
||||
await s.commit()
|
||||
|
||||
|
||||
async def test_the_command_only_previews_without_yes(seeded, tmp_path):
|
||||
ids, payload, _ = seeded
|
||||
path = write_archive(payload, tmp_path, datetime.now(timezone.utc), app_version="test")
|
||||
new_id = await _change_everything(ids)
|
||||
|
||||
assert await run(path) == 4
|
||||
|
||||
async with async_session() as s:
|
||||
assert (await s.get(Note, ids["kept"])).title == "after"
|
||||
assert await s.get(Note, new_id) is not None
|
||||
await backup.replace_from_backup(payload)
|
||||
|
||||
|
||||
async def test_the_command_rolls_back_from_the_tar_with_yes(seeded, tmp_path):
|
||||
"""The scheduled backup's own file, read back through the command: the
|
||||
whole store returns to it, after the safety backup is taken."""
|
||||
ids, payload, _ = seeded
|
||||
path = write_archive(payload, tmp_path, datetime.now(timezone.utc), app_version="test")
|
||||
new_id = await _change_everything(ids)
|
||||
saved = {"file": "grimoire-safety.tar", "size_bytes": 1, "where": str(tmp_path)}
|
||||
|
||||
with patch.object(backup_job, "save_state_before_restore",
|
||||
AsyncMock(return_value=saved)) as save:
|
||||
assert await run(path, yes=True) == 0
|
||||
|
||||
save.assert_awaited_once()
|
||||
async with async_session() as s:
|
||||
kept = await s.get(Note, ids["kept"])
|
||||
assert (kept.title, kept.body) == ("before", "as backed up")
|
||||
assert await s.get(Note, ids["doomed"]) is not None
|
||||
assert await s.get(Note, new_id) is None
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Real-Postgres proof that every instance has one id lineage, that a full
|
||||
backup carries it, and that adopting a backup's id replaces this one's."""
|
||||
import io
|
||||
|
||||
import pytest
|
||||
|
||||
from grimoire.models import async_session
|
||||
from grimoire.services import backup
|
||||
from grimoire.services.backup_file import read_archive, write_archive_to
|
||||
from grimoire.services.instance import adopt_instance_id, get_instance_id
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
|
||||
|
||||
|
||||
async def test_the_migration_gave_this_instance_an_id_and_it_stays_put():
|
||||
first = await get_instance_id()
|
||||
assert first and len(first) >= 32
|
||||
assert await get_instance_id() == first
|
||||
|
||||
|
||||
async def test_a_full_backup_carries_the_instance_id_through_the_tar():
|
||||
payload = await backup.export_full_backup()
|
||||
assert payload["instance_id"] == await get_instance_id()
|
||||
|
||||
out = io.BytesIO()
|
||||
write_archive_to(payload, out, app_version="test")
|
||||
out.seek(0)
|
||||
assert read_archive(out)["instance_id"] == payload["instance_id"]
|
||||
|
||||
|
||||
async def test_adopting_a_backups_id_replaces_this_instances():
|
||||
own = await get_instance_id()
|
||||
try:
|
||||
async with async_session() as s:
|
||||
await adopt_instance_id(s, "adopted-lineage")
|
||||
await s.commit()
|
||||
assert await get_instance_id() == "adopted-lineage"
|
||||
finally:
|
||||
async with async_session() as s:
|
||||
await adopt_instance_id(s, own)
|
||||
await s.commit()
|
||||
assert await get_instance_id() == own
|
||||
@@ -93,6 +93,22 @@ async def test_naming_rules_confirms_them_and_both_sides_read_the_link(world):
|
||||
assert [(l["id"], l["state"]) for l in lessons] == [(world["lesson"], "confirmed")]
|
||||
|
||||
|
||||
async def test_a_rule_list_counts_confirmed_lessons_only(world):
|
||||
"""The bulk count a rule list reads: a rejected link is not a lesson
|
||||
proving the rule, a rule with no links reads 0, and a stranger counts
|
||||
nothing through a lesson they cannot read."""
|
||||
owner = world["owner"]
|
||||
await links_svc.set_lesson_rules(owner, world["lesson"], [world["r1"], world["r2"]])
|
||||
await links_svc.judge_link(owner, world["lesson"], world["r2"], "reject", "not it")
|
||||
rules = [world["r1"], world["r2"], world["foreign"]]
|
||||
|
||||
assert await links_svc.confirmed_lesson_counts(owner, rules) == {
|
||||
world["r1"]: 1, world["r2"]: 0, world["foreign"]: 0,
|
||||
}
|
||||
assert (await links_svc.confirmed_lesson_counts(world["stranger"], rules))[world["r1"]] == 0
|
||||
assert await links_svc.confirmed_lesson_counts(owner, []) == {}
|
||||
|
||||
|
||||
async def test_a_rule_left_out_of_the_set_is_rejected_not_forgotten(world):
|
||||
owner = world["owner"]
|
||||
await links_svc.set_lesson_rules(owner, world["lesson"], [world["r1"], world["r2"]])
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
"""The saved proxy hop count against a real database.
|
||||
|
||||
What only a real database shows: the count is read back after a save, saving
|
||||
leaves exactly one copy however many admins there are, saving None removes it,
|
||||
and a saved count reaches `client_ip()` through the worker's cache.
|
||||
"""
|
||||
import pytest
|
||||
from quart import Quart
|
||||
from sqlalchemy import select
|
||||
|
||||
from grimoire import proxy
|
||||
from grimoire.config import Config
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.setting import Setting
|
||||
from grimoire.proxy import client_ip
|
||||
from grimoire.services import proxy_settings
|
||||
from tests.helpers import ensure_user
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
|
||||
|
||||
|
||||
async def _copies() -> list[Setting]:
|
||||
async with async_session() as s:
|
||||
return list((await s.execute(
|
||||
select(Setting).where(Setting.key == proxy_settings.SETTING_KEY)
|
||||
)).scalars().all())
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def two_admins():
|
||||
async with async_session() as s:
|
||||
first = await ensure_user(s, "proxy_hops_admin_a", role="admin")
|
||||
second = await ensure_user(s, "proxy_hops_admin_b", role="admin")
|
||||
await s.commit()
|
||||
ids = (first.id, second.id)
|
||||
yield ids
|
||||
await proxy_settings.save_hops(None)
|
||||
proxy.set_saved_hops(None)
|
||||
|
||||
|
||||
async def test_a_saved_count_reads_back_and_leaves_one_copy(two_admins):
|
||||
# A stale copy on the second admin, as an older write path could leave.
|
||||
async with async_session() as s:
|
||||
s.add(Setting(user_id=two_admins[1], key=proxy_settings.SETTING_KEY, value="4"))
|
||||
await s.commit()
|
||||
|
||||
await proxy_settings.save_hops(2)
|
||||
assert await proxy_settings.saved_hops() == 2
|
||||
copies = await _copies()
|
||||
assert len(copies) == 1 and copies[0].value == "2"
|
||||
|
||||
|
||||
async def test_saving_none_removes_the_count(two_admins):
|
||||
await proxy_settings.save_hops(3)
|
||||
await proxy_settings.save_hops(None)
|
||||
assert await proxy_settings.saved_hops() is None
|
||||
assert await _copies() == []
|
||||
assert proxy.saved_hops_in_force() is None
|
||||
|
||||
|
||||
async def test_a_saved_count_reaches_the_client_address(two_admins, monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
await proxy_settings.save_hops(2)
|
||||
proxy.set_saved_hops(None) # as in another worker, before its refresh
|
||||
await proxy_settings.refresh()
|
||||
|
||||
app = Quart(__name__)
|
||||
headers = {"Remote-Addr": "10.0.0.5", "X-Forwarded-For": "203.0.113.7, 172.70.1.1"}
|
||||
async with app.test_request_context("/", headers=headers):
|
||||
assert client_ip() == "203.0.113.7"
|
||||
@@ -143,6 +143,8 @@ async def test_the_signal_counts_sessions_and_asks_once_the_bar_is_crossed(world
|
||||
for situation in ("s1", "s1", "s2"):
|
||||
assert await judgments_svc.co_occurred(uid, done.id, ["work.verify"],
|
||||
situation=situation) == ""
|
||||
# Below the bar the pair is gathering evidence, not waiting on anyone.
|
||||
assert (await judgments_svc.pending(uid, rule_id=done.id))["rules"] == []
|
||||
line = await judgments_svc.co_occurred(uid, done.id, ["work.verify"], situation="s3")
|
||||
assert f"#{done.id}" in line and "`work.verify`" in line and "3 distinct sessions" in line
|
||||
assert "judge_rule_moments" in line
|
||||
@@ -193,7 +195,23 @@ async def test_the_open_resolves_acts_through_the_installs_mappings(world):
|
||||
out = await judgments_svc.opened_after(uid, done.id, acts, session_id="s1")
|
||||
assert "work.deliver" in out["moments"]
|
||||
assert "work.run" not in out["moments"]
|
||||
assert (await _row(done.id, "work.deliver")).source == "signal"
|
||||
row = await _row(done.id, "work.deliver")
|
||||
assert row.source == "signal"
|
||||
# The action that reached the moment is kept, and shown as the why.
|
||||
assert row.evidence["reached_by"] == {"git push": 1}
|
||||
assert row.note.startswith("Opened just after `work.deliver` (reached by `git push`)")
|
||||
|
||||
|
||||
async def test_a_rule_already_mounted_gathers_no_signal(world):
|
||||
"""A mounted rule arrives by lookup at its own moments, so an open of it
|
||||
is no evidence it is missing another."""
|
||||
uid, done = world["uid"], world["done"]
|
||||
await rulebooks_svc.set_rule_moments(done.id, uid, ["work.finish"])
|
||||
for situation in ("a", "b", "c", "d"):
|
||||
assert await judgments_svc.co_occurred(
|
||||
uid, done.id, ["work.verify"], situation=situation,
|
||||
) == ""
|
||||
assert await _row(done.id, "work.verify") is None
|
||||
|
||||
|
||||
# ── The misfire ──────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -238,9 +238,10 @@ async def test_the_delivery_lookup_answers_by_home(world):
|
||||
|
||||
|
||||
async def test_moment_usage_counts_deliveries_and_the_agent_opens_that_followed(world):
|
||||
"""The per-moment readout, on real SQL: an EXISTS inside a
|
||||
COUNT(CASE …) is a shape no mock can vouch for, and the block's own guard
|
||||
would turn a database refusal into a quiet `moment_usage_failed`."""
|
||||
"""The per-moment readout, on real SQL: a correlated EXISTS selected as a
|
||||
column beside an outer join is a shape no mock can vouch for, and the
|
||||
block's own guard would turn a database refusal into a quiet
|
||||
`moment_usage_failed`."""
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import delete
|
||||
@@ -289,3 +290,13 @@ async def test_moment_usage_counts_deliveries_and_the_agent_opens_that_followed(
|
||||
report = block["by_moment"]["reply.report"]
|
||||
assert (report["delivered"], report["rules"], report["opened"]) == (1, 1, 0)
|
||||
assert report["last_delivered_at"]
|
||||
|
||||
# Two rules share work.finish: which one arrived, and which was opened.
|
||||
assert finish["by_rule"] == [
|
||||
{"rule_id": rule.id, "title": rule.title, "delivered": 2, "opened": True},
|
||||
{"rule_id": other.id, "title": "Close the task with the push",
|
||||
"delivered": 1, "opened": False},
|
||||
]
|
||||
assert report["by_rule"] == [
|
||||
{"rule_id": rule.id, "title": rule.title, "delivered": 1, "opened": False},
|
||||
]
|
||||
|
||||
@@ -294,6 +294,43 @@ async def test_get_snippet_carries_the_structured_consumer_map(seeded):
|
||||
assert consumers == {"instances": [], "variants": [], "uses": []}
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_a_reread_takes_back_the_reference_edges_a_looser_rule_wrote(seeded):
|
||||
"""record_uses only adds; a tightened by-name rule must also remove what
|
||||
the old rule wrote. A judged edge on the same row survives."""
|
||||
from grimoire.services import snippets as snippets_svc
|
||||
from grimoire.services.shape_ledger import record_uses, replace_reference_uses
|
||||
|
||||
owner, pid, sid = seeded["owner"], seeded["pid"], seeded["snippet"]
|
||||
helper = await snippets_svc.create_snippet(
|
||||
owner, name="cls_reread_helper", code="def hash_token(raw):\n return raw\n",
|
||||
language="python", repo="Widget", path="src/hash.py", symbol="hash_token",
|
||||
project_id=pid,
|
||||
)
|
||||
hid = int(helper.id)
|
||||
|
||||
async def make_app_row(s):
|
||||
return (await s.execute(select(CodeShape).where(
|
||||
CodeShape.project_id == pid, CodeShape.symbol == "make_app",
|
||||
))).scalar_one()
|
||||
|
||||
async with async_session() as s:
|
||||
await record_uses(s, await make_app_row(s), [sid, hid], basis="reference")
|
||||
await s.commit()
|
||||
await classify_shapes(owner, pid, [
|
||||
{"path": "src/app.py", "symbol": "make_app", "status": "exempt",
|
||||
"reason": "its own thing", "uses": [hid]},
|
||||
], via="audit")
|
||||
|
||||
async with async_session() as s:
|
||||
await replace_reference_uses(s, await make_app_row(s), [])
|
||||
await s.commit()
|
||||
_, n_sid = await list_project_shapes(owner, pid, uses=sid)
|
||||
rows_hid, n_hid = await list_project_shapes(owner, pid, uses=hid)
|
||||
assert n_sid == 0
|
||||
assert n_hid == 1 and rows_hid[0].symbol == "make_app"
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_sync_refiles_rows_whose_snippet_was_purged(seeded):
|
||||
"""The SET NULL companion: a judgment whose target is hard-deleted
|
||||
@@ -665,6 +702,28 @@ async def test_write_time_derive_names_the_family_or_the_canon_for_a_name(seeded
|
||||
assert await write_time_derive(pid, "v/B.vue", [("css", "log-empty")]) == []
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_new_shapes_at_names_only_the_definitions_the_file_does_not_hold(seeded):
|
||||
"""A shape the ledger holds at the path is an edit inside it; one it
|
||||
does not hold there — even under a name living elsewhere — is new."""
|
||||
from grimoire.services.shape_ledger import new_shapes_at
|
||||
|
||||
pid = seeded["pid"]
|
||||
await sync_repo_shapes(pid, REPO, _defs(
|
||||
("src/a.py", "sym", "load", "def load():", "def load():\n return 1"),
|
||||
("src/b.py", "sym", "save", "def save():", "def save():\n return 2"),
|
||||
("v/A.vue", "css", "log-empty", ".log-empty {", ".log-empty { color: red }"),
|
||||
), seen_marker="m1")
|
||||
|
||||
assert await new_shapes_at(pid, "src/a.py", [("sym", "load"), ("sym", "save")]) == [
|
||||
("sym", "save"),
|
||||
]
|
||||
# CSS compares without the leading dot, as the rest of the ledger does.
|
||||
assert await new_shapes_at(pid, "v/A.vue", [("css", ".log-empty")]) == []
|
||||
assert await new_shapes_at(pid, "src/new.py", [("sym", "load")]) == [("sym", "load")]
|
||||
assert await new_shapes_at(pid, "src/a.py", [("sym", "")]) == []
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_derive_new_names_the_copy_that_joined_a_family_since_the_stamp(seeded):
|
||||
"""The first sync seeds one `slug`; a later sync adds an identical
|
||||
@@ -1193,6 +1252,81 @@ async def test_a_shape_written_this_turn_is_judged_before_the_sync_sees_it(seede
|
||||
assert row.vanished_at is None and row.last_seen_commit == "c2"
|
||||
|
||||
|
||||
async def _rows_at(pid: int, path: str) -> list[CodeShape]:
|
||||
"""Every row at ``path``, vanished ones included: what the identity
|
||||
constraint sees."""
|
||||
async with async_session() as s:
|
||||
return list((await s.execute(select(CodeShape).where(
|
||||
CodeShape.project_id == pid, CodeShape.path == path,
|
||||
))).scalars().all())
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_judging_a_vanished_shape_revives_its_row(seeded):
|
||||
"""A verdict with `repo` on a shape whose row has vanished lands on that
|
||||
row and brings it back. It does not insert a second row under the same
|
||||
identity, which the database refuses, taking the whole batch with it."""
|
||||
from grimoire.services import repo_bindings as repo_bindings_svc
|
||||
|
||||
owner, pid, sid = seeded["owner"], seeded["pid"], seeded["snippet"]
|
||||
await repo_bindings_svc.set_binding(owner, REPO, pid)
|
||||
await sync_repo_shapes(
|
||||
pid, REPO, [s for s in SHAPES if s[0] != "src/util.py"], seen_marker="c1",
|
||||
)
|
||||
[gone] = await _rows_at(pid, "src/util.py")
|
||||
assert gone.vanished_at is not None
|
||||
|
||||
out = await classify_shapes(owner, pid, [
|
||||
{"path": "src/util.py", "symbol": "helper", "kind": "sym", "status": "exempt",
|
||||
"reason": "the module's own parse step", "reason_code": "pure-helper"},
|
||||
{"path": "src/app.py", "symbol": "make_app", "status": "instance", "snippet_id": sid},
|
||||
], repo=REPO)
|
||||
assert out["classified"] == 2 and out["unmatched"] == []
|
||||
|
||||
[row] = await _rows_at(pid, "src/util.py")
|
||||
assert row.id == gone.id and row.vanished_at is None
|
||||
assert (row.status, row.reason_code) == ("exempt", "pure-helper")
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_a_write_path_stamp_on_a_vanished_shape_revives_its_row(seeded):
|
||||
"""The write-path hook meets a shape being written again after a sync
|
||||
stamped it vanished. Its suggestion lands on the existing row, with no
|
||||
second insert, and a standing verdict on that row still stands."""
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from grimoire.services.shape_ledger import suggest_write_path_instances
|
||||
|
||||
owner, pid, sid = seeded["owner"], seeded["pid"], seeded["snippet"]
|
||||
pulled = {sid: datetime.now(timezone.utc)}
|
||||
code = "def helper():\n return factory()\n"
|
||||
await sync_repo_shapes(
|
||||
pid, REPO, [s for s in SHAPES if s[0] != "src/util.py"], seen_marker="c1",
|
||||
)
|
||||
|
||||
suggested = await suggest_write_path_instances(
|
||||
owner, pid, path="src/util.py", shapes=[("sym", "helper")],
|
||||
code=code, pulled=pulled, repo_key=REPO,
|
||||
)
|
||||
assert [s["symbol"] for s in suggested] == ["helper"]
|
||||
[row] = await _rows_at(pid, "src/util.py")
|
||||
assert row.vanished_at is None and row.proposed_snippet_id == sid
|
||||
|
||||
# A verdict on the row, then another vanish: the next stamp leaves it be.
|
||||
await classify_shapes(owner, pid, [
|
||||
{"path": "src/util.py", "symbol": "helper", "status": "exempt", "reason": "local"},
|
||||
])
|
||||
await sync_repo_shapes(
|
||||
pid, REPO, [s for s in SHAPES if s[0] != "src/util.py"], seen_marker="c2",
|
||||
)
|
||||
assert await suggest_write_path_instances(
|
||||
owner, pid, path="src/util.py", shapes=[("sym", "helper")],
|
||||
code=code, pulled=pulled, repo_key=REPO,
|
||||
) == []
|
||||
[row] = await _rows_at(pid, "src/util.py")
|
||||
assert row.status == "exempt" and row.classified_by == "agent"
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
async def test_unjudged_is_what_nobody_read(seeded):
|
||||
"""The end-of-turn question: a shape with no row, an unclassified one, a
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Real-Postgres proof that a write is traced to the project it lands in.
|
||||
|
||||
Two projects of one user: the session's own and another. Each locator kind is
|
||||
resolved from a real row, and check_act answers for a session standing in
|
||||
the first.
|
||||
"""
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from sqlalchemy import select
|
||||
|
||||
from grimoire.models import async_session
|
||||
from grimoire.models.milestone import Milestone
|
||||
from grimoire.models.note import Note
|
||||
from grimoire.models.project import Project
|
||||
from grimoire.models.system import System
|
||||
from grimoire.services import write_scope
|
||||
from tests.helpers import ensure_user
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
|
||||
|
||||
OWNER = "write_scope_owner"
|
||||
BATCH = "write-scope-test-batch"
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def two_projects():
|
||||
async with async_session() as s:
|
||||
user = await ensure_user(s, OWNER)
|
||||
for model in (Note, Milestone, System, Project):
|
||||
for row in (await s.execute(select(model).where(model.user_id == user.id))).scalars():
|
||||
await s.delete(row)
|
||||
await s.flush()
|
||||
home = Project(user_id=user.id, title="Home")
|
||||
other = Project(user_id=user.id, title="Elsewhere")
|
||||
s.add_all([home, other])
|
||||
await s.flush()
|
||||
plan = Milestone(user_id=user.id, project_id=other.id, title="Their plan")
|
||||
area = System(user_id=user.id, project_id=other.id, name="Their area")
|
||||
s.add_all([plan, area])
|
||||
await s.flush()
|
||||
theirs = Note(user_id=user.id, project_id=other.id, milestone_id=plan.id,
|
||||
title="Their task", body="", note_type="note", task_kind="work",
|
||||
status="todo")
|
||||
mine = Note(user_id=user.id, project_id=home.id, title="My task", body="",
|
||||
note_type="note", task_kind="work", status="todo")
|
||||
trashed = Note(user_id=user.id, project_id=other.id, title="Trashed", body="",
|
||||
note_type="note", deleted_batch_id=BATCH)
|
||||
s.add_all([theirs, mine, trashed])
|
||||
await s.commit()
|
||||
return {"user": user.id, "home": home.id, "other": other.id, "plan": plan.id,
|
||||
"area": area.id, "theirs": theirs.id, "mine": mine.id}
|
||||
|
||||
|
||||
async def test_each_locator_finds_the_project_of_the_row_it_names(two_projects):
|
||||
ids = two_projects
|
||||
other, home, uid = ids["other"], ids["home"], ids["user"]
|
||||
|
||||
async def lands(tool, **args):
|
||||
return await write_scope.tool_targets(uid, tool, args)
|
||||
|
||||
assert await lands("add_task_log", task_id=ids["theirs"]) == {other}
|
||||
assert await lands("update_task", task_id=ids["mine"]) == {home}
|
||||
assert await lands("update_task", task_id=ids["mine"], milestone_id=ids["plan"]) == {home, other}
|
||||
assert await lands("delete_milestone", milestone_id=ids["plan"]) == {other}
|
||||
assert await lands("update_system", system_id=ids["area"]) == {other}
|
||||
assert await lands("restore", deleted_batch_id=BATCH) == {other}
|
||||
assert await lands("create_rule", topic_id=1) == set()
|
||||
|
||||
|
||||
async def test_a_session_in_one_project_is_asked_before_writing_in_the_other(two_projects):
|
||||
ids = two_projects
|
||||
uid, home, other = ids["user"], ids["home"], ids["other"]
|
||||
|
||||
own = await write_scope.check_act(uid, home, tool="add_task_log",
|
||||
args={"task_id": ids["mine"]})
|
||||
assert own["allowed"] is True
|
||||
|
||||
theirs = await write_scope.check_act(uid, home, tool="add_task_log",
|
||||
args={"task_id": ids["theirs"]})
|
||||
assert theirs["allowed"] is False
|
||||
assert theirs["targets"] == [{"id": other, "title": "Elsewhere"}]
|
||||
assert "Home" in theirs["reason"] and "Elsewhere" in theirs["reason"]
|
||||
|
||||
issue = await write_scope.check_act(uid, home, tool="create_task",
|
||||
args={"project_id": other, "kind": "issue"})
|
||||
assert issue["allowed"] is True and issue["issue"] is True
|
||||
@@ -573,3 +573,36 @@ def test_rule_outcome_is_registered_and_is_not_read_only():
|
||||
assert "rule_outcome" in mcp.names
|
||||
assert "rule_outcome" in _WRITE_TOOLS
|
||||
assert "rule_outcome" not in _READ_ONLY_TOOLS
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_rules_carries_usage_only_when_asked():
|
||||
"""A listing is for applying rules, so it stays lean; `with_usage` adds
|
||||
each rule's telemetry and confirmed-lesson count for judging them, with
|
||||
`days` passed down as a window."""
|
||||
from grimoire.mcp.tools import rulebooks as tools
|
||||
from grimoire.services.rule_usage import empty_rule_usage
|
||||
|
||||
rows = [fake_rule(id=1, title="dev is home"), fake_rule(id=2, title="ask first")]
|
||||
seen = {**empty_rule_usage(), "surfaced_count": 4, "pull_count": 1}
|
||||
usage = AsyncMock(return_value={1: seen})
|
||||
lessons = AsyncMock(return_value={1: 3})
|
||||
with patch.object(tools.rulebooks_svc, "list_rules", AsyncMock(return_value=rows)), \
|
||||
patch.object(tools.rulebooks_svc, "list_rule_moments", AsyncMock(return_value={})), \
|
||||
patch.object(tools, "usage_for_rules", usage), \
|
||||
patch.object(tools, "confirmed_lesson_counts", lessons):
|
||||
lean = await tools.list_rules()
|
||||
assert all("usage" not in r and "confirmed_lessons" not in r for r in lean["rules"])
|
||||
usage.assert_not_awaited()
|
||||
|
||||
out = await tools.list_rules(with_usage=True, days=14)
|
||||
|
||||
by_id = {r["id"]: r for r in out["rules"]}
|
||||
assert by_id[1]["usage"]["surfaced_count"] == 4
|
||||
assert by_id[1]["confirmed_lessons"] == 3
|
||||
# A rule missing from either read is zero-filled, never absent.
|
||||
assert by_id[2]["usage"] == empty_rule_usage()
|
||||
assert by_id[2]["confirmed_lessons"] == 0
|
||||
assert usage.await_args.args[0] == [1, 2]
|
||||
assert usage.await_args.kwargs["since"] is not None
|
||||
lessons.assert_awaited_once_with(7, [1, 2])
|
||||
|
||||
@@ -139,7 +139,7 @@ async def test_a_named_record_comes_first_and_is_not_repeated_as_a_match():
|
||||
other = fake_note(id=11, title="something that matched the words", user_id=1)
|
||||
hits = [(0.80, named), (0.78, other)]
|
||||
out, surfaced, retrieval = await _hint(
|
||||
"go ahead with 4448", hits=hits, records={4448: named},
|
||||
"go ahead with 4448 and the filing fix", hits=hits, records={4448: named},
|
||||
)
|
||||
|
||||
ctx = out["context"]
|
||||
@@ -156,6 +156,23 @@ async def test_a_named_record_comes_first_and_is_not_repeated_as_a_match():
|
||||
assert row["suppressed"] == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_prompt_with_no_subject_still_shows_what_it_names():
|
||||
""""go ahead with 4448" names a record and nothing else: the record is
|
||||
looked up and shown, and no ranked menu is searched for."""
|
||||
rec = _task(4448, "the named one")
|
||||
other = fake_note(id=11, title="what the reply was about", user_id=1)
|
||||
out, surfaced, retrieval = await _hint(
|
||||
"go ahead with 4448", hits=[(0.8, other)], records={4448: rec},
|
||||
context="I'll start on the importer next.",
|
||||
)
|
||||
|
||||
assert out["note_ids"] == [4448]
|
||||
assert "Possibly relevant" not in out["context"]
|
||||
assert "auto_inject" not in surfaced
|
||||
retrieval.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_named_record_already_shown_is_a_pointer():
|
||||
rec = _task(4448, "the named one", "a long body that is already in context")
|
||||
|
||||
@@ -0,0 +1,214 @@
|
||||
"""The project fence, end to end: plugin/hooks/grimoire_project_fence.sh run
|
||||
through a real shell against real git repos and a stand-in server.
|
||||
|
||||
A session started in one repo edits, commits and pushes in another, and makes
|
||||
Grimoire writes. Each act aimed elsewhere is asked about in the server's
|
||||
words; acts in the session's own repo never reach the server.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from tests.helpers import http_sink, need_tools
|
||||
|
||||
HOOK = Path(__file__).resolve().parents[1] / "plugin" / "hooks" / "grimoire_project_fence.sh"
|
||||
PATH = "/api/plugin/write-scope"
|
||||
HOME_URL = "https://forge.example/team/home.git"
|
||||
OTHER_URL = "https://forge.example/team/other.git"
|
||||
ASK = json.dumps({"allowed": False, "targets": [{"id": 9, "title": "Other"}], "issue": False,
|
||||
"reason": "This session belongs to Home. Editing a file in Other, a different project."}).encode()
|
||||
OK = b'{"allowed": true, "targets": [], "issue": false, "reason": ""}'
|
||||
|
||||
|
||||
def _git(*args, cwd):
|
||||
subprocess.run(["git", *args], cwd=cwd, check=True, capture_output=True,
|
||||
env={**os.environ, "GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@example.com",
|
||||
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@example.com"})
|
||||
|
||||
|
||||
def _repo(tmp_path: Path, name: str, remote: str) -> Path:
|
||||
path = tmp_path / name
|
||||
path.mkdir()
|
||||
_git("init", "-q", cwd=path)
|
||||
_git("remote", "add", "origin", remote, cwd=path)
|
||||
return path
|
||||
|
||||
|
||||
def _run(tmp_path, port, event, project_dir):
|
||||
need_tools("bash", "curl", "awk", "git")
|
||||
env = {"PATH": os.environ["PATH"], "GRIMOIRE_URL": f"http://127.0.0.1:{port}",
|
||||
"GRIMOIRE_TOKEN": "t", "TMPDIR": str(tmp_path / "tmp"), "HOME": str(tmp_path),
|
||||
"CLAUDE_PROJECT_DIR": str(project_dir)}
|
||||
(tmp_path / "tmp").mkdir(exist_ok=True)
|
||||
event = {"session_id": "s-fence", "hook_event_name": "PreToolUse", **event}
|
||||
out = subprocess.run(["bash", str(HOOK)], input=json.dumps(event), capture_output=True,
|
||||
text=True, env=env, timeout=30)
|
||||
assert out.returncode == 0, out.stderr
|
||||
return out.stdout.strip()
|
||||
|
||||
|
||||
def _asked(out: str) -> str:
|
||||
decision = json.loads(out)["hookSpecificOutput"]
|
||||
assert decision["permissionDecision"] == "ask"
|
||||
return decision["permissionDecisionReason"]
|
||||
|
||||
|
||||
def _edit(path, cwd):
|
||||
return {"tool_name": "Edit", "cwd": str(cwd),
|
||||
"tool_input": {"file_path": str(path), "old_string": "a", "new_string": "b"}}
|
||||
|
||||
|
||||
def _bash(command, cwd):
|
||||
return {"tool_name": "Bash", "cwd": str(cwd), "tool_input": {"command": command}}
|
||||
|
||||
|
||||
def test_an_edit_in_the_sessions_own_repo_never_asks_the_server(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
out = _run(tmp_path, port, _edit(home / "src" / "a.py", home), home)
|
||||
assert out == "" and seen == []
|
||||
|
||||
|
||||
def test_an_edit_in_another_projects_repo_is_asked_about_in_the_servers_words(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
other = _repo(tmp_path, "other", OTHER_URL)
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
out = _run(tmp_path, port, _edit(other / "new_dir" / "b.py", home), home)
|
||||
assert "a different project" in _asked(out)
|
||||
q = seen[0]
|
||||
assert q["_path"] == PATH
|
||||
assert q["repo"] == [HOME_URL] and q["target_repo"] == [OTHER_URL]
|
||||
assert q["act"] == ["Editing a file in"]
|
||||
|
||||
|
||||
def test_an_edit_the_server_allows_goes_ahead_silently(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
other = _repo(tmp_path, "other", OTHER_URL)
|
||||
with http_sink(by_path={PATH: OK}) as (port, seen):
|
||||
out = _run(tmp_path, port, _edit(other / "b.py", home), home)
|
||||
assert out == "" and len(seen) == 1
|
||||
|
||||
|
||||
def test_an_edit_outside_every_repo_is_no_projects_business(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
loose = tmp_path / "loose"
|
||||
loose.mkdir()
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
out = _run(tmp_path, port, _edit(loose / "notes.txt", home), home)
|
||||
assert out == "" and seen == []
|
||||
|
||||
|
||||
def test_a_commit_through_git_dash_c_is_checked_against_that_repo(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
other = _repo(tmp_path, "other", OTHER_URL)
|
||||
command = f'git -C {other} commit -m "tidy && push later"'
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
out = _run(tmp_path, port, _bash(command, home), home)
|
||||
_asked(out)
|
||||
assert seen[0]["target_repo"] == [OTHER_URL]
|
||||
assert seen[0]["act"] == ["git commit in"]
|
||||
|
||||
|
||||
def test_a_push_after_cd_is_checked_against_the_repo_it_moved_to(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
_repo(tmp_path, "other", OTHER_URL)
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
out = _run(tmp_path, port, _bash("cd ../other && git push origin dev", home), home)
|
||||
_asked(out)
|
||||
assert seen[0]["target_repo"] == [OTHER_URL] and seen[0]["act"] == ["git push in"]
|
||||
|
||||
|
||||
def test_git_commands_that_change_nothing_and_writes_at_home_stay_local(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
other = _repo(tmp_path, "other", OTHER_URL)
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
assert _run(tmp_path, port, _bash(f"git -C {other} status && git -C {other} log -1", home), home) == ""
|
||||
assert _run(tmp_path, port, _bash("git commit -m x && git push", home), home) == ""
|
||||
assert _run(tmp_path, port, _bash("echo hello", home), home) == ""
|
||||
assert seen == []
|
||||
|
||||
|
||||
def test_a_worktree_of_another_repo_belongs_to_that_repo(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
other = _repo(tmp_path, "other", OTHER_URL)
|
||||
_git("commit", "-q", "--allow-empty", "-m", "start", cwd=other)
|
||||
tree = tmp_path / "other-tree"
|
||||
_git("worktree", "add", "-q", str(tree), cwd=other)
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
out = _run(tmp_path, port, _edit(tree / "c.py", home), home)
|
||||
_asked(out)
|
||||
assert seen[0]["target_repo"] == [OTHER_URL]
|
||||
|
||||
|
||||
def test_a_grimoire_write_sends_its_short_arguments_and_is_asked_about(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
event = {"tool_name": "mcp__plugin_grimoire_grimoire__update_task", "cwd": str(home),
|
||||
"tool_input": {"task_id": 5, "status": "done", "body": "x" * 500,
|
||||
"system_ids": [3, 4]}}
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
out = _run(tmp_path, port, event, home)
|
||||
_asked(out)
|
||||
q = seen[0]
|
||||
assert q["tool"] == ["update_task"]
|
||||
assert json.loads(q["input"][0]) == {"task_id": "5", "status": "done", "system_ids": ["3", "4"]}
|
||||
|
||||
|
||||
def test_a_grimoire_read_never_asks_the_server(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
event = {"tool_name": "mcp__plugin_grimoire_grimoire__get_task", "cwd": str(home),
|
||||
"tool_input": {"task_id": 5}}
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
assert _run(tmp_path, port, event, home) == ""
|
||||
assert seen == []
|
||||
|
||||
|
||||
def test_a_session_started_outside_any_project_takes_the_first_one_it_enters(tmp_path):
|
||||
loose = tmp_path / "loose"
|
||||
loose.mkdir()
|
||||
write = {"tool_name": "mcp__plugin_grimoire_grimoire__add_task_log", "cwd": str(loose),
|
||||
"tool_input": {"task_id": 5, "content": "c"}}
|
||||
with http_sink(by_path={PATH: ASK}) as (port, seen):
|
||||
# Nothing to hold the session to yet.
|
||||
assert _run(tmp_path, port, write, loose) == ""
|
||||
assert seen == []
|
||||
for pid in (7, 8):
|
||||
_run(tmp_path, port, {"hook_event_name": "PostToolUse", "cwd": str(loose),
|
||||
"tool_name": "mcp__plugin_grimoire_grimoire__enter_project",
|
||||
"tool_input": {"project_id": pid}}, loose)
|
||||
_asked(_run(tmp_path, port, write, loose))
|
||||
assert seen[0]["fallback_project_id"] == ["7"]
|
||||
assert "repo" not in seen[0]
|
||||
|
||||
|
||||
def test_an_unanswered_check_lets_the_act_through_and_says_so_once(tmp_path):
|
||||
home = _repo(tmp_path, "home", HOME_URL)
|
||||
other = _repo(tmp_path, "other", OTHER_URL)
|
||||
first = _run(tmp_path, 1, _edit(other / "b.py", home), home)
|
||||
ctx = json.loads(first)["hookSpecificOutput"]["additionalContext"]
|
||||
assert ctx.startswith("> Grimoire did not answer the project fence check")
|
||||
assert _run(tmp_path, 1, _edit(other / "b.py", home), home) == ""
|
||||
|
||||
|
||||
def test_the_entered_project_outlives_a_compaction_and_ends_with_the_session(tmp_path):
|
||||
# A compaction loses the context, not the session: the project it entered
|
||||
# stays its project. Session end clears it.
|
||||
loose = tmp_path / "loose"
|
||||
loose.mkdir()
|
||||
_run(tmp_path, 1, {"hook_event_name": "PostToolUse", "cwd": str(loose),
|
||||
"tool_name": "mcp__plugin_grimoire_grimoire__enter_project",
|
||||
"tool_input": {"project_id": 7}}, loose)
|
||||
entered = tmp_path / "tmp" / "grimoire-fence" / "s-fence.entered"
|
||||
assert entered.read_text() == "7"
|
||||
|
||||
env = {"PATH": os.environ["PATH"], "TMPDIR": str(tmp_path / "tmp"), "HOME": str(tmp_path)}
|
||||
hooks = HOOK.parent
|
||||
for hook, event in (("grimoire_session_context.sh", {"source": "compact"}),
|
||||
("grimoire_session_end.sh", {"reason": "other"})):
|
||||
subprocess.run(["bash", str(hooks / hook)], input=json.dumps({"session_id": "s-fence", **event}),
|
||||
capture_output=True, text=True, env=env, timeout=60, check=True)
|
||||
if hook == "grimoire_session_context.sh":
|
||||
assert entered.exists(), "a compaction cleared the project the session entered"
|
||||
assert not entered.exists()
|
||||
@@ -0,0 +1,186 @@
|
||||
"""The trusted proxy decides the client address and whether a request is HTTPS.
|
||||
|
||||
Only the entry TRUSTED_PROXY_HOPS in from the right of a forwarding header was
|
||||
written by a proxy the app trusts; a client can put anything to its left. The
|
||||
session cookie's Secure flag and HSTS follow the trusted X-Forwarded-Proto, and
|
||||
plain HTTP is left untouched.
|
||||
"""
|
||||
import pytest
|
||||
from quart import Quart, Response, session
|
||||
|
||||
from grimoire import config, proxy
|
||||
from grimoire.config import Config
|
||||
from grimoire.proxy import (
|
||||
HSTS_VALUE, PerRequestSecureSession, apply_https_headers, client_ip, is_https,
|
||||
trusted_entry,
|
||||
)
|
||||
|
||||
CLIENT = "203.0.113.7"
|
||||
FORGED = "198.51.100.1"
|
||||
PEER = "192.0.2.10"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("header, hops, expected", [
|
||||
("", 1, None),
|
||||
(CLIENT, 0, None),
|
||||
(CLIENT, 1, CLIENT),
|
||||
(f"{FORGED}, {CLIENT}", 1, CLIENT),
|
||||
(f"{FORGED}, {CLIENT}, {PEER}", 2, CLIENT),
|
||||
# Shorter than the hop count: the leftmost entry, written by the first proxy.
|
||||
(CLIENT, 2, CLIENT),
|
||||
(" , ", 1, None),
|
||||
])
|
||||
def test_trusted_entry_counts_in_from_the_right(header, hops, expected):
|
||||
assert trusted_entry(header, hops) == expected
|
||||
|
||||
|
||||
def _app() -> Quart:
|
||||
app = Quart(__name__)
|
||||
app.secret_key = "test-only-key"
|
||||
app.session_interface = PerRequestSecureSession()
|
||||
|
||||
@app.route("/")
|
||||
async def index():
|
||||
session["seen"] = True
|
||||
response = Response("ok")
|
||||
apply_https_headers(response)
|
||||
return response
|
||||
|
||||
return app
|
||||
|
||||
|
||||
def _context(app, headers=None, scheme="http"):
|
||||
# Quart's ASGI layer writes the socket peer into a Remote-Addr header,
|
||||
# replacing any the client sent; a test request context takes it as given.
|
||||
return app.test_request_context(
|
||||
"/", headers={"Remote-Addr": PEER, **(headers or {})}, scheme=scheme,
|
||||
)
|
||||
|
||||
|
||||
async def test_a_client_forged_address_left_of_the_proxys_is_ignored(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
async with _context(_app(), {"X-Forwarded-For": f"{FORGED}, {CLIENT}"}):
|
||||
assert client_ip() == CLIENT
|
||||
|
||||
|
||||
async def test_zero_hops_ignores_the_header_and_reads_the_socket_peer(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 0)
|
||||
async with _context(_app(), {"X-Forwarded-For": CLIENT}):
|
||||
assert client_ip() == PEER
|
||||
|
||||
|
||||
async def test_no_header_reads_the_socket_peer(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
async with _context(_app()):
|
||||
assert client_ip() == PEER
|
||||
|
||||
|
||||
async def test_a_header_sent_twice_is_read_as_one_chain(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
headers = [("Remote-Addr", PEER), ("X-Forwarded-For", FORGED), ("X-Forwarded-For", CLIENT)]
|
||||
async with _app().test_request_context("/", headers=headers):
|
||||
assert client_ip() == CLIENT
|
||||
|
||||
|
||||
async def test_a_short_proto_header_is_read_where_the_first_proxy_wrote_it(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 2)
|
||||
async with _context(_app(), {"X-Forwarded-Proto": "https"}):
|
||||
assert is_https()
|
||||
|
||||
|
||||
async def test_a_forged_https_left_of_the_proxys_http_is_ignored(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
async with _context(_app(), {"X-Forwarded-Proto": "https, http"}):
|
||||
assert not is_https()
|
||||
|
||||
|
||||
async def test_a_direct_https_request_is_https_at_any_hop_count(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 0)
|
||||
async with _context(_app(), scheme="https"):
|
||||
assert is_https()
|
||||
|
||||
|
||||
def test_outside_a_request_nothing_is_trusted():
|
||||
assert client_ip() == ""
|
||||
assert not is_https()
|
||||
|
||||
|
||||
async def _get(monkeypatch, hops, headers):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", hops)
|
||||
response = await _app().test_client().get("/", headers=headers)
|
||||
cookie = " ".join(response.headers.getlist("Set-Cookie"))
|
||||
assert "session=" in cookie
|
||||
return cookie, response.headers.get("Strict-Transport-Security")
|
||||
|
||||
|
||||
async def test_hops_zero_with_https_header_sends_no_secure_flag_and_no_hsts(monkeypatch):
|
||||
cookie, hsts = await _get(monkeypatch, 0, {"X-Forwarded-Proto": "https"})
|
||||
assert "Secure" not in cookie
|
||||
assert hsts is None
|
||||
|
||||
|
||||
async def test_hops_one_with_https_header_sends_secure_flag_and_hsts(monkeypatch):
|
||||
cookie, hsts = await _get(monkeypatch, 1, {"X-Forwarded-Proto": "https"})
|
||||
assert "Secure" in cookie
|
||||
assert hsts == HSTS_VALUE
|
||||
|
||||
|
||||
async def test_a_plain_request_sends_neither(monkeypatch):
|
||||
cookie, hsts = await _get(monkeypatch, 1, {})
|
||||
assert "Secure" not in cookie
|
||||
assert hsts is None
|
||||
|
||||
|
||||
def test_the_app_decides_secure_per_request():
|
||||
from grimoire.app import create_app
|
||||
assert isinstance(create_app().session_interface, PerRequestSecureSession)
|
||||
|
||||
|
||||
def test_the_app_reads_forwarding_headers_only_through_this_module():
|
||||
"""A second reader of X-Forwarded-* or the socket peer would be a second
|
||||
trust rule. Matches the reads themselves, not prose that names a header."""
|
||||
import pathlib
|
||||
import re
|
||||
read = re.compile(
|
||||
r"""headers\.(?:get|getlist|__getitem__)\(\s*["']X-(?:Forwarded|Real)"""
|
||||
r"""|headers\[\s*["']X-(?:Forwarded|Real)"""
|
||||
r"""|\.remote_addr\b""",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
src = pathlib.Path(proxy.__file__).parent
|
||||
readers = sorted(
|
||||
str(path.relative_to(src)) for path in src.rglob("*.py")
|
||||
if path.name != "proxy.py" and read.search(path.read_text())
|
||||
)
|
||||
assert readers == []
|
||||
assert read.search((src / "proxy.py").read_text()), "the pattern no longer finds the real reader"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("env, expected", [
|
||||
({}, 1),
|
||||
({"TRUSTED_PROXY_HOPS": "2"}, 2),
|
||||
({"TRUSTED_PROXY_HOPS": "0"}, 0),
|
||||
({"TRUSTED_PROXY_HOPS": "two"}, -1),
|
||||
({"TRUST_PROXY_HEADERS": "false"}, 0),
|
||||
({"TRUST_PROXY_HEADERS": "true"}, 1),
|
||||
({"TRUST_PROXY_HEADERS": "false", "TRUSTED_PROXY_HOPS": "1"}, 1),
|
||||
])
|
||||
def test_the_hop_count_from_the_environment(monkeypatch, env, expected):
|
||||
monkeypatch.delenv("TRUSTED_PROXY_HOPS", raising=False)
|
||||
monkeypatch.delenv("TRUST_PROXY_HEADERS", raising=False)
|
||||
for name, value in env.items():
|
||||
monkeypatch.setenv(name, value)
|
||||
assert config._trusted_proxy_hops() == expected
|
||||
|
||||
|
||||
def test_a_hop_count_that_is_not_a_number_refuses_to_start(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", -1)
|
||||
with pytest.raises(ValueError, match="TRUSTED_PROXY_HOPS"):
|
||||
Config.validate()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("key", sorted(config.PLACEHOLDER_SECRET_KEYS))
|
||||
def test_a_placeholder_secret_key_refuses_to_start(monkeypatch, key):
|
||||
monkeypatch.setattr(Config, "SECRET_KEY", key)
|
||||
with pytest.raises(ValueError, match="public placeholder"):
|
||||
Config.validate()
|
||||
@@ -0,0 +1,232 @@
|
||||
"""The reverse-proxy hop count an admin sets in Settings → Config.
|
||||
|
||||
Behind Cloudflare's proxied DNS the right count is 2. At the default of 1,
|
||||
every visitor reads as a Cloudflare edge address and shares one sign-in rate
|
||||
limit, so a few bad sign-ins block everyone. The count is therefore saved where
|
||||
an admin can find it, beside a readout of what each count makes of their own
|
||||
request. Covered here:
|
||||
- the saved count wins over the environment's;
|
||||
- the readout picks out the Cloudflare road;
|
||||
- the admin route validates what it saves;
|
||||
- a "too many attempts" reply says where to look.
|
||||
"""
|
||||
import asyncio
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
from quart import Quart, g
|
||||
|
||||
from grimoire import proxy
|
||||
from grimoire.config import Config
|
||||
from grimoire.proxy import client_ip, readout, request_readout, trusted_hops
|
||||
from grimoire.rate_limit import SHARED_ADDRESS_HINT
|
||||
from grimoire.services import proxy_settings
|
||||
|
||||
VISITOR = "203.0.113.7"
|
||||
CLOUDFLARE_EDGE = "172.70.1.1"
|
||||
LAN_VISITOR = "192.168.1.20"
|
||||
TRAEFIK = "10.0.0.5"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _no_saved_count():
|
||||
proxy.set_saved_hops(None)
|
||||
yield
|
||||
proxy.set_saved_hops(None)
|
||||
|
||||
|
||||
def _context(headers: dict):
|
||||
app = Quart(__name__)
|
||||
return app.test_request_context("/", headers={"Remote-Addr": TRAEFIK, **headers})
|
||||
|
||||
|
||||
@pytest.mark.parametrize("raw, expected", [
|
||||
(0, 0), (2, 2), (5, 5), ("3", 3), (" 1 ", 1),
|
||||
(6, None), (-1, None), ("x", None), ("", None), (True, None), (1.5, None), (None, None),
|
||||
])
|
||||
def test_a_hop_count_is_a_whole_number_from_zero_to_five(raw, expected):
|
||||
assert proxy_settings.parse_hops(raw) == expected
|
||||
|
||||
|
||||
def test_the_saved_count_wins_over_the_environments(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
assert trusted_hops() == 1
|
||||
proxy.set_saved_hops(2)
|
||||
assert trusted_hops() == 2
|
||||
proxy.set_saved_hops(0)
|
||||
assert trusted_hops() == 0, "a saved 0 is a count, not an absence"
|
||||
|
||||
|
||||
async def test_the_client_address_follows_a_saved_count(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
async with _context({"X-Forwarded-For": f"{VISITOR}, {CLOUDFLARE_EDGE}"}):
|
||||
assert client_ip() == CLOUDFLARE_EDGE
|
||||
proxy.set_saved_hops(2)
|
||||
assert client_ip() == VISITOR
|
||||
|
||||
|
||||
@pytest.mark.parametrize("saved, from_env, expected", [
|
||||
(2, False, "setting"), (2, True, "setting"), (None, True, "env"), (None, False, "default"),
|
||||
])
|
||||
def test_the_source_names_where_the_count_came_from(monkeypatch, saved, from_env, expected):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS_FROM_ENV", from_env)
|
||||
proxy.set_saved_hops(saved)
|
||||
assert proxy_settings.source() == expected
|
||||
|
||||
|
||||
def test_the_readout_suggests_the_count_that_reads_cloudflares_visitor():
|
||||
out = readout(TRAEFIK, f"{VISITOR}, {CLOUDFLARE_EDGE}", "https, https", VISITOR)
|
||||
by_hops = {row["hops"]: row for row in out["rows"]}
|
||||
assert by_hops[0]["client"] == TRAEFIK
|
||||
assert by_hops[1]["client"] == CLOUDFLARE_EDGE
|
||||
assert by_hops[2]["client"] == VISITOR and by_hops[2]["https"] is True
|
||||
assert out["cloudflare"] is True and out["suggested"] == 2
|
||||
assert out["forwarded_for"] == [VISITOR, CLOUDFLARE_EDGE]
|
||||
|
||||
|
||||
def test_the_readout_suggests_nothing_off_cloudflare():
|
||||
# Only the admin knows which address is theirs on a road with no
|
||||
# Cloudflare header to compare against.
|
||||
out = readout(TRAEFIK, LAN_VISITOR, "http", None)
|
||||
assert out["cloudflare"] is False and out["suggested"] is None
|
||||
assert [row["client"] for row in out["rows"]] == [TRAEFIK, LAN_VISITOR, LAN_VISITOR, LAN_VISITOR]
|
||||
assert not any(row["https"] for row in out["rows"])
|
||||
|
||||
|
||||
def test_the_readout_rows_can_disagree():
|
||||
# The guard above passes vacuously if every row reads the same address.
|
||||
out = readout(TRAEFIK, f"{VISITOR}, {CLOUDFLARE_EDGE}", "", VISITOR)
|
||||
assert len({row["client"] for row in out["rows"]}) == 3
|
||||
|
||||
|
||||
async def test_the_request_readout_reads_the_cloudflare_headers():
|
||||
headers = {"X-Forwarded-For": f"{VISITOR}, {CLOUDFLARE_EDGE}", "CF-Connecting-IP": VISITOR}
|
||||
async with _context(headers):
|
||||
out = request_readout()
|
||||
assert out["peer"] == TRAEFIK and out["cloudflare_client"] == VISITOR and out["suggested"] == 2
|
||||
|
||||
async with _context({"CF-Ray": "8a1b2c3d4e5f-LHR"}):
|
||||
out = request_readout()
|
||||
assert out["cloudflare"] is True and out["suggested"] is None
|
||||
|
||||
|
||||
async def test_a_failed_refresh_keeps_the_count_in_force():
|
||||
proxy.set_saved_hops(2)
|
||||
with patch.object(proxy_settings, "saved_hops", AsyncMock(side_effect=OSError("db down"))):
|
||||
await proxy_settings.refresh()
|
||||
assert trusted_hops() == 2
|
||||
|
||||
|
||||
async def test_a_hanging_startup_read_does_not_hold_up_the_boot(monkeypatch):
|
||||
async def _never():
|
||||
await asyncio.Event().wait()
|
||||
|
||||
monkeypatch.setattr(proxy_settings, "STARTUP_READ_TIMEOUT", 0.05)
|
||||
with patch.object(proxy_settings, "refresh", _never), \
|
||||
patch.object(proxy_settings.logger, "warning") as warn:
|
||||
await asyncio.wait_for(proxy_settings.load_at_startup(), timeout=2)
|
||||
assert warn.called
|
||||
|
||||
|
||||
def test_the_startup_read_has_a_deadline():
|
||||
assert 0 < proxy_settings.STARTUP_READ_TIMEOUT < 60
|
||||
|
||||
|
||||
# ── the admin route ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def _call(method: str, body=None):
|
||||
from grimoire.app import create_app
|
||||
from grimoire.routes import admin
|
||||
|
||||
app = create_app()
|
||||
view = admin.get_proxy_setting if method == "GET" else admin.update_proxy_setting
|
||||
async with app.test_request_context("/api/admin/proxy", method=method, json=body,
|
||||
headers={"Remote-Addr": TRAEFIK}):
|
||||
g.user = SimpleNamespace(id=1, username="admin", role="admin")
|
||||
resp = await view.__wrapped__()
|
||||
if isinstance(resp, tuple):
|
||||
return resp[1], await resp[0].get_json()
|
||||
return resp.status_code, await resp.get_json()
|
||||
|
||||
|
||||
async def _fake_save(hops):
|
||||
proxy.set_saved_hops(hops)
|
||||
|
||||
|
||||
async def test_the_route_reports_the_count_its_source_and_the_readout(monkeypatch):
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
proxy.set_saved_hops(2)
|
||||
status, body = await _call("GET")
|
||||
assert status == 200
|
||||
assert body["hops"] == 2 and body["source"] == "setting" and body["env_hops"] == 1
|
||||
assert body["max_hops"] == proxy.MAX_HOPS
|
||||
assert body["readout"]["peer"] == TRAEFIK and len(body["readout"]["rows"]) == 4
|
||||
|
||||
|
||||
async def test_saving_a_count_applies_it_and_audits_the_change(monkeypatch):
|
||||
from grimoire.routes import admin
|
||||
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
save = AsyncMock(side_effect=_fake_save)
|
||||
audit = AsyncMock()
|
||||
with patch.object(admin.proxy_settings, "save_hops", save), patch.object(admin, "log_audit", audit):
|
||||
status, body = await _call("PUT", {"hops": 2})
|
||||
assert status == 200 and body["hops"] == 2
|
||||
save.assert_awaited_once_with(2)
|
||||
details = audit.call_args.kwargs["details"]
|
||||
assert details == {"before": 1, "after": 2, "saved": 2}
|
||||
|
||||
|
||||
async def test_saving_null_goes_back_to_the_environments_count(monkeypatch):
|
||||
from grimoire.routes import admin
|
||||
|
||||
monkeypatch.setattr(Config, "TRUSTED_PROXY_HOPS", 1)
|
||||
proxy.set_saved_hops(3)
|
||||
save = AsyncMock(side_effect=_fake_save)
|
||||
with patch.object(admin.proxy_settings, "save_hops", save), patch.object(admin, "log_audit", AsyncMock()):
|
||||
status, body = await _call("PUT", {"hops": None})
|
||||
assert status == 200 and body["hops"] == 1
|
||||
save.assert_awaited_once_with(None)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad", [6, -1, "two", True, 1.5])
|
||||
async def test_a_count_out_of_range_is_refused_and_nothing_is_saved(bad):
|
||||
from grimoire.routes import admin
|
||||
|
||||
save = AsyncMock()
|
||||
with patch.object(admin.proxy_settings, "save_hops", save):
|
||||
status, body = await _call("PUT", {"hops": bad})
|
||||
assert status == 400 and "0 to 5" in body["error"]
|
||||
save.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("method", ["GET", "PUT"])
|
||||
async def test_the_route_is_for_admins_only(method):
|
||||
from grimoire.app import create_app
|
||||
from grimoire.routes import admin
|
||||
|
||||
view = admin.get_proxy_setting if method == "GET" else admin.update_proxy_setting
|
||||
app = create_app()
|
||||
async with app.test_request_context("/api/admin/proxy", method=method, json={"hops": 2}):
|
||||
resp = await view()
|
||||
assert resp[1] == 401
|
||||
|
||||
|
||||
# ── where a lockout shows up ─────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def test_a_rate_limited_sign_in_says_where_an_admin_should_look(monkeypatch):
|
||||
from grimoire.app import create_app
|
||||
from grimoire.routes import auth
|
||||
|
||||
monkeypatch.setattr(Config, "LOCAL_AUTH_ENABLED", True)
|
||||
app = create_app()
|
||||
with patch.object(auth, "is_rate_limited", AsyncMock(return_value=True)):
|
||||
async with app.test_request_context("/api/auth/login", method="POST", json={}):
|
||||
resp, status = await auth.login()
|
||||
body = await resp.get_json()
|
||||
assert status == 429
|
||||
assert SHARED_ADDRESS_HINT in body["error"]
|
||||
assert "Settings → Config" in SHARED_ADDRESS_HINT
|
||||
@@ -81,7 +81,7 @@ async def test_the_re_run_searches_the_way_the_arm_does():
|
||||
rerun_kw.update(kw)
|
||||
return []
|
||||
|
||||
row = SimpleNamespace(query="q", threshold=0.5, limit_n=3, project_id=None,
|
||||
row = SimpleNamespace(source="auto_inject", query="q", threshold=0.5, limit_n=3, project_id=None,
|
||||
created_at=CALL)
|
||||
with patch("grimoire.services.embeddings.semantic_search_notes", side_effect=rerun_search):
|
||||
await review._rerun(1, row, 3)
|
||||
@@ -99,7 +99,7 @@ BEFORE = CALL - timedelta(days=1)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_re_run_ranks_from_one_and_marks_the_budget_cut():
|
||||
row = SimpleNamespace(query="q", threshold=0.55, limit_n=2, project_id=None,
|
||||
row = SimpleNamespace(source="auto_inject", query="q", threshold=0.55, limit_n=2, project_id=None,
|
||||
created_at=CALL)
|
||||
hits = [
|
||||
(0.81, fake_note(id=11, title="first", created_at=BEFORE, updated_at=BEFORE)),
|
||||
@@ -127,7 +127,7 @@ async def test_a_record_written_after_the_call_is_dropped_before_ranking():
|
||||
"""The session that made a call goes on to write about it, and that record
|
||||
outranks everything in a re-run. The call could never have been offered
|
||||
it, so it must not take a rank — or a verdict — from what was."""
|
||||
row = SimpleNamespace(query="q", threshold=0.55, limit_n=1, project_id=None,
|
||||
row = SimpleNamespace(source="auto_inject", query="q", threshold=0.55, limit_n=1, project_id=None,
|
||||
created_at=CALL)
|
||||
after = CALL + timedelta(hours=1)
|
||||
hits = [
|
||||
@@ -147,6 +147,103 @@ async def test_a_record_written_after_the_call_is_dropped_before_ranking():
|
||||
for ln in lines] == [(1, 22, True, False), (2, 23, False, True)]
|
||||
|
||||
|
||||
# ── the rule arms ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _rule(rid, title, created=BEFORE, updated=BEFORE):
|
||||
return SimpleNamespace(
|
||||
id=rid, title=title, kind="rule", when_to_apply=f"when {title}",
|
||||
statement="s", created_at=created, updated_at=updated,
|
||||
)
|
||||
|
||||
|
||||
def test_the_reviewable_rule_arms_are_the_ones_that_annotate():
|
||||
"""Every rule arm that shows lines can be reviewed; the reply backstop
|
||||
shows none, so there is no menu to judge."""
|
||||
from grimoire.services import retrieval_pipeline as rp
|
||||
|
||||
assert set(review.RULE_REVIEWABLE) == {
|
||||
a.source for a in rp.RULE_ARMS if not a.stop_only
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_rule_re_run_searches_the_way_the_arm_does():
|
||||
"""The floor and the project the arm searched with, run and compared."""
|
||||
from grimoire.services import retrieval_pipeline as rp
|
||||
|
||||
arm_kw: dict = {}
|
||||
|
||||
async def arm_search(_uid, _q, **kw):
|
||||
if "kind" not in kw:
|
||||
arm_kw.update(kw)
|
||||
return []
|
||||
|
||||
await rp.run_rule_arm(
|
||||
rp.PROMPT_RULE, rp.RuleMoment(user_id=1, query="q", project_id=7),
|
||||
floor=0.7, budget=3,
|
||||
io=rp.RuleIO(search=arm_search, record_retrieval=MagicMock(),
|
||||
record_rule_surfaced=MagicMock()),
|
||||
)
|
||||
rerun_kw: dict = {}
|
||||
|
||||
async def rerun_search(_uid, _q, **kw):
|
||||
rerun_kw.update(kw)
|
||||
return []
|
||||
|
||||
row = SimpleNamespace(source="prompt_rule", query="q", threshold=0.7, limit_n=3,
|
||||
project_id=7, created_at=CALL)
|
||||
with patch("grimoire.services.embeddings.semantic_search_rules", side_effect=rerun_search):
|
||||
await review._rerun(1, row, 3)
|
||||
|
||||
for key in ("threshold", "project_id"):
|
||||
assert rerun_kw[key] == arm_kw[key], key
|
||||
assert "kind" not in rerun_kw
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(("source", "within"), [
|
||||
# No band: the budget alone is the cut.
|
||||
("prompt_rule", [True, True, True, False]),
|
||||
# Banded: the third sits outside the band of the top hit, so it was
|
||||
# not shown though the budget had room for it.
|
||||
("pre_tool_rule", [True, True, False, False]),
|
||||
])
|
||||
async def test_a_rule_line_is_inside_the_cut_only_where_the_arm_showed_it(source, within):
|
||||
row = SimpleNamespace(source=source, query="git push", threshold=0.7, limit_n=3,
|
||||
project_id=None, created_at=CALL)
|
||||
hits = [(0.86, _rule(1, "one")), (0.84, _rule(2, "two")),
|
||||
(0.76, _rule(3, "three")), (0.74, _rule(4, "four"))]
|
||||
|
||||
async def search(_uid, _q, **kw):
|
||||
assert kw["threshold"] == 0.7 and kw["project_id"] is None
|
||||
return hits
|
||||
|
||||
with patch("grimoire.services.embeddings.semantic_search_rules", side_effect=search):
|
||||
lines = await review._rerun(1, row, 4)
|
||||
assert [ln["within_budget"] for ln in lines] == within
|
||||
assert lines[0]["passage"] == "when one"
|
||||
assert lines[0]["kind"] == "rule"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_rule_written_after_the_call_is_dropped_before_ranking():
|
||||
row = SimpleNamespace(source="write_path_rule", query="q", threshold=0.7, limit_n=2,
|
||||
project_id=None, created_at=CALL)
|
||||
after = CALL + timedelta(hours=1)
|
||||
hits = [(0.9, _rule(31, "written after", created=after, updated=after)),
|
||||
(0.8, _rule(32, "held then"))]
|
||||
|
||||
async def search(_uid, _q, **kw):
|
||||
return hits
|
||||
|
||||
rep: dict = {}
|
||||
with patch("grimoire.services.embeddings.semantic_search_rules", side_effect=search):
|
||||
lines = await review._rerun(1, row, 2, report=rep)
|
||||
assert rep["postdated"] == [31]
|
||||
assert [(ln["rank"], ln["record_id"]) for ln in lines] == [(1, 32)]
|
||||
|
||||
|
||||
# ── against Postgres ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -127,7 +127,7 @@ async def test_injected_menu_attributes_another_users_note():
|
||||
patch.object(plugin_context, "record_retrieval", MagicMock()), \
|
||||
patch.object(plugin_context, "owner_names_for",
|
||||
AsyncMock(return_value={9: "alex"})):
|
||||
out = await plugin_context.build_autoinject_hint(7, "anything")
|
||||
out = await plugin_context.build_autoinject_hint(7, "debounce helper")
|
||||
|
||||
lines = out["context"].splitlines()
|
||||
theirs_line = next(ln for ln in lines if "#2" in ln)
|
||||
@@ -160,7 +160,7 @@ async def test_injected_menu_labels_the_record_kind():
|
||||
AsyncMock(return_value={"enabled": True, "threshold": 0.5,
|
||||
"top_k": 5})), \
|
||||
patch.object(plugin_context, "record_retrieval", MagicMock()):
|
||||
out = await plugin_context.build_autoinject_hint(7, "anything")
|
||||
out = await plugin_context.build_autoinject_hint(7, "debounce helper")
|
||||
|
||||
lines = out["context"].splitlines()
|
||||
by_id = {n: next(ln for ln in lines if f"#{n} " in ln) for n in (1, 2, 3, 4, 5)}
|
||||
|
||||
@@ -71,13 +71,29 @@ def test_every_surface_name_is_a_real_telemetry_source():
|
||||
)
|
||||
|
||||
|
||||
def test_no_two_surfaces_share_a_settings_key():
|
||||
"""One dial, one label."""
|
||||
floors = [s.floor_key for s in rs.SURFACES.values()]
|
||||
def test_a_shared_settings_key_is_one_bar_with_one_default():
|
||||
"""One dial, one label. Surfaces may share a floor key, as the rule arms
|
||||
do, but then they must ship the same default: otherwise the one number
|
||||
means something different on each surface until somebody sets it."""
|
||||
by_key: dict[str, set[float]] = {}
|
||||
for s in rs.SURFACES.values():
|
||||
by_key.setdefault(s.floor_key, set()).add(s.floor_default)
|
||||
split = {k: v for k, v in by_key.items() if len(v) > 1}
|
||||
assert not split, f"a shared floor key ships two defaults: {split}"
|
||||
|
||||
budgets = [s.budget_key for s in rs.SURFACES.values()]
|
||||
assert len(set(floors)) == len(floors), f"duplicate floor key in {floors}"
|
||||
assert len(set(budgets)) == len(budgets), f"duplicate budget key in {budgets}"
|
||||
assert not (set(floors) & set(budgets)), "a floor key doubles as a budget key"
|
||||
assert not (set(by_key) & set(budgets)), "a floor key doubles as a budget key"
|
||||
|
||||
|
||||
def test_the_rule_arms_that_annotate_share_one_bar_and_the_reply_stop_does_not():
|
||||
from grimoire.services.retrieval_pipeline import RULE_FLOOR_KEY
|
||||
|
||||
assert sorted(rs.sharing_floor("prompt_rule")) == [
|
||||
"pre_tool_rule", "prompt_rule", "write_path_rule",
|
||||
]
|
||||
assert rs.SURFACES["prompt_rule"].floor_key == RULE_FLOOR_KEY
|
||||
assert rs.sharing_floor("reply_rule") == ["reply_rule"]
|
||||
|
||||
|
||||
def test_every_surface_says_what_it_asks_over_what_and_how_often():
|
||||
|
||||
@@ -86,9 +86,9 @@ async def test_a_good_change_writes_the_setting_and_the_event():
|
||||
|
||||
setter.assert_awaited_once()
|
||||
_uid, key, stored = setter.await_args.args
|
||||
assert key == "kb_promptrule_threshold" and stored == "0.66"
|
||||
assert key == "kb_rule_threshold" and stored == "0.66"
|
||||
|
||||
event = session.add.call_args.args[0]
|
||||
event = session.add.call_args_list[0].args[0]
|
||||
assert event.surface == "prompt_rule" and event.dial == "floor"
|
||||
# The before-value is what makes the history answerable.
|
||||
assert event.old_value == 0.72 and event.new_value == 0.66
|
||||
@@ -97,6 +97,37 @@ async def test_a_good_change_writes_the_setting_and_the_event():
|
||||
assert out["clamped"] is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_shared_floor_records_the_move_on_every_surface_that_reads_it():
|
||||
"""One key, one write, an event per surface. A rule arm whose bar moved
|
||||
with no event of its own would show a telemetry shift nothing explains."""
|
||||
session, ctx = _patches(floor=0.70)
|
||||
with ctx[0], patch.object(rt, "set_setting", AsyncMock()) as setter, \
|
||||
ctx[2], ctx[3]:
|
||||
out = await rt.set_dial(
|
||||
1, "pre_tool_rule", "floor", 0.74,
|
||||
reason="read the 10 highest declines: every one was a real miss",
|
||||
)
|
||||
|
||||
setter.assert_awaited_once()
|
||||
recorded = {c.args[0].surface for c in session.add.call_args_list}
|
||||
assert recorded == {"pre_tool_rule", "prompt_rule", "write_path_rule"}
|
||||
assert sorted(out["moved_with"]) == ["prompt_rule", "write_path_rule"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_budget_moves_only_its_own_surface():
|
||||
session, ctx = _patches(budget=3)
|
||||
with ctx[0], patch.object(rt, "set_setting", AsyncMock()), ctx[2], ctx[3]:
|
||||
out = await rt.set_dial(
|
||||
1, "prompt_rule", "budget", 4,
|
||||
reason="on-point hits sit at ranks 4 and 5 as often as rank 1",
|
||||
)
|
||||
|
||||
assert [c.args[0].surface for c in session.add.call_args_list] == ["prompt_rule"]
|
||||
assert out["moved_with"] == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("dial, sent, applied", [
|
||||
("floor", 1.4, 1.0),
|
||||
|
||||
@@ -279,6 +279,19 @@ def test_a_point_seen_only_in_usage_counts_as_having_emitted() -> None:
|
||||
assert "enter_project" not in quiet
|
||||
|
||||
|
||||
def test_a_getter_seen_only_by_its_pulls_counts_as_having_emitted() -> None:
|
||||
"""A getter surfaces nothing, so no `by_source` ever names it; its pulls
|
||||
are the only sign it records. Without them every pull door reads silent
|
||||
on every active window."""
|
||||
quiet = {p["source"] for p in _silent_surfaces(
|
||||
{}, {"pulled_by_source": {"mcp_get_note": 3}},
|
||||
{"pulled_by_source": {"mcp_get_rule": 2}}, active=True)}
|
||||
assert "mcp_get_note" not in quiet
|
||||
assert "mcp_get_rule" not in quiet
|
||||
# A door with no pulls is still reported.
|
||||
assert "mcp_get_snippet" in quiet
|
||||
|
||||
|
||||
# ── settings parsing ──────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.parametrize("raw,fallback,want", [
|
||||
|
||||
@@ -52,3 +52,9 @@ def test_service_surface_and_allowlist():
|
||||
def test_set_admin_setting_exists():
|
||||
from grimoire.services.settings import set_admin_setting
|
||||
assert callable(set_admin_setting)
|
||||
|
||||
|
||||
def test_restore_routes_registered():
|
||||
from grimoire.app import create_app
|
||||
rules = {r.rule for r in create_app().url_map.iter_rules()}
|
||||
assert {"/api/admin/restore", "/api/admin/restore/preview", "/api/admin/restore/sources"} <= rules
|
||||
|
||||
@@ -76,10 +76,15 @@ def test_the_route_does_not_take_the_actor_from_the_caller():
|
||||
def test_every_registry_key_is_recognised_as_a_dial():
|
||||
"""Derived from the registry, both directions. A seventh surface added
|
||||
without a row here is a floor that can be written silently again."""
|
||||
from grimoire.services.retrieval_surfaces import SURFACES, dial_for_key
|
||||
from grimoire.services.retrieval_surfaces import (
|
||||
SURFACES, dial_for_key, sharing_floor,
|
||||
)
|
||||
|
||||
for name, surface in SURFACES.items():
|
||||
assert dial_for_key(surface.floor_key) == (name, "floor")
|
||||
# A shared floor resolves to one of the surfaces that read it, and
|
||||
# `set_dial` records the move on all of them.
|
||||
owner, dial = dial_for_key(surface.floor_key)
|
||||
assert dial == "floor" and name in sharing_floor(owner)
|
||||
assert dial_for_key(surface.budget_key) == (name, "budget")
|
||||
|
||||
|
||||
|
||||
@@ -61,6 +61,29 @@ def test_the_proposal_line_names_the_tool_that_answers_it():
|
||||
assert "judge_rule_moments" in line and '"reject"' in line
|
||||
|
||||
|
||||
def test_the_proposal_line_is_a_question_to_check_not_advice_to_relay():
|
||||
rule = SimpleNamespace(id=11, title="Definition of done", kind="rule")
|
||||
evidence = {"situations": ["a", "b", "c"],
|
||||
"reached_by": {"git push": 3, "git push --tags": 1}}
|
||||
line = svc._proposal_line(rule, "work.deliver", evidence)
|
||||
assert "co-occurrence, not a recommendation" in line
|
||||
assert "`list_moments`" in line and "if it is about a subject, leave it" in line
|
||||
assert "(reached by `git push`, `git push --tags`)" in line
|
||||
note = svc._signal_note("work.deliver", evidence)
|
||||
assert note.startswith("Opened just after `work.deliver` (reached by `git push`")
|
||||
assert "3 distinct sessions" in note
|
||||
|
||||
|
||||
def test_the_actions_a_signal_remembers_are_bounded():
|
||||
ev: dict = {}
|
||||
for i in range(svc._SIGNAL_ACTIONS_CAP + 3):
|
||||
ev = svc._add_reached(ev, f"cmd {i}")
|
||||
ev = svc._add_reached(ev, "cmd 0")
|
||||
assert len(ev["reached_by"]) == svc._SIGNAL_ACTIONS_CAP
|
||||
assert ev["reached_by"]["cmd 0"] == 2
|
||||
assert svc._add_reached({}, "")["reached_by"] == {}
|
||||
|
||||
|
||||
# ── The misfire ──────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -133,6 +133,49 @@ def test_an_open_sends_the_window_and_prints_the_answer(sandbox):
|
||||
assert (tmp / "grimoire-priorart" / "s1.opened.ids").read_text().startswith("11\t")
|
||||
|
||||
|
||||
def test_the_window_is_the_last_three_acts(sandbox):
|
||||
tmp, env = sandbox
|
||||
ledger = tmp / "grimoire-moment"
|
||||
ledger.mkdir()
|
||||
now = int(time.time())
|
||||
acts = [{"tool_name": "Bash", "tool_input": {"command": f"step {i}"}} for i in range(5)]
|
||||
(ledger / "s1.acts").write_text(
|
||||
"".join(f"{now - 10 + i}\t{json.dumps(a)}\n" for i, a in enumerate(acts))
|
||||
)
|
||||
_run("grimoire_record_opened.sh",
|
||||
{"session_id": "s1", "tool_name": "mcp__grimoire__get_rule",
|
||||
"tool_input": {"rule_id": 11}}, env)
|
||||
body = json.loads((tmp / "curl.body").read_text())
|
||||
assert body["acts"] == acts[-3:]
|
||||
|
||||
|
||||
def test_an_open_of_a_rule_a_hook_just_named_is_not_reported(sandbox):
|
||||
"""A ranked arm or a moment line explains that open; the moment beside it
|
||||
does not, so it is no evidence for a mount."""
|
||||
tmp, env = sandbox
|
||||
ledger = tmp / "grimoire-moment"
|
||||
ledger.mkdir()
|
||||
now = int(time.time())
|
||||
(ledger / "s1.acts").write_text(
|
||||
f"{now - 5}\t{json.dumps({'tool_name': 'Bash', 'tool_input': {'command': 'git push'}})}\n"
|
||||
)
|
||||
named = tmp / "grimoire-priorart"
|
||||
named.mkdir()
|
||||
(named / "s1.rules.ids").write_text(f"7\t{now - 30}\n11\t{now - 20}\n")
|
||||
out = _run("grimoire_record_opened.sh",
|
||||
{"session_id": "s1", "tool_name": "mcp__grimoire__get_rule",
|
||||
"tool_input": {"rule_id": 11}}, env)
|
||||
assert out.stdout == ""
|
||||
assert not (tmp / "curl.log").exists()
|
||||
# The open itself is still recorded.
|
||||
assert (named / "s1.opened.ids").read_text().startswith("11\t")
|
||||
# A rule no hook named is still reported.
|
||||
_run("grimoire_record_opened.sh",
|
||||
{"session_id": "s1", "tool_name": "mcp__grimoire__get_rule",
|
||||
"tool_input": {"rule_id": 1}}, env)
|
||||
assert json.loads((tmp / "curl.body").read_text())["rule_id"] == 1
|
||||
|
||||
|
||||
def test_an_open_with_nothing_in_the_window_sends_nothing(sandbox):
|
||||
tmp, env = sandbox
|
||||
ledger = tmp / "grimoire-moment"
|
||||
|
||||
@@ -33,16 +33,8 @@ pytestmark = pytest.mark.usefixtures("_bind_user")
|
||||
# each of these does.
|
||||
|
||||
|
||||
# The write-path hint returns early when a write matched nothing at all — no
|
||||
# staleness, no synced record, no prior-art menu, no shape signal. The rule arm
|
||||
# sits deliberately on the FAR side of that guard, because it runs a semantic
|
||||
# search and moving it above would mean an embedding query on every write in
|
||||
# the session (which is why its gating is a separate question from
|
||||
# precision).
|
||||
#
|
||||
# So a fixture that stubs every other arm to empty never reaches the rule arm
|
||||
# at all — which is what the first run of this file did. The note hit below is
|
||||
# not decoration: it is the condition the arm requires in order to fire.
|
||||
# A prior-art hit for the fixtures that want the menu beside the rule lines.
|
||||
# The rule arm runs on every write with a path, whether or not this matched.
|
||||
_PRIOR_ART = [(0.72, fake_note(id=9, title="debounce helper", user_id=1,
|
||||
note_type="snippet"))]
|
||||
|
||||
@@ -161,7 +153,8 @@ async def test_the_arm_searches_on_its_OWN_bar_not_the_code_one():
|
||||
1, "frontend/src/api/client.ts", code="x" * 400,
|
||||
)
|
||||
|
||||
kw = search.await_args.kwargs
|
||||
# The first search is the arm's own; the preference slot asks after it.
|
||||
kw = search.await_args_list[0].kwargs
|
||||
assert kw["threshold"] == 0.81, "the arm is still using the code threshold"
|
||||
assert kw["limit"] == pc.RULEHINT_LIMIT
|
||||
# NO CATEGORY FILTER, on the axis that exists: the act arms narrow by
|
||||
@@ -179,21 +172,16 @@ async def test_the_arm_searches_on_its_OWN_bar_not_the_code_one():
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_arm_does_not_fire_on_a_write_that_matched_nothing():
|
||||
"""The gate, pinned — because the fixture above now depends on it and a
|
||||
silent change would make every other test here pass vacuously.
|
||||
|
||||
A write matching no prior art returns before the rule arm runs. That is
|
||||
deliberate: the arm is a semantic search, and ungating it means an
|
||||
embedding query on every write in the session. The gate stays until the
|
||||
arm's precision is fixed, so this failing is a reason to look at precision
|
||||
rather than to update the assertion.
|
||||
"""
|
||||
async def test_a_write_that_matched_no_prior_art_still_gets_its_rule_check():
|
||||
"""A rule about the file being written does not depend on a snippet
|
||||
resembling it, so the arm runs before the prior-art guard, not behind it."""
|
||||
rec = MagicMock()
|
||||
hits = [(0.71, fake_rule(id=156, title="A wait with no deadline is a bug"))]
|
||||
await _run_arm(hits, rec, prior_art=[])
|
||||
out = await _run_arm(hits, rec, prior_art=[])
|
||||
|
||||
assert rec.call_count == 0
|
||||
assert rec.call_args.kwargs["rule_ids"] == [156]
|
||||
assert "get_rule(156)" in out["context"]
|
||||
assert out["rule_ids"] == [156]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -834,10 +822,11 @@ async def test_the_tool_arm_logs_the_call_that_found_nothing():
|
||||
out = await _run_tool_arm([], rec, retrieval_log=log)
|
||||
|
||||
assert out == {"context": "", "rule_ids": [], "checkpoint": {}}
|
||||
assert log.call_count == 1
|
||||
assert log.call_args.kwargs["source"] == "pre_tool_rule"
|
||||
assert log.call_args.kwargs["results"] == []
|
||||
assert log.call_args.kwargs["query"] == "curl -s https://git.example/api/v1/runs", (
|
||||
# The arm's own row; the preference slot logs beside it under its own name.
|
||||
rows = [c for c in log.call_args_list if c.kwargs.get("source") == "pre_tool_rule"]
|
||||
assert len(rows) == 1
|
||||
assert rows[0].kwargs["results"] == []
|
||||
assert rows[0].kwargs["query"] == "curl -s https://git.example/api/v1/runs", (
|
||||
"the query is the point of the row: it is what a threshold is tuned against"
|
||||
)
|
||||
rec.assert_not_called()
|
||||
@@ -850,8 +839,9 @@ async def test_the_tool_arm_logs_a_call_whose_only_hit_was_already_shown():
|
||||
out = await _run_tool_arm(hits, rec, retrieval_log=log, exclude_rule_ids=[161])
|
||||
|
||||
assert out["rule_ids"] == []
|
||||
assert log.call_count == 1
|
||||
assert log.call_args.kwargs["results"] == []
|
||||
rows = [c for c in log.call_args_list if c.kwargs.get("source") == "pre_tool_rule"]
|
||||
assert len(rows) == 1
|
||||
assert rows[0].kwargs["results"] == []
|
||||
rec.assert_not_called()
|
||||
|
||||
|
||||
@@ -982,14 +972,21 @@ async def test_a_genuine_ranker_decline_reports_zero_suppression():
|
||||
assert row.kwargs["suppressed"] is not None
|
||||
|
||||
|
||||
def _tool_row(log):
|
||||
"""The tool arm's own call row; the preference slot logs beside it."""
|
||||
rows = [c for c in log.call_args_list if c.kwargs.get("source") == "pre_tool_rule"]
|
||||
assert len(rows) == 1, rows
|
||||
return rows[0].kwargs
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_tool_arm_reports_suppression_too():
|
||||
log, rec = MagicMock(), MagicMock()
|
||||
hits = [(0.75, fake_rule(id=161, title="Reach the forge through its MCP tools"))]
|
||||
await _run_tool_arm(hits, rec, retrieval_log=log, exclude_rule_ids=[161])
|
||||
|
||||
assert log.call_args.kwargs["results"] == []
|
||||
assert log.call_args.kwargs["suppressed"] == 1
|
||||
assert _tool_row(log)["results"] == []
|
||||
assert _tool_row(log)["suppressed"] == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -1002,8 +999,8 @@ async def test_a_shown_hit_is_not_counted_as_suppressed():
|
||||
out = await _run_tool_arm(hits, rec, retrieval_log=log, exclude_rule_ids=[12])
|
||||
|
||||
assert out["rule_ids"] == [161]
|
||||
assert log.call_args.kwargs["suppressed"] == 1
|
||||
assert len(log.call_args.kwargs["results"]) == 1
|
||||
assert _tool_row(log)["suppressed"] == 1
|
||||
assert len(_tool_row(log)["results"]) == 1
|
||||
|
||||
|
||||
# ── The identity between the two tables ─────────────────────────────────
|
||||
|
||||
@@ -11,7 +11,7 @@ The full FK-remapping round-trip needs real Postgres and lives in the
|
||||
"""
|
||||
from datetime import datetime, timezone
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -22,7 +22,7 @@ def test_backup_version_is_current():
|
||||
"""The bump is the point of the test — a payload section added without
|
||||
moving the version produces backups that are structurally different and
|
||||
indistinguishable by inspection."""
|
||||
assert backup.BACKUP_VERSION == 27
|
||||
assert backup.BACKUP_VERSION == 28
|
||||
|
||||
|
||||
def _exportable_note(**over):
|
||||
@@ -561,11 +561,15 @@ class _CM:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_export_full_backup_contains_every_declared_section():
|
||||
with patch("grimoire.services.backup.async_session", lambda: _CM()):
|
||||
with patch("grimoire.services.backup.async_session", lambda: _CM()), patch(
|
||||
"grimoire.services.instance.get_instance_id", AsyncMock(return_value="test-lineage"),
|
||||
):
|
||||
out = await backup.export_full_backup()
|
||||
|
||||
assert out["version"] == backup.BACKUP_VERSION
|
||||
assert out["scope"] == "full"
|
||||
# v28: the id lineage a live rollback is checked against.
|
||||
assert out["instance_id"] == "test-lineage"
|
||||
assert "api_keys" in out["_not_included"]
|
||||
# The sections v2 silently dropped, the six v5 added, v6's
|
||||
# note_supersessions, and v7's code_shapes (all empty here).
|
||||
@@ -681,3 +685,41 @@ def test_rule_rows_carry_the_kind_so_a_preference_does_not_restore_as_a_rule():
|
||||
arose_from_id=None, created_at=stamp, updated_at=stamp,
|
||||
)
|
||||
assert backup._rule_rows([row])[0]["kind"] == "preference"
|
||||
|
||||
|
||||
def test_a_rollback_never_cascades_into_a_row_it_keeps_for_outside_references():
|
||||
"""A rollback replaces most backed-up tables whole, and keeps an identity
|
||||
table's rows in place so what points at them from OUTSIDE the backup (API
|
||||
keys, shares, forge connections) survives. If such a kept row had an
|
||||
ON DELETE CASCADE foreign key into a table replaced whole, clearing that
|
||||
table would delete the kept row, and its keys and shares with it."""
|
||||
from grimoire.models import Base
|
||||
|
||||
tables = Base.metadata.tables
|
||||
backed_up = set(backup._BACKED_UP)
|
||||
referenced_from_outside = {
|
||||
fk.column.table.name
|
||||
for table in Base.metadata.sorted_tables if table.name not in backed_up
|
||||
for fk in table.foreign_keys if fk.column.table.name in backed_up
|
||||
}
|
||||
assert {"users", "projects", "notes"} <= referenced_from_outside
|
||||
assert referenced_from_outside <= set(backup._IDENTITY_TABLES)
|
||||
safe_targets = set(backup._IDENTITY_TABLES) | set(backup._CATALOG_TABLES) | (
|
||||
set(tables) - backed_up
|
||||
)
|
||||
unsafe = sorted(
|
||||
f"{name}.{fk.parent.name} -> {fk.column.table.name}"
|
||||
for name in referenced_from_outside
|
||||
for fk in tables[name].foreign_keys
|
||||
if (fk.ondelete or "").upper() == "CASCADE" and fk.column.table.name not in safe_targets
|
||||
)
|
||||
assert unsafe == []
|
||||
|
||||
|
||||
def test_every_rollback_table_list_names_real_backed_up_tables():
|
||||
from grimoire.models import Base
|
||||
|
||||
assert set(backup._IDENTITY_TABLES) <= set(backup._BACKED_UP)
|
||||
assert set(backup._CATALOG_TABLES) <= set(backup._BACKED_UP)
|
||||
assert set(backup._DERIVED_TABLES) <= set(backup._NOT_INCLUDED)
|
||||
assert set(backup._DERIVED_TABLES) <= set(Base.metadata.tables)
|
||||
|
||||
@@ -188,3 +188,47 @@ def test_checking_s3_only_lists():
|
||||
said = asyncio.run(check_destination(S3, "secret", transport=httpx.MockTransport(handler)))
|
||||
assert methods == ["GET", "GET"]
|
||||
assert "holds 2 Grimoire backup" in said
|
||||
|
||||
|
||||
# ── Restoring from what is already at the destination ──────────────────────
|
||||
|
||||
|
||||
def test_a_folder_lists_its_backups_newest_first_and_nothing_else(tmp_path):
|
||||
for name in ("grimoire-20261012T020000Z.tar", "scribe-20261011T020000Z.tar",
|
||||
"grimoire-20261013T020000Z.tar", "notes.txt", ".grimoire-20261014T020000Z.tar.part"):
|
||||
(tmp_path / name).write_bytes(b"x" * 10)
|
||||
found = asyncio.run(backup_destination.list_backups(BackupConfig(folder_path=str(tmp_path)), None))
|
||||
assert [b.name for b in found] == [
|
||||
"grimoire-20261013T020000Z.tar", "grimoire-20261012T020000Z.tar",
|
||||
"scribe-20261011T020000Z.tar",
|
||||
]
|
||||
assert found[0].size_bytes == 10
|
||||
|
||||
|
||||
def test_a_missing_folder_says_so_when_listing(tmp_path):
|
||||
with pytest.raises(DestinationError, match="isn't there"):
|
||||
asyncio.run(backup_destination.list_backups(
|
||||
BackupConfig(folder_path=str(tmp_path / "absent")), None))
|
||||
|
||||
|
||||
def test_opening_a_folder_backup_is_the_file_itself(tmp_path):
|
||||
(tmp_path / "grimoire-20261012T020000Z.tar").write_bytes(b"x")
|
||||
path = asyncio.run(backup_destination.open_backup(
|
||||
BackupConfig(folder_path=str(tmp_path)), None, "grimoire-20261012T020000Z.tar", tmp_path))
|
||||
assert path == tmp_path / "grimoire-20261012T020000Z.tar"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", [
|
||||
"../etc/passwd", "grimoire-x/../../secret.tar", "secret_key", "/data/secret_key",
|
||||
"grimoire-20261012T020000Z.json",
|
||||
])
|
||||
def test_only_a_backup_name_can_be_opened(tmp_path, name):
|
||||
with pytest.raises(DestinationError, match="isn't the name of a Grimoire backup"):
|
||||
asyncio.run(backup_destination.open_backup(
|
||||
BackupConfig(folder_path=str(tmp_path)), None, name, tmp_path))
|
||||
|
||||
|
||||
def test_a_backup_not_in_the_folder_says_so(tmp_path):
|
||||
with pytest.raises(DestinationError, match="isn't in"):
|
||||
asyncio.run(backup_destination.open_backup(
|
||||
BackupConfig(folder_path=str(tmp_path)), None, "grimoire-20261012T020000Z.tar", tmp_path))
|
||||
|
||||
@@ -221,3 +221,124 @@ def test_routes_and_scheduler_are_wired():
|
||||
assert callable(backup_job.start_backup_scheduler)
|
||||
assert callable(backup_job.stop_backup_scheduler)
|
||||
assert backup_destination.SECRET_SETTING == "backup_s3_secret"
|
||||
|
||||
|
||||
# ── The backup a rollback takes of what it replaces ────────────────────────
|
||||
|
||||
|
||||
def _safety_patches(tmp_path, *, config, deliver=None):
|
||||
from grimoire.config import Config
|
||||
|
||||
async def load_config():
|
||||
return config
|
||||
|
||||
async def load_secret():
|
||||
return (None, False)
|
||||
|
||||
async def export():
|
||||
return {"version": 28, "scope": "full", "users": []}
|
||||
|
||||
async def delivered(payload, config, secret, at, **kw):
|
||||
return Delivered("grimoire-20261015T020000Z.tar", 1234, "/backups")
|
||||
|
||||
return [
|
||||
patch.object(backup_job, "load_config", load_config),
|
||||
patch.object(backup_job, "load_secret", load_secret),
|
||||
patch.object(backup_job, "deliver", deliver or delivered),
|
||||
patch("grimoire.services.backup.export_full_backup", export),
|
||||
patch.object(Config, "DATA_DIR", str(tmp_path)),
|
||||
]
|
||||
|
||||
|
||||
def test_the_state_before_a_rollback_goes_to_the_backup_destination(tmp_path):
|
||||
saved = _run(backup_job.save_state_before_restore(),
|
||||
_safety_patches(tmp_path, config=BackupConfig(enabled=True)))
|
||||
assert saved == {"file": "grimoire-20261015T020000Z.tar", "size_bytes": 1234,
|
||||
"where": "/backups"}
|
||||
assert not (tmp_path / backup_job.PRE_RESTORE_DIR).exists()
|
||||
|
||||
|
||||
def test_with_backups_off_it_is_kept_on_the_instance(tmp_path):
|
||||
saved = _run(backup_job.save_state_before_restore(),
|
||||
_safety_patches(tmp_path, config=BackupConfig(enabled=False)))
|
||||
folder = tmp_path / backup_job.PRE_RESTORE_DIR
|
||||
assert saved["where"] == str(folder)
|
||||
assert (folder / saved["file"]).stat().st_size == saved["size_bytes"] > 0
|
||||
assert "note" not in saved
|
||||
|
||||
|
||||
def test_a_destination_that_refuses_it_falls_back_to_the_instance_and_says_so(tmp_path):
|
||||
async def refuse(*a, **kw):
|
||||
raise DestinationError("/backups isn't there.")
|
||||
|
||||
saved = _run(backup_job.save_state_before_restore(),
|
||||
_safety_patches(tmp_path, config=BackupConfig(enabled=True), deliver=refuse))
|
||||
assert saved["where"] == str(tmp_path / backup_job.PRE_RESTORE_DIR)
|
||||
assert "/backups isn't there." in saved["note"]
|
||||
|
||||
|
||||
def test_a_rollback_that_cannot_save_the_current_state_does_not_run(tmp_path):
|
||||
from grimoire.services import backup
|
||||
|
||||
calls = []
|
||||
|
||||
async def check(data):
|
||||
return "rollback"
|
||||
|
||||
async def replace(data):
|
||||
calls.append("replace")
|
||||
return {"mode": "rollback"}
|
||||
|
||||
async def fail():
|
||||
raise OSError("disk full")
|
||||
|
||||
patches = [
|
||||
patch.object(backup, "check_replace", check),
|
||||
patch.object(backup, "replace_from_backup", replace),
|
||||
patch.object(backup_job, "save_state_before_restore", fail),
|
||||
]
|
||||
with pytest.raises(OSError):
|
||||
_run(backup_job.rollback_to_backup({}), patches)
|
||||
assert calls == []
|
||||
|
||||
|
||||
def test_a_refused_rollback_writes_no_safety_backup():
|
||||
from grimoire.services import backup
|
||||
|
||||
calls = []
|
||||
|
||||
async def check(data):
|
||||
raise backup.RestoreRefused("from another instance")
|
||||
|
||||
async def save():
|
||||
calls.append("save")
|
||||
|
||||
patches = [
|
||||
patch.object(backup, "check_replace", check),
|
||||
patch.object(backup_job, "save_state_before_restore", save),
|
||||
]
|
||||
with pytest.raises(backup.RestoreRefused):
|
||||
_run(backup_job.rollback_to_backup({}), patches)
|
||||
assert calls == []
|
||||
|
||||
|
||||
def test_a_rollback_reports_where_the_state_it_replaced_went():
|
||||
from grimoire.services import backup
|
||||
|
||||
async def check(data):
|
||||
return "rollback"
|
||||
|
||||
async def replace(data):
|
||||
return {"mode": "rollback", "restored": {"notes": 3}}
|
||||
|
||||
async def save():
|
||||
return {"file": "grimoire-x.tar", "size_bytes": 9, "where": "/data/pre-restore"}
|
||||
|
||||
patches = [
|
||||
patch.object(backup, "check_replace", check),
|
||||
patch.object(backup, "replace_from_backup", replace),
|
||||
patch.object(backup_job, "save_state_before_restore", save),
|
||||
]
|
||||
result = _run(backup_job.rollback_to_backup({}), patches)
|
||||
assert result["safety_backup"]["where"] == "/data/pre-restore"
|
||||
assert result["restored"] == {"notes": 3}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user