bvandeusenandClaude Opus 5.5
0f5ef0be63
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 56s
CI & Build / integration (push) Successful in 1m4s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 44s
Step 2 of the Scribe -> Grimoire rename. - src/scribe -> src/grimoire, and every import, patch target, module path, hypercorn entry point and `python -m grimoire.restore` with it; pyproject, uv.lock and the frontend package are named grimoire. - Image git.fabledsword.com/bvandeusen/fabledgrimoire; CI's Postgres service runs as grimoire / grimoire_test. - Compose files default the DB user, database and (prod) volume to grimoire, each overridable from .env, so an install created as Scribe keeps its data by setting POSTGRES_USER, POSTGRES_DB and PGDATA_VOLUME. - Backups are written as grimoire-<ts>.tar with format grimoire-backup-tar. Archives named scribe-* with the scribe-backup-tar manifest still restore and are still counted at the destination (folder and S3), with tests. - Exports, the MCP auth realm, the ASGI scope key and the persistent logger carry the new name. Kept on purpose: the secret-sealing key label stays "scribe.secrets.v1", because every stored secret was derived under it. Plugin identity (plugin name, hooks, env vars, .scribe marker, skills) is step 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
96 lines
7.3 KiB
Markdown
96 lines
7.3 KiB
Markdown
# Architecture
|
|
|
|
Grimoire is one app container and one PostgreSQL container. There is no language model in the stack: Claude works with Grimoire from outside, through the MCP endpoint and the Claude Code plugin, and people use the web UI.
|
|
|
|
## Stack
|
|
|
|
| Layer | Technology |
|
|
|-------|-----------|
|
|
| Frontend | Vue 3, TypeScript, Vite, Pinia, Vue Router; Tiptap for the editor |
|
|
| Backend | Python 3.14, Quart (async), served by Hypercorn |
|
|
| MCP | The `mcp` SDK's server, mounted in the same app at `/mcp` (streamable HTTP) |
|
|
| Database | PostgreSQL 17 with pgvector; SQLAlchemy 2 (async, asyncpg); Alembic migrations |
|
|
| Embeddings | fastembed running in-process (`BAAI/bge-small-en-v1.5`, 384 dimensions) |
|
|
| Plugin | Claude Code hooks (POSIX shell + awk), skills and a slash command, shipped from this repo |
|
|
|
|
## How the pieces fit
|
|
|
|
```
|
|
Claude Code ──── plugin hooks ──── /api/plugin/* ┐
|
|
│ │
|
|
└────────── MCP tools ──────── /mcp ├── Quart app ── PostgreSQL + pgvector
|
|
│ │
|
|
Browser ──────── Vue SPA ────────── /api/* ┘ └── in-process jobs and embedder
|
|
```
|
|
|
|
- **The web UI** is a Vue single-page app, built into the image and served by Quart. Every unknown path that isn't under `/api/` falls back to `index.html`; `/api/*` always answers JSON.
|
|
- **The REST API** (`src/grimoire/routes/`) backs the UI, one blueprint per resource.
|
|
- **The MCP server** (`src/grimoire/mcp/`) is how Claude reads and writes records. Each module in `mcp/tools/` registers its tools, and the server's instructions and each tool's docstring are the contract an agent reads. Requests authenticate with an API key as a Bearer token.
|
|
- **The plugin** (`plugin/`) runs in the user's Claude Code. Its hooks call `/api/plugin/*` to put the right context in front of Claude at the right moment: the project at session start, matching rules and prior art before a prompt or a tool call, the rules mounted on a moment of work, and a check on the reply that ends a turn. Its skills describe the working practices. See [API Keys & MCP](api-keys-and-mcp.md) to install it.
|
|
- **Services** (`src/grimoire/services/`) hold the logic both doors share. Routes and MCP tools stay thin: parse the request, check access, call a service, return a plain dict.
|
|
|
|
## Data model
|
|
|
|
Most records are rows in one `notes` table, told apart by kind:
|
|
|
|
- **Notes**, and the specialised kinds that are notes underneath: **snippets** (recorded prior art, with a canonical code location), **lessons** (transferable insights, linked to the rules they instance) and **processes** (stored procedures, synced to Claude Code as skills).
|
|
- **Tasks**: a note with a status. A task's kind is `work`, `issue` or `spike`. Tasks carry work logs, sub-tasks, recurrence and a session claim.
|
|
|
|
Around them:
|
|
|
|
- **Projects** group work and carry a goal. **Milestones** are plans: a design body plus ordered step tasks.
|
|
- **Systems** are a project's named areas; records are tagged with them, and each maps to a canonical system shared across projects.
|
|
- **Rulebooks → topics → rules.** A rule in a rulebook is global; a rule on a project applies there only. Rules have a trigger (`when_to_apply`), versions, relations to other rules, and can be mounted on **moments** of work (`work.change`, `reply.report`, ...), which plugin actions reach through moment mappings.
|
|
- **Family canon**: ideas shared across projects on the same platform, and each project's answer to them (adopted, variant, owed).
|
|
- **Design systems**: tokens and guidance, with inheritance; the stylesheet is generated from them.
|
|
- **The shape ledger**: code shapes extracted from bound repos and their classification against snippets.
|
|
- **Embeddings** for notes (chunked), rules, milestones and systems, in pgvector columns.
|
|
- **Telemetry**: retrieval logs, note and rule usage, and the tuning history of retrieval thresholds.
|
|
- **Users, groups, shares and API keys.** Access is decided in the app (`services/access.py`): owners, direct shares and group shares, with read or write permission.
|
|
|
|
Deleted records go to the trash first and can be restored until retention purges them.
|
|
|
|
## Retrieval
|
|
|
|
Search and the plugin's context injection run on the same pipeline (`services/retrieval_pipeline.py`): embed the query, rank stored chunks by cosine similarity in pgvector, keep what clears the surface's threshold, and log what was shown. Each surface (MCP search, auto-inject, the pre-tool rule arm, the write-path prior-art arm, ...) is a spec over those stages with its own thresholds and budget. The thresholds are settings an admin can tune, and every change is recorded with its reason. When the embedder is unavailable, search falls back to keyword matching.
|
|
|
|
The embedding model downloads into `FASTEMBED_CACHE_DIR` (default `/data/fastembed-cache`) on first use, so the `/data` volume keeps it across restarts.
|
|
|
|
## Background work
|
|
|
|
All in-process; there is no separate worker container.
|
|
|
|
- At startup the app records any shipped default that moved in this release, starts its schedulers, and only then backfills missing embeddings, so the backfill never competes with startup for the database.
|
|
- Recurring tasks spawn every 15 minutes. Daily jobs: version auto-pinning (03:00 UTC), trash purge (03:30 UTC), targeted `VACUUM ANALYZE` (04:00 UTC by default). Scheduled backups check hourly whether one is due ([Backups](backups.md)).
|
|
- Log, notification and auth-token retention run on their own loops.
|
|
- A diagnostics heartbeat writes to `/data/diagnostics`.
|
|
|
|
Every database wait on startup has a deadline. Migrations wait for the database to accept connections (up to five minutes) before they run, so the app starts cleanly whatever order the containers come up in.
|
|
|
|
## Authentication
|
|
|
|
- **Sessions**: `HttpOnly`, `SameSite=Lax` cookies, `Secure` when `SECURE_COOKIES=true`. Changing a password bumps the user's session version and signs out their other sessions.
|
|
- **API keys**: `Authorization: Bearer fmcp_...`, stored hashed. A `read` key may only call read endpoints and read MCP tools.
|
|
- **OIDC** single sign-on with PKCE, optionally replacing local logins ([SSO / OAuth](sso-oauth.md)).
|
|
- **Rate limits** on register, login and forgot-password, per client IP (`TRUST_PROXY_HEADERS` reads the IP from the proxy's headers).
|
|
|
|
Credentials Grimoire stores for outside services (forge tokens, the backup destination's S3 secret) are sealed under a key derived from `SECRET_KEY`.
|
|
|
|
## Repository layout
|
|
|
|
```
|
|
├── alembic/versions/ database migrations
|
|
├── src/grimoire/
|
|
│ ├── app.py app factory, blueprints, startup and shutdown
|
|
│ ├── config.py environment configuration
|
|
│ ├── models/ SQLAlchemy models
|
|
│ ├── routes/ REST blueprints
|
|
│ ├── mcp/ MCP server and its tool modules
|
|
│ ├── services/ shared logic
|
|
│ └── restore.py `python -m grimoire.restore` for a fresh instance
|
|
├── frontend/src/ views, components, stores, composables, api client
|
|
├── plugin/ Claude Code hooks, skills and commands
|
|
├── scripts/ CI checks and the plugin version minter
|
|
└── tests/ unit tests, plus `-m integration` against real Postgres
|
|
```
|