FabledGrimoire/docs/architecture.md

Architecture

Grimoire is one app container and one PostgreSQL container. There is no language model in the stack: Claude works with Grimoire from outside, through the MCP endpoint and the Claude Code plugin, and people use the web UI.

Stack

Layer	Technology
Frontend	Vue 3, TypeScript, Vite, Pinia, Vue Router; Tiptap for the editor
Backend	Python 3.14, Quart (async), served by Hypercorn
MCP	The mcp SDK's server, mounted in the same app at /mcp (streamable HTTP)
Database	PostgreSQL 17 with pgvector; SQLAlchemy 2 (async, asyncpg); Alembic migrations
Embeddings	fastembed running in-process (BAAI/bge-small-en-v1.5, 384 dimensions)
Plugin	Claude Code hooks (POSIX shell + awk), skills and a slash command, shipped from this repo

How the pieces fit

 Claude Code ──── plugin hooks ──── /api/plugin/*  ┐
      │                                            │
      └────────── MCP tools ──────── /mcp          ├── Quart app ── PostgreSQL + pgvector
                                                   │      │
 Browser ──────── Vue SPA ────────── /api/*        ┘      └── in-process jobs and embedder

• The web UI is a Vue single-page app, built into the image and served by Quart. Every unknown path that isn't under /api/ falls back to index.html; /api/* always answers JSON.
• The REST API (src/grimoire/routes/) backs the UI, one blueprint per resource.
• The MCP server (src/grimoire/mcp/) is how Claude reads and writes records. Each module in mcp/tools/ registers its tools, and the server's instructions and each tool's docstring are the contract an agent reads. Requests authenticate with an API key as a Bearer token.
• The plugin (plugin/) runs in the user's Claude Code. Its hooks call /api/plugin/* to put the right context in front of Claude at the right moment: the project at session start, matching rules and prior art before a prompt or a tool call, the rules mounted on a moment of work, and a check on the reply that ends a turn. Its skills describe the working practices. See API Keys & MCP to install it.
• Services (src/grimoire/services/) hold the logic both doors share. Routes and MCP tools stay thin: parse the request, check access, call a service, return a plain dict.

Data model

Most records are rows in one notes table, told apart by kind:

• Notes, and the specialised kinds that are notes underneath: snippets (recorded prior art, with a canonical code location), lessons (transferable insights, linked to the rules they instance) and processes (stored procedures, synced to Claude Code as skills).
• Tasks: a note with a status. A task's kind is work, issue or spike. Tasks carry work logs, sub-tasks, recurrence and a session claim.

Around them:

• Projects group work and carry a goal. Milestones are plans: a design body plus ordered step tasks.
• Systems are a project's named areas; records are tagged with them, and each maps to a canonical system shared across projects.
• Rulebooks → topics → rules. A rule in a rulebook is global; a rule on a project applies there only. Rules have a trigger (when_to_apply), versions, relations to other rules, and can be mounted on moments of work (work.change, reply.report, ...), which plugin actions reach through moment mappings.
• Family canon: ideas shared across projects on the same platform, and each project's answer to them (adopted, variant, owed).
• Design systems: tokens and guidance, with inheritance; the stylesheet is generated from them.
• The shape ledger: code shapes extracted from bound repos and their classification against snippets.
• Embeddings for notes (chunked), rules, milestones and systems, in pgvector columns.
• Telemetry: retrieval logs, note and rule usage, and the tuning history of retrieval thresholds.
• Users, groups, shares and API keys. Access is decided in the app (services/access.py): owners, direct shares and group shares, with read or write permission.

Deleted records go to the trash first and can be restored until retention purges them.

Retrieval

Search and the plugin's context injection run on the same pipeline (services/retrieval_pipeline.py): embed the query, rank stored chunks by cosine similarity in pgvector, keep what clears the surface's threshold, and log what was shown. Each surface (MCP search, auto-inject, the pre-tool rule arm, the write-path prior-art arm, ...) is a spec over those stages with its own thresholds and budget. The thresholds are settings an admin can tune, and every change is recorded with its reason. When the embedder is unavailable, search falls back to keyword matching.

The embedding model downloads into FASTEMBED_CACHE_DIR (default /data/fastembed-cache) on first use, so the /data volume keeps it across restarts.

Background work

All in-process; there is no separate worker container.

• At startup the app records any shipped default that moved in this release, starts its schedulers, and only then backfills missing embeddings, so the backfill never competes with startup for the database.
• Recurring tasks spawn every 15 minutes. Daily jobs: version auto-pinning (03:00 UTC), trash purge (03:30 UTC), targeted VACUUM ANALYZE (04:00 UTC by default). Scheduled backups check hourly whether one is due (Backups).
• Log, notification and auth-token retention run on their own loops.
• A diagnostics heartbeat writes to /data/diagnostics.

Every database wait on startup has a deadline. Migrations wait for the database to accept connections (up to five minutes) before they run, so the app starts cleanly whatever order the containers come up in.

Authentication

• Sessions: HttpOnly, SameSite=Lax cookies, Secure on a request the trusted proxy reports as HTTPS, which also gets HSTS (grimoire/proxy.py). Changing a password bumps the user's session version and signs out their other sessions.
• API keys: Authorization: Bearer fmcp_..., stored hashed. A read key may only call read endpoints and read MCP tools.
• OIDC single sign-on with PKCE, optionally replacing local logins (SSO / OAuth).
• Rate limits on register, login and forgot-password, per client IP, read from X-Forwarded-For by TRUSTED_PROXY_HOPS.

Credentials Grimoire stores for outside services (forge tokens, the backup destination's S3 secret) are sealed under a key derived from SECRET_KEY, which the instance makes on first start and keeps in /data/secret_key unless one is set.

Repository layout

├── alembic/versions/      database migrations
├── src/grimoire/
│   ├── app.py             app factory, blueprints, startup and shutdown
│   ├── config.py          environment configuration
│   ├── models/            SQLAlchemy models
│   ├── routes/            REST blueprints
│   ├── mcp/               MCP server and its tool modules
│   ├── services/          shared logic
│   └── restore.py         `python -m grimoire.restore`: the restore, from the command line
├── frontend/src/          views, components, stores, composables, api client
├── plugin/                Claude Code hooks, skills and commands
├── scripts/               CI checks and the plugin version minter
└── tests/                 unit tests, plus `-m integration` against real Postgres