bvandeusen/FabledGrimoire
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases 53 Wiki Activity

docs: architecture, configuration, development and features describe Scribe as it is today

Browse Source 
Milestone 544, step 6217 (follow-up to #1778, which rewrote features.md and
api-reference.md in July but left architecture.md and configuration.md alone).

- architecture.md, configuration.md, development.md: rewritten against the
  code. They still described Ollama, chat, SSE generation, voice, project
  summaries, image caching, Docker secrets that the compose files don't use,
  a raw-SQL migration convention the migrations no longer follow, a 27-entry
  migration list, CalVer YY.MM.DD.N tags and the removed infra/ files.
- features.md: adds what shipped since July (spikes, claims, moments, rule
  versions and verification, lessons, reply shapes, snippets and the shape
  ledger, forge integration, family canon, design systems, retrieval tuning)
  and corrects the Settings tabs and keyboard shortcuts (g g is the graph).
- api-reference.md: adds the endpoints for snippets, lessons, design
  systems, family canon, platforms, shared areas, retrieval, rule versions,
  relations and verification, forge connections, webhooks, scheduled
  backups and the newer plugin hooks.
- api-keys-and-mcp.md: Settings tab is "MCP Access"; a plugin install
  section; the tool-family table matches the tools that exist.
- design-system.md: the dated "decisions in progress" changelog (April
  PR log, chat/journal/calendar surfaces) becomes a short present-tense
  section; the stored design system is named as the authority.
- sso-oauth.md: the app is Scribe, not "Fabled Assistant".
- README: feature summary and doc table brought current.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
bvandeusen
2026-10-21 19:23:56 -04:00
co-authored by Claude Opus 5.5
parent 82e49176b1
commit 47d40bffd8
10 changed files with 410 additions and 964 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/development.md
+48 -140
View File
@@ -1,179 +1,87 @@
# Development
## Workflow
All development is Docker-based. Do not install Python or Node dependencies locally.
## Running it
```bash
# Start the full stack (app + PostgreSQL + Ollama)
docker compose up --build
docker compose up --build # app + PostgreSQL at http://localhost:5000
docker compose up --build app # rebuild after a change
docker compose down -v # stop and wipe the database
```
# Rebuild after backend changes (frontend changes require rebuild too)
docker compose up --build app
The image builds the frontend, so a frontend change needs a rebuild too. For quicker frontend work, run Vite against the running app; it proxies `/api` to port 5000:
# Reset everything (wipes database)
docker compose down -v && docker compose up --build
```bash
cd frontend && npm ci && npm run dev # http://localhost:5173
```
# Run checks (lint, format, typecheck, tests)
make check
## Checks
# Individual checks
CI runs these on every push; locally they need `uv sync --extra dev` and `cd frontend && npm ci`.
```bash
make lint # ruff check src/
make fmt # ruff format src/
make typecheck # vue-tsc --noEmit
make test # pytest tests/
make check # all three
```
## Frontend Hot Reload
Unit tests run without a database. Tests marked `integration` need a real PostgreSQL and run in CI's integration job, against a schema built by the real migrations.
The Docker setup does not include Vite's hot-reload dev server. After frontend changes, rebuild the image. For faster iteration during active frontend work, you can run Vite locally:
## Database migrations
The container runs `alembic upgrade head` on every start, after waiting for the database to accept connections.
```bash
cd frontend
npm install
npm run dev # Vite dev server at http://localhost:5173
docker compose exec app alembic revision -m "short_description"
```
Point the Vite dev server at the backend by setting `VITE_API_BASE_URL=http://localhost:5000` (or configure `vite.config.ts` proxy).
- Number revisions sequentially (`0123_...`). Revision ids must fit in 32 characters, the width of `alembic_version.version_num`.
- Write both `upgrade()` and `downgrade()`.
- Adding a value to a `CHECK`-constrained column means dropping and re-adding the constraint in the same migration.
- Keep migrations to schema changes. Filling a derived table from a large one belongs in a background backfill after startup, where its cost doesn't hold up the boot.
- A migration that depends on application code (a key derivation, a serialiser) copies what it needs rather than importing it, so a later edit can't change what an old migration does.
## Database Migrations
## The plugin
Alembic migrations run automatically on container startup (`alembic upgrade head` in the `CMD`).
`plugin/` ships straight from git: installs fetch it through `.claude-plugin/marketplace.json`, and there is no build step. After changing anything under `plugin/` or `.claude-plugin/`, run `make mint-plugin` before committing. Claude Code only refreshes an installed plugin when its version changes, and CI fails a plugin change that didn't bump it.
To create a new migration:
```bash
# Inside the running app container
docker compose exec app alembic revision -m "description_of_change"
# Edit the generated file in alembic/versions/
```
The hooks must run on a bare machine: POSIX shell, awk, sed and the usual coreutils, plus `git` and `curl`. They fail open, so a slow or unreachable Scribe never blocks Claude Code.
Migration conventions:
- Use raw SQL with `IF NOT EXISTS` guards for idempotency
- Use `DO $$ BEGIN CREATE TYPE … EXCEPTION WHEN duplicate_object THEN NULL; END $$` for enum types
- Number migrations sequentially (e.g. `0027_add_something.py`)
- Always provide both `upgrade()` and `downgrade()`
## CI and releases
## CI/CD
CI runs on Forgejo Actions in the shared `ci-python:3.14` image ([`ci-requirements.md`](../ci-requirements.md) lists what it relies on). Six jobs: TypeScript typecheck, plugin hooks, Python lint (plus design-token and dangling-style checks), unit tests, integration tests, and the image build, which waits on all of them except the plugin job.
### Pipeline
| Push | Image tags |
|------|-----------|
| `dev` | `:dev`, `:<sha>` |
| `main` | `:latest`, `:<sha>` |
| a `v*` tag | `:latest`, `:<version>`, `:<sha>` |
CI runs on Forgejo Actions, consuming the shared
[`ci-python:3.14`](https://git.fabledsword.com/bvandeusen/CI-runner) image
via `container.image` (Python 3.14 + Node 24 + ruff + uv + Docker CLI):
`:<sha>` never moves, so it is the rollback point. Images publish to `git.fabledsword.com/bvandeusen/fabledscribe`, with a registry layer cache at `:cache`. The build needs the `REGISTRY_USER` and `REGISTRY_TOKEN` (a token with `write:packages`) repository secrets.
| Trigger | Jobs | Docker tags pushed |
|---------|------|--------------------|
| Push to `dev` | typecheck + lint + test → build | `:dev`, `:<sha>` |
| Tag `v*` on `main` | typecheck + lint + test → build | `:latest`, `:<version>`, `:<sha>` |
| Push to `main` | typecheck + lint + test | (no build) |
Work happens on `dev`. A release merges `dev` into `main` with a plain merge commit; a dated `vYYYY.MM.DD.HHMM` tag, matching the version the image reports, can mark it.
### Release Process
1. Work on `dev` branch — CI validates on every push
2. When ready, open a PR from `dev` → `main` in Forgejo
3. Merge the PR
4. Create a release via the Forgejo UI on `main` with a `v*` tag (e.g. `v26.03.23.1` — CalVer: `YY.MM.DD.N`)
5. The tag push triggers CI → build job pushes `:latest` + `:<version>` Docker images
6. After merging to main, sync dev back:
```bash
git checkout dev && git merge main && git push origin dev
```
### Runner
CI jobs schedule against the `python-ci` runner label and run inside the
shared `git.fabledsword.com/bvandeusen/ci-python:3.14` image (see
`ci-requirements.md` for what this project relies on from the image).
The runner deployment lives outside this repo; image bumps happen in
[CI-Runner](https://git.fabledsword.com/bvandeusen/CI-runner) via Renovate.
`infra/runner-compose.yml` + `infra/act-runner-config.yml` document the
runner-host deployment shape; the source of truth is the deployed
config on the runner host.
### Docker Registry
Images pushed to: `git.fabledsword.com/bvandeusen/scribe`
Cache tag: `:cache` (reduces build time ~80%)
Required secrets (repo → Settings → Secrets → Actions):
- `REGISTRY_USER` — Forgejo username
- `REGISTRY_TOKEN` — Forgejo PAT with `write:packages` scope
## Migration Chain
Current migration sequence (all idempotent raw SQL):
```
0001 create_notes_table
0002 create_tasks_table
0003 task_note_companion (data migration)
0004 merge_tasks_into_notes
0005 add_chat_tables
0006 add_settings_table
0007 add_title_and_updated_at_indexes
0008 add_users_and_user_id
0009 add_message_status
0010 add_app_logs_table
0011 add_password_reset_tokens
0012 add_invitation_tokens
0013 add_tool_calls_to_messages
0014 add_note_embeddings
0015 add_oauth_fields
0016 add_image_cache
0017 add_projects
0018 add_push_subscriptions
0019 add_events (dead code — internal CalDAV/Radicale table; Radicale was removed)
0020 add_milestones
0021 add_task_logs
0022 add_note_versions_and_drafts
0023 add_tags_to_note_versions
0024 add_session_version
0025 add_sharing_and_notifications
0026 add_briefing_tables
0027 add_api_keys
```
**Important:** Do NOT use `op.create_table()` or `sa.Enum()` — SQLAlchemy's event system can fire `CREATE TYPE` even with `create_type=False`, causing failures on re-run. Always use raw SQL with `IF NOT EXISTS` / `DO $$ BEGIN ... EXCEPTION WHEN duplicate_object` guards.
## Project Conventions
## Conventions
### Backend
- Services: `async with async_session() as session:` — import from `scribe.models`
- No `scribe.database` module
- Blueprint per resource: `routes/notes.py`, `routes/tasks.py`, etc.
- All business logic in `services/`; routes are thin wrappers
- Permission checks via `services/access.py` — never inline ownership checks in routes
- Services open their own session: `async with async_session() as session:` (from `scribe.models`).
- Routes and MCP tools stay thin; the logic lives in `services/`, shared by both.
- Access checks go through `services/access.py`, never inline ownership checks.
- Every wait on something outside the process has a deadline.
### Frontend
- API calls via `frontend/src/api/client.ts` typed helpers (`apiGet`, `apiPost`, `apiPatch`, `apiDelete`)
- Pinia stores for shared state; local `ref()` for component-only state
- Composables in `composables/` for reusable behaviour (autosave, keyboard nav, tag suggestions, …)
- Views are page-level components in `views/`; reusable UI in `components/`
- API calls go through the typed helpers in `frontend/src/api/client.ts`.
- Pinia stores for shared state, `ref()` for state local to a component; composables for reusable behaviour.
- Colours, spacing and type come from the design system's tokens in `theme.css`, which is generated, so CI rejects an unknown `var()`.
### Commit Style
### Commits
```
type(scope): short description
type(scope): what the change does
Longer body if needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Why, and anything a reader tracing this change later needs.
```
Types: `feat`, `fix`, `refactor`, `docs`, `chore`, `test`
Scopes: feature area (e.g. `chat`, `journal`, `mcp`, `notes`)
## Testing
```bash
# Run all tests
make test
# Run specific test file
docker compose exec app /opt/venv/bin/pytest tests/test_auth.py -v
```
Tests are in `tests/`. They run against a real PostgreSQL instance in CI (not mocked). Keep tests integration-style where possible — mock failures have historically masked real migration bugs.
Types: `feat`, `fix`, `refactor`, `docs`, `chore`, `test`. Comments in the code describe how it works today; the history of a change belongs in its commit message.