16 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	95a37318fc	feat(rules): rules become findable by meaning (#3030, milestone 307 step 4) ... Rules were the only major record type with no vector, so `search` could never return one and a rule could arrive only by being preloaded. That single fact is what made every rule compete for one always-on budget. THE DECISION THE TASK ASKED FOR, made explicitly: a sibling rule_embeddings table, not a polymorphic embedding row. The ROW could have been generalised; the SEARCH could not. semantic_search_notes is Note-specific scoping end to end — the visibility clause, the supersession penalty, note_type/task_kind/system filters — and a rule shares none of it, scoping instead by rulebook ownership or project. Generalising the row while still needing two searches is the worst of both: a key with referential integrity to neither table, on the path every session start runs, to share four columns. What is genuinely common is BEHAVIOUR — get_embedding, chunk_document, embedding_text, CHUNKER_VERSION — and those are reused as-is. Sharing them is the DRY win; sharing the table would have been the DRY costume. The document shape is measured, not chosen (note 2485). That pass found the snippet was the only discriminative record in the corpus — a 0.153 top-to-second gap against 0.010-0.023 — and that the cause was its SHAPE: purpose stated twice in a short single-topic document. rule_document reproduces it: the trigger in the title AND as the body's first line. And it excludes `why`, which matters more than any of it. `why` is dated incident narrative — rule 46's runs to 4,300 characters — and long multi-topic prose is exactly what made sixteen dev-logs mutually indistinguishable. Adding it would not give the vector more to work with; it would give every rule the SAME thing to work with. rule_document takes no `why` parameter at all, so a well-meaning caller cannot pass one. A rule with no trigger degrades to title + statement — findable, less sharp. That is an argument for backfilling triggers (step 6), not for padding the document with whatever text is nearby. search(content_type="rule") returns the rule WITH its why and how_to_apply: they are its operational half, the session payload never carries them, and a caller who went looking should not have to re-fetch. Writes re-index fire-and-forget like notes; startup backfills in its own try block so neither backfill can skip the other. rule_embeddings is derived, so it joins note_embeddings in the backup's explicitly-NOT-included list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 15:00:49 -04:00
bvandeusenandClaude Opus 5	6ddb8bf859	feat(rules): a rule can say when it applies, which area it is about, and what it belongs with (#3029, milestone 307 step 3, schema) ... A rule could not state its trigger, its area, or its siblings, so all three were being written as prose instead: a System's charter restating rule text, a `why` naming the note that caused it, and two halves of one shape merged into a single row because either could surface without the other. Migration 0088 adds the four fields those workarounds stood in for: - `when_to_apply` — the trigger. Nullable in the DB and required at the service layer: existing rules have none and a migration cannot invent one. - `tier` — always_on | conditional, defaulting to always_on. This migration therefore changes NOTHING about which rules bind; an install upgrades and every rule keeps arriving exactly as before. Getting that backwards is the one failure this milestone exists to prevent, so _valid_tier falls back to always_on rather than silently un-binding a rule with a typo'd tier. - `arose_from_id` — the record that caused the rule, the edge notes and tasks already have. SET NULL: trashing the source does not repeal the rule. - `rule_systems` / `rule_relations` — the canon tag and the typed edges (co_surfaces / overrides / elaborates), each earned from a workaround its absence forced. rule_brief() replaces the THREE hand-written trim dicts that had already diverged — two carried topic_id, one didn't, and none carried the timestamps the model has held all along. That omission is why a rule written before the capability it duplicates was indistinguishable at read time from one still doing work. It now carries updated_at as a DATE: the question is "how old is this", and a full stamp across the always-on set is ~2k characters for precision nobody reads. The two callers select the ENTITY rather than a column list, so rule_brief stays the single place deciding what a surfaced rule says. Backup: both new tables carried, area tags by canonical SLUG (ids are per-install). The rule-relation restore runs after ALL rules exist and after the catalog, because an edge names two rules and a tag names a global row — sections renumbered so the file reads in dependency order. A pre-0088 payload restores with tier=always_on, i.e. binding exactly as when it was taken. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 14:11:32 -04:00
bvandeusenandClaude Opus 5	a97547fbc6	feat(systems): the area vocabulary becomes a global table so a rule can point at one (#3027, milestone 307 step 1) ... The eight standard area names already existed — as STANDARD_SYSTEMS, a tuple in services/systems.py that milestone 297 seeds at inception. A constant cannot be a foreign key, so nothing outside a project could reference an area: systems. project_id is NOT NULL, and a rule that spans projects would have to chain itself to one project's row. And because the list only ever applied on the inception-seed path, three spellings of one area reached this instance anyway (CI & runners / CI and Release / CI & release). - canonical_systems: global, no user_id — a shared project inherits the vocabulary instead of re-earning it. Migration 0087 seeds the same eight. - systems.canonical_id: nullable, SET NULL. Association only — no System is renamed and record_systems is untouched, so no record's tags move. - canonical_slug folds &/and, case and punctuation, so spelling variants map mechanically and a real difference ("CI & runners") becomes a proposal a human confirms. propose_mappings reports; set_system_canonical is the only writer. - seed_standard_systems now reads the catalog and maps as it mints, so a project born standard never needs a reconciliation pass. - Catalog writes are admin-only; reads are open — a global list anyone can extend stops being shared. - backup: carried by SLUG, not id (ids are per-install). Restore reuses the target's own rows and only creates entries an admin added on the source; an unknown slug restores unmapped rather than failing. Rule 22: STANDARD_SYSTEMS is removed, not deprecated. Rule 115: nothing seeded names an app, repo or house convention. Design in note 3026. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 12:31:18 -04:00
bvandeusenandClaude Fable 5	ffbdf19116	feat(ledger): the CSS consumer map — code_shape_consumers edges (shape → file whose markup names the class, count), migration 0086, resolve_consumers (own-file row when the template defines the class, else every other definition), sync_repo_consumers rebuilt from the archive on every refresh, consumers_of; derived, so not backed up (milestone 302 step 2, #2935) ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-23 13:59:55 -04:00
bvandeusenandClaude Fable 5	e9b8f525c8	feat(inception): projects.inception record + project_rulebook_exclusions — migration 0085 with legacy backfill; backup v10 (#2879, milestone 297 step 1) ... A project's inheritance becomes a decision, not a default (milestone 297). - projects.inception (JSONB, NULL = undecided): {decided_at, decided_by, via mcp|ui|legacy, choices {exclude_always_on_rulebooks, subscribe_rulebooks, design_system_id, seed_systems}}; on to_dict. - project_rulebook_exclusions: a project's opt-out of a whole always-on rulebook — the sibling of the rule/topic suppressions, CASCADE both ways. - services/inception.py (first cut): the vocabulary, validate_inception (pure, all-or-nothing), normalize_choices, is_decided. Effects come in step 3. - Migration 0085 backfills every existing project via="legacy" with its current standing (no exclusions, its subscriptions, its design_system_id, no seed) so the ask fires only for projects created after this ships. - Backup v10: rulebook_exclusions section; project rows carry inception and design_system_id, restored in a post-pass once rulebooks/design systems are mapped (design_system_id was not restored before — fixed in passing). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-21 21:57:02 -04:00
bvandeusenandClaude Fable 5	6a0f8ad328	feat(backup): code_shape_uses travels (v9); consumer-map test expects uses (#2870) ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-21 15:21:13 -04:00
bvandeusenandClaude Fable 5	7d48eb0b1b	refactor(services): one periodic-task shape, one APScheduler job shape, one token hash, one summary rule — the services pass of the shape audit (#2830, milestone 296) ... - background.start_periodic(interval, work, label=) replaces the three hand-rolled while-True/sleep/try loops in logging, auth and notifications. - services/scheduler.ScheduledJob replaces the four private BackgroundScheduler copies in recurrence/version_pinning/trash/db_maintenance schedulers; public start_/stop_/reschedule_ surfaces unchanged. - api_keys.hash_token is the one sha256 helper; auth.py used to inline it 5x. - auth.is_registration_open reads via settings.get_admin_setting; notification prefs read via settings.get_setting; _fire_share_email uses _get_user_email. - projects.get_project_summary / milestones.get_project_milestone_summary are now the one-id view of their batch siblings instead of a second copy of the queries; sharing.best_permission_by (was _deduplicate_by_permission) is the one rank-dedup, now also used by list_projects_for_user. - backup: the row builders for every section both exporters carry are named functions, so a column added to one export cannot silently miss the other. - iso() from models.base replaces the attr.isoformat()-if-attr-else-None idiom and db_maintenance._iso across services; backup keeps its explicit shape. - trash.py hoists the sql_delete/timedelta imports it re-imported per function. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-21 12:34:28 -04:00
bvandeusenandClaude Fable 5	d74a244b3a	feat(ledger): divergence readout — button B where button A is canon, shape history, and judged-shape recheck (#2793, milestone 294 step 7) ... Every judgment now goes through one helper that remembers the fingerprint judged (classified_sha) and writes a code_shape_events row; the sync writes vanished / reappeared / drifted events and flags recheck_at when a body moves under an instance/variant. The refresh flags diverges_from on shapes new since the previous computation that sit where one canon dominates the judged siblings of their directory+kind and were not proposed as that canon (a first seed flags nothing); the write-path hint asks the same question in-band for the shapes the hook names. list_shapes(flag=divergence|recheck), shape_history(project_id, path, symbol) (read-only), coverage line/payload/ card carry divergent + recheck. Backup v8 carries the history. Plugin 0.1.36. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-20 23:13:39 -04:00
bvandeusenandClaude Fable 5	ba0030e51d	feat(ledger): mechanical proposer — every refresh proposes instances against canon and groups derive-first candidates; agents confirm in batches (#2792, milestone 294 step 6) ... Shapes now carry a content fingerprint (signature + whitespace/comment- insensitive body_sha; migration 0080) and the proposer runs inside the coverage refresh, the one moment bodies exist: symbol elsewhere → textual containment → body references the canon → signature resemblance → semantic (capped per refresh, unreached rows stay unexamined for the next). A hit is a proposal on the row (proposed_snippet_id/basis/score), never a classification; rows with no canon hit group by the derive-first rule (identical body in ≥2 places, same name in ≥3 files) as proposal_basis= derive + a group key. list_shapes(proposal=any|canon|derive|<basis>) is the queue; confirm_shape_proposals(project_id, snippet_id|path|basis) confirms in batches as agent instances; any classify_shapes/hook stamp retires the proposal. Readout carries proposed + derive_groups (line, payload, card). Plugin 0.1.35 (skill: the machine proposes, judgment classifies). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-20 21:30:35 -04:00
bvandeusenandClaude Fable 5	19fdc9aa89	feat(ledger): code_shapes — the shape ledger schema (#2787, milestone 294 step 1) ... The accounting half of the pattern system (governing note 2786): the snippet library records canon (small), this table accounts for EVERY extracted shape (total). Identity is (project, repo_key, path, symbol, kind) — kind included because one file can define '.foo' (css) and 'foo' (sym) as distinct shapes. Status vocabulary: canonical / instance / variant / exempt / unclassified, with unclassified as the default and THE todo state; classifications carry who judged (agent|audit|hook|mechanical|import), when, and the why for variants/exemptions. first/last-seen commits + vanished_at keep history instead of deleting it; a rename reads as vanish+new (accepted for v1). snippet_id is SET NULL on snippet deletion so accounting rows outlive their target and rejoin the todo via the step-2 sync, never dangle silently. Backups: v7 carries code_shapes (judgment data, worth moving) — full and per-user export sections, and a restore that keeps a judgment only when its snippet survives the id re-mapping, downgrading to unclassified otherwise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 19:25:25 -04:00
bvandeusenandClaude Fable 5	1faf8f3ece	feat(forge): per-user forge connections — keyring, host-keyed resolution, project pin (#2778) ... A forge token is a user's credential, not an instance's. The single admin-settings config is replaced by per-user keyring rows (one per forge host), and every server-side forge read runs on the PROJECT OWNER's keyring: - forge_connections table + projects.forge_connection_id pin (migration 0078, which also carries the existing admin config into the first admin's row and deletes the old setting keys — no legacy dual-read) - get_forge() replaced by get_forges(owner_id, project_id) -> ForgeSelector; resolve(repo) picks the connection whose host serves the repo. A pinned project uses ONLY its pinned connection; a stale pin (ownership moved) is ignored, never honored across users - env FORGE_* config survives as an implicit entry for admin owners only; a stored row for the same host beats it - consumers threaded: pull-time freshness (owner of the note), coverage (owner of the project), coverage routes' configured flag - routes: /api/settings/forge-connections CRUD + per-connection test (own-rows only, tokens never returned); /api/admin/forge shrinks to /api/admin/forge-webhook (secret only); PUT /api/projects/<id>/forge pins, owner-or-admin asking, owner's connections only - UI: Git Forges card moves to Settings -> Integrations as a connection list; webhook secret stays in the admin Config tab; owner-only forge select on the project coverage card - backups exclude forge_connections (credentials, api_keys precedent) and the pin, so restores fall back to keyring resolution Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 11:23:22 -04:00
bvandeusen	5dcb738ce8	fix(backup): carry note_supersessions — the coverage guard caught the omission ... CI failed on 45c6b1c. `tests/test_services_backup.py` asserts every table is either in `_BACKED_UP` or explicitly in `_NOT_INCLUDED`, and the new table was in neither. That guard exists for exactly this: a new table gets a model and a migration — both of which fail loudly if wrong — and then silently never gets a backup section. No error, no warning, and a restore that reports success. Its comment says it was written to "stop the seventh" table slipping through. This was the seventh. Supersession claims are backed up rather than excluded because they are a JUDGEMENT. Someone decided this note replaced that one, and nothing in either note's text records the decision. Lose them and a restored corpus silently returns to ranking stale material alongside current material — with no symptom that says why. Scoping differs by backup kind, deliberately. A whole-instance export takes every row. A single-user export takes only claims where BOTH ends are that user's notes: a claim spanning out to someone else's record cannot be restored into a single-user import, since the far id is not in the map, so exporting it would write a row that silently vanishes on the way back in. Restore maps both ids and skips the row unless both resolve. A claim is about a PAIR — half of one is not a weaker claim, it is a dangling row pointing at whatever note holds that id next. Guarded by `data.get` like every post-v2 section, so v5 and older payloads restore cleanly without it. BACKUP_VERSION 5 -> 6, and the version test moved with it. A payload section added without moving the version produces backups that are structurally different and indistinguishable by inspection. Refs #278	2026-08-07 21:49:08 -04:00
bvandeusenandClaude Opus 5	84541f392b	fix(backup): six tables were silently absent, and nothing would catch a seventh ... services/backup.py enumerated its tables as hand-maintained literals with nothing tying them to the schema. Tables added since that list was last extended were absent from every backup — no error, no warning, and a restore that reports success. Missing: systems, record_systems (0065), note_usage_events (0071), design_systems, design_tokens (0072), and repo_bindings — which the issue itself had not spotted, found only by diffing the model tablenames against the two lists instead of trusting either. _NOT_INCLUDED was worse than incomplete: it named "embeddings", "invitations" and "password_resets", none of which are tables. It read as coverage while naming nothing the schema could confirm. Now real names, plus retrieval_logs — observational telemetry that grows per query and that nothing reads for correctness. THE DELIVERABLE IS THE GUARD, not the six sections. Extending a list fixes today and changes nothing about the next table; a new one now fails a test until someone either backs it up or states that it shouldn't be. It checks both directions — an unaccounted table, and a listed name that no longer exists, which is what the three phantom entries above would have tripped. Design systems need ordering care: parent_id is a self-FK. The export orders parent-first (parent_id NULLS FIRST, then id — a parent always has the smaller id), so restore resolves each parent from the map as it goes, with no second pass. A child whose parent is missing lands as a root rather than failing the whole restore. Usage events are kept because pull-through is the evidence base for whether recall works, and it only ever accumulates — a restore that dropped it would reset that measurement to zero while everything still looked fine. BACKUP_VERSION 4 -> 5. Every new restore section is data.get()-guarded, so v2/v3/v4 payloads restore unchanged. Closes #2293. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 23:09:25 -04:00
bvandeusenandClaude Opus 4.8	b49efdcb11	refactor(scribe): retire calendar/events + person/place/list entities (backend) ... Narrow Scribe to a Claude-Code work system-of-record (milestone #194, decision note #1759). Wholesale removal per rule #22 — backend + schema half. Calendar/events + CalDAV: delete models/event, services/{events,caldav, caldav_sync}, routes/events, mcp/tools/events; strip event branches from backup (bump v3->v4), dashboard (upcoming_events), trash, recent, and the mcp server read-only allowlist + instructions. Typed entities (person/place/list): delete mcp/tools/entities; drop the notes.metadata (entity_meta) column from model/service/routes and the knowledge browse service. note_type STAYS — it also marks 'process' notes. Scheduler: event_scheduler -> recurrence_scheduler, keeping only the recurring-task spawn job (drops event reminders + CalDAV sync). Schema: migration 0069 drops the events table + notes.metadata column + orphan caldav settings rows (faithful downgrade recreates them). KEEP: recurrence.py (task recurrence), notifications task reminders, graph view, and every work surface. Frontend + plugin/docs true-up follow next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPtbSzA4JLMAKgFZ8VTg7Q	2026-07-19 13:29:14 -04:00
bvandeusenandClaude Opus 4.8	d6c8470ab2	feat(backup): v3 backup covers rulebooks, rules, events + join tables ... The v2 backup silently dropped the entire rulebook system (rulebooks, topics, rules), the project subscription/suppression join tables, and events — so a 'full' backup wasn't. v3 adds all of them with FK re-mapping on restore, and a _not_included field that names the still-deferred tables (ACL groups/shares, api_keys, embeddings, transient/operational) so the gap is explicit, not silent. restore_full_backup routes v2 and v3 through one path; v3-only sections are guarded by data.get so a v2 payload still restores cleanly. Tests: version/coverage constants, pure join-table row helpers, and the export contract via a mocked session (CI has no DB; full round-trip is a manual check). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-09 23:52:14 -04:00
bvandeusenandClaude Opus 4.8	b255a0f90e	refactor: rename package fabledassistant -> scribe (code-only) ... Renames src/fabledassistant -> src/scribe and all imports, plus the default DB name and DB user/password (fabled -> scribe) in config + compose. 952 refs / 154 files. Reverses the old 'internal name stays fabledassistant' convention. Code-only: live databases are still physically named 'fabledassistant'. Deployed environments must set POSTGRES_DB / POSTGRES_USER (or rename the DB) since the defaults now resolve to 'scribe'. Repo (FabledScribe), git host (fabledsword), MCP (fabled-git) and the image name (fabledscribe) are intentionally unchanged. ruff check src/ clean locally; CI (typecheck + pytest) is the gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 15:48:35 -04:00