Commit Graph
21 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 9006affda8 feat(rules): a rule keeps what it used to say — rule_versions (#3240, milestone 323 step 1)
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Failing after 31s
CI & Build / Python tests (push) Successful in 1m3s
CI & Build / Build & push image (push) Successful in 23s
The sibling notes already had. `note_versions` snapshots a note's every
meaningful edit; a RULE, which binds behaviour on every session that loads
it, had nothing — an edit destroyed the previous wording with no record
anywhere. Rescoping rule 79 meant hand-copying the superseded statement into
a task log to keep it (#3237). The more consequential record had the weaker
protection.

Schema and transport only. Nothing writes a version yet — that is step 2.

Three guards are deliberately NOT copied from note_versions, each defending
against autosave, which rules do not have: no pruning or MAX_VERSIONS, no
pin columns, no minimum interval. A rule is edited a handful of times in its
life, and capping invites losing the one edit somebody needed.

`user_id` is the ACTOR rather than the owner, and SET NULL rather than
CASCADE: deleting a user must not erase the history of the rules they
edited. The restore diverges from its NoteVersion sibling accordingly — an
unmappable user leaves the row with a null actor instead of dropping it,
which is the whole point of choosing SET NULL. The integration round trip
pins that, because nothing in the code says which of the two shapes is
intended and "make it match the sibling" would silently delete the record.

Backup goes to v13. Both export paths carry the table; the per-user one
scopes through the rule rather than the version's user_id, or it would carry
the versions this user wrote on someone else's rule and drop the ones
someone else wrote on theirs. The restore remaps rule_id through
rule_id_map — #3182's arose_from_id trap on a new table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 18:38:16 -04:00
bvandeusen a6ef3a6a5a fix(backup): a restore stops flattening the record vocabulary, and a column guard stops the next one (#3182)
CI & Build / Python lint (push) Successful in 6s
CI & Build / Plugin hooks (push) Successful in 18s
CI & Build / integration (push) Successful in 48s
CI & Build / Python tests (push) Failing after 56s
CI & Build / Build & push image (push) Skipped
CI & Build / TypeScript typecheck (push) Successful in 39s
`_note_rows` carried 16 of the `notes` table's 27 columns. A backup -> restore
cycle reported success and handed back a corpus with every snippet and process
flattened into a plain note, every issue and spike into `work`, every
provenance edge gone, and recurring tasks no longer recurring. The record-type
and kind vocabulary is what #3128 and milestone 312 were about, and a restore
erased it.

Two more found by auditing every row helper rather than only the one being
edited: `_milestone_rows` dropped `body` — a milestone IS the plan (0066), so
every plan restored as a title with no reasoning behind it — and
`_repo_binding_rows` dropped `ref`, the branch a ledger follows (#2873), so a
restored binding silently accounts for a different tree.

`arose_from_id` is deferred to a second pass beside `parent_id`, never written
in the constructor: it is an id in the SOURCE database, so copying it through
lands the edge on whatever record happens to hold that number here. An edge
whose target did not survive stays NULL rather than being guessed at. This is
the trap that kept the fix out of milestone 317 step 1.

THE STRUCTURAL HALF. The coverage guard from #2293 checks TABLES against
Base.metadata; nothing checked COLUMNS, which is how nine went missing from a
table that had been "covered" for years — added to the model and the migration,
both of which fail loudly, and never to the serialiser, which fails silently.
`_COLUMN_EXCLUSIONS` now declares, per table, every column deliberately not
exported and why, and a parameterised guard walks all 23 helpers and asserts
the two agree. Forgetting is no longer expressible.

Reconciling all 23 turned up one more deliberate exclusion worth naming: the
`code_shapes` proposal columns are the machine's standing suggestion, cleared
by judgment and recomputed by every refresh, so carrying them would restore
stale guesses over a tree the proposer has not seen.

Tests: the round trip drives the REAL restore_full_backup against Postgres,
not a reimplementation of its loop — a test that re-derives the remap it is
checking would agree with whatever the product does, including nothing.

Backup v12.
2026-08-28 15:22:50 -04:00
bvandeusen 2263fd04a4 fix(tests): the notes table has 27 columns — deleted_batch_id is the second deliberate exclusion (#3165)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / integration (push) Successful in 26s
CI & Build / TypeScript typecheck (push) Successful in 38s
CI & Build / Python tests (push) Successful in 1m18s
CI & Build / Build & push image (push) Successful in 24s
The pin test caught its own inaccuracy on the first run, which is what it is
for. `deleted_batch_id` comes from SoftDeleteMixin alongside `deleted_at` and
is excluded for the same reason: trashed rows are not exported, so neither is
the batch id that groups them for restore(). The nine-field gap #3182 tracks
is unchanged.
2026-08-28 14:55:44 -04:00
bvandeusen 2065781302 feat(notes): a note can carry its own check — verify_with, expires_when, verified_at (#3165, milestone 317 step 1)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Failing after 45s
CI & Build / Build & push image (push) Skipped
The sibling of migration 0090, one table over. Same distinction: a NORM is a
decision with no truth value; a CONSTRAINT asserts a fact about someone
else's software and goes false with nobody watching. Notes hold far more
constraints than rules do and hold them longer — a cross-project reference
asserting what a signing service does on a duplicate upload is believed by
every project that reads it, and nothing in the record says when anyone last
looked. note_supersessions only fires once a human has already believed it.

Three nullable columns, no backfill, no index. The index margin is thinner
than 0090's — thousands of note rows against hundreds of rules — so the
comment says to decide it in step 3 against a real query plan rather than
guessing here.

The columns land on every row in `notes`, but only non-task, non-snippet
records will be OFFERED them (gated at the service in step 2): a task's decay
is its status, and a snippet already carries a richer location-aware verdict
in data.verification. A schema-level gate would have meant a CHECK across
three columns to say what the write path says in two lines.

Backup carries the trio (v11), with `verified_at` restored through
_dt_or_none — _dt substitutes now(), which would restore every never-checked
note as checked at the moment of the restore, inverting the one signal the
sweep reads.

Found while doing that, NOT fixed here, and now pinned by a test: `_note_rows`
carries 16 of the `notes` table's 26 columns. note_type, task_kind,
arose_from_id, the recurrence pair, the lifecycle stamps, description and data
have all been missing for a long time, so a restore flattens every snippet and
process into a plain note and every issue and spike into `work`. The coverage
guard cannot see it — it checks TABLES, not columns, which is #2293's failure
mode one level down. #3182 tracks it; arose_from_id needs the second
id-remapping pass parent_id gets, which is why it is not a drive-by fix.
2026-08-28 14:54:07 -04:00
bvandeusenandClaude Opus 5 e08e999406 feat(rules): a rule can carry its own check — verify_with, expires_when, verified_at (#3095, milestone 312 step 1)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / integration (push) Successful in 28s
CI & Build / Build & push image (push) Successful in 25s
A rulebook holds two kinds of row in one table. A NORM is a decision: no
truth value, changes only when its author changes it, and they know they
did. A CONSTRAINT asserts a fact about someone else's software, and goes
false with nobody present. Milestone 307's audit found nine stale sites;
every one was a constraint, and not one norm had rotted.

Three nullable columns so a rule can say how to check itself. expires_when
is a STATE, not a date — constraints expire when the ground moves, not on a
schedule. verified_at NULL means never checked and sorts FIRST in the sweep
to come: unexamined outranks examined-long-ago. Most rules set none of the
three; a null verify_with is the marker for "this is a decision, there is
nothing to go and check," and it only reads that way while it stays honest.

Nothing is backfilled and nothing is indexed. A migration cannot invent a
check any more than 0088 could invent a trigger, and the sweep reads a whole
rulebook — hundreds of rows, on operator demand, never on a request path.

Also, in the backup service the fields had to pass through:

- Restore now remaps arose_from_id through note_id_map. It has been exported
  since 0088 and silently dropped on the way back in ever since, so every
  restore lost every rule's provenance link.
- _dt_or_none, because _dt substitutes now() for an absent value. That is
  right for created_at/updated_at and wrong here: a rule nobody ever checked
  would restore looking freshly checked and fall to the bottom of the sweep
  it should top.

Column additions do not move BACKUP_VERSION; only new sections do, as when
0088 added when_to_apply/tier/arose_from_id to the same helper.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 07:44:13 -04:00
bvandeusenandClaude Opus 5 95a37318fc feat(rules): rules become findable by meaning (#3030, milestone 307 step 4)
CI & Build / Python lint (push) Failing after 9s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 44s
CI & Build / integration (push) Successful in 45s
CI & Build / Python tests (push) Successful in 1m26s
CI & Build / Build & push image (push) Skipped
Rules were the only major record type with no vector, so `search` could never
return one and a rule could arrive only by being preloaded. That single fact is
what made every rule compete for one always-on budget.

THE DECISION THE TASK ASKED FOR, made explicitly: a sibling rule_embeddings
table, not a polymorphic embedding row. The ROW could have been generalised;
the SEARCH could not. semantic_search_notes is Note-specific scoping end to end
— the visibility clause, the supersession penalty, note_type/task_kind/system
filters — and a rule shares none of it, scoping instead by rulebook ownership
or project. Generalising the row while still needing two searches is the worst
of both: a key with referential integrity to neither table, on the path every
session start runs, to share four columns. What is genuinely common is
BEHAVIOUR — get_embedding, chunk_document, embedding_text, CHUNKER_VERSION —
and those are reused as-is. Sharing them is the DRY win; sharing the table
would have been the DRY costume.

The document shape is measured, not chosen (note 2485). That pass found the
snippet was the only discriminative record in the corpus — a 0.153
top-to-second gap against 0.010-0.023 — and that the cause was its SHAPE:
purpose stated twice in a short single-topic document. rule_document
reproduces it: the trigger in the title AND as the body's first line.

And it excludes `why`, which matters more than any of it. `why` is dated
incident narrative — rule 46's runs to 4,300 characters — and long multi-topic
prose is exactly what made sixteen dev-logs mutually indistinguishable. Adding
it would not give the vector more to work with; it would give every rule the
SAME thing to work with. rule_document takes no `why` parameter at all, so a
well-meaning caller cannot pass one.

A rule with no trigger degrades to title + statement — findable, less sharp.
That is an argument for backfilling triggers (step 6), not for padding the
document with whatever text is nearby.

search(content_type="rule") returns the rule WITH its why and how_to_apply:
they are its operational half, the session payload never carries them, and a
caller who went looking should not have to re-fetch. Writes re-index
fire-and-forget like notes; startup backfills in its own try block so neither
backfill can skip the other. rule_embeddings is derived, so it joins
note_embeddings in the backup's explicitly-NOT-included list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:00:49 -04:00
bvandeusenandClaude Opus 5 6ddb8bf859 feat(rules): a rule can say when it applies, which area it is about, and what it belongs with (#3029, milestone 307 step 3, schema)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 25s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Failing after 47s
CI & Build / Build & push image (push) Skipped
A rule could not state its trigger, its area, or its siblings, so all three
were being written as prose instead: a System's charter restating rule text,
a `why` naming the note that caused it, and two halves of one shape merged
into a single row because either could surface without the other.

Migration 0088 adds the four fields those workarounds stood in for:

- `when_to_apply` — the trigger. Nullable in the DB and required at the
  service layer: existing rules have none and a migration cannot invent one.
- `tier` — always_on | conditional, defaulting to always_on. This migration
  therefore changes NOTHING about which rules bind; an install upgrades and
  every rule keeps arriving exactly as before. Getting that backwards is the
  one failure this milestone exists to prevent, so _valid_tier falls back to
  always_on rather than silently un-binding a rule with a typo'd tier.
- `arose_from_id` — the record that caused the rule, the edge notes and tasks
  already have. SET NULL: trashing the source does not repeal the rule.
- `rule_systems` / `rule_relations` — the canon tag and the typed edges
  (co_surfaces / overrides / elaborates), each earned from a workaround its
  absence forced.

rule_brief() replaces the THREE hand-written trim dicts that had already
diverged — two carried topic_id, one didn't, and none carried the timestamps
the model has held all along. That omission is why a rule written before the
capability it duplicates was indistinguishable at read time from one still
doing work. It now carries updated_at as a DATE: the question is "how old is
this", and a full stamp across the always-on set is ~2k characters for
precision nobody reads. The two callers select the ENTITY rather than a column
list, so rule_brief stays the single place deciding what a surfaced rule says.

Backup: both new tables carried, area tags by canonical SLUG (ids are
per-install). The rule-relation restore runs after ALL rules exist and after
the catalog, because an edge names two rules and a tag names a global row —
sections renumbered so the file reads in dependency order. A pre-0088 payload
restores with tier=always_on, i.e. binding exactly as when it was taken.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 14:11:32 -04:00
bvandeusenandClaude Opus 5 a97547fbc6 feat(systems): the area vocabulary becomes a global table so a rule can point at one (#3027, milestone 307 step 1)
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 39s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 30s
CI & Build / Python tests (push) Failing after 56s
CI & Build / Build & push image (push) Skipped
The eight standard area names already existed — as STANDARD_SYSTEMS, a tuple in
services/systems.py that milestone 297 seeds at inception. A constant cannot be
a foreign key, so nothing outside a project could reference an area: systems.
project_id is NOT NULL, and a rule that spans projects would have to chain
itself to one project's row. And because the list only ever applied on the
inception-seed path, three spellings of one area reached this instance anyway
(CI & runners / CI and Release / CI & release).

- canonical_systems: global, no user_id — a shared project inherits the
  vocabulary instead of re-earning it. Migration 0087 seeds the same eight.
- systems.canonical_id: nullable, SET NULL. Association only — no System is
  renamed and record_systems is untouched, so no record's tags move.
- canonical_slug folds &/and, case and punctuation, so spelling variants map
  mechanically and a real difference ("CI & runners") becomes a proposal a
  human confirms. propose_mappings reports; set_system_canonical is the only
  writer.
- seed_standard_systems now reads the catalog and maps as it mints, so a
  project born standard never needs a reconciliation pass.
- Catalog writes are admin-only; reads are open — a global list anyone can
  extend stops being shared.
- backup: carried by SLUG, not id (ids are per-install). Restore reuses the
  target's own rows and only creates entries an admin added on the source; an
  unknown slug restores unmapped rather than failing.

Rule 22: STANDARD_SYSTEMS is removed, not deprecated. Rule 115: nothing seeded
names an app, repo or house convention. Design in note 3026.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 12:31:18 -04:00
bvandeusenandClaude Fable 5 ffbdf19116 feat(ledger): the CSS consumer map — code_shape_consumers edges (shape → file whose markup names the class, count), migration 0086, resolve_consumers (own-file row when the template defines the class, else every other definition), sync_repo_consumers rebuilt from the archive on every refresh, consumers_of; derived, so not backed up (milestone 302 step 2, #2935)
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Failing after 29s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Failing after 52s
CI & Build / Build & push image (push) Skipped
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 13:59:55 -04:00
bvandeusenandClaude Fable 5 e9b8f525c8 feat(inception): projects.inception record + project_rulebook_exclusions — migration 0085 with legacy backfill; backup v10 (#2879, milestone 297 step 1)
CI & Build / Python lint (push) Successful in 6s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 42s
CI & Build / integration (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m36s
CI & Build / Build & push image (push) Successful in 36s
A project's inheritance becomes a decision, not a default (milestone 297).
- projects.inception (JSONB, NULL = undecided): {decided_at, decided_by, via
  mcp|ui|legacy, choices {exclude_always_on_rulebooks, subscribe_rulebooks,
  design_system_id, seed_systems}}; on to_dict.
- project_rulebook_exclusions: a project's opt-out of a whole always-on
  rulebook — the sibling of the rule/topic suppressions, CASCADE both ways.
- services/inception.py (first cut): the vocabulary, validate_inception
  (pure, all-or-nothing), normalize_choices, is_decided. Effects come in
  step 3.
- Migration 0085 backfills every existing project via="legacy" with its
  current standing (no exclusions, its subscriptions, its design_system_id,
  no seed) so the ask fires only for projects created after this ships.
- Backup v10: rulebook_exclusions section; project rows carry inception and
  design_system_id, restored in a post-pass once rulebooks/design systems are
  mapped (design_system_id was not restored before — fixed in passing).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 21:57:02 -04:00
bvandeusenandClaude Fable 5 6a0f8ad328 feat(backup): code_shape_uses travels (v9); consumer-map test expects uses (#2870)
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 22s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 25s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 15:21:13 -04:00
bvandeusenandClaude Fable 5 7d48eb0b1b refactor(services): one periodic-task shape, one APScheduler job shape, one token hash, one summary rule — the services pass of the shape audit (#2830, milestone 296)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / TypeScript typecheck (push) Successful in 12s
CI & Build / integration (push) Successful in 24s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 24s
- background.start_periodic(interval, work, label=) replaces the three hand-rolled
  while-True/sleep/try loops in logging, auth and notifications.
- services/scheduler.ScheduledJob replaces the four private BackgroundScheduler
  copies in recurrence/version_pinning/trash/db_maintenance schedulers; public
  start_/stop_/reschedule_ surfaces unchanged.
- api_keys.hash_token is the one sha256 helper; auth.py used to inline it 5x.
- auth.is_registration_open reads via settings.get_admin_setting; notification
  prefs read via settings.get_setting; _fire_share_email uses _get_user_email.
- projects.get_project_summary / milestones.get_project_milestone_summary are
  now the one-id view of their batch siblings instead of a second copy of the
  queries; sharing.best_permission_by (was _deduplicate_by_permission) is the
  one rank-dedup, now also used by list_projects_for_user.
- backup: the row builders for every section both exporters carry are named
  functions, so a column added to one export cannot silently miss the other.
- iso() from models.base replaces the attr.isoformat()-if-attr-else-None idiom
  and db_maintenance._iso across services; backup keeps its explicit shape.
- trash.py hoists the sql_delete/timedelta imports it re-imported per function.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 12:34:28 -04:00
bvandeusenandClaude Fable 5 d74a244b3a feat(ledger): divergence readout — button B where button A is canon, shape history, and judged-shape recheck (#2793, milestone 294 step 7)
CI & Build / TypeScript typecheck (push) Failing after 2s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 28s
CI & Build / Python tests (push) Failing after 37s
CI & Build / Build & push image (push) Skipped
Every judgment now goes through one helper that remembers the fingerprint
judged (classified_sha) and writes a code_shape_events row; the sync writes
vanished / reappeared / drifted events and flags recheck_at when a body
moves under an instance/variant. The refresh flags diverges_from on shapes
new since the previous computation that sit where one canon dominates the
judged siblings of their directory+kind and were not proposed as that canon
(a first seed flags nothing); the write-path hint asks the same question
in-band for the shapes the hook names. list_shapes(flag=divergence|recheck),
shape_history(project_id, path, symbol) (read-only), coverage line/payload/
card carry divergent + recheck. Backup v8 carries the history. Plugin 0.1.36.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 23:13:39 -04:00
bvandeusenandClaude Fable 5 ba0030e51d feat(ledger): mechanical proposer — every refresh proposes instances against canon and groups derive-first candidates; agents confirm in batches (#2792, milestone 294 step 6)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / integration (push) Failing after 34s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python tests (push) Successful in 1m17s
CI & Build / Build & push image (push) Successful in 48s
Shapes now carry a content fingerprint (signature + whitespace/comment-
insensitive body_sha; migration 0080) and the proposer runs inside the
coverage refresh, the one moment bodies exist: symbol elsewhere → textual
containment → body references the canon → signature resemblance → semantic
(capped per refresh, unreached rows stay unexamined for the next). A hit is
a proposal on the row (proposed_snippet_id/basis/score), never a
classification; rows with no canon hit group by the derive-first rule
(identical body in ≥2 places, same name in ≥3 files) as proposal_basis=
derive + a group key. list_shapes(proposal=any|canon|derive|<basis>) is the
queue; confirm_shape_proposals(project_id, snippet_id|path|basis) confirms
in batches as agent instances; any classify_shapes/hook stamp retires the
proposal. Readout carries proposed + derive_groups (line, payload, card).
Plugin 0.1.35 (skill: the machine proposes, judgment classifies).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 21:30:35 -04:00
bvandeusenandClaude Fable 5 19fdc9aa89 feat(ledger): code_shapes — the shape ledger schema (#2787, milestone 294 step 1)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 13s
CI & Build / integration (push) Successful in 25s
CI & Build / Python tests (push) Successful in 54s
CI & Build / Build & push image (push) Successful in 45s
The accounting half of the pattern system (governing note 2786): the snippet
library records canon (small), this table accounts for EVERY extracted shape
(total). Identity is (project, repo_key, path, symbol, kind) — kind included
because one file can define '.foo' (css) and 'foo' (sym) as distinct shapes.
Status vocabulary: canonical / instance / variant / exempt / unclassified,
with unclassified as the default and THE todo state; classifications carry
who judged (agent|audit|hook|mechanical|import), when, and the why for
variants/exemptions. first/last-seen commits + vanished_at keep history
instead of deleting it; a rename reads as vanish+new (accepted for v1).

snippet_id is SET NULL on snippet deletion so accounting rows outlive their
target and rejoin the todo via the step-2 sync, never dangle silently.

Backups: v7 carries code_shapes (judgment data, worth moving) — full and
per-user export sections, and a restore that keeps a judgment only when its
snippet survives the id re-mapping, downgrading to unclassified otherwise.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 19:25:25 -04:00
bvandeusenandClaude Fable 5 1faf8f3ece feat(forge): per-user forge connections — keyring, host-keyed resolution, project pin (#2778)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / TypeScript typecheck (push) Successful in 41s
CI & Build / integration (push) Successful in 37s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Successful in 40s
A forge token is a user's credential, not an instance's. The single
admin-settings config is replaced by per-user keyring rows (one per forge
host), and every server-side forge read runs on the PROJECT OWNER's keyring:

- forge_connections table + projects.forge_connection_id pin (migration 0078,
  which also carries the existing admin config into the first admin's row and
  deletes the old setting keys — no legacy dual-read)
- get_forge() replaced by get_forges(owner_id, project_id) -> ForgeSelector;
  resolve(repo) picks the connection whose host serves the repo. A pinned
  project uses ONLY its pinned connection; a stale pin (ownership moved) is
  ignored, never honored across users
- env FORGE_* config survives as an implicit entry for admin owners only;
  a stored row for the same host beats it
- consumers threaded: pull-time freshness (owner of the note), coverage
  (owner of the project), coverage routes' configured flag
- routes: /api/settings/forge-connections CRUD + per-connection test
  (own-rows only, tokens never returned); /api/admin/forge shrinks to
  /api/admin/forge-webhook (secret only); PUT /api/projects/<id>/forge pins,
  owner-or-admin asking, owner's connections only
- UI: Git Forges card moves to Settings -> Integrations as a connection
  list; webhook secret stays in the admin Config tab; owner-only forge
  select on the project coverage card
- backups exclude forge_connections (credentials, api_keys precedent) and
  the pin, so restores fall back to keyring resolution

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 11:23:22 -04:00
bvandeusen 5dcb738ce8 fix(backup): carry note_supersessions — the coverage guard caught the omission
CI & Build / integration (push) Successful in 13s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 48s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Build & push image (push) Successful in 36s
CI failed on 45c6b1c. `tests/test_services_backup.py` asserts every table is
either in `_BACKED_UP` or explicitly in `_NOT_INCLUDED`, and the new table was
in neither.

That guard exists for exactly this: a new table gets a model and a migration —
both of which fail loudly if wrong — and then silently never gets a backup
section. No error, no warning, and a restore that reports success. Its comment
says it was written to "stop the seventh" table slipping through. This was the
seventh.

Supersession claims are backed up rather than excluded because they are a
JUDGEMENT. Someone decided this note replaced that one, and nothing in either
note's text records the decision. Lose them and a restored corpus silently
returns to ranking stale material alongside current material — with no symptom
that says why.

Scoping differs by backup kind, deliberately. A whole-instance export takes
every row. A single-user export takes only claims where BOTH ends are that
user's notes: a claim spanning out to someone else's record cannot be restored
into a single-user import, since the far id is not in the map, so exporting it
would write a row that silently vanishes on the way back in.

Restore maps both ids and skips the row unless both resolve. A claim is about a
PAIR — half of one is not a weaker claim, it is a dangling row pointing at
whatever note holds that id next. Guarded by `data.get` like every post-v2
section, so v5 and older payloads restore cleanly without it.

BACKUP_VERSION 5 -> 6, and the version test moved with it. A payload section
added without moving the version produces backups that are structurally
different and indistinguishable by inspection.

Refs #278
2026-08-07 21:49:08 -04:00
bvandeusenandClaude Opus 5 84541f392b fix(backup): six tables were silently absent, and nothing would catch a seventh
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / integration (push) Successful in 24s
CI & Build / TypeScript typecheck (push) Successful in 25s
CI & Build / Python tests (push) Failing after 38s
CI & Build / Build & push image (push) Skipped
services/backup.py enumerated its tables as hand-maintained literals with
nothing tying them to the schema. Tables added since that list was last
extended were absent from every backup — no error, no warning, and a restore
that reports success.

Missing: systems, record_systems (0065), note_usage_events (0071),
design_systems, design_tokens (0072), and repo_bindings — which the issue
itself had not spotted, found only by diffing the model tablenames against the
two lists instead of trusting either.

_NOT_INCLUDED was worse than incomplete: it named "embeddings", "invitations"
and "password_resets", none of which are tables. It read as coverage while
naming nothing the schema could confirm. Now real names, plus retrieval_logs —
observational telemetry that grows per query and that nothing reads for
correctness.

THE DELIVERABLE IS THE GUARD, not the six sections. Extending a list fixes
today and changes nothing about the next table; a new one now fails a test
until someone either backs it up or states that it shouldn't be. It checks
both directions — an unaccounted table, and a listed name that no longer
exists, which is what the three phantom entries above would have tripped.

Design systems need ordering care: parent_id is a self-FK. The export orders
parent-first (parent_id NULLS FIRST, then id — a parent always has the smaller
id), so restore resolves each parent from the map as it goes, with no second
pass. A child whose parent is missing lands as a root rather than failing the
whole restore.

Usage events are kept because pull-through is the evidence base for whether
recall works, and it only ever accumulates — a restore that dropped it would
reset that measurement to zero while everything still looked fine.

BACKUP_VERSION 4 -> 5. Every new restore section is data.get()-guarded, so
v2/v3/v4 payloads restore unchanged.

Closes #2293.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-07-31 23:09:25 -04:00
bvandeusenandClaude Opus 4.8 b49efdcb11 refactor(scribe): retire calendar/events + person/place/list entities (backend)
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / integration (push) Successful in 30s
CI & Build / Python tests (push) Failing after 31s
CI & Build / Build & push image (push) Has been skipped
Narrow Scribe to a Claude-Code work system-of-record (milestone #194,
decision note #1759). Wholesale removal per rule #22 — backend + schema half.

Calendar/events + CalDAV: delete models/event, services/{events,caldav,
caldav_sync}, routes/events, mcp/tools/events; strip event branches from
backup (bump v3->v4), dashboard (upcoming_events), trash, recent, and the
mcp server read-only allowlist + instructions.

Typed entities (person/place/list): delete mcp/tools/entities; drop the
notes.metadata (entity_meta) column from model/service/routes and the
knowledge browse service. note_type STAYS — it also marks 'process' notes.

Scheduler: event_scheduler -> recurrence_scheduler, keeping only the
recurring-task spawn job (drops event reminders + CalDAV sync).

Schema: migration 0069 drops the events table + notes.metadata column +
orphan caldav settings rows (faithful downgrade recreates them).

KEEP: recurrence.py (task recurrence), notifications task reminders, graph
view, and every work surface. Frontend + plugin/docs true-up follow next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BPtbSzA4JLMAKgFZ8VTg7Q
2026-07-19 13:29:14 -04:00
bvandeusenandClaude Opus 4.8 d6c8470ab2 feat(backup): v3 backup covers rulebooks, rules, events + join tables
The v2 backup silently dropped the entire rulebook system (rulebooks, topics,
rules), the project subscription/suppression join tables, and events — so a
'full' backup wasn't. v3 adds all of them with FK re-mapping on restore, and a
_not_included field that names the still-deferred tables (ACL groups/shares,
api_keys, embeddings, transient/operational) so the gap is explicit, not silent.

restore_full_backup routes v2 and v3 through one path; v3-only sections are
guarded by data.get so a v2 payload still restores cleanly.

Tests: version/coverage constants, pure join-table row helpers, and the export
contract via a mocked session (CI has no DB; full round-trip is a manual check).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:52:14 -04:00
bvandeusenandClaude Opus 4.8 b255a0f90e refactor: rename package fabledassistant -> scribe (code-only)
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python tests (push) Successful in 54s
CI & Build / Build & push image (push) Successful in 1m14s
Renames src/fabledassistant -> src/scribe and all imports, plus the
default DB name and DB user/password (fabled -> scribe) in config +
compose. 952 refs / 154 files. Reverses the old 'internal name stays
fabledassistant' convention.

Code-only: live databases are still physically named 'fabledassistant'.
Deployed environments must set POSTGRES_DB / POSTGRES_USER (or rename the
DB) since the defaults now resolve to 'scribe'. Repo (FabledScribe), git
host (fabledsword), MCP (fabled-git) and the image name (fabledscribe)
are intentionally unchanged.

ruff check src/ clean locally; CI (typecheck + pytest) is the gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 15:48:35 -04:00