99 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Fable 5	cbccb6bd5d	feat(coverage): pattern-library coverage measurement (#2692, milestone 288 step 7) ... Server-side shape enumeration per bound repo — one archive download via the forge adapter, definitions extracted with a Python mirror of the write-path hook's awk rules (shared test vectors pin the two together) — compared against recorded snippet locations by path+symbol. Summary is cached in the settings KV with a freshness stamp; recomputed on webhook push (spawned off the delivery path) or explicit refresh, never in a request path. Surfaces: GET/POST /api/projects/<id>/coverage[/refresh], a project-page card (estimate-labeled, largest-gaps chips), and a one-line evidence-carrying entry in enter_project read from cache only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 16:05:43 -04:00
bvandeusenandClaude Fable 5	89b07f7857	feat(forge): push webhook flags drift at the moment the repo moves (#2691) ... Second adapter consumer. POST /api/webhooks/forge validates Gitea's X-Gitea-Signature (HMAC-SHA256, constant-time; no secret configured = the endpoint 404s out of existence), extracts changed/removed paths, and flags matched snippets by writing verification.invalidated_by {commit_sha, at, path, removed} — the existing attention vocabulary extended, not a new flag: needs_attention includes it, both filter dialects (Python + jsonpath SQL) include it in 'attention' and exclude it from 'ok', and recording ANY fresh verdict clears it by construction because compose_verification builds a new dict. Unverified snippets are skipped (already in their own bucket); replayed deliveries at the same head commit are no-ops; processing failures return 200 with a WARNING + AppLog canary so the forge never marks deliveries failed and operators never disable the hook over a transient (#2663's lesson). Matching goes through repo BINDINGS: recorded location repos are free-form names ('Scribe') that cannot address a forge, so a snippet reaches its forge repo through its project's binding — which also fixes step 5's pull-time resolution for every real record via the same fallback. O(bindings + snippets-in-project + changed files). Settings: webhook secret beside the forge config (masked, sentinel- skipped, Docker-secret env channel, endpoint documented in the UI). Tests: signature gate, payload parsing, path semantics, both filter dialects extended in the drift-check guard file, and real-Postgres end-to-end (flag lands, attention lists it, replay quiet, re-verify clears, unbound repo untouched). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 13:05:00 -04:00
bvandeusenandClaude Fable 5	eb760eb440	fix(snippets): read the stored provenance before writing the fresh stamp into the response ... fields aliases data['snippet'], so stamping the response first made the staleness check compare the new stamp to itself — the background persist never fired. Caught by test_current_code_confirms_and_refreshes_provenance on run 3811, which exists for exactly this write. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:53:34 -04:00
bvandeusenandClaude Fable 5	2fce57847b	feat(snippets): pull-time freshness — the forge confirms the cache at the moment it's trusted (#2690) ... First consumer of the forge adapter. attach_live_body decorates both pull surfaces (MCP get_snippet, REST detail) with body_source + body_freshness when the instance has a forge: 'current' means the cached code was just found verbatim (whitespace-normalized, the same normalization the verdict hash uses) in the fetched file, and provenance restamps to the file's last commit — reflected in the response and persisted in the background. A snippet body is a FRAGMENT of its file, so a fetch can honestly CONFIRM the cache or report divergence, never clobber the record with the whole file: 'diverged' is the reader's information, and a 404 stamps the mechanically-true 'missing' verdict into the existing attention state — once, not on every pull of an already-flagged record. The probe never raises and never blocks past 2.5s (tighter than the adapter's own timeout — the pull is where a session decides whether pulling is worth it, #2663's finding); a hung forge costs bounded time and the cache serves. A no-forge instance's response stays byte-identical to today's (rule #115 baseline, pinned by test). services/background.py is the new one home for fire-and-forget tasks with strong references (the #2663 GC footgun) — telemetry's two copies predate it and keep their bespoke canaries; new callers use this. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:46:45 -04:00
bvandeusenandClaude Fable 5	13e428c596	feat(forge): adapter seam + Gitea implementation — optional read access to the operator's forge (#2689) ... Step 4 of milestone 288 (decision #2686). services/forge.py defines the contract steps 5-7 consume — read_file (content + last_commit_sha, the provenance stamp), default_branch, resolve_repo, check — with GiteaForge as the first implementation over the REST contents/repo/version/user endpoints. Repo identity reuses normalize_repo_key: the host segment selects whether this forge serves a recorded repo, the remainder is the API path, so no new identity scheme exists. Read-only by construction; errors never carry the token; first outbound-HTTP timeout convention (5s total, no retries — the consumer's fallback is the retry policy). OPTIONAL per instance (rule #115): get_forge() returns None when unconfigured and every consumer treats None as today's behavior. Config lives in admin settings (Settings → Config → Git Forge: kind/base URL/token, save + test-connection probe reporting version + identity), with FORGE_* env / Docker-secret fallbacks; DB wins so a UI edit can't silently lose to an env var. Token treatment follows the smtp_password convention (masked on read, mask-sentinel skipped on write, absent from audit details) — and wiring it surfaced that the generic GET/PUT /api/settings dump bypassed that masking for the owning admin's raw KV rows, so secret keys are now masked there too (fixes the same exposure for smtp_password). Contract tests run against httpx.MockTransport as the fake forge — the reference behaviors the GitHub adapter (step 8) must reproduce — plus the off-by-default gate, partial-config-is-off, env-vs-DB precedence, and route/mask structural checks. Also: the step-2 definition detector learned to skip dunders after flagging __init__ as 'already defined in 4 files' on this step's own build — guaranteed noise for a hint that must stay trustworthy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:37:27 -04:00
bvandeusenandClaude Fable 5	1e7f66e72d	feat(snippets): body provenance — the cache-with-provenance half of the pointer model (#2688) ... Decision #2686: the recorded location is the source of truth for a snippet's code; the stored body is a cache of it. data.provenance now records what the cache is a cache OF — commit_sha + fetched_at — as a carried JSONB field following the verification precedent, so no migration is needed and absence keeps today's exact semantics. The rules: provenance follows the code (fresh SHA restamps it, a code edit without one drops it, a metadata edit carries it); writes ABOUT the code carry it — record_verification rebuilds data from scratch and would otherwise erase it silently; an ok verdict at a known commit restamps it, since the checker just proved the cache matches the source there. verify_snippet verdicts also record the commit they ran at, making "the repo moved on since the check" computable once the forge integration lands. create/update/verify MCP tools take commit_sha (git rev-parse HEAD — free for any session with a checkout). Unit tests pin the compose/carry logic; real-Postgres integration tests run create→verify→update end-to-end (#2663: DB paths get no mocked-only coverage). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:00:40 -04:00
bvandeusenandClaude Fable 5	4107b17727	fix(telemetry): usage readout grouped by a rebuilt CASE — group by the label instead (#2663 root cause) ... CI's new integration tests reproduced the outage on a clean database and named the half: the writes land fine, and usage_for_notes fails on EVERY call. The GROUP BY rebuilt the ambient case() expression, and asyncpg's expanding IN-parameters give each instance its own bind names — so Postgres sees a SELECT expression the GROUP BY doesn't cover and rejects the query with a GroupingError, which the old code swallowed into zeros. One labelled expression, grouped by its label. The deployed table has been accumulating events all along; history appears as soon as this deploys. Also: the two hook-execution tests now run with the real PATH and skip when the hook's tools are absent (the restricted-PATH convention next door is for silence contracts, where empty-for-the-wrong-reason still passes) — and the unit lane installs jq so 'skip' never quietly becomes 'nowhere'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-14 21:49:48 -04:00
bvandeusenandClaude Fable 5	77acee9239	fix(telemetry): usage counters get canaries, task references, and the missing integration tests (#2663) ... The deployed instance ran with every usage counter at zero while surfacing demonstrably fired. Every unit test was green because every unit test mocked either _schedule or the session — the two functions that touch the database ran against real Postgres nowhere. Both telemetry writers also held no reference to their fire-and-forget tasks (the loop keeps only weak ones), and swallowed every failure into logger.debug, so a total outage was indistinguishable from an unused corpus. - note_usage + retrieval_telemetry keep strong task references until done - failures log at WARNING; note_usage additionally drops one AppLog error row per process per site, so the admin UI shows the outage without host access - integration tests cover _insert_events -> usage_for_notes and the full record_pulled chain on a running loop, splitting the write and read halves so a failure names its side Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-14 21:45:32 -04:00
bvandeusenandClaude Fable 5	9c68faa0bd	feat(processes): composition contract — the process is the skeleton, the conversation supplies the parameters ... Extends #2582's fix per the operator's direction: a stored Process must incorporate the context it was triggered with, not displace it. No new plumbing needed — the live context is already in the session; what was missing is the stated contract. get_process now carries it: live constraints/scope/focus fold into the procedure and override its defaults where they disagree; a clarify step asks only what the conversation has NOT already answered (confirm interpretations, don't re-ask); stated concerns become lenses the procedure applies, not text it discards. The sync-generated skill stubs state the short form. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-10 08:39:02 -04:00
bvandeusenandClaude Fable 5	66c4fd1b88	fix(sync): process-skill triggers invoke by name only — resemblance offers, never substitutes ... Scribe issue #2582: the sync wrote "Use when {title}-type work is requested" into every generated process skill, so a bespoke live prompt that merely resembled a stored Process got the Process's canned procedure — and inherited approvals embedded in its body (the Drift Audit's fan-out opt-in) that the conversation never granted. Observed live: an explicit systems-review request with its own constraints was answered with the Drift Audit clarify menu recommending ~15 parallel auditors. New trigger contract, both variants: follow verbatim only when invoked by name; on mere resemblance the live instructions govern — offer the process, ask before following, and never inherit embedded approvals the operator hasn't granted in this conversation. (The Drift Audit process record itself was also rewritten to a sequential in-context walk — data change, live already, recorded in the same issue.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-10 08:33:40 -04:00
bvandeusenandClaude Fable 5	272b7dbddf	feat(dedup): per-kind duplicate-report floors — notes/tasks default 0.93 ... At chunk grain (#280) a note-pair's similarity is its closest chunk pair, so the shared 0.82 floor saturated the note/task reports with related families (38 note / 155 task groups against the 200-pair cap, measured 2026-08-09). Split kb_duplicate_threshold into per-kind settings keys with per-kind defaults: snippet 0.82 (single-chunk, scale unchanged), note/task 0.93 (points the report at genuinely-alike records). Settings UI grows the two new knobs; report entrypoints inherit the change via get_duplicate_threshold(user_id, kind). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-09 10:33:11 -04:00
bvandeusenandClaude Fable 5	23cb396e65	fix(recurrence): drop the now-unused child binding ... The spawn's explicit embed call went with #280 step 3; the create_note result had no other reader. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:52:33 -04:00
bvandeusenandClaude Fable 5	041d8defbc	feat(embeddings): best-chunk-per-note on every retrieval surface (#280 step 4) ... A note's relevance is now its best chunk's similarity, everywhere: - semantic_search_notes keeps the indexed raw-distance top-k and over-fetches chunk rows (x4, composing with the x3 supersession over-fetch), then collapses to first-appearance-per-note — rows arrive distance-ordered, so first is best. Every ranked consumer (MCP/REST search, Browse, auto-inject, write-path, gate) inherits through the one function. - list_notes semantic q swaps its join for a correlated MIN-distance subquery — the join would have repeated a long note once per matching chunk and made total count chunks. - the duplicate report groups its self-join by note pair on MIN(distance): pair similarity = closest chunk pair, and the < join now also drops cross-chunk self-pairs that would flag every long note against itself. - the write gate queries once per chunk of the candidate (capped at 8), so a note duplicating an existing record in ONE SECTION is caught — the whole-document query diluted exactly the section that mattered. Integration test now seeds a two-chunk note and pins the collapse against real pgvector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:51:01 -04:00
bvandeusenandClaude Fable 5	0e70a3896b	feat(embeddings): per-chunk rows — schema, write path, version-aware backfill (#280 steps 2+3) ... note_embeddings becomes one row per chunk: PK (note_id, chunk_index), plus chunk_text (what this vector actually encodes) and chunker_version. Migration 0077 clears the table — embeddings are derived (0067 precedent) and the old whole-document rows are indistinguishable from single-chunk notes, so the startup backfill regenerates the corpus at the new shape. The backfill is now version-aware: a future shape change is a CHUNKER_VERSION bump that re-embeds exactly the stale notes, not another wipe. upsert_note_embedding takes (title, body) and chunks internally — one path for the write path, the recurrence spawn and the backfill. The recurrence spawn's own embed call is deleted outright: create_note already embeds via embed_note (#2056), so the spawn was a second copy of the rule. An emptied record now CLEARS its stale vectors instead of leaving them findable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:47:34 -04:00
bvandeusenandClaude Fable 5	6b5043a69c	feat(embeddings): chunk_document — the chunked document shape (#280 step 1) ... bge-small reads 512 tokens and fastembed truncates silently, so a single whole-document vector permanently lost everything past ~400 words. The new shape: split at markdown headings (fence-aware), merge small sections, split oversize ones at paragraph boundaries, title-anchor every chunk, repeat the section heading on continuation pieces. A record that fits the window yields exactly one chunk identical to the historical title\nbody shape, so the corpus's sharpest records are byte-for-byte unaffected. CHUNKER_VERSION added so later shape changes re-embed by version comparison instead of a table wipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:43:51 -04:00
bvandeusen	2d1e26f38f	feat(telemetry): ambient surfacings count, apart — enter_project and the skill sync emit ... #2477, option (a) as decided, with the readout changed in the same commit. ## The two silent surfaces enter_project returns open tasks + recent notes on every project entry — probably the largest surfacing by volume — and emitted nothing, so the pulls it caused floated unattributed and the surfaced:pulled ratio ran against a denominator missing its biggest contributor. Now source "enter_project". build_process_manifest installs every reachable Process as an auto-surfacing skill on the operator's machine — its own docstring calls it the most consequential passive surface Scribe has — and emitted nothing, so a Process matched on every relevant turn and never opened was indistinguishable from one never installed. Now source "process_skill_sync": the honest event is "installed", which is a surfacing in effect since the description sits in front of the model each session. ## The readout, same commit — the condition option (a) carried Both surfaces are AMBIENT: top-N-by-recency and install-everything are not ranked choices. Pooling them into surfaced_count would make a note's number dominated by "recently updated in a project you opened", and dead-weight detection would read that as popularity — the wrong number read confidently, which is the corrupts-data tier the survey ranked above everything else. So usage_for_notes splits: surfaced_count stays RANKED-ONLY (every existing consumer's reading — "surfaced often, never pulled → dead weight" — keeps meaning what it meant), and ambient_count is new. Classified in SQL via a CASE on AMBIENT_SOURCES so the group count stays three rows per note, not one per distinct source. Pulls stay pooled: "did anyone ever open this?" does not depend on how it was found. #1038 and #2085 read agent pulls and ranked surfacings; both are unaffected by ambient volume, which is the point. Refs #2477	2026-08-08 22:39:13 -04:00
bvandeusen	45ba4aab25	fix(telemetry): the two invisible retrievals log, and /api/search takes scope ... #2463, the remaining findings (finding 3 landed with f11a547). ## The reuse slot logs (source: reuse_slot) A real semantic query competing for an auto-inject menu slot, and the ledger was asymmetric: the scored hit it DISPLACED was in retrieval_logs, the query that displaced it was not — so the slot could never be evaluated against what it replaced. #1038 and #2085 are gated on this ledger being complete. ## Browse search logs (source: browse_search) The human's main search surface, and it logged nothing — so retrieval_logs claimed the web's search was /api/search. Measured while fixing: /api/search has ZERO frontend consumers; the web UI searches through /api/knowledge exclusively. The table wasn't just under-describing the UI, it was describing a surface the UI never touches. The task's check — does folding human queries into the corpus skew the precision signal thresholds are tuned against? — is answered by the source column: distinct values (browse_search, reuse_slot beside the existing four) mean tuning includes or excludes human traffic deliberately rather than by accident. Same resolution as the mcp_/rest_ split in note_usage_events. ## /api/search takes project_id (and logs it) The route logged project_id=None unconditionally while the MCP tool insists the agent pass the active project. Now optional, default global: with no frontend consumer, this route serves API callers, and an API caller states its scope explicitly — the default-scope UI decision the task flagged is moot until a UI actually consumes the route, which is recorded rather than guessed. Refs #2463	2026-08-08 20:00:22 -04:00
bvandeusen	f11a547cd2	fix(lists): shared-project records appear, and a list's q means what search means ... #2462, both decided halves. ## The ACL defect list_notes filtered on Note.user_id == user_id with no scope parameter at all — never a per-call decision, the capability was absent. query_knowledge beside it was deliberately browse-scoped with a comment saying why. So a task in a shared project was invisible in list_tasks, enter_project's open-task list, the SessionStart todo count and the web UI's task views, while the same project's notes and snippets appeared. Now the shared clause: notes_visibility_clause(user_id, "browse"). Browse and not read, per decision #2094 — an ambient list must never surface a record someone shared one-to-one; those stay search-only. This is the remaining half of a fix made twice before (#2159 widened fetch, #2092 widened meaning), and the guard below is what stops a fourth half appearing. Guarded by source inspection in test_retrieval_scopes: every list-shaped service references a shared visibility clause AND carries no bare Note.user_id comparison that would quietly re-narrow it. An owner-only list returns correct-looking rows and simply omits the shared ones — the shape no behavioural test catches. ## q is semantic (operator: "make it match") The UI's note list keyword-matched while the UI's Browse search semantic-matched, over the same records, with nothing saying so. Now one meaning: q joins the embedding index and orders by cosine distance at the interactive floor, with every lifecycle filter still applied in the same indexed query. Relevance ordering wins over `sort` when q is present — a query is a relevance claim, and sorting its results by date would shuffle the answer. ILIKE survives only as the embedder-down fallback: degraded, never empty. Stated position: superseded records are NOT demoted in list-q. The penalty reorders a top-k; reordering a paginated, counted list would make page boundaries lie. The search surfaces carry the demotion. ## The interactive floor becomes one constant INTERACTIVE_SEARCH_THRESHOLD = 0.3 in embeddings.py, consumed by routes/search.py (was a commented constant), knowledge.py (was a bare literal), and the new list-q. This closes #2463's finding 3 early — the same number lived in two files with the reasoning attached to only one. ## Deferred, deliberately The return-shape unification (ORM objects vs dicts) stays undone. It is a 14-caller refactor whose motivation — callers being unable to swap paths — shrinks now that both paths share the clause and the meaning of q. If swap pressure recurs it deserves its own change, not a rider on an ACL fix. Refs #2462, #2463	2026-08-08 19:58:01 -04:00
bvandeusen	d7039dc17c	feat(dedup): the duplicate report reaches notes and tasks, with per-kind cures ... Step 5 of #278, folding in #2534. The operator's no-gate decision for the web UI (#2482 — "an llm attached to this surface is the corrections system") has a precondition nobody had built: the corrector has to be able to SEE what needs correcting. find_duplicate_snippets had no equivalent for notes or tasks, so a duplicate note was only ever noticed by accident. find_duplicate_records(kind="snippet"|"note"|"task") — the same indexed self-join, parameterised. Tasks are notes with a status, not a note_type, so the kind split is a status predicate; mixing them would propose folding a to-do into a write-up. find_duplicate_snippets stays as a wrapper because both surfaces and SnippetListView consume it by name. What differs by kind is the CURE, and the report says so in a `suggestion` field rather than leaving the caller to guess: snippet merge — lossless, the survivor keeps every call site note NEVER merge. A correction pair → supersedes on the newer; state smeared across dated records → extract to the System's reference note; genuinely parallel → leave alone. Choosing needs the records READ, which is the agent's job — so non-snippet groups carry `members` with dates and any `existing_supersessions` already declared inside the group. A pair someone ruled on is not an open question. task usually the same work opened twice — keep the one with the history, cancel the other with a pointer. The snippet sibling filter stays snippet-only: it keys on symbol/code_sha, which other kinds don't carry — and for them a look-alike is a finding. Surfaces: MCP find_duplicate_records (classified into _READ_ONLY_TOOLS — the completeness test would have caught the omission), REST /api/notes/duplicates, and a KnowledgeView panel mirroring SnippetListView's — links only, no merge button, because for notes the report proposes and the correction is a read- and-decide act. The panel follows the type filter and clears when it changes, so a note report can't linger under a task view. Correcting the task's own premise: it claimed the snippet report had "no view consuming it" — stale; SnippetListView has consumed it since it shipped. The UI gap was only ever notes/tasks. Answers the question carried from #2482: yes, the update routes on BOTH surfaces can turn a record into a duplicate — the gate is create-time by design. This report is the mechanism that catches it after the fact, which is the model the operator chose. Refs #278, #2547	2026-08-08 18:51:49 -04:00
bvandeusen	3f1523b19f	feat(systems): read-side teeth — the vocabulary at session start, a search filter, and the state/chronicle instructions ... Step 4 of #278, product half. The audit that motivated it: one System in project 2, thirty records tagged, nothing since July 28 — three days after the feature landed. Not a discipline failure; retrieval was completely blind to the association (zero references in embeddings, knowledge, search, auto-inject, or enter_project), so tagging was a write-side label with no read-side payoff, and labels nobody reads don't get maintained. Three changes, ordered by what makes the others workable: 1. enter_project returns the project's Systems (id, name, first line of the charter). Load-bearing for the tagging instruction: you cannot ask an agent to check a record against a vocabulary it never sees. Trimmed because it rides on every session start; the full charter stays get_system's job. Present-and-empty rather than absent when a project has none — "no named areas yet" is information the create-the-System instruction acts on. 2. search accepts system_id, MCP and REST (#33). Implemented once in semantic_search_notes as an EXISTS against record_systems — an association filter deciding candidate-set membership before scoring, like project_id, not a ranking signal. The REST route's missing project filter stays #2463's: it carries a default-scope UI decision this change must not preempt. 3. The instructions (#119, _INSTRUCTIONS + using-scribe skill; plugin 0.1.25 for the cache): - Tag as you write, with an executable test — "would someone investigating that subsystem want this in the pile list_system_records returns?" — rather than "tag appropriately", which is what died. - Create the System when the area has no record: the two-or-more test snippets use, plus "don't wait to be asked to name an area that plainly exists", because the agent's default was leaving un-modelled areas un-modelled forever. - State vs chronicle: dev-logs are written once and never rewritten; durable findings live in the System's reference note, updated in place — safe because note versions are the changelog, which has existed since the feature shipped and was never named as one. list_system_records' docstring now sells it as the way to READ a subsystem, reference note first. No auto-inject boost by System — vocabulary and filter first, measure before adding ranking behaviour (the #2486 lesson). Refs #278, #2546	2026-08-08 18:19:58 -04:00
bvandeusen	f20c019f2a	feat(supersession): demote what a later note overtook, and label it ... Step 3 of #278. First step with visible effect. ## Where the demotion happens, and why not in SQL Applied AFTER the pgvector fetch, over an over-fetched candidate set, not as part of the ORDER BY. Ordering by `distance + penalty` would be exact and would turn an indexed top-k into a scan-and-sort of every embedded note — the HNSW index from migration 0067 can only serve a raw-distance ordering. So the query fetches 3x the requested rows by raw distance and the re-rank happens in Python. Demoting after a LIMIT k with no over-fetch would have been theatre: the cut already happened, so a superseded record pushed down still sits in the results and the live record that should have replaced it was never fetched. The cost is stated in the code: a live record outside the over-fetch window cannot be promoted in. With a 0.05 penalty against neighbours ~0.014 apart, that needs the true answer more than three ranks down, which no observed query approaches. ## Demote, never hide — enforced in three places The penalty applies to the RANKING score, not to the relevance threshold. The floor decides whether a record is relevant at all; the penalty decides which relevant record comes first. Applying it to the floor would drop a superseded record out of the results entirely, which is the one thing this must not do. It is small on purpose. Supersession is a claim about SOME of a record's content, so one that strongly answers a question nothing else answers still surfaces — just behind anything comparable that is current. Its test asserts both bounds, the upper one citing the operator's constraint rather than an optimisation. And every test here that could be satisfied by dropping a record instead asserts the record is still present. ## The dedup gate opts out A superseded record is still a duplicate of what you are about to write — the claim is that it is no longer current, not that it is gone. Demoting it there would let the same note be recorded a second time, and the second copy would be the one nothing warns about. ## The label Auto-inject marks a superseded line SUPERSEDED with a pointer to check the later record. One query for the whole menu. An agent handed stale material with nothing marking it acts on it with full confidence, which is worse than never having surfaced it — the ranking is only half the fix. Fails open: a supersession lookup error returns unpenalised results rather than none, because unpenalised ranking is the behaviour that shipped for months and a broken search is not. Refs #278	2026-08-08 02:00:06 -04:00
bvandeusen	984407f931	fix(supersession): one query for both directions, not two per note read ... CI failed on 8d9e96c — eight tests in test_mcp_tool_notes.py, all "Connect call failed (127.0.0.1, 5432)". The proximate cause is that `_attach_supersession` runs on every note read/write and those are unit tests of the tool layer with no database. But the test failure exposed a worse decision underneath it. I had written the two directions as two service calls, so every `get_note` made TWO extra round trips plus TWO ACL checks — on the hottest path in the product — to save a two-line partition in Python. That is the wrong trade whether or not a test noticed. `get_relations` replaces both: one OR query, one ACL check, partitioned by which column holds the note's id. Its test asserts `execute.await_count == 1`, so the collapse can't quietly come apart later. The tests then get an autouse stub rather than the code getting a swallow. The tool genuinely has a new dependency; hiding that behind a try/except to keep unit tests green would be arranging for the code to lie about what it does. This file already records the same hazard for note 2109, so the stub sits next to that precedent. Added the test that matters, which the first pass missed: a superseded record still surfaces, so an agent WILL read stale material — and it must arrive with a plain-language warning, not just a numeric field to notice. Also pinned that both keys are ABSENT rather than present-and-empty when there are no relations. Refs #278	2026-08-07 22:45:56 -04:00
bvandeusen	8d9e96cc6d	feat(supersession): declare it — supersedes on both write paths ... Step 2 of #278. Records and reads the claim; the demotion that makes it matter is step 3. `services/supersession.py` with set/get on both directions, following the set_record_systems shape since this is the same kind of mutable M2M at the tool/route layer rather than inside notes_svc. ## Both directions are exposed, and only one is obvious `supersedes` is what the author claimed. `superseded_by` is what a READER needs and what the note itself cannot know — a stale record handed over with no marker gets acted on confidently, which is worse than never surfacing it. So get_note carries it, says so in its docstring, and adds a plain-language line telling the reader to open the newer note first. Both are OMITTED when empty rather than serialised as empty lists. A field that always says nothing trains readers to skip fields — the lesson consolidated_at cost, removed in the previous commit. ## Refuse vs drop, which is the one real judgement here Dropped silently: a target that doesn't exist, is trashed, is the note itself, or would close a cycle. Each is a claim with no subject or no meaning; none is something the caller can act on. REFUSED with PermissionError: a target the caller can read but not write. That is the single case where the caller could believe they succeeded and be wrong in a way that matters — demoting someone else's record out of their retrieval is damage invisible from the outside, with no symptom for the owner to trace. Rule #47, and PermissionError because services/snippets.py already uses it for read-but-not-write with both surfaces catching it. The PATCH/PUT routes scope by the CALLER, not owner_uid: an editor-share holder may edit the note and must not thereby inherit the owner's write access to whatever they name as superseded. ## Cycles A ring claims every member is obsolete. Under flat demotion that demotes them all equally, so the set drops out of ranked retrieval together with nothing in the data saying why. Refused by walking the existing graph from the proposed target — iteratively with a visited set, because the graph is user-supplied and a deep chain must not become a stack overflow on a write path. The visited set also makes the walk terminate on a ring that already exists, which is pinned by its own test rather than trusted. Both surfaces (#33), the instruction surface per #119 — framed as the third answer beside update-instead and force=true: not everything resembling an existing record should be folded into it, and not everything distinct should compete with it forever. Refs #278	2026-08-07 22:38:09 -04:00
bvandeusen	5dcb738ce8	fix(backup): carry note_supersessions — the coverage guard caught the omission ... CI failed on 45c6b1c. `tests/test_services_backup.py` asserts every table is either in `_BACKED_UP` or explicitly in `_NOT_INCLUDED`, and the new table was in neither. That guard exists for exactly this: a new table gets a model and a migration — both of which fail loudly if wrong — and then silently never gets a backup section. No error, no warning, and a restore that reports success. Its comment says it was written to "stop the seventh" table slipping through. This was the seventh. Supersession claims are backed up rather than excluded because they are a JUDGEMENT. Someone decided this note replaced that one, and nothing in either note's text records the decision. Lose them and a restored corpus silently returns to ranking stale material alongside current material — with no symptom that says why. Scoping differs by backup kind, deliberately. A whole-instance export takes every row. A single-user export takes only claims where BOTH ends are that user's notes: a claim spanning out to someone else's record cannot be restored into a single-user import, since the far id is not in the map, so exporting it would write a row that silently vanishes on the way back in. Restore maps both ids and skips the row unless both resolve. A claim is about a PAIR — half of one is not a weaker claim, it is a dangling row pointing at whatever note holds that id next. Guarded by `data.get` like every post-v2 section, so v5 and older payloads restore cleanly without it. BACKUP_VERSION 5 -> 6, and the version test moved with it. A payload section added without moving the version produces backups that are structurally different and indistinguishable by inspection. Refs #278	2026-08-07 21:49:08 -04:00
bvandeusen	45c6b1c88a	feat(supersession): the relation, and the dead column that stood where it should ... Step 1 of #278. Structure only — nothing reads or writes the new table yet. Old records outrank newer ones on the same subject because a similarity score cannot tell time. A note that accurately described how something worked in June is still accurate ABOUT June; it is just no longer the answer. Nothing recorded that, so nothing could act on it. `note_supersessions(superseder_id, superseded_id)`. The claim points FORWARD — the newer record names what it overtakes — because the older one cannot know it has been overtaken; asking it to record its own obsolescence is asking it to predict the future. A table rather than a column because the relation is genuinely many-to-many and partial, and both directions are hot: superseded_id answers "has this been overtaken?" at ranking time, superseder_id answers "what does this replace?" in a record view. An array column serves one and not the other. CASCADE is safe because trashing is not a delete — trash_svc stamps deleted_at, so a trashed note keeps its claims and restore brings them back. It fires only on purge_trash, where a claim about the row would be unactionable anyway. A CHECK rejects self-supersession, which under flat demotion would let a record demote itself. ## consolidated_at, and what it actually was Dropped. Written by nothing while serialised into every note and task payload as null — and worse, it implied a capability. The survey (#2483) read it as note consolidation modelled and abandoned. That was wrong, and the frontend is what says so: `TaskViewerView` rendered "✦ Auto-summarized from work logs" gated on this column. It is a survivor of the pre-pivot auto-summary subsystem (migration 0030), whose own column #599 removed. Not an unbuilt feature — an outlived one. So four more remnants went with it: the banner, its CSS, a `consolidatedAt` ref in TaskEditorView assigned and never read, and `.auto-summary-banner-editor` styling with zero template usage. That last one is presence-without-reference in the same family as the column itself. Dropped rather than repurposed for supersession, and the distinction is the point: consolidation folds records into one survivor and destroys the originals. Supersession is the opposite — both survive, the older ranks behind. Smuggling one in under a column named for the other would bury that in schema. ## The hard delete_note Removed, with a comment where it stood. Zero callers, and the danger was never that it ran — it is that it was findable by name. Someone wanting to delete a note greps `delete_note`, finds a function in the notes service with exactly the right signature, and permanently destroys a record every path downstream expects to be recoverable. The MCP tool of the same name already went through trash_svc; only the service function was the trap. Refs #278, #2483	2026-08-07 21:40:51 -04:00
bvandeusen	bbba0b3ae3	refactor(embeddings): one definition of the document a record is embedded as ... `f"{title}\n{body}"` was written out four times. #2486 found three — the write path, the recurring-task spawn, the startup backfill. The guard added here found the fourth immediately, and it was the one that mattered most. `dedup.find_duplicate_note` built the same string as a QUERY, compared against embedded documents. Shaped differently from the corpus it searches, the gate degrades silently: it still returns neighbours, just less apt ones, and nothing says the query and the index stopped agreeing. The spawn path has the same shape of risk — a recurring task embedded differently from everything else is ranked against documents it doesn't match. None of the four had diverged. That is what makes this worth doing now rather than after: they are identical today, so collapsing them is a no-op, and the whole point is that the next change to the shape can't hit three of four. Which is imminent. #2486 measured a dev-log separating from five unrelated dev-logs by 0.023 where a snippet separates by 0.153 — the difference being that a snippet states its purpose twice in a short document. Whether that shape is right is the open question; testing an alternative against four copies would mean testing a shape that isn't the one in production. This is the precondition the issue named. The guard is source inspection, matching the f-string pattern rather than a variable name, so a copy that renames its locals is still caught. A behavioural test cannot see this: an inlined copy produces the same string today and diverges the day the shape changes. Refs #2486	2026-08-07 13:01:45 -04:00
bvandeusen	24d071619b	fix(dedup): compare the artefact, not the prose describing it ... The snippet gate was reading the wrong field, and #2464's UI recipes made it measurable in both directions at once: .btn-danger vs .btn-danger-outline 0.92 siblings, BLOCKED .btn-primary re-recorded verbatim under a different name <0.90 a literal copy, ALLOWED The second is what settles it. Identical code at an identical repo·path·symbol sailed through because the description differed, while two deliberately parallel variants were refused because theirs did not. A snippet's embedded document is mostly prose ABOUT the code, so no threshold fixes this: lowering it blocks more siblings, raising it admits more copies. So structure decides. Two exact signals, both index-served off the notes.data mirror that already exists, no migration and no backfill: location the same named thing in the same file. Requires BOTH path and symbol — a path alone is a directory of artefacts, and matching on it would refuse every second recipe from one stylesheet. code byte-identical code anywhere, via the same fingerprint the drift check uses. The semantic arm survives as a backstop for a genuine reword that shares neither, raised to 0.96 so it sits above the 0.92 band where real variants live. Structural hits say what they matched instead of hedging with "similar", and point at merge rather than update — two records of one artefact is what merge exists to fold back together. find_duplicate_snippets gets the same correction: pairs where both snippets name a symbol, name DIFFERENT symbols, and hold different code are variants, not copies. Without it a design system's button family reports as one merge set — eight recipes, every direct pair over the floor, top score 0.92, one click from collapsing a component family. The cost is real and stated in the code: a helper recorded twice under two names no longer reports. That trade favours the report being usable, and same-symbol and unnamed duplicates — how re-recording usually looks — still surface. The filter fails open, so a lookup failure degrades to the old unfiltered report rather than to a reassuring empty one. resolve_locations extracted: compose_body, create_snippet and now the gate each had their own copy of the repo/path/symbol shorthand fallback, and the gate is the one where a disagreement would mean matching a location the record won't be stored with. Applied to both create surfaces (#33) — the web UI must not be the way to record what the agent was stopped from writing. Refs #2518, #2464	2026-08-06 11:31:02 -04:00
bvandeusenandClaude Opus 5	dcd4efcea0	refactor(design): retire /design — a surface that could only inspect itself ... The design surface is for the projects an install tracks. /design read the running app's own stylesheet — names out of a bundled theme.css, values out of getComputedStyle(document.documentElement) — so it could only ever describe the instance serving the page. Scribe is one project among the projects Scribe tracks; it gets no view hardcoded into every install. The mechanism that makes this a tool rather than a mirror already existed and already covers Scribe: scripts/check_design_tokens.py runs in CI against a sheet path it knows nothing about, using check_code_against_tokens — the same engine behind check_snippets_against_system. /design was redundant even here. Removed: DesignView, DesignTabs (nothing left to tab between), api/design.ts, routes/design.py and its blueprint, the /design route, ui_design_system() and its setting, and the Settings picker that designated "this app's UI". utils/designTokens.ts and utils/designDrift.ts go with it — between them they were the browser-reading half. What survives is utils/designValues.ts, which works on a record rather than a document: valueForMode, modesPresent, and resolveDeclared. resolveDeclared gained real isolation in the move. Custom properties inherit and `all: initial` does not reset them, so a probe sitting in this page would resolve any reference a record leaves undeclared against the SURROUNDING app's tokens — previewing another project's system would quietly borrow this one's palette wherever that system was incomplete, and a token already reported under unknown_refs would render as though it were fine. Undeclared references are now blanked on the probe first, so they resolve to nothing, which is what the record says they are. Migration 0075 absorbs ui_design_system_id alongside design_rulebook_id rather than an 0076 undoing it: 0075 has not run anywhere, since dev is unmerged and deploys come from main. Both keys named a design source for the running install, and a project already carries its own pointer. This retires the agreement panel shipped yesterday. It asked whether the sheet was actually loaded and applied — the one question a record cannot answer about itself — but only ever about the app you are already inside. Nothing replaces it; recorded in #2430 rather than quietly dropped. Step 1 of milestone #274. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-04 10:35:52 -04:00
bvandeusenandClaude Opus 5	5b824c1626	feat(design): the panel now asks whether the app agrees with its own sheet ... Retiring rulebook #2 left the /design drift panel with no data source, and because its empty state was well-written the feature read as working while it could only ever render "nothing designated" (#2419). The original question is genuinely gone: theme.css is generated from design system 2, so checking the system against a sheet derived from it would be a tautology. The question that survives is the one no server can answer. A generated sheet still has to be LOADED and APPLIED, and nothing checked that it was: absent the record declares a token the app doesn't have — the sheet was never regenerated after the record changed, or never loaded differs the app has it with another value — a stale sheet, or a later rule that overrode it unrecorded the app declares a token in the record's own family that the record has never heard of Both sides go through the same engine so the comparison is honest: declared values are set on an offscreen probe and read back, which performs the same var() substitution the browser already did to the live values. Comparing raw strings would mark every derived token as drift. The designation moved with the feature — design_rulebook_id becomes ui_design_system_id, with a migration deleting the retired key rather than leaving an inert row. The prose extractor it fed goes too (#2288 said its runtime role ended when the import landed). Three orphans of the same shape, found alongside and fixed here: - darkOverriddenNames hardcoded [data-theme="dark"]. The sheet went dark-first months ago, so it matched nothing and the "mode-aware" flag silently left the gallery. Now matches the SHAPE of a mode selector, which also holds for an install whose modes aren't light and dark. - groupFor's prefix table never heard of --fs-, so 110 tokens sat under "other". Groups now come from the record where there is one; the table can only know families that shipped with the product (rule #115). - The type scale was a hand-written table of nine sizes marked "no token", true when written and false since the scale was recorded. Now rendered from whatever size tokens the sheet declares, so it can't go stale twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-03 20:50:03 -04:00
bvandeusenandClaude Opus 5	22f907c44d	feat(design): offer starter token ROLES at creation, never values ... A literal gets written into a stylesheet when there is no role to reach for. This codebase demonstrated it: the house style had no "text on a filled colour" role, so 76 call sites wrote a pure-white literal — not out of defiance, but because nothing existed to write instead. The correction was not a better ban list; it was declaring the missing role (#2275, #2349). So the useful moment is creation. A system whose roles are named on day one never presents the occasion. Ten groups, ~40 roles: surface, text, action, semantic, border, accent, radius, space, motion, state. Operator's call was one flat list, every group individually skippable — presets keyed to app shape (web / CLI / docs) were rejected because they need the product to hold opinions about app categories, and a wrong category is worse than a list someone prunes once. TWO BOUNDARIES THIS HAS TO HOLD, both rule #115: - The ROLES ship; the VALUES never do. Every seeded token has an empty value_by_mode, so a fresh system is a set of named, deliberately-unanswered questions. A test asserts no hex appears anywhere in the module — not just that tokens are blank, but that no palette hides in a comment waiting to be pasted in. - The PREFIX is the install's. `--fs-` is FabledSword's convention, not the product's; the default is a neutral `--ds-` and callers pass their own. Valueless roles are already legible downstream — render_stylesheet emits them as commented-out declarations and stylesheet_for_system reports them under `valueless` (#2299) — so "declared but undecided" reads correctly with nothing new built. Both surfaces, per rule #33: MCP gains starter_role_groups/token_prefix plus list_starter_role_groups(); REST gains the same on POST plus GET /api/design-systems/starter-roles. The parity enumeration is extended rather than loosened. Note create_design_system treats None and [] alike (seed nothing), while starter_tokens treats None as "all". Deliberate: creation must never write 40 rows into a system whose caller never asked, and the everything-checked default belongs in the UI where the operator can see it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-03 11:37:20 -04:00
bvandeusenandClaude Opus 5	be3a0ffaf9	fix(projects): batch the summary queries — the fan-out was exhausting the pool ... Reported live: Projects and Snippets showed skeletons that never resolved, /knowledge worked intermittently. The logs named it exactly: QueuePool limit of size 5 overflow 10 reached, connection timed out, 30.00 GET /api/settings 500 30584.0ms GET /api/projects 200 30882.9ms /api/projects was not hanging — it was waiting out the 30-second checkout timeout and then returning 200 with summaries silently missing, because _attach swallowed the TimeoutError. Nobody waits 31 seconds, so it read as a hang. THE SHAPE: routes/projects.py ran asyncio.gather over every project. Each _attach called get_project_summary, which opened its own session for three queries and then called get_project_milestone_summary — which opened one more session PER MILESTONE. So 25 projects asked for roughly 250 concurrent checkouts against a pool of 15 (SQLAlchemy's default 5 + 10 overflow). That is why unrelated routes failed too. Snippets and /knowledge were never broken; they queued behind the burst and inherited its timeout. /api/settings returning 500 while /api/projects returned 200 is the same cause wearing two faces. The comment above the gather said "one backend pass instead of N+1 frontend calls". It did remove the N+1 from the network — and recreated it against the connection pool, where it is worse, because the browser had at least been serialising those calls. Now: get_project_summaries() does all projects in four queries and one session, and get_project_milestone_summaries() does all milestones in two. Two sessions total for the whole page, independent of how many projects exist. The progress calculation is extracted to _progress_from_counts and shared by both the batch and single paths, so the cancelled-exclusion rule cannot drift into two versions that disagree about whether a milestone is finished. Tests assert the SESSION COUNT, not just the values. An implementation that returned identical output while opening a session per project would pass a correctness test and reproduce the outage. Deliberately NOT done: raising pool_size. It would move the cliff rather than remove it, and this endpoint now needs two connections regardless of scale. Closes #2384. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-02 19:09:36 -04:00
bvandeusenandClaude Opus 5	84541f392b	fix(backup): six tables were silently absent, and nothing would catch a seventh ... services/backup.py enumerated its tables as hand-maintained literals with nothing tying them to the schema. Tables added since that list was last extended were absent from every backup — no error, no warning, and a restore that reports success. Missing: systems, record_systems (0065), note_usage_events (0071), design_systems, design_tokens (0072), and repo_bindings — which the issue itself had not spotted, found only by diffing the model tablenames against the two lists instead of trusting either. _NOT_INCLUDED was worse than incomplete: it named "embeddings", "invitations" and "password_resets", none of which are tables. It read as coverage while naming nothing the schema could confirm. Now real names, plus retrieval_logs — observational telemetry that grows per query and that nothing reads for correctness. THE DELIVERABLE IS THE GUARD, not the six sections. Extending a list fixes today and changes nothing about the next table; a new one now fails a test until someone either backs it up or states that it shouldn't be. It checks both directions — an unaccounted table, and a listed name that no longer exists, which is what the three phantom entries above would have tripped. Design systems need ordering care: parent_id is a self-FK. The export orders parent-first (parent_id NULLS FIRST, then id — a parent always has the smaller id), so restore resolves each parent from the map as it goes, with no second pass. A child whose parent is missing lands as a root rather than failing the whole restore. Usage events are kept because pull-through is the evidence base for whether recall works, and it only ever accumulates — a restore that dropped it would reset that measurement to zero while everything still looked fine. BACKUP_VERSION 4 -> 5. Every new restore section is data.get()-guarded, so v2/v3/v4 payloads restore unchanged. Closes #2293. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 23:09:25 -04:00
bvandeusenandClaude Opus 5	da2383b079	fix(retrieval): verify the reserved slot's kind instead of trusting the query ... CI caught six failures on f8522fb. Five were fixtures; one was a real assumption. THE REAL ONE: _reserve_slot_for_reuse trusted that a query filtered by note_type could only return reuse kinds. It now checks _record_kind on the way in. That slot exists FOR snippets and processes — one silently spent on something else is worse than no slot at all, because the resulting line is indistinguishable from one that earned its place on score. THE FIXTURES, all the same shape: MagicMock notes with is_task left to auto-create. It is truthy, and _record_kind reads task-ness FIRST — so every mock snippet in three test modules was rendering as "task". Two of those fixtures already carried a comment explaining this exact hazard about `.data`; the same reasoning applies to `.is_task` and nobody had needed it until the menu started naming kinds. One assertion was genuinely stale rather than broken: test_write_path_trigger pinned note_type == "snippet", which was the behaviour the widening replaced. Updated to the new contract, including the task_kind="issue" filter that keeps the open to-do list out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 23:01:31 -04:00
bvandeusenandClaude Opus 5	f8522fb28f	fix(retrieval): give reuse a slot, and let experience reach the write path ... Two mirror-image scoping mistakes, neither deliberate (#2246). AUTO-INJECT let every kind compete on raw cosine. That is fatal rather than merely imperfect here, because Scribe's project records are ABOUT software work: a task titled "surface snippets before the agent writes code" is a near-perfect lexical match for "write a function…" while answering none of it. Measured live, a prompt asking for a helper returned three records about BUILDING the retrieval system and zero snippets. Snippets are ~0.5% of the corpus, and the ratio worsens as the project record grows — which is the direction Scribe is meant to grow, so no threshold tuning fixes it. Now the best snippet or process takes the LAST slot when none won on score. Deliberately NOT held to the margin band: that band measures distance from the top overall score, and the top score is the very thing snippets lose to. It still must clear the configured threshold, so a weak snippet cannot buy the slot — silence stays the default. Skipped entirely when reuse already won, so the fix is invisible in the case it isn't needed. WRITE-PATH was snippets-only — the same mistake inverted. An issue recording "we tried this and it deadlocked" could never reach the moment that code was about to be written, though it is arguably the better prior art: it says what NOT to do. Widened to snippets plus recorded experience. That needed a filter the search layer couldn't express. "Experience" is issues plus dev-logs, which differ on is_task, so neither note_type nor is_task alone covers it. semantic_search_notes gains task_kind, which restricts TASKS to the given kinds while leaving non-task notes untouched — so note_type=("snippet", "note") + task_kind="issue" yields snippets, fixed problems and durable notes, without the open to-do list. note_type now accepts a sequence too. Non-snippet hits are labelled with their kind, because an unlabelled issue on that menu reads as "here is code to reuse", the opposite of what it says. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 22:56:31 -04:00
bvandeusenandClaude Opus 5	5c51e29f26	fix(embeddings): embed in the service, so every caller gets it (#2056) ... A note or task created through MCP was not semantically searchable until the next restart's backfill ran. Embedding fired at the five REST route handlers and nowhere else; the MCP tools call the service directly, so they skipped it. The shape of this bug is the reason to care: it is invisible on an instance that redeploys constantly (this one does, per rule 46) and permanent on one that doesn't. Rule 115 — the product has to stand up for the install that restarts twice a year, not just for the one that restarts hourly. Moved to services/notes.embed_note(), called from create_note and update_note, and deleted from all five routes. Every caller — REST, MCP, recurrence, snippets — now gets it by construction rather than by remembering. Two things fall out of having one implementation instead of six: - It uses note.user_id, the OWNER. The routes were inconsistent: some passed the caller's uid, some the owner's. On a shared record the caller's id mints a second embedding row that nothing reads. - services/snippets.py's _embed_snippet existed only because snippets are created via MCP and the routes couldn't cover them. Every one of its four call sites goes through notes_svc, so the helper and its four calls are gone, along with the eight test patches that existed to neutralise it. RuntimeError (no running loop — unit tests, scripts) and any indexing failure are both swallowed: a write that succeeded must not be failed by its index. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 22:46:57 -04:00
bvandeusenandClaude Opus 5	731ca284c3	feat(design-systems): give a design system a way to reach the session ... Storing a design system never made a session aware of one. Rules get pushed into every session by the SessionStart hook and returned by enter_project; a design system had neither, so its standards were reachable only by an agent that already knew to call resolve_design_system — the same silent failure as a token nobody declares. That gap was invisible while the operator's visual standards also lived in a rulebook. Retiring that rulebook (which is what this unblocks) would have deleted design guidance from every session with nothing to say so. - services/design_systems.design_context() — the delivery side. Guidance is chain-merged ANCESTOR-FIRST: a child system holds only what it CHANGES, so its own guidance describes a departure from a house style it never restates, and the leaf alone is a fragment. Tokens are summarised (count + group names), not listed — a hundred declarations would crowd out the context they are meant to inform. - enter_project returns `design_system`, null when the project has none. - The SessionStart context gains a Design system block with pointers to the values, alongside the always-on rules. - server.py's entity list gains Design system, including the negative: do NOT record one as a rulebook, because a token kept as prose cannot be resolved, inherited, rendered or checked. - The rulebook-tier passage used "a design-system rulebook" as its worked example of a subscribed rulebook — it now teaches the opposite, plus a new "is this a rule at all?" test pointing at design systems, processes and snippets. - using-scribe gains a section on building UI against the project's system. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 22:10:09 -04:00
bvandeusen	378a4b8f99	feat(ci): check the app's own components against the tokens, not just snippets ... Closes the gap the theme.css repoint exposed (#2319, part of #2277). `check_code_against_tokens` could always answer "does this code use the sheet correctly?" — it was only ever fed recorded SNIPPETS. The app's own components, where sixteen unresolvable references were living quietly, were checked by nothing at all. That was structural rather than an oversight: the drift panel runs in the browser and cannot read source files, and the server has no repo access. CI is the only place holding both the sources and the ability to run the check — and it only became cheap once theme.css became a generated artifact, so the source of truth is a committed file with no network and no credentials. **The sheet now carries its own SUPERSEDES block.** That is what keeps the checker instance-agnostic (rule #115): it knows nothing about any palette, and reads both the declarations and the discouraged literals out of whatever stylesheet it is pointed at. Hardcoding "#fff means use the text token" would have baked one install's kit into the tool. Two severities, split on whether the count is already zero: FAIL an unresolvable var() reference — zero today, so this is a ratchet holding a line already reached. It cannot false-positive either: the name is declared or it is not. REPORT superseded literals (32 files) and raw colour literals (246). Gating those means a permanently-red job, and a check nobody reads is worse than no check. **Comments are stripped before scanning, and that fired on the first real run.** A comment explaining why a literal is avoided necessarily contains that literal — DesignSystemsView's stylesheet documents exactly that about `#fff`, and the checker reported the explanation as a violation. A checker that flags the documentation of a rule teaches people to stop documenting rules. Also narrowed `--fs-weight-medium`'s supersedes to the keywords. `600` and `700` are real violations of the two-weight rule, but a bare number matches too much to find by literal scan — `z-index: 600` is not a font weight. That needs a property-aware check, which is a different tool. Verified end to end: the generator's output parses back through the checker's reader, so the two halves cannot drift into disagreeing about the format.	2026-07-31 14:52:06 -04:00
bvandeusen	473280e690	chore(design-systems): stop teaching one install's kit in product copy ... The operator's check: this must be a system for managing design systems, not one with the FabledSword family built into it. No LOGIC was coupled — the audit found zero behavioural dependencies. But every docstring example, every UI placeholder and several comments named this install's palette, so a stranger creating their first design system was shown "FabledSword" as the expected shape and `--fs-obsidian` as the expected token. Examples teach, and these taught the wrong thing. Placeholders now describe the SHAPE ("Your house style", "--surface-page") rather than naming one instance's contents, and the token-name placeholder now says the thing worth saying: name it for its purpose, because `--obsidian` and `--button-bg` both stop being true the moment the value or the element changes. Not fixed here, and it is the one real coupling left: DesignView.vue hardcodes rule 65's button variants and rule 60's type scale as literal arrays, so a stranger's Design page would display this family's specs. Those arrays exist because there was no design system to read from — which there now is. They go when the panel is repointed (#2295), not before. Scribe's own stylesheet comments ("Moss action-primary per Hybrid") are left alone: that is the app CONSUMING the family style, which is what dogfooding looks like, not the tool assuming it.	2026-07-31 10:18:53 -04:00
bvandeusen	1fde646c60	feat(design-systems): formulas — derived tokens that follow their source ... Operator: "build in a way to support formulas like this so that the colors shift as expected and have less to clean up when testing color changes." The storage needed no change at all, which is the good news. A formula is just a value: --fs-accent-soft: color-mix(in srgb, var(--fs-accent) 15%, transparent) It passes the value sanitiser untouched (verified, and now pinned by a test — had `color-mix(... var(...) ...)` been rejected as unsafe, derivation would have needed a storage shape of its own), and the browser resolves the `var()` at use time. Change `--fs-accent` and everything derived from it shifts. **One declaration covers every mode**, and that is the "less to clean up" part. A derived token written once in the base layer follows its source through dark mode automatically, because `var()` resolves where it is USED rather than where it is written. A stored computed literal would need a row per mode and would silently stop tracking the source the moment the source changed — the whole problem this avoids. What derivation DID need is the check. A formula pointing at a token that does not exist is invalid-at-computed-value-time: the browser drops the declaration outright and the token has no value. No error, no warning, nothing in the toolchain notices — the same family as `--color-accent`, `_parent_map`, and the scripted edit whose anchor matched nothing. So `derivation_report` returns three things alongside the sheet: which tokens are computed and from what, which formulas point at nothing, and which derive from each other in a loop. CSS resolves a loop to nothing rather than hanging, so the cycle check is about telling the operator, not protecting the renderer — but a token that quietly resolves to nothing is exactly what is worth being told. A self-reference with a fallback (`var(--fs-x, 8px)`) is deliberately not a dependency; counting it would report every such token as a one-node loop. The UI leads with broken formulas, then loops, then the healthy derived set — the first two are unambiguously wrong, where a duplicate value is a judgement call.	2026-07-31 09:51:54 -04:00
bvandeusen	23a385e2db	revert(design-systems): drop the rulebook import — a migration, not a feature ... Operator's call, and it corrects a scope error rather than a bug: "this is a path for a user to go from a rulebook to a design system. we don't need to build this path in the app itself ... you should be the one that does the import ... going forward no one else should have to do such a migration." Right. Nobody starting from a design system will ever go rulebook -> system, so the whole path was permanent product code serving a single act on one install. Rule #22: remove it, don't flag it off. Gone from the service, the REST route, the MCP tool, the UI panel, the API client and its tests. There is a second consequence I had missed, and it is the better argument. The parser was WORSE at this than doing it by hand. `propose_tokens` leaves radius steps and type sizes valueless because "Small 4px" is not a hex and nothing here parses it — a limitation I documented carefully and shipped anyway. But that limitation only exists because the importer had to run unattended. Done as work rather than as a feature, those values are just read and written, and the result is a complete design system instead of one with a dozen blanks and a count explaining them. Scaffolding built around my own absence from the loop, when I am the loop. KEPT: `extract_expectations` and `design_expectations` in services/design_rulebook_import.py. The live drift panel still reads them until it is repointed at a resolved design system (#2295), and removing them now would take the /design page's only content with it. They go with that change, not this one.	2026-07-31 09:46:12 -04:00
bvandeusen	0f80b790c7	feat(design-systems): central prose — guidance on the system, rationale on tokens ... Last piece of the architecture in #2296. The operator: "the prose doesn't have to live as one offs, there's a central system for managing it." Two fields, both free-form: design_systems.guidance the narrative a token table cannot hold — aesthetic, voice and tone, what is deliberately out of scope. design_tokens.rationale WHY a token is this value, which is a different question from `purpose` (what it is FOR). "Success equals Moss, aligned by design" is a rationale; "page bg, deepest surface" is a purpose. Rules carry the first routinely and a token row had nowhere to put it. Free-form rather than a column per category, deliberately. A schema with `voice`, `aesthetic` and `scope` columns would bake one rulebook's table of contents into every install (rule #115), leaving the next install three empty columns and nowhere for what it actually cares about. Both nullable: a design system with no prose at all is complete, not a draft. `rationale` cascades like `purpose` — deepest non-empty wins — so an app overriding a colour keeps the family's reasoning rather than blanking it. Same argument as `supersedes`: the override was about the value, not the meaning. In the generated sheet the inline comment prefers `purpose` and falls back to `rationale`, so a token carrying only the why still says something instead of rendering bare.	2026-07-30 21:52:16 -04:00
bvandeusen	46d88f9e7e	feat(design-systems): check the components against the sheet they claim to use ... "The snippets use the tags from the sheet" was a relation nobody could verify. Now it is three checks, and all three currently fail SILENTLY in this codebase: unknown `var(--x)` where the system declares no `--x`. Renders as nothing at all — no error, no failing test, no visual clue beyond the element quietly not being styled. superseded literals a value the sheet said to stop writing, paired with the token to write instead. Only possible because `supersedes` is declared rather than inferred. local definitions custom properties a snippet mints for itself instead of reusing the sheet's — the bloat a shared sheet exists to prevent, where a value stops being reused and starts being restated per component. The first is not hypothetical. Writing DesignSystemsView.vue earlier in this same session I used `--color-accent` throughout; it does not exist, and nothing in the toolchain noticed. This check is the thing that would have. A token that is both defined and read locally is reported ONCE, as an unknown reference — "--btn-bg does not exist in the sheet" is the more precise statement of the same problem, and reporting both would double-count one fact. Literal matching is boundary-aware and case-insensitive: `#fff` must not fire inside `#ffffff` (different colours, and a finding on the wrong one sends someone to change correct code), while `#FFFFFF` in a rulebook has to match `#ffffff` in a stylesheet — the same trap `normalize_hex` exists for. Snippets with nothing to report are omitted entirely. A list of everything that is fine is a list nobody reads twice — the same principle the auto-inject menu and the drift panel are both built on. Two integration mistakes fixed while wiring it: `list_snippets` returns `(rows, total)` and caps its limit at 100, and `get_snippet` returns a Note model rather than a dict. The list rows carry a preview, not the code, so the check reads each full body — checking the preview would have reported on a truncation.	2026-07-30 21:47:47 -04:00
bvandeusen	b0a7d9e89b	feat(design-systems): the master sheet — purpose tokens, not per-element values ... Operator's new requirement (#2299, architecture in #2296): a design system does not just hold tokens, it generates and manages a master CSS sheet. That settles the milestone's open "authority mechanism" question — the record is authoritative because the stylesheet comes out of it. **The sheet is shaped by purpose and styles no elements.** It declares custom properties, grouped by what they mean, and contains no `.btn-primary`, no `table`, no `input`. That is the design, not a shortcut: a sheet that styled elements would restate the same handful of values once per element and grow with the UI, where purpose-named values are stated once and reused. Components live as SNIPPETS that reference these names — a surface that already exists and already carries prose, locations, drift checks, merge and write-path recall. A token named after an element (`--fs-button-bg`) is the smell that the two have been mixed; a purpose name (`--fs-action-primary`) is reused across all of them. Alongside the CSS the endpoint returns what the text cannot say for itself: which tokens are still valueless, and which VALUES are declared under more than one name. The second is the operator's "reuse consistent values" constraint made checkable — and it reports rather than refuses, because a design system legitimately aligns colours on purpose ("Success = Moss, by design") and only a human knows which case it is. Mode maps to selector the way the codebase already does it: base on the root selector, every other mode layered on `[data-theme="…"]`. The root selector is a PARAMETER — #251 recorded that a container-scoped preview cannot use `:root`, so hardcoding it would have made the generator useless to the preview surface. A token the rulebook names but states no value for is emitted as a commented-out declaration IN ITS GROUP rather than dropped. Its absence is the finding, and a comment puts that finding where the reader already is. Values are validated, not escaped, and this is a real boundary rather than tidiness: design systems are shareable records (rule #47), so `red; } body { display: none` in a system shared with you would otherwise inject CSS into your page. A value containing `{ } ; @ < >`, a comment delimiter or a newline is REFUSED and rendered as a comment saying so — rejecting beats stripping, since a partially-sanitised value is one the operator never wrote and the sheet's whole claim is that it is the record. Not in scope, and deliberately: serving this as the app's actual stylesheet. Generating and exposing a sheet is reversible; swapping theme.css for a generated one is not, and it should be an explicit call rather than a side effect.	2026-07-30 21:42:08 -04:00
bvandeusen	4dc57f8ab2	feat(design-systems): import a design system out of a rulebook's prose ... Milestone #254 step 3 (#2288). Reuses #251's prose extractor as the reader and adds the part that makes it an import rather than a list of claims. **The join is the whole trick.** A rulebook states a design system in two places and neither half is a token: one rule names the colours ("Obsidian #14171A (page bg, deepest surface)"), another names the custom properties (`--fs-obsidian/iron/slate`). The import pairs them on the word — `--fs-obsidian` ends with `obsidian` — which is the only reason it produces something usable instead of seventy empty names. The parenthetical becomes the token's purpose, which is the field a bare hex could never carry. **Prohibitions arrive as replacements, per the operator's reframe.** Rule 52 declares Parchment and forbids pure white in one breath, so the import emits "write --fs-parchment instead of #ffffff" — the same fact stated forwards. It attaches to the FIRST token that rule supplied a value for, not to every token of that rule, because claiming Vellum is also the replacement for white would be putting words in the rulebook's mouth. **A token the rulebook names but states no readable value for is still proposed, with an empty value.** Radius steps and type sizes are prose ("Small 4px") and nothing here parses them; inventing a parse per shape would be guessing. The name is real and the value needs a human, so the proposal says exactly that — and the UI leads with the COUNT of those, because an import that hid them would look more complete than it is. Preview is the default on both surfaces and in the UI. An import is a proposal: rulebooks are written aspirationally and some of what they describe was never built, so every entry carries the rule id and the sentence it came from and a reviewer can check the claim rather than trust it. Existing token names are never overwritten. A value already in the record was put there deliberately — most likely correcting this importer — so a re-run fills gaps and lists the rest as skipped, which also makes it safe to repeat. Colours the rulebook names but never exposes as a custom property produce no token: it never asked for one, and inventing a name would put something in the record no rule sanctions.	2026-07-30 21:23:32 -04:00
bvandeusen	3da40abcb8	feat(design-systems): declare what to write instead, rather than what not to ... Milestone #254 step 6, first half (#2295) — and this reframes the task rather than answering it. The operator's call: "in this case we should declare what should be used in place of pure white, it's not a prohibition it's what should be used in its place." None of the three options on the table (a constraints record / the panel reads both sources / negative token rows) was right, because all three kept the prohibition as a KIND OF THING. It isn't one. "Pure white is never text" is the shadow cast by a positive fact — text is Parchment — and a design system that stores what things ARE has no row for a ban because it never needed one. So `design_tokens` gains `supersedes`: the literal values this token should be written instead of. `--color-text-on-action` supersedes `#fff` / `#ffffff`. Same fact as the rule, stated forwards, and now actionable — a finding can say what to write rather than only objecting. It has to be DECLARED, not derived, and that is the crux: `#fff` and Parchment `#E8E4D8` are different colours, so no value-matching check could ever have connected them. That mismatch is precisely why the prohibition looked unrepresentable until it was turned around. `supersedes` cascades on EMPTINESS rather than on None. A child overriding a colour says nothing about which literals it replaces, and blanking the family's declaration there would silently disarm the check for every app that customises the token — while a child that states its own list replaces it wholesale. Two things this deliberately does NOT do: - It does not feed the drift panel. Superseded literals live in component CSS, which `designDrift.ts` cannot see and already documents as a blind spot. This is input for the source lint (#2277). Declaring it with nothing consuming it yet is honest; wiring it to a panel that cannot check it would not be. - It does not remove the panel's `prohibited_color` arm yet — that happens when the panel is repointed at a resolved system, which needs #2288 first. The declaration also exposes a missing token: most of the 67 hardcoded `color: #fff` (#2275) are text on a coloured action button, and the system has no token for that role at all. Every view hardcodes it. Declaring the token that was never there is the first real output of the operator's framing.	2026-07-30 21:16:45 -04:00
bvandeusen	143b968c5d	feat(design-systems): REST + MCP surfaces, at parity ... Milestone #254 step 4 (#2290). Eleven capabilities, both surfaces, one service. Design systems are owner-scoped top-level records rather than project-scoped ones, so these do not nest under /api/projects/ the way systems do — routes/rulebooks.py was the closer shape. The one exception is the project pointer, which is genuinely about a project: PUT /api/projects/<id>/design-system, PUT rather than PATCH because clearing it is a first-class outcome and not an omission. `/resolved` and `/tokens` are deliberately separate endpoints. One answers "what does this system CHANGE", the other "what does it end up BEING", and a system that overrides nothing has an empty token list and a full resolved set. Shipping only one would have made the other a client-side computation of exactly the kind the record model exists to remove. ResolvedToken.to_dict carries the SHADOWED contributions, not just the winner. Dropping them at the serialisation boundary would have discarded the one thing step 2 was built to preserve, and it would have been invisible — the payload still looks complete. Three sentinel translations on the MCP side, each tested, because an agent cannot omit an argument and a wrong mapping here is silent: - parent_id: 0 = unchanged, -1 = clear (become a family system), positive = set. Renaming a system must not silently re-root it. - order_index: -1 = unchanged, since 0 is a valid position. - value_by_mode: guarded on `is not None`, not truthiness, so `{}` can strip every mode from a token instead of being unreachable. DesignSystemCycle maps to 400 on REST and to a ValueError carrying the message on MCP — kept apart from 404 throughout. An agent told "not found" retries the same call; one told what the loop is can fix it. Two structural guards beyond the parity list: every endpoint must be reachable on the app (catching a decorator copied without its path, where the second handler silently never runs), and every public coroutine in the tools module must be registered (a tool written but never registered is invisible to an agent, and nothing else would notice).	2026-07-30 17:09:25 -04:00
bvandeusen	839d6902ad	feat(design-systems): resolve the chain, and keep the argument not the verdict ... Milestone #254 step 2 (#2287). `resolve_tokens` flattens a system's inheritance chain into its effective token set — walk to the root, deepest wins by token name. Pure and duck-typed, so a test states a whole hierarchy in literals and the service hands the same function ORM rows. **Provenance is stored as the contest, not the winner.** A ResolvedToken carries every system that offered a value, per mode, deepest first — `[0]` won and `[1:]` are what it shadowed. "Which system supplied this?" and "what did it override?" are then two reads of one list and cannot disagree, where a winner plus a separate provenance field would be two things to keep in step. **Merging is per (name, MODE), and that is the storage decision paying off.** A system that deepens one accent for light backgrounds while leaving dark alone owns `base` and still inherits `dark`. A token-level "overridden here" flag would have to lie about one of them, and the two-column shape could not have represented it at all. Metadata cascades separately by the same deepest-wins rule, with one exception: `order_index` treats 0 as UNSTATED rather than "first", because 0 is the column default. Reading it as a real value would let a colour-only override drag its token to the top of its group — a visible reshuffle in return for a change that touched nothing structural. One fix to step 1 while wiring this up: `_parent_map` is now scoped to the SYSTEM'S OWNER rather than the caller. A caller reading through a shared project owns no link in the chain, so the caller-scoped version would have handed them an empty forest and truncated the cascade to a single system — a page rendering with plausible wrong values and no error anywhere. The ACL already grants read along the whole chain; this is the loading side keeping that promise, and it now has a test naming the shared-project case. `BASE_MODE` moves from the model to the cascade module, where it belongs: it is a resolution rule, not a storage fact, and design_cascade.py deliberately imports nothing so both access.py and the service can depend on it.	2026-07-30 17:02:31 -04:00
bvandeusen	03b3998585	feat(design-systems): the model, the parent chain, and the guard on it ... Milestone #254 step 1 (#2286). A design system becomes a record Scribe holds rather than prose in a rulebook: a named set of tokens with an OPTIONAL parent, so a family system carries the house style and an app system carries only what it changes. Answering "what does this app alter?" is then `list its tokens` — nothing to compute. `parent_id` is the whole model. It replaces both an `always_on` flag (a family system is one with no parent) and a subscription join table (a project points at ONE system; the chain supplies the rest) — less schema than the rulebook shape it mirrors. Two decisions the task left open, settled here: - **Token values are JSONB keyed by mode**, not `value_light`/`value_dark` columns. The deciding argument was not flexibility, it was ambiguity: in a child system an unset mode means "inherit", in a root it means "not mode-dependent", and as columns both are NULL and the resolver cannot tell them apart. As a map, resolution is `{**parent, **child}` at every level with no special case for roots. Against it: queryability — but nothing filters tokens by value in SQL, so that buys a query no caller makes. - **`group_name` is free text, no CHECK enum.** Groupings are each design system's own vocabulary; a whitelist would bake one install's kit into the schema. No CHECK is introduced anywhere, so rule #36 does not fire. The cascade lives in `services/design_cascade.py` as pure functions over a `{id: parent_id}` map, importing nothing — which is what lets both the service and `access.py` use it without a cycle, and lets a test state a whole hierarchy in one literal. Cycles are refused on WRITE by walking up from the proposed parent (the cheap direction), and survived on READ by a visited-set, because a loop from a direct DB edit must truncate rather than hang. ACL (rule #78) is deliberately asymmetric: owning a system grants write, reaching one through a project you can see grants READ ONLY. An editor on a shared project must not be able to rewrite the family system every other project in that family resolves through. Also renames `services/design_system.py` -> `design_rulebook_import.py`. It is the #251 prose extractor, whose role is already scheduled to become a one-shot importer (#2288), and leaving it one character away from the new `design_systems.py` was a trap for every later session. Rule #115 throughout: nothing seeds a system or implies a default. An install with zero design systems is ordinary, not degraded.	2026-07-30 16:54:41 -04:00
bvandeusenandClaude Opus 5	d3ee24f239	feat(design-explorer): rulebook binding + prose→claims extraction ... Milestone #251 step 2 (#2259). The half of the drift panel that needed to be testable, which is why it is Python: the frontend has no test runner, so the fiddly extraction lives server-side and the browser only does set arithmetic over live token values. BINDING. A per-user setting `design_rulebook_id` names the rulebook that describes this install's design system. A setting rather than a column: no migration, discoverable in the Settings UI (rule #25), and honest about being a per-install choice rather than a property of the rulebook. No rulebook designated returns an empty set with rulebook_id: null — the NORMAL case for any install but the one that set it up (rule #115), which the client renders as an explanatory empty state rather than an error. The id comes back alongside the list so "not designated" and "designated but empty" stay distinguishable. EXTRACTION. No NLP. Rule statements are prose written for humans and should stay that way, so this takes only what is unambiguous in any prose — the hex colours and custom-property names a rule mentions. Anything subtler needs a rule author to opt into a structured form, deliberately left for when someone wants it. Three things earn their complexity: - SENTENCE-SCOPED NEGATION. A rule routinely states what the palette requires and what it forbids in consecutive sentences ("Parchment #E8E4D8 …, Vellum #C2BFB4 …. Pure white #FFFFFF is NEVER used."). Detecting negation across the whole statement would mark the required colours as forbidden — inverting the finding rather than missing it, which is worse. Per sentence, all four come out right. - HEX NORMALISATION is load-bearing, not tidiness. The rulebook writes #FFFFFF and components write #fff; if those don't compare equal the largest drift finding in the codebase — 67 hardcoded white text colours (#2275) — reads as zero. Alpha forms keep their alpha, since #fff and #ffff are different colours and collapsing them would manufacture equality. - SLASH SHORTHAND. Rulebooks write token families as --fs-radius-sm/md/lg/xl and --fs-obsidian/iron/slate/pewter. Both expand under one rule — prefix is everything up to and including the LAST hyphen of the first segment — which also handles --fs-dur-fast/base/slow. Verified against the real rule text: 18 tokens from three different shorthand shapes. how_to_apply is read alongside statement, because rulebooks routinely keep the statement declarative and put the concrete values in how_to_apply; ignoring it would miss the checkable half. Claims dedupe on (kind, value), first source winning, so a colour named by several rules is one expectation attributed to the rule that introduced it. Prose with nothing checkable yields nothing — most rules are judgement, not specification, and a panel that reported unparseable rules as problems would be unusable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-30 15:25:36 -04:00
bvandeusenandClaude Opus 5	293a14361a	fix(db): bind datetimes, not strings, when filtering timestamptz columns ... Closes #1727 and #2257 — the same bug in two services written months apart. AppLog.created_at is `timestamp with time zone`. asyncpg binds a Python str as VARCHAR and Postgres has no `timestamptz >= text` operator, so both of these raised when Postgres planned the query: notifications.check_due_tasks AppLog.created_at >= today.isoformat() logging.get_logs AppLog.created_at >= <raw request.args str> #1727 was the worse of the two because a per-user `except Exception` swallowed it: reminder emails silently never sent, and the only outward trace was an hourly traceback in the Postgres log. It has been open since 2026-07-19 with the diagnosis written and the fix never applied. #2257 has no swallowing handler, so it merely breaks the admin log viewer's date filters outright. notifications: `utc_day_start(day)` returns midnight UTC as an AWARE datetime. Deliberately not the bare `date` the original diagnosis suggested — comparing timestamptz to date does work via an implicit cast, but Postgres resolves that cast in the SESSION's TimeZone, so the dedup window would drift with a server setting nobody remembers is load-bearing. logging: `parse_filter_datetime()` converts the query-string value to an aware UTC datetime; unparseable input returns None so the filter is skipped rather than 500ing the viewer. It also fixes a bug the naive fix would have introduced — `date_to=2026-07-30` parses to midnight, so `<=` would exclude the entire day the user asked for. Date-only upper bounds now run to 23:59:59.999999, while a value carrying an explicit time is left as given. The guard is the point. This class is invisible to ordinary testing: the failure happens when Postgres plans the query, not when Python builds it, so no unit test that doesn't execute SQL can see it. tests/test_timestamp_filters.py fails CI on two shapes — 1. a local bound to .isoformat() compared against a *_at column (#1727) 2. a str-ANNOTATED PARAMETER compared against a *_at column (#2257) Shape 2 is the one that matters. Nothing in logging.py looks date-ish, so a guard built only from #1727's shape finds nothing there — which is exactly how the second instance survived. Verified by replaying both checks against the pre-fix files out of git: shape 1 catches `today_str`, shape 2 catches `date_from`/`date_to`, and the current tree is clean. Found by grepping for siblings after fixing #1727 — the third instance today of "the second place nobody checked", after #2245. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-30 12:53:57 -04:00