Compare commits

..
2 Commits
Author SHA1 Message Date
bvandeusen 93d660b710 Tool disambiguators, kind badges, and a badge layer that clears AA (#3123, #3124, #3132)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 31s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m17s
CI & Build / Build & push image (push) Successful in 15s
2026-08-27 21:28:48 -04:00
bvandeusen 056c7c75da A task's kind is correctable — the Kind select stops lying (#3129)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 28s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 15s
2026-08-27 18:05:16 -04:00
135 changed files with 731 additions and 13652 deletions
+9 -61
View File
@@ -46,6 +46,8 @@ on:
- "alembic/**"
- "alembic.ini"
- "Dockerfile"
- "assets/**"
- "fable-mcp/**"
# The plugin ships straight from this repo — installs fetch it via
# .claude-plugin/marketplace.json, NOT from the image. So a push here is
# the release, with no build step in between. Omitting these paths meant
@@ -277,21 +279,6 @@ jobs:
env:
UV_PROJECT_ENVIRONMENT: /opt/venv
run: uv sync --locked --extra dev
# Standing answers to the checks carried by rules 81 and 79 — two facts
# about THIS runner that conditional rules assert as fact, and that
# otherwise need a throwaway job to confirm (#3237). Printing them on
# every integration run makes the next rulebook sweep a log read.
# Rule 80's evidence is the container listing the next step already
# prints. Every command is guarded: a diagnostic that can break the lane
# it observes is worse than no diagnostic.
- name: Runner facts (rules 79 and 81)
run: |
echo "--- rule 81: which shell runs a run: step ---"
readlink -f /bin/sh || echo "/bin/sh: not a symlink"
ps -p $$ -o comm= || true
echo "--- rule 79: is a service reachable by its hostname yet? ---"
getent hosts postgres \
|| echo "no — 'postgres' does not resolve; the bridge-IP lookup is still required"
- name: Integration suite (resolve service IP, migrate, test)
run: |
set -eux
@@ -302,9 +289,8 @@ jobs:
PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
test -n "$PG_IP"
export DATABASE_URL="postgresql+asyncpg://scribe:ci_integration@${PG_IP}:5432/scribe_test"
# Wait for Postgres to accept connections. The run: shell is dash
# (/bin/sh -> /usr/bin/dash on this Debian-based image, confirmed by
# the step above) — no bash /dev/tcp, so use Python.
# Wait for Postgres to accept connections (busybox sh — the runner
# default — has no bash /dev/tcp, so use Python).
/opt/venv/bin/python - "$PG_IP" <<'PY'
import socket, sys, time
for _ in range(30):
@@ -341,14 +327,6 @@ jobs:
packages: write
steps:
- uses: actions/checkout@v6
with:
# Rule 149 asks for this on any job deriving the version NAME. The
# name here comes from HEAD's commit TIME, which a depth-1 clone
# already has — but the rule states it unconditionally because the
# failure it guards is silent (a too-low value, every lane green),
# and a later change to how the name is derived would inherit the
# landmine rather than the guard.
fetch-depth: 0
- name: Generate image tags and version
id: tags
@@ -361,27 +339,7 @@ jobs:
# the runner log on commit 2a374d9.
run: |
TAGS="${{ env.IMAGE }}:${{ github.sha }}"
# THREE VALUES, NEVER FOLDED TOGETHER (rule 149). Until 2026-08-31
# BUILD_VERSION was the CHANNEL — "dev" / "main" / the tag — so the
# image self-reported {"version":"main"}, a channel name where a
# build identifier belongs. That cost a debugging session: with the
# deploy misbehaving, nothing on the running instance could say
# which commit was serving it.
# 1. ORDERING KEY — BUILD time, monotonic by construction. Minutes
# since 2020-01-01. Never a commit count (not monotonic across
# branches) and never commit time (goes DOWN when an older
# commit is rebuilt).
BUILD_KEY=$(( ( $(date -u +%s) - 1577836800 ) / 60 ))
# 2. NAME — COMMIT time, so the same source reports the same string
# on every lane and the channel is the only thing that differs.
COMMIT_TS=$(git log --format=%ct -1 HEAD)
BUILD_NAME=$(date -u -d "@$COMMIT_TS" +%Y.%m.%d.%H%M)
# 3. CHANNEL — its own value. Never a suffix, never a segment.
CHANNEL="dev"
BUILD_VERSION="dev"
case "${{ github.ref }}" in
refs/heads/dev)
TAGS="$TAGS,${{ env.IMAGE }}:dev"
@@ -390,17 +348,15 @@ jobs:
# main IS the production line: publish :latest (plus the :<sha>
# set above). No separate :main tag.
TAGS="$TAGS,${{ env.IMAGE }}:latest"
CHANNEL="stable"
BUILD_VERSION="main"
;;
refs/tags/*)
TAGS="$TAGS,${{ env.IMAGE }}:latest,${{ env.IMAGE }}:${{ github.ref_name }}"
CHANNEL="stable"
BUILD_VERSION="${{ github.ref_name }}"
;;
esac
echo "value=$TAGS" >> $GITHUB_OUTPUT
echo "build_name=$BUILD_NAME" >> $GITHUB_OUTPUT
echo "build_key=$BUILD_KEY" >> $GITHUB_OUTPUT
echo "channel=$CHANNEL" >> $GITHUB_OUTPUT
echo "build_version=$BUILD_VERSION" >> $GITHUB_OUTPUT
- name: Free disk space
# Self-hosted runner housekeeping. Two-step cleanup:
@@ -430,15 +386,7 @@ jobs:
push: true
provenance: false
tags: ${{ steps.tags.outputs.value }}
# All three, plus the commit — rule 145: the registry's identity for
# a build (:<sha>) and the artifact's identity for itself must
# agree, and they can only be checked against each other if the
# artifact says which commit it is.
build-args: |
BUILD_VERSION=${{ steps.tags.outputs.build_name }}
BUILD_KEY=${{ steps.tags.outputs.build_key }}
BUILD_CHANNEL=${{ steps.tags.outputs.channel }}
BUILD_COMMIT=${{ github.sha }}
build-args: BUILD_VERSION=${{ steps.tags.outputs.build_version }}
# Registry-backed layer cache. Pull from :cache to prime
# BuildKit, push updated layers back to :cache so the next
# build starts warm even if the runner's local cache was
+2 -21
View File
@@ -41,29 +41,10 @@ COPY alembic/ alembic/
# Ensure Python finds the source tree (where static files live) before site-packages
ENV PYTHONPATH=/app/src
# THREE VALUES, NEVER FOLDED TOGETHER (rule 149), plus the commit.
#
# BUILD_VERSION is the NAME (YYYY.MM.DD.HHMM, from COMMIT time) — the same
# string on every lane for the same source, so it answers "is this the same
# code?" rather than "which lane built it?".
# BUILD_KEY is the ORDERING KEY (minutes since 2020-01-01, from BUILD time) —
# the only value anything may compare to decide what is newer.
# BUILD_CHANNEL is its own field. Never a suffix, never a segment of the name.
# BUILD_COMMIT lets the artifact's self-report be checked against the :<sha>
# it was published under (rule 145).
#
# Each defaults to empty rather than to a placeholder, EXCEPT the name: a
# local build genuinely has no ordering key or channel, and the endpoint says
# so by omitting them. Inventing values would make a local image claim a
# position in an update order it is not part of.
# Version is injected at build time via --build-arg BUILD_VERSION=YY.MM.DD.N
# Falls back to "dev" for local / untagged builds
ARG BUILD_VERSION=dev
ARG BUILD_KEY=
ARG BUILD_CHANNEL=
ARG BUILD_COMMIT=
ENV APP_VERSION=$BUILD_VERSION
ENV APP_BUILD_KEY=$BUILD_KEY
ENV APP_CHANNEL=$BUILD_CHANNEL
ENV APP_COMMIT=$BUILD_COMMIT
EXPOSE 5000
CMD ["sh", "-c", "alembic upgrade head && hypercorn 'scribe.app:create_app()' --bind 0.0.0.0:5000 --keep-alive 600"]
+1 -10
View File
@@ -1,4 +1,4 @@
.PHONY: build up down logs health migrate lint typecheck test fmt mint-plugin
.PHONY: build up down logs health migrate lint typecheck test fmt
# --- Docker ---
@@ -36,12 +36,3 @@ test:
# Run all checks in one shot (mirrors what CI does)
check: lint typecheck test
# --- Plugin ---
# Run this after changing anything under plugin/ or .claude-plugin/, BEFORE
# committing. The plugin ships straight from git with no build step, so its
# version is minted here rather than stamped by CI; the lane fails if you
# forget, but this is what makes remembering cheap.
mint-plugin:
python3 scripts/mint_plugin_version.py
@@ -1,80 +0,0 @@
"""a note can carry its own check — verify_with, expires_when, verified_at
(milestone 317 step 1)
Revision ID: 0092
Revises: 0091
Create Date: 2026-08-28
The sibling of 0090, which gave rules the same three columns. Same
distinction, one table over:
A NORM is a decision — no truth value, and it changes only when its author
changes it, which they know they did. A CONSTRAINT is a fact about someone
else's software, and nobody is present when it goes false.
Notes hold far more constraints than rules do, and hold them for longer. A
cross-project reference note asserting what a signing service does on a
duplicate upload, or how a forge numbers its CI runs, is believed by every
project that reads it, and there is nothing in the record that says when
anyone last looked. `note_supersessions` only fires once a human has read
the note, disagreed, and written the correction — which is the case where
the note was already believed.
Three nullable columns:
- `verify_with` — how to tell whether this is still true. A command, a path,
a URL, a query. Prose is allowed; something runnable is better.
- `expires_when` — the STATE under which it stops being true. Deliberately
not a date: constraints do not expire on a schedule, they expire when the
world underneath them moves.
- `verified_at` — when the check last passed. NULL means never checked, and
sorts FIRST in the sweep: unexamined outranks examined-long-ago.
WHICH ROWS THESE ARE FOR. `notes` is one table holding notes, tasks,
snippets and processes, so these columns land on all of them. Only non-task,
non-snippet records are OFFERED them (milestone 317 decisions 1 and 2, gated
at the service in step 2): a task's decay is its status, and a snippet
already carries a richer, location-aware verdict in `data.verification`. The
columns exist on the other rows and stay null there; a gate that lives in
the schema would have meant a partial index or a CHECK across three columns
to express something the write path can say in two lines.
All three optional, because most notes should set none of them — the whole
value of the sweep is that its output is short. A null `verify_with` is not
an omission; it is the honest marker of "this one is a decision, and there
is nothing to go and check."
No CHECK constraint is involved, so rule 36 does not apply. Nothing is
backfilled: a migration cannot invent a check.
"""
import sqlalchemy as sa
from alembic import op
revision = "0092"
down_revision = "0091"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("notes", sa.Column("verify_with", sa.Text(), nullable=True))
op.add_column("notes", sa.Column("expires_when", sa.Text(), nullable=True))
op.add_column(
"notes",
sa.Column("verified_at", sa.DateTime(timezone=True), nullable=True),
)
# No index, for 0090's reason — the sweep runs when a human asks, never on
# a request path — but the margin is thinner here and worth naming. `rules`
# is hundreds of rows; `notes` is thousands and grows with every session.
#
# Still a sequential scan's job at this size, and an index on
# (verified_at) filtered to `verify_with IS NOT NULL` would be maintained
# on every note write to serve one operator-initiated query. If step 3's
# live acceptance measures otherwise, add it there against a real plan
# rather than guessing here.
def downgrade() -> None:
op.drop_column("notes", "verified_at")
op.drop_column("notes", "expires_when")
op.drop_column("notes", "verify_with")
-83
View File
@@ -1,83 +0,0 @@
"""rules gain an edit history — rule_versions (milestone 323 step 1)
Revision ID: 0093
Revises: 0092
Create Date: 2026-08-29
The sibling `note_versions` has had for a long time. A note's every meaningful
edit is snapshotted, and the design-system note calls that history "the
changelog". A RULE — which binds behaviour on every session that loads it —
had nothing: an edit destroyed what it used to say, with no record anywhere.
Rescoping rule 79 on 2026-08-29 is what surfaced it. The superseded statement
had to be hand-copied into a task log to survive the edit (#3237), which is
not a process, it is a person remembering. The more consequential record had
the weaker protection.
Three things are deliberately NOT copied from note_versions, and each is a
guard that exists there for a reason that does not hold here:
- **No pruning, and no MAX_VERSIONS.** That cap defends against note autosave
filling every slot. Rules have no autosave; every edit is a deliberate
update_rule. A rule is edited a handful of times in its life, and capping
invites losing the one edit somebody needed.
- **No pin columns.** `pin_kind`/`pin_label` exist so a note's version can
survive that pruning. With nothing pruning, a pin protects a row that was
never at risk.
- **No minimum interval.** 300 seconds between snapshots is also an autosave
defence; here it would only ever discard a second deliberate edit.
`user_id` is the ACTOR rather than the owner, and is SET NULL rather than
CASCADE: deleting a user must not erase the history of the rules they edited.
The edit still happened and the rule still binds because of it.
No CHECK constraint, so rule 36 does not apply. Nothing is backfilled — a
migration cannot invent the text a rule used to have, and inventing "the
current text, as of now" would be worse than an empty history, because it
would look like a record of an edit that never occurred.
"""
import sqlalchemy as sa
from alembic import op
revision = "0093"
down_revision = "0092"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"rule_versions",
sa.Column("id", sa.BigInteger(), primary_key=True),
sa.Column(
"rule_id",
sa.BigInteger(),
sa.ForeignKey("rules.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"user_id",
sa.BigInteger(),
sa.ForeignKey("users.id", ondelete="SET NULL"),
nullable=True,
),
sa.Column("title", sa.Text(), nullable=False, server_default=""),
sa.Column("statement", sa.Text(), nullable=False, server_default=""),
sa.Column("why", sa.Text(), nullable=True),
sa.Column("how_to_apply", sa.Text(), nullable=True),
sa.Column("when_to_apply", sa.Text(), nullable=True),
sa.Column("tier", sa.Text(), nullable=True),
sa.Column("verify_with", sa.Text(), nullable=True),
sa.Column("expires_when", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
)
# The only query this table serves is "the history of THIS rule, newest
# first" — unlike 0092's columns, which are read by an operator-initiated
# sweep over the whole set. Every read here is keyed on rule_id, so the
# index earns its write cost immediately rather than on a hunch.
op.create_index("ix_rule_versions_rule_id", "rule_versions", ["rule_id"])
def downgrade() -> None:
op.drop_index("ix_rule_versions_rule_id", table_name="rule_versions")
op.drop_table("rule_versions")
@@ -1,86 +0,0 @@
"""add rule_usage_events — was a surfaced rule ever read? (milestone 333 step 1)
Revision ID: 0094
Revises: 0093
Create Date: 2026-09-02
The sibling `note_usage_events` has had since 0071, and the third rule-side
table to arrive after `rule_embeddings` and `rule_versions` — each one added
because the rule side kept inheriting machinery built for notes and getting
the weaker version of it.
WHAT IT MEASURES. The write-path standing-rule arm is the only retrieval
surface in Scribe whose usefulness cannot be observed, and — not coincidentally
— the only one that has never declined to fire. Over 30 days it took 296 calls,
returned something on every one, and cleared its threshold 100% of the time,
while every other surface declines most of the time (#3311). That is either a
perfectly tuned surface or a bar it cannot fail to clear, and `retrieval_logs`
cannot tell them apart: it records what the ranker scored, never whether the
hint was any use.
WHY NOT A rule_id COLUMN ON note_usage_events. The row shares no note-specific
fields and the aggregate readout is the same shape, which is the strongest case
for sharing that note #3163 admits. What decides against it is identity at
RESTORE: `note_usage_events`'s importer maps `note_id` through `note_id_map`
and drops what does not resolve. A rule id parked in that column would come
back from a backup silently reattached to whatever note took that number —
telemetry not merely lost but wrong, and wrong in a way nothing downstream
could detect. `rule_versions` made the same call for the same reason.
FK-free on `rule_id` and `user_id`, matching note_usage_events, retrieval_logs
and app_logs — and deliberately unlike `rule_versions`, which does carry FKs.
The difference is what the row is for: a version belongs to a rule's history
and dies with it; telemetry outlives the row it describes. Deleting a rule must
not erase the evidence that it was surfaced forty times and opened never, since
that evidence is exactly the case for having deleted it.
No CHECK on `event`, matching the note twin. Rule 36 governs adding a value to
a column that is already gated; it does not require gating one that never was,
and a two-member enum whose members are written by two functions in one module
is not where that discipline earns its cost.
Downgrade drops the table outright. The data is purely observational — nothing
reads it for correctness, so losing it costs history and no behaviour.
"""
from alembic import op
import sqlalchemy as sa
revision = "0094"
down_revision = "0093"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"rule_usage_events",
# BigInteger throughout where the note twin uses Integer: rules.id is
# BigInteger, so rule_id must be, and a high-churn append-only table is
# a poor place to discover an id ceiling.
sa.Column("id", sa.BigInteger(), primary_key=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
nullable=False,
server_default=sa.text("now()"),
),
sa.Column("user_id", sa.BigInteger(), nullable=True),
sa.Column("rule_id", sa.BigInteger(), nullable=False),
sa.Column("event", sa.Text(), nullable=False),
sa.Column("source", sa.Text(), nullable=False),
)
# Every readout is "these rule ids, split by event", so the composite is the
# one that actually gets used; the others serve pruning and per-user views.
op.create_index(
"ix_rule_usage_rule_event", "rule_usage_events", ["rule_id", "event"]
)
op.create_index("ix_rule_usage_created_at", "rule_usage_events", ["created_at"])
op.create_index("ix_rule_usage_user_id", "rule_usage_events", ["user_id"])
def downgrade() -> None:
op.drop_index("ix_rule_usage_user_id", table_name="rule_usage_events")
op.drop_index("ix_rule_usage_created_at", table_name="rule_usage_events")
op.drop_index("ix_rule_usage_rule_event", table_name="rule_usage_events")
op.drop_table("rule_usage_events")
@@ -1,52 +0,0 @@
"""add retrieval_logs.suppressed_count — tell a ranker decline from a repeat (#3497)
Revision ID: 0095
Revises: 0094
Create Date: 2026-09-03
`result_count == 0` has always meant "this surface said nothing", which is the
right number for "was the hint any use" and the wrong one for tuning a
threshold. It folds together two unrelated events:
- the ranker found nothing above the bar — the ONLY evidence a threshold is
set too high; and
- the ranker found something the session had already been shown — a decline
that says nothing whatever about the bar.
The rule arms filter in Python after the search, so they can count the second
kind exactly. The note arms pass `exclude_ids` INTO semantic_search_notes, so
the dropped rows never come back and there is nothing to count.
NULLABLE, AND THE NULL IS THE POINT. A surface that does not measure
suppression stores NULL, not 0, and the readout renders it as "not measured"
rather than "none". Defaulting to 0 would make an unmeasured surface look like
a perfectly clean one — the exact substitution of an artifact for a
measurement that #3311 made and that #3497 exists to correct. Doing it again,
in the migration that fixes it, would be its own small joke.
No backfill for the same reason: existing rows genuinely do not know, and
saying so is the honest state. `retrieval_logs` is not restored from backup,
so no importer changes.
Downgrade drops the column. Purely observational — nothing reads it for
correctness.
"""
from alembic import op
import sqlalchemy as sa
revision = "0095"
down_revision = "0094"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"retrieval_logs",
sa.Column("suppressed_count", sa.Integer(), nullable=True),
)
def downgrade() -> None:
op.drop_column("retrieval_logs", "suppressed_count")
@@ -1,62 +0,0 @@
"""add retrieval_logs.best_available_score — the score the bar rejected (#3670)
Revision ID: 0096
Revises: 0095
Create Date: 2026-09-08
`cleared_threshold` was documented as the number to read FIRST — "a surface
that clears its bar on nearly every call is either well-tuned or too loose,
and p10 says which". It was never a measurement. The search applies the
threshold before returning, so every returned result cleared the bar by
construction and a call with no results has no `top_score` to compare:
the condition is true exactly when `result_count > 0`.
`zero_result_calls + cleared_threshold == calls` held on all nineteen
source/window readings ever taken. It was `calls - zero_result_calls`
wearing a name that promised a second opinion, and a reading procedure was
built on top of it that asked the reader to compare a number against itself.
THE MISSING NUMBER, and the reason this is a column rather than a deletion.
The question the table exists to answer is "is the bar in the right place",
and that question is only answerable from the calls that returned NOTHING:
how close did the best rejected candidate come? A bar at 0.72 turning away
a stream of 0.71s is set too high by a hair. A bar turning away 0.30s is
doing its job. Those two are indistinguishable today — both render as a
zero-result call — and no arrangement of the existing columns separates
them, because the losing score is discarded inside the search.
So the searches now rank without the bar and apply it in Python, which
costs nothing (the rows were already ordered by distance, and the qualifying
set is provably identical — above-threshold rows sort first), and the best
score seen becomes observable.
NULLABLE, AND UNBACKFILLED, for the reason 0095 spells out: a row written
before this shipped genuinely does not know what its best rejected candidate
scored, and saying so is the honest state. A 0.0 default would read as "the
corpus had nothing remotely relevant" — an artifact standing in for a
measurement, which is the whole defect this milestone corrects.
`retrieval_logs` is not restored from backup, so no importer changes.
Downgrade drops the column. Purely observational — nothing reads it for
correctness.
"""
from alembic import op
import sqlalchemy as sa
revision = "0096"
down_revision = "0095"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"retrieval_logs",
sa.Column("best_available_score", sa.Float(), nullable=True),
)
def downgrade() -> None:
op.drop_column("retrieval_logs", "best_available_score")
@@ -1,60 +0,0 @@
"""add retrieval_logs.best_available_id — WHICH record the bar refused (#3807)
Revision ID: 0097
Revises: 0096
Create Date: 2026-09-09
0096 added `best_available_score` so a threshold could be judged from what it
rejected. It records how CLOSE the bar came to firing and not WHAT it turned
away, and that turns out to be the half a decision actually needs.
Live, `pre_tool_rule` shows a bar of ~0.72 with a near-miss p90 of 0.7071 —
about 117 declines a day sitting within 0.013 of firing. Dropping the bar to
0.707 would take that arm from 22 hits a day to roughly 139, a six-fold change
on a surface that runs before every Bash call. The percentile says the mass is
there. Nothing says whether it is worth showing.
AND THE TWO OBVIOUS INSTRUMENTS DO NOT ANSWER IT. Pull-through cannot: the
injected rule line already carries title and trigger, so a session can comply
without ever calling `get_rule`, and rule pull-through therefore understates
usefulness by construction. Reading the rejected records can — and `result_ids`
holds only what was RETURNED, so on a zero-result call it is empty and the
near-missed record has no name.
So: the id, beside the score, from the SAME ranked candidate. The two must
never be able to describe different records — a score paired with its
neighbour's id would be worse than no id at all, because it invites a reader to
judge the wrong record and conclude the bar is fine.
NULLABLE AND UNBACKFILLED, for the reason 0095 and 0096 both spell out: a row
written before this genuinely does not know, and inventing a value would put an
artifact where a measurement belongs. Null here means "not measured", never
"nothing was close".
NOT A FOREIGN KEY, deliberately. `retrieval_logs` spans record types — the
rules arms store rule ids, the note arms store note ids — and `source` is what
says which table an id belongs to, exactly as `result_ids` has always worked.
A constraint would have to point at one table and would be wrong for the other.
Downgrade drops the column. Purely observational — nothing reads it for
correctness.
"""
from alembic import op
import sqlalchemy as sa
revision = "0097"
down_revision = "0096"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"retrieval_logs",
sa.Column("best_available_id", sa.Integer(), nullable=True),
)
def downgrade() -> None:
op.drop_column("retrieval_logs", "best_available_id")
+6 -19
View File
@@ -7,20 +7,12 @@ import { useTheme } from "@/composables/useTheme";
import { useShortcuts } from "@/composables/useShortcuts";
import { useAuthStore } from "@/stores/auth";
import { useSettingsStore } from "@/stores/settings";
import { apiPut } from "@/api/client";
import { fetchVersion } from "@/api/version";
import { apiGet, apiPut } from "@/api/client";
useTheme();
const router = useRouter();
// THREE states, not two (#3127 checklist 12). `null` is "not answered yet" and
// renders nothing; a string renders; `appVersionFailed` renders its own thing.
// This used to default to the literal "dev" and swallow the error, which meant
// an instance that could not answer was indistinguishable from a local build
// that genuinely reports "dev" — a blank standing in for `unknown`, in the one
// readout whose whole job is to say what is running.
const appVersion = ref<string | null>(null);
const appVersionFailed = ref(false);
const appVersion = ref("dev");
const authStore = useAuthStore();
const settingsStore = useSettingsStore();
const { showShortcuts, toggleShortcuts, closeShortcuts } = useShortcuts();
@@ -127,12 +119,10 @@ onMounted(async () => {
startAppServices();
}
try {
appVersion.value = (await fetchVersion()).version;
const data = await apiGet<{ version: string }>("/api/version");
appVersion.value = data.version;
} catch {
// Not silent any more: the footer says it could not find out, rather than
// showing a version it never received. The full readout (version, channel,
// commit, build) lives in Settings → Config.
appVersionFailed.value = true;
// silent — version display is non-critical
}
});
@@ -161,10 +151,7 @@ onUnmounted(() => {
<div id="main-content" class="app-content">
<router-view />
</div>
<footer class="app-footer">
<span v-if="appVersion">v{{ appVersion }}</span>
<span v-else-if="appVersionFailed">version unknown</span>
</footer>
<footer class="app-footer">v{{ appVersion }}</footer>
</div>
<!-- Keyboard shortcuts overlay -->
+32 -127
View File
@@ -52,121 +52,41 @@ export function apiErrorMessage(e: unknown, fallback: string): string {
return fallback;
}
/**
* How long an ordinary JSON call may wait before it is declared failed.
*
* Rule 156: a wait with no deadline is a bug. `fetch`'s own default is to wait
* as long as the browser will, which is not a deadline — it is the absence of
* one, and it renders as a spinner that never resolves. There is no state a
* surface can show for "pending forever" that is not a lie.
*
* 30s is chosen to be longer than anything healthy: it has to clear a cold
* embedding call and a list view under connection-pool contention (#2384 had
* /api/projects fanning 25 concurrent sessions at a 15-connection pool), so
* tripping it means something is genuinely wrong rather than merely busy. Slow
* BY DESIGN is a different case and passes its own value — see the callers in
* SettingsView that do.
*/
const DEFAULT_TIMEOUT_MS = 30_000;
/** HTTP 408. Not a status any Scribe route returns, so it unambiguously means
* "the client gave up" rather than anything the server said. */
const CLIENT_TIMEOUT_STATUS = 408;
/**
* How long a STREAM may take to answer with its headers.
*
* Streams are the one case a wall-clock deadline would break: a long-lived SSE
* connection is *supposed* to stay open, and `AbortSignal.timeout` would kill
* it mid-flight along with the body. But that does not exempt them from rule
* 156 — it relocates the deadline. Two different waits are involved:
*
* connect — the server answering with headers. CAN fail to answer, so it
* carries this deadline, cleared the moment headers arrive.
* stream — the body, open indefinitely on purpose. Its failure mode is
* going quiet, which a timeout cannot tell from being idle; that
* is what reconnection and Last-Event-ID are for, not this.
*
* Reading the connect as exempt because "the stream is long-lived" is the easy
* mistake here, and it leaves an unreachable server looking like a quiet one.
*/
const STREAM_CONNECT_TIMEOUT_MS = 15_000;
/**
* A signal that aborts if headers do not arrive in time, plus the `settle` to
* call once they do. After `settle()` the returned signal never fires, so the
* stream body runs unbounded — which is the intent.
*/
function connectDeadline(base: AbortSignal): { signal: AbortSignal; settle: () => void } {
const gate = new AbortController();
const timer = setTimeout(
() => gate.abort(new DOMException("stream did not connect in time", "TimeoutError")),
STREAM_CONNECT_TIMEOUT_MS,
);
return {
signal: AbortSignal.any([base, gate.signal]),
settle: () => clearTimeout(timer),
};
}
export interface RequestOpts {
/** Override the deadline. Pass one when the call is slow BY DESIGN. */
timeoutMs?: number;
}
/**
* The one place a request is actually made — every verb below goes through
* here, so the deadline cannot be forgotten by adding a sixth.
*
* EXPIRY SURFACES AS AN `ApiError`, which is rule 156's second half: the
* failure has to arrive in the shape the caller already handles. A bare
* `DOMException: TimeoutError` would reach `apiErrorMessage(e, fallback)` as
* an object with no `body`, so every catch site in the app would report its
* generic fallback and the timeout would be invisible in the very situation it
* exists to expose. Rethrowing as `ApiError` means ~330 existing call sites
* report it correctly without being touched.
*
* Only a TIMEOUT is converted. A deliberate cancellation aborts with
* `AbortError` and is left alone — a caller that cancelled its own request
* does not want it reported as a server failure.
*/
async function request<T>(path: string, init: RequestInit, opts?: RequestOpts): Promise<T> {
const timeoutMs = opts?.timeoutMs ?? DEFAULT_TIMEOUT_MS;
let res: Response;
try {
res = await fetch(path, { ...init, signal: AbortSignal.timeout(timeoutMs) });
} catch (e) {
if (e instanceof DOMException && e.name === "TimeoutError") {
throw new ApiError(CLIENT_TIMEOUT_STATUS, {
error: `The server did not answer within ${Math.round(timeoutMs / 1000)}s.`,
});
}
throw e;
}
export async function apiGet<T>(path: string): Promise<T> {
const res = await fetch(path);
return handleResponse<T>(res, path);
}
/** JSON body headers — the three write verbs sent an identical literal each. */
const JSON_HEADERS = { "Content-Type": "application/json" };
export function apiGet<T>(path: string, opts?: RequestOpts): Promise<T> {
return request<T>(path, {}, opts);
export async function apiPost<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
return handleResponse<T>(res, path);
}
export function apiPost<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
return request<T>(path, { method: "POST", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
export async function apiPut<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
return handleResponse<T>(res, path);
}
export function apiPut<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
return request<T>(path, { method: "PUT", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
export async function apiPatch<T>(path: string, body: unknown): Promise<T> {
const res = await fetch(path, {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
return handleResponse<T>(res, path);
}
export function apiPatch<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
return request<T>(path, { method: "PATCH", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
}
export function apiDelete(path: string, opts?: RequestOpts): Promise<void> {
return request<void>(path, { method: "DELETE" }, opts);
export async function apiDelete(path: string): Promise<void> {
const res = await fetch(path, { method: "DELETE" });
return handleResponse<void>(res, path);
}
// ---------------------------------------------------------------------------
@@ -301,14 +221,7 @@ export function apiSSEStream(
}
const done = (async () => {
// Bounded connect, unbounded stream — see STREAM_CONNECT_TIMEOUT_MS.
const connect = connectDeadline(combinedSignal);
let res: Response;
try {
res = await fetch(path, { headers, signal: connect.signal });
} finally {
connect.settle();
}
const res = await fetch(path, { headers, signal: combinedSignal });
if (!res.ok) {
let body: Record<string, unknown> = {};
try {
@@ -405,19 +318,11 @@ export async function apiStreamPost(
body: unknown,
onChunk: (data: Record<string, unknown>) => void
): Promise<void> {
// Bounded connect, unbounded stream — see STREAM_CONNECT_TIMEOUT_MS.
const connect = connectDeadline(new AbortController().signal);
let res: Response;
try {
res = await fetch(path, {
method: "POST",
headers: JSON_HEADERS,
body: JSON.stringify(body),
signal: connect.signal,
});
} finally {
connect.settle();
}
const res = await fetch(path, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
if (!res.ok) {
let errBody: Record<string, unknown> = {};
try {
-52
View File
@@ -1,5 +1,3 @@
import type { RecordUsage } from "@/types/usage";
import { apiGet, apiPost, apiPatch, apiDelete } from "@/api/client";
/** How a rule reaches a session (milestone 307). */
@@ -98,13 +96,6 @@ export interface RuleHeader {
* A date (YYYY-MM-DD), or the literal "never".
*/
last_verified?: string;
/**
* Surfaced-vs-opened counts from `rule_usage_events` (milestone 333).
* Zero-filled by the list route, so a rule predating the table reads as
* "never surfaced" rather than as a missing field — which for a while is
* every rule on every install.
*/
usage?: RecordUsage;
}
export interface ApplicableRules {
@@ -237,49 +228,6 @@ export async function unrelateRules(relationId: number): Promise<void> {
return apiDelete(`/api/rule-relations/${relationId}`);
}
/**
* One entry in a rule's edit history.
*
* Each entry holds the text the edit REPLACED, not the text it introduced —
* so the newest entry is what the rule said before its most recent change,
* and what that change produced is the rule as it stands now. Read the other
* way round, every diff comes out backwards.
*
* The listing form omits the long fields; open one to get them.
*/
export interface RuleVersion {
id: number;
rule_id: number;
/** Who made the edit. Null when that account has since been deleted. */
user_id: number | null;
title: string;
created_at: string;
statement?: string;
why?: string;
how_to_apply?: string;
when_to_apply?: string;
tier?: string;
verify_with?: string;
expires_when?: string;
}
export async function listRuleVersions(ruleId: number): Promise<RuleVersion[]> {
const data = await apiGet<{ versions: RuleVersion[] }>(
`/api/rules/${ruleId}/versions`,
);
return data.versions;
}
export async function getRuleVersion(
ruleId: number, versionId: number,
): Promise<RuleVersion> {
return apiGet<RuleVersion>(`/api/rules/${ruleId}/versions/${versionId}`);
}
// No restoreRuleVersion, deliberately (milestone 323). Putting an old wording
// back goes through updateRule, which snapshots what it replaces — so the
// undo stays visible in the history like any other edit.
export async function deleteRule(id: number): Promise<void> {
return apiDelete(`/api/rules/${id}`);
}
+9 -7
View File
@@ -1,5 +1,3 @@
import type { RecordUsage } from "@/types/usage";
import { apiGet, apiPost, apiPatch, apiDelete } from "@/api/client";
/** One canonical location of a reusable thing. A snippet that unifies several
@@ -52,11 +50,15 @@ export interface Snippet {
owner?: string | null;
}
/** Kept as a name because every consumer here says "snippet usage" — but it IS
* the shared shape, since rules answer the same question off their own table
* (milestone 333). The reasoning lives on `RecordUsage`; duplicating the four
* fields here is how the two drift. */
export type SnippetUsage = RecordUsage;
/** How often a record was put in front of an agent versus actually opened.
* A high `surfaced_count` with `pull_count: 0` is dead weight — it occupies a
* slot in every future auto-inject menu while never being used. */
export interface SnippetUsage {
surfaced_count: number;
pull_count: number;
last_surfaced_at: string | null;
last_pulled_at: string | null;
}
/** Result of the last drift check — does the recorded location and code still
* match source? The check runs agent-side (Scribe has no checkout); this is the
-44
View File
@@ -1,44 +0,0 @@
import { apiGet } from "./client";
/**
* What `/api/version` answers — the client's half of `build_version_payload`
* (`src/scribe/routes/api.py`), which is where the reasoning for the shape is
* written down.
*
* EVERY FIELD BUT `version` IS OPTIONAL, and an absent one means "this build
* does not know", not "empty". A local build has no ordering key and no
* channel, and the server says so by omitting the keys rather than sending
* `""` — emitting a placeholder would let it claim a position in an update
* order it is not part of.
*
* So a renderer must read ABSENCE, never falsiness. `build` is a number and
* `0` is a legitimate ordering key, so `v.build || "unknown"` would report a
* real value as unknown; `v.build ?? "unknown"` is the correct form.
*/
export interface VersionPayload {
/** The NAME — `YYYY.MM.DD.HHMM` from commit time. Answers "is this the same code?" */
version: string;
/** The ORDERING KEY — minutes since 2020-01-01, from build time. Absent on a local build. */
build?: number;
/** `dev` / `main` / a tag. Its own field, never folded into the name. */
channel?: string;
/** The commit the artifact was published under, so its claim can be checked against the registry. */
commit?: string;
}
/**
* SHORTER than the client's 30s default, deliberately.
*
* This readout answers "what is running?" during an incident, which is exactly
* when the server may be the thing that is unwell — and it is one static field
* off a route that does no work, so a healthy instance answers it immediately.
* Waiting the full default before saying so would leave a person staring at
* "still loading" for half a minute in the moment they are trying to find out
* whether the instance is alive at all. Eight seconds clears a slow-but-alive
* instance and tells them something quickly when it is not.
*/
const VERSION_TIMEOUT_MS = 8000;
export function fetchVersion(): Promise<VersionPayload> {
return apiGet<VersionPayload>("/api/version", { timeoutMs: VERSION_TIMEOUT_MS });
}
-26
View File
@@ -351,29 +351,3 @@
.required { color: var(--fs-error); }
.field-hint { margin: 0.3rem 0 0; font-size: 0.8rem; color: var(--fs-text-tertiary); }
/* --- usage badge ----------------------------------------------------------
"surfaced N×, opened M×" on a list row, for any record kind the retrieval
surfaces can choose: snippets and notes from note_usage_events, rules from
rule_usage_events. Promoted here from SnippetListView's scoped block when
the rule list needed the same chip (milestone 333 step 5) — a second scoped
copy is how the ninth duplicated CSS family starts (#3207).
Geometry and colour only. A view keeps its own spacing as a scoped
remainder, the way it does for every other recipe in this file. */
.usage-tag {
font-size: 0.7rem;
padding: 0.1rem 0.4rem;
border-radius: 4px;
white-space: nowrap;
font-variant-numeric: tabular-nums;
background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent);
color: var(--fs-text-tertiary-fg);
}
/* Dead weight is a nudge, not an error — it warns in the warning colour rather
than the danger one, because the record isn't broken, just unearned. */
.usage-tag.usage-dead {
background: color-mix(in srgb, var(--fs-warning) 18%, transparent);
color: var(--fs-warning-fg);
}
+3 -3
View File
@@ -127,7 +127,7 @@
border: 1px solid var(--fs-error);
border-radius: var(--fs-radius-sm);
font-size: 0.85rem;
color: var(--fs-error-fg);
color: var(--fs-error);
}
.diff-view {
border: 1px solid var(--fs-border-color);
@@ -147,11 +147,11 @@
}
.diff-delete {
background: color-mix(in srgb, var(--fs-error) 12%, transparent);
color: var(--fs-error-fg);
color: var(--fs-error);
}
.diff-insert {
background: color-mix(in srgb, var(--fs-success) 12%, transparent);
color: var(--fs-success-fg);
color: var(--fs-success);
}
.diff-equal {
color: var(--fs-text-tertiary);
-6
View File
@@ -31,7 +31,6 @@
--fs-accent-faint: color-mix(in srgb, var(--fs-accent) 8%, transparent); /* The faintest accent wash */
--fs-accent-deep: color-mix(in srgb, var(--fs-accent) 70%, black); /* The accent, darkened */
--fs-accent-wash: color-mix(in srgb, var(--fs-accent) 22%, transparent); /* Heaviest accent tint */
--fs-accent-fg: color-mix(in srgb, var(--fs-accent) 45%, var(--fs-text-primary)); /* Accent TEXT on an accent tint */
--fs-gradient-cta: linear-gradient(135deg, var(--fs-accent), var(--fs-accent-deep));
--fs-glow-cta: 0 2px 10px color-mix(in srgb, var(--fs-accent) 35%, transparent);
--fs-glow-cta-hover: 0 4px 24px color-mix(in srgb, var(--fs-accent) 65%, transparent);
@@ -103,11 +102,8 @@
/* semantic */
--fs-success: var(--fs-action-primary);
--fs-success-fg: color-mix(in srgb, var(--fs-success) 45%, var(--fs-text-primary)); /* Success TEXT on a success tint */
--fs-warning: #8B6F1E;
--fs-warning-fg: color-mix(in srgb, var(--fs-warning) 50%, var(--fs-text-primary)); /* Warning TEXT on a warning tint */
--fs-error: #C04A1F;
--fs-error-fg: color-mix(in srgb, var(--fs-error) 50%, var(--fs-text-primary)); /* Error TEXT on an error tint */
--fs-info: #3D5A6E;
--fs-destructive: #6B2118; /* irreversible — deliberately not the error colour */
@@ -152,9 +148,7 @@
/* text */
--fs-text-primary: #E8E4D8; /* body, headings, labels — inverts by mode */
--fs-text-secondary: #C2BFB4;
--fs-text-secondary-fg: color-mix(in srgb, var(--fs-text-secondary) 90%, var(--fs-text-primary)); /* Secondary TEXT on a secondary tint (barely moves; no exceptions) */
--fs-text-tertiary: #9C9A92;
--fs-text-tertiary-fg: color-mix(in srgb, var(--fs-text-tertiary) 55%, var(--fs-text-primary)); /* Tertiary TEXT on a tertiary tint */
--fs-text-on-action: #E8E4D8; /* text on a filled colour — NOT mode-dependent */
/* type */
+1 -1
View File
@@ -25,7 +25,7 @@
border-color: var(--fs-accent);
}
.ctx-crumb-project {
color: var(--fs-accent-fg);
color: var(--fs-accent);
background: color-mix(in srgb, var(--fs-accent) 10%, transparent);
border: 1px solid color-mix(in srgb, var(--fs-accent) 30%, transparent);
text-decoration: none;
+2 -2
View File
@@ -206,7 +206,7 @@ router.afterEach(() => {
background: var(--fs-accent-soft);
}
.nav-link.router-link-active {
color: var(--fs-accent-fg);
color: var(--fs-accent);
font-weight: 500;
background: color-mix(in srgb, var(--fs-accent) 25%, transparent);
box-shadow: 0 0 16px color-mix(in srgb, var(--fs-accent) 30%, transparent);
@@ -257,7 +257,7 @@ router.afterEach(() => {
font-weight: 500;
text-transform: uppercase;
letter-spacing: 0.05em;
color: var(--fs-accent-fg);
color: var(--fs-accent);
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
padding: 0.1rem 0.35rem;
border-radius: var(--fs-radius-sm);
+2 -2
View File
@@ -137,12 +137,12 @@ function markerFor(type: DiffLine['type']): string {
.diff-delete {
background: color-mix(in srgb, var(--fs-error) 12%, transparent);
color: var(--fs-error-fg);
color: var(--fs-error);
}
.diff-insert {
background: color-mix(in srgb, var(--fs-success) 12%, transparent);
color: var(--fs-success-fg);
color: var(--fs-success);
}
.diff-equal {
+22 -2
View File
@@ -2,7 +2,7 @@
import { ref, computed, onMounted } from "vue";
import { apiGet, pinNoteVersion, unpinNoteVersion } from "@/api/client";
import DiffView from "@/components/DiffView.vue";
import { computeDiff, type DiffLine } from "@/utils/diff";
import type { DiffLine } from "@/composables/useAssist";
import { fmtStamp } from "@/utils/dateFormat";
interface NoteVersion {
@@ -33,8 +33,28 @@ const loadingDetail = ref(false);
const diff = computed<DiffLine[]>(() => {
if (!selectedVersion.value?.body) return [];
const a = props.currentBody;
const b = selectedVersion.value.body;
return computeDiff(props.currentBody, selectedVersion.value.body);
const aLines = a.split('\n');
const bLines = b.split('\n');
const m = aLines.length, n = bLines.length;
const dp: number[][] = Array.from({ length: m + 1 }, () => new Array(n + 1).fill(0));
for (let i = m - 1; i >= 0; i--)
for (let j = n - 1; j >= 0; j--)
dp[i][j] = aLines[i] === bLines[j]
? dp[i+1][j+1] + 1
: Math.max(dp[i+1][j], dp[i][j+1]);
const result: DiffLine[] = [];
let i = 0, j = 0;
while (i < m && j < n) {
if (aLines[i] === bLines[j]) { result.push({ type: 'equal', text: aLines[i++] }); j++; }
else if (dp[i+1][j] >= dp[i][j+1]) result.push({ type: 'delete', text: aLines[i++] });
else result.push({ type: 'insert', text: bLines[j++] });
}
while (i < m) result.push({ type: 'delete', text: aLines[i++] });
while (j < n) result.push({ type: 'insert', text: bLines[j++] });
return result;
});
async function loadVersions() {
@@ -227,11 +227,11 @@ const markers: Record<DiffLine["type"], string> = {
.iap-diff-equal { color: var(--fs-text-tertiary); }
.iap-diff-delete {
background: color-mix(in srgb, var(--fs-error) 10%, transparent);
color: var(--fs-error-fg);
color: var(--fs-error);
}
.iap-diff-insert {
background: color-mix(in srgb, var(--fs-success) 10%, transparent);
color: var(--fs-success-fg);
color: var(--fs-success);
}
.iap-diff-marker {
+1 -1
View File
@@ -156,7 +156,7 @@ const groups = [
.md-btn.active {
background: color-mix(in srgb, var(--fs-accent) 14%, transparent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
box-shadow: 0 0 0 1px color-mix(in srgb, var(--fs-accent) 35%, transparent);
}
-198
View File
@@ -1,198 +0,0 @@
<script setup lang="ts">
/**
* The staleness sweep for NOTES: notes that assert a fact, oldest first.
*
* Sibling of RuleSweepPane, not a shared component — the two read differently
* enough that merging them would mean a prop for every difference (a rule has
* a tier and a statement; a note has a project and opens at a route). What
* they share is the SHAPE of the judgement, and that is worth copying
* deliberately rather than abstracting: the ordering carries urgency, "never"
* is categorically different from a date, and a failed check writes nothing.
*
* Lives in the Knowledge view rather than beside the rules sweep (operator's
* call, milestone 317 step 4): notes stay where notes live. The cost, accepted
* knowingly, is that there is no single screen showing every record anyone has
* left unconfirmed — /rules keeps its own.
*/
import { onMounted, ref } from "vue";
import { apiGet, apiPost } from "@/api/client";
import { useToastStore } from "@/stores/toast";
interface DueNote {
id: number;
title: string;
project_id: number | null;
verify_with: string;
expires_when: string;
last_verified: string | null;
days_since_verified: number | null;
}
const emit = defineEmits<{ "open-note": [id: number] }>();
const toast = useToastStore();
const rows = ref<DueNote[]>([]);
const loading = ref(false);
const neverOnly = ref(false);
const busyId = ref<number | null>(null);
async function reload() {
loading.value = true;
try {
const p = new URLSearchParams();
if (neverOnly.value) p.set("never_only", "1");
const data = await apiGet<{ notes: DueNote[] }>(
`/api/notes/due-for-verification?${p}`,
);
rows.value = data.notes;
} catch {
toast.show("Could not load the sweep", "error");
} finally {
loading.value = false;
}
}
async function verify(id: number, stillTrue: boolean) {
busyId.value = id;
try {
await apiPost(`/api/notes/${id}/verify`, { still_true: stillTrue });
if (stillTrue) {
// It has been confirmed, so it leaves the list — the sweep shows what
// still needs looking at, and leaving it in place would invite a second
// stamp nobody earned.
rows.value = rows.value.filter((r) => r.id !== id);
toast.show("Recorded — checked today");
} else {
// It stays. A failed check writes nothing on purpose: the note is wrong
// rather than in a state worth recording, so it keeps its place until
// someone corrects, supersedes, or unhooks it.
toast.show("Recorded as no longer true — the note keeps its place here");
}
} catch {
toast.show("Could not record that", "error");
} finally {
busyId.value = null;
}
}
onMounted(reload);
defineExpose({ reload });
</script>
<template>
<section class="sweep">
<header>
<h2>Due for verification</h2>
<p class="lede">
Notes that assert a fact about something outside your control what a
service does, how a tool behaves. Most notes are decisions and never
appear here; they have no truth value to go stale.
</p>
</header>
<div class="filters">
<label class="filter">
<input v-model="neverOnly" type="checkbox" @change="reload" />
<span>Never checked only</span>
</label>
</div>
<p v-if="loading" class="state">Loading</p>
<!-- An empty sweep is GOOD NEWS and must not read like a broken page. -->
<p v-else-if="!rows.length" class="state empty">
Nothing to check.
{{ neverOnly
? "Every note that carries a check has been confirmed at least once."
: "No note carries a check yet add one to a note that asserts a fact." }}
</p>
<ol v-else class="rows">
<li v-for="n in rows" :key="n.id" class="row">
<div class="row-head">
<button class="row-title" @click="emit('open-note', n.id)">{{ n.title }}</button>
<span class="age" :class="{ unchecked: n.days_since_verified === null }">
{{ n.days_since_verified === null
? "never checked"
: `${n.days_since_verified}d ago` }}
</span>
</div>
<dl class="check">
<dt>Check</dt>
<dd>{{ n.verify_with }}</dd>
<template v-if="n.expires_when">
<dt>Ends when</dt>
<dd>{{ n.expires_when }}</dd>
</template>
</dl>
<div class="actions">
<button :disabled="busyId === n.id" @click="verify(n.id, true)">Still true</button>
<button :disabled="busyId === n.id" @click="verify(n.id, false)">No longer true</button>
</div>
</li>
</ol>
<p v-if="rows.length" class="footnote">
Record a result only after actually running the check. No longer true stores nothing
on purpose the note is wrong rather than in a state worth recording, so it keeps its
place here until you correct it, supersede it, or remove its check.
</p>
</section>
</template>
<style scoped>
.sweep { display: flex; flex-direction: column; gap: var(--fs-space-3); }
h2 { margin: 0; font-size: 1.05rem; }
.lede {
margin: 0.35rem 0 0;
max-width: 62ch;
font-size: 0.85rem;
color: var(--fs-text-secondary);
line-height: 1.5;
}
.filters { display: flex; gap: var(--fs-space-5); align-items: center; flex-wrap: wrap; }
.filter { display: flex; align-items: center; gap: var(--fs-space-2); font-size: 0.82rem; color: var(--fs-text-secondary); }
.filter input[type="checkbox"] { accent-color: var(--fs-accent); }
.state { margin: 0; font-size: 0.9rem; color: var(--fs-text-secondary); }
.state.empty { color: var(--fs-text-tertiary); }
.rows { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: var(--fs-space-3); }
.row {
background: var(--fs-surface-raised);
border-radius: var(--fs-radius-md);
padding: var(--fs-space-3);
}
.row-head { display: flex; align-items: baseline; gap: var(--fs-space-2); flex-wrap: wrap; }
.row-title {
background: none; border: none; padding: 0; cursor: pointer;
font-family: Fraunces, serif; font-style: italic; font-size: 1.02rem;
color: var(--fs-text-primary); text-align: left;
}
.row-title:hover { text-decoration: underline; }
/* The ORDER carries urgency — the top of this list is the least-confirmed
thing in the corpus. No red/amber ramp: it would restate the ordering and
force an invented "stale after N days" threshold. "Never" is marked because
it is categorically DIFFERENT from a date, not a worse one. */
.age { margin-left: auto; font-size: 0.78rem; color: var(--fs-text-secondary); font-variant-numeric: tabular-nums; }
.age.unchecked { font-style: italic; color: var(--fs-text-tertiary); }
.check { display: grid; grid-template-columns: auto 1fr; gap: 0.15rem var(--fs-space-3); margin: var(--fs-space-3) 0 0; }
.check dt { font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.05em; color: var(--fs-text-tertiary); }
.check dd { margin: 0; font-size: 0.82rem; color: var(--fs-text-primary); min-width: 0; overflow-wrap: anywhere; }
.actions { display: flex; gap: var(--fs-space-2); margin-top: var(--fs-space-3); }
.actions button {
cursor: pointer; font: inherit; font-size: 0.78rem;
background: var(--fs-surface-page); color: var(--fs-text-primary);
border: 1px solid var(--fs-border-color); border-radius: var(--fs-radius-md);
padding: 0.25rem 0.6rem;
}
.actions button:hover:not(:disabled) { background: var(--fs-surface-hover); }
.actions button:disabled { opacity: var(--fs-disabled-opacity); cursor: default; }
.footnote { margin: 0; max-width: 62ch; font-size: 0.78rem; color: var(--fs-text-tertiary); line-height: 1.45; }
</style>
+2 -2
View File
@@ -679,7 +679,7 @@ async function confirmDelete() {
font-weight: 500;
background: color-mix(in srgb, var(--fs-accent) 12%, transparent);
border: 1px solid color-mix(in srgb, var(--fs-accent) 30%, transparent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
border-radius: 999px;
padding: 0.05rem 0.45rem;
flex-shrink: 0;
@@ -689,7 +689,7 @@ async function confirmDelete() {
font-weight: 500;
text-transform: uppercase;
letter-spacing: 0.04em;
color: var(--fs-text-tertiary-fg);
color: var(--fs-text-tertiary);
background: color-mix(in srgb, var(--fs-text-tertiary) 12%, transparent);
border-radius: 999px;
padding: 0.05rem 0.45rem;
+1 -1
View File
@@ -168,7 +168,7 @@ function focusInput() {
border-radius: 999px;
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
border: 1px solid var(--fs-accent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
font-size: 0.8rem;
white-space: nowrap;
}
-62
View File
@@ -1,62 +0,0 @@
<script setup lang="ts">
/**
* "N/M used" on a list row — surfaced vs opened, for any record kind.
*
* Extracted from SnippetListView when the rule list needed the same chip
* (milestone 333 step 5). The counts read identically for both; what differs
* is the ADVICE, which is why that is a prop. A snippet surfaced repeatedly
* and never opened should probably go; a rule in the same position may simply
* have a `when_to_apply` that fires on the wrong thing, and telling an
* operator to delete it would be the wrong nudge half the time.
*/
import type { RecordUsage } from "@/types/usage";
const props = defineProps<{
usage?: RecordUsage | null;
/** What to suggest when this record looks like dead weight. Appended to the
* tooltip; kind-specific, because the remedies are. */
deadWeightAdvice: string;
/** What the record is called in the tooltip's own sentence. */
noun?: string;
}>();
/** Offered repeatedly and never opened. Three rather than one because one or
* two surfacings is noise — the record may simply not have come up in a
* relevant context yet. */
const isDeadWeight = () =>
!!props.usage && props.usage.pull_count === 0 && props.usage.surfaced_count >= 3;
/** "" renders nothing. A record nobody has surfaced yet gets no badge at all:
* "0/0" would read as a verdict when it is an absence of evidence — and on a
* freshly-migrated install that is every row. */
const label = () => {
const u = props.usage;
if (!u || u.surfaced_count === 0) return "";
return `${u.pull_count}/${u.surfaced_count} used`;
};
const title = () => {
const u = props.usage;
if (!u) return "";
const last = u.last_pulled_at
? `Last opened ${new Date(u.last_pulled_at).toLocaleDateString()}.`
: "Never opened.";
const verdict = isDeadWeight() ? ` ${props.deadWeightAdvice}` : "";
return (
`Surfaced to an agent ${u.surfaced_count}×, opened in full ` +
`${u.pull_count}×. ${last}${verdict}`
);
};
</script>
<template>
<span
v-if="label()"
class="usage-tag"
:class="{ 'usage-dead': isDeadWeight() }"
:title="title()"
>{{ label() }}</span>
</template>
<!-- The look lives in components.css (canon). Nothing scoped here on purpose:
a view that needs different spacing keeps that as its own remainder. -->
@@ -2,7 +2,7 @@
import { ref, computed } from "vue";
import { apiGet } from "@/api/client";
import DiffView from "@/components/DiffView.vue";
import { computeDiff, type DiffLine } from "@/utils/diff";
import type { DiffLine } from "@/composables/useAssist";
interface NoteVersion {
id: number;
@@ -31,7 +31,25 @@ const loadingDetail = ref(false);
const diff = computed<DiffLine[]>(() => {
if (!selectedVersion.value?.body) return [];
return computeDiff(props.currentBody, selectedVersion.value.body);
const aLines = props.currentBody.split("\n");
const bLines = selectedVersion.value.body.split("\n");
const m = aLines.length, n = bLines.length;
const dp: number[][] = Array.from({ length: m + 1 }, () => new Array(n + 1).fill(0));
for (let i = m - 1; i >= 0; i--)
for (let j = n - 1; j >= 0; j--)
dp[i][j] = aLines[i] === bLines[j]
? dp[i + 1][j + 1] + 1
: Math.max(dp[i + 1][j], dp[i][j + 1]);
const result: DiffLine[] = [];
let i = 0, j = 0;
while (i < m && j < n) {
if (aLines[i] === bLines[j]) { result.push({ type: "equal", text: aLines[i++] }); j++; }
else if (dp[i + 1][j] >= dp[i][j + 1]) result.push({ type: "delete", text: aLines[i++] });
else result.push({ type: "insert", text: bLines[j++] });
}
while (i < m) result.push({ type: "delete", text: aLines[i++] });
while (j < n) result.push({ type: "insert", text: bLines[j++] });
return result;
});
function formatDate(iso: string): string {
@@ -548,7 +548,7 @@ defineExpose({ reload: loadProjectNotes });
.note-tag-pill {
font-size: 0.58rem;
color: var(--fs-accent-fg);
color: var(--fs-accent);
background: color-mix(in srgb, var(--fs-accent) 10%, transparent);
border-radius: 999px;
padding: 0 0.3rem;
@@ -645,7 +645,7 @@ defineExpose({ reload: loadProjectNotes });
.btn-tag-suggestion.applied {
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
border-color: var(--fs-accent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
}
.link-suggest-strip {
@@ -424,8 +424,8 @@ defineExpose({ reload: loadAll });
border-radius: 10px;
text-transform: capitalize;
}
.ms-status-active { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent-fg); }
.ms-status-completed { background: color-mix(in srgb, var(--fs-success) 15%, transparent); color: var(--fs-success-fg); }
.ms-status-active { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent); }
.ms-status-completed { background: color-mix(in srgb, var(--fs-success) 15%, transparent); color: var(--fs-success); }
.task-items {
list-style: none;
@@ -516,8 +516,8 @@ defineExpose({ reload: loadAll });
user-select: none;
margin-left: auto;
}
.status-cycler.status-in_progress { border-color: var(--fs-accent); color: var(--fs-accent-fg); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
.status-cycler.status-done { border-color: var(--fs-success); color: var(--fs-success-fg); background: color-mix(in srgb, var(--fs-success) 10%, transparent); }
.status-cycler.status-in_progress { border-color: var(--fs-accent); color: var(--fs-accent); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
.status-cycler.status-done { border-color: var(--fs-success); color: var(--fs-success); background: color-mix(in srgb, var(--fs-success) 10%, transparent); }
.btn-edit-task { margin-left: 0.25rem; }
.btn-edit-task:hover { text-decoration: underline; }
@@ -3,7 +3,6 @@ import { computed, ref, watch, onMounted } from "vue";
import { useRulebooksStore } from "@/stores/rulebooks";
import { useCanonicalSystemsStore } from "@/stores/canonicalSystems";
import type { RuleTier } from "@/api/rulebooks";
import RuleHistoryPanel from "@/components/rules/RuleHistoryPanel.vue";
const props = defineProps<{ ruleId: number | null; topicId: number | null }>();
const emit = defineEmits<{ close: [] }>();
@@ -257,17 +256,6 @@ watch(() => props.ruleId, load);
How to apply
<textarea v-model="howToApply" rows="4" placeholder="When / where this kicks in." />
</label>
<!-- Only on an existing rule: a rule being created has no past, and an
"Edit history — none" line on a blank form reads as a broken panel.
Keyed on ruleId so switching rules reloads rather than showing the
previous rule's history under the new one's text. -->
<RuleHistoryPanel
v-if="!isCreating && ruleId !== null"
:key="ruleId"
:rule-id="ruleId"
:current="store.currentRule"
/>
</aside>
</div>
</template>
@@ -1,274 +0,0 @@
<script setup lang="ts">
/**
* What a rule USED TO SAY — inside the slide-over, where a rule is read in
* full. Not on the list row: a history entry point there would compete with
* the row's actual job.
*
* A SIBLING OF HistoryPanel.vue, NOT A REUSE OF IT, and the reason is in its
* props: `noteId` + `currentBody`, a `NoteVersion` carrying tags and pin
* columns, a fetch of /api/notes/…, a `restore` emit, and pin/unpin buttons.
* Every one of those is note-shaped. Rules have no tags, no pins, and
* deliberately no restore, and a rule's text is EIGHT fields rather than one
* body — which changes the central question from "what changed" to "which
* fields moved".
*
* What was genuinely shared is shared: DiffView.vue takes DiffLine[] and
* nothing note-shaped, and the LCS walk now lives in utils/diff.ts, which
* this file uses rather than copying a fourth time (#3207).
*/
import { computed, onMounted, ref, watch } from "vue";
import DiffView from "@/components/DiffView.vue";
import { computeDiff } from "@/utils/diff";
import {
listRuleVersions, getRuleVersion, type Rule, type RuleVersion,
} from "@/api/rulebooks";
import { useToastStore } from "@/stores/toast";
const props = defineProps<{ ruleId: number; current: Rule | null }>();
const toast = useToastStore();
const versions = ref<RuleVersion[]>([]);
const selected = ref<RuleVersion | null>(null);
const expanded = ref(false);
const loading = ref(false);
const loadingDetail = ref(false);
// The eight TEXT fields a version carries, in the order the editor shows
// them. Narrowed to its own type rather than `keyof RuleVersion`, which would
// also admit id/rule_id/user_id/created_at — none of which is text a reader
// compares, and all of which would widen every lookup below to `number`.
// Labels rather than column names: a reader is deciding whether to open a
// row, and "How to apply" reads where "how_to_apply" has to be decoded.
type TextField =
| "title" | "statement" | "when_to_apply" | "tier"
| "why" | "how_to_apply" | "verify_with" | "expires_when";
const FIELDS: Array<[TextField, string]> = [
["title", "Title"],
["statement", "Statement"],
["when_to_apply", "When to apply"],
["tier", "Tier"],
["why", "Why"],
["how_to_apply", "How to apply"],
["verify_with", "Check"],
["expires_when", "Ends when"],
];
/**
* Which fields this edit moved.
*
* A version holds the text the edit REPLACED, so the edit is the step from
* this row to the NEXT NEWER state — the version above it in the list, or,
* for the newest row, the rule as it stands now. Comparing against the row
* below instead would attribute every change to the wrong edit.
*/
function changedFields(index: number): string[] {
const before = versions.value[index];
// `Rule` carries all eight as required strings; a RuleVersion carries them
// only once opened, which is what the undefined check below is about.
const after: Pick<Rule, TextField> | RuleVersion | null =
index === 0 ? props.current : versions.value[index - 1] ?? null;
if (!before || !after) return [];
return FIELDS
.filter(([key]) => {
// A listing row carries only the title; the rest arrive when opened.
// Undefined means NOT LOADED, which is not the same as unchanged — so a
// field nobody has fetched is claimed as neither.
const a = before[key];
const b = after[key];
if (a === undefined || b === undefined) return false;
return (a ?? "") !== (b ?? "");
})
.map(([, label]) => label);
}
/** True when this edit rewrote or removed the rule's check.
*
* Worth its own marker because editing `verify_with` silently drops
* `verified_at` (milestone 312) — the moment a rule re-entered the staleness
* sweep. That happens nowhere a reader can see it, and this row is the only
* surface that can say when it happened. */
function checkChanged(index: number): boolean {
return changedFields(index).includes("Check");
}
const diff = computed(() => {
if (!selected.value || selected.value.statement === undefined) return [];
const now = props.current?.statement ?? "";
return computeDiff(now, selected.value.statement);
});
function stamp(iso: string): string {
return iso.slice(0, 10);
}
async function load() {
loading.value = true;
try {
versions.value = await listRuleVersions(props.ruleId);
} catch {
toast.show("Could not load this rule's history", "error");
} finally {
loading.value = false;
}
}
async function open(v: RuleVersion) {
if (selected.value?.id === v.id) {
selected.value = null;
return;
}
loadingDetail.value = true;
try {
const full = await getRuleVersion(props.ruleId, v.id);
// Merged back into the list so `changedFields` can compare against real
// text once a neighbour has been opened, instead of staying blind.
const at = versions.value.findIndex((x) => x.id === v.id);
if (at >= 0) versions.value[at] = { ...versions.value[at], ...full };
selected.value = versions.value[at] ?? full;
} catch {
toast.show("Could not open that version", "error");
} finally {
loadingDetail.value = false;
}
}
onMounted(load);
watch(() => props.ruleId, () => { selected.value = null; load(); });
</script>
<template>
<section class="history">
<button class="toggle" :aria-expanded="expanded" @click="expanded = !expanded">
<span>Edit history</span>
<span class="count">{{ versions.length || "none" }}</span>
</button>
<div v-if="expanded" class="body">
<p v-if="loading" class="state">Loading</p>
<!-- Never reworded is the ordinary case, and must not read as a fault. -->
<p v-else-if="!versions.length" class="state empty">
This rule has never been reworded. Nothing was recorded before the history
existed, so an older rule starts empty too.
</p>
<template v-else>
<p class="lede">
Each entry is what the rule said <em>before</em> that edit. The wording it
was changed to is the rule as it stands above.
</p>
<ol class="rows">
<li v-for="(v, i) in versions" :key="v.id" class="row">
<button
class="row-head"
:class="{ open: selected?.id === v.id }"
@click="open(v)"
>
<span class="when">{{ stamp(v.created_at) }}</span>
<span class="fields">
{{ changedFields(i).join(", ") || "opened to compare" }}
</span>
<span v-if="checkChanged(i)" class="check-moved">check reset</span>
</button>
<div v-if="selected?.id === v.id" class="detail">
<p v-if="loadingDetail" class="state">Loading</p>
<template v-else>
<p v-if="checkChanged(i)" class="warn">
This edit changed the rule's check, which cleared its verification
stamp the rule went back to the top of the staleness sweep here.
</p>
<dl class="fields-list">
<template v-for="[key, label] in FIELDS" :key="key">
<template v-if="key !== 'statement' && v[key]">
<dt>{{ label }}</dt>
<dd>{{ v[key] }}</dd>
</template>
</template>
</dl>
<h4>Statement</h4>
<DiffView v-if="diff.length" :diff="diff" />
<p v-else class="state">The statement did not change in this edit.</p>
</template>
</div>
</li>
</ol>
</template>
</div>
</section>
</template>
<style scoped>
.history { border-top: 1px solid var(--fs-border-color); padding-top: var(--fs-space-3); }
.toggle {
display: flex; align-items: center; gap: var(--fs-space-2); width: 100%;
background: none; border: none; padding: 0; cursor: pointer;
font: inherit; font-size: var(--fs-size-body-sm); color: var(--fs-text-secondary);
}
.toggle:hover { color: var(--fs-text-primary); }
.count {
margin-left: auto; font-size: var(--fs-size-tiny); color: var(--fs-text-tertiary);
font-variant-numeric: tabular-nums;
}
.body { margin-top: var(--fs-space-3); display: flex; flex-direction: column; gap: var(--fs-space-3); }
.state { margin: 0; font-size: var(--fs-size-body-sm); color: var(--fs-text-secondary); }
.state.empty { color: var(--fs-text-tertiary); }
.lede {
margin: 0; max-width: 62ch; font-size: var(--fs-size-tiny);
color: var(--fs-text-tertiary); line-height: var(--fs-leading-body);
}
.rows { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: var(--fs-space-2); }
.row { background: var(--fs-surface-raised); border-radius: var(--fs-radius-md); }
.row-head {
display: flex; align-items: baseline; gap: var(--fs-space-3); width: 100%;
background: none; border: none; cursor: pointer; text-align: left;
padding: var(--fs-space-2) var(--fs-space-3);
font: inherit; font-size: var(--fs-size-body-sm); color: var(--fs-text-primary);
}
.row-head:hover { background: var(--fs-surface-hover); border-radius: var(--fs-radius-md); }
.when {
font-variant-numeric: tabular-nums; color: var(--fs-text-secondary);
font-size: var(--fs-size-tiny);
}
.fields { color: var(--fs-text-primary); min-width: 0; overflow-wrap: anywhere; }
/* A TINT, not the solid token. `--fs-warning-fg` is defined as "warning text
ON A WARNING TINT" — painting it over solid `--fs-warning` is the same-hue
contrast failure #3141 records. The 12% mix is how theme.css builds its own
`-bg` pairs, and it keeps the value a resolvable var() rather than a raw hex
that check_design_tokens.py cannot see at all. */
.check-moved {
margin-left: auto; flex: none;
background: color-mix(in srgb, var(--fs-warning) 12%, transparent);
color: var(--fs-warning-fg);
border-radius: var(--fs-radius-pill);
padding: 0.1rem 0.5rem;
font-size: var(--fs-size-tiny); letter-spacing: var(--fs-tracking-tiny);
}
.detail {
padding: 0 var(--fs-space-3) var(--fs-space-3);
display: flex; flex-direction: column; gap: var(--fs-space-2);
}
.warn {
margin: 0; font-size: var(--fs-size-tiny); line-height: var(--fs-leading-body);
color: var(--fs-warning-fg);
background: color-mix(in srgb, var(--fs-warning) 12%, transparent);
border-radius: var(--fs-radius-sm); padding: var(--fs-space-2);
}
.fields-list { display: grid; grid-template-columns: auto 1fr; gap: 0.15rem var(--fs-space-3); margin: 0; }
.fields-list dt {
font-size: var(--fs-size-tiny); text-transform: uppercase;
letter-spacing: var(--fs-tracking-tiny); color: var(--fs-text-tertiary);
}
.fields-list dd {
margin: 0; font-size: var(--fs-size-body-sm);
color: var(--fs-text-primary); min-width: 0; overflow-wrap: anywhere;
}
h4 { margin: var(--fs-space-2) 0 0; font-size: var(--fs-size-tiny); color: var(--fs-text-tertiary); }
</style>
@@ -1,16 +1,5 @@
<script setup lang="ts">
import type { RuleHeader } from "@/api/rulebooks";
import UsageBadge from "@/components/UsageBadge.vue";
/** The dead-weight nudge for a RULE — two remedies, not one, which is the
* whole reason this advice is per-kind. A snippet nobody opens should
* probably go. A rule nobody opens may be perfectly good and simply firing on
* the wrong thing, so "delete it" would be the wrong nudge half the time and
* the operator has to be the one who picks. */
const RULE_DEAD_WEIGHT =
"Kept arriving without being read. Either its trigger fires on the wrong " +
"work — reword “when to apply” so it says when — or it is not wanted here. " +
"Until one or the other, it takes a slot in every write it matches.";
defineProps<{ topicId: number; rules: RuleHeader[] }>();
const emit = defineEmits<{
@@ -39,7 +28,6 @@ const emit = defineEmits<{
? 'Asserts a fact nobody has confirmed yet'
: `Check last passed ${r.last_verified}`"
>{{ r.last_verified === "never" ? "unverified" : `checked ${r.last_verified}` }}</span>
<UsageBadge :usage="r.usage" :dead-weight-advice="RULE_DEAD_WEIGHT" />
</div>
<div class="statement">{{ r.statement }}</div>
<div v-if="r.when_to_apply || r.updated_at" class="meta">
+26 -5
View File
@@ -1,5 +1,4 @@
import { ref, computed, watch, type Ref } from "vue";
import { computeDiff, type DiffLine } from "@/utils/diff";
import { apiPost, apiPut, apiDelete, apiSSEStream, type SSEStreamHandle } from "@/api/client";
import { useToastStore } from "@/stores/toast";
import {
@@ -10,16 +9,17 @@ import {
export type AssistState = "idle" | "streaming" | "review";
export type ScopeMode = "document" | "section";
// Re-exported: this composable was where DiffLine lived before the diff
// moved to a shared util, and every consumer still imports the type from here.
export type { DiffLine };
export interface AssistTarget {
text: string;
startOffset: number;
endOffset: number;
}
export interface DiffLine {
type: 'equal' | 'delete' | 'insert';
text: string;
}
export interface NoteDraft {
id: number;
note_id: number;
@@ -31,6 +31,27 @@ export interface NoteDraft {
updated_at: string;
}
function computeDiff(a: string, b: string): DiffLine[] {
const aLines = a.split('\n');
const bLines = b.split('\n');
const m = aLines.length, n = bLines.length;
const dp: number[][] = Array.from({ length: m + 1 }, () => new Array(n + 1).fill(0));
for (let i = m - 1; i >= 0; i--)
for (let j = n - 1; j >= 0; j--)
dp[i][j] = aLines[i] === bLines[j]
? dp[i+1][j+1] + 1
: Math.max(dp[i+1][j], dp[i][j+1]);
const result: DiffLine[] = [];
let i = 0, j = 0;
while (i < m && j < n) {
if (aLines[i] === bLines[j]) { result.push({ type: 'equal', text: aLines[i++] }); j++; }
else if (dp[i+1][j] >= dp[i][j+1]) result.push({ type: 'delete', text: aLines[i++] });
else result.push({ type: 'insert', text: bLines[j++] });
}
while (i < m) result.push({ type: 'delete', text: aLines[i++] });
while (j < n) result.push({ type: 'insert', text: bLines[j++] });
return result;
}
export function useAssist(body: Ref<string>, noteId?: Ref<number | null>, projectId?: Ref<number | null>) {
const toast = useToastStore();
+1 -7
View File
@@ -31,8 +31,6 @@ export const useNotesStore = defineStore("notes", () => {
project_id?: number | null;
milestone_id?: number | null;
note_type?: string;
verify_with?: string;
expires_when?: string;
}): Promise<Note> {
try {
return await apiPost<Note>("/api/notes", data);
@@ -44,11 +42,7 @@ export const useNotesStore = defineStore("notes", () => {
async function updateNote(
id: number,
data: Partial<Pick<
Note,
"title" | "body" | "tags" | "project_id" | "milestone_id" | "note_type"
| "verify_with" | "expires_when"
>>
data: Partial<Pick<Note, "title" | "body" | "tags" | "project_id" | "milestone_id" | "note_type">>
): Promise<Note> {
try {
const note = await apiPut<Note>(`/api/notes/${id}`, data);
-9
View File
@@ -34,15 +34,6 @@ export interface Note {
is_task: boolean;
note_type: NoteType;
task_kind?: TaskKind;
// The note's own check (milestone 317). Empty on almost every note — that
// is the normal case: a note with no `verify_with` is a DECISION, and there
// is nothing to go and check. Only a note asserting a fact about something
// outside the operator's control carries one. `verified_at` null while
// `verify_with` is set means NOBODY HAS EVER CONFIRMED IT, which is the
// state the sweep ranks first.
verify_with?: string;
expires_when?: string;
verified_at?: string | null;
systems?: System[];
arose_from_id?: number | null;
created_at: string;
-23
View File
@@ -1,23 +0,0 @@
/**
* How often a record was put in front of an agent, and how often one then
* opened it in full.
*
* One shape for every record kind the retrieval surfaces can choose. Snippets
* and notes are counted in `note_usage_events`; rules in `rule_usage_events`,
* which is a separate table because a note id and a rule id are different
* namespaces resolved through different maps at restore (milestone 333). The
* TABLES are separate for that reason; the READOUT is the same question, so
* the client type is one.
*
* A high `surfaced_count` with `pull_count: 0` is dead weight — it occupies a
* slot in every future menu while never being used. What to DO about that
* differs by kind, which is why the advice is a prop on the badge rather than
* a property of this type: a snippet nobody opens should probably be deleted,
* while a rule nobody opens may just be mis-triggered.
*/
export interface RecordUsage {
surfaced_count: number;
pull_count: number;
last_surfaced_at: string | null;
last_pulled_at: string | null;
}
-50
View File
@@ -1,50 +0,0 @@
/**
* Line diff — one copy, for every surface that shows what changed.
*
* WHY THIS FILE EXISTS. The same LCS walk was written out three times:
* privately in `useAssist.ts`, and again inside `HistoryPanel.vue` and
* `VersionHistorySection.vue`. The three were character-identical apart from
* quote style — nobody had diverged them on purpose, they were simply copied
* because `computeDiff` was never exported. Milestone 323 needed a fourth
* consumer (a rule's edit history), and a fourth copy is the cost #3207
* records: a fix or an improvement now has to be found in N places by someone
* who does not know N.
*/
export interface DiffLine {
type: "equal" | "delete" | "insert";
text: string;
}
/**
* Diff `a` against `b`, line by line.
*
* `delete` lines come from `a`, `insert` lines from `b` — so the caller
* decides which side reads as "before" by which argument it passes. Every
* caller here passes the CURRENT text as `a` and the older text as `b`, so a
* deletion is what the old version had and an insertion is what replaced it.
*
* O(m·n) in time and memory: fine for a note or a rule, and deliberately not
* generalised further, since nothing here diffs a file of thousands of lines.
*/
export function computeDiff(a: string, b: string): DiffLine[] {
const aLines = a.split("\n");
const bLines = b.split("\n");
const m = aLines.length, n = bLines.length;
const dp: number[][] = Array.from({ length: m + 1 }, () => new Array(n + 1).fill(0));
for (let i = m - 1; i >= 0; i--)
for (let j = n - 1; j >= 0; j--)
dp[i][j] = aLines[i] === bLines[j]
? dp[i + 1][j + 1] + 1
: Math.max(dp[i + 1][j], dp[i][j + 1]);
const result: DiffLine[] = [];
let i = 0, j = 0;
while (i < m && j < n) {
if (aLines[i] === bLines[j]) { result.push({ type: "equal", text: aLines[i++] }); j++; }
else if (dp[i + 1][j] >= dp[i][j + 1]) result.push({ type: "delete", text: aLines[i++] });
else result.push({ type: "insert", text: bLines[j++] });
}
while (i < m) result.push({ type: "delete", text: aLines[i++] });
while (j < n) result.push({ type: "insert", text: bLines[j++] });
return result;
}
+2 -2
View File
@@ -774,7 +774,7 @@ onUnmounted(() => {
.tag-chip {
font-size: 0.7rem;
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
border-radius: 999px;
padding: 0.1rem 0.4rem;
}
@@ -928,7 +928,7 @@ onUnmounted(() => {
}
.peek-linked-item:hover {
background: color-mix(in srgb, var(--fs-accent) 8%, var(--fs-surface-raised));
color: var(--fs-accent-fg);
color: var(--fs-accent);
}
.peek-linked-type {
+12 -94
View File
@@ -4,7 +4,6 @@ import { useRouter } from "vue-router";
import { apiGet } from "@/api/client";
import type { TaskKind, TaskStatus, TaskPriority } from "@/types/note";
import KindBadge from "@/components/KindBadge.vue";
import NoteSweepPane from "@/components/NoteSweepPane.vue";
import StatusBadge from "@/components/StatusBadge.vue";
import PriorityBadge from "@/components/PriorityBadge.vue";
import GraphView from "@/views/GraphView.vue";
@@ -14,7 +13,6 @@ import {
Workflow,
Search,
Share2,
ShieldCheck,
ChevronLeft,
ChevronRight,
X,
@@ -26,7 +24,7 @@ const router = useRouter();
interface KnowledgeItem {
id: number;
note_type: "note" | "task" | "process" | "snippet";
note_type: "note" | "task" | "process";
title: string;
snippet: string;
tags: string[];
@@ -44,41 +42,9 @@ interface KnowledgeItem {
task_kind?: TaskKind;
}
// ─── The facet vocabulary ─────────────────────────────────────────────────────
// Mirrors services/knowledge._FACETS, which is where it is defined for real.
// A facet spans BOTH typing axes — a record TYPE (note / process / snippet) or
// a task KIND (`task` for any, else issue / spike) — because that is what this
// feed actually holds.
//
// `plan` is still a valid facet at the API, for the 90 legacy plan-tasks, but
// it has no chip: retired in 0066, it kept a chip of its own for longer than
// `issue` — 17% of every task here — went without one (#3128). Those rows are
// still reachable under Tasks, wearing a Plan badge.
type Facet = "" | "note" | "task" | "issue" | "spike" | "snippet" | "process";
// The facets that select TASKS. Kinds are subsets of `task`, so any of them
// means the duplicate report should be comparing tasks.
const TASK_FACETS = new Set<Facet>(["task", "issue", "spike"]);
const FACET_CHIPS: [Exclude<Facet, "">, string][] = [
["note", "Notes"],
["task", "Tasks"],
["issue", "Issues"],
["spike", "Spikes"],
["snippet", "Snippets"],
["process", "Processes"],
];
// ─── View mode ────────────────────────────────────────────────────────────────
// The sweep is cross-cutting — a note that has gone false does not care which
// facet it sits under — so it REPLACES the browse list rather than filtering
// it. Filtering would mean the answer depended on which chip was active, which
// is the under-reporting the sweep exists to prevent (milestone 317 step 4).
const sweepActive = ref(false);
// ─── Filter state ─────────────────────────────────────────────────────────────
const activeType = ref<Facet>("");
const activeType = ref<"" | "note" | "task" | "plan" | "process">("");
const activeTag = ref("");
const sortMode = ref<"modified" | "created" | "alpha" | "type">("modified");
const searchQuery = ref("");
@@ -104,10 +70,9 @@ const dupGroups = ref<DupGroup[]>([]);
const dupSuggestion = ref("");
const dupLoading = ref(false);
const dupChecked = ref(false);
// The report follows the type filter: viewing tasks — under ANY task facet,
// including a single kind — checks tasks. Everything else checks notes, the
// kind with the most to find.
const dupKind = computed(() => (TASK_FACETS.has(activeType.value) ? "task" : "note"));
// The report follows the type filter: viewing tasks checks tasks. Anything
// else (all / plan / process) checks notes — the kind with the most to find.
const dupKind = computed(() => (activeType.value === "task" ? "task" : "note"));
async function loadDuplicates() {
dupLoading.value = true;
@@ -131,11 +96,8 @@ watch(dupKind, () => { dupChecked.value = false; dupGroups.value = []; });
// ─── Type counts ──────────────────────────────────────────────────────────────
// One number per facet, plus the grand total. Partial because the server sends
// a key only for a facet it has rows for. Kinds are subsets of `task` and are
// deliberately absent from `total` — including them would count an issue twice.
type KnowledgeCounts = Partial<Record<Exclude<Facet, "">, number>> & { total: number };
const typeCounts = ref<KnowledgeCounts>({ total: 0 });
interface KnowledgeCounts { note: number; task: number; plan: number; process: number; total: number }
const typeCounts = ref<KnowledgeCounts>({ note: 0, task: 0, plan: 0, process: 0, total: 0 });
async function fetchCounts() {
try {
@@ -272,10 +234,6 @@ function onSearchInput() {
}
watch([activeType, sortMode], () => resetAndReobserve());
// Closing the sweep remounts the feed, and with it the scroll sentinel — a
// fresh element the old observer is not watching. Without this the list loads
// its first page and then never loads another.
watch(sweepActive, (open) => { if (!open) resetAndReobserve(); });
watch(activeTag, () => { fetchCounts(); resetAndReobserve(); });
// ─── Today bar ────────────────────────────────────────────────────────────────
@@ -316,18 +274,9 @@ function isOverdue(item: KnowledgeItem): boolean {
return new Date(item.due_date) < new Date(new Date().toDateString());
}
// Each record kind opens in ITS OWN editor. A snippet used to fall through to
// /notes/:id, whose save is a plain PATCH of the body — which left the snippet's
// derived `data` mirror describing the previous version (#3128). The service now
// recomposes the mirror either way, so this is no longer the guard; it is simply
// that the note editor cannot edit a snippet's signature, language or locations,
// and offering it as the way in was always wrong. Processes stay here on
// purpose: they have no editor of their own and the note editor knows the type.
function openItem(item: KnowledgeItem) {
if (item.note_type === 'task') {
router.push(`/tasks/${item.id}`);
} else if (item.note_type === 'snippet') {
router.push(`/snippets/${item.id}`);
} else {
router.push(`/notes/${item.id}`);
}
@@ -431,14 +380,14 @@ onUnmounted(() => {
<span v-if="typeCounts.total > 1" class="filter-count">{{ typeCounts.total }}</span>
</button>
<button
v-for="[val, label] in FACET_CHIPS"
v-for="[val, label, key] in ([['note','Notes','note'],['task','Tasks','task'],['plan','Plans','plan'],['process','Processes','process']] as [string,string,string][])"
:key="val"
class="filter-btn"
:class="{ active: activeType === val }"
@click="activeType = val"
@click="activeType = (val as '' | 'note' | 'task' | 'plan' | 'process')"
>
<span class="filter-btn-label">{{ label }}</span>
<span v-if="(typeCounts[val] ?? 0) > 1" class="filter-count">{{ typeCounts[val] }}</span>
<span v-if="typeCounts[key as keyof KnowledgeCounts] > 1" class="filter-count">{{ typeCounts[key as keyof KnowledgeCounts] }}</span>
</button>
</div>
@@ -484,15 +433,6 @@ onUnmounted(() => {
<Share2 :size="16" />
Graph
</button>
<button
class="btn-ghost btn-compact"
:class="{ active: sweepActive }"
title="Notes that assert a fact about something outside your control, least-recently-confirmed first. Most notes are decisions and never appear."
@click="sweepActive = !sweepActive"
>
<ShieldCheck :size="16" />
Due
</button>
<button
class="btn-ghost btn-compact"
:disabled="dupLoading"
@@ -503,12 +443,6 @@ onUnmounted(() => {
</button>
</div>
<!-- The sweep replaces the feed. It is not a facet: a facet answers
"show me this kind", and this answers "show me what nobody has
confirmed" a question the type chips cannot narrow without
under-reporting it. -->
<NoteSweepPane v-if="sweepActive" @open-note="(id) => router.push(`/notes/${id}`)" />
<!-- Near-duplicate report. A proposal surface only: unlike snippets
(which merge losslessly), notes are never merged the right fix is
supersession, extraction into a reference note, or leaving parallel
@@ -546,12 +480,6 @@ onUnmounted(() => {
</template>
</div>
<!-- The whole feed stands down while the sweep is open: two answers
to two different questions on one screen is neither. Wrapped
rather than given an extra v-if branch, because the scroll
sentinel lives inside the grid and the observer must not be left
holding a ref to something that never renders. -->
<template v-if="!sweepActive">
<!-- Loading / empty -->
<div v-if="loading && items.length === 0" class="knowledge-empty">Loading…</div>
<div v-else-if="!loading && items.length === 0" class="knowledge-empty">
@@ -573,7 +501,6 @@ onUnmounted(() => {
<span v-if="item.note_type === 'note'">Note</span>
<span v-else-if="item.note_type === 'task'">{{ item.task_kind === 'plan' ? 'Plan' : 'Task' }}</span>
<span v-else-if="item.note_type === 'process'">Process</span>
<span v-else-if="item.note_type === 'snippet'">Snippet</span>
</span>
<!-- Kind sits BESIDE the type badge, not inside it: the type badge
speaks the vocabulary of this view's type filter (note / task /
@@ -625,7 +552,6 @@ onUnmounted(() => {
<span v-if="contentFetching" class="sentinel-loading">Loading</span>
</div>
</div>
</template>
</div>
<!-- Graph panel -->
@@ -827,7 +753,7 @@ onUnmounted(() => {
}
.filter-btn.active .filter-count {
background: color-mix(in srgb, var(--fs-accent) 20%, transparent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
}
.filter-tag { font-size: 0.78rem; }
@@ -955,14 +881,6 @@ onUnmounted(() => {
.badge--note { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: #7A6DA8; }
.badge--task { background: rgba(212,160,23,0.15); color: #fbbf24; }
.badge--plan { background: rgba(99,102,241,0.18); color: #818cf8; }
/* Snippet and process are NEUTRAL on purpose. Both were unstyled — and the
snippet had no label either, so all 90 of them rendered an empty chip in
this feed (#3128). Giving them hues would put a third and fourth colour
beside KindBadge's warm/cool pair on the same card; a record type that
isn't an alarm reads better as plain. Standard body pair, so the contrast
is the one the palette already guarantees. */
.badge--snippet,
.badge--process { background: var(--fs-surface-raised); color: var(--fs-text-secondary); }
.k-card-body { flex: 1; padding-right: 40px; }
.k-card-title {
@@ -1008,7 +926,7 @@ onUnmounted(() => {
border-radius: 4px;
white-space: nowrap;
background: color-mix(in srgb, var(--fs-text-secondary) 15%, transparent);
color: var(--fs-text-secondary-fg);
color: var(--fs-text-secondary);
}
/* ── Task card ──────────────────────────────────────────── */
+44 -110
View File
@@ -34,14 +34,6 @@ const tags = ref<string[]>([]);
const projectId = ref<number | null>(null);
const milestoneId = ref<number | null>(null);
const noteType = ref<NoteType>("note");
// The note's own check (milestone 317). Offered only for a plain note: a
// task's decay is its status, and a snippet has verify_snippet — the service
// refuses both, so the form must not ask for what the save would reject.
const verifyWith = ref("");
const expiresWhen = ref("");
const verifiedAt = ref<string | null>(null);
const canCarryCheck = computed(() => noteType.value === "note");
const dirty = ref(false);
const saving = ref(false);
const showPreview = ref(false);
@@ -206,41 +198,6 @@ let savedTags: string[] = [];
let savedProjectId: number | null = null;
let savedMilestoneId: number | null = null;
let savedNoteType: NoteType = "note";
let savedVerifyWith = "";
let savedExpiresWhen = "";
/** The write, in one place. Three call sites (save, create, auto-save) each
* spelled this out, so every new field had to be added three times — which is
* how one of them ends up not carrying it. */
function payload() {
return {
title: title.value,
body: body.value,
tags: tags.value,
project_id: projectId.value,
milestone_id: milestoneId.value,
note_type: noteType.value,
// "" clears the check: the REST door reads an empty string as NULL
// (NULLABLE_NOTE_TEXT), which is how a cleared form input says "remove
// this" without needing the MCP door's explicit `clear` list.
verify_with: canCarryCheck.value ? verifyWith.value : "",
expires_when: canCarryCheck.value ? expiresWhen.value : "",
};
}
/** What the form last agreed with the server about — the other half of the
* same list, and for the same reason. */
function snapshot() {
savedTitle = title.value;
savedBody = body.value;
savedTags = [...tags.value];
savedProjectId = projectId.value;
savedMilestoneId = milestoneId.value;
savedNoteType = noteType.value;
savedVerifyWith = verifyWith.value;
savedExpiresWhen = expiresWhen.value;
dirty.value = false;
}
function markDirty() {
dirty.value =
@@ -249,9 +206,7 @@ function markDirty() {
JSON.stringify(tags.value) !== JSON.stringify(savedTags) ||
projectId.value !== savedProjectId ||
milestoneId.value !== savedMilestoneId ||
noteType.value !== savedNoteType ||
verifyWith.value !== savedVerifyWith ||
expiresWhen.value !== savedExpiresWhen;
noteType.value !== savedNoteType;
}
function onBodyUpdate(newVal: string) {
@@ -269,10 +224,12 @@ onMounted(async () => {
projectId.value = store.currentNote.project_id ?? null;
milestoneId.value = store.currentNote.milestone_id ?? null;
noteType.value = (store.currentNote.note_type as NoteType) || "note";
verifyWith.value = store.currentNote.verify_with || "";
expiresWhen.value = store.currentNote.expires_when || "";
verifiedAt.value = store.currentNote.verified_at ?? null;
snapshot();
savedTitle = title.value;
savedBody = body.value;
savedTags = [...tags.value];
savedProjectId = projectId.value;
savedMilestoneId = milestoneId.value;
savedNoteType = noteType.value;
}
} else {
// New note: read type from query param
@@ -303,11 +260,31 @@ async function save() {
const finalBody = body.value;
try {
if (isEditing.value) {
await store.updateNote(noteId.value!, { ...payload(), body: finalBody });
snapshot();
await store.updateNote(noteId.value!, {
title: title.value,
body: finalBody,
tags: tags.value,
project_id: projectId.value,
milestone_id: milestoneId.value,
note_type: noteType.value,
});
savedTitle = title.value;
savedBody = body.value;
savedTags = [...tags.value];
savedProjectId = projectId.value;
savedMilestoneId = milestoneId.value;
savedNoteType = noteType.value;
dirty.value = false;
toast.show("Note saved");
} else {
const note = await store.createNote({ ...payload(), body: finalBody });
const note = await store.createNote({
title: title.value,
body: finalBody,
tags: tags.value,
project_id: projectId.value,
milestone_id: milestoneId.value,
note_type: noteType.value,
});
dirty.value = false;
toast.show("Note created");
router.push(`/notes/${note.id}`);
@@ -344,8 +321,18 @@ async function doAutoSave() {
saving.value = true;
const finalBody = body.value;
try {
await store.updateNote(noteId.value!, { ...payload(), body: finalBody });
snapshot();
await store.updateNote(noteId.value!, {
title: title.value, body: finalBody, tags: tags.value,
project_id: projectId.value, milestone_id: milestoneId.value,
note_type: noteType.value,
});
savedTitle = title.value;
savedBody = body.value;
savedTags = [...tags.value];
savedProjectId = projectId.value;
savedMilestoneId = milestoneId.value;
savedNoteType = noteType.value;
dirty.value = false;
toast.show("Auto-saved");
} catch {
// Silent
@@ -509,41 +496,6 @@ onUnmounted(() => assist.clearSelection());
</select>
</div>
<!-- The note's own check (milestone 317). Shown only for a plain
note: the service refuses a check on a task or a snippet, so
offering the fields there would be a form whose save fails. -->
<template v-if="canCarryCheck">
<div class="sb-field">
<div class="sb-label-row">
<span class="sb-label">Check</span>
<span v-if="verifyWith" class="check-age" :class="{ unchecked: !verifiedAt }">
{{ verifiedAt ? `checked ${verifiedAt.slice(0, 10)}` : "never checked" }}
</span>
</div>
<!-- Phrased as the question that decides, not as a field name.
"Verify with" would get filled in on every note; "could this
become false without anyone editing it?" gets filled in on
the few that can. -->
<textarea
v-model="verifyWith"
class="sb-textarea"
rows="2"
placeholder="How would someone check this is still true? Leave empty unless this note could become false without anyone editing it."
@input="markDirty"
></textarea>
</div>
<div v-if="verifyWith" class="sb-field">
<label class="sb-label">Ends when</label>
<textarea
v-model="expiresWhen"
class="sb-textarea"
rows="2"
placeholder="What state ends it? A state, not a date — “when the forge numbers runs per workflow”, not “in six months”."
@input="markDirty"
></textarea>
</div>
</template>
<!-- Link Suggestions -->
<div v-if="linkSuggestions.length > 0" class="sb-field link-suggest-field">
<div class="sb-label-row">
@@ -726,7 +678,7 @@ onUnmounted(() => assist.clearSelection());
flex-direction: column;
}
.sb-select, .sb-input, .sb-textarea {
.sb-select, .sb-input {
width: 100%;
padding: 5px 8px;
border-radius: var(--fs-radius-sm);
@@ -738,27 +690,9 @@ onUnmounted(() => assist.clearSelection());
outline: none;
transition: border-color 0.15s;
}
.sb-select:focus, .sb-input:focus, .sb-textarea:focus {
.sb-select:focus, .sb-input:focus {
border-color: var(--fs-accent);
}
.sb-textarea {
resize: vertical;
line-height: 1.4;
box-sizing: border-box;
}
/* No red/amber ramp, matching RuleSweepPane: a colour scale would restate the
sweep's ordering and force an invented "stale after N days" threshold.
"Never" is marked because it is categorically different from a date, not a
worse one it means nobody has ever confirmed the claim. */
.check-age {
font-size: 0.7rem;
color: var(--fs-text-secondary);
font-variant-numeric: tabular-nums;
}
.check-age.unchecked {
font-style: italic;
color: var(--fs-text-tertiary);
}
/* Link Suggestions */
.link-suggest-field { gap: 0.4rem; }
+1 -1
View File
@@ -416,7 +416,7 @@ async function convertToTask() {
}
.badge-note {
background: color-mix(in srgb, var(--fs-accent) 12%, transparent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
border: 1px solid color-mix(in srgb, var(--fs-accent) 25%, transparent);
}
.badge-task {
+4 -4
View File
@@ -1287,8 +1287,8 @@ async function confirmDelete() {
.stat-todo { background: color-mix(in srgb, var(--fs-text-tertiary) 8%, transparent); color: var(--fs-text-secondary); border-color: var(--fs-border-color); }
.stat-inprogress { background: color-mix(in srgb, #3b82f6 10%, transparent); color: #3b82f6; border-color: color-mix(in srgb, #3b82f6 28%, transparent); }
.stat-done { background: color-mix(in srgb, var(--fs-success) 10%, transparent); color: var(--fs-success-fg); border-color: color-mix(in srgb, var(--fs-success) 28%, transparent); }
.stat-notes { background: color-mix(in srgb, var(--fs-accent) 8%, transparent); color: var(--fs-accent-fg); border-color: color-mix(in srgb, var(--fs-accent) 22%, transparent); }
.stat-done { background: color-mix(in srgb, var(--fs-success) 10%, transparent); color: var(--fs-success); border-color: color-mix(in srgb, var(--fs-success) 28%, transparent); }
.stat-notes { background: color-mix(in srgb, var(--fs-accent) 8%, transparent); color: var(--fs-accent); border-color: color-mix(in srgb, var(--fs-accent) 22%, transparent); }
/* ── Pattern-library coverage card ───────────────────────────── */
.coverage-card {
@@ -1471,7 +1471,7 @@ async function confirmDelete() {
.tab-btn.active .tab-count {
background: color-mix(in srgb, var(--fs-accent) 12%, transparent);
border-color: color-mix(in srgb, var(--fs-accent) 30%, transparent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
}
/* ── Tasks view ──────────────────────────────────────────────── */
@@ -1707,7 +1707,7 @@ async function confirmDelete() {
border-radius: 3px;
margin-left: auto;
}
.col-add-btn:hover { color: var(--fs-accent-fg); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
.col-add-btn:hover { color: var(--fs-accent); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
.kanban-cards { display: flex; flex-direction: column; gap: 0.3rem; }
+15 -171
View File
@@ -9,7 +9,6 @@ import type { User } from "@/types/auth";
import PaginationBar from "@/components/PaginationBar.vue";
import TagInput from "@/components/TagInput.vue";
import { fmtDate, fmtLogStamp } from "@/utils/dateFormat";
import { fetchVersion, type VersionPayload } from "@/api/version";
const store = useSettingsStore();
const authStore = useAuthStore();
@@ -86,7 +85,6 @@ const kbWritePathEnabled = ref(true);
// code embeddings sit on a much higher similarity floor than prose, so 0.55 let
// unrelated code through (#2223). Shares top-k, not the threshold.
const kbWritePathThreshold = ref("0.68");
const kbRuleHintThreshold = ref("0.72");
// Near-duplicate report floors, one per record kind (services/dedup.py).
// Snippets are single-chunk, so their floor sits below the 0.90 write-time
// gate and catches what it lets through. Notes/tasks are scored at chunk
@@ -149,17 +147,12 @@ async function saveKbInject() {
// Same `|| default` reasoning: falling back to 0 would surface every
// snippet in the corpus on every edit, which is the failure this knob fixes.
const wpT = Math.min(1, Math.max(0, Number(kbWritePathThreshold.value) || 0.68));
// Same `|| default` reasoning again, and it bites harder here: a rule hint
// fires on every write, so a fallback of 0 would attach a standing rule to
// every edit in the session.
const rhT = Math.min(1, Math.max(0, Number(kbRuleHintThreshold.value) || 0.72));
kbInjectThreshold.value = String(t);
kbInjectTopK.value = String(k);
kbDupThresholdSnippet.value = String(dupSnip);
kbDupThresholdNote.value = String(dupNote);
kbDupThresholdTask.value = String(dupTask);
kbWritePathThreshold.value = String(wpT);
kbRuleHintThreshold.value = String(rhT);
savingKbInject.value = true;
kbInjectSaved.value = false;
try {
@@ -172,10 +165,6 @@ async function saveKbInject() {
// measurements that split them.
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
kb_writepath_threshold: String(wpT),
// A THIRD corpus with a third bar — see RULEHINT_DEFAULT_THRESHOLD
// in services/plugin_context.py for why rules cannot share the
// code threshold any more than code could share the prose one.
kb_rulehint_threshold: String(rhT),
kb_duplicate_threshold_snippet: String(dupSnip),
kb_duplicate_threshold_note: String(dupNote),
kb_duplicate_threshold_task: String(dupTask),
@@ -198,47 +187,7 @@ const changingPassword = ref(false);
const invalidatingSessions = ref(false);
const exporting = ref(false);
const restoring = ref(false);
// Backup, export and restore walk the whole store, so they are slow BY DESIGN
// and the client's ordinary 30s default would cut them off mid-work. They are
// still bounded: rule 156 asks for a deadline, not a short one, and "no ceiling
// at all" is what leaves a restore that died server-side spinning forever.
const BULK_TRANSFER_TIMEOUT_MS = 10 * 60 * 1000;
function bulkDeadline(): AbortSignal {
return AbortSignal.timeout(BULK_TRANSFER_TIMEOUT_MS);
}
// ── What's running (#3127 checklist 12) ─────────────────────────────────
// Three states kept apart, because collapsing any two of them is the defect
// this readout exists to remove: `null` + no error = not asked yet (the Config
// tab has not been opened); a payload = answered, with each ABSENT field shown
// as "unknown"; `versionError` = the fetch itself failed, which is its own
// thing and must never render as a blank or as a plausible-looking value.
const versionInfo = ref<VersionPayload | null>(null);
const versionLoading = ref(false);
const versionError = ref("");
const commitCopied = ref(false);
async function loadVersionPanel() {
if (versionLoading.value) return;
versionLoading.value = true;
versionError.value = "";
try {
versionInfo.value = await fetchVersion();
} catch (e) {
versionInfo.value = null;
versionError.value = apiErrorMessage(e, "Could not reach the instance to ask what it is running.");
} finally {
versionLoading.value = false;
}
}
async function copyCommit() {
if (!versionInfo.value?.commit) return;
await copyToClipboard(versionInfo.value.commit);
commitCopied.value = true;
setTimeout(() => { commitCopied.value = false; }, 2000);
}
const appVersion = ref('dev');
const restoreFileInput = ref<HTMLInputElement | null>(null);
// Migrate stored "admin" → "config"; unknown tabs fall back to "general"
@@ -252,7 +201,6 @@ function _loadTabContent(tab: string) {
else if (tab === "logs") loadLogsPanel();
else if (tab === "groups") loadGroupsPanel();
else if (tab === "areas") canonStore.fetchCatalog(true);
else if (tab === "config" && !versionInfo.value) loadVersionPanel();
}
if (tab === "apikeys") { fetchApiKeys(); }
}
@@ -606,6 +554,10 @@ function toggleProfileWorkDay(day: string) {
function emptyTagsFetch(): Promise<string[]> { return Promise.resolve([]) }
onMounted(async () => {
try {
const v = await apiGet<{ version: string }>('/api/version')
appVersion.value = v.version
} catch { /* non-critical */ }
await store.fetchSettings();
newEmail.value = authStore.user?.email ?? "";
@@ -621,9 +573,6 @@ onMounted(async () => {
kbInjectTopK.value = allSettings.kb_autoinject_top_k;
}
kbWritePathEnabled.value = allSettings.kb_writepath_enabled !== "false";
if (allSettings.kb_rulehint_threshold !== undefined) {
kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
}
if (allSettings.kb_writepath_threshold !== undefined) {
kbWritePathThreshold.value = allSettings.kb_writepath_threshold;
}
@@ -778,7 +727,7 @@ async function exportData(scope: "user" | "full") {
exporting.value = true;
try {
const url = scope === "full" ? "/api/admin/backup" : "/api/admin/backup?scope=user";
const res = await fetch(url, { signal: bulkDeadline() });
const res = await fetch(url);
if (!res.ok) {
const body = await res.json().catch(() => ({ error: `Error ${res.status}` }));
throw new Error((body as Record<string, string>).error || `Error ${res.status}`);
@@ -803,7 +752,7 @@ const exportingNotes = ref(false);
async function exportNotes(format: "markdown" | "json") {
exportingNotes.value = true;
try {
const res = await fetch(`/api/export?format=${format}`, { signal: bulkDeadline() });
const res = await fetch(`/api/export?format=${format}`);
if (!res.ok) throw new Error(`Error ${res.status}`);
const blob = await res.blob();
const ext = format === "json" ? "json" : "zip";
@@ -1032,7 +981,6 @@ async function handleRestoreFile(event: Event) {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(data),
signal: bulkDeadline(),
});
if (!res.ok) {
const body = await res.json().catch(() => ({ error: `Error ${res.status}` }));
@@ -1469,29 +1417,6 @@ async function deleteUser(userId: number) {
location, not by resemblance.
</p>
</div>
<div class="field">
<label for="kb-rulehint-threshold">Standing-rule confidence threshold (01)</label>
<input
id="kb-rulehint-threshold"
v-model="kbRuleHintThreshold"
type="number"
min="0"
max="1"
step="0.01"
class="fs-input input"
style="max-width: 8rem"
/>
<p class="field-hint">
The same hint can mention a standing rule whose trigger resembles what's
being written — only rules marked <em>conditional</em>, since always-on
ones are already loaded. Stricter again than the threshold above, because
there are far fewer rules than snippets: with a small set, something
always ranks first, so the bar has to carry more of the judgement.
Raise it if rules keep arriving unread; lower it if a rule you needed
never showed up. Settings → check the pull-through in
<code>retrieval_telemetry</code> to see which is happening.
</p>
</div>
<!-- A design system belongs to a PROJECT, and the picker for it lives on
the project. There was a setting here that designated the system
this install's own interface was built from; it only ever described
@@ -2184,48 +2109,6 @@ async function deleteUser(userId: number) {
<!-- ── Admin ── -->
<div v-if="authStore.isAdmin" v-show="activeTab === 'config'" class="settings-grid">
<section class="settings-section full-width">
<h2>What's running</h2>
<p class="section-desc">
The build serving this page. Paste the commit into a <code>:sha</code> image
lookup to check the registry and the app agree about what was published.
</p>
<div v-if="versionLoading" class="state-msg">Reading the ledger&hellip;</div>
<div v-else-if="versionError" class="error-msg">
{{ versionError }}
<button class="btn-ghost btn-compact version-retry" @click="loadVersionPanel">Try again</button>
</div>
<dl v-else-if="versionInfo" class="version-grid">
<dt>Version</dt>
<dd class="version-value">{{ versionInfo.version }}</dd>
<dt>Channel</dt>
<dd :class="versionInfo.channel === undefined ? 'version-unknown' : 'version-value'">
{{ versionInfo.channel ?? "unknown" }}
</dd>
<dt>Commit</dt>
<dd v-if="versionInfo.commit" class="version-value version-commit">
<span class="version-sha">{{ versionInfo.commit }}</span>
<button class="btn-ghost btn-compact" @click="copyCommit">
{{ commitCopied ? "Copied" : "Copy" }}
</button>
</dd>
<dd v-else class="version-unknown">unknown</dd>
<dt>Build</dt>
<!-- The ordering key, kept because its ABSENCE is the diagnostic one:
no key means this build is not part of any update order, which is
what a local or hand-built image looks like. `??` not `||` 0 is
a legitimate key. -->
<dd :class="versionInfo.build === undefined ? 'version-unknown' : 'version-value'">
{{ versionInfo.build ?? "unknown" }}
</dd>
</dl>
<div v-else class="empty-msg">Nothing asked yet.</div>
</section>
<section class="settings-section full-width">
<h2>Application URL</h2>
<p class="section-desc">
@@ -2849,7 +2732,7 @@ async function deleteUser(userId: number) {
background: var(--fs-surface-raised);
}
.sidebar-item.active {
color: var(--fs-accent-fg);
color: var(--fs-accent);
background: color-mix(in srgb, var(--fs-accent) 8%, transparent);
border-left-color: var(--fs-accent);
font-weight: 500;
@@ -2885,45 +2768,6 @@ async function deleteUser(userId: number) {
letter-spacing: 0.07em;
color: var(--fs-text-tertiary);
}
/* What's running — a definition list of instance facts. Spacing/geometry only;
colour and type come from the tokens. */
.version-grid {
display: grid;
grid-template-columns: max-content 1fr;
gap: 0.4rem 1rem;
margin: 0;
align-items: baseline;
}
.version-grid dt {
font-size: 0.8rem;
color: var(--fs-text-secondary);
}
.version-grid dd {
margin: 0;
font-size: 0.875rem;
font-family: var(--fs-font-mono);
color: var(--fs-text-primary);
}
/* An absent field reads as absent — never as a blank, and never styled to look
like a value it does not have (#3127 checklist 12). */
.version-grid dd.version-unknown {
font-family: inherit;
font-style: italic;
color: var(--fs-text-tertiary);
}
.version-commit {
display: flex;
align-items: center;
gap: 0.5rem;
flex-wrap: wrap;
}
.version-sha {
overflow-wrap: anywhere;
}
.version-retry {
margin-left: 0.5rem;
}
.section-desc {
margin: 0 0 1rem;
font-size: 0.875rem;
@@ -3255,7 +3099,7 @@ async function deleteUser(userId: number) {
border-radius: var(--fs-radius-sm);
}
.role-admin {
color: var(--fs-accent-fg);
color: var(--fs-accent);
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
}
.role-user {
@@ -3335,9 +3179,9 @@ async function deleteUser(userId: number) {
text-transform: uppercase; letter-spacing: 0.05em;
padding: 0.1rem 0.35rem; border-radius: var(--fs-radius-sm);
}
.cat-audit { color: var(--fs-accent-fg); background: color-mix(in srgb, var(--fs-accent) 15%, transparent); }
.cat-usage { color: var(--fs-success-fg); background: color-mix(in srgb, var(--fs-success) 15%, transparent); }
.cat-error { color: var(--fs-error-fg); background: color-mix(in srgb, var(--fs-error) 15%, transparent); }
.cat-audit { color: var(--fs-accent); background: color-mix(in srgb, var(--fs-accent) 15%, transparent); }
.cat-usage { color: var(--fs-success); background: color-mix(in srgb, var(--fs-success) 15%, transparent); }
.cat-error { color: var(--fs-error); background: color-mix(in srgb, var(--fs-error) 15%, transparent); }
.method-tag {
display: inline-block;
font-size: 0.65rem; font-weight: 500; font-family: monospace;
@@ -3502,8 +3346,8 @@ async function deleteUser(userId: number) {
padding: 0.15rem 0.4rem;
border-radius: 4px;
}
.role-owner { background: color-mix(in srgb, var(--fs-warning) 15%, transparent); color: var(--fs-warning-fg); }
.role-member { background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent); color: var(--fs-text-tertiary-fg); }
.role-owner { background: color-mix(in srgb, var(--fs-warning) 15%, transparent); color: var(--fs-warning); }
.role-member { background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent); color: var(--fs-text-tertiary); }
.members-empty {
color: var(--fs-text-tertiary);
@@ -3684,7 +3528,7 @@ async function deleteUser(userId: number) {
.day-btn.active {
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
border-color: var(--fs-accent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
font-weight: 500;
}
+3 -3
View File
@@ -242,9 +242,9 @@ onMounted(async () => {
border-radius: 4px;
white-space: nowrap;
}
.perm-viewer { background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent); color: var(--fs-text-tertiary-fg); }
.perm-editor { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent-fg); }
.perm-admin { background: color-mix(in srgb, var(--fs-warning) 15%, transparent); color: var(--fs-warning-fg); }
.perm-viewer { background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent); color: var(--fs-text-tertiary); }
.perm-editor { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent); }
.perm-admin { background: color-mix(in srgb, var(--fs-warning) 15%, transparent); color: var(--fs-warning); }
.empty-msg {
margin: 0;
+1 -1
View File
@@ -288,7 +288,7 @@ async function confirmDelete() {
font-family: var(--fs-font-mono);
font-size: 0.82rem;
background: color-mix(in srgb, var(--fs-accent) 12%, transparent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
padding: 0.08rem 0.35rem;
border-radius: var(--fs-radius-sm);
word-break: break-all;
+61 -12
View File
@@ -9,7 +9,6 @@ import {
type SnippetListItem,
} from "@/api/snippets";
import { useToastStore } from "@/stores/toast";
import UsageBadge from "@/components/UsageBadge.vue";
const router = useRouter();
const toast = useToastStore();
@@ -199,6 +198,23 @@ function languageOf(tags: string[]): string {
return tags.find((t) => t && t !== "snippet") ?? "";
}
/** A snippet that has been offered repeatedly and never opened. The threshold
* is 3 rather than 1 because one or two surfacings is noise — the record may
* simply not have come up in a relevant context yet. */
function isDeadWeight(s: SnippetListItem): boolean {
const u = s.usage;
return !!u && u.pull_count === 0 && u.surfaced_count >= 3;
}
/** Short badge text, or "" to render nothing. A record nobody has surfaced yet
* gets no badge at all: "0 / 0" would read as a verdict when it's an absence
* of evidence. */
function usageBadge(s: SnippetListItem): string {
const u = s.usage;
if (!u || u.surfaced_count === 0) return "";
return `${u.pull_count}/${u.surfaced_count} used`;
}
/** Short label for the drift verdict, or "" when there's nothing to say.
* An expired verdict is reported as "unchecked" whatever it used to say —
* it was about code that is no longer in the record. */
@@ -238,13 +254,22 @@ function driftTitle(s: SnippetListItem): string {
return v.detail ? `${when}: ${what}. ${v.detail}` : `${when}: ${what}.`;
}
/** The dead-weight nudge for a SNIPPET, passed to the shared badge. Kept here
* rather than inside the component because the remedy is kind-specific — a
* rule in the same position gets different advice (milestone 333 step 5). */
const SNIPPET_DEAD_WEIGHT =
"Offered repeatedly without ever being opened — consider rewriting its " +
"“when to reach for it” so it says when, or deleting it. It takes a slot " +
"in every future auto-inject menu.";
function usageTitle(s: SnippetListItem): string {
const u = s.usage;
if (!u) return "";
const last = u.last_pulled_at
? `Last opened ${new Date(u.last_pulled_at).toLocaleDateString()}.`
: "Never opened.";
const verdict = isDeadWeight(s)
? " Offered repeatedly without ever being opened — consider rewriting its" +
" “when to reach for it” so it says when, or deleting it. It takes a slot" +
" in every future auto-inject menu."
: "";
return (
`Surfaced to an agent ${u.surfaced_count}×, opened in full ` +
`${u.pull_count}×. ${last}${verdict}`
);
}
</script>
<template>
@@ -431,7 +456,14 @@ const SNIPPET_DEAD_WEIGHT =
<span v-if="driftBadge(s)" class="drift-tag" :title="driftTitle(s)">
{{ driftBadge(s) }}
</span>
<UsageBadge :usage="s.usage" :dead-weight-advice="SNIPPET_DEAD_WEIGHT" />
<span
v-if="usageBadge(s)"
class="usage-tag"
:class="{ 'usage-dead': isDeadWeight(s) }"
:title="usageTitle(s)"
>
{{ usageBadge(s) }}
</span>
<span v-if="s.shared" class="shared-tag" :title="`Shared by ${s.owner ?? 'another user'} — a suggestion, not your own record`">
by {{ s.owner ?? "another user" }}
</span>
@@ -678,7 +710,7 @@ const SNIPPET_DEAD_WEIGHT =
flex-shrink: 0;
white-space: nowrap;
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
color: var(--fs-accent-fg);
color: var(--fs-accent);
}
.snippet-when {
@@ -707,7 +739,7 @@ const SNIPPET_DEAD_WEIGHT =
border-radius: 4px;
white-space: nowrap;
background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent);
color: var(--fs-text-tertiary-fg);
color: var(--fs-text-tertiary);
}
.dup-action {
@@ -722,7 +754,24 @@ const SNIPPET_DEAD_WEIGHT =
border-radius: 4px;
white-space: nowrap;
background: color-mix(in srgb, var(--fs-error) 15%, transparent);
color: var(--fs-error-fg);
color: var(--fs-error);
}
.usage-tag {
font-size: 0.7rem;
padding: 0.1rem 0.4rem;
border-radius: 4px;
white-space: nowrap;
font-variant-numeric: tabular-nums;
background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent);
color: var(--fs-text-tertiary);
}
/* Dead weight is a nudge, not an error — it warns in the warning colour rather
than the danger one, because the record isn't broken, just unearned. */
.usage-tag.usage-dead {
background: color-mix(in srgb, var(--fs-warning) 18%, transparent);
color: var(--fs-warning);
}
/* Header + select-mode */
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "scribe",
"description": "Scribe system-of-record for Claude Code: MCP tools over your notes/tasks/projects/rules, a session-start push channel that surfaces your always-on rules + active-project context, process-skills (writing-plans, systematic-debugging, verification, brainstorming, reusing-code), and your saved Scribe Processes auto-surfaced as skills (/scribe:sync). Replaces superpowers + file-memory with one app-backed plugin.",
"version": "2026.09.10.0221",
"version": "0.1.47",
"author": {
"name": "Bryan Van Deusen"
},
+2 -7
View File
@@ -78,13 +78,8 @@ On install you'll be asked for:
## Notes
- **Do not hand-edit `version` in `.claude-plugin/plugin.json`.** It is minted
from the clock — run `python3 scripts/mint_plugin_version.py` (or `make
mint-plugin`, where `make` is installed) after changing anything under
`plugin/`, and commit the result. The installer decides whether to refresh the cache it
executes from by comparing that string, so content that ships without a new
version reaches the repo and stops there (#2209). CI fails the lane if you
forget.
- Set a `version` bump in `.claude-plugin/plugin.json` per release so clients
pick up changes.
- The session-start, auto-inject and prior-art hooks need only a **read**-scoped
key; the MCP tools need **write** scope to create/update. Every hook is a GET
for that reason — a read key cannot POST.
-9
View File
@@ -33,15 +33,6 @@
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_prior_art.sh\""
}
]
},
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_tool_rules.sh\""
}
]
}
],
"PostToolUse": [
+1 -11
View File
@@ -175,16 +175,6 @@ while IFS= read -r rel_path; do
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
fi
# The rules marker the SessionStart hook stored, handed back so the server
# can say whether those rules moved since (milestone 323). Nothing stored
# means nothing sent, which the server reads as silence rather than as a
# mismatch — an install that never reached /api/plugin/context must not
# start claiming its rules changed.
etag_q=""
if [ -f "$state_dir/${safe_sid}.rules_etag" ]; then
held=$(jq -sRr '@uri' < "$state_dir/${safe_sid}.rules_etag" 2>/dev/null) || held=""
[ -n "$held" ] && etag_q="&rules_etag=${held}"
fi
if [ -n "$path_enc" ]; then
# 8s, not the pre-write hook's 5: this hook runs AFTER the tool, so it
# gates nothing the session is waiting on, and the first prior-art call
@@ -194,7 +184,7 @@ while IFS= read -r rel_path; do
reached=1
body=$(curl -fsS --max-time 8 \
-H "Authorization: Bearer ${token}" \
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${derive_exclude_q}${shapes_q}${etag_q}" 2>/dev/null) || { body=""; reached=0; }
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${derive_exclude_q}${shapes_q}" 2>/dev/null) || { body=""; reached=0; }
# A call that was owed and didn't come back is said, once per outage
# (#2932) — shared marker with the pre-write hook, so one outage is one
# line however the code was written.
-103
View File
@@ -16,9 +16,6 @@
# scribe_reached STATE SID the server answered: the next outage speaks again
# scribe_config sets `url` + `token` from the env, returns 0
# only if BOTH are usable (#2278)
# scribe_rules_live FILE live rule ids from the exclusion ledger,
# comma-joined; entries age out (#3751)
# scribe_rules_append FILE stdin ids -> the ledger, timestamped
#
# Sourced, not executed: `. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"`.
@@ -178,103 +175,3 @@ scribe_unreached() {
scribe_reached() {
rm -f "$1/$2.unreached" 2>/dev/null || true
}
# ---------------------------------------------------------------------------
# THE RULE EXCLUSION LEDGER, and how entries in it AGE (#3751).
#
# Two hooks write this file and two read it, which is the whole reason the
# parsing lives here: `scribe_prior_art.sh` and `scribe_tool_rules.sh` share
# one ledger so a rule named by one arm is not re-offered by the other, and a
# format only one of them understood would break that on the first read.
#
# WHAT PROBLEM AGEING SOLVES. #3749 clears the ledger when an EVENT destroys
# context — a compaction or a /clear. This is the case with no event at all: a
# long session where the rule was named two hundred turns ago and has simply
# fallen out of attention. It is #3702's argument at the tier level (present in
# context and salient at the moment are different properties) applied to time
# instead of to tier.
#
# PER-ENTRY TIMESTAMPS, NOT A FILE MTIME. Clearing the whole ledger when the
# file is old is one line of shell and wrong in exactly the session that needs
# it: a single recent write keeps every stale id alive, and the ids that go
# stale first are the ones from the rules that fire most.
#
# WALL TIME, NOT A TURN COUNT, and the trade is real rather than dismissed. A
# turn count is a truer model of salience — an idle session does not forget —
# but a hook has no turn number without keeping its own counter, which is a
# second piece of session state to write, read, clear on compaction and get
# wrong. Wall time is available from `date` and costs nothing. The failure mode
# it accepts is a session left idle over lunch treating its rules as forgotten,
# which produces one extra full line per rule and no other harm.
_SCRIBE_RULE_TTL=2700
# 45 MINUTES, and the reasoning rather than the number (rule 32).
#
# There is no data on this yet, so it is a judgement made to be revised — the
# telemetry that would settle it is the one #3807 just built, and a reading of
# how often an aged-out rule gets PULLED after it returns is what should move
# this.
#
# Too short and the exclusion stops existing and the repetition it prevents
# comes back. Too long and it never fires at all in a session short enough to
# matter. 45 minutes is about one working stretch on a single task: long enough
# that a rule does not re-announce itself while you are still doing the thing
# it governs, short enough that a multi-hour session gets a genuine refresh
# rather than one 9am mention.
#
# Being wrong on the short side is now the cheaper error, which is why this
# leans short. Since #3750 an excluded rule is REFERENCED rather than withheld,
# so the ledger is no longer the only thing standing between a session and a
# rule it has forgotten — an expired entry costs one full line instead of one
# short one, and the exclusion re-arms the moment it is spent.
# Live ids from a ledger, comma-joined for `exclude_rule_ids`. Empty output for
# a missing, empty or fully-aged file — the callers already treat "" as "send
# no exclusions".
#
# THE LAST ENTRY FOR AN ID WINS, and this is what stops a rule ping-ponging.
# The file is append-only, so a rule that ages out, gets surfaced fresh and is
# appended again has TWO lines. Reading the first would leave it permanently
# expired and it would re-announce itself on every single call from then on —
# the loudest possible failure, from the mechanism meant to quieten things.
# Appends are chronological, so the last line for an id is its most recent.
#
# A BARE ID — no tab, no timestamp — IS LIVE. That is the pre-#3751 format, and
# a session in flight when this ships has a ledger full of them. Treating
# unknown as expired would make every one of those sessions re-announce every
# rule it had already been told, all at once, which is precisely the noise this
# exists to prevent. Unknown means "not measured" and never "old" — the same
# null discipline the retrieval_logs columns use. Those entries simply never
# age, which is bounded: the session ends.
scribe_rules_live() {
local f="$1" now
[ -n "$f" ] && [ -f "$f" ] || return 0
now=$(date +%s 2>/dev/null) || now=0
awk -F'\t' -v now="$now" -v ttl="$_SCRIBE_RULE_TTL" '
{
id = $1
gsub(/[^0-9]/, "", id)
if (id == "") next
if (!(id in seen)) { seen[id] = 1; seq[++n] = id }
stamp[id] = ($2 ~ /^[0-9]+$/) ? $2 : ""
}
END {
out = ""
for (i = 1; i <= n; i++) {
id = seq[i]
if (stamp[id] != "" && now > 0 && (now - stamp[id]) > ttl) continue
out = out (out == "" ? "" : ",") id
}
print out
}
' "$f" 2>/dev/null || true
}
# Append surfaced ids, stamped. Reads ids on stdin, one per line — the shape
# `jq -r '(.rule_ids // [])[]?'` already produces at both call sites.
scribe_rules_append() {
local f="$1" now
[ -n "$f" ] || return 0
now=$(date +%s 2>/dev/null) || now=0
awk -v ts="$now" 'NF { print $1 "\t" ts }' >> "$f" 2>/dev/null || true
}
+5 -7
View File
@@ -204,11 +204,10 @@ if [ -n "$session_id" ]; then
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
fi
# Ageing, not a flat read (#3751), and the ONLY ledger here that ages: the
# note channels above are a different question with a different answer, and
# this arm's sibling hook reads the same rule file through the same helper.
rule_seen=$(scribe_rules_live "$rulefile")
[ -n "$rule_seen" ] && rule_exclude_q="&exclude_rule_ids=${rule_seen}"
if [ -f "$rulefile" ]; then
rule_seen=$(tr '\n' ',' < "$rulefile" 2>/dev/null | sed 's/,$//')
[ -n "$rule_seen" ] && rule_exclude_q="&exclude_rule_ids=${rule_seen}"
fi
fi
# Not `|| exit 0`: an unreachable instance must not discard a local finding
@@ -240,8 +239,7 @@ if [ -n "$body" ]; then
printf '%s' "$body" | jq -r '(.sync_note_ids // [])[]?' 2>/dev/null >> "$syncfile" || true
fi
if [ -n "$rulefile" ]; then
printf '%s' "$body" | jq -r '(.rule_ids // [])[]?' 2>/dev/null \
| scribe_rules_append "$rulefile"
printf '%s' "$body" | jq -r '(.rule_ids // [])[]?' 2>/dev/null >> "$rulefile" || true
fi
if [ -n "$derivefile" ]; then
printf '%s' "$body" | jq -r '(.derive_keys // [])[]?' 2>/dev/null >> "$derivefile" || true
-69
View File
@@ -55,53 +55,6 @@ here=$(CDPATH= cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) || exit 0
event=$(cat 2>/dev/null || true)
source=$(printf '%s' "$event" | jq -r '.source // empty' 2>/dev/null) || source=""
# --- The rule ledger outlives the context it describes (#3749) ---
#
# scribe_prior_art.sh and scribe_tool_rules.sh record every rule id they have
# named in <state>/<sid>.rules.ids and hand it back as exclude_rule_ids, so a
# rule is named once per session and then goes quiet. That is right while the
# session still HOLDS what it was told, and wrong the moment it does not.
#
# A compaction summarizes the earlier injections away and does not touch the
# filesystem, so the rule ends up absent from context AND still excluded —
# unreachable for the rest of the session. The banner below tells the model to
# re-pull its ALWAYS-ON rules, but a rule an arm surfaced is conditional and is
# not in that set, so it has no other way back. The rules most likely to be in
# this state are the ones that fire most often, which is to say the ones that
# apply most.
#
# The session id survives a compaction — the etag marker further down is
# rewritten on `compact` and keyed by session_id, which is only meaningful if
# the id is stable — so the stale ledger is genuinely found again, not orphaned.
#
# CLEARED ON THE SOURCES THAT DESTROY CONTEXT, AND ONLY THOSE:
#
# compact CLEAR — summarized away; the file survived.
# clear CLEAR — context wiped.
# startup nothing to do: a new session id means a new, empty file.
# resume KEEP. The context was genuinely restored, so the ledger still
# describes what the session holds. Clearing here would re-surface
# every rule after a restore that lost nothing — the mirror error.
# fork KEEP, and the answer is the same whichever way forks are keyed: a
# fork carries the conversation, so if it inherits the id the ledger
# is accurate, and if it gets a new one the file is empty anyway.
#
# ONLY the rules ledger. The same directory holds .ids / .sync.ids /
# .derive.ids for the note arms. Whether a surfaced NOTE should return after a
# compaction is a different question with a different answer, and leaving those
# alone is a decision rather than an oversight.
case "$source" in
compact|clear)
sid=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || sid=""
if [ -n "$sid" ]; then
safe_sid=$(printf '%s' "$sid" | tr -c 'A-Za-z0-9._-' '_')
# Best-effort, like every other filesystem touch in these hooks: a ledger
# that cannot be removed costs a repeated exclusion, never a session.
rm -f "${TMPDIR:-/tmp}/scribe-priorart/${safe_sid}.rules.ids" 2>/dev/null || true
fi
;;
esac
out=""
# Append $1 to $out, separated by a horizontal rule when $out already has content.
append() { if [ -n "$out" ]; then out="${out}"$'\n\n---\n\n'"$1"; else out="$1"; fi; }
@@ -156,28 +109,6 @@ if [ -n "$url" ] && [ -n "$token" ] && command -v curl >/dev/null 2>&1; then
-H "Authorization: Bearer ${token}" \
"${url%/}/api/plugin/context${q}" 2>/dev/null) || body=""
[ -n "$body" ] && dyn=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null)
# Stash the rules marker for the write-path hook (milestone 323). THIS is
# where it has to be captured: the model receives one from
# list_always_on_rules too, but a hook cannot see an MCP tool's result. Stored
# under the same state dir the prior-art hook already uses, keyed by session,
# so "changed since" means since THIS session loaded its rules.
#
# Written on `compact` as well as `startup`, and that is correct rather than
# convenient: a compact tells the session to re-pull its rules, so the marker
# should describe the set it is about to hold. It is also why this cannot
# cover the compaction case — see the table in services/plugin_context.py.
if [ -n "$body" ]; then
etag=$(printf '%s' "$body" | jq -r '.rules_etag // empty' 2>/dev/null) || etag=""
if [ -n "$etag" ]; then
sid=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || sid=""
safe_sid=$(printf '%s' "${sid:-nosession}" | tr -c 'A-Za-z0-9._-' '_')
etag_dir="${TMPDIR:-/tmp}/scribe-priorart"
# Best-effort throughout: a marker that cannot be stored costs a hint,
# never the session.
mkdir -p "$etag_dir" 2>/dev/null \
&& printf '%s' "$etag" > "$etag_dir/${safe_sid}.rules_etag" 2>/dev/null || true
fi
fi
[ -z "$dyn" ] && status="> ⚠️ Scribe: live rules/project context could not be loaded this session (instance unreachable or request failed). The standing guidance above still applies — pull rules with \`list_always_on_rules()\` and project context with \`enter_project()\` as needed."
elif [ -n "$url" ] && [ -z "$token" ]; then
status="> ⚠️ Scribe: live context disabled this session — the API key is not configured (Scribe base URL is). Set it with \`/plugin\` → Scribe → configure, or export SCRIBE_TOKEN. Tools still work; pull rules with \`list_always_on_rules()\` and project context with \`enter_project()\`."
-10
View File
@@ -21,16 +21,6 @@ for the operator's work, and as your own working memory across sessions.
compaction — call `list_always_on_rules()` (and `enter_project()` when a
project is in scope) BEFORE acting. When a loaded rule and a default habit
disagree, the rule wins; if no rule speaks to it, ask rather than assume.
- **What you loaded is not all of the rules.** Only the always-on tier arrives
that way; conditional rules are RETRIEVED, and one you were never handed
binds exactly as hard. So before a consequential act, `search` for a rule
about it (`content_type="rule"`) rather than concluding from an empty
loaded set that nothing applies. "I was not told" is not the same as "there
is no rule," and only one of those is checkable.
This bites hardest on which TOOL to reach for — curling an API that has an
MCP client, standing up a local stack, running a suite CI owns. Those feel
like mechanics rather than decisions, so they raise no doubt and generate no
query; the moment you are most confident is the moment to look.
- **Recall before acting** — before you answer anything about the operator's
work or start a task, `search` Scribe first; assume a related note, task, or
decision already exists. Concretely, reach for recall whenever a request
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
# Scribe — PreToolUse rule arm for ACTIONS (#3476).
#
# The sibling of scribe_prior_art.sh. That hook is registered on Write|Edit and
# asks "what is recorded about the file being written". This one asks "does a
# standing rule speak to the command about to be run" — the question nothing
# could ask before, and the reason every rule about which tool to reach for had
# to live in the always-on preload instead.
#
# WHY A HOOK AND NOT AN INSTRUCTION. A reflex generates no query (note #3089):
# you reach for `curl` confidently, with no moment of doubt, so a surface that
# waits to be asked never fires. Here nothing is asked — the tool call IS the
# query, and the reflex has to become a tool call before it can do anything.
#
# SILENT ON OUTAGE, deliberately, unlike the prior-art hook. A write is
# occasional; a Bash call is not, and an "instance did not answer" line before
# every command is the noise that gets a channel muted. scribe_prior_art.sh
# still speaks for both when the instance is down.
#
# Env:
# SCRIBE_URL / SCRIBE_TOKEN override for the settings.json dogfooding path.
command -v jq >/dev/null 2>&1 || exit 0
command -v curl >/dev/null 2>&1 || exit 0
# PreToolUse delivers { session_id, cwd, tool_name, tool_input: {...}, ... }
event=$(cat 2>/dev/null || true)
tool_name=$(printf '%s' "$event" | jq -r '.tool_name // empty' 2>/dev/null) || exit 0
session_id=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || session_id=""
event_cwd=$(printf '%s' "$event" | jq -r '.cwd // empty' 2>/dev/null) || event_cwd=""
[ -n "$tool_name" ] || exit 0
# The action, as text. `.command` is Bash's field; the fallbacks let the matcher
# in hooks.json widen to other tools without this script changing — which is the
# whole reason the server side takes a name and a string rather than a schema.
command_text=$(printf '%s' "$event" | jq -r '
.tool_input.command //
.tool_input.url //
.tool_input.prompt //
empty' 2>/dev/null) || command_text=""
[ -n "$command_text" ] || exit 0
# shellcheck source=plugin/hooks/scribe_defs.sh
. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
# scribe_config, not a hand-rolled pair of parameter expansions: it also treats
# an UNEXPANDED `${...}` placeholder as unset, which would otherwise be sent as
# a garbage Bearer token and 401 on every call (#2198's class).
scribe_config || exit 0
# Bounded before encoding: a heredoc or a pasted script can be enormous, and
# the verb and its target — the part a rule is about — sit at the front. The
# server bounds it again; this keeps a huge payload off the wire in the first
# place. `head -c`, never `cut -c`: cut truncates each LINE and caps nothing.
command_text=$(printf '%s' "$command_text" | head -c 2000)
# -sRr, never -rR: jq -R without -s reads LINE BY LINE, so a multi-line command
# would encode per line and join with raw newlines — an invalid URL.
cmd_enc=$(printf '%s' "$command_text" | jq -sRr '@uri' 2>/dev/null) || exit 0
tool_enc=$(printf '%s' "$tool_name" | jq -sRr '@uri' 2>/dev/null) || exit 0
repo_q=""
lookup_dir=${event_cwd:-${CLAUDE_PROJECT_DIR:-$PWD}}
repo_remote=$(git -C "$lookup_dir" remote get-url origin 2>/dev/null || true)
if [ -n "$repo_remote" ]; then
repo_enc=$(printf '%s' "$repo_remote" | jq -sRr '@uri' 2>/dev/null) || repo_enc=""
[ -n "$repo_enc" ] && repo_q="&repo=${repo_enc}"
fi
# THE SHARED SESSION LEDGER, and the thing most worth getting right here.
#
# scribe_prior_art.sh keeps the rules it has already named in
# <state>/<sid>.rules.ids and passes them as exclude_rule_ids. This hook reads
# and appends to that SAME file rather than keeping its own: two ledgers would
# mean a rule named by one arm gets re-offered by the other, and the hint that
# fires most often is exactly the one that must not repeat itself.
#
# The directory keeps the prior-art name on purpose — renaming it would orphan
# every live session's state for a cosmetic gain.
state_dir="${TMPDIR:-/tmp}/scribe-priorart"
mkdir -p "$state_dir" 2>/dev/null || true
rulefile=""
rule_exclude_q=""
if [ -n "$session_id" ]; then
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
rulefile="$state_dir/${safe_sid}.rules.ids"
# Ageing, not a flat read (#3751): an id named two hours ago is not one the
# session is still holding. scribe_rules_live carries the reasoning.
rule_seen=$(scribe_rules_live "$rulefile")
[ -n "$rule_seen" ] && rule_exclude_q="&exclude_rule_ids=${rule_seen}"
fi
# `|| exit 0` here, unlike the prior-art hook: there is no local arm whose
# finding would be discarded, and an outage line before every command is worse
# than silence. See the header.
body=$(curl -fsS --max-time 5 \
-H "Authorization: Bearer ${token}" \
"${url%/}/api/plugin/tool-rules?tool=${tool_enc}&command=${cmd_enc}${repo_q}${rule_exclude_q}" 2>/dev/null) || exit 0
context=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null) || exit 0
[ -n "$context" ] || exit 0
# Remember what was named so it is not repeated this session.
if [ -n "$rulefile" ]; then
printf '%s' "$body" | jq -r '(.rule_ids // [])[]?' 2>/dev/null \
| scribe_rules_append "$rulefile"
fi
jq -cn --arg ctx "$context" '{
hookSpecificOutput: {
hookEventName: "PreToolUse",
additionalContext: $ctx
}
}' 2>/dev/null || true
exit 0
+5 -86
View File
@@ -56,31 +56,10 @@ Two constraints on *how* that's achieved:
re-deriving it or opening a duplicate. When a project is in scope, pass its
`project_id` so results stay scoped.
2. **Standing rules are binding — and the ones you were handed are not all of
them.** Load the resident set via `list_always_on_rules()` at session start
(see "Do this first"); treat every one as binding. Pull a rule's full
statement with `get_rule(id)` when it's about to bite. When a project is in
scope, `enter_project(id)` also returns its applicable rules.
Rules come in two tiers. **Always-on** rules are delivered — they arrive
whether or not you ask. **Conditional** rules are RETRIEVED, and one binds
just as hard for never having been handed to you. So before a consequential
act, `search(content_type="rule")` on what you are about to do. An empty
loaded set is not evidence that no rule applies; it is only evidence that
none was pushed, and those are different claims.
The tier split exists because delivery does not scale: every resident rule
costs tokens in every session forever, so a rulebook that grows past a few
dozen either stops growing or stops fitting. Retrieval is what lets the
rulebook keep growing — but retrieval only fires if something asks.
**Ask hardest where you feel most certain.** Rules about which TOOL to reach
for — use the forge's MCP client rather than curling its API, don't stand up
a local stack, don't run the suite CI owns — govern moves that feel like
mechanics rather than decisions. A reflex raises no doubt, so it generates
no query, so the rule that would have stopped it is never retrieved. That is
the failure this instruction exists to prevent, and confidence is its only
warning sign.
2. **Standing rules are binding.** Load them via `list_always_on_rules()` at
session start (see "Do this first"); treat every one as binding. Pull a
rule's full statement with `get_rule(id)` when it's about to bite. When a
project is in scope, `enter_project(id)` also returns its applicable rules.
3. **Update over duplicate.** When recording, prefer updating an existing
note/rule/task over creating a new one. Search first; revise what's there.
@@ -117,25 +96,7 @@ Two constraints on *how* that's achieved:
not restraint. Only a record genuinely about no particular area goes
untagged.
8. **Name the record, never just its number.** Whenever you refer to a Scribe
record — in a message to the operator, a commit message, a task body, a
work-log — write the id *and* its title: `#3244 "the staleness signal"`,
`milestone 323 "rule versioning"`. Not `#3244`.
You have the record open; the operator does not. A bare id reads as
complete to you and as homework to them — they have to look it up to know
what their own conversation is about, or guess. Scribe's own duplicate gate
already writes `id 412: "debounce helper"` for exactly this reason; match
it everywhere else.
The first mention in a message carries the title; later mentions of the
same record can use the bare id. If you don't know the title, look it up
before citing the number — an id you can't name is one you haven't checked.
This matters most in the places read later by someone with even less
context than the operator has now: commit messages, task bodies, and any
record that cites another.
9. **State updates in place; chronicles don't.** A dev-log records what
8. **State updates in place; chronicles don't.** A dev-log records what
*happened* — write it once, never rewrite it. A durable finding (how a
subsystem works, a measured number) lives in that System's **reference
note** ("«System» — reference"), which you UPDATE as facts change — safe,
@@ -145,48 +106,6 @@ Two constraints on *how* that's achieved:
re-measurement, a reversed decision), pass the old id in `supersedes` so the
stale record is demoted and labelled rather than left competing.
10. **A few notes assert a FACT, and those can carry their own check.**
Supersession only fires once somebody has read a note and disagreed — which
is the case where it was already believed. A note asserting something about
*someone else's* software — what a service does on a duplicate upload, how a
forge numbers its CI runs, what an updater compares — can instead carry
`verify_with` (how to check it) and `expires_when` (the STATE that ends it:
"when the forge numbers runs per workflow", never "in six months").
`notes_due_for_verification` lists them least-recently-confirmed first, with
never-checked at the top; `mark_note_verified` records what you found, and
`still_true=False` deliberately writes nothing — a note whose check failed
is wrong rather than in a state worth recording, so it keeps its place.
**The test is one question: could this note become false without anyone
editing it?** If no, leave both fields empty. That is the normal case, and
an empty `verify_with` is the positive marker for "this is a decision, there
is nothing to go and check" — not an unfinished record. The sweep is only
worth reading while almost nothing is on it, so a check added out of
tidiness costs the whole surface, not just that note.
**The sharper form of the same test: is the thing this note describes yours
to change?** If yes it is a decision — editing your own software is how it
changes, and you will know you did it. Measured against a real corpus, every
note that earned a check was about somebody ELSE's software: a signing
service, a forge, a hub, an SDK, a model, a dependency set.
**Three that look like candidates and are not:**
- **Resume pointers and "current state" notes.** They go stale fastest of
anything, which is exactly why they tempt — but the cure is to update or
delete them, not to schedule a check. A sweep full of pointers is a sweep
nobody reads.
- **Measurements of your own system.** They go false because you changed
something, and you knew. A measurement earns a check only when what it
measures is outside your control.
- **A decision that RESTS on somebody else's behaviour.** The decision is
still a decision. Put the check on the note asserting the fact, and link
the decision to it.
Not for tasks — a task's decay is its status, and a done issue records what
happened rather than asserting something that can go false. Not for snippets
either: `verify_snippet` compares the recorded location and code against the
repo, which is richer and already wired to drift detection.
## Stay inside the active project's scope
Once a project is in scope — you called `enter_project`, or the working repo is
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
# Bump the patch segment of fable-mcp/pyproject.toml version and stage the file.
# Usage: called automatically by the Claude Code pre-commit hook, or manually.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
FILE="$REPO_ROOT/fable-mcp/pyproject.toml"
current=$(grep '^version = ' "$FILE" | sed 's/version = "\(.*\)"/\1/')
major=$(echo "$current" | cut -d. -f1)
minor=$(echo "$current" | cut -d. -f2)
patch=$(echo "$current" | cut -d. -f3)
new_version="$major.$minor.$((patch + 1))"
sed -i "s/^version = \"$current\"/version = \"$new_version\"/" "$FILE"
git -C "$REPO_ROOT" add "$FILE"
echo "fable-mcp: $current$new_version"
+15 -22
View File
@@ -119,20 +119,15 @@ def same_hue_text_on_tint(css: str) -> tuple[list[str], list[str]]:
dangerous of the two — it does not even name a `-bg` token, so nothing
about it looks like the pattern until you measure it.
Returned separately because they were paid down separately — the token
form first (7 badge pairs), then the inline form (46 sites across 18
files, 26 of them --fs-accent). Both are clean now, so BOTH gate. The
split is kept because the two spellings need different error text: one
names a -bg token you can search for, the other names nothing at all.
Returned separately because they are at different stages. The token form
is CLEAN and therefore gates. The inline form has a live backlog (48 sites
when this split was written, 26 of them --fs-accent), so it reports with a
count: a gate nobody can satisfy today gets switched off, and then it
guards nothing.
"""
token_form, inline_form = [], []
for body in re.findall(r"\{([^{}]*)\}", css):
# (?<![-\w]) or `border-color`, `border-left-color` and `outline-color`
# all match as if they were text. They are not: a border is a non-text
# graphic and its floor is 3:1, not 4.5. Without this the check reported
# seven rules that were already correct — and a check that cries wolf on
# correct code is one that gets muted.
fg = set(re.findall(r"(?<![-\w])color\s*:\s*var\(\s*(--fs-[\w-]+?)\s*\)", body))
fg = set(re.findall(r"color\s*:\s*var\(\s*(--fs-[\w-]+?)\s*\)", body))
bg_tok = set(re.findall(r"background(?:-color)?\s*:\s*var\(\s*(--fs-[\w-]+?)-bg\s*\)", body))
bg_inl = set(re.findall(
r"background(?:-color)?\s*:\s*color-mix\([^;]*?var\(\s*(--fs-[\w-]+?)\s*\)[^;]*?\)",
@@ -237,23 +232,21 @@ def main() -> int:
print("OK — no text painted with a token on a tint of its own -bg.\n")
if inline_tint_hits:
print(f"FAIL — {len(inline_tint_hits)} rule(s) paint text with a token on an "
f"INLINE color-mix tint of that same token.")
print(" Identical defect to the block above, spelled without a -bg token —")
print(" which is what let it hide: nothing about it LOOKS like the pattern.")
print(" Use the token's -fg sibling.\n")
for path, tok in inline_tint_hits:
print(f" {path}: color: var({tok}) on an inline tint -> var({tok}-fg)")
print()
else:
print("OK — no text painted with a token on an inline tint of itself.\n")
by_tok: dict[str, int] = {}
for _p, tok in inline_tint_hits:
by_tok[tok] = by_tok.get(tok, 0) + 1
print(f"REPORT — {len(inline_tint_hits)} rule(s) paint text with a token on "
f"an INLINE color-mix tint of that same token.")
print(" Same defect, spelled without a -bg token so it does not gate yet.")
print(" Worst offenders: " + ", ".join(
f"{t} x{n}" for t, n in sorted(by_tok.items(), key=lambda kv: -kv[1])[:4]) + "\n")
if args.report_literals:
print(f"REPORT — {literal_count} raw colour literal(s) in component CSS.")
print(" Advisory: a literal is a value stated outside the system, so it "
"cannot follow a palette change.\n")
return 1 if (unresolved or same_hue_hits or inline_tint_hits) else 0
return 1 if (unresolved or same_hue_hits) else 0
if __name__ == "__main__":
+45 -278
View File
@@ -13,20 +13,9 @@ separate defects have reached a live install through that path:
install, because `plugin.json`'s version wasn't bumped and the installer
compares versions to decide whether to refresh its cache.
Both were fixed. The second was fixed TWICE — once by bumping the number, and
then properly, by removing the class it came from: `plugin.json`'s version is
no longer a value anybody chooses. `scripts/mint_plugin_version.py` derives it
from the clock (`make mint-plugin`), and `check_version_is_minted` below fails
the lane when shipped content moved and the version did not.
State exactly what that did and did not remove, because a rationale that
overstates its own control is how the control gets trusted past its limit, and
because the paragraph this replaces was itself read that way. Gone: having to
remember which NUMBER to write, and the whole question of whether a chosen
number was the right one. Not gone: the mint still has to be RUN, and
forgetting to run it is still possible. What changed is that forgetting is now
LOUD — a red lane on the batch that forgot, instead of a silent no-op found
weeks later when somebody says "I don't think it updated" (#2220).
The rule for the second one was already written down and was still missed. A
written rule that depends on being remembered during a long session is not a
control; this is.
shellcheck and jq are NOT in `ci-python` (verified against CI-runner's Dockerfile
and scripts/install-common.sh, not from memory — rule #37). CI installs both
@@ -42,15 +31,8 @@ itself loudly, because a check that quietly no-ops is the failure mode this
whole file exists to prevent.
Usage:
python3 scripts/check_plugin.py # all checks
python3 scripts/check_plugin.py --no-version # on `main` only — see below
`--no-version` exists for ONE case. The version is measured against
`origin/main`, so on `main` itself the comparison is against itself and answers
nothing; the syntax, pattern and marker checks are the only ones that mean
anything there. It is NOT a way past a red lane — see
`check_version_is_minted`, whose whole design is shaped by keeping this flag
out of anyone's muscle memory.
python3 scripts/check_plugin.py # all checks
python3 scripts/check_plugin.py --no-version # skip the bump check
"""
from __future__ import annotations
@@ -61,90 +43,16 @@ import re
import shutil
import subprocess
import sys
from datetime import datetime, timedelta, timezone
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
# The shape contract is ONE definition, shared with the script that mints it —
# a checker carrying its own copy of the format would drift from the minter
# and pass values the minter can no longer produce. Explicit path insert
# because this file runs both as `python3 scripts/check_plugin.py` (which puts
# `scripts/` on the path, not the root) and as an import from the test suite.
sys.path.insert(0, str(ROOT))
from scripts.mint_plugin_version import VERSION_RE # noqa: E402
PLUGIN_DIR = ROOT / "plugin"
HOOKS_DIR = PLUGIN_DIR / "hooks"
MANIFEST = PLUGIN_DIR / ".claude-plugin" / "plugin.json"
# ── What ships, and what decides what it says about itself ─────────────────
#
# ONE definition (#3127 §3, milestone 334 step 2). It has TWO consumers that
# need different granularities, and conflating them is the bug:
#
# the workflow's `paths:` trigger whole paths should CI run at all?
# the version check paths MINUS should the version
# the manifest have moved?
# `version`
#
# The second one is why this is not just a tuple of paths. `plugin.json` lives
# INSIDE `plugin/`, so a version bump is itself a change to the shipped set —
# and a check that reads the set naively then treats the bump as its own
# justification. Any bump passes, no bump fails, and it has proved nothing.
# `shipped_content_changed` below is the exclusion-aware reader.
#
# The exclusion is that ONE FIELD, never the whole file: `plugin.json` also
# carries description, mcpServers and userConfig, all of which reach an
# install and all of which matter. Excluding the file wholesale would mean a
# userConfig-only edit computes an unchanged version and never refreshes —
# #2209 again with a narrower trigger.
SHIPPED_PATHS = ("plugin", ".claude-plugin")
# Files that decide what a published artifact SAYS ABOUT ITSELF — kept as a
# table so the next artifact is a one-line addition rather than a third
# bespoke guard (#3127 §3). The membership test is NOT "is this copied into
# the artifact?" but "can changing this file change the published bytes, or
# what the artifact says about itself?" — FC learned that twice in four days
# (#3156, #3202), and a deriver is never in the COPY list.
#
# Note what is absent: a CHECKER does not belong here. Whatever validates a
# version decides whether the lane goes red, not what any artifact reports,
# so `check_plugin.py` itself is not a deriver, while the script that mints
# the plugin version is.
DERIVERS: dict[str, tuple[str, ...]] = {
# The "Generate image tags and version" step computes the server image's
# name, ordering key and channel (#3298).
".forgejo/workflows/ci.yml": ("server-image",),
# Decides the plugin's version FORMAT, so it decides what every future
# manifest says about itself (milestone 334 step 3).
"scripts/mint_plugin_version.py": ("plugin",),
}
def version_relevant_paths() -> tuple[str, ...]:
"""Everything a change to which must produce a NEW plugin version.
Wider than `SHIPPED_PATHS`, and #3127 §3's asymmetry is why it has to be:
A change to how the VERSION is computed is compared against nothing at
all. Left out, the published artifact goes on reporting the OLD value
indefinitely.
Concretely — change the mint script's format string, change nothing else,
and a diff over the shipped paths alone reports "no content change, the
version need not move". The manifest then keeps a value in the old format
forever and nothing ever says so. The mint script reaches no install and
belongs here anyway; that is #3156's exact shape.
A CHECKER is deliberately not here. Whatever validates the version decides
whether the lane goes red, not what any artifact reports — so this file is
absent from its own set, and that is not an oversight.
"""
return SHIPPED_PATHS + tuple(
path for path, artifacts in DERIVERS.items() if "plugin" in artifacts
)
# Paths whose contents reach an install. Keep in step with the workflow's
# `paths:` filter — a path that ships but isn't checked here is the gap again.
SHIPPED = ("plugin", ".claude-plugin")
failures: list[str] = []
@@ -235,6 +143,8 @@ def check_patterns() -> None:
ok(f"{rel}: no known-bad patterns")
# --- the version bump ------------------------------------------------------
# --- shellcheck ------------------------------------------------------------
def check_shellcheck() -> None:
@@ -305,16 +215,6 @@ SMOKE_EVENTS: dict[str, str] = {
"tool_input": {"file_path": "src/x.py",
"new_string": f"def {_ABSENT_SYM}():\n pass\n"}}
),
# The pre-tool rule arm (#3476). A real Bash call, and one whose whole
# point is that it looks harmless: reaching for curl against the forge API
# is the reflex the arm exists to catch. With no instance it must stay
# SILENT — it is deliberately not an OUTAGE_SPEAKER, because a Bash call is
# not occasional and an outage line before every command gets the channel
# muted.
"scribe_tool_rules.sh": json.dumps(
{"session_id": "smoke", "cwd": ".", "tool_name": "Bash",
"tool_input": {"command": "curl -s https://example.invalid/api/v1/runs"}}
),
"scribe_sync_processes.sh": json.dumps({"source": "startup"}),
"scribe_session_context.sh": json.dumps({"source": "startup"}),
# The after-write hook (#2901) diffs the working tree; on CI's clean
@@ -491,213 +391,80 @@ def _git(*args: str) -> tuple[int, str]:
return proc.returncode, (proc.stdout or proc.stderr).strip()
def manifest_text(ref: str | None = None) -> str | None:
"""The manifest's RAW TEXT at `ref`, or in the working tree when ref is None.
Split out from `manifest_version` because the exclusion below needs every
field except one, not the one field.
"""
def manifest_version(ref: str | None = None) -> str | None:
"""The manifest version at `ref`, or in the working tree when ref is None."""
if ref is None:
try:
return MANIFEST.read_text()
except OSError:
return json.loads(MANIFEST.read_text()).get("version")
except Exception:
return None
rel = MANIFEST.relative_to(ROOT).as_posix()
code, out = _git("show", f"{ref}:{rel}")
return out if code == 0 else None
def manifest_version(ref: str | None = None) -> str | None:
"""The manifest version at `ref`, or in the working tree when ref is None."""
text = manifest_text(ref)
if text is None:
if code != 0:
return None
try:
return json.loads(text).get("version")
return json.loads(out).get("version")
except Exception:
return None
# Distinct from None, which is a legitimate "this manifest does not exist".
_UNREADABLE = object()
def check_version_bump(base: str = "origin/main") -> None:
"""If shipped plugin content differs from `base`, the version must too.
def manifest_differs_beyond_version(a: str | None, b: str | None) -> bool:
"""Do two `plugin.json` texts differ in anything OTHER than `version`?
THE exclusion, and it is kept pure — no git, no filesystem — because this
is the half worth testing hard and it needs no repository to exercise.
Compares PARSED objects rather than text, so reformatting, key reordering
and whitespace do not read as content changes. `version` is dropped from
both sides; everything else counts, which is what keeps a userConfig-only
or mcpServers-only edit demanding a new version.
Unreadable input answers True. The conservative direction is "demand a new
version": a spurious bump costs one cache refresh, while a missed one is
#2209 — the fix reaches the repo and stops there.
"""
def without_version(text: str | None):
if text is None:
return None
try:
data = json.loads(text)
except Exception:
return _UNREADABLE
if not isinstance(data, dict):
return _UNREADABLE
return {k: v for k, v in data.items() if k != "version"}
left, right = without_version(a), without_version(b)
if left is _UNREADABLE or right is _UNREADABLE:
return True
return left != right
def shipped_content_changed(base: str) -> tuple[bool | None, list[str]]:
"""Has anything that REACHES AN INSTALL changed against `base`?
Returns `(changed, paths)`. `changed` is **None** when the question could
not be answered — a caller must never read that as "no", which is the
distinction #2663 cost weeks of zeroed telemetry to learn.
The manifest is special-cased, not excluded: if it is the ONLY thing that
moved and the only difference is `version`, nothing that reaches an
install has changed. Any other manifest field, or any other file, counts.
Reads `version_relevant_paths`, which is the shipped set PLUS the files
that decide the version — see there for why the deriver has to be in it.
"""
code, out = _git("diff", "--name-only", base, "--", *version_relevant_paths())
if code != 0:
return None, []
paths = [p for p in out.splitlines() if p.strip()]
if not paths:
return False, []
rel_manifest = MANIFEST.relative_to(ROOT).as_posix()
if paths == [rel_manifest]:
return manifest_differs_beyond_version(
manifest_text(), manifest_text(base)
), paths
return True, paths
def check_version_is_minted(base: str = "origin/main") -> None:
"""THE control (#3127 checklist 4), replacing "somebody remembers".
The checklist asks, of any hand-set component: *say what happens the
release somebody forgets it.* This is the answer — the lane goes red,
deterministically, because CI can compute whether the value should have
moved. Its predecessor could only ask "did the number move at all", which
any bump satisfied and which therefore proved nothing.
Four verdicts:
content changed, version did not FAIL — this is #2209, exactly
version not in canonical shape FAIL — see below
version implausibly in the future FAIL — a bad clock or a hand-edit
version moved, content did not pass, and say so
THE LAST ROW IS NOT A FAILURE, DELIBERATELY. A needless re-mint costs one
cache refresh and nothing else. Failing the lane over a harmless act is how
a check earns a `--no-version` in somebody's muscle memory and stops
running at all — which is the failure mode this whole file exists to
prevent. The implication that matters is one-directional: content changed
IMPLIES version moved.
A malformed version is worth failing on even though the installer would
accept it. `K4` returns the manifest string verbatim, and `H == "unknown"`
sets `forceOverwrite`, so a broken value either sorts as a normal string
or reinstalls the plugin every single session (#3325). Neither is loud.
Stated against the BASE BRANCH rather than the last commit, as its
predecessor was: a per-commit rule would demand a fresh mint from every
commit in a batch, when what matters is that whatever reaches an install
differs from what is cached. One mint per batch, which is also how a person
would do it.
Stated against the BASE BRANCH rather than the last commit on purpose. A
per-commit rule would demand a bump from every commit in a batch; what
actually matters is that whatever reaches an install carries a version the
installer can tell apart from the one already cached. One bump per batch,
which is also how a human would do it.
"""
code, _ = _git("rev-parse", "--verify", base)
if code != 0:
# Do NOT pass silently — a check that quietly no-ops is how this class
# of bug survives in the first place.
fail(
f"cannot resolve {base}, so the minted-version check could not run. "
f"cannot resolve {base}, so the version-bump check could not run. "
f"Fetch it first — `git fetch --depth=1 origin main:refs/remotes/"
f"origin/main` is enough, since this diffs two trees and needs no "
f"common ancestor — or pass --no-version deliberately."
)
return
here = manifest_version()
code, changed = _git("diff", "--name-only", base, "--", *SHIPPED)
if code != 0:
fail(f"git diff against {base} failed: {changed}")
return
if not changed.strip():
ok(f"no shipped plugin changes against {base} — version bump not required")
return
here, there = manifest_version(), manifest_version(base)
if here is None:
fail(f"could not read a version from {MANIFEST.relative_to(ROOT)}")
return
if not VERSION_RE.match(here):
fail(
f"the manifest version is {here!r}, which is not YYYY.MM.DD.HHMM.\n"
f" One shape for every version in the family (#3127 checklist "
f"10), zero-padded so the midnight case renders 2026.01.05.0000.\n"
f" Run `make mint-plugin`."
)
return
minted = datetime.strptime(here, "%Y.%m.%d.%H%M").replace(tzinfo=timezone.utc)
# A day of slack: the mint happens on a workstation and the lane runs
# later, so a *small* skew is ordinary. A value further out than that is
# a wrong clock or a typed year, and it makes the version lie about when
# it was minted.
if minted > datetime.now(timezone.utc) + timedelta(days=1):
fail(
f"the manifest version {here} is in the future. Either the clock "
f"that minted it is wrong, or it was typed by hand."
)
return
changed, paths = shipped_content_changed(base)
if changed is None:
fail(f"git diff against {base} failed, so the version check could not run")
return
there = manifest_version(base)
if there is None:
ok(f"no manifest on {base} — treating as a new plugin (version {here})")
return
if changed and here == there:
files = "\n ".join(paths)
if here == there:
files = "\n ".join(changed.splitlines())
fail(
f"plugin content changed but the version is still {here}.\n"
f" The installer decides whether to refresh its cache by "
f"comparing this string, so an unchanged version means these edits "
f"reach the repo and stop there — the marketplace clone updates, the "
f"cache that actually executes does not (#2209, #1040, #2220).\n"
f" Run `make mint-plugin`.\n"
f"plugin content changed but the manifest version is still {here}.\n"
f" The installer compares versions to decide whether to refresh "
f"its cache, so an unchanged version means these edits reach the repo "
f"and stop there — the marketplace clone updates, the cache that "
f"actually executes does not (issue #2209).\n"
f" Bump `version` in {MANIFEST.relative_to(ROOT)}.\n"
f" Changed:\n {files}"
)
elif changed:
ok(f"plugin content changed and the version was minted {there} -> {here}")
elif here != there:
# Not a failure — see the docstring. Named rather than silent, because
# the uninteresting cause (minted twice) and the interesting one (the
# version-relevant set is too narrow to see what actually changed)
# produce the same line, and only a person can tell them apart.
ok(
f"the version moved {there} -> {here} with no version-relevant "
f"change — harmless, unless something DID change that the set "
f"cannot see"
)
else:
ok(f"nothing version-relevant changed against {base} — no mint required")
ok(f"plugin content changed and version moved {there} -> {here}")
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--no-version", action="store_true",
help="skip the minted-version check; for `main`, where "
"it would be measured against itself")
help="skip the manifest version-bump check")
parser.add_argument("--base", default="origin/main",
help="branch the version is measured against")
help="branch the version bump is measured against")
args = parser.parse_args()
if not HOOKS_DIR.is_dir():
@@ -711,7 +478,7 @@ def main() -> int:
check_local_prior_art_needs_no_instance()
check_session_context_reports_its_version()
if not args.no_version:
check_version_is_minted(args.base)
check_version_bump(args.base)
print()
if failures:
-144
View File
@@ -1,144 +0,0 @@
#!/usr/bin/env python3
"""Mint the plugin's version — `YYYY.MM.DD.HHMM`, UTC, zero-padded.
Run this whenever you change something under `plugin/` or `.claude-plugin/`,
before you commit:
make mint-plugin # or: python3 scripts/mint_plugin_version.py
WHY A SCRIPT AND NOT A BUILD STEP. `plugin/` is not in the Docker image.
Installs fetch it straight from this git repo via `.claude-plugin/
marketplace.json`, so **a push IS the release** — there is no build between
you committing and a user fetching, and therefore no moment at which CI could
stamp a version in. Every other artifact in the family derives its version
during a build (note #3127 §2). This one has no build to derive during.
WHICH CLOCK, AND WHY IT DIFFERS FROM THE SERVER IMAGE — the divergence is
deliberate, and it lives one directory away from its opposite, so it is
exactly what a later "let's make these consistent" change would collapse:
server image name from COMMIT time, ordering key from BUILD time
(two lanes building one source must report one string;
a rebuild of an older commit must not go backwards)
plugin one value, from MINT time
§2's reason for commit time is that two lanes build one source. The plugin has
one lane and no build, so that reason does not reach it and paying its cost
buys nothing. What is given up is reproducibility-from-history: you cannot
recompute this value later, only verify that it moved when it had to.
That trade is acceptable ONLY because of what #3325 established by reading the
installer's code: the refresh test is `P.version === H`, plain string
equality, with no ordering comparison anywhere. Where a comparator ORDERS, an
unreproducible version is dangerous — nothing can check it is right. Where it
only tests equality, "did it change when it should have" is the entire
specification, and `check_plugin.py` checks that completely.
The manifest is rewritten with a surgical replacement of the `version` line
rather than `json.dump`, because its formatting and key order are not this
script's to decide and a whole-file reformat would make every mint an
unreadable diff.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from datetime import datetime, timezone
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
MANIFEST = ROOT / "plugin" / ".claude-plugin" / "plugin.json"
# Four dot-separated numeric fields, zero-padded, and nothing else — one shape
# for every human-readable version in the family (#3127 checklist 10). The
# padding is load-bearing for the midnight case the checklist names by hand:
# 2026.01.05.0000, which an unpadded `%-H%M` would render as `0` and silently
# shorten. Harmless while nothing orders these, wrong the moment anything does.
VERSION_RE = re.compile(r"^\d{4}\.\d{2}\.\d{2}\.\d{4}$")
VERSION_FORMAT = "%Y.%m.%d.%H%M"
# The `version` line, captured so its surroundings survive byte-for-byte.
VERSION_LINE_RE = re.compile(r'^(\s*"version"\s*:\s*")([^"]*)(".*)$', re.M)
def mint(now: datetime | None = None) -> str:
"""The version for this moment. UTC, always.
The conversion is not decoration: `strftime` renders whatever offset the
datetime carries, so without it two people minting the same instant in
different zones produce different strings — and the string IS the
artifact's identity. A naive datetime is read as UTC rather than as the
machine's zone, because that is this function's stated contract and
guessing the host's offset is how the bug comes back by another route.
"""
moment = now or datetime.now(timezone.utc)
if moment.tzinfo is None:
moment = moment.replace(tzinfo=timezone.utc)
return moment.astimezone(timezone.utc).strftime(VERSION_FORMAT)
def rewrite(text: str, version: str) -> str:
"""`text` with its `version` value replaced, and everything else untouched.
Raises rather than falling back to a JSON round-trip: a manifest this
cannot match is one whose shape changed, and quietly reformatting the file
to cope would be a much larger edit than the caller asked for.
"""
# Counted BEFORE substituting, not via subn's return: a capped `subn`
# reports the replacements it made, so a manifest with two `version` lines
# would look like a clean single match while the second one — the real one,
# perhaps — kept its old value.
matches = VERSION_LINE_RE.findall(text)
if len(matches) != 1:
raise ValueError(
f"expected exactly one `version` line in the manifest, found {len(matches)}"
)
return VERSION_LINE_RE.sub(
lambda m: f"{m.group(1)}{version}{m.group(3)}", text, count=1
)
def main() -> int:
parser = argparse.ArgumentParser(description="Mint the plugin's version.")
parser.add_argument(
"--check", action="store_true",
help="print the version that WOULD be minted and change nothing",
)
args = parser.parse_args()
version = mint()
if args.check:
print(version)
return 0
try:
text = MANIFEST.read_text()
except OSError as exc:
print(f"cannot read {MANIFEST.relative_to(ROOT)}: {exc}", file=sys.stderr)
return 1
try:
previous = json.loads(text).get("version")
except Exception:
previous = None
if previous == version:
# Same minute. Not an error — the value is already correct for now, and
# failing here would turn "I ran it twice" into a problem to solve.
print(f"plugin version already {version} (same minute) — unchanged")
return 0
try:
MANIFEST.write_text(rewrite(text, version))
except ValueError as exc:
print(f"{MANIFEST.relative_to(ROOT)}: {exc}", file=sys.stderr)
return 1
print(f"plugin version {previous} -> {version}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# Claude Code PreToolUse hook for Bash.
# Reads the tool input JSON from stdin; if the command is a git commit
# and fable-mcp files (other than pyproject.toml) are staged, bumps
# the fable-mcp patch version before the commit proceeds.
#
# Exits 0 always so it never blocks the commit.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
input=$(cat)
command=$(echo "$input" | python3 -c "
import sys, json
data = json.load(sys.stdin)
# Claude Code sends {tool_input: {command: ...}}
ti = data.get('tool_input', data)
print(ti.get('command', ''))
" 2>/dev/null || echo "")
# Only act on git commit commands
if ! echo "$command" | grep -qE "git commit"; then
exit 0
fi
cd "$REPO_ROOT"
# Check if fable-mcp files other than pyproject.toml are staged
fable_staged=$(git diff --cached --name-only 2>/dev/null \
| grep "^fable-mcp/" \
| grep -v "^fable-mcp/pyproject.toml$" \
|| true)
if [ -n "$fable_staged" ]; then
bash "$REPO_ROOT/scripts/bump_fable_mcp_version.sh"
fi
exit 0
+3 -50
View File
@@ -30,46 +30,6 @@ from quart import Quart
# duplicate gate, the untagged-record systems_hint) act in-band in tool
# responses, at the moment they apply.
# Grow one of those, not this block.
# BUDGET: ~1980 of the client's ~2048-char cap (#2562). Everything below is
# competing for the last ~68 characters, so an addition here is a trade, never
# an append.
#
# Milestone 317 (a note's own verify_with / expires_when, and the sweep over
# them) was DECLINED a line, deliberately, by the operator — not overlooked.
# The reasoning, so it is not re-litigated blind: this is a map, and its own
# closing line says each tool's description carries the full contract. The
# sweep is a curation act, not a session-start reflex like enter_project or
# list_always_on_rules. Spending the last of the budget on it would leave the
# map unable to grow for something more central later.
#
# The accepted cost: an agent that never opens create_note's docstring never
# learns the field exists. Guidance lives in the create_note / update_note
# docstrings and the using-scribe skill instead.
#
# Milestone 333 step 3 (2026-09-04) bought the HOW bullet's second clause —
# search(content_type="rule") before a consequential act — by TRADING OUT
# "Processes are saved procedures (follow verbatim)" and "Deletes are
# trash-recoverable". Recorded so the trade is not silently reversed:
# - Both were already in test_instruction_surfaces_agree's DISPLACED_TOPICS
# and already stated on a delivered surface, so nothing fell off: the
# process reflex is in every scribe-proc-* skill listing (each says the
# process governs and is followed verbatim), and trash recovery is in the
# delete_*/list_trash/restore docstrings, which is where per-tool guidance
# belongs by this block's own doctrine.
# - What it bought is not per-tool guidance and has nowhere else to live at
# session-start altitude. Rules were retrievable only by RESIDENCY: the
# always-on preload put them in front of the agent, and nothing told a
# session to go looking for one it had not been handed. The tier split is
# therefore load-bearing on ANY install (rule 115): a delivered rule costs
# tokens in every session forever, so a rulebook that only delivers cannot
# grow past what one session can hold, and every rule worth keeping has to
# become resident to bind at all. Retrieval is what lets it keep growing —
# and retrieval fires only if something asks, which nothing told a session
# to do. A tool-choice reflex asks least of all (#3476, #161).
# - This states the PULL for conditional rules, exactly as the surrounding
# line states it for always-on ones. Rule 119 makes these surfaces the
# specification, so the same sentence lands on all three session-start
# surfaces, and test_instruction_surfaces_agree pins it.
_INSTRUCTIONS = """
Scribe is the operator's self-hosted second brain and system of record — and
yours: recall from it before acting, record as you go. Keep no parallel copy
@@ -88,13 +48,13 @@ Hierarchy: Project -> Milestone -> Task/Note. The map, by purpose:
active project_id to stay in scope.
- WHERE work happens: Systems. Tag records with system_ids as you write;
create_system when the area is unmodelled.
- HOW: rules bind. list_always_on_rules() at start; before a consequential
act, search(content_type="rule") — the resident set is not all of them.
- HOW: rules are binding — list_always_on_rules() at session start.
- UI: the project's design system is binding — resolve_design_system /
get_design_system_stylesheet before hand-writing a value.
- REUSE: search snippets before writing a helper; record what you build with
create_snippet; classify shapes against canon (classify_shapes) — a
consumer map is rows, never prose.
consumer map is rows, never prose. Processes are saved procedures (follow
verbatim). Deletes are trash-recoverable.
A task is a note with status (*_note vs *_task tools).
Creates are duplicate-gated: a near-match BLOCKS and returns the existing
@@ -160,13 +120,6 @@ _READ_ONLY_TOOLS = frozenset({
# prefix, so the completeness test below cannot derive it — the same
# reason `enter_project` is spelled out above.
"retrieval_telemetry",
# The note staleness sweep (milestone 317). A pure read — mark_note_verified
# is the write, and it is deliberately NOT here. Spelled out for
# retrieval_telemetry's reason: `notes_due_for_verification` matches none of
# the prefixes the completeness test derives from, so nothing would have
# prompted this decision. `rules_due_for_verification` is in the same
# position and is NOT listed — see #3191.
"notes_due_for_verification",
})
# Read-SHAPED tools that must NOT be reachable with a read key — a getter that
+3 -15
View File
@@ -57,7 +57,7 @@ async def get_milestone(milestone_id: int) -> dict:
return {
"milestone": out,
"steps": [t.to_dict() for t in steps],
**rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_milestone"),
**rulebooks_svc.rules_payload(applicable),
}
@@ -137,23 +137,11 @@ async def delete_milestone(milestone_id: int) -> dict:
"""Move a milestone to the trash (recoverable). Its tasks go with it as one batch.
Restore via restore(batch_id)."""
uid = current_user_id()
# Read the title BEFORE the delete: afterwards the row is trashed and the
# confirmation could only echo the number back. A deletion the operator
# cannot recognise is one they cannot tell was the wrong one.
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
# that errors must not stop the delete happening. Same posture the
# staleness marker takes — a decoration may never break its payload.
try:
doomed = await milestones_svc.get_milestone(uid, milestone_id)
title = getattr(doomed, "title", "") if doomed else ""
except Exception:
title = ""
batch = await trash_svc.delete(uid, "milestone", milestone_id)
if batch is None:
raise ValueError(f"milestone {milestone_id} not found")
return {"deleted": milestone_id, "title": title, "deleted_batch_id": batch,
"message": f'Milestone {milestone_id} ("{title}") and its tasks '
f"moved to trash. Restore with restore('{batch}')."}
return {"deleted_batch_id": batch,
"message": f"Milestone {milestone_id} + its tasks moved to trash. Restore with restore('{batch}')."}
def register(mcp) -> None:
+3 -168
View File
@@ -103,8 +103,6 @@ async def create_note(
project_id: int = 0,
system_ids: list[int] | None = None,
supersedes: list[int] | None = None,
verify_with: str = "",
expires_when: str = "",
force: bool = False,
) -> dict:
"""Create a new note in Scribe.
@@ -136,33 +134,6 @@ async def create_note(
arrives labelled when it does surface. This records a CLAIM, not a
verdict: it never says the older note was wrong, only that it is no
longer the current answer.
verify_with: HOW TO CHECK this note is still true. Leave empty for
almost every note — that is the normal case, not an unfinished
one.
A note is a NORM or a CONSTRAINT. A norm is a decision ("we derive
versions from commit time"): it has no truth value and changes only
when its author changes it, which they know they did. A CONSTRAINT
asserts a fact about someone else's software ("AMO refuses to
re-sign a version", "this forge numbers CI runs per repository"),
and it goes false with nobody watching. Only constraints get a
check.
The test, in one question: COULD THIS NOTE BECOME FALSE WITHOUT
ANYONE EDITING IT? If no, leave this empty. Sharper still: is the
thing this note describes YOURS TO CHANGE? If yes it is a
decision. Measured against a real corpus, every note that earned a
check was about somebody else's software.
Three that look like candidates and are not: a resume pointer or
"current state" note (goes stale fastest, but the cure is to
update it, not to check it); a measurement of your own system (it
goes false because you changed something, and you knew); and a
decision that RESTS on someone else's behaviour (check the note
asserting the fact, not the decision).
A command, a path, a URL, a query. Prose is allowed; something
runnable is better.
expires_when: The STATE that ends it — deliberately not a date.
"When Forgejo issues run numbers per workflow rather than per
repository", not "in six months". Constraints expire when the
ground moves, not on a schedule.
force: Bypass the near-duplicate gate. By default, if a title- or
meaning-similar note already exists in the same project, creation is
BLOCKED and the existing note's id is returned so you update it
@@ -190,8 +161,6 @@ async def create_note(
body=body,
tags=tags,
project_id=project_id or None,
verify_with=verify_with,
expires_when=expires_when,
)
if system_ids:
await systems_svc.set_record_systems(uid, note.id, system_ids)
@@ -216,9 +185,6 @@ async def update_note(
project_id: int = 0,
system_ids: list[int] | None = None,
supersedes: list[int] | None = None,
verify_with: str = "",
expires_when: str = "",
clear: list[str] | None = None,
) -> dict:
"""Update an existing Scribe note. Only explicitly provided fields are changed.
@@ -233,27 +199,6 @@ async def update_note(
supersedes: Replace the ids of earlier notes this one replaces
(set-semantics). None = leave unchanged; [] = clear all. See
create_note for when to reach for it.
verify_with: How to check this note is still true. Almost every note
should leave this empty — that is the normal, finished state, not
a gap: an empty check is the marker for "this is a decision, there
is nothing to go and check". See create_note for the full
norm-vs-constraint test; the short form is "could this become
false without anyone editing it?".
expires_when: The STATE that ends it, not a date.
clear: Names of fields to UNSET — "verify_with", "expires_when".
Needed because "" means "leave this alone" here, so there is no
value that removes a field: an agent updating a body must not
silently wipe a check it was not asked about. A note that stops
being a constraint is cleared by naming the field, which cannot
happen by accident.
Rewriting `verify_with` drops the note's verification stamp: a stamp
certifies a particular check, and carrying it across a rewrite would vouch
for something nobody has looked at.
A task and a snippet are both REFUSED a check, with a message saying where
to go instead — a task's decay is its status, and a snippet has
verify_snippet.
"""
uid = current_user_id()
fields: dict = {}
@@ -265,13 +210,7 @@ async def update_note(
fields["tags"] = tags
if project_id:
fields["project_id"] = project_id
if verify_with:
fields["verify_with"] = verify_with
if expires_when:
fields["expires_when"] = expires_when
note = await notes_svc.update_note(
uid, note_id, clear=clear or (), **fields
)
note = await notes_svc.update_note(uid, note_id, **fields)
if note is None:
raise ValueError(f"note {note_id} not found")
if system_ids is not None:
@@ -324,113 +263,11 @@ async def find_duplicate_records(kind: str = "note", threshold: float = 0.0) ->
async def delete_note(note_id: int) -> dict:
"""Move a Scribe note to the trash (recoverable). Restore via restore(batch_id)."""
uid = current_user_id()
# Read the title BEFORE the delete: afterwards the row is trashed and the
# confirmation could only echo the number back. A deletion the operator
# cannot recognise is one they cannot tell was the wrong one.
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
# that errors must not stop the delete happening. Same posture the
# staleness marker takes — a decoration may never break its payload.
try:
loaded = await notes_svc.get_note_for_user(uid, note_id)
title = getattr(loaded[0], "title", "") if loaded else ""
except Exception:
title = ""
batch = await trash_svc.delete(uid, "note", note_id)
if batch is None:
raise ValueError(f"note {note_id} not found")
return {"deleted": note_id, "title": title, "deleted_batch_id": batch,
"message": f'Note {note_id} ("{title}") moved to trash. '
f"Restore with restore('{batch}')."}
async def notes_due_for_verification(
older_than_days: int = 0, project_id: int = 0, never_only: bool = False,
) -> dict:
"""Which notes assert a FACT that nobody has confirmed lately.
A corpus of notes holds two kinds of thing. Most are DECISIONS or records
of what happened — they have no truth value and cannot rot. A few assert a
fact about someone else's software: what a signing service does on a
duplicate upload, how a forge numbers its CI runs, what an updater
compares. Those go false silently, with nobody present, and a
cross-project reference note keeps being read as current by every project
that cites it.
This lists the second kind, oldest verification first, NEVER-CHECKED AT
THE TOP — a note nobody has ever confirmed is a claim with no evidence
behind it at all. Each row carries `verify_with` in full, because you are
about to go and run it, plus `expires_when` and `days_since_verified`.
Reach for it when curating, when a note's claim just contradicted what you
observed, or periodically. Then, per row: run the check, and call
mark_note_verified with what you found.
Notes with no `verify_with` never appear, and that is correct — they are
decisions, and there is nothing to go and check. Do not "fix" their
absence by giving them checks: this list is only worth reading while
everything on it genuinely can go false.
Args:
older_than_days: only notes last verified longer ago than this.
Never-checked notes always qualify — they are the most overdue
thing there is. 0 = no age filter.
project_id: narrow to one project. 0 = every project. Unlike the rules
sweep, this filter is safe: a note belongs to at most one project
outright, with none of the subscription and always-on paths that
would make a project filter UNDER-report a rule.
never_only: only notes nobody has ever verified.
"""
uid = current_user_id()
notes = await notes_svc.notes_due_for_verification(
uid,
older_than_days=older_than_days,
project_id=project_id or None,
never_only=never_only,
)
return {
"notes": [notes_svc.verification_row(n) for n in notes],
"total": len(notes),
}
async def mark_note_verified(note_id: int, still_true: bool = True) -> dict:
"""Record that you ran a note's check — and what it said.
Call this AFTER actually running the note's `verify_with`, never on the
strength of the claim sounding plausible. A stamp nobody earned is worse
than no stamp: it moves the note to the bottom of the sweep and buys the
claim another long silence.
`still_true=False` writes NOTHING. A note whose check failed is not in a
special state to be recorded — it is WRONG, and the honest next moves are
to correct it, supersede it, or find out why. So it stays at the top of
the sweep until someone deals with it, and the response tells you what the
note said would end it.
Args:
note_id: the note whose check you ran.
still_true: True if the check passed. False if the fact it asserts is
no longer true — say so, that is the outcome worth having.
"""
uid = current_user_id()
note = await notes_svc.mark_note_verified(note_id, uid, still_true)
if note is None:
raise ValueError(
f"note {note_id} not found, not writable by you, or carries no "
f"verify_with (nothing to verify is not the same as verified)"
)
data = notes_svc.verification_row(note)
data["verified"] = bool(still_true)
if not still_true:
data["next"] = (
"This note is no longer true and is still being read as current "
"by anything that cites it. Correct it with update_note, write "
"the replacement with create_note(supersedes=[...]), or clear its "
"check with update_note(clear=[\"verify_with\"]) if it has stopped "
"asserting a fact at all. It stays at the top of "
"notes_due_for_verification until one of those happens."
)
return data
return {"deleted_batch_id": batch,
"message": f"Note {note_id} moved to trash. Restore with restore('{batch}')."}
def register(mcp) -> None:
@@ -441,7 +278,5 @@ def register(mcp) -> None:
update_note,
find_duplicate_records,
delete_note,
notes_due_for_verification,
mark_note_verified,
):
mcp.tool(name=fn.__name__)(fn)
+2 -2
View File
@@ -207,7 +207,7 @@ async def enter_project(project_id: int) -> dict:
],
"design_system": design_system,
"milestone_summary": milestone_summary,
**rulebooks_svc.rules_payload(applicable, user_id=uid, source="enter_project"),
**rulebooks_svc.rules_payload(applicable),
"open_tasks": [
{
"id": t.id, "title": t.title, "status": t.status,
@@ -251,7 +251,7 @@ async def get_project(project_id: int) -> dict:
applicable = await rulebooks_svc.get_applicable_rules(
project_id=project_id, user_id=uid,
)
data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_project"))
data.update(rulebooks_svc.rules_payload(applicable))
return data
+7 -167
View File
@@ -18,7 +18,6 @@ from scribe.mcp._context import current_user_id
from scribe.services import dedup as dedup_svc
from scribe.services import rulebooks as rulebooks_svc
from scribe.services import trash as trash_svc
from scribe.services.rule_usage import record_rule_pulled, record_rule_surfaced
# ── Rulebook CRUD ───────────────────────────────────────────────────────
@@ -123,9 +122,8 @@ async def delete_rulebook(rulebook_id: int, confirmed: bool = False) -> dict:
"confirmed_required": True,
}
batch = await trash_svc.delete(uid, "rulebook", rulebook_id)
return {"deleted": rulebook_id, "title": rb.title, "deleted_batch_id": batch,
"message": f'Rulebook {rulebook_id} ("{rb.title}") moved to trash. '
f"Restore with restore('{batch}')."}
return {"deleted": rulebook_id, "deleted_batch_id": batch,
"message": f"Moved to trash. Restore with restore('{batch}')."}
# ── Topic CRUD ─────────────────────────────────────────────────────────
@@ -194,9 +192,8 @@ async def delete_topic(topic_id: int, confirmed: bool = False) -> dict:
"confirmed_required": True,
}
batch = await trash_svc.delete(uid, "topic", topic_id)
return {"deleted": topic_id, "title": topic.title, "deleted_batch_id": batch,
"message": f'Topic {topic_id} ("{topic.title}") moved to trash. '
f"Restore with restore('{batch}')."}
return {"deleted": topic_id, "deleted_batch_id": batch,
"message": f"Moved to trash. Restore with restore('{batch}')."}
# ── Rule CRUD ──────────────────────────────────────────────────────────
@@ -265,25 +262,7 @@ async def list_always_on_rules(project_id: int = 0) -> dict:
"""
uid = current_user_id()
rules = await rulebooks_svc.list_always_on_rules(uid, project_id=project_id)
# AMBIENT source: the resident set, handed over whole. No ranker chose
# these, so they must not land in the pull-through numerator's denominator
# — but they must land SOMEWHERE, or the largest rule surface in the
# product stays the one surface its own scoreboard cannot see (#3473).
record_rule_surfaced(
user_id=uid,
rule_ids=[r.id for r in rules],
source="list_always_on_rules",
)
return {
"rules": [_rule_summary(r) for r in rules],
"total": len(rules),
# A marker for the set you are now holding. It is not for you to read:
# the write-path hook carries it back and is told if these rules have
# moved since. Deliberately NOT on rules_payload's applicable_rules —
# that is a DIFFERENT set (subscription-derived), and one key name
# over two sets is how a comparison starts reporting phantom changes.
"rules_etag": rulebooks_svc.rules_etag(rules),
}
return {"rules": [_rule_summary(r) for r in rules], "total": len(rules)}
async def get_rule(rule_id: int) -> dict:
@@ -298,11 +277,6 @@ async def get_rule(rule_id: int) -> dict:
rule = await rulebooks_svc.get_rule(rule_id, uid)
if rule is None:
raise ValueError(f"rule {rule_id} not found")
# THE pull that matters. The write-path rule arm's own message ends "Read
# it with get_rule(N)", so this is the exact action the hint asks for and
# the only evidence that one landed. Recorded after the access check, so a
# refused read is not counted as a pull.
record_rule_pulled(user_id=uid, rule_id=int(rule.id), source="mcp_get_rule")
return await rulebooks_svc.rule_detail(uid, rule)
@@ -315,61 +289,6 @@ async def create_rule(
) -> dict:
"""Create a new rule in a rulebook (a SHARED rule — keep it general).
PROPOSE RULES READILY, AND WRITE ONE WHEN THE OPERATOR SAYS YES. Noticing
that something has hardened into a standing instruction is valuable work,
and a session that notices it and says nothing has thrown the observation
away. So raise it whenever you see one. The single step that belongs
between noticing and writing is the operator's yes: a rule binds every
future session, and they are the person it binds.
Their yes is also the only moment the rule is reliably IN FRONT of them.
After the write it may not be again for months — a conditional rule is not
read aloud at session start, and a project-scoped one does not appear in
an unfiltered list_rules() at all. So the proposal is the review.
When the operator asks for a rule in so many words, that IS the yes —
write it and move on. The loop below is for the rule you thought of.
A PROPOSAL CARRIES FOUR THINGS, and the fourth is the one that decides it:
1. WHAT it would require — the statement, in the words it would carry,
not a gloss of them. The operator is agreeing to text.
2. INTENT — what it changes about how work gets done, and what goes
wrong today without it. "Be careful about X" is not an intent; the
behaviour that would differ tomorrow is.
3. WHY NOW — the incident, observation or decision behind it. Pass that
record as arose_from_id, and say it in the conversation too: the
field is for the reader six months out, the sentence is for the
person deciding.
4. HOW IT WOULD BE ENFORCED — a test, a CI check, a hook, a schema
constraint, a duplicate gate, a review step... or nothing, in which
case say so plainly: "nothing — this is prose a session has to
remember." Answer this one honestly and it will sometimes dissolve
the rule, which is the point rather than a side effect. What a test
can assert should BE that test; a rule is what remains when nothing
mechanical can hold the thing. A rulebook grows by default and
shrinks only on purpose, so a question that prevents a rule is worth
more than any question that improves one's wording.
THEN CLOSE WITH A QUESTION THEY CAN ANSWER IN ONE WORD. Offer three
answers, and make the middle one the easy one:
* "Approve it AS WRITTEN" — you create it with the statement exactly as
shown. This is what makes element 1 load-bearing: they approved TEXT,
so that text is what gets stored, verbatim.
* "LET'S TALK ABOUT IT" — the wording, the scope, the tier, whether it
wants to be a rule at all. Most good rules arrive this way, so treat
this answer as the expected one rather than a setback.
* "NO" — let it go. If the observation is still worth keeping, it is a
note (create_note): recorded, findable, and binding on nobody.
Where the interface offers structured choices, ask it that way — a
question with named options is answered in a click, while the same
question inside a paragraph is answered by scrolling past. Where it does
not, write the three options out as three options. Either way ask once
and let the answer stand; re-raising a declined proposal argues a rule
into existence, which is the thing this whole loop exists to prevent.
A rulebook rule is shared by every project that gets the rulebook: an
always_on rulebook binds ALL your projects; a subscribed rulebook binds the
projects that opt in. So a rulebook rule must read as a general standard —
@@ -488,16 +407,6 @@ async def create_project_rule(
the rule is returned in get_project's applicable_rules (under
project_rules) and in list_rules(project_id=...).
PROPOSE, THEN WRITE ON A YES — create_rule's opening carries the whole
loop: the four things a proposal states (what it would require, its
intent, why now, and how it would be enforced) and the one-word question
that closes it (approve as written / talk about it / no). All of it
applies here unchanged. Reach for that loop MORE readily on this surface,
not less: a project rule stays out of an unfiltered list_rules(), and a
conditional one stays out of session start too, so the operator's yes is
the one moment this rule is certain to have been seen by the person it
binds.
Check first whether a rule is the right shape at all — create_rule's
opening asks that question and it applies identically here. A visual
standard is a design system; a procedure is a process (create_process);
@@ -639,73 +548,6 @@ async def update_rule(
return await rulebooks_svc.rule_detail(uid, rule, system_ids)
async def rule_history(rule_id: int, version_id: int = 0) -> dict:
"""What a rule USED TO SAY, newest change first.
Read this before you argue with a rule, and before you rewrite one. A
rule that has been reworded may have been reworded for a reason you are
about to rediscover the hard way — and the wording it replaced is often
the fastest way to see what the current one is guarding against. The
rescoping of rule 79 is the case this exists for: the superseded
statement had to be hand-copied into a task log to survive the edit.
EACH ENTRY HOLDS THE TEXT THE EDIT REPLACED, not the text it introduced.
So "what did this say before the most recent change?" is the first entry,
and the text the change PRODUCED is the rule as it stands now — read that
with get_rule. Pair the two and you have the diff.
An empty history is ordinary and means the rule has never been reworded,
not that its history was lost. Nothing is written before milestone 323,
so a rule edited before then starts empty too.
Args:
rule_id: The rule whose history to read.
version_id: 0 (default) lists the history — when each change
happened, by whom, and the title as it then stood. Pass an id
from that list to read that snapshot IN FULL. The list omits
statement and why on purpose: a rule's statement runs to
thousands of characters, and a history carrying every field would
cost more to read than the answer is worth.
There is deliberately no restore. Putting an old wording back is a
decision, so it goes through update_rule — which snapshots what it
replaces, leaving the undo visible in the history like any other edit. A
one-click revert would erase the only record of why the rewrite happened.
"""
uid = current_user_id()
if version_id:
version = await rulebooks_svc.get_rule_version(rule_id, version_id, uid)
if version is None:
raise ValueError(
f"version {version_id} not found on rule {rule_id}"
)
return version.to_dict(include_text=True)
versions = await rulebooks_svc.list_rule_versions(rule_id, uid)
if versions is None:
raise ValueError(f"rule {rule_id} not found")
# Fail-open, like the deletes: a missing title must not turn a readable
# history into an error.
try:
rule = await rulebooks_svc.get_rule(rule_id, uid)
except Exception:
rule = None
return {
"rule_id": rule_id,
"title": rule.title if rule else "",
"versions": [v.to_dict(include_text=False) for v in versions],
"total": len(versions),
# Said in-band because an empty list is the ordinary case and reads
# like a missing feature otherwise.
"note": (
"Each entry holds the text the edit REPLACED. The current wording "
"is on the rule itself — get_rule(%d)." % rule_id
if versions else
"This rule has never been reworded."
),
}
async def delete_rule(rule_id: int, confirmed: bool = False) -> dict:
"""Move a rule to the trash (recoverable). Requires confirmed=True."""
uid = current_user_id()
@@ -721,9 +563,8 @@ async def delete_rule(rule_id: int, confirmed: bool = False) -> dict:
"confirmed_required": True,
}
batch = await trash_svc.delete(uid, "rule", rule_id)
return {"deleted": rule_id, "title": rule.title, "deleted_batch_id": batch,
"message": f'Rule {rule_id} ("{rule.title}") moved to trash. '
f"Restore with restore('{batch}')."}
return {"deleted": rule_id, "deleted_batch_id": batch,
"message": f"Moved to trash. Restore with restore('{batch}')."}
# ── Subscriptions ──────────────────────────────────────────────────────
@@ -991,6 +832,5 @@ def register(mcp) -> None:
suppress_topic_for_project, unsuppress_topic_for_project,
exclude_always_on_rulebook, include_always_on_rulebook,
rules_due_for_verification, mark_rule_verified,
rule_history,
):
mcp.tool(name=fn.__name__)(fn)
+9 -166
View File
@@ -112,7 +112,6 @@ async def search(
return await _search_rules(uid, q, limit)
is_task = {"note": False, "task": True}.get(content_type) # None => any
t0 = time.perf_counter()
report: dict = {}
raw = await semantic_search_notes(
uid, q, limit=limit, is_task=is_task,
project_id=project_id or None,
@@ -120,16 +119,12 @@ async def search(
# An explicit search reaches everything the operator may read, including
# records shared with them one-to-one.
scope="read",
report=report,
)
record_retrieval(
user_id=uid, source="mcp_search", query=q,
threshold=DEFAULT_SIMILARITY_THRESHOLD, limit=limit,
project_id=project_id or None, is_task=is_task, results=raw,
duration_ms=(time.perf_counter() - t0) * 1000.0,
best_available=report.get("best_available_score"),
best_available_id=report.get("best_available_id"),
searched=bool(report.get("searched", True)),
)
owners = await owner_names_for(
{int(note.user_id) for _s, note in raw if note.user_id != uid}
@@ -154,9 +149,7 @@ async def search(
}
async def retrieval_telemetry(
days: int = 30, near_miss_samples: int = 0,
) -> dict:
async def retrieval_telemetry(days: int = 30) -> dict:
"""What the retrieval telemetry says about YOUR surfaces, over a window.
The read half of the loop the ranker's thresholds are meant to be tuned
@@ -165,82 +158,17 @@ async def retrieval_telemetry(
hand-probing the live instance, which is how the last such decision had to
be made.
Three readouts, from the three tables built for them:
Two readouts, from the two tables built for them:
`sources` — per retrieval surface (`auto_inject`, `write_path`,
`mcp_search`, …), from `retrieval_logs`: `calls`, `zero_result_calls`,
`near_misses`, the `top_score` spread (p10/p50/p90/min/max),
`avg_result_count` and `p90_duration_ms`.
`cleared_threshold` (how often the best hit beat the threshold in force for
that call), the `top_score` spread (p10/p50/p90/min/max), `avg_result_count`
and `p90_duration_ms`. THE number to read first is `cleared_threshold`
against `calls`, with the spread beside it: a surface that clears its bar
on nearly every call is either well-tuned or too loose, and p10 says which.
THE NUMBER TO READ FIRST IS `near_misses.p90`, AGAINST THE THRESHOLD IN
FORCE FOR THAT SURFACE. It is measured on the calls the BAR turned away —
zero-result calls, minus the ones whose zero was a repeat the reader had
already been shown — using the best score the ranker reached before the bar
rejected it. So it is the one figure here that says something the bar
cannot make true by construction, and `max` is always below the threshold:
an above-bar candidate nobody excluded would have been returned. A bar at 0.72 turning away a stream of 0.71s is set too
high by a hair and the surface is losing hits it should have had. The same
bar turning away 0.30s is working, and the corpus simply had nothing. Both
render as a zero-result call, and nothing else in this readout tells them
apart.
`near_miss_samples` (0-20, default 0) TURNS THE PERCENTILES INTO RECORDS
YOU CAN READ. Each source then carries `near_miss_records`: its highest
scoring declines, each with the `record_id` the bar refused and the `query`
that asked. Reach for it whenever you are about to move a threshold.
THE PERCENTILE CANNOT SETTLE A BAR ON ITS OWN, and this is the whole reason
the parameter exists. `near_misses.p90` says mass is sitting just under the
line; it says nothing about whether that mass is RELEVANT, and those are
different questions. Lowering a bar to where the mass is, without reading
what is there, is choosing a firing rate rather than a quality. Pull-through
cannot referee it either — the injected rule line already carries title and
trigger, so a session can comply without ever calling `get_rule`, which
makes rule pull-through understate usefulness by construction. Reading the
rejected records is the method that actually answers it.
Off by default because it is a LISTING, not a statistic: it is for the
moment you are making a decision, not for every readout.
`near_misses` is `null` when no declining call in the window measured it —
rows written before #3670 shipped cannot know. That is "not measured", not
"nothing came close"; a 0.0 there would be a claim about the corpus
invented out of a caller's silence.
A NULL HERE NOW MEANS ONE THING, which it did not at first. A semantic
search returns nothing three ways WITHOUT having run — an empty query, an
unavailable embedder, and a failed database query — and each used to write
a row indistinguishable from a ranker that declined (#3765). Those calls no
longer write a row at all, on the same reasoning that already keeps a blank
command out of the log: a row there reports a call that never happened and
drags the clear rate down with phantom declines. So a null is "searched,
and nothing came close", and a broken search shows up as a WARNING in the
application log rather than as a quiet zero in here.
THERE IS NO `cleared_threshold` ANY MORE, and if you remember one, that
memory is of a tautology (#3670). The search applies the bar before
returning, so every returned result cleared it by construction and a call
with no results has no score to compare: the field was true exactly when
`result_count > 0`, i.e. it was `calls - zero_result_calls` under a name
that promised a second opinion. `zero_result_calls + cleared_threshold ==
calls` held on all nineteen readings ever taken. The reading procedure
built on it — "clears its bar on nearly every call" — asked you to compare
a number with itself.
CHECK `suppression` BEFORE CONCLUDING ANYTHING FROM `zero_result_calls`. A
zero-result call is two different events wearing one number: the ranker
found nothing above the bar, or it found only what this session had already
been shown. Just the first is evidence about the bar. `suppression` splits
them where the surface can tell — `zero_because_already_shown` comes off
`zero_result_calls` to leave the true ranker declines.
`suppression` is `null` when NO row in the window reported it, and that is
"not measured here", NOT "none suppressed". Surfaces that pass their
exclusions into the search never see what was dropped, so they cannot say.
Do not read a null as a zero: reading an artifact as a measurement is how
this surface got mis-scoped once already (#3311, #3497).
`usage` — NOTES ONLY, from `note_usage_events`, at the per-note grain
`usage` — from `note_usage_events`, at the per-note grain
`retrieval_logs` cannot be indexed at: `surfaced` (ranked surfacings — a
scored surface CHOSE the record), `ambient` (the rest), `pulled` split into
`pulled_by_agent` / `pulled_by_human`, the distinct-note counts, and
@@ -254,84 +182,6 @@ async def retrieval_telemetry(
tuned against — only by a pull the agent made. Aggregating across the
mcp_/rest_ prefix would silently answer the wrong one.
`usage["by_source"]` — THE number to tune a threshold against, because the
top-level `pull_through` is a corpus average and averages the surfaces
together. Per surface: `notes_surfaced`, `notes_pulled`, `pull_through`,
and `ambient: true` on surfaces whose surfacings were not scored choices
(their ratio is null — "surfaced often, opened never" is not a judgment
about a record nothing chose). Read it as: of the distinct notes THIS
surface put in front of the agent, how many did the agent then open?
It is an UPPER BOUND per surface: a pull records the door it came
through, not the surface that led there, so a note surfaced by two surfaces
and opened once counts for both — attribution would need the session
identity #2085 declined to invent. `by_source_failed: true` means that one
query failed while the rest of the readout stood.
`rule_usage` — the same question for RULES, from `rule_usage_events`:
`surfaced` and `ambient`, `pulled` split into `pulled_by_agent` /
`pulled_by_human`, the distinct-rule counts, and `pull_through` on the same
definition (agent pulls over RANKED surfacings).
A SEPARATE BLOCK, not folded into `usage`, and reading it as one number
with that is the mistake to avoid. The corpora differ by orders of
magnitude — a few dozen eligible rules against thousands of notes — so a
blended ratio would be the note ratio with noise on it and would hide the
rule arm entirely.
`surfaced` VS `ambient` IS THE READING THAT MATTERS HERE. `surfaced` counts
rules a ranker chose — today only the write-path arm — and those are claims
a pull can settle. `ambient` counts BULK DELIVERIES: the SessionStart
preload, `list_always_on_rules`, and the `rules_payload` surfaces
(`enter_project`, `get_project`, `get_milestone`, `start_planning`,
`get_task`), which hand over the whole applicable set at once with nobody
choosing anything. A large `ambient` says the resident set is big and
arrives often — never that it is useful, and never that it is read.
`pull_through` therefore divides by `surfaced` alone. Fold the preload in
and growing the always-on set would depress the arm's measured precision
while trimming it would flatter it, for reasons having nothing to do with
the arm. To judge the PRELOAD instead, compare `ambient` against pulls of
those same rules over time: a resident set surfaced thousands of times and
opened never is the dead-weight signal, one tier up.
Read it against `sources["write_path_rule"]`. That arm was once believed
never to decline — the reading that scoped #3311 — but it was the arm's
`retrieval_logs` row being written only on calls that FOUND something, so
the zeros were missing rather than absent (#3497). Measured since, it
declines the large majority of its calls like any other surface.
EVERY COUNTER BLOCK CARRIES ITS OWN COVERAGE — `complete_from` and
`covers_window`. `complete_from` is when the number became trustworthy:
for one source, its first recorded row; for a section that sums several,
the LATEST of theirs, because a total is complete only once every
contributor was being written. `covers_window: false` means the window
reaches back further than the recording does, so the count is a fraction
of the period it appears to describe.
READ IT BEFORE COMPARING TWO NUMBERS, and especially before comparing
across a deploy. A counter added last week, read over a 30-day window,
reports a real count against an imagined denominator — and the result is
a plausible fraction rather than an obvious zero, which is what makes it
dangerous. That reading cost milestone #379 five steps aimed at a defect
that did not exist.
`covers_window` is null, never false, when nothing was ever recorded:
"no measurement" is not "partial measurement", the same distinction
`suppression`'s null carries a few paragraphs up.
A SOURCE SHOWING `calls: 0` WAS RECORDING AND MADE NO CALLS. `sources`
lists every source the table has ever held, not only those active in the
window, so a surface that stopped firing stays visible rather than
disappearing — being absent is reserved for a source that has never
recorded at all. Its score fields are null, not zero: the calls are a
real observation, the distribution is not one.
`rule_usage_failed: true` means that read failed while the rest of the
readout stood. The counts are still present so a caller can render, but
they are zeros meaning "could not find out", not "nothing happened" — do
not report a pull-through from a block carrying that flag.
Scoped to your own telemetry — a retrieval log records what your agent
asked for, query text included, and is not a shared record kind.
@@ -341,15 +191,8 @@ It is an UPPER BOUND per surface: a pull records the door it came
Args:
days: window size, default 30. Clamped to at least 1.
near_miss_samples: 0-20, default 0. How many of each source's highest
scoring DECLINES to list by record, with the query that asked.
Pass it when you are about to move a threshold; leave it off
otherwise. See the near-miss section above for why a percentile
alone cannot settle a bar.
"""
return await retrieval_summary(
current_user_id(), days=days, near_miss_samples=near_miss_samples,
)
return await retrieval_summary(current_user_id(), days=days)
def register(mcp) -> None:
+1 -11
View File
@@ -494,19 +494,9 @@ async def delete_snippet(snippet_id: int) -> dict:
that should survive, prefer merge_snippets — that keeps the call sites.
"""
uid = current_user_id()
# Read before deleting so the confirmation can NAME what went — an id
# alone leaves the operator unable to tell which snippet this was.
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
# that errors must not stop the delete happening. Same posture the
# staleness marker takes — a decoration may never break its payload.
try:
doomed = await snippets_svc.get_snippet(uid, snippet_id)
title = getattr(doomed, "title", "") if doomed else ""
except Exception:
title = ""
if not await snippets_svc.delete_snippet(uid, snippet_id):
raise ValueError(f"snippet {snippet_id} not found")
return {"deleted": True, "id": snippet_id, "title": title}
return {"deleted": True, "id": snippet_id}
async def merge_snippets(target_id: int, source_ids: list[int]) -> dict:
+3 -15
View File
@@ -103,7 +103,7 @@ async def get_task(task_id: int) -> dict:
applicable = await rulebooks_svc.get_applicable_rules(
project_id=note.project_id, user_id=uid,
)
data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_task"))
data.update(rulebooks_svc.rules_payload(applicable))
data.update(await access_svc.describe_provenance(uid, note))
# Same reasoning as get_note's record_pulled, and this is the tool where it
# matters MOST: auto-inject ranks kind-blind over a corpus that is
@@ -364,23 +364,11 @@ async def delete_task(task_id: int) -> dict:
"""Move a Scribe task (or plan) to the trash (recoverable). Sub-tasks go with it.
Restore via restore(batch_id)."""
uid = current_user_id()
# Read the title BEFORE the delete: afterwards the row is trashed and the
# confirmation could only echo the number back. A deletion the operator
# cannot recognise is one they cannot tell was the wrong one.
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
# that errors must not stop the delete happening. Same posture the
# staleness marker takes — a decoration may never break its payload.
try:
loaded = await notes_svc.get_note_for_user(uid, task_id)
title = getattr(loaded[0], "title", "") if loaded else ""
except Exception:
title = ""
batch = await trash_svc.delete(uid, "task", task_id)
if batch is None:
raise ValueError(f"task {task_id} not found")
return {"deleted": task_id, "title": title, "deleted_batch_id": batch,
"message": f'Task {task_id} ("{title}") moved to trash. '
f"Restore with restore('{batch}')."}
return {"deleted_batch_id": batch,
"message": f"Task {task_id} moved to trash. Restore with restore('{batch}')."}
def register(mcp) -> None:
-2
View File
@@ -28,13 +28,11 @@ from scribe.models.invitation import InvitationToken # noqa: E402, F401
from scribe.models.embedding import NoteEmbedding, RuleEmbedding # noqa: E402, F401
from scribe.models.retrieval_log import RetrievalLog # noqa: E402, F401
from scribe.models.note_usage import NoteUsageEvent # noqa: E402, F401
from scribe.models.rule_usage import RuleUsageEvent # noqa: E402, F401
from scribe.models.project import Project # noqa: E402, F401
from scribe.models.milestone import Milestone # noqa: E402, F401
from scribe.models.task_log import TaskLog # noqa: E402, F401
from scribe.models.note_draft import NoteDraft # noqa: E402, F401
from scribe.models.note_version import NoteVersion # noqa: E402, F401
from scribe.models.rule_version import RuleVersion # noqa: E402, F401
from scribe.models.note_supersession import NoteSupersession # noqa: E402, F401
from scribe.models.group import Group, GroupMembership # noqa: E402, F401
from scribe.models.share import NoteShare, ProjectShare # noqa: E402, F401
+1 -5
View File
@@ -51,11 +51,7 @@ class RuleEmbedding(Base):
"""One embedding vector per CHUNK of a rule (milestone 307, note 3026).
A SIBLING of NoteEmbedding rather than a generalisation of it, decided
deliberately. The reasoning below turned out to be the only written
statement of a rule Scribe applies to every record type, so it is now also
NOTE 3163 — "When a record type earns its own table" — with the worked
cases and the cost of leaving `notes`. Read that before splitting a record
type off; this stays here because it is where the decision was made.
deliberately:
- The embedding ROW could have been made polymorphic. The SEARCH could not.
`semantic_search_notes` is a long function of Note-specific scoping —
+2 -41
View File
@@ -94,40 +94,9 @@ class Note(Base, TimestampMixin, SoftDeleteMixin):
# name/language/signature/locations live here so they can be INDEXED. The
# body keeps the same facts in readable markdown and remains what gets
# embedded; this is a mirror for querying, not the source of truth for
# display — and it is DERIVED, so every path that writes a snippet's body
# rewrites it too (services/snippets.recompose_data, called from
# notes.update_note). 0070 left it NULL on existing rows and
# snippets.backfill_snippet_data filled them at startup; readers still fall
# back to parsing the body when it is absent (snippet_fields).
# display. NULL on every row written before migration 0070, so readers fall
# back to parsing the body (see services/snippets.snippet_fields).
data: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
# The three fields that tell a CONSTRAINT apart from a NORM (milestone
# 317, migration 0092) — the same trio `rules` carries, and for the same
# reason. A norm is a decision: no truth value, changes only when its
# author changes it. A constraint asserts a fact about someone else's
# software and goes false with nobody watching. Only constraints get a
# check.
#
# `verify_with` is how to check it is still true; `expires_when` is the
# STATE that ends it, deliberately not a date — constraints expire when
# the ground moves, not on a schedule. `verified_at` NULL means never
# checked and sorts FIRST in the sweep: unexamined outranks
# examined-long-ago.
#
# These sit on `notes`, so every kind of row in this table has them, but
# only non-task, non-snippet records are OFFERED them (gated in
# services/notes.py). A task's decay is its status — a done issue records
# what happened and cannot go false — and a snippet already carries a
# richer, location-aware verdict in `data.verification`. On those rows
# these stay null, which is also what they mean.
#
# Most notes should leave all three empty. A null `verify_with` is not a
# gap; it is the marker for "this is a decision, there is nothing to go
# and check", and the sweep is only worth reading while that holds.
verify_with: Mapped[str | None] = mapped_column(Text, nullable=True)
expires_when: Mapped[str | None] = mapped_column(Text, nullable=True)
verified_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
__table_args__ = (
Index("ix_notes_tags", "tags", postgresql_using="gin"),
@@ -168,14 +137,6 @@ class Note(Base, TimestampMixin, SoftDeleteMixin):
"is_task": self.is_task,
"note_type": self.note_type or "note",
"task_kind": self.task_kind,
# Serialized unconditionally, like every other field a given row
# kind may not use (recurrence, started_at, the task fields). The
# DERIVED "last_verified" label is the one that appears only when
# a check exists — a raw projection of the row should not make a
# client branch on which keys are present.
"verify_with": self.verify_with or "",
"expires_when": self.expires_when or "",
"verified_at": iso(self.verified_at),
"created_at": iso(self.created_at),
"updated_at": iso(self.updated_at),
}
-28
View File
@@ -42,35 +42,8 @@ class RetrievalLog(Base):
# False=notes, NULL=any.
is_task: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
result_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
# How many scored hits this call DROPPED because the session had already
# been shown them. NULLABLE, and the null is load-bearing: it means "this
# surface does not report suppression", which must not read as "nothing was
# suppressed". `result_count == 0` alone conflates two different events —
# the ranker found nothing above threshold, and the ranker found something
# the reader already had — and only the first says a threshold is too high.
# Reading a zero as a ranker decline is how #3311 mis-scoped a milestone;
# an unmeasured value that renders as 0 is the same mistake with a nicer
# face, so surfaces that filter INSIDE the search leave this null.
suppressed_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
top_score: Mapped[float | None] = mapped_column(Float, nullable=True)
min_score: Mapped[float | None] = mapped_column(Float, nullable=True)
# The best score the ranker COULD have offered, before the threshold — as
# against `top_score`, which is the best it DID offer. They are equal on
# any call that returned something, and only this one exists on a call
# that returned nothing, which is the only place a bar can be judged from
# (#3670). Null means the caller did not measure it, never "nothing was
# close": a 0.0 there would read as a corpus with no relevant records at
# all, which is an artifact standing in for a measurement.
best_available_score: Mapped[float | None] = mapped_column(Float, nullable=True)
# WHICH record scored that, so a reader can judge what the bar refused
# rather than only how close it came (#3807). Written from the same ranked
# candidate as the score above — the two describing different records would
# be worse than no id at all, because it invites judging the wrong one.
#
# Not a foreign key on purpose: this table spans record types (the rule arms
# store rule ids, the note arms store note ids) and `source` is what says
# which, exactly as `result_ids` has always worked.
best_available_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
# [{"id": int, "score": float, "rank": int}, ...], highest-first.
result_ids: Mapped[list] = mapped_column(JSONB, nullable=False, default=list)
duration_ms: Mapped[float | None] = mapped_column(Float, nullable=True)
@@ -94,7 +67,6 @@ class RetrievalLog(Base):
"project_id": self.project_id,
"is_task": self.is_task,
"result_count": self.result_count,
"suppressed_count": self.suppressed_count,
"top_score": self.top_score,
"min_score": self.min_score,
"result_ids": self.result_ids,
-95
View File
@@ -1,95 +0,0 @@
from sqlalchemy import BigInteger, Index, Text
from sqlalchemy.orm import Mapped, mapped_column
from scribe.models import Base
from scribe.models.base import CreatedAtMixin, iso
SURFACED = "surfaced"
PULLED = "pulled"
class RuleUsageEvent(Base, CreatedAtMixin):
"""One row per time a rule was SURFACED to the agent, or PULLED in full.
The sibling `note_usage_events` has had since 2026-07, third in the line
after `rule_embeddings` and `rule_versions` — and, like those, it exists
because the rule side kept inheriting machinery built for notes and
quietly getting the weaker version of it.
WHY RULES NEED THEIR OWN AND CANNOT SHARE THE NOTE TABLE. Not squeamishness
about a polymorphic column — the row shares no note-specific fields and the
aggregate readout is the same shape, which is the strongest case for
sharing that note #3163 admits. What decides it is IDENTITY AT RESTORE. A
note id and a rule id are different namespaces resolved through different
maps, and `note_usage_events`'s importer maps `note_id` through
`note_id_map` and drops what does not resolve. A rule id parked in that
column would come back from a backup silently reattached to whatever note
happened to take that number — telemetry that is not merely lost but wrong,
and wrong in a way nothing downstream could detect.
WHAT THIS MEASURES, AND WHY IT DID NOT EXIST. The write-path standing-rule
arm is the only retrieval surface in Scribe whose usefulness cannot be
observed — and, not coincidentally, the only one that has never declined to
fire (#3311: 296 calls, zero zero-result, 100% clearing its threshold).
`retrieval_logs` gives it scores; scores say what the ranker thought, never
whether the hint landed. Without a pull counter no install can tune the arm
from evidence, only from the shape of a histogram.
Deliberately FK-FREE on `rule_id` and `user_id`, matching `note_usage_events`,
`retrieval_logs` and `app_logs` — and diverging from `rule_versions`, which
does carry FKs. The difference is what the row is FOR: a version is part of
a rule's history and dies with it, while telemetry outlives the row it
describes. Deleting a rule must not erase the evidence that it was surfaced
forty times and opened never, because that evidence is precisely the case
for having deleted it.
Cells left deliberately empty (note #3163's step 3): no share ACL — rules
have none of their own; no soft delete — nothing restores a telemetry row,
and the table is append-only; no embedding — an event is not a document.
"""
__tablename__ = "rule_usage_events"
# BigInteger throughout, where the note twin uses Integer. `rule_id` has to
# be, since `rules.id` is BigInteger — and once one column is, matching the
# rest costs nothing and keeps the row uniform. A high-churn append-only
# telemetry table is a poor place to discover an id ceiling.
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
user_id: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
rule_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
# 'surfaced' | 'pulled'
event: Mapped[str] = mapped_column(Text, nullable=False)
# Which surface produced it. A CONVENTION, not a fixed vocabulary, and the
# note twin's comment explains why this one deliberately does not enumerate
# its members: the previous such list went stale, naming a source nothing
# wrote while omitting ones that existed, and a half-true enumeration reads
# as authoritative in exactly the way that misleads (#2476).
# `grep -rn record_rule_pulled\|record_rule_surfaced src/` is the
# authoritative list, and unlike a comment it cannot drift.
#
# The mcp_/rest_ prefix split is load-bearing here for the same reason it is
# for notes, and more so: "is this rule dead weight?" is served by any pull,
# but "did that injected hint land?" — the question this arm exists to
# answer — is served by AGENT pulls only. Never aggregate across the prefix
# without saying why.
source: Mapped[str] = mapped_column(Text, nullable=False)
__table_args__ = (
# Every readout is "these rule ids, split by event" — a covering
# composite beats separate single-column indexes for it.
Index("ix_rule_usage_rule_event", "rule_id", "event"),
Index("ix_rule_usage_created_at", "created_at"),
Index("ix_rule_usage_user_id", "user_id"),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"created_at": iso(self.created_at),
"user_id": self.user_id,
"rule_id": self.rule_id,
"event": self.event,
"source": self.source,
}
-91
View File
@@ -1,91 +0,0 @@
from datetime import datetime
from sqlalchemy import BigInteger, ForeignKey, Text
from sqlalchemy.orm import Mapped, mapped_column
from scribe.models import Base
from scribe.models.base import CreatedAtMixin, iso
class RuleVersion(Base, CreatedAtMixin):
"""One snapshot of a rule's text, taken before an edit overwrote it.
THE SIBLING NOTES ALREADY HAD. `note_versions` has existed for a long
time, and the design-system note calls its history "the changelog". Rules
— which BIND BEHAVIOUR on every session that loads them — had nothing, so
an edit destroyed what the rule used to say. Rescoping rule 79 on
2026-08-29 meant hand-copying the superseded statement into a task log to
keep it (#3237). The more consequential record had the weaker protection.
`CreatedAtMixin`, not `TimestampMixin`: a version is an EVENT. It is
written once and never updated, so an `updated_at` on it would be a column
that can only ever lie.
WHAT IS DELIBERATELY DIFFERENT FROM NoteVersion (milestone 323):
- `user_id` is the ACTOR — who made the edit — where NoteVersion's is the
owner, because `update_note` passes an owner-scoped id. For an audit
trail over a binding instruction, "who changed this" is the question
being asked, and a rule is editable by anyone with rulebook access.
- No `pin_kind` / `pin_label`. Those exist so a note's version can survive
autosave pruning. Nothing prunes here, so a pin would protect a row that
was never at risk.
- TEXT ONLY. A rule's Systems and its typed relations are edges with their
own lifecycle; folding them in would make one word, "version", mean two
different things — the rule's wording, and the rule's place in the
graph. `verified_at` is likewise absent: it is a stamp about a check,
not a property of the text, and step 2's snapshot is taken before
update_rule clears it precisely so the history shows the check that was
in force when this wording was written.
"""
__tablename__ = "rule_versions"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
rule_id: Mapped[int] = mapped_column(
BigInteger, ForeignKey("rules.id", ondelete="CASCADE"), index=True
)
# The actor. SET NULL rather than CASCADE: deleting a user must not erase
# the history of the rules they edited — the edit still happened, and the
# rule is still binding because of it. NoteVersion cascades because a
# note's versions belong to its owner; a rule's belong to the rule.
user_id: Mapped[int | None] = mapped_column(
BigInteger, ForeignKey("users.id", ondelete="SET NULL"), nullable=True
)
title: Mapped[str] = mapped_column(Text, default="")
statement: Mapped[str] = mapped_column(Text, default="")
why: Mapped[str | None] = mapped_column(Text, nullable=True)
how_to_apply: Mapped[str | None] = mapped_column(Text, nullable=True)
when_to_apply: Mapped[str | None] = mapped_column(Text, nullable=True)
tier: Mapped[str | None] = mapped_column(Text, nullable=True)
verify_with: Mapped[str | None] = mapped_column(Text, nullable=True)
expires_when: Mapped[str | None] = mapped_column(Text, nullable=True)
def to_dict(self, include_text: bool = True) -> dict:
"""The row. `include_text=False` gives the listing form.
A rule's `statement` and `why` run to thousands of characters — rule
149's `why` alone is longer than most notes — so a history LIST that
carried every field would be unreadable and expensive. The listing
answers "when, and by whom"; opening one answers "and what did it
say". Same split NoteVersion makes with `include_body`.
"""
out: dict = {
"id": self.id,
"rule_id": self.rule_id,
"user_id": self.user_id,
"title": self.title,
"created_at": iso(self.created_at),
}
if include_text:
out.update({
"statement": self.statement,
"why": self.why or "",
"how_to_apply": self.how_to_apply or "",
"when_to_apply": self.when_to_apply or "",
"tier": self.tier or "",
"verify_with": self.verify_with or "",
"expires_when": self.expires_when or "",
})
return out
+1 -56
View File
@@ -10,61 +10,6 @@ async def health():
return jsonify({"status": "ok"})
def build_version_payload() -> dict:
"""What build is this, separated into the values that answer different
questions (rule 149).
UNTIL 2026-08-31 THIS RETURNED THE CHANNEL. `BUILD_VERSION` in CI was
literally "dev" / "main" / the tag, so a running instance reported
`{"version": "main"}` — a channel name sitting where a build identifier
belongs. The cost was concrete: with a deploy misbehaving, nothing on the
instance could say which commit was serving it, and the one endpoint whose
job that is answered with the name of a branch.
The three values, and why they are three:
- `version` — the NAME, `YYYY.MM.DD.HHMM` from COMMIT time. Answers "is
this the same code?", so two channels carrying one commit report the
same string.
- `build` — the ORDERING KEY, minutes since 2020-01-01 from BUILD time.
Answers "may this be installed over that?". The ONLY value anything may
compare; it is monotonic by construction, which neither a commit count
(branches diverge) nor a commit time (rebuilds go backwards) is.
- `channel` — its own field, never folded into the name.
Plus `commit`, so the artifact's claim about itself can be checked against
the `:<sha>` it was published under (rule 145).
ABSENT RATHER THAN EMPTY when unknown. A local build has no ordering key
and no channel, and saying so is honest; emitting `""` or a placeholder
would let it claim a position in an update order it is not part of. A
reader must treat a missing `build` as "cannot be ordered", not as zero.
"""
payload: dict = {"version": os.environ.get("APP_VERSION", "dev")}
# Reported verbatim, never validated against an enum — a build claiming
# something unexpected is better shown than dropped (rule 149).
for key, env in (("channel", "APP_CHANNEL"), ("commit", "APP_COMMIT")):
value = (os.environ.get(env) or "").strip()
if value:
payload[key] = value
raw_key = (os.environ.get("APP_BUILD_KEY") or "").strip()
if raw_key:
try:
# An INTEGER, not a string. A string ordering key is how a
# comparison silently becomes lexicographic — "9" > "10" — which
# is the same class of fault as folding the channel in: it reads
# fine and orders wrong.
payload["build"] = int(raw_key)
except ValueError:
# A malformed key is omitted rather than passed through: a reader
# that cannot order is correct, one that orders on garbage is not.
pass
return payload
@api.route("/version")
async def version():
return jsonify(build_version_payload())
return jsonify({"version": os.environ.get("APP_VERSION", "dev")})
+4 -12
View File
@@ -1,4 +1,4 @@
"""Unified Knowledge endpoint — every record kind in one queryable feed."""
"""Unified Knowledge endpoint — notes, tasks, plans, and processes in one queryable feed."""
import logging
from quart import Blueprint, jsonify, request
@@ -6,18 +6,12 @@ from quart import Blueprint, jsonify, request
from scribe.auth import get_current_user_id, login_required
from scribe.routes.utils import parse_pagination
from scribe.services.access import label_shared_items
from scribe.services.knowledge import FACET_TYPES
logger = logging.getLogger(__name__)
knowledge_bp = Blueprint("knowledge", __name__, url_prefix="/api/knowledge")
# Derived from the service's facet table, never re-listed here. This set was a
# hand-kept copy and had drifted three kinds behind it: it admitted `plan`
# (retired in 0066) and rejected `issue` (shipped in 0065, 435 rows) and
# `snippet` — so the browse surface could not filter to the kinds it was
# already rendering badges for (#3128).
_VALID_TYPES = FACET_TYPES
_VALID_TYPES = {"note", "task", "plan", "process"}
_VALID_SORTS = {"modified", "created", "alpha", "type"}
@@ -27,9 +21,7 @@ async def list_knowledge():
"""Return paginated knowledge objects with optional filtering.
Query params:
type — a facet from services.knowledge._FACETS: a record type
(note|process|snippet) or a task kind (task for any,
else work|issue|spike|plan). Omit for all.
type — one of note|task|plan|process (omit for all)
tags — comma-separated tag filter (AND logic)
sort — modified|created|alpha|type (default: modified)
q — search query (semantic when provided, keyword fallback)
@@ -135,7 +127,7 @@ async def get_knowledge_batch():
@knowledge_bp.route("/tags", methods=["GET"])
@login_required
async def list_knowledge_tags():
"""Return all tags used across knowledge objects, narrowed to one facet."""
"""Return all tags used across knowledge objects (excludes tasks)."""
uid = get_current_user_id()
note_type = request.args.get("type", "").strip().lower() or None
+1 -72
View File
@@ -19,10 +19,7 @@ from scribe.services.notes import (
get_note_for_user,
get_or_create_note_by_title,
list_notes,
mark_note_verified,
notes_due_for_verification,
update_note,
verification_row,
)
from scribe.services.note_drafts import upsert_draft, get_draft, delete_draft
from scribe.services import dedup as dedup_svc
@@ -115,8 +112,6 @@ async def create_note_route():
priority=priority,
due_date=due_date,
note_type=note_type,
verify_with=data.get("verify_with"),
expires_when=data.get("expires_when"),
)
except ValueError as e:
return jsonify({"error": str(e)}), 400
@@ -253,14 +248,7 @@ async def update_note_route(note_id: int):
owner_uid = note_obj.user_id
data = await request.get_json()
fields = {}
for key in (
"title", "body", "description", "parent_id", "project_id",
"milestone_id", "status", "priority", "note_type",
# A cleared form input arrives as "" and the service reads that as
# NULL (NULLABLE_NOTE_TEXT), so this door expresses "remove the check"
# with its own idiom and needs no `clear` list (milestone 317).
"verify_with", "expires_when",
):
for key in ("title", "body", "description", "parent_id", "project_id", "milestone_id", "status", "priority", "note_type"):
if key in data:
fields[key] = data[key]
if "due_date" in data:
@@ -502,62 +490,3 @@ async def graph_route():
shared_tags = request.args.get("shared_tags", "false").lower() == "true"
graph = await build_note_graph(uid, project_id=project_id, include_shared_tags=shared_tags)
return jsonify(graph)
# ── The staleness sweep (milestone 317) ──────────────────────────────────────
# The web half of notes_due_for_verification / mark_note_verified. Same
# contract as the MCP door and the rules routes beside it — the service holds
# the behaviour, these two just parse and serialise.
@notes_bp.route("/due-for-verification", methods=["GET"])
@login_required
async def notes_due_route():
"""Notes that carry a check, oldest verification first, never-checked top.
Query params: older_than_days, project_id, never_only. A note with no
`verify_with` never appears — it is a decision, not a fact.
"""
uid = get_current_user_id()
args = request.args
try:
older = int(args.get("older_than_days", 0) or 0)
project = int(args.get("project_id", 0) or 0)
except ValueError:
return jsonify({"error": "older_than_days and project_id must be integers"}), 400
try:
notes = await notes_due_for_verification(
uid,
older_than_days=older,
project_id=project or None,
never_only=args.get("never_only", "").lower() in ("1", "true", "yes"),
)
except ValueError as exc:
# A 400, not a silently narrowed result: a filter that quietly answers
# a different question is the failure this whole surface exists to
# catch.
return jsonify({"error": str(exc)}), 400
return jsonify({
"notes": [verification_row(n) for n in notes],
"total": len(notes),
})
@notes_bp.route("/<int:note_id>/verify", methods=["POST"])
@login_required
async def mark_note_verified_route(note_id: int):
"""Record that the note's check was run. Body: {"still_true": bool}.
`still_true: false` writes nothing — a note whose check failed is wrong,
not in a recordable state — so it keeps its place at the top of the sweep.
"""
data = await request.get_json() or {}
uid = get_current_user_id()
still_true = bool(data.get("still_true", True))
note = await mark_note_verified(note_id, uid, still_true)
if note is None:
return jsonify({
"error": "note not found, not writable by you, or carries no verify_with"
}), 404
payload = verification_row(note)
payload["verified"] = still_true
return jsonify(payload)
-47
View File
@@ -101,46 +101,6 @@ async def autoinject_retrieve():
return jsonify(result)
@plugin_bp.get("/tool-rules")
@login_required
async def pre_tool_rules():
"""Standing rules for the plugin's PreToolUse hook on ACTIONS (#3476).
Answers "does a recorded rule speak to the command about to be run?" — the
sibling of /prior-art, which can only answer that question about a code
write. Rules about which tool to reach for (don't curl the forge, don't
stand up a stack, don't run the suite locally) had no retrieval surface at
all before this, which is why they all had to live in the resident preload.
Titles + trigger only, never the statement: the hint says a rule may apply
and hands over `get_rule(id)`. One rule at most (RULEHINT_LIMIT), and empty
most of the time.
Query:
tool (str) — the tool about to run, e.g. `Bash`. Used in
the hint's wording, not in the search: a
rule is about the action, not the harness.
command (str) — the command about to run; the semantic query.
Absent or blank → empty, no search.
repo (optional) — working repo remote, resolved to the bound
project exactly as /retrieve and /prior-art.
exclude_rule_ids (opt) — comma-separated rule ids already surfaced
this session. SHARED with /prior-art's
ledger on purpose: one session keeps one
list, so a rule named by either arm is not
re-offered by the other.
"""
tool = (request.args.get("tool") or "tool").strip()
command = request.args.get("command") or ""
project_id, _repo, _unbound = await _project_scope()
exclude_rule_ids = _int_list(request.args.get("exclude_rule_ids"))
result = await plugin_ctx_svc.build_tool_rule_hint(
g.user.id, tool, command,
project_id=project_id, exclude_rule_ids=exclude_rule_ids,
)
return jsonify(result)
@plugin_bp.get("/prior-art")
@login_required
async def write_path_prior_art():
@@ -178,11 +138,6 @@ async def write_path_prior_art():
or `canon:<snippet_id>`) already named this
session by the ledger arm (#2900); its own
channel, like the two above.
rules_etag (opt) — the marker the session was given when it loaded
its always-on rules (milestone 323). Sent back
so the server can say whether those rules have
MOVED since. Absent means the hook has nothing
stored, which is silence, not a mismatch.
shapes (opt) — comma-separated `kind:name` definitions the hook
found in (or enclosing) the payload, kind being
css|sym. The shape ledger's write-path feed
@@ -202,7 +157,6 @@ async def write_path_prior_art():
p.strip() for p in (request.args.get("exclude_derive") or "").split(",") if p.strip()
]
exclude_rule_ids = _int_list(request.args.get("exclude_rule_ids"))
rules_etag = (request.args.get("rules_etag") or "").strip()
shapes = _parse_shapes(request.args.get("shapes") or "")
api_key = getattr(g, "api_key", None)
may_stamp = api_key is None or getattr(api_key, "scope", "") == "write"
@@ -214,7 +168,6 @@ async def write_path_prior_art():
repo_key=repo_bindings_svc.normalize_repo_key(repo) if repo else "",
exclude_derive=exclude_derive,
exclude_rule_ids=exclude_rule_ids,
rules_etag=rules_etag,
)
return jsonify(result)
+2 -57
View File
@@ -10,9 +10,6 @@ from quart import Blueprint, jsonify, request
from scribe.auth import get_current_user_id, login_required
import scribe.services.rulebooks as rulebooks_svc
from scribe.services.trash import delete as trash_delete
from scribe.services.rule_usage import (
empty_rule_usage, record_rule_pulled, usage_for_rules,
)
rulebooks_bp = Blueprint("rulebooks", __name__, url_prefix="/api")
@@ -139,24 +136,13 @@ async def list_rules():
except ValueError:
return jsonify({"error": "rulebook_id, topic_id, project_id must be integers"}), 400
uid = get_current_user_id()
rows = await rulebooks_svc.list_rules(
user_id=uid,
user_id=get_current_user_id(),
rulebook_id=rulebook_id,
topic_id=topic_id,
project_id=project_id,
)
items = [r.to_dict() for r in rows]
# One aggregate for the whole page — a per-row lookup here would be N+1 by
# construction, the same reason the snippet list does it this way. Every
# row gets the key, zero-filled, so the UI renders "never surfaced" rather
# than having to treat a missing field as a state. That matters more here
# than for snippets: every rule on every install predates this table, so
# for a while the zero-filled shape IS the common case.
usage = await usage_for_rules([int(it["id"]) for it in items])
for it in items:
it["usage"] = usage.get(int(it["id"]), empty_rule_usage())
return jsonify({"rules": items})
return jsonify({"rules": [r.to_dict() for r in rows]})
@rulebooks_bp.post("/rulebook-topics/<int:topic_id>/rules")
@@ -196,11 +182,6 @@ async def get_rule(rule_id: int):
rule = await rulebooks_svc.get_rule(rule_id, uid)
if rule is None:
return jsonify({"error": "rule not found"}), 404
# `rest_` rather than `mcp_`, and the prefix is load-bearing: "is this rule
# dead weight?" is served by any pull, but "did that injected hint land?"
# — the question this arm exists to answer — is served by AGENT pulls only.
# A person clicking through the rule list says nothing about the hint.
record_rule_pulled(user_id=uid, rule_id=int(rule.id), source="rest_rule")
return jsonify(await rulebooks_svc.rule_detail(uid, rule))
@@ -225,42 +206,6 @@ async def update_rule(rule_id: int):
return jsonify(await rulebooks_svc.rule_detail(uid, rule, data.get("system_ids")))
@rulebooks_bp.get("/rules/<int:rule_id>/versions")
@login_required
async def list_rule_versions(rule_id: int):
"""A rule's edit history, newest first.
Listing form only — a rule's `statement` and `why` run to thousands of
characters, so a history list carrying every field would be unreadable
and expensive to send. Open one for the text.
"""
uid = get_current_user_id()
versions = await rulebooks_svc.list_rule_versions(rule_id, uid)
if versions is None:
return jsonify({"error": "rule not found"}), 404
return jsonify({
"versions": [v.to_dict(include_text=False) for v in versions],
})
@rulebooks_bp.get("/rules/<int:rule_id>/versions/<int:version_id>")
@login_required
async def get_rule_version(rule_id: int, version_id: int):
"""One snapshot in full — what the rule said before that edit."""
uid = get_current_user_id()
version = await rulebooks_svc.get_rule_version(rule_id, version_id, uid)
if version is None:
return jsonify({"error": "version not found"}), 404
return jsonify(version.to_dict(include_text=True))
# NO restore route, deliberately (milestone 323). A note version can be
# restored; a binding instruction should not be revertible in one click.
# Putting a rewrite back goes through update_rule, which takes its own
# snapshot and leaves the undo in the history like any other edit — a silent
# revert would erase the only record of why the rewrite happened.
@rulebooks_bp.post("/rules/<int:rule_id>/relations")
@login_required
async def relate_rules(rule_id: int):
-5
View File
@@ -44,22 +44,17 @@ async def search_route():
project_id = request.args.get("project_id", type=int)
t0 = time.perf_counter()
report: dict = {}
results = await semantic_search_notes(
uid, q, limit=limit, is_task=is_task, threshold=_REST_SEARCH_THRESHOLD,
project_id=project_id, system_id=system_id,
# The user typed this, so it reaches everything they may read.
scope="read",
report=report,
)
record_retrieval(
user_id=uid, source="rest_search", query=q,
threshold=_REST_SEARCH_THRESHOLD, limit=limit,
project_id=project_id, is_task=is_task, results=results,
duration_ms=(time.perf_counter() - t0) * 1000.0,
best_available=report.get("best_available_score"),
best_available_id=report.get("best_available_id"),
searched=bool(report.get("searched", True)),
)
owners = await owner_names_for(
{int(note.user_id) for _s, note in results if note.user_id != uid}
+3 -46
View File
@@ -6,63 +6,20 @@ write that never errors and never lands (the #2663 GC footgun). This module is
the one place that gets the pattern right: strong references in ``_pending``,
discarded on completion, with failures logged at WARNING instead of vanishing.
``retrieval_telemetry`` predates this module and keeps its own copy, because
its canary is a genuinely different shape — one process-wide flag and no
AppLog row. ``note_usage`` and ``rule_usage`` share ``report_telemetry_failure``
below. New fire-and-forget callers use ``spawn`` rather than writing another
copy of the strong-reference dance.
``note_usage`` and ``retrieval_telemetry`` predate this module and carry their
own copies with bespoke canary semantics; new fire-and-forget callers use this
instead of writing a fourth copy.
"""
from __future__ import annotations
import asyncio
import logging
import traceback
from collections.abc import Coroutine
logger = logging.getLogger(__name__)
_pending: set[asyncio.Task] = set()
# Sites that have already dropped their once-per-process AppLog row, keyed
# "<subsystem>:<site>". A readout can run on every list render — without this,
# a broken table turns the error log into a firehose that buries the finding it
# exists to surface.
_reported: set[str] = set()
async def report_telemetry_failure(subsystem: str, site: str) -> None:
"""Make a swallowed telemetry failure visible. Call from an except block.
WARNING to the process log every time; one AppLog error row per process per
(subsystem, site) so the admin UI shows the outage without host access.
THIS IS NOT DECORATION. #2663 is the record of a telemetry subsystem running
at zero for weeks — every counter reading empty, indistinguishable from
"nobody uses this" — because every failure went to ``logger.debug``. A
subsystem whose failures are all invisible cannot report its own death.
The AppLog write is itself guarded: when the database is down it fails too,
and that is fine. The WARNING already said so, and a canary must never take
down the surface it watches.
"""
logger.warning("%s telemetry %s failed", subsystem, site, exc_info=True)
key = f"{subsystem}:{site}"
if key in _reported:
return
_reported.add(key)
try:
from scribe.services.logging import log_error
await log_error(
endpoint=subsystem,
error_type=f"{subsystem}_{site}_failed",
error_message=f"{subsystem} telemetry {site} is failing; "
"usage counters will read zero until this is fixed",
traceback=traceback.format_exc(),
)
except Exception:
logger.debug("%s canary write failed", subsystem, exc_info=True)
def spawn(coro: Coroutine, *, site: str) -> None:
"""Schedule ``coro`` fire-and-forget; ``site`` names it in failure logs.
+10 -318
View File
@@ -9,10 +9,8 @@ from scribe.models.note import Note
from scribe.models.note_draft import NoteDraft
from scribe.models.note_supersession import NoteSupersession
from scribe.models.note_version import NoteVersion
from scribe.models.rule_version import RuleVersion
from scribe.models.design_system import DesignSystem, DesignToken
from scribe.models.note_usage import NoteUsageEvent
from scribe.models.rule_usage import RuleUsageEvent
from scribe.models.canonical_system import CanonicalSystem
from scribe.models.rulebook import RuleRelation, rule_systems as rule_systems_t
from scribe.models.code_shape import CodeShape, CodeShapeEvent, CodeShapeUse
@@ -52,23 +50,8 @@ logger = logging.getLogger(__name__)
# ones (reference/hook) travel too, cheaply, and the next refresh refreshes them.
# v10 (2026-08) added projects.inception + project_rulebook_exclusions
# (milestone 297): the WHY a project inherits what it does, and its opt-outs.
# v11 (2026-08) added the note verification trio — notes.verify_with /
# expires_when / verified_at (milestone 317).
# v12 (2026-08) closed #3182: the nine Note columns that had been missing for
# years (note_type, task_kind, arose_from_id, data, description, the
# recurrence pair, started_at, completed_at), plus milestones.body — which IS
# the plan — and repo_bindings.ref. Until v12 a restore reported success and
# handed back a corpus with every snippet and process flattened into a plain
# note, every issue and spike into `work`, and every plan reduced to a title.
# _COLUMN_EXCLUSIONS and its guard landed with it, so the next such column
# fails the build instead.
# v13 (2026-08) added rule_versions — a rule's edit history (milestone 323).
# v14 (2026-09) added rule_usage_events — the rule twin of note_usage_events
# (milestone 333). Carrying it is the WHOLE REASON the table is separate: the
# note importer maps note_id through note_id_map, so a rule id parked there
# would restore attached to whatever note took that number.
# Bump when the serialized schema changes.
BACKUP_VERSION = 14
BACKUP_VERSION = 10
# Every table this backup carries, by its REAL name. Paired with _NOT_INCLUDED
# below, these two lists must together account for the entire schema — which is
@@ -94,14 +77,6 @@ _BACKED_UP = [
"canonical_systems",
# v10 (2026-08): a rule's area tag and its typed edges (milestone 307).
"rule_systems", "rule_relations",
# v13 (2026-08): a rule's edit history (milestone 323). note_versions has
# always travelled; its sibling has no excuse not to.
"rule_versions",
# v14 (2026-09): rule usage telemetry (milestone 333). Same argument
# note_usage_events makes for itself — pull-through is only ever
# accumulated, so a restore that dropped it would silently reset the
# measurement to zero while everything still looked fine.
"rule_usage_events",
]
# Tables intentionally NOT in the backup, surfaced in the payload so the gap is
@@ -133,88 +108,6 @@ _NOT_INCLUDED = [
]
# Columns a backed-up table deliberately does NOT export, per table. Paired
# with the column-coverage guard in tests/test_services_backup.py, this is
# _NOT_INCLUDED's shape one level down — and it exists because the table guard
# could not see the failure it was written to stop.
#
# #2293 was six whole tables missing. #3182 was NINE COLUMNS missing from a
# table that had been "covered" for years: note_type and task_kind, so every
# snippet and process restored as a plain note and every issue and spike as
# `work`; arose_from_id, so every provenance edge vanished; the recurrence
# pair, so recurring tasks stopped recurring; milestones.body, which IS the
# plan; repo_bindings.ref, the branch a ledger follows. Every one arrived the
# same way — a column added to the model and the migration, both of which fail
# loudly, and then never added to the serialiser, which fails silently.
#
# So: a new column on a backed-up table now fails the build unless it is either
# exported or named here with a reason. "I forgot" is no longer expressible.
_COLUMN_EXCLUSIONS: dict[str, set[str]] = {
# Trash is not exported at all, so neither is the batch id that groups a
# deletion for restore(). Uniform across every soft-deletable table.
"users": set(),
"projects": {
"deleted_at", "deleted_batch_id",
# Credentials-adjacent, same reasoning as api_keys and
# forge_connections: a restored project falls back to keyring-by-host
# resolution, which is the documented unpinned behaviour (#2778).
"forge_connection_id",
},
"milestones": {"deleted_at", "deleted_batch_id"},
"notes": {"deleted_at", "deleted_batch_id"},
"task_logs": set(),
"note_drafts": set(),
"note_versions": set(),
"settings": set(),
"rulebooks": {"deleted_at", "deleted_batch_id"},
"rulebook_topics": {"deleted_at", "deleted_batch_id"},
"rules": {"deleted_at", "deleted_batch_id"},
"systems": {
"deleted_at", "deleted_batch_id",
# Travels as `canonical_slug`: the catalog is global and its ids are
# per-install, so an id would restore pointing at whatever area
# happened to land on that number.
"canonical_id",
"created_at", "updated_at",
},
"canonical_systems": {
"deleted_at", "deleted_batch_id",
# Matched on SLUG at restore, so a target install that already seeded
# the standard vocabulary reuses its own rows rather than colliding.
"id", "created_at", "updated_at",
},
# Edge tables: the id is regenerated on insert, and the pair IS the row.
"record_systems": {"id", "created_at"},
"note_supersessions": {"id", "created_at"},
"rule_relations": {"id", "created_at"},
"note_usage_events": {"id"},
# Same as the note twin: the surrogate key is re-issued on insert.
"rule_usage_events": {"id"},
"design_systems": {"deleted_at", "deleted_batch_id", "created_at", "updated_at"},
"design_tokens": {"deleted_at", "deleted_batch_id", "created_at", "updated_at"},
"repo_bindings": {"id", "created_at", "updated_at"},
# Everything travels. A version row IS the audit trail, so a column left
# behind is a fact about a binding instruction that no longer exists
# anywhere.
"rule_versions": set(),
# Serialised via the model's own to_dict(), so a column reaches the backup
# the moment it reaches that method — and the guard still catches one that
# reaches neither.
"code_shapes": {
# The PROPOSER's standing suggestion for an unclassified row, not a
# judgment: "looks like an instance of #N", or "repeats with no canon".
# Every refresh recomputes it and a judgment clears it, so carrying it
# would restore stale machine guesses over a tree the proposer has not
# seen. Same reasoning as code_shape_consumers in _NOT_INCLUDED —
# derived data is regenerated, never restored.
"proposed_snippet_id", "proposal_basis", "proposal_score",
"proposal_group", "proposed_at", "proposed_sha",
},
"code_shape_events": set(),
"code_shape_uses": set(),
}
def _dt(val: str | None) -> datetime:
return datetime.fromisoformat(val) if val else datetime.now(timezone.utc)
@@ -333,17 +226,6 @@ def _usage_event_rows(rows) -> list[dict]:
]
def _rule_usage_event_rows(rows) -> list[dict]:
return [
{
"user_id": r.user_id, "rule_id": r.rule_id, "event": r.event,
"source": r.source,
"created_at": r.created_at.isoformat() if r.created_at else None,
}
for r in rows
]
def _code_shape_rows(rows) -> list[dict]:
return [r.to_dict() for r in rows]
@@ -358,13 +240,7 @@ def _code_shape_use_rows(rows) -> list[dict]:
def _repo_binding_rows(rows) -> list[dict]:
return [
# `ref` is the branch this binding's ledger follows (#2873). Without
# it a restored binding silently falls back to the default branch, and
# the shape ledger starts accounting for a different tree (#3182).
{
"user_id": r.user_id, "project_id": r.project_id,
"repo_key": r.repo_key, "ref": r.ref,
}
{"user_id": r.user_id, "project_id": r.project_id, "repo_key": r.repo_key}
for r in rows
]
@@ -406,11 +282,6 @@ def _milestone_rows(rows) -> list[dict]:
{
"id": m.id, "user_id": m.user_id, "project_id": m.project_id,
"title": m.title, "description": m.description, "status": m.status,
# THE PLAN. A milestone IS the plan (0066) and `body` is its
# design and intent; `description` is only the one-line summary.
# Dropping this restored every plan as a title with no reasoning
# behind it (#3182).
"body": m.body,
"order_index": m.order_index,
"created_at": m.created_at.isoformat(),
"updated_at": m.updated_at.isoformat(),
@@ -423,45 +294,12 @@ def _note_rows(rows) -> list[dict]:
return [
{
"id": n.id, "user_id": n.user_id, "title": n.title, "body": n.body,
"description": n.description,
"tags": n.tags or [], "parent_id": n.parent_id,
"project_id": n.project_id, "milestone_id": n.milestone_id,
"status": n.status, "priority": n.priority,
"due_date": n.due_date.isoformat() if n.due_date else None,
"created_at": n.created_at.isoformat(),
"updated_at": n.updated_at.isoformat(),
# WHAT KIND OF RECORD THIS IS — both typing axes (#3182). Missing
# until now, which meant a restore reported success and handed back
# a corpus with every snippet and process flattened into a plain
# note and every issue and spike into `work`. Nothing recomputes
# these; the vocabulary is simply gone.
"note_type": n.note_type,
"task_kind": n.task_kind,
# Provenance — which record caused this one. Re-mapped in the
# second pass beside parent_id, never here: the value is an id in
# the SOURCE database.
"arose_from_id": n.arose_from_id,
# The queryable mirror. The only one of these that would self-heal
# (backfill_snippet_data rebuilds it from the body at startup), but
# a restore should not hand back a corpus that needs a restart to
# become searchable by location.
"data": n.data,
# Lifecycle: when the work actually started and finished, and the
# recurrence rule that makes a task come back. Without these a
# restored recurring task simply stops recurring.
"started_at": n.started_at.isoformat() if n.started_at else None,
"completed_at": n.completed_at.isoformat() if n.completed_at else None,
"recurrence_rule": n.recurrence_rule,
"recurrence_next_spawn_at": (
n.recurrence_next_spawn_at.isoformat()
if n.recurrence_next_spawn_at else None
),
# The verification trio (milestone 317, migration 0092). Operator
# judgment — "somebody checked this fact, and this is when" —
# which nothing can recompute.
"verify_with": n.verify_with,
"expires_when": n.expires_when,
"verified_at": n.verified_at.isoformat() if n.verified_at else None,
}
for n in rows
]
@@ -504,23 +342,6 @@ def _note_version_rows(rows) -> list[dict]:
]
def _rule_version_rows(rows) -> list[dict]:
"""A rule's edit history. Sibling of _note_version_rows, and it travels for
the same reason: a version is the only record of what a binding
instruction used to say, and nothing can recompute it."""
return [
{
"id": rv.id, "rule_id": rv.rule_id, "user_id": rv.user_id,
"title": rv.title, "statement": rv.statement, "why": rv.why,
"how_to_apply": rv.how_to_apply, "when_to_apply": rv.when_to_apply,
"tier": rv.tier, "verify_with": rv.verify_with,
"expires_when": rv.expires_when,
"created_at": rv.created_at.isoformat(),
}
for rv in rows
]
def _setting_rows(rows) -> list[dict]:
return [{"user_id": s.user_id, "key": s.key, "value": s.value} for s in rows]
@@ -602,9 +423,6 @@ async def export_full_backup() -> dict:
note_versions = (await session.execute(
select(NoteVersion).order_by(NoteVersion.note_id, NoteVersion.id)
)).scalars().all()
rule_versions = (await session.execute(
select(RuleVersion).order_by(RuleVersion.rule_id, RuleVersion.id)
)).scalars().all()
settings = (await session.execute(select(Setting))).scalars().all()
systems = (await session.execute(select(System))).scalars().all()
canonical_systems = (await session.execute(
@@ -629,9 +447,6 @@ async def export_full_backup() -> dict:
)).scalars().all()
design_tokens = (await session.execute(select(DesignToken))).scalars().all()
usage_events = (await session.execute(select(NoteUsageEvent))).scalars().all()
rule_usage_events = (
await session.execute(select(RuleUsageEvent))
).scalars().all()
repo_bindings = (await session.execute(select(RepoBinding))).scalars().all()
code_shapes = (await session.execute(select(CodeShape))).scalars().all()
code_shape_events = (await session.execute(
@@ -672,7 +487,6 @@ async def export_full_backup() -> dict:
"task_logs": _task_log_rows(task_logs),
"note_drafts": _note_draft_rows(note_drafts),
"note_versions": _note_version_rows(note_versions),
"rule_versions": _rule_version_rows(rule_versions),
"settings": _setting_rows(settings),
"rulebooks": _rulebook_rows(rulebooks),
"rulebook_topics": _topic_rows(topics),
@@ -691,7 +505,6 @@ async def export_full_backup() -> dict:
"design_systems": _design_system_rows(design_systems),
"design_tokens": _design_token_rows(design_tokens),
"note_usage_events": _usage_event_rows(usage_events),
"rule_usage_events": _rule_usage_event_rows(rule_usage_events),
"repo_bindings": _repo_binding_rows(repo_bindings),
"note_supersessions": _note_supersession_rows(supersessions),
"code_shapes": _code_shape_rows(code_shapes),
@@ -810,22 +623,6 @@ async def export_user_backup(user_id: int) -> dict:
.join(CanonicalSystem, CanonicalSystem.id == rule_systems_t.c.canonical_id)
.where(rule_systems_t.c.rule_id.in_(_rule_ids))
)).all() if _rule_ids else []
# Scoped through the RULE, not the version's user_id. That column is
# the ACTOR (milestone 323), so filtering on it would carry the
# versions this user wrote on someone ELSE's rule and drop the ones
# someone else wrote on theirs — the opposite of a per-user export.
rule_versions = (await session.execute(
select(RuleVersion).where(RuleVersion.rule_id.in_(_rule_ids))
.order_by(RuleVersion.rule_id, RuleVersion.id)
)).scalars().all() if _rule_ids else []
# Scoped through the RULE for the same reason the versions above are,
# and it is worth restating because the column that looks right is
# wrong: `user_id` here is whoever the arm fired FOR, not who owns the
# rule. Filtering on it would carry this user's surfacings of someone
# ELSE's rule and drop the ones fired for someone else on theirs.
rule_usage_events = (await session.execute(
select(RuleUsageEvent).where(RuleUsageEvent.rule_id.in_(_rule_ids))
)).scalars().all() if _rule_ids else []
rule_relations = (await session.execute(
select(RuleRelation).where(
RuleRelation.from_rule_id.in_(_rule_ids),
@@ -874,7 +671,6 @@ async def export_user_backup(user_id: int) -> dict:
"task_logs": _task_log_rows(task_logs),
"note_drafts": _note_draft_rows(note_drafts),
"note_versions": _note_version_rows(note_versions),
"rule_versions": _rule_version_rows(rule_versions),
"settings": _setting_rows(settings),
"rulebooks": _rulebook_rows(rulebooks),
"rulebook_topics": _topic_rows(topics),
@@ -893,7 +689,6 @@ async def export_user_backup(user_id: int) -> dict:
"design_systems": _design_system_rows(design_systems),
"design_tokens": _design_token_rows(design_tokens),
"note_usage_events": _usage_event_rows(usage_events),
"rule_usage_events": _rule_usage_event_rows(rule_usage_events),
"repo_bindings": _repo_binding_rows(repo_bindings),
"note_supersessions": _note_supersession_rows(supersessions),
"code_shapes": _code_shape_rows(code_shapes),
@@ -952,29 +747,11 @@ async def _restore_v1(data: dict) -> dict:
body=n_data.get("body", ""),
tags=n_data.get("tags", []),
parent_id=None, # patched below
arose_from_id=None, # patched below, same reason
description=n_data.get("description"),
note_type=n_data.get("note_type") or "note",
task_kind=n_data.get("task_kind") or "work",
data=n_data.get("data"),
started_at=_dt_or_none(n_data.get("started_at")),
completed_at=_dt_or_none(n_data.get("completed_at")),
recurrence_rule=n_data.get("recurrence_rule"),
recurrence_next_spawn_at=_dt_or_none(
n_data.get("recurrence_next_spawn_at")
),
status=n_data.get("status"),
priority=n_data.get("priority"),
due_date=_d(n_data.get("due_date")),
created_at=_dt(n_data.get("created_at")),
updated_at=_dt(n_data.get("updated_at")),
verify_with=n_data.get("verify_with"),
expires_when=n_data.get("expires_when"),
# _dt_or_none, NOT _dt: an absent stamp must stay absent. _dt
# substitutes now(), which would restore every never-checked
# note as checked at the moment of the restore — inverting the
# one signal the sweep reads.
verified_at=_dt_or_none(n_data.get("verified_at")),
)
session.add(note)
await session.flush()
@@ -982,25 +759,14 @@ async def _restore_v1(data: dict) -> dict:
note_id_map[old_id] = note.id
stats["notes"] += 1
# Patch the two note->note edges now that every note has a new id.
# Both are ids in the SOURCE database, so writing either straight into
# the constructor would point at whatever record happens to hold that
# number here — a restore that succeeds and silently re-parents (#3182).
# An edge whose target did not survive the import is left NULL rather
# than guessed at.
# Patch parent_id now that all notes have new IDs
for n_data in data.get("notes", []):
old_id = n_data.get("id")
if not old_id or old_id not in note_id_map:
continue
note_row = await session.get(Note, note_id_map[old_id])
if note_row is None:
continue
old_parent = n_data.get("parent_id")
if old_parent and old_parent in note_id_map:
note_row.parent_id = note_id_map[old_parent]
old_origin = n_data.get("arose_from_id")
if old_origin and old_origin in note_id_map:
note_row.arose_from_id = note_id_map[old_origin]
if old_id and old_parent and old_id in note_id_map and old_parent in note_id_map:
note_row = await session.get(Note, note_id_map[old_id])
if note_row:
note_row.parent_id = note_id_map[old_parent]
for s_data in data.get("settings", []):
mapped_user_id = user_id_map.get(s_data.get("user_id", 0))
@@ -1030,11 +796,10 @@ async def _restore_v2(data: dict) -> dict:
"rulebook_subscriptions": 0, "rule_suppressions": 0,
"topic_suppressions": 0, "rulebook_exclusions": 0,
"systems": 0, "record_systems": 0, "design_systems": 0,
"design_tokens": 0, "note_usage_events": 0, "rule_usage_events": 0,
"repo_bindings": 0,
"design_tokens": 0, "note_usage_events": 0, "repo_bindings": 0,
"note_supersessions": 0, "code_shapes": 0, "code_shape_events": 0,
"code_shape_uses": 0, "canonical_systems": 0,
"rule_systems": 0, "rule_relations": 0, "rule_versions": 0,
"rule_systems": 0, "rule_relations": 0,
}
async with async_session() as session:
@@ -1094,7 +859,6 @@ async def _restore_v2(data: dict) -> dict:
project_id=mapped_pid,
title=m_data.get("title", ""),
description=m_data.get("description"),
body=m_data.get("body"),
status=m_data.get("status", "active"),
order_index=m_data.get("order_index", 0),
created_at=_dt(m_data.get("created_at")),
@@ -1107,7 +871,6 @@ async def _restore_v2(data: dict) -> dict:
# 4a. Notes — first pass (no parent_id yet)
notes_with_parents: list[tuple[int, int]] = [] # (new_note_id, old_parent_id)
notes_with_origins: list[tuple[int, int]] = [] # (new_note_id, old_arose_from_id)
for n_data in data.get("notes", []):
mapped_uid = user_id_map.get(n_data.get("user_id", 0))
if mapped_uid is None:
@@ -1118,17 +881,6 @@ async def _restore_v2(data: dict) -> dict:
body=n_data.get("body", ""),
tags=n_data.get("tags", []),
parent_id=None,
arose_from_id=None, # patched below, same reason as parent_id
description=n_data.get("description"),
note_type=n_data.get("note_type") or "note",
task_kind=n_data.get("task_kind") or "work",
data=n_data.get("data"),
started_at=_dt_or_none(n_data.get("started_at")),
completed_at=_dt_or_none(n_data.get("completed_at")),
recurrence_rule=n_data.get("recurrence_rule"),
recurrence_next_spawn_at=_dt_or_none(
n_data.get("recurrence_next_spawn_at")
),
project_id=project_id_map.get(n_data["project_id"]) if n_data.get("project_id") else None,
milestone_id=milestone_id_map.get(n_data["milestone_id"]) if n_data.get("milestone_id") else None,
status=n_data.get("status"),
@@ -1136,35 +888,21 @@ async def _restore_v2(data: dict) -> dict:
due_date=_d(n_data.get("due_date")),
created_at=_dt(n_data.get("created_at")),
updated_at=_dt(n_data.get("updated_at")),
verify_with=n_data.get("verify_with"),
expires_when=n_data.get("expires_when"),
# _dt_or_none — see the note on the other restore path.
verified_at=_dt_or_none(n_data.get("verified_at")),
)
session.add(note)
await session.flush()
note_id_map[n_data["id"]] = note.id
if n_data.get("parent_id"):
notes_with_parents.append((note.id, n_data["parent_id"]))
if n_data.get("arose_from_id"):
notes_with_origins.append((note.id, n_data["arose_from_id"]))
stats["notes"] += 1
# 4b. Patch the note->note edges. Deferred for the same reason
# parent_id always has been: these are ids in the SOURCE database
# (#3182). An edge whose target did not survive stays NULL.
# 4b. Patch parent_id
for new_note_id, old_parent_id in notes_with_parents:
new_parent_id = note_id_map.get(old_parent_id)
if new_parent_id:
note_row = await session.get(Note, new_note_id)
if note_row:
note_row.parent_id = new_parent_id
for new_note_id, old_origin_id in notes_with_origins:
new_origin_id = note_id_map.get(old_origin_id)
if new_origin_id:
note_row = await session.get(Note, new_note_id)
if note_row:
note_row.arose_from_id = new_origin_id
# 5. TaskLogs
for tl_data in data.get("task_logs", []):
@@ -1403,32 +1141,6 @@ async def _restore_v2(data: dict) -> dict:
))
stats["rule_relations"] += 1
# A rule's edit history (milestone 323). Must come after the rules
# themselves — rule_id_map is only populated above — and both ids are
# ids in the SOURCE database, which is #3182's arose_from_id trap.
for rv in data.get("rule_versions", []):
mapped_rid = rule_id_map.get(rv.get("rule_id", 0))
if mapped_rid is None:
continue
# Unlike NoteVersion, an unmappable user does NOT drop the row.
# user_id is the ACTOR and is nullable by design: the column is
# SET NULL precisely so history outlives the account that wrote
# it. Skipping here would delete the record the FK preserves.
session.add(RuleVersion(
rule_id=mapped_rid,
user_id=user_id_map.get(rv.get("user_id") or 0),
title=rv.get("title", ""),
statement=rv.get("statement", ""),
why=rv.get("why"),
how_to_apply=rv.get("how_to_apply"),
when_to_apply=rv.get("when_to_apply"),
tier=rv.get("tier"),
verify_with=rv.get("verify_with"),
expires_when=rv.get("expires_when"),
created_at=_dt(rv.get("created_at")),
))
stats["rule_versions"] += 1
# 15. Systems
for sy_data in data.get("systems", []):
mapped_uid = user_id_map.get(sy_data.get("user_id", 0))
@@ -1533,25 +1245,6 @@ async def _restore_v2(data: dict) -> dict:
))
stats["note_usage_events"] += 1
# The rule twin — and the reason it is a separate table at all.
# Resolved through rule_id_map, NOT note_id_map. A rule id run through
# the note map would either drop (best case) or land on whatever note
# took that number, producing telemetry that is wrong rather than
# missing and that nothing downstream could detect (milestone 333).
# Must come after the rules themselves; rule_id_map is populated there.
for ev in data.get("rule_usage_events", []):
mapped_rid = rule_id_map.get(ev.get("rule_id", 0))
if mapped_rid is None:
continue
session.add(RuleUsageEvent(
user_id=user_id_map.get(ev.get("user_id") or 0),
rule_id=mapped_rid,
event=ev.get("event", ""),
source=ev.get("source", ""),
created_at=_dt(ev.get("created_at")),
))
stats["rule_usage_events"] += 1
# 20. Repo bindings — small, but losing them means every bound repo
# quietly stops loading its project at session start.
for rb_data in data.get("repo_bindings", []):
@@ -1562,7 +1255,6 @@ async def _restore_v2(data: dict) -> dict:
session.add(RepoBinding(
user_id=mapped_uid, project_id=mapped_pid,
repo_key=rb_data.get("repo_key", ""),
ref=rb_data.get("ref"),
))
stats["repo_bindings"] += 1
+13 -151
View File
@@ -344,52 +344,6 @@ def chunk_document(title: str | None, body: str | None) -> list[str]:
return chunks
async def _claim_parent_row(session, id_column, row_id: int, label: str) -> bool:
"""Lock the record a vector belongs to BEFORE rewriting that vector (#3262).
An embedding write and a cascading delete of the same record take the same
two row locks in OPPOSITE orders. The embedder deletes the old chunk rows
and then, on INSERT, needs the foreign key's lock on the parent; a delete
of the parent — or of the rulebook, topic or project above it — locks the
parent first and cascades down into the chunk rows. That is a cycle, and
Postgres breaks it by killing one side at random: sometimes the embedding
write, which is swallowed and invisible, and sometimes the operator's
delete, which surfaces as a 500 on an operation that should have worked.
Claiming the parent first REMOVES the cycle rather than narrowing it.
Either the embedder arrives first and the delete waits its turn behind it,
or the delete already holds the row and NOWAIT makes the embedder lose at
once. The embedder is the side that should lose: a skipped refresh costs a
stale vector until the next write or the startup backfill, and the other
outcome costs a person their request.
FOR KEY SHARE, not FOR UPDATE — it is precisely the lock the INSERT's
foreign key would take anyway, so it conflicts with a delete of the parent
and with nothing else. An ordinary edit of the same record, or a second
refresh racing this one, is unaffected.
Returns False when the row is locked or already gone; the caller skips.
"""
try:
held = (await session.execute(
select(id_column)
.where(id_column == row_id)
# BOTH flags: SQLAlchemy spells the four Postgres row locks as a
# read/key_share pair, and key_share alone is FOR NO KEY UPDATE —
# which would make two refreshes of one record fight each other.
.with_for_update(read=True, key_share=True, nowait=True)
)).scalar_one_or_none()
except Exception:
# LockNotAvailable: this record is being deleted right now. Not an
# error — the delete wins by design.
logger.debug("Skipping embedding for %s %d — row is being deleted", label, row_id)
return False
if held is None:
logger.debug("Skipping embedding for %s %d — row is gone", label, row_id)
return False
return True
async def upsert_note_embedding(
note_id: int, user_id: int, title: str | None, body: str | None
) -> None:
@@ -426,8 +380,6 @@ async def upsert_note_embedding(
try:
async with async_session() as session:
if not await _claim_parent_row(session, Note.id, note_id, "note"):
return
await session.execute(
delete(NoteEmbedding).where(NoteEmbedding.note_id == note_id)
)
@@ -448,23 +400,6 @@ async def upsert_note_embedding(
logger.warning("Failed to persist embedding for note %d", note_id, exc_info=True)
# Both searches rank WITHOUT the threshold and apply it in Python, so the best
# rejected score stays observable (#3670). The qualifying set is provably
# unchanged: rows arrive ordered by distance ascending, so every above-bar row
# sorts ahead of every below-bar one, and an over-fetch that used to return N
# above-bar rows returns the same N plus some losers. What changes is only that
# the losers are now visible instead of discarded inside the query.
#
# That visibility is the entire point. A bar can only be judged from the calls
# it TURNED AWAY — a 0.72 bar rejecting a stream of 0.71s is set too high by a
# hair, one rejecting 0.30s is working — and those two are indistinguishable
# from any arrangement of the columns that survive the filter.
#
# `report` is how the score gets out without changing what a search RETURNS.
# Eight of the eleven call sites want hits and nothing else; the three that
# write telemetry pass a dict and read `best_available_score` back out of it.
async def semantic_search_notes(
user_id: int,
query: str,
@@ -479,27 +414,12 @@ async def semantic_search_notes(
scope: str = "own",
demote_superseded: bool = True,
system_id: int | None = None,
report: dict | None = None,
) -> list[tuple[float, Note]]:
"""Return up to *limit* (score, note) pairs most relevant to *query*.
Scores are cosine similarities in [-1, 1]; only notes at or above
*threshold* are returned, sorted highest-first.
Pass `report` (an empty dict) to learn what the threshold turned away:
the function sets `report["best_available_score"]` to the highest score
anything reached, or None when the corpus offered nothing at all. It is
the only figure that survives a call returning nothing, and therefore the
only one a bar can be judged from (#3670).
It also sets `report["searched"]`: False before anything can return, True
only where a real result set exists. So an empty query, an unavailable
embedder and a failed database query all leave it FALSE, and a caller can
tell a search that found nothing from one that never ran. A caller logging
telemetry must check it — recording a failed search as a zero-result call
reports a decline the ranker never made (#3765). ABSENT means no search
touched the dict at all, which is a stand-in in a test, not a real call.
`note_type` narrows to a record kind, or several (e.g. "snippet", or
("snippet", "note")), for callers that want prior art rather than everything
embedded.
@@ -534,13 +454,6 @@ async def semantic_search_notes(
Returns an empty list if the embedder is unavailable or on any error.
"""
# Stamped FALSE before anything can return, flipped True only where a real
# result set exists (#3765). Every early return below leaves it false, so a
# caller can tell a search that found nothing from one that never ran. It
# has to be the first thing done to `report`: a return added above this
# line would leave the key ABSENT, which reads as "no caller asked".
if report is not None:
report["searched"] = False
if not query or not query.strip():
return []
try:
@@ -552,6 +465,7 @@ async def semantic_search_notes(
# Distance ceiling equivalent to the similarity floor. Clamp to the valid
# cosine-distance range [0, 2] so a threshold of, say, -1 doesn't produce a
# nonsensical ceiling.
max_distance = min(2.0, max(0.0, 1.0 - threshold))
distance = NoteEmbedding.embedding.cosine_distance(query_vec)
try:
@@ -626,10 +540,11 @@ async def semantic_search_notes(
fetch = limit * _CHUNK_OVERFETCH * (
_SUPERSESSION_OVERFETCH if demote_superseded else 1
)
# NO threshold predicate — see the note above this function. The
# bar is applied after the collapse, where the rejected scores can
# still be seen.
stmt = stmt.order_by(distance.asc()).limit(fetch)
stmt = (
stmt.where(distance <= max_distance)
.order_by(distance.asc())
.limit(fetch)
)
rows = list((await session.execute(stmt)).all())
except Exception:
logger.warning("Failed to query note embeddings", exc_info=True)
@@ -648,23 +563,6 @@ async def semantic_search_notes(
continue
seen.add(int(note.id))
scored.append((1.0 - float(dist), note))
# The best score anything reached, bar or no bar. Recorded BEFORE the
# filter because a call that returns nothing is exactly when it matters.
if report is not None:
# `searched` is what stops a null score meaning four things (#3765).
# Every early return above — empty query, embedder down, and the broad
# `except` around the query itself — leaves this key ABSENT, so a
# caller can tell "I looked and there was nothing" from "I never
# looked" and from "the query failed". Set here, at the one point past
# which a real result set exists.
report["searched"] = True
# ONE unpack, so the score and the id cannot describe different records
# (#3807). Splitting these into two expressions is how a later edit
# pairs a score with its neighbour's id.
best = scored[0] if scored else None
report["best_available_score"] = best[0] if best else None
report["best_available_id"] = int(best[1].id) if best else None
scored = [pair for pair in scored if pair[0] >= threshold]
if not demote_superseded:
return scored[:limit]
return await _apply_supersession_penalty(scored, limit)
@@ -768,8 +666,6 @@ async def upsert_rule_embedding(
replacement is atomic per rule so a concurrent read sees the old chunk set
or the new one, never a mixture.
"""
from scribe.models.rulebook import Rule # runtime import: see TYPE_CHECKING above
doc_title, doc_body = rule_document(title, statement, when_to_apply)
chunks = chunk_document(doc_title, doc_body)
try:
@@ -792,8 +688,6 @@ async def upsert_rule_embedding(
try:
async with async_session() as session:
if not await _claim_parent_row(session, Rule.id, rule_id, "rule"):
return
await session.execute(
delete(RuleEmbedding).where(RuleEmbedding.rule_id == rule_id)
)
@@ -818,24 +712,9 @@ async def semantic_search_rules(
limit: int = 5,
threshold: float = _SIMILARITY_THRESHOLD,
tier: str | None = None,
report: dict | None = None,
) -> list[tuple[float, "Rule"]]:
"""Return up to *limit* (score, rule) pairs most relevant to *query*.
Pass `report` (an empty dict) to learn what the threshold turned away:
the function sets `report["best_available_score"]` to the highest score
anything reached, or None when the corpus offered nothing at all. It is
the only figure that survives a call returning nothing, and therefore the
only one a bar can be judged from (#3670).
It also sets `report["searched"]`: False before anything can return, True
only where a real result set exists. So an empty query, an unavailable
embedder and a failed database query all leave it FALSE, and a caller can
tell a search that found nothing from one that never ran. A caller logging
telemetry must check it — recording a failed search as a zero-result call
reports a decline the ranker never made (#3765). ABSENT means no search
touched the dict at all, which is a stand-in in a test, not a real call.
Scoped by OWNERSHIP — a rule is the caller's if they own its rulebook or
its project. Deliberately not filtered to what currently BINDS a given
project: this answers "is there a rule about this", which a person asking
@@ -843,17 +722,10 @@ async def semantic_search_rules(
is the surfacing question, and it has its own machinery
(get_applicable_rules) rather than a second, subtly different copy here.
`tier` narrows to one tier, and NONE is the ordinary case. The write-path
and pre-tool hints deliberately pass nothing: an always-on rule is already
in the session, but being in a list from turn zero is not the same as being
in front of the reader when the action it governs is taken, and filtering
on tier made a whole class of rules permanently ineligible for the one
mechanism that surfaces a rule AT the moment. Relevance is the threshold's
job; see the block above RULEHINT_LIMIT in services/plugin_context.py for
the argument and for what the resulting scores are being read against.
Pass a tier when a caller genuinely wants one class — a listing, an audit,
a UI that renders the tiers apart. Not to approximate relevance.
`tier` narrows to one tier. The write-path hint passes "conditional",
because an always-on rule is ALREADY in the session — surfacing it again as
a suggestion is pure noise, and noise on a hint that fires on every write
is how a hint gets ignored.
Collapses to best-chunk-per-rule like the note search, so a long rule split
across chunks competes once rather than crowding the results with itself.
@@ -863,9 +735,6 @@ async def semantic_search_rules(
from scribe.models.project import Project
from scribe.models.rulebook import Rule, Rulebook, RulebookTopic
# See the sibling search: stamped before anything can return (#3765).
if report is not None:
report["searched"] = False
if not query or not query.strip():
return []
try:
@@ -874,6 +743,7 @@ async def semantic_search_rules(
logger.debug("Rule search skipped — embedder unavailable")
return []
max_distance = min(2.0, max(0.0, 1.0 - threshold))
distance = RuleEmbedding.embedding.cosine_distance(query_vec)
try:
@@ -887,8 +757,7 @@ async def semantic_search_rules(
.outerjoin(Project, Rule.project_id == Project.id)
.where(
Rule.deleted_at.is_(None),
# No threshold predicate — see the note above
# semantic_search_notes. Applied below, after the collapse.
distance <= max_distance,
# topic_id XOR project_id, so exactly one arm can match.
or_(
Rulebook.owner_user_id == user_id,
@@ -911,14 +780,7 @@ async def semantic_search_rules(
if rule.id not in best or score > best[rule.id][0]:
best[rule.id] = (score, rule)
ranked = sorted(best.values(), key=lambda pair: pair[0], reverse=True)
if report is not None:
# See the sibling search: absent means the search never ran (#3765).
report["searched"] = True
# One unpack — see the sibling search (#3807).
best = ranked[0] if ranked else None
report["best_available_score"] = best[0] if best else None
report["best_available_id"] = int(best[1].id) if best else None
return [pair for pair in ranked if pair[0] >= threshold][:limit]
return ranked[:limit]
async def backfill_rule_embeddings() -> None:
+60 -117
View File
@@ -1,4 +1,4 @@
"""Knowledge service — one query across every record kind Scribe holds.
"""Knowledge service — unified query across notes, tasks, plans, and processes.
ACL (rules #47/#78, decision note 2094): these queries were owner-only until
2026-07-25, which meant a record shared with you could be opened by id but never
@@ -265,94 +265,22 @@ def _note_to_item(note: Note) -> dict:
return item
# What each type facet MEANS, once, for every arm that has to know.
#
# The vocabulary spans BOTH typing axes — `note_type` for non-task records and
# `task_kind` for tasks — so a facet cannot be a filter on one column, which is
# why this is a table rather than a chain of ifs. Each entry is
# (is_task, the value pinned on that axis); None pins nothing, i.e. every task.
#
# It is a table because the alternative had already gone wrong. The predicate
# was written three times — a SQL if-chain, a Python if-chain over semantic
# candidates, and a ternary computing the `is_task` pre-filter — and the three
# only agreed by luck. Adding `issue` to the SQL arm alone (the obvious edit,
# and the one #3128 was about to make) would have set the pre-filter to
# is_task=False, handed the Python arm a candidate set containing no tasks at
# all, and returned an empty semantic half for the Issues facet forever, with
# nothing red anywhere. A new facet is now one row here.
#
# `plan` is retired (0066) but kept: 90 legacy plan-tasks exist and a facet
# they answer to costs one line. It simply has no chip in the UI any more.
_FACETS: dict[str, tuple[bool, str | None]] = {
"task": (True, None),
"work": (True, "work"),
"issue": (True, "issue"),
"spike": (True, "spike"),
"plan": (True, "plan"),
"note": (False, "note"),
"process": (False, "process"),
"snippet": (False, "snippet"),
}
# The non-task record types, for the counts query. Derived so it cannot drift
# from the table above.
NON_TASK_FACETS = tuple(
value for _is_task, value in _FACETS.values() if not _is_task and value
)
# The whole vocabulary, for the door's request validation — public so the route
# validates against the same table the query reads instead of a hand-kept copy.
FACET_TYPES = frozenset(_FACETS)
# An unrecognised facet resolves to "a non-task note whose note_type is that
# string" — which matches nothing, since no row stores an unknown type. That is
# the behaviour the old if-chain had by falling through, and it is the right
# one: a typo should return an empty list, never the whole corpus.
def _facet(note_type: str) -> tuple[bool, str | None]:
return _FACETS.get(note_type, (False, note_type))
def facet_is_task(note_type: str | None) -> bool | None:
"""The `is_task` pre-filter a facet implies — None when it spans both.
Used to narrow the semantic candidate set before it is fetched. Reads the
same table `_apply_type_filter` and `matches_facet` read, so the pre-filter
can no longer disagree with the predicate it is meant to anticipate.
"""
if not note_type:
return None
return _facet(note_type)[0]
def matches_facet(note, note_type: str | None) -> bool:
"""The Python dialect of `_apply_type_filter`, for candidates the vector
search has already fetched — there is no query left to narrow.
Generated from the same table, so this is a translation rather than a
second implementation. Note the `not note.is_task` arm: the hand-written
version omitted it and was saved only by the upstream pre-filter.
"""
if not note_type:
return True
is_task, value = _facet(note_type)
if is_task:
return note.is_task and (value is None or note.task_kind == value)
return not note.is_task and note.note_type == value
def _apply_type_filter(stmt, note_type: str | None):
"""Apply the type facet to a Note select. Trashed rows are always excluded."""
"""Apply the type facet to a Note select.
'task' = any task (status not null); 'plan' = a task with task_kind='plan';
any other non-empty type = a non-task note of that note_type; None = all.
Trashed rows (deleted_at set) are always excluded.
"""
stmt = stmt.where(Note.deleted_at.is_(None))
if not note_type:
return stmt
is_task, value = _facet(note_type)
if is_task:
stmt = stmt.where(Note.status.isnot(None))
if value is not None:
stmt = stmt.where(Note.task_kind == value)
return stmt
return stmt.where(Note.status.is_(None)).where(Note.note_type == value)
if note_type == "task":
return stmt.where(Note.status.isnot(None))
if note_type == "plan":
return stmt.where(Note.status.isnot(None)).where(Note.task_kind == "plan")
if note_type:
return stmt.where(Note.note_type == note_type).where(Note.status.is_(None))
return stmt
async def query_knowledge(
@@ -367,7 +295,7 @@ async def query_knowledge(
locations: dict[str, str] | None = None,
verification: str = "",
) -> tuple[list[dict], int]:
"""Query knowledge objects with filters.
"""Query knowledge objects (non-task notes) with filters.
`project_id` narrows to one project (None = every project).
@@ -496,7 +424,7 @@ async def _semantic_knowledge_search(
INTERACTIVE_SEARCH_THRESHOLD,
semantic_search_notes,
)
is_task_filter = facet_is_task(note_type)
is_task_filter = True if note_type in ("task", "plan") else (False if note_type else None)
import time as _time
_t0 = _time.perf_counter()
candidates = await semantic_search_notes(
@@ -526,7 +454,11 @@ async def _semantic_knowledge_search(
for _score, note in candidates:
if note.deleted_at is not None:
continue
if not matches_facet(note, note_type):
if note_type == "task" and not note.is_task:
continue
elif note_type == "plan" and (not note.is_task or note.task_kind != "plan"):
continue
elif note_type and note_type not in ("task", "plan") and note.note_type != note_type:
continue
if tags and not all(t in (note.tags or []) for t in tags):
continue
@@ -582,40 +514,51 @@ async def get_knowledge_counts(user_id: int, tags: list[str] | None = None) -> d
search would surface."""
visible = browsable_notes_clause(user_id)
async with async_session() as session:
def _scoped(stmt):
stmt = stmt.where(visible).where(Note.deleted_at.is_(None))
for tag in tags or []:
stmt = stmt.where(Note.tags.contains([tag]))
return stmt
# One grouped query per typing axis. The task half used to be a count
# for 'task' plus a second count for 'plan', which is why 'issue' —
# 17% of every task here — had no number to show: each kind needed its
# own query and nobody added one. Grouping by task_kind counts every
# kind, including ones added later, for the same two round-trips.
non_task = _scoped(
# Count non-task types
stmt = (
select(Note.note_type, func.count(Note.id))
.where(visible)
.where(Note.status.is_(None))
.where(Note.note_type.in_(NON_TASK_FACETS))
).group_by(Note.note_type)
counts = {t: n for t, n in (await session.execute(non_task)).all()}
.where(Note.deleted_at.is_(None))
.where(Note.note_type.in_(["note", "process"]))
.group_by(Note.note_type)
)
if tags:
for tag in tags:
stmt = stmt.where(Note.tags.contains([tag]))
rows = list((await session.execute(stmt)).all())
counts = {row[0]: row[1] for row in rows}
by_kind = _scoped(
select(Note.task_kind, func.count(Note.id))
# Count tasks separately (is_task = status IS NOT NULL)
task_stmt = (
select(func.count(Note.id))
.where(visible)
.where(Note.status.isnot(None))
).group_by(Note.task_kind)
kind_counts = {k: n for k, n in (await session.execute(by_kind)).all()}
.where(Note.deleted_at.is_(None))
)
if tags:
for tag in tags:
task_stmt = task_stmt.where(Note.tags.contains([tag]))
task_count: int = (await session.execute(task_stmt)).scalar_one()
counts["task"] = task_count
# Kinds are SUBSETS of 'task' and are deliberately left out of the total —
# adding them would count every task twice.
counts["task"] = sum(kind_counts.values())
for kind, value in _FACETS.items():
if value[0] and value[1] is not None:
counts[kind] = kind_counts.get(kind, 0)
# Plans are a subset of tasks (task_kind='plan'); counted for the facet
# but NOT added to total to avoid double-counting against "task".
plan_stmt = (
select(func.count(Note.id))
.where(visible)
.where(Note.status.isnot(None))
.where(Note.task_kind == "plan")
.where(Note.deleted_at.is_(None))
)
if tags:
for tag in tags:
plan_stmt = plan_stmt.where(Note.tags.contains([tag]))
counts["plan"] = (await session.execute(plan_stmt)).scalar_one()
for t in NON_TASK_FACETS:
for t in ("note", "task", "plan", "process"):
counts.setdefault(t, 0)
counts["total"] = counts["task"] + sum(counts[t] for t in NON_TASK_FACETS)
counts["total"] = sum(counts[t] for t in ("note", "task", "process"))
return counts
+28 -10
View File
@@ -30,13 +30,13 @@ from __future__ import annotations
import asyncio
import logging
import traceback
from sqlalchemy import case, func, select
from scribe.models import async_session
from scribe.models.note_usage import PULLED, SURFACED, NoteUsageEvent
from scribe.models.base import iso
from scribe.services.background import report_telemetry_failure
logger = logging.getLogger(__name__)
@@ -46,19 +46,37 @@ logger = logging.getLogger(__name__)
# never lands. The done-callback discard keeps the set from growing.
_pending: set[asyncio.Task] = set()
# Sites that already dropped their once-per-process AppLog row. The readout
# runs on every snippet list render — without this, a broken table would turn
# the error log into a firehose that buries the finding it exists to surface.
_reported: set[str] = set()
async def _report_failure(site: str) -> None:
"""This subsystem's canary, now the shared one.
"""Make a swallowed telemetry failure visible. Called from an except block.
The per-site dedup, the WARNING and the single AppLog row all moved to
`background.report_telemetry_failure` unchanged when `rule_usage` needed
the identical behaviour — two hand-kept copies of a thing whose whole job
is to be reliable is the wrong number. `retrieval_telemetry` deliberately
still has its own: its canary is a different shape (one process-wide flag,
no AppLog row), so repointing it would change behaviour rather than
consolidate it.
WARNING to the process log every time; one AppLog error row per process per
site so the admin UI shows the outage without host access. The AppLog write
is itself guarded — when the whole database is down it fails too, and that
is fine: the WARNING already said so, and a canary must never take down the
surface it watches.
"""
await report_telemetry_failure("note_usage", site)
logger.warning("note usage telemetry %s failed", site, exc_info=True)
if site in _reported:
return
_reported.add(site)
try:
from scribe.services.logging import log_error
await log_error(
endpoint="note_usage",
error_type=f"note_usage_{site}_failed",
error_message=f"note usage telemetry {site} is failing; "
"usage counters will read zero until this is fixed",
traceback=traceback.format_exc(),
)
except Exception:
logger.debug("note usage canary write failed", exc_info=True)
async def _insert_events(rows: list[dict]) -> None:
+1 -265
View File
@@ -1,6 +1,5 @@
import logging
import re
from collections.abc import Iterable
from datetime import date, datetime, timezone
from sqlalchemy import func, or_, select, text
@@ -10,52 +9,6 @@ from scribe.models.note import Note, TaskKind, TaskPriority, TaskStatus
logger = logging.getLogger(__name__)
# The fields `snippets.parse_snippet_fields` reads. Writing any of them can
# change what a snippet's derived `data` mirror should say, so update_note
# recomposes the mirror when one moves. Kept here as a set of NAMES rather
# than imported, because it describes update_note's own `fields` dict, not the
# parser's signature.
_PARSED_FROM_BODY = frozenset({"title", "body", "tags"})
# Text fields where EMPTY MEANS NULL (milestone 317). The sweep's whole signal
# is `verify_with IS NULL` = "this is a decision, there is nothing to go and
# check". An empty string that is not NULL makes a norm look like a constraint
# nobody has verified, forever — and it would sit at the top of the sweep,
# since never-checked sorts first. Sibling of rulebooks.NULLABLE_RULE_TEXT.
NULLABLE_NOTE_TEXT = ("verify_with", "expires_when")
def guard_check_fields(status: str | None, note_type: str | None) -> None:
"""Raise unless a record in this shape may carry verify_with/expires_when.
Stated as an INVARIANT over the resulting record rather than a filter on
which fields a caller passed, so it also catches the sideways route: a
checked note being turned into a task, which no per-field gate would see.
Raises rather than dropping silently, for minted_kind's reason (#3129) — a
silently-corrected write is the defect that reasoning exists to end, and a
caller putting a check on the wrong record has an idea an error corrects
and a default hides. Lives at the service, not either door, so REST and MCP
cannot come to disagree about it.
"""
if status is not None:
raise ValueError(
"a task cannot carry verify_with/expires_when: a task's decay is "
"its status, and a done issue records what happened rather than "
"asserting something that can later go false. Put the check on the "
"note the fact lives in, or clear the check before making this a "
"task."
)
if note_type == "snippet":
raise ValueError(
"a snippet already has a check: verify_snippet(), which compares "
"the recorded location and code against the repo and expires its "
"own verdict when the code moves. verify_with/expires_when are the "
"free-text form, for prose notes that assert a fact about "
"something outside the repo."
)
def embed_note(note) -> None:
"""Refresh a note's embedding, fire-and-forget.
@@ -76,10 +29,6 @@ def embed_note(note) -> None:
exceptions are swallowed because a record that saved must not fail on its
index refresh. No running loop (unit tests, scripts) is an ordinary case,
not an error.
Detaching also means this task races anything that deletes the note out
from under it. That is not handled here: `upsert_note_embedding` claims
the note's row before touching its vectors, and loses if it can't (#3262).
"""
try:
import asyncio
@@ -152,16 +101,7 @@ async def create_note(
task_kind: str = "work",
arose_from_id: int | None = None,
data: dict | None = None,
verify_with: str | None = None,
expires_when: str | None = None,
) -> Note:
# Empty means empty (NULLABLE_NOTE_TEXT), then the invariant. Both run
# before anything is written, so an illegal shape never reaches the table.
verify_with = verify_with or None
expires_when = expires_when or None
if verify_with or expires_when:
guard_check_fields(status, note_type)
# Validate status/priority here so the MCP create_task path (which passes
# them straight through) can't persist an out-of-enum value that the REST
# route would have rejected — there's no DB CHECK on notes.status.
@@ -205,8 +145,6 @@ async def create_note(
task_kind=task_kind,
arose_from_id=arose_from_id,
data=data,
verify_with=verify_with,
expires_when=expires_when,
)
session.add(note)
await session.commit()
@@ -459,20 +397,7 @@ def minted_kind(kind: str) -> str:
raise ValueError(f"kind must be one of {MINTABLE_KINDS}, got {kind!r}")
async def update_note(
user_id: int, note_id: int, clear: Iterable[str] = (), **fields: object,
) -> Note | None:
"""Partial update. `clear` names fields to UNSET; **fields carries values.
Clearing is explicit and separate because a nullable field cannot be
emptied by passing it: the MCP door reads "" as "leave this alone", so an
agent filling two fields does not wipe the others, and a note that stops
being a constraint genuinely needs its check removed. Naming the field is
the one form that cannot happen by accident. The REST door, where a
cleared form input arrives as "", reaches the same place through the
NULLABLE_NOTE_TEXT normalisation below — two idioms, one outcome.
(Same shape as rulebooks.update_rule, milestone 312 step 2.)
"""
async def update_note(user_id: int, note_id: int, **fields: object) -> Note | None:
async with async_session() as session:
result = await session.execute(
select(Note).where(Note.id == note_id, Note.user_id == user_id)
@@ -484,10 +409,6 @@ async def update_note(
old_body = note.body
old_title = note.title
old_tags = list(note.tags or [])
check_before = note.verify_with
for key in clear:
if key in NULLABLE_NOTE_TEXT:
setattr(note, key, None)
for key, value in fields.items():
if not hasattr(note, key):
continue
@@ -517,44 +438,7 @@ async def update_note(
)
elif key == "tags" and isinstance(value, list):
value = _normalize_tags(value)
elif key in NULLABLE_NOTE_TEXT:
value = value or None
elif key == "verified_at":
# Not settable here. A stamp says somebody performed THIS
# check, so it is written by the verification path and by a
# restore, never by an ordinary edit that could mint one for a
# check nobody ran.
continue
setattr(note, key, value)
# The invariant, over the RESULTING record rather than over what was
# passed — which is what catches a checked note being turned into a
# task. Raised before commit, so nothing is persisted.
if note.verify_with or note.expires_when:
guard_check_fields(note.status, note.note_type)
# A stamp certifies A CHECK, not a record. Rewrite or remove the check
# and the old stamp certifies something that no longer exists, so it is
# dropped and the note re-enters the sweep. The safe direction: a note
# wrongly listed as due costs one look; a note wrongly vouched for
# costs exactly what the sweep exists to catch.
if note.verify_with != check_before:
note.verified_at = None
# A snippet's `data` is DERIVED from its body — so a write that moves
# the body through this generic door must move the mirror with it
# (#3128). Without this, PATCH /api/notes/<snippet_id> {body} left the
# mirror behind, and snippet_fields PREFERS the mirror: the row went on
# reporting its old repo/path/symbol to prior-art recall while showing
# its new body. `update_snippet` composes the mirror itself and passes
# it explicitly, so an explicit `data` always wins — the caller that
# knows the field set beats the one that can only re-read the body.
if "data" not in fields and not _PARSED_FROM_BODY.isdisjoint(fields):
# Imported here, not at module scope: services/snippets.py calls
# back into this module (update_snippet -> update_note), so a
# top-level import is a cycle.
from scribe.services.snippets import (
SNIPPET_NOTE_TYPE, recompose_data,
)
if note.note_type == SNIPPET_NOTE_TYPE:
note.data = recompose_data(note)
# Auto-set lifecycle timestamps on status transitions
if "status" in fields:
_now = datetime.now(timezone.utc)
@@ -603,154 +487,6 @@ async def update_note(
# permanent deletion. Both are reachable; neither is spelled `delete_note`.
# ── The sweep (milestone 317 step 3) ─────────────────────────────────────────
#
# A SIBLING of rulebooks.rules_due_for_verification, not a shared
# implementation, and deliberately so (note 3163). The row could have been
# shared; the QUERY cannot. That sweep scopes by rulebook ownership XOR project
# ownership because rules have no sharing ACL at all — no rule_shares, no
# can_read_rule. A note scopes by the note ACL, which is a different question
# with a different answer. What IS common — how a stamp reads, how old it is —
# lives in services/verification.py and is imported by both.
async def notes_due_for_verification(
user_id: int,
older_than_days: int = 0,
project_id: int | None = None,
never_only: bool = False,
) -> list[Note]:
"""Notes that carry a check, oldest verification first, never-checked top.
THE QUERY THE COLUMNS EXIST FOR. `verify_with` and `expires_when` are
storage; this is what turns them into something that gets acted on.
Without it, note decay is caught only when a human reads the note and
disagrees — which is the case where the note was already believed.
Ordered `verified_at` ASC **NULLS FIRST**: never-checked outranks
checked-long-ago, because a note nobody has ever confirmed is a claim with
no evidence behind it at all. Postgres sorts NULLs LAST on ASC by default,
so this is explicit — and getting it wrong would not error, it would
silently invert the one signal the sweep exists to carry.
Notes with no `verify_with` never appear. Not an omission: they are
decisions, there is nothing to go and check, and listing them would dilute
the result until nobody reads it.
Scoped with `browsable_notes_clause`, NOT the read scope (decision note
2094): a sweep is a passive surface, and a record shared one-to-one must
not arrive in one unasked.
Deliberately NOT filtered to non-task, non-snippet records even though the
write path (step 2) permits a check on nothing else. A row in that state
would be a row in an ILLEGAL state, and this is the one surface that could
tell somebody about it. Hiding it here to match the invariant would make
the sweep agree with a database it had stopped describing.
Args:
user_id: whose notes.
older_than_days: only notes last verified longer ago than this.
Never-checked notes always qualify — they are the most overdue
thing there is. 0 = no age filter. Negative raises: it would mean
"everything", which is a different question than the one asked,
answered silently.
project_id: narrow to one project. None = every project.
never_only: only notes that have never been verified.
"""
from datetime import timedelta
from scribe.services.access import browsable_notes_clause
if older_than_days < 0:
raise ValueError(
f"older_than_days must be >= 0, got {older_than_days}. A negative "
f"window silently means 'everything', which is not what any caller "
f"of a staleness sweep is asking."
)
async with async_session() as session:
# One statement, not a fetch-then-filter: the ordering below is the
# database's, so it cannot disagree with itself across two halves.
stmt = (
select(Note)
.where(
browsable_notes_clause(user_id),
Note.deleted_at.is_(None),
Note.verify_with.is_not(None),
)
)
if project_id is not None:
stmt = stmt.where(Note.project_id == project_id)
if never_only:
stmt = stmt.where(Note.verified_at.is_(None))
elif older_than_days > 0:
cutoff = datetime.now(timezone.utc) - timedelta(days=older_than_days)
stmt = stmt.where(
or_(Note.verified_at.is_(None), Note.verified_at < cutoff)
)
stmt = stmt.order_by(Note.verified_at.asc().nullsfirst(), Note.id)
return list((await session.execute(stmt)).scalars().all())
def verification_row(note: Note) -> dict:
"""One row of the sweep — the CHECK in full, unlike a listing.
The opposite call from a browse: here the caller is about to go and run the
check, so the text they need IS the payload rather than the bloat.
"""
from scribe.services.verification import (
days_since_verified,
last_verified_label,
)
return {
"id": note.id,
"title": note.title,
"project_id": note.project_id,
"verify_with": note.verify_with or "",
"expires_when": note.expires_when or "",
"last_verified": last_verified_label(note),
"days_since_verified": days_since_verified(note),
}
async def mark_note_verified(
note_id: int, user_id: int, still_true: bool = True,
) -> Note | None:
"""Stamp a note as verified — or, when the check FAILED, refuse to.
The asymmetry is the design: passing writes a stamp, failing writes
nothing. There is no "verified false" state, because a note whose check
failed is not a note in a special condition — it is a note that is WRONG,
and the honest resolutions are to correct it, supersede it, or find out
why. Recording the failure as a flag would let it sit there being false
with the sweep quietly satisfied that somebody had looked.
So a failed check leaves `verified_at` untouched and the note stays at the
top of the sweep until someone actually deals with it.
Write access, not read (rules 47/78): stamping is a mutation, and an
editor-share holder may make it while a viewer may not.
Returns None when the note is not found, not writable, or carries no
`verify_with` — nothing to verify is a different answer from verified.
"""
from scribe.services.access import can_write_note
async with async_session() as session:
note = (await session.execute(
select(Note).where(Note.id == note_id, Note.deleted_at.is_(None))
)).scalars().first()
if note is None or not note.verify_with:
return None
if not await can_write_note(user_id, note_id):
return None
if still_true:
note.verified_at = datetime.now(timezone.utc)
await session.commit()
await session.refresh(note)
return note
async def get_all_tags(user_id: int, q: str | None = None) -> list[str]:
async with async_session() as session:
if q:
+1 -1
View File
@@ -60,7 +60,7 @@ async def start_planning(user_id: int, project_id: int, title: str) -> dict:
return {
"milestone": milestone.to_dict(),
**rulebooks_svc.rules_payload(applicable, user_id=user_id, source="start_planning"),
**rulebooks_svc.rules_payload(applicable),
"project_goal": getattr(project, "goal", "") or "",
"open_task_count": open_count,
}
+25 -547
View File
@@ -32,7 +32,6 @@ from scribe.services import snippets as snippets_svc
from scribe.services.access import label_shared_items, owner_names_for
from scribe.services.embeddings import semantic_search_notes, semantic_search_rules
from scribe.services.note_usage import record_surfaced
from scribe.services.rule_usage import record_rule_surfaced
from scribe.services.supersession import superseded_ids
from scribe.services.retrieval_telemetry import record_retrieval
from scribe.services.settings import get_setting
@@ -86,148 +85,6 @@ WRITEPATH_THRESHOLD_KEY = "kb_writepath_threshold"
WRITEPATH_DEFAULT_ENABLED = True
WRITEPATH_DEFAULT_THRESHOLD = 0.68
# The standing-rule arm (milestone 307) gets its own bar — the split #2223 made
# one surface down, now made for the THIRD corpus. It inherited 0.68 above, and
# that number was measured against code-vs-note-PROSE. It was never re-derived
# for code-vs-RULE-TEXT.
#
# THE STRUCTURAL ARGUMENT, which is the only kind admissible here (rule 115).
# Two facts hold on any install, including one with six rules and no telemetry:
#
# 1. The eligible corpus is SMALL — every rule an install owns, still only
# a few dozen documents against thousands of notes. A top-k over a small
# pool always returns something, so "the best match cleared the bar"
# drifts from "a good match exists" toward "N things were ranked". A bar
# calibrated for best-of-thousands is cleared by best-of-forty as
# arithmetic rather than relevance.
# This argument WEAKENED when the arms stopped filtering to one tier
# (see the note on that below): a larger pool makes clearing the bar
# mean more, not less. The threshold was deliberately left where it was
# anyway — moving two variables at once would make the resulting
# distribution unreadable, and this one errs toward silence on purpose.
# 2. Rules are short imperative technical English — a far more HOMOGENEOUS
# corpus than note prose. #2223 measured the floor for code against prose
# at 0.55-0.63 and set 0.68 above it. A more homogeneous corpus has a
# HIGHER floor, so 0.68 is not merely inherited, it is below where this
# corpus's noise sits.
#
# WHY 0.72 AND NOT A NUMBER OFF A HISTOGRAM. The exact offset between prose's
# floor and rule-text's is not derivable in general — it depends on how an
# install writes its rules — so the default errs deliberately toward SILENCE
# rather than toward recall, on an asymmetry that is itself structural: this
# hint fires on EVERY write. A missed rule is recoverable, because the rule is
# still in Scribe and the agent can search it. A hint that cries wolf is not:
# it teaches the reader to skip the whole block, and the surface is lost along
# with the true positives it would have carried. The arm's own comment already
# says "noise on a hint that fires on every write is how a hint gets ignored".
#
# TUNE IT FROM YOUR OWN INSTANCE, which is now possible: `retrieval_telemetry`
# reports `rule_usage.pull_through` (milestone 333 step 3). Raise this if rules
# arrive unread; lower it if rules you needed never arrived. What would RETIRE
# it: a cross-encoder rerank (#1038), which would make a similarity bar the
# wrong control entirely.
RULEHINT_THRESHOLD_KEY = "kb_rulehint_threshold"
RULEHINT_DEFAULT_THRESHOLD = 0.72
# ONE rule per write, not two — and this is deliberately NOT a knob.
#
# With a corpus this small, top-k does as much damage as the threshold: k=2
# over a few dozen candidates means the second line is almost always the
# second-best noise, arriving with the same confident framing as the first.
# Halving k halves that regardless of where the bar sits.
#
# It also BOUNDS the blast radius of widening the pool (below): with k=1 a
# wider corpus can change WHICH rule surfaces and how often one does, but it
# can never make a single hint longer. The loudness of one hint and the
# eligibility of a rule are separate controls, and only one of them moved.
#
# It stays a constant because it is a decision about how LOUD one hint may be,
# not a per-install tuning question. The hint already carries prior art, shape
# signals and staleness; rules are the fourth voice in it, and a fourth voice
# that speaks twice is where a reader stops reading. Nothing suggests an
# operator wants this different, and a knob nobody turns is a knob that only
# adds a way to misconfigure the surface (rule 25 cuts both ways).
RULEHINT_LIMIT = 1
# AND A REPEAT COMPETES FOR THAT ONE SLOT ON RANK ALONE (#3750).
#
# Since #3750 a hit already on the session's exclusion ledger is RENDERED
# rather than dropped, which raises a question the old behaviour never had to
# answer: when the top-ranked hit is one the session has already seen, does it
# take the slot, or step aside for a fresh rule behind it?
#
# It takes the slot, and nothing is fetched behind it. Two reasons.
#
# RANK IS THE ANSWER TO "WHAT IS RELEVANT NOW". If the repeat scores 0.85 and
# the best fresh candidate 0.73, the repeat is the better match for the action
# actually being taken. Overfetching in order to promote the fresh one past it
# would reinstate exactly the withholding this milestone exists to remove, one
# rank deeper and harder to see — recency is not a reason to show a worse
# match, and "you have seen this" is not the same claim as "you are holding
# this".
#
# AND A SECOND LINE IS THE ONE THING THE LIMIT ABOVE FORBIDS. Letting a repeat
# ride alongside a fresh rule means two hint lines, and the paragraph above is
# entirely about why a fourth voice that speaks twice is where a reader stops
# reading. A reference costs the same ~40 tokens as a first surfacing, so
# "it is only a short extra line" is not available as an argument: the budget
# is one line because of what a second line does to the whole hint, not
# because of what it costs.
#
# The consequence is deliberate and worth naming: a rule that keeps ranking
# first for a recurring situation keeps being referenced, every time the
# situation recurs. That is the intended behaviour — the situation recurring
# IS the trigger — and its decay belongs to exclusion ageing (#3751), not to
# a rule that ranks first being quietly demoted for having won before.
# WHY THE ARMS NO LONGER FILTER TO ONE TIER (#3702).
#
# Both arms used to pass `tier="conditional"`, on the reasoning that an
# always-on rule is already in the session, so surfacing it again is pure
# noise. That reasoning conflates two different things:
#
# PRESENT IN CONTEXT — the rule was delivered at session start.
# SALIENT AT THE MOMENT — the rule is in front of the reader when the
# action it governs is about to be taken.
#
# A rule handed over in a list at turn zero is present while a session writes
# a config value three hundred turns later. It is not surfaced. So the filter
# did not merely skip a redundant hint — it made a whole class of rules
# permanently ineligible for the only mechanism that puts a rule in front of
# an agent AT the moment, and the more important a rule is, the more likely
# it was in that class.
#
# The deeper defect is that the filter was doing the THRESHOLD's job. Whether
# a rule belongs in this hint is a relevance question, and a similarity bar is
# the control for relevance. A categorical exclusion standing in for a
# relevance judgment cannot be tuned, cannot be measured, and cannot be wrong
# in a way anybody notices.
#
# THIS IS A MEASURED CHANGE, NOT A SETTLED ONE. The old comment's fear is
# real — a hint that fires on every write and says obvious things teaches the
# reader to skip the block, and the surface is then lost along with its true
# positives. That fear had simply never been checked. `retrieval_logs` already
# records top_score, result_count and the query for every call, so the
# evidence now arrives on its own:
#
# - rules clear the bar often and at high scores -> the fear was justified,
# the filter was a crude proxy for a bar set too low, and the WORK IS THE
# BAR. Any reinstated filter should then carry a measured reason.
# - rules clear rarely, in a thin band near the bar -> the filter was never
# the right instrument and relevance was always sufficient.
#
# Only the eligibility moved. The bar and k=1 were both left exactly where
# they were, so the resulting distribution has one cause.
# How much of a command reaches the embedding (#3476). A shell call is not a
# file: most are short, and the ones that are not are usually a heredoc or a
# pasted script whose bulk says nothing about which rule applies. The VERB AND
# ITS TARGET sit at the front — `curl https://git.fabledsword.com/api/...`,
# `docker compose up`, `git checkout -b` — and that head is the whole signal.
# Sending the tail as well would push it out of a 512-token window and let a
# heredoc's prose decide the match.
_TOOL_QUERY_CHARS = 400
# Minimum SUBSTANCE (non-whitespace chars) a payload must carry before the
# semantic arm will run at all — the cheap half of the operator's #89 idea
# ("a sliding scale between number of characters and semantic threshold").
@@ -494,7 +351,6 @@ async def _reserve_slot_for_reuse(
top_k = cfg["top_k"]
_t0 = time.perf_counter()
_rep: dict = {}
reuse = await semantic_search_notes(
user_id, query,
limit=1,
@@ -503,7 +359,6 @@ async def _reserve_slot_for_reuse(
exclude_ids=exclude_ids | {int(n.id) for _s, n in kept},
note_type=_REUSE_KINDS,
scope="browse",
report=_rep,
)
# A real semantic query competing for a menu slot — logged like the scored
# arm it displaces. Before this, the hit it PUSHED OUT was in
@@ -514,9 +369,6 @@ async def _reserve_slot_for_reuse(
user_id=user_id, source="reuse_slot", query=query,
threshold=cfg["threshold"], limit=1, project_id=project_id,
is_task=None, results=reuse,
best_available=_rep.get("best_available_score"),
best_available_id=_rep.get("best_available_id"),
searched=bool(_rep.get("searched", True)),
duration_ms=(time.perf_counter() - _t0) * 1000.0,
)
# Verify the kind rather than trusting the query that asked for it, and
@@ -566,7 +418,6 @@ async def build_autoinject_hint(
return empty
t0 = time.perf_counter()
_rep_ai: dict = {}
hits = await semantic_search_notes(
user_id, q,
limit=cfg["top_k"],
@@ -578,15 +429,11 @@ async def build_autoinject_hint(
# still appear is a collaborator's note inside a shared project — legible
# only because the line below names its owner.
scope="browse",
report=_rep_ai,
)
record_retrieval(
user_id=user_id, source="auto_inject", query=q,
threshold=cfg["threshold"], limit=cfg["top_k"],
project_id=(project_id or None), is_task=None, results=hits,
best_available=_rep_ai.get("best_available_score"),
best_available_id=_rep_ai.get("best_available_id"),
searched=bool(_rep_ai.get("searched", True)),
duration_ms=(time.perf_counter() - t0) * 1000.0,
)
if not hits:
@@ -843,67 +690,12 @@ async def get_writepath_config(user_id: int) -> dict:
threshold = WRITEPATH_DEFAULT_THRESHOLD
threshold = min(1.0, max(0.0, threshold))
try:
rule_threshold = float(await get_setting(
user_id, RULEHINT_THRESHOLD_KEY, str(RULEHINT_DEFAULT_THRESHOLD)))
except (TypeError, ValueError):
rule_threshold = RULEHINT_DEFAULT_THRESHOLD
rule_threshold = min(1.0, max(0.0, rule_threshold))
return {
**cfg,
"enabled": enabled_raw.strip().lower() in ("true", "1", "yes", "on"),
"threshold": threshold,
# Its own bar, for a third corpus — see RULEHINT_DEFAULT_THRESHOLD.
"rule_threshold": rule_threshold,
}
def _rule_hint_line(rule, *, where: str, seen: bool) -> str:
"""One rule hint line — both arms, both tails (#3750).
ONE FUNCTION BECAUSE THE TAILS MUST NOT DRIFT. The two arms phrase their
heads differently ("may apply here" vs "may apply to this Bash call") and
that difference is deliberate. Everything after it must not differ, and
#3497's history is that the pre-tool arm inherited a defect from its
sibling by being modelled on it rather than sharing with it. Two copies of
a two-branch string is how one branch gets fixed and the other does not.
WHY A REPEAT GETS A LINE AT ALL. Both arms used to drop a hit whose id was
already on the session's exclusion ledger and emit nothing. That is correct
only while the session still HOLDS what it was told, and a compaction
breaks exactly that: the earlier injection is summarized away while the id
stays on the ledger, so the rule is absent from context AND unreachable for
the rest of the session (#3749 closes the compaction half; this closes the
ordinary half, where a session simply stops holding a line it read an hour
ago).
Only ONE CLAUSE of the original line is false on a repeat — the claim that
the rule is not in the session's loaded set. So only that clause changes.
Title, trigger and pull pointer are identical either way, the statement is
never injected either way, and a repeat therefore costs the same ~40 tokens
as a first surfacing and no more.
DELIBERATELY NOT ASKING THE SESSION WHETHER IT HOLDS THE RULE. A model
asked "do you still hold rule 156?" will say yes, and the claim is
unverifiable self-report about its own context. The answer is also not
needed: the line is cheap enough to always emit and carries its own remedy
in both branches. Removing the question removes the fragility rather than
managing it.
"""
trigger = (rule.when_to_apply or "").strip()
tail = (
f"You saw it earlier this session; pull it with get_rule({rule.id}) "
"if you no longer hold it."
if seen else
f"Read it with get_rule({rule.id}) before deciding it does not "
"apply; it is not in this session's loaded set."
)
return (
f"Standing rule that may apply {where} \u2014 \u201c{rule.title}\u201d"
+ (f" ({trigger})" if trigger else "")
+ f". {tail}"
)
async def build_write_path_hint(
user_id: int,
@@ -916,7 +708,6 @@ async def build_write_path_hint(
repo_key: str = "",
exclude_derive: list[str] | None = None,
exclude_rule_ids: list[int] | None = None,
rules_etag: str = "",
) -> dict:
"""Prior-art hint for the plugin's PreToolUse hook on Write/Edit.
@@ -1059,7 +850,6 @@ async def build_write_path_hint(
# Pulled-and-seen ids stay in the query (as evidence) but never in
# the menu — the dedup contract holds, the resemblance still lands.
pulled_seen = seen & set(pulled)
_rep_wp: dict = {}
hits = await semantic_search_notes(
user_id, query,
limit=remaining + len(pulled_seen),
@@ -1083,40 +873,12 @@ async def build_write_path_hint(
# Same reasoning as auto-inject: nobody asked for this, so it takes
# the browse scope and never surfaces a one-to-one direct share.
scope="browse",
report=_rep_wp,
)
resembles = {
int(note.id): float(score) for score, note in hits
if int(note.id) in pulled
}
shown = [(s, n) for s, n in hits if int(n.id) not in seen]
# WHAT THIS ARM WITHHELD AFTER THE SEARCH ANSWERED, and the reason
# `best_available_score` cannot always be reported here (#3739 again,
# from the side its fix did not reach).
#
# This arm is the one note arm that filters TWICE. `exclude_ids` takes
# `seen - pulled_seen` into the search, but the pulled-and-seen ids stay
# in the query deliberately — `resembles` above needs them — and are
# dropped in the line above instead. So the score the search reported is
# PRE that drop while the row's `result_count` is POST it, and a record
# the session had already been shown could be logged as something the
# BAR turned away. Live proof on the first read after #3739 shipped:
# write_path's near-miss max was 0.822 while the lowest score it ever
# RETURNED was 0.6857 — a "rejection" that beat every acceptance.
#
# The suppression column cannot rescue it the way it does for the rule
# arms: this arm's count would be PARTIAL, covering only the drops made
# here and not the ones `exclude_ids` made inside the search, and a
# partial number under a name that reads as complete is the substitution
# this whole milestone exists to stop.
#
# So the honest answer is null — "not measured on this call" — whenever
# this filter removed anything, because then the bar is not the only
# thing that turned something away and the reported score may belong to
# a record we withheld ourselves. Calls where nothing was dropped keep
# reporting it, which is most of them.
withheld_here = len(hits) - len(shown)
hits = shown[:remaining]
hits = [(s, n) for s, n in hits if int(n.id) not in seen][:remaining]
record_retrieval(
user_id=user_id, source="write_path", query=query,
threshold=cfg["threshold"], limit=remaining,
@@ -1124,16 +886,6 @@ async def build_write_path_hint(
# recording it as a notes-only retrieval would misdescribe the
# candidate set the threshold is being tuned against.
project_id=scope_project, is_task=None, results=hits,
best_available=(
None if withheld_here else _rep_wp.get("best_available_score")
),
# Withheld on the SAME condition as the score. A surviving id
# beside a null score would name a record without saying what it
# scored, which is the pair disagreeing in the other direction.
best_available_id=(
None if withheld_here else _rep_wp.get("best_available_id")
),
searched=bool(_rep_wp.get("searched", True)),
duration_ms=(time.perf_counter() - t0) * 1000.0,
)
if hits:
@@ -1199,74 +951,7 @@ async def build_write_path_hint(
derive = [d for d in found if d.get("key") not in skip]
except Exception:
logger.warning("write-time derive check failed", exc_info=True)
staleness: list[str] = []
# ── Have the rules moved under this session? (milestone 323) ───────
#
# THE CARRIER IS THE POINT. This hook already fires before a write — the
# moment acting on a stale rule actually costs something — and the check
# is one comparison against a marker the session already holds. No
# payload, no extra round trip, and nothing said when nothing moved.
#
# WHAT THIS CANNOT SEE, and a reader who finds an etag here will assume
# otherwise:
#
# what goes wrong | caught?
# ---------------------------------------------------|--------
# another session edits a rule mid-flight | yes
# the session is misremembering a rule read hours ago | yes
# compaction summarised the rules out of context | NO
#
# The third is the most common and this is blind to it: the etag was in
# context too and went with the rules. The SessionStart nudge is that
# case's only mechanism and must not be softened because this shipped.
#
# Fails open, like every other arm here: a staleness hint must never
# break a write.
if rules_etag:
try:
current = await rulebooks_svc.list_always_on_rules(
user_id, project_id=project_id or 0,
)
if rulebooks_svc.rules_etag(current) != rules_etag:
moved = rulebooks_svc.rules_moved_since(current, rules_etag)
held = rulebooks_svc.etag_count(rules_etag)
bits = []
if moved:
named = ", ".join(
f"#{r.id} \u201c{r.title}\u201d" for r in moved[:3]
)
more = len(moved) - 3
bits.append(
f"{named}" + (f", and {more} more" if more > 0 else "")
)
# A DELETED rule moves no timestamp and leaves no row to name,
# so the count is the only thing that can report the one change
# that takes an instruction OUT of force.
if held is not None and held != len(current):
delta = len(current) - held
bits.append(
f"{abs(delta)} rule(s) {'added' if delta > 0 else 'no longer in force'}"
)
if bits:
staleness.append(
"Your loaded rules have changed since this session "
"started — " + "; ".join(bits) + ". Re-read them with "
"list_always_on_rules() before relying on the set you "
"are holding."
)
except Exception:
logger.debug("write-path rules-etag arm failed", exc_info=True)
# The guard sits BELOW the staleness arm on purpose. A rules change is
# unconditional news — it does not become less true because this
# particular write happened to match no prior art — and this arm is one
# indexed query, only when the session actually sent a marker.
#
# The standing-rule arm further down is deliberately left on the far side
# of this guard: that one runs a SEMANTIC search, and moving it here would
# run an embedding query on every write in the session. Its gating is a
# separate question from this one (see the note on #3244).
if not staleness and not synced and not menu and not stamped and not divergence and not derive:
if not synced and not menu and not stamped and not divergence and not derive:
return empty
owners = await owner_names_for({
@@ -1285,9 +970,7 @@ async def build_write_path_hint(
for marker, item in menu:
rendered.append((item, marker, _owner_of(item), _foreign_language(item, target_lang)))
# Seeded with the staleness line, which is decided above the early
# return and so cannot wait for this list to exist.
lines: list[str] = list(staleness)
lines: list[str] = []
sync_note_ids: list[int] = []
if synced:
# The sync framing (#2708). Deliberately imperative about the record —
@@ -1364,94 +1047,37 @@ async def build_write_path_hint(
# resembles what is being written, noticed at the moment it is relevant
# rather than by being resident in every session.
#
# EVERY TIER, since #3702 — see the note at RULEHINT_LIMIT. This comment
# used to read "CONDITIONAL ONLY: an always-on rule is already in the
# session, so repeating it here would be noise". That conflated being
# PRESENT in context with being SALIENT at the moment the action is taken,
# and it is the same conflation #3750 corrects one layer up: a rule the
# session was told about an hour ago is not a rule in front of the reader
# now.
# CONDITIONAL ONLY. An always-on rule is already in the session; repeating
# it here would be noise, and noise on a hint that fires on every write is
# how a hint gets ignored.
#
# Fails open like every other arm: a rule hint must never break a write.
rule_ids: list[int] = []
try:
already = set(exclude_rule_ids or [])
# Timed like the notes arm above. Without this the rule row was the one
# source in the whole readout reporting a null p90_duration_ms (#3311)
# — a gap that reads as "this surface is somehow not measurable" rather
# than "nobody passed the number".
rule_t0 = time.perf_counter()
_rep_wpr: dict = {}
hits = await semantic_search_rules(
user_id, code or path, limit=RULEHINT_LIMIT,
threshold=cfg["rule_threshold"],
report=_rep_wpr,
user_id, code or path, limit=2,
threshold=cfg["threshold"], tier="conditional",
)
rule_ms = (time.perf_counter() - rule_t0) * 1000.0
fresh = [(score, rule) for score, rule in hits if rule.id not in already]
# EVERY hit gets a line; `already` only changes the tail (#3750).
for _score, rule in hits:
for _score, rule in fresh:
trigger = (rule.when_to_apply or "").strip()
lines.append(
_rule_hint_line(rule, where="here", seen=rule.id in already)
f"Standing rule that may apply here — \u201c{rule.title}\u201d"
+ (f" ({trigger})" if trigger else "")
+ f". Read it with get_rule({rule.id}) before deciding it "
"does not apply; it is not in this session's loaded set."
)
# `rule_ids` stays FRESH-ONLY, and that is the whole telemetry story of
# this change (#3752). It is what the hook writes to the exclusion
# ledger and what `record_rule_surfaced` counts; a referenced rule is
# already on the ledger by definition, and counting it as a surfacing
# would inflate pull_through's denominator with a choice this arm never
# made. A reference is a RENDERING decision, not a retrieval outcome.
rule_ids.extend(rule.id for _score, rule in fresh)
# TWO tables, and the split is not arbitrary. retrieval_logs is one
# row per CALL, keyed on the score distribution a threshold is tuned
# from. rule_usage_events is one row per RULE per event, which is the
# grain "was this hint ever acted on" needs and the grain a JSONB
# result_ids array cannot be indexed at.
#
# This comment used to say rule ids had nowhere to go — that
# note_usage_events remaps ids on restore, so a rule id there would
# return attached to whatever note took that number. That is still
# true of the NOTE table, and it is exactly why rule_usage_events is
# its own (milestone 333 step 1). The gap it described is closed.
#
# THE CALL LOG IS UNCONDITIONAL; THE SURFACING LOG IS NOT, and the
# asymmetry is the correction #3497 exists to make. Both used to sit
# inside an `if fresh:`, which is how this arm came to report
# `zero_result_calls: 0` and `cleared_threshold: 133/133` — not a
# perfectly tuned surface but one structurally unable to record its
# own misses. #3311 read that artifact as a measurement and a whole
# milestone was scoped on it. A call that found nothing is the ONLY
# evidence a threshold is set too high, and it is the row every note
# surface has always written (write_path: 421 zeroes of 613 calls;
# auto_inject: 114 of 326). A SURFACING is different in kind: nothing
# was shown, so no such event occurred, and its log stays guarded.
#
# `results=fresh`, not `hits`: the note arms pass their exclusions
# INTO semantic_search_notes, so what they log is already
# post-exclusion. semantic_search_rules takes no such parameter and
# this filter is where the equivalent happens — logging `hits` would
# quietly make this row mean something other than every other row in
# the same readout.
record_retrieval(
user_id=user_id, source="write_path_rule", query=code or path,
threshold=cfg["rule_threshold"], limit=RULEHINT_LIMIT,
project_id=project_id,
is_task=None, results=fresh, duration_ms=rule_ms,
best_available=_rep_wpr.get("best_available_score"),
best_available_id=_rep_wpr.get("best_available_id"),
searched=bool(_rep_wpr.get("searched", True)),
# What the ranker found and this session had already been told.
# Without it a zero row cannot say whether the bar was too high or
# the reader was simply ahead of it — and only the first is a
# reason to move the threshold.
suppressed=len(hits) - len(fresh),
)
rule_ids.append(rule.id)
if fresh:
# `rule_ids` is `fresh`, i.e. AFTER exclude_rule_ids. A rule the
# session already holds was considered and not shown, and counting
# it would inflate the denominator with claims the agent never saw
# — which reads as a precision problem this arm does not have.
record_rule_surfaced(
user_id=user_id, rule_ids=rule_ids, source="write_path_rule",
# retrieval_logs, NOT note_usage_events: that table's ids are
# remapped on a backup restore, so a rule id there would return
# attached to whatever note took that number. This one is never
# restored, and `source` already separates the surfaces.
record_retrieval(
user_id=user_id, source="write_path_rule", query=code or path,
threshold=cfg["threshold"], limit=2, project_id=project_id,
is_task=None, results=fresh,
)
except Exception:
logger.debug("write-path rule arm failed", exc_info=True)
@@ -1469,125 +1095,6 @@ async def build_write_path_hint(
}
async def build_tool_rule_hint(
user_id: int,
tool_name: str,
command: str,
*,
project_id: int = 0,
exclude_rule_ids: list[int] | None = None,
) -> dict:
"""Standing rules that may apply to the ACTION about to be taken (#3476).
The sibling of the write-path rule arm, and the surface that was missing.
That arm is keyed on `code or path`, so a rule can only be retrieved at the
moment of a code WRITE. Every rule about which tool to reach for — don't
curl the forge, don't stand up a stack, don't run the suite locally, don't
branch — was therefore unreachable at the moment it mattered, and residency
in the always-on preload was the only surface it had.
WHY A MECHANICAL TRIGGER AND NOT AN INSTRUCTION. Note #3089's finding is
that a reflex generates no query: you reach for `curl` confidently, with no
moment of doubt, so any surface that waits to be asked never fires. Here
nothing has to be asked — the tool call IS the query, and the reflex has to
become a tool call before it can do anything.
Deliberately TOOL-AGNOSTIC: takes a name and a string. The hook decides
which tools it watches, so widening the matcher is a `hooks.json` edit with
no change here.
EVERY TIER, since #3702 — the tier filter this docstring used to describe
is gone from both arms, for the reason recorded at RULEHINT_LIMIT: present
in context and salient at the moment are different properties, and only the
second is what this arm is for.
Fails open and returns an empty context on any error: a recall aid may
never break the operator's action.
"""
out: dict = {"context": "", "rule_ids": []}
command = (command or "").strip()
if not command:
return out
try:
cfg = await get_writepath_config(user_id)
if not cfg.get("enabled"):
return out
# The command text is the query. A long heredoc or a pasted script
# would otherwise push the meaningful head of the command out of the
# embedding window, so it is bounded — the verb and its target sit at
# the front, which is the part a rule is about.
query = command[:_TOOL_QUERY_CHARS]
t0 = time.perf_counter()
_rep_ptr: dict = {}
hits = await semantic_search_rules(
user_id, query, limit=RULEHINT_LIMIT,
threshold=cfg["rule_threshold"],
report=_rep_ptr,
)
duration_ms = (time.perf_counter() - t0) * 1000.0
already = set(exclude_rule_ids or [])
fresh = [(score, rule) for score, rule in hits if rule.id not in already]
# Logged BEFORE the early return, for the reason spelled out at length
# on the write-path arm above: a call that found nothing is the only
# evidence a threshold is too high, and an arm that logs only the calls
# it liked reports a flawless clear-rate however badly it is tuned.
# This arm shipped with the same defect inherited from its sibling, and
# it mattered more here — a surface with no rows at all cannot be told
# apart from a hook that never fired, which is precisely the silent
# failure the arm was built to stop.
record_retrieval(
user_id=user_id, source="pre_tool_rule", query=query,
threshold=cfg["rule_threshold"], limit=RULEHINT_LIMIT,
project_id=project_id,
is_task=None, results=fresh, duration_ms=duration_ms,
best_available=_rep_ptr.get("best_available_score"),
best_available_id=_rep_ptr.get("best_available_id"),
searched=bool(_rep_ptr.get("searched", True)),
# See the sibling arm. It matters more here: this arm fires on every
# Bash call, so a long session excludes its way to an all-zero row
# and the threshold looks wrong when nothing about it is.
suppressed=len(hits) - len(fresh),
)
# `hits`, not `fresh` (#3750). A call whose only hit is a repeat still
# has something to say — the arm just says it differently.
if not hits:
return out
lines = [
_rule_hint_line(
rule, where=f"to this {tool_name} call",
seen=rule.id in already,
)
for _score, rule in hits
]
# FRESH-ONLY, for the reason given on the sibling arm: a reference is a
# rendering decision, not a retrieval outcome, and counting it here
# would inflate the denominator pull_through is read from.
rule_ids = [rule.id for _score, rule in fresh]
# RANKED, not ambient: this arm chose what it showed, so a pull can
# settle whether the choice was any good. `rule_usage.RANKED_SOURCES`
# carries the same name.
#
# GUARDED, which it did not need to be before #3750: `fresh` can now be
# empty on a call that still emitted a line, and recording a surfacing
# of nothing would write an event with no rules in it.
if rule_ids:
record_rule_surfaced(
user_id=user_id, rule_ids=rule_ids, source="pre_tool_rule",
)
out["context"] = "\n".join(lines)
out["rule_ids"] = rule_ids
except Exception:
logger.debug("pre-tool rule arm failed", exc_info=True)
return out
def _derive_line(path: str, derive: list[dict]) -> str:
"""The ledger's word on the names being written (#2900): a duplicate
family to derive, or a canon to reuse — said at the write."""
@@ -1701,10 +1208,7 @@ async def build_session_context(
its normalized key — triggers a one-line "bind this repo" hint so
the binding is self-healing.
Returns {"context": str, "rule_count": int, "project": dict | None,
"rules_etag": str}. The etag is for the HOOK, not for the model — the
hook stores it and hands it back on each write so the server can say
whether these rules have moved since the session loaded them.
Returns {"context": str, "rule_count": int, "project": dict | None}.
`context` is markdown ready to drop into `additionalContext`; it is capped
at _MAX_CHARS with an explicit truncation note so the hook can pass it
through verbatim.
@@ -1713,24 +1217,6 @@ async def build_session_context(
# exclusion (milestone 297) takes a rulebook out of this block, and is
# named below so the departure is visible rather than silent.
rules = await rulebooks_svc.list_always_on_rules(user_id, project_id=project_id)
# AMBIENT source, and the one that matters most: this is the preload — the
# block every session opens with, chosen by nobody, paid for every turn.
#
# It emitted nothing until 2026-09-03, which made the resident set's cost
# certain and its usefulness unfalsifiable at the same time (#3473). Note
# #3089 is the argument this measurement finally lets someone test: that a
# rule arriving with thirty others, none of them relevant, is read as
# preamble rather than as a claim — so presence is not surfacing, and a
# tier-1 set can grow without anybody noticing it stopped working.
#
# Recorded even when the hook truncates the block below: the rules WERE
# delivered, and counting only the untruncated ones would quietly shrink
# the denominator exactly where the set is too big to read.
record_rule_surfaced(
user_id=user_id,
rule_ids=[r.id for r in rules],
source="session_start",
)
excluded = (
await rulebooks_svc.excluded_always_on_rulebooks(user_id, project_id)
if project_id else []
@@ -1831,12 +1317,4 @@ async def build_session_context(
if len(context) > _MAX_CHARS:
context = context[:_MAX_CHARS].rstrip() + "\n\n…(truncated — call list_always_on_rules())"
return {
"context": context,
"rule_count": len(rules),
"project": project_dict,
# Computed from the rules THIS payload was built from, not re-queried:
# the marker has to describe the set the session is actually holding,
# and a second query could disagree with the first.
"rules_etag": rulebooks_svc.rules_etag(rules),
}
return {"context": context, "rule_count": len(rules), "project": project_dict}
+19 -586
View File
@@ -27,10 +27,6 @@ from scribe.models import async_session
from scribe.models.base import iso
from scribe.models.note import Note
from scribe.models.note_usage import PULLED, SURFACED, NoteUsageEvent
from scribe.models.rule_usage import PULLED as RULE_PULLED
from scribe.models.rule_usage import SURFACED as RULE_SURFACED
from scribe.models.rule_usage import RuleUsageEvent
from scribe.services.rule_usage import is_ambient
from scribe.models.retrieval_log import RetrievalLog
logger = logging.getLogger(__name__)
@@ -55,27 +51,12 @@ def _build_payload(
is_task: bool | None,
results: list[tuple[float, Note]],
duration_ms: float | None,
suppressed: int | None = None,
best_available: float | None = None,
best_available_id: int | None = None,
) -> dict:
"""Reduce a retrieval call to a flat, JSON-safe RetrievalLog payload.
Pure and synchronous (no DB, no event loop) so it is unit-testable and safe
to run inline before scheduling the write. `results` is the
`(score, Note)` list from semantic_search_notes, already highest-first.
`suppressed` is how many scored hits the caller dropped because the session
had already been shown them, and it stays None for callers that cannot
know. See the column's comment: None means "not measured here", which is a
different fact from 0 and must never render as one.
`best_available` is the highest score the ranker reached BEFORE the
threshold, and it carries the same null discipline for a sharper reason: it
is the only field that still says something on a call that returned
nothing, so a 0.0 standing in for "not measured" would read as "the corpus
held nothing remotely relevant" — a claim about the corpus invented out of
a caller's silence.
"""
items = [
{"id": int(note.id), "score": round(float(score), 5), "rank": rank}
@@ -91,16 +72,8 @@ def _build_payload(
"project_id": project_id,
"is_task": is_task,
"result_count": len(items),
"suppressed_count": (None if suppressed is None else int(suppressed)),
"top_score": (scores[0] if scores else None),
"min_score": (scores[-1] if scores else None),
"best_available_score": (
None if best_available is None else round(float(best_available), 5)
),
# The record that scored it, so a reader can go and look (#3807).
"best_available_id": (
None if best_available_id is None else int(best_available_id)
),
"result_ids": items,
"duration_ms": (round(duration_ms, 2) if duration_ms is not None else None),
}
@@ -138,10 +111,6 @@ def record_retrieval(
is_task: bool | None,
results: list[tuple[float, Any]],
duration_ms: float | None = None,
suppressed: int | None = None,
best_available: float | None = None,
best_available_id: int | None = None,
searched: bool = True,
) -> None:
"""Fire-and-forget: record one retrieval call.
@@ -153,35 +122,9 @@ def record_retrieval(
provide. retrieval_logs is not restored at all, so it has no such hazard,
and `source` already distinguishes the surfaces.
`searched=False` WRITES NO ROW, and that is the point rather than an
optimisation. A semantic search has three ways to return nothing without
having run — an empty query, an unavailable embedder, and the broad
`except` around the query itself — and each one currently arrives here
looking exactly like a ranker that declined. Logging it would report a
decline nobody made, drag `zero_result_calls` down with phantom evidence
about a threshold, and leave `best_available_score` null for a reason that
has nothing to do with the corpus. That last ambiguity is #3765: the field
added to judge a bar was null on four unrelated causes, one of them a
swallowed failure, and no reader could tell them apart.
Dropping the row is what makes the remaining nulls mean ONE thing —
"searched, and there was nothing".
The same convention already governs the pre-tool arm: a blank command costs
no embedding query, so it writes no row, because "a row here would report a
call that never happened and drag the clear-rate down with phantom
declines". This extends it from a case the caller could see in advance to
the ones only the search knows about.
A FAILURE IS NOT MADE INVISIBLE BY THIS. `semantic_search_notes` logs a
WARNING on a query failure, which is where a broken search belongs — a
counter cannot say "I am broken" without a reader already trusting it.
Builds the payload inline (synchronously) then schedules the insert so the
caller returns immediately. Never raises — telemetry must not affect search.
"""
if not searched:
return
try:
payload = _build_payload(
user_id=user_id,
@@ -193,9 +136,6 @@ def record_retrieval(
is_task=is_task,
results=results,
duration_ms=duration_ms,
suppressed=suppressed,
best_available=best_available,
best_available_id=best_available_id,
)
except Exception:
logger.debug("retrieval telemetry payload build failed", exc_info=True)
@@ -222,147 +162,36 @@ def record_retrieval(
def _bucket(rows: list) -> dict:
"""A score readout a human can act on, from one aggregate row."""
(calls, zero, p10, p50, p90, lo, hi, avg_n, dur,
measured, supp_calls, supp_zero,
miss_calls, miss_p50, miss_p90, miss_max) = rows
calls, zero, cleared, p10, p50, p90, lo, hi, avg_n, dur = rows
return {
"calls": int(calls or 0),
# A call that returned nothing is not a low-scoring call — it is a
# different failure (nothing indexed, filter too narrow), and averaging
# it into the score distribution would hide both.
"zero_result_calls": int(zero or 0),
# `cleared_threshold` USED TO LIVE HERE and it was a tautology (#3670).
# The search applies the bar before returning, so every returned result
# cleared it by construction and a call with nothing has no score to
# compare — the condition was true exactly when `result_count > 0`.
# `zero_result_calls + cleared_threshold == calls` held on all nineteen
# readings ever taken. It was `calls - zero_result_calls` wearing a name
# that promised a second opinion, and the docstring built a reading
# procedure on it that asked the reader to compare a number with itself.
# Its replacement is `near_misses` below, which the bar cannot fix by
# construction because it is measured on the calls the bar REJECTED.
# Of the zeros above, which were the RANKER declining and which were
# the reader having seen it already? `zero_result_calls` cannot say,
# and only the first kind is evidence about the threshold.
#
# None — not a zeroed dict — when no row in the window reported it. A
# surface that filters inside the search genuinely does not know, and
# rendering that as `{"calls": 0}` would state a measurement nobody
# made. That substitution is the whole of #3311.
"suppression": (
None if not int(measured or 0) else {
"measured_calls": int(measured or 0),
"calls_with_suppression": int(supp_calls or 0),
# Subtract from zero_result_calls for the true ranker declines.
"zero_because_already_shown": int(supp_zero or 0),
}
),
# How often the best hit actually cleared the threshold in force for
# that call. THE precision-adjacent number: a surface that clears its
# bar on almost every call is either well-tuned or too loose, and the
# score spread below says which.
"cleared_threshold": int(cleared or 0),
"top_score": {
"p10": _round(p10), "p50": _round(p50), "p90": _round(p90),
"min": _round(lo), "max": _round(hi),
},
# WHAT THE BAR TURNED AWAY, and the only figure here a threshold can
# actually be tuned from. Measured over the calls that returned
# NOTHING, on the best score the ranker reached before the filter.
#
# Read `p90` against the threshold in force. A bar at 0.72 rejecting a
# stream of 0.71s is set too high by a hair and the surface is losing
# hits it should have had; the same bar rejecting 0.30s is doing its
# job and the corpus simply had nothing. Both render as a zero-result
# call, and nothing else in this readout separates them.
#
# None — not a zeroed block — when no declining call in the window
# measured it. Old rows predate the column, and a 0.0 would assert that
# the corpus held nothing relevant, which is a claim about the corpus
# invented out of a caller's silence.
"near_misses": (
None if not int(miss_calls or 0) else {
"measured_calls": int(miss_calls or 0),
"p50": _round(miss_p50),
"p90": _round(miss_p90),
"max": _round(miss_max),
}
),
"avg_result_count": _round(avg_n),
"p90_duration_ms": _round(dur, 1),
}
# The aggregate row Postgres would have returned for a source with no rows in
# the window: nothing counted, nothing scored. Positional, matching the SELECT
# `_bucket` unpacks — calls, zero, p10, p50, p90, min, max, avg_n,
# dur, measured, supp_calls, supp_zero, miss_calls, miss_p50, miss_p90,
# miss_max. The counts are 0 because zero calls is a real observation;
# everything else is None because a distribution nobody sampled has no value,
# and rendering it as 0.0 would state one.
_NO_ROWS_IN_WINDOW = [0, 0, None, None, None, None, None, None, None,
0, 0, 0, 0, None, None, None]
def _round(v, places: int = 4):
return None if v is None else round(float(v), places)
async def _complete_from(session, model, user_id) -> dict[str, Any]:
"""When each source in `model` started being recorded, and the instant the
WHOLE table is complete from. Returns {source: earliest_row, "*": latest}.
THE GRAIN IS THE SOURCE, and that is the whole point. `retrieval_logs` has
rows going back months, so a table-level "earliest row" says months and
tells a reader their window is fully covered — while a source added last
week has a week of rows and a counter that silently means something else.
Per-source is the only grain at which partial coverage is visible.
THE AGGREGATE USES THE LATEST, NOT THE EARLIEST. A number that sums several
sources is complete only once EVERY contributor was recording, so "*" is a
max over the sources, not a min. Taking the min here would reproduce the
exact reading this exists to prevent: the oldest source vouching for the
youngest.
All-time, deliberately unfiltered by the window — a query bounded by
`since` can only ever report something at or after `since`, which answers
nothing.
"""
rows = (
await session.execute(
select(model.source, func.min(model.created_at))
.where(model.user_id == user_id)
.group_by(model.source)
)
).all()
out: dict[str, Any] = {src: ts for src, ts in rows if ts is not None}
stamps = list(out.values())
out["*"] = max(stamps) if stamps else None
return out
def _coverage(complete_from, since) -> dict:
"""The two keys every counter block carries, from one timestamp.
`covers_window` is None — never False — when nothing was ever recorded.
"No rows at all" is not "partial coverage", it is no measurement, and the
null convention #3497 established for `suppression` holds here for the
same reason: absent must not read as a verdict.
"""
return {
# iso() already returns None for an unset value (#2845) — the guard
# belongs on covers_window, which is a verdict, not a serialisation.
"complete_from": iso(complete_from),
"covers_window": (
None if complete_from is None else complete_from <= since
),
}
async def retrieval_summary(
user_id: int | None, *, days: int = 30, near_miss_samples: int = 0,
) -> dict:
async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
"""What the retrieval telemetry says, per surface, over a window.
Three aggregates side by side, each read from the table built for it — NOT
a join. `usage` is notes, `rule_usage` is rules, and they stay apart
because a few dozen eligible rules blended into thousands of notes is the
note ratio with noise on it (milestone 333). `NoteUsageEvent`'s own docstring is explicit that the two are
Two aggregates side by side, each read from the table built for it — NOT a
join. `NoteUsageEvent`'s own docstring is explicit that the two are
complements ("RetrievalLog tunes the threshold, this tunes the corpus") and
that RetrievalLog's JSONB `result_ids` "can't be indexed at" the per-note
grain. So the score distribution comes from `retrieval_logs` on its indexed
@@ -370,12 +199,6 @@ async def retrieval_summary(
it was built for. Reading each from its own table is both cheaper and more
honest than correlating them through JSONB.
`usage["by_source"]` is the one join, and it stays INSIDE
`note_usage_events` — surfaced rows against pulled rows on note_id. That
answers "of the notes this surface chose, how many were opened", which the
top-level ratio averages away. It does not cross into `retrieval_logs`, so
the sentence above still holds.
Scoped to one user's own telemetry. There is no sharing model for a
retrieval log — it records what THIS user's agent asked for, including the
query text — so an owner filter is the whole access rule here rather than a
@@ -391,77 +214,23 @@ async def retrieval_summary(
"since": iso(since),
"sources": {},
"usage": {},
"rule_usage": {},
"read_failed": False,
}
zero = case((RetrievalLog.result_count == 0, 1), else_=0)
# THE NEAR-MISS POPULATION: calls that returned nothing BECAUSE THE BAR
# TURNED SOMETHING AWAY, and recorded what it was. Three conditions, and
# the third was missing for one deploy (#3739).
#
# Zero-result only: on a call that returned something,
# `best_available_score` equals `top_score` and adds nothing.
#
# Non-null only: rows written before #3670 genuinely do not know, and must
# not read as scoreless declines.
#
# AND NOT A REPEAT. A zero-result call is two unrelated events — the ranker
# found nothing above the bar, or it found only what this session had
# already been shown — and just the first says anything about the bar. That
# is the whole of #3497, and #3670 reintroduced the conflation one level up:
# the rule arms filter exclusions in PYTHON, after the search, so a rule
# that cleared the bar and was dropped as a repeat still reported a high
# `best_available_score` on a zero-result row. Live proof, first read after
# deploy: pre_tool_rule's near-miss max was 0.7457 while the lowest score it
# ever RETURNED was 0.7204 — a "rejection" that outscored acceptances.
#
# The NULL arm is principled, not permissive: `suppressed_count IS NULL`
# means the caller passed its exclusions INTO the search, which is exactly
# the case where the reported score is already post-exclusion and cannot be
# contaminated. Note arms stay measured; rule arms get cleaned.
#
# Deliberately conservative: a call carrying both a repeat and a lower
# genuine miss is dropped whole, losing that point. It undercounts; it
# cannot corrupt — the right way round for a number read against a bar.
#
# This also makes `near_misses.max < threshold` true BY CONSTRUCTION. An
# above-bar candidate that was not excluded would have been returned, so
# its call is not in this population at all.
declined = (
(RetrievalLog.result_count == 0)
& (RetrievalLog.best_available_score.isnot(None))
& (
RetrievalLog.suppressed_count.is_(None)
| (RetrievalLog.suppressed_count == 0)
)
)
miss = case((declined, 1), else_=0)
# `best_available_score` only for those rows; NULL elsewhere, and
# percentile_cont ignores NULLs, so the distribution is over the declines
# alone without a second pass over the table.
miss_score = case((declined, RetrievalLog.best_available_score), else_=None)
# Three sums rather than one, because "not measured" and "measured as zero"
# are different answers and a single counter cannot hold both.
measured = case((RetrievalLog.suppressed_count.isnot(None), 1), else_=0)
supp_calls = case((RetrievalLog.suppressed_count > 0, 1), else_=0)
supp_zero = case(
((RetrievalLog.result_count == 0) & (RetrievalLog.suppressed_count > 0), 1),
cleared = case(
(
(RetrievalLog.threshold.isnot(None))
& (RetrievalLog.top_score.isnot(None))
& (RetrievalLog.top_score >= RetrievalLog.threshold),
1,
),
else_=0,
)
zero = case((RetrievalLog.result_count == 0, 1), else_=0)
def pct(p: float):
return func.percentile_cont(p).within_group(RetrievalLog.top_score.asc())
# Assigned inside the try below; named here so the readout can tell
# "this query failed" from "this window has no rows" (#2663).
by_source_rows = None
rule_rows = None
distinct_rules_surfaced = distinct_rules_pulled = 0
# None means the coverage read did not happen — distinct from a table with
# no rows, which is {"*": None}. Same reason `read_failed` exists.
note_complete = rule_complete = None
try:
async with async_session() as session:
rows = (
@@ -470,6 +239,7 @@ async def retrieval_summary(
RetrievalLog.source,
func.count().label("calls"),
func.sum(zero).label("zero"),
func.sum(cleared).label("cleared"),
pct(0.1), pct(0.5), pct(0.9),
func.min(RetrievalLog.top_score),
func.max(RetrievalLog.top_score),
@@ -477,13 +247,6 @@ async def retrieval_summary(
func.percentile_cont(0.9).within_group(
RetrievalLog.duration_ms.asc()
),
func.sum(measured).label("measured"),
func.sum(supp_calls).label("supp_calls"),
func.sum(supp_zero).label("supp_zero"),
func.sum(miss).label("miss_calls"),
func.percentile_cont(0.5).within_group(miss_score.asc()),
func.percentile_cont(0.9).within_group(miss_score.asc()),
func.max(miss_score),
)
.where(
RetrievalLog.created_at >= since,
@@ -492,82 +255,8 @@ async def retrieval_summary(
.group_by(RetrievalLog.source)
)
).all()
log_complete = await _complete_from(session, RetrievalLog, user_id)
for row in rows:
source = row[0]
bucket = _bucket(list(row[1:]))
# Per SOURCE, not per table: retrieval_logs goes back months
# while any individual arm may be days old, and the table's
# age would vouch for an arm that has barely started.
bucket.update(_coverage(log_complete.get(source), since))
out["sources"][source] = bucket
# A source with rows in the table but NONE in this window would
# otherwise be absent from the readout — and absent is exactly how
# a source that never existed renders, so a surface that WAS
# recording and went silent is unreadable (#3720). That is #2663
# one level up: the failure that looks like the correct answer.
#
# Zero here is a real measurement, not a manufactured one. The
# all-time query proves the source was recording, and it made no
# calls across a window it fully covers — which is why no
# `covers_window` special case is needed: a source whose first row
# fell after `since` would have that row IN the window and already
# hold a bucket, so anything reaching here began before it.
for src, first_row in log_complete.items():
if src == "*" or first_row is None or src in out["sources"]:
continue
quiet = _bucket(list(_NO_ROWS_IN_WINDOW))
quiet.update(_coverage(first_row, since))
out["sources"][src] = quiet
# WHAT THE BAR REFUSED, by name (#3807). Opt-in, because it is a
# LISTING and not a statistic: an id cannot be percentiled, and a
# reader tuning a threshold needs to go and read the records rather
# than see another number about them. Off by default so the
# ordinary readout keeps its size.
#
# Deliberately NOT a window function. This module's one production
# outage (#2663) was a grouped query the database rejected, swallowed
# by the broad except, every counter reading zero while the mocked
# tests passed — and the lesson recorded then was to group on a raw
# column and classify in Python rather than push cleverness into the
# SQL. So: one flat ordered query, overfetched, bucketed here.
if near_miss_samples > 0:
want = max(1, min(int(near_miss_samples), 20))
rows = (
await session.execute(
select(
RetrievalLog.source,
RetrievalLog.best_available_score,
RetrievalLog.best_available_id,
RetrievalLog.query,
)
.where(
declined,
RetrievalLog.created_at >= since,
RetrievalLog.user_id == user_id,
RetrievalLog.best_available_id.isnot(None),
)
.order_by(RetrievalLog.best_available_score.desc())
# Overfetch so every source can fill its own quota even
# when one of them holds all the highest scores.
.limit(want * 40)
)
).all()
for src, score, rec_id, q in rows:
bucket = out["sources"].get(src)
if bucket is None:
continue
samples = bucket.setdefault("near_miss_records", [])
if len(samples) >= want:
continue
samples.append({
"score": _round(score),
"record_id": int(rec_id),
# Enough to recognise the ask, not the whole prompt.
"query": (q or "")[:120],
})
out["sources"][row[0]] = _bucket(list(row[1:]))
# The corpus side, at its own grain. `ambient` mirrors
# note_usage.usage_for_notes: an ambient surfacing was not a scored
@@ -594,7 +283,6 @@ async def retrieval_summary(
.group_by(NoteUsageEvent.event, NoteUsageEvent.source)
)
).all()
note_complete = await _complete_from(session, NoteUsageEvent, user_id)
# Distinct-note counts need their OWN queries, and this is not
# fussiness: count(distinct note_id) per (event, source) group
@@ -622,142 +310,6 @@ async def retrieval_summary(
)
)
).scalar_one()
# Per-source pull-through, at the NOTE grain (#3311).
#
# The `urows` query above already groups by source and the loop
# below then throws the source away, so until now this readout
# could say what the corpus's overall pull-through was and nothing
# about WHICH surface earned it. The data was always here; only
# the aggregation discarded it.
#
# It cannot be had by grouping the PULLED rows by source: a pull
# records the door it came through (`mcp_get_note`), not the
# surface that put the record in front of the agent. Correlating
# those within a session is what #2085 ruled out — there is no
# session identity server-side and inventing one would mean
# threading a client-supplied token through every read path. The
# note grain answers the question without one: of the distinct
# notes surface X chose, how many did an agent open in this window?
#
# Guarded separately from the reads above, on #2663's actual
# lesson. That outage was a NOVEL SQL SHAPE the database rejected
# inside a broad except. This join is the novel shape here, and a
# failure in it must not take down two readouts that already work.
try:
pulled_ids = (
select(NoteUsageEvent.note_id)
.where(
NoteUsageEvent.created_at >= since,
NoteUsageEvent.user_id == user_id,
NoteUsageEvent.event == PULLED,
# autoescape because `_` is a LIKE wildcard: a bare
# like("mcp_%") also matches "mcpX…". The Python half
# of this readout uses str.startswith and has no such
# hazard; this is the SQL half's version of it.
NoteUsageEvent.source.startswith("mcp_", autoescape=True),
)
.distinct()
.subquery()
)
surfaced_pairs = (
select(NoteUsageEvent.source, NoteUsageEvent.note_id)
.where(
NoteUsageEvent.created_at >= since,
NoteUsageEvent.user_id == user_id,
NoteUsageEvent.event == SURFACED,
)
.distinct()
.subquery()
)
# DISTINCT on (source, note_id) FIRST, which is what lets the
# outer aggregate be a plain count(): the pairs are already
# unique, so the left join cannot multiply them and no
# count(DISTINCT) is needed to undo damage that never happens.
by_source_rows = (
await session.execute(
select(
surfaced_pairs.c.source,
func.count().label("notes_surfaced"),
func.count(pulled_ids.c.note_id).label("notes_pulled"),
)
.select_from(
surfaced_pairs.outerjoin(
pulled_ids,
pulled_ids.c.note_id == surfaced_pairs.c.note_id,
)
)
.group_by(surfaced_pairs.c.source)
)
).all()
except Exception:
logger.warning("per-source pull-through read failed", exc_info=True)
by_source_rows = None
# Rules, at their own grain and in their own block (milestone 333).
#
# Guarded separately from the reads above for the reason `by_source`
# is: this table is NEW, and an instance running upgraded code
# against un-migrated schema would otherwise take down two readouts
# that work perfectly in order to report a third that cannot.
#
# The queries themselves are the note block's shapes, not novel
# ones — a group-by on two indexed columns and two count(distinct).
# The distinct counts need their own queries for the same reason
# the note ones do: count(distinct rule_id) per group cannot be
# summed across groups without double-counting a rule two sources
# both touched.
try:
rule_rows = (
await session.execute(
select(
RuleUsageEvent.event,
RuleUsageEvent.source,
func.count().label("n"),
)
.where(
RuleUsageEvent.created_at >= since,
RuleUsageEvent.user_id == user_id,
)
.group_by(RuleUsageEvent.event, RuleUsageEvent.source)
)
).all()
rule_complete = await _complete_from(
session, RuleUsageEvent, user_id,
)
# The rows carry `source`, so the ranked/ambient split is done
# below rather than in SQL — the bulk surfaces started emitting
# on 2026-09-03 (#3473), so there IS an ambient class now.
#
# `distinct_rules_surfaced` deliberately counts BOTH classes. It
# answers "how many distinct rules did this install put in front
# of an agent at all", which is the denominator for dead weight
# — and a rule delivered by the preload a hundred times and
# never opened is the most important case that question has.
distinct_rules_surfaced = (
await session.execute(
select(func.count(func.distinct(RuleUsageEvent.rule_id)))
.where(
RuleUsageEvent.created_at >= since,
RuleUsageEvent.user_id == user_id,
RuleUsageEvent.event == RULE_SURFACED,
)
)
).scalar_one()
distinct_rules_pulled = (
await session.execute(
select(func.count(func.distinct(RuleUsageEvent.rule_id)))
.where(
RuleUsageEvent.created_at >= since,
RuleUsageEvent.user_id == user_id,
RuleUsageEvent.event == RULE_PULLED,
)
)
).scalar_one()
except Exception:
logger.warning("rule usage read failed", exc_info=True)
rule_rows = None
distinct_rules_surfaced = distinct_rules_pulled = 0
except Exception:
logger.warning("retrieval summary read failed", exc_info=True)
out["read_failed"] = True
@@ -798,124 +350,5 @@ async def retrieval_summary(
round(usage["pulled_by_agent"] / usage["surfaced"], 4)
if usage["surfaced"] else None
)
# The same question, per surface — which is the one the top-level ratio
# cannot answer. A corpus average of 0.05 is compatible with one surface
# earning its noise and another producing none, and tuning a threshold
# needs to know which.
#
# UPPER BOUND, and say so where it will be read: a pull records the door,
# not the surface that led to it, so a note surfaced by two surfaces and
# opened once counts as pulled for both. Attribution would need the session
# identity #2085 declined to invent. The bound is still decisive in the
# direction that matters — a surface reading near zero here is not being
# flattered by the double-count.
if by_source_rows is None:
usage["by_source"] = {}
# Distinct from an empty window, for the same reason `read_failed` is.
usage["by_source_failed"] = True
else:
by_source: dict[str, dict] = {}
for source, n_surfaced, n_pulled in by_source_rows:
n_surfaced, n_pulled = int(n_surfaced or 0), int(n_pulled or 0)
ambient = source in AMBIENT_SOURCES
by_source[source] = {
"notes_surfaced": n_surfaced,
"notes_pulled": n_pulled,
# None rather than a number on an ambient surface: nothing
# CHOSE those records, so "surfaced often, opened never" is not
# a judgment about them. The counts stay visible; the ratio
# that would be misread does not.
"pull_through": (
None if ambient or not n_surfaced
else round(n_pulled / n_surfaced, 4)
),
"ambient": ambient,
}
usage["by_source"] = by_source
# The SECTION's coverage, from the latest source to start recording — a
# figure that sums several sources is complete only once every one of them
# was being written. `_complete_from` computes that as "*".
usage.update(_coverage((note_complete or {}).get("*"), since))
out["usage"] = usage
# ── Rules, deliberately a SEPARATE block ────────────────────────────
#
# Not folded into `usage`, for two reasons and the second is the one that
# bites. The corpora differ by orders of magnitude — a few dozen eligible
# rules against thousands of notes — so one blended ratio would be the note
# ratio with a little noise on it, and the rule arm's own behaviour would
# be undetectable inside it. And `usage` is what existing callers already
# read: silently changing what it counts would move a number people have
# been comparing across windows, without telling them it now measures
# something else.
#
# `ambient` now carries the bulk deliveries — the SessionStart preload,
# `list_always_on_rules`, and every `rules_payload` surface (#3473). Before
# they emitted, this block had no ambient key and said the absence was a
# fact about the data. It was, and it was also the thing that made the
# always-on set impossible to judge: the largest rule surface in the
# product was the one surface its own scoreboard could not see.
#
# READ THE TWO SEPARATELY, ALWAYS. `surfaced` is a claim a ranker made and
# a pull can settle. `ambient` is a delivery nobody chose, so a high count
# says the set is large and resident, never that it is useful.
rule_usage = {
"surfaced": 0, "ambient": 0,
"pulled": 0, "pulled_by_agent": 0, "pulled_by_human": 0,
"distinct_rules_surfaced": int(distinct_rules_surfaced or 0),
"distinct_rules_pulled": int(distinct_rules_pulled or 0),
}
if rule_rows is None:
# The FLAG is added, the shape is kept — matching `by_source_failed`
# one block up. A caller that renders this must not have to choose
# between crashing on a missing key and quietly showing zeros it has no
# right to: the keys let it render, and the flag tells it the zeros are
# "we could not find out" rather than "nothing happened" (#2663).
rule_usage["rule_usage_failed"] = True
else:
for event, source, n in rule_rows:
n = int(n)
if event == RULE_SURFACED:
# One definition of ranked-vs-ambient, imported rather than
# restated — the per-rule badge readout reads the same
# predicate, and two spellings of "what counts as surfaced" is
# precisely the uneven wiring #3246 found across this system.
if is_ambient(source):
rule_usage["ambient"] += n
else:
rule_usage["surfaced"] += n
elif event == RULE_PULLED:
rule_usage["pulled"] += n
# Same split, and it carries MORE weight here than for notes.
# The arm's whole claim is "this rule may apply to what you are
# writing", and only an agent opening it says the claim landed.
# A person browsing the rule list says nothing about the hint.
if source.startswith("mcp_"):
rule_usage["pulled_by_agent"] += n
else:
rule_usage["pulled_by_human"] += n
# None, not 0.0, when nothing was surfaced — matching the note block. A
# ratio of zero asserts "we showed rules and none were opened"; with an
# empty numerator AND denominator that is a claim the data does not
# support, and it is the reading that would make a brand-new install look
# like a broken one.
#
# RANKED SURFACINGS ONLY in the denominator, and this is the load-bearing
# line of the whole change. Pull-through asks "was that hint any use", and
# only a surface that CHOSE what it showed can be judged by it. Folding the
# preload in would divide the same pulls by a number that grows with every
# session and every rule added to the resident set — so enlarging the
# always-on set would DEPRESS the arm's measured precision, and trimming it
# would flatter it, neither for any reason to do with the arm. The ambient
# count sits beside it, unaveraged, and is read as size rather than skill.
rule_usage["pull_through"] = (
round(rule_usage["pulled_by_agent"] / rule_usage["surfaced"], 4)
if rule_usage["surfaced"] else None
)
rule_usage.update(_coverage((rule_complete or {}).get("*"), since))
out["rule_usage"] = rule_usage
return out
-295
View File
@@ -1,295 +0,0 @@
"""Rule usage telemetry — did a surfaced rule ever get read?
The sibling of `note_usage`, for the one retrieval surface in Scribe that
could not be measured at all.
Two event streams, deliberately independent:
- SURFACED: the write-path standing-rule arm put this rule in front of the
agent, unbidden, during a write.
- PULLED: someone then opened it in full (`get_rule`, or the REST detail
route).
WHY THIS ARM AND NOT ANOTHER. Every other surface declines most of the time —
`write_path` returns nothing on 78% of calls, `reuse_slot` on 79%, auto-inject
on 39%. The rule arm APPEARED never to have returned nothing (#3311), and this
docstring used to put that forward as the puzzle worth measuring: "either a
perfectly tuned surface or a bar it cannot fail to clear".
It was neither, and the correction belongs here rather than being quietly
deleted. The arm wrote its `retrieval_logs` row only on calls that FOUND
something (#3497), so `zero_result_calls` sat at 0 and `cleared_threshold` at
`calls` because of the shape of the code — at any threshold whatsoever. A
statistic that could not vary was read as a finding about the corpus. It is the
#2663 failure mode one level up: there the broken readout was a zero, here it
was a hundred percent, which is far better camouflage.
The reason to measure this arm survives the correction, and is stronger for it.
`retrieval_logs` records what the ranker scored, never whether the hint was any
use, so even an honest clear-rate would not settle the question. The ratio these
two streams produce is the missing half, and without it any threshold change is
a number picked off a histogram.
Design notes, mirroring `note_usage`:
- Writes are fire-and-forget through `background.spawn`, so telemetry never
adds latency to — or can break — the surface it observes. This module does
NOT carry its own copy of the strong-reference dance; `background` is the
one place that gets it right, and a fourth copy is how one of them drifts.
- Failures degrade, but never SILENTLY. `report_telemetry_failure` logs at
WARNING and drops one AppLog row per process per site. #2663 is the record
of this exact subsystem class running at zero for weeks — indistinguishable
from "nobody uses this" — because every failure went to `logger.debug`.
- Reads (`usage_for_rules`) are awaited and aggregated in one round-trip for
a whole page, never per row.
AMBIENT VS RANKED. The note twin splits ranked surfacings from ambient ones
because `enter_project` and the skill sync put records in front of the agent
without choosing them, and counting those as surfacings makes recency read as
popularity (#2477). Rules have exactly that shape: the SessionStart preload,
`list_always_on_rules`, and every `rules_payload` surface hand over the whole
applicable set at once, chosen by nobody.
Until 2026-09-03 those bulk surfaces emitted nothing, and this module said so —
"an empty `AMBIENT_SOURCES` would be machinery pretending to a distinction the
data does not yet contain". True as far as it went, but it had a consequence
worth naming, because it is the reason the bucket exists now: the always-on
set's token cost was certain and its usefulness was UNFALSIFIABLE, permanently
and by construction. The one surface whose value was actually in question was
the one surface exempt from the scoreboard that judges every other.
They emit now. The split is the readout-level change the old note promised — a
`case()`, no migration, because `event` and `source` are plain Text with no
CHECK constraint. `source` stays granular so a reader can still tell the
preload from `enter_project` from the ranked arm.
WHY THIS NAMES THE RANKED SOURCES AND THE TWIN NAMES THE AMBIENT ONES. A
deliberate divergence, on the failure mode rather than on symmetry. Both shapes
fail silently when someone adds a surface and forgets the list, so the question
is which list changes more often — and here it is emphatically the ambient one:
there are TWO ranked rule sources (the write-path arm and the pre-tool arm)
against the seven bulk ones the preload alone contributes. Ranked sources are
added when somebody builds a ranker, which is rare and deliberate; bulk ones
appear whenever a surface hands rules over, which is most of them. Naming the
rare, slow-moving half means a newly-added bulk surface defaults to
`ambient`, which merely under-counts it, instead of defaulting to `ranked`,
which would quietly pad the pull-through denominator with surfacings nobody
chose and make the arm look imprecise. Same argument #3191 and #3430 make
against hand-kept lists: keep the list that must be remembered as short and as
slow-moving as possible.
"""
from __future__ import annotations
import logging
from sqlalchemy import case, func, select
from scribe.models import async_session
from scribe.models.base import iso
from scribe.models.rule_usage import PULLED, SURFACED, RuleUsageEvent
from scribe.services.background import report_telemetry_failure, spawn
logger = logging.getLogger(__name__)
# The surfaces that CHOSE the rules they showed. Everything else is ambient —
# see the module docstring for why the rare half is the half that gets named.
#
# Membership is the whole definition of the pull-through denominator: a ranked
# surfacing is a claim ("this rule may apply to what you are doing") that a pull
# can confirm or refute, while an ambient one is a delivery nobody decided on.
# Add a source here only when a ranker picked it.
RANKED_SOURCES = ("write_path_rule", "pre_tool_rule")
def is_ambient(source: str) -> bool:
"""Was this surfacing a bulk delivery rather than a ranked choice?
One definition, read by both the per-rule badge readout and the aggregate
in `retrieval_telemetry` — the two used to be able to disagree about what
"surfaced" counted, which is the class of drift #3246 found across the
rules system.
Sync and pure, per the service canon (#2860), but deliberately PUBLIC where
that canon says such helpers stay `_private`. The departure is the point:
a module-private copy in each caller is exactly the second definition this
exists to prevent.
"""
return source not in RANKED_SOURCES
async def _report_failure(site: str) -> None:
await report_telemetry_failure("rule_usage", site)
async def _insert_events(rows: list[dict]) -> None:
"""Persist usage rows. Best-effort: failures degrade, visibly."""
try:
async with async_session() as session:
session.add_all([RuleUsageEvent(**row) for row in rows])
await session.commit()
except Exception:
await _report_failure("write")
def _schedule(rows: list[dict]) -> None:
if not rows:
return
spawn(_insert_events(rows), site="rule_usage_write")
def record_rule_surfaced(
*, user_id: int | None, rule_ids: list[int] | set[int], source: str
) -> None:
"""Fire-and-forget: record that these rules were shown to the agent.
Takes the whole delivery at once — one insert per surfacing event, not per
rule — because a hint is a single decision and its rows should land
together.
Record what was actually SHOWN, never what was considered. For the ranked
arm that means the post-filter hits: it drops what the session already
holds (`exclude_rule_ids`) before it speaks, and a rule considered and not
shown was not surfaced. Counting those would inflate the denominator with
claims the agent never saw, which reads as a precision problem the arm does
not have.
Bulk surfaces pass their whole delivered set, which is the same rule read
from the other end — everything in a preload IS shown. `source` is what
separates the two afterwards (see `RANKED_SOURCES`); this function does not
care which kind it is recording.
"""
try:
rows = [
{
"user_id": user_id,
"rule_id": int(rid),
"event": SURFACED,
"source": source,
}
for rid in rule_ids
]
except Exception:
logger.debug("rule usage payload build failed", exc_info=True)
return
_schedule(rows)
def record_rule_pulled(*, user_id: int | None, rule_id: int, source: str) -> None:
"""Fire-and-forget: record that a rule was opened in full.
A PULL is somebody choosing to open one record. `list_always_on_rules` and
`enter_project` are NOT pulls — they are bulk resident loads that hand over
every applicable rule at once, and counting them would swamp the signal
with the very ambient delivery the ratio exists to distinguish from.
"""
try:
rows = [
{
"user_id": user_id,
"rule_id": int(rule_id),
"event": PULLED,
"source": source,
}
]
except Exception:
logger.debug("rule usage payload build failed", exc_info=True)
return
_schedule(rows)
def empty_rule_usage() -> dict:
"""The zero readout — what a rule with no recorded events looks like.
Callers render this shape unconditionally, so a rule predating the table
reads as "never surfaced, never pulled" rather than as a missing key. That
distinction matters more here than for notes: every rule in an install
predates this table, so for a while "no events" is the normal state and it
must not look like a broken readout.
`surfaced_count` is RANKED surfacings only; `ambient_count` is the bulk
deliveries (see `RANKED_SOURCES`). The split is what keeps the badge's
"shown often, opened never → dead weight" reading honest: every rule in an
always-on set is delivered every session, so an unsplit counter would rank
the resident set as the most-surfaced rules in the install purely for being
resident.
"""
return {
"surfaced_count": 0,
"ambient_count": 0,
"pull_count": 0,
"last_surfaced_at": None,
"last_pulled_at": None,
}
async def usage_for_rules(rule_ids: list[int]) -> dict[int, dict]:
"""Aggregate usage for a set of rules: {rule_id: {counts + timestamps}}.
One GROUP BY for the whole page rather than a query per row — this feeds a
list view, so the per-row shape would be N+1 by construction. Rules with no
events come back with `empty_rule_usage()`, so the caller never has to tell
"no events" from "not in the result".
"""
ids = [int(r) for r in rule_ids]
out: dict[int, dict] = {rid: empty_rule_usage() for rid in ids}
if not ids:
return out
# Classified in SQL so the group stays small: per rule we get at most
# (surfaced-ranked, surfaced-ambient, pulled) rather than a row per distinct
# source. ONE labelled expression, bound to a variable and reused in the
# GROUP BY — a second `case()` instance there renders its own expanding-IN
# bind names under asyncpg, so the database sees two DIFFERENT expressions
# and rejects the query with a GroupingError. The note twin carries the
# same warning for the same reason, and #2663 is what it cost: the
# rejection was swallowed and every counter read zero in production while
# the writes were landing fine.
ambient = case(
(RuleUsageEvent.source.notin_(RANKED_SOURCES), True),
else_=False,
).label("ambient")
try:
async with async_session() as session:
rows = (
await session.execute(
select(
RuleUsageEvent.rule_id,
RuleUsageEvent.event,
func.count().label("n"),
func.max(RuleUsageEvent.created_at).label("last_at"),
ambient,
)
.where(RuleUsageEvent.rule_id.in_(ids))
.group_by(
RuleUsageEvent.rule_id,
RuleUsageEvent.event,
ambient,
)
)
).all()
except Exception:
# A telemetry readout must not be able to break the list it decorates —
# but it must say it failed, or a broken readout is indistinguishable
# from a corpus nobody uses (#2663).
await _report_failure("readout")
return out
for rule_id, event, n, last_at, is_amb in rows:
slot = out.get(int(rule_id))
if slot is None:
continue
if event == SURFACED and is_amb:
slot["ambient_count"] = int(n)
elif event == SURFACED:
slot["surfaced_count"] = int(n)
slot["last_surfaced_at"] = iso(last_at)
elif event == PULLED:
# Pulls are pulls regardless of what surfaced the rule — "did
# anyone ever open this?" does not depend on how it was found. Both
# halves accumulate, so this ADDS rather than assigns: a rule can
# now be pulled after a ranked hint and after a preload, and the
# split arrives as two rows.
slot["pull_count"] = slot["pull_count"] + int(n)
latest = iso(last_at)
if latest and (slot["last_pulled_at"] or "") < latest:
slot["last_pulled_at"] = latest
return out
-82
View File
@@ -1,82 +0,0 @@
"""A rule's edit history — the snapshot taken when an edit overwrites text.
Sibling of `note_versions`, and the differences are the whole design.
WHAT A VERSION IS FOR. It records what a rule USED TO SAY, because an update
overwrites that and nothing can recompute it. It is not a record of the rule's
lifecycle: a soft delete keeps the rule row whole and restorable, so there is
nothing to preserve at delete time, and anything that outlived a purge would be
data the operator explicitly asked to be gone.
WHAT IS DELIBERATELY NOT COPIED FROM note_versions (milestone 323). Both of its
guards defend against note autosave, which fires every 60 seconds and would
otherwise burn every slot. Rules have no autosave — every version here comes
from a deliberate `update_rule` — so both guards would only ever discard a real
edit:
- No `MIN_VERSION_INTERVAL_SECONDS`. Two deliberate edits four minutes apart
are two edits.
- No `MAX_VERSIONS` and no pruning. A rule is edited a handful of times in its
life; a cap invites losing the one edit somebody needed.
TEXT ONLY. A rule's topic, its Systems tags and its typed edges are not here.
They have their own lifecycle, and folding them in would make one word,
"version", mean two things — the rule's wording, and the rule's place in the
graph.
"""
from sqlalchemy import select
from scribe.models import async_session
from scribe.models.rule_version import RuleVersion
# The columns a version carries, and therefore the ones whose change is worth
# a snapshot. `order_index` and `arose_from_id` are editable through
# update_rule and are deliberately absent: reordering a rulebook is not an
# edit to what any rule says, and a run of "moved rule 4 above rule 3"
# snapshots would bury the edits somebody is actually looking for.
SNAPSHOT_FIELDS = (
"title", "statement", "why", "how_to_apply", "when_to_apply",
"tier", "verify_with", "expires_when",
)
def snapshot(rule) -> dict:
"""The rule's text right now, as a plain dict.
Plain values rather than the ORM object because the caller has to hold
this ACROSS the mutation — a live `rule` would show the new text by the
time it was read, which is the one thing the snapshot must not do.
"""
return {f: getattr(rule, f) for f in SNAPSHOT_FIELDS}
def record_if_changed(session, rule, actor_user_id: int | None, before: dict):
"""Add a version holding `before`, unless the edit changed no text.
Git semantics, and not a nicety: `update_rule` is reached by callers that
resend every field, so without this a form saved twice would file a second
identical snapshot and the history would stop reading as a list of edits.
Session-BOUND, unlike `note_versions.create_version` which opens its own.
The version and the edit that caused it commit together or not at all;
a separate session could leave a snapshot behind after the update it
describes had failed, which is a history entry for an edit that never
happened.
"""
if snapshot(rule) == before:
return None
version = RuleVersion(
rule_id=rule.id, user_id=actor_user_id, **before,
)
session.add(version)
return version
async def list_versions(rule_id: int) -> list[RuleVersion]:
"""Newest first — the reader's question is "what did this just say?"."""
async with async_session() as session:
return list((await session.execute(
select(RuleVersion)
.where(RuleVersion.rule_id == rule_id)
.order_by(RuleVersion.created_at.desc(), RuleVersion.id.desc())
)).scalars().all())
+22 -193
View File
@@ -9,7 +9,6 @@ from __future__ import annotations
import logging
from collections.abc import Iterable
from datetime import datetime
from typing import Optional
from sqlalchemy import and_, delete as sql_delete, insert, or_, select
@@ -17,13 +16,6 @@ from sqlalchemy import and_, delete as sql_delete, insert, or_, select
from scribe.models import async_session
from scribe.models.system import System
from scribe.models.rulebook import Rulebook
from scribe.services.verification import (
days_since_verified as _days_since_verified,
last_verified_label as _last_verified_label,
)
from scribe.services import rule_versions
from scribe.models.rule_version import RuleVersion
from scribe.services.rule_usage import record_rule_surfaced
logger = logging.getLogger(__name__)
@@ -319,12 +311,19 @@ def _valid_tier(tier: str) -> str:
return tier if tier in TIERS else "always_on"
# Re-exported, not redefined. Notes gained the same trio in milestone 317 and
# this reading of it is genuinely common, so it moved to services/verification
# — the DRY win note 3163 names, as against sharing the QUERY, which the two
# record types cannot (a rule scopes by rulebook ownership, a note by the note
# ACL). Kept importable from here because callers already reach for it here.
last_verified_label = _last_verified_label
def last_verified_label(rule: Rule) -> str | None:
"""How long ago the rule's check passed — None when it carries no check.
One helper because two surfaces need the same answer and the brief-dict
lesson in rule_brief's docstring is what happens otherwise: three copies
that had already drifted. `None` means "this rule is a decision, the
question does not apply"; "never" means "it is a fact and nobody has
confirmed it" — a distinction worth keeping, because the second is the
one worth acting on.
"""
if not rule.verify_with:
return None
return rule.verified_at.date().isoformat() if rule.verified_at else "never"
def rule_brief(rule: Rule, **extra) -> dict:
@@ -380,10 +379,6 @@ def _refresh_rule_embedding(rule: Rule) -> None:
swallowed because a rule that SAVED must not fail on its index refresh —
a stale vector costs a missed search hit, a raised exception costs the
write. No running loop (unit tests, scripts) is ordinary, not an error.
Detaching also means this task races anything that deletes the rule out
from under it. That is not handled here: `upsert_rule_embedding` claims
the rule's row before touching its vectors, and loses if it can't (#3262).
"""
try:
import asyncio
@@ -756,10 +751,6 @@ async def update_rule(
"verify_with", "expires_when",
}
check_before = rule.verify_with
# Captured BEFORE anything is written, and as plain values — this has
# to survive the mutation below. A rule's history is the only record
# of what it used to say; the edit itself destroys that.
text_before = rule_versions.snapshot(rule)
for key in clear:
if key in allowed and key in NULLABLE_RULE_TEXT:
setattr(rule, key, None)
@@ -783,59 +774,12 @@ async def update_rule(
# wrongly vouched for costs the thing the sweep exists to catch.
if rule.verify_with != check_before:
rule.verified_at = None
# Same session as the edit, so the two commit together. The snapshot
# holds the OLD verify_with — the check that was in force when that
# wording was written — which is why it is taken before the loop and
# not here.
rule_versions.record_if_changed(session, rule, user_id, text_before)
await session.commit()
await session.refresh(rule)
_refresh_rule_embedding(rule)
return rule
# ── Edit history (milestone 323) ───────────────────────────────────────
#
# The ACL-scoped reads live HERE rather than in services/rule_versions.py,
# and not by preference: rulebooks imports rule_versions for the write path,
# so the reverse import would be a cycle. The split is also the honest one —
# rule_versions owns what a version IS, this module owns who may read one.
async def list_rule_versions(rule_id: int, user_id: int):
"""A rule's history, newest first. None when the rule is not readable.
Scoped through the rule itself, never through the version's `user_id`:
that column is the ACTOR. Reading a rule's history is a question about
the RULE, so anyone who can read the rule can read what it used to say,
and anyone who cannot read the rule gets nothing — including the versions
they personally wrote, if the rule has since moved out of their reach.
"""
async with async_session() as session:
if await _fetch_owned_rule(session, rule_id, user_id) is None:
return None
return await rule_versions.list_versions(rule_id)
async def get_rule_version(rule_id: int, version_id: int, user_id: int):
"""One snapshot in full. None when the rule or the version is not found.
Takes the rule id as well as the version id so the ownership check has
something to run against BEFORE the version is read, and so a version id
from another rule cannot be read through a rule the caller does happen to
own — the check and the fetch have to agree about which rule is in play.
"""
async with async_session() as session:
if await _fetch_owned_rule(session, rule_id, user_id) is None:
return None
return (await session.execute(
select(RuleVersion).where(
RuleVersion.id == version_id,
RuleVersion.rule_id == rule_id,
)
)).scalar_one_or_none()
# ── Canon tags + typed edges (milestone 307) ───────────────────────────
async def set_rule_systems(
@@ -1396,7 +1340,7 @@ async def get_applicable_rules(
}
def rules_payload(applicable: dict, *, user_id: int | None, source: str) -> dict:
def rules_payload(applicable: dict) -> dict:
"""The caller-facing shape of a get_applicable_rules() result.
Every surface that hands rules to an agent (enter_project, get_project,
@@ -1407,34 +1351,7 @@ def rules_payload(applicable: dict, *, user_id: int | None, source: str) -> dict
`excluded_always_on` (milestone 297) names the always-on rulebooks this
project decided NOT to inherit, so the departure is visible wherever the
rules are.
IT ALSO RECORDS THE SURFACING, which is why it now takes a caller and a
source. Every one of those surfaces is a bulk delivery — the applicable set
handed over whole, chosen by nobody — so this is the one place that has to
emit for all of them. Doing it per-caller instead would be five sites to
remember, and #3430 gap 2 is what that costs: the process→skill sync went
un-emitted through an entire dedicated telemetry survey because nothing
forced its surface to be accounted for.
`source` stays the CALLER's name rather than a constant, so the readout can
still separate the session handshake from a mid-session milestone read;
`RANKED_SOURCES` in `rule_usage` is what folds them back together.
Emitting from here is safe in a way emitting from `get_applicable_rules`
would not be: this function is only ever called to BUILD A REPLY. The two
other callers of the rules machinery — the write-path etag arm
(`plugin_context`) and `rules_etag_for` — compute a marker and show nobody
anything, and counting those would put rules in the denominator that no
agent ever saw.
"""
record_rule_surfaced(
user_id=user_id,
rule_ids=(
[r["id"] for r in applicable.get("rules", [])]
+ [r["id"] for r in applicable.get("project_rules", [])]
),
source=source,
)
return {
"applicable_rules": applicable["rules"],
"applicable_rules_truncated": applicable["truncated"],
@@ -1446,101 +1363,6 @@ def rules_payload(applicable: dict, *, user_id: int | None, source: str) -> dict
}
# ── The staleness marker (milestone 323 step 5) ────────────────────────
#
# WHAT THIS CAN AND CANNOT SEE. An etag catches a rule that MOVED after a
# session loaded it. It is not a general staleness check, and a reader who
# finds one here will assume it is:
#
# what goes wrong | caught?
# ---------------------------------------------------|--------
# another session edits a rule mid-flight | yes
# the session is misremembering a rule read hours ago | yes
# compaction summarised the rules out of context | NO
#
# The third is the most common and the marker is blind to it, because the
# etag was in context too and went with the rules. The SessionStart nudge is
# that case's only mechanism, and MUST NOT be softened because this exists —
# retiring something that covers the common case in favour of something that
# does not is the plausible mistake here.
_ETAG_EMPTY = "empty|0"
def rules_etag(rules: list) -> str:
"""A marker for "is the set you are holding still the current one?".
`max(updated_at)` alone is not enough: DELETING a rule moves no timestamp,
and that is the single change that takes an instruction OUT of force —
the one a session most needs to hear about. The count catches it.
Instance-agnostic (rule 115): it knows nothing about any particular
rulebook, and an install with one rule or none produces a stable marker
rather than an error. "No rules" must read as a state, not as a change,
or every session on a fresh install would be told its rules had moved.
"""
if not rules:
return _ETAG_EMPTY
# A decoration must not be able to break what it decorates. This is
# computed on the SessionStart path, where raising would cost the whole
# context payload to save a hint — so a row with no usable timestamp is
# skipped rather than compared, and a set with none degrades to a
# count-only marker instead of failing. Count-only still catches a rule
# added or deleted; it just cannot see an edit, which is the right way
# round to lose information.
stamps = [
r.updated_at for r in rules
if isinstance(getattr(r, "updated_at", None), datetime)
]
if not stamps:
return f"unknown|{len(rules)}"
return f"{max(stamps).isoformat()}|{len(rules)}"
async def rules_etag_for(user_id: int, project_id: int = 0) -> str:
"""The current marker for the set a session at this scope would hold.
Deliberately built from `list_always_on_rules` rather than from a
`max()/count()` aggregate. An aggregate would be cheaper, and would have
to restate that function's definition of the set — the always_on flag,
the project's inception exclusions, the tier filter. Two definitions of
"the session's rules" is how the marker starts disagreeing with the
rules, which is worse than materialising a few dozen rows.
"""
rules = await list_always_on_rules(user_id, project_id=project_id)
return rules_etag(rules)
def rules_moved_since(rules: list, held_etag: str) -> list:
"""The rules whose text changed after `held_etag` was issued.
Returns [] when the marker matches, is unparseable, or is absent — a
caller cannot act on "something is different but I cannot say what", and
a garbled marker must not be reported as a change.
A count difference is real news that this list cannot show: a rule
DELETED since the marker was issued has no row left to return. Callers
compare counts separately.
"""
if not held_etag or held_etag == _ETAG_EMPTY:
return []
stamp, _, _count = held_etag.partition("|")
try:
held_at = datetime.fromisoformat(stamp)
except ValueError:
return []
return [r for r in rules if r.updated_at and r.updated_at > held_at]
def etag_count(held_etag: str) -> int | None:
"""How many rules the holder had. None when the marker cannot be read."""
_stamp, _, count = (held_etag or "").partition("|")
try:
return int(count)
except ValueError:
return None
# ── The staleness sweep (milestone 312) ────────────────────────────────
async def rules_due_for_verification(
@@ -1633,7 +1455,14 @@ def verification_row(rule: Rule) -> dict:
because "2026-06-14" and "74 days" prompt different reactions and only
one of them is the question being asked.
"""
days = _days_since_verified(rule)
from datetime import datetime, timezone
days = None
if rule.verified_at is not None:
stamp = rule.verified_at
if stamp.tzinfo is None:
stamp = stamp.replace(tzinfo=timezone.utc)
days = (datetime.now(timezone.utc) - stamp).days
return {
"id": rule.id,
"title": rule.title,
+4 -50
View File
@@ -540,60 +540,14 @@ def compose_data(
return out
def recompose_data(note) -> dict:
"""Rebuild a snippet's `data` mirror from its own body, title and tags.
For the GENERIC note door. `update_snippet` composes the mirror itself from
the field set it just merged and never needs this; a plain
`update_note(body=...)` which the Knowledge feed's editor issues, because
a snippet card there routes to /notes/:id has no idea the mirror exists,
and left it stale. `snippet_fields` then PREFERS the stale mirror, so the
row reported its old repo/path/symbol to prior-art recall while displaying
its new body: confidently wrong, which is worse than no record (#3128).
The body is the authority; the mirror is derived. That is already the rule
this file states it just had no enforcement on the path that bypasses
`update_snippet`.
`verification` and `provenance` are CARRIED, not recomposed, because
neither is in the body to parse the same carry `compose_data` does for
the snippet service's own writes. Note that a verdict does not need
invalidating here: `code_sha` is recomputed from the new code, so a stale
verdict expires itself on read exactly as it does after any other edit.
"""
parsed = parse_snippet_fields(note.title, note.body, note.tags)
prior = note.data or {}
return compose_data(
name=parsed["name"],
when_to_use=parsed["when_to_use"],
signature=parsed["signature"],
language=parsed["language"],
code=parsed["code"],
locations=parsed["locations"],
merged_from=parsed["merged_from"],
verification=prior.get("verification"),
provenance=prior.get("provenance"),
)
def snippet_fields(note) -> dict:
"""Structured fields for a snippet, preferring the indexed `data` column and
falling back to parsing the body.
THE BODY IS THE AUTHORITY; `data` is a mirror derived from it. Every writer
keeps them in step the snippet service composes the mirror from the field
set it just merged, `update_note` recomposes it when a body reaches the
generic door (#3128), and `backfill_snippet_data` filled the pre-0070 rows
at startup. The fallback below is therefore a belt to that braces, not a
second source of truth: it is what a row looks like before the backfill has
run, and it must keep agreeing with the mirror.
(This docstring used to say those rows were "never backfilled". That was
true when 0070 landed and stopped being true when the backfill shipped; it
is corrected here because the sentence read as licence for a stale mirror,
which is exactly the bug #3128 found.)
`code` only ever comes from the body, since `data` doesn't carry it.
Both paths must agree, because rows written before migration 0070 have no
`data` and are never backfilled a hand-edited body is the authority for
those, and there is no deadline by which they must be converted. `code` only
ever comes from the body, since `data` doesn't carry it.
"""
parsed = parse_snippet_fields(note.title, note.body, note.tags)
stored = getattr(note, "data", None)
-57
View File
@@ -1,57 +0,0 @@
"""What a record's own check MEANS — shared by rules and notes.
Two record types carry `verify_with` / `expires_when` / `verified_at`: rules
(milestone 312) and notes (milestone 317). What they share is BEHAVIOUR how
a stamp is read, how old it is, what "never" means not storage and not the
query. Note 3163 is the rule this file is an instance of: `semantic_search_*`
could never have been shared between them because a rule scopes by rulebook
ownership and a note scopes by the note ACL, so the two sweeps are siblings.
These functions are the part that genuinely is common, factored so it exists
once rather than twice.
Everything here is pure, sync and DUCK-TYPED: it reads `verify_with` and
`verified_at` off whatever it is handed. That is deliberate. A shared base
class or a protocol would tie two SQLAlchemy models together to share four
lines of date arithmetic the DRY costume, in the same note's words.
"""
from datetime import datetime, timezone
def last_verified_label(record) -> str | None:
"""How long ago this record's check passed — None when it carries none.
Three states, and the distinction between the last two is the whole point:
- `None` this record is a DECISION. There is nothing to go and check,
and the question does not apply. Most records.
- "never" it asserts a fact and NOBODY HAS EVER CONFIRMED IT. The one
worth acting on, and why the sweep sorts these first.
- a date somebody checked, then.
Callers attach this to a payload only when it is not None, so "no key" and
"never verified" do not become two states every client has to tell apart.
"""
if not getattr(record, "verify_with", None):
return None
stamp = getattr(record, "verified_at", None)
return stamp.date().isoformat() if stamp else "never"
def days_since_verified(record) -> int | None:
"""Whole days since the check last passed; None if it never has.
Computed rather than left to the reader, because "2026-06-14" and "74
days" prompt different reactions and only one of them is the question
being asked.
Naive stamps are read as UTC. Postgres hands these back tz-aware, but a
restore, a fixture or a sqlite-backed test may not, and subtracting an
aware datetime from a naive one raises which would make the sweep fail
on exactly the rows it exists to surface.
"""
stamp = getattr(record, "verified_at", None)
if stamp is None:
return None
if stamp.tzinfo is None:
stamp = stamp.replace(tzinfo=timezone.utc)
return (datetime.now(timezone.utc) - stamp).days
+1 -68
View File
@@ -9,68 +9,7 @@ from __future__ import annotations
from contextlib import contextmanager
from datetime import datetime, timezone
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
async def drive_update_note(note, **kwargs):
"""Run `services/notes.update_note` against a stand-in row.
The patch stack is the point: update_note reaches for a version snapshot,
an embedding refresh and a project reactivation on its way out, none of
which a unit test has. Written twice once for the snippet mirror
(#3128) and once for the verification fields (#3182/317) — before being
consolidated here.
Returns whatever update_note returned; assert on the `note` you passed in.
"""
from unittest.mock import AsyncMock as _AsyncMock
session = make_mock_session()
result = MagicMock()
result.scalars.return_value.first.return_value = note
session.execute = _AsyncMock(return_value=result)
with patch("scribe.services.notes.async_session") as cls, \
patch("scribe.services.notes.embed_note", MagicMock()), \
patch("scribe.services.notes._maybe_reactivate_project", _AsyncMock()), \
patch("scribe.services.note_versions.create_version", _AsyncMock()):
cls.return_value = session
from scribe.services.notes import update_note
return await update_note(user_id=7, note_id=note.id, **kwargs)
def tool_doc(module: str, name: str) -> str:
"""An MCP tool's docstring, whitespace-flattened.
Flattened because these are hard-wrapped at ~76 characters, so any phrase
worth asserting on is liable to straddle a line break a property of the
formatter, not of the guidance. The disambiguator guard (#3123) learned
that on its own first run, matching raw text and reporting a phrase absent
that was plainly there.
Used by every test that pins the docstring CONTRACT rather than its
wording. The tool docstring is the agent-facing contract (rule 119), so
these guards exist to catch it being tidied down to a parameter list.
"""
import importlib
import re as _re
fn = getattr(importlib.import_module(module), name)
assert fn.__doc__, f"{name} has no docstring at all"
return _re.sub(r"\s+", " ", fn.__doc__)
def compiled_sql(element, dialect=None) -> str:
"""A SQLAlchemy clause or statement rendered as literal SQL text.
For asserting on the shape of a predicate without a database which is how
the visibility clauses and the knowledge facets are both tested. Was a
private copy in each of those modules before #3128 needed a third.
Pass `dialect` when the assertion is about something only one backend
renders a Postgres row-lock mode, say. The generic dialect is enough for
a predicate's shape and would quietly drop the rest.
"""
return str(element.compile(dialect=dialect, compile_kwargs={"literal_binds": True}))
from unittest.mock import AsyncMock, MagicMock
def make_mock_session() -> AsyncMock:
@@ -153,9 +92,6 @@ def fake_note(**attrs) -> MagicMock:
"id": 1, "title": "t", "body": "", "tags": [], "user_id": 7,
"note_type": "note", "is_task": False, "task_kind": "work",
"data": None, "deleted_at": None,
# Milestone 317: a truthy mock here reads as "this note carries a
# check", which trips the guard on records that may not have one.
"verify_with": None, "expires_when": None, "verified_at": None,
}, attrs)
@@ -165,7 +101,6 @@ def fake_task(**attrs) -> MagicMock:
"id": 1, "title": "t", "body": "", "status": "todo", "priority": "none",
"tags": [], "parent_id": None, "project_id": None, "is_task": True,
"task_kind": "work", "user_id": 7, "deleted_at": None,
"verify_with": None, "expires_when": None, "verified_at": None,
}, attrs)
@@ -177,8 +112,6 @@ def fake_snippet(**attrs) -> MagicMock:
"body": "```js\nreturn 1\n```\n", "tags": ["js", "snippet"],
"note_type": "snippet", "is_task": False, "task_kind": "work",
"user_id": 7, "data": None, "deleted_at": None,
"status": None,
"verify_with": None, "expires_when": None, "verified_at": None,
}, attrs)
+15 -1
View File
@@ -29,7 +29,6 @@ parameters, with the "is this even the right tool" paragraph quietly gone.
import re
import pytest
from tests.helpers import tool_doc as _doc
# The create surfaces and where they live. `start_planning` is here because
# it is a create in everything but name — it is how a plan comes into being.
@@ -51,6 +50,21 @@ _ALTERNATIVES = [
]
def _doc(module: str, name: str) -> str:
"""The docstring with its whitespace flattened.
Flattened because a docstring is hard-wrapped: "design system" spans a
line break in at least one of these, and matching the raw text would
report it absent. The first draft of this check did exactly that, and
caught it on itself.
"""
import importlib
fn = getattr(importlib.import_module(module), name)
assert fn.__doc__, f"{name} has no docstring at all"
return re.sub(r"\s+", " ", fn.__doc__)
@pytest.mark.parametrize(("module", "name"), _SURFACES)
def test_a_create_surface_names_at_least_two_alternatives(module, name):
"""Reaching for the wrong tool must still put the right one in view."""
-118
View File
@@ -1,118 +0,0 @@
"""The embedding refresh must LOSE to a delete, not race it (#3262).
Both upserts replace a record's vectors as delete-then-insert. That takes two
row locks the chunk rows, then the parent row via the insert's foreign key —
in the exact reverse of the order a cascading delete of the parent takes them.
Postgres calls that a deadlock and kills one side at random, which sometimes
means killing the user's delete.
These pin the ORDER, not the outcome: the claim on the parent goes first, and
when the claim fails nothing else in the transaction runs. Compiling the
statement is the only way to assert on a lock mode without a database the
integration twin (test_integration_embedding_yields_to_delete.py) proves the
behaviour against a real one.
"""
from unittest.mock import AsyncMock, MagicMock, patch
from sqlalchemy.dialects import postgresql
from sqlalchemy.exc import OperationalError
from scribe.services import embeddings as emb
from tests.helpers import compiled_sql
ONE_VECTOR = [[0.0] * 384]
# The lock mode is a Postgres extension — the generic dialect renders a plain
# FOR UPDATE and would pass an assertion that proves nothing.
PG = postgresql.dialect()
def _mock_session(lock_result: object = 7, execute_side_effect=None):
"""A session stand-in whose first execute answers the parent-row claim."""
session = MagicMock()
claimed = MagicMock()
claimed.scalar_one_or_none.return_value = lock_result
if execute_side_effect is not None:
session.execute = AsyncMock(side_effect=execute_side_effect)
else:
session.execute = AsyncMock(return_value=claimed)
session.commit = AsyncMock()
session.add = MagicMock()
ctx = MagicMock()
ctx.__aenter__ = AsyncMock(return_value=session)
ctx.__aexit__ = AsyncMock(return_value=False)
return session, ctx
def _lock_unavailable() -> OperationalError:
"""What asyncpg raises through SQLAlchemy when NOWAIT can't take the row."""
return OperationalError("SELECT ...", {}, Exception("lock not available"))
async def test_a_note_refresh_claims_the_row_before_rewriting_its_vectors():
"""The claim is FIRST, and it is FOR KEY SHARE NOWAIT.
FOR KEY SHARE because that is exactly the lock the insert's foreign key
takes anyway it conflicts with a delete of the note and with nothing
else, so an ordinary edit is unaffected. NOWAIT because the whole point is
to lose immediately rather than queue up behind the delete and hold the
chunk rows while doing it.
"""
session, ctx = _mock_session()
with (
patch.object(emb, "async_session", return_value=ctx),
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
):
await emb.upsert_note_embedding(7, 42, "T", "a short body")
claim, replace = [c.args[0] for c in session.execute.call_args_list][:2]
assert compiled_sql(claim, dialect=PG).startswith("SELECT notes.id")
assert "FOR KEY SHARE NOWAIT" in compiled_sql(claim, dialect=PG)
assert compiled_sql(replace, dialect=PG).startswith("DELETE FROM note_embeddings")
session.add.assert_called()
async def test_a_rule_refresh_claims_the_row_before_rewriting_its_vectors():
"""The rule twin — the path the reported deadlock actually took."""
session, ctx = _mock_session()
with (
patch.object(emb, "async_session", return_value=ctx),
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
):
await emb.upsert_rule_embedding(9, "T", "a short statement", "on write")
claim, replace = [c.args[0] for c in session.execute.call_args_list][:2]
assert compiled_sql(claim, dialect=PG).startswith("SELECT rules.id")
assert "FOR KEY SHARE NOWAIT" in compiled_sql(claim, dialect=PG)
assert compiled_sql(replace, dialect=PG).startswith("DELETE FROM rule_embeddings")
session.add.assert_called()
async def test_a_record_being_deleted_is_left_alone_rather_than_raced():
"""The claim failing ends the write — it does not fall through to the
delete-and-insert that would take the locks in the losing order."""
session, ctx = _mock_session(execute_side_effect=_lock_unavailable())
with (
patch.object(emb, "async_session", return_value=ctx),
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
):
await emb.upsert_rule_embedding(9, "T", "a short statement", "on write")
assert session.execute.await_count == 1, "it stopped at the claim"
session.add.assert_not_called()
session.commit.assert_not_awaited()
async def test_a_record_already_gone_is_not_re_embedded():
"""A vector inserted for a row that no longer exists is either a foreign
key violation or, worse, a resurrected chunk. Nothing to refresh."""
session, ctx = _mock_session(lock_result=None)
with (
patch.object(emb, "async_session", return_value=ctx),
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
):
await emb.upsert_note_embedding(7, 42, "T", "a short body")
assert session.execute.await_count == 1
session.add.assert_not_called()
session.commit.assert_not_awaited()
-128
View File
@@ -1,128 +0,0 @@
"""Every request the web UI makes has a deadline (rule 156).
A source-inspection guard in the unit lane there is no frontend test runner,
and this is a property of the source rather than of a rendered result, so
reading the source is the honest way to check it.
WHY. `fetch`'s default is to wait as long as the browser will. That is not a
long timeout, it is the absence of one, and there is no state a surface can
render for "pending forever" that is not a lie the spinner that never
resolves is indistinguishable from work still in progress. Rule 156 names
`fetch` specifically:
When a library's default is "wait indefinitely" — `fetch`, most HTTP
clients, a bare `await` on a stream supplying the deadline is part of
using it, not a hardening pass for later.
Before this guard, no request in the app carried one.
"""
from __future__ import annotations
import re
from pathlib import Path
FRONTEND = Path(__file__).resolve().parents[1] / "frontend" / "src"
CLIENT = FRONTEND / "api" / "client.ts"
def _call_text(src: str, start: int) -> str:
"""The source of one `fetch(...)` call, from its open paren to its close.
Naive paren balancing. Adequate because every call site here passes an
object literal, and a construct complex enough to defeat it is one worth
looking at by hand anyway.
"""
depth = 0
for i in range(start, len(src)):
if src[i] == "(":
depth += 1
elif src[i] == ")":
depth -= 1
if depth == 0:
return src[start:i + 1]
return src[start:]
def _fetch_calls() -> list[tuple[Path, str]]:
calls: list[tuple[Path, str]] = []
for path in list(FRONTEND.rglob("*.ts")) + list(FRONTEND.rglob("*.vue")):
src = path.read_text()
for m in re.finditer(r"\bfetch\(", src):
calls.append((path, _call_text(src, m.end() - 1)))
return calls
def test_every_fetch_passes_a_signal():
"""No bare `fetch` anywhere in the frontend.
Stated on the SIGNAL rather than on a timeout value, because the two
legitimate shapes here produce different values and only share this: an
ordinary call takes the client's default, a stream bounds its CONNECT and
then deliberately runs unbounded, and a bulk transfer passes minutes. What
they must all do is pass something.
"""
naked = [
f"{path.relative_to(FRONTEND)}: {call[:70]}"
for path, call in _fetch_calls()
if "signal:" not in call
]
assert not naked, (
"these fetch calls carry no AbortSignal, so they wait forever "
"(rule 156):\n " + "\n ".join(naked)
)
def test_the_client_applies_its_deadline_by_default():
"""The specific regression that would silently undo this.
An earlier pass (#3329) made `timeoutMs` OPT-IN and used it at exactly one
call site, which left ~330 others waiting forever while the mechanism
looked present. Reverting to that shape would not fail the guard above
every call would still reach `fetch` through `request()` so the default
is pinned here separately.
`??` is the operative character: `opts?.timeoutMs || DEFAULT` would treat
an explicit 0 as "use the default", and `opts?.timeoutMs` alone would
reinstate the opt-in bug.
"""
src = CLIENT.read_text()
assert "opts?.timeoutMs ?? DEFAULT_TIMEOUT_MS" in src, (
"request() must fall back to DEFAULT_TIMEOUT_MS — without it the "
"deadline is opt-in again and almost nothing opts in"
)
def test_a_timeout_arrives_as_the_error_shape_callers_already_handle():
"""Rule 156's second half: expiry surfaces as a NAMED failure.
A raw `DOMException: TimeoutError` reaches `apiErrorMessage(e, fallback)`
as an object with no `body`, so every catch site in the app would print its
generic fallback and the timeout would be invisible in exactly the
situation it exists to expose. Rethrowing as `ApiError` is what makes the
other ~330 call sites report it without being edited.
"""
src = CLIENT.read_text()
assert 'e.name === "TimeoutError"' in src, (
"request() must recognise a timeout specifically"
)
assert "new ApiError(CLIENT_TIMEOUT_STATUS" in src, (
"a timeout must be rethrown as ApiError so apiErrorMessage can read it"
)
def test_a_deliberate_cancellation_is_not_reported_as_a_timeout():
"""Only `TimeoutError` is converted, never `AbortError`.
A caller that cancelled its own request a superseded search, a closed
stream must not have that surfaced to the user as a server failure. The
guard is that the conversion is gated on the name, which the assertion
above already pins; this states the intent so the gate is not "simplified"
into catching every abort.
"""
src = CLIENT.read_text()
convert = src[src.index("async function request<"):]
convert = convert[:convert.index("\n}")]
assert "AbortError" not in convert, (
"request() must not convert AbortError — a deliberate cancellation is "
"not a timeout"
)

Some files were not shown because too many files have changed in this diff Show More