Compare commits
62
Commits
93d660b710
...
dev
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
277f5df515 | ||
|
|
ab14f783e1 | ||
|
|
1cfbf43ccd | ||
|
|
a165483b92 | ||
|
|
e7c1af32a0 | ||
|
|
277aea58e4 | ||
|
|
5e19a1b028 | ||
|
|
7a2aff7bc1 | ||
|
|
0808e8259a | ||
|
|
950c93c5d4 | ||
|
|
21a5831479 | ||
|
|
dd1e6e2645 | ||
|
|
c3ecdf0972 | ||
|
|
1a34363059 | ||
|
|
30d87e461a | ||
|
|
8be555d6dd | ||
|
|
48804c437d | ||
|
|
154a5de13e | ||
|
|
2ee24b9d2b | ||
|
|
8b9b3a1d9b | ||
|
|
6627cfc2f0 | ||
|
|
238510080e | ||
|
|
8901c904a9 | ||
|
|
70761b16d9 | ||
|
|
8f7f447fda | ||
|
|
111eef7e30 | ||
|
|
8826be7a91 | ||
|
|
e029a7db64 | ||
|
|
9bb59b73ba | ||
|
|
f5a3643da8 | ||
|
|
64cb719a12 | ||
|
|
f1896bfe9d | ||
|
|
ea972ac3f7 | ||
|
|
0d4b155699 | ||
|
|
05da26eb24 | ||
|
|
70d84fbfd7 | ||
|
|
9d8104f7a5 | ||
|
|
7827b4ce63 | ||
|
|
69ce7afc45 | ||
|
|
7985f8c7d7 | ||
|
|
efabba58dd | ||
|
|
5c9bb40777 | ||
|
|
a8b2040216 | ||
|
|
0704988528 | ||
|
|
255c43a8fe | ||
|
|
6fa66f202b | ||
|
|
7a0dc93270 | ||
|
|
9006affda8 | ||
|
|
9657478500 | ||
|
|
1d65e98ac2 | ||
|
|
1ec44071d2 | ||
|
|
b51621fca7 | ||
|
|
8489206224 | ||
|
|
4736a0a0ba | ||
|
|
700ef20eb0 | ||
|
|
b134fe9aa1 | ||
|
|
a6ef3a6a5a | ||
|
|
2263fd04a4 | ||
|
|
2065781302 | ||
|
|
454c617ca0 | ||
|
|
f80401d58e | ||
|
|
d0a2733cb6 |
@@ -46,8 +46,6 @@ on:
|
||||
- "alembic/**"
|
||||
- "alembic.ini"
|
||||
- "Dockerfile"
|
||||
- "assets/**"
|
||||
- "fable-mcp/**"
|
||||
# The plugin ships straight from this repo — installs fetch it via
|
||||
# .claude-plugin/marketplace.json, NOT from the image. So a push here is
|
||||
# the release, with no build step in between. Omitting these paths meant
|
||||
@@ -279,6 +277,21 @@ jobs:
|
||||
env:
|
||||
UV_PROJECT_ENVIRONMENT: /opt/venv
|
||||
run: uv sync --locked --extra dev
|
||||
# Standing answers to the checks carried by rules 81 and 79 — two facts
|
||||
# about THIS runner that conditional rules assert as fact, and that
|
||||
# otherwise need a throwaway job to confirm (#3237). Printing them on
|
||||
# every integration run makes the next rulebook sweep a log read.
|
||||
# Rule 80's evidence is the container listing the next step already
|
||||
# prints. Every command is guarded: a diagnostic that can break the lane
|
||||
# it observes is worse than no diagnostic.
|
||||
- name: Runner facts (rules 79 and 81)
|
||||
run: |
|
||||
echo "--- rule 81: which shell runs a run: step ---"
|
||||
readlink -f /bin/sh || echo "/bin/sh: not a symlink"
|
||||
ps -p $$ -o comm= || true
|
||||
echo "--- rule 79: is a service reachable by its hostname yet? ---"
|
||||
getent hosts postgres \
|
||||
|| echo "no — 'postgres' does not resolve; the bridge-IP lookup is still required"
|
||||
- name: Integration suite (resolve service IP, migrate, test)
|
||||
run: |
|
||||
set -eux
|
||||
@@ -289,8 +302,9 @@ jobs:
|
||||
PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
|
||||
test -n "$PG_IP"
|
||||
export DATABASE_URL="postgresql+asyncpg://scribe:ci_integration@${PG_IP}:5432/scribe_test"
|
||||
# Wait for Postgres to accept connections (busybox sh — the runner
|
||||
# default — has no bash /dev/tcp, so use Python).
|
||||
# Wait for Postgres to accept connections. The run: shell is dash
|
||||
# (/bin/sh -> /usr/bin/dash on this Debian-based image, confirmed by
|
||||
# the step above) — no bash /dev/tcp, so use Python.
|
||||
/opt/venv/bin/python - "$PG_IP" <<'PY'
|
||||
import socket, sys, time
|
||||
for _ in range(30):
|
||||
@@ -327,6 +341,14 @@ jobs:
|
||||
packages: write
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
# Rule 149 asks for this on any job deriving the version NAME. The
|
||||
# name here comes from HEAD's commit TIME, which a depth-1 clone
|
||||
# already has — but the rule states it unconditionally because the
|
||||
# failure it guards is silent (a too-low value, every lane green),
|
||||
# and a later change to how the name is derived would inherit the
|
||||
# landmine rather than the guard.
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Generate image tags and version
|
||||
id: tags
|
||||
@@ -339,7 +361,27 @@ jobs:
|
||||
# the runner log on commit 2a374d9.
|
||||
run: |
|
||||
TAGS="${{ env.IMAGE }}:${{ github.sha }}"
|
||||
BUILD_VERSION="dev"
|
||||
|
||||
# THREE VALUES, NEVER FOLDED TOGETHER (rule 149). Until 2026-08-31
|
||||
# BUILD_VERSION was the CHANNEL — "dev" / "main" / the tag — so the
|
||||
# image self-reported {"version":"main"}, a channel name where a
|
||||
# build identifier belongs. That cost a debugging session: with the
|
||||
# deploy misbehaving, nothing on the running instance could say
|
||||
# which commit was serving it.
|
||||
|
||||
# 1. ORDERING KEY — BUILD time, monotonic by construction. Minutes
|
||||
# since 2020-01-01. Never a commit count (not monotonic across
|
||||
# branches) and never commit time (goes DOWN when an older
|
||||
# commit is rebuilt).
|
||||
BUILD_KEY=$(( ( $(date -u +%s) - 1577836800 ) / 60 ))
|
||||
|
||||
# 2. NAME — COMMIT time, so the same source reports the same string
|
||||
# on every lane and the channel is the only thing that differs.
|
||||
COMMIT_TS=$(git log --format=%ct -1 HEAD)
|
||||
BUILD_NAME=$(date -u -d "@$COMMIT_TS" +%Y.%m.%d.%H%M)
|
||||
|
||||
# 3. CHANNEL — its own value. Never a suffix, never a segment.
|
||||
CHANNEL="dev"
|
||||
case "${{ github.ref }}" in
|
||||
refs/heads/dev)
|
||||
TAGS="$TAGS,${{ env.IMAGE }}:dev"
|
||||
@@ -348,15 +390,17 @@ jobs:
|
||||
# main IS the production line: publish :latest (plus the :<sha>
|
||||
# set above). No separate :main tag.
|
||||
TAGS="$TAGS,${{ env.IMAGE }}:latest"
|
||||
BUILD_VERSION="main"
|
||||
CHANNEL="stable"
|
||||
;;
|
||||
refs/tags/*)
|
||||
TAGS="$TAGS,${{ env.IMAGE }}:latest,${{ env.IMAGE }}:${{ github.ref_name }}"
|
||||
BUILD_VERSION="${{ github.ref_name }}"
|
||||
CHANNEL="stable"
|
||||
;;
|
||||
esac
|
||||
echo "value=$TAGS" >> $GITHUB_OUTPUT
|
||||
echo "build_version=$BUILD_VERSION" >> $GITHUB_OUTPUT
|
||||
echo "build_name=$BUILD_NAME" >> $GITHUB_OUTPUT
|
||||
echo "build_key=$BUILD_KEY" >> $GITHUB_OUTPUT
|
||||
echo "channel=$CHANNEL" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Free disk space
|
||||
# Self-hosted runner housekeeping. Two-step cleanup:
|
||||
@@ -386,7 +430,15 @@ jobs:
|
||||
push: true
|
||||
provenance: false
|
||||
tags: ${{ steps.tags.outputs.value }}
|
||||
build-args: BUILD_VERSION=${{ steps.tags.outputs.build_version }}
|
||||
# All three, plus the commit — rule 145: the registry's identity for
|
||||
# a build (:<sha>) and the artifact's identity for itself must
|
||||
# agree, and they can only be checked against each other if the
|
||||
# artifact says which commit it is.
|
||||
build-args: |
|
||||
BUILD_VERSION=${{ steps.tags.outputs.build_name }}
|
||||
BUILD_KEY=${{ steps.tags.outputs.build_key }}
|
||||
BUILD_CHANNEL=${{ steps.tags.outputs.channel }}
|
||||
BUILD_COMMIT=${{ github.sha }}
|
||||
# Registry-backed layer cache. Pull from :cache to prime
|
||||
# BuildKit, push updated layers back to :cache so the next
|
||||
# build starts warm even if the runner's local cache was
|
||||
|
||||
+21
-2
@@ -41,10 +41,29 @@ COPY alembic/ alembic/
|
||||
# Ensure Python finds the source tree (where static files live) before site-packages
|
||||
ENV PYTHONPATH=/app/src
|
||||
|
||||
# Version is injected at build time via --build-arg BUILD_VERSION=YY.MM.DD.N
|
||||
# Falls back to "dev" for local / untagged builds
|
||||
# THREE VALUES, NEVER FOLDED TOGETHER (rule 149), plus the commit.
|
||||
#
|
||||
# BUILD_VERSION is the NAME (YYYY.MM.DD.HHMM, from COMMIT time) — the same
|
||||
# string on every lane for the same source, so it answers "is this the same
|
||||
# code?" rather than "which lane built it?".
|
||||
# BUILD_KEY is the ORDERING KEY (minutes since 2020-01-01, from BUILD time) —
|
||||
# the only value anything may compare to decide what is newer.
|
||||
# BUILD_CHANNEL is its own field. Never a suffix, never a segment of the name.
|
||||
# BUILD_COMMIT lets the artifact's self-report be checked against the :<sha>
|
||||
# it was published under (rule 145).
|
||||
#
|
||||
# Each defaults to empty rather than to a placeholder, EXCEPT the name: a
|
||||
# local build genuinely has no ordering key or channel, and the endpoint says
|
||||
# so by omitting them. Inventing values would make a local image claim a
|
||||
# position in an update order it is not part of.
|
||||
ARG BUILD_VERSION=dev
|
||||
ARG BUILD_KEY=
|
||||
ARG BUILD_CHANNEL=
|
||||
ARG BUILD_COMMIT=
|
||||
ENV APP_VERSION=$BUILD_VERSION
|
||||
ENV APP_BUILD_KEY=$BUILD_KEY
|
||||
ENV APP_CHANNEL=$BUILD_CHANNEL
|
||||
ENV APP_COMMIT=$BUILD_COMMIT
|
||||
|
||||
EXPOSE 5000
|
||||
CMD ["sh", "-c", "alembic upgrade head && hypercorn 'scribe.app:create_app()' --bind 0.0.0.0:5000 --keep-alive 600"]
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
.PHONY: build up down logs health migrate lint typecheck test fmt
|
||||
.PHONY: build up down logs health migrate lint typecheck test fmt mint-plugin
|
||||
|
||||
# --- Docker ---
|
||||
|
||||
@@ -36,3 +36,12 @@ test:
|
||||
|
||||
# Run all checks in one shot (mirrors what CI does)
|
||||
check: lint typecheck test
|
||||
|
||||
# --- Plugin ---
|
||||
|
||||
# Run this after changing anything under plugin/ or .claude-plugin/, BEFORE
|
||||
# committing. The plugin ships straight from git with no build step, so its
|
||||
# version is minted here rather than stamped by CI; the lane fails if you
|
||||
# forget, but this is what makes remembering cheap.
|
||||
mint-plugin:
|
||||
python3 scripts/mint_plugin_version.py
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
"""a note can carry its own check — verify_with, expires_when, verified_at
|
||||
(milestone 317 step 1)
|
||||
|
||||
Revision ID: 0092
|
||||
Revises: 0091
|
||||
Create Date: 2026-08-28
|
||||
|
||||
The sibling of 0090, which gave rules the same three columns. Same
|
||||
distinction, one table over:
|
||||
|
||||
A NORM is a decision — no truth value, and it changes only when its author
|
||||
changes it, which they know they did. A CONSTRAINT is a fact about someone
|
||||
else's software, and nobody is present when it goes false.
|
||||
|
||||
Notes hold far more constraints than rules do, and hold them for longer. A
|
||||
cross-project reference note asserting what a signing service does on a
|
||||
duplicate upload, or how a forge numbers its CI runs, is believed by every
|
||||
project that reads it, and there is nothing in the record that says when
|
||||
anyone last looked. `note_supersessions` only fires once a human has read
|
||||
the note, disagreed, and written the correction — which is the case where
|
||||
the note was already believed.
|
||||
|
||||
Three nullable columns:
|
||||
|
||||
- `verify_with` — how to tell whether this is still true. A command, a path,
|
||||
a URL, a query. Prose is allowed; something runnable is better.
|
||||
- `expires_when` — the STATE under which it stops being true. Deliberately
|
||||
not a date: constraints do not expire on a schedule, they expire when the
|
||||
world underneath them moves.
|
||||
- `verified_at` — when the check last passed. NULL means never checked, and
|
||||
sorts FIRST in the sweep: unexamined outranks examined-long-ago.
|
||||
|
||||
WHICH ROWS THESE ARE FOR. `notes` is one table holding notes, tasks,
|
||||
snippets and processes, so these columns land on all of them. Only non-task,
|
||||
non-snippet records are OFFERED them (milestone 317 decisions 1 and 2, gated
|
||||
at the service in step 2): a task's decay is its status, and a snippet
|
||||
already carries a richer, location-aware verdict in `data.verification`. The
|
||||
columns exist on the other rows and stay null there; a gate that lives in
|
||||
the schema would have meant a partial index or a CHECK across three columns
|
||||
to express something the write path can say in two lines.
|
||||
|
||||
All three optional, because most notes should set none of them — the whole
|
||||
value of the sweep is that its output is short. A null `verify_with` is not
|
||||
an omission; it is the honest marker of "this one is a decision, and there
|
||||
is nothing to go and check."
|
||||
|
||||
No CHECK constraint is involved, so rule 36 does not apply. Nothing is
|
||||
backfilled: a migration cannot invent a check.
|
||||
"""
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0092"
|
||||
down_revision = "0091"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("notes", sa.Column("verify_with", sa.Text(), nullable=True))
|
||||
op.add_column("notes", sa.Column("expires_when", sa.Text(), nullable=True))
|
||||
op.add_column(
|
||||
"notes",
|
||||
sa.Column("verified_at", sa.DateTime(timezone=True), nullable=True),
|
||||
)
|
||||
# No index, for 0090's reason — the sweep runs when a human asks, never on
|
||||
# a request path — but the margin is thinner here and worth naming. `rules`
|
||||
# is hundreds of rows; `notes` is thousands and grows with every session.
|
||||
#
|
||||
# Still a sequential scan's job at this size, and an index on
|
||||
# (verified_at) filtered to `verify_with IS NOT NULL` would be maintained
|
||||
# on every note write to serve one operator-initiated query. If step 3's
|
||||
# live acceptance measures otherwise, add it there against a real plan
|
||||
# rather than guessing here.
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("notes", "verified_at")
|
||||
op.drop_column("notes", "expires_when")
|
||||
op.drop_column("notes", "verify_with")
|
||||
@@ -0,0 +1,83 @@
|
||||
"""rules gain an edit history — rule_versions (milestone 323 step 1)
|
||||
|
||||
Revision ID: 0093
|
||||
Revises: 0092
|
||||
Create Date: 2026-08-29
|
||||
|
||||
The sibling `note_versions` has had for a long time. A note's every meaningful
|
||||
edit is snapshotted, and the design-system note calls that history "the
|
||||
changelog". A RULE — which binds behaviour on every session that loads it —
|
||||
had nothing: an edit destroyed what it used to say, with no record anywhere.
|
||||
|
||||
Rescoping rule 79 on 2026-08-29 is what surfaced it. The superseded statement
|
||||
had to be hand-copied into a task log to survive the edit (#3237), which is
|
||||
not a process, it is a person remembering. The more consequential record had
|
||||
the weaker protection.
|
||||
|
||||
Three things are deliberately NOT copied from note_versions, and each is a
|
||||
guard that exists there for a reason that does not hold here:
|
||||
|
||||
- **No pruning, and no MAX_VERSIONS.** That cap defends against note autosave
|
||||
filling every slot. Rules have no autosave; every edit is a deliberate
|
||||
update_rule. A rule is edited a handful of times in its life, and capping
|
||||
invites losing the one edit somebody needed.
|
||||
- **No pin columns.** `pin_kind`/`pin_label` exist so a note's version can
|
||||
survive that pruning. With nothing pruning, a pin protects a row that was
|
||||
never at risk.
|
||||
- **No minimum interval.** 300 seconds between snapshots is also an autosave
|
||||
defence; here it would only ever discard a second deliberate edit.
|
||||
|
||||
`user_id` is the ACTOR rather than the owner, and is SET NULL rather than
|
||||
CASCADE: deleting a user must not erase the history of the rules they edited.
|
||||
The edit still happened and the rule still binds because of it.
|
||||
|
||||
No CHECK constraint, so rule 36 does not apply. Nothing is backfilled — a
|
||||
migration cannot invent the text a rule used to have, and inventing "the
|
||||
current text, as of now" would be worse than an empty history, because it
|
||||
would look like a record of an edit that never occurred.
|
||||
"""
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision = "0093"
|
||||
down_revision = "0092"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"rule_versions",
|
||||
sa.Column("id", sa.BigInteger(), primary_key=True),
|
||||
sa.Column(
|
||||
"rule_id",
|
||||
sa.BigInteger(),
|
||||
sa.ForeignKey("rules.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"user_id",
|
||||
sa.BigInteger(),
|
||||
sa.ForeignKey("users.id", ondelete="SET NULL"),
|
||||
nullable=True,
|
||||
),
|
||||
sa.Column("title", sa.Text(), nullable=False, server_default=""),
|
||||
sa.Column("statement", sa.Text(), nullable=False, server_default=""),
|
||||
sa.Column("why", sa.Text(), nullable=True),
|
||||
sa.Column("how_to_apply", sa.Text(), nullable=True),
|
||||
sa.Column("when_to_apply", sa.Text(), nullable=True),
|
||||
sa.Column("tier", sa.Text(), nullable=True),
|
||||
sa.Column("verify_with", sa.Text(), nullable=True),
|
||||
sa.Column("expires_when", sa.Text(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
)
|
||||
# The only query this table serves is "the history of THIS rule, newest
|
||||
# first" — unlike 0092's columns, which are read by an operator-initiated
|
||||
# sweep over the whole set. Every read here is keyed on rule_id, so the
|
||||
# index earns its write cost immediately rather than on a hunch.
|
||||
op.create_index("ix_rule_versions_rule_id", "rule_versions", ["rule_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_rule_versions_rule_id", table_name="rule_versions")
|
||||
op.drop_table("rule_versions")
|
||||
@@ -0,0 +1,86 @@
|
||||
"""add rule_usage_events — was a surfaced rule ever read? (milestone 333 step 1)
|
||||
|
||||
Revision ID: 0094
|
||||
Revises: 0093
|
||||
Create Date: 2026-09-02
|
||||
|
||||
The sibling `note_usage_events` has had since 0071, and the third rule-side
|
||||
table to arrive after `rule_embeddings` and `rule_versions` — each one added
|
||||
because the rule side kept inheriting machinery built for notes and getting
|
||||
the weaker version of it.
|
||||
|
||||
WHAT IT MEASURES. The write-path standing-rule arm is the only retrieval
|
||||
surface in Scribe whose usefulness cannot be observed, and — not coincidentally
|
||||
— the only one that has never declined to fire. Over 30 days it took 296 calls,
|
||||
returned something on every one, and cleared its threshold 100% of the time,
|
||||
while every other surface declines most of the time (#3311). That is either a
|
||||
perfectly tuned surface or a bar it cannot fail to clear, and `retrieval_logs`
|
||||
cannot tell them apart: it records what the ranker scored, never whether the
|
||||
hint was any use.
|
||||
|
||||
WHY NOT A rule_id COLUMN ON note_usage_events. The row shares no note-specific
|
||||
fields and the aggregate readout is the same shape, which is the strongest case
|
||||
for sharing that note #3163 admits. What decides against it is identity at
|
||||
RESTORE: `note_usage_events`'s importer maps `note_id` through `note_id_map`
|
||||
and drops what does not resolve. A rule id parked in that column would come
|
||||
back from a backup silently reattached to whatever note took that number —
|
||||
telemetry not merely lost but wrong, and wrong in a way nothing downstream
|
||||
could detect. `rule_versions` made the same call for the same reason.
|
||||
|
||||
FK-free on `rule_id` and `user_id`, matching note_usage_events, retrieval_logs
|
||||
and app_logs — and deliberately unlike `rule_versions`, which does carry FKs.
|
||||
The difference is what the row is for: a version belongs to a rule's history
|
||||
and dies with it; telemetry outlives the row it describes. Deleting a rule must
|
||||
not erase the evidence that it was surfaced forty times and opened never, since
|
||||
that evidence is exactly the case for having deleted it.
|
||||
|
||||
No CHECK on `event`, matching the note twin. Rule 36 governs adding a value to
|
||||
a column that is already gated; it does not require gating one that never was,
|
||||
and a two-member enum whose members are written by two functions in one module
|
||||
is not where that discipline earns its cost.
|
||||
|
||||
Downgrade drops the table outright. The data is purely observational — nothing
|
||||
reads it for correctness, so losing it costs history and no behaviour.
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0094"
|
||||
down_revision = "0093"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"rule_usage_events",
|
||||
# BigInteger throughout where the note twin uses Integer: rules.id is
|
||||
# BigInteger, so rule_id must be, and a high-churn append-only table is
|
||||
# a poor place to discover an id ceiling.
|
||||
sa.Column("id", sa.BigInteger(), primary_key=True),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
nullable=False,
|
||||
server_default=sa.text("now()"),
|
||||
),
|
||||
sa.Column("user_id", sa.BigInteger(), nullable=True),
|
||||
sa.Column("rule_id", sa.BigInteger(), nullable=False),
|
||||
sa.Column("event", sa.Text(), nullable=False),
|
||||
sa.Column("source", sa.Text(), nullable=False),
|
||||
)
|
||||
# Every readout is "these rule ids, split by event", so the composite is the
|
||||
# one that actually gets used; the others serve pruning and per-user views.
|
||||
op.create_index(
|
||||
"ix_rule_usage_rule_event", "rule_usage_events", ["rule_id", "event"]
|
||||
)
|
||||
op.create_index("ix_rule_usage_created_at", "rule_usage_events", ["created_at"])
|
||||
op.create_index("ix_rule_usage_user_id", "rule_usage_events", ["user_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_rule_usage_user_id", table_name="rule_usage_events")
|
||||
op.drop_index("ix_rule_usage_created_at", table_name="rule_usage_events")
|
||||
op.drop_index("ix_rule_usage_rule_event", table_name="rule_usage_events")
|
||||
op.drop_table("rule_usage_events")
|
||||
@@ -0,0 +1,52 @@
|
||||
"""add retrieval_logs.suppressed_count — tell a ranker decline from a repeat (#3497)
|
||||
|
||||
Revision ID: 0095
|
||||
Revises: 0094
|
||||
Create Date: 2026-09-03
|
||||
|
||||
`result_count == 0` has always meant "this surface said nothing", which is the
|
||||
right number for "was the hint any use" and the wrong one for tuning a
|
||||
threshold. It folds together two unrelated events:
|
||||
|
||||
- the ranker found nothing above the bar — the ONLY evidence a threshold is
|
||||
set too high; and
|
||||
- the ranker found something the session had already been shown — a decline
|
||||
that says nothing whatever about the bar.
|
||||
|
||||
The rule arms filter in Python after the search, so they can count the second
|
||||
kind exactly. The note arms pass `exclude_ids` INTO semantic_search_notes, so
|
||||
the dropped rows never come back and there is nothing to count.
|
||||
|
||||
NULLABLE, AND THE NULL IS THE POINT. A surface that does not measure
|
||||
suppression stores NULL, not 0, and the readout renders it as "not measured"
|
||||
rather than "none". Defaulting to 0 would make an unmeasured surface look like
|
||||
a perfectly clean one — the exact substitution of an artifact for a
|
||||
measurement that #3311 made and that #3497 exists to correct. Doing it again,
|
||||
in the migration that fixes it, would be its own small joke.
|
||||
|
||||
No backfill for the same reason: existing rows genuinely do not know, and
|
||||
saying so is the honest state. `retrieval_logs` is not restored from backup,
|
||||
so no importer changes.
|
||||
|
||||
Downgrade drops the column. Purely observational — nothing reads it for
|
||||
correctness.
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0095"
|
||||
down_revision = "0094"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"retrieval_logs",
|
||||
sa.Column("suppressed_count", sa.Integer(), nullable=True),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("retrieval_logs", "suppressed_count")
|
||||
@@ -0,0 +1,62 @@
|
||||
"""add retrieval_logs.best_available_score — the score the bar rejected (#3670)
|
||||
|
||||
Revision ID: 0096
|
||||
Revises: 0095
|
||||
Create Date: 2026-09-08
|
||||
|
||||
`cleared_threshold` was documented as the number to read FIRST — "a surface
|
||||
that clears its bar on nearly every call is either well-tuned or too loose,
|
||||
and p10 says which". It was never a measurement. The search applies the
|
||||
threshold before returning, so every returned result cleared the bar by
|
||||
construction and a call with no results has no `top_score` to compare:
|
||||
the condition is true exactly when `result_count > 0`.
|
||||
|
||||
`zero_result_calls + cleared_threshold == calls` held on all nineteen
|
||||
source/window readings ever taken. It was `calls - zero_result_calls`
|
||||
wearing a name that promised a second opinion, and a reading procedure was
|
||||
built on top of it that asked the reader to compare a number against itself.
|
||||
|
||||
THE MISSING NUMBER, and the reason this is a column rather than a deletion.
|
||||
The question the table exists to answer is "is the bar in the right place",
|
||||
and that question is only answerable from the calls that returned NOTHING:
|
||||
how close did the best rejected candidate come? A bar at 0.72 turning away
|
||||
a stream of 0.71s is set too high by a hair. A bar turning away 0.30s is
|
||||
doing its job. Those two are indistinguishable today — both render as a
|
||||
zero-result call — and no arrangement of the existing columns separates
|
||||
them, because the losing score is discarded inside the search.
|
||||
|
||||
So the searches now rank without the bar and apply it in Python, which
|
||||
costs nothing (the rows were already ordered by distance, and the qualifying
|
||||
set is provably identical — above-threshold rows sort first), and the best
|
||||
score seen becomes observable.
|
||||
|
||||
NULLABLE, AND UNBACKFILLED, for the reason 0095 spells out: a row written
|
||||
before this shipped genuinely does not know what its best rejected candidate
|
||||
scored, and saying so is the honest state. A 0.0 default would read as "the
|
||||
corpus had nothing remotely relevant" — an artifact standing in for a
|
||||
measurement, which is the whole defect this milestone corrects.
|
||||
|
||||
`retrieval_logs` is not restored from backup, so no importer changes.
|
||||
|
||||
Downgrade drops the column. Purely observational — nothing reads it for
|
||||
correctness.
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision = "0096"
|
||||
down_revision = "0095"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"retrieval_logs",
|
||||
sa.Column("best_available_score", sa.Float(), nullable=True),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("retrieval_logs", "best_available_score")
|
||||
+19
-6
@@ -7,12 +7,20 @@ import { useTheme } from "@/composables/useTheme";
|
||||
import { useShortcuts } from "@/composables/useShortcuts";
|
||||
import { useAuthStore } from "@/stores/auth";
|
||||
import { useSettingsStore } from "@/stores/settings";
|
||||
import { apiGet, apiPut } from "@/api/client";
|
||||
import { apiPut } from "@/api/client";
|
||||
import { fetchVersion } from "@/api/version";
|
||||
|
||||
useTheme();
|
||||
|
||||
const router = useRouter();
|
||||
const appVersion = ref("dev");
|
||||
// THREE states, not two (#3127 checklist 12). `null` is "not answered yet" and
|
||||
// renders nothing; a string renders; `appVersionFailed` renders its own thing.
|
||||
// This used to default to the literal "dev" and swallow the error, which meant
|
||||
// an instance that could not answer was indistinguishable from a local build
|
||||
// that genuinely reports "dev" — a blank standing in for `unknown`, in the one
|
||||
// readout whose whole job is to say what is running.
|
||||
const appVersion = ref<string | null>(null);
|
||||
const appVersionFailed = ref(false);
|
||||
const authStore = useAuthStore();
|
||||
const settingsStore = useSettingsStore();
|
||||
const { showShortcuts, toggleShortcuts, closeShortcuts } = useShortcuts();
|
||||
@@ -119,10 +127,12 @@ onMounted(async () => {
|
||||
startAppServices();
|
||||
}
|
||||
try {
|
||||
const data = await apiGet<{ version: string }>("/api/version");
|
||||
appVersion.value = data.version;
|
||||
appVersion.value = (await fetchVersion()).version;
|
||||
} catch {
|
||||
// silent — version display is non-critical
|
||||
// Not silent any more: the footer says it could not find out, rather than
|
||||
// showing a version it never received. The full readout (version, channel,
|
||||
// commit, build) lives in Settings → Config.
|
||||
appVersionFailed.value = true;
|
||||
}
|
||||
});
|
||||
|
||||
@@ -151,7 +161,10 @@ onUnmounted(() => {
|
||||
<div id="main-content" class="app-content">
|
||||
<router-view />
|
||||
</div>
|
||||
<footer class="app-footer">v{{ appVersion }}</footer>
|
||||
<footer class="app-footer">
|
||||
<span v-if="appVersion">v{{ appVersion }}</span>
|
||||
<span v-else-if="appVersionFailed">version unknown</span>
|
||||
</footer>
|
||||
</div>
|
||||
|
||||
<!-- Keyboard shortcuts overlay -->
|
||||
|
||||
+127
-32
@@ -52,41 +52,121 @@ export function apiErrorMessage(e: unknown, fallback: string): string {
|
||||
return fallback;
|
||||
}
|
||||
|
||||
export async function apiGet<T>(path: string): Promise<T> {
|
||||
const res = await fetch(path);
|
||||
/**
|
||||
* How long an ordinary JSON call may wait before it is declared failed.
|
||||
*
|
||||
* Rule 156: a wait with no deadline is a bug. `fetch`'s own default is to wait
|
||||
* as long as the browser will, which is not a deadline — it is the absence of
|
||||
* one, and it renders as a spinner that never resolves. There is no state a
|
||||
* surface can show for "pending forever" that is not a lie.
|
||||
*
|
||||
* 30s is chosen to be longer than anything healthy: it has to clear a cold
|
||||
* embedding call and a list view under connection-pool contention (#2384 had
|
||||
* /api/projects fanning 25 concurrent sessions at a 15-connection pool), so
|
||||
* tripping it means something is genuinely wrong rather than merely busy. Slow
|
||||
* BY DESIGN is a different case and passes its own value — see the callers in
|
||||
* SettingsView that do.
|
||||
*/
|
||||
const DEFAULT_TIMEOUT_MS = 30_000;
|
||||
|
||||
/** HTTP 408. Not a status any Scribe route returns, so it unambiguously means
|
||||
* "the client gave up" rather than anything the server said. */
|
||||
const CLIENT_TIMEOUT_STATUS = 408;
|
||||
|
||||
/**
|
||||
* How long a STREAM may take to answer with its headers.
|
||||
*
|
||||
* Streams are the one case a wall-clock deadline would break: a long-lived SSE
|
||||
* connection is *supposed* to stay open, and `AbortSignal.timeout` would kill
|
||||
* it mid-flight along with the body. But that does not exempt them from rule
|
||||
* 156 — it relocates the deadline. Two different waits are involved:
|
||||
*
|
||||
* connect — the server answering with headers. CAN fail to answer, so it
|
||||
* carries this deadline, cleared the moment headers arrive.
|
||||
* stream — the body, open indefinitely on purpose. Its failure mode is
|
||||
* going quiet, which a timeout cannot tell from being idle; that
|
||||
* is what reconnection and Last-Event-ID are for, not this.
|
||||
*
|
||||
* Reading the connect as exempt because "the stream is long-lived" is the easy
|
||||
* mistake here, and it leaves an unreachable server looking like a quiet one.
|
||||
*/
|
||||
const STREAM_CONNECT_TIMEOUT_MS = 15_000;
|
||||
|
||||
/**
|
||||
* A signal that aborts if headers do not arrive in time, plus the `settle` to
|
||||
* call once they do. After `settle()` the returned signal never fires, so the
|
||||
* stream body runs unbounded — which is the intent.
|
||||
*/
|
||||
function connectDeadline(base: AbortSignal): { signal: AbortSignal; settle: () => void } {
|
||||
const gate = new AbortController();
|
||||
const timer = setTimeout(
|
||||
() => gate.abort(new DOMException("stream did not connect in time", "TimeoutError")),
|
||||
STREAM_CONNECT_TIMEOUT_MS,
|
||||
);
|
||||
return {
|
||||
signal: AbortSignal.any([base, gate.signal]),
|
||||
settle: () => clearTimeout(timer),
|
||||
};
|
||||
}
|
||||
|
||||
export interface RequestOpts {
|
||||
/** Override the deadline. Pass one when the call is slow BY DESIGN. */
|
||||
timeoutMs?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* The one place a request is actually made — every verb below goes through
|
||||
* here, so the deadline cannot be forgotten by adding a sixth.
|
||||
*
|
||||
* EXPIRY SURFACES AS AN `ApiError`, which is rule 156's second half: the
|
||||
* failure has to arrive in the shape the caller already handles. A bare
|
||||
* `DOMException: TimeoutError` would reach `apiErrorMessage(e, fallback)` as
|
||||
* an object with no `body`, so every catch site in the app would report its
|
||||
* generic fallback and the timeout would be invisible in the very situation it
|
||||
* exists to expose. Rethrowing as `ApiError` means ~330 existing call sites
|
||||
* report it correctly without being touched.
|
||||
*
|
||||
* Only a TIMEOUT is converted. A deliberate cancellation aborts with
|
||||
* `AbortError` and is left alone — a caller that cancelled its own request
|
||||
* does not want it reported as a server failure.
|
||||
*/
|
||||
async function request<T>(path: string, init: RequestInit, opts?: RequestOpts): Promise<T> {
|
||||
const timeoutMs = opts?.timeoutMs ?? DEFAULT_TIMEOUT_MS;
|
||||
let res: Response;
|
||||
try {
|
||||
res = await fetch(path, { ...init, signal: AbortSignal.timeout(timeoutMs) });
|
||||
} catch (e) {
|
||||
if (e instanceof DOMException && e.name === "TimeoutError") {
|
||||
throw new ApiError(CLIENT_TIMEOUT_STATUS, {
|
||||
error: `The server did not answer within ${Math.round(timeoutMs / 1000)}s.`,
|
||||
});
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
return handleResponse<T>(res, path);
|
||||
}
|
||||
|
||||
export async function apiPost<T>(path: string, body: unknown): Promise<T> {
|
||||
const res = await fetch(path, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
return handleResponse<T>(res, path);
|
||||
/** JSON body headers — the three write verbs sent an identical literal each. */
|
||||
const JSON_HEADERS = { "Content-Type": "application/json" };
|
||||
|
||||
export function apiGet<T>(path: string, opts?: RequestOpts): Promise<T> {
|
||||
return request<T>(path, {}, opts);
|
||||
}
|
||||
|
||||
export async function apiPut<T>(path: string, body: unknown): Promise<T> {
|
||||
const res = await fetch(path, {
|
||||
method: "PUT",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
return handleResponse<T>(res, path);
|
||||
export function apiPost<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
|
||||
return request<T>(path, { method: "POST", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
|
||||
}
|
||||
|
||||
export async function apiPatch<T>(path: string, body: unknown): Promise<T> {
|
||||
const res = await fetch(path, {
|
||||
method: "PATCH",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
return handleResponse<T>(res, path);
|
||||
export function apiPut<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
|
||||
return request<T>(path, { method: "PUT", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
|
||||
}
|
||||
|
||||
export async function apiDelete(path: string): Promise<void> {
|
||||
const res = await fetch(path, { method: "DELETE" });
|
||||
return handleResponse<void>(res, path);
|
||||
export function apiPatch<T>(path: string, body: unknown, opts?: RequestOpts): Promise<T> {
|
||||
return request<T>(path, { method: "PATCH", headers: JSON_HEADERS, body: JSON.stringify(body) }, opts);
|
||||
}
|
||||
|
||||
export function apiDelete(path: string, opts?: RequestOpts): Promise<void> {
|
||||
return request<void>(path, { method: "DELETE" }, opts);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -221,7 +301,14 @@ export function apiSSEStream(
|
||||
}
|
||||
|
||||
const done = (async () => {
|
||||
const res = await fetch(path, { headers, signal: combinedSignal });
|
||||
// Bounded connect, unbounded stream — see STREAM_CONNECT_TIMEOUT_MS.
|
||||
const connect = connectDeadline(combinedSignal);
|
||||
let res: Response;
|
||||
try {
|
||||
res = await fetch(path, { headers, signal: connect.signal });
|
||||
} finally {
|
||||
connect.settle();
|
||||
}
|
||||
if (!res.ok) {
|
||||
let body: Record<string, unknown> = {};
|
||||
try {
|
||||
@@ -318,11 +405,19 @@ export async function apiStreamPost(
|
||||
body: unknown,
|
||||
onChunk: (data: Record<string, unknown>) => void
|
||||
): Promise<void> {
|
||||
const res = await fetch(path, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
// Bounded connect, unbounded stream — see STREAM_CONNECT_TIMEOUT_MS.
|
||||
const connect = connectDeadline(new AbortController().signal);
|
||||
let res: Response;
|
||||
try {
|
||||
res = await fetch(path, {
|
||||
method: "POST",
|
||||
headers: JSON_HEADERS,
|
||||
body: JSON.stringify(body),
|
||||
signal: connect.signal,
|
||||
});
|
||||
} finally {
|
||||
connect.settle();
|
||||
}
|
||||
if (!res.ok) {
|
||||
let errBody: Record<string, unknown> = {};
|
||||
try {
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { RecordUsage } from "@/types/usage";
|
||||
|
||||
import { apiGet, apiPost, apiPatch, apiDelete } from "@/api/client";
|
||||
|
||||
/** How a rule reaches a session (milestone 307). */
|
||||
@@ -96,6 +98,13 @@ export interface RuleHeader {
|
||||
* A date (YYYY-MM-DD), or the literal "never".
|
||||
*/
|
||||
last_verified?: string;
|
||||
/**
|
||||
* Surfaced-vs-opened counts from `rule_usage_events` (milestone 333).
|
||||
* Zero-filled by the list route, so a rule predating the table reads as
|
||||
* "never surfaced" rather than as a missing field — which for a while is
|
||||
* every rule on every install.
|
||||
*/
|
||||
usage?: RecordUsage;
|
||||
}
|
||||
|
||||
export interface ApplicableRules {
|
||||
@@ -228,6 +237,49 @@ export async function unrelateRules(relationId: number): Promise<void> {
|
||||
return apiDelete(`/api/rule-relations/${relationId}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* One entry in a rule's edit history.
|
||||
*
|
||||
* Each entry holds the text the edit REPLACED, not the text it introduced —
|
||||
* so the newest entry is what the rule said before its most recent change,
|
||||
* and what that change produced is the rule as it stands now. Read the other
|
||||
* way round, every diff comes out backwards.
|
||||
*
|
||||
* The listing form omits the long fields; open one to get them.
|
||||
*/
|
||||
export interface RuleVersion {
|
||||
id: number;
|
||||
rule_id: number;
|
||||
/** Who made the edit. Null when that account has since been deleted. */
|
||||
user_id: number | null;
|
||||
title: string;
|
||||
created_at: string;
|
||||
statement?: string;
|
||||
why?: string;
|
||||
how_to_apply?: string;
|
||||
when_to_apply?: string;
|
||||
tier?: string;
|
||||
verify_with?: string;
|
||||
expires_when?: string;
|
||||
}
|
||||
|
||||
export async function listRuleVersions(ruleId: number): Promise<RuleVersion[]> {
|
||||
const data = await apiGet<{ versions: RuleVersion[] }>(
|
||||
`/api/rules/${ruleId}/versions`,
|
||||
);
|
||||
return data.versions;
|
||||
}
|
||||
|
||||
export async function getRuleVersion(
|
||||
ruleId: number, versionId: number,
|
||||
): Promise<RuleVersion> {
|
||||
return apiGet<RuleVersion>(`/api/rules/${ruleId}/versions/${versionId}`);
|
||||
}
|
||||
|
||||
// No restoreRuleVersion, deliberately (milestone 323). Putting an old wording
|
||||
// back goes through updateRule, which snapshots what it replaces — so the
|
||||
// undo stays visible in the history like any other edit.
|
||||
|
||||
export async function deleteRule(id: number): Promise<void> {
|
||||
return apiDelete(`/api/rules/${id}`);
|
||||
}
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { RecordUsage } from "@/types/usage";
|
||||
|
||||
import { apiGet, apiPost, apiPatch, apiDelete } from "@/api/client";
|
||||
|
||||
/** One canonical location of a reusable thing. A snippet that unifies several
|
||||
@@ -50,15 +52,11 @@ export interface Snippet {
|
||||
owner?: string | null;
|
||||
}
|
||||
|
||||
/** How often a record was put in front of an agent versus actually opened.
|
||||
* A high `surfaced_count` with `pull_count: 0` is dead weight — it occupies a
|
||||
* slot in every future auto-inject menu while never being used. */
|
||||
export interface SnippetUsage {
|
||||
surfaced_count: number;
|
||||
pull_count: number;
|
||||
last_surfaced_at: string | null;
|
||||
last_pulled_at: string | null;
|
||||
}
|
||||
/** Kept as a name because every consumer here says "snippet usage" — but it IS
|
||||
* the shared shape, since rules answer the same question off their own table
|
||||
* (milestone 333). The reasoning lives on `RecordUsage`; duplicating the four
|
||||
* fields here is how the two drift. */
|
||||
export type SnippetUsage = RecordUsage;
|
||||
|
||||
/** Result of the last drift check — does the recorded location and code still
|
||||
* match source? The check runs agent-side (Scribe has no checkout); this is the
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import { apiGet } from "./client";
|
||||
|
||||
/**
|
||||
* What `/api/version` answers — the client's half of `build_version_payload`
|
||||
* (`src/scribe/routes/api.py`), which is where the reasoning for the shape is
|
||||
* written down.
|
||||
*
|
||||
* EVERY FIELD BUT `version` IS OPTIONAL, and an absent one means "this build
|
||||
* does not know", not "empty". A local build has no ordering key and no
|
||||
* channel, and the server says so by omitting the keys rather than sending
|
||||
* `""` — emitting a placeholder would let it claim a position in an update
|
||||
* order it is not part of.
|
||||
*
|
||||
* So a renderer must read ABSENCE, never falsiness. `build` is a number and
|
||||
* `0` is a legitimate ordering key, so `v.build || "unknown"` would report a
|
||||
* real value as unknown; `v.build ?? "unknown"` is the correct form.
|
||||
*/
|
||||
export interface VersionPayload {
|
||||
/** The NAME — `YYYY.MM.DD.HHMM` from commit time. Answers "is this the same code?" */
|
||||
version: string;
|
||||
/** The ORDERING KEY — minutes since 2020-01-01, from build time. Absent on a local build. */
|
||||
build?: number;
|
||||
/** `dev` / `main` / a tag. Its own field, never folded into the name. */
|
||||
channel?: string;
|
||||
/** The commit the artifact was published under, so its claim can be checked against the registry. */
|
||||
commit?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* SHORTER than the client's 30s default, deliberately.
|
||||
*
|
||||
* This readout answers "what is running?" during an incident, which is exactly
|
||||
* when the server may be the thing that is unwell — and it is one static field
|
||||
* off a route that does no work, so a healthy instance answers it immediately.
|
||||
* Waiting the full default before saying so would leave a person staring at
|
||||
* "still loading" for half a minute in the moment they are trying to find out
|
||||
* whether the instance is alive at all. Eight seconds clears a slow-but-alive
|
||||
* instance and tells them something quickly when it is not.
|
||||
*/
|
||||
const VERSION_TIMEOUT_MS = 8000;
|
||||
|
||||
export function fetchVersion(): Promise<VersionPayload> {
|
||||
return apiGet<VersionPayload>("/api/version", { timeoutMs: VERSION_TIMEOUT_MS });
|
||||
}
|
||||
@@ -351,3 +351,29 @@
|
||||
|
||||
.required { color: var(--fs-error); }
|
||||
.field-hint { margin: 0.3rem 0 0; font-size: 0.8rem; color: var(--fs-text-tertiary); }
|
||||
|
||||
/* --- usage badge ----------------------------------------------------------
|
||||
"surfaced N×, opened M×" on a list row, for any record kind the retrieval
|
||||
surfaces can choose: snippets and notes from note_usage_events, rules from
|
||||
rule_usage_events. Promoted here from SnippetListView's scoped block when
|
||||
the rule list needed the same chip (milestone 333 step 5) — a second scoped
|
||||
copy is how the ninth duplicated CSS family starts (#3207).
|
||||
|
||||
Geometry and colour only. A view keeps its own spacing as a scoped
|
||||
remainder, the way it does for every other recipe in this file. */
|
||||
.usage-tag {
|
||||
font-size: 0.7rem;
|
||||
padding: 0.1rem 0.4rem;
|
||||
border-radius: 4px;
|
||||
white-space: nowrap;
|
||||
font-variant-numeric: tabular-nums;
|
||||
background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent);
|
||||
color: var(--fs-text-tertiary-fg);
|
||||
}
|
||||
|
||||
/* Dead weight is a nudge, not an error — it warns in the warning colour rather
|
||||
than the danger one, because the record isn't broken, just unearned. */
|
||||
.usage-tag.usage-dead {
|
||||
background: color-mix(in srgb, var(--fs-warning) 18%, transparent);
|
||||
color: var(--fs-warning-fg);
|
||||
}
|
||||
|
||||
@@ -127,7 +127,7 @@
|
||||
border: 1px solid var(--fs-error);
|
||||
border-radius: var(--fs-radius-sm);
|
||||
font-size: 0.85rem;
|
||||
color: var(--fs-error);
|
||||
color: var(--fs-error-fg);
|
||||
}
|
||||
.diff-view {
|
||||
border: 1px solid var(--fs-border-color);
|
||||
@@ -147,11 +147,11 @@
|
||||
}
|
||||
.diff-delete {
|
||||
background: color-mix(in srgb, var(--fs-error) 12%, transparent);
|
||||
color: var(--fs-error);
|
||||
color: var(--fs-error-fg);
|
||||
}
|
||||
.diff-insert {
|
||||
background: color-mix(in srgb, var(--fs-success) 12%, transparent);
|
||||
color: var(--fs-success);
|
||||
color: var(--fs-success-fg);
|
||||
}
|
||||
.diff-equal {
|
||||
color: var(--fs-text-tertiary);
|
||||
|
||||
@@ -31,6 +31,7 @@
|
||||
--fs-accent-faint: color-mix(in srgb, var(--fs-accent) 8%, transparent); /* The faintest accent wash */
|
||||
--fs-accent-deep: color-mix(in srgb, var(--fs-accent) 70%, black); /* The accent, darkened */
|
||||
--fs-accent-wash: color-mix(in srgb, var(--fs-accent) 22%, transparent); /* Heaviest accent tint */
|
||||
--fs-accent-fg: color-mix(in srgb, var(--fs-accent) 45%, var(--fs-text-primary)); /* Accent TEXT on an accent tint */
|
||||
--fs-gradient-cta: linear-gradient(135deg, var(--fs-accent), var(--fs-accent-deep));
|
||||
--fs-glow-cta: 0 2px 10px color-mix(in srgb, var(--fs-accent) 35%, transparent);
|
||||
--fs-glow-cta-hover: 0 4px 24px color-mix(in srgb, var(--fs-accent) 65%, transparent);
|
||||
@@ -102,8 +103,11 @@
|
||||
|
||||
/* semantic */
|
||||
--fs-success: var(--fs-action-primary);
|
||||
--fs-success-fg: color-mix(in srgb, var(--fs-success) 45%, var(--fs-text-primary)); /* Success TEXT on a success tint */
|
||||
--fs-warning: #8B6F1E;
|
||||
--fs-warning-fg: color-mix(in srgb, var(--fs-warning) 50%, var(--fs-text-primary)); /* Warning TEXT on a warning tint */
|
||||
--fs-error: #C04A1F;
|
||||
--fs-error-fg: color-mix(in srgb, var(--fs-error) 50%, var(--fs-text-primary)); /* Error TEXT on an error tint */
|
||||
--fs-info: #3D5A6E;
|
||||
--fs-destructive: #6B2118; /* irreversible — deliberately not the error colour */
|
||||
|
||||
@@ -148,7 +152,9 @@
|
||||
/* text */
|
||||
--fs-text-primary: #E8E4D8; /* body, headings, labels — inverts by mode */
|
||||
--fs-text-secondary: #C2BFB4;
|
||||
--fs-text-secondary-fg: color-mix(in srgb, var(--fs-text-secondary) 90%, var(--fs-text-primary)); /* Secondary TEXT on a secondary tint (barely moves; no exceptions) */
|
||||
--fs-text-tertiary: #9C9A92;
|
||||
--fs-text-tertiary-fg: color-mix(in srgb, var(--fs-text-tertiary) 55%, var(--fs-text-primary)); /* Tertiary TEXT on a tertiary tint */
|
||||
--fs-text-on-action: #E8E4D8; /* text on a filled colour — NOT mode-dependent */
|
||||
|
||||
/* type */
|
||||
|
||||
@@ -25,7 +25,7 @@
|
||||
border-color: var(--fs-accent);
|
||||
}
|
||||
.ctx-crumb-project {
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
background: color-mix(in srgb, var(--fs-accent) 10%, transparent);
|
||||
border: 1px solid color-mix(in srgb, var(--fs-accent) 30%, transparent);
|
||||
text-decoration: none;
|
||||
|
||||
@@ -206,7 +206,7 @@ router.afterEach(() => {
|
||||
background: var(--fs-accent-soft);
|
||||
}
|
||||
.nav-link.router-link-active {
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
font-weight: 500;
|
||||
background: color-mix(in srgb, var(--fs-accent) 25%, transparent);
|
||||
box-shadow: 0 0 16px color-mix(in srgb, var(--fs-accent) 30%, transparent);
|
||||
@@ -257,7 +257,7 @@ router.afterEach(() => {
|
||||
font-weight: 500;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.05em;
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
|
||||
padding: 0.1rem 0.35rem;
|
||||
border-radius: var(--fs-radius-sm);
|
||||
|
||||
@@ -137,12 +137,12 @@ function markerFor(type: DiffLine['type']): string {
|
||||
|
||||
.diff-delete {
|
||||
background: color-mix(in srgb, var(--fs-error) 12%, transparent);
|
||||
color: var(--fs-error);
|
||||
color: var(--fs-error-fg);
|
||||
}
|
||||
|
||||
.diff-insert {
|
||||
background: color-mix(in srgb, var(--fs-success) 12%, transparent);
|
||||
color: var(--fs-success);
|
||||
color: var(--fs-success-fg);
|
||||
}
|
||||
|
||||
.diff-equal {
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
import { ref, computed, onMounted } from "vue";
|
||||
import { apiGet, pinNoteVersion, unpinNoteVersion } from "@/api/client";
|
||||
import DiffView from "@/components/DiffView.vue";
|
||||
import type { DiffLine } from "@/composables/useAssist";
|
||||
import { computeDiff, type DiffLine } from "@/utils/diff";
|
||||
import { fmtStamp } from "@/utils/dateFormat";
|
||||
|
||||
interface NoteVersion {
|
||||
@@ -33,28 +33,8 @@ const loadingDetail = ref(false);
|
||||
|
||||
const diff = computed<DiffLine[]>(() => {
|
||||
if (!selectedVersion.value?.body) return [];
|
||||
const a = props.currentBody;
|
||||
const b = selectedVersion.value.body;
|
||||
|
||||
const aLines = a.split('\n');
|
||||
const bLines = b.split('\n');
|
||||
const m = aLines.length, n = bLines.length;
|
||||
const dp: number[][] = Array.from({ length: m + 1 }, () => new Array(n + 1).fill(0));
|
||||
for (let i = m - 1; i >= 0; i--)
|
||||
for (let j = n - 1; j >= 0; j--)
|
||||
dp[i][j] = aLines[i] === bLines[j]
|
||||
? dp[i+1][j+1] + 1
|
||||
: Math.max(dp[i+1][j], dp[i][j+1]);
|
||||
const result: DiffLine[] = [];
|
||||
let i = 0, j = 0;
|
||||
while (i < m && j < n) {
|
||||
if (aLines[i] === bLines[j]) { result.push({ type: 'equal', text: aLines[i++] }); j++; }
|
||||
else if (dp[i+1][j] >= dp[i][j+1]) result.push({ type: 'delete', text: aLines[i++] });
|
||||
else result.push({ type: 'insert', text: bLines[j++] });
|
||||
}
|
||||
while (i < m) result.push({ type: 'delete', text: aLines[i++] });
|
||||
while (j < n) result.push({ type: 'insert', text: bLines[j++] });
|
||||
return result;
|
||||
return computeDiff(props.currentBody, selectedVersion.value.body);
|
||||
});
|
||||
|
||||
async function loadVersions() {
|
||||
|
||||
@@ -227,11 +227,11 @@ const markers: Record<DiffLine["type"], string> = {
|
||||
.iap-diff-equal { color: var(--fs-text-tertiary); }
|
||||
.iap-diff-delete {
|
||||
background: color-mix(in srgb, var(--fs-error) 10%, transparent);
|
||||
color: var(--fs-error);
|
||||
color: var(--fs-error-fg);
|
||||
}
|
||||
.iap-diff-insert {
|
||||
background: color-mix(in srgb, var(--fs-success) 10%, transparent);
|
||||
color: var(--fs-success);
|
||||
color: var(--fs-success-fg);
|
||||
}
|
||||
|
||||
.iap-diff-marker {
|
||||
|
||||
@@ -156,7 +156,7 @@ const groups = [
|
||||
|
||||
.md-btn.active {
|
||||
background: color-mix(in srgb, var(--fs-accent) 14%, transparent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
box-shadow: 0 0 0 1px color-mix(in srgb, var(--fs-accent) 35%, transparent);
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
<script setup lang="ts">
|
||||
/**
|
||||
* The staleness sweep for NOTES: notes that assert a fact, oldest first.
|
||||
*
|
||||
* Sibling of RuleSweepPane, not a shared component — the two read differently
|
||||
* enough that merging them would mean a prop for every difference (a rule has
|
||||
* a tier and a statement; a note has a project and opens at a route). What
|
||||
* they share is the SHAPE of the judgement, and that is worth copying
|
||||
* deliberately rather than abstracting: the ordering carries urgency, "never"
|
||||
* is categorically different from a date, and a failed check writes nothing.
|
||||
*
|
||||
* Lives in the Knowledge view rather than beside the rules sweep (operator's
|
||||
* call, milestone 317 step 4): notes stay where notes live. The cost, accepted
|
||||
* knowingly, is that there is no single screen showing every record anyone has
|
||||
* left unconfirmed — /rules keeps its own.
|
||||
*/
|
||||
import { onMounted, ref } from "vue";
|
||||
import { apiGet, apiPost } from "@/api/client";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
|
||||
interface DueNote {
|
||||
id: number;
|
||||
title: string;
|
||||
project_id: number | null;
|
||||
verify_with: string;
|
||||
expires_when: string;
|
||||
last_verified: string | null;
|
||||
days_since_verified: number | null;
|
||||
}
|
||||
|
||||
const emit = defineEmits<{ "open-note": [id: number] }>();
|
||||
|
||||
const toast = useToastStore();
|
||||
const rows = ref<DueNote[]>([]);
|
||||
const loading = ref(false);
|
||||
const neverOnly = ref(false);
|
||||
const busyId = ref<number | null>(null);
|
||||
|
||||
async function reload() {
|
||||
loading.value = true;
|
||||
try {
|
||||
const p = new URLSearchParams();
|
||||
if (neverOnly.value) p.set("never_only", "1");
|
||||
const data = await apiGet<{ notes: DueNote[] }>(
|
||||
`/api/notes/due-for-verification?${p}`,
|
||||
);
|
||||
rows.value = data.notes;
|
||||
} catch {
|
||||
toast.show("Could not load the sweep", "error");
|
||||
} finally {
|
||||
loading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function verify(id: number, stillTrue: boolean) {
|
||||
busyId.value = id;
|
||||
try {
|
||||
await apiPost(`/api/notes/${id}/verify`, { still_true: stillTrue });
|
||||
if (stillTrue) {
|
||||
// It has been confirmed, so it leaves the list — the sweep shows what
|
||||
// still needs looking at, and leaving it in place would invite a second
|
||||
// stamp nobody earned.
|
||||
rows.value = rows.value.filter((r) => r.id !== id);
|
||||
toast.show("Recorded — checked today");
|
||||
} else {
|
||||
// It stays. A failed check writes nothing on purpose: the note is wrong
|
||||
// rather than in a state worth recording, so it keeps its place until
|
||||
// someone corrects, supersedes, or unhooks it.
|
||||
toast.show("Recorded as no longer true — the note keeps its place here");
|
||||
}
|
||||
} catch {
|
||||
toast.show("Could not record that", "error");
|
||||
} finally {
|
||||
busyId.value = null;
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(reload);
|
||||
defineExpose({ reload });
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section class="sweep">
|
||||
<header>
|
||||
<h2>Due for verification</h2>
|
||||
<p class="lede">
|
||||
Notes that assert a fact about something outside your control — what a
|
||||
service does, how a tool behaves. Most notes are decisions and never
|
||||
appear here; they have no truth value to go stale.
|
||||
</p>
|
||||
</header>
|
||||
|
||||
<div class="filters">
|
||||
<label class="filter">
|
||||
<input v-model="neverOnly" type="checkbox" @change="reload" />
|
||||
<span>Never checked only</span>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<p v-if="loading" class="state">Loading…</p>
|
||||
|
||||
<!-- An empty sweep is GOOD NEWS and must not read like a broken page. -->
|
||||
<p v-else-if="!rows.length" class="state empty">
|
||||
Nothing to check.
|
||||
{{ neverOnly
|
||||
? "Every note that carries a check has been confirmed at least once."
|
||||
: "No note carries a check yet — add one to a note that asserts a fact." }}
|
||||
</p>
|
||||
|
||||
<ol v-else class="rows">
|
||||
<li v-for="n in rows" :key="n.id" class="row">
|
||||
<div class="row-head">
|
||||
<button class="row-title" @click="emit('open-note', n.id)">{{ n.title }}</button>
|
||||
<span class="age" :class="{ unchecked: n.days_since_verified === null }">
|
||||
{{ n.days_since_verified === null
|
||||
? "never checked"
|
||||
: `${n.days_since_verified}d ago` }}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<dl class="check">
|
||||
<dt>Check</dt>
|
||||
<dd>{{ n.verify_with }}</dd>
|
||||
<template v-if="n.expires_when">
|
||||
<dt>Ends when</dt>
|
||||
<dd>{{ n.expires_when }}</dd>
|
||||
</template>
|
||||
</dl>
|
||||
|
||||
<div class="actions">
|
||||
<button :disabled="busyId === n.id" @click="verify(n.id, true)">Still true</button>
|
||||
<button :disabled="busyId === n.id" @click="verify(n.id, false)">No longer true</button>
|
||||
</div>
|
||||
</li>
|
||||
</ol>
|
||||
|
||||
<p v-if="rows.length" class="footnote">
|
||||
Record a result only after actually running the check. “No longer true” stores nothing
|
||||
on purpose — the note is wrong rather than in a state worth recording, so it keeps its
|
||||
place here until you correct it, supersede it, or remove its check.
|
||||
</p>
|
||||
</section>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.sweep { display: flex; flex-direction: column; gap: var(--fs-space-3); }
|
||||
h2 { margin: 0; font-size: 1.05rem; }
|
||||
.lede {
|
||||
margin: 0.35rem 0 0;
|
||||
max-width: 62ch;
|
||||
font-size: 0.85rem;
|
||||
color: var(--fs-text-secondary);
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.filters { display: flex; gap: var(--fs-space-5); align-items: center; flex-wrap: wrap; }
|
||||
.filter { display: flex; align-items: center; gap: var(--fs-space-2); font-size: 0.82rem; color: var(--fs-text-secondary); }
|
||||
.filter input[type="checkbox"] { accent-color: var(--fs-accent); }
|
||||
|
||||
.state { margin: 0; font-size: 0.9rem; color: var(--fs-text-secondary); }
|
||||
.state.empty { color: var(--fs-text-tertiary); }
|
||||
|
||||
.rows { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: var(--fs-space-3); }
|
||||
.row {
|
||||
background: var(--fs-surface-raised);
|
||||
border-radius: var(--fs-radius-md);
|
||||
padding: var(--fs-space-3);
|
||||
}
|
||||
.row-head { display: flex; align-items: baseline; gap: var(--fs-space-2); flex-wrap: wrap; }
|
||||
.row-title {
|
||||
background: none; border: none; padding: 0; cursor: pointer;
|
||||
font-family: Fraunces, serif; font-style: italic; font-size: 1.02rem;
|
||||
color: var(--fs-text-primary); text-align: left;
|
||||
}
|
||||
.row-title:hover { text-decoration: underline; }
|
||||
/* The ORDER carries urgency — the top of this list is the least-confirmed
|
||||
thing in the corpus. No red/amber ramp: it would restate the ordering and
|
||||
force an invented "stale after N days" threshold. "Never" is marked because
|
||||
it is categorically DIFFERENT from a date, not a worse one. */
|
||||
.age { margin-left: auto; font-size: 0.78rem; color: var(--fs-text-secondary); font-variant-numeric: tabular-nums; }
|
||||
.age.unchecked { font-style: italic; color: var(--fs-text-tertiary); }
|
||||
|
||||
.check { display: grid; grid-template-columns: auto 1fr; gap: 0.15rem var(--fs-space-3); margin: var(--fs-space-3) 0 0; }
|
||||
.check dt { font-size: 0.7rem; text-transform: uppercase; letter-spacing: 0.05em; color: var(--fs-text-tertiary); }
|
||||
.check dd { margin: 0; font-size: 0.82rem; color: var(--fs-text-primary); min-width: 0; overflow-wrap: anywhere; }
|
||||
|
||||
.actions { display: flex; gap: var(--fs-space-2); margin-top: var(--fs-space-3); }
|
||||
.actions button {
|
||||
cursor: pointer; font: inherit; font-size: 0.78rem;
|
||||
background: var(--fs-surface-page); color: var(--fs-text-primary);
|
||||
border: 1px solid var(--fs-border-color); border-radius: var(--fs-radius-md);
|
||||
padding: 0.25rem 0.6rem;
|
||||
}
|
||||
.actions button:hover:not(:disabled) { background: var(--fs-surface-hover); }
|
||||
.actions button:disabled { opacity: var(--fs-disabled-opacity); cursor: default; }
|
||||
|
||||
.footnote { margin: 0; max-width: 62ch; font-size: 0.78rem; color: var(--fs-text-tertiary); line-height: 1.45; }
|
||||
</style>
|
||||
@@ -679,7 +679,7 @@ async function confirmDelete() {
|
||||
font-weight: 500;
|
||||
background: color-mix(in srgb, var(--fs-accent) 12%, transparent);
|
||||
border: 1px solid color-mix(in srgb, var(--fs-accent) 30%, transparent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
border-radius: 999px;
|
||||
padding: 0.05rem 0.45rem;
|
||||
flex-shrink: 0;
|
||||
@@ -689,7 +689,7 @@ async function confirmDelete() {
|
||||
font-weight: 500;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
color: var(--fs-text-tertiary);
|
||||
color: var(--fs-text-tertiary-fg);
|
||||
background: color-mix(in srgb, var(--fs-text-tertiary) 12%, transparent);
|
||||
border-radius: 999px;
|
||||
padding: 0.05rem 0.45rem;
|
||||
|
||||
@@ -168,7 +168,7 @@ function focusInput() {
|
||||
border-radius: 999px;
|
||||
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
|
||||
border: 1px solid var(--fs-accent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
font-size: 0.8rem;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
<script setup lang="ts">
|
||||
/**
|
||||
* "N/M used" on a list row — surfaced vs opened, for any record kind.
|
||||
*
|
||||
* Extracted from SnippetListView when the rule list needed the same chip
|
||||
* (milestone 333 step 5). The counts read identically for both; what differs
|
||||
* is the ADVICE, which is why that is a prop. A snippet surfaced repeatedly
|
||||
* and never opened should probably go; a rule in the same position may simply
|
||||
* have a `when_to_apply` that fires on the wrong thing, and telling an
|
||||
* operator to delete it would be the wrong nudge half the time.
|
||||
*/
|
||||
import type { RecordUsage } from "@/types/usage";
|
||||
|
||||
const props = defineProps<{
|
||||
usage?: RecordUsage | null;
|
||||
/** What to suggest when this record looks like dead weight. Appended to the
|
||||
* tooltip; kind-specific, because the remedies are. */
|
||||
deadWeightAdvice: string;
|
||||
/** What the record is called in the tooltip's own sentence. */
|
||||
noun?: string;
|
||||
}>();
|
||||
|
||||
/** Offered repeatedly and never opened. Three rather than one because one or
|
||||
* two surfacings is noise — the record may simply not have come up in a
|
||||
* relevant context yet. */
|
||||
const isDeadWeight = () =>
|
||||
!!props.usage && props.usage.pull_count === 0 && props.usage.surfaced_count >= 3;
|
||||
|
||||
/** "" renders nothing. A record nobody has surfaced yet gets no badge at all:
|
||||
* "0/0" would read as a verdict when it is an absence of evidence — and on a
|
||||
* freshly-migrated install that is every row. */
|
||||
const label = () => {
|
||||
const u = props.usage;
|
||||
if (!u || u.surfaced_count === 0) return "";
|
||||
return `${u.pull_count}/${u.surfaced_count} used`;
|
||||
};
|
||||
|
||||
const title = () => {
|
||||
const u = props.usage;
|
||||
if (!u) return "";
|
||||
const last = u.last_pulled_at
|
||||
? `Last opened ${new Date(u.last_pulled_at).toLocaleDateString()}.`
|
||||
: "Never opened.";
|
||||
const verdict = isDeadWeight() ? ` ${props.deadWeightAdvice}` : "";
|
||||
return (
|
||||
`Surfaced to an agent ${u.surfaced_count}×, opened in full ` +
|
||||
`${u.pull_count}×. ${last}${verdict}`
|
||||
);
|
||||
};
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<span
|
||||
v-if="label()"
|
||||
class="usage-tag"
|
||||
:class="{ 'usage-dead': isDeadWeight() }"
|
||||
:title="title()"
|
||||
>{{ label() }}</span>
|
||||
</template>
|
||||
|
||||
<!-- The look lives in components.css (canon). Nothing scoped here on purpose:
|
||||
a view that needs different spacing keeps that as its own remainder. -->
|
||||
@@ -2,7 +2,7 @@
|
||||
import { ref, computed } from "vue";
|
||||
import { apiGet } from "@/api/client";
|
||||
import DiffView from "@/components/DiffView.vue";
|
||||
import type { DiffLine } from "@/composables/useAssist";
|
||||
import { computeDiff, type DiffLine } from "@/utils/diff";
|
||||
|
||||
interface NoteVersion {
|
||||
id: number;
|
||||
@@ -31,25 +31,7 @@ const loadingDetail = ref(false);
|
||||
|
||||
const diff = computed<DiffLine[]>(() => {
|
||||
if (!selectedVersion.value?.body) return [];
|
||||
const aLines = props.currentBody.split("\n");
|
||||
const bLines = selectedVersion.value.body.split("\n");
|
||||
const m = aLines.length, n = bLines.length;
|
||||
const dp: number[][] = Array.from({ length: m + 1 }, () => new Array(n + 1).fill(0));
|
||||
for (let i = m - 1; i >= 0; i--)
|
||||
for (let j = n - 1; j >= 0; j--)
|
||||
dp[i][j] = aLines[i] === bLines[j]
|
||||
? dp[i + 1][j + 1] + 1
|
||||
: Math.max(dp[i + 1][j], dp[i][j + 1]);
|
||||
const result: DiffLine[] = [];
|
||||
let i = 0, j = 0;
|
||||
while (i < m && j < n) {
|
||||
if (aLines[i] === bLines[j]) { result.push({ type: "equal", text: aLines[i++] }); j++; }
|
||||
else if (dp[i + 1][j] >= dp[i][j + 1]) result.push({ type: "delete", text: aLines[i++] });
|
||||
else result.push({ type: "insert", text: bLines[j++] });
|
||||
}
|
||||
while (i < m) result.push({ type: "delete", text: aLines[i++] });
|
||||
while (j < n) result.push({ type: "insert", text: bLines[j++] });
|
||||
return result;
|
||||
return computeDiff(props.currentBody, selectedVersion.value.body);
|
||||
});
|
||||
|
||||
function formatDate(iso: string): string {
|
||||
|
||||
@@ -548,7 +548,7 @@ defineExpose({ reload: loadProjectNotes });
|
||||
|
||||
.note-tag-pill {
|
||||
font-size: 0.58rem;
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
background: color-mix(in srgb, var(--fs-accent) 10%, transparent);
|
||||
border-radius: 999px;
|
||||
padding: 0 0.3rem;
|
||||
@@ -645,7 +645,7 @@ defineExpose({ reload: loadProjectNotes });
|
||||
.btn-tag-suggestion.applied {
|
||||
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
|
||||
border-color: var(--fs-accent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
}
|
||||
|
||||
.link-suggest-strip {
|
||||
|
||||
@@ -424,8 +424,8 @@ defineExpose({ reload: loadAll });
|
||||
border-radius: 10px;
|
||||
text-transform: capitalize;
|
||||
}
|
||||
.ms-status-active { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent); }
|
||||
.ms-status-completed { background: color-mix(in srgb, var(--fs-success) 15%, transparent); color: var(--fs-success); }
|
||||
.ms-status-active { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent-fg); }
|
||||
.ms-status-completed { background: color-mix(in srgb, var(--fs-success) 15%, transparent); color: var(--fs-success-fg); }
|
||||
|
||||
.task-items {
|
||||
list-style: none;
|
||||
@@ -516,8 +516,8 @@ defineExpose({ reload: loadAll });
|
||||
user-select: none;
|
||||
margin-left: auto;
|
||||
}
|
||||
.status-cycler.status-in_progress { border-color: var(--fs-accent); color: var(--fs-accent); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
|
||||
.status-cycler.status-done { border-color: var(--fs-success); color: var(--fs-success); background: color-mix(in srgb, var(--fs-success) 10%, transparent); }
|
||||
.status-cycler.status-in_progress { border-color: var(--fs-accent); color: var(--fs-accent-fg); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
|
||||
.status-cycler.status-done { border-color: var(--fs-success); color: var(--fs-success-fg); background: color-mix(in srgb, var(--fs-success) 10%, transparent); }
|
||||
|
||||
.btn-edit-task { margin-left: 0.25rem; }
|
||||
.btn-edit-task:hover { text-decoration: underline; }
|
||||
|
||||
@@ -3,6 +3,7 @@ import { computed, ref, watch, onMounted } from "vue";
|
||||
import { useRulebooksStore } from "@/stores/rulebooks";
|
||||
import { useCanonicalSystemsStore } from "@/stores/canonicalSystems";
|
||||
import type { RuleTier } from "@/api/rulebooks";
|
||||
import RuleHistoryPanel from "@/components/rules/RuleHistoryPanel.vue";
|
||||
|
||||
const props = defineProps<{ ruleId: number | null; topicId: number | null }>();
|
||||
const emit = defineEmits<{ close: [] }>();
|
||||
@@ -256,6 +257,17 @@ watch(() => props.ruleId, load);
|
||||
How to apply
|
||||
<textarea v-model="howToApply" rows="4" placeholder="When / where this kicks in." />
|
||||
</label>
|
||||
|
||||
<!-- Only on an existing rule: a rule being created has no past, and an
|
||||
"Edit history — none" line on a blank form reads as a broken panel.
|
||||
Keyed on ruleId so switching rules reloads rather than showing the
|
||||
previous rule's history under the new one's text. -->
|
||||
<RuleHistoryPanel
|
||||
v-if="!isCreating && ruleId !== null"
|
||||
:key="ruleId"
|
||||
:rule-id="ruleId"
|
||||
:current="store.currentRule"
|
||||
/>
|
||||
</aside>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
@@ -0,0 +1,274 @@
|
||||
<script setup lang="ts">
|
||||
/**
|
||||
* What a rule USED TO SAY — inside the slide-over, where a rule is read in
|
||||
* full. Not on the list row: a history entry point there would compete with
|
||||
* the row's actual job.
|
||||
*
|
||||
* A SIBLING OF HistoryPanel.vue, NOT A REUSE OF IT, and the reason is in its
|
||||
* props: `noteId` + `currentBody`, a `NoteVersion` carrying tags and pin
|
||||
* columns, a fetch of /api/notes/…, a `restore` emit, and pin/unpin buttons.
|
||||
* Every one of those is note-shaped. Rules have no tags, no pins, and
|
||||
* deliberately no restore, and a rule's text is EIGHT fields rather than one
|
||||
* body — which changes the central question from "what changed" to "which
|
||||
* fields moved".
|
||||
*
|
||||
* What was genuinely shared is shared: DiffView.vue takes DiffLine[] and
|
||||
* nothing note-shaped, and the LCS walk now lives in utils/diff.ts, which
|
||||
* this file uses rather than copying a fourth time (#3207).
|
||||
*/
|
||||
import { computed, onMounted, ref, watch } from "vue";
|
||||
import DiffView from "@/components/DiffView.vue";
|
||||
import { computeDiff } from "@/utils/diff";
|
||||
import {
|
||||
listRuleVersions, getRuleVersion, type Rule, type RuleVersion,
|
||||
} from "@/api/rulebooks";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
|
||||
const props = defineProps<{ ruleId: number; current: Rule | null }>();
|
||||
|
||||
const toast = useToastStore();
|
||||
const versions = ref<RuleVersion[]>([]);
|
||||
const selected = ref<RuleVersion | null>(null);
|
||||
const expanded = ref(false);
|
||||
const loading = ref(false);
|
||||
const loadingDetail = ref(false);
|
||||
|
||||
// The eight TEXT fields a version carries, in the order the editor shows
|
||||
// them. Narrowed to its own type rather than `keyof RuleVersion`, which would
|
||||
// also admit id/rule_id/user_id/created_at — none of which is text a reader
|
||||
// compares, and all of which would widen every lookup below to `number`.
|
||||
// Labels rather than column names: a reader is deciding whether to open a
|
||||
// row, and "How to apply" reads where "how_to_apply" has to be decoded.
|
||||
type TextField =
|
||||
| "title" | "statement" | "when_to_apply" | "tier"
|
||||
| "why" | "how_to_apply" | "verify_with" | "expires_when";
|
||||
|
||||
const FIELDS: Array<[TextField, string]> = [
|
||||
["title", "Title"],
|
||||
["statement", "Statement"],
|
||||
["when_to_apply", "When to apply"],
|
||||
["tier", "Tier"],
|
||||
["why", "Why"],
|
||||
["how_to_apply", "How to apply"],
|
||||
["verify_with", "Check"],
|
||||
["expires_when", "Ends when"],
|
||||
];
|
||||
|
||||
/**
|
||||
* Which fields this edit moved.
|
||||
*
|
||||
* A version holds the text the edit REPLACED, so the edit is the step from
|
||||
* this row to the NEXT NEWER state — the version above it in the list, or,
|
||||
* for the newest row, the rule as it stands now. Comparing against the row
|
||||
* below instead would attribute every change to the wrong edit.
|
||||
*/
|
||||
function changedFields(index: number): string[] {
|
||||
const before = versions.value[index];
|
||||
// `Rule` carries all eight as required strings; a RuleVersion carries them
|
||||
// only once opened, which is what the undefined check below is about.
|
||||
const after: Pick<Rule, TextField> | RuleVersion | null =
|
||||
index === 0 ? props.current : versions.value[index - 1] ?? null;
|
||||
if (!before || !after) return [];
|
||||
return FIELDS
|
||||
.filter(([key]) => {
|
||||
// A listing row carries only the title; the rest arrive when opened.
|
||||
// Undefined means NOT LOADED, which is not the same as unchanged — so a
|
||||
// field nobody has fetched is claimed as neither.
|
||||
const a = before[key];
|
||||
const b = after[key];
|
||||
if (a === undefined || b === undefined) return false;
|
||||
return (a ?? "") !== (b ?? "");
|
||||
})
|
||||
.map(([, label]) => label);
|
||||
}
|
||||
|
||||
/** True when this edit rewrote or removed the rule's check.
|
||||
*
|
||||
* Worth its own marker because editing `verify_with` silently drops
|
||||
* `verified_at` (milestone 312) — the moment a rule re-entered the staleness
|
||||
* sweep. That happens nowhere a reader can see it, and this row is the only
|
||||
* surface that can say when it happened. */
|
||||
function checkChanged(index: number): boolean {
|
||||
return changedFields(index).includes("Check");
|
||||
}
|
||||
|
||||
const diff = computed(() => {
|
||||
if (!selected.value || selected.value.statement === undefined) return [];
|
||||
const now = props.current?.statement ?? "";
|
||||
return computeDiff(now, selected.value.statement);
|
||||
});
|
||||
|
||||
function stamp(iso: string): string {
|
||||
return iso.slice(0, 10);
|
||||
}
|
||||
|
||||
async function load() {
|
||||
loading.value = true;
|
||||
try {
|
||||
versions.value = await listRuleVersions(props.ruleId);
|
||||
} catch {
|
||||
toast.show("Could not load this rule's history", "error");
|
||||
} finally {
|
||||
loading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function open(v: RuleVersion) {
|
||||
if (selected.value?.id === v.id) {
|
||||
selected.value = null;
|
||||
return;
|
||||
}
|
||||
loadingDetail.value = true;
|
||||
try {
|
||||
const full = await getRuleVersion(props.ruleId, v.id);
|
||||
// Merged back into the list so `changedFields` can compare against real
|
||||
// text once a neighbour has been opened, instead of staying blind.
|
||||
const at = versions.value.findIndex((x) => x.id === v.id);
|
||||
if (at >= 0) versions.value[at] = { ...versions.value[at], ...full };
|
||||
selected.value = versions.value[at] ?? full;
|
||||
} catch {
|
||||
toast.show("Could not open that version", "error");
|
||||
} finally {
|
||||
loadingDetail.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
onMounted(load);
|
||||
watch(() => props.ruleId, () => { selected.value = null; load(); });
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<section class="history">
|
||||
<button class="toggle" :aria-expanded="expanded" @click="expanded = !expanded">
|
||||
<span>Edit history</span>
|
||||
<span class="count">{{ versions.length || "none" }}</span>
|
||||
</button>
|
||||
|
||||
<div v-if="expanded" class="body">
|
||||
<p v-if="loading" class="state">Loading…</p>
|
||||
|
||||
<!-- Never reworded is the ordinary case, and must not read as a fault. -->
|
||||
<p v-else-if="!versions.length" class="state empty">
|
||||
This rule has never been reworded. Nothing was recorded before the history
|
||||
existed, so an older rule starts empty too.
|
||||
</p>
|
||||
|
||||
<template v-else>
|
||||
<p class="lede">
|
||||
Each entry is what the rule said <em>before</em> that edit. The wording it
|
||||
was changed to is the rule as it stands above.
|
||||
</p>
|
||||
<ol class="rows">
|
||||
<li v-for="(v, i) in versions" :key="v.id" class="row">
|
||||
<button
|
||||
class="row-head"
|
||||
:class="{ open: selected?.id === v.id }"
|
||||
@click="open(v)"
|
||||
>
|
||||
<span class="when">{{ stamp(v.created_at) }}</span>
|
||||
<span class="fields">
|
||||
{{ changedFields(i).join(", ") || "opened to compare" }}
|
||||
</span>
|
||||
<span v-if="checkChanged(i)" class="check-moved">check reset</span>
|
||||
</button>
|
||||
|
||||
<div v-if="selected?.id === v.id" class="detail">
|
||||
<p v-if="loadingDetail" class="state">Loading…</p>
|
||||
<template v-else>
|
||||
<p v-if="checkChanged(i)" class="warn">
|
||||
This edit changed the rule's check, which cleared its verification
|
||||
stamp — the rule went back to the top of the staleness sweep here.
|
||||
</p>
|
||||
<dl class="fields-list">
|
||||
<template v-for="[key, label] in FIELDS" :key="key">
|
||||
<template v-if="key !== 'statement' && v[key]">
|
||||
<dt>{{ label }}</dt>
|
||||
<dd>{{ v[key] }}</dd>
|
||||
</template>
|
||||
</template>
|
||||
</dl>
|
||||
<h4>Statement</h4>
|
||||
<DiffView v-if="diff.length" :diff="diff" />
|
||||
<p v-else class="state">The statement did not change in this edit.</p>
|
||||
</template>
|
||||
</div>
|
||||
</li>
|
||||
</ol>
|
||||
</template>
|
||||
</div>
|
||||
</section>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.history { border-top: 1px solid var(--fs-border-color); padding-top: var(--fs-space-3); }
|
||||
|
||||
.toggle {
|
||||
display: flex; align-items: center; gap: var(--fs-space-2); width: 100%;
|
||||
background: none; border: none; padding: 0; cursor: pointer;
|
||||
font: inherit; font-size: var(--fs-size-body-sm); color: var(--fs-text-secondary);
|
||||
}
|
||||
.toggle:hover { color: var(--fs-text-primary); }
|
||||
.count {
|
||||
margin-left: auto; font-size: var(--fs-size-tiny); color: var(--fs-text-tertiary);
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.body { margin-top: var(--fs-space-3); display: flex; flex-direction: column; gap: var(--fs-space-3); }
|
||||
.state { margin: 0; font-size: var(--fs-size-body-sm); color: var(--fs-text-secondary); }
|
||||
.state.empty { color: var(--fs-text-tertiary); }
|
||||
.lede {
|
||||
margin: 0; max-width: 62ch; font-size: var(--fs-size-tiny);
|
||||
color: var(--fs-text-tertiary); line-height: var(--fs-leading-body);
|
||||
}
|
||||
|
||||
.rows { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: var(--fs-space-2); }
|
||||
.row { background: var(--fs-surface-raised); border-radius: var(--fs-radius-md); }
|
||||
|
||||
.row-head {
|
||||
display: flex; align-items: baseline; gap: var(--fs-space-3); width: 100%;
|
||||
background: none; border: none; cursor: pointer; text-align: left;
|
||||
padding: var(--fs-space-2) var(--fs-space-3);
|
||||
font: inherit; font-size: var(--fs-size-body-sm); color: var(--fs-text-primary);
|
||||
}
|
||||
.row-head:hover { background: var(--fs-surface-hover); border-radius: var(--fs-radius-md); }
|
||||
.when {
|
||||
font-variant-numeric: tabular-nums; color: var(--fs-text-secondary);
|
||||
font-size: var(--fs-size-tiny);
|
||||
}
|
||||
.fields { color: var(--fs-text-primary); min-width: 0; overflow-wrap: anywhere; }
|
||||
|
||||
/* A TINT, not the solid token. `--fs-warning-fg` is defined as "warning text
|
||||
ON A WARNING TINT" — painting it over solid `--fs-warning` is the same-hue
|
||||
contrast failure #3141 records. The 12% mix is how theme.css builds its own
|
||||
`-bg` pairs, and it keeps the value a resolvable var() rather than a raw hex
|
||||
that check_design_tokens.py cannot see at all. */
|
||||
.check-moved {
|
||||
margin-left: auto; flex: none;
|
||||
background: color-mix(in srgb, var(--fs-warning) 12%, transparent);
|
||||
color: var(--fs-warning-fg);
|
||||
border-radius: var(--fs-radius-pill);
|
||||
padding: 0.1rem 0.5rem;
|
||||
font-size: var(--fs-size-tiny); letter-spacing: var(--fs-tracking-tiny);
|
||||
}
|
||||
|
||||
.detail {
|
||||
padding: 0 var(--fs-space-3) var(--fs-space-3);
|
||||
display: flex; flex-direction: column; gap: var(--fs-space-2);
|
||||
}
|
||||
.warn {
|
||||
margin: 0; font-size: var(--fs-size-tiny); line-height: var(--fs-leading-body);
|
||||
color: var(--fs-warning-fg);
|
||||
background: color-mix(in srgb, var(--fs-warning) 12%, transparent);
|
||||
border-radius: var(--fs-radius-sm); padding: var(--fs-space-2);
|
||||
}
|
||||
.fields-list { display: grid; grid-template-columns: auto 1fr; gap: 0.15rem var(--fs-space-3); margin: 0; }
|
||||
.fields-list dt {
|
||||
font-size: var(--fs-size-tiny); text-transform: uppercase;
|
||||
letter-spacing: var(--fs-tracking-tiny); color: var(--fs-text-tertiary);
|
||||
}
|
||||
.fields-list dd {
|
||||
margin: 0; font-size: var(--fs-size-body-sm);
|
||||
color: var(--fs-text-primary); min-width: 0; overflow-wrap: anywhere;
|
||||
}
|
||||
h4 { margin: var(--fs-space-2) 0 0; font-size: var(--fs-size-tiny); color: var(--fs-text-tertiary); }
|
||||
</style>
|
||||
@@ -1,5 +1,16 @@
|
||||
<script setup lang="ts">
|
||||
import type { RuleHeader } from "@/api/rulebooks";
|
||||
import UsageBadge from "@/components/UsageBadge.vue";
|
||||
|
||||
/** The dead-weight nudge for a RULE — two remedies, not one, which is the
|
||||
* whole reason this advice is per-kind. A snippet nobody opens should
|
||||
* probably go. A rule nobody opens may be perfectly good and simply firing on
|
||||
* the wrong thing, so "delete it" would be the wrong nudge half the time and
|
||||
* the operator has to be the one who picks. */
|
||||
const RULE_DEAD_WEIGHT =
|
||||
"Kept arriving without being read. Either its trigger fires on the wrong " +
|
||||
"work — reword “when to apply” so it says when — or it is not wanted here. " +
|
||||
"Until one or the other, it takes a slot in every write it matches.";
|
||||
|
||||
defineProps<{ topicId: number; rules: RuleHeader[] }>();
|
||||
const emit = defineEmits<{
|
||||
@@ -28,6 +39,7 @@ const emit = defineEmits<{
|
||||
? 'Asserts a fact nobody has confirmed yet'
|
||||
: `Check last passed ${r.last_verified}`"
|
||||
>{{ r.last_verified === "never" ? "unverified" : `checked ${r.last_verified}` }}</span>
|
||||
<UsageBadge :usage="r.usage" :dead-weight-advice="RULE_DEAD_WEIGHT" />
|
||||
</div>
|
||||
<div class="statement">{{ r.statement }}</div>
|
||||
<div v-if="r.when_to_apply || r.updated_at" class="meta">
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { ref, computed, watch, type Ref } from "vue";
|
||||
import { computeDiff, type DiffLine } from "@/utils/diff";
|
||||
import { apiPost, apiPut, apiDelete, apiSSEStream, type SSEStreamHandle } from "@/api/client";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
import {
|
||||
@@ -9,17 +10,16 @@ import {
|
||||
export type AssistState = "idle" | "streaming" | "review";
|
||||
export type ScopeMode = "document" | "section";
|
||||
|
||||
// Re-exported: this composable was where DiffLine lived before the diff
|
||||
// moved to a shared util, and every consumer still imports the type from here.
|
||||
export type { DiffLine };
|
||||
|
||||
export interface AssistTarget {
|
||||
text: string;
|
||||
startOffset: number;
|
||||
endOffset: number;
|
||||
}
|
||||
|
||||
export interface DiffLine {
|
||||
type: 'equal' | 'delete' | 'insert';
|
||||
text: string;
|
||||
}
|
||||
|
||||
export interface NoteDraft {
|
||||
id: number;
|
||||
note_id: number;
|
||||
@@ -31,27 +31,6 @@ export interface NoteDraft {
|
||||
updated_at: string;
|
||||
}
|
||||
|
||||
function computeDiff(a: string, b: string): DiffLine[] {
|
||||
const aLines = a.split('\n');
|
||||
const bLines = b.split('\n');
|
||||
const m = aLines.length, n = bLines.length;
|
||||
const dp: number[][] = Array.from({ length: m + 1 }, () => new Array(n + 1).fill(0));
|
||||
for (let i = m - 1; i >= 0; i--)
|
||||
for (let j = n - 1; j >= 0; j--)
|
||||
dp[i][j] = aLines[i] === bLines[j]
|
||||
? dp[i+1][j+1] + 1
|
||||
: Math.max(dp[i+1][j], dp[i][j+1]);
|
||||
const result: DiffLine[] = [];
|
||||
let i = 0, j = 0;
|
||||
while (i < m && j < n) {
|
||||
if (aLines[i] === bLines[j]) { result.push({ type: 'equal', text: aLines[i++] }); j++; }
|
||||
else if (dp[i+1][j] >= dp[i][j+1]) result.push({ type: 'delete', text: aLines[i++] });
|
||||
else result.push({ type: 'insert', text: bLines[j++] });
|
||||
}
|
||||
while (i < m) result.push({ type: 'delete', text: aLines[i++] });
|
||||
while (j < n) result.push({ type: 'insert', text: bLines[j++] });
|
||||
return result;
|
||||
}
|
||||
|
||||
export function useAssist(body: Ref<string>, noteId?: Ref<number | null>, projectId?: Ref<number | null>) {
|
||||
const toast = useToastStore();
|
||||
|
||||
@@ -31,6 +31,8 @@ export const useNotesStore = defineStore("notes", () => {
|
||||
project_id?: number | null;
|
||||
milestone_id?: number | null;
|
||||
note_type?: string;
|
||||
verify_with?: string;
|
||||
expires_when?: string;
|
||||
}): Promise<Note> {
|
||||
try {
|
||||
return await apiPost<Note>("/api/notes", data);
|
||||
@@ -42,7 +44,11 @@ export const useNotesStore = defineStore("notes", () => {
|
||||
|
||||
async function updateNote(
|
||||
id: number,
|
||||
data: Partial<Pick<Note, "title" | "body" | "tags" | "project_id" | "milestone_id" | "note_type">>
|
||||
data: Partial<Pick<
|
||||
Note,
|
||||
"title" | "body" | "tags" | "project_id" | "milestone_id" | "note_type"
|
||||
| "verify_with" | "expires_when"
|
||||
>>
|
||||
): Promise<Note> {
|
||||
try {
|
||||
const note = await apiPut<Note>(`/api/notes/${id}`, data);
|
||||
|
||||
@@ -34,6 +34,15 @@ export interface Note {
|
||||
is_task: boolean;
|
||||
note_type: NoteType;
|
||||
task_kind?: TaskKind;
|
||||
// The note's own check (milestone 317). Empty on almost every note — that
|
||||
// is the normal case: a note with no `verify_with` is a DECISION, and there
|
||||
// is nothing to go and check. Only a note asserting a fact about something
|
||||
// outside the operator's control carries one. `verified_at` null while
|
||||
// `verify_with` is set means NOBODY HAS EVER CONFIRMED IT, which is the
|
||||
// state the sweep ranks first.
|
||||
verify_with?: string;
|
||||
expires_when?: string;
|
||||
verified_at?: string | null;
|
||||
systems?: System[];
|
||||
arose_from_id?: number | null;
|
||||
created_at: string;
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
/**
|
||||
* How often a record was put in front of an agent, and how often one then
|
||||
* opened it in full.
|
||||
*
|
||||
* One shape for every record kind the retrieval surfaces can choose. Snippets
|
||||
* and notes are counted in `note_usage_events`; rules in `rule_usage_events`,
|
||||
* which is a separate table because a note id and a rule id are different
|
||||
* namespaces resolved through different maps at restore (milestone 333). The
|
||||
* TABLES are separate for that reason; the READOUT is the same question, so
|
||||
* the client type is one.
|
||||
*
|
||||
* A high `surfaced_count` with `pull_count: 0` is dead weight — it occupies a
|
||||
* slot in every future menu while never being used. What to DO about that
|
||||
* differs by kind, which is why the advice is a prop on the badge rather than
|
||||
* a property of this type: a snippet nobody opens should probably be deleted,
|
||||
* while a rule nobody opens may just be mis-triggered.
|
||||
*/
|
||||
export interface RecordUsage {
|
||||
surfaced_count: number;
|
||||
pull_count: number;
|
||||
last_surfaced_at: string | null;
|
||||
last_pulled_at: string | null;
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
/**
|
||||
* Line diff — one copy, for every surface that shows what changed.
|
||||
*
|
||||
* WHY THIS FILE EXISTS. The same LCS walk was written out three times:
|
||||
* privately in `useAssist.ts`, and again inside `HistoryPanel.vue` and
|
||||
* `VersionHistorySection.vue`. The three were character-identical apart from
|
||||
* quote style — nobody had diverged them on purpose, they were simply copied
|
||||
* because `computeDiff` was never exported. Milestone 323 needed a fourth
|
||||
* consumer (a rule's edit history), and a fourth copy is the cost #3207
|
||||
* records: a fix or an improvement now has to be found in N places by someone
|
||||
* who does not know N.
|
||||
*/
|
||||
|
||||
export interface DiffLine {
|
||||
type: "equal" | "delete" | "insert";
|
||||
text: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Diff `a` against `b`, line by line.
|
||||
*
|
||||
* `delete` lines come from `a`, `insert` lines from `b` — so the caller
|
||||
* decides which side reads as "before" by which argument it passes. Every
|
||||
* caller here passes the CURRENT text as `a` and the older text as `b`, so a
|
||||
* deletion is what the old version had and an insertion is what replaced it.
|
||||
*
|
||||
* O(m·n) in time and memory: fine for a note or a rule, and deliberately not
|
||||
* generalised further, since nothing here diffs a file of thousands of lines.
|
||||
*/
|
||||
export function computeDiff(a: string, b: string): DiffLine[] {
|
||||
const aLines = a.split("\n");
|
||||
const bLines = b.split("\n");
|
||||
const m = aLines.length, n = bLines.length;
|
||||
const dp: number[][] = Array.from({ length: m + 1 }, () => new Array(n + 1).fill(0));
|
||||
for (let i = m - 1; i >= 0; i--)
|
||||
for (let j = n - 1; j >= 0; j--)
|
||||
dp[i][j] = aLines[i] === bLines[j]
|
||||
? dp[i + 1][j + 1] + 1
|
||||
: Math.max(dp[i + 1][j], dp[i][j + 1]);
|
||||
const result: DiffLine[] = [];
|
||||
let i = 0, j = 0;
|
||||
while (i < m && j < n) {
|
||||
if (aLines[i] === bLines[j]) { result.push({ type: "equal", text: aLines[i++] }); j++; }
|
||||
else if (dp[i + 1][j] >= dp[i][j + 1]) result.push({ type: "delete", text: aLines[i++] });
|
||||
else result.push({ type: "insert", text: bLines[j++] });
|
||||
}
|
||||
while (i < m) result.push({ type: "delete", text: aLines[i++] });
|
||||
while (j < n) result.push({ type: "insert", text: bLines[j++] });
|
||||
return result;
|
||||
}
|
||||
@@ -774,7 +774,7 @@ onUnmounted(() => {
|
||||
.tag-chip {
|
||||
font-size: 0.7rem;
|
||||
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
border-radius: 999px;
|
||||
padding: 0.1rem 0.4rem;
|
||||
}
|
||||
@@ -928,7 +928,7 @@ onUnmounted(() => {
|
||||
}
|
||||
.peek-linked-item:hover {
|
||||
background: color-mix(in srgb, var(--fs-accent) 8%, var(--fs-surface-raised));
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
}
|
||||
|
||||
.peek-linked-type {
|
||||
|
||||
@@ -4,6 +4,7 @@ import { useRouter } from "vue-router";
|
||||
import { apiGet } from "@/api/client";
|
||||
import type { TaskKind, TaskStatus, TaskPriority } from "@/types/note";
|
||||
import KindBadge from "@/components/KindBadge.vue";
|
||||
import NoteSweepPane from "@/components/NoteSweepPane.vue";
|
||||
import StatusBadge from "@/components/StatusBadge.vue";
|
||||
import PriorityBadge from "@/components/PriorityBadge.vue";
|
||||
import GraphView from "@/views/GraphView.vue";
|
||||
@@ -13,6 +14,7 @@ import {
|
||||
Workflow,
|
||||
Search,
|
||||
Share2,
|
||||
ShieldCheck,
|
||||
ChevronLeft,
|
||||
ChevronRight,
|
||||
X,
|
||||
@@ -24,7 +26,7 @@ const router = useRouter();
|
||||
|
||||
interface KnowledgeItem {
|
||||
id: number;
|
||||
note_type: "note" | "task" | "process";
|
||||
note_type: "note" | "task" | "process" | "snippet";
|
||||
title: string;
|
||||
snippet: string;
|
||||
tags: string[];
|
||||
@@ -42,9 +44,41 @@ interface KnowledgeItem {
|
||||
task_kind?: TaskKind;
|
||||
}
|
||||
|
||||
// ─── The facet vocabulary ─────────────────────────────────────────────────────
|
||||
// Mirrors services/knowledge._FACETS, which is where it is defined for real.
|
||||
// A facet spans BOTH typing axes — a record TYPE (note / process / snippet) or
|
||||
// a task KIND (`task` for any, else issue / spike) — because that is what this
|
||||
// feed actually holds.
|
||||
//
|
||||
// `plan` is still a valid facet at the API, for the 90 legacy plan-tasks, but
|
||||
// it has no chip: retired in 0066, it kept a chip of its own for longer than
|
||||
// `issue` — 17% of every task here — went without one (#3128). Those rows are
|
||||
// still reachable under Tasks, wearing a Plan badge.
|
||||
type Facet = "" | "note" | "task" | "issue" | "spike" | "snippet" | "process";
|
||||
|
||||
// The facets that select TASKS. Kinds are subsets of `task`, so any of them
|
||||
// means the duplicate report should be comparing tasks.
|
||||
const TASK_FACETS = new Set<Facet>(["task", "issue", "spike"]);
|
||||
|
||||
const FACET_CHIPS: [Exclude<Facet, "">, string][] = [
|
||||
["note", "Notes"],
|
||||
["task", "Tasks"],
|
||||
["issue", "Issues"],
|
||||
["spike", "Spikes"],
|
||||
["snippet", "Snippets"],
|
||||
["process", "Processes"],
|
||||
];
|
||||
|
||||
// ─── View mode ────────────────────────────────────────────────────────────────
|
||||
// The sweep is cross-cutting — a note that has gone false does not care which
|
||||
// facet it sits under — so it REPLACES the browse list rather than filtering
|
||||
// it. Filtering would mean the answer depended on which chip was active, which
|
||||
// is the under-reporting the sweep exists to prevent (milestone 317 step 4).
|
||||
const sweepActive = ref(false);
|
||||
|
||||
// ─── Filter state ─────────────────────────────────────────────────────────────
|
||||
|
||||
const activeType = ref<"" | "note" | "task" | "plan" | "process">("");
|
||||
const activeType = ref<Facet>("");
|
||||
const activeTag = ref("");
|
||||
const sortMode = ref<"modified" | "created" | "alpha" | "type">("modified");
|
||||
const searchQuery = ref("");
|
||||
@@ -70,9 +104,10 @@ const dupGroups = ref<DupGroup[]>([]);
|
||||
const dupSuggestion = ref("");
|
||||
const dupLoading = ref(false);
|
||||
const dupChecked = ref(false);
|
||||
// The report follows the type filter: viewing tasks checks tasks. Anything
|
||||
// else (all / plan / process) checks notes — the kind with the most to find.
|
||||
const dupKind = computed(() => (activeType.value === "task" ? "task" : "note"));
|
||||
// The report follows the type filter: viewing tasks — under ANY task facet,
|
||||
// including a single kind — checks tasks. Everything else checks notes, the
|
||||
// kind with the most to find.
|
||||
const dupKind = computed(() => (TASK_FACETS.has(activeType.value) ? "task" : "note"));
|
||||
|
||||
async function loadDuplicates() {
|
||||
dupLoading.value = true;
|
||||
@@ -96,8 +131,11 @@ watch(dupKind, () => { dupChecked.value = false; dupGroups.value = []; });
|
||||
|
||||
// ─── Type counts ──────────────────────────────────────────────────────────────
|
||||
|
||||
interface KnowledgeCounts { note: number; task: number; plan: number; process: number; total: number }
|
||||
const typeCounts = ref<KnowledgeCounts>({ note: 0, task: 0, plan: 0, process: 0, total: 0 });
|
||||
// One number per facet, plus the grand total. Partial because the server sends
|
||||
// a key only for a facet it has rows for. Kinds are subsets of `task` and are
|
||||
// deliberately absent from `total` — including them would count an issue twice.
|
||||
type KnowledgeCounts = Partial<Record<Exclude<Facet, "">, number>> & { total: number };
|
||||
const typeCounts = ref<KnowledgeCounts>({ total: 0 });
|
||||
|
||||
async function fetchCounts() {
|
||||
try {
|
||||
@@ -234,6 +272,10 @@ function onSearchInput() {
|
||||
}
|
||||
|
||||
watch([activeType, sortMode], () => resetAndReobserve());
|
||||
// Closing the sweep remounts the feed, and with it the scroll sentinel — a
|
||||
// fresh element the old observer is not watching. Without this the list loads
|
||||
// its first page and then never loads another.
|
||||
watch(sweepActive, (open) => { if (!open) resetAndReobserve(); });
|
||||
watch(activeTag, () => { fetchCounts(); resetAndReobserve(); });
|
||||
|
||||
// ─── Today bar ────────────────────────────────────────────────────────────────
|
||||
@@ -274,9 +316,18 @@ function isOverdue(item: KnowledgeItem): boolean {
|
||||
return new Date(item.due_date) < new Date(new Date().toDateString());
|
||||
}
|
||||
|
||||
// Each record kind opens in ITS OWN editor. A snippet used to fall through to
|
||||
// /notes/:id, whose save is a plain PATCH of the body — which left the snippet's
|
||||
// derived `data` mirror describing the previous version (#3128). The service now
|
||||
// recomposes the mirror either way, so this is no longer the guard; it is simply
|
||||
// that the note editor cannot edit a snippet's signature, language or locations,
|
||||
// and offering it as the way in was always wrong. Processes stay here on
|
||||
// purpose: they have no editor of their own and the note editor knows the type.
|
||||
function openItem(item: KnowledgeItem) {
|
||||
if (item.note_type === 'task') {
|
||||
router.push(`/tasks/${item.id}`);
|
||||
} else if (item.note_type === 'snippet') {
|
||||
router.push(`/snippets/${item.id}`);
|
||||
} else {
|
||||
router.push(`/notes/${item.id}`);
|
||||
}
|
||||
@@ -380,14 +431,14 @@ onUnmounted(() => {
|
||||
<span v-if="typeCounts.total > 1" class="filter-count">{{ typeCounts.total }}</span>
|
||||
</button>
|
||||
<button
|
||||
v-for="[val, label, key] in ([['note','Notes','note'],['task','Tasks','task'],['plan','Plans','plan'],['process','Processes','process']] as [string,string,string][])"
|
||||
v-for="[val, label] in FACET_CHIPS"
|
||||
:key="val"
|
||||
class="filter-btn"
|
||||
:class="{ active: activeType === val }"
|
||||
@click="activeType = (val as '' | 'note' | 'task' | 'plan' | 'process')"
|
||||
@click="activeType = val"
|
||||
>
|
||||
<span class="filter-btn-label">{{ label }}</span>
|
||||
<span v-if="typeCounts[key as keyof KnowledgeCounts] > 1" class="filter-count">{{ typeCounts[key as keyof KnowledgeCounts] }}</span>
|
||||
<span v-if="(typeCounts[val] ?? 0) > 1" class="filter-count">{{ typeCounts[val] }}</span>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
@@ -433,6 +484,15 @@ onUnmounted(() => {
|
||||
<Share2 :size="16" />
|
||||
Graph
|
||||
</button>
|
||||
<button
|
||||
class="btn-ghost btn-compact"
|
||||
:class="{ active: sweepActive }"
|
||||
title="Notes that assert a fact about something outside your control, least-recently-confirmed first. Most notes are decisions and never appear."
|
||||
@click="sweepActive = !sweepActive"
|
||||
>
|
||||
<ShieldCheck :size="16" />
|
||||
Due
|
||||
</button>
|
||||
<button
|
||||
class="btn-ghost btn-compact"
|
||||
:disabled="dupLoading"
|
||||
@@ -443,6 +503,12 @@ onUnmounted(() => {
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<!-- The sweep replaces the feed. It is not a facet: a facet answers
|
||||
"show me this kind", and this answers "show me what nobody has
|
||||
confirmed" — a question the type chips cannot narrow without
|
||||
under-reporting it. -->
|
||||
<NoteSweepPane v-if="sweepActive" @open-note="(id) => router.push(`/notes/${id}`)" />
|
||||
|
||||
<!-- Near-duplicate report. A proposal surface only: unlike snippets
|
||||
(which merge losslessly), notes are never merged — the right fix is
|
||||
supersession, extraction into a reference note, or leaving parallel
|
||||
@@ -480,6 +546,12 @@ onUnmounted(() => {
|
||||
</template>
|
||||
</div>
|
||||
|
||||
<!-- The whole feed stands down while the sweep is open: two answers
|
||||
to two different questions on one screen is neither. Wrapped
|
||||
rather than given an extra v-if branch, because the scroll
|
||||
sentinel lives inside the grid and the observer must not be left
|
||||
holding a ref to something that never renders. -->
|
||||
<template v-if="!sweepActive">
|
||||
<!-- Loading / empty -->
|
||||
<div v-if="loading && items.length === 0" class="knowledge-empty">Loading…</div>
|
||||
<div v-else-if="!loading && items.length === 0" class="knowledge-empty">
|
||||
@@ -501,6 +573,7 @@ onUnmounted(() => {
|
||||
<span v-if="item.note_type === 'note'">Note</span>
|
||||
<span v-else-if="item.note_type === 'task'">{{ item.task_kind === 'plan' ? 'Plan' : 'Task' }}</span>
|
||||
<span v-else-if="item.note_type === 'process'">Process</span>
|
||||
<span v-else-if="item.note_type === 'snippet'">Snippet</span>
|
||||
</span>
|
||||
<!-- Kind sits BESIDE the type badge, not inside it: the type badge
|
||||
speaks the vocabulary of this view's type filter (note / task /
|
||||
@@ -552,6 +625,7 @@ onUnmounted(() => {
|
||||
<span v-if="contentFetching" class="sentinel-loading">Loading…</span>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
</div>
|
||||
|
||||
<!-- Graph panel -->
|
||||
@@ -753,7 +827,7 @@ onUnmounted(() => {
|
||||
}
|
||||
.filter-btn.active .filter-count {
|
||||
background: color-mix(in srgb, var(--fs-accent) 20%, transparent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
}
|
||||
.filter-tag { font-size: 0.78rem; }
|
||||
|
||||
@@ -881,6 +955,14 @@ onUnmounted(() => {
|
||||
.badge--note { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: #7A6DA8; }
|
||||
.badge--task { background: rgba(212,160,23,0.15); color: #fbbf24; }
|
||||
.badge--plan { background: rgba(99,102,241,0.18); color: #818cf8; }
|
||||
/* Snippet and process are NEUTRAL on purpose. Both were unstyled — and the
|
||||
snippet had no label either, so all 90 of them rendered an empty chip in
|
||||
this feed (#3128). Giving them hues would put a third and fourth colour
|
||||
beside KindBadge's warm/cool pair on the same card; a record type that
|
||||
isn't an alarm reads better as plain. Standard body pair, so the contrast
|
||||
is the one the palette already guarantees. */
|
||||
.badge--snippet,
|
||||
.badge--process { background: var(--fs-surface-raised); color: var(--fs-text-secondary); }
|
||||
|
||||
.k-card-body { flex: 1; padding-right: 40px; }
|
||||
.k-card-title {
|
||||
@@ -926,7 +1008,7 @@ onUnmounted(() => {
|
||||
border-radius: 4px;
|
||||
white-space: nowrap;
|
||||
background: color-mix(in srgb, var(--fs-text-secondary) 15%, transparent);
|
||||
color: var(--fs-text-secondary);
|
||||
color: var(--fs-text-secondary-fg);
|
||||
}
|
||||
|
||||
/* ── Task card ──────────────────────────────────────────── */
|
||||
|
||||
@@ -34,6 +34,14 @@ const tags = ref<string[]>([]);
|
||||
const projectId = ref<number | null>(null);
|
||||
const milestoneId = ref<number | null>(null);
|
||||
const noteType = ref<NoteType>("note");
|
||||
|
||||
// The note's own check (milestone 317). Offered only for a plain note: a
|
||||
// task's decay is its status, and a snippet has verify_snippet — the service
|
||||
// refuses both, so the form must not ask for what the save would reject.
|
||||
const verifyWith = ref("");
|
||||
const expiresWhen = ref("");
|
||||
const verifiedAt = ref<string | null>(null);
|
||||
const canCarryCheck = computed(() => noteType.value === "note");
|
||||
const dirty = ref(false);
|
||||
const saving = ref(false);
|
||||
const showPreview = ref(false);
|
||||
@@ -198,6 +206,41 @@ let savedTags: string[] = [];
|
||||
let savedProjectId: number | null = null;
|
||||
let savedMilestoneId: number | null = null;
|
||||
let savedNoteType: NoteType = "note";
|
||||
let savedVerifyWith = "";
|
||||
let savedExpiresWhen = "";
|
||||
|
||||
/** The write, in one place. Three call sites (save, create, auto-save) each
|
||||
* spelled this out, so every new field had to be added three times — which is
|
||||
* how one of them ends up not carrying it. */
|
||||
function payload() {
|
||||
return {
|
||||
title: title.value,
|
||||
body: body.value,
|
||||
tags: tags.value,
|
||||
project_id: projectId.value,
|
||||
milestone_id: milestoneId.value,
|
||||
note_type: noteType.value,
|
||||
// "" clears the check: the REST door reads an empty string as NULL
|
||||
// (NULLABLE_NOTE_TEXT), which is how a cleared form input says "remove
|
||||
// this" without needing the MCP door's explicit `clear` list.
|
||||
verify_with: canCarryCheck.value ? verifyWith.value : "",
|
||||
expires_when: canCarryCheck.value ? expiresWhen.value : "",
|
||||
};
|
||||
}
|
||||
|
||||
/** What the form last agreed with the server about — the other half of the
|
||||
* same list, and for the same reason. */
|
||||
function snapshot() {
|
||||
savedTitle = title.value;
|
||||
savedBody = body.value;
|
||||
savedTags = [...tags.value];
|
||||
savedProjectId = projectId.value;
|
||||
savedMilestoneId = milestoneId.value;
|
||||
savedNoteType = noteType.value;
|
||||
savedVerifyWith = verifyWith.value;
|
||||
savedExpiresWhen = expiresWhen.value;
|
||||
dirty.value = false;
|
||||
}
|
||||
|
||||
function markDirty() {
|
||||
dirty.value =
|
||||
@@ -206,7 +249,9 @@ function markDirty() {
|
||||
JSON.stringify(tags.value) !== JSON.stringify(savedTags) ||
|
||||
projectId.value !== savedProjectId ||
|
||||
milestoneId.value !== savedMilestoneId ||
|
||||
noteType.value !== savedNoteType;
|
||||
noteType.value !== savedNoteType ||
|
||||
verifyWith.value !== savedVerifyWith ||
|
||||
expiresWhen.value !== savedExpiresWhen;
|
||||
}
|
||||
|
||||
function onBodyUpdate(newVal: string) {
|
||||
@@ -224,12 +269,10 @@ onMounted(async () => {
|
||||
projectId.value = store.currentNote.project_id ?? null;
|
||||
milestoneId.value = store.currentNote.milestone_id ?? null;
|
||||
noteType.value = (store.currentNote.note_type as NoteType) || "note";
|
||||
savedTitle = title.value;
|
||||
savedBody = body.value;
|
||||
savedTags = [...tags.value];
|
||||
savedProjectId = projectId.value;
|
||||
savedMilestoneId = milestoneId.value;
|
||||
savedNoteType = noteType.value;
|
||||
verifyWith.value = store.currentNote.verify_with || "";
|
||||
expiresWhen.value = store.currentNote.expires_when || "";
|
||||
verifiedAt.value = store.currentNote.verified_at ?? null;
|
||||
snapshot();
|
||||
}
|
||||
} else {
|
||||
// New note: read type from query param
|
||||
@@ -260,31 +303,11 @@ async function save() {
|
||||
const finalBody = body.value;
|
||||
try {
|
||||
if (isEditing.value) {
|
||||
await store.updateNote(noteId.value!, {
|
||||
title: title.value,
|
||||
body: finalBody,
|
||||
tags: tags.value,
|
||||
project_id: projectId.value,
|
||||
milestone_id: milestoneId.value,
|
||||
note_type: noteType.value,
|
||||
});
|
||||
savedTitle = title.value;
|
||||
savedBody = body.value;
|
||||
savedTags = [...tags.value];
|
||||
savedProjectId = projectId.value;
|
||||
savedMilestoneId = milestoneId.value;
|
||||
savedNoteType = noteType.value;
|
||||
dirty.value = false;
|
||||
await store.updateNote(noteId.value!, { ...payload(), body: finalBody });
|
||||
snapshot();
|
||||
toast.show("Note saved");
|
||||
} else {
|
||||
const note = await store.createNote({
|
||||
title: title.value,
|
||||
body: finalBody,
|
||||
tags: tags.value,
|
||||
project_id: projectId.value,
|
||||
milestone_id: milestoneId.value,
|
||||
note_type: noteType.value,
|
||||
});
|
||||
const note = await store.createNote({ ...payload(), body: finalBody });
|
||||
dirty.value = false;
|
||||
toast.show("Note created");
|
||||
router.push(`/notes/${note.id}`);
|
||||
@@ -321,18 +344,8 @@ async function doAutoSave() {
|
||||
saving.value = true;
|
||||
const finalBody = body.value;
|
||||
try {
|
||||
await store.updateNote(noteId.value!, {
|
||||
title: title.value, body: finalBody, tags: tags.value,
|
||||
project_id: projectId.value, milestone_id: milestoneId.value,
|
||||
note_type: noteType.value,
|
||||
});
|
||||
savedTitle = title.value;
|
||||
savedBody = body.value;
|
||||
savedTags = [...tags.value];
|
||||
savedProjectId = projectId.value;
|
||||
savedMilestoneId = milestoneId.value;
|
||||
savedNoteType = noteType.value;
|
||||
dirty.value = false;
|
||||
await store.updateNote(noteId.value!, { ...payload(), body: finalBody });
|
||||
snapshot();
|
||||
toast.show("Auto-saved");
|
||||
} catch {
|
||||
// Silent
|
||||
@@ -496,6 +509,41 @@ onUnmounted(() => assist.clearSelection());
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<!-- The note's own check (milestone 317). Shown only for a plain
|
||||
note: the service refuses a check on a task or a snippet, so
|
||||
offering the fields there would be a form whose save fails. -->
|
||||
<template v-if="canCarryCheck">
|
||||
<div class="sb-field">
|
||||
<div class="sb-label-row">
|
||||
<span class="sb-label">Check</span>
|
||||
<span v-if="verifyWith" class="check-age" :class="{ unchecked: !verifiedAt }">
|
||||
{{ verifiedAt ? `checked ${verifiedAt.slice(0, 10)}` : "never checked" }}
|
||||
</span>
|
||||
</div>
|
||||
<!-- Phrased as the question that decides, not as a field name.
|
||||
"Verify with" would get filled in on every note; "could this
|
||||
become false without anyone editing it?" gets filled in on
|
||||
the few that can. -->
|
||||
<textarea
|
||||
v-model="verifyWith"
|
||||
class="sb-textarea"
|
||||
rows="2"
|
||||
placeholder="How would someone check this is still true? Leave empty unless this note could become false without anyone editing it."
|
||||
@input="markDirty"
|
||||
></textarea>
|
||||
</div>
|
||||
<div v-if="verifyWith" class="sb-field">
|
||||
<label class="sb-label">Ends when</label>
|
||||
<textarea
|
||||
v-model="expiresWhen"
|
||||
class="sb-textarea"
|
||||
rows="2"
|
||||
placeholder="What state ends it? A state, not a date — “when the forge numbers runs per workflow”, not “in six months”."
|
||||
@input="markDirty"
|
||||
></textarea>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<!-- Link Suggestions -->
|
||||
<div v-if="linkSuggestions.length > 0" class="sb-field link-suggest-field">
|
||||
<div class="sb-label-row">
|
||||
@@ -678,7 +726,7 @@ onUnmounted(() => assist.clearSelection());
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.sb-select, .sb-input {
|
||||
.sb-select, .sb-input, .sb-textarea {
|
||||
width: 100%;
|
||||
padding: 5px 8px;
|
||||
border-radius: var(--fs-radius-sm);
|
||||
@@ -690,9 +738,27 @@ onUnmounted(() => assist.clearSelection());
|
||||
outline: none;
|
||||
transition: border-color 0.15s;
|
||||
}
|
||||
.sb-select:focus, .sb-input:focus {
|
||||
.sb-select:focus, .sb-input:focus, .sb-textarea:focus {
|
||||
border-color: var(--fs-accent);
|
||||
}
|
||||
.sb-textarea {
|
||||
resize: vertical;
|
||||
line-height: 1.4;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
/* No red/amber ramp, matching RuleSweepPane: a colour scale would restate the
|
||||
sweep's ordering and force an invented "stale after N days" threshold.
|
||||
"Never" is marked because it is categorically different from a date, not a
|
||||
worse one — it means nobody has ever confirmed the claim. */
|
||||
.check-age {
|
||||
font-size: 0.7rem;
|
||||
color: var(--fs-text-secondary);
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
.check-age.unchecked {
|
||||
font-style: italic;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
|
||||
/* Link Suggestions */
|
||||
.link-suggest-field { gap: 0.4rem; }
|
||||
|
||||
@@ -416,7 +416,7 @@ async function convertToTask() {
|
||||
}
|
||||
.badge-note {
|
||||
background: color-mix(in srgb, var(--fs-accent) 12%, transparent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
border: 1px solid color-mix(in srgb, var(--fs-accent) 25%, transparent);
|
||||
}
|
||||
.badge-task {
|
||||
|
||||
@@ -1287,8 +1287,8 @@ async function confirmDelete() {
|
||||
|
||||
.stat-todo { background: color-mix(in srgb, var(--fs-text-tertiary) 8%, transparent); color: var(--fs-text-secondary); border-color: var(--fs-border-color); }
|
||||
.stat-inprogress { background: color-mix(in srgb, #3b82f6 10%, transparent); color: #3b82f6; border-color: color-mix(in srgb, #3b82f6 28%, transparent); }
|
||||
.stat-done { background: color-mix(in srgb, var(--fs-success) 10%, transparent); color: var(--fs-success); border-color: color-mix(in srgb, var(--fs-success) 28%, transparent); }
|
||||
.stat-notes { background: color-mix(in srgb, var(--fs-accent) 8%, transparent); color: var(--fs-accent); border-color: color-mix(in srgb, var(--fs-accent) 22%, transparent); }
|
||||
.stat-done { background: color-mix(in srgb, var(--fs-success) 10%, transparent); color: var(--fs-success-fg); border-color: color-mix(in srgb, var(--fs-success) 28%, transparent); }
|
||||
.stat-notes { background: color-mix(in srgb, var(--fs-accent) 8%, transparent); color: var(--fs-accent-fg); border-color: color-mix(in srgb, var(--fs-accent) 22%, transparent); }
|
||||
|
||||
/* ── Pattern-library coverage card ───────────────────────────── */
|
||||
.coverage-card {
|
||||
@@ -1471,7 +1471,7 @@ async function confirmDelete() {
|
||||
.tab-btn.active .tab-count {
|
||||
background: color-mix(in srgb, var(--fs-accent) 12%, transparent);
|
||||
border-color: color-mix(in srgb, var(--fs-accent) 30%, transparent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
}
|
||||
|
||||
/* ── Tasks view ──────────────────────────────────────────────── */
|
||||
@@ -1707,7 +1707,7 @@ async function confirmDelete() {
|
||||
border-radius: 3px;
|
||||
margin-left: auto;
|
||||
}
|
||||
.col-add-btn:hover { color: var(--fs-accent); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
|
||||
.col-add-btn:hover { color: var(--fs-accent-fg); background: color-mix(in srgb, var(--fs-accent) 10%, transparent); }
|
||||
|
||||
.kanban-cards { display: flex; flex-direction: column; gap: 0.3rem; }
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ import type { User } from "@/types/auth";
|
||||
import PaginationBar from "@/components/PaginationBar.vue";
|
||||
import TagInput from "@/components/TagInput.vue";
|
||||
import { fmtDate, fmtLogStamp } from "@/utils/dateFormat";
|
||||
import { fetchVersion, type VersionPayload } from "@/api/version";
|
||||
|
||||
const store = useSettingsStore();
|
||||
const authStore = useAuthStore();
|
||||
@@ -85,6 +86,7 @@ const kbWritePathEnabled = ref(true);
|
||||
// code embeddings sit on a much higher similarity floor than prose, so 0.55 let
|
||||
// unrelated code through (#2223). Shares top-k, not the threshold.
|
||||
const kbWritePathThreshold = ref("0.68");
|
||||
const kbRuleHintThreshold = ref("0.72");
|
||||
// Near-duplicate report floors, one per record kind (services/dedup.py).
|
||||
// Snippets are single-chunk, so their floor sits below the 0.90 write-time
|
||||
// gate and catches what it lets through. Notes/tasks are scored at chunk
|
||||
@@ -147,12 +149,17 @@ async function saveKbInject() {
|
||||
// Same `|| default` reasoning: falling back to 0 would surface every
|
||||
// snippet in the corpus on every edit, which is the failure this knob fixes.
|
||||
const wpT = Math.min(1, Math.max(0, Number(kbWritePathThreshold.value) || 0.68));
|
||||
// Same `|| default` reasoning again, and it bites harder here: a rule hint
|
||||
// fires on every write, so a fallback of 0 would attach a standing rule to
|
||||
// every edit in the session.
|
||||
const rhT = Math.min(1, Math.max(0, Number(kbRuleHintThreshold.value) || 0.72));
|
||||
kbInjectThreshold.value = String(t);
|
||||
kbInjectTopK.value = String(k);
|
||||
kbDupThresholdSnippet.value = String(dupSnip);
|
||||
kbDupThresholdNote.value = String(dupNote);
|
||||
kbDupThresholdTask.value = String(dupTask);
|
||||
kbWritePathThreshold.value = String(wpT);
|
||||
kbRuleHintThreshold.value = String(rhT);
|
||||
savingKbInject.value = true;
|
||||
kbInjectSaved.value = false;
|
||||
try {
|
||||
@@ -165,6 +172,10 @@ async function saveKbInject() {
|
||||
// measurements that split them.
|
||||
kb_writepath_enabled: kbWritePathEnabled.value ? 'true' : 'false',
|
||||
kb_writepath_threshold: String(wpT),
|
||||
// A THIRD corpus with a third bar — see RULEHINT_DEFAULT_THRESHOLD
|
||||
// in services/plugin_context.py for why rules cannot share the
|
||||
// code threshold any more than code could share the prose one.
|
||||
kb_rulehint_threshold: String(rhT),
|
||||
kb_duplicate_threshold_snippet: String(dupSnip),
|
||||
kb_duplicate_threshold_note: String(dupNote),
|
||||
kb_duplicate_threshold_task: String(dupTask),
|
||||
@@ -187,7 +198,47 @@ const changingPassword = ref(false);
|
||||
const invalidatingSessions = ref(false);
|
||||
const exporting = ref(false);
|
||||
const restoring = ref(false);
|
||||
const appVersion = ref('dev');
|
||||
// Backup, export and restore walk the whole store, so they are slow BY DESIGN
|
||||
// and the client's ordinary 30s default would cut them off mid-work. They are
|
||||
// still bounded: rule 156 asks for a deadline, not a short one, and "no ceiling
|
||||
// at all" is what leaves a restore that died server-side spinning forever.
|
||||
const BULK_TRANSFER_TIMEOUT_MS = 10 * 60 * 1000;
|
||||
|
||||
function bulkDeadline(): AbortSignal {
|
||||
return AbortSignal.timeout(BULK_TRANSFER_TIMEOUT_MS);
|
||||
}
|
||||
|
||||
// ── What's running (#3127 checklist 12) ─────────────────────────────────
|
||||
// Three states kept apart, because collapsing any two of them is the defect
|
||||
// this readout exists to remove: `null` + no error = not asked yet (the Config
|
||||
// tab has not been opened); a payload = answered, with each ABSENT field shown
|
||||
// as "unknown"; `versionError` = the fetch itself failed, which is its own
|
||||
// thing and must never render as a blank or as a plausible-looking value.
|
||||
const versionInfo = ref<VersionPayload | null>(null);
|
||||
const versionLoading = ref(false);
|
||||
const versionError = ref("");
|
||||
const commitCopied = ref(false);
|
||||
|
||||
async function loadVersionPanel() {
|
||||
if (versionLoading.value) return;
|
||||
versionLoading.value = true;
|
||||
versionError.value = "";
|
||||
try {
|
||||
versionInfo.value = await fetchVersion();
|
||||
} catch (e) {
|
||||
versionInfo.value = null;
|
||||
versionError.value = apiErrorMessage(e, "Could not reach the instance to ask what it is running.");
|
||||
} finally {
|
||||
versionLoading.value = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function copyCommit() {
|
||||
if (!versionInfo.value?.commit) return;
|
||||
await copyToClipboard(versionInfo.value.commit);
|
||||
commitCopied.value = true;
|
||||
setTimeout(() => { commitCopied.value = false; }, 2000);
|
||||
}
|
||||
const restoreFileInput = ref<HTMLInputElement | null>(null);
|
||||
|
||||
// Migrate stored "admin" → "config"; unknown tabs fall back to "general"
|
||||
@@ -201,6 +252,7 @@ function _loadTabContent(tab: string) {
|
||||
else if (tab === "logs") loadLogsPanel();
|
||||
else if (tab === "groups") loadGroupsPanel();
|
||||
else if (tab === "areas") canonStore.fetchCatalog(true);
|
||||
else if (tab === "config" && !versionInfo.value) loadVersionPanel();
|
||||
}
|
||||
if (tab === "apikeys") { fetchApiKeys(); }
|
||||
}
|
||||
@@ -554,10 +606,6 @@ function toggleProfileWorkDay(day: string) {
|
||||
function emptyTagsFetch(): Promise<string[]> { return Promise.resolve([]) }
|
||||
|
||||
onMounted(async () => {
|
||||
try {
|
||||
const v = await apiGet<{ version: string }>('/api/version')
|
||||
appVersion.value = v.version
|
||||
} catch { /* non-critical */ }
|
||||
await store.fetchSettings();
|
||||
newEmail.value = authStore.user?.email ?? "";
|
||||
|
||||
@@ -573,6 +621,9 @@ onMounted(async () => {
|
||||
kbInjectTopK.value = allSettings.kb_autoinject_top_k;
|
||||
}
|
||||
kbWritePathEnabled.value = allSettings.kb_writepath_enabled !== "false";
|
||||
if (allSettings.kb_rulehint_threshold !== undefined) {
|
||||
kbRuleHintThreshold.value = allSettings.kb_rulehint_threshold;
|
||||
}
|
||||
if (allSettings.kb_writepath_threshold !== undefined) {
|
||||
kbWritePathThreshold.value = allSettings.kb_writepath_threshold;
|
||||
}
|
||||
@@ -727,7 +778,7 @@ async function exportData(scope: "user" | "full") {
|
||||
exporting.value = true;
|
||||
try {
|
||||
const url = scope === "full" ? "/api/admin/backup" : "/api/admin/backup?scope=user";
|
||||
const res = await fetch(url);
|
||||
const res = await fetch(url, { signal: bulkDeadline() });
|
||||
if (!res.ok) {
|
||||
const body = await res.json().catch(() => ({ error: `Error ${res.status}` }));
|
||||
throw new Error((body as Record<string, string>).error || `Error ${res.status}`);
|
||||
@@ -752,7 +803,7 @@ const exportingNotes = ref(false);
|
||||
async function exportNotes(format: "markdown" | "json") {
|
||||
exportingNotes.value = true;
|
||||
try {
|
||||
const res = await fetch(`/api/export?format=${format}`);
|
||||
const res = await fetch(`/api/export?format=${format}`, { signal: bulkDeadline() });
|
||||
if (!res.ok) throw new Error(`Error ${res.status}`);
|
||||
const blob = await res.blob();
|
||||
const ext = format === "json" ? "json" : "zip";
|
||||
@@ -981,6 +1032,7 @@ async function handleRestoreFile(event: Event) {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(data),
|
||||
signal: bulkDeadline(),
|
||||
});
|
||||
if (!res.ok) {
|
||||
const body = await res.json().catch(() => ({ error: `Error ${res.status}` }));
|
||||
@@ -1417,6 +1469,29 @@ async function deleteUser(userId: number) {
|
||||
location, not by resemblance.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-rulehint-threshold">Standing-rule confidence threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-rulehint-threshold"
|
||||
v-model="kbRuleHintThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The same hint can mention a standing rule whose trigger resembles what's
|
||||
being written — only rules marked <em>conditional</em>, since always-on
|
||||
ones are already loaded. Stricter again than the threshold above, because
|
||||
there are far fewer rules than snippets: with a small set, something
|
||||
always ranks first, so the bar has to carry more of the judgement.
|
||||
Raise it if rules keep arriving unread; lower it if a rule you needed
|
||||
never showed up. Settings → check the pull-through in
|
||||
<code>retrieval_telemetry</code> to see which is happening.
|
||||
</p>
|
||||
</div>
|
||||
<!-- A design system belongs to a PROJECT, and the picker for it lives on
|
||||
the project. There was a setting here that designated the system
|
||||
this install's own interface was built from; it only ever described
|
||||
@@ -2109,6 +2184,48 @@ async function deleteUser(userId: number) {
|
||||
<!-- ── Admin ── -->
|
||||
<div v-if="authStore.isAdmin" v-show="activeTab === 'config'" class="settings-grid">
|
||||
|
||||
<section class="settings-section full-width">
|
||||
<h2>What's running</h2>
|
||||
<p class="section-desc">
|
||||
The build serving this page. Paste the commit into a <code>:sha</code> image
|
||||
lookup to check the registry and the app agree about what was published.
|
||||
</p>
|
||||
|
||||
<div v-if="versionLoading" class="state-msg">Reading the ledger…</div>
|
||||
<div v-else-if="versionError" class="error-msg">
|
||||
{{ versionError }}
|
||||
<button class="btn-ghost btn-compact version-retry" @click="loadVersionPanel">Try again</button>
|
||||
</div>
|
||||
<dl v-else-if="versionInfo" class="version-grid">
|
||||
<dt>Version</dt>
|
||||
<dd class="version-value">{{ versionInfo.version }}</dd>
|
||||
|
||||
<dt>Channel</dt>
|
||||
<dd :class="versionInfo.channel === undefined ? 'version-unknown' : 'version-value'">
|
||||
{{ versionInfo.channel ?? "unknown" }}
|
||||
</dd>
|
||||
|
||||
<dt>Commit</dt>
|
||||
<dd v-if="versionInfo.commit" class="version-value version-commit">
|
||||
<span class="version-sha">{{ versionInfo.commit }}</span>
|
||||
<button class="btn-ghost btn-compact" @click="copyCommit">
|
||||
{{ commitCopied ? "Copied" : "Copy" }}
|
||||
</button>
|
||||
</dd>
|
||||
<dd v-else class="version-unknown">unknown</dd>
|
||||
|
||||
<dt>Build</dt>
|
||||
<!-- The ordering key, kept because its ABSENCE is the diagnostic one:
|
||||
no key means this build is not part of any update order, which is
|
||||
what a local or hand-built image looks like. `??` not `||` — 0 is
|
||||
a legitimate key. -->
|
||||
<dd :class="versionInfo.build === undefined ? 'version-unknown' : 'version-value'">
|
||||
{{ versionInfo.build ?? "unknown" }}
|
||||
</dd>
|
||||
</dl>
|
||||
<div v-else class="empty-msg">Nothing asked yet.</div>
|
||||
</section>
|
||||
|
||||
<section class="settings-section full-width">
|
||||
<h2>Application URL</h2>
|
||||
<p class="section-desc">
|
||||
@@ -2732,7 +2849,7 @@ async function deleteUser(userId: number) {
|
||||
background: var(--fs-surface-raised);
|
||||
}
|
||||
.sidebar-item.active {
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
background: color-mix(in srgb, var(--fs-accent) 8%, transparent);
|
||||
border-left-color: var(--fs-accent);
|
||||
font-weight: 500;
|
||||
@@ -2768,6 +2885,45 @@ async function deleteUser(userId: number) {
|
||||
letter-spacing: 0.07em;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
/* What's running — a definition list of instance facts. Spacing/geometry only;
|
||||
colour and type come from the tokens. */
|
||||
.version-grid {
|
||||
display: grid;
|
||||
grid-template-columns: max-content 1fr;
|
||||
gap: 0.4rem 1rem;
|
||||
margin: 0;
|
||||
align-items: baseline;
|
||||
}
|
||||
.version-grid dt {
|
||||
font-size: 0.8rem;
|
||||
color: var(--fs-text-secondary);
|
||||
}
|
||||
.version-grid dd {
|
||||
margin: 0;
|
||||
font-size: 0.875rem;
|
||||
font-family: var(--fs-font-mono);
|
||||
color: var(--fs-text-primary);
|
||||
}
|
||||
/* An absent field reads as absent — never as a blank, and never styled to look
|
||||
like a value it does not have (#3127 checklist 12). */
|
||||
.version-grid dd.version-unknown {
|
||||
font-family: inherit;
|
||||
font-style: italic;
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
.version-commit {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.version-sha {
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
.version-retry {
|
||||
margin-left: 0.5rem;
|
||||
}
|
||||
|
||||
.section-desc {
|
||||
margin: 0 0 1rem;
|
||||
font-size: 0.875rem;
|
||||
@@ -3099,7 +3255,7 @@ async function deleteUser(userId: number) {
|
||||
border-radius: var(--fs-radius-sm);
|
||||
}
|
||||
.role-admin {
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
|
||||
}
|
||||
.role-user {
|
||||
@@ -3179,9 +3335,9 @@ async function deleteUser(userId: number) {
|
||||
text-transform: uppercase; letter-spacing: 0.05em;
|
||||
padding: 0.1rem 0.35rem; border-radius: var(--fs-radius-sm);
|
||||
}
|
||||
.cat-audit { color: var(--fs-accent); background: color-mix(in srgb, var(--fs-accent) 15%, transparent); }
|
||||
.cat-usage { color: var(--fs-success); background: color-mix(in srgb, var(--fs-success) 15%, transparent); }
|
||||
.cat-error { color: var(--fs-error); background: color-mix(in srgb, var(--fs-error) 15%, transparent); }
|
||||
.cat-audit { color: var(--fs-accent-fg); background: color-mix(in srgb, var(--fs-accent) 15%, transparent); }
|
||||
.cat-usage { color: var(--fs-success-fg); background: color-mix(in srgb, var(--fs-success) 15%, transparent); }
|
||||
.cat-error { color: var(--fs-error-fg); background: color-mix(in srgb, var(--fs-error) 15%, transparent); }
|
||||
.method-tag {
|
||||
display: inline-block;
|
||||
font-size: 0.65rem; font-weight: 500; font-family: monospace;
|
||||
@@ -3346,8 +3502,8 @@ async function deleteUser(userId: number) {
|
||||
padding: 0.15rem 0.4rem;
|
||||
border-radius: 4px;
|
||||
}
|
||||
.role-owner { background: color-mix(in srgb, var(--fs-warning) 15%, transparent); color: var(--fs-warning); }
|
||||
.role-member { background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent); color: var(--fs-text-tertiary); }
|
||||
.role-owner { background: color-mix(in srgb, var(--fs-warning) 15%, transparent); color: var(--fs-warning-fg); }
|
||||
.role-member { background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent); color: var(--fs-text-tertiary-fg); }
|
||||
|
||||
.members-empty {
|
||||
color: var(--fs-text-tertiary);
|
||||
@@ -3528,7 +3684,7 @@ async function deleteUser(userId: number) {
|
||||
.day-btn.active {
|
||||
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
|
||||
border-color: var(--fs-accent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
|
||||
@@ -242,9 +242,9 @@ onMounted(async () => {
|
||||
border-radius: 4px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.perm-viewer { background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent); color: var(--fs-text-tertiary); }
|
||||
.perm-editor { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent); }
|
||||
.perm-admin { background: color-mix(in srgb, var(--fs-warning) 15%, transparent); color: var(--fs-warning); }
|
||||
.perm-viewer { background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent); color: var(--fs-text-tertiary-fg); }
|
||||
.perm-editor { background: color-mix(in srgb, var(--fs-accent) 15%, transparent); color: var(--fs-accent-fg); }
|
||||
.perm-admin { background: color-mix(in srgb, var(--fs-warning) 15%, transparent); color: var(--fs-warning-fg); }
|
||||
|
||||
.empty-msg {
|
||||
margin: 0;
|
||||
|
||||
@@ -288,7 +288,7 @@ async function confirmDelete() {
|
||||
font-family: var(--fs-font-mono);
|
||||
font-size: 0.82rem;
|
||||
background: color-mix(in srgb, var(--fs-accent) 12%, transparent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
padding: 0.08rem 0.35rem;
|
||||
border-radius: var(--fs-radius-sm);
|
||||
word-break: break-all;
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
type SnippetListItem,
|
||||
} from "@/api/snippets";
|
||||
import { useToastStore } from "@/stores/toast";
|
||||
import UsageBadge from "@/components/UsageBadge.vue";
|
||||
|
||||
const router = useRouter();
|
||||
const toast = useToastStore();
|
||||
@@ -198,23 +199,6 @@ function languageOf(tags: string[]): string {
|
||||
return tags.find((t) => t && t !== "snippet") ?? "";
|
||||
}
|
||||
|
||||
/** A snippet that has been offered repeatedly and never opened. The threshold
|
||||
* is 3 rather than 1 because one or two surfacings is noise — the record may
|
||||
* simply not have come up in a relevant context yet. */
|
||||
function isDeadWeight(s: SnippetListItem): boolean {
|
||||
const u = s.usage;
|
||||
return !!u && u.pull_count === 0 && u.surfaced_count >= 3;
|
||||
}
|
||||
|
||||
/** Short badge text, or "" to render nothing. A record nobody has surfaced yet
|
||||
* gets no badge at all: "0 / 0" would read as a verdict when it's an absence
|
||||
* of evidence. */
|
||||
function usageBadge(s: SnippetListItem): string {
|
||||
const u = s.usage;
|
||||
if (!u || u.surfaced_count === 0) return "";
|
||||
return `${u.pull_count}/${u.surfaced_count} used`;
|
||||
}
|
||||
|
||||
/** Short label for the drift verdict, or "" when there's nothing to say.
|
||||
* An expired verdict is reported as "unchecked" whatever it used to say —
|
||||
* it was about code that is no longer in the record. */
|
||||
@@ -254,22 +238,13 @@ function driftTitle(s: SnippetListItem): string {
|
||||
return v.detail ? `${when}: ${what}. ${v.detail}` : `${when}: ${what}.`;
|
||||
}
|
||||
|
||||
function usageTitle(s: SnippetListItem): string {
|
||||
const u = s.usage;
|
||||
if (!u) return "";
|
||||
const last = u.last_pulled_at
|
||||
? `Last opened ${new Date(u.last_pulled_at).toLocaleDateString()}.`
|
||||
: "Never opened.";
|
||||
const verdict = isDeadWeight(s)
|
||||
? " Offered repeatedly without ever being opened — consider rewriting its" +
|
||||
" “when to reach for it” so it says when, or deleting it. It takes a slot" +
|
||||
" in every future auto-inject menu."
|
||||
: "";
|
||||
return (
|
||||
`Surfaced to an agent ${u.surfaced_count}×, opened in full ` +
|
||||
`${u.pull_count}×. ${last}${verdict}`
|
||||
);
|
||||
}
|
||||
/** The dead-weight nudge for a SNIPPET, passed to the shared badge. Kept here
|
||||
* rather than inside the component because the remedy is kind-specific — a
|
||||
* rule in the same position gets different advice (milestone 333 step 5). */
|
||||
const SNIPPET_DEAD_WEIGHT =
|
||||
"Offered repeatedly without ever being opened — consider rewriting its " +
|
||||
"“when to reach for it” so it says when, or deleting it. It takes a slot " +
|
||||
"in every future auto-inject menu.";
|
||||
</script>
|
||||
|
||||
<template>
|
||||
@@ -456,14 +431,7 @@ function usageTitle(s: SnippetListItem): string {
|
||||
<span v-if="driftBadge(s)" class="drift-tag" :title="driftTitle(s)">
|
||||
{{ driftBadge(s) }}
|
||||
</span>
|
||||
<span
|
||||
v-if="usageBadge(s)"
|
||||
class="usage-tag"
|
||||
:class="{ 'usage-dead': isDeadWeight(s) }"
|
||||
:title="usageTitle(s)"
|
||||
>
|
||||
{{ usageBadge(s) }}
|
||||
</span>
|
||||
<UsageBadge :usage="s.usage" :dead-weight-advice="SNIPPET_DEAD_WEIGHT" />
|
||||
<span v-if="s.shared" class="shared-tag" :title="`Shared by ${s.owner ?? 'another user'} — a suggestion, not your own record`">
|
||||
by {{ s.owner ?? "another user" }}
|
||||
</span>
|
||||
@@ -710,7 +678,7 @@ function usageTitle(s: SnippetListItem): string {
|
||||
flex-shrink: 0;
|
||||
white-space: nowrap;
|
||||
background: color-mix(in srgb, var(--fs-accent) 15%, transparent);
|
||||
color: var(--fs-accent);
|
||||
color: var(--fs-accent-fg);
|
||||
}
|
||||
|
||||
.snippet-when {
|
||||
@@ -739,7 +707,7 @@ function usageTitle(s: SnippetListItem): string {
|
||||
border-radius: 4px;
|
||||
white-space: nowrap;
|
||||
background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent);
|
||||
color: var(--fs-text-tertiary);
|
||||
color: var(--fs-text-tertiary-fg);
|
||||
}
|
||||
|
||||
.dup-action {
|
||||
@@ -754,24 +722,7 @@ function usageTitle(s: SnippetListItem): string {
|
||||
border-radius: 4px;
|
||||
white-space: nowrap;
|
||||
background: color-mix(in srgb, var(--fs-error) 15%, transparent);
|
||||
color: var(--fs-error);
|
||||
}
|
||||
|
||||
.usage-tag {
|
||||
font-size: 0.7rem;
|
||||
padding: 0.1rem 0.4rem;
|
||||
border-radius: 4px;
|
||||
white-space: nowrap;
|
||||
font-variant-numeric: tabular-nums;
|
||||
background: color-mix(in srgb, var(--fs-text-tertiary) 15%, transparent);
|
||||
color: var(--fs-text-tertiary);
|
||||
}
|
||||
|
||||
/* Dead weight is a nudge, not an error — it warns in the warning colour rather
|
||||
than the danger one, because the record isn't broken, just unearned. */
|
||||
.usage-tag.usage-dead {
|
||||
background: color-mix(in srgb, var(--fs-warning) 18%, transparent);
|
||||
color: var(--fs-warning);
|
||||
color: var(--fs-error-fg);
|
||||
}
|
||||
|
||||
/* Header + select-mode */
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "scribe",
|
||||
"description": "Scribe system-of-record for Claude Code: MCP tools over your notes/tasks/projects/rules, a session-start push channel that surfaces your always-on rules + active-project context, process-skills (writing-plans, systematic-debugging, verification, brainstorming, reusing-code), and your saved Scribe Processes auto-surfaced as skills (/scribe:sync). Replaces superpowers + file-memory with one app-backed plugin.",
|
||||
"version": "0.1.47",
|
||||
"version": "2026.09.09.0408",
|
||||
"author": {
|
||||
"name": "Bryan Van Deusen"
|
||||
},
|
||||
|
||||
+7
-2
@@ -78,8 +78,13 @@ On install you'll be asked for:
|
||||
|
||||
## Notes
|
||||
|
||||
- Set a `version` bump in `.claude-plugin/plugin.json` per release so clients
|
||||
pick up changes.
|
||||
- **Do not hand-edit `version` in `.claude-plugin/plugin.json`.** It is minted
|
||||
from the clock — run `python3 scripts/mint_plugin_version.py` (or `make
|
||||
mint-plugin`, where `make` is installed) after changing anything under
|
||||
`plugin/`, and commit the result. The installer decides whether to refresh the cache it
|
||||
executes from by comparing that string, so content that ships without a new
|
||||
version reaches the repo and stops there (#2209). CI fails the lane if you
|
||||
forget.
|
||||
- The session-start, auto-inject and prior-art hooks need only a **read**-scoped
|
||||
key; the MCP tools need **write** scope to create/update. Every hook is a GET
|
||||
for that reason — a read key cannot POST.
|
||||
|
||||
@@ -33,6 +33,15 @@
|
||||
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_prior_art.sh\""
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": "Bash",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_tool_rules.sh\""
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"PostToolUse": [
|
||||
|
||||
@@ -175,6 +175,16 @@ while IFS= read -r rel_path; do
|
||||
derive_seen=$(tr '\n' ',' < "$derivefile" 2>/dev/null | sed 's/,$//' | jq -sRr '@uri' 2>/dev/null) || derive_seen=""
|
||||
[ -n "$derive_seen" ] && derive_exclude_q="&exclude_derive=${derive_seen}"
|
||||
fi
|
||||
# The rules marker the SessionStart hook stored, handed back so the server
|
||||
# can say whether those rules moved since (milestone 323). Nothing stored
|
||||
# means nothing sent, which the server reads as silence rather than as a
|
||||
# mismatch — an install that never reached /api/plugin/context must not
|
||||
# start claiming its rules changed.
|
||||
etag_q=""
|
||||
if [ -f "$state_dir/${safe_sid}.rules_etag" ]; then
|
||||
held=$(jq -sRr '@uri' < "$state_dir/${safe_sid}.rules_etag" 2>/dev/null) || held=""
|
||||
[ -n "$held" ] && etag_q="&rules_etag=${held}"
|
||||
fi
|
||||
if [ -n "$path_enc" ]; then
|
||||
# 8s, not the pre-write hook's 5: this hook runs AFTER the tool, so it
|
||||
# gates nothing the session is waiting on, and the first prior-art call
|
||||
@@ -184,7 +194,7 @@ while IFS= read -r rel_path; do
|
||||
reached=1
|
||||
body=$(curl -fsS --max-time 8 \
|
||||
-H "Authorization: Bearer ${token}" \
|
||||
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${derive_exclude_q}${shapes_q}" 2>/dev/null) || { body=""; reached=0; }
|
||||
"${url%/}/api/plugin/prior-art?path=${path_enc}&code=${code_enc}${repo_q}${exclude_q}${sync_exclude_q}${derive_exclude_q}${shapes_q}${etag_q}" 2>/dev/null) || { body=""; reached=0; }
|
||||
# A call that was owed and didn't come back is said, once per outage
|
||||
# (#2932) — shared marker with the pre-write hook, so one outage is one
|
||||
# line however the code was written.
|
||||
|
||||
@@ -55,6 +55,53 @@ here=$(CDPATH= cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) || exit 0
|
||||
event=$(cat 2>/dev/null || true)
|
||||
source=$(printf '%s' "$event" | jq -r '.source // empty' 2>/dev/null) || source=""
|
||||
|
||||
# --- The rule ledger outlives the context it describes (#3749) ---
|
||||
#
|
||||
# scribe_prior_art.sh and scribe_tool_rules.sh record every rule id they have
|
||||
# named in <state>/<sid>.rules.ids and hand it back as exclude_rule_ids, so a
|
||||
# rule is named once per session and then goes quiet. That is right while the
|
||||
# session still HOLDS what it was told, and wrong the moment it does not.
|
||||
#
|
||||
# A compaction summarizes the earlier injections away and does not touch the
|
||||
# filesystem, so the rule ends up absent from context AND still excluded —
|
||||
# unreachable for the rest of the session. The banner below tells the model to
|
||||
# re-pull its ALWAYS-ON rules, but a rule an arm surfaced is conditional and is
|
||||
# not in that set, so it has no other way back. The rules most likely to be in
|
||||
# this state are the ones that fire most often, which is to say the ones that
|
||||
# apply most.
|
||||
#
|
||||
# The session id survives a compaction — the etag marker further down is
|
||||
# rewritten on `compact` and keyed by session_id, which is only meaningful if
|
||||
# the id is stable — so the stale ledger is genuinely found again, not orphaned.
|
||||
#
|
||||
# CLEARED ON THE SOURCES THAT DESTROY CONTEXT, AND ONLY THOSE:
|
||||
#
|
||||
# compact CLEAR — summarized away; the file survived.
|
||||
# clear CLEAR — context wiped.
|
||||
# startup nothing to do: a new session id means a new, empty file.
|
||||
# resume KEEP. The context was genuinely restored, so the ledger still
|
||||
# describes what the session holds. Clearing here would re-surface
|
||||
# every rule after a restore that lost nothing — the mirror error.
|
||||
# fork KEEP, and the answer is the same whichever way forks are keyed: a
|
||||
# fork carries the conversation, so if it inherits the id the ledger
|
||||
# is accurate, and if it gets a new one the file is empty anyway.
|
||||
#
|
||||
# ONLY the rules ledger. The same directory holds .ids / .sync.ids /
|
||||
# .derive.ids for the note arms. Whether a surfaced NOTE should return after a
|
||||
# compaction is a different question with a different answer, and leaving those
|
||||
# alone is a decision rather than an oversight.
|
||||
case "$source" in
|
||||
compact|clear)
|
||||
sid=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || sid=""
|
||||
if [ -n "$sid" ]; then
|
||||
safe_sid=$(printf '%s' "$sid" | tr -c 'A-Za-z0-9._-' '_')
|
||||
# Best-effort, like every other filesystem touch in these hooks: a ledger
|
||||
# that cannot be removed costs a repeated exclusion, never a session.
|
||||
rm -f "${TMPDIR:-/tmp}/scribe-priorart/${safe_sid}.rules.ids" 2>/dev/null || true
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
out=""
|
||||
# Append $1 to $out, separated by a horizontal rule when $out already has content.
|
||||
append() { if [ -n "$out" ]; then out="${out}"$'\n\n---\n\n'"$1"; else out="$1"; fi; }
|
||||
@@ -109,6 +156,28 @@ if [ -n "$url" ] && [ -n "$token" ] && command -v curl >/dev/null 2>&1; then
|
||||
-H "Authorization: Bearer ${token}" \
|
||||
"${url%/}/api/plugin/context${q}" 2>/dev/null) || body=""
|
||||
[ -n "$body" ] && dyn=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null)
|
||||
# Stash the rules marker for the write-path hook (milestone 323). THIS is
|
||||
# where it has to be captured: the model receives one from
|
||||
# list_always_on_rules too, but a hook cannot see an MCP tool's result. Stored
|
||||
# under the same state dir the prior-art hook already uses, keyed by session,
|
||||
# so "changed since" means since THIS session loaded its rules.
|
||||
#
|
||||
# Written on `compact` as well as `startup`, and that is correct rather than
|
||||
# convenient: a compact tells the session to re-pull its rules, so the marker
|
||||
# should describe the set it is about to hold. It is also why this cannot
|
||||
# cover the compaction case — see the table in services/plugin_context.py.
|
||||
if [ -n "$body" ]; then
|
||||
etag=$(printf '%s' "$body" | jq -r '.rules_etag // empty' 2>/dev/null) || etag=""
|
||||
if [ -n "$etag" ]; then
|
||||
sid=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || sid=""
|
||||
safe_sid=$(printf '%s' "${sid:-nosession}" | tr -c 'A-Za-z0-9._-' '_')
|
||||
etag_dir="${TMPDIR:-/tmp}/scribe-priorart"
|
||||
# Best-effort throughout: a marker that cannot be stored costs a hint,
|
||||
# never the session.
|
||||
mkdir -p "$etag_dir" 2>/dev/null \
|
||||
&& printf '%s' "$etag" > "$etag_dir/${safe_sid}.rules_etag" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
[ -z "$dyn" ] && status="> ⚠️ Scribe: live rules/project context could not be loaded this session (instance unreachable or request failed). The standing guidance above still applies — pull rules with \`list_always_on_rules()\` and project context with \`enter_project()\` as needed."
|
||||
elif [ -n "$url" ] && [ -z "$token" ]; then
|
||||
status="> ⚠️ Scribe: live context disabled this session — the API key is not configured (Scribe base URL is). Set it with \`/plugin\` → Scribe → configure, or export SCRIBE_TOKEN. Tools still work; pull rules with \`list_always_on_rules()\` and project context with \`enter_project()\`."
|
||||
|
||||
@@ -21,6 +21,16 @@ for the operator's work, and as your own working memory across sessions.
|
||||
compaction — call `list_always_on_rules()` (and `enter_project()` when a
|
||||
project is in scope) BEFORE acting. When a loaded rule and a default habit
|
||||
disagree, the rule wins; if no rule speaks to it, ask rather than assume.
|
||||
- **What you loaded is not all of the rules.** Only the always-on tier arrives
|
||||
that way; conditional rules are RETRIEVED, and one you were never handed
|
||||
binds exactly as hard. So before a consequential act, `search` for a rule
|
||||
about it (`content_type="rule"`) rather than concluding from an empty
|
||||
loaded set that nothing applies. "I was not told" is not the same as "there
|
||||
is no rule," and only one of those is checkable.
|
||||
This bites hardest on which TOOL to reach for — curling an API that has an
|
||||
MCP client, standing up a local stack, running a suite CI owns. Those feel
|
||||
like mechanics rather than decisions, so they raise no doubt and generate no
|
||||
query; the moment you are most confident is the moment to look.
|
||||
- **Recall before acting** — before you answer anything about the operator's
|
||||
work or start a task, `search` Scribe first; assume a related note, task, or
|
||||
decision already exists. Concretely, reach for recall whenever a request
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
#!/usr/bin/env bash
|
||||
# Scribe — PreToolUse rule arm for ACTIONS (#3476).
|
||||
#
|
||||
# The sibling of scribe_prior_art.sh. That hook is registered on Write|Edit and
|
||||
# asks "what is recorded about the file being written". This one asks "does a
|
||||
# standing rule speak to the command about to be run" — the question nothing
|
||||
# could ask before, and the reason every rule about which tool to reach for had
|
||||
# to live in the always-on preload instead.
|
||||
#
|
||||
# WHY A HOOK AND NOT AN INSTRUCTION. A reflex generates no query (note #3089):
|
||||
# you reach for `curl` confidently, with no moment of doubt, so a surface that
|
||||
# waits to be asked never fires. Here nothing is asked — the tool call IS the
|
||||
# query, and the reflex has to become a tool call before it can do anything.
|
||||
#
|
||||
# SILENT ON OUTAGE, deliberately, unlike the prior-art hook. A write is
|
||||
# occasional; a Bash call is not, and an "instance did not answer" line before
|
||||
# every command is the noise that gets a channel muted. scribe_prior_art.sh
|
||||
# still speaks for both when the instance is down.
|
||||
#
|
||||
# Env:
|
||||
# SCRIBE_URL / SCRIBE_TOKEN override for the settings.json dogfooding path.
|
||||
|
||||
command -v jq >/dev/null 2>&1 || exit 0
|
||||
command -v curl >/dev/null 2>&1 || exit 0
|
||||
|
||||
# PreToolUse delivers { session_id, cwd, tool_name, tool_input: {...}, ... }
|
||||
event=$(cat 2>/dev/null || true)
|
||||
tool_name=$(printf '%s' "$event" | jq -r '.tool_name // empty' 2>/dev/null) || exit 0
|
||||
session_id=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || session_id=""
|
||||
event_cwd=$(printf '%s' "$event" | jq -r '.cwd // empty' 2>/dev/null) || event_cwd=""
|
||||
|
||||
[ -n "$tool_name" ] || exit 0
|
||||
|
||||
# The action, as text. `.command` is Bash's field; the fallbacks let the matcher
|
||||
# in hooks.json widen to other tools without this script changing — which is the
|
||||
# whole reason the server side takes a name and a string rather than a schema.
|
||||
command_text=$(printf '%s' "$event" | jq -r '
|
||||
.tool_input.command //
|
||||
.tool_input.url //
|
||||
.tool_input.prompt //
|
||||
empty' 2>/dev/null) || command_text=""
|
||||
|
||||
[ -n "$command_text" ] || exit 0
|
||||
|
||||
# shellcheck source=plugin/hooks/scribe_defs.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
|
||||
|
||||
# scribe_config, not a hand-rolled pair of parameter expansions: it also treats
|
||||
# an UNEXPANDED `${...}` placeholder as unset, which would otherwise be sent as
|
||||
# a garbage Bearer token and 401 on every call (#2198's class).
|
||||
scribe_config || exit 0
|
||||
|
||||
# Bounded before encoding: a heredoc or a pasted script can be enormous, and
|
||||
# the verb and its target — the part a rule is about — sit at the front. The
|
||||
# server bounds it again; this keeps a huge payload off the wire in the first
|
||||
# place. `head -c`, never `cut -c`: cut truncates each LINE and caps nothing.
|
||||
command_text=$(printf '%s' "$command_text" | head -c 2000)
|
||||
|
||||
# -sRr, never -rR: jq -R without -s reads LINE BY LINE, so a multi-line command
|
||||
# would encode per line and join with raw newlines — an invalid URL.
|
||||
cmd_enc=$(printf '%s' "$command_text" | jq -sRr '@uri' 2>/dev/null) || exit 0
|
||||
tool_enc=$(printf '%s' "$tool_name" | jq -sRr '@uri' 2>/dev/null) || exit 0
|
||||
|
||||
repo_q=""
|
||||
lookup_dir=${event_cwd:-${CLAUDE_PROJECT_DIR:-$PWD}}
|
||||
repo_remote=$(git -C "$lookup_dir" remote get-url origin 2>/dev/null || true)
|
||||
if [ -n "$repo_remote" ]; then
|
||||
repo_enc=$(printf '%s' "$repo_remote" | jq -sRr '@uri' 2>/dev/null) || repo_enc=""
|
||||
[ -n "$repo_enc" ] && repo_q="&repo=${repo_enc}"
|
||||
fi
|
||||
|
||||
# THE SHARED SESSION LEDGER, and the thing most worth getting right here.
|
||||
#
|
||||
# scribe_prior_art.sh keeps the rules it has already named in
|
||||
# <state>/<sid>.rules.ids and passes them as exclude_rule_ids. This hook reads
|
||||
# and appends to that SAME file rather than keeping its own: two ledgers would
|
||||
# mean a rule named by one arm gets re-offered by the other, and the hint that
|
||||
# fires most often is exactly the one that must not repeat itself.
|
||||
#
|
||||
# The directory keeps the prior-art name on purpose — renaming it would orphan
|
||||
# every live session's state for a cosmetic gain.
|
||||
state_dir="${TMPDIR:-/tmp}/scribe-priorart"
|
||||
mkdir -p "$state_dir" 2>/dev/null || true
|
||||
rulefile=""
|
||||
rule_exclude_q=""
|
||||
if [ -n "$session_id" ]; then
|
||||
safe_sid=$(printf '%s' "$session_id" | tr -c 'A-Za-z0-9._-' '_')
|
||||
rulefile="$state_dir/${safe_sid}.rules.ids"
|
||||
if [ -f "$rulefile" ]; then
|
||||
rule_seen=$(tr '\n' ',' < "$rulefile" 2>/dev/null | sed 's/,$//')
|
||||
[ -n "$rule_seen" ] && rule_exclude_q="&exclude_rule_ids=${rule_seen}"
|
||||
fi
|
||||
fi
|
||||
|
||||
# `|| exit 0` here, unlike the prior-art hook: there is no local arm whose
|
||||
# finding would be discarded, and an outage line before every command is worse
|
||||
# than silence. See the header.
|
||||
body=$(curl -fsS --max-time 5 \
|
||||
-H "Authorization: Bearer ${token}" \
|
||||
"${url%/}/api/plugin/tool-rules?tool=${tool_enc}&command=${cmd_enc}${repo_q}${rule_exclude_q}" 2>/dev/null) || exit 0
|
||||
|
||||
context=$(printf '%s' "$body" | jq -r '.context // empty' 2>/dev/null) || exit 0
|
||||
[ -n "$context" ] || exit 0
|
||||
|
||||
# Remember what was named so it is not repeated this session.
|
||||
if [ -n "$rulefile" ]; then
|
||||
printf '%s' "$body" | jq -r '(.rule_ids // [])[]?' 2>/dev/null >> "$rulefile" || true
|
||||
fi
|
||||
|
||||
jq -cn --arg ctx "$context" '{
|
||||
hookSpecificOutput: {
|
||||
hookEventName: "PreToolUse",
|
||||
additionalContext: $ctx
|
||||
}
|
||||
}' 2>/dev/null || true
|
||||
exit 0
|
||||
@@ -56,10 +56,31 @@ Two constraints on *how* that's achieved:
|
||||
re-deriving it or opening a duplicate. When a project is in scope, pass its
|
||||
`project_id` so results stay scoped.
|
||||
|
||||
2. **Standing rules are binding.** Load them via `list_always_on_rules()` at
|
||||
session start (see "Do this first"); treat every one as binding. Pull a
|
||||
rule's full statement with `get_rule(id)` when it's about to bite. When a
|
||||
project is in scope, `enter_project(id)` also returns its applicable rules.
|
||||
2. **Standing rules are binding — and the ones you were handed are not all of
|
||||
them.** Load the resident set via `list_always_on_rules()` at session start
|
||||
(see "Do this first"); treat every one as binding. Pull a rule's full
|
||||
statement with `get_rule(id)` when it's about to bite. When a project is in
|
||||
scope, `enter_project(id)` also returns its applicable rules.
|
||||
|
||||
Rules come in two tiers. **Always-on** rules are delivered — they arrive
|
||||
whether or not you ask. **Conditional** rules are RETRIEVED, and one binds
|
||||
just as hard for never having been handed to you. So before a consequential
|
||||
act, `search(content_type="rule")` on what you are about to do. An empty
|
||||
loaded set is not evidence that no rule applies; it is only evidence that
|
||||
none was pushed, and those are different claims.
|
||||
|
||||
The tier split exists because delivery does not scale: every resident rule
|
||||
costs tokens in every session forever, so a rulebook that grows past a few
|
||||
dozen either stops growing or stops fitting. Retrieval is what lets the
|
||||
rulebook keep growing — but retrieval only fires if something asks.
|
||||
|
||||
**Ask hardest where you feel most certain.** Rules about which TOOL to reach
|
||||
for — use the forge's MCP client rather than curling its API, don't stand up
|
||||
a local stack, don't run the suite CI owns — govern moves that feel like
|
||||
mechanics rather than decisions. A reflex raises no doubt, so it generates
|
||||
no query, so the rule that would have stopped it is never retrieved. That is
|
||||
the failure this instruction exists to prevent, and confidence is its only
|
||||
warning sign.
|
||||
|
||||
3. **Update over duplicate.** When recording, prefer updating an existing
|
||||
note/rule/task over creating a new one. Search first; revise what's there.
|
||||
@@ -96,7 +117,25 @@ Two constraints on *how* that's achieved:
|
||||
not restraint. Only a record genuinely about no particular area goes
|
||||
untagged.
|
||||
|
||||
8. **State updates in place; chronicles don't.** A dev-log records what
|
||||
8. **Name the record, never just its number.** Whenever you refer to a Scribe
|
||||
record — in a message to the operator, a commit message, a task body, a
|
||||
work-log — write the id *and* its title: `#3244 "the staleness signal"`,
|
||||
`milestone 323 "rule versioning"`. Not `#3244`.
|
||||
|
||||
You have the record open; the operator does not. A bare id reads as
|
||||
complete to you and as homework to them — they have to look it up to know
|
||||
what their own conversation is about, or guess. Scribe's own duplicate gate
|
||||
already writes `id 412: "debounce helper"` for exactly this reason; match
|
||||
it everywhere else.
|
||||
|
||||
The first mention in a message carries the title; later mentions of the
|
||||
same record can use the bare id. If you don't know the title, look it up
|
||||
before citing the number — an id you can't name is one you haven't checked.
|
||||
This matters most in the places read later by someone with even less
|
||||
context than the operator has now: commit messages, task bodies, and any
|
||||
record that cites another.
|
||||
|
||||
9. **State updates in place; chronicles don't.** A dev-log records what
|
||||
*happened* — write it once, never rewrite it. A durable finding (how a
|
||||
subsystem works, a measured number) lives in that System's **reference
|
||||
note** ("«System» — reference"), which you UPDATE as facts change — safe,
|
||||
@@ -106,6 +145,48 @@ Two constraints on *how* that's achieved:
|
||||
re-measurement, a reversed decision), pass the old id in `supersedes` so the
|
||||
stale record is demoted and labelled rather than left competing.
|
||||
|
||||
10. **A few notes assert a FACT, and those can carry their own check.**
|
||||
Supersession only fires once somebody has read a note and disagreed — which
|
||||
is the case where it was already believed. A note asserting something about
|
||||
*someone else's* software — what a service does on a duplicate upload, how a
|
||||
forge numbers its CI runs, what an updater compares — can instead carry
|
||||
`verify_with` (how to check it) and `expires_when` (the STATE that ends it:
|
||||
"when the forge numbers runs per workflow", never "in six months").
|
||||
`notes_due_for_verification` lists them least-recently-confirmed first, with
|
||||
never-checked at the top; `mark_note_verified` records what you found, and
|
||||
`still_true=False` deliberately writes nothing — a note whose check failed
|
||||
is wrong rather than in a state worth recording, so it keeps its place.
|
||||
|
||||
**The test is one question: could this note become false without anyone
|
||||
editing it?** If no, leave both fields empty. That is the normal case, and
|
||||
an empty `verify_with` is the positive marker for "this is a decision, there
|
||||
is nothing to go and check" — not an unfinished record. The sweep is only
|
||||
worth reading while almost nothing is on it, so a check added out of
|
||||
tidiness costs the whole surface, not just that note.
|
||||
|
||||
**The sharper form of the same test: is the thing this note describes yours
|
||||
to change?** If yes it is a decision — editing your own software is how it
|
||||
changes, and you will know you did it. Measured against a real corpus, every
|
||||
note that earned a check was about somebody ELSE's software: a signing
|
||||
service, a forge, a hub, an SDK, a model, a dependency set.
|
||||
|
||||
**Three that look like candidates and are not:**
|
||||
- **Resume pointers and "current state" notes.** They go stale fastest of
|
||||
anything, which is exactly why they tempt — but the cure is to update or
|
||||
delete them, not to schedule a check. A sweep full of pointers is a sweep
|
||||
nobody reads.
|
||||
- **Measurements of your own system.** They go false because you changed
|
||||
something, and you knew. A measurement earns a check only when what it
|
||||
measures is outside your control.
|
||||
- **A decision that RESTS on somebody else's behaviour.** The decision is
|
||||
still a decision. Put the check on the note asserting the fact, and link
|
||||
the decision to it.
|
||||
|
||||
Not for tasks — a task's decay is its status, and a done issue records what
|
||||
happened rather than asserting something that can go false. Not for snippets
|
||||
either: `verify_snippet` compares the recorded location and code against the
|
||||
repo, which is richer and already wired to drift detection.
|
||||
|
||||
## Stay inside the active project's scope
|
||||
|
||||
Once a project is in scope — you called `enter_project`, or the working repo is
|
||||
|
||||
@@ -1,17 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bump the patch segment of fable-mcp/pyproject.toml version and stage the file.
|
||||
# Usage: called automatically by the Claude Code pre-commit hook, or manually.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
FILE="$REPO_ROOT/fable-mcp/pyproject.toml"
|
||||
|
||||
current=$(grep '^version = ' "$FILE" | sed 's/version = "\(.*\)"/\1/')
|
||||
major=$(echo "$current" | cut -d. -f1)
|
||||
minor=$(echo "$current" | cut -d. -f2)
|
||||
patch=$(echo "$current" | cut -d. -f3)
|
||||
new_version="$major.$minor.$((patch + 1))"
|
||||
|
||||
sed -i "s/^version = \"$current\"/version = \"$new_version\"/" "$FILE"
|
||||
git -C "$REPO_ROOT" add "$FILE"
|
||||
echo "fable-mcp: $current → $new_version"
|
||||
@@ -119,15 +119,20 @@ def same_hue_text_on_tint(css: str) -> tuple[list[str], list[str]]:
|
||||
dangerous of the two — it does not even name a `-bg` token, so nothing
|
||||
about it looks like the pattern until you measure it.
|
||||
|
||||
Returned separately because they are at different stages. The token form
|
||||
is CLEAN and therefore gates. The inline form has a live backlog (48 sites
|
||||
when this split was written, 26 of them --fs-accent), so it reports with a
|
||||
count: a gate nobody can satisfy today gets switched off, and then it
|
||||
guards nothing.
|
||||
Returned separately because they were paid down separately — the token
|
||||
form first (7 badge pairs), then the inline form (46 sites across 18
|
||||
files, 26 of them --fs-accent). Both are clean now, so BOTH gate. The
|
||||
split is kept because the two spellings need different error text: one
|
||||
names a -bg token you can search for, the other names nothing at all.
|
||||
"""
|
||||
token_form, inline_form = [], []
|
||||
for body in re.findall(r"\{([^{}]*)\}", css):
|
||||
fg = set(re.findall(r"color\s*:\s*var\(\s*(--fs-[\w-]+?)\s*\)", body))
|
||||
# (?<![-\w]) or `border-color`, `border-left-color` and `outline-color`
|
||||
# all match as if they were text. They are not: a border is a non-text
|
||||
# graphic and its floor is 3:1, not 4.5. Without this the check reported
|
||||
# seven rules that were already correct — and a check that cries wolf on
|
||||
# correct code is one that gets muted.
|
||||
fg = set(re.findall(r"(?<![-\w])color\s*:\s*var\(\s*(--fs-[\w-]+?)\s*\)", body))
|
||||
bg_tok = set(re.findall(r"background(?:-color)?\s*:\s*var\(\s*(--fs-[\w-]+?)-bg\s*\)", body))
|
||||
bg_inl = set(re.findall(
|
||||
r"background(?:-color)?\s*:\s*color-mix\([^;]*?var\(\s*(--fs-[\w-]+?)\s*\)[^;]*?\)",
|
||||
@@ -232,21 +237,23 @@ def main() -> int:
|
||||
print("OK — no text painted with a token on a tint of its own -bg.\n")
|
||||
|
||||
if inline_tint_hits:
|
||||
by_tok: dict[str, int] = {}
|
||||
for _p, tok in inline_tint_hits:
|
||||
by_tok[tok] = by_tok.get(tok, 0) + 1
|
||||
print(f"REPORT — {len(inline_tint_hits)} rule(s) paint text with a token on "
|
||||
f"an INLINE color-mix tint of that same token.")
|
||||
print(" Same defect, spelled without a -bg token so it does not gate yet.")
|
||||
print(" Worst offenders: " + ", ".join(
|
||||
f"{t} x{n}" for t, n in sorted(by_tok.items(), key=lambda kv: -kv[1])[:4]) + "\n")
|
||||
print(f"FAIL — {len(inline_tint_hits)} rule(s) paint text with a token on an "
|
||||
f"INLINE color-mix tint of that same token.")
|
||||
print(" Identical defect to the block above, spelled without a -bg token —")
|
||||
print(" which is what let it hide: nothing about it LOOKS like the pattern.")
|
||||
print(" Use the token's -fg sibling.\n")
|
||||
for path, tok in inline_tint_hits:
|
||||
print(f" {path}: color: var({tok}) on an inline tint -> var({tok}-fg)")
|
||||
print()
|
||||
else:
|
||||
print("OK — no text painted with a token on an inline tint of itself.\n")
|
||||
|
||||
if args.report_literals:
|
||||
print(f"REPORT — {literal_count} raw colour literal(s) in component CSS.")
|
||||
print(" Advisory: a literal is a value stated outside the system, so it "
|
||||
"cannot follow a palette change.\n")
|
||||
|
||||
return 1 if (unresolved or same_hue_hits) else 0
|
||||
return 1 if (unresolved or same_hue_hits or inline_tint_hits) else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+278
-45
@@ -13,9 +13,20 @@ separate defects have reached a live install through that path:
|
||||
install, because `plugin.json`'s version wasn't bumped and the installer
|
||||
compares versions to decide whether to refresh its cache.
|
||||
|
||||
The rule for the second one was already written down and was still missed. A
|
||||
written rule that depends on being remembered during a long session is not a
|
||||
control; this is.
|
||||
Both were fixed. The second was fixed TWICE — once by bumping the number, and
|
||||
then properly, by removing the class it came from: `plugin.json`'s version is
|
||||
no longer a value anybody chooses. `scripts/mint_plugin_version.py` derives it
|
||||
from the clock (`make mint-plugin`), and `check_version_is_minted` below fails
|
||||
the lane when shipped content moved and the version did not.
|
||||
|
||||
State exactly what that did and did not remove, because a rationale that
|
||||
overstates its own control is how the control gets trusted past its limit, and
|
||||
because the paragraph this replaces was itself read that way. Gone: having to
|
||||
remember which NUMBER to write, and the whole question of whether a chosen
|
||||
number was the right one. Not gone: the mint still has to be RUN, and
|
||||
forgetting to run it is still possible. What changed is that forgetting is now
|
||||
LOUD — a red lane on the batch that forgot, instead of a silent no-op found
|
||||
weeks later when somebody says "I don't think it updated" (#2220).
|
||||
|
||||
shellcheck and jq are NOT in `ci-python` (verified against CI-runner's Dockerfile
|
||||
and scripts/install-common.sh, not from memory — rule #37). CI installs both
|
||||
@@ -31,8 +42,15 @@ itself loudly, because a check that quietly no-ops is the failure mode this
|
||||
whole file exists to prevent.
|
||||
|
||||
Usage:
|
||||
python3 scripts/check_plugin.py # all checks
|
||||
python3 scripts/check_plugin.py --no-version # skip the bump check
|
||||
python3 scripts/check_plugin.py # all checks
|
||||
python3 scripts/check_plugin.py --no-version # on `main` only — see below
|
||||
|
||||
`--no-version` exists for ONE case. The version is measured against
|
||||
`origin/main`, so on `main` itself the comparison is against itself and answers
|
||||
nothing; the syntax, pattern and marker checks are the only ones that mean
|
||||
anything there. It is NOT a way past a red lane — see
|
||||
`check_version_is_minted`, whose whole design is shaped by keeping this flag
|
||||
out of anyone's muscle memory.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -43,16 +61,90 @@ import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
# The shape contract is ONE definition, shared with the script that mints it —
|
||||
# a checker carrying its own copy of the format would drift from the minter
|
||||
# and pass values the minter can no longer produce. Explicit path insert
|
||||
# because this file runs both as `python3 scripts/check_plugin.py` (which puts
|
||||
# `scripts/` on the path, not the root) and as an import from the test suite.
|
||||
sys.path.insert(0, str(ROOT))
|
||||
from scripts.mint_plugin_version import VERSION_RE # noqa: E402
|
||||
|
||||
PLUGIN_DIR = ROOT / "plugin"
|
||||
HOOKS_DIR = PLUGIN_DIR / "hooks"
|
||||
MANIFEST = PLUGIN_DIR / ".claude-plugin" / "plugin.json"
|
||||
|
||||
# Paths whose contents reach an install. Keep in step with the workflow's
|
||||
# `paths:` filter — a path that ships but isn't checked here is the gap again.
|
||||
SHIPPED = ("plugin", ".claude-plugin")
|
||||
# ── What ships, and what decides what it says about itself ─────────────────
|
||||
#
|
||||
# ONE definition (#3127 §3, milestone 334 step 2). It has TWO consumers that
|
||||
# need different granularities, and conflating them is the bug:
|
||||
#
|
||||
# the workflow's `paths:` trigger whole paths should CI run at all?
|
||||
# the version check paths MINUS should the version
|
||||
# the manifest have moved?
|
||||
# `version`
|
||||
#
|
||||
# The second one is why this is not just a tuple of paths. `plugin.json` lives
|
||||
# INSIDE `plugin/`, so a version bump is itself a change to the shipped set —
|
||||
# and a check that reads the set naively then treats the bump as its own
|
||||
# justification. Any bump passes, no bump fails, and it has proved nothing.
|
||||
# `shipped_content_changed` below is the exclusion-aware reader.
|
||||
#
|
||||
# The exclusion is that ONE FIELD, never the whole file: `plugin.json` also
|
||||
# carries description, mcpServers and userConfig, all of which reach an
|
||||
# install and all of which matter. Excluding the file wholesale would mean a
|
||||
# userConfig-only edit computes an unchanged version and never refreshes —
|
||||
# #2209 again with a narrower trigger.
|
||||
SHIPPED_PATHS = ("plugin", ".claude-plugin")
|
||||
|
||||
# Files that decide what a published artifact SAYS ABOUT ITSELF — kept as a
|
||||
# table so the next artifact is a one-line addition rather than a third
|
||||
# bespoke guard (#3127 §3). The membership test is NOT "is this copied into
|
||||
# the artifact?" but "can changing this file change the published bytes, or
|
||||
# what the artifact says about itself?" — FC learned that twice in four days
|
||||
# (#3156, #3202), and a deriver is never in the COPY list.
|
||||
#
|
||||
# Note what is absent: a CHECKER does not belong here. Whatever validates a
|
||||
# version decides whether the lane goes red, not what any artifact reports,
|
||||
# so `check_plugin.py` itself is not a deriver, while the script that mints
|
||||
# the plugin version is.
|
||||
DERIVERS: dict[str, tuple[str, ...]] = {
|
||||
# The "Generate image tags and version" step computes the server image's
|
||||
# name, ordering key and channel (#3298).
|
||||
".forgejo/workflows/ci.yml": ("server-image",),
|
||||
# Decides the plugin's version FORMAT, so it decides what every future
|
||||
# manifest says about itself (milestone 334 step 3).
|
||||
"scripts/mint_plugin_version.py": ("plugin",),
|
||||
}
|
||||
|
||||
|
||||
def version_relevant_paths() -> tuple[str, ...]:
|
||||
"""Everything a change to which must produce a NEW plugin version.
|
||||
|
||||
Wider than `SHIPPED_PATHS`, and #3127 §3's asymmetry is why it has to be:
|
||||
|
||||
A change to how the VERSION is computed is compared against nothing at
|
||||
all. Left out, the published artifact goes on reporting the OLD value
|
||||
indefinitely.
|
||||
|
||||
Concretely — change the mint script's format string, change nothing else,
|
||||
and a diff over the shipped paths alone reports "no content change, the
|
||||
version need not move". The manifest then keeps a value in the old format
|
||||
forever and nothing ever says so. The mint script reaches no install and
|
||||
belongs here anyway; that is #3156's exact shape.
|
||||
|
||||
A CHECKER is deliberately not here. Whatever validates the version decides
|
||||
whether the lane goes red, not what any artifact reports — so this file is
|
||||
absent from its own set, and that is not an oversight.
|
||||
"""
|
||||
return SHIPPED_PATHS + tuple(
|
||||
path for path, artifacts in DERIVERS.items() if "plugin" in artifacts
|
||||
)
|
||||
|
||||
|
||||
failures: list[str] = []
|
||||
|
||||
@@ -143,8 +235,6 @@ def check_patterns() -> None:
|
||||
ok(f"{rel}: no known-bad patterns")
|
||||
|
||||
|
||||
# --- the version bump ------------------------------------------------------
|
||||
|
||||
# --- shellcheck ------------------------------------------------------------
|
||||
|
||||
def check_shellcheck() -> None:
|
||||
@@ -215,6 +305,16 @@ SMOKE_EVENTS: dict[str, str] = {
|
||||
"tool_input": {"file_path": "src/x.py",
|
||||
"new_string": f"def {_ABSENT_SYM}():\n pass\n"}}
|
||||
),
|
||||
# The pre-tool rule arm (#3476). A real Bash call, and one whose whole
|
||||
# point is that it looks harmless: reaching for curl against the forge API
|
||||
# is the reflex the arm exists to catch. With no instance it must stay
|
||||
# SILENT — it is deliberately not an OUTAGE_SPEAKER, because a Bash call is
|
||||
# not occasional and an outage line before every command gets the channel
|
||||
# muted.
|
||||
"scribe_tool_rules.sh": json.dumps(
|
||||
{"session_id": "smoke", "cwd": ".", "tool_name": "Bash",
|
||||
"tool_input": {"command": "curl -s https://example.invalid/api/v1/runs"}}
|
||||
),
|
||||
"scribe_sync_processes.sh": json.dumps({"source": "startup"}),
|
||||
"scribe_session_context.sh": json.dumps({"source": "startup"}),
|
||||
# The after-write hook (#2901) diffs the working tree; on CI's clean
|
||||
@@ -391,80 +491,213 @@ def _git(*args: str) -> tuple[int, str]:
|
||||
return proc.returncode, (proc.stdout or proc.stderr).strip()
|
||||
|
||||
|
||||
def manifest_version(ref: str | None = None) -> str | None:
|
||||
"""The manifest version at `ref`, or in the working tree when ref is None."""
|
||||
def manifest_text(ref: str | None = None) -> str | None:
|
||||
"""The manifest's RAW TEXT at `ref`, or in the working tree when ref is None.
|
||||
|
||||
Split out from `manifest_version` because the exclusion below needs every
|
||||
field except one, not the one field.
|
||||
"""
|
||||
if ref is None:
|
||||
try:
|
||||
return json.loads(MANIFEST.read_text()).get("version")
|
||||
except Exception:
|
||||
return MANIFEST.read_text()
|
||||
except OSError:
|
||||
return None
|
||||
rel = MANIFEST.relative_to(ROOT).as_posix()
|
||||
code, out = _git("show", f"{ref}:{rel}")
|
||||
if code != 0:
|
||||
return out if code == 0 else None
|
||||
|
||||
|
||||
def manifest_version(ref: str | None = None) -> str | None:
|
||||
"""The manifest version at `ref`, or in the working tree when ref is None."""
|
||||
text = manifest_text(ref)
|
||||
if text is None:
|
||||
return None
|
||||
try:
|
||||
return json.loads(out).get("version")
|
||||
return json.loads(text).get("version")
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def check_version_bump(base: str = "origin/main") -> None:
|
||||
"""If shipped plugin content differs from `base`, the version must too.
|
||||
# Distinct from None, which is a legitimate "this manifest does not exist".
|
||||
_UNREADABLE = object()
|
||||
|
||||
Stated against the BASE BRANCH rather than the last commit on purpose. A
|
||||
per-commit rule would demand a bump from every commit in a batch; what
|
||||
actually matters is that whatever reaches an install carries a version the
|
||||
installer can tell apart from the one already cached. One bump per batch,
|
||||
which is also how a human would do it.
|
||||
|
||||
def manifest_differs_beyond_version(a: str | None, b: str | None) -> bool:
|
||||
"""Do two `plugin.json` texts differ in anything OTHER than `version`?
|
||||
|
||||
THE exclusion, and it is kept pure — no git, no filesystem — because this
|
||||
is the half worth testing hard and it needs no repository to exercise.
|
||||
|
||||
Compares PARSED objects rather than text, so reformatting, key reordering
|
||||
and whitespace do not read as content changes. `version` is dropped from
|
||||
both sides; everything else counts, which is what keeps a userConfig-only
|
||||
or mcpServers-only edit demanding a new version.
|
||||
|
||||
Unreadable input answers True. The conservative direction is "demand a new
|
||||
version": a spurious bump costs one cache refresh, while a missed one is
|
||||
#2209 — the fix reaches the repo and stops there.
|
||||
"""
|
||||
def without_version(text: str | None):
|
||||
if text is None:
|
||||
return None
|
||||
try:
|
||||
data = json.loads(text)
|
||||
except Exception:
|
||||
return _UNREADABLE
|
||||
if not isinstance(data, dict):
|
||||
return _UNREADABLE
|
||||
return {k: v for k, v in data.items() if k != "version"}
|
||||
|
||||
left, right = without_version(a), without_version(b)
|
||||
if left is _UNREADABLE or right is _UNREADABLE:
|
||||
return True
|
||||
return left != right
|
||||
|
||||
|
||||
def shipped_content_changed(base: str) -> tuple[bool | None, list[str]]:
|
||||
"""Has anything that REACHES AN INSTALL changed against `base`?
|
||||
|
||||
Returns `(changed, paths)`. `changed` is **None** when the question could
|
||||
not be answered — a caller must never read that as "no", which is the
|
||||
distinction #2663 cost weeks of zeroed telemetry to learn.
|
||||
|
||||
The manifest is special-cased, not excluded: if it is the ONLY thing that
|
||||
moved and the only difference is `version`, nothing that reaches an
|
||||
install has changed. Any other manifest field, or any other file, counts.
|
||||
|
||||
Reads `version_relevant_paths`, which is the shipped set PLUS the files
|
||||
that decide the version — see there for why the deriver has to be in it.
|
||||
"""
|
||||
code, out = _git("diff", "--name-only", base, "--", *version_relevant_paths())
|
||||
if code != 0:
|
||||
return None, []
|
||||
paths = [p for p in out.splitlines() if p.strip()]
|
||||
if not paths:
|
||||
return False, []
|
||||
|
||||
rel_manifest = MANIFEST.relative_to(ROOT).as_posix()
|
||||
if paths == [rel_manifest]:
|
||||
return manifest_differs_beyond_version(
|
||||
manifest_text(), manifest_text(base)
|
||||
), paths
|
||||
return True, paths
|
||||
|
||||
|
||||
def check_version_is_minted(base: str = "origin/main") -> None:
|
||||
"""THE control (#3127 checklist 4), replacing "somebody remembers".
|
||||
|
||||
The checklist asks, of any hand-set component: *say what happens the
|
||||
release somebody forgets it.* This is the answer — the lane goes red,
|
||||
deterministically, because CI can compute whether the value should have
|
||||
moved. Its predecessor could only ask "did the number move at all", which
|
||||
any bump satisfied and which therefore proved nothing.
|
||||
|
||||
Four verdicts:
|
||||
|
||||
content changed, version did not FAIL — this is #2209, exactly
|
||||
version not in canonical shape FAIL — see below
|
||||
version implausibly in the future FAIL — a bad clock or a hand-edit
|
||||
version moved, content did not pass, and say so
|
||||
|
||||
THE LAST ROW IS NOT A FAILURE, DELIBERATELY. A needless re-mint costs one
|
||||
cache refresh and nothing else. Failing the lane over a harmless act is how
|
||||
a check earns a `--no-version` in somebody's muscle memory and stops
|
||||
running at all — which is the failure mode this whole file exists to
|
||||
prevent. The implication that matters is one-directional: content changed
|
||||
IMPLIES version moved.
|
||||
|
||||
A malformed version is worth failing on even though the installer would
|
||||
accept it. `K4` returns the manifest string verbatim, and `H == "unknown"`
|
||||
sets `forceOverwrite`, so a broken value either sorts as a normal string
|
||||
or reinstalls the plugin every single session (#3325). Neither is loud.
|
||||
|
||||
Stated against the BASE BRANCH rather than the last commit, as its
|
||||
predecessor was: a per-commit rule would demand a fresh mint from every
|
||||
commit in a batch, when what matters is that whatever reaches an install
|
||||
differs from what is cached. One mint per batch, which is also how a person
|
||||
would do it.
|
||||
"""
|
||||
code, _ = _git("rev-parse", "--verify", base)
|
||||
if code != 0:
|
||||
# Do NOT pass silently — a check that quietly no-ops is how this class
|
||||
# of bug survives in the first place.
|
||||
fail(
|
||||
f"cannot resolve {base}, so the version-bump check could not run. "
|
||||
f"cannot resolve {base}, so the minted-version check could not run. "
|
||||
f"Fetch it first — `git fetch --depth=1 origin main:refs/remotes/"
|
||||
f"origin/main` is enough, since this diffs two trees and needs no "
|
||||
f"common ancestor — or pass --no-version deliberately."
|
||||
)
|
||||
return
|
||||
|
||||
code, changed = _git("diff", "--name-only", base, "--", *SHIPPED)
|
||||
if code != 0:
|
||||
fail(f"git diff against {base} failed: {changed}")
|
||||
return
|
||||
if not changed.strip():
|
||||
ok(f"no shipped plugin changes against {base} — version bump not required")
|
||||
return
|
||||
|
||||
here, there = manifest_version(), manifest_version(base)
|
||||
here = manifest_version()
|
||||
if here is None:
|
||||
fail(f"could not read a version from {MANIFEST.relative_to(ROOT)}")
|
||||
return
|
||||
if not VERSION_RE.match(here):
|
||||
fail(
|
||||
f"the manifest version is {here!r}, which is not YYYY.MM.DD.HHMM.\n"
|
||||
f" One shape for every version in the family (#3127 checklist "
|
||||
f"10), zero-padded so the midnight case renders 2026.01.05.0000.\n"
|
||||
f" Run `make mint-plugin`."
|
||||
)
|
||||
return
|
||||
|
||||
minted = datetime.strptime(here, "%Y.%m.%d.%H%M").replace(tzinfo=timezone.utc)
|
||||
# A day of slack: the mint happens on a workstation and the lane runs
|
||||
# later, so a *small* skew is ordinary. A value further out than that is
|
||||
# a wrong clock or a typed year, and it makes the version lie about when
|
||||
# it was minted.
|
||||
if minted > datetime.now(timezone.utc) + timedelta(days=1):
|
||||
fail(
|
||||
f"the manifest version {here} is in the future. Either the clock "
|
||||
f"that minted it is wrong, or it was typed by hand."
|
||||
)
|
||||
return
|
||||
|
||||
changed, paths = shipped_content_changed(base)
|
||||
if changed is None:
|
||||
fail(f"git diff against {base} failed, so the version check could not run")
|
||||
return
|
||||
|
||||
there = manifest_version(base)
|
||||
if there is None:
|
||||
ok(f"no manifest on {base} — treating as a new plugin (version {here})")
|
||||
return
|
||||
if here == there:
|
||||
files = "\n ".join(changed.splitlines())
|
||||
|
||||
if changed and here == there:
|
||||
files = "\n ".join(paths)
|
||||
fail(
|
||||
f"plugin content changed but the manifest version is still {here}.\n"
|
||||
f" The installer compares versions to decide whether to refresh "
|
||||
f"its cache, so an unchanged version means these edits reach the repo "
|
||||
f"and stop there — the marketplace clone updates, the cache that "
|
||||
f"actually executes does not (issue #2209).\n"
|
||||
f" Bump `version` in {MANIFEST.relative_to(ROOT)}.\n"
|
||||
f"plugin content changed but the version is still {here}.\n"
|
||||
f" The installer decides whether to refresh its cache by "
|
||||
f"comparing this string, so an unchanged version means these edits "
|
||||
f"reach the repo and stop there — the marketplace clone updates, the "
|
||||
f"cache that actually executes does not (#2209, #1040, #2220).\n"
|
||||
f" Run `make mint-plugin`.\n"
|
||||
f" Changed:\n {files}"
|
||||
)
|
||||
elif changed:
|
||||
ok(f"plugin content changed and the version was minted {there} -> {here}")
|
||||
elif here != there:
|
||||
# Not a failure — see the docstring. Named rather than silent, because
|
||||
# the uninteresting cause (minted twice) and the interesting one (the
|
||||
# version-relevant set is too narrow to see what actually changed)
|
||||
# produce the same line, and only a person can tell them apart.
|
||||
ok(
|
||||
f"the version moved {there} -> {here} with no version-relevant "
|
||||
f"change — harmless, unless something DID change that the set "
|
||||
f"cannot see"
|
||||
)
|
||||
else:
|
||||
ok(f"plugin content changed and version moved {there} -> {here}")
|
||||
ok(f"nothing version-relevant changed against {base} — no mint required")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--no-version", action="store_true",
|
||||
help="skip the manifest version-bump check")
|
||||
help="skip the minted-version check; for `main`, where "
|
||||
"it would be measured against itself")
|
||||
parser.add_argument("--base", default="origin/main",
|
||||
help="branch the version bump is measured against")
|
||||
help="branch the version is measured against")
|
||||
args = parser.parse_args()
|
||||
|
||||
if not HOOKS_DIR.is_dir():
|
||||
@@ -478,7 +711,7 @@ def main() -> int:
|
||||
check_local_prior_art_needs_no_instance()
|
||||
check_session_context_reports_its_version()
|
||||
if not args.no_version:
|
||||
check_version_bump(args.base)
|
||||
check_version_is_minted(args.base)
|
||||
|
||||
print()
|
||||
if failures:
|
||||
|
||||
@@ -0,0 +1,144 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Mint the plugin's version — `YYYY.MM.DD.HHMM`, UTC, zero-padded.
|
||||
|
||||
Run this whenever you change something under `plugin/` or `.claude-plugin/`,
|
||||
before you commit:
|
||||
|
||||
make mint-plugin # or: python3 scripts/mint_plugin_version.py
|
||||
|
||||
WHY A SCRIPT AND NOT A BUILD STEP. `plugin/` is not in the Docker image.
|
||||
Installs fetch it straight from this git repo via `.claude-plugin/
|
||||
marketplace.json`, so **a push IS the release** — there is no build between
|
||||
you committing and a user fetching, and therefore no moment at which CI could
|
||||
stamp a version in. Every other artifact in the family derives its version
|
||||
during a build (note #3127 §2). This one has no build to derive during.
|
||||
|
||||
WHICH CLOCK, AND WHY IT DIFFERS FROM THE SERVER IMAGE — the divergence is
|
||||
deliberate, and it lives one directory away from its opposite, so it is
|
||||
exactly what a later "let's make these consistent" change would collapse:
|
||||
|
||||
server image name from COMMIT time, ordering key from BUILD time
|
||||
(two lanes building one source must report one string;
|
||||
a rebuild of an older commit must not go backwards)
|
||||
plugin one value, from MINT time
|
||||
|
||||
§2's reason for commit time is that two lanes build one source. The plugin has
|
||||
one lane and no build, so that reason does not reach it and paying its cost
|
||||
buys nothing. What is given up is reproducibility-from-history: you cannot
|
||||
recompute this value later, only verify that it moved when it had to.
|
||||
|
||||
That trade is acceptable ONLY because of what #3325 established by reading the
|
||||
installer's code: the refresh test is `P.version === H`, plain string
|
||||
equality, with no ordering comparison anywhere. Where a comparator ORDERS, an
|
||||
unreproducible version is dangerous — nothing can check it is right. Where it
|
||||
only tests equality, "did it change when it should have" is the entire
|
||||
specification, and `check_plugin.py` checks that completely.
|
||||
|
||||
The manifest is rewritten with a surgical replacement of the `version` line
|
||||
rather than `json.dump`, because its formatting and key order are not this
|
||||
script's to decide and a whole-file reformat would make every mint an
|
||||
unreadable diff.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
MANIFEST = ROOT / "plugin" / ".claude-plugin" / "plugin.json"
|
||||
|
||||
# Four dot-separated numeric fields, zero-padded, and nothing else — one shape
|
||||
# for every human-readable version in the family (#3127 checklist 10). The
|
||||
# padding is load-bearing for the midnight case the checklist names by hand:
|
||||
# 2026.01.05.0000, which an unpadded `%-H%M` would render as `0` and silently
|
||||
# shorten. Harmless while nothing orders these, wrong the moment anything does.
|
||||
VERSION_RE = re.compile(r"^\d{4}\.\d{2}\.\d{2}\.\d{4}$")
|
||||
VERSION_FORMAT = "%Y.%m.%d.%H%M"
|
||||
|
||||
# The `version` line, captured so its surroundings survive byte-for-byte.
|
||||
VERSION_LINE_RE = re.compile(r'^(\s*"version"\s*:\s*")([^"]*)(".*)$', re.M)
|
||||
|
||||
|
||||
def mint(now: datetime | None = None) -> str:
|
||||
"""The version for this moment. UTC, always.
|
||||
|
||||
The conversion is not decoration: `strftime` renders whatever offset the
|
||||
datetime carries, so without it two people minting the same instant in
|
||||
different zones produce different strings — and the string IS the
|
||||
artifact's identity. A naive datetime is read as UTC rather than as the
|
||||
machine's zone, because that is this function's stated contract and
|
||||
guessing the host's offset is how the bug comes back by another route.
|
||||
"""
|
||||
moment = now or datetime.now(timezone.utc)
|
||||
if moment.tzinfo is None:
|
||||
moment = moment.replace(tzinfo=timezone.utc)
|
||||
return moment.astimezone(timezone.utc).strftime(VERSION_FORMAT)
|
||||
|
||||
|
||||
def rewrite(text: str, version: str) -> str:
|
||||
"""`text` with its `version` value replaced, and everything else untouched.
|
||||
|
||||
Raises rather than falling back to a JSON round-trip: a manifest this
|
||||
cannot match is one whose shape changed, and quietly reformatting the file
|
||||
to cope would be a much larger edit than the caller asked for.
|
||||
"""
|
||||
# Counted BEFORE substituting, not via subn's return: a capped `subn`
|
||||
# reports the replacements it made, so a manifest with two `version` lines
|
||||
# would look like a clean single match while the second one — the real one,
|
||||
# perhaps — kept its old value.
|
||||
matches = VERSION_LINE_RE.findall(text)
|
||||
if len(matches) != 1:
|
||||
raise ValueError(
|
||||
f"expected exactly one `version` line in the manifest, found {len(matches)}"
|
||||
)
|
||||
return VERSION_LINE_RE.sub(
|
||||
lambda m: f"{m.group(1)}{version}{m.group(3)}", text, count=1
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Mint the plugin's version.")
|
||||
parser.add_argument(
|
||||
"--check", action="store_true",
|
||||
help="print the version that WOULD be minted and change nothing",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
version = mint()
|
||||
if args.check:
|
||||
print(version)
|
||||
return 0
|
||||
|
||||
try:
|
||||
text = MANIFEST.read_text()
|
||||
except OSError as exc:
|
||||
print(f"cannot read {MANIFEST.relative_to(ROOT)}: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
try:
|
||||
previous = json.loads(text).get("version")
|
||||
except Exception:
|
||||
previous = None
|
||||
|
||||
if previous == version:
|
||||
# Same minute. Not an error — the value is already correct for now, and
|
||||
# failing here would turn "I ran it twice" into a problem to solve.
|
||||
print(f"plugin version already {version} (same minute) — unchanged")
|
||||
return 0
|
||||
|
||||
try:
|
||||
MANIFEST.write_text(rewrite(text, version))
|
||||
except ValueError as exc:
|
||||
print(f"{MANIFEST.relative_to(ROOT)}: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
print(f"plugin version {previous} -> {version}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,39 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Claude Code PreToolUse hook for Bash.
|
||||
# Reads the tool input JSON from stdin; if the command is a git commit
|
||||
# and fable-mcp files (other than pyproject.toml) are staged, bumps
|
||||
# the fable-mcp patch version before the commit proceeds.
|
||||
#
|
||||
# Exits 0 always so it never blocks the commit.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
|
||||
input=$(cat)
|
||||
command=$(echo "$input" | python3 -c "
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
# Claude Code sends {tool_input: {command: ...}}
|
||||
ti = data.get('tool_input', data)
|
||||
print(ti.get('command', ''))
|
||||
" 2>/dev/null || echo "")
|
||||
|
||||
# Only act on git commit commands
|
||||
if ! echo "$command" | grep -qE "git commit"; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cd "$REPO_ROOT"
|
||||
|
||||
# Check if fable-mcp files other than pyproject.toml are staged
|
||||
fable_staged=$(git diff --cached --name-only 2>/dev/null \
|
||||
| grep "^fable-mcp/" \
|
||||
| grep -v "^fable-mcp/pyproject.toml$" \
|
||||
|| true)
|
||||
|
||||
if [ -n "$fable_staged" ]; then
|
||||
bash "$REPO_ROOT/scripts/bump_fable_mcp_version.sh"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -30,6 +30,46 @@ from quart import Quart
|
||||
# duplicate gate, the untagged-record systems_hint) act in-band in tool
|
||||
# responses, at the moment they apply.
|
||||
# Grow one of those, not this block.
|
||||
# BUDGET: ~1980 of the client's ~2048-char cap (#2562). Everything below is
|
||||
# competing for the last ~68 characters, so an addition here is a trade, never
|
||||
# an append.
|
||||
#
|
||||
# Milestone 317 (a note's own verify_with / expires_when, and the sweep over
|
||||
# them) was DECLINED a line, deliberately, by the operator — not overlooked.
|
||||
# The reasoning, so it is not re-litigated blind: this is a map, and its own
|
||||
# closing line says each tool's description carries the full contract. The
|
||||
# sweep is a curation act, not a session-start reflex like enter_project or
|
||||
# list_always_on_rules. Spending the last of the budget on it would leave the
|
||||
# map unable to grow for something more central later.
|
||||
#
|
||||
# The accepted cost: an agent that never opens create_note's docstring never
|
||||
# learns the field exists. Guidance lives in the create_note / update_note
|
||||
# docstrings and the using-scribe skill instead.
|
||||
#
|
||||
# Milestone 333 step 3 (2026-09-04) bought the HOW bullet's second clause —
|
||||
# search(content_type="rule") before a consequential act — by TRADING OUT
|
||||
# "Processes are saved procedures (follow verbatim)" and "Deletes are
|
||||
# trash-recoverable". Recorded so the trade is not silently reversed:
|
||||
# - Both were already in test_instruction_surfaces_agree's DISPLACED_TOPICS
|
||||
# and already stated on a delivered surface, so nothing fell off: the
|
||||
# process reflex is in every scribe-proc-* skill listing (each says the
|
||||
# process governs and is followed verbatim), and trash recovery is in the
|
||||
# delete_*/list_trash/restore docstrings, which is where per-tool guidance
|
||||
# belongs by this block's own doctrine.
|
||||
# - What it bought is not per-tool guidance and has nowhere else to live at
|
||||
# session-start altitude. Rules were retrievable only by RESIDENCY: the
|
||||
# always-on preload put them in front of the agent, and nothing told a
|
||||
# session to go looking for one it had not been handed. The tier split is
|
||||
# therefore load-bearing on ANY install (rule 115): a delivered rule costs
|
||||
# tokens in every session forever, so a rulebook that only delivers cannot
|
||||
# grow past what one session can hold, and every rule worth keeping has to
|
||||
# become resident to bind at all. Retrieval is what lets it keep growing —
|
||||
# and retrieval fires only if something asks, which nothing told a session
|
||||
# to do. A tool-choice reflex asks least of all (#3476, #161).
|
||||
# - This states the PULL for conditional rules, exactly as the surrounding
|
||||
# line states it for always-on ones. Rule 119 makes these surfaces the
|
||||
# specification, so the same sentence lands on all three session-start
|
||||
# surfaces, and test_instruction_surfaces_agree pins it.
|
||||
_INSTRUCTIONS = """
|
||||
Scribe is the operator's self-hosted second brain and system of record — and
|
||||
yours: recall from it before acting, record as you go. Keep no parallel copy
|
||||
@@ -48,13 +88,13 @@ Hierarchy: Project -> Milestone -> Task/Note. The map, by purpose:
|
||||
active project_id to stay in scope.
|
||||
- WHERE work happens: Systems. Tag records with system_ids as you write;
|
||||
create_system when the area is unmodelled.
|
||||
- HOW: rules are binding — list_always_on_rules() at session start.
|
||||
- HOW: rules bind. list_always_on_rules() at start; before a consequential
|
||||
act, search(content_type="rule") — the resident set is not all of them.
|
||||
- UI: the project's design system is binding — resolve_design_system /
|
||||
get_design_system_stylesheet before hand-writing a value.
|
||||
- REUSE: search snippets before writing a helper; record what you build with
|
||||
create_snippet; classify shapes against canon (classify_shapes) — a
|
||||
consumer map is rows, never prose. Processes are saved procedures (follow
|
||||
verbatim). Deletes are trash-recoverable.
|
||||
consumer map is rows, never prose.
|
||||
|
||||
A task is a note with status (*_note vs *_task tools).
|
||||
Creates are duplicate-gated: a near-match BLOCKS and returns the existing
|
||||
@@ -120,6 +160,13 @@ _READ_ONLY_TOOLS = frozenset({
|
||||
# prefix, so the completeness test below cannot derive it — the same
|
||||
# reason `enter_project` is spelled out above.
|
||||
"retrieval_telemetry",
|
||||
# The note staleness sweep (milestone 317). A pure read — mark_note_verified
|
||||
# is the write, and it is deliberately NOT here. Spelled out for
|
||||
# retrieval_telemetry's reason: `notes_due_for_verification` matches none of
|
||||
# the prefixes the completeness test derives from, so nothing would have
|
||||
# prompted this decision. `rules_due_for_verification` is in the same
|
||||
# position and is NOT listed — see #3191.
|
||||
"notes_due_for_verification",
|
||||
})
|
||||
|
||||
# Read-SHAPED tools that must NOT be reachable with a read key — a getter that
|
||||
|
||||
@@ -57,7 +57,7 @@ async def get_milestone(milestone_id: int) -> dict:
|
||||
return {
|
||||
"milestone": out,
|
||||
"steps": [t.to_dict() for t in steps],
|
||||
**rulebooks_svc.rules_payload(applicable),
|
||||
**rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_milestone"),
|
||||
}
|
||||
|
||||
|
||||
@@ -137,11 +137,23 @@ async def delete_milestone(milestone_id: int) -> dict:
|
||||
"""Move a milestone to the trash (recoverable). Its tasks go with it as one batch.
|
||||
Restore via restore(batch_id)."""
|
||||
uid = current_user_id()
|
||||
# Read the title BEFORE the delete: afterwards the row is trashed and the
|
||||
# confirmation could only echo the number back. A deletion the operator
|
||||
# cannot recognise is one they cannot tell was the wrong one.
|
||||
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
|
||||
# that errors must not stop the delete happening. Same posture the
|
||||
# staleness marker takes — a decoration may never break its payload.
|
||||
try:
|
||||
doomed = await milestones_svc.get_milestone(uid, milestone_id)
|
||||
title = getattr(doomed, "title", "") if doomed else ""
|
||||
except Exception:
|
||||
title = ""
|
||||
batch = await trash_svc.delete(uid, "milestone", milestone_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"milestone {milestone_id} not found")
|
||||
return {"deleted_batch_id": batch,
|
||||
"message": f"Milestone {milestone_id} + its tasks moved to trash. Restore with restore('{batch}')."}
|
||||
return {"deleted": milestone_id, "title": title, "deleted_batch_id": batch,
|
||||
"message": f'Milestone {milestone_id} ("{title}") and its tasks '
|
||||
f"moved to trash. Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
|
||||
@@ -103,6 +103,8 @@ async def create_note(
|
||||
project_id: int = 0,
|
||||
system_ids: list[int] | None = None,
|
||||
supersedes: list[int] | None = None,
|
||||
verify_with: str = "",
|
||||
expires_when: str = "",
|
||||
force: bool = False,
|
||||
) -> dict:
|
||||
"""Create a new note in Scribe.
|
||||
@@ -134,6 +136,33 @@ async def create_note(
|
||||
arrives labelled when it does surface. This records a CLAIM, not a
|
||||
verdict: it never says the older note was wrong, only that it is no
|
||||
longer the current answer.
|
||||
verify_with: HOW TO CHECK this note is still true. Leave empty for
|
||||
almost every note — that is the normal case, not an unfinished
|
||||
one.
|
||||
A note is a NORM or a CONSTRAINT. A norm is a decision ("we derive
|
||||
versions from commit time"): it has no truth value and changes only
|
||||
when its author changes it, which they know they did. A CONSTRAINT
|
||||
asserts a fact about someone else's software ("AMO refuses to
|
||||
re-sign a version", "this forge numbers CI runs per repository"),
|
||||
and it goes false with nobody watching. Only constraints get a
|
||||
check.
|
||||
The test, in one question: COULD THIS NOTE BECOME FALSE WITHOUT
|
||||
ANYONE EDITING IT? If no, leave this empty. Sharper still: is the
|
||||
thing this note describes YOURS TO CHANGE? If yes it is a
|
||||
decision. Measured against a real corpus, every note that earned a
|
||||
check was about somebody else's software.
|
||||
Three that look like candidates and are not: a resume pointer or
|
||||
"current state" note (goes stale fastest, but the cure is to
|
||||
update it, not to check it); a measurement of your own system (it
|
||||
goes false because you changed something, and you knew); and a
|
||||
decision that RESTS on someone else's behaviour (check the note
|
||||
asserting the fact, not the decision).
|
||||
A command, a path, a URL, a query. Prose is allowed; something
|
||||
runnable is better.
|
||||
expires_when: The STATE that ends it — deliberately not a date.
|
||||
"When Forgejo issues run numbers per workflow rather than per
|
||||
repository", not "in six months". Constraints expire when the
|
||||
ground moves, not on a schedule.
|
||||
force: Bypass the near-duplicate gate. By default, if a title- or
|
||||
meaning-similar note already exists in the same project, creation is
|
||||
BLOCKED and the existing note's id is returned so you update it
|
||||
@@ -161,6 +190,8 @@ async def create_note(
|
||||
body=body,
|
||||
tags=tags,
|
||||
project_id=project_id or None,
|
||||
verify_with=verify_with,
|
||||
expires_when=expires_when,
|
||||
)
|
||||
if system_ids:
|
||||
await systems_svc.set_record_systems(uid, note.id, system_ids)
|
||||
@@ -185,6 +216,9 @@ async def update_note(
|
||||
project_id: int = 0,
|
||||
system_ids: list[int] | None = None,
|
||||
supersedes: list[int] | None = None,
|
||||
verify_with: str = "",
|
||||
expires_when: str = "",
|
||||
clear: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Update an existing Scribe note. Only explicitly provided fields are changed.
|
||||
|
||||
@@ -199,6 +233,27 @@ async def update_note(
|
||||
supersedes: Replace the ids of earlier notes this one replaces
|
||||
(set-semantics). None = leave unchanged; [] = clear all. See
|
||||
create_note for when to reach for it.
|
||||
verify_with: How to check this note is still true. Almost every note
|
||||
should leave this empty — that is the normal, finished state, not
|
||||
a gap: an empty check is the marker for "this is a decision, there
|
||||
is nothing to go and check". See create_note for the full
|
||||
norm-vs-constraint test; the short form is "could this become
|
||||
false without anyone editing it?".
|
||||
expires_when: The STATE that ends it, not a date.
|
||||
clear: Names of fields to UNSET — "verify_with", "expires_when".
|
||||
Needed because "" means "leave this alone" here, so there is no
|
||||
value that removes a field: an agent updating a body must not
|
||||
silently wipe a check it was not asked about. A note that stops
|
||||
being a constraint is cleared by naming the field, which cannot
|
||||
happen by accident.
|
||||
|
||||
Rewriting `verify_with` drops the note's verification stamp: a stamp
|
||||
certifies a particular check, and carrying it across a rewrite would vouch
|
||||
for something nobody has looked at.
|
||||
|
||||
A task and a snippet are both REFUSED a check, with a message saying where
|
||||
to go instead — a task's decay is its status, and a snippet has
|
||||
verify_snippet.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
fields: dict = {}
|
||||
@@ -210,7 +265,13 @@ async def update_note(
|
||||
fields["tags"] = tags
|
||||
if project_id:
|
||||
fields["project_id"] = project_id
|
||||
note = await notes_svc.update_note(uid, note_id, **fields)
|
||||
if verify_with:
|
||||
fields["verify_with"] = verify_with
|
||||
if expires_when:
|
||||
fields["expires_when"] = expires_when
|
||||
note = await notes_svc.update_note(
|
||||
uid, note_id, clear=clear or (), **fields
|
||||
)
|
||||
if note is None:
|
||||
raise ValueError(f"note {note_id} not found")
|
||||
if system_ids is not None:
|
||||
@@ -263,11 +324,113 @@ async def find_duplicate_records(kind: str = "note", threshold: float = 0.0) ->
|
||||
async def delete_note(note_id: int) -> dict:
|
||||
"""Move a Scribe note to the trash (recoverable). Restore via restore(batch_id)."""
|
||||
uid = current_user_id()
|
||||
# Read the title BEFORE the delete: afterwards the row is trashed and the
|
||||
# confirmation could only echo the number back. A deletion the operator
|
||||
# cannot recognise is one they cannot tell was the wrong one.
|
||||
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
|
||||
# that errors must not stop the delete happening. Same posture the
|
||||
# staleness marker takes — a decoration may never break its payload.
|
||||
try:
|
||||
loaded = await notes_svc.get_note_for_user(uid, note_id)
|
||||
title = getattr(loaded[0], "title", "") if loaded else ""
|
||||
except Exception:
|
||||
title = ""
|
||||
batch = await trash_svc.delete(uid, "note", note_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"note {note_id} not found")
|
||||
return {"deleted_batch_id": batch,
|
||||
"message": f"Note {note_id} moved to trash. Restore with restore('{batch}')."}
|
||||
return {"deleted": note_id, "title": title, "deleted_batch_id": batch,
|
||||
"message": f'Note {note_id} ("{title}") moved to trash. '
|
||||
f"Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
async def notes_due_for_verification(
|
||||
older_than_days: int = 0, project_id: int = 0, never_only: bool = False,
|
||||
) -> dict:
|
||||
"""Which notes assert a FACT that nobody has confirmed lately.
|
||||
|
||||
A corpus of notes holds two kinds of thing. Most are DECISIONS or records
|
||||
of what happened — they have no truth value and cannot rot. A few assert a
|
||||
fact about someone else's software: what a signing service does on a
|
||||
duplicate upload, how a forge numbers its CI runs, what an updater
|
||||
compares. Those go false silently, with nobody present, and a
|
||||
cross-project reference note keeps being read as current by every project
|
||||
that cites it.
|
||||
|
||||
This lists the second kind, oldest verification first, NEVER-CHECKED AT
|
||||
THE TOP — a note nobody has ever confirmed is a claim with no evidence
|
||||
behind it at all. Each row carries `verify_with` in full, because you are
|
||||
about to go and run it, plus `expires_when` and `days_since_verified`.
|
||||
|
||||
Reach for it when curating, when a note's claim just contradicted what you
|
||||
observed, or periodically. Then, per row: run the check, and call
|
||||
mark_note_verified with what you found.
|
||||
|
||||
Notes with no `verify_with` never appear, and that is correct — they are
|
||||
decisions, and there is nothing to go and check. Do not "fix" their
|
||||
absence by giving them checks: this list is only worth reading while
|
||||
everything on it genuinely can go false.
|
||||
|
||||
Args:
|
||||
older_than_days: only notes last verified longer ago than this.
|
||||
Never-checked notes always qualify — they are the most overdue
|
||||
thing there is. 0 = no age filter.
|
||||
project_id: narrow to one project. 0 = every project. Unlike the rules
|
||||
sweep, this filter is safe: a note belongs to at most one project
|
||||
outright, with none of the subscription and always-on paths that
|
||||
would make a project filter UNDER-report a rule.
|
||||
never_only: only notes nobody has ever verified.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
notes = await notes_svc.notes_due_for_verification(
|
||||
uid,
|
||||
older_than_days=older_than_days,
|
||||
project_id=project_id or None,
|
||||
never_only=never_only,
|
||||
)
|
||||
return {
|
||||
"notes": [notes_svc.verification_row(n) for n in notes],
|
||||
"total": len(notes),
|
||||
}
|
||||
|
||||
|
||||
async def mark_note_verified(note_id: int, still_true: bool = True) -> dict:
|
||||
"""Record that you ran a note's check — and what it said.
|
||||
|
||||
Call this AFTER actually running the note's `verify_with`, never on the
|
||||
strength of the claim sounding plausible. A stamp nobody earned is worse
|
||||
than no stamp: it moves the note to the bottom of the sweep and buys the
|
||||
claim another long silence.
|
||||
|
||||
`still_true=False` writes NOTHING. A note whose check failed is not in a
|
||||
special state to be recorded — it is WRONG, and the honest next moves are
|
||||
to correct it, supersede it, or find out why. So it stays at the top of
|
||||
the sweep until someone deals with it, and the response tells you what the
|
||||
note said would end it.
|
||||
|
||||
Args:
|
||||
note_id: the note whose check you ran.
|
||||
still_true: True if the check passed. False if the fact it asserts is
|
||||
no longer true — say so, that is the outcome worth having.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
note = await notes_svc.mark_note_verified(note_id, uid, still_true)
|
||||
if note is None:
|
||||
raise ValueError(
|
||||
f"note {note_id} not found, not writable by you, or carries no "
|
||||
f"verify_with (nothing to verify is not the same as verified)"
|
||||
)
|
||||
data = notes_svc.verification_row(note)
|
||||
data["verified"] = bool(still_true)
|
||||
if not still_true:
|
||||
data["next"] = (
|
||||
"This note is no longer true and is still being read as current "
|
||||
"by anything that cites it. Correct it with update_note, write "
|
||||
"the replacement with create_note(supersedes=[...]), or clear its "
|
||||
"check with update_note(clear=[\"verify_with\"]) if it has stopped "
|
||||
"asserting a fact at all. It stays at the top of "
|
||||
"notes_due_for_verification until one of those happens."
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
@@ -278,5 +441,7 @@ def register(mcp) -> None:
|
||||
update_note,
|
||||
find_duplicate_records,
|
||||
delete_note,
|
||||
notes_due_for_verification,
|
||||
mark_note_verified,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
|
||||
@@ -207,7 +207,7 @@ async def enter_project(project_id: int) -> dict:
|
||||
],
|
||||
"design_system": design_system,
|
||||
"milestone_summary": milestone_summary,
|
||||
**rulebooks_svc.rules_payload(applicable),
|
||||
**rulebooks_svc.rules_payload(applicable, user_id=uid, source="enter_project"),
|
||||
"open_tasks": [
|
||||
{
|
||||
"id": t.id, "title": t.title, "status": t.status,
|
||||
@@ -251,7 +251,7 @@ async def get_project(project_id: int) -> dict:
|
||||
applicable = await rulebooks_svc.get_applicable_rules(
|
||||
project_id=project_id, user_id=uid,
|
||||
)
|
||||
data.update(rulebooks_svc.rules_payload(applicable))
|
||||
data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_project"))
|
||||
return data
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ from scribe.mcp._context import current_user_id
|
||||
from scribe.services import dedup as dedup_svc
|
||||
from scribe.services import rulebooks as rulebooks_svc
|
||||
from scribe.services import trash as trash_svc
|
||||
from scribe.services.rule_usage import record_rule_pulled, record_rule_surfaced
|
||||
|
||||
|
||||
# ── Rulebook CRUD ───────────────────────────────────────────────────────
|
||||
@@ -122,8 +123,9 @@ async def delete_rulebook(rulebook_id: int, confirmed: bool = False) -> dict:
|
||||
"confirmed_required": True,
|
||||
}
|
||||
batch = await trash_svc.delete(uid, "rulebook", rulebook_id)
|
||||
return {"deleted": rulebook_id, "deleted_batch_id": batch,
|
||||
"message": f"Moved to trash. Restore with restore('{batch}')."}
|
||||
return {"deleted": rulebook_id, "title": rb.title, "deleted_batch_id": batch,
|
||||
"message": f'Rulebook {rulebook_id} ("{rb.title}") moved to trash. '
|
||||
f"Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
# ── Topic CRUD ─────────────────────────────────────────────────────────
|
||||
@@ -192,8 +194,9 @@ async def delete_topic(topic_id: int, confirmed: bool = False) -> dict:
|
||||
"confirmed_required": True,
|
||||
}
|
||||
batch = await trash_svc.delete(uid, "topic", topic_id)
|
||||
return {"deleted": topic_id, "deleted_batch_id": batch,
|
||||
"message": f"Moved to trash. Restore with restore('{batch}')."}
|
||||
return {"deleted": topic_id, "title": topic.title, "deleted_batch_id": batch,
|
||||
"message": f'Topic {topic_id} ("{topic.title}") moved to trash. '
|
||||
f"Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
# ── Rule CRUD ──────────────────────────────────────────────────────────
|
||||
@@ -262,7 +265,25 @@ async def list_always_on_rules(project_id: int = 0) -> dict:
|
||||
"""
|
||||
uid = current_user_id()
|
||||
rules = await rulebooks_svc.list_always_on_rules(uid, project_id=project_id)
|
||||
return {"rules": [_rule_summary(r) for r in rules], "total": len(rules)}
|
||||
# AMBIENT source: the resident set, handed over whole. No ranker chose
|
||||
# these, so they must not land in the pull-through numerator's denominator
|
||||
# — but they must land SOMEWHERE, or the largest rule surface in the
|
||||
# product stays the one surface its own scoreboard cannot see (#3473).
|
||||
record_rule_surfaced(
|
||||
user_id=uid,
|
||||
rule_ids=[r.id for r in rules],
|
||||
source="list_always_on_rules",
|
||||
)
|
||||
return {
|
||||
"rules": [_rule_summary(r) for r in rules],
|
||||
"total": len(rules),
|
||||
# A marker for the set you are now holding. It is not for you to read:
|
||||
# the write-path hook carries it back and is told if these rules have
|
||||
# moved since. Deliberately NOT on rules_payload's applicable_rules —
|
||||
# that is a DIFFERENT set (subscription-derived), and one key name
|
||||
# over two sets is how a comparison starts reporting phantom changes.
|
||||
"rules_etag": rulebooks_svc.rules_etag(rules),
|
||||
}
|
||||
|
||||
|
||||
async def get_rule(rule_id: int) -> dict:
|
||||
@@ -277,6 +298,11 @@ async def get_rule(rule_id: int) -> dict:
|
||||
rule = await rulebooks_svc.get_rule(rule_id, uid)
|
||||
if rule is None:
|
||||
raise ValueError(f"rule {rule_id} not found")
|
||||
# THE pull that matters. The write-path rule arm's own message ends "Read
|
||||
# it with get_rule(N)", so this is the exact action the hint asks for and
|
||||
# the only evidence that one landed. Recorded after the access check, so a
|
||||
# refused read is not counted as a pull.
|
||||
record_rule_pulled(user_id=uid, rule_id=int(rule.id), source="mcp_get_rule")
|
||||
return await rulebooks_svc.rule_detail(uid, rule)
|
||||
|
||||
|
||||
@@ -289,6 +315,61 @@ async def create_rule(
|
||||
) -> dict:
|
||||
"""Create a new rule in a rulebook (a SHARED rule — keep it general).
|
||||
|
||||
PROPOSE RULES READILY, AND WRITE ONE WHEN THE OPERATOR SAYS YES. Noticing
|
||||
that something has hardened into a standing instruction is valuable work,
|
||||
and a session that notices it and says nothing has thrown the observation
|
||||
away. So raise it whenever you see one. The single step that belongs
|
||||
between noticing and writing is the operator's yes: a rule binds every
|
||||
future session, and they are the person it binds.
|
||||
|
||||
Their yes is also the only moment the rule is reliably IN FRONT of them.
|
||||
After the write it may not be again for months — a conditional rule is not
|
||||
read aloud at session start, and a project-scoped one does not appear in
|
||||
an unfiltered list_rules() at all. So the proposal is the review.
|
||||
|
||||
When the operator asks for a rule in so many words, that IS the yes —
|
||||
write it and move on. The loop below is for the rule you thought of.
|
||||
|
||||
A PROPOSAL CARRIES FOUR THINGS, and the fourth is the one that decides it:
|
||||
|
||||
1. WHAT it would require — the statement, in the words it would carry,
|
||||
not a gloss of them. The operator is agreeing to text.
|
||||
2. INTENT — what it changes about how work gets done, and what goes
|
||||
wrong today without it. "Be careful about X" is not an intent; the
|
||||
behaviour that would differ tomorrow is.
|
||||
3. WHY NOW — the incident, observation or decision behind it. Pass that
|
||||
record as arose_from_id, and say it in the conversation too: the
|
||||
field is for the reader six months out, the sentence is for the
|
||||
person deciding.
|
||||
4. HOW IT WOULD BE ENFORCED — a test, a CI check, a hook, a schema
|
||||
constraint, a duplicate gate, a review step... or nothing, in which
|
||||
case say so plainly: "nothing — this is prose a session has to
|
||||
remember." Answer this one honestly and it will sometimes dissolve
|
||||
the rule, which is the point rather than a side effect. What a test
|
||||
can assert should BE that test; a rule is what remains when nothing
|
||||
mechanical can hold the thing. A rulebook grows by default and
|
||||
shrinks only on purpose, so a question that prevents a rule is worth
|
||||
more than any question that improves one's wording.
|
||||
|
||||
THEN CLOSE WITH A QUESTION THEY CAN ANSWER IN ONE WORD. Offer three
|
||||
answers, and make the middle one the easy one:
|
||||
|
||||
* "Approve it AS WRITTEN" — you create it with the statement exactly as
|
||||
shown. This is what makes element 1 load-bearing: they approved TEXT,
|
||||
so that text is what gets stored, verbatim.
|
||||
* "LET'S TALK ABOUT IT" — the wording, the scope, the tier, whether it
|
||||
wants to be a rule at all. Most good rules arrive this way, so treat
|
||||
this answer as the expected one rather than a setback.
|
||||
* "NO" — let it go. If the observation is still worth keeping, it is a
|
||||
note (create_note): recorded, findable, and binding on nobody.
|
||||
|
||||
Where the interface offers structured choices, ask it that way — a
|
||||
question with named options is answered in a click, while the same
|
||||
question inside a paragraph is answered by scrolling past. Where it does
|
||||
not, write the three options out as three options. Either way ask once
|
||||
and let the answer stand; re-raising a declined proposal argues a rule
|
||||
into existence, which is the thing this whole loop exists to prevent.
|
||||
|
||||
A rulebook rule is shared by every project that gets the rulebook: an
|
||||
always_on rulebook binds ALL your projects; a subscribed rulebook binds the
|
||||
projects that opt in. So a rulebook rule must read as a general standard —
|
||||
@@ -407,6 +488,16 @@ async def create_project_rule(
|
||||
the rule is returned in get_project's applicable_rules (under
|
||||
project_rules) and in list_rules(project_id=...).
|
||||
|
||||
PROPOSE, THEN WRITE ON A YES — create_rule's opening carries the whole
|
||||
loop: the four things a proposal states (what it would require, its
|
||||
intent, why now, and how it would be enforced) and the one-word question
|
||||
that closes it (approve as written / talk about it / no). All of it
|
||||
applies here unchanged. Reach for that loop MORE readily on this surface,
|
||||
not less: a project rule stays out of an unfiltered list_rules(), and a
|
||||
conditional one stays out of session start too, so the operator's yes is
|
||||
the one moment this rule is certain to have been seen by the person it
|
||||
binds.
|
||||
|
||||
Check first whether a rule is the right shape at all — create_rule's
|
||||
opening asks that question and it applies identically here. A visual
|
||||
standard is a design system; a procedure is a process (create_process);
|
||||
@@ -548,6 +639,73 @@ async def update_rule(
|
||||
return await rulebooks_svc.rule_detail(uid, rule, system_ids)
|
||||
|
||||
|
||||
async def rule_history(rule_id: int, version_id: int = 0) -> dict:
|
||||
"""What a rule USED TO SAY, newest change first.
|
||||
|
||||
Read this before you argue with a rule, and before you rewrite one. A
|
||||
rule that has been reworded may have been reworded for a reason you are
|
||||
about to rediscover the hard way — and the wording it replaced is often
|
||||
the fastest way to see what the current one is guarding against. The
|
||||
rescoping of rule 79 is the case this exists for: the superseded
|
||||
statement had to be hand-copied into a task log to survive the edit.
|
||||
|
||||
EACH ENTRY HOLDS THE TEXT THE EDIT REPLACED, not the text it introduced.
|
||||
So "what did this say before the most recent change?" is the first entry,
|
||||
and the text the change PRODUCED is the rule as it stands now — read that
|
||||
with get_rule. Pair the two and you have the diff.
|
||||
|
||||
An empty history is ordinary and means the rule has never been reworded,
|
||||
not that its history was lost. Nothing is written before milestone 323,
|
||||
so a rule edited before then starts empty too.
|
||||
|
||||
Args:
|
||||
rule_id: The rule whose history to read.
|
||||
version_id: 0 (default) lists the history — when each change
|
||||
happened, by whom, and the title as it then stood. Pass an id
|
||||
from that list to read that snapshot IN FULL. The list omits
|
||||
statement and why on purpose: a rule's statement runs to
|
||||
thousands of characters, and a history carrying every field would
|
||||
cost more to read than the answer is worth.
|
||||
|
||||
There is deliberately no restore. Putting an old wording back is a
|
||||
decision, so it goes through update_rule — which snapshots what it
|
||||
replaces, leaving the undo visible in the history like any other edit. A
|
||||
one-click revert would erase the only record of why the rewrite happened.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
if version_id:
|
||||
version = await rulebooks_svc.get_rule_version(rule_id, version_id, uid)
|
||||
if version is None:
|
||||
raise ValueError(
|
||||
f"version {version_id} not found on rule {rule_id}"
|
||||
)
|
||||
return version.to_dict(include_text=True)
|
||||
|
||||
versions = await rulebooks_svc.list_rule_versions(rule_id, uid)
|
||||
if versions is None:
|
||||
raise ValueError(f"rule {rule_id} not found")
|
||||
# Fail-open, like the deletes: a missing title must not turn a readable
|
||||
# history into an error.
|
||||
try:
|
||||
rule = await rulebooks_svc.get_rule(rule_id, uid)
|
||||
except Exception:
|
||||
rule = None
|
||||
return {
|
||||
"rule_id": rule_id,
|
||||
"title": rule.title if rule else "",
|
||||
"versions": [v.to_dict(include_text=False) for v in versions],
|
||||
"total": len(versions),
|
||||
# Said in-band because an empty list is the ordinary case and reads
|
||||
# like a missing feature otherwise.
|
||||
"note": (
|
||||
"Each entry holds the text the edit REPLACED. The current wording "
|
||||
"is on the rule itself — get_rule(%d)." % rule_id
|
||||
if versions else
|
||||
"This rule has never been reworded."
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
async def delete_rule(rule_id: int, confirmed: bool = False) -> dict:
|
||||
"""Move a rule to the trash (recoverable). Requires confirmed=True."""
|
||||
uid = current_user_id()
|
||||
@@ -563,8 +721,9 @@ async def delete_rule(rule_id: int, confirmed: bool = False) -> dict:
|
||||
"confirmed_required": True,
|
||||
}
|
||||
batch = await trash_svc.delete(uid, "rule", rule_id)
|
||||
return {"deleted": rule_id, "deleted_batch_id": batch,
|
||||
"message": f"Moved to trash. Restore with restore('{batch}')."}
|
||||
return {"deleted": rule_id, "title": rule.title, "deleted_batch_id": batch,
|
||||
"message": f'Rule {rule_id} ("{rule.title}") moved to trash. '
|
||||
f"Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
# ── Subscriptions ──────────────────────────────────────────────────────
|
||||
@@ -832,5 +991,6 @@ def register(mcp) -> None:
|
||||
suppress_topic_for_project, unsuppress_topic_for_project,
|
||||
exclude_always_on_rulebook, include_always_on_rulebook,
|
||||
rules_due_for_verification, mark_rule_verified,
|
||||
rule_history,
|
||||
):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
|
||||
@@ -112,6 +112,7 @@ async def search(
|
||||
return await _search_rules(uid, q, limit)
|
||||
is_task = {"note": False, "task": True}.get(content_type) # None => any
|
||||
t0 = time.perf_counter()
|
||||
report: dict = {}
|
||||
raw = await semantic_search_notes(
|
||||
uid, q, limit=limit, is_task=is_task,
|
||||
project_id=project_id or None,
|
||||
@@ -119,12 +120,14 @@ async def search(
|
||||
# An explicit search reaches everything the operator may read, including
|
||||
# records shared with them one-to-one.
|
||||
scope="read",
|
||||
report=report,
|
||||
)
|
||||
record_retrieval(
|
||||
user_id=uid, source="mcp_search", query=q,
|
||||
threshold=DEFAULT_SIMILARITY_THRESHOLD, limit=limit,
|
||||
project_id=project_id or None, is_task=is_task, results=raw,
|
||||
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
||||
best_available=report.get("best_available_score"),
|
||||
)
|
||||
owners = await owner_names_for(
|
||||
{int(note.user_id) for _s, note in raw if note.user_id != uid}
|
||||
@@ -158,17 +161,54 @@ async def retrieval_telemetry(days: int = 30) -> dict:
|
||||
hand-probing the live instance, which is how the last such decision had to
|
||||
be made.
|
||||
|
||||
Two readouts, from the two tables built for them:
|
||||
Three readouts, from the three tables built for them:
|
||||
|
||||
`sources` — per retrieval surface (`auto_inject`, `write_path`,
|
||||
`mcp_search`, …), from `retrieval_logs`: `calls`, `zero_result_calls`,
|
||||
`cleared_threshold` (how often the best hit beat the threshold in force for
|
||||
that call), the `top_score` spread (p10/p50/p90/min/max), `avg_result_count`
|
||||
and `p90_duration_ms`. THE number to read first is `cleared_threshold`
|
||||
against `calls`, with the spread beside it: a surface that clears its bar
|
||||
on nearly every call is either well-tuned or too loose, and p10 says which.
|
||||
`near_misses`, the `top_score` spread (p10/p50/p90/min/max),
|
||||
`avg_result_count` and `p90_duration_ms`.
|
||||
|
||||
`usage` — from `note_usage_events`, at the per-note grain
|
||||
THE NUMBER TO READ FIRST IS `near_misses.p90`, AGAINST THE THRESHOLD IN
|
||||
FORCE FOR THAT SURFACE. It is measured on the calls the BAR turned away —
|
||||
zero-result calls, minus the ones whose zero was a repeat the reader had
|
||||
already been shown — using the best score the ranker reached before the bar
|
||||
rejected it. So it is the one figure here that says something the bar
|
||||
cannot make true by construction, and `max` is always below the threshold:
|
||||
an above-bar candidate nobody excluded would have been returned. A bar at 0.72 turning away a stream of 0.71s is set too
|
||||
high by a hair and the surface is losing hits it should have had. The same
|
||||
bar turning away 0.30s is working, and the corpus simply had nothing. Both
|
||||
render as a zero-result call, and nothing else in this readout tells them
|
||||
apart.
|
||||
|
||||
`near_misses` is `null` when no declining call in the window measured it —
|
||||
rows written before #3670 shipped cannot know. That is "not measured", not
|
||||
"nothing came close"; a 0.0 there would be a claim about the corpus
|
||||
invented out of a caller's silence.
|
||||
|
||||
THERE IS NO `cleared_threshold` ANY MORE, and if you remember one, that
|
||||
memory is of a tautology (#3670). The search applies the bar before
|
||||
returning, so every returned result cleared it by construction and a call
|
||||
with no results has no score to compare: the field was true exactly when
|
||||
`result_count > 0`, i.e. it was `calls - zero_result_calls` under a name
|
||||
that promised a second opinion. `zero_result_calls + cleared_threshold ==
|
||||
calls` held on all nineteen readings ever taken. The reading procedure
|
||||
built on it — "clears its bar on nearly every call" — asked you to compare
|
||||
a number with itself.
|
||||
|
||||
CHECK `suppression` BEFORE CONCLUDING ANYTHING FROM `zero_result_calls`. A
|
||||
zero-result call is two different events wearing one number: the ranker
|
||||
found nothing above the bar, or it found only what this session had already
|
||||
been shown. Just the first is evidence about the bar. `suppression` splits
|
||||
them where the surface can tell — `zero_because_already_shown` comes off
|
||||
`zero_result_calls` to leave the true ranker declines.
|
||||
|
||||
`suppression` is `null` when NO row in the window reported it, and that is
|
||||
"not measured here", NOT "none suppressed". Surfaces that pass their
|
||||
exclusions into the search never see what was dropped, so they cannot say.
|
||||
Do not read a null as a zero: reading an artifact as a measurement is how
|
||||
this surface got mis-scoped once already (#3311, #3497).
|
||||
|
||||
`usage` — NOTES ONLY, from `note_usage_events`, at the per-note grain
|
||||
`retrieval_logs` cannot be indexed at: `surfaced` (ranked surfacings — a
|
||||
scored surface CHOSE the record), `ambient` (the rest), `pulled` split into
|
||||
`pulled_by_agent` / `pulled_by_human`, the distinct-note counts, and
|
||||
@@ -182,6 +222,84 @@ async def retrieval_telemetry(days: int = 30) -> dict:
|
||||
tuned against — only by a pull the agent made. Aggregating across the
|
||||
mcp_/rest_ prefix would silently answer the wrong one.
|
||||
|
||||
`usage["by_source"]` — THE number to tune a threshold against, because the
|
||||
top-level `pull_through` is a corpus average and averages the surfaces
|
||||
together. Per surface: `notes_surfaced`, `notes_pulled`, `pull_through`,
|
||||
and `ambient: true` on surfaces whose surfacings were not scored choices
|
||||
(their ratio is null — "surfaced often, opened never" is not a judgment
|
||||
about a record nothing chose). Read it as: of the distinct notes THIS
|
||||
surface put in front of the agent, how many did the agent then open?
|
||||
|
||||
It is an UPPER BOUND per surface: a pull records the door it came
|
||||
through, not the surface that led there, so a note surfaced by two surfaces
|
||||
and opened once counts for both — attribution would need the session
|
||||
identity #2085 declined to invent. `by_source_failed: true` means that one
|
||||
query failed while the rest of the readout stood.
|
||||
|
||||
`rule_usage` — the same question for RULES, from `rule_usage_events`:
|
||||
`surfaced` and `ambient`, `pulled` split into `pulled_by_agent` /
|
||||
`pulled_by_human`, the distinct-rule counts, and `pull_through` on the same
|
||||
definition (agent pulls over RANKED surfacings).
|
||||
|
||||
A SEPARATE BLOCK, not folded into `usage`, and reading it as one number
|
||||
with that is the mistake to avoid. The corpora differ by orders of
|
||||
magnitude — a few dozen eligible rules against thousands of notes — so a
|
||||
blended ratio would be the note ratio with noise on it and would hide the
|
||||
rule arm entirely.
|
||||
|
||||
`surfaced` VS `ambient` IS THE READING THAT MATTERS HERE. `surfaced` counts
|
||||
rules a ranker chose — today only the write-path arm — and those are claims
|
||||
a pull can settle. `ambient` counts BULK DELIVERIES: the SessionStart
|
||||
preload, `list_always_on_rules`, and the `rules_payload` surfaces
|
||||
(`enter_project`, `get_project`, `get_milestone`, `start_planning`,
|
||||
`get_task`), which hand over the whole applicable set at once with nobody
|
||||
choosing anything. A large `ambient` says the resident set is big and
|
||||
arrives often — never that it is useful, and never that it is read.
|
||||
|
||||
`pull_through` therefore divides by `surfaced` alone. Fold the preload in
|
||||
and growing the always-on set would depress the arm's measured precision
|
||||
while trimming it would flatter it, for reasons having nothing to do with
|
||||
the arm. To judge the PRELOAD instead, compare `ambient` against pulls of
|
||||
those same rules over time: a resident set surfaced thousands of times and
|
||||
opened never is the dead-weight signal, one tier up.
|
||||
|
||||
Read it against `sources["write_path_rule"]`. That arm was once believed
|
||||
never to decline — the reading that scoped #3311 — but it was the arm's
|
||||
`retrieval_logs` row being written only on calls that FOUND something, so
|
||||
the zeros were missing rather than absent (#3497). Measured since, it
|
||||
declines the large majority of its calls like any other surface.
|
||||
|
||||
EVERY COUNTER BLOCK CARRIES ITS OWN COVERAGE — `complete_from` and
|
||||
`covers_window`. `complete_from` is when the number became trustworthy:
|
||||
for one source, its first recorded row; for a section that sums several,
|
||||
the LATEST of theirs, because a total is complete only once every
|
||||
contributor was being written. `covers_window: false` means the window
|
||||
reaches back further than the recording does, so the count is a fraction
|
||||
of the period it appears to describe.
|
||||
|
||||
READ IT BEFORE COMPARING TWO NUMBERS, and especially before comparing
|
||||
across a deploy. A counter added last week, read over a 30-day window,
|
||||
reports a real count against an imagined denominator — and the result is
|
||||
a plausible fraction rather than an obvious zero, which is what makes it
|
||||
dangerous. That reading cost milestone #379 five steps aimed at a defect
|
||||
that did not exist.
|
||||
|
||||
`covers_window` is null, never false, when nothing was ever recorded:
|
||||
"no measurement" is not "partial measurement", the same distinction
|
||||
`suppression`'s null carries a few paragraphs up.
|
||||
|
||||
A SOURCE SHOWING `calls: 0` WAS RECORDING AND MADE NO CALLS. `sources`
|
||||
lists every source the table has ever held, not only those active in the
|
||||
window, so a surface that stopped firing stays visible rather than
|
||||
disappearing — being absent is reserved for a source that has never
|
||||
recorded at all. Its score fields are null, not zero: the calls are a
|
||||
real observation, the distribution is not one.
|
||||
|
||||
`rule_usage_failed: true` means that read failed while the rest of the
|
||||
readout stood. The counts are still present so a caller can render, but
|
||||
they are zeros meaning "could not find out", not "nothing happened" — do
|
||||
not report a pull-through from a block carrying that flag.
|
||||
|
||||
Scoped to your own telemetry — a retrieval log records what your agent
|
||||
asked for, query text included, and is not a shared record kind.
|
||||
|
||||
|
||||
@@ -494,9 +494,19 @@ async def delete_snippet(snippet_id: int) -> dict:
|
||||
that should survive, prefer merge_snippets — that keeps the call sites.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
# Read before deleting so the confirmation can NAME what went — an id
|
||||
# alone leaves the operator unable to tell which snippet this was.
|
||||
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
|
||||
# that errors must not stop the delete happening. Same posture the
|
||||
# staleness marker takes — a decoration may never break its payload.
|
||||
try:
|
||||
doomed = await snippets_svc.get_snippet(uid, snippet_id)
|
||||
title = getattr(doomed, "title", "") if doomed else ""
|
||||
except Exception:
|
||||
title = ""
|
||||
if not await snippets_svc.delete_snippet(uid, snippet_id):
|
||||
raise ValueError(f"snippet {snippet_id} not found")
|
||||
return {"deleted": True, "id": snippet_id}
|
||||
return {"deleted": True, "id": snippet_id, "title": title}
|
||||
|
||||
|
||||
async def merge_snippets(target_id: int, source_ids: list[int]) -> dict:
|
||||
|
||||
@@ -103,7 +103,7 @@ async def get_task(task_id: int) -> dict:
|
||||
applicable = await rulebooks_svc.get_applicable_rules(
|
||||
project_id=note.project_id, user_id=uid,
|
||||
)
|
||||
data.update(rulebooks_svc.rules_payload(applicable))
|
||||
data.update(rulebooks_svc.rules_payload(applicable, user_id=uid, source="get_task"))
|
||||
data.update(await access_svc.describe_provenance(uid, note))
|
||||
# Same reasoning as get_note's record_pulled, and this is the tool where it
|
||||
# matters MOST: auto-inject ranks kind-blind over a corpus that is
|
||||
@@ -364,11 +364,23 @@ async def delete_task(task_id: int) -> dict:
|
||||
"""Move a Scribe task (or plan) to the trash (recoverable). Sub-tasks go with it.
|
||||
Restore via restore(batch_id)."""
|
||||
uid = current_user_id()
|
||||
# Read the title BEFORE the delete: afterwards the row is trashed and the
|
||||
# confirmation could only echo the number back. A deletion the operator
|
||||
# cannot recognise is one they cannot tell was the wrong one.
|
||||
# Fail-open: the title is a COURTESY on top of the delete, so a lookup
|
||||
# that errors must not stop the delete happening. Same posture the
|
||||
# staleness marker takes — a decoration may never break its payload.
|
||||
try:
|
||||
loaded = await notes_svc.get_note_for_user(uid, task_id)
|
||||
title = getattr(loaded[0], "title", "") if loaded else ""
|
||||
except Exception:
|
||||
title = ""
|
||||
batch = await trash_svc.delete(uid, "task", task_id)
|
||||
if batch is None:
|
||||
raise ValueError(f"task {task_id} not found")
|
||||
return {"deleted_batch_id": batch,
|
||||
"message": f"Task {task_id} moved to trash. Restore with restore('{batch}')."}
|
||||
return {"deleted": task_id, "title": title, "deleted_batch_id": batch,
|
||||
"message": f'Task {task_id} ("{title}") moved to trash. '
|
||||
f"Restore with restore('{batch}')."}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
|
||||
@@ -28,11 +28,13 @@ from scribe.models.invitation import InvitationToken # noqa: E402, F401
|
||||
from scribe.models.embedding import NoteEmbedding, RuleEmbedding # noqa: E402, F401
|
||||
from scribe.models.retrieval_log import RetrievalLog # noqa: E402, F401
|
||||
from scribe.models.note_usage import NoteUsageEvent # noqa: E402, F401
|
||||
from scribe.models.rule_usage import RuleUsageEvent # noqa: E402, F401
|
||||
from scribe.models.project import Project # noqa: E402, F401
|
||||
from scribe.models.milestone import Milestone # noqa: E402, F401
|
||||
from scribe.models.task_log import TaskLog # noqa: E402, F401
|
||||
from scribe.models.note_draft import NoteDraft # noqa: E402, F401
|
||||
from scribe.models.note_version import NoteVersion # noqa: E402, F401
|
||||
from scribe.models.rule_version import RuleVersion # noqa: E402, F401
|
||||
from scribe.models.note_supersession import NoteSupersession # noqa: E402, F401
|
||||
from scribe.models.group import Group, GroupMembership # noqa: E402, F401
|
||||
from scribe.models.share import NoteShare, ProjectShare # noqa: E402, F401
|
||||
|
||||
@@ -51,7 +51,11 @@ class RuleEmbedding(Base):
|
||||
"""One embedding vector per CHUNK of a rule (milestone 307, note 3026).
|
||||
|
||||
A SIBLING of NoteEmbedding rather than a generalisation of it, decided
|
||||
deliberately:
|
||||
deliberately. The reasoning below turned out to be the only written
|
||||
statement of a rule Scribe applies to every record type, so it is now also
|
||||
NOTE 3163 — "When a record type earns its own table" — with the worked
|
||||
cases and the cost of leaving `notes`. Read that before splitting a record
|
||||
type off; this stays here because it is where the decision was made.
|
||||
|
||||
- The embedding ROW could have been made polymorphic. The SEARCH could not.
|
||||
`semantic_search_notes` is a long function of Note-specific scoping —
|
||||
|
||||
@@ -94,9 +94,40 @@ class Note(Base, TimestampMixin, SoftDeleteMixin):
|
||||
# name/language/signature/locations live here so they can be INDEXED. The
|
||||
# body keeps the same facts in readable markdown and remains what gets
|
||||
# embedded; this is a mirror for querying, not the source of truth for
|
||||
# display. NULL on every row written before migration 0070, so readers fall
|
||||
# back to parsing the body (see services/snippets.snippet_fields).
|
||||
# display — and it is DERIVED, so every path that writes a snippet's body
|
||||
# rewrites it too (services/snippets.recompose_data, called from
|
||||
# notes.update_note). 0070 left it NULL on existing rows and
|
||||
# snippets.backfill_snippet_data filled them at startup; readers still fall
|
||||
# back to parsing the body when it is absent (snippet_fields).
|
||||
data: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
||||
# The three fields that tell a CONSTRAINT apart from a NORM (milestone
|
||||
# 317, migration 0092) — the same trio `rules` carries, and for the same
|
||||
# reason. A norm is a decision: no truth value, changes only when its
|
||||
# author changes it. A constraint asserts a fact about someone else's
|
||||
# software and goes false with nobody watching. Only constraints get a
|
||||
# check.
|
||||
#
|
||||
# `verify_with` is how to check it is still true; `expires_when` is the
|
||||
# STATE that ends it, deliberately not a date — constraints expire when
|
||||
# the ground moves, not on a schedule. `verified_at` NULL means never
|
||||
# checked and sorts FIRST in the sweep: unexamined outranks
|
||||
# examined-long-ago.
|
||||
#
|
||||
# These sit on `notes`, so every kind of row in this table has them, but
|
||||
# only non-task, non-snippet records are OFFERED them (gated in
|
||||
# services/notes.py). A task's decay is its status — a done issue records
|
||||
# what happened and cannot go false — and a snippet already carries a
|
||||
# richer, location-aware verdict in `data.verification`. On those rows
|
||||
# these stay null, which is also what they mean.
|
||||
#
|
||||
# Most notes should leave all three empty. A null `verify_with` is not a
|
||||
# gap; it is the marker for "this is a decision, there is nothing to go
|
||||
# and check", and the sweep is only worth reading while that holds.
|
||||
verify_with: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
expires_when: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
verified_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True
|
||||
)
|
||||
|
||||
__table_args__ = (
|
||||
Index("ix_notes_tags", "tags", postgresql_using="gin"),
|
||||
@@ -137,6 +168,14 @@ class Note(Base, TimestampMixin, SoftDeleteMixin):
|
||||
"is_task": self.is_task,
|
||||
"note_type": self.note_type or "note",
|
||||
"task_kind": self.task_kind,
|
||||
# Serialized unconditionally, like every other field a given row
|
||||
# kind may not use (recurrence, started_at, the task fields). The
|
||||
# DERIVED "last_verified" label is the one that appears only when
|
||||
# a check exists — a raw projection of the row should not make a
|
||||
# client branch on which keys are present.
|
||||
"verify_with": self.verify_with or "",
|
||||
"expires_when": self.expires_when or "",
|
||||
"verified_at": iso(self.verified_at),
|
||||
"created_at": iso(self.created_at),
|
||||
"updated_at": iso(self.updated_at),
|
||||
}
|
||||
|
||||
@@ -42,8 +42,26 @@ class RetrievalLog(Base):
|
||||
# False=notes, NULL=any.
|
||||
is_task: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
||||
result_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
# How many scored hits this call DROPPED because the session had already
|
||||
# been shown them. NULLABLE, and the null is load-bearing: it means "this
|
||||
# surface does not report suppression", which must not read as "nothing was
|
||||
# suppressed". `result_count == 0` alone conflates two different events —
|
||||
# the ranker found nothing above threshold, and the ranker found something
|
||||
# the reader already had — and only the first says a threshold is too high.
|
||||
# Reading a zero as a ranker decline is how #3311 mis-scoped a milestone;
|
||||
# an unmeasured value that renders as 0 is the same mistake with a nicer
|
||||
# face, so surfaces that filter INSIDE the search leave this null.
|
||||
suppressed_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
top_score: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||
min_score: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||
# The best score the ranker COULD have offered, before the threshold — as
|
||||
# against `top_score`, which is the best it DID offer. They are equal on
|
||||
# any call that returned something, and only this one exists on a call
|
||||
# that returned nothing, which is the only place a bar can be judged from
|
||||
# (#3670). Null means the caller did not measure it, never "nothing was
|
||||
# close": a 0.0 there would read as a corpus with no relevant records at
|
||||
# all, which is an artifact standing in for a measurement.
|
||||
best_available_score: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||
# [{"id": int, "score": float, "rank": int}, ...], highest-first.
|
||||
result_ids: Mapped[list] = mapped_column(JSONB, nullable=False, default=list)
|
||||
duration_ms: Mapped[float | None] = mapped_column(Float, nullable=True)
|
||||
@@ -67,6 +85,7 @@ class RetrievalLog(Base):
|
||||
"project_id": self.project_id,
|
||||
"is_task": self.is_task,
|
||||
"result_count": self.result_count,
|
||||
"suppressed_count": self.suppressed_count,
|
||||
"top_score": self.top_score,
|
||||
"min_score": self.min_score,
|
||||
"result_ids": self.result_ids,
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
from sqlalchemy import BigInteger, Index, Text
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from scribe.models import Base
|
||||
from scribe.models.base import CreatedAtMixin, iso
|
||||
|
||||
SURFACED = "surfaced"
|
||||
PULLED = "pulled"
|
||||
|
||||
|
||||
class RuleUsageEvent(Base, CreatedAtMixin):
|
||||
"""One row per time a rule was SURFACED to the agent, or PULLED in full.
|
||||
|
||||
The sibling `note_usage_events` has had since 2026-07, third in the line
|
||||
after `rule_embeddings` and `rule_versions` — and, like those, it exists
|
||||
because the rule side kept inheriting machinery built for notes and
|
||||
quietly getting the weaker version of it.
|
||||
|
||||
WHY RULES NEED THEIR OWN AND CANNOT SHARE THE NOTE TABLE. Not squeamishness
|
||||
about a polymorphic column — the row shares no note-specific fields and the
|
||||
aggregate readout is the same shape, which is the strongest case for
|
||||
sharing that note #3163 admits. What decides it is IDENTITY AT RESTORE. A
|
||||
note id and a rule id are different namespaces resolved through different
|
||||
maps, and `note_usage_events`'s importer maps `note_id` through
|
||||
`note_id_map` and drops what does not resolve. A rule id parked in that
|
||||
column would come back from a backup silently reattached to whatever note
|
||||
happened to take that number — telemetry that is not merely lost but wrong,
|
||||
and wrong in a way nothing downstream could detect.
|
||||
|
||||
WHAT THIS MEASURES, AND WHY IT DID NOT EXIST. The write-path standing-rule
|
||||
arm is the only retrieval surface in Scribe whose usefulness cannot be
|
||||
observed — and, not coincidentally, the only one that has never declined to
|
||||
fire (#3311: 296 calls, zero zero-result, 100% clearing its threshold).
|
||||
`retrieval_logs` gives it scores; scores say what the ranker thought, never
|
||||
whether the hint landed. Without a pull counter no install can tune the arm
|
||||
from evidence, only from the shape of a histogram.
|
||||
|
||||
Deliberately FK-FREE on `rule_id` and `user_id`, matching `note_usage_events`,
|
||||
`retrieval_logs` and `app_logs` — and diverging from `rule_versions`, which
|
||||
does carry FKs. The difference is what the row is FOR: a version is part of
|
||||
a rule's history and dies with it, while telemetry outlives the row it
|
||||
describes. Deleting a rule must not erase the evidence that it was surfaced
|
||||
forty times and opened never, because that evidence is precisely the case
|
||||
for having deleted it.
|
||||
|
||||
Cells left deliberately empty (note #3163's step 3): no share ACL — rules
|
||||
have none of their own; no soft delete — nothing restores a telemetry row,
|
||||
and the table is append-only; no embedding — an event is not a document.
|
||||
"""
|
||||
|
||||
__tablename__ = "rule_usage_events"
|
||||
|
||||
# BigInteger throughout, where the note twin uses Integer. `rule_id` has to
|
||||
# be, since `rules.id` is BigInteger — and once one column is, matching the
|
||||
# rest costs nothing and keeps the row uniform. A high-churn append-only
|
||||
# telemetry table is a poor place to discover an id ceiling.
|
||||
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
|
||||
user_id: Mapped[int | None] = mapped_column(BigInteger, nullable=True)
|
||||
rule_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
|
||||
|
||||
# 'surfaced' | 'pulled'
|
||||
event: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
|
||||
# Which surface produced it. A CONVENTION, not a fixed vocabulary, and the
|
||||
# note twin's comment explains why this one deliberately does not enumerate
|
||||
# its members: the previous such list went stale, naming a source nothing
|
||||
# wrote while omitting ones that existed, and a half-true enumeration reads
|
||||
# as authoritative in exactly the way that misleads (#2476).
|
||||
# `grep -rn record_rule_pulled\|record_rule_surfaced src/` is the
|
||||
# authoritative list, and unlike a comment it cannot drift.
|
||||
#
|
||||
# The mcp_/rest_ prefix split is load-bearing here for the same reason it is
|
||||
# for notes, and more so: "is this rule dead weight?" is served by any pull,
|
||||
# but "did that injected hint land?" — the question this arm exists to
|
||||
# answer — is served by AGENT pulls only. Never aggregate across the prefix
|
||||
# without saying why.
|
||||
source: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
|
||||
__table_args__ = (
|
||||
# Every readout is "these rule ids, split by event" — a covering
|
||||
# composite beats separate single-column indexes for it.
|
||||
Index("ix_rule_usage_rule_event", "rule_id", "event"),
|
||||
Index("ix_rule_usage_created_at", "created_at"),
|
||||
Index("ix_rule_usage_user_id", "user_id"),
|
||||
)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"id": self.id,
|
||||
"created_at": iso(self.created_at),
|
||||
"user_id": self.user_id,
|
||||
"rule_id": self.rule_id,
|
||||
"event": self.event,
|
||||
"source": self.source,
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import BigInteger, ForeignKey, Text
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from scribe.models import Base
|
||||
from scribe.models.base import CreatedAtMixin, iso
|
||||
|
||||
|
||||
class RuleVersion(Base, CreatedAtMixin):
|
||||
"""One snapshot of a rule's text, taken before an edit overwrote it.
|
||||
|
||||
THE SIBLING NOTES ALREADY HAD. `note_versions` has existed for a long
|
||||
time, and the design-system note calls its history "the changelog". Rules
|
||||
— which BIND BEHAVIOUR on every session that loads them — had nothing, so
|
||||
an edit destroyed what the rule used to say. Rescoping rule 79 on
|
||||
2026-08-29 meant hand-copying the superseded statement into a task log to
|
||||
keep it (#3237). The more consequential record had the weaker protection.
|
||||
|
||||
`CreatedAtMixin`, not `TimestampMixin`: a version is an EVENT. It is
|
||||
written once and never updated, so an `updated_at` on it would be a column
|
||||
that can only ever lie.
|
||||
|
||||
WHAT IS DELIBERATELY DIFFERENT FROM NoteVersion (milestone 323):
|
||||
|
||||
- `user_id` is the ACTOR — who made the edit — where NoteVersion's is the
|
||||
owner, because `update_note` passes an owner-scoped id. For an audit
|
||||
trail over a binding instruction, "who changed this" is the question
|
||||
being asked, and a rule is editable by anyone with rulebook access.
|
||||
- No `pin_kind` / `pin_label`. Those exist so a note's version can survive
|
||||
autosave pruning. Nothing prunes here, so a pin would protect a row that
|
||||
was never at risk.
|
||||
- TEXT ONLY. A rule's Systems and its typed relations are edges with their
|
||||
own lifecycle; folding them in would make one word, "version", mean two
|
||||
different things — the rule's wording, and the rule's place in the
|
||||
graph. `verified_at` is likewise absent: it is a stamp about a check,
|
||||
not a property of the text, and step 2's snapshot is taken before
|
||||
update_rule clears it precisely so the history shows the check that was
|
||||
in force when this wording was written.
|
||||
"""
|
||||
|
||||
__tablename__ = "rule_versions"
|
||||
|
||||
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
|
||||
rule_id: Mapped[int] = mapped_column(
|
||||
BigInteger, ForeignKey("rules.id", ondelete="CASCADE"), index=True
|
||||
)
|
||||
# The actor. SET NULL rather than CASCADE: deleting a user must not erase
|
||||
# the history of the rules they edited — the edit still happened, and the
|
||||
# rule is still binding because of it. NoteVersion cascades because a
|
||||
# note's versions belong to its owner; a rule's belong to the rule.
|
||||
user_id: Mapped[int | None] = mapped_column(
|
||||
BigInteger, ForeignKey("users.id", ondelete="SET NULL"), nullable=True
|
||||
)
|
||||
|
||||
title: Mapped[str] = mapped_column(Text, default="")
|
||||
statement: Mapped[str] = mapped_column(Text, default="")
|
||||
why: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
how_to_apply: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
when_to_apply: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
tier: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
verify_with: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
expires_when: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
|
||||
def to_dict(self, include_text: bool = True) -> dict:
|
||||
"""The row. `include_text=False` gives the listing form.
|
||||
|
||||
A rule's `statement` and `why` run to thousands of characters — rule
|
||||
149's `why` alone is longer than most notes — so a history LIST that
|
||||
carried every field would be unreadable and expensive. The listing
|
||||
answers "when, and by whom"; opening one answers "and what did it
|
||||
say". Same split NoteVersion makes with `include_body`.
|
||||
"""
|
||||
out: dict = {
|
||||
"id": self.id,
|
||||
"rule_id": self.rule_id,
|
||||
"user_id": self.user_id,
|
||||
"title": self.title,
|
||||
"created_at": iso(self.created_at),
|
||||
}
|
||||
if include_text:
|
||||
out.update({
|
||||
"statement": self.statement,
|
||||
"why": self.why or "",
|
||||
"how_to_apply": self.how_to_apply or "",
|
||||
"when_to_apply": self.when_to_apply or "",
|
||||
"tier": self.tier or "",
|
||||
"verify_with": self.verify_with or "",
|
||||
"expires_when": self.expires_when or "",
|
||||
})
|
||||
return out
|
||||
@@ -10,6 +10,61 @@ async def health():
|
||||
return jsonify({"status": "ok"})
|
||||
|
||||
|
||||
def build_version_payload() -> dict:
|
||||
"""What build is this, separated into the values that answer different
|
||||
questions (rule 149).
|
||||
|
||||
UNTIL 2026-08-31 THIS RETURNED THE CHANNEL. `BUILD_VERSION` in CI was
|
||||
literally "dev" / "main" / the tag, so a running instance reported
|
||||
`{"version": "main"}` — a channel name sitting where a build identifier
|
||||
belongs. The cost was concrete: with a deploy misbehaving, nothing on the
|
||||
instance could say which commit was serving it, and the one endpoint whose
|
||||
job that is answered with the name of a branch.
|
||||
|
||||
The three values, and why they are three:
|
||||
|
||||
- `version` — the NAME, `YYYY.MM.DD.HHMM` from COMMIT time. Answers "is
|
||||
this the same code?", so two channels carrying one commit report the
|
||||
same string.
|
||||
- `build` — the ORDERING KEY, minutes since 2020-01-01 from BUILD time.
|
||||
Answers "may this be installed over that?". The ONLY value anything may
|
||||
compare; it is monotonic by construction, which neither a commit count
|
||||
(branches diverge) nor a commit time (rebuilds go backwards) is.
|
||||
- `channel` — its own field, never folded into the name.
|
||||
|
||||
Plus `commit`, so the artifact's claim about itself can be checked against
|
||||
the `:<sha>` it was published under (rule 145).
|
||||
|
||||
ABSENT RATHER THAN EMPTY when unknown. A local build has no ordering key
|
||||
and no channel, and saying so is honest; emitting `""` or a placeholder
|
||||
would let it claim a position in an update order it is not part of. A
|
||||
reader must treat a missing `build` as "cannot be ordered", not as zero.
|
||||
"""
|
||||
payload: dict = {"version": os.environ.get("APP_VERSION", "dev")}
|
||||
|
||||
# Reported verbatim, never validated against an enum — a build claiming
|
||||
# something unexpected is better shown than dropped (rule 149).
|
||||
for key, env in (("channel", "APP_CHANNEL"), ("commit", "APP_COMMIT")):
|
||||
value = (os.environ.get(env) or "").strip()
|
||||
if value:
|
||||
payload[key] = value
|
||||
|
||||
raw_key = (os.environ.get("APP_BUILD_KEY") or "").strip()
|
||||
if raw_key:
|
||||
try:
|
||||
# An INTEGER, not a string. A string ordering key is how a
|
||||
# comparison silently becomes lexicographic — "9" > "10" — which
|
||||
# is the same class of fault as folding the channel in: it reads
|
||||
# fine and orders wrong.
|
||||
payload["build"] = int(raw_key)
|
||||
except ValueError:
|
||||
# A malformed key is omitted rather than passed through: a reader
|
||||
# that cannot order is correct, one that orders on garbage is not.
|
||||
pass
|
||||
|
||||
return payload
|
||||
|
||||
|
||||
@api.route("/version")
|
||||
async def version():
|
||||
return jsonify({"version": os.environ.get("APP_VERSION", "dev")})
|
||||
return jsonify(build_version_payload())
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Unified Knowledge endpoint — notes, tasks, plans, and processes in one queryable feed."""
|
||||
"""Unified Knowledge endpoint — every record kind in one queryable feed."""
|
||||
import logging
|
||||
|
||||
from quart import Blueprint, jsonify, request
|
||||
@@ -6,12 +6,18 @@ from quart import Blueprint, jsonify, request
|
||||
from scribe.auth import get_current_user_id, login_required
|
||||
from scribe.routes.utils import parse_pagination
|
||||
from scribe.services.access import label_shared_items
|
||||
from scribe.services.knowledge import FACET_TYPES
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
knowledge_bp = Blueprint("knowledge", __name__, url_prefix="/api/knowledge")
|
||||
|
||||
_VALID_TYPES = {"note", "task", "plan", "process"}
|
||||
# Derived from the service's facet table, never re-listed here. This set was a
|
||||
# hand-kept copy and had drifted three kinds behind it: it admitted `plan`
|
||||
# (retired in 0066) and rejected `issue` (shipped in 0065, 435 rows) and
|
||||
# `snippet` — so the browse surface could not filter to the kinds it was
|
||||
# already rendering badges for (#3128).
|
||||
_VALID_TYPES = FACET_TYPES
|
||||
_VALID_SORTS = {"modified", "created", "alpha", "type"}
|
||||
|
||||
|
||||
@@ -21,7 +27,9 @@ async def list_knowledge():
|
||||
"""Return paginated knowledge objects with optional filtering.
|
||||
|
||||
Query params:
|
||||
type — one of note|task|plan|process (omit for all)
|
||||
type — a facet from services.knowledge._FACETS: a record type
|
||||
(note|process|snippet) or a task kind (task for any,
|
||||
else work|issue|spike|plan). Omit for all.
|
||||
tags — comma-separated tag filter (AND logic)
|
||||
sort — modified|created|alpha|type (default: modified)
|
||||
q — search query (semantic when provided, keyword fallback)
|
||||
@@ -127,7 +135,7 @@ async def get_knowledge_batch():
|
||||
@knowledge_bp.route("/tags", methods=["GET"])
|
||||
@login_required
|
||||
async def list_knowledge_tags():
|
||||
"""Return all tags used across knowledge objects (excludes tasks)."""
|
||||
"""Return all tags used across knowledge objects, narrowed to one facet."""
|
||||
uid = get_current_user_id()
|
||||
note_type = request.args.get("type", "").strip().lower() or None
|
||||
|
||||
|
||||
@@ -19,7 +19,10 @@ from scribe.services.notes import (
|
||||
get_note_for_user,
|
||||
get_or_create_note_by_title,
|
||||
list_notes,
|
||||
mark_note_verified,
|
||||
notes_due_for_verification,
|
||||
update_note,
|
||||
verification_row,
|
||||
)
|
||||
from scribe.services.note_drafts import upsert_draft, get_draft, delete_draft
|
||||
from scribe.services import dedup as dedup_svc
|
||||
@@ -112,6 +115,8 @@ async def create_note_route():
|
||||
priority=priority,
|
||||
due_date=due_date,
|
||||
note_type=note_type,
|
||||
verify_with=data.get("verify_with"),
|
||||
expires_when=data.get("expires_when"),
|
||||
)
|
||||
except ValueError as e:
|
||||
return jsonify({"error": str(e)}), 400
|
||||
@@ -248,7 +253,14 @@ async def update_note_route(note_id: int):
|
||||
owner_uid = note_obj.user_id
|
||||
data = await request.get_json()
|
||||
fields = {}
|
||||
for key in ("title", "body", "description", "parent_id", "project_id", "milestone_id", "status", "priority", "note_type"):
|
||||
for key in (
|
||||
"title", "body", "description", "parent_id", "project_id",
|
||||
"milestone_id", "status", "priority", "note_type",
|
||||
# A cleared form input arrives as "" and the service reads that as
|
||||
# NULL (NULLABLE_NOTE_TEXT), so this door expresses "remove the check"
|
||||
# with its own idiom and needs no `clear` list (milestone 317).
|
||||
"verify_with", "expires_when",
|
||||
):
|
||||
if key in data:
|
||||
fields[key] = data[key]
|
||||
if "due_date" in data:
|
||||
@@ -490,3 +502,62 @@ async def graph_route():
|
||||
shared_tags = request.args.get("shared_tags", "false").lower() == "true"
|
||||
graph = await build_note_graph(uid, project_id=project_id, include_shared_tags=shared_tags)
|
||||
return jsonify(graph)
|
||||
|
||||
|
||||
# ── The staleness sweep (milestone 317) ──────────────────────────────────────
|
||||
# The web half of notes_due_for_verification / mark_note_verified. Same
|
||||
# contract as the MCP door and the rules routes beside it — the service holds
|
||||
# the behaviour, these two just parse and serialise.
|
||||
|
||||
@notes_bp.route("/due-for-verification", methods=["GET"])
|
||||
@login_required
|
||||
async def notes_due_route():
|
||||
"""Notes that carry a check, oldest verification first, never-checked top.
|
||||
|
||||
Query params: older_than_days, project_id, never_only. A note with no
|
||||
`verify_with` never appears — it is a decision, not a fact.
|
||||
"""
|
||||
uid = get_current_user_id()
|
||||
args = request.args
|
||||
try:
|
||||
older = int(args.get("older_than_days", 0) or 0)
|
||||
project = int(args.get("project_id", 0) or 0)
|
||||
except ValueError:
|
||||
return jsonify({"error": "older_than_days and project_id must be integers"}), 400
|
||||
try:
|
||||
notes = await notes_due_for_verification(
|
||||
uid,
|
||||
older_than_days=older,
|
||||
project_id=project or None,
|
||||
never_only=args.get("never_only", "").lower() in ("1", "true", "yes"),
|
||||
)
|
||||
except ValueError as exc:
|
||||
# A 400, not a silently narrowed result: a filter that quietly answers
|
||||
# a different question is the failure this whole surface exists to
|
||||
# catch.
|
||||
return jsonify({"error": str(exc)}), 400
|
||||
return jsonify({
|
||||
"notes": [verification_row(n) for n in notes],
|
||||
"total": len(notes),
|
||||
})
|
||||
|
||||
|
||||
@notes_bp.route("/<int:note_id>/verify", methods=["POST"])
|
||||
@login_required
|
||||
async def mark_note_verified_route(note_id: int):
|
||||
"""Record that the note's check was run. Body: {"still_true": bool}.
|
||||
|
||||
`still_true: false` writes nothing — a note whose check failed is wrong,
|
||||
not in a recordable state — so it keeps its place at the top of the sweep.
|
||||
"""
|
||||
data = await request.get_json() or {}
|
||||
uid = get_current_user_id()
|
||||
still_true = bool(data.get("still_true", True))
|
||||
note = await mark_note_verified(note_id, uid, still_true)
|
||||
if note is None:
|
||||
return jsonify({
|
||||
"error": "note not found, not writable by you, or carries no verify_with"
|
||||
}), 404
|
||||
payload = verification_row(note)
|
||||
payload["verified"] = still_true
|
||||
return jsonify(payload)
|
||||
|
||||
@@ -101,6 +101,46 @@ async def autoinject_retrieve():
|
||||
return jsonify(result)
|
||||
|
||||
|
||||
@plugin_bp.get("/tool-rules")
|
||||
@login_required
|
||||
async def pre_tool_rules():
|
||||
"""Standing rules for the plugin's PreToolUse hook on ACTIONS (#3476).
|
||||
|
||||
Answers "does a recorded rule speak to the command about to be run?" — the
|
||||
sibling of /prior-art, which can only answer that question about a code
|
||||
write. Rules about which tool to reach for (don't curl the forge, don't
|
||||
stand up a stack, don't run the suite locally) had no retrieval surface at
|
||||
all before this, which is why they all had to live in the resident preload.
|
||||
|
||||
Titles + trigger only, never the statement: the hint says a rule may apply
|
||||
and hands over `get_rule(id)`. One rule at most (RULEHINT_LIMIT), and empty
|
||||
most of the time.
|
||||
|
||||
Query:
|
||||
tool (str) — the tool about to run, e.g. `Bash`. Used in
|
||||
the hint's wording, not in the search: a
|
||||
rule is about the action, not the harness.
|
||||
command (str) — the command about to run; the semantic query.
|
||||
Absent or blank → empty, no search.
|
||||
repo (optional) — working repo remote, resolved to the bound
|
||||
project exactly as /retrieve and /prior-art.
|
||||
exclude_rule_ids (opt) — comma-separated rule ids already surfaced
|
||||
this session. SHARED with /prior-art's
|
||||
ledger on purpose: one session keeps one
|
||||
list, so a rule named by either arm is not
|
||||
re-offered by the other.
|
||||
"""
|
||||
tool = (request.args.get("tool") or "tool").strip()
|
||||
command = request.args.get("command") or ""
|
||||
project_id, _repo, _unbound = await _project_scope()
|
||||
exclude_rule_ids = _int_list(request.args.get("exclude_rule_ids"))
|
||||
result = await plugin_ctx_svc.build_tool_rule_hint(
|
||||
g.user.id, tool, command,
|
||||
project_id=project_id, exclude_rule_ids=exclude_rule_ids,
|
||||
)
|
||||
return jsonify(result)
|
||||
|
||||
|
||||
@plugin_bp.get("/prior-art")
|
||||
@login_required
|
||||
async def write_path_prior_art():
|
||||
@@ -138,6 +178,11 @@ async def write_path_prior_art():
|
||||
or `canon:<snippet_id>`) already named this
|
||||
session by the ledger arm (#2900); its own
|
||||
channel, like the two above.
|
||||
rules_etag (opt) — the marker the session was given when it loaded
|
||||
its always-on rules (milestone 323). Sent back
|
||||
so the server can say whether those rules have
|
||||
MOVED since. Absent means the hook has nothing
|
||||
stored, which is silence, not a mismatch.
|
||||
shapes (opt) — comma-separated `kind:name` definitions the hook
|
||||
found in (or enclosing) the payload, kind being
|
||||
css|sym. The shape ledger's write-path feed
|
||||
@@ -157,6 +202,7 @@ async def write_path_prior_art():
|
||||
p.strip() for p in (request.args.get("exclude_derive") or "").split(",") if p.strip()
|
||||
]
|
||||
exclude_rule_ids = _int_list(request.args.get("exclude_rule_ids"))
|
||||
rules_etag = (request.args.get("rules_etag") or "").strip()
|
||||
shapes = _parse_shapes(request.args.get("shapes") or "")
|
||||
api_key = getattr(g, "api_key", None)
|
||||
may_stamp = api_key is None or getattr(api_key, "scope", "") == "write"
|
||||
@@ -168,6 +214,7 @@ async def write_path_prior_art():
|
||||
repo_key=repo_bindings_svc.normalize_repo_key(repo) if repo else "",
|
||||
exclude_derive=exclude_derive,
|
||||
exclude_rule_ids=exclude_rule_ids,
|
||||
rules_etag=rules_etag,
|
||||
)
|
||||
return jsonify(result)
|
||||
|
||||
|
||||
@@ -10,6 +10,9 @@ from quart import Blueprint, jsonify, request
|
||||
from scribe.auth import get_current_user_id, login_required
|
||||
import scribe.services.rulebooks as rulebooks_svc
|
||||
from scribe.services.trash import delete as trash_delete
|
||||
from scribe.services.rule_usage import (
|
||||
empty_rule_usage, record_rule_pulled, usage_for_rules,
|
||||
)
|
||||
|
||||
rulebooks_bp = Blueprint("rulebooks", __name__, url_prefix="/api")
|
||||
|
||||
@@ -136,13 +139,24 @@ async def list_rules():
|
||||
except ValueError:
|
||||
return jsonify({"error": "rulebook_id, topic_id, project_id must be integers"}), 400
|
||||
|
||||
uid = get_current_user_id()
|
||||
rows = await rulebooks_svc.list_rules(
|
||||
user_id=get_current_user_id(),
|
||||
user_id=uid,
|
||||
rulebook_id=rulebook_id,
|
||||
topic_id=topic_id,
|
||||
project_id=project_id,
|
||||
)
|
||||
return jsonify({"rules": [r.to_dict() for r in rows]})
|
||||
items = [r.to_dict() for r in rows]
|
||||
# One aggregate for the whole page — a per-row lookup here would be N+1 by
|
||||
# construction, the same reason the snippet list does it this way. Every
|
||||
# row gets the key, zero-filled, so the UI renders "never surfaced" rather
|
||||
# than having to treat a missing field as a state. That matters more here
|
||||
# than for snippets: every rule on every install predates this table, so
|
||||
# for a while the zero-filled shape IS the common case.
|
||||
usage = await usage_for_rules([int(it["id"]) for it in items])
|
||||
for it in items:
|
||||
it["usage"] = usage.get(int(it["id"]), empty_rule_usage())
|
||||
return jsonify({"rules": items})
|
||||
|
||||
|
||||
@rulebooks_bp.post("/rulebook-topics/<int:topic_id>/rules")
|
||||
@@ -182,6 +196,11 @@ async def get_rule(rule_id: int):
|
||||
rule = await rulebooks_svc.get_rule(rule_id, uid)
|
||||
if rule is None:
|
||||
return jsonify({"error": "rule not found"}), 404
|
||||
# `rest_` rather than `mcp_`, and the prefix is load-bearing: "is this rule
|
||||
# dead weight?" is served by any pull, but "did that injected hint land?"
|
||||
# — the question this arm exists to answer — is served by AGENT pulls only.
|
||||
# A person clicking through the rule list says nothing about the hint.
|
||||
record_rule_pulled(user_id=uid, rule_id=int(rule.id), source="rest_rule")
|
||||
return jsonify(await rulebooks_svc.rule_detail(uid, rule))
|
||||
|
||||
|
||||
@@ -206,6 +225,42 @@ async def update_rule(rule_id: int):
|
||||
return jsonify(await rulebooks_svc.rule_detail(uid, rule, data.get("system_ids")))
|
||||
|
||||
|
||||
@rulebooks_bp.get("/rules/<int:rule_id>/versions")
|
||||
@login_required
|
||||
async def list_rule_versions(rule_id: int):
|
||||
"""A rule's edit history, newest first.
|
||||
|
||||
Listing form only — a rule's `statement` and `why` run to thousands of
|
||||
characters, so a history list carrying every field would be unreadable
|
||||
and expensive to send. Open one for the text.
|
||||
"""
|
||||
uid = get_current_user_id()
|
||||
versions = await rulebooks_svc.list_rule_versions(rule_id, uid)
|
||||
if versions is None:
|
||||
return jsonify({"error": "rule not found"}), 404
|
||||
return jsonify({
|
||||
"versions": [v.to_dict(include_text=False) for v in versions],
|
||||
})
|
||||
|
||||
|
||||
@rulebooks_bp.get("/rules/<int:rule_id>/versions/<int:version_id>")
|
||||
@login_required
|
||||
async def get_rule_version(rule_id: int, version_id: int):
|
||||
"""One snapshot in full — what the rule said before that edit."""
|
||||
uid = get_current_user_id()
|
||||
version = await rulebooks_svc.get_rule_version(rule_id, version_id, uid)
|
||||
if version is None:
|
||||
return jsonify({"error": "version not found"}), 404
|
||||
return jsonify(version.to_dict(include_text=True))
|
||||
|
||||
|
||||
# NO restore route, deliberately (milestone 323). A note version can be
|
||||
# restored; a binding instruction should not be revertible in one click.
|
||||
# Putting a rewrite back goes through update_rule, which takes its own
|
||||
# snapshot and leaves the undo in the history like any other edit — a silent
|
||||
# revert would erase the only record of why the rewrite happened.
|
||||
|
||||
|
||||
@rulebooks_bp.post("/rules/<int:rule_id>/relations")
|
||||
@login_required
|
||||
async def relate_rules(rule_id: int):
|
||||
|
||||
@@ -44,17 +44,20 @@ async def search_route():
|
||||
project_id = request.args.get("project_id", type=int)
|
||||
|
||||
t0 = time.perf_counter()
|
||||
report: dict = {}
|
||||
results = await semantic_search_notes(
|
||||
uid, q, limit=limit, is_task=is_task, threshold=_REST_SEARCH_THRESHOLD,
|
||||
project_id=project_id, system_id=system_id,
|
||||
# The user typed this, so it reaches everything they may read.
|
||||
scope="read",
|
||||
report=report,
|
||||
)
|
||||
record_retrieval(
|
||||
user_id=uid, source="rest_search", query=q,
|
||||
threshold=_REST_SEARCH_THRESHOLD, limit=limit,
|
||||
project_id=project_id, is_task=is_task, results=results,
|
||||
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
||||
best_available=report.get("best_available_score"),
|
||||
)
|
||||
owners = await owner_names_for(
|
||||
{int(note.user_id) for _s, note in results if note.user_id != uid}
|
||||
|
||||
@@ -6,20 +6,63 @@ write that never errors and never lands (the #2663 GC footgun). This module is
|
||||
the one place that gets the pattern right: strong references in ``_pending``,
|
||||
discarded on completion, with failures logged at WARNING instead of vanishing.
|
||||
|
||||
``note_usage`` and ``retrieval_telemetry`` predate this module and carry their
|
||||
own copies with bespoke canary semantics; new fire-and-forget callers use this
|
||||
instead of writing a fourth copy.
|
||||
``retrieval_telemetry`` predates this module and keeps its own copy, because
|
||||
its canary is a genuinely different shape — one process-wide flag and no
|
||||
AppLog row. ``note_usage`` and ``rule_usage`` share ``report_telemetry_failure``
|
||||
below. New fire-and-forget callers use ``spawn`` rather than writing another
|
||||
copy of the strong-reference dance.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import traceback
|
||||
from collections.abc import Coroutine
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_pending: set[asyncio.Task] = set()
|
||||
|
||||
# Sites that have already dropped their once-per-process AppLog row, keyed
|
||||
# "<subsystem>:<site>". A readout can run on every list render — without this,
|
||||
# a broken table turns the error log into a firehose that buries the finding it
|
||||
# exists to surface.
|
||||
_reported: set[str] = set()
|
||||
|
||||
|
||||
async def report_telemetry_failure(subsystem: str, site: str) -> None:
|
||||
"""Make a swallowed telemetry failure visible. Call from an except block.
|
||||
|
||||
WARNING to the process log every time; one AppLog error row per process per
|
||||
(subsystem, site) so the admin UI shows the outage without host access.
|
||||
|
||||
THIS IS NOT DECORATION. #2663 is the record of a telemetry subsystem running
|
||||
at zero for weeks — every counter reading empty, indistinguishable from
|
||||
"nobody uses this" — because every failure went to ``logger.debug``. A
|
||||
subsystem whose failures are all invisible cannot report its own death.
|
||||
|
||||
The AppLog write is itself guarded: when the database is down it fails too,
|
||||
and that is fine. The WARNING already said so, and a canary must never take
|
||||
down the surface it watches.
|
||||
"""
|
||||
logger.warning("%s telemetry %s failed", subsystem, site, exc_info=True)
|
||||
key = f"{subsystem}:{site}"
|
||||
if key in _reported:
|
||||
return
|
||||
_reported.add(key)
|
||||
try:
|
||||
from scribe.services.logging import log_error
|
||||
|
||||
await log_error(
|
||||
endpoint=subsystem,
|
||||
error_type=f"{subsystem}_{site}_failed",
|
||||
error_message=f"{subsystem} telemetry {site} is failing; "
|
||||
"usage counters will read zero until this is fixed",
|
||||
traceback=traceback.format_exc(),
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("%s canary write failed", subsystem, exc_info=True)
|
||||
|
||||
|
||||
def spawn(coro: Coroutine, *, site: str) -> None:
|
||||
"""Schedule ``coro`` fire-and-forget; ``site`` names it in failure logs.
|
||||
|
||||
+318
-10
@@ -9,8 +9,10 @@ from scribe.models.note import Note
|
||||
from scribe.models.note_draft import NoteDraft
|
||||
from scribe.models.note_supersession import NoteSupersession
|
||||
from scribe.models.note_version import NoteVersion
|
||||
from scribe.models.rule_version import RuleVersion
|
||||
from scribe.models.design_system import DesignSystem, DesignToken
|
||||
from scribe.models.note_usage import NoteUsageEvent
|
||||
from scribe.models.rule_usage import RuleUsageEvent
|
||||
from scribe.models.canonical_system import CanonicalSystem
|
||||
from scribe.models.rulebook import RuleRelation, rule_systems as rule_systems_t
|
||||
from scribe.models.code_shape import CodeShape, CodeShapeEvent, CodeShapeUse
|
||||
@@ -50,8 +52,23 @@ logger = logging.getLogger(__name__)
|
||||
# ones (reference/hook) travel too, cheaply, and the next refresh refreshes them.
|
||||
# v10 (2026-08) added projects.inception + project_rulebook_exclusions
|
||||
# (milestone 297): the WHY a project inherits what it does, and its opt-outs.
|
||||
# v11 (2026-08) added the note verification trio — notes.verify_with /
|
||||
# expires_when / verified_at (milestone 317).
|
||||
# v12 (2026-08) closed #3182: the nine Note columns that had been missing for
|
||||
# years (note_type, task_kind, arose_from_id, data, description, the
|
||||
# recurrence pair, started_at, completed_at), plus milestones.body — which IS
|
||||
# the plan — and repo_bindings.ref. Until v12 a restore reported success and
|
||||
# handed back a corpus with every snippet and process flattened into a plain
|
||||
# note, every issue and spike into `work`, and every plan reduced to a title.
|
||||
# _COLUMN_EXCLUSIONS and its guard landed with it, so the next such column
|
||||
# fails the build instead.
|
||||
# v13 (2026-08) added rule_versions — a rule's edit history (milestone 323).
|
||||
# v14 (2026-09) added rule_usage_events — the rule twin of note_usage_events
|
||||
# (milestone 333). Carrying it is the WHOLE REASON the table is separate: the
|
||||
# note importer maps note_id through note_id_map, so a rule id parked there
|
||||
# would restore attached to whatever note took that number.
|
||||
# Bump when the serialized schema changes.
|
||||
BACKUP_VERSION = 10
|
||||
BACKUP_VERSION = 14
|
||||
|
||||
# Every table this backup carries, by its REAL name. Paired with _NOT_INCLUDED
|
||||
# below, these two lists must together account for the entire schema — which is
|
||||
@@ -77,6 +94,14 @@ _BACKED_UP = [
|
||||
"canonical_systems",
|
||||
# v10 (2026-08): a rule's area tag and its typed edges (milestone 307).
|
||||
"rule_systems", "rule_relations",
|
||||
# v13 (2026-08): a rule's edit history (milestone 323). note_versions has
|
||||
# always travelled; its sibling has no excuse not to.
|
||||
"rule_versions",
|
||||
# v14 (2026-09): rule usage telemetry (milestone 333). Same argument
|
||||
# note_usage_events makes for itself — pull-through is only ever
|
||||
# accumulated, so a restore that dropped it would silently reset the
|
||||
# measurement to zero while everything still looked fine.
|
||||
"rule_usage_events",
|
||||
]
|
||||
|
||||
# Tables intentionally NOT in the backup, surfaced in the payload so the gap is
|
||||
@@ -108,6 +133,88 @@ _NOT_INCLUDED = [
|
||||
]
|
||||
|
||||
|
||||
# Columns a backed-up table deliberately does NOT export, per table. Paired
|
||||
# with the column-coverage guard in tests/test_services_backup.py, this is
|
||||
# _NOT_INCLUDED's shape one level down — and it exists because the table guard
|
||||
# could not see the failure it was written to stop.
|
||||
#
|
||||
# #2293 was six whole tables missing. #3182 was NINE COLUMNS missing from a
|
||||
# table that had been "covered" for years: note_type and task_kind, so every
|
||||
# snippet and process restored as a plain note and every issue and spike as
|
||||
# `work`; arose_from_id, so every provenance edge vanished; the recurrence
|
||||
# pair, so recurring tasks stopped recurring; milestones.body, which IS the
|
||||
# plan; repo_bindings.ref, the branch a ledger follows. Every one arrived the
|
||||
# same way — a column added to the model and the migration, both of which fail
|
||||
# loudly, and then never added to the serialiser, which fails silently.
|
||||
#
|
||||
# So: a new column on a backed-up table now fails the build unless it is either
|
||||
# exported or named here with a reason. "I forgot" is no longer expressible.
|
||||
_COLUMN_EXCLUSIONS: dict[str, set[str]] = {
|
||||
# Trash is not exported at all, so neither is the batch id that groups a
|
||||
# deletion for restore(). Uniform across every soft-deletable table.
|
||||
"users": set(),
|
||||
"projects": {
|
||||
"deleted_at", "deleted_batch_id",
|
||||
# Credentials-adjacent, same reasoning as api_keys and
|
||||
# forge_connections: a restored project falls back to keyring-by-host
|
||||
# resolution, which is the documented unpinned behaviour (#2778).
|
||||
"forge_connection_id",
|
||||
},
|
||||
"milestones": {"deleted_at", "deleted_batch_id"},
|
||||
"notes": {"deleted_at", "deleted_batch_id"},
|
||||
"task_logs": set(),
|
||||
"note_drafts": set(),
|
||||
"note_versions": set(),
|
||||
"settings": set(),
|
||||
"rulebooks": {"deleted_at", "deleted_batch_id"},
|
||||
"rulebook_topics": {"deleted_at", "deleted_batch_id"},
|
||||
"rules": {"deleted_at", "deleted_batch_id"},
|
||||
"systems": {
|
||||
"deleted_at", "deleted_batch_id",
|
||||
# Travels as `canonical_slug`: the catalog is global and its ids are
|
||||
# per-install, so an id would restore pointing at whatever area
|
||||
# happened to land on that number.
|
||||
"canonical_id",
|
||||
"created_at", "updated_at",
|
||||
},
|
||||
"canonical_systems": {
|
||||
"deleted_at", "deleted_batch_id",
|
||||
# Matched on SLUG at restore, so a target install that already seeded
|
||||
# the standard vocabulary reuses its own rows rather than colliding.
|
||||
"id", "created_at", "updated_at",
|
||||
},
|
||||
# Edge tables: the id is regenerated on insert, and the pair IS the row.
|
||||
"record_systems": {"id", "created_at"},
|
||||
"note_supersessions": {"id", "created_at"},
|
||||
"rule_relations": {"id", "created_at"},
|
||||
"note_usage_events": {"id"},
|
||||
# Same as the note twin: the surrogate key is re-issued on insert.
|
||||
"rule_usage_events": {"id"},
|
||||
"design_systems": {"deleted_at", "deleted_batch_id", "created_at", "updated_at"},
|
||||
"design_tokens": {"deleted_at", "deleted_batch_id", "created_at", "updated_at"},
|
||||
"repo_bindings": {"id", "created_at", "updated_at"},
|
||||
# Everything travels. A version row IS the audit trail, so a column left
|
||||
# behind is a fact about a binding instruction that no longer exists
|
||||
# anywhere.
|
||||
"rule_versions": set(),
|
||||
# Serialised via the model's own to_dict(), so a column reaches the backup
|
||||
# the moment it reaches that method — and the guard still catches one that
|
||||
# reaches neither.
|
||||
"code_shapes": {
|
||||
# The PROPOSER's standing suggestion for an unclassified row, not a
|
||||
# judgment: "looks like an instance of #N", or "repeats with no canon".
|
||||
# Every refresh recomputes it and a judgment clears it, so carrying it
|
||||
# would restore stale machine guesses over a tree the proposer has not
|
||||
# seen. Same reasoning as code_shape_consumers in _NOT_INCLUDED —
|
||||
# derived data is regenerated, never restored.
|
||||
"proposed_snippet_id", "proposal_basis", "proposal_score",
|
||||
"proposal_group", "proposed_at", "proposed_sha",
|
||||
},
|
||||
"code_shape_events": set(),
|
||||
"code_shape_uses": set(),
|
||||
}
|
||||
|
||||
|
||||
def _dt(val: str | None) -> datetime:
|
||||
return datetime.fromisoformat(val) if val else datetime.now(timezone.utc)
|
||||
|
||||
@@ -226,6 +333,17 @@ def _usage_event_rows(rows) -> list[dict]:
|
||||
]
|
||||
|
||||
|
||||
def _rule_usage_event_rows(rows) -> list[dict]:
|
||||
return [
|
||||
{
|
||||
"user_id": r.user_id, "rule_id": r.rule_id, "event": r.event,
|
||||
"source": r.source,
|
||||
"created_at": r.created_at.isoformat() if r.created_at else None,
|
||||
}
|
||||
for r in rows
|
||||
]
|
||||
|
||||
|
||||
def _code_shape_rows(rows) -> list[dict]:
|
||||
return [r.to_dict() for r in rows]
|
||||
|
||||
@@ -240,7 +358,13 @@ def _code_shape_use_rows(rows) -> list[dict]:
|
||||
|
||||
def _repo_binding_rows(rows) -> list[dict]:
|
||||
return [
|
||||
{"user_id": r.user_id, "project_id": r.project_id, "repo_key": r.repo_key}
|
||||
# `ref` is the branch this binding's ledger follows (#2873). Without
|
||||
# it a restored binding silently falls back to the default branch, and
|
||||
# the shape ledger starts accounting for a different tree (#3182).
|
||||
{
|
||||
"user_id": r.user_id, "project_id": r.project_id,
|
||||
"repo_key": r.repo_key, "ref": r.ref,
|
||||
}
|
||||
for r in rows
|
||||
]
|
||||
|
||||
@@ -282,6 +406,11 @@ def _milestone_rows(rows) -> list[dict]:
|
||||
{
|
||||
"id": m.id, "user_id": m.user_id, "project_id": m.project_id,
|
||||
"title": m.title, "description": m.description, "status": m.status,
|
||||
# THE PLAN. A milestone IS the plan (0066) and `body` is its
|
||||
# design and intent; `description` is only the one-line summary.
|
||||
# Dropping this restored every plan as a title with no reasoning
|
||||
# behind it (#3182).
|
||||
"body": m.body,
|
||||
"order_index": m.order_index,
|
||||
"created_at": m.created_at.isoformat(),
|
||||
"updated_at": m.updated_at.isoformat(),
|
||||
@@ -294,12 +423,45 @@ def _note_rows(rows) -> list[dict]:
|
||||
return [
|
||||
{
|
||||
"id": n.id, "user_id": n.user_id, "title": n.title, "body": n.body,
|
||||
"description": n.description,
|
||||
"tags": n.tags or [], "parent_id": n.parent_id,
|
||||
"project_id": n.project_id, "milestone_id": n.milestone_id,
|
||||
"status": n.status, "priority": n.priority,
|
||||
"due_date": n.due_date.isoformat() if n.due_date else None,
|
||||
"created_at": n.created_at.isoformat(),
|
||||
"updated_at": n.updated_at.isoformat(),
|
||||
# WHAT KIND OF RECORD THIS IS — both typing axes (#3182). Missing
|
||||
# until now, which meant a restore reported success and handed back
|
||||
# a corpus with every snippet and process flattened into a plain
|
||||
# note and every issue and spike into `work`. Nothing recomputes
|
||||
# these; the vocabulary is simply gone.
|
||||
"note_type": n.note_type,
|
||||
"task_kind": n.task_kind,
|
||||
# Provenance — which record caused this one. Re-mapped in the
|
||||
# second pass beside parent_id, never here: the value is an id in
|
||||
# the SOURCE database.
|
||||
"arose_from_id": n.arose_from_id,
|
||||
# The queryable mirror. The only one of these that would self-heal
|
||||
# (backfill_snippet_data rebuilds it from the body at startup), but
|
||||
# a restore should not hand back a corpus that needs a restart to
|
||||
# become searchable by location.
|
||||
"data": n.data,
|
||||
# Lifecycle: when the work actually started and finished, and the
|
||||
# recurrence rule that makes a task come back. Without these a
|
||||
# restored recurring task simply stops recurring.
|
||||
"started_at": n.started_at.isoformat() if n.started_at else None,
|
||||
"completed_at": n.completed_at.isoformat() if n.completed_at else None,
|
||||
"recurrence_rule": n.recurrence_rule,
|
||||
"recurrence_next_spawn_at": (
|
||||
n.recurrence_next_spawn_at.isoformat()
|
||||
if n.recurrence_next_spawn_at else None
|
||||
),
|
||||
# The verification trio (milestone 317, migration 0092). Operator
|
||||
# judgment — "somebody checked this fact, and this is when" —
|
||||
# which nothing can recompute.
|
||||
"verify_with": n.verify_with,
|
||||
"expires_when": n.expires_when,
|
||||
"verified_at": n.verified_at.isoformat() if n.verified_at else None,
|
||||
}
|
||||
for n in rows
|
||||
]
|
||||
@@ -342,6 +504,23 @@ def _note_version_rows(rows) -> list[dict]:
|
||||
]
|
||||
|
||||
|
||||
def _rule_version_rows(rows) -> list[dict]:
|
||||
"""A rule's edit history. Sibling of _note_version_rows, and it travels for
|
||||
the same reason: a version is the only record of what a binding
|
||||
instruction used to say, and nothing can recompute it."""
|
||||
return [
|
||||
{
|
||||
"id": rv.id, "rule_id": rv.rule_id, "user_id": rv.user_id,
|
||||
"title": rv.title, "statement": rv.statement, "why": rv.why,
|
||||
"how_to_apply": rv.how_to_apply, "when_to_apply": rv.when_to_apply,
|
||||
"tier": rv.tier, "verify_with": rv.verify_with,
|
||||
"expires_when": rv.expires_when,
|
||||
"created_at": rv.created_at.isoformat(),
|
||||
}
|
||||
for rv in rows
|
||||
]
|
||||
|
||||
|
||||
def _setting_rows(rows) -> list[dict]:
|
||||
return [{"user_id": s.user_id, "key": s.key, "value": s.value} for s in rows]
|
||||
|
||||
@@ -423,6 +602,9 @@ async def export_full_backup() -> dict:
|
||||
note_versions = (await session.execute(
|
||||
select(NoteVersion).order_by(NoteVersion.note_id, NoteVersion.id)
|
||||
)).scalars().all()
|
||||
rule_versions = (await session.execute(
|
||||
select(RuleVersion).order_by(RuleVersion.rule_id, RuleVersion.id)
|
||||
)).scalars().all()
|
||||
settings = (await session.execute(select(Setting))).scalars().all()
|
||||
systems = (await session.execute(select(System))).scalars().all()
|
||||
canonical_systems = (await session.execute(
|
||||
@@ -447,6 +629,9 @@ async def export_full_backup() -> dict:
|
||||
)).scalars().all()
|
||||
design_tokens = (await session.execute(select(DesignToken))).scalars().all()
|
||||
usage_events = (await session.execute(select(NoteUsageEvent))).scalars().all()
|
||||
rule_usage_events = (
|
||||
await session.execute(select(RuleUsageEvent))
|
||||
).scalars().all()
|
||||
repo_bindings = (await session.execute(select(RepoBinding))).scalars().all()
|
||||
code_shapes = (await session.execute(select(CodeShape))).scalars().all()
|
||||
code_shape_events = (await session.execute(
|
||||
@@ -487,6 +672,7 @@ async def export_full_backup() -> dict:
|
||||
"task_logs": _task_log_rows(task_logs),
|
||||
"note_drafts": _note_draft_rows(note_drafts),
|
||||
"note_versions": _note_version_rows(note_versions),
|
||||
"rule_versions": _rule_version_rows(rule_versions),
|
||||
"settings": _setting_rows(settings),
|
||||
"rulebooks": _rulebook_rows(rulebooks),
|
||||
"rulebook_topics": _topic_rows(topics),
|
||||
@@ -505,6 +691,7 @@ async def export_full_backup() -> dict:
|
||||
"design_systems": _design_system_rows(design_systems),
|
||||
"design_tokens": _design_token_rows(design_tokens),
|
||||
"note_usage_events": _usage_event_rows(usage_events),
|
||||
"rule_usage_events": _rule_usage_event_rows(rule_usage_events),
|
||||
"repo_bindings": _repo_binding_rows(repo_bindings),
|
||||
"note_supersessions": _note_supersession_rows(supersessions),
|
||||
"code_shapes": _code_shape_rows(code_shapes),
|
||||
@@ -623,6 +810,22 @@ async def export_user_backup(user_id: int) -> dict:
|
||||
.join(CanonicalSystem, CanonicalSystem.id == rule_systems_t.c.canonical_id)
|
||||
.where(rule_systems_t.c.rule_id.in_(_rule_ids))
|
||||
)).all() if _rule_ids else []
|
||||
# Scoped through the RULE, not the version's user_id. That column is
|
||||
# the ACTOR (milestone 323), so filtering on it would carry the
|
||||
# versions this user wrote on someone ELSE's rule and drop the ones
|
||||
# someone else wrote on theirs — the opposite of a per-user export.
|
||||
rule_versions = (await session.execute(
|
||||
select(RuleVersion).where(RuleVersion.rule_id.in_(_rule_ids))
|
||||
.order_by(RuleVersion.rule_id, RuleVersion.id)
|
||||
)).scalars().all() if _rule_ids else []
|
||||
# Scoped through the RULE for the same reason the versions above are,
|
||||
# and it is worth restating because the column that looks right is
|
||||
# wrong: `user_id` here is whoever the arm fired FOR, not who owns the
|
||||
# rule. Filtering on it would carry this user's surfacings of someone
|
||||
# ELSE's rule and drop the ones fired for someone else on theirs.
|
||||
rule_usage_events = (await session.execute(
|
||||
select(RuleUsageEvent).where(RuleUsageEvent.rule_id.in_(_rule_ids))
|
||||
)).scalars().all() if _rule_ids else []
|
||||
rule_relations = (await session.execute(
|
||||
select(RuleRelation).where(
|
||||
RuleRelation.from_rule_id.in_(_rule_ids),
|
||||
@@ -671,6 +874,7 @@ async def export_user_backup(user_id: int) -> dict:
|
||||
"task_logs": _task_log_rows(task_logs),
|
||||
"note_drafts": _note_draft_rows(note_drafts),
|
||||
"note_versions": _note_version_rows(note_versions),
|
||||
"rule_versions": _rule_version_rows(rule_versions),
|
||||
"settings": _setting_rows(settings),
|
||||
"rulebooks": _rulebook_rows(rulebooks),
|
||||
"rulebook_topics": _topic_rows(topics),
|
||||
@@ -689,6 +893,7 @@ async def export_user_backup(user_id: int) -> dict:
|
||||
"design_systems": _design_system_rows(design_systems),
|
||||
"design_tokens": _design_token_rows(design_tokens),
|
||||
"note_usage_events": _usage_event_rows(usage_events),
|
||||
"rule_usage_events": _rule_usage_event_rows(rule_usage_events),
|
||||
"repo_bindings": _repo_binding_rows(repo_bindings),
|
||||
"note_supersessions": _note_supersession_rows(supersessions),
|
||||
"code_shapes": _code_shape_rows(code_shapes),
|
||||
@@ -747,11 +952,29 @@ async def _restore_v1(data: dict) -> dict:
|
||||
body=n_data.get("body", ""),
|
||||
tags=n_data.get("tags", []),
|
||||
parent_id=None, # patched below
|
||||
arose_from_id=None, # patched below, same reason
|
||||
description=n_data.get("description"),
|
||||
note_type=n_data.get("note_type") or "note",
|
||||
task_kind=n_data.get("task_kind") or "work",
|
||||
data=n_data.get("data"),
|
||||
started_at=_dt_or_none(n_data.get("started_at")),
|
||||
completed_at=_dt_or_none(n_data.get("completed_at")),
|
||||
recurrence_rule=n_data.get("recurrence_rule"),
|
||||
recurrence_next_spawn_at=_dt_or_none(
|
||||
n_data.get("recurrence_next_spawn_at")
|
||||
),
|
||||
status=n_data.get("status"),
|
||||
priority=n_data.get("priority"),
|
||||
due_date=_d(n_data.get("due_date")),
|
||||
created_at=_dt(n_data.get("created_at")),
|
||||
updated_at=_dt(n_data.get("updated_at")),
|
||||
verify_with=n_data.get("verify_with"),
|
||||
expires_when=n_data.get("expires_when"),
|
||||
# _dt_or_none, NOT _dt: an absent stamp must stay absent. _dt
|
||||
# substitutes now(), which would restore every never-checked
|
||||
# note as checked at the moment of the restore — inverting the
|
||||
# one signal the sweep reads.
|
||||
verified_at=_dt_or_none(n_data.get("verified_at")),
|
||||
)
|
||||
session.add(note)
|
||||
await session.flush()
|
||||
@@ -759,14 +982,25 @@ async def _restore_v1(data: dict) -> dict:
|
||||
note_id_map[old_id] = note.id
|
||||
stats["notes"] += 1
|
||||
|
||||
# Patch parent_id now that all notes have new IDs
|
||||
# Patch the two note->note edges now that every note has a new id.
|
||||
# Both are ids in the SOURCE database, so writing either straight into
|
||||
# the constructor would point at whatever record happens to hold that
|
||||
# number here — a restore that succeeds and silently re-parents (#3182).
|
||||
# An edge whose target did not survive the import is left NULL rather
|
||||
# than guessed at.
|
||||
for n_data in data.get("notes", []):
|
||||
old_id = n_data.get("id")
|
||||
if not old_id or old_id not in note_id_map:
|
||||
continue
|
||||
note_row = await session.get(Note, note_id_map[old_id])
|
||||
if note_row is None:
|
||||
continue
|
||||
old_parent = n_data.get("parent_id")
|
||||
if old_id and old_parent and old_id in note_id_map and old_parent in note_id_map:
|
||||
note_row = await session.get(Note, note_id_map[old_id])
|
||||
if note_row:
|
||||
note_row.parent_id = note_id_map[old_parent]
|
||||
if old_parent and old_parent in note_id_map:
|
||||
note_row.parent_id = note_id_map[old_parent]
|
||||
old_origin = n_data.get("arose_from_id")
|
||||
if old_origin and old_origin in note_id_map:
|
||||
note_row.arose_from_id = note_id_map[old_origin]
|
||||
|
||||
for s_data in data.get("settings", []):
|
||||
mapped_user_id = user_id_map.get(s_data.get("user_id", 0))
|
||||
@@ -796,10 +1030,11 @@ async def _restore_v2(data: dict) -> dict:
|
||||
"rulebook_subscriptions": 0, "rule_suppressions": 0,
|
||||
"topic_suppressions": 0, "rulebook_exclusions": 0,
|
||||
"systems": 0, "record_systems": 0, "design_systems": 0,
|
||||
"design_tokens": 0, "note_usage_events": 0, "repo_bindings": 0,
|
||||
"design_tokens": 0, "note_usage_events": 0, "rule_usage_events": 0,
|
||||
"repo_bindings": 0,
|
||||
"note_supersessions": 0, "code_shapes": 0, "code_shape_events": 0,
|
||||
"code_shape_uses": 0, "canonical_systems": 0,
|
||||
"rule_systems": 0, "rule_relations": 0,
|
||||
"rule_systems": 0, "rule_relations": 0, "rule_versions": 0,
|
||||
}
|
||||
|
||||
async with async_session() as session:
|
||||
@@ -859,6 +1094,7 @@ async def _restore_v2(data: dict) -> dict:
|
||||
project_id=mapped_pid,
|
||||
title=m_data.get("title", ""),
|
||||
description=m_data.get("description"),
|
||||
body=m_data.get("body"),
|
||||
status=m_data.get("status", "active"),
|
||||
order_index=m_data.get("order_index", 0),
|
||||
created_at=_dt(m_data.get("created_at")),
|
||||
@@ -871,6 +1107,7 @@ async def _restore_v2(data: dict) -> dict:
|
||||
|
||||
# 4a. Notes — first pass (no parent_id yet)
|
||||
notes_with_parents: list[tuple[int, int]] = [] # (new_note_id, old_parent_id)
|
||||
notes_with_origins: list[tuple[int, int]] = [] # (new_note_id, old_arose_from_id)
|
||||
for n_data in data.get("notes", []):
|
||||
mapped_uid = user_id_map.get(n_data.get("user_id", 0))
|
||||
if mapped_uid is None:
|
||||
@@ -881,6 +1118,17 @@ async def _restore_v2(data: dict) -> dict:
|
||||
body=n_data.get("body", ""),
|
||||
tags=n_data.get("tags", []),
|
||||
parent_id=None,
|
||||
arose_from_id=None, # patched below, same reason as parent_id
|
||||
description=n_data.get("description"),
|
||||
note_type=n_data.get("note_type") or "note",
|
||||
task_kind=n_data.get("task_kind") or "work",
|
||||
data=n_data.get("data"),
|
||||
started_at=_dt_or_none(n_data.get("started_at")),
|
||||
completed_at=_dt_or_none(n_data.get("completed_at")),
|
||||
recurrence_rule=n_data.get("recurrence_rule"),
|
||||
recurrence_next_spawn_at=_dt_or_none(
|
||||
n_data.get("recurrence_next_spawn_at")
|
||||
),
|
||||
project_id=project_id_map.get(n_data["project_id"]) if n_data.get("project_id") else None,
|
||||
milestone_id=milestone_id_map.get(n_data["milestone_id"]) if n_data.get("milestone_id") else None,
|
||||
status=n_data.get("status"),
|
||||
@@ -888,21 +1136,35 @@ async def _restore_v2(data: dict) -> dict:
|
||||
due_date=_d(n_data.get("due_date")),
|
||||
created_at=_dt(n_data.get("created_at")),
|
||||
updated_at=_dt(n_data.get("updated_at")),
|
||||
verify_with=n_data.get("verify_with"),
|
||||
expires_when=n_data.get("expires_when"),
|
||||
# _dt_or_none — see the note on the other restore path.
|
||||
verified_at=_dt_or_none(n_data.get("verified_at")),
|
||||
)
|
||||
session.add(note)
|
||||
await session.flush()
|
||||
note_id_map[n_data["id"]] = note.id
|
||||
if n_data.get("parent_id"):
|
||||
notes_with_parents.append((note.id, n_data["parent_id"]))
|
||||
if n_data.get("arose_from_id"):
|
||||
notes_with_origins.append((note.id, n_data["arose_from_id"]))
|
||||
stats["notes"] += 1
|
||||
|
||||
# 4b. Patch parent_id
|
||||
# 4b. Patch the note->note edges. Deferred for the same reason
|
||||
# parent_id always has been: these are ids in the SOURCE database
|
||||
# (#3182). An edge whose target did not survive stays NULL.
|
||||
for new_note_id, old_parent_id in notes_with_parents:
|
||||
new_parent_id = note_id_map.get(old_parent_id)
|
||||
if new_parent_id:
|
||||
note_row = await session.get(Note, new_note_id)
|
||||
if note_row:
|
||||
note_row.parent_id = new_parent_id
|
||||
for new_note_id, old_origin_id in notes_with_origins:
|
||||
new_origin_id = note_id_map.get(old_origin_id)
|
||||
if new_origin_id:
|
||||
note_row = await session.get(Note, new_note_id)
|
||||
if note_row:
|
||||
note_row.arose_from_id = new_origin_id
|
||||
|
||||
# 5. TaskLogs
|
||||
for tl_data in data.get("task_logs", []):
|
||||
@@ -1141,6 +1403,32 @@ async def _restore_v2(data: dict) -> dict:
|
||||
))
|
||||
stats["rule_relations"] += 1
|
||||
|
||||
# A rule's edit history (milestone 323). Must come after the rules
|
||||
# themselves — rule_id_map is only populated above — and both ids are
|
||||
# ids in the SOURCE database, which is #3182's arose_from_id trap.
|
||||
for rv in data.get("rule_versions", []):
|
||||
mapped_rid = rule_id_map.get(rv.get("rule_id", 0))
|
||||
if mapped_rid is None:
|
||||
continue
|
||||
# Unlike NoteVersion, an unmappable user does NOT drop the row.
|
||||
# user_id is the ACTOR and is nullable by design: the column is
|
||||
# SET NULL precisely so history outlives the account that wrote
|
||||
# it. Skipping here would delete the record the FK preserves.
|
||||
session.add(RuleVersion(
|
||||
rule_id=mapped_rid,
|
||||
user_id=user_id_map.get(rv.get("user_id") or 0),
|
||||
title=rv.get("title", ""),
|
||||
statement=rv.get("statement", ""),
|
||||
why=rv.get("why"),
|
||||
how_to_apply=rv.get("how_to_apply"),
|
||||
when_to_apply=rv.get("when_to_apply"),
|
||||
tier=rv.get("tier"),
|
||||
verify_with=rv.get("verify_with"),
|
||||
expires_when=rv.get("expires_when"),
|
||||
created_at=_dt(rv.get("created_at")),
|
||||
))
|
||||
stats["rule_versions"] += 1
|
||||
|
||||
# 15. Systems
|
||||
for sy_data in data.get("systems", []):
|
||||
mapped_uid = user_id_map.get(sy_data.get("user_id", 0))
|
||||
@@ -1245,6 +1533,25 @@ async def _restore_v2(data: dict) -> dict:
|
||||
))
|
||||
stats["note_usage_events"] += 1
|
||||
|
||||
# The rule twin — and the reason it is a separate table at all.
|
||||
# Resolved through rule_id_map, NOT note_id_map. A rule id run through
|
||||
# the note map would either drop (best case) or land on whatever note
|
||||
# took that number, producing telemetry that is wrong rather than
|
||||
# missing and that nothing downstream could detect (milestone 333).
|
||||
# Must come after the rules themselves; rule_id_map is populated there.
|
||||
for ev in data.get("rule_usage_events", []):
|
||||
mapped_rid = rule_id_map.get(ev.get("rule_id", 0))
|
||||
if mapped_rid is None:
|
||||
continue
|
||||
session.add(RuleUsageEvent(
|
||||
user_id=user_id_map.get(ev.get("user_id") or 0),
|
||||
rule_id=mapped_rid,
|
||||
event=ev.get("event", ""),
|
||||
source=ev.get("source", ""),
|
||||
created_at=_dt(ev.get("created_at")),
|
||||
))
|
||||
stats["rule_usage_events"] += 1
|
||||
|
||||
# 20. Repo bindings — small, but losing them means every bound repo
|
||||
# quietly stops loading its project at session start.
|
||||
for rb_data in data.get("repo_bindings", []):
|
||||
@@ -1255,6 +1562,7 @@ async def _restore_v2(data: dict) -> dict:
|
||||
session.add(RepoBinding(
|
||||
user_id=mapped_uid, project_id=mapped_pid,
|
||||
repo_key=rb_data.get("repo_key", ""),
|
||||
ref=rb_data.get("ref"),
|
||||
))
|
||||
stats["repo_bindings"] += 1
|
||||
|
||||
|
||||
@@ -344,6 +344,52 @@ def chunk_document(title: str | None, body: str | None) -> list[str]:
|
||||
return chunks
|
||||
|
||||
|
||||
async def _claim_parent_row(session, id_column, row_id: int, label: str) -> bool:
|
||||
"""Lock the record a vector belongs to BEFORE rewriting that vector (#3262).
|
||||
|
||||
An embedding write and a cascading delete of the same record take the same
|
||||
two row locks in OPPOSITE orders. The embedder deletes the old chunk rows
|
||||
and then, on INSERT, needs the foreign key's lock on the parent; a delete
|
||||
of the parent — or of the rulebook, topic or project above it — locks the
|
||||
parent first and cascades down into the chunk rows. That is a cycle, and
|
||||
Postgres breaks it by killing one side at random: sometimes the embedding
|
||||
write, which is swallowed and invisible, and sometimes the operator's
|
||||
delete, which surfaces as a 500 on an operation that should have worked.
|
||||
|
||||
Claiming the parent first REMOVES the cycle rather than narrowing it.
|
||||
Either the embedder arrives first and the delete waits its turn behind it,
|
||||
or the delete already holds the row and NOWAIT makes the embedder lose at
|
||||
once. The embedder is the side that should lose: a skipped refresh costs a
|
||||
stale vector until the next write or the startup backfill, and the other
|
||||
outcome costs a person their request.
|
||||
|
||||
FOR KEY SHARE, not FOR UPDATE — it is precisely the lock the INSERT's
|
||||
foreign key would take anyway, so it conflicts with a delete of the parent
|
||||
and with nothing else. An ordinary edit of the same record, or a second
|
||||
refresh racing this one, is unaffected.
|
||||
|
||||
Returns False when the row is locked or already gone; the caller skips.
|
||||
"""
|
||||
try:
|
||||
held = (await session.execute(
|
||||
select(id_column)
|
||||
.where(id_column == row_id)
|
||||
# BOTH flags: SQLAlchemy spells the four Postgres row locks as a
|
||||
# read/key_share pair, and key_share alone is FOR NO KEY UPDATE —
|
||||
# which would make two refreshes of one record fight each other.
|
||||
.with_for_update(read=True, key_share=True, nowait=True)
|
||||
)).scalar_one_or_none()
|
||||
except Exception:
|
||||
# LockNotAvailable: this record is being deleted right now. Not an
|
||||
# error — the delete wins by design.
|
||||
logger.debug("Skipping embedding for %s %d — row is being deleted", label, row_id)
|
||||
return False
|
||||
if held is None:
|
||||
logger.debug("Skipping embedding for %s %d — row is gone", label, row_id)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
async def upsert_note_embedding(
|
||||
note_id: int, user_id: int, title: str | None, body: str | None
|
||||
) -> None:
|
||||
@@ -380,6 +426,8 @@ async def upsert_note_embedding(
|
||||
|
||||
try:
|
||||
async with async_session() as session:
|
||||
if not await _claim_parent_row(session, Note.id, note_id, "note"):
|
||||
return
|
||||
await session.execute(
|
||||
delete(NoteEmbedding).where(NoteEmbedding.note_id == note_id)
|
||||
)
|
||||
@@ -400,6 +448,23 @@ async def upsert_note_embedding(
|
||||
logger.warning("Failed to persist embedding for note %d", note_id, exc_info=True)
|
||||
|
||||
|
||||
# Both searches rank WITHOUT the threshold and apply it in Python, so the best
|
||||
# rejected score stays observable (#3670). The qualifying set is provably
|
||||
# unchanged: rows arrive ordered by distance ascending, so every above-bar row
|
||||
# sorts ahead of every below-bar one, and an over-fetch that used to return N
|
||||
# above-bar rows returns the same N plus some losers. What changes is only that
|
||||
# the losers are now visible instead of discarded inside the query.
|
||||
#
|
||||
# That visibility is the entire point. A bar can only be judged from the calls
|
||||
# it TURNED AWAY — a 0.72 bar rejecting a stream of 0.71s is set too high by a
|
||||
# hair, one rejecting 0.30s is working — and those two are indistinguishable
|
||||
# from any arrangement of the columns that survive the filter.
|
||||
#
|
||||
# `report` is how the score gets out without changing what a search RETURNS.
|
||||
# Eight of the eleven call sites want hits and nothing else; the three that
|
||||
# write telemetry pass a dict and read `best_available_score` back out of it.
|
||||
|
||||
|
||||
async def semantic_search_notes(
|
||||
user_id: int,
|
||||
query: str,
|
||||
@@ -414,12 +479,19 @@ async def semantic_search_notes(
|
||||
scope: str = "own",
|
||||
demote_superseded: bool = True,
|
||||
system_id: int | None = None,
|
||||
report: dict | None = None,
|
||||
) -> list[tuple[float, Note]]:
|
||||
"""Return up to *limit* (score, note) pairs most relevant to *query*.
|
||||
|
||||
Scores are cosine similarities in [-1, 1]; only notes at or above
|
||||
*threshold* are returned, sorted highest-first.
|
||||
|
||||
Pass `report` (an empty dict) to learn what the threshold turned away:
|
||||
the function sets `report["best_available_score"]` to the highest score
|
||||
anything reached, or None when the corpus offered nothing at all. It is
|
||||
the only figure that survives a call returning nothing, and therefore the
|
||||
only one a bar can be judged from (#3670).
|
||||
|
||||
`note_type` narrows to a record kind, or several (e.g. "snippet", or
|
||||
("snippet", "note")), for callers that want prior art rather than everything
|
||||
embedded.
|
||||
@@ -465,7 +537,6 @@ async def semantic_search_notes(
|
||||
# Distance ceiling equivalent to the similarity floor. Clamp to the valid
|
||||
# cosine-distance range [0, 2] so a threshold of, say, -1 doesn't produce a
|
||||
# nonsensical ceiling.
|
||||
max_distance = min(2.0, max(0.0, 1.0 - threshold))
|
||||
distance = NoteEmbedding.embedding.cosine_distance(query_vec)
|
||||
|
||||
try:
|
||||
@@ -540,11 +611,10 @@ async def semantic_search_notes(
|
||||
fetch = limit * _CHUNK_OVERFETCH * (
|
||||
_SUPERSESSION_OVERFETCH if demote_superseded else 1
|
||||
)
|
||||
stmt = (
|
||||
stmt.where(distance <= max_distance)
|
||||
.order_by(distance.asc())
|
||||
.limit(fetch)
|
||||
)
|
||||
# NO threshold predicate — see the note above this function. The
|
||||
# bar is applied after the collapse, where the rejected scores can
|
||||
# still be seen.
|
||||
stmt = stmt.order_by(distance.asc()).limit(fetch)
|
||||
rows = list((await session.execute(stmt)).all())
|
||||
except Exception:
|
||||
logger.warning("Failed to query note embeddings", exc_info=True)
|
||||
@@ -563,6 +633,11 @@ async def semantic_search_notes(
|
||||
continue
|
||||
seen.add(int(note.id))
|
||||
scored.append((1.0 - float(dist), note))
|
||||
# The best score anything reached, bar or no bar. Recorded BEFORE the
|
||||
# filter because a call that returns nothing is exactly when it matters.
|
||||
if report is not None:
|
||||
report["best_available_score"] = scored[0][0] if scored else None
|
||||
scored = [pair for pair in scored if pair[0] >= threshold]
|
||||
if not demote_superseded:
|
||||
return scored[:limit]
|
||||
return await _apply_supersession_penalty(scored, limit)
|
||||
@@ -666,6 +741,8 @@ async def upsert_rule_embedding(
|
||||
replacement is atomic per rule so a concurrent read sees the old chunk set
|
||||
or the new one, never a mixture.
|
||||
"""
|
||||
from scribe.models.rulebook import Rule # runtime import: see TYPE_CHECKING above
|
||||
|
||||
doc_title, doc_body = rule_document(title, statement, when_to_apply)
|
||||
chunks = chunk_document(doc_title, doc_body)
|
||||
try:
|
||||
@@ -688,6 +765,8 @@ async def upsert_rule_embedding(
|
||||
|
||||
try:
|
||||
async with async_session() as session:
|
||||
if not await _claim_parent_row(session, Rule.id, rule_id, "rule"):
|
||||
return
|
||||
await session.execute(
|
||||
delete(RuleEmbedding).where(RuleEmbedding.rule_id == rule_id)
|
||||
)
|
||||
@@ -712,9 +791,16 @@ async def semantic_search_rules(
|
||||
limit: int = 5,
|
||||
threshold: float = _SIMILARITY_THRESHOLD,
|
||||
tier: str | None = None,
|
||||
report: dict | None = None,
|
||||
) -> list[tuple[float, "Rule"]]:
|
||||
"""Return up to *limit* (score, rule) pairs most relevant to *query*.
|
||||
|
||||
Pass `report` (an empty dict) to learn what the threshold turned away:
|
||||
the function sets `report["best_available_score"]` to the highest score
|
||||
anything reached, or None when the corpus offered nothing at all. It is
|
||||
the only figure that survives a call returning nothing, and therefore the
|
||||
only one a bar can be judged from (#3670).
|
||||
|
||||
Scoped by OWNERSHIP — a rule is the caller's if they own its rulebook or
|
||||
its project. Deliberately not filtered to what currently BINDS a given
|
||||
project: this answers "is there a rule about this", which a person asking
|
||||
@@ -722,10 +808,17 @@ async def semantic_search_rules(
|
||||
is the surfacing question, and it has its own machinery
|
||||
(get_applicable_rules) rather than a second, subtly different copy here.
|
||||
|
||||
`tier` narrows to one tier. The write-path hint passes "conditional",
|
||||
because an always-on rule is ALREADY in the session — surfacing it again as
|
||||
a suggestion is pure noise, and noise on a hint that fires on every write
|
||||
is how a hint gets ignored.
|
||||
`tier` narrows to one tier, and NONE is the ordinary case. The write-path
|
||||
and pre-tool hints deliberately pass nothing: an always-on rule is already
|
||||
in the session, but being in a list from turn zero is not the same as being
|
||||
in front of the reader when the action it governs is taken, and filtering
|
||||
on tier made a whole class of rules permanently ineligible for the one
|
||||
mechanism that surfaces a rule AT the moment. Relevance is the threshold's
|
||||
job; see the block above RULEHINT_LIMIT in services/plugin_context.py for
|
||||
the argument and for what the resulting scores are being read against.
|
||||
|
||||
Pass a tier when a caller genuinely wants one class — a listing, an audit,
|
||||
a UI that renders the tiers apart. Not to approximate relevance.
|
||||
|
||||
Collapses to best-chunk-per-rule like the note search, so a long rule split
|
||||
across chunks competes once rather than crowding the results with itself.
|
||||
@@ -743,7 +836,6 @@ async def semantic_search_rules(
|
||||
logger.debug("Rule search skipped — embedder unavailable")
|
||||
return []
|
||||
|
||||
max_distance = min(2.0, max(0.0, 1.0 - threshold))
|
||||
distance = RuleEmbedding.embedding.cosine_distance(query_vec)
|
||||
|
||||
try:
|
||||
@@ -757,7 +849,8 @@ async def semantic_search_rules(
|
||||
.outerjoin(Project, Rule.project_id == Project.id)
|
||||
.where(
|
||||
Rule.deleted_at.is_(None),
|
||||
distance <= max_distance,
|
||||
# No threshold predicate — see the note above
|
||||
# semantic_search_notes. Applied below, after the collapse.
|
||||
# topic_id XOR project_id, so exactly one arm can match.
|
||||
or_(
|
||||
Rulebook.owner_user_id == user_id,
|
||||
@@ -780,7 +873,9 @@ async def semantic_search_rules(
|
||||
if rule.id not in best or score > best[rule.id][0]:
|
||||
best[rule.id] = (score, rule)
|
||||
ranked = sorted(best.values(), key=lambda pair: pair[0], reverse=True)
|
||||
return ranked[:limit]
|
||||
if report is not None:
|
||||
report["best_available_score"] = ranked[0][0] if ranked else None
|
||||
return [pair for pair in ranked if pair[0] >= threshold][:limit]
|
||||
|
||||
|
||||
async def backfill_rule_embeddings() -> None:
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Knowledge service — unified query across notes, tasks, plans, and processes.
|
||||
"""Knowledge service — one query across every record kind Scribe holds.
|
||||
|
||||
ACL (rules #47/#78, decision note 2094): these queries were owner-only until
|
||||
2026-07-25, which meant a record shared with you could be opened by id but never
|
||||
@@ -265,22 +265,94 @@ def _note_to_item(note: Note) -> dict:
|
||||
return item
|
||||
|
||||
|
||||
def _apply_type_filter(stmt, note_type: str | None):
|
||||
"""Apply the type facet to a Note select.
|
||||
# What each type facet MEANS, once, for every arm that has to know.
|
||||
#
|
||||
# The vocabulary spans BOTH typing axes — `note_type` for non-task records and
|
||||
# `task_kind` for tasks — so a facet cannot be a filter on one column, which is
|
||||
# why this is a table rather than a chain of ifs. Each entry is
|
||||
# (is_task, the value pinned on that axis); None pins nothing, i.e. every task.
|
||||
#
|
||||
# It is a table because the alternative had already gone wrong. The predicate
|
||||
# was written three times — a SQL if-chain, a Python if-chain over semantic
|
||||
# candidates, and a ternary computing the `is_task` pre-filter — and the three
|
||||
# only agreed by luck. Adding `issue` to the SQL arm alone (the obvious edit,
|
||||
# and the one #3128 was about to make) would have set the pre-filter to
|
||||
# is_task=False, handed the Python arm a candidate set containing no tasks at
|
||||
# all, and returned an empty semantic half for the Issues facet forever, with
|
||||
# nothing red anywhere. A new facet is now one row here.
|
||||
#
|
||||
# `plan` is retired (0066) but kept: 90 legacy plan-tasks exist and a facet
|
||||
# they answer to costs one line. It simply has no chip in the UI any more.
|
||||
_FACETS: dict[str, tuple[bool, str | None]] = {
|
||||
"task": (True, None),
|
||||
"work": (True, "work"),
|
||||
"issue": (True, "issue"),
|
||||
"spike": (True, "spike"),
|
||||
"plan": (True, "plan"),
|
||||
"note": (False, "note"),
|
||||
"process": (False, "process"),
|
||||
"snippet": (False, "snippet"),
|
||||
}
|
||||
|
||||
'task' = any task (status not null); 'plan' = a task with task_kind='plan';
|
||||
any other non-empty type = a non-task note of that note_type; None = all.
|
||||
# The non-task record types, for the counts query. Derived so it cannot drift
|
||||
# from the table above.
|
||||
NON_TASK_FACETS = tuple(
|
||||
value for _is_task, value in _FACETS.values() if not _is_task and value
|
||||
)
|
||||
|
||||
Trashed rows (deleted_at set) are always excluded.
|
||||
# The whole vocabulary, for the door's request validation — public so the route
|
||||
# validates against the same table the query reads instead of a hand-kept copy.
|
||||
FACET_TYPES = frozenset(_FACETS)
|
||||
|
||||
|
||||
# An unrecognised facet resolves to "a non-task note whose note_type is that
|
||||
# string" — which matches nothing, since no row stores an unknown type. That is
|
||||
# the behaviour the old if-chain had by falling through, and it is the right
|
||||
# one: a typo should return an empty list, never the whole corpus.
|
||||
def _facet(note_type: str) -> tuple[bool, str | None]:
|
||||
return _FACETS.get(note_type, (False, note_type))
|
||||
|
||||
|
||||
def facet_is_task(note_type: str | None) -> bool | None:
|
||||
"""The `is_task` pre-filter a facet implies — None when it spans both.
|
||||
|
||||
Used to narrow the semantic candidate set before it is fetched. Reads the
|
||||
same table `_apply_type_filter` and `matches_facet` read, so the pre-filter
|
||||
can no longer disagree with the predicate it is meant to anticipate.
|
||||
"""
|
||||
if not note_type:
|
||||
return None
|
||||
return _facet(note_type)[0]
|
||||
|
||||
|
||||
def matches_facet(note, note_type: str | None) -> bool:
|
||||
"""The Python dialect of `_apply_type_filter`, for candidates the vector
|
||||
search has already fetched — there is no query left to narrow.
|
||||
|
||||
Generated from the same table, so this is a translation rather than a
|
||||
second implementation. Note the `not note.is_task` arm: the hand-written
|
||||
version omitted it and was saved only by the upstream pre-filter.
|
||||
"""
|
||||
if not note_type:
|
||||
return True
|
||||
is_task, value = _facet(note_type)
|
||||
if is_task:
|
||||
return note.is_task and (value is None or note.task_kind == value)
|
||||
return not note.is_task and note.note_type == value
|
||||
|
||||
|
||||
def _apply_type_filter(stmt, note_type: str | None):
|
||||
"""Apply the type facet to a Note select. Trashed rows are always excluded."""
|
||||
stmt = stmt.where(Note.deleted_at.is_(None))
|
||||
if note_type == "task":
|
||||
return stmt.where(Note.status.isnot(None))
|
||||
if note_type == "plan":
|
||||
return stmt.where(Note.status.isnot(None)).where(Note.task_kind == "plan")
|
||||
if note_type:
|
||||
return stmt.where(Note.note_type == note_type).where(Note.status.is_(None))
|
||||
return stmt
|
||||
if not note_type:
|
||||
return stmt
|
||||
is_task, value = _facet(note_type)
|
||||
if is_task:
|
||||
stmt = stmt.where(Note.status.isnot(None))
|
||||
if value is not None:
|
||||
stmt = stmt.where(Note.task_kind == value)
|
||||
return stmt
|
||||
return stmt.where(Note.status.is_(None)).where(Note.note_type == value)
|
||||
|
||||
|
||||
async def query_knowledge(
|
||||
@@ -295,7 +367,7 @@ async def query_knowledge(
|
||||
locations: dict[str, str] | None = None,
|
||||
verification: str = "",
|
||||
) -> tuple[list[dict], int]:
|
||||
"""Query knowledge objects (non-task notes) with filters.
|
||||
"""Query knowledge objects with filters.
|
||||
|
||||
`project_id` narrows to one project (None = every project).
|
||||
|
||||
@@ -424,7 +496,7 @@ async def _semantic_knowledge_search(
|
||||
INTERACTIVE_SEARCH_THRESHOLD,
|
||||
semantic_search_notes,
|
||||
)
|
||||
is_task_filter = True if note_type in ("task", "plan") else (False if note_type else None)
|
||||
is_task_filter = facet_is_task(note_type)
|
||||
import time as _time
|
||||
_t0 = _time.perf_counter()
|
||||
candidates = await semantic_search_notes(
|
||||
@@ -454,11 +526,7 @@ async def _semantic_knowledge_search(
|
||||
for _score, note in candidates:
|
||||
if note.deleted_at is not None:
|
||||
continue
|
||||
if note_type == "task" and not note.is_task:
|
||||
continue
|
||||
elif note_type == "plan" and (not note.is_task or note.task_kind != "plan"):
|
||||
continue
|
||||
elif note_type and note_type not in ("task", "plan") and note.note_type != note_type:
|
||||
if not matches_facet(note, note_type):
|
||||
continue
|
||||
if tags and not all(t in (note.tags or []) for t in tags):
|
||||
continue
|
||||
@@ -514,51 +582,40 @@ async def get_knowledge_counts(user_id: int, tags: list[str] | None = None) -> d
|
||||
search would surface."""
|
||||
visible = browsable_notes_clause(user_id)
|
||||
async with async_session() as session:
|
||||
# Count non-task types
|
||||
stmt = (
|
||||
select(Note.note_type, func.count(Note.id))
|
||||
.where(visible)
|
||||
.where(Note.status.is_(None))
|
||||
.where(Note.deleted_at.is_(None))
|
||||
.where(Note.note_type.in_(["note", "process"]))
|
||||
.group_by(Note.note_type)
|
||||
)
|
||||
if tags:
|
||||
for tag in tags:
|
||||
def _scoped(stmt):
|
||||
stmt = stmt.where(visible).where(Note.deleted_at.is_(None))
|
||||
for tag in tags or []:
|
||||
stmt = stmt.where(Note.tags.contains([tag]))
|
||||
rows = list((await session.execute(stmt)).all())
|
||||
counts = {row[0]: row[1] for row in rows}
|
||||
return stmt
|
||||
|
||||
# Count tasks separately (is_task = status IS NOT NULL)
|
||||
task_stmt = (
|
||||
select(func.count(Note.id))
|
||||
.where(visible)
|
||||
# One grouped query per typing axis. The task half used to be a count
|
||||
# for 'task' plus a second count for 'plan', which is why 'issue' —
|
||||
# 17% of every task here — had no number to show: each kind needed its
|
||||
# own query and nobody added one. Grouping by task_kind counts every
|
||||
# kind, including ones added later, for the same two round-trips.
|
||||
non_task = _scoped(
|
||||
select(Note.note_type, func.count(Note.id))
|
||||
.where(Note.status.is_(None))
|
||||
.where(Note.note_type.in_(NON_TASK_FACETS))
|
||||
).group_by(Note.note_type)
|
||||
counts = {t: n for t, n in (await session.execute(non_task)).all()}
|
||||
|
||||
by_kind = _scoped(
|
||||
select(Note.task_kind, func.count(Note.id))
|
||||
.where(Note.status.isnot(None))
|
||||
.where(Note.deleted_at.is_(None))
|
||||
)
|
||||
if tags:
|
||||
for tag in tags:
|
||||
task_stmt = task_stmt.where(Note.tags.contains([tag]))
|
||||
task_count: int = (await session.execute(task_stmt)).scalar_one()
|
||||
counts["task"] = task_count
|
||||
).group_by(Note.task_kind)
|
||||
kind_counts = {k: n for k, n in (await session.execute(by_kind)).all()}
|
||||
|
||||
# Plans are a subset of tasks (task_kind='plan'); counted for the facet
|
||||
# but NOT added to total to avoid double-counting against "task".
|
||||
plan_stmt = (
|
||||
select(func.count(Note.id))
|
||||
.where(visible)
|
||||
.where(Note.status.isnot(None))
|
||||
.where(Note.task_kind == "plan")
|
||||
.where(Note.deleted_at.is_(None))
|
||||
)
|
||||
if tags:
|
||||
for tag in tags:
|
||||
plan_stmt = plan_stmt.where(Note.tags.contains([tag]))
|
||||
counts["plan"] = (await session.execute(plan_stmt)).scalar_one()
|
||||
# Kinds are SUBSETS of 'task' and are deliberately left out of the total —
|
||||
# adding them would count every task twice.
|
||||
counts["task"] = sum(kind_counts.values())
|
||||
for kind, value in _FACETS.items():
|
||||
if value[0] and value[1] is not None:
|
||||
counts[kind] = kind_counts.get(kind, 0)
|
||||
|
||||
for t in ("note", "task", "plan", "process"):
|
||||
for t in NON_TASK_FACETS:
|
||||
counts.setdefault(t, 0)
|
||||
counts["total"] = sum(counts[t] for t in ("note", "task", "process"))
|
||||
counts["total"] = counts["task"] + sum(counts[t] for t in NON_TASK_FACETS)
|
||||
return counts
|
||||
|
||||
|
||||
|
||||
@@ -30,13 +30,13 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import traceback
|
||||
|
||||
from sqlalchemy import case, func, select
|
||||
|
||||
from scribe.models import async_session
|
||||
from scribe.models.note_usage import PULLED, SURFACED, NoteUsageEvent
|
||||
from scribe.models.base import iso
|
||||
from scribe.services.background import report_telemetry_failure
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -46,37 +46,19 @@ logger = logging.getLogger(__name__)
|
||||
# never lands. The done-callback discard keeps the set from growing.
|
||||
_pending: set[asyncio.Task] = set()
|
||||
|
||||
# Sites that already dropped their once-per-process AppLog row. The readout
|
||||
# runs on every snippet list render — without this, a broken table would turn
|
||||
# the error log into a firehose that buries the finding it exists to surface.
|
||||
_reported: set[str] = set()
|
||||
|
||||
|
||||
async def _report_failure(site: str) -> None:
|
||||
"""Make a swallowed telemetry failure visible. Called from an except block.
|
||||
"""This subsystem's canary, now the shared one.
|
||||
|
||||
WARNING to the process log every time; one AppLog error row per process per
|
||||
site so the admin UI shows the outage without host access. The AppLog write
|
||||
is itself guarded — when the whole database is down it fails too, and that
|
||||
is fine: the WARNING already said so, and a canary must never take down the
|
||||
surface it watches.
|
||||
The per-site dedup, the WARNING and the single AppLog row all moved to
|
||||
`background.report_telemetry_failure` unchanged when `rule_usage` needed
|
||||
the identical behaviour — two hand-kept copies of a thing whose whole job
|
||||
is to be reliable is the wrong number. `retrieval_telemetry` deliberately
|
||||
still has its own: its canary is a different shape (one process-wide flag,
|
||||
no AppLog row), so repointing it would change behaviour rather than
|
||||
consolidate it.
|
||||
"""
|
||||
logger.warning("note usage telemetry %s failed", site, exc_info=True)
|
||||
if site in _reported:
|
||||
return
|
||||
_reported.add(site)
|
||||
try:
|
||||
from scribe.services.logging import log_error
|
||||
|
||||
await log_error(
|
||||
endpoint="note_usage",
|
||||
error_type=f"note_usage_{site}_failed",
|
||||
error_message=f"note usage telemetry {site} is failing; "
|
||||
"usage counters will read zero until this is fixed",
|
||||
traceback=traceback.format_exc(),
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("note usage canary write failed", exc_info=True)
|
||||
await report_telemetry_failure("note_usage", site)
|
||||
|
||||
|
||||
async def _insert_events(rows: list[dict]) -> None:
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import logging
|
||||
import re
|
||||
from collections.abc import Iterable
|
||||
from datetime import date, datetime, timezone
|
||||
|
||||
from sqlalchemy import func, or_, select, text
|
||||
@@ -9,6 +10,52 @@ from scribe.models.note import Note, TaskKind, TaskPriority, TaskStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# The fields `snippets.parse_snippet_fields` reads. Writing any of them can
|
||||
# change what a snippet's derived `data` mirror should say, so update_note
|
||||
# recomposes the mirror when one moves. Kept here as a set of NAMES rather
|
||||
# than imported, because it describes update_note's own `fields` dict, not the
|
||||
# parser's signature.
|
||||
_PARSED_FROM_BODY = frozenset({"title", "body", "tags"})
|
||||
|
||||
|
||||
# Text fields where EMPTY MEANS NULL (milestone 317). The sweep's whole signal
|
||||
# is `verify_with IS NULL` = "this is a decision, there is nothing to go and
|
||||
# check". An empty string that is not NULL makes a norm look like a constraint
|
||||
# nobody has verified, forever — and it would sit at the top of the sweep,
|
||||
# since never-checked sorts first. Sibling of rulebooks.NULLABLE_RULE_TEXT.
|
||||
NULLABLE_NOTE_TEXT = ("verify_with", "expires_when")
|
||||
|
||||
|
||||
def guard_check_fields(status: str | None, note_type: str | None) -> None:
|
||||
"""Raise unless a record in this shape may carry verify_with/expires_when.
|
||||
|
||||
Stated as an INVARIANT over the resulting record rather than a filter on
|
||||
which fields a caller passed, so it also catches the sideways route: a
|
||||
checked note being turned into a task, which no per-field gate would see.
|
||||
|
||||
Raises rather than dropping silently, for minted_kind's reason (#3129) — a
|
||||
silently-corrected write is the defect that reasoning exists to end, and a
|
||||
caller putting a check on the wrong record has an idea an error corrects
|
||||
and a default hides. Lives at the service, not either door, so REST and MCP
|
||||
cannot come to disagree about it.
|
||||
"""
|
||||
if status is not None:
|
||||
raise ValueError(
|
||||
"a task cannot carry verify_with/expires_when: a task's decay is "
|
||||
"its status, and a done issue records what happened rather than "
|
||||
"asserting something that can later go false. Put the check on the "
|
||||
"note the fact lives in, or clear the check before making this a "
|
||||
"task."
|
||||
)
|
||||
if note_type == "snippet":
|
||||
raise ValueError(
|
||||
"a snippet already has a check: verify_snippet(), which compares "
|
||||
"the recorded location and code against the repo and expires its "
|
||||
"own verdict when the code moves. verify_with/expires_when are the "
|
||||
"free-text form, for prose notes that assert a fact about "
|
||||
"something outside the repo."
|
||||
)
|
||||
|
||||
|
||||
def embed_note(note) -> None:
|
||||
"""Refresh a note's embedding, fire-and-forget.
|
||||
@@ -29,6 +76,10 @@ def embed_note(note) -> None:
|
||||
exceptions are swallowed because a record that saved must not fail on its
|
||||
index refresh. No running loop (unit tests, scripts) is an ordinary case,
|
||||
not an error.
|
||||
|
||||
Detaching also means this task races anything that deletes the note out
|
||||
from under it. That is not handled here: `upsert_note_embedding` claims
|
||||
the note's row before touching its vectors, and loses if it can't (#3262).
|
||||
"""
|
||||
try:
|
||||
import asyncio
|
||||
@@ -101,7 +152,16 @@ async def create_note(
|
||||
task_kind: str = "work",
|
||||
arose_from_id: int | None = None,
|
||||
data: dict | None = None,
|
||||
verify_with: str | None = None,
|
||||
expires_when: str | None = None,
|
||||
) -> Note:
|
||||
# Empty means empty (NULLABLE_NOTE_TEXT), then the invariant. Both run
|
||||
# before anything is written, so an illegal shape never reaches the table.
|
||||
verify_with = verify_with or None
|
||||
expires_when = expires_when or None
|
||||
if verify_with or expires_when:
|
||||
guard_check_fields(status, note_type)
|
||||
|
||||
# Validate status/priority here so the MCP create_task path (which passes
|
||||
# them straight through) can't persist an out-of-enum value that the REST
|
||||
# route would have rejected — there's no DB CHECK on notes.status.
|
||||
@@ -145,6 +205,8 @@ async def create_note(
|
||||
task_kind=task_kind,
|
||||
arose_from_id=arose_from_id,
|
||||
data=data,
|
||||
verify_with=verify_with,
|
||||
expires_when=expires_when,
|
||||
)
|
||||
session.add(note)
|
||||
await session.commit()
|
||||
@@ -397,7 +459,20 @@ def minted_kind(kind: str) -> str:
|
||||
raise ValueError(f"kind must be one of {MINTABLE_KINDS}, got {kind!r}")
|
||||
|
||||
|
||||
async def update_note(user_id: int, note_id: int, **fields: object) -> Note | None:
|
||||
async def update_note(
|
||||
user_id: int, note_id: int, clear: Iterable[str] = (), **fields: object,
|
||||
) -> Note | None:
|
||||
"""Partial update. `clear` names fields to UNSET; **fields carries values.
|
||||
|
||||
Clearing is explicit and separate because a nullable field cannot be
|
||||
emptied by passing it: the MCP door reads "" as "leave this alone", so an
|
||||
agent filling two fields does not wipe the others, and a note that stops
|
||||
being a constraint genuinely needs its check removed. Naming the field is
|
||||
the one form that cannot happen by accident. The REST door, where a
|
||||
cleared form input arrives as "", reaches the same place through the
|
||||
NULLABLE_NOTE_TEXT normalisation below — two idioms, one outcome.
|
||||
(Same shape as rulebooks.update_rule, milestone 312 step 2.)
|
||||
"""
|
||||
async with async_session() as session:
|
||||
result = await session.execute(
|
||||
select(Note).where(Note.id == note_id, Note.user_id == user_id)
|
||||
@@ -409,6 +484,10 @@ async def update_note(user_id: int, note_id: int, **fields: object) -> Note | No
|
||||
old_body = note.body
|
||||
old_title = note.title
|
||||
old_tags = list(note.tags or [])
|
||||
check_before = note.verify_with
|
||||
for key in clear:
|
||||
if key in NULLABLE_NOTE_TEXT:
|
||||
setattr(note, key, None)
|
||||
for key, value in fields.items():
|
||||
if not hasattr(note, key):
|
||||
continue
|
||||
@@ -438,7 +517,44 @@ async def update_note(user_id: int, note_id: int, **fields: object) -> Note | No
|
||||
)
|
||||
elif key == "tags" and isinstance(value, list):
|
||||
value = _normalize_tags(value)
|
||||
elif key in NULLABLE_NOTE_TEXT:
|
||||
value = value or None
|
||||
elif key == "verified_at":
|
||||
# Not settable here. A stamp says somebody performed THIS
|
||||
# check, so it is written by the verification path and by a
|
||||
# restore, never by an ordinary edit that could mint one for a
|
||||
# check nobody ran.
|
||||
continue
|
||||
setattr(note, key, value)
|
||||
# The invariant, over the RESULTING record rather than over what was
|
||||
# passed — which is what catches a checked note being turned into a
|
||||
# task. Raised before commit, so nothing is persisted.
|
||||
if note.verify_with or note.expires_when:
|
||||
guard_check_fields(note.status, note.note_type)
|
||||
# A stamp certifies A CHECK, not a record. Rewrite or remove the check
|
||||
# and the old stamp certifies something that no longer exists, so it is
|
||||
# dropped and the note re-enters the sweep. The safe direction: a note
|
||||
# wrongly listed as due costs one look; a note wrongly vouched for
|
||||
# costs exactly what the sweep exists to catch.
|
||||
if note.verify_with != check_before:
|
||||
note.verified_at = None
|
||||
# A snippet's `data` is DERIVED from its body — so a write that moves
|
||||
# the body through this generic door must move the mirror with it
|
||||
# (#3128). Without this, PATCH /api/notes/<snippet_id> {body} left the
|
||||
# mirror behind, and snippet_fields PREFERS the mirror: the row went on
|
||||
# reporting its old repo/path/symbol to prior-art recall while showing
|
||||
# its new body. `update_snippet` composes the mirror itself and passes
|
||||
# it explicitly, so an explicit `data` always wins — the caller that
|
||||
# knows the field set beats the one that can only re-read the body.
|
||||
if "data" not in fields and not _PARSED_FROM_BODY.isdisjoint(fields):
|
||||
# Imported here, not at module scope: services/snippets.py calls
|
||||
# back into this module (update_snippet -> update_note), so a
|
||||
# top-level import is a cycle.
|
||||
from scribe.services.snippets import (
|
||||
SNIPPET_NOTE_TYPE, recompose_data,
|
||||
)
|
||||
if note.note_type == SNIPPET_NOTE_TYPE:
|
||||
note.data = recompose_data(note)
|
||||
# Auto-set lifecycle timestamps on status transitions
|
||||
if "status" in fields:
|
||||
_now = datetime.now(timezone.utc)
|
||||
@@ -487,6 +603,154 @@ async def update_note(user_id: int, note_id: int, **fields: object) -> Note | No
|
||||
# permanent deletion. Both are reachable; neither is spelled `delete_note`.
|
||||
|
||||
|
||||
# ── The sweep (milestone 317 step 3) ─────────────────────────────────────────
|
||||
#
|
||||
# A SIBLING of rulebooks.rules_due_for_verification, not a shared
|
||||
# implementation, and deliberately so (note 3163). The row could have been
|
||||
# shared; the QUERY cannot. That sweep scopes by rulebook ownership XOR project
|
||||
# ownership because rules have no sharing ACL at all — no rule_shares, no
|
||||
# can_read_rule. A note scopes by the note ACL, which is a different question
|
||||
# with a different answer. What IS common — how a stamp reads, how old it is —
|
||||
# lives in services/verification.py and is imported by both.
|
||||
|
||||
async def notes_due_for_verification(
|
||||
user_id: int,
|
||||
older_than_days: int = 0,
|
||||
project_id: int | None = None,
|
||||
never_only: bool = False,
|
||||
) -> list[Note]:
|
||||
"""Notes that carry a check, oldest verification first, never-checked top.
|
||||
|
||||
THE QUERY THE COLUMNS EXIST FOR. `verify_with` and `expires_when` are
|
||||
storage; this is what turns them into something that gets acted on.
|
||||
Without it, note decay is caught only when a human reads the note and
|
||||
disagrees — which is the case where the note was already believed.
|
||||
|
||||
Ordered `verified_at` ASC **NULLS FIRST**: never-checked outranks
|
||||
checked-long-ago, because a note nobody has ever confirmed is a claim with
|
||||
no evidence behind it at all. Postgres sorts NULLs LAST on ASC by default,
|
||||
so this is explicit — and getting it wrong would not error, it would
|
||||
silently invert the one signal the sweep exists to carry.
|
||||
|
||||
Notes with no `verify_with` never appear. Not an omission: they are
|
||||
decisions, there is nothing to go and check, and listing them would dilute
|
||||
the result until nobody reads it.
|
||||
|
||||
Scoped with `browsable_notes_clause`, NOT the read scope (decision note
|
||||
2094): a sweep is a passive surface, and a record shared one-to-one must
|
||||
not arrive in one unasked.
|
||||
|
||||
Deliberately NOT filtered to non-task, non-snippet records even though the
|
||||
write path (step 2) permits a check on nothing else. A row in that state
|
||||
would be a row in an ILLEGAL state, and this is the one surface that could
|
||||
tell somebody about it. Hiding it here to match the invariant would make
|
||||
the sweep agree with a database it had stopped describing.
|
||||
|
||||
Args:
|
||||
user_id: whose notes.
|
||||
older_than_days: only notes last verified longer ago than this.
|
||||
Never-checked notes always qualify — they are the most overdue
|
||||
thing there is. 0 = no age filter. Negative raises: it would mean
|
||||
"everything", which is a different question than the one asked,
|
||||
answered silently.
|
||||
project_id: narrow to one project. None = every project.
|
||||
never_only: only notes that have never been verified.
|
||||
"""
|
||||
from datetime import timedelta
|
||||
|
||||
from scribe.services.access import browsable_notes_clause
|
||||
|
||||
if older_than_days < 0:
|
||||
raise ValueError(
|
||||
f"older_than_days must be >= 0, got {older_than_days}. A negative "
|
||||
f"window silently means 'everything', which is not what any caller "
|
||||
f"of a staleness sweep is asking."
|
||||
)
|
||||
|
||||
async with async_session() as session:
|
||||
# One statement, not a fetch-then-filter: the ordering below is the
|
||||
# database's, so it cannot disagree with itself across two halves.
|
||||
stmt = (
|
||||
select(Note)
|
||||
.where(
|
||||
browsable_notes_clause(user_id),
|
||||
Note.deleted_at.is_(None),
|
||||
Note.verify_with.is_not(None),
|
||||
)
|
||||
)
|
||||
if project_id is not None:
|
||||
stmt = stmt.where(Note.project_id == project_id)
|
||||
if never_only:
|
||||
stmt = stmt.where(Note.verified_at.is_(None))
|
||||
elif older_than_days > 0:
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=older_than_days)
|
||||
stmt = stmt.where(
|
||||
or_(Note.verified_at.is_(None), Note.verified_at < cutoff)
|
||||
)
|
||||
stmt = stmt.order_by(Note.verified_at.asc().nullsfirst(), Note.id)
|
||||
return list((await session.execute(stmt)).scalars().all())
|
||||
|
||||
|
||||
def verification_row(note: Note) -> dict:
|
||||
"""One row of the sweep — the CHECK in full, unlike a listing.
|
||||
|
||||
The opposite call from a browse: here the caller is about to go and run the
|
||||
check, so the text they need IS the payload rather than the bloat.
|
||||
"""
|
||||
from scribe.services.verification import (
|
||||
days_since_verified,
|
||||
last_verified_label,
|
||||
)
|
||||
|
||||
return {
|
||||
"id": note.id,
|
||||
"title": note.title,
|
||||
"project_id": note.project_id,
|
||||
"verify_with": note.verify_with or "",
|
||||
"expires_when": note.expires_when or "",
|
||||
"last_verified": last_verified_label(note),
|
||||
"days_since_verified": days_since_verified(note),
|
||||
}
|
||||
|
||||
|
||||
async def mark_note_verified(
|
||||
note_id: int, user_id: int, still_true: bool = True,
|
||||
) -> Note | None:
|
||||
"""Stamp a note as verified — or, when the check FAILED, refuse to.
|
||||
|
||||
The asymmetry is the design: passing writes a stamp, failing writes
|
||||
nothing. There is no "verified false" state, because a note whose check
|
||||
failed is not a note in a special condition — it is a note that is WRONG,
|
||||
and the honest resolutions are to correct it, supersede it, or find out
|
||||
why. Recording the failure as a flag would let it sit there being false
|
||||
with the sweep quietly satisfied that somebody had looked.
|
||||
|
||||
So a failed check leaves `verified_at` untouched and the note stays at the
|
||||
top of the sweep until someone actually deals with it.
|
||||
|
||||
Write access, not read (rules 47/78): stamping is a mutation, and an
|
||||
editor-share holder may make it while a viewer may not.
|
||||
|
||||
Returns None when the note is not found, not writable, or carries no
|
||||
`verify_with` — nothing to verify is a different answer from verified.
|
||||
"""
|
||||
from scribe.services.access import can_write_note
|
||||
|
||||
async with async_session() as session:
|
||||
note = (await session.execute(
|
||||
select(Note).where(Note.id == note_id, Note.deleted_at.is_(None))
|
||||
)).scalars().first()
|
||||
if note is None or not note.verify_with:
|
||||
return None
|
||||
if not await can_write_note(user_id, note_id):
|
||||
return None
|
||||
if still_true:
|
||||
note.verified_at = datetime.now(timezone.utc)
|
||||
await session.commit()
|
||||
await session.refresh(note)
|
||||
return note
|
||||
|
||||
|
||||
async def get_all_tags(user_id: int, q: str | None = None) -> list[str]:
|
||||
async with async_session() as session:
|
||||
if q:
|
||||
|
||||
@@ -60,7 +60,7 @@ async def start_planning(user_id: int, project_id: int, title: str) -> dict:
|
||||
|
||||
return {
|
||||
"milestone": milestone.to_dict(),
|
||||
**rulebooks_svc.rules_payload(applicable),
|
||||
**rulebooks_svc.rules_payload(applicable, user_id=user_id, source="start_planning"),
|
||||
"project_goal": getattr(project, "goal", "") or "",
|
||||
"open_task_count": open_count,
|
||||
}
|
||||
|
||||
@@ -32,6 +32,7 @@ from scribe.services import snippets as snippets_svc
|
||||
from scribe.services.access import label_shared_items, owner_names_for
|
||||
from scribe.services.embeddings import semantic_search_notes, semantic_search_rules
|
||||
from scribe.services.note_usage import record_surfaced
|
||||
from scribe.services.rule_usage import record_rule_surfaced
|
||||
from scribe.services.supersession import superseded_ids
|
||||
from scribe.services.retrieval_telemetry import record_retrieval
|
||||
from scribe.services.settings import get_setting
|
||||
@@ -85,6 +86,117 @@ WRITEPATH_THRESHOLD_KEY = "kb_writepath_threshold"
|
||||
WRITEPATH_DEFAULT_ENABLED = True
|
||||
WRITEPATH_DEFAULT_THRESHOLD = 0.68
|
||||
|
||||
# The standing-rule arm (milestone 307) gets its own bar — the split #2223 made
|
||||
# one surface down, now made for the THIRD corpus. It inherited 0.68 above, and
|
||||
# that number was measured against code-vs-note-PROSE. It was never re-derived
|
||||
# for code-vs-RULE-TEXT.
|
||||
#
|
||||
# THE STRUCTURAL ARGUMENT, which is the only kind admissible here (rule 115).
|
||||
# Two facts hold on any install, including one with six rules and no telemetry:
|
||||
#
|
||||
# 1. The eligible corpus is SMALL — every rule an install owns, still only
|
||||
# a few dozen documents against thousands of notes. A top-k over a small
|
||||
# pool always returns something, so "the best match cleared the bar"
|
||||
# drifts from "a good match exists" toward "N things were ranked". A bar
|
||||
# calibrated for best-of-thousands is cleared by best-of-forty as
|
||||
# arithmetic rather than relevance.
|
||||
# This argument WEAKENED when the arms stopped filtering to one tier
|
||||
# (see the note on that below): a larger pool makes clearing the bar
|
||||
# mean more, not less. The threshold was deliberately left where it was
|
||||
# anyway — moving two variables at once would make the resulting
|
||||
# distribution unreadable, and this one errs toward silence on purpose.
|
||||
# 2. Rules are short imperative technical English — a far more HOMOGENEOUS
|
||||
# corpus than note prose. #2223 measured the floor for code against prose
|
||||
# at 0.55-0.63 and set 0.68 above it. A more homogeneous corpus has a
|
||||
# HIGHER floor, so 0.68 is not merely inherited, it is below where this
|
||||
# corpus's noise sits.
|
||||
#
|
||||
# WHY 0.72 AND NOT A NUMBER OFF A HISTOGRAM. The exact offset between prose's
|
||||
# floor and rule-text's is not derivable in general — it depends on how an
|
||||
# install writes its rules — so the default errs deliberately toward SILENCE
|
||||
# rather than toward recall, on an asymmetry that is itself structural: this
|
||||
# hint fires on EVERY write. A missed rule is recoverable, because the rule is
|
||||
# still in Scribe and the agent can search it. A hint that cries wolf is not:
|
||||
# it teaches the reader to skip the whole block, and the surface is lost along
|
||||
# with the true positives it would have carried. The arm's own comment already
|
||||
# says "noise on a hint that fires on every write is how a hint gets ignored".
|
||||
#
|
||||
# TUNE IT FROM YOUR OWN INSTANCE, which is now possible: `retrieval_telemetry`
|
||||
# reports `rule_usage.pull_through` (milestone 333 step 3). Raise this if rules
|
||||
# arrive unread; lower it if rules you needed never arrived. What would RETIRE
|
||||
# it: a cross-encoder rerank (#1038), which would make a similarity bar the
|
||||
# wrong control entirely.
|
||||
RULEHINT_THRESHOLD_KEY = "kb_rulehint_threshold"
|
||||
RULEHINT_DEFAULT_THRESHOLD = 0.72
|
||||
|
||||
# ONE rule per write, not two — and this is deliberately NOT a knob.
|
||||
#
|
||||
# With a corpus this small, top-k does as much damage as the threshold: k=2
|
||||
# over a few dozen candidates means the second line is almost always the
|
||||
# second-best noise, arriving with the same confident framing as the first.
|
||||
# Halving k halves that regardless of where the bar sits.
|
||||
#
|
||||
# It also BOUNDS the blast radius of widening the pool (below): with k=1 a
|
||||
# wider corpus can change WHICH rule surfaces and how often one does, but it
|
||||
# can never make a single hint longer. The loudness of one hint and the
|
||||
# eligibility of a rule are separate controls, and only one of them moved.
|
||||
#
|
||||
# It stays a constant because it is a decision about how LOUD one hint may be,
|
||||
# not a per-install tuning question. The hint already carries prior art, shape
|
||||
# signals and staleness; rules are the fourth voice in it, and a fourth voice
|
||||
# that speaks twice is where a reader stops reading. Nothing suggests an
|
||||
# operator wants this different, and a knob nobody turns is a knob that only
|
||||
# adds a way to misconfigure the surface (rule 25 cuts both ways).
|
||||
RULEHINT_LIMIT = 1
|
||||
|
||||
# WHY THE ARMS NO LONGER FILTER TO ONE TIER (#3702).
|
||||
#
|
||||
# Both arms used to pass `tier="conditional"`, on the reasoning that an
|
||||
# always-on rule is already in the session, so surfacing it again is pure
|
||||
# noise. That reasoning conflates two different things:
|
||||
#
|
||||
# PRESENT IN CONTEXT — the rule was delivered at session start.
|
||||
# SALIENT AT THE MOMENT — the rule is in front of the reader when the
|
||||
# action it governs is about to be taken.
|
||||
#
|
||||
# A rule handed over in a list at turn zero is present while a session writes
|
||||
# a config value three hundred turns later. It is not surfaced. So the filter
|
||||
# did not merely skip a redundant hint — it made a whole class of rules
|
||||
# permanently ineligible for the only mechanism that puts a rule in front of
|
||||
# an agent AT the moment, and the more important a rule is, the more likely
|
||||
# it was in that class.
|
||||
#
|
||||
# The deeper defect is that the filter was doing the THRESHOLD's job. Whether
|
||||
# a rule belongs in this hint is a relevance question, and a similarity bar is
|
||||
# the control for relevance. A categorical exclusion standing in for a
|
||||
# relevance judgment cannot be tuned, cannot be measured, and cannot be wrong
|
||||
# in a way anybody notices.
|
||||
#
|
||||
# THIS IS A MEASURED CHANGE, NOT A SETTLED ONE. The old comment's fear is
|
||||
# real — a hint that fires on every write and says obvious things teaches the
|
||||
# reader to skip the block, and the surface is then lost along with its true
|
||||
# positives. That fear had simply never been checked. `retrieval_logs` already
|
||||
# records top_score, result_count and the query for every call, so the
|
||||
# evidence now arrives on its own:
|
||||
#
|
||||
# - rules clear the bar often and at high scores -> the fear was justified,
|
||||
# the filter was a crude proxy for a bar set too low, and the WORK IS THE
|
||||
# BAR. Any reinstated filter should then carry a measured reason.
|
||||
# - rules clear rarely, in a thin band near the bar -> the filter was never
|
||||
# the right instrument and relevance was always sufficient.
|
||||
#
|
||||
# Only the eligibility moved. The bar and k=1 were both left exactly where
|
||||
# they were, so the resulting distribution has one cause.
|
||||
|
||||
# How much of a command reaches the embedding (#3476). A shell call is not a
|
||||
# file: most are short, and the ones that are not are usually a heredoc or a
|
||||
# pasted script whose bulk says nothing about which rule applies. The VERB AND
|
||||
# ITS TARGET sit at the front — `curl https://git.fabledsword.com/api/...`,
|
||||
# `docker compose up`, `git checkout -b` — and that head is the whole signal.
|
||||
# Sending the tail as well would push it out of a 512-token window and let a
|
||||
# heredoc's prose decide the match.
|
||||
_TOOL_QUERY_CHARS = 400
|
||||
|
||||
# Minimum SUBSTANCE (non-whitespace chars) a payload must carry before the
|
||||
# semantic arm will run at all — the cheap half of the operator's #89 idea
|
||||
# ("a sliding scale between number of characters and semantic threshold").
|
||||
@@ -351,6 +463,7 @@ async def _reserve_slot_for_reuse(
|
||||
|
||||
top_k = cfg["top_k"]
|
||||
_t0 = time.perf_counter()
|
||||
_rep: dict = {}
|
||||
reuse = await semantic_search_notes(
|
||||
user_id, query,
|
||||
limit=1,
|
||||
@@ -359,6 +472,7 @@ async def _reserve_slot_for_reuse(
|
||||
exclude_ids=exclude_ids | {int(n.id) for _s, n in kept},
|
||||
note_type=_REUSE_KINDS,
|
||||
scope="browse",
|
||||
report=_rep,
|
||||
)
|
||||
# A real semantic query competing for a menu slot — logged like the scored
|
||||
# arm it displaces. Before this, the hit it PUSHED OUT was in
|
||||
@@ -369,6 +483,7 @@ async def _reserve_slot_for_reuse(
|
||||
user_id=user_id, source="reuse_slot", query=query,
|
||||
threshold=cfg["threshold"], limit=1, project_id=project_id,
|
||||
is_task=None, results=reuse,
|
||||
best_available=_rep.get("best_available_score"),
|
||||
duration_ms=(time.perf_counter() - _t0) * 1000.0,
|
||||
)
|
||||
# Verify the kind rather than trusting the query that asked for it, and
|
||||
@@ -418,6 +533,7 @@ async def build_autoinject_hint(
|
||||
return empty
|
||||
|
||||
t0 = time.perf_counter()
|
||||
_rep_ai: dict = {}
|
||||
hits = await semantic_search_notes(
|
||||
user_id, q,
|
||||
limit=cfg["top_k"],
|
||||
@@ -429,11 +545,13 @@ async def build_autoinject_hint(
|
||||
# still appear is a collaborator's note inside a shared project — legible
|
||||
# only because the line below names its owner.
|
||||
scope="browse",
|
||||
report=_rep_ai,
|
||||
)
|
||||
record_retrieval(
|
||||
user_id=user_id, source="auto_inject", query=q,
|
||||
threshold=cfg["threshold"], limit=cfg["top_k"],
|
||||
project_id=(project_id or None), is_task=None, results=hits,
|
||||
best_available=_rep_ai.get("best_available_score"),
|
||||
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
||||
)
|
||||
if not hits:
|
||||
@@ -690,10 +808,19 @@ async def get_writepath_config(user_id: int) -> dict:
|
||||
threshold = WRITEPATH_DEFAULT_THRESHOLD
|
||||
threshold = min(1.0, max(0.0, threshold))
|
||||
|
||||
try:
|
||||
rule_threshold = float(await get_setting(
|
||||
user_id, RULEHINT_THRESHOLD_KEY, str(RULEHINT_DEFAULT_THRESHOLD)))
|
||||
except (TypeError, ValueError):
|
||||
rule_threshold = RULEHINT_DEFAULT_THRESHOLD
|
||||
rule_threshold = min(1.0, max(0.0, rule_threshold))
|
||||
|
||||
return {
|
||||
**cfg,
|
||||
"enabled": enabled_raw.strip().lower() in ("true", "1", "yes", "on"),
|
||||
"threshold": threshold,
|
||||
# Its own bar, for a third corpus — see RULEHINT_DEFAULT_THRESHOLD.
|
||||
"rule_threshold": rule_threshold,
|
||||
}
|
||||
|
||||
|
||||
@@ -708,6 +835,7 @@ async def build_write_path_hint(
|
||||
repo_key: str = "",
|
||||
exclude_derive: list[str] | None = None,
|
||||
exclude_rule_ids: list[int] | None = None,
|
||||
rules_etag: str = "",
|
||||
) -> dict:
|
||||
"""Prior-art hint for the plugin's PreToolUse hook on Write/Edit.
|
||||
|
||||
@@ -850,6 +978,7 @@ async def build_write_path_hint(
|
||||
# Pulled-and-seen ids stay in the query (as evidence) but never in
|
||||
# the menu — the dedup contract holds, the resemblance still lands.
|
||||
pulled_seen = seen & set(pulled)
|
||||
_rep_wp: dict = {}
|
||||
hits = await semantic_search_notes(
|
||||
user_id, query,
|
||||
limit=remaining + len(pulled_seen),
|
||||
@@ -873,12 +1002,40 @@ async def build_write_path_hint(
|
||||
# Same reasoning as auto-inject: nobody asked for this, so it takes
|
||||
# the browse scope and never surfaces a one-to-one direct share.
|
||||
scope="browse",
|
||||
report=_rep_wp,
|
||||
)
|
||||
resembles = {
|
||||
int(note.id): float(score) for score, note in hits
|
||||
if int(note.id) in pulled
|
||||
}
|
||||
hits = [(s, n) for s, n in hits if int(n.id) not in seen][:remaining]
|
||||
shown = [(s, n) for s, n in hits if int(n.id) not in seen]
|
||||
# WHAT THIS ARM WITHHELD AFTER THE SEARCH ANSWERED, and the reason
|
||||
# `best_available_score` cannot always be reported here (#3739 again,
|
||||
# from the side its fix did not reach).
|
||||
#
|
||||
# This arm is the one note arm that filters TWICE. `exclude_ids` takes
|
||||
# `seen - pulled_seen` into the search, but the pulled-and-seen ids stay
|
||||
# in the query deliberately — `resembles` above needs them — and are
|
||||
# dropped in the line above instead. So the score the search reported is
|
||||
# PRE that drop while the row's `result_count` is POST it, and a record
|
||||
# the session had already been shown could be logged as something the
|
||||
# BAR turned away. Live proof on the first read after #3739 shipped:
|
||||
# write_path's near-miss max was 0.822 while the lowest score it ever
|
||||
# RETURNED was 0.6857 — a "rejection" that beat every acceptance.
|
||||
#
|
||||
# The suppression column cannot rescue it the way it does for the rule
|
||||
# arms: this arm's count would be PARTIAL, covering only the drops made
|
||||
# here and not the ones `exclude_ids` made inside the search, and a
|
||||
# partial number under a name that reads as complete is the substitution
|
||||
# this whole milestone exists to stop.
|
||||
#
|
||||
# So the honest answer is null — "not measured on this call" — whenever
|
||||
# this filter removed anything, because then the bar is not the only
|
||||
# thing that turned something away and the reported score may belong to
|
||||
# a record we withheld ourselves. Calls where nothing was dropped keep
|
||||
# reporting it, which is most of them.
|
||||
withheld_here = len(hits) - len(shown)
|
||||
hits = shown[:remaining]
|
||||
record_retrieval(
|
||||
user_id=user_id, source="write_path", query=query,
|
||||
threshold=cfg["threshold"], limit=remaining,
|
||||
@@ -886,6 +1043,9 @@ async def build_write_path_hint(
|
||||
# recording it as a notes-only retrieval would misdescribe the
|
||||
# candidate set the threshold is being tuned against.
|
||||
project_id=scope_project, is_task=None, results=hits,
|
||||
best_available=(
|
||||
None if withheld_here else _rep_wp.get("best_available_score")
|
||||
),
|
||||
duration_ms=(time.perf_counter() - t0) * 1000.0,
|
||||
)
|
||||
if hits:
|
||||
@@ -951,7 +1111,74 @@ async def build_write_path_hint(
|
||||
derive = [d for d in found if d.get("key") not in skip]
|
||||
except Exception:
|
||||
logger.warning("write-time derive check failed", exc_info=True)
|
||||
if not synced and not menu and not stamped and not divergence and not derive:
|
||||
staleness: list[str] = []
|
||||
# ── Have the rules moved under this session? (milestone 323) ───────
|
||||
#
|
||||
# THE CARRIER IS THE POINT. This hook already fires before a write — the
|
||||
# moment acting on a stale rule actually costs something — and the check
|
||||
# is one comparison against a marker the session already holds. No
|
||||
# payload, no extra round trip, and nothing said when nothing moved.
|
||||
#
|
||||
# WHAT THIS CANNOT SEE, and a reader who finds an etag here will assume
|
||||
# otherwise:
|
||||
#
|
||||
# what goes wrong | caught?
|
||||
# ---------------------------------------------------|--------
|
||||
# another session edits a rule mid-flight | yes
|
||||
# the session is misremembering a rule read hours ago | yes
|
||||
# compaction summarised the rules out of context | NO
|
||||
#
|
||||
# The third is the most common and this is blind to it: the etag was in
|
||||
# context too and went with the rules. The SessionStart nudge is that
|
||||
# case's only mechanism and must not be softened because this shipped.
|
||||
#
|
||||
# Fails open, like every other arm here: a staleness hint must never
|
||||
# break a write.
|
||||
if rules_etag:
|
||||
try:
|
||||
current = await rulebooks_svc.list_always_on_rules(
|
||||
user_id, project_id=project_id or 0,
|
||||
)
|
||||
if rulebooks_svc.rules_etag(current) != rules_etag:
|
||||
moved = rulebooks_svc.rules_moved_since(current, rules_etag)
|
||||
held = rulebooks_svc.etag_count(rules_etag)
|
||||
bits = []
|
||||
if moved:
|
||||
named = ", ".join(
|
||||
f"#{r.id} \u201c{r.title}\u201d" for r in moved[:3]
|
||||
)
|
||||
more = len(moved) - 3
|
||||
bits.append(
|
||||
f"{named}" + (f", and {more} more" if more > 0 else "")
|
||||
)
|
||||
# A DELETED rule moves no timestamp and leaves no row to name,
|
||||
# so the count is the only thing that can report the one change
|
||||
# that takes an instruction OUT of force.
|
||||
if held is not None and held != len(current):
|
||||
delta = len(current) - held
|
||||
bits.append(
|
||||
f"{abs(delta)} rule(s) {'added' if delta > 0 else 'no longer in force'}"
|
||||
)
|
||||
if bits:
|
||||
staleness.append(
|
||||
"Your loaded rules have changed since this session "
|
||||
"started — " + "; ".join(bits) + ". Re-read them with "
|
||||
"list_always_on_rules() before relying on the set you "
|
||||
"are holding."
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("write-path rules-etag arm failed", exc_info=True)
|
||||
|
||||
# The guard sits BELOW the staleness arm on purpose. A rules change is
|
||||
# unconditional news — it does not become less true because this
|
||||
# particular write happened to match no prior art — and this arm is one
|
||||
# indexed query, only when the session actually sent a marker.
|
||||
#
|
||||
# The standing-rule arm further down is deliberately left on the far side
|
||||
# of this guard: that one runs a SEMANTIC search, and moving it here would
|
||||
# run an embedding query on every write in the session. Its gating is a
|
||||
# separate question from this one (see the note on #3244).
|
||||
if not staleness and not synced and not menu and not stamped and not divergence and not derive:
|
||||
return empty
|
||||
|
||||
owners = await owner_names_for({
|
||||
@@ -970,7 +1197,9 @@ async def build_write_path_hint(
|
||||
for marker, item in menu:
|
||||
rendered.append((item, marker, _owner_of(item), _foreign_language(item, target_lang)))
|
||||
|
||||
lines: list[str] = []
|
||||
# Seeded with the staleness line, which is decided above the early
|
||||
# return and so cannot wait for this list to exist.
|
||||
lines: list[str] = list(staleness)
|
||||
sync_note_ids: list[int] = []
|
||||
if synced:
|
||||
# The sync framing (#2708). Deliberately imperative about the record —
|
||||
@@ -1055,10 +1284,18 @@ async def build_write_path_hint(
|
||||
rule_ids: list[int] = []
|
||||
try:
|
||||
already = set(exclude_rule_ids or [])
|
||||
# Timed like the notes arm above. Without this the rule row was the one
|
||||
# source in the whole readout reporting a null p90_duration_ms (#3311)
|
||||
# — a gap that reads as "this surface is somehow not measurable" rather
|
||||
# than "nobody passed the number".
|
||||
rule_t0 = time.perf_counter()
|
||||
_rep_wpr: dict = {}
|
||||
hits = await semantic_search_rules(
|
||||
user_id, code or path, limit=2,
|
||||
threshold=cfg["threshold"], tier="conditional",
|
||||
user_id, code or path, limit=RULEHINT_LIMIT,
|
||||
threshold=cfg["rule_threshold"],
|
||||
report=_rep_wpr,
|
||||
)
|
||||
rule_ms = (time.perf_counter() - rule_t0) * 1000.0
|
||||
fresh = [(score, rule) for score, rule in hits if rule.id not in already]
|
||||
for _score, rule in fresh:
|
||||
trigger = (rule.when_to_apply or "").strip()
|
||||
@@ -1069,15 +1306,55 @@ async def build_write_path_hint(
|
||||
"does not apply; it is not in this session's loaded set."
|
||||
)
|
||||
rule_ids.append(rule.id)
|
||||
# TWO tables, and the split is not arbitrary. retrieval_logs is one
|
||||
# row per CALL, keyed on the score distribution a threshold is tuned
|
||||
# from. rule_usage_events is one row per RULE per event, which is the
|
||||
# grain "was this hint ever acted on" needs and the grain a JSONB
|
||||
# result_ids array cannot be indexed at.
|
||||
#
|
||||
# This comment used to say rule ids had nowhere to go — that
|
||||
# note_usage_events remaps ids on restore, so a rule id there would
|
||||
# return attached to whatever note took that number. That is still
|
||||
# true of the NOTE table, and it is exactly why rule_usage_events is
|
||||
# its own (milestone 333 step 1). The gap it described is closed.
|
||||
#
|
||||
# THE CALL LOG IS UNCONDITIONAL; THE SURFACING LOG IS NOT, and the
|
||||
# asymmetry is the correction #3497 exists to make. Both used to sit
|
||||
# inside an `if fresh:`, which is how this arm came to report
|
||||
# `zero_result_calls: 0` and `cleared_threshold: 133/133` — not a
|
||||
# perfectly tuned surface but one structurally unable to record its
|
||||
# own misses. #3311 read that artifact as a measurement and a whole
|
||||
# milestone was scoped on it. A call that found nothing is the ONLY
|
||||
# evidence a threshold is set too high, and it is the row every note
|
||||
# surface has always written (write_path: 421 zeroes of 613 calls;
|
||||
# auto_inject: 114 of 326). A SURFACING is different in kind: nothing
|
||||
# was shown, so no such event occurred, and its log stays guarded.
|
||||
#
|
||||
# `results=fresh`, not `hits`: the note arms pass their exclusions
|
||||
# INTO semantic_search_notes, so what they log is already
|
||||
# post-exclusion. semantic_search_rules takes no such parameter and
|
||||
# this filter is where the equivalent happens — logging `hits` would
|
||||
# quietly make this row mean something other than every other row in
|
||||
# the same readout.
|
||||
record_retrieval(
|
||||
user_id=user_id, source="write_path_rule", query=code or path,
|
||||
threshold=cfg["rule_threshold"], limit=RULEHINT_LIMIT,
|
||||
project_id=project_id,
|
||||
is_task=None, results=fresh, duration_ms=rule_ms,
|
||||
best_available=_rep_wpr.get("best_available_score"),
|
||||
# What the ranker found and this session had already been told.
|
||||
# Without it a zero row cannot say whether the bar was too high or
|
||||
# the reader was simply ahead of it — and only the first is a
|
||||
# reason to move the threshold.
|
||||
suppressed=len(hits) - len(fresh),
|
||||
)
|
||||
if fresh:
|
||||
# retrieval_logs, NOT note_usage_events: that table's ids are
|
||||
# remapped on a backup restore, so a rule id there would return
|
||||
# attached to whatever note took that number. This one is never
|
||||
# restored, and `source` already separates the surfaces.
|
||||
record_retrieval(
|
||||
user_id=user_id, source="write_path_rule", query=code or path,
|
||||
threshold=cfg["threshold"], limit=2, project_id=project_id,
|
||||
is_task=None, results=fresh,
|
||||
# `rule_ids` is `fresh`, i.e. AFTER exclude_rule_ids. A rule the
|
||||
# session already holds was considered and not shown, and counting
|
||||
# it would inflate the denominator with claims the agent never saw
|
||||
# — which reads as a precision problem this arm does not have.
|
||||
record_rule_surfaced(
|
||||
user_id=user_id, rule_ids=rule_ids, source="write_path_rule",
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("write-path rule arm failed", exc_info=True)
|
||||
@@ -1095,6 +1372,117 @@ async def build_write_path_hint(
|
||||
}
|
||||
|
||||
|
||||
async def build_tool_rule_hint(
|
||||
user_id: int,
|
||||
tool_name: str,
|
||||
command: str,
|
||||
*,
|
||||
project_id: int = 0,
|
||||
exclude_rule_ids: list[int] | None = None,
|
||||
) -> dict:
|
||||
"""Standing rules that may apply to the ACTION about to be taken (#3476).
|
||||
|
||||
The sibling of the write-path rule arm, and the surface that was missing.
|
||||
That arm is keyed on `code or path`, so a rule can only be retrieved at the
|
||||
moment of a code WRITE. Every rule about which tool to reach for — don't
|
||||
curl the forge, don't stand up a stack, don't run the suite locally, don't
|
||||
branch — was therefore unreachable at the moment it mattered, and residency
|
||||
in the always-on preload was the only surface it had.
|
||||
|
||||
WHY A MECHANICAL TRIGGER AND NOT AN INSTRUCTION. Note #3089's finding is
|
||||
that a reflex generates no query: you reach for `curl` confidently, with no
|
||||
moment of doubt, so any surface that waits to be asked never fires. Here
|
||||
nothing has to be asked — the tool call IS the query, and the reflex has to
|
||||
become a tool call before it can do anything.
|
||||
|
||||
Deliberately TOOL-AGNOSTIC: takes a name and a string. The hook decides
|
||||
which tools it watches, so widening the matcher is a `hooks.json` edit with
|
||||
no change here.
|
||||
|
||||
CONDITIONAL ONLY, exactly as the write-path arm — an always-on rule is
|
||||
already resident and repeating it is noise. That filter is also the
|
||||
transition this arm exists to enable: re-tier a rule to `conditional` and
|
||||
it starts arriving here instead of in every session's preamble.
|
||||
|
||||
Fails open and returns an empty context on any error: a recall aid may
|
||||
never break the operator's action.
|
||||
"""
|
||||
out: dict = {"context": "", "rule_ids": []}
|
||||
command = (command or "").strip()
|
||||
if not command:
|
||||
return out
|
||||
|
||||
try:
|
||||
cfg = await get_writepath_config(user_id)
|
||||
if not cfg.get("enabled"):
|
||||
return out
|
||||
|
||||
# The command text is the query. A long heredoc or a pasted script
|
||||
# would otherwise push the meaningful head of the command out of the
|
||||
# embedding window, so it is bounded — the verb and its target sit at
|
||||
# the front, which is the part a rule is about.
|
||||
query = command[:_TOOL_QUERY_CHARS]
|
||||
|
||||
t0 = time.perf_counter()
|
||||
_rep_ptr: dict = {}
|
||||
hits = await semantic_search_rules(
|
||||
user_id, query, limit=RULEHINT_LIMIT,
|
||||
threshold=cfg["rule_threshold"],
|
||||
report=_rep_ptr,
|
||||
)
|
||||
duration_ms = (time.perf_counter() - t0) * 1000.0
|
||||
|
||||
already = set(exclude_rule_ids or [])
|
||||
fresh = [(score, rule) for score, rule in hits if rule.id not in already]
|
||||
|
||||
# Logged BEFORE the early return, for the reason spelled out at length
|
||||
# on the write-path arm above: a call that found nothing is the only
|
||||
# evidence a threshold is too high, and an arm that logs only the calls
|
||||
# it liked reports a flawless clear-rate however badly it is tuned.
|
||||
# This arm shipped with the same defect inherited from its sibling, and
|
||||
# it mattered more here — a surface with no rows at all cannot be told
|
||||
# apart from a hook that never fired, which is precisely the silent
|
||||
# failure the arm was built to stop.
|
||||
record_retrieval(
|
||||
user_id=user_id, source="pre_tool_rule", query=query,
|
||||
threshold=cfg["rule_threshold"], limit=RULEHINT_LIMIT,
|
||||
project_id=project_id,
|
||||
is_task=None, results=fresh, duration_ms=duration_ms,
|
||||
best_available=_rep_ptr.get("best_available_score"),
|
||||
# See the sibling arm. It matters more here: this arm fires on every
|
||||
# Bash call, so a long session excludes its way to an all-zero row
|
||||
# and the threshold looks wrong when nothing about it is.
|
||||
suppressed=len(hits) - len(fresh),
|
||||
)
|
||||
if not fresh:
|
||||
return out
|
||||
|
||||
lines: list[str] = []
|
||||
rule_ids: list[int] = []
|
||||
for _score, rule in fresh:
|
||||
trigger = (rule.when_to_apply or "").strip()
|
||||
lines.append(
|
||||
f"Standing rule that may apply to this {tool_name} call — "
|
||||
f"“{rule.title}”"
|
||||
+ (f" ({trigger})" if trigger else "")
|
||||
+ f". Read it with get_rule({rule.id}) before deciding it "
|
||||
"does not apply; it is not in this session's loaded set."
|
||||
)
|
||||
rule_ids.append(rule.id)
|
||||
|
||||
# RANKED, not ambient: this arm chose what it showed, so a pull can
|
||||
# settle whether the choice was any good. `rule_usage.RANKED_SOURCES`
|
||||
# carries the same name.
|
||||
record_rule_surfaced(
|
||||
user_id=user_id, rule_ids=rule_ids, source="pre_tool_rule",
|
||||
)
|
||||
out["context"] = "\n".join(lines)
|
||||
out["rule_ids"] = rule_ids
|
||||
except Exception:
|
||||
logger.debug("pre-tool rule arm failed", exc_info=True)
|
||||
return out
|
||||
|
||||
|
||||
def _derive_line(path: str, derive: list[dict]) -> str:
|
||||
"""The ledger's word on the names being written (#2900): a duplicate
|
||||
family to derive, or a canon to reuse — said at the write."""
|
||||
@@ -1208,7 +1596,10 @@ async def build_session_context(
|
||||
its normalized key — triggers a one-line "bind this repo" hint so
|
||||
the binding is self-healing.
|
||||
|
||||
Returns {"context": str, "rule_count": int, "project": dict | None}.
|
||||
Returns {"context": str, "rule_count": int, "project": dict | None,
|
||||
"rules_etag": str}. The etag is for the HOOK, not for the model — the
|
||||
hook stores it and hands it back on each write so the server can say
|
||||
whether these rules have moved since the session loaded them.
|
||||
`context` is markdown ready to drop into `additionalContext`; it is capped
|
||||
at _MAX_CHARS with an explicit truncation note so the hook can pass it
|
||||
through verbatim.
|
||||
@@ -1217,6 +1608,24 @@ async def build_session_context(
|
||||
# exclusion (milestone 297) takes a rulebook out of this block, and is
|
||||
# named below so the departure is visible rather than silent.
|
||||
rules = await rulebooks_svc.list_always_on_rules(user_id, project_id=project_id)
|
||||
# AMBIENT source, and the one that matters most: this is the preload — the
|
||||
# block every session opens with, chosen by nobody, paid for every turn.
|
||||
#
|
||||
# It emitted nothing until 2026-09-03, which made the resident set's cost
|
||||
# certain and its usefulness unfalsifiable at the same time (#3473). Note
|
||||
# #3089 is the argument this measurement finally lets someone test: that a
|
||||
# rule arriving with thirty others, none of them relevant, is read as
|
||||
# preamble rather than as a claim — so presence is not surfacing, and a
|
||||
# tier-1 set can grow without anybody noticing it stopped working.
|
||||
#
|
||||
# Recorded even when the hook truncates the block below: the rules WERE
|
||||
# delivered, and counting only the untruncated ones would quietly shrink
|
||||
# the denominator exactly where the set is too big to read.
|
||||
record_rule_surfaced(
|
||||
user_id=user_id,
|
||||
rule_ids=[r.id for r in rules],
|
||||
source="session_start",
|
||||
)
|
||||
excluded = (
|
||||
await rulebooks_svc.excluded_always_on_rulebooks(user_id, project_id)
|
||||
if project_id else []
|
||||
@@ -1317,4 +1726,12 @@ async def build_session_context(
|
||||
if len(context) > _MAX_CHARS:
|
||||
context = context[:_MAX_CHARS].rstrip() + "\n\n…(truncated — call list_always_on_rules())"
|
||||
|
||||
return {"context": context, "rule_count": len(rules), "project": project_dict}
|
||||
return {
|
||||
"context": context,
|
||||
"rule_count": len(rules),
|
||||
"project": project_dict,
|
||||
# Computed from the rules THIS payload was built from, not re-queried:
|
||||
# the marker has to describe the set the session is actually holding,
|
||||
# and a second query could disagree with the first.
|
||||
"rules_etag": rulebooks_svc.rules_etag(rules),
|
||||
}
|
||||
|
||||
@@ -27,6 +27,10 @@ from scribe.models import async_session
|
||||
from scribe.models.base import iso
|
||||
from scribe.models.note import Note
|
||||
from scribe.models.note_usage import PULLED, SURFACED, NoteUsageEvent
|
||||
from scribe.models.rule_usage import PULLED as RULE_PULLED
|
||||
from scribe.models.rule_usage import SURFACED as RULE_SURFACED
|
||||
from scribe.models.rule_usage import RuleUsageEvent
|
||||
from scribe.services.rule_usage import is_ambient
|
||||
from scribe.models.retrieval_log import RetrievalLog
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -51,12 +55,26 @@ def _build_payload(
|
||||
is_task: bool | None,
|
||||
results: list[tuple[float, Note]],
|
||||
duration_ms: float | None,
|
||||
suppressed: int | None = None,
|
||||
best_available: float | None = None,
|
||||
) -> dict:
|
||||
"""Reduce a retrieval call to a flat, JSON-safe RetrievalLog payload.
|
||||
|
||||
Pure and synchronous (no DB, no event loop) so it is unit-testable and safe
|
||||
to run inline before scheduling the write. `results` is the
|
||||
`(score, Note)` list from semantic_search_notes, already highest-first.
|
||||
|
||||
`suppressed` is how many scored hits the caller dropped because the session
|
||||
had already been shown them, and it stays None for callers that cannot
|
||||
know. See the column's comment: None means "not measured here", which is a
|
||||
different fact from 0 and must never render as one.
|
||||
|
||||
`best_available` is the highest score the ranker reached BEFORE the
|
||||
threshold, and it carries the same null discipline for a sharper reason: it
|
||||
is the only field that still says something on a call that returned
|
||||
nothing, so a 0.0 standing in for "not measured" would read as "the corpus
|
||||
held nothing remotely relevant" — a claim about the corpus invented out of
|
||||
a caller's silence.
|
||||
"""
|
||||
items = [
|
||||
{"id": int(note.id), "score": round(float(score), 5), "rank": rank}
|
||||
@@ -72,8 +90,12 @@ def _build_payload(
|
||||
"project_id": project_id,
|
||||
"is_task": is_task,
|
||||
"result_count": len(items),
|
||||
"suppressed_count": (None if suppressed is None else int(suppressed)),
|
||||
"top_score": (scores[0] if scores else None),
|
||||
"min_score": (scores[-1] if scores else None),
|
||||
"best_available_score": (
|
||||
None if best_available is None else round(float(best_available), 5)
|
||||
),
|
||||
"result_ids": items,
|
||||
"duration_ms": (round(duration_ms, 2) if duration_ms is not None else None),
|
||||
}
|
||||
@@ -111,6 +133,8 @@ def record_retrieval(
|
||||
is_task: bool | None,
|
||||
results: list[tuple[float, Any]],
|
||||
duration_ms: float | None = None,
|
||||
suppressed: int | None = None,
|
||||
best_available: float | None = None,
|
||||
) -> None:
|
||||
"""Fire-and-forget: record one retrieval call.
|
||||
|
||||
@@ -136,6 +160,8 @@ def record_retrieval(
|
||||
is_task=is_task,
|
||||
results=results,
|
||||
duration_ms=duration_ms,
|
||||
suppressed=suppressed,
|
||||
best_available=best_available,
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("retrieval telemetry payload build failed", exc_info=True)
|
||||
@@ -162,36 +188,145 @@ def record_retrieval(
|
||||
|
||||
def _bucket(rows: list) -> dict:
|
||||
"""A score readout a human can act on, from one aggregate row."""
|
||||
calls, zero, cleared, p10, p50, p90, lo, hi, avg_n, dur = rows
|
||||
(calls, zero, p10, p50, p90, lo, hi, avg_n, dur,
|
||||
measured, supp_calls, supp_zero,
|
||||
miss_calls, miss_p50, miss_p90, miss_max) = rows
|
||||
return {
|
||||
"calls": int(calls or 0),
|
||||
# A call that returned nothing is not a low-scoring call — it is a
|
||||
# different failure (nothing indexed, filter too narrow), and averaging
|
||||
# it into the score distribution would hide both.
|
||||
"zero_result_calls": int(zero or 0),
|
||||
# How often the best hit actually cleared the threshold in force for
|
||||
# that call. THE precision-adjacent number: a surface that clears its
|
||||
# bar on almost every call is either well-tuned or too loose, and the
|
||||
# score spread below says which.
|
||||
"cleared_threshold": int(cleared or 0),
|
||||
# `cleared_threshold` USED TO LIVE HERE and it was a tautology (#3670).
|
||||
# The search applies the bar before returning, so every returned result
|
||||
# cleared it by construction and a call with nothing has no score to
|
||||
# compare — the condition was true exactly when `result_count > 0`.
|
||||
# `zero_result_calls + cleared_threshold == calls` held on all nineteen
|
||||
# readings ever taken. It was `calls - zero_result_calls` wearing a name
|
||||
# that promised a second opinion, and the docstring built a reading
|
||||
# procedure on it that asked the reader to compare a number with itself.
|
||||
# Its replacement is `near_misses` below, which the bar cannot fix by
|
||||
# construction because it is measured on the calls the bar REJECTED.
|
||||
# Of the zeros above, which were the RANKER declining and which were
|
||||
# the reader having seen it already? `zero_result_calls` cannot say,
|
||||
# and only the first kind is evidence about the threshold.
|
||||
#
|
||||
# None — not a zeroed dict — when no row in the window reported it. A
|
||||
# surface that filters inside the search genuinely does not know, and
|
||||
# rendering that as `{"calls": 0}` would state a measurement nobody
|
||||
# made. That substitution is the whole of #3311.
|
||||
"suppression": (
|
||||
None if not int(measured or 0) else {
|
||||
"measured_calls": int(measured or 0),
|
||||
"calls_with_suppression": int(supp_calls or 0),
|
||||
# Subtract from zero_result_calls for the true ranker declines.
|
||||
"zero_because_already_shown": int(supp_zero or 0),
|
||||
}
|
||||
),
|
||||
"top_score": {
|
||||
"p10": _round(p10), "p50": _round(p50), "p90": _round(p90),
|
||||
"min": _round(lo), "max": _round(hi),
|
||||
},
|
||||
# WHAT THE BAR TURNED AWAY, and the only figure here a threshold can
|
||||
# actually be tuned from. Measured over the calls that returned
|
||||
# NOTHING, on the best score the ranker reached before the filter.
|
||||
#
|
||||
# Read `p90` against the threshold in force. A bar at 0.72 rejecting a
|
||||
# stream of 0.71s is set too high by a hair and the surface is losing
|
||||
# hits it should have had; the same bar rejecting 0.30s is doing its
|
||||
# job and the corpus simply had nothing. Both render as a zero-result
|
||||
# call, and nothing else in this readout separates them.
|
||||
#
|
||||
# None — not a zeroed block — when no declining call in the window
|
||||
# measured it. Old rows predate the column, and a 0.0 would assert that
|
||||
# the corpus held nothing relevant, which is a claim about the corpus
|
||||
# invented out of a caller's silence.
|
||||
"near_misses": (
|
||||
None if not int(miss_calls or 0) else {
|
||||
"measured_calls": int(miss_calls or 0),
|
||||
"p50": _round(miss_p50),
|
||||
"p90": _round(miss_p90),
|
||||
"max": _round(miss_max),
|
||||
}
|
||||
),
|
||||
"avg_result_count": _round(avg_n),
|
||||
"p90_duration_ms": _round(dur, 1),
|
||||
}
|
||||
|
||||
|
||||
# The aggregate row Postgres would have returned for a source with no rows in
|
||||
# the window: nothing counted, nothing scored. Positional, matching the SELECT
|
||||
# `_bucket` unpacks — calls, zero, p10, p50, p90, min, max, avg_n,
|
||||
# dur, measured, supp_calls, supp_zero, miss_calls, miss_p50, miss_p90,
|
||||
# miss_max. The counts are 0 because zero calls is a real observation;
|
||||
# everything else is None because a distribution nobody sampled has no value,
|
||||
# and rendering it as 0.0 would state one.
|
||||
_NO_ROWS_IN_WINDOW = [0, 0, None, None, None, None, None, None, None,
|
||||
0, 0, 0, 0, None, None, None]
|
||||
|
||||
|
||||
def _round(v, places: int = 4):
|
||||
return None if v is None else round(float(v), places)
|
||||
|
||||
|
||||
async def _complete_from(session, model, user_id) -> dict[str, Any]:
|
||||
"""When each source in `model` started being recorded, and the instant the
|
||||
WHOLE table is complete from. Returns {source: earliest_row, "*": latest}.
|
||||
|
||||
THE GRAIN IS THE SOURCE, and that is the whole point. `retrieval_logs` has
|
||||
rows going back months, so a table-level "earliest row" says months and
|
||||
tells a reader their window is fully covered — while a source added last
|
||||
week has a week of rows and a counter that silently means something else.
|
||||
Per-source is the only grain at which partial coverage is visible.
|
||||
|
||||
THE AGGREGATE USES THE LATEST, NOT THE EARLIEST. A number that sums several
|
||||
sources is complete only once EVERY contributor was recording, so "*" is a
|
||||
max over the sources, not a min. Taking the min here would reproduce the
|
||||
exact reading this exists to prevent: the oldest source vouching for the
|
||||
youngest.
|
||||
|
||||
All-time, deliberately unfiltered by the window — a query bounded by
|
||||
`since` can only ever report something at or after `since`, which answers
|
||||
nothing.
|
||||
"""
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(model.source, func.min(model.created_at))
|
||||
.where(model.user_id == user_id)
|
||||
.group_by(model.source)
|
||||
)
|
||||
).all()
|
||||
out: dict[str, Any] = {src: ts for src, ts in rows if ts is not None}
|
||||
stamps = list(out.values())
|
||||
out["*"] = max(stamps) if stamps else None
|
||||
return out
|
||||
|
||||
|
||||
def _coverage(complete_from, since) -> dict:
|
||||
"""The two keys every counter block carries, from one timestamp.
|
||||
|
||||
`covers_window` is None — never False — when nothing was ever recorded.
|
||||
"No rows at all" is not "partial coverage", it is no measurement, and the
|
||||
null convention #3497 established for `suppression` holds here for the
|
||||
same reason: absent must not read as a verdict.
|
||||
"""
|
||||
return {
|
||||
# iso() already returns None for an unset value (#2845) — the guard
|
||||
# belongs on covers_window, which is a verdict, not a serialisation.
|
||||
"complete_from": iso(complete_from),
|
||||
"covers_window": (
|
||||
None if complete_from is None else complete_from <= since
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
"""What the retrieval telemetry says, per surface, over a window.
|
||||
|
||||
Two aggregates side by side, each read from the table built for it — NOT a
|
||||
join. `NoteUsageEvent`'s own docstring is explicit that the two are
|
||||
Three aggregates side by side, each read from the table built for it — NOT
|
||||
a join. `usage` is notes, `rule_usage` is rules, and they stay apart
|
||||
because a few dozen eligible rules blended into thousands of notes is the
|
||||
note ratio with noise on it (milestone 333). `NoteUsageEvent`'s own docstring is explicit that the two are
|
||||
complements ("RetrievalLog tunes the threshold, this tunes the corpus") and
|
||||
that RetrievalLog's JSONB `result_ids` "can't be indexed at" the per-note
|
||||
grain. So the score distribution comes from `retrieval_logs` on its indexed
|
||||
@@ -199,6 +334,12 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
it was built for. Reading each from its own table is both cheaper and more
|
||||
honest than correlating them through JSONB.
|
||||
|
||||
`usage["by_source"]` is the one join, and it stays INSIDE
|
||||
`note_usage_events` — surfaced rows against pulled rows on note_id. That
|
||||
answers "of the notes this surface chose, how many were opened", which the
|
||||
top-level ratio averages away. It does not cross into `retrieval_logs`, so
|
||||
the sentence above still holds.
|
||||
|
||||
Scoped to one user's own telemetry. There is no sharing model for a
|
||||
retrieval log — it records what THIS user's agent asked for, including the
|
||||
query text — so an owner filter is the whole access rule here rather than a
|
||||
@@ -214,23 +355,77 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
"since": iso(since),
|
||||
"sources": {},
|
||||
"usage": {},
|
||||
"rule_usage": {},
|
||||
"read_failed": False,
|
||||
}
|
||||
|
||||
cleared = case(
|
||||
(
|
||||
(RetrievalLog.threshold.isnot(None))
|
||||
& (RetrievalLog.top_score.isnot(None))
|
||||
& (RetrievalLog.top_score >= RetrievalLog.threshold),
|
||||
1,
|
||||
),
|
||||
zero = case((RetrievalLog.result_count == 0, 1), else_=0)
|
||||
# THE NEAR-MISS POPULATION: calls that returned nothing BECAUSE THE BAR
|
||||
# TURNED SOMETHING AWAY, and recorded what it was. Three conditions, and
|
||||
# the third was missing for one deploy (#3739).
|
||||
#
|
||||
# Zero-result only: on a call that returned something,
|
||||
# `best_available_score` equals `top_score` and adds nothing.
|
||||
#
|
||||
# Non-null only: rows written before #3670 genuinely do not know, and must
|
||||
# not read as scoreless declines.
|
||||
#
|
||||
# AND NOT A REPEAT. A zero-result call is two unrelated events — the ranker
|
||||
# found nothing above the bar, or it found only what this session had
|
||||
# already been shown — and just the first says anything about the bar. That
|
||||
# is the whole of #3497, and #3670 reintroduced the conflation one level up:
|
||||
# the rule arms filter exclusions in PYTHON, after the search, so a rule
|
||||
# that cleared the bar and was dropped as a repeat still reported a high
|
||||
# `best_available_score` on a zero-result row. Live proof, first read after
|
||||
# deploy: pre_tool_rule's near-miss max was 0.7457 while the lowest score it
|
||||
# ever RETURNED was 0.7204 — a "rejection" that outscored acceptances.
|
||||
#
|
||||
# The NULL arm is principled, not permissive: `suppressed_count IS NULL`
|
||||
# means the caller passed its exclusions INTO the search, which is exactly
|
||||
# the case where the reported score is already post-exclusion and cannot be
|
||||
# contaminated. Note arms stay measured; rule arms get cleaned.
|
||||
#
|
||||
# Deliberately conservative: a call carrying both a repeat and a lower
|
||||
# genuine miss is dropped whole, losing that point. It undercounts; it
|
||||
# cannot corrupt — the right way round for a number read against a bar.
|
||||
#
|
||||
# This also makes `near_misses.max < threshold` true BY CONSTRUCTION. An
|
||||
# above-bar candidate that was not excluded would have been returned, so
|
||||
# its call is not in this population at all.
|
||||
declined = (
|
||||
(RetrievalLog.result_count == 0)
|
||||
& (RetrievalLog.best_available_score.isnot(None))
|
||||
& (
|
||||
RetrievalLog.suppressed_count.is_(None)
|
||||
| (RetrievalLog.suppressed_count == 0)
|
||||
)
|
||||
)
|
||||
miss = case((declined, 1), else_=0)
|
||||
# `best_available_score` only for those rows; NULL elsewhere, and
|
||||
# percentile_cont ignores NULLs, so the distribution is over the declines
|
||||
# alone without a second pass over the table.
|
||||
miss_score = case((declined, RetrievalLog.best_available_score), else_=None)
|
||||
# Three sums rather than one, because "not measured" and "measured as zero"
|
||||
# are different answers and a single counter cannot hold both.
|
||||
measured = case((RetrievalLog.suppressed_count.isnot(None), 1), else_=0)
|
||||
supp_calls = case((RetrievalLog.suppressed_count > 0, 1), else_=0)
|
||||
supp_zero = case(
|
||||
((RetrievalLog.result_count == 0) & (RetrievalLog.suppressed_count > 0), 1),
|
||||
else_=0,
|
||||
)
|
||||
zero = case((RetrievalLog.result_count == 0, 1), else_=0)
|
||||
|
||||
def pct(p: float):
|
||||
return func.percentile_cont(p).within_group(RetrievalLog.top_score.asc())
|
||||
|
||||
# Assigned inside the try below; named here so the readout can tell
|
||||
# "this query failed" from "this window has no rows" (#2663).
|
||||
by_source_rows = None
|
||||
rule_rows = None
|
||||
distinct_rules_surfaced = distinct_rules_pulled = 0
|
||||
# None means the coverage read did not happen — distinct from a table with
|
||||
# no rows, which is {"*": None}. Same reason `read_failed` exists.
|
||||
note_complete = rule_complete = None
|
||||
|
||||
try:
|
||||
async with async_session() as session:
|
||||
rows = (
|
||||
@@ -239,7 +434,6 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
RetrievalLog.source,
|
||||
func.count().label("calls"),
|
||||
func.sum(zero).label("zero"),
|
||||
func.sum(cleared).label("cleared"),
|
||||
pct(0.1), pct(0.5), pct(0.9),
|
||||
func.min(RetrievalLog.top_score),
|
||||
func.max(RetrievalLog.top_score),
|
||||
@@ -247,6 +441,13 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
func.percentile_cont(0.9).within_group(
|
||||
RetrievalLog.duration_ms.asc()
|
||||
),
|
||||
func.sum(measured).label("measured"),
|
||||
func.sum(supp_calls).label("supp_calls"),
|
||||
func.sum(supp_zero).label("supp_zero"),
|
||||
func.sum(miss).label("miss_calls"),
|
||||
func.percentile_cont(0.5).within_group(miss_score.asc()),
|
||||
func.percentile_cont(0.9).within_group(miss_score.asc()),
|
||||
func.max(miss_score),
|
||||
)
|
||||
.where(
|
||||
RetrievalLog.created_at >= since,
|
||||
@@ -255,8 +456,34 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
.group_by(RetrievalLog.source)
|
||||
)
|
||||
).all()
|
||||
log_complete = await _complete_from(session, RetrievalLog, user_id)
|
||||
for row in rows:
|
||||
out["sources"][row[0]] = _bucket(list(row[1:]))
|
||||
source = row[0]
|
||||
bucket = _bucket(list(row[1:]))
|
||||
# Per SOURCE, not per table: retrieval_logs goes back months
|
||||
# while any individual arm may be days old, and the table's
|
||||
# age would vouch for an arm that has barely started.
|
||||
bucket.update(_coverage(log_complete.get(source), since))
|
||||
out["sources"][source] = bucket
|
||||
|
||||
# A source with rows in the table but NONE in this window would
|
||||
# otherwise be absent from the readout — and absent is exactly how
|
||||
# a source that never existed renders, so a surface that WAS
|
||||
# recording and went silent is unreadable (#3720). That is #2663
|
||||
# one level up: the failure that looks like the correct answer.
|
||||
#
|
||||
# Zero here is a real measurement, not a manufactured one. The
|
||||
# all-time query proves the source was recording, and it made no
|
||||
# calls across a window it fully covers — which is why no
|
||||
# `covers_window` special case is needed: a source whose first row
|
||||
# fell after `since` would have that row IN the window and already
|
||||
# hold a bucket, so anything reaching here began before it.
|
||||
for src, first_row in log_complete.items():
|
||||
if src == "*" or first_row is None or src in out["sources"]:
|
||||
continue
|
||||
quiet = _bucket(list(_NO_ROWS_IN_WINDOW))
|
||||
quiet.update(_coverage(first_row, since))
|
||||
out["sources"][src] = quiet
|
||||
|
||||
# The corpus side, at its own grain. `ambient` mirrors
|
||||
# note_usage.usage_for_notes: an ambient surfacing was not a scored
|
||||
@@ -283,6 +510,7 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
.group_by(NoteUsageEvent.event, NoteUsageEvent.source)
|
||||
)
|
||||
).all()
|
||||
note_complete = await _complete_from(session, NoteUsageEvent, user_id)
|
||||
|
||||
# Distinct-note counts need their OWN queries, and this is not
|
||||
# fussiness: count(distinct note_id) per (event, source) group
|
||||
@@ -310,6 +538,142 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
)
|
||||
)
|
||||
).scalar_one()
|
||||
|
||||
# Per-source pull-through, at the NOTE grain (#3311).
|
||||
#
|
||||
# The `urows` query above already groups by source and the loop
|
||||
# below then throws the source away, so until now this readout
|
||||
# could say what the corpus's overall pull-through was and nothing
|
||||
# about WHICH surface earned it. The data was always here; only
|
||||
# the aggregation discarded it.
|
||||
#
|
||||
# It cannot be had by grouping the PULLED rows by source: a pull
|
||||
# records the door it came through (`mcp_get_note`), not the
|
||||
# surface that put the record in front of the agent. Correlating
|
||||
# those within a session is what #2085 ruled out — there is no
|
||||
# session identity server-side and inventing one would mean
|
||||
# threading a client-supplied token through every read path. The
|
||||
# note grain answers the question without one: of the distinct
|
||||
# notes surface X chose, how many did an agent open in this window?
|
||||
#
|
||||
# Guarded separately from the reads above, on #2663's actual
|
||||
# lesson. That outage was a NOVEL SQL SHAPE the database rejected
|
||||
# inside a broad except. This join is the novel shape here, and a
|
||||
# failure in it must not take down two readouts that already work.
|
||||
try:
|
||||
pulled_ids = (
|
||||
select(NoteUsageEvent.note_id)
|
||||
.where(
|
||||
NoteUsageEvent.created_at >= since,
|
||||
NoteUsageEvent.user_id == user_id,
|
||||
NoteUsageEvent.event == PULLED,
|
||||
# autoescape because `_` is a LIKE wildcard: a bare
|
||||
# like("mcp_%") also matches "mcpX…". The Python half
|
||||
# of this readout uses str.startswith and has no such
|
||||
# hazard; this is the SQL half's version of it.
|
||||
NoteUsageEvent.source.startswith("mcp_", autoescape=True),
|
||||
)
|
||||
.distinct()
|
||||
.subquery()
|
||||
)
|
||||
surfaced_pairs = (
|
||||
select(NoteUsageEvent.source, NoteUsageEvent.note_id)
|
||||
.where(
|
||||
NoteUsageEvent.created_at >= since,
|
||||
NoteUsageEvent.user_id == user_id,
|
||||
NoteUsageEvent.event == SURFACED,
|
||||
)
|
||||
.distinct()
|
||||
.subquery()
|
||||
)
|
||||
# DISTINCT on (source, note_id) FIRST, which is what lets the
|
||||
# outer aggregate be a plain count(): the pairs are already
|
||||
# unique, so the left join cannot multiply them and no
|
||||
# count(DISTINCT) is needed to undo damage that never happens.
|
||||
by_source_rows = (
|
||||
await session.execute(
|
||||
select(
|
||||
surfaced_pairs.c.source,
|
||||
func.count().label("notes_surfaced"),
|
||||
func.count(pulled_ids.c.note_id).label("notes_pulled"),
|
||||
)
|
||||
.select_from(
|
||||
surfaced_pairs.outerjoin(
|
||||
pulled_ids,
|
||||
pulled_ids.c.note_id == surfaced_pairs.c.note_id,
|
||||
)
|
||||
)
|
||||
.group_by(surfaced_pairs.c.source)
|
||||
)
|
||||
).all()
|
||||
except Exception:
|
||||
logger.warning("per-source pull-through read failed", exc_info=True)
|
||||
by_source_rows = None
|
||||
|
||||
# Rules, at their own grain and in their own block (milestone 333).
|
||||
#
|
||||
# Guarded separately from the reads above for the reason `by_source`
|
||||
# is: this table is NEW, and an instance running upgraded code
|
||||
# against un-migrated schema would otherwise take down two readouts
|
||||
# that work perfectly in order to report a third that cannot.
|
||||
#
|
||||
# The queries themselves are the note block's shapes, not novel
|
||||
# ones — a group-by on two indexed columns and two count(distinct).
|
||||
# The distinct counts need their own queries for the same reason
|
||||
# the note ones do: count(distinct rule_id) per group cannot be
|
||||
# summed across groups without double-counting a rule two sources
|
||||
# both touched.
|
||||
try:
|
||||
rule_rows = (
|
||||
await session.execute(
|
||||
select(
|
||||
RuleUsageEvent.event,
|
||||
RuleUsageEvent.source,
|
||||
func.count().label("n"),
|
||||
)
|
||||
.where(
|
||||
RuleUsageEvent.created_at >= since,
|
||||
RuleUsageEvent.user_id == user_id,
|
||||
)
|
||||
.group_by(RuleUsageEvent.event, RuleUsageEvent.source)
|
||||
)
|
||||
).all()
|
||||
rule_complete = await _complete_from(
|
||||
session, RuleUsageEvent, user_id,
|
||||
)
|
||||
# The rows carry `source`, so the ranked/ambient split is done
|
||||
# below rather than in SQL — the bulk surfaces started emitting
|
||||
# on 2026-09-03 (#3473), so there IS an ambient class now.
|
||||
#
|
||||
# `distinct_rules_surfaced` deliberately counts BOTH classes. It
|
||||
# answers "how many distinct rules did this install put in front
|
||||
# of an agent at all", which is the denominator for dead weight
|
||||
# — and a rule delivered by the preload a hundred times and
|
||||
# never opened is the most important case that question has.
|
||||
distinct_rules_surfaced = (
|
||||
await session.execute(
|
||||
select(func.count(func.distinct(RuleUsageEvent.rule_id)))
|
||||
.where(
|
||||
RuleUsageEvent.created_at >= since,
|
||||
RuleUsageEvent.user_id == user_id,
|
||||
RuleUsageEvent.event == RULE_SURFACED,
|
||||
)
|
||||
)
|
||||
).scalar_one()
|
||||
distinct_rules_pulled = (
|
||||
await session.execute(
|
||||
select(func.count(func.distinct(RuleUsageEvent.rule_id)))
|
||||
.where(
|
||||
RuleUsageEvent.created_at >= since,
|
||||
RuleUsageEvent.user_id == user_id,
|
||||
RuleUsageEvent.event == RULE_PULLED,
|
||||
)
|
||||
)
|
||||
).scalar_one()
|
||||
except Exception:
|
||||
logger.warning("rule usage read failed", exc_info=True)
|
||||
rule_rows = None
|
||||
distinct_rules_surfaced = distinct_rules_pulled = 0
|
||||
except Exception:
|
||||
logger.warning("retrieval summary read failed", exc_info=True)
|
||||
out["read_failed"] = True
|
||||
@@ -350,5 +714,124 @@ async def retrieval_summary(user_id: int | None, *, days: int = 30) -> dict:
|
||||
round(usage["pulled_by_agent"] / usage["surfaced"], 4)
|
||||
if usage["surfaced"] else None
|
||||
)
|
||||
|
||||
# The same question, per surface — which is the one the top-level ratio
|
||||
# cannot answer. A corpus average of 0.05 is compatible with one surface
|
||||
# earning its noise and another producing none, and tuning a threshold
|
||||
# needs to know which.
|
||||
#
|
||||
# UPPER BOUND, and say so where it will be read: a pull records the door,
|
||||
# not the surface that led to it, so a note surfaced by two surfaces and
|
||||
# opened once counts as pulled for both. Attribution would need the session
|
||||
# identity #2085 declined to invent. The bound is still decisive in the
|
||||
# direction that matters — a surface reading near zero here is not being
|
||||
# flattered by the double-count.
|
||||
if by_source_rows is None:
|
||||
usage["by_source"] = {}
|
||||
# Distinct from an empty window, for the same reason `read_failed` is.
|
||||
usage["by_source_failed"] = True
|
||||
else:
|
||||
by_source: dict[str, dict] = {}
|
||||
for source, n_surfaced, n_pulled in by_source_rows:
|
||||
n_surfaced, n_pulled = int(n_surfaced or 0), int(n_pulled or 0)
|
||||
ambient = source in AMBIENT_SOURCES
|
||||
by_source[source] = {
|
||||
"notes_surfaced": n_surfaced,
|
||||
"notes_pulled": n_pulled,
|
||||
# None rather than a number on an ambient surface: nothing
|
||||
# CHOSE those records, so "surfaced often, opened never" is not
|
||||
# a judgment about them. The counts stay visible; the ratio
|
||||
# that would be misread does not.
|
||||
"pull_through": (
|
||||
None if ambient or not n_surfaced
|
||||
else round(n_pulled / n_surfaced, 4)
|
||||
),
|
||||
"ambient": ambient,
|
||||
}
|
||||
usage["by_source"] = by_source
|
||||
|
||||
# The SECTION's coverage, from the latest source to start recording — a
|
||||
# figure that sums several sources is complete only once every one of them
|
||||
# was being written. `_complete_from` computes that as "*".
|
||||
usage.update(_coverage((note_complete or {}).get("*"), since))
|
||||
out["usage"] = usage
|
||||
|
||||
# ── Rules, deliberately a SEPARATE block ────────────────────────────
|
||||
#
|
||||
# Not folded into `usage`, for two reasons and the second is the one that
|
||||
# bites. The corpora differ by orders of magnitude — a few dozen eligible
|
||||
# rules against thousands of notes — so one blended ratio would be the note
|
||||
# ratio with a little noise on it, and the rule arm's own behaviour would
|
||||
# be undetectable inside it. And `usage` is what existing callers already
|
||||
# read: silently changing what it counts would move a number people have
|
||||
# been comparing across windows, without telling them it now measures
|
||||
# something else.
|
||||
#
|
||||
# `ambient` now carries the bulk deliveries — the SessionStart preload,
|
||||
# `list_always_on_rules`, and every `rules_payload` surface (#3473). Before
|
||||
# they emitted, this block had no ambient key and said the absence was a
|
||||
# fact about the data. It was, and it was also the thing that made the
|
||||
# always-on set impossible to judge: the largest rule surface in the
|
||||
# product was the one surface its own scoreboard could not see.
|
||||
#
|
||||
# READ THE TWO SEPARATELY, ALWAYS. `surfaced` is a claim a ranker made and
|
||||
# a pull can settle. `ambient` is a delivery nobody chose, so a high count
|
||||
# says the set is large and resident, never that it is useful.
|
||||
rule_usage = {
|
||||
"surfaced": 0, "ambient": 0,
|
||||
"pulled": 0, "pulled_by_agent": 0, "pulled_by_human": 0,
|
||||
"distinct_rules_surfaced": int(distinct_rules_surfaced or 0),
|
||||
"distinct_rules_pulled": int(distinct_rules_pulled or 0),
|
||||
}
|
||||
if rule_rows is None:
|
||||
# The FLAG is added, the shape is kept — matching `by_source_failed`
|
||||
# one block up. A caller that renders this must not have to choose
|
||||
# between crashing on a missing key and quietly showing zeros it has no
|
||||
# right to: the keys let it render, and the flag tells it the zeros are
|
||||
# "we could not find out" rather than "nothing happened" (#2663).
|
||||
rule_usage["rule_usage_failed"] = True
|
||||
else:
|
||||
for event, source, n in rule_rows:
|
||||
n = int(n)
|
||||
if event == RULE_SURFACED:
|
||||
# One definition of ranked-vs-ambient, imported rather than
|
||||
# restated — the per-rule badge readout reads the same
|
||||
# predicate, and two spellings of "what counts as surfaced" is
|
||||
# precisely the uneven wiring #3246 found across this system.
|
||||
if is_ambient(source):
|
||||
rule_usage["ambient"] += n
|
||||
else:
|
||||
rule_usage["surfaced"] += n
|
||||
elif event == RULE_PULLED:
|
||||
rule_usage["pulled"] += n
|
||||
# Same split, and it carries MORE weight here than for notes.
|
||||
# The arm's whole claim is "this rule may apply to what you are
|
||||
# writing", and only an agent opening it says the claim landed.
|
||||
# A person browsing the rule list says nothing about the hint.
|
||||
if source.startswith("mcp_"):
|
||||
rule_usage["pulled_by_agent"] += n
|
||||
else:
|
||||
rule_usage["pulled_by_human"] += n
|
||||
|
||||
# None, not 0.0, when nothing was surfaced — matching the note block. A
|
||||
# ratio of zero asserts "we showed rules and none were opened"; with an
|
||||
# empty numerator AND denominator that is a claim the data does not
|
||||
# support, and it is the reading that would make a brand-new install look
|
||||
# like a broken one.
|
||||
#
|
||||
# RANKED SURFACINGS ONLY in the denominator, and this is the load-bearing
|
||||
# line of the whole change. Pull-through asks "was that hint any use", and
|
||||
# only a surface that CHOSE what it showed can be judged by it. Folding the
|
||||
# preload in would divide the same pulls by a number that grows with every
|
||||
# session and every rule added to the resident set — so enlarging the
|
||||
# always-on set would DEPRESS the arm's measured precision, and trimming it
|
||||
# would flatter it, neither for any reason to do with the arm. The ambient
|
||||
# count sits beside it, unaveraged, and is read as size rather than skill.
|
||||
rule_usage["pull_through"] = (
|
||||
round(rule_usage["pulled_by_agent"] / rule_usage["surfaced"], 4)
|
||||
if rule_usage["surfaced"] else None
|
||||
)
|
||||
rule_usage.update(_coverage((rule_complete or {}).get("*"), since))
|
||||
out["rule_usage"] = rule_usage
|
||||
|
||||
return out
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
"""Rule usage telemetry — did a surfaced rule ever get read?
|
||||
|
||||
The sibling of `note_usage`, for the one retrieval surface in Scribe that
|
||||
could not be measured at all.
|
||||
|
||||
Two event streams, deliberately independent:
|
||||
|
||||
- SURFACED: the write-path standing-rule arm put this rule in front of the
|
||||
agent, unbidden, during a write.
|
||||
- PULLED: someone then opened it in full (`get_rule`, or the REST detail
|
||||
route).
|
||||
|
||||
WHY THIS ARM AND NOT ANOTHER. Every other surface declines most of the time —
|
||||
`write_path` returns nothing on 78% of calls, `reuse_slot` on 79%, auto-inject
|
||||
on 39%. The rule arm APPEARED never to have returned nothing (#3311), and this
|
||||
docstring used to put that forward as the puzzle worth measuring: "either a
|
||||
perfectly tuned surface or a bar it cannot fail to clear".
|
||||
|
||||
It was neither, and the correction belongs here rather than being quietly
|
||||
deleted. The arm wrote its `retrieval_logs` row only on calls that FOUND
|
||||
something (#3497), so `zero_result_calls` sat at 0 and `cleared_threshold` at
|
||||
`calls` because of the shape of the code — at any threshold whatsoever. A
|
||||
statistic that could not vary was read as a finding about the corpus. It is the
|
||||
#2663 failure mode one level up: there the broken readout was a zero, here it
|
||||
was a hundred percent, which is far better camouflage.
|
||||
|
||||
The reason to measure this arm survives the correction, and is stronger for it.
|
||||
`retrieval_logs` records what the ranker scored, never whether the hint was any
|
||||
use, so even an honest clear-rate would not settle the question. The ratio these
|
||||
two streams produce is the missing half, and without it any threshold change is
|
||||
a number picked off a histogram.
|
||||
|
||||
Design notes, mirroring `note_usage`:
|
||||
- Writes are fire-and-forget through `background.spawn`, so telemetry never
|
||||
adds latency to — or can break — the surface it observes. This module does
|
||||
NOT carry its own copy of the strong-reference dance; `background` is the
|
||||
one place that gets it right, and a fourth copy is how one of them drifts.
|
||||
- Failures degrade, but never SILENTLY. `report_telemetry_failure` logs at
|
||||
WARNING and drops one AppLog row per process per site. #2663 is the record
|
||||
of this exact subsystem class running at zero for weeks — indistinguishable
|
||||
from "nobody uses this" — because every failure went to `logger.debug`.
|
||||
- Reads (`usage_for_rules`) are awaited and aggregated in one round-trip for
|
||||
a whole page, never per row.
|
||||
|
||||
AMBIENT VS RANKED. The note twin splits ranked surfacings from ambient ones
|
||||
because `enter_project` and the skill sync put records in front of the agent
|
||||
without choosing them, and counting those as surfacings makes recency read as
|
||||
popularity (#2477). Rules have exactly that shape: the SessionStart preload,
|
||||
`list_always_on_rules`, and every `rules_payload` surface hand over the whole
|
||||
applicable set at once, chosen by nobody.
|
||||
|
||||
Until 2026-09-03 those bulk surfaces emitted nothing, and this module said so —
|
||||
"an empty `AMBIENT_SOURCES` would be machinery pretending to a distinction the
|
||||
data does not yet contain". True as far as it went, but it had a consequence
|
||||
worth naming, because it is the reason the bucket exists now: the always-on
|
||||
set's token cost was certain and its usefulness was UNFALSIFIABLE, permanently
|
||||
and by construction. The one surface whose value was actually in question was
|
||||
the one surface exempt from the scoreboard that judges every other.
|
||||
|
||||
They emit now. The split is the readout-level change the old note promised — a
|
||||
`case()`, no migration, because `event` and `source` are plain Text with no
|
||||
CHECK constraint. `source` stays granular so a reader can still tell the
|
||||
preload from `enter_project` from the ranked arm.
|
||||
|
||||
WHY THIS NAMES THE RANKED SOURCES AND THE TWIN NAMES THE AMBIENT ONES. A
|
||||
deliberate divergence, on the failure mode rather than on symmetry. Both shapes
|
||||
fail silently when someone adds a surface and forgets the list, so the question
|
||||
is which list changes more often — and here it is emphatically the ambient one:
|
||||
there are TWO ranked rule sources (the write-path arm and the pre-tool arm)
|
||||
against the seven bulk ones the preload alone contributes. Ranked sources are
|
||||
added when somebody builds a ranker, which is rare and deliberate; bulk ones
|
||||
appear whenever a surface hands rules over, which is most of them. Naming the
|
||||
rare, slow-moving half means a newly-added bulk surface defaults to
|
||||
`ambient`, which merely under-counts it, instead of defaulting to `ranked`,
|
||||
which would quietly pad the pull-through denominator with surfacings nobody
|
||||
chose and make the arm look imprecise. Same argument #3191 and #3430 make
|
||||
against hand-kept lists: keep the list that must be remembered as short and as
|
||||
slow-moving as possible.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from sqlalchemy import case, func, select
|
||||
|
||||
from scribe.models import async_session
|
||||
from scribe.models.base import iso
|
||||
from scribe.models.rule_usage import PULLED, SURFACED, RuleUsageEvent
|
||||
from scribe.services.background import report_telemetry_failure, spawn
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# The surfaces that CHOSE the rules they showed. Everything else is ambient —
|
||||
# see the module docstring for why the rare half is the half that gets named.
|
||||
#
|
||||
# Membership is the whole definition of the pull-through denominator: a ranked
|
||||
# surfacing is a claim ("this rule may apply to what you are doing") that a pull
|
||||
# can confirm or refute, while an ambient one is a delivery nobody decided on.
|
||||
# Add a source here only when a ranker picked it.
|
||||
RANKED_SOURCES = ("write_path_rule", "pre_tool_rule")
|
||||
|
||||
|
||||
def is_ambient(source: str) -> bool:
|
||||
"""Was this surfacing a bulk delivery rather than a ranked choice?
|
||||
|
||||
One definition, read by both the per-rule badge readout and the aggregate
|
||||
in `retrieval_telemetry` — the two used to be able to disagree about what
|
||||
"surfaced" counted, which is the class of drift #3246 found across the
|
||||
rules system.
|
||||
|
||||
Sync and pure, per the service canon (#2860), but deliberately PUBLIC where
|
||||
that canon says such helpers stay `_private`. The departure is the point:
|
||||
a module-private copy in each caller is exactly the second definition this
|
||||
exists to prevent.
|
||||
"""
|
||||
return source not in RANKED_SOURCES
|
||||
|
||||
|
||||
async def _report_failure(site: str) -> None:
|
||||
await report_telemetry_failure("rule_usage", site)
|
||||
|
||||
|
||||
async def _insert_events(rows: list[dict]) -> None:
|
||||
"""Persist usage rows. Best-effort: failures degrade, visibly."""
|
||||
try:
|
||||
async with async_session() as session:
|
||||
session.add_all([RuleUsageEvent(**row) for row in rows])
|
||||
await session.commit()
|
||||
except Exception:
|
||||
await _report_failure("write")
|
||||
|
||||
|
||||
def _schedule(rows: list[dict]) -> None:
|
||||
if not rows:
|
||||
return
|
||||
spawn(_insert_events(rows), site="rule_usage_write")
|
||||
|
||||
|
||||
def record_rule_surfaced(
|
||||
*, user_id: int | None, rule_ids: list[int] | set[int], source: str
|
||||
) -> None:
|
||||
"""Fire-and-forget: record that these rules were shown to the agent.
|
||||
|
||||
Takes the whole delivery at once — one insert per surfacing event, not per
|
||||
rule — because a hint is a single decision and its rows should land
|
||||
together.
|
||||
|
||||
Record what was actually SHOWN, never what was considered. For the ranked
|
||||
arm that means the post-filter hits: it drops what the session already
|
||||
holds (`exclude_rule_ids`) before it speaks, and a rule considered and not
|
||||
shown was not surfaced. Counting those would inflate the denominator with
|
||||
claims the agent never saw, which reads as a precision problem the arm does
|
||||
not have.
|
||||
|
||||
Bulk surfaces pass their whole delivered set, which is the same rule read
|
||||
from the other end — everything in a preload IS shown. `source` is what
|
||||
separates the two afterwards (see `RANKED_SOURCES`); this function does not
|
||||
care which kind it is recording.
|
||||
"""
|
||||
try:
|
||||
rows = [
|
||||
{
|
||||
"user_id": user_id,
|
||||
"rule_id": int(rid),
|
||||
"event": SURFACED,
|
||||
"source": source,
|
||||
}
|
||||
for rid in rule_ids
|
||||
]
|
||||
except Exception:
|
||||
logger.debug("rule usage payload build failed", exc_info=True)
|
||||
return
|
||||
_schedule(rows)
|
||||
|
||||
|
||||
def record_rule_pulled(*, user_id: int | None, rule_id: int, source: str) -> None:
|
||||
"""Fire-and-forget: record that a rule was opened in full.
|
||||
|
||||
A PULL is somebody choosing to open one record. `list_always_on_rules` and
|
||||
`enter_project` are NOT pulls — they are bulk resident loads that hand over
|
||||
every applicable rule at once, and counting them would swamp the signal
|
||||
with the very ambient delivery the ratio exists to distinguish from.
|
||||
"""
|
||||
try:
|
||||
rows = [
|
||||
{
|
||||
"user_id": user_id,
|
||||
"rule_id": int(rule_id),
|
||||
"event": PULLED,
|
||||
"source": source,
|
||||
}
|
||||
]
|
||||
except Exception:
|
||||
logger.debug("rule usage payload build failed", exc_info=True)
|
||||
return
|
||||
_schedule(rows)
|
||||
|
||||
|
||||
def empty_rule_usage() -> dict:
|
||||
"""The zero readout — what a rule with no recorded events looks like.
|
||||
|
||||
Callers render this shape unconditionally, so a rule predating the table
|
||||
reads as "never surfaced, never pulled" rather than as a missing key. That
|
||||
distinction matters more here than for notes: every rule in an install
|
||||
predates this table, so for a while "no events" is the normal state and it
|
||||
must not look like a broken readout.
|
||||
|
||||
`surfaced_count` is RANKED surfacings only; `ambient_count` is the bulk
|
||||
deliveries (see `RANKED_SOURCES`). The split is what keeps the badge's
|
||||
"shown often, opened never → dead weight" reading honest: every rule in an
|
||||
always-on set is delivered every session, so an unsplit counter would rank
|
||||
the resident set as the most-surfaced rules in the install purely for being
|
||||
resident.
|
||||
"""
|
||||
return {
|
||||
"surfaced_count": 0,
|
||||
"ambient_count": 0,
|
||||
"pull_count": 0,
|
||||
"last_surfaced_at": None,
|
||||
"last_pulled_at": None,
|
||||
}
|
||||
|
||||
|
||||
async def usage_for_rules(rule_ids: list[int]) -> dict[int, dict]:
|
||||
"""Aggregate usage for a set of rules: {rule_id: {counts + timestamps}}.
|
||||
|
||||
One GROUP BY for the whole page rather than a query per row — this feeds a
|
||||
list view, so the per-row shape would be N+1 by construction. Rules with no
|
||||
events come back with `empty_rule_usage()`, so the caller never has to tell
|
||||
"no events" from "not in the result".
|
||||
"""
|
||||
ids = [int(r) for r in rule_ids]
|
||||
out: dict[int, dict] = {rid: empty_rule_usage() for rid in ids}
|
||||
if not ids:
|
||||
return out
|
||||
|
||||
# Classified in SQL so the group stays small: per rule we get at most
|
||||
# (surfaced-ranked, surfaced-ambient, pulled) rather than a row per distinct
|
||||
# source. ONE labelled expression, bound to a variable and reused in the
|
||||
# GROUP BY — a second `case()` instance there renders its own expanding-IN
|
||||
# bind names under asyncpg, so the database sees two DIFFERENT expressions
|
||||
# and rejects the query with a GroupingError. The note twin carries the
|
||||
# same warning for the same reason, and #2663 is what it cost: the
|
||||
# rejection was swallowed and every counter read zero in production while
|
||||
# the writes were landing fine.
|
||||
ambient = case(
|
||||
(RuleUsageEvent.source.notin_(RANKED_SOURCES), True),
|
||||
else_=False,
|
||||
).label("ambient")
|
||||
try:
|
||||
async with async_session() as session:
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(
|
||||
RuleUsageEvent.rule_id,
|
||||
RuleUsageEvent.event,
|
||||
func.count().label("n"),
|
||||
func.max(RuleUsageEvent.created_at).label("last_at"),
|
||||
ambient,
|
||||
)
|
||||
.where(RuleUsageEvent.rule_id.in_(ids))
|
||||
.group_by(
|
||||
RuleUsageEvent.rule_id,
|
||||
RuleUsageEvent.event,
|
||||
ambient,
|
||||
)
|
||||
)
|
||||
).all()
|
||||
except Exception:
|
||||
# A telemetry readout must not be able to break the list it decorates —
|
||||
# but it must say it failed, or a broken readout is indistinguishable
|
||||
# from a corpus nobody uses (#2663).
|
||||
await _report_failure("readout")
|
||||
return out
|
||||
|
||||
for rule_id, event, n, last_at, is_amb in rows:
|
||||
slot = out.get(int(rule_id))
|
||||
if slot is None:
|
||||
continue
|
||||
if event == SURFACED and is_amb:
|
||||
slot["ambient_count"] = int(n)
|
||||
elif event == SURFACED:
|
||||
slot["surfaced_count"] = int(n)
|
||||
slot["last_surfaced_at"] = iso(last_at)
|
||||
elif event == PULLED:
|
||||
# Pulls are pulls regardless of what surfaced the rule — "did
|
||||
# anyone ever open this?" does not depend on how it was found. Both
|
||||
# halves accumulate, so this ADDS rather than assigns: a rule can
|
||||
# now be pulled after a ranked hint and after a preload, and the
|
||||
# split arrives as two rows.
|
||||
slot["pull_count"] = slot["pull_count"] + int(n)
|
||||
latest = iso(last_at)
|
||||
if latest and (slot["last_pulled_at"] or "") < latest:
|
||||
slot["last_pulled_at"] = latest
|
||||
return out
|
||||
@@ -0,0 +1,82 @@
|
||||
"""A rule's edit history — the snapshot taken when an edit overwrites text.
|
||||
|
||||
Sibling of `note_versions`, and the differences are the whole design.
|
||||
|
||||
WHAT A VERSION IS FOR. It records what a rule USED TO SAY, because an update
|
||||
overwrites that and nothing can recompute it. It is not a record of the rule's
|
||||
lifecycle: a soft delete keeps the rule row whole and restorable, so there is
|
||||
nothing to preserve at delete time, and anything that outlived a purge would be
|
||||
data the operator explicitly asked to be gone.
|
||||
|
||||
WHAT IS DELIBERATELY NOT COPIED FROM note_versions (milestone 323). Both of its
|
||||
guards defend against note autosave, which fires every 60 seconds and would
|
||||
otherwise burn every slot. Rules have no autosave — every version here comes
|
||||
from a deliberate `update_rule` — so both guards would only ever discard a real
|
||||
edit:
|
||||
|
||||
- No `MIN_VERSION_INTERVAL_SECONDS`. Two deliberate edits four minutes apart
|
||||
are two edits.
|
||||
- No `MAX_VERSIONS` and no pruning. A rule is edited a handful of times in its
|
||||
life; a cap invites losing the one edit somebody needed.
|
||||
|
||||
TEXT ONLY. A rule's topic, its Systems tags and its typed edges are not here.
|
||||
They have their own lifecycle, and folding them in would make one word,
|
||||
"version", mean two things — the rule's wording, and the rule's place in the
|
||||
graph.
|
||||
"""
|
||||
from sqlalchemy import select
|
||||
|
||||
from scribe.models import async_session
|
||||
from scribe.models.rule_version import RuleVersion
|
||||
|
||||
# The columns a version carries, and therefore the ones whose change is worth
|
||||
# a snapshot. `order_index` and `arose_from_id` are editable through
|
||||
# update_rule and are deliberately absent: reordering a rulebook is not an
|
||||
# edit to what any rule says, and a run of "moved rule 4 above rule 3"
|
||||
# snapshots would bury the edits somebody is actually looking for.
|
||||
SNAPSHOT_FIELDS = (
|
||||
"title", "statement", "why", "how_to_apply", "when_to_apply",
|
||||
"tier", "verify_with", "expires_when",
|
||||
)
|
||||
|
||||
|
||||
def snapshot(rule) -> dict:
|
||||
"""The rule's text right now, as a plain dict.
|
||||
|
||||
Plain values rather than the ORM object because the caller has to hold
|
||||
this ACROSS the mutation — a live `rule` would show the new text by the
|
||||
time it was read, which is the one thing the snapshot must not do.
|
||||
"""
|
||||
return {f: getattr(rule, f) for f in SNAPSHOT_FIELDS}
|
||||
|
||||
|
||||
def record_if_changed(session, rule, actor_user_id: int | None, before: dict):
|
||||
"""Add a version holding `before`, unless the edit changed no text.
|
||||
|
||||
Git semantics, and not a nicety: `update_rule` is reached by callers that
|
||||
resend every field, so without this a form saved twice would file a second
|
||||
identical snapshot and the history would stop reading as a list of edits.
|
||||
|
||||
Session-BOUND, unlike `note_versions.create_version` which opens its own.
|
||||
The version and the edit that caused it commit together or not at all;
|
||||
a separate session could leave a snapshot behind after the update it
|
||||
describes had failed, which is a history entry for an edit that never
|
||||
happened.
|
||||
"""
|
||||
if snapshot(rule) == before:
|
||||
return None
|
||||
version = RuleVersion(
|
||||
rule_id=rule.id, user_id=actor_user_id, **before,
|
||||
)
|
||||
session.add(version)
|
||||
return version
|
||||
|
||||
|
||||
async def list_versions(rule_id: int) -> list[RuleVersion]:
|
||||
"""Newest first — the reader's question is "what did this just say?"."""
|
||||
async with async_session() as session:
|
||||
return list((await session.execute(
|
||||
select(RuleVersion)
|
||||
.where(RuleVersion.rule_id == rule_id)
|
||||
.order_by(RuleVersion.created_at.desc(), RuleVersion.id.desc())
|
||||
)).scalars().all())
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Iterable
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
from sqlalchemy import and_, delete as sql_delete, insert, or_, select
|
||||
@@ -16,6 +17,13 @@ from sqlalchemy import and_, delete as sql_delete, insert, or_, select
|
||||
from scribe.models import async_session
|
||||
from scribe.models.system import System
|
||||
from scribe.models.rulebook import Rulebook
|
||||
from scribe.services.verification import (
|
||||
days_since_verified as _days_since_verified,
|
||||
last_verified_label as _last_verified_label,
|
||||
)
|
||||
from scribe.services import rule_versions
|
||||
from scribe.models.rule_version import RuleVersion
|
||||
from scribe.services.rule_usage import record_rule_surfaced
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -311,19 +319,12 @@ def _valid_tier(tier: str) -> str:
|
||||
return tier if tier in TIERS else "always_on"
|
||||
|
||||
|
||||
def last_verified_label(rule: Rule) -> str | None:
|
||||
"""How long ago the rule's check passed — None when it carries no check.
|
||||
|
||||
One helper because two surfaces need the same answer and the brief-dict
|
||||
lesson in rule_brief's docstring is what happens otherwise: three copies
|
||||
that had already drifted. `None` means "this rule is a decision, the
|
||||
question does not apply"; "never" means "it is a fact and nobody has
|
||||
confirmed it" — a distinction worth keeping, because the second is the
|
||||
one worth acting on.
|
||||
"""
|
||||
if not rule.verify_with:
|
||||
return None
|
||||
return rule.verified_at.date().isoformat() if rule.verified_at else "never"
|
||||
# Re-exported, not redefined. Notes gained the same trio in milestone 317 and
|
||||
# this reading of it is genuinely common, so it moved to services/verification
|
||||
# — the DRY win note 3163 names, as against sharing the QUERY, which the two
|
||||
# record types cannot (a rule scopes by rulebook ownership, a note by the note
|
||||
# ACL). Kept importable from here because callers already reach for it here.
|
||||
last_verified_label = _last_verified_label
|
||||
|
||||
|
||||
def rule_brief(rule: Rule, **extra) -> dict:
|
||||
@@ -379,6 +380,10 @@ def _refresh_rule_embedding(rule: Rule) -> None:
|
||||
swallowed because a rule that SAVED must not fail on its index refresh —
|
||||
a stale vector costs a missed search hit, a raised exception costs the
|
||||
write. No running loop (unit tests, scripts) is ordinary, not an error.
|
||||
|
||||
Detaching also means this task races anything that deletes the rule out
|
||||
from under it. That is not handled here: `upsert_rule_embedding` claims
|
||||
the rule's row before touching its vectors, and loses if it can't (#3262).
|
||||
"""
|
||||
try:
|
||||
import asyncio
|
||||
@@ -751,6 +756,10 @@ async def update_rule(
|
||||
"verify_with", "expires_when",
|
||||
}
|
||||
check_before = rule.verify_with
|
||||
# Captured BEFORE anything is written, and as plain values — this has
|
||||
# to survive the mutation below. A rule's history is the only record
|
||||
# of what it used to say; the edit itself destroys that.
|
||||
text_before = rule_versions.snapshot(rule)
|
||||
for key in clear:
|
||||
if key in allowed and key in NULLABLE_RULE_TEXT:
|
||||
setattr(rule, key, None)
|
||||
@@ -774,12 +783,59 @@ async def update_rule(
|
||||
# wrongly vouched for costs the thing the sweep exists to catch.
|
||||
if rule.verify_with != check_before:
|
||||
rule.verified_at = None
|
||||
# Same session as the edit, so the two commit together. The snapshot
|
||||
# holds the OLD verify_with — the check that was in force when that
|
||||
# wording was written — which is why it is taken before the loop and
|
||||
# not here.
|
||||
rule_versions.record_if_changed(session, rule, user_id, text_before)
|
||||
await session.commit()
|
||||
await session.refresh(rule)
|
||||
_refresh_rule_embedding(rule)
|
||||
return rule
|
||||
|
||||
|
||||
# ── Edit history (milestone 323) ───────────────────────────────────────
|
||||
#
|
||||
# The ACL-scoped reads live HERE rather than in services/rule_versions.py,
|
||||
# and not by preference: rulebooks imports rule_versions for the write path,
|
||||
# so the reverse import would be a cycle. The split is also the honest one —
|
||||
# rule_versions owns what a version IS, this module owns who may read one.
|
||||
|
||||
|
||||
async def list_rule_versions(rule_id: int, user_id: int):
|
||||
"""A rule's history, newest first. None when the rule is not readable.
|
||||
|
||||
Scoped through the rule itself, never through the version's `user_id`:
|
||||
that column is the ACTOR. Reading a rule's history is a question about
|
||||
the RULE, so anyone who can read the rule can read what it used to say,
|
||||
and anyone who cannot read the rule gets nothing — including the versions
|
||||
they personally wrote, if the rule has since moved out of their reach.
|
||||
"""
|
||||
async with async_session() as session:
|
||||
if await _fetch_owned_rule(session, rule_id, user_id) is None:
|
||||
return None
|
||||
return await rule_versions.list_versions(rule_id)
|
||||
|
||||
|
||||
async def get_rule_version(rule_id: int, version_id: int, user_id: int):
|
||||
"""One snapshot in full. None when the rule or the version is not found.
|
||||
|
||||
Takes the rule id as well as the version id so the ownership check has
|
||||
something to run against BEFORE the version is read, and so a version id
|
||||
from another rule cannot be read through a rule the caller does happen to
|
||||
own — the check and the fetch have to agree about which rule is in play.
|
||||
"""
|
||||
async with async_session() as session:
|
||||
if await _fetch_owned_rule(session, rule_id, user_id) is None:
|
||||
return None
|
||||
return (await session.execute(
|
||||
select(RuleVersion).where(
|
||||
RuleVersion.id == version_id,
|
||||
RuleVersion.rule_id == rule_id,
|
||||
)
|
||||
)).scalar_one_or_none()
|
||||
|
||||
|
||||
# ── Canon tags + typed edges (milestone 307) ───────────────────────────
|
||||
|
||||
async def set_rule_systems(
|
||||
@@ -1340,7 +1396,7 @@ async def get_applicable_rules(
|
||||
}
|
||||
|
||||
|
||||
def rules_payload(applicable: dict) -> dict:
|
||||
def rules_payload(applicable: dict, *, user_id: int | None, source: str) -> dict:
|
||||
"""The caller-facing shape of a get_applicable_rules() result.
|
||||
|
||||
Every surface that hands rules to an agent (enter_project, get_project,
|
||||
@@ -1351,7 +1407,34 @@ def rules_payload(applicable: dict) -> dict:
|
||||
`excluded_always_on` (milestone 297) names the always-on rulebooks this
|
||||
project decided NOT to inherit, so the departure is visible wherever the
|
||||
rules are.
|
||||
|
||||
IT ALSO RECORDS THE SURFACING, which is why it now takes a caller and a
|
||||
source. Every one of those surfaces is a bulk delivery — the applicable set
|
||||
handed over whole, chosen by nobody — so this is the one place that has to
|
||||
emit for all of them. Doing it per-caller instead would be five sites to
|
||||
remember, and #3430 gap 2 is what that costs: the process→skill sync went
|
||||
un-emitted through an entire dedicated telemetry survey because nothing
|
||||
forced its surface to be accounted for.
|
||||
|
||||
`source` stays the CALLER's name rather than a constant, so the readout can
|
||||
still separate the session handshake from a mid-session milestone read;
|
||||
`RANKED_SOURCES` in `rule_usage` is what folds them back together.
|
||||
|
||||
Emitting from here is safe in a way emitting from `get_applicable_rules`
|
||||
would not be: this function is only ever called to BUILD A REPLY. The two
|
||||
other callers of the rules machinery — the write-path etag arm
|
||||
(`plugin_context`) and `rules_etag_for` — compute a marker and show nobody
|
||||
anything, and counting those would put rules in the denominator that no
|
||||
agent ever saw.
|
||||
"""
|
||||
record_rule_surfaced(
|
||||
user_id=user_id,
|
||||
rule_ids=(
|
||||
[r["id"] for r in applicable.get("rules", [])]
|
||||
+ [r["id"] for r in applicable.get("project_rules", [])]
|
||||
),
|
||||
source=source,
|
||||
)
|
||||
return {
|
||||
"applicable_rules": applicable["rules"],
|
||||
"applicable_rules_truncated": applicable["truncated"],
|
||||
@@ -1363,6 +1446,101 @@ def rules_payload(applicable: dict) -> dict:
|
||||
}
|
||||
|
||||
|
||||
# ── The staleness marker (milestone 323 step 5) ────────────────────────
|
||||
#
|
||||
# WHAT THIS CAN AND CANNOT SEE. An etag catches a rule that MOVED after a
|
||||
# session loaded it. It is not a general staleness check, and a reader who
|
||||
# finds one here will assume it is:
|
||||
#
|
||||
# what goes wrong | caught?
|
||||
# ---------------------------------------------------|--------
|
||||
# another session edits a rule mid-flight | yes
|
||||
# the session is misremembering a rule read hours ago | yes
|
||||
# compaction summarised the rules out of context | NO
|
||||
#
|
||||
# The third is the most common and the marker is blind to it, because the
|
||||
# etag was in context too and went with the rules. The SessionStart nudge is
|
||||
# that case's only mechanism, and MUST NOT be softened because this exists —
|
||||
# retiring something that covers the common case in favour of something that
|
||||
# does not is the plausible mistake here.
|
||||
|
||||
_ETAG_EMPTY = "empty|0"
|
||||
|
||||
|
||||
def rules_etag(rules: list) -> str:
|
||||
"""A marker for "is the set you are holding still the current one?".
|
||||
|
||||
`max(updated_at)` alone is not enough: DELETING a rule moves no timestamp,
|
||||
and that is the single change that takes an instruction OUT of force —
|
||||
the one a session most needs to hear about. The count catches it.
|
||||
|
||||
Instance-agnostic (rule 115): it knows nothing about any particular
|
||||
rulebook, and an install with one rule or none produces a stable marker
|
||||
rather than an error. "No rules" must read as a state, not as a change,
|
||||
or every session on a fresh install would be told its rules had moved.
|
||||
"""
|
||||
if not rules:
|
||||
return _ETAG_EMPTY
|
||||
# A decoration must not be able to break what it decorates. This is
|
||||
# computed on the SessionStart path, where raising would cost the whole
|
||||
# context payload to save a hint — so a row with no usable timestamp is
|
||||
# skipped rather than compared, and a set with none degrades to a
|
||||
# count-only marker instead of failing. Count-only still catches a rule
|
||||
# added or deleted; it just cannot see an edit, which is the right way
|
||||
# round to lose information.
|
||||
stamps = [
|
||||
r.updated_at for r in rules
|
||||
if isinstance(getattr(r, "updated_at", None), datetime)
|
||||
]
|
||||
if not stamps:
|
||||
return f"unknown|{len(rules)}"
|
||||
return f"{max(stamps).isoformat()}|{len(rules)}"
|
||||
|
||||
|
||||
async def rules_etag_for(user_id: int, project_id: int = 0) -> str:
|
||||
"""The current marker for the set a session at this scope would hold.
|
||||
|
||||
Deliberately built from `list_always_on_rules` rather than from a
|
||||
`max()/count()` aggregate. An aggregate would be cheaper, and would have
|
||||
to restate that function's definition of the set — the always_on flag,
|
||||
the project's inception exclusions, the tier filter. Two definitions of
|
||||
"the session's rules" is how the marker starts disagreeing with the
|
||||
rules, which is worse than materialising a few dozen rows.
|
||||
"""
|
||||
rules = await list_always_on_rules(user_id, project_id=project_id)
|
||||
return rules_etag(rules)
|
||||
|
||||
|
||||
def rules_moved_since(rules: list, held_etag: str) -> list:
|
||||
"""The rules whose text changed after `held_etag` was issued.
|
||||
|
||||
Returns [] when the marker matches, is unparseable, or is absent — a
|
||||
caller cannot act on "something is different but I cannot say what", and
|
||||
a garbled marker must not be reported as a change.
|
||||
|
||||
A count difference is real news that this list cannot show: a rule
|
||||
DELETED since the marker was issued has no row left to return. Callers
|
||||
compare counts separately.
|
||||
"""
|
||||
if not held_etag or held_etag == _ETAG_EMPTY:
|
||||
return []
|
||||
stamp, _, _count = held_etag.partition("|")
|
||||
try:
|
||||
held_at = datetime.fromisoformat(stamp)
|
||||
except ValueError:
|
||||
return []
|
||||
return [r for r in rules if r.updated_at and r.updated_at > held_at]
|
||||
|
||||
|
||||
def etag_count(held_etag: str) -> int | None:
|
||||
"""How many rules the holder had. None when the marker cannot be read."""
|
||||
_stamp, _, count = (held_etag or "").partition("|")
|
||||
try:
|
||||
return int(count)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
# ── The staleness sweep (milestone 312) ────────────────────────────────
|
||||
|
||||
async def rules_due_for_verification(
|
||||
@@ -1455,14 +1633,7 @@ def verification_row(rule: Rule) -> dict:
|
||||
because "2026-06-14" and "74 days" prompt different reactions and only
|
||||
one of them is the question being asked.
|
||||
"""
|
||||
from datetime import datetime, timezone
|
||||
|
||||
days = None
|
||||
if rule.verified_at is not None:
|
||||
stamp = rule.verified_at
|
||||
if stamp.tzinfo is None:
|
||||
stamp = stamp.replace(tzinfo=timezone.utc)
|
||||
days = (datetime.now(timezone.utc) - stamp).days
|
||||
days = _days_since_verified(rule)
|
||||
return {
|
||||
"id": rule.id,
|
||||
"title": rule.title,
|
||||
|
||||
@@ -540,14 +540,60 @@ def compose_data(
|
||||
return out
|
||||
|
||||
|
||||
def recompose_data(note) -> dict:
|
||||
"""Rebuild a snippet's `data` mirror from its own body, title and tags.
|
||||
|
||||
For the GENERIC note door. `update_snippet` composes the mirror itself from
|
||||
the field set it just merged and never needs this; a plain
|
||||
`update_note(body=...)` — which the Knowledge feed's editor issues, because
|
||||
a snippet card there routes to /notes/:id — has no idea the mirror exists,
|
||||
and left it stale. `snippet_fields` then PREFERS the stale mirror, so the
|
||||
row reported its old repo/path/symbol to prior-art recall while displaying
|
||||
its new body: confidently wrong, which is worse than no record (#3128).
|
||||
|
||||
The body is the authority; the mirror is derived. That is already the rule
|
||||
this file states — it just had no enforcement on the path that bypasses
|
||||
`update_snippet`.
|
||||
|
||||
`verification` and `provenance` are CARRIED, not recomposed, because
|
||||
neither is in the body to parse — the same carry `compose_data` does for
|
||||
the snippet service's own writes. Note that a verdict does not need
|
||||
invalidating here: `code_sha` is recomputed from the new code, so a stale
|
||||
verdict expires itself on read exactly as it does after any other edit.
|
||||
"""
|
||||
parsed = parse_snippet_fields(note.title, note.body, note.tags)
|
||||
prior = note.data or {}
|
||||
return compose_data(
|
||||
name=parsed["name"],
|
||||
when_to_use=parsed["when_to_use"],
|
||||
signature=parsed["signature"],
|
||||
language=parsed["language"],
|
||||
code=parsed["code"],
|
||||
locations=parsed["locations"],
|
||||
merged_from=parsed["merged_from"],
|
||||
verification=prior.get("verification"),
|
||||
provenance=prior.get("provenance"),
|
||||
)
|
||||
|
||||
|
||||
def snippet_fields(note) -> dict:
|
||||
"""Structured fields for a snippet, preferring the indexed `data` column and
|
||||
falling back to parsing the body.
|
||||
|
||||
Both paths must agree, because rows written before migration 0070 have no
|
||||
`data` and are never backfilled — a hand-edited body is the authority for
|
||||
those, and there is no deadline by which they must be converted. `code` only
|
||||
ever comes from the body, since `data` doesn't carry it.
|
||||
THE BODY IS THE AUTHORITY; `data` is a mirror derived from it. Every writer
|
||||
keeps them in step — the snippet service composes the mirror from the field
|
||||
set it just merged, `update_note` recomposes it when a body reaches the
|
||||
generic door (#3128), and `backfill_snippet_data` filled the pre-0070 rows
|
||||
at startup. The fallback below is therefore a belt to that braces, not a
|
||||
second source of truth: it is what a row looks like before the backfill has
|
||||
run, and it must keep agreeing with the mirror.
|
||||
|
||||
(This docstring used to say those rows were "never backfilled". That was
|
||||
true when 0070 landed and stopped being true when the backfill shipped; it
|
||||
is corrected here because the sentence read as licence for a stale mirror,
|
||||
which is exactly the bug #3128 found.)
|
||||
|
||||
`code` only ever comes from the body, since `data` doesn't carry it.
|
||||
"""
|
||||
parsed = parse_snippet_fields(note.title, note.body, note.tags)
|
||||
stored = getattr(note, "data", None)
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
"""What a record's own check MEANS — shared by rules and notes.
|
||||
|
||||
Two record types carry `verify_with` / `expires_when` / `verified_at`: rules
|
||||
(milestone 312) and notes (milestone 317). What they share is BEHAVIOUR — how
|
||||
a stamp is read, how old it is, what "never" means — not storage and not the
|
||||
query. Note 3163 is the rule this file is an instance of: `semantic_search_*`
|
||||
could never have been shared between them because a rule scopes by rulebook
|
||||
ownership and a note scopes by the note ACL, so the two sweeps are siblings.
|
||||
These functions are the part that genuinely is common, factored so it exists
|
||||
once rather than twice.
|
||||
|
||||
Everything here is pure, sync and DUCK-TYPED: it reads `verify_with` and
|
||||
`verified_at` off whatever it is handed. That is deliberate. A shared base
|
||||
class or a protocol would tie two SQLAlchemy models together to share four
|
||||
lines of date arithmetic — the DRY costume, in the same note's words.
|
||||
"""
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
def last_verified_label(record) -> str | None:
|
||||
"""How long ago this record's check passed — None when it carries none.
|
||||
|
||||
Three states, and the distinction between the last two is the whole point:
|
||||
|
||||
- `None` — this record is a DECISION. There is nothing to go and check,
|
||||
and the question does not apply. Most records.
|
||||
- "never" — it asserts a fact and NOBODY HAS EVER CONFIRMED IT. The one
|
||||
worth acting on, and why the sweep sorts these first.
|
||||
- a date — somebody checked, then.
|
||||
|
||||
Callers attach this to a payload only when it is not None, so "no key" and
|
||||
"never verified" do not become two states every client has to tell apart.
|
||||
"""
|
||||
if not getattr(record, "verify_with", None):
|
||||
return None
|
||||
stamp = getattr(record, "verified_at", None)
|
||||
return stamp.date().isoformat() if stamp else "never"
|
||||
|
||||
|
||||
def days_since_verified(record) -> int | None:
|
||||
"""Whole days since the check last passed; None if it never has.
|
||||
|
||||
Computed rather than left to the reader, because "2026-06-14" and "74
|
||||
days" prompt different reactions and only one of them is the question
|
||||
being asked.
|
||||
|
||||
Naive stamps are read as UTC. Postgres hands these back tz-aware, but a
|
||||
restore, a fixture or a sqlite-backed test may not, and subtracting an
|
||||
aware datetime from a naive one raises — which would make the sweep fail
|
||||
on exactly the rows it exists to surface.
|
||||
"""
|
||||
stamp = getattr(record, "verified_at", None)
|
||||
if stamp is None:
|
||||
return None
|
||||
if stamp.tzinfo is None:
|
||||
stamp = stamp.replace(tzinfo=timezone.utc)
|
||||
return (datetime.now(timezone.utc) - stamp).days
|
||||
+68
-1
@@ -9,7 +9,68 @@ from __future__ import annotations
|
||||
from contextlib import contextmanager
|
||||
from datetime import datetime, timezone
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, MagicMock
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
|
||||
async def drive_update_note(note, **kwargs):
|
||||
"""Run `services/notes.update_note` against a stand-in row.
|
||||
|
||||
The patch stack is the point: update_note reaches for a version snapshot,
|
||||
an embedding refresh and a project reactivation on its way out, none of
|
||||
which a unit test has. Written twice — once for the snippet mirror
|
||||
(#3128) and once for the verification fields (#3182/317) — before being
|
||||
consolidated here.
|
||||
|
||||
Returns whatever update_note returned; assert on the `note` you passed in.
|
||||
"""
|
||||
from unittest.mock import AsyncMock as _AsyncMock
|
||||
|
||||
session = make_mock_session()
|
||||
result = MagicMock()
|
||||
result.scalars.return_value.first.return_value = note
|
||||
session.execute = _AsyncMock(return_value=result)
|
||||
with patch("scribe.services.notes.async_session") as cls, \
|
||||
patch("scribe.services.notes.embed_note", MagicMock()), \
|
||||
patch("scribe.services.notes._maybe_reactivate_project", _AsyncMock()), \
|
||||
patch("scribe.services.note_versions.create_version", _AsyncMock()):
|
||||
cls.return_value = session
|
||||
from scribe.services.notes import update_note
|
||||
return await update_note(user_id=7, note_id=note.id, **kwargs)
|
||||
|
||||
|
||||
def tool_doc(module: str, name: str) -> str:
|
||||
"""An MCP tool's docstring, whitespace-flattened.
|
||||
|
||||
Flattened because these are hard-wrapped at ~76 characters, so any phrase
|
||||
worth asserting on is liable to straddle a line break — a property of the
|
||||
formatter, not of the guidance. The disambiguator guard (#3123) learned
|
||||
that on its own first run, matching raw text and reporting a phrase absent
|
||||
that was plainly there.
|
||||
|
||||
Used by every test that pins the docstring CONTRACT rather than its
|
||||
wording. The tool docstring is the agent-facing contract (rule 119), so
|
||||
these guards exist to catch it being tidied down to a parameter list.
|
||||
"""
|
||||
import importlib
|
||||
import re as _re
|
||||
|
||||
fn = getattr(importlib.import_module(module), name)
|
||||
assert fn.__doc__, f"{name} has no docstring at all"
|
||||
return _re.sub(r"\s+", " ", fn.__doc__)
|
||||
|
||||
|
||||
def compiled_sql(element, dialect=None) -> str:
|
||||
"""A SQLAlchemy clause or statement rendered as literal SQL text.
|
||||
|
||||
For asserting on the shape of a predicate without a database — which is how
|
||||
the visibility clauses and the knowledge facets are both tested. Was a
|
||||
private copy in each of those modules before #3128 needed a third.
|
||||
|
||||
Pass `dialect` when the assertion is about something only one backend
|
||||
renders — a Postgres row-lock mode, say. The generic dialect is enough for
|
||||
a predicate's shape and would quietly drop the rest.
|
||||
"""
|
||||
return str(element.compile(dialect=dialect, compile_kwargs={"literal_binds": True}))
|
||||
|
||||
|
||||
def make_mock_session() -> AsyncMock:
|
||||
@@ -92,6 +153,9 @@ def fake_note(**attrs) -> MagicMock:
|
||||
"id": 1, "title": "t", "body": "", "tags": [], "user_id": 7,
|
||||
"note_type": "note", "is_task": False, "task_kind": "work",
|
||||
"data": None, "deleted_at": None,
|
||||
# Milestone 317: a truthy mock here reads as "this note carries a
|
||||
# check", which trips the guard on records that may not have one.
|
||||
"verify_with": None, "expires_when": None, "verified_at": None,
|
||||
}, attrs)
|
||||
|
||||
|
||||
@@ -101,6 +165,7 @@ def fake_task(**attrs) -> MagicMock:
|
||||
"id": 1, "title": "t", "body": "", "status": "todo", "priority": "none",
|
||||
"tags": [], "parent_id": None, "project_id": None, "is_task": True,
|
||||
"task_kind": "work", "user_id": 7, "deleted_at": None,
|
||||
"verify_with": None, "expires_when": None, "verified_at": None,
|
||||
}, attrs)
|
||||
|
||||
|
||||
@@ -112,6 +177,8 @@ def fake_snippet(**attrs) -> MagicMock:
|
||||
"body": "```js\nreturn 1\n```\n", "tags": ["js", "snippet"],
|
||||
"note_type": "snippet", "is_task": False, "task_kind": "work",
|
||||
"user_id": 7, "data": None, "deleted_at": None,
|
||||
"status": None,
|
||||
"verify_with": None, "expires_when": None, "verified_at": None,
|
||||
}, attrs)
|
||||
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ parameters, with the "is this even the right tool" paragraph quietly gone.
|
||||
import re
|
||||
|
||||
import pytest
|
||||
from tests.helpers import tool_doc as _doc
|
||||
|
||||
# The create surfaces and where they live. `start_planning` is here because
|
||||
# it is a create in everything but name — it is how a plan comes into being.
|
||||
@@ -50,21 +51,6 @@ _ALTERNATIVES = [
|
||||
]
|
||||
|
||||
|
||||
def _doc(module: str, name: str) -> str:
|
||||
"""The docstring with its whitespace flattened.
|
||||
|
||||
Flattened because a docstring is hard-wrapped: "design system" spans a
|
||||
line break in at least one of these, and matching the raw text would
|
||||
report it absent. The first draft of this check did exactly that, and
|
||||
caught it on itself.
|
||||
"""
|
||||
import importlib
|
||||
|
||||
fn = getattr(importlib.import_module(module), name)
|
||||
assert fn.__doc__, f"{name} has no docstring at all"
|
||||
return re.sub(r"\s+", " ", fn.__doc__)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("module", "name"), _SURFACES)
|
||||
def test_a_create_surface_names_at_least_two_alternatives(module, name):
|
||||
"""Reaching for the wrong tool must still put the right one in view."""
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
"""The embedding refresh must LOSE to a delete, not race it (#3262).
|
||||
|
||||
Both upserts replace a record's vectors as delete-then-insert. That takes two
|
||||
row locks — the chunk rows, then the parent row via the insert's foreign key —
|
||||
in the exact reverse of the order a cascading delete of the parent takes them.
|
||||
Postgres calls that a deadlock and kills one side at random, which sometimes
|
||||
means killing the user's delete.
|
||||
|
||||
These pin the ORDER, not the outcome: the claim on the parent goes first, and
|
||||
when the claim fails nothing else in the transaction runs. Compiling the
|
||||
statement is the only way to assert on a lock mode without a database — the
|
||||
integration twin (test_integration_embedding_yields_to_delete.py) proves the
|
||||
behaviour against a real one.
|
||||
"""
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
from sqlalchemy.dialects import postgresql
|
||||
from sqlalchemy.exc import OperationalError
|
||||
|
||||
from scribe.services import embeddings as emb
|
||||
from tests.helpers import compiled_sql
|
||||
|
||||
ONE_VECTOR = [[0.0] * 384]
|
||||
|
||||
# The lock mode is a Postgres extension — the generic dialect renders a plain
|
||||
# FOR UPDATE and would pass an assertion that proves nothing.
|
||||
PG = postgresql.dialect()
|
||||
|
||||
|
||||
def _mock_session(lock_result: object = 7, execute_side_effect=None):
|
||||
"""A session stand-in whose first execute answers the parent-row claim."""
|
||||
session = MagicMock()
|
||||
claimed = MagicMock()
|
||||
claimed.scalar_one_or_none.return_value = lock_result
|
||||
if execute_side_effect is not None:
|
||||
session.execute = AsyncMock(side_effect=execute_side_effect)
|
||||
else:
|
||||
session.execute = AsyncMock(return_value=claimed)
|
||||
session.commit = AsyncMock()
|
||||
session.add = MagicMock()
|
||||
ctx = MagicMock()
|
||||
ctx.__aenter__ = AsyncMock(return_value=session)
|
||||
ctx.__aexit__ = AsyncMock(return_value=False)
|
||||
return session, ctx
|
||||
|
||||
|
||||
def _lock_unavailable() -> OperationalError:
|
||||
"""What asyncpg raises through SQLAlchemy when NOWAIT can't take the row."""
|
||||
return OperationalError("SELECT ...", {}, Exception("lock not available"))
|
||||
|
||||
|
||||
async def test_a_note_refresh_claims_the_row_before_rewriting_its_vectors():
|
||||
"""The claim is FIRST, and it is FOR KEY SHARE NOWAIT.
|
||||
|
||||
FOR KEY SHARE because that is exactly the lock the insert's foreign key
|
||||
takes anyway — it conflicts with a delete of the note and with nothing
|
||||
else, so an ordinary edit is unaffected. NOWAIT because the whole point is
|
||||
to lose immediately rather than queue up behind the delete and hold the
|
||||
chunk rows while doing it.
|
||||
"""
|
||||
session, ctx = _mock_session()
|
||||
with (
|
||||
patch.object(emb, "async_session", return_value=ctx),
|
||||
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
|
||||
):
|
||||
await emb.upsert_note_embedding(7, 42, "T", "a short body")
|
||||
|
||||
claim, replace = [c.args[0] for c in session.execute.call_args_list][:2]
|
||||
assert compiled_sql(claim, dialect=PG).startswith("SELECT notes.id")
|
||||
assert "FOR KEY SHARE NOWAIT" in compiled_sql(claim, dialect=PG)
|
||||
assert compiled_sql(replace, dialect=PG).startswith("DELETE FROM note_embeddings")
|
||||
session.add.assert_called()
|
||||
|
||||
|
||||
async def test_a_rule_refresh_claims_the_row_before_rewriting_its_vectors():
|
||||
"""The rule twin — the path the reported deadlock actually took."""
|
||||
session, ctx = _mock_session()
|
||||
with (
|
||||
patch.object(emb, "async_session", return_value=ctx),
|
||||
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
|
||||
):
|
||||
await emb.upsert_rule_embedding(9, "T", "a short statement", "on write")
|
||||
|
||||
claim, replace = [c.args[0] for c in session.execute.call_args_list][:2]
|
||||
assert compiled_sql(claim, dialect=PG).startswith("SELECT rules.id")
|
||||
assert "FOR KEY SHARE NOWAIT" in compiled_sql(claim, dialect=PG)
|
||||
assert compiled_sql(replace, dialect=PG).startswith("DELETE FROM rule_embeddings")
|
||||
session.add.assert_called()
|
||||
|
||||
|
||||
async def test_a_record_being_deleted_is_left_alone_rather_than_raced():
|
||||
"""The claim failing ends the write — it does not fall through to the
|
||||
delete-and-insert that would take the locks in the losing order."""
|
||||
session, ctx = _mock_session(execute_side_effect=_lock_unavailable())
|
||||
with (
|
||||
patch.object(emb, "async_session", return_value=ctx),
|
||||
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
|
||||
):
|
||||
await emb.upsert_rule_embedding(9, "T", "a short statement", "on write")
|
||||
|
||||
assert session.execute.await_count == 1, "it stopped at the claim"
|
||||
session.add.assert_not_called()
|
||||
session.commit.assert_not_awaited()
|
||||
|
||||
|
||||
async def test_a_record_already_gone_is_not_re_embedded():
|
||||
"""A vector inserted for a row that no longer exists is either a foreign
|
||||
key violation or, worse, a resurrected chunk. Nothing to refresh."""
|
||||
session, ctx = _mock_session(lock_result=None)
|
||||
with (
|
||||
patch.object(emb, "async_session", return_value=ctx),
|
||||
patch.object(emb, "get_embeddings", AsyncMock(return_value=ONE_VECTOR)),
|
||||
):
|
||||
await emb.upsert_note_embedding(7, 42, "T", "a short body")
|
||||
|
||||
assert session.execute.await_count == 1
|
||||
session.add.assert_not_called()
|
||||
session.commit.assert_not_awaited()
|
||||
@@ -0,0 +1,128 @@
|
||||
"""Every request the web UI makes has a deadline (rule 156).
|
||||
|
||||
A source-inspection guard in the unit lane — there is no frontend test runner,
|
||||
and this is a property of the source rather than of a rendered result, so
|
||||
reading the source is the honest way to check it.
|
||||
|
||||
WHY. `fetch`'s default is to wait as long as the browser will. That is not a
|
||||
long timeout, it is the absence of one, and there is no state a surface can
|
||||
render for "pending forever" that is not a lie — the spinner that never
|
||||
resolves is indistinguishable from work still in progress. Rule 156 names
|
||||
`fetch` specifically:
|
||||
|
||||
When a library's default is "wait indefinitely" — `fetch`, most HTTP
|
||||
clients, a bare `await` on a stream — supplying the deadline is part of
|
||||
using it, not a hardening pass for later.
|
||||
|
||||
Before this guard, no request in the app carried one.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
FRONTEND = Path(__file__).resolve().parents[1] / "frontend" / "src"
|
||||
CLIENT = FRONTEND / "api" / "client.ts"
|
||||
|
||||
|
||||
def _call_text(src: str, start: int) -> str:
|
||||
"""The source of one `fetch(...)` call, from its open paren to its close.
|
||||
|
||||
Naive paren balancing. Adequate because every call site here passes an
|
||||
object literal, and a construct complex enough to defeat it is one worth
|
||||
looking at by hand anyway.
|
||||
"""
|
||||
depth = 0
|
||||
for i in range(start, len(src)):
|
||||
if src[i] == "(":
|
||||
depth += 1
|
||||
elif src[i] == ")":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return src[start:i + 1]
|
||||
return src[start:]
|
||||
|
||||
|
||||
def _fetch_calls() -> list[tuple[Path, str]]:
|
||||
calls: list[tuple[Path, str]] = []
|
||||
for path in list(FRONTEND.rglob("*.ts")) + list(FRONTEND.rglob("*.vue")):
|
||||
src = path.read_text()
|
||||
for m in re.finditer(r"\bfetch\(", src):
|
||||
calls.append((path, _call_text(src, m.end() - 1)))
|
||||
return calls
|
||||
|
||||
|
||||
def test_every_fetch_passes_a_signal():
|
||||
"""No bare `fetch` anywhere in the frontend.
|
||||
|
||||
Stated on the SIGNAL rather than on a timeout value, because the two
|
||||
legitimate shapes here produce different values and only share this: an
|
||||
ordinary call takes the client's default, a stream bounds its CONNECT and
|
||||
then deliberately runs unbounded, and a bulk transfer passes minutes. What
|
||||
they must all do is pass something.
|
||||
"""
|
||||
naked = [
|
||||
f"{path.relative_to(FRONTEND)}: {call[:70]}"
|
||||
for path, call in _fetch_calls()
|
||||
if "signal:" not in call
|
||||
]
|
||||
assert not naked, (
|
||||
"these fetch calls carry no AbortSignal, so they wait forever "
|
||||
"(rule 156):\n " + "\n ".join(naked)
|
||||
)
|
||||
|
||||
|
||||
def test_the_client_applies_its_deadline_by_default():
|
||||
"""The specific regression that would silently undo this.
|
||||
|
||||
An earlier pass (#3329) made `timeoutMs` OPT-IN and used it at exactly one
|
||||
call site, which left ~330 others waiting forever while the mechanism
|
||||
looked present. Reverting to that shape would not fail the guard above —
|
||||
every call would still reach `fetch` through `request()` — so the default
|
||||
is pinned here separately.
|
||||
|
||||
`??` is the operative character: `opts?.timeoutMs || DEFAULT` would treat
|
||||
an explicit 0 as "use the default", and `opts?.timeoutMs` alone would
|
||||
reinstate the opt-in bug.
|
||||
"""
|
||||
src = CLIENT.read_text()
|
||||
assert "opts?.timeoutMs ?? DEFAULT_TIMEOUT_MS" in src, (
|
||||
"request() must fall back to DEFAULT_TIMEOUT_MS — without it the "
|
||||
"deadline is opt-in again and almost nothing opts in"
|
||||
)
|
||||
|
||||
|
||||
def test_a_timeout_arrives_as_the_error_shape_callers_already_handle():
|
||||
"""Rule 156's second half: expiry surfaces as a NAMED failure.
|
||||
|
||||
A raw `DOMException: TimeoutError` reaches `apiErrorMessage(e, fallback)`
|
||||
as an object with no `body`, so every catch site in the app would print its
|
||||
generic fallback and the timeout would be invisible in exactly the
|
||||
situation it exists to expose. Rethrowing as `ApiError` is what makes the
|
||||
other ~330 call sites report it without being edited.
|
||||
"""
|
||||
src = CLIENT.read_text()
|
||||
assert 'e.name === "TimeoutError"' in src, (
|
||||
"request() must recognise a timeout specifically"
|
||||
)
|
||||
assert "new ApiError(CLIENT_TIMEOUT_STATUS" in src, (
|
||||
"a timeout must be rethrown as ApiError so apiErrorMessage can read it"
|
||||
)
|
||||
|
||||
|
||||
def test_a_deliberate_cancellation_is_not_reported_as_a_timeout():
|
||||
"""Only `TimeoutError` is converted, never `AbortError`.
|
||||
|
||||
A caller that cancelled its own request — a superseded search, a closed
|
||||
stream — must not have that surfaced to the user as a server failure. The
|
||||
guard is that the conversion is gated on the name, which the assertion
|
||||
above already pins; this states the intent so the gate is not "simplified"
|
||||
into catching every abort.
|
||||
"""
|
||||
src = CLIENT.read_text()
|
||||
convert = src[src.index("async function request<"):]
|
||||
convert = convert[:convert.index("\n}")]
|
||||
assert "AbortError" not in convert, (
|
||||
"request() must not convert AbortError — a deliberate cancellation is "
|
||||
"not a timeout"
|
||||
)
|
||||
@@ -15,14 +15,17 @@ def test_rules_payload_carries_excluded_always_on_as_the_seventh_key():
|
||||
out = rules_payload({
|
||||
"rules": [], "truncated": False, "subscribed_rulebooks": [],
|
||||
"excluded_always_on": [{"id": 1, "title": "Family"}],
|
||||
})
|
||||
}, user_id=1, source="enter_project")
|
||||
assert set(out) == {
|
||||
"applicable_rules", "applicable_rules_truncated", "subscribed_rulebooks",
|
||||
"project_rules", "suppressed_rules", "suppressed_topics", "excluded_always_on",
|
||||
}
|
||||
assert out["excluded_always_on"] == [{"id": 1, "title": "Family"}]
|
||||
# An older applicable dict without the key still renders (empty list).
|
||||
assert rules_payload({"rules": [], "truncated": False, "subscribed_rulebooks": []})["excluded_always_on"] == []
|
||||
assert rules_payload(
|
||||
{"rules": [], "truncated": False, "subscribed_rulebooks": []},
|
||||
user_id=1, source="enter_project",
|
||||
)["excluded_always_on"] == []
|
||||
|
||||
|
||||
def test_list_always_on_rules_service_and_tool_take_a_project_id():
|
||||
|
||||
@@ -195,6 +195,46 @@ def test_displaced_topics_live_on_a_delivered_surface():
|
||||
)
|
||||
|
||||
|
||||
# The SECOND pull (milestone 333 step 3). `list_always_on_rules()` fetches the
|
||||
# resident tier; this one says that tier is not all of them, and that a
|
||||
# conditional rule has to be gone looking for. However a surface words the
|
||||
# surrounding prose, it names the call.
|
||||
RETRIEVE = 'content_type="rule"'
|
||||
|
||||
|
||||
def test_every_session_start_surface_states_the_conditional_retrieval():
|
||||
"""The push/pull asymmetry, one level in.
|
||||
|
||||
The tests above pin that a session PULLS the resident rules rather than
|
||||
trusting the SessionStart push. This pins the same shape between the two
|
||||
TIERS: an always-on rule is delivered, a conditional one is retrieved, and
|
||||
a surface that states only the first leaves a session reading its loaded
|
||||
set as the whole rulebook.
|
||||
|
||||
That reading is wrong in the direction that costs something. "Nothing was
|
||||
pushed" and "no rule applies" are different claims, and only one of them
|
||||
has been checked — the same asymmetry as #2198, now between tiers instead
|
||||
of between channels.
|
||||
|
||||
It is also what made the always-on tier the only one that worked, on any
|
||||
install rather than this one (rule 115). A rule nothing retrieves has to be
|
||||
resident to bind at all, so every rule worth keeping becomes resident; and
|
||||
a resident rule costs tokens in every session forever, so a rulebook that
|
||||
only delivers cannot grow past what one session can hold. Retrieval is what
|
||||
lifts that ceiling — and it only fires if something asks.
|
||||
"""
|
||||
missing = []
|
||||
for path in SESSION_START_SURFACES:
|
||||
if RETRIEVE not in path.read_text():
|
||||
missing.append(str(path.relative_to(ROOT)))
|
||||
assert not missing, (
|
||||
f"these surfaces state the always-on pull but never tell the agent to "
|
||||
f"retrieve a conditional rule ({RETRIEVE}): {missing}. A session that "
|
||||
f"reads its loaded set as the whole rulebook will act on \"I was not "
|
||||
f"told\" as if it meant \"there is no rule\" (milestone 333 step 3)."
|
||||
)
|
||||
|
||||
|
||||
def test_no_surface_names_the_push_without_stating_the_pull():
|
||||
"""The exact shape #2497 took.
|
||||
|
||||
|
||||
@@ -0,0 +1,207 @@
|
||||
"""Real-Postgres round trip for the note fields #3182 restored.
|
||||
|
||||
The unit lane can prove a serialiser EMITS a key. It cannot prove a restore
|
||||
puts the value back on the right row, and the note->note edges are exactly
|
||||
where that distinction bites: `parent_id` and `arose_from_id` hold ids from
|
||||
the SOURCE database, so a restore that writes them straight through succeeds,
|
||||
reports success, and silently points every edge at whatever record happens to
|
||||
hold that number here.
|
||||
|
||||
That is why #3182 was not fixed in passing.
|
||||
|
||||
These drive the REAL `restore_full_backup`, not a reimplementation of its
|
||||
loop. A test that re-derives the remap it is checking would agree with
|
||||
whatever the product does, including nothing.
|
||||
"""
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from sqlalchemy import select
|
||||
|
||||
from scribe.models import async_session
|
||||
from scribe.models.note import Note
|
||||
from scribe.models.user import User
|
||||
from scribe.services import backup
|
||||
from tests.helpers import ensure_user
|
||||
|
||||
pytestmark = [pytest.mark.integration, pytest.mark.usefixtures("_dispose_engine")]
|
||||
|
||||
RESTORED_USERNAME = "backup_roundtrip_restored"
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def source():
|
||||
"""A snippet, a process, and an issue that arose from a work task — the
|
||||
shapes whose identity the backup used to drop — exported as backup rows.
|
||||
|
||||
Deleted children-first: `arose_from_id` is a real FK, so removing the
|
||||
origin while the issue still points at it is asking the database a
|
||||
question the test has no reason to ask.
|
||||
"""
|
||||
async with async_session() as s:
|
||||
owner = await ensure_user(s, "backup_roundtrip_owner")
|
||||
uid = owner.id
|
||||
await s.commit()
|
||||
|
||||
async with async_session() as s:
|
||||
origin = Note(
|
||||
user_id=uid, title="the work that broke", body="",
|
||||
status="done", task_kind="work",
|
||||
)
|
||||
snippet = Note(
|
||||
user_id=uid, title="debounce — rate-limit", body="```js\n1\n```",
|
||||
note_type="snippet", data={"name": "debounce", "language": "js"},
|
||||
)
|
||||
process = Note(
|
||||
user_id=uid, title="DRY pass", body="steps", note_type="process",
|
||||
)
|
||||
s.add_all([origin, snippet, process])
|
||||
await s.commit()
|
||||
origin_id = origin.id
|
||||
|
||||
issue = Note(
|
||||
user_id=uid, title="the fix", body="symptom -> cause -> fix",
|
||||
status="done", task_kind="issue", arose_from_id=origin_id,
|
||||
description="one-liner",
|
||||
)
|
||||
s.add(issue)
|
||||
await s.commit()
|
||||
order = [issue.id, origin_id, snippet.id, process.id]
|
||||
|
||||
async with async_session() as s:
|
||||
rows = (await s.execute(select(Note).where(Note.id.in_(order)))).scalars().all()
|
||||
note_rows = backup._note_rows(list(rows))
|
||||
user_rows = backup._user_rows(
|
||||
[(await s.execute(select(User).where(User.id == uid))).scalars().one()]
|
||||
)
|
||||
# Restore mints a NEW user from the payload, so the restored corpus is
|
||||
# entirely separate from the source — which is what makes the id
|
||||
# assertions meaningful. Renamed here rather than in a sibling fixture:
|
||||
# `restored` depends on this one, and a rename elsewhere might not have
|
||||
# run by the time the restore does.
|
||||
user_rows[0]["username"] = RESTORED_USERNAME
|
||||
|
||||
yield {
|
||||
"payload": {
|
||||
"version": backup.BACKUP_VERSION,
|
||||
"users": user_rows,
|
||||
"notes": note_rows,
|
||||
},
|
||||
"origin_id": origin_id,
|
||||
"owner_id": uid,
|
||||
}
|
||||
|
||||
async with async_session() as s:
|
||||
for nid in order:
|
||||
row = await s.get(Note, nid)
|
||||
if row is not None:
|
||||
await s.delete(row)
|
||||
await s.commit()
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def restored(source):
|
||||
"""Run the real restore, then hand back the new rows by title.
|
||||
|
||||
The restore mints a NEW user from the payload, so the restored corpus is
|
||||
entirely separate from the source one — which is what makes the id
|
||||
assertions below meaningful.
|
||||
"""
|
||||
await backup.restore_full_backup(source["payload"])
|
||||
|
||||
async with async_session() as s:
|
||||
user = (await s.execute(
|
||||
select(User).where(User.username == RESTORED_USERNAME)
|
||||
)).scalars().first()
|
||||
assert user is not None, "the payload's user was not restored"
|
||||
rows = (await s.execute(
|
||||
select(Note).where(Note.user_id == user.id)
|
||||
)).scalars().all()
|
||||
by_title = {n.title: n for n in rows}
|
||||
new_user_id = user.id
|
||||
|
||||
yield by_title, source
|
||||
|
||||
async with async_session() as s:
|
||||
fresh = [await s.get(Note, r.id) for r in by_title.values()]
|
||||
for row in fresh:
|
||||
if row is not None:
|
||||
row.arose_from_id = None
|
||||
row.parent_id = None
|
||||
await s.flush()
|
||||
for row in fresh:
|
||||
if row is not None:
|
||||
await s.delete(row)
|
||||
await s.commit()
|
||||
async with async_session() as s:
|
||||
user = await s.get(User, new_user_id)
|
||||
if user is not None:
|
||||
await s.delete(user)
|
||||
await s.commit()
|
||||
|
||||
|
||||
@pytest_asyncio.fixture(autouse=True)
|
||||
async def _no_leftover_restored_user():
|
||||
"""The payload's username is fixed, so a previous failed run would leave a
|
||||
row that makes `restored` pick the wrong user. Clear it first."""
|
||||
async with async_session() as s:
|
||||
stale = (await s.execute(
|
||||
select(User).where(User.username == RESTORED_USERNAME)
|
||||
)).scalars().all()
|
||||
for user in stale:
|
||||
notes = (await s.execute(
|
||||
select(Note).where(Note.user_id == user.id)
|
||||
)).scalars().all()
|
||||
for n in notes:
|
||||
n.arose_from_id = None
|
||||
n.parent_id = None
|
||||
await s.flush()
|
||||
for n in notes:
|
||||
await s.delete(n)
|
||||
await s.delete(user)
|
||||
await s.commit()
|
||||
|
||||
|
||||
async def test_a_restored_record_keeps_what_it_IS(restored):
|
||||
"""#3182's headline. Without note_type and task_kind a restore reported
|
||||
success and handed back a corpus where every snippet and process was a
|
||||
plain note and every issue and spike was `work` — the whole vocabulary
|
||||
milestone 312 and #3128 were about, gone, with nothing to notice it by."""
|
||||
by_title, _ = restored
|
||||
|
||||
assert by_title["debounce — rate-limit"].note_type == "snippet"
|
||||
assert by_title["DRY pass"].note_type == "process"
|
||||
assert by_title["the fix"].task_kind == "issue"
|
||||
assert by_title["the work that broke"].task_kind == "work"
|
||||
|
||||
|
||||
async def test_a_restored_snippet_keeps_its_queryable_mirror(restored):
|
||||
"""The one field that would self-heal — backfill_snippet_data rebuilds it
|
||||
from the body at startup — but a restore should not hand back a corpus
|
||||
that needs a restart before it is findable by location."""
|
||||
by_title, _ = restored
|
||||
assert by_title["debounce — rate-limit"].data == {
|
||||
"name": "debounce", "language": "js",
|
||||
}
|
||||
|
||||
|
||||
async def test_the_provenance_edge_is_remapped_not_copied(restored):
|
||||
"""THE regression, and the reason this needed a real database.
|
||||
|
||||
The payload's `arose_from_id` is an id in the SOURCE database. Copying it
|
||||
through would leave the restored issue pointing at whatever record happens
|
||||
to hold that number — a restore that succeeds and silently rewires
|
||||
history. The edge must land on the RESTORED origin instead.
|
||||
"""
|
||||
by_title, src = restored
|
||||
issue = by_title["the fix"]
|
||||
origin = by_title["the work that broke"]
|
||||
|
||||
assert issue.arose_from_id == origin.id
|
||||
# ...and that is a different row from the one the payload named.
|
||||
assert issue.arose_from_id != src["origin_id"]
|
||||
|
||||
|
||||
async def test_description_and_status_survive(restored):
|
||||
by_title, _ = restored
|
||||
assert by_title["the fix"].description == "one-liner"
|
||||
assert by_title["the fix"].status == "done"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user