Compare commits

...
6 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 32ebc2b1d4 docs(6205): retrieval_surfaces and family_adoption comments drop citations and the decision quote
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m25s
CI & Build / Python tests (push) Successful in 2m9s
CI & Build / Build & push image (push) Successful in 20s
retrieval_surfaces' header states why the arms read their numbers from one
table and why a floor is moved by reading the refused record, without the
quoted decision, the retired arm or the declined-rule instance.
family_adoption drops milestone/step labels from its section headers and
states the no-ids reason instead of naming the trap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:29:52 -04:00
bvandeusenandClaude Opus 5.5 dd4bc234e9 docs(6205): inception and family comments drop record, milestone and date citations
inception states the legacy stamp and the retired rulebook keys as facts
about stored records; family keeps the repeat-trigger calibration numbers
without the date and record they were measured on, and drops the instance
that taught the per-kind comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:29:03 -04:00
bvandeusenandClaude Opus 5.5 5f96545ae1 docs(6205): record_refs and moment_delivery comments drop record and milestone citations
record_refs states why a guessed id is refused without quoting the
operator; moment_delivery's docstrings name what each door does without the
milestone and step that built it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:28:23 -04:00
bvandeusenandClaude Opus 5.5 5382e0a7da docs(6205): systems comments state the reason, not the instance that taught it
The name gate, catalog and starter-set docstrings drop milestone and record
citations and the history of the tuple constant; each states why it is
shaped this way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:27:44 -04:00
bvandeusenandClaude Opus 5.5 6dab239eb2 docs(6205): note_usage comments state the reason, not the instance that taught it
The silent-outage story (#2663), the ranked/ambient split (#2477), the
transfer evidence (#3735) and the attach_usage consolidation (#4230) are
stated as the property each guards, without the incident.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:27:19 -04:00
bvandeusenandClaude Opus 5.5 757d016a0c docs(6205): milestones comments state the reason, not the instance that taught it
The pool-exhaustion, handshake-size and stepless-roadmap comments state the
condition the code guards instead of the measured incident; citations go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:26:50 -04:00
9 changed files with 135 additions and 171 deletions
+14 -17
View File
@@ -1,4 +1,4 @@
"""Family canon's promotion engine (milestone 463 step 3).
"""Family canon's promotion engine.
A family idea moves between three states — candidate, canon, retired — and
every move is a decision with a reason, written to `family_decisions`. No
@@ -62,7 +62,7 @@ logger = logging.getLogger(__name__)
#
# These are what an evaluation is judged against, on every install. They live
# here — and in the tool docstrings that quote them — rather than in any
# instance's rulebook (rules 115, 119).
# instance's rulebook, so every install judges by the same text.
CRITERIA = (
{
@@ -97,13 +97,11 @@ CRITERIA_KEYS = tuple(c["key"] for c in CRITERIA)
TRIGGERS = ("citation", "repeat", "milestone")
# The repeat trigger's similarity floor. MEASURED, not guessed (2026-10-06,
# #4989): against a description of one pattern known to have been built four
# times in four projects, the records that implement it scored 0.79-0.82, and
# the best match in a project that never built it scored 0.65. A trigger only
# opens an evaluation, so a false positive costs one judgment, not a wrong
# promotion. Step 5 measures the cross-language threshold against known pairs
# and supersedes this.
# The repeat trigger's similarity floor. MEASURED, not guessed: against a
# description of one pattern known to have been built four times in four
# projects, the records that implement it scored 0.79-0.82, and the best match
# in a project that never built it scored 0.65. A trigger only opens an
# evaluation, so a false positive costs one judgment, not a wrong promotion.
REPEAT_THRESHOLD = 0.80
# Words that say a citation is the SOURCE of a pattern rather than background.
@@ -119,12 +117,11 @@ _LINEAGE = re.compile(
_LINEAGE_WINDOW = 80
# Note kinds the repeat trigger compares: recorded knowledge and shapes, not
# the to-do list — and each only against its own kind (#5130). A repeat is the
# same KIND of thing made twice: two shapes built, or two ideas written down.
# the to-do list — and each only against its own kind. A repeat is the same
# KIND of thing made twice: two shapes built, or two ideas written down.
# Across kinds the score stops meaning that. One chunk of a long narrative
# note that mentions a shape's vocabulary scores like the shape itself: a
# dated release log matched a signed-APK CI snippet at 0.83, above this
# trigger's floor, and opened the log as a candidate. The cross-kind relation
# note that mentions a shape's vocabulary scores like the shape itself — above
# this trigger's floor — and would open the note as a candidate. The cross-kind relation
# that IS real — code built from an idea written elsewhere — has its own
# routes: the citation trigger when the lineage is named, and the shape ledger
# once the idea is canon.
@@ -172,7 +169,7 @@ async def _topic(session, user_id: int, topic_id: int | None, *, rules: bool) ->
async def _snapshot(session, idea: FamilyIdea | None) -> dict | None:
"""The idea's state, as a decision's before/after. Slugs, never ids: an id
inside JSON cannot be remapped by a restore (the #3182 trap)."""
inside JSON cannot be remapped by a restore."""
if idea is None:
return None
return {
@@ -219,7 +216,7 @@ async def get_idea(user_id: int, note_id: int) -> dict | None:
"platforms": await _platform_slugs(session, note_id),
"topic": await _topic(session, user_id, idea.topic_id, rules=True),
"adoptions": counts,
# Brief: the history is read in bulk; the full rows are list_decisions (#5131).
# Brief: the history is read in bulk; the full rows are list_decisions.
"decisions": [d.to_brief() for d in decisions],
"undoable_decision_id": _undoable_id(decisions),
})
@@ -478,7 +475,7 @@ async def propose(
Idempotent: an existing idea comes back unchanged and nothing is logged —
a trigger firing on every edit of a record must not grow the log. Write-
gated on the note: an idea is state on someone's record (rule 78).
gated on the note: an idea is state on someone's record.
"""
if not (reason or "").strip():
raise ValueError("a proposal needs a reason — what makes this a family idea?")
+9 -8
View File
@@ -1,4 +1,4 @@
"""Family canon's adoption ledger (milestone 463 step 4).
"""Family canon's adoption ledger.
Promotion (services/family.py) decides that an idea is canon for some
platforms. This module decides what each project on those platforms does
@@ -44,7 +44,7 @@ The losing side's reasoning is folded into the idea's note — as a trap, an
alternative, or the branch for its condition — and is never dropped. The
substance changed, so the version moves.
SHAPES (step 5). The shape ledger answers too: a shape classified against an
SHAPES. The shape ledger answers too: a shape classified against an
idea's reference — in any language — is the project's code saying `adopted`
(an instance) or `variant` (with the shape's reason). The two ledgers never
disagree: a classification moves the row, an assessment or undo the shapes
@@ -77,7 +77,7 @@ logger = logging.getLogger(__name__)
# --- the assessment and the conflict order, as product text --------------------
#
# Judged on every install (rules 115, 119); the tool docstrings quote them.
# Judged on every install by the same text; the tool docstrings quote them.
OUTCOMES = (
{
@@ -241,8 +241,9 @@ def needs_recheck(row_status: str, row_version: int | None, idea_status: str,
def _row_state(row: FamilyAdoption) -> dict:
"""An answer as a decision's before/after. No ids (the #3182 trap): the
owed task is the row's column, not part of the snapshot."""
"""An answer as a decision's before/after. No ids, because an id inside
JSON cannot be remapped by a restore: the owed task is the row's column,
not part of the snapshot."""
return {"status": row.status, "reason": row.reason or "", "canon_version": row.canon_version}
@@ -294,7 +295,7 @@ async def _project_languages(session, project_id: int) -> list[str]:
return [r[0] for r in rows]
# --- the shape ledger as evidence (milestone 463 step 5) ----------------------------
# --- the shape ledger as evidence ---------------------------------------------------
#
# A shape classified against an idea's reference IS an answer to the idea: an
# `instance` — or the reference's own `canonical` location — says the project
@@ -618,7 +619,7 @@ async def adoption_matrix(
select(Note).where(Note.id.in_(task_ids), Note.deleted_at.is_(None))
)).scalars().all()
} if task_ids else {}
# The code that answers each cell (step 5): live shapes classified
# The code that answers each cell: live shapes classified
# against one of the idea's references.
evidence: dict[tuple[int, int], list[dict]] = {}
for iid, shape in (await session.execute(
@@ -1215,7 +1216,7 @@ async def _shapes_follow(user_id: int, project_id: int, idea_id: int, outcome: s
return int(result.get("classified") or 0)
# --- delivery (milestone 463 step 6) -------------------------------------------------
# --- delivery -----------------------------------------------------------------------
#
# The ledger reaches a session three ways: a count on entering a project, the
# ideas themselves by retrieval, and the owed answers filed while a task was
+12 -13
View File
@@ -1,4 +1,4 @@
"""Project inception — what a project was decided to inherit (milestone 297).
"""Project inception — what a project was decided to inherit.
A project's inheritance is a decision, not a default. The record lives on
``projects.inception``::
@@ -14,14 +14,13 @@ A project's inheritance is a decision, not a default. The record lives on
}
NULL = undecided → enter_project asks. ``legacy`` is the migration's stamp on
projects that existed before the step did (inherit-all / no design system /
no seed), so the ask fires only for projects created after this shipped.
projects that predate inception records (inherit-all / no design system /
no seed), so the ask fires only for projects created since.
Rules are not a choice any more. ``exclude_always_on_rulebooks`` went with the
always-on tier (milestone 394), and ``subscribe_rulebooks`` went with
subscriptions (milestone 414): a rule in a rulebook is global and applies to
every project, and a project's own rules are written on it directly. Migration
0101 strips both keys from stored records.
Rules are not a choice: a rule in a rulebook is global and applies to every
project, and a project's own rules are written on it directly. Migration 0101
strips the retired ``exclude_always_on_rulebooks`` and ``subscribe_rulebooks``
keys from stored records.
The shape and its validator are pure; ``decide`` composes the existing
services — set_project_design_system and the standard Systems seed — checks
@@ -31,7 +30,7 @@ half-applied decision is re-runnable rather than recorded as done.
``current_defaults`` is what the enter_project ask shows: what binds today
if nobody decides.
``platforms`` (milestone 463) is which platforms the project IS — the answer
``platforms`` is which platforms the project IS — the answer
that decides which family ideas reach it. A list of catalog SLUGS, not ids:
the record is JSON, and a slug survives a backup restore onto an install whose
ids differ, where an id inside JSON would come back naming another platform.
@@ -54,7 +53,7 @@ CHOICE_KEYS = ("design_system_id", "seed_systems", "platforms")
def validate_inception(choices) -> str | None:
"""The structural error an inception ``choices`` object would earn, or
None. Pure and checked BEFORE any effect is applied: a decision either
applies whole or errors whole (the StrictArgs lesson, #2709).
applies whole or errors whole.
Accepts three keys, each optional: ``design_system_id`` an int or None,
``seed_systems`` a bool, ``platforms`` a list of slugs or None. Unknown
@@ -152,7 +151,7 @@ async def decide(
choices: dict | None,
via: str,
) -> dict:
"""Record a project's inception decision and apply it (milestone 297).
"""Record a project's inception decision and apply it.
Owner-only. Validates the choices (pure) and every target (readable)
first; then, each idempotent: point the project at the design system
@@ -243,8 +242,8 @@ def _platform_line(defaults: dict) -> str:
async def inception_ask(user_id: int, project_id: int) -> dict:
"""The enter_project ask for an undecided project (milestone 297) — the
sibling of the systems-bootstrap ask (#2683): the project's OWN current
"""The enter_project ask for an undecided project — the sibling of the
systems-bootstrap ask: the project's OWN current
defaults, what to ask the operator, and the exact call that answers it.
Fail-open: a hint must never break the call it rides on."""
try:
+22 -23
View File
@@ -19,14 +19,14 @@ OPEN_STEP_STATUSES = ("todo", "in_progress")
def embed_milestone(milestone: Milestone) -> None:
"""Refresh a milestone's vectors, fire-and-forget (milestone 415).
"""Refresh a milestone's vectors, fire-and-forget.
At the service, so every path that writes a milestone gets it — the lesson
embed_note records (#2056): a record written through a door that forgot the
call stays out of search until a restart. Exceptions are swallowed because
At the service, so every path that writes a milestone gets it: a record
written through a door that forgot the call stays out of search until a
restart. Exceptions are swallowed because
a milestone that saved must not fail on its index refresh; no running loop
(a script, a unit test) is ordinary. A delete racing the refresh wins: the
upsert claims the milestone's row first (#3262).
upsert claims the milestone's row first.
"""
try:
import asyncio
@@ -250,12 +250,12 @@ async def get_project_milestone_summaries(
The per-project version below is a nested fan-out: one query to list a
project's milestones, then one more per milestone for its progress. Called
for 25 projects concurrently it asked for ~250 pooled connections against a
pool of 15, and every one of them waited out the 30-second checkout timeout
(#2384). This does the same work in a fixed number of queries and one
for many projects concurrently it asks for far more pooled connections than
the pool holds, and every one of them waits out the checkout timeout. This
does the same work in a fixed number of queries and one
session: the milestones, their step counts, and their open steps.
Each row carries `next_step` — the earliest open step, or None (#4154).
Each row carries `next_step` — the earliest open step, or None.
That is NOT the same question `services/placement.py` answers: placement
knows which step you are on and names the next one AFTER it, while a
listing has no current step, so the earliest open one is the whole answer.
@@ -279,7 +279,7 @@ async def get_project_milestone_summaries(
next_step: dict[int, dict] = {}
if milestones:
milestone_ids = [m.id for m in milestones]
# Both step queries below take the SAME visibility clause (rule 78).
# Both step queries below take the SAME visibility clause.
# They have to: `next_step` names a step and the counts beside it
# say how many there are, so a row that could name a step its own
# progress excludes would be reporting two different milestones.
@@ -304,8 +304,8 @@ async def get_project_milestone_summaries(
step_touched[milestone_id] = latest
# ONE query for every milestone in the batch, not one per row.
# #2384 was exactly this listing fanned out per milestone, and it
# drained the connection pool; a third flat query keeps the cost
# Fanned out per milestone, this listing drains the connection
# pool; a third flat query keeps the cost
# constant in the number of plans. Ordered the way
# services/placement.py orders steps — creation order, the order a
# plan is written and a batch create inserts — so the first row per
@@ -331,14 +331,14 @@ async def get_project_milestone_summaries(
entry.update(_progress_from_counts(counts.get(m.id, {})))
# A milestone's own updated_at doesn't move when its steps do, so a
# plan whose steps closed today would read as untouched since it was
# written (#4045). Touched is the later of the two.
# written. Touched is the later of the two.
touched = [t for t in (m.updated_at, step_touched.get(m.id)) if t]
entry["last_touched_at"] = max(touched).isoformat() if touched else None
# Always present, None included. "8 of 9 done" tells a reader there is
# an open step and not WHICH, and a gap that shape gets filled from
# whatever id is nearest to hand — a retrieval hint carries an id and a
# title and no status, and that is how a done step was reported as the
# open one (#4154). A listing that names it leaves nothing to guess.
# title and no status, so a done step can be reported as the open one.
# A listing that names it leaves nothing to guess.
entry["next_step"] = next_step.get(m.id)
out.setdefault(m.project_id, []).append(entry)
return out
@@ -352,11 +352,10 @@ async def get_project_milestone_summary(user_id: int, project_id: int) -> list[d
# What a milestone LISTING needs: enough to say what each plan is, how far
# along it is, and which step is next. The plan itself (`body`) is
# get_milestone's job. Summaries once carried it, and on a project with 39
# milestones enter_project came to ~222k characters, 168k of them plan bodies. That is past what an MCP client will
# accept as a tool result, so the session handshake arrived as a file to page
# through (#4045). user_id / project_id / timestamps repeat what the caller
# already knows.
# get_milestone's job: on a project with a few dozen milestones, plan bodies
# would push enter_project past what an MCP client accepts as a tool result,
# and the session handshake would arrive as a file to page through.
# user_id / project_id / timestamps repeat what the caller already knows.
_BRIEF_FIELDS = (
"id", "title", "description", "status", "order_index",
"total", "completed", "pct", "status_counts", "next_step",
@@ -391,9 +390,9 @@ def unplanned_milestones(
A plan written as a milestone with a description and no steps is open work
that nothing else names. It is never "touched" — touching is a step
changing — so the recency list that brief_milestone_summary(limit=) builds
can never reach it, and progress reads 0% either way. A project whose
roadmap was written that way ended up with every later plan opened as a
new milestone beside the one that already described it (milestone 415).
can never reach it, and progress reads 0% either way. Unlisted, a roadmap
written that way gets every later plan opened as a new milestone beside the
one that already described it.
`exclude_ids` drops milestones a caller already listed. Kept in roadmap
order (order_index, then creation), the order they were written in.
+8 -8
View File
@@ -1,4 +1,4 @@
"""Delivering the rules mounted on a moment, through every door (milestone 458 step 4).
"""Delivering the rules mounted on a moment, through every door.
One function decides what an act reaches and what arrives with it; the doors
differ only in how they carry the answer:
@@ -45,7 +45,7 @@ async def reachable_tools(user_id: int) -> list[str]:
"""
from scribe.services import rulebooks
# A moment that carries a default reply shape (milestone 500) is worth a
# A moment that carries a default reply shape is worth a
# request whether or not anything is mounted on it: the shape is product,
# so every install has it.
shaped = {s.moment for s in reply_shapes.SHAPES.values() if s.key != reply_shapes.CORE_KEY}
@@ -91,7 +91,7 @@ async def deliver_for_act(
def shapes_for_act(reached: list[dict], seen: frozenset[str] = frozenset()) -> tuple[list[str], dict[str, str]]:
"""The default reply shapes riding the moments this act reached (milestone 500).
"""The default reply shapes riding the moments this act reached.
Closing a task comes just before a completion report, a structured
question is an ask, opening a plan comes before a plan is put up for
@@ -114,7 +114,7 @@ async def deliver_for_turn(
user_id: int, *, seen: frozenset[str] = frozenset(), project_id: int | None = None,
exclude: frozenset[int] = frozenset(), held: frozenset[int] = frozenset(),
) -> dict:
"""What every turn carries before its reply is written (milestone 500 step 3).
"""What every turn carries before its reply is written.
The core reply shape — in full once per session and again after a
compaction (the ledger is swept then), otherwise its one-line reminder —
@@ -147,7 +147,7 @@ async def attach_moment_rules(
) -> dict:
"""Carry the rules mounted on this tool call's moments in its own response.
The MCP door (#4286's attach shape): an in-band decoration on a payload
The MCP door: an in-band decoration on a payload
already being returned, fail-open, and absent when there is nothing to
say. The project is the record's own when it has one, else the caller's
argument — a project's mounted rules belong to work in that project.
@@ -184,7 +184,7 @@ async def attach_moment_rules(
# ── The reply moment: the backstop at the end of a turn ─────────────────
#
# The Stop hook's door, folded in from milestone 456 step 8. A reply is the
# The Stop hook's door. A reply is the
# one act no tool call marks, and the moment where "please verify this on
# your end" gets said — so it is checked twice over: the rules MOUNTED on the
# reply moments (deterministic, somebody said they belong here), and the
@@ -229,7 +229,7 @@ def _reply_query(reply: str) -> str:
def reply_hold_reason(held: list[dict]) -> str:
"""The text the agent reads INSTEAD of its reply going out.
A practice, not a prohibition (rule 165), on the act checkpoint's model:
A practice, not a prohibition, on the act checkpoint's model:
nothing here knows the reply is wrong. It names each rule with why it was
raised, gives the remedy as one call per rule, and says the reply may go
out unchanged — so a reader who finds the rules beside the point is out in
@@ -245,7 +245,7 @@ def reply_hold_reason(held: list[dict]) -> str:
parts.append(f"“{item['title']}” ({why}{trigger}) — get_rule({item['rule_id']})")
noun = "a standing rule" if len(held) == 1 else "standing rules"
# A mount that does not bear on this reply is a misfire worth one call:
# reports gather into an unmount proposal for the operator (step 7b).
# reports gather into an unmount proposal for the operator.
mounted = [item for item in held if item.get("moment")]
misfire = (
" If one mounted on a moment has nothing to do with this reply, "
+24 -31
View File
@@ -15,13 +15,12 @@ Design notes (mirrors retrieval_telemetry, for the same reasons):
- Writes are fire-and-forget. `record_surfaced` / `record_pulled` extract
plain ints synchronously and schedule the insert as a background task, so
telemetry never adds latency to — or can break — the surface it observes.
- Failures degrade, but they must not degrade SILENTLY. The original version
swallowed everything into logger.debug, and the deployed instance ran with
every counter at zero for weeks while surfacing demonstrably fired — a
total outage indistinguishable from "nobody uses this" (#2663). A
subsystem whose every failure mode is invisible cannot report its own
death, so failures now log at WARNING and drop one AppLog error row per
process per site, where the admin UI shows it.
- Failures degrade, but they must not degrade SILENTLY. Swallowed into a
debug log, a failure leaves every counter at zero while surfacing fires —
a total outage indistinguishable from "nobody uses this". A subsystem
whose every failure mode is invisible cannot report its own death, so
failures log at WARNING and drop one AppLog error row per process per
site, where the admin UI shows it.
- Reads (`usage_for_notes`) are NOT fire-and-forget — a readout the caller
awaits, aggregated in one round-trip for a whole page of snippets rather
than per row.
@@ -151,7 +150,7 @@ def record_pulled(
`project_id` is where the READER was — the caller's active project, never
the record's own. Compared against the record's `project_id`, it answers
whether this was opened somewhere other than where it was written, which
is the evidence that a record TRANSFERRED (milestone 385, #3735).
is the evidence that a record TRANSFERRED.
Unlike a surfacing arm, a getter only knows what it was handed, so this
stays optional and null is an ordinary answer meaning "not reported". A
@@ -178,7 +177,7 @@ def record_pulled(
# top-N-by-recency happen to be; the skill sync installs every Process the
# operator can reach. Counting those alongside auto-inject would make a note's
# surfaced_count dominated by "it was recently updated in a project you
# opened", and dead-weight detection would read that as popularity (#2477).
# opened", and dead-weight detection would read that as popularity.
# They still matter — a pull that follows one must not float unattributed — so
# they land in their own bucket rather than not landing at all.
AMBIENT_SOURCES = ("enter_project", "process_skill_sync")
@@ -191,14 +190,13 @@ def empty_usage() -> dict:
reads as "never surfaced, never pulled" rather than as a missing key.
`surfaced_count` is RANKED surfacings only — a scored surface chose this
record. `ambient_count` is the rest (see AMBIENT_SOURCES). The split is the
readout half of #2477: the "high surfaced, zero pulls → dead weight"
reading is only valid over surfacings that were choices.
record. `ambient_count` is the rest (see AMBIENT_SOURCES). The split exists
because the "high surfaced, zero pulls → dead weight" reading is only valid over surfacings that were choices.
`surfaced_away_count` / `pulled_away_count` are the subsets that happened
on a project other than the one the record was written on — the evidence
that a record TRANSFERRED, which is the claim the lesson kind rests on
(milestone 385, #3735). Counted only where both projects are known: an
that a record TRANSFERRED, which is the claim the lesson kind rests on.
Counted only where both projects are known: an
event with no reader project, or a record with no project of its own,
cannot speak to "away" and is left out rather than guessed.
"""
@@ -231,9 +229,8 @@ async def usage_for_notes(note_ids: list[int]) -> dict[int, dict]:
# distinct source. ONE labelled expression, grouped by its label — a second
# case() instance in GROUP BY renders with its own expanding-IN bind names
# under asyncpg, so the database sees two DIFFERENT expressions and rejects
# the query with a GroupingError. That rejection was swallowed, which is
# how every counter read zero in production while the writes were landing
# fine (#2663).
# the query with a GroupingError — and a swallowed rejection reads as
# every counter at zero while the writes land fine.
ambient = case(
(NoteUsageEvent.source.in_(AMBIENT_SOURCES), True),
else_=False,
@@ -262,7 +259,7 @@ async def usage_for_notes(note_ids: list[int]) -> dict[int, dict]:
# it needs the join to `notes` and the first must keep counting
# events whose note has since been deleted (the table is FK-free
# so that evidence outlives the row). Ranked surfacings only, for
# the same reason surfaced_count is (#2477).
# the same reason surfaced_count is.
away_rows = (
await session.execute(
select(
@@ -284,7 +281,7 @@ async def usage_for_notes(note_ids: list[int]) -> dict[int, dict]:
except Exception:
# A telemetry readout must not be able to break the list it decorates —
# but it must say it failed, or a broken readout is indistinguishable
# from a corpus nobody uses (#2663).
# from a corpus nobody uses.
await _report_failure("readout")
return out
@@ -335,15 +332,12 @@ def _row_id(row: dict, key: str) -> int | None:
async def attach_usage(rows: Sequence[dict], *, key: str = "id") -> None:
"""Add `usage` to every row of a payload a door is about to return (#4230).
"""Add `usage` to every row of a payload a door is about to return.
The one seam both doors and every record kind share. Before this, four call
sites carried their own copy of the same lines — two list routes and two
detail routes — and `/api/knowledge`, which is the list a person ACTUALLY
browses notes and lessons in, was about to become a fifth. That is how the
chip came to reach two record kinds out of four while a service named
`usage_for_notes` worked on all of them: each door read fine on its own,
and nobody was comparing them.
The one seam both doors and every record kind share — list routes, detail
routes and `/api/knowledge`, the list a person actually browses notes and
lessons in. A copy per call site lets the chip reach some record kinds and
not others while each door reads fine on its own.
ONE AGGREGATE FOR THE WHOLE PAGE. `usage_for_notes` is a single GROUP BY
over the id set; calling it per row would be N+1 by construction, which is
@@ -360,11 +354,10 @@ async def attach_usage(rows: Sequence[dict], *, key: str = "id") -> None:
reports it through `_report_failure("readout")` and returns the zero-filled
map, so a broken readout degrades without breaking the list it decorates.
Wrapping it again would swallow the REPORT along with the error, and a
silently-swallowed readout failure is exactly #2663 — every counter reading
zero in production for weeks while the writes landed fine.
silently-swallowed readout failure reads as every counter at zero while
the writes land fine.
Mutates in place and returns None, matching how the call sites already used
it: these rows are the payload, not a copy of it.
Mutates in place and returns None: these rows are the payload, not a copy of it.
A detail payload is just a one-row list — `await attach_usage([data])` —
so the single-record doors share this seam rather than keeping a second
+8 -8
View File
@@ -1,6 +1,6 @@
"""Record references written into prose: refusing guessed ids, resolving placeholders, reading the ids a message names.
THE DEFECT THIS EXISTS FOR (#4016)
THE DEFECT THIS EXISTS FOR
A session about to create several records guesses the ids they will get — the
last id it saw, plus one, plus two — and writes those guesses into a plan body
@@ -17,8 +17,8 @@ Two halves, and they are not alternatives:
rewrites the placeholders inside ONE transaction, so there is no moment at
which an id has to be predicted.
- REFUSING A GUESSED ID is the backstop for a session that guesses anyway. It
is a refusal, not a warning, because a warning is an instruction and the
operator's point was exactly that instructions are not always followed.
is a refusal, not a warning, because a warning is an instruction and
instructions are not always followed.
HOW A GUESS IS RECOGNISED
@@ -131,12 +131,12 @@ def resolve_placeholders(text: str | None, refs: dict[str, str]) -> str | None:
return PLACEHOLDER_RE.sub(lambda m: refs[m.group(1)], text)
# ── Records an operator names in a message (#4796) ─────────────────────────
# ── Records an operator names in a message ─────────────────────────────────
#
# "yes go ahead with 4448" says exactly which record is meant, and the prompt
# menu could not use it: a number carries no meaning to an embedding, so the
# menu filled with whatever resembled the words AROUND it. These ids are looked
# up directly instead (plugin_context.build_autoinject_hint).
# "yes go ahead with 4448" says exactly which record is meant, and a ranked
# menu cannot use it: a number carries no meaning to an embedding, so the menu
# fills with whatever resembles the words AROUND it. These ids are looked up
# directly instead (plugin_context.build_autoinject_hint).
#
# A guard against a false READ, not a false lookup. A number that is not a
# record id still has to resolve to a record this user can open before anything
+24 -48
View File
@@ -1,34 +1,19 @@
"""One registry of the retrieval surfaces, and the two numbers each one has (#4102).
"""One registry of the retrieval surfaces, and the two numbers each one has.
WHY THIS EXISTS
Six push arms each carried their own loose copy of the same shape: a settings
key, a default, and a limit that was usually a module constant nobody could
change. The read-and-clamp was written out separately in `plugin_context` (twice
over, for auto-inject and the write path), in three rule arms, and again in
the completion-report arm (since retired, milestone 500). That was survivable
while the numbers were shipped constants an operator occasionally edited.
Every push arm has the same shape: a settings key, a default, and a limit. A
floor is chosen and adjusted by the model that uses it, with the operator able
to touch it — and a tuning surface cannot be consistent for the model if every
arm spells its own configuration differently. So the arms do not own their
numbers; they read them from here, and the tuning tool, the routes and the
Settings UI all enumerate THIS table rather than hard-coding special cases.
It stops being survivable once the numbers are meant to MOVE. The operator's
decision for this step:
"the floor should be chosen and adjusted by the model using it. we've come
back to something either fails or has to be looked at by the user we need a
model consistent surface for the adjustment of these floor values. the user
should be able to touch it but the model should be the thing handling it 9
times out of 10."
A tuning surface cannot be "model consistent" if every arm spells its own
configuration differently. So the arms stop owning their numbers and read them
from here instead, and the tuning tool, the routes and the Settings UI all
enumerate THIS table rather than hard-coding six special cases.
WHAT A FLOOR IS NOW, AND WHAT IT IS NOT
WHAT A FLOOR IS, AND WHAT IT IS NOT
It is not a relevance judgement. Relevance is decided by the reader, which is
the only participant that can read a trigger against a situation — that is the
milestone's whole argument, and the injected line has always said so out loud
("read it before deciding it does not apply").
the only participant that can read a trigger against a situation — and the
injected line says so out loud ("read it before deciding it does not apply").
A floor answers the cheaper question: *is this worth ranking at all*. `k` is
what binds, and `k` is a BUDGET — how much of this surface's attention a
@@ -38,29 +23,21 @@ afford what an arm that fires once a turn can.
WHY THE DEFAULTS BELOW ARE STARTING POINTS AND NOT ANSWERS
A cosine score is a distance in BAAI/bge-small-en-v1.5's vector space, measured
against THIS corpus. It cannot transfer to an install with different records,
and rule 115 forbids defending a shipped default from this instance's
telemetry — which is what made the old design unbuildable: every number was a
guess everywhere except here, and there was no mechanism that could ever
improve it.
The mechanism is the fix. Scribe ships a starting point and the means to
correct it, so the values below carry the measurement that motivated them (see
the long comments in `plugin_context.py`, which are kept where they are because
they record how each number was first arrived at) without claiming to be right
for anybody else.
against one corpus. It cannot transfer to an install with different records,
and a shipped default cannot be defended from any one install's telemetry.
Scribe ships a starting point and the means to correct it, so the values carry
the measurement that motivated them without claiming to be right for anybody
else.
HOW A FLOOR SHOULD ACTUALLY BE MOVED
By reading the records the floor refused — `retrieval_telemetry(
near_miss_samples=N)` returns them by id — and never by the percentile alone.
That is not a style preference; it is the one case where the two disagreed and
was checked. `report_preference` logged 69 consecutive declines with the
refused score 0.0006 under the bar, and every percentile said "lower it".
Reading the refused record showed it was rule 77 "Extract intent from loose
phrasing", a false positive, so lowering the bar would have delivered that rule
on every completion report ever written. The statistic and the correct action
pointed in opposite directions, and only opening the record could tell.
An arm can log dozens of consecutive declines with the refused score a hair
under the bar while every percentile says "lower it", and the refused record,
once read, turns out to be a false positive that a lower bar would deliver on
every matching act. The statistic and the correct action can point in opposite
directions, and only opening the record tells them apart.
"""
from __future__ import annotations
@@ -76,11 +53,10 @@ from scribe.services.retrieval_pipeline import ( # noqa: F401 - Surface re-expo
MAX_BUDGET = 10
# THE TABLE IS READ OFF THE SPECS (milestone 456 step 5). Each ranked arm in
# THE TABLE IS READ OFF THE SPECS. Each ranked arm in
# `retrieval_pipeline` carries its own `Surface` — the floor and budget pair,
# its settings keys and defaults, and the prose a tuner reads — so an arm added
# there is tunable by construction, and nothing here can drift from it. The
# keys and defaults are the ones this table held before the move, unchanged.
# there is tunable by construction, and nothing here can drift from it.
SURFACES: dict[str, Surface] = {
arm.tuning.name: arm.tuning for arm in TUNED_ARMS if arm.tuning is not None
}
@@ -88,10 +64,10 @@ SURFACES: dict[str, Surface] = {
# Reserved slots are deliberately absent. `preference_slot`, `reuse_slot` and
# `lesson_slot` borrow their parent arm's floor and are hard-limited to one hit
# each, because their entire purpose is to guarantee a single line to a kind of
# record that keeps losing a general score contest (#2246, #3894) — or, for
# record that keeps losing a general score contest — or, for
# `lesson_slot`, to a kind whose loss is total rather than merely a lost
# convenience, since a lesson has no act arm to fall back on and nobody browses
# lessons looking for one (milestone 385). A budget of "1" is the feature;
# lessons looking for one. A budget of "1" is the feature;
# exposing it as tunable would invite setting it to 0 and silently removing the
# guarantee.
#
+14 -15
View File
@@ -114,11 +114,11 @@ def matching_patterns(patterns, path: str) -> list[str]:
async def assess_system_name(user_id: int, project_id: int, name: str) -> dict:
"""What BOTH doors must know before minting a System name (milestone 307).
"""What BOTH doors must know before minting a System name.
Lived in the MCP tool alone until now, which is how the web UI shipped
without a gate the agent surface enforced (#2482). One service function, so
the two doors cannot answer the same question differently (rule 33).
One service function, so the MCP tool and the web UI cannot answer the
same question differently — a gate living in one door is a gate the other
ships without.
Returns `{"duplicate": …|None, "canonical": …|None}`:
@@ -162,21 +162,20 @@ async def assess_system_name(user_id: int, project_id: int, name: str) -> dict:
async def standard_systems() -> list[tuple[str, str]]:
"""The standard cross-project vocabulary (#2798) as (name, charter) pairs.
"""The standard cross-project vocabulary as (name, charter) pairs.
Reads the GLOBAL canonical catalog (milestone 307). This was a tuple
constant in this module until the catalog became a table: a constant
cannot be a foreign key, so nothing outside a project could reference an
area, and the list only ever applied on the inception-seed path — which is
how three spellings of "CI & Release" reached one instance anyway.
Reads the GLOBAL canonical catalog. The catalog is a table rather than a
constant because a constant cannot be a foreign key: projects map their
own Systems onto a catalog entry, so variant spellings of one area still
resolve to it.
"""
return [(entry.name, entry.description or "") for entry in
await canonical_systems_svc.list_canonical_systems()]
async def seed_standard_systems(user_id: int, project_id: int) -> list[System]:
"""Mint the standard starter set for a project that has NO Systems yet
(milestone 297). Idempotent: a project with any System — the vocabulary
"""Mint the standard starter set for a project that has NO Systems yet.
Idempotent: a project with any System — the vocabulary
already started, standard or not — gets nothing; the duplicate gate and
the project's own judgment take it from there. [] without write access.
@@ -197,9 +196,9 @@ async def seed_standard_systems(user_id: int, project_id: int) -> list[System]:
def embed_system(system) -> None:
"""Refresh a System's charter vectors, fire-and-forget (#4251).
"""Refresh a System's charter vectors, fire-and-forget.
The twin of `notes.embed_note`, and here for the same reason (#2056): at
The twin of `notes.embed_note`, and here for the same reason: at
the service, so every door gets it by construction rather than each route
and tool remembering. A charter edited through one door and not another
would stay findable by what it used to say, and nothing would report it.
@@ -430,7 +429,7 @@ async def system_names_for(note_ids: set[int]) -> dict[int, list[str]]:
For decorating a result set the caller was allowed to see — an injected
menu line says which part of the project a record is about, so the reader
can place it without opening it (#4364). No access check here for that
can place it without opening it. No access check here for that
reason: the ids come from a search that applied one, and a system name is
metadata of the record, not a record of its own.