Commit Graph
2 Commits
Author SHA1 Message Date
bvandeusenandClaude Fable 5 bbee0d0db1 refactor(tests): one definition each for the copied fixtures and fakes (#2825, milestone 296 area 1)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 24s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 54s
CI & Build / Build & push image (push) Successful in 18s
The shape ledger showed the same test scaffolding defined over and over:
_bind_user x12 (byte-identical), _dispose_engine x10 in three wordings,
_no_supersession x3, _make_mock_session x7 in three subsets, a get-or-create
User helper x2 (+3 inlined), and fifteen hand-rolled MagicMock note factories
each re-explaining the same "an auto-MagicMock attribute is truthy" hazard
(note 2109).

Now: conftest.py carries _bind_user / _dispose_engine / _no_supersession as
opt-in fixtures (pytestmark = usefixtures(...) per module, so unit tests pay
nothing), and tests/helpers.py carries make_mock_session(), ensure_user() and
fake_note(**attrs) — the hazard documented once, real values on every
attribute the product reads. Call sites were rewritten by AST so titles with
dashes and commas survived; the three SimpleNamespace _note stand-ins that
only feed a single function stay local.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 11:03:48 -04:00
bvandeusen f20c019f2a feat(supersession): demote what a later note overtook, and label it
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / TypeScript typecheck (push) Successful in 10s
CI & Build / integration (push) Successful in 18s
CI & Build / Python tests (push) Failing after 30s
CI & Build / Build & push image (push) Skipped
Step 3 of #278. First step with visible effect.

## Where the demotion happens, and why not in SQL

Applied AFTER the pgvector fetch, over an over-fetched candidate set, not as
part of the ORDER BY.

Ordering by `distance + penalty` would be exact and would turn an indexed
top-k into a scan-and-sort of every embedded note — the HNSW index from
migration 0067 can only serve a raw-distance ordering. So the query fetches
3x the requested rows by raw distance and the re-rank happens in Python.

Demoting after a LIMIT k with no over-fetch would have been theatre: the cut
already happened, so a superseded record pushed down still sits in the results
and the live record that should have replaced it was never fetched.

The cost is stated in the code: a live record outside the over-fetch window
cannot be promoted in. With a 0.05 penalty against neighbours ~0.014 apart,
that needs the true answer more than three ranks down, which no observed query
approaches.

## Demote, never hide — enforced in three places

The penalty applies to the RANKING score, not to the relevance threshold. The
floor decides whether a record is relevant at all; the penalty decides which
relevant record comes first. Applying it to the floor would drop a superseded
record out of the results entirely, which is the one thing this must not do.

It is small on purpose. Supersession is a claim about SOME of a record's
content, so one that strongly answers a question nothing else answers still
surfaces — just behind anything comparable that is current. Its test asserts
both bounds, the upper one citing the operator's constraint rather than an
optimisation.

And every test here that could be satisfied by dropping a record instead
asserts the record is still present.

## The dedup gate opts out

A superseded record is still a duplicate of what you are about to write — the
claim is that it is no longer current, not that it is gone. Demoting it there
would let the same note be recorded a second time, and the second copy would be
the one nothing warns about.

## The label

Auto-inject marks a superseded line SUPERSEDED with a pointer to check the
later record. One query for the whole menu. An agent handed stale material with
nothing marking it acts on it with full confidence, which is worse than never
having surfaced it — the ranking is only half the fix.

Fails open: a supersession lookup error returns unpenalised results rather than
none, because unpenalised ranking is the behaviour that shipped for months and
a broken search is not.

Refs #278
2026-08-08 02:00:06 -04:00