fix(dedup): compare the artefact, not the prose describing it
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 48s
CI & Build / Build & push image (push) Successful in 25s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 48s
CI & Build / Build & push image (push) Successful in 25s
The snippet gate was reading the wrong field, and #2464's UI recipes made it measurable in both directions at once: .btn-danger vs .btn-danger-outline 0.92 siblings, BLOCKED .btn-primary re-recorded verbatim under a different name <0.90 a literal copy, ALLOWED The second is what settles it. Identical code at an identical repo·path·symbol sailed through because the description differed, while two deliberately parallel variants were refused because theirs did not. A snippet's embedded document is mostly prose ABOUT the code, so no threshold fixes this: lowering it blocks more siblings, raising it admits more copies. So structure decides. Two exact signals, both index-served off the notes.data mirror that already exists, no migration and no backfill: location the same named thing in the same file. Requires BOTH path and symbol — a path alone is a directory of artefacts, and matching on it would refuse every second recipe from one stylesheet. code byte-identical code anywhere, via the same fingerprint the drift check uses. The semantic arm survives as a backstop for a genuine reword that shares neither, raised to 0.96 so it sits above the 0.92 band where real variants live. Structural hits say what they matched instead of hedging with "similar", and point at merge rather than update — two records of one artefact is what merge exists to fold back together. find_duplicate_snippets gets the same correction: pairs where both snippets name a symbol, name DIFFERENT symbols, and hold different code are variants, not copies. Without it a design system's button family reports as one merge set — eight recipes, every direct pair over the floor, top score 0.92, one click from collapsing a component family. The cost is real and stated in the code: a helper recorded twice under two names no longer reports. That trade favours the report being usable, and same-symbol and unnamed duplicates — how re-recording usually looks — still surface. The filter fails open, so a lookup failure degrades to the old unfiltered report rather than to a reassuring empty one. resolve_locations extracted: compose_body, create_snippet and now the gate each had their own copy of the repo/path/symbol shorthand fallback, and the gate is the one where a disagreement would mean matching a location the record won't be stored with. Applied to both create surfaces (#33) — the web UI must not be the way to record what the agent was stopped from writing. Refs #2518, #2464
This commit is contained in:
@@ -110,3 +110,175 @@ def test_duplicate_response_shape():
|
||||
assert r["match"] == "title"
|
||||
assert "force=true" in r["message"]
|
||||
assert "update_task" in r["message"]
|
||||
|
||||
|
||||
# --- snippet structural identity (#2518) -------------------------------------
|
||||
#
|
||||
# The gate used to compare a snippet's rendered DOCUMENT, which is mostly prose
|
||||
# about the code. Measured on the button corpus, that failed in both directions
|
||||
# at once: two deliberately-parallel variants were refused at 0.92, while a
|
||||
# verbatim re-record of one snippet under a different name scored below 0.90 and
|
||||
# was created. These tests pin the structural signals that replaced it.
|
||||
|
||||
|
||||
def _session_sequence(results):
|
||||
"""A mocked async_session() whose successive execute() calls yield `results`.
|
||||
|
||||
The single-result helper above can't express this: the structural check runs
|
||||
a location query and then a code query, and the whole point is that they
|
||||
answer differently.
|
||||
"""
|
||||
s = AsyncMock()
|
||||
s.__aenter__ = AsyncMock(return_value=s)
|
||||
s.__aexit__ = AsyncMock(return_value=False)
|
||||
wrapped = []
|
||||
for note in results:
|
||||
r = MagicMock()
|
||||
r.scalars.return_value.first.return_value = note
|
||||
wrapped.append(r)
|
||||
s.execute = AsyncMock(side_effect=wrapped)
|
||||
return s
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_same_location_is_a_duplicate_however_it_is_described():
|
||||
"""The measured false NEGATIVE: identical code at an identical
|
||||
repo·path·symbol was created because the prose around it differed."""
|
||||
existing = _fake_note(id=30, title=".btn-primary — a page's main action",
|
||||
note_type="snippet")
|
||||
sem = AsyncMock()
|
||||
with patch("scribe.services.dedup.async_session",
|
||||
return_value=_session_sequence([None, existing])), \
|
||||
patch("scribe.services.dedup.embeddings_svc.semantic_search_notes", sem):
|
||||
dup = await find_duplicate_note(
|
||||
7, "primaryButton — something else entirely", body="x" * 400,
|
||||
project_id=2, is_task=False, note_type="snippet",
|
||||
code=".btn-primary { color: red; }",
|
||||
locations=[{"repo": "Scribe", "path": "a/b.css", "symbol": ".btn-primary"}],
|
||||
)
|
||||
assert dup is not None
|
||||
assert dup.reason == "location"
|
||||
assert dup.similarity == 1.0
|
||||
# Structural identity is certain, so it must not be diluted by asking the
|
||||
# embedder for a second opinion.
|
||||
sem.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_identical_code_is_a_duplicate_at_a_different_location():
|
||||
existing = _fake_note(id=31, title="group_pairs", note_type="snippet")
|
||||
with patch("scribe.services.dedup.async_session",
|
||||
return_value=_session_sequence([None, None, existing])), \
|
||||
patch("scribe.services.dedup.embeddings_svc.semantic_search_notes",
|
||||
AsyncMock(return_value=[])):
|
||||
dup = await find_duplicate_note(
|
||||
7, "unionFind", body="x" * 400, project_id=2, is_task=False,
|
||||
note_type="snippet", code="def f():\n return 1",
|
||||
locations=[{"repo": "Scribe", "path": "z.py", "symbol": "f"}],
|
||||
)
|
||||
assert dup is not None
|
||||
assert dup.reason == "code"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_location_without_a_symbol_is_not_an_identity():
|
||||
"""A path alone is a DIRECTORY of artefacts. Matching on it would refuse
|
||||
every second snippet recorded from one file — which is exactly the corpus
|
||||
the button recipes form."""
|
||||
session = _session_sequence([None])
|
||||
sem = AsyncMock(return_value=[])
|
||||
with patch("scribe.services.dedup.async_session", return_value=session), \
|
||||
patch("scribe.services.dedup.embeddings_svc.semantic_search_notes", sem):
|
||||
dup = await find_duplicate_note(
|
||||
7, "Some recipe", body="x" * 400, project_id=2, is_task=False,
|
||||
note_type="snippet", code="",
|
||||
locations=[{"repo": "Scribe", "path": "a/b.css", "symbol": ""}],
|
||||
)
|
||||
assert dup is None
|
||||
# Exactly one query — the title check. With no symbol and no code there is
|
||||
# nothing to match structurally, and the sequence above would raise
|
||||
# StopIteration if a second query were issued.
|
||||
assert session.execute.await_count == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_snippets_use_the_raised_semantic_threshold():
|
||||
"""Variants of one component legitimately reach 0.92. The semantic arm has
|
||||
to sit above that band or it refuses the corpus it exists to protect."""
|
||||
from scribe.services.dedup import (
|
||||
_SEMANTIC_THRESHOLD,
|
||||
_SNIPPET_SEMANTIC_THRESHOLD,
|
||||
)
|
||||
sem = AsyncMock(return_value=[])
|
||||
with patch("scribe.services.dedup.async_session",
|
||||
return_value=_session_sequence([None, None, None])), \
|
||||
patch("scribe.services.dedup.embeddings_svc.semantic_search_notes", sem):
|
||||
await find_duplicate_note(
|
||||
7, "A recipe", body="x" * 400, project_id=2, is_task=False,
|
||||
note_type="snippet", code="x",
|
||||
locations=[{"repo": "R", "path": "p", "symbol": "s"}],
|
||||
)
|
||||
assert sem.await_args.kwargs["threshold"] == _SNIPPET_SEMANTIC_THRESHOLD
|
||||
assert _SNIPPET_SEMANTIC_THRESHOLD > 0.92, (
|
||||
"the observed sibling band tops out at 0.92 (.btn-danger vs "
|
||||
".btn-danger-outline); a threshold at or below it blocks legitimate "
|
||||
"variants again"
|
||||
)
|
||||
assert _SNIPPET_SEMANTIC_THRESHOLD > _SEMANTIC_THRESHOLD
|
||||
|
||||
|
||||
def test_sibling_variants_are_not_reported_as_merge_candidates():
|
||||
"""The measured false POSITIVE: eight button recipes, every direct pair over
|
||||
the floor, proposed as ONE merge set."""
|
||||
from scribe.services.dedup import _drop_sibling_pairs
|
||||
|
||||
records = {
|
||||
1: {"locations": [{"repo": "S", "path": "c.css", "symbol": ".btn-primary"}],
|
||||
"code_sha": "aaa"},
|
||||
2: {"locations": [{"repo": "S", "path": "c.css", "symbol": ".btn-secondary"}],
|
||||
"code_sha": "bbb"},
|
||||
}
|
||||
assert _drop_sibling_pairs([(1, 2, 0.87)], records) == []
|
||||
|
||||
|
||||
def test_identical_code_still_reports_even_with_different_symbols():
|
||||
"""The filter keys on "the author named these apart", but a shared code
|
||||
fingerprint overrides that — the same code under two names IS the
|
||||
copy-paste the report exists to surface."""
|
||||
from scribe.services.dedup import _drop_sibling_pairs
|
||||
|
||||
records = {
|
||||
1: {"locations": [{"repo": "S", "path": "a.py", "symbol": "debounce"}],
|
||||
"code_sha": "same"},
|
||||
2: {"locations": [{"repo": "S", "path": "b.py", "symbol": "useDebounced"}],
|
||||
"code_sha": "same"},
|
||||
}
|
||||
assert _drop_sibling_pairs([(1, 2, 0.9)], records) == [(1, 2, 0.9)]
|
||||
|
||||
|
||||
def test_unnamed_snippets_still_report():
|
||||
"""A snippet with no recorded symbol made no identity claim, so the filter
|
||||
must not protect it — re-recording without a location is a common way to
|
||||
duplicate."""
|
||||
from scribe.services.dedup import _drop_sibling_pairs
|
||||
|
||||
records = {1: {"code_sha": "aaa"}, 2: {"code_sha": "bbb"}}
|
||||
assert _drop_sibling_pairs([(1, 2, 0.9)], records) == [(1, 2, 0.9)]
|
||||
|
||||
|
||||
def test_duplicate_response_names_what_matched_for_structural_hits():
|
||||
"""A structural hit is certain, so the message must not hedge with
|
||||
"similar" — and it points at merge, which is what two records of one
|
||||
artefact actually need."""
|
||||
r = duplicate_response(
|
||||
DuplicateMatch(id=9, title=".btn-primary", similarity=1.0, reason="location"),
|
||||
"snippet",
|
||||
)
|
||||
assert "repo · path · symbol" in r["message"]
|
||||
assert "merge_snippets" in r["message"]
|
||||
assert "similar" not in r["message"]
|
||||
|
||||
r = duplicate_response(
|
||||
DuplicateMatch(id=9, title="x", similarity=1.0, reason="code"), "snippet",
|
||||
)
|
||||
assert "identical code" in r["message"]
|
||||
|
||||
Reference in New Issue
Block a user