fix(embeddings): the index refresh loses the race it used to deadlock (#3262)
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / integration (push) Successful in 38s
CI & Build / Python tests (push) Failing after 55s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / integration (push) Successful in 38s
CI & Build / Python tests (push) Failing after 55s
CI & Build / Build & push image (push) Skipped
An embedding refresh replaces a record's vectors as delete-then-insert, which takes the chunk rows first and the parent row second (via the insert's foreign key). A cascading delete of the parent takes exactly those two locks in the other order. Postgres calls the cycle a deadlock and kills one side: sometimes the detached embedder, silently, and sometimes the user's delete, as a 500 on an operation that should have worked. Both upserts now claim the parent row with FOR KEY SHARE NOWAIT before touching any chunk row. That removes the cycle instead of narrowing it — either the embedder is first and the delete queues behind it, or the delete already holds the row and the embedder loses at once, which is the side designed to lose. FOR KEY SHARE is the lock the insert would take anyway, so an ordinary edit is unaffected. The note twin, recorded as unverified on the issue, has the same shape and the same fix; a trash purge is the hard delete that reaches it. Unit tests pin the ORDER and the lock mode by compiling the statement; the integration pair holds a real delete open in one transaction and proves the embedder returns having written nothing, with a deadline so a regression fails instead of hanging.
This commit is contained in:
@@ -379,6 +379,10 @@ def _refresh_rule_embedding(rule: Rule) -> None:
|
||||
swallowed because a rule that SAVED must not fail on its index refresh —
|
||||
a stale vector costs a missed search hit, a raised exception costs the
|
||||
write. No running loop (unit tests, scripts) is ordinary, not an error.
|
||||
|
||||
Detaching also means this task races anything that deletes the rule out
|
||||
from under it. That is not handled here: `upsert_rule_embedding` claims
|
||||
the rule's row before touching its vectors, and loses if it can't (#3262).
|
||||
"""
|
||||
try:
|
||||
import asyncio
|
||||
|
||||
Reference in New Issue
Block a user