fix(snippets): backfill must also catch JSON null, not just SQL NULL
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 45s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 1m11s

Integration lane caught it (run 2984): the backfill reported 0 rows to fill
and left `data` unset. `IS NULL` was the whole predicate, but a JSONB column
has two empty states. SQLAlchemy's JSON types default to
`none_as_null=False`, so assigning Python `None` persists the JSON encoding
of null — `IS NULL` walks straight past it.

Migration 0070 left genuine SQL NULLs, so the product path was right; the
test was constructing the wrong shape with `data=None`. Fixed both ways,
because both states mean "no usable mirror":
- predicate is now `data IS NULL OR jsonb_typeof(data) = 'null'`;
- the legacy-row test OMITS `data` (a real SQL NULL, 0070's actual shape),
  and a second test covers the JSON-null shape and asserts the premise with
  `jsonb_typeof` rather than assuming it.

Refs #2083.
This commit is contained in:
2026-07-27 23:12:48 -04:00
parent dd1b5e5ddb
commit 083944f0fd
2 changed files with 34 additions and 9 deletions
+10 -2
View File
@@ -32,7 +32,7 @@ import asyncio
import logging
import re
from sqlalchemy import select
from sqlalchemy import func, or_, select
from scribe.models import async_session
from scribe.models.note import Note
@@ -386,8 +386,16 @@ async def backfill_snippet_data(*, batch: int = 500) -> int:
filled row is skipped forever after, and a snippet with no structured fields
at all settles at `{}` rather than staying NULL and being re-scanned. Trashed
rows are included so a later restore comes back queryable.
"Unfilled" means SQL NULL *or* JSON `null` — two different states in a JSONB
column, and only the first is what migration 0070 left behind. SQLAlchemy's
JSON types default to ``none_as_null=False``, so assigning Python ``None`` to
this column persists the JSON encoding of null rather than SQL NULL; an
``IS NULL`` test alone walks straight past such a row and reports nothing to
do. Both mean "no usable mirror", so both are filled.
"""
filled = 0
unfilled = or_(Note.data.is_(None), func.jsonb_typeof(Note.data) == "null")
async with async_session() as session:
while True:
rows = list(
@@ -395,7 +403,7 @@ async def backfill_snippet_data(*, batch: int = 500) -> int:
await session.execute(
select(Note)
.where(Note.note_type == SNIPPET_NOTE_TYPE)
.where(Note.data.is_(None))
.where(unfilled)
.limit(batch)
)
)