fix(snippets): backfill must also catch JSON null, not just SQL NULL
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 45s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 1m11s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 45s
CI & Build / integration (push) Successful in 36s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 1m11s
Integration lane caught it (run 2984): the backfill reported 0 rows to fill and left `data` unset. `IS NULL` was the whole predicate, but a JSONB column has two empty states. SQLAlchemy's JSON types default to `none_as_null=False`, so assigning Python `None` persists the JSON encoding of null — `IS NULL` walks straight past it. Migration 0070 left genuine SQL NULLs, so the product path was right; the test was constructing the wrong shape with `data=None`. Fixed both ways, because both states mean "no usable mirror": - predicate is now `data IS NULL OR jsonb_typeof(data) = 'null'`; - the legacy-row test OMITS `data` (a real SQL NULL, 0070's actual shape), and a second test covers the JSON-null shape and asserts the premise with `jsonb_typeof` rather than assuming it. Refs #2083.
This commit is contained in:
@@ -32,7 +32,7 @@ import asyncio
|
||||
import logging
|
||||
import re
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import func, or_, select
|
||||
|
||||
from scribe.models import async_session
|
||||
from scribe.models.note import Note
|
||||
@@ -386,8 +386,16 @@ async def backfill_snippet_data(*, batch: int = 500) -> int:
|
||||
filled row is skipped forever after, and a snippet with no structured fields
|
||||
at all settles at `{}` rather than staying NULL and being re-scanned. Trashed
|
||||
rows are included so a later restore comes back queryable.
|
||||
|
||||
"Unfilled" means SQL NULL *or* JSON `null` — two different states in a JSONB
|
||||
column, and only the first is what migration 0070 left behind. SQLAlchemy's
|
||||
JSON types default to ``none_as_null=False``, so assigning Python ``None`` to
|
||||
this column persists the JSON encoding of null rather than SQL NULL; an
|
||||
``IS NULL`` test alone walks straight past such a row and reports nothing to
|
||||
do. Both mean "no usable mirror", so both are filled.
|
||||
"""
|
||||
filled = 0
|
||||
unfilled = or_(Note.data.is_(None), func.jsonb_typeof(Note.data) == "null")
|
||||
async with async_session() as session:
|
||||
while True:
|
||||
rows = list(
|
||||
@@ -395,7 +403,7 @@ async def backfill_snippet_data(*, batch: int = 500) -> int:
|
||||
await session.execute(
|
||||
select(Note)
|
||||
.where(Note.note_type == SNIPPET_NOTE_TYPE)
|
||||
.where(Note.data.is_(None))
|
||||
.where(unfilled)
|
||||
.limit(batch)
|
||||
)
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user