Every backfill selected record text alongside the id and then iterated that snapshot — minutes, over a real corpus. Anything edited inside the window got correct vectors from its own write path and then had them overwritten with the pre-edit text, stamped at the current CHUNKER_VERSION so no later boot would ever repair it. The record stays findable by what it used to say, silently, until someone happens to edit it again.
All four backfills had it. Three pre-existing; the fourth was written an hour earlier for #4251 by copying the third.
Two halves
Stop it — the scan takes ids only; _current_row re-reads each record at the moment it is embedded. One indexed PK read against a forward pass. The race shrinks from the length of the run to the width of one record.
Repair it — _vectors_older_than_their_record adds a third staleness condition beside "no vectors" and "old chunker": vectors older than the text they encode. A guard that refuses to write a bad value does not undo the values already stored (#4202).
That check is worth more than the bug. It equally catches a record whose embed_note never ran — no event loop, a swallowed refresh, a process that died between the commit and the index. All of those are invisible to the version column and invisible from outside.
_tasks_logged_since_embedding covers the case the timestamp comparison cannot see: a work log is part of a task's document but lives in its own table, so writing one never moves notes.updated_at — and those are the records the race was most likely to hit.
Deploy notes
No migration, no version bump, no corpus change. Smaller and lower-risk than #175.
The repair half executes on the next boot, as part of the existing startup backfill.
Whether the previous run clobbered anything is unknown and was not measured; the repair sweeps it either way without needing a diagnosis first.
Verification
tests/test_backfill_reads_current_text.py — the regression, deletion between scan and loop, both repair conditions, and a quiet corpus embedding nothing. The structural guard was checked against the OLD shape (reintroducing select(Note.id, Note.title, Note.body) makes it report 2 offenders) as well as the new, so it can catch the next copy rather than merely agree with today's code (#167).
One commit. CI 7185 green on `a58a225`.
## The bug
Every backfill selected record text alongside the id and then iterated that snapshot — minutes, over a real corpus. Anything edited inside the window got correct vectors from its own write path and then had them **overwritten with the pre-edit text**, stamped at the current `CHUNKER_VERSION` so no later boot would ever repair it. The record stays findable by what it used to say, silently, until someone happens to edit it again.
All four backfills had it. Three pre-existing; the fourth was written an hour earlier for #4251 by copying the third.
## Two halves
**Stop it** — the scan takes ids only; `_current_row` re-reads each record at the moment it is embedded. One indexed PK read against a forward pass. The race shrinks from the length of the run to the width of one record.
**Repair it** — `_vectors_older_than_their_record` adds a third staleness condition beside "no vectors" and "old chunker": vectors older than the text they encode. A guard that refuses to write a bad value does not undo the values already stored (#4202).
That check is worth more than the bug. It equally catches a record whose `embed_note` never ran — no event loop, a swallowed refresh, a process that died between the commit and the index. All of those are invisible to the version column and invisible from outside.
`_tasks_logged_since_embedding` covers the case the timestamp comparison cannot see: a work log is part of a task's document but lives in its own table, so writing one never moves `notes.updated_at` — and those are the records the race was most likely to hit.
## Deploy notes
- **No migration, no version bump, no corpus change.** Smaller and lower-risk than #175.
- The repair half executes **on the next boot**, as part of the existing startup backfill.
- Whether the previous run clobbered anything is unknown and was not measured; the repair sweeps it either way without needing a diagnosis first.
## Verification
`tests/test_backfill_reads_current_text.py` — the regression, deletion between scan and loop, both repair conditions, and a quiet corpus embedding nothing. The structural guard was checked against the OLD shape (reintroducing `select(Note.id, Note.title, Note.body)` makes it report 2 offenders) as well as the new, so it can catch the next copy rather than merely agree with today's code (#167).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Every backfill selected the text alongside the id and then iterated that
snapshot, sleeping between records. Over a real corpus that is minutes, and
anything edited inside the window was re-embedded by its own write path and
then OVERWRITTEN with the pre-edit text the scan had captured — stamped at the
current chunker version, so the next boot considered it current and never
repaired it. The record stayed findable by what it used to say, indefinitely,
until someone happened to edit it again. Nothing reported it, and no existing
signal separated it from a correct record: the version was right and the
vectors were there.
All four backfills had it. Three were pre-existing; I wrote the fourth an hour
ago for #4251 by copying the third.
#4251 also widened the exposure. The race used to need a note-body edit; now
every add_task_log re-embeds its task, so any session recording work during a
backfill can hit it — which is exactly what "forward work continues while the
backfill runs" means.
TWO HALVES, SEPARATE ON PURPOSE.
Stopping it: the scan takes IDS ONLY and `_current_row` re-reads each record at
the moment it is embedded. One indexed primary-key read, negligible beside the
forward pass that follows, and it turns a backfill from "replay a snapshot"
into "repair to current truth". The residual race shrinks from the length of
the run to the width of one record, which the parent-row claim and the
single-transaction replacement narrow further.
Repairing it: `_vectors_older_than_their_record` adds a third staleness
condition beside "no vectors" and "old chunker" — vectors older than the text
they encode. A guard that refuses to write a bad value does not undo the bad
values already stored (#4202); the rows are what has to change.
That check earns its place beyond the bug that prompted it. It equally catches
a record whose `embed_note` never ran — no event loop, a swallowed refresh, a
process that died between the commit and the index. Every one of those looks
identical from the version column and identical from outside.
And a work log is part of a task's document while living in its own table, so
writing one never moves `notes.updated_at` and the timestamp comparison cannot
see it. `_tasks_logged_since_embedding` covers exactly the records the race was
most likely to have hit.
The live instance has already run #4251's backfill — the prior-art arm is
surfacing `## Work log —` passages, which is the thing itself rather than a
banner about it. Whether that run clobbered anything is unknown and was not
measured; the repair half picks up whatever it did on the next boot, without
needing it diagnosed first.
The structural guard in the tests was checked against the old shape, not just
the new one: reintroducing `select(Note.id, Note.title, Note.body)` makes it
fail, so it can catch the next copy of this rather than only agreeing with
today's code (#167).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
One commit. CI 7185 green on
a58a225.The bug
Every backfill selected record text alongside the id and then iterated that snapshot — minutes, over a real corpus. Anything edited inside the window got correct vectors from its own write path and then had them overwritten with the pre-edit text, stamped at the current
CHUNKER_VERSIONso no later boot would ever repair it. The record stays findable by what it used to say, silently, until someone happens to edit it again.All four backfills had it. Three pre-existing; the fourth was written an hour earlier for #4251 by copying the third.
Two halves
Stop it — the scan takes ids only;
_current_rowre-reads each record at the moment it is embedded. One indexed PK read against a forward pass. The race shrinks from the length of the run to the width of one record.Repair it —
_vectors_older_than_their_recordadds a third staleness condition beside "no vectors" and "old chunker": vectors older than the text they encode. A guard that refuses to write a bad value does not undo the values already stored (#4202).That check is worth more than the bug. It equally catches a record whose
embed_notenever ran — no event loop, a swallowed refresh, a process that died between the commit and the index. All of those are invisible to the version column and invisible from outside._tasks_logged_since_embeddingcovers the case the timestamp comparison cannot see: a work log is part of a task's document but lives in its own table, so writing one never movesnotes.updated_at— and those are the records the race was most likely to hit.Deploy notes
Verification
tests/test_backfill_reads_current_text.py— the regression, deletion between scan and loop, both repair conditions, and a quiet corpus embedding nothing. The structural guard was checked against the OLD shape (reintroducingselect(Note.id, Note.title, Note.body)makes it report 2 offenders) as well as the new, so it can catch the next copy rather than merely agree with today's code (#167).🤖 Generated with Claude Code
https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy