fix(ml): per-task async engine for recompute_centroid (#881) #114

2026-06-16T20:24:32-04:00

bvandeusen commented

2026-06-16 20:24:32 -04:00

Fixes the flood of recompute_centroid failures in the activity view: RuntimeError: ... got Future ... attached to a different loop.

Root cause

recompute_centroid + recompute_centroids were the only tasks still using the process-wide singleton extensions.get_session() under asyncio.run(). The async engine's asyncpg pool is bound to the loop it was created on; each Celery task runs a fresh loop, so after the first invocation the cached engine handed old-loop connections to the new loop → fails on the first DB await (~35ms).

Fix

Convert both to the established per-task async_session_factory() pattern (NullPool engine created + disposed inside the task's own loop), matching scan/download/admin tasks. No get_session usages remain in tasks/.

CI green on dev (run 1179, 51201b4). Deploy note: this runs on the ml queue → re-pull the ml-worker image.

🤖 Generated with Claude Code

Fixes the flood of `recompute_centroid` failures in the activity view: `RuntimeError: ... got Future ... attached to a different loop`. ## Root cause `recompute_centroid` + `recompute_centroids` were the only tasks still using the process-wide singleton `extensions.get_session()` under `asyncio.run()`. The async engine's asyncpg pool is bound to the loop it was created on; each Celery task runs a fresh loop, so after the first invocation the cached engine handed old-loop connections to the new loop → fails on the first DB await (~35ms). ## Fix Convert both to the established per-task `async_session_factory()` pattern (NullPool engine created + disposed inside the task's own loop), matching scan/download/admin tasks. No `get_session` usages remain in `tasks/`. CI green on dev (run 1179, 51201b4). Deploy note: this runs on the `ml` queue → re-pull the **ml-worker** image. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 1 commit 2026-06-16 20:24:32 -04:00

fix(ml): per-task async engine for recompute_centroid (#881 )

CI / lint (push) Successful in 3s

Details

CI / frontend-build (push) Successful in 19s

Details

CI / backend-lint-and-test (push) Successful in 28s

Details

CI / integration (push) Successful in 3m14s

Details

51201b459e

recompute_centroid + recompute_centroids were the only tasks still using
the process-wide singleton extensions.get_session() under asyncio.run().
The async engine's asyncpg pool is bound to the loop it was created on;
each Celery task runs a fresh asyncio.run() loop, so after the first
invocation the cached engine handed loop-A connections to loop B and raised
"Future attached to a different loop" — every recompute after the first in
a worker process failed (~35ms, fails on first DB await).

Convert both to the established per-task async_session_factory() pattern
(NullPool engine created + disposed inside the task's own loop), matching
scan/download/admin tasks. No get_session usages remain in tasks/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bvandeusen merged commit 18d5c05639 into main

2026-06-16 20:24:37 -04:00

bvandeusen referenced this pull request

2026-06-16 20:34:16 -04:00

test(artist-dir): fix flaky uq_image_record_sha256 collision #115

bvandeusen referenced this issue from a commit

2026-06-27 22:49:12 -04:00

feat(ml): tag-eval backend — head-vs-centroid learning-curve eval (persisted)

bvandeusen referenced this pull request

2026-06-27 23:02:58 -04:00

Tag-eval (heads vs centroid) + focus/provenance/layout/sticky-header fixes + tag-gaps cleanup #136

bvandeusen referenced this issue from a commit