refactor(services): shared race-safe get_or_create helper (DRY backend sweep)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m19s

The find-or-create dance — SELECT, then a SAVEPOINT INSERT that recovers (not a
full rollback) on IntegrityError when a concurrent worker inserted first — was
hand-rolled identically in 4 async sites: ArtistService.find_or_create,
TagService.find_or_create, ExtensionService._find_or_create_artist and
._find_or_create_source. Divergent copies of exactly this pattern are how the
duplicate-row/race bugs in reference_scalar_one_or_none_duplicates crept in, so
it now lives once in services/db_helpers.get_or_create (returns (row, created);
factory adds+flushes+returns the row; caller owns the outer commit).

Over-DRY guard: SourceService's IntegrityError sites RAISE DuplicateSourceError
(reject-on-conflict, a different concept) — left alone. Importer._get_or_create
is the lone SYNC consumer (already shared by 2 callers) — stays separate, can't
cross the sync/async boundary. §8b: no hand-rolled async find-or-create remains.
Test: get_or_create creates then returns existing without re-invoking the factory.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-09 23:46:01 -04:00
parent 9deebfa133
commit 7b2a2051e9
5 changed files with 131 additions and 70 deletions
+23 -40
View File
@@ -11,11 +11,11 @@ from __future__ import annotations
import re
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession
from ..models import Artist, Source
from ..utils.slug import slugify
from .db_helpers import get_or_create
from .source_service import BACKFILL_MAX_CHUNKS
@@ -169,24 +169,16 @@ class ExtensionService:
500 against uq_artist_slug.
"""
slug = slugify(raw_name)
existing = (await self.session.execute(
select(Artist).where(Artist.slug == slug)
)).scalar_one_or_none()
if existing is not None:
return existing, False
sp = await self.session.begin_nested()
try:
async def _create() -> Artist:
artist = Artist(name=raw_name, slug=slug, is_subscription=True)
self.session.add(artist)
await self.session.flush()
await sp.commit()
return artist, True
except IntegrityError:
await sp.rollback()
recovered = (await self.session.execute(
select(Artist).where(Artist.slug == slug)
)).scalar_one()
return recovered, False
return artist
return await get_or_create(
self.session, select(Artist).where(Artist.slug == slug), _create
)
async def _find_or_create_source(
self, *, artist_id: int, platform: str, url: str,
@@ -194,17 +186,13 @@ class ExtensionService:
"""Race-safe — same pattern as _find_or_create_artist above. The
uq_source_artist_platform_url constraint catches the duplicate
insert; we roll the savepoint back and re-select."""
existing = (await self.session.execute(
select(Source).where(
Source.artist_id == artist_id,
Source.platform == platform,
Source.url == url,
)
)).scalar_one_or_none()
if existing is not None:
return existing, False
sp = await self.session.begin_nested()
try:
select_existing = select(Source).where(
Source.artist_id == artist_id,
Source.platform == platform,
Source.url == url,
)
async def _create() -> Source:
# New subscription sources arm run-until-done backfill (plan #693)
# so the first ticks walk the full history (otherwise gallery-dl's
# exit:20 short-circuits before the archive is built). Mirrors
@@ -219,16 +207,11 @@ class ExtensionService:
)
self.session.add(src)
await self.session.flush()
await sp.commit()
except IntegrityError:
await sp.rollback()
recovered = (await self.session.execute(
select(Source).where(
Source.artist_id == artist_id,
Source.platform == platform,
Source.url == url,
)
)).scalar_one()
return recovered, False
await self.session.commit()
return src, True
return src
src, created = await get_or_create(
self.session, select_existing, _create
)
if created:
await self.session.commit()
return src, created