feat(ml): read suggestions + allowlist from image_prediction (#768 step 2)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m13s

Switch every prediction READER off the JSON column onto the normalized
image_prediction table. Parity by construction: each reader loads the same
{raw_name: {category, confidence}} dict it consumed before (via small
_load_predictions helpers), so all downstream threshold/alias/merge/consensus
logic is byte-identical — only the data source changed.

- suggestions.SuggestionService.for_image (and for_selection via it)
- ml.apply_allowlist_tags (iterates images that have prediction rows)
- importer re-import reset deletes the image's prediction rows
The tagger_predictions JSON column is still dual-written (step 1) so it stays
valid during transition; the backfill task's NULL check still works. Removing
the JSON write + DROP column + retiring the #764 prune is the cleanup
follow-up (needs a quiesced-worker window for the DROP lock).

Tests: shared tests/_prediction_helpers.seed_predictions seeds the table;
read-path tests (suggestions, bulk consensus, allowlist apply, API) seed there
instead of ImageRecord.tagger_predictions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 16:03:58 -04:00
parent 79089b50b0
commit 22cdf0f334
8 changed files with 142 additions and 48 deletions
+12 -6
View File
@@ -64,18 +64,21 @@ async def test_apply_allowlist_applies_above_threshold(db):
from backend.app.services.tag_service import TagService
from backend.app.tasks import ml as ml_tasks
from tests._prediction_helpers import seed_predictions
tag = await TagService(db).find_or_create("autohero", TagKind.character)
db.add(TagAllowlist(tag_id=tag.id, min_confidence=0.95))
img = ImageRecord(
path="/images/al.jpg", sha256="al" + "0" * 62, size_bytes=1,
mime="image/jpeg", width=1, height=1,
origin="imported_filesystem", integrity_status="unknown",
tagger_predictions={
"autohero": {"category": "character", "confidence": 0.97}
},
)
db.add(img)
await db.commit()
await seed_predictions(
db, img.id, {"autohero": {"category": "character", "confidence": 0.97}}
)
await db.commit()
n = ml_tasks.apply_allowlist_tags(tag_id=tag.id)
assert n >= 1
@@ -99,18 +102,21 @@ async def test_apply_allowlist_skips_below_threshold(db):
from backend.app.services.tag_service import TagService
from backend.app.tasks import ml as ml_tasks
from tests._prediction_helpers import seed_predictions
tag = await TagService(db).find_or_create("lowconf", TagKind.character)
db.add(TagAllowlist(tag_id=tag.id, min_confidence=0.95))
img = ImageRecord(
path="/images/lc.jpg", sha256="lc" + "0" * 62, size_bytes=1,
mime="image/jpeg", width=1, height=1,
origin="imported_filesystem", integrity_status="unknown",
tagger_predictions={
"lowconf": {"category": "character", "confidence": 0.40}
},
)
db.add(img)
await db.commit()
await seed_predictions(
db, img.id, {"lowconf": {"category": "character", "confidence": 0.40}}
)
await db.commit()
ml_tasks.apply_allowlist_tags(tag_id=tag.id)
applied = (
await db.execute(