feat(ml): DB-backed tagger_store_floor (default 0.70), the ingest confidence floor
Promotes the prediction store-floor from the TAGGER_STORE_FLOOR env (default 0.05) to a DB-backed, Settings-UI-tunable ml_settings column (default 0.70). Storing every tag down to 0.05 from a ~10k-tag tagger is what grew image_record's TOAST to ~100 GB; the suggestion path already filters at 0.70 and the centroid/learned path covers lower-confidence preferred tags, so the sub-0.70 tail is redundant. Foundation for plan-task #764 (backfill + reclaim land next; this only changes the write gate for NEW imports). - ml_settings.tagger_store_floor (migration 0044, default 0.70) - tagger.Tagger.infer(store_floor=...); ml task passes settings.tagger_store_floor - ML admin GET/PATCH expose it; PATCH rejects a category suggestion threshold below the floor (nothing below the floor is stored, so the gap surfaces nothing) — server backstop for the UI slider clamp - Settings → ML: store-floor slider + caption; category sliders min-bound to it Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -33,8 +33,13 @@ _MODEL_DIR = Path(os.environ.get("ML_MODEL_DIR", "/models")) / "camie"
|
||||
_MODEL_FILE = f"{MODEL_NAME}.onnx"
|
||||
_METADATA_FILE = f"{MODEL_NAME}-metadata.json"
|
||||
|
||||
# Below this confidence, predictions aren't stored (keeps the JSON compact).
|
||||
STORE_FLOOR = float(os.environ.get("TAGGER_STORE_FLOOR", "0.05"))
|
||||
# Ingest floor below which predictions aren't stored (keeps the JSON compact).
|
||||
# DEFAULT/fallback only — the live value is DB-backed
|
||||
# (ml_settings.tagger_store_floor) and passed into infer() per call by the ml
|
||||
# task. 0.70: the suggestion path already filters there and the centroid path
|
||||
# covers lower-confidence preferred tags, so the sub-0.70 tail is redundant
|
||||
# (it had bloated image_record's TOAST to ~100 GB; plan-task #764).
|
||||
DEFAULT_STORE_FLOOR = 0.70
|
||||
|
||||
# The categories FC-2b surfaces in the UI. Others (meta/rating/year) are
|
||||
# still stored but the suggestion service filters them out.
|
||||
@@ -145,10 +150,13 @@ class Tagger:
|
||||
arr = arr.transpose(2, 0, 1) # HWC -> CHW
|
||||
return arr[np.newaxis, :, :, :] # NCHW
|
||||
|
||||
def infer(self, image_path: Path) -> dict[str, TagPrediction]:
|
||||
def infer(
|
||||
self, image_path: Path, *, store_floor: float = DEFAULT_STORE_FLOOR,
|
||||
) -> dict[str, TagPrediction]:
|
||||
"""Run Camie v2 on one image. Returns {name: TagPrediction} with
|
||||
confidence >= STORE_FLOOR (across all categories — the suggestion
|
||||
service does category filtering later).
|
||||
confidence >= store_floor (across all categories — the suggestion
|
||||
service does category filtering later). store_floor is the DB-backed
|
||||
ml_settings.tagger_store_floor, passed in by the ml task.
|
||||
|
||||
v2 emits multiple outputs; we use the refined predictions
|
||||
(output[1] per onnx_inference.py). Sigmoid is applied to raw
|
||||
@@ -167,7 +175,7 @@ class Tagger:
|
||||
cats = self._tag_categories
|
||||
for idx, score in enumerate(probs):
|
||||
conf = float(score)
|
||||
if conf < STORE_FLOOR:
|
||||
if conf < store_floor:
|
||||
continue
|
||||
if idx >= len(names):
|
||||
# Output longer than metadata declared — shouldn't happen but
|
||||
|
||||
Reference in New Issue
Block a user