refactor(ml): DRY pass — shared sweep helpers + table-driven settings (#161)
Consolidate duplication accrued across the ML tagging + settings backend, behavior-preserving (over-DRY guard applied — the three auto-apply sweep BODIES stay separate; only their shared inner helpers are extracted). - _sigmoid / _conflict_scores / _insert_presentation_review (heads.py): the score→prob transform (6 inlined sites), the presentation conflict signal (2 sites), and the ring-loud PresentationReview insert (2 sites, single- sourced so the mode column can't drift on the shared composite PK). - _applied_or_rejected (training_data.py): the per-tag "applied ∪ rejected" skip-set, byte-identical at 3 sweep sites (heads.py x2, tasks/ml.py ccip). - ccip sweep divergence fixes: import ccip._FIGURE_KINDS + training_data._l2norm instead of local copies that silently drift when the canonical changes. - MLSettings.load / .load_sync classmethods (mirror ImportSettings); route all 8 scalar_one singleton reads through them (the session.get None-path stays). - GET serializers for MLSettings + ImportSettings are now table-driven off the same _EDITABLE tuples PATCH writes, so a new field can't be silently absent from GET (the split that historically dropped fields). - AUTO_APPLY_THRESHOLD_MIN/MAX constant single-sources the [0.5,0.999] operating range across the service clamp + the 5 API validators. - test_ml_dry_helpers.py pins _applied_or_rejected + _sigmoid. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi
This commit is contained in:
@@ -94,6 +94,24 @@ def _rejected_ids(session: Session, tag_id: int) -> list[int]:
|
||||
]
|
||||
|
||||
|
||||
def _applied_or_rejected(session: Session, tag_ids) -> dict[int, set[int]]:
|
||||
"""Per-tag skip set for the auto-apply sweeps: every image that ALREADY carries
|
||||
the tag (ANY source — not just training positives) OR has rejected it. A sweep
|
||||
never re-applies to these. Shared by auto_apply_sweep + system_tag_auto_apply_sweep
|
||||
(heads.py) and scheduled_ccip_auto_apply (tasks/ml.py). Callers mutate the returned
|
||||
sets in-place to also dedupe within a single run."""
|
||||
skip: dict[int, set[int]] = {}
|
||||
for tid in tag_ids:
|
||||
ids = {
|
||||
r[0] for r in session.execute(
|
||||
select(image_tag.c.image_record_id).where(image_tag.c.tag_id == tid)
|
||||
).all()
|
||||
}
|
||||
ids.update(_rejected_ids(session, tid))
|
||||
skip[tid] = ids
|
||||
return skip
|
||||
|
||||
|
||||
def _sample_unlabeled(session: Session, exclude: set[int], limit: int) -> list[int]:
|
||||
"""Random image ids (with an embedding) NOT carrying the tag. Concepts are
|
||||
sparse, so an untagged image is almost always a true negative."""
|
||||
|
||||
Reference in New Issue
Block a user