feat(heads): auto-apply observability + on by default (#114 auto-apply B)
Auto-apply is now ON by default (operator-asked: opt-OUT, not opt-in) — migration 0059 + model default flipped. The support (>=30) + measured-precision gates keep it safe and every auto-tag is reversible. Observability so the operator can tune from real data: - MISFIRE = an auto-applied (source='head_auto') tag the operator later removes. UNDER-FIRE = a tag with a head the operator adds by hand (the head missed it). Both captured at correction time in TagService.add_to_image/remove_from_image (source is lost on delete) into durable per-tag counters (head_metric), keyed by tag so they survive head retrain/prune. - Daily snapshot_head_metrics writes a per-concept time-series point (head_metrics_snapshot): auto-applied volume + cumulative misfires/under-fires + head quality; 180-day retention; daily beat. - GET /api/heads/metrics: per-concept current counts + realized misfire rate + head quality, plus the snapshot time-series — the report to tune the precision target + support floor. Migration 0060. Tests: misfire/under-fire counting (and the negatives — manual removal isn't a misfire, headless manual add isn't an under-fire), snapshot time-series, metrics API. What's the autofire threshold? There's no single number — each graduated head derives its OWN probability cutoff from its PR curve: the operating point that holds precision >= head_auto_apply_precision (0.97) at max recall. The global knobs are that target + the >=30 support floor. NEXT (slice 3): UI — enable toggle, dry-run preview, per-concept trends. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -9,6 +9,8 @@ from .credential import Credential
|
||||
from .download_event import DownloadEvent
|
||||
from .external_link import ExternalLink
|
||||
from .head_auto_apply_run import HeadAutoApplyRun
|
||||
from .head_metric import HeadMetric
|
||||
from .head_metrics_snapshot import HeadMetricsSnapshot
|
||||
from .head_training_run import HeadTrainingRun
|
||||
from .image_prediction import ImagePrediction
|
||||
from .image_provenance import ImageProvenance
|
||||
@@ -69,6 +71,8 @@ __all__ = [
|
||||
"LibraryAuditRun",
|
||||
"MLSettings",
|
||||
"HeadAutoApplyRun",
|
||||
"HeadMetric",
|
||||
"HeadMetricsSnapshot",
|
||||
"HeadTrainingRun",
|
||||
"TagAlias",
|
||||
"TagAllowlist",
|
||||
|
||||
Reference in New Issue
Block a user