tag-eval: auto-apply operating point + server-side top-N concept discovery #139
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: bvandeusen/FabledCurator#139
Reference in New Issue
Block a user
head.auto_apply = {threshold, precision, recall}or null. Makes per-concept auto-apply readiness measurable (the eval's "commit" Bar 4).auto_top_nparam unions the explicit list with the N most-tagged general concepts via one fast DB query (replaces slow HTTP directory paging). Card gains "+ auto-add top-N" and precision-target inputs.No migration. (
5143f4c)🤖 Generated with Claude Code
Two additions driven by "what's the commit threshold?" + "find more tags": 1. High-precision operating point (Bar 4). Per concept, report the threshold that maximizes recall while holding precision >= a target (default 0.97, configurable via `precision_target`) — i.e. "could this fire without a human, and how much would it catch?" `head.auto_apply` = {target, threshold, precision, recall} or null if the target is unreachable. Surfaced on the card. 2. Server-side concept auto-discovery. `auto_top_n` param unions the explicit concept list with the N most-tagged general tags (one fast DB query) so the eval can broaden itself without hand-listing — replaces the slow HTTP directory paging. Card gains "+ auto-add top-N" and precision-target inputs. No migration; numpy/sklearn stay lazy. Existing _normalize_params test still holds (new keys additive; None still falls back to DEFAULT_CONCEPTS). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>