tag-eval: auto-apply operating point + server-side top-N concept discovery #139

2026-06-28T00:54:10-04:00

bvandeusen commented

2026-06-28 00:54:10 -04:00

Auto-apply operating point — per concept, the threshold maximizing recall while holding precision ≥ target (default 0.97, configurable). head.auto_apply = {threshold, precision, recall} or null. Makes per-concept auto-apply readiness measurable (the eval's "commit" Bar 4).
Top-N concept discovery — auto_top_n param unions the explicit list with the N most-tagged general concepts via one fast DB query (replaces slow HTTP directory paging). Card gains "+ auto-add top-N" and precision-target inputs.

No migration. (5143f4c)

🤖 Generated with Claude Code

- **Auto-apply operating point** — per concept, the threshold maximizing recall while holding precision ≥ target (default 0.97, configurable). `head.auto_apply = {threshold, precision, recall}` or null. Makes per-concept auto-apply readiness measurable (the eval's "commit" Bar 4). - **Top-N concept discovery** — `auto_top_n` param unions the explicit list with the N most-tagged general concepts via one fast DB query (replaces slow HTTP directory paging). Card gains "+ auto-add top-N" and precision-target inputs. No migration. (`5143f4c`) 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 1 commit 2026-06-28 00:54:10 -04:00

feat(tag-eval): auto-apply operating point + server-side top-N concept discovery

CI / lint (push) Successful in 2s

Details

CI / frontend-build (push) Successful in 17s

Details

CI / backend-lint-and-test (push) Successful in 28s

Details

CI / integration (push) Successful in 3m24s

Details

5143f4c34f

Two additions driven by "what's the commit threshold?" + "find more tags":

1. High-precision operating point (Bar 4). Per concept, report the threshold that
   maximizes recall while holding precision >= a target (default 0.97, configurable
   via `precision_target`) — i.e. "could this fire without a human, and how much
   would it catch?" `head.auto_apply` = {target, threshold, precision, recall} or
   null if the target is unreachable. Surfaced on the card.

2. Server-side concept auto-discovery. `auto_top_n` param unions the explicit
   concept list with the N most-tagged general tags (one fast DB query) so the
   eval can broaden itself without hand-listing — replaces the slow HTTP directory
   paging. Card gains "+ auto-add top-N" and precision-target inputs.

No migration; numpy/sklearn stay lazy. Existing _normalize_params test still
holds (new keys additive; None still falls back to DEFAULT_CONCEPTS).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

bvandeusen merged commit abf88b1a15 into main

2026-06-28 00:54:14 -04:00

bvandeusen referenced this pull request

2026-07-06 21:03:23 -04:00

Soft auto-apply (retract + confirm, no self-training) + tagging UX (reject-rest, tag-input race, modal playlist) #197

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#139