feat(tag-eval): auto-apply operating point + server-side top-N concept discovery
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m24s

Two additions driven by "what's the commit threshold?" + "find more tags":

1. High-precision operating point (Bar 4). Per concept, report the threshold that
   maximizes recall while holding precision >= a target (default 0.97, configurable
   via `precision_target`) — i.e. "could this fire without a human, and how much
   would it catch?" `head.auto_apply` = {target, threshold, precision, recall} or
   null if the target is unreachable. Surfaced on the card.

2. Server-side concept auto-discovery. `auto_top_n` param unions the explicit
   concept list with the N most-tagged general tags (one fast DB query) so the
   eval can broaden itself without hand-listing — replaces the slow HTTP directory
   paging. Card gains "+ auto-add top-N" and precision-target inputs.

No migration; numpy/sklearn stay lazy. Existing _normalize_params test still
holds (new keys additive; None still falls back to DEFAULT_CONCEPTS).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-28 00:50:28 -04:00
parent fc64f130b8
commit 5143f4c34f
2 changed files with 98 additions and 7 deletions
@@ -19,6 +19,19 @@
:disabled="running"
/>
<div class="d-flex mb-3" style="gap: 12px;">
<v-text-field
v-model.number="autoTopN" label="+ auto-add top-N concepts"
type="number" min="0" max="200" density="compact" hide-details
:disabled="running" style="max-width: 220px;"
/>
<v-text-field
v-model.number="precisionTarget" label="Auto-apply precision target"
type="number" min="0.5" max="0.999" step="0.01" density="compact" hide-details
:disabled="running" style="max-width: 220px;"
/>
</div>
<v-btn
v-if="!running"
color="accent" variant="flat" rounded="pill"
@@ -78,6 +91,16 @@
(head centroid)
</div>
<div class="text-caption mb-2">
<span class="fc-muted">Auto-apply:</span>
<template v-if="c.head.auto_apply">
<span class="fc-up">ready</span> at P{{ c.head.auto_apply.target }}
catches recall <strong>{{ c.head.auto_apply.recall }}</strong>
(thr {{ c.head.auto_apply.threshold }})
</template>
<span v-else class="fc-down">not reachable at P{{ report.params.precision_target }}</span>
</div>
<div v-if="c.curve && c.curve.length" class="fc-curve">
<span class="fc-muted text-caption">Learning curve (AP @ N positives):</span>
<span v-for="p in c.curve" :key="p.n_pos" class="fc-curve__pt">
@@ -145,6 +168,8 @@ const store = useTagEvalStore()
const modal = useModalStore()
const run = ref(null)
const conceptsText = ref(DEFAULT_CONCEPTS)
const autoTopN = ref(0)
const precisionTarget = ref(0.97)
const busy = ref(false)
let pollTimer = null
@@ -186,7 +211,11 @@ async function onStart() {
busy.value = true
try {
const concepts = conceptsText.value.split(',').map(s => s.trim()).filter(Boolean)
const res = await store.start({ concepts })
const res = await store.start({
concepts,
auto_top_n: Number(autoTopN.value) || 0,
precision_target: Number(precisionTarget.value) || 0.97,
})
run.value = await store.getRun(res.run_id)
startPoll(res.run_id)
} catch (e) {