Files
minstrel/web/src/routes/admin/tuning/+page.svelte
T
bvandeusenandClaude Opus 5 f367eeaa9d
test-web / test (push) Successful in 1m7s
test-go / test (push) Successful in 1m31s
test-go / integration (push) Failing after 4m21s
release / Build signed APK (releases and dev) (push) Successful in 5m11s
release / Build + push container image (push) Successful in 1m52s
release / Verify release artifacts (tag releases only) (push) Skipped
fix(recommendation): Songs-like gets its own profile so it stops wandering
Operator, 2026-09-10: "when I play it I'm expecting to get a consistent
sound and style from the experience... I was getting a seeming wide variety
of music from each one when I was hoping to stay in a certain neighborhood."

Songs-like shared the `daily_mix` weight profile with For-You, and that
sharing WAS the bug. The two surfaces want opposite things: For-You answers
"what will they enjoy today" and is supposed to roam; Songs-like answers
"what sounds like THIS". Under one profile the broad answer wins.

The arithmetic, from the shared weights:

    unrelated track, liked, not played recently → 1.0 + 2.0 + 1.0 = 4.0
    PERFECT similarity match, not liked         → 1.0 + 1.5       = 2.5

Liking something outranked sounding like the seed, because LikeBoost (2.0)
exceeded SimilarityWeight's whole range (1.5) and TasteWeight (1.5, and
seed-INDEPENDENT) matched it outright. Under the new profile the same pair
scores 5.00 vs 2.00.

Two levers, because either alone leaves the other's failure intact:

POOL. Songs-like now takes its own CandidateSourceLimits. The default gave
~29% of candidates a sim_score of literally zero — `taste_overlap` and
`random_fill` are both `0.0::float8` in recommendation.sql, seed-independent
by construction. Same total pool size; composition shifts to arms that
measure distance from the seed, LBSimilar doubled.

WEIGHTS. A third profile beside radio and daily_mix, DB-backed and live per
rule 25, with the property that similarity's range exceeds the combined
range of every seed-independent differentiator — so a closer match cannot
be beaten on likes, freshness and taste alone, while tracks within ~0.39
similarity of each other still get ordered by what the user likes.

Rule 131 changed the pool design mid-way and for the better. Zeroing the
two seed-independent arms was the first instinct and is exactly the
vanish-or-nothing shape that rule forbids: a seed with thin ListenBrainz
coverage would yield a short mix or none. They are the tier-3 FLOOR — cut
hard, never removed — and the weights keep them at the bottom of the
ranking rather than out of the pool. "A few tracks further from the seed
than we'd like" beats "no playlist".

Caught while wiring it: switching only pickTopN's final Score would have
been nearly INERT. scoreAndSortCandidates does the selection sort, and the
caller caps and truncates in that order — so the playlist would still have
been chosen by daily_mix and merely relabelled with songs_like numbers. It
now takes the profile as a parameter, and each surface passes its own.

Also corrects the daily_mix card's blurb, which claimed Songs-like as one
of its surfaces and no longer is.

Guards pin behaviour rather than the numbers, since numbers get retuned:
that similarity beats an unrelated liked track, that daily_mix still
DOESN'T (or the split buys nothing), that the tier-3 floor is non-zero,
and that the UI card shows its own values rather than falling back. Each
falsified against its named regression first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 20:51:15 -04:00

538 lines
22 KiB
Svelte
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<script lang="ts">
import { pageTitle } from '$lib/branding';
import {
getTuning,
patchTuning,
resetTuning,
getTrends,
type TuningScope,
type WeightProfileScope,
type TuningSnapshot,
type WeightProfile,
type TasteTuning,
type DiscoverTuning,
type TrendsResponse,
type TrendSeries,
type TrendMarker
} from '$lib/api/tuning';
import { errMessage } from '$lib/api/errors';
import { pushToast } from '$lib/stores/toast.svelte';
// The defaults-discovery lab (#1250): DB-backed scoring-weight
// profiles + taste-build knobs with live effect. This card exists to
// FIND good values — found-good values get baked into shipped
// defaults, so end users and other operators never need it.
const weightFields: { key: keyof WeightProfile; label: string; hint: string }[] = [
{ key: 'base_weight', label: 'Base weight', hint: 'Floor score every candidate starts from.' },
{ key: 'like_boost', label: 'Like boost', hint: 'Added when the track is liked.' },
{ key: 'recency_weight', label: 'Recency weight', hint: 'Rewards tracks not played recently (030d ramp).' },
{ key: 'skip_penalty', label: 'Skip penalty', hint: 'Subtracts skips/plays ratio.' },
{ key: 'jitter_magnitude', label: 'Jitter', hint: 'Random reshuffle magnitude for near-ties.' },
{ key: 'context_weight', label: 'Context weight', hint: 'Session-vector similarity contribution.' },
{ key: 'similarity_weight', label: 'Similarity weight', hint: 'Seed-similarity contribution.' },
{ key: 'taste_weight', label: 'Taste weight', hint: 'Learned taste-profile fit, in [-1, +1].' },
{ key: 'context_time_weight', label: 'Time-of-day weight', hint: "Artist's time-of-day/weekday affinity for the current context, in [-1, +1]. 0 = ignore when you listen." }
];
const tasteFields: { key: keyof TasteTuning; label: string; hint: string }[] = [
{ key: 'half_life_days', label: 'Half-life (days)', hint: "A play's influence halves every this-many days." },
{ key: 'engagement_hard_skip', label: 'Hard-skip point', hint: 'Completion at/below which a play reads 1.' },
{ key: 'engagement_neutral', label: 'Neutral point', hint: 'Completion at which a play reads 0.' },
{ key: 'engagement_full', label: 'Full point', hint: 'Completion at/above which a play reads +1.' },
{ key: 'enriched_tag_scale', label: 'Enriched tag weight', hint: 'How much folksonomy tags (MusicBrainz/Last.fm) count vs raw file genre, in [0, 1]. 0 = genre only.' },
{ key: 'era_scale', label: 'Era weight', hint: 'How strongly a decade-play imprints on the era facet, in [0, 1]. 0 = era ignored.' },
{ key: 'mood_scale', label: 'Mood weight', hint: 'How strongly a mood-tagged play imprints on the mood facet (from folksonomy tags), in [0, 1]. 0 = mood ignored.' }
];
const discoverFields: { key: keyof DiscoverTuning; label: string; hint: string }[] = [
{ key: 'tag_overlap_weight', label: 'Taste-tag weight', hint: "How strongly a candidate's tags matching your taste boosts it. score x (1 + w x overlap), so 0 turns the tag term off and ranks on similarity alone. An artist with no cached tags is never penalised." },
{ key: 'snooze_days', label: 'Snooze length (days)', hint: 'How long "not right now" parks a suggestion before it returns on its own. Records no opinion about the artist and never feeds the taste profile.' }
];
const profileScopes: { scope: WeightProfileScope; label: string; blurb: string }[] = [
{ scope: 'radio', label: 'Radio', blurb: 'Seed-directed listening — the user picked a direction.' },
{ scope: 'daily_mix', label: 'Daily mixes', blurb: 'For You, the discovery mixes, and "You might like".' },
{
scope: 'songs_like',
label: 'Songs like…',
blurb:
'The tightest surface: everything here should sound like the seed track. Similarity dominates on purpose — raising like/taste/recency here is what makes these mixes wander.'
}
];
let snapshot = $state<TuningSnapshot | null>(null);
let loadFailed = $state(false);
// Editable copies, string-typed for the inputs; parsed on save.
let form = $state<Record<string, Record<string, string>>>({});
let saving = $state<TuningScope | null>(null);
function fillForm(snap: TuningSnapshot) {
const f: Record<string, Record<string, string>> = {
radio: {},
daily_mix: {},
songs_like: {},
taste: {},
discover: {}
};
for (const p of profileScopes.map((s) => s.scope)) {
for (const { key } of weightFields) f[p][key] = String(snap.profiles[p][key]);
}
for (const { key } of tasteFields) f.taste[key] = String(snap.taste[key]);
for (const { key } of discoverFields) f.discover[key] = String(snap.discover[key]);
form = f;
}
$effect(() => {
getTuning()
.then((snap) => {
fillForm(snap);
snapshot = snap;
})
.catch(() => {
loadFailed = true;
});
});
// A knob deviates when its CURRENT SAVED value differs from shipped;
// the dot marks where this install has drifted from defaults.
function deviates(scope: TuningScope, key: string): boolean {
if (!snapshot) return false;
if (scope === 'taste') {
return snapshot.taste[key as keyof TasteTuning] !== snapshot.shipped.taste[key as keyof TasteTuning];
}
if (scope === 'discover') {
return (
snapshot.discover[key as keyof DiscoverTuning] !==
snapshot.shipped.discover[key as keyof DiscoverTuning]
);
}
return (
snapshot.profiles[scope][key as keyof WeightProfile] !==
snapshot.shipped.profiles[scope][key as keyof WeightProfile]
);
}
function currentValue(scope: TuningScope, key: string): number {
if (!snapshot) return 0;
if (scope === 'taste') return snapshot.taste[key as keyof TasteTuning];
if (scope === 'discover') return snapshot.discover[key as keyof DiscoverTuning];
return snapshot.profiles[scope][key as keyof WeightProfile];
}
async function save(scope: TuningScope) {
if (!snapshot) return;
const values: Record<string, number> = {};
for (const [key, raw] of Object.entries(form[scope] ?? {})) {
const v = Number(raw);
if (!Number.isFinite(v)) {
pushToast(`${key} is not a number.`, 'error');
return;
}
if (v !== currentValue(scope, key)) values[key] = v;
}
if (Object.keys(values).length === 0) {
pushToast('Nothing changed.');
return;
}
saving = scope;
try {
snapshot = await patchTuning(scope, values);
fillForm(snapshot);
pushToast('Saved — takes effect on the next scoring pass.');
} catch (e: unknown) {
pushToast(`Save failed: ${errMessage(e)}`, 'error');
} finally {
saving = null;
}
}
async function reset(scope: TuningScope) {
saving = scope;
try {
snapshot = await resetTuning(scope);
fillForm(snapshot);
pushToast('Reset to shipped defaults.');
} catch (e: unknown) {
pushToast(`Reset failed: ${errMessage(e)}`, 'error');
} finally {
saving = null;
}
}
// Trends (#1251): weekly skip-rate sparklines per surface with
// knob-turn markers — the verify half of the tune→verify loop.
let trends = $state<TrendsResponse | null>(null);
let trendsFailed = $state(false);
$effect(() => {
getTrends()
.then((t) => {
trends = t;
})
.catch(() => {
trendsFailed = true;
});
});
const SPARK_W = 220;
const SPARK_H = 36;
const TREND_LOW_VOLUME = 20;
// The week axis is the sorted union of every series' buckets, so all
// sparklines and markers share one x scale.
const weekAxis = $derived.by(() => {
if (!trends) return [] as string[];
const set = new Set<string>();
for (const s of trends.series) for (const p of s.points) set.add(p.week_start);
return [...set].sort();
});
function xFor(week: string): number {
const i = weekAxis.indexOf(week);
if (i < 0 || weekAxis.length < 2) return 0;
return (i / (weekAxis.length - 1)) * SPARK_W;
}
// Polyline of the series' weekly skip rate on a fixed [0,1] y-scale
// (higher = worse, drawn upward) so rows are visually comparable.
function sparkPoints(s: TrendSeries): string {
return s.points
.map((p) => `${xFor(p.week_start).toFixed(1)},${(SPARK_H - p.skip_rate * SPARK_H).toFixed(1)}`)
.join(' ');
}
// A marker lands on the latest axis week that starts at/before it.
function markerX(m: TrendMarker): number {
const day = m.changed_at.slice(0, 10);
let idx = -1;
for (let i = 0; i < weekAxis.length; i++) {
if (weekAxis[i] <= day) idx = i;
}
if (idx < 0 || weekAxis.length < 2) return 0;
return (idx / (weekAxis.length - 1)) * SPARK_W;
}
function windowTasteHitRate(s: TrendSeries): number {
let plays = 0;
let hits = 0;
for (const p of s.points) {
plays += p.plays;
hits += p.taste_hit_rate * p.plays;
}
return plays > 0 ? hits / plays : 0;
}
// The "latest" columns are ONE WEEK while the Plays column is the whole
// window, which is a trap: a 40% skip rate off 17 plays sat next to a
// four-figure Plays total and read as a solid signal. It isn't — I misread
// exactly this and briefly concluded Deep cuts was the worst surface, when
// over 180 days it's one of the best (#2495). So the week's own play count
// comes back with the rates and is rendered beside them.
function latest(s: TrendSeries): { skip: number; completion: number; plays: number } {
const last = s.points[s.points.length - 1];
return last
? { skip: last.skip_rate, completion: last.avg_completion, plays: last.plays }
: { skip: 0, completion: 0, plays: 0 };
}
function pct(v: number): string {
return `${(v * 100).toFixed(0)}%`;
}
function markerSummary(m: TrendMarker): string {
const when = m.changed_at.slice(0, 10);
if (m.action === 'reset') return `${when}${m.scope} reset to defaults`;
const fields = (m.changes ?? []).map((c) => `${c.field} ${c.old}${c.new}`).join(', ');
return `${when}${m.scope}: ${fields}`;
}
</script>
<svelte:head>
<title>{pageTitle('Tuning')}</title>
</svelte:head>
<div class="space-y-6 p-4">
<div>
<h1 class="text-xl font-semibold">Recommendation tuning</h1>
<p class="mt-1 max-w-3xl text-sm text-text-secondary">
The defaults-discovery lab. Changes apply live — radio on the next request, daily mixes on
the next rebuild — and every change is recorded so the metrics page can tie outcome shifts
to knob turns. Found-good values get baked into shipped defaults; a dot marks knobs that
currently deviate from them.
</p>
</div>
{#if loadFailed}
<p class="text-sm text-text-secondary">Couldn't load tuning settings.</p>
{:else if !snapshot}
<p class="text-sm text-text-secondary">Loading…</p>
{:else}
<section class="grid gap-4 lg:grid-cols-2">
{#each profileScopes as p (p.scope)}
<div class="space-y-3 rounded border border-border bg-surface p-4">
<div>
<h2 class="text-lg font-semibold">{p.label}</h2>
<p class="text-xs text-text-secondary">{p.blurb}</p>
</div>
<div class="space-y-2">
{#each weightFields as f (f.key)}
<div class="grid grid-cols-[1fr_7rem] items-center gap-2">
<label class="text-sm" for="{p.scope}-{f.key}" title={f.hint}>
{f.label}
{#if deviates(p.scope, f.key)}
<span
class="ml-1 inline-block h-1.5 w-1.5 rounded-full bg-accent align-middle"
title="Deviates from the shipped default ({snapshot.shipped.profiles[p.scope][f.key]})"
></span>
{/if}
</label>
<input
id="{p.scope}-{f.key}"
type="number"
step="0.1"
bind:value={form[p.scope][f.key]}
class="w-full rounded border border-border bg-background px-2 py-1 text-right text-sm tabular-nums outline-none focus:border-accent"
/>
</div>
{/each}
</div>
<div class="flex gap-2">
<button
type="button"
disabled={saving !== null}
onclick={() => save(p.scope)}
class="rounded bg-accent px-3 py-1.5 text-sm text-white disabled:opacity-50"
>
Save {p.label.toLowerCase()}
</button>
<button
type="button"
disabled={saving !== null}
onclick={() => reset(p.scope)}
aria-label="Reset {p.label} to defaults"
class="rounded border border-border px-3 py-1.5 text-sm text-text-secondary hover:text-text-primary disabled:opacity-50"
>
Reset to defaults
</button>
</div>
</div>
{/each}
</section>
<section class="space-y-3 rounded border border-border bg-surface p-4 lg:max-w-xl">
<div>
<h2 class="text-lg font-semibold">Taste profile build</h2>
<p class="text-xs text-text-secondary">
How plays become the learned taste profile: the influence half-life and the
completion→engagement curve (must stay ordered: hard-skip &lt; neutral &lt; full).
</p>
</div>
<div class="space-y-2">
{#each tasteFields as f (f.key)}
<div class="grid grid-cols-[1fr_7rem] items-center gap-2">
<label class="text-sm" for="taste-{f.key}" title={f.hint}>
{f.label}
{#if deviates('taste', f.key)}
<span
class="ml-1 inline-block h-1.5 w-1.5 rounded-full bg-accent align-middle"
title="Deviates from the shipped default ({snapshot.shipped.taste[f.key]})"
></span>
{/if}
</label>
<input
id="taste-{f.key}"
type="number"
step={f.key === 'half_life_days' ? '1' : '0.05'}
bind:value={form.taste[f.key]}
class="w-full rounded border border-border bg-background px-2 py-1 text-right text-sm tabular-nums outline-none focus:border-accent"
/>
</div>
{/each}
</div>
<div class="flex gap-2">
<button
type="button"
disabled={saving !== null}
onclick={() => save('taste')}
class="rounded bg-accent px-3 py-1.5 text-sm text-white disabled:opacity-50"
>
Save taste
</button>
<button
type="button"
disabled={saving !== null}
onclick={() => reset('taste')}
aria-label="Reset taste profile build to defaults"
class="rounded border border-border px-3 py-1.5 text-sm text-text-secondary hover:text-text-primary disabled:opacity-50"
>
Reset to defaults
</button>
</div>
</section>
<!-- Discover request surface (#2377). Its own scope, not part of taste:
the snooze length lives here, and a snooze deliberately carries no
taste signal (#2374). -->
<section class="space-y-3 rounded border border-border bg-surface p-4 lg:max-w-xl">
<div>
<h2 class="text-lg font-semibold">Discover requests</h2>
<p class="text-xs text-text-secondary">
How the Discover suggestion deck ranks out-of-library artists. Tag coverage for
artists you don't own is partial by nature, so an untagged candidate keeps its
similarity score rather than being pushed down.
</p>
</div>
<div class="space-y-2">
{#each discoverFields as f (f.key)}
<div class="grid grid-cols-[1fr_7rem] items-center gap-2">
<label class="text-sm" for="discover-{f.key}" title={f.hint}>
{f.label}
{#if deviates('discover', f.key)}
<span
class="ml-1 inline-block h-1.5 w-1.5 rounded-full bg-accent align-middle"
title="Deviates from the shipped default ({snapshot.shipped.discover[f.key]})"
></span>
{/if}
</label>
<input
id="discover-{f.key}"
type="number"
step={f.key === 'snooze_days' ? '1' : '0.1'}
min="0"
bind:value={form.discover[f.key]}
class="w-full rounded border border-border bg-background px-2 py-1 text-right text-sm tabular-nums outline-none focus:border-accent"
/>
</div>
{/each}
</div>
<div class="flex gap-2">
<button
type="button"
disabled={saving !== null}
onclick={() => save('discover')}
class="rounded bg-accent px-3 py-1.5 text-sm text-white disabled:opacity-50"
>
Save discover
</button>
<button
type="button"
disabled={saving !== null}
onclick={() => reset('discover')}
aria-label="Reset discover requests to defaults"
class="rounded border border-border px-3 py-1.5 text-sm text-text-secondary hover:text-text-primary disabled:opacity-50"
>
Reset to defaults
</button>
</div>
</section>
{/if}
<!-- Weekly trends (#1251): the verify half. Sparklines share one
week axis; dashed ticks mark knob turns so cause→effect reads
off the chart. -->
<section class="space-y-3 rounded border border-border bg-surface p-4">
<div>
<h2 class="text-lg font-semibold">Weekly trends</h2>
<p class="text-xs text-text-secondary">
Skip rate per surface over the last {trends?.weeks ?? 12} weeks (lower is better; all
users aggregated, rates only). Dashed ticks mark tuning changes. Taste hit is the share
of plays whose artist fits the current taste profile.
<span class="font-medium">The skip and completion columns show the most recent week
alone</span>, not the whole window — the figure after the skip rate is that week's
play count, so a rate drawn from a handful of listens reads as what it is.
</p>
</div>
{#if trendsFailed}
<p class="text-sm text-text-secondary">Couldn't load trends.</p>
{:else if !trends}
<p class="text-sm text-text-secondary">Loading…</p>
{:else if trends.series.length === 0}
<p class="text-sm text-text-secondary">
No plays recorded yet — trends appear once listening accumulates.
</p>
{:else}
<table class="w-full text-sm">
<thead>
<tr class="text-left text-text-secondary">
<th class="py-1 font-medium">Surface</th>
<th class="py-1 font-medium">Skip rate by week</th>
<th class="py-1 text-right font-medium">Plays<span class="font-normal text-xs"> (window)</span></th>
<th class="py-1 text-right font-medium">Skip<span class="font-normal text-xs"> (last wk)</span></th>
<th class="py-1 text-right font-medium">Completion<span class="font-normal text-xs"> (last wk)</span></th>
<th class="py-1 text-right font-medium">Taste hit<span class="font-normal text-xs"> (window)</span></th>
</tr>
</thead>
<tbody>
{#each trends.series as s (s.key)}
<tr
class="border-t border-border"
class:opacity-60={s.plays < TREND_LOW_VOLUME}
title={s.plays < TREND_LOW_VOLUME
? 'Fewer than 20 plays in the window — treat as anecdote, not signal.'
: undefined}
>
<td class="py-1.5 pr-2">{s.label}</td>
<td class="py-1.5">
<svg
width={SPARK_W}
height={SPARK_H}
viewBox="0 0 {SPARK_W} {SPARK_H}"
role="img"
aria-label="{s.label} weekly skip rate"
data-testid="sparkline-{s.key}"
>
<line x1="0" y1={SPARK_H - 0.5} x2={SPARK_W} y2={SPARK_H - 0.5}
stroke="currentColor" opacity="0.15" />
{#each trends.markers as m, i (i)}
<line
x1={markerX(m)} y1="0" x2={markerX(m)} y2={SPARK_H}
stroke="currentColor" opacity="0.35" stroke-dasharray="2,2"
>
<title>{markerSummary(m)}</title>
</line>
{/each}
{#if s.points.length > 1}
<polyline
class="text-accent"
points={sparkPoints(s)}
fill="none"
stroke="currentColor"
stroke-width="1.5"
/>
{:else if s.points.length === 1}
<circle
class="text-accent"
cx={xFor(s.points[0].week_start)}
cy={SPARK_H - s.points[0].skip_rate * SPARK_H}
r="2" fill="currentColor"
/>
{/if}
</svg>
</td>
<td class="py-1.5 text-right tabular-nums">{s.plays}</td>
<td class="py-1.5 text-right tabular-nums">
{pct(latest(s).skip)}
<span class="text-xs text-text-secondary">/{latest(s).plays}</span>
</td>
<td class="py-1.5 text-right tabular-nums">{pct(latest(s).completion)}</td>
<td class="py-1.5 text-right tabular-nums">{pct(windowTasteHitRate(s))}</td>
</tr>
{/each}
</tbody>
</table>
{#if trends.markers.length > 0}
<div class="space-y-1">
<h3 class="text-sm font-medium">Tuning changes in this window</h3>
<ul class="space-y-0.5 text-xs text-text-secondary">
{#each trends.markers as m, i (i)}
<li>{markerSummary(m)}</li>
{/each}
</ul>
</div>
{/if}
{/if}
</section>
</div>