Files
minstrel/web/src/routes/admin/tuning/tuning.test.ts
T
bvandeusenandClaude Opus 5 f367eeaa9d
test-web / test (push) Successful in 1m7s
test-go / test (push) Successful in 1m31s
test-go / integration (push) Failing after 4m21s
release / Build signed APK (releases and dev) (push) Successful in 5m11s
release / Build + push container image (push) Successful in 1m52s
release / Verify release artifacts (tag releases only) (push) Skipped
fix(recommendation): Songs-like gets its own profile so it stops wandering
Operator, 2026-09-10: "when I play it I'm expecting to get a consistent
sound and style from the experience... I was getting a seeming wide variety
of music from each one when I was hoping to stay in a certain neighborhood."

Songs-like shared the `daily_mix` weight profile with For-You, and that
sharing WAS the bug. The two surfaces want opposite things: For-You answers
"what will they enjoy today" and is supposed to roam; Songs-like answers
"what sounds like THIS". Under one profile the broad answer wins.

The arithmetic, from the shared weights:

    unrelated track, liked, not played recently → 1.0 + 2.0 + 1.0 = 4.0
    PERFECT similarity match, not liked         → 1.0 + 1.5       = 2.5

Liking something outranked sounding like the seed, because LikeBoost (2.0)
exceeded SimilarityWeight's whole range (1.5) and TasteWeight (1.5, and
seed-INDEPENDENT) matched it outright. Under the new profile the same pair
scores 5.00 vs 2.00.

Two levers, because either alone leaves the other's failure intact:

POOL. Songs-like now takes its own CandidateSourceLimits. The default gave
~29% of candidates a sim_score of literally zero — `taste_overlap` and
`random_fill` are both `0.0::float8` in recommendation.sql, seed-independent
by construction. Same total pool size; composition shifts to arms that
measure distance from the seed, LBSimilar doubled.

WEIGHTS. A third profile beside radio and daily_mix, DB-backed and live per
rule 25, with the property that similarity's range exceeds the combined
range of every seed-independent differentiator — so a closer match cannot
be beaten on likes, freshness and taste alone, while tracks within ~0.39
similarity of each other still get ordered by what the user likes.

Rule 131 changed the pool design mid-way and for the better. Zeroing the
two seed-independent arms was the first instinct and is exactly the
vanish-or-nothing shape that rule forbids: a seed with thin ListenBrainz
coverage would yield a short mix or none. They are the tier-3 FLOOR — cut
hard, never removed — and the weights keep them at the bottom of the
ranking rather than out of the pool. "A few tracks further from the seed
than we'd like" beats "no playlist".

Caught while wiring it: switching only pickTopN's final Score would have
been nearly INERT. scoreAndSortCandidates does the selection sort, and the
caller caps and truncates in that order — so the playlist would still have
been chosen by daily_mix and merely relabelled with songs_like numbers. It
now takes the profile as a parameter, and each surface passes its own.

Also corrects the daily_mix card's blurb, which claimed Songs-like as one
of its surfaces and no longer is.

Guards pin behaviour rather than the numbers, since numbers get retuned:
that similarity beats an unrelated liked track, that daily_mix still
DOESN'T (or the split buys nothing), that the tier-3 floor is non-zero,
and that the UI card shows its own values rather than falling back. Each
falsified against its named regression first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
2026-09-10 20:51:15 -04:00

326 lines
14 KiB
TypeScript

import { describe, expect, test, vi, beforeEach } from 'vitest';
import { render, screen, fireEvent, waitFor } from '@testing-library/svelte';
import type { TuningSnapshot, TrendsResponse } from '$lib/api/tuning';
vi.mock('$lib/api/tuning', async (orig) => {
const actual = (await orig()) as Record<string, unknown>;
return {
...actual,
getTuning: vi.fn(),
patchTuning: vi.fn(),
resetTuning: vi.fn(),
getTrends: vi.fn()
};
});
const pushToast = vi.fn();
vi.mock('$lib/stores/toast.svelte', () => ({ pushToast: (...a: unknown[]) => pushToast(...a) }));
import TuningPage from './+page.svelte';
import { getTuning, patchTuning, resetTuning, getTrends } from '$lib/api/tuning';
const weights = (over: Partial<Record<string, number>> = {}) => ({
base_weight: 1,
like_boost: 2,
recency_weight: 1,
skip_penalty: 2,
jitter_magnitude: 0.1,
context_weight: 0.5,
similarity_weight: 1.5,
taste_weight: 1.5,
context_time_weight: 1,
...over
});
const taste = (over: Partial<Record<string, number>> = {}) => ({
half_life_days: 75,
engagement_hard_skip: 0.05,
engagement_neutral: 0.3,
engagement_full: 0.9,
enriched_tag_scale: 0.5,
era_scale: 0.5,
mood_scale: 0.5,
...over
});
const discover = (over: Partial<Record<string, number>> = {}) => ({
tag_overlap_weight: 1,
snooze_days: 90,
...over
});
// NOTE the `as TuningSnapshot` cast below: it silences the type error when the
// server snapshot grows a field, so a missing key here surfaces as every test
// on this page throwing inside fillForm rather than as a compile failure. Add
// new scopes to BOTH this fixture and `shipped`.
function snapshot(over: Partial<TuningSnapshot> = {}): TuningSnapshot {
return {
profiles: {
radio: weights({ taste_weight: 1 }),
daily_mix: weights(),
songs_like: weights({ similarity_weight: 4, like_boost: 0.5, taste_weight: 0.25 })
},
taste: taste(),
discover: discover(),
shipped: {
profiles: {
radio: weights({ taste_weight: 1 }),
daily_mix: weights(),
songs_like: weights({ similarity_weight: 4, like_boost: 0.5, taste_weight: 0.25 })
},
taste: taste(),
discover: discover()
},
...over
} as TuningSnapshot;
}
const emptyTrends: TrendsResponse = { weeks: 12, series: [], markers: [] };
beforeEach(() => {
vi.clearAllMocks();
(getTrends as ReturnType<typeof vi.fn>).mockResolvedValue(emptyTrends);
});
describe('Admin tuning page', () => {
test('renders every weight profile and the taste card with current values', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Radio')).toBeInTheDocument());
expect(screen.getByText('Daily mixes')).toBeInTheDocument();
expect(screen.getByText('Songs like…')).toBeInTheDocument();
expect(screen.getByText('Taste profile build')).toBeInTheDocument();
const radioTaste = screen.getByLabelText(/taste weight/i, {
selector: '#radio-taste_weight'
}) as HTMLInputElement;
expect(radioTaste.value).toBe('1');
const halfLife = document.getElementById('taste-half_life_days') as HTMLInputElement;
expect(halfLife.value).toBe('75');
});
// Songs-like is a separate weight profile precisely so it can be tuned
// apart from For-You (#3881). If its card stopped rendering its OWN values
// — or quietly fell back to daily_mix's — the split would exist in the
// backend and be unreachable in the UI, which is the same as not shipping
// it (rule 27).
test('the songs-like card carries its own weights, not daily_mix\'s', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Songs like…')).toBeInTheDocument());
const sim = document.getElementById('songs_like-similarity_weight') as HTMLInputElement;
const like = document.getElementById('songs_like-like_boost') as HTMLInputElement;
expect(sim.value).toBe('4');
expect(like.value).toBe('0.5');
// The contrast that makes the card worth having: daily_mix must still show
// its own, different numbers on the same page.
const dailySim = document.getElementById('daily_mix-similarity_weight') as HTMLInputElement;
expect(dailySim.value).not.toBe(sim.value);
});
test('save sends only the changed fields for the scope', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
(patchTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Radio')).toBeInTheDocument());
const input = document.getElementById('radio-taste_weight') as HTMLInputElement;
await fireEvent.input(input, { target: { value: '2.5' } });
await fireEvent.click(screen.getByRole('button', { name: /save radio/i }));
await waitFor(() =>
expect(patchTuning).toHaveBeenCalledWith('radio', { taste_weight: 2.5 })
);
});
test('save with no changes calls nothing and says so', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Radio')).toBeInTheDocument());
await fireEvent.click(screen.getByRole('button', { name: /save radio/i }));
expect(patchTuning).not.toHaveBeenCalled();
await waitFor(() => expect(pushToast).toHaveBeenCalledWith('Nothing changed.'));
});
// Targets the button by its accessible name, not its position. This used to
// click the LAST reset button and assume that meant taste — which silently
// became the wrong scope the moment a card was added below it.
test('reset calls resetTuning for the scope', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
(resetTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Taste profile build')).toBeInTheDocument());
await fireEvent.click(
screen.getByRole('button', { name: /reset taste profile build to defaults/i })
);
await waitFor(() => expect(resetTuning).toHaveBeenCalledWith('taste'));
});
test('marks knobs that deviate from shipped defaults', async () => {
const snap = snapshot();
snap.profiles.daily_mix = weights({ similarity_weight: 4 });
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snap);
render(TuningPage);
await waitFor(() => expect(screen.getByText('Daily mixes')).toBeInTheDocument());
expect(
screen.getByTitle(/deviates from the shipped default \(1\.5\)/i)
).toBeInTheDocument();
});
test('renders weekly trend rows with sparklines and knob-turn markers (#1251)', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
(getTrends as ReturnType<typeof vi.fn>).mockResolvedValue({
weeks: 12,
series: [
{
key: 'radio',
label: 'Radio',
intent: 'go_to',
plays: 40,
points: [
{
week_start: '2026-06-22',
plays: 25,
skips: 5,
skip_rate: 0.2,
avg_completion: 0.8,
taste_hit_rate: 0.6
},
{
week_start: '2026-06-29',
plays: 15,
skips: 6,
skip_rate: 0.4,
avg_completion: 0.7,
taste_hit_rate: 0.5
}
]
},
{
key: 'discover',
label: 'Discover',
intent: 'discovery',
plays: 5,
points: [
{
week_start: '2026-06-29',
plays: 5,
skips: 3,
skip_rate: 0.6,
avg_completion: 0.4,
taste_hit_rate: 0.2
}
]
}
],
markers: [
{
changed_at: '2026-06-30T10:00:00Z',
scope: 'radio',
action: 'update',
changes: [{ field: 'taste_weight', old: 1, new: 2 }]
}
]
} satisfies TrendsResponse);
render(TuningPage);
await waitFor(() => expect(screen.getByText('Weekly trends')).toBeInTheDocument());
expect(screen.getByTestId('sparkline-radio')).toBeInTheDocument();
expect(screen.getByTestId('sparkline-discover')).toBeInTheDocument();
// The skip column is the LAST WEEK's rate, now carrying that week's play
// count so a rate off a handful of plays reads as what it is (#2495).
// Radio's latest week: 40% skip over 15 plays; Discover's: 60% over 5.
expect(screen.getByText('/15')).toBeInTheDocument();
expect(screen.getByText('/5')).toBeInTheDocument();
// Completion columns are unchanged and still bare percentages — radio 70%.
expect(screen.getByText('70%')).toBeInTheDocument();
// '40%' is now genuinely ambiguous: radio's latest SKIP rate and discover's
// latest COMPLETION are both 40%. testing-library matches an element's own
// direct text nodes, so the skip cell still matches despite its trailing
// play-count span. Assert the count rather than pretending it's unique.
expect(screen.getAllByText('40%')).toHaveLength(2);
// The window/last-week distinction has to be visible in the headers, or the
// Plays total reads as the denominator of the skip rate. That misreading is
// what #2495 was filed over.
//
// Queried as column headers rather than by text: the caption below also
// mentions "Plays", and matching on the word finds the prose too.
// Exact accessible names: /^Skip/ also matches the "Skip rate by week"
// sparkline column, and /Plays/ matched the caption prose before that.
expect(screen.getByRole('columnheader', { name: 'Plays (window)' })).toBeInTheDocument();
expect(screen.getByRole('columnheader', { name: 'Skip (last wk)' })).toBeInTheDocument();
expect(screen.getByRole('columnheader', { name: 'Completion (last wk)' })).toBeInTheDocument();
// The knob turn is listed under the chart AND tooltipped on each
// sparkline's marker tick, hence getAllBy.
expect(
screen.getAllByText(/2026-06-30 — radio: taste_weight 1→2/).length
).toBeGreaterThan(0);
// Low-volume series (5 plays) is dimmed as anecdote.
expect(screen.getByTitle(/fewer than 20 plays in the window/i)).toBeInTheDocument();
});
test('empty trends show the accumulation hint', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() =>
expect(screen.getByText(/trends appear once listening accumulates/i)).toBeInTheDocument()
);
});
// --- Discover scope (#2377) ---
test('renders the Discover card with its two knobs', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Discover requests')).toBeInTheDocument());
expect(screen.getByLabelText(/taste-tag weight/i)).toBeInTheDocument();
expect(screen.getByLabelText(/snooze length/i)).toBeInTheDocument();
});
test('saving Discover patches only the changed field under the discover scope', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
(patchTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Discover requests')).toBeInTheDocument());
await fireEvent.input(screen.getByLabelText(/taste-tag weight/i), {
target: { value: '2.5' }
});
await fireEvent.click(screen.getByRole('button', { name: /save discover/i }));
await waitFor(() =>
expect(patchTuning).toHaveBeenCalledWith('discover', { tag_overlap_weight: 2.5 })
);
});
// 0 is the operator's off switch for the whole tag term, so the form must be
// able to send it — a falsy-value bug here would silently make the feature
// impossible to disable.
test('a Discover weight of 0 is sent, not dropped as falsy', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
(patchTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Discover requests')).toBeInTheDocument());
await fireEvent.input(screen.getByLabelText(/taste-tag weight/i), { target: { value: '0' } });
await fireEvent.click(screen.getByRole('button', { name: /save discover/i }));
await waitFor(() =>
expect(patchTuning).toHaveBeenCalledWith('discover', { tag_overlap_weight: 0 })
);
});
test('reset targets the discover scope', async () => {
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
(resetTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snapshot());
render(TuningPage);
await waitFor(() => expect(screen.getByText('Discover requests')).toBeInTheDocument());
await fireEvent.click(
screen.getByRole('button', { name: /reset discover requests to defaults/i })
);
await waitFor(() => expect(resetTuning).toHaveBeenCalledWith('discover'));
});
test('a deviating Discover knob is dotted', async () => {
const snap = snapshot();
snap.discover = discover({ tag_overlap_weight: 3 });
(getTuning as ReturnType<typeof vi.fn>).mockResolvedValue(snap);
render(TuningPage);
await waitFor(() => expect(screen.getByText('Discover requests')).toBeInTheDocument());
expect(screen.getByTitle(/deviates from the shipped default \(1\)/i)).toBeInTheDocument();
});
});