fix(server/playlists): redistributeSlots double-counting + tieBreakHash avalanche

Two real algorithm bugs in F-T1's For-You composition + Discover
allocator. Both surfaced as failing unit tests under go test -race.

1. redistributeSlots was re-redistributing a bucket's full deficit
   on every pass instead of just the residual. The loop computed
   `deficit = b.want - final[i]` each iteration, but final[i] for
   a deficit bucket never increases (its supply is exhausted), so
   pass N saw the same deficit as pass N-1 and kept shoveling it
   to peers. For [want:40 avail:100, want:30 avail:0, want:30 avail:100],
   four passes pushed cross-user's deficit into dormant+random four
   times each, hitting the 100-slot clamp at the end and producing
   [50, 0, 50] instead of the spec'd [55, 0, 45].

   Fix: track per-source `redistributed[i]` and subtract it from the
   deficit each pass. Multi-pass behavior still works for the case
   where a peer's supply runs out mid-distribution.

2. tieBreakHash used FNV-1a 64-bit with trackID + dateStr appended.
   For dateStrs differing only in the last character ("2026-05-07"
   vs "2026-05-08"), the FNV state diverged only in low bits at the
   final byte; multiplication by FNV_prime propagates upward but the
   relative ordering of 60 small candidate UUIDs (which differ only
   in their last byte) ended up identical across the two dates. The
   For-You head/tail test asserted that the tail's first 5 should
   change across days; it didn't.

   Fix: switch to SHA-256 truncated to 8 bytes. SHA-256 has full
   avalanche, so any single-bit input change roughly half-flips the
   output bits and meaningfully reorders.

The hash isn't security-load-bearing; we just need strong avalanche
for tiny dateStr deltas. Determinism (same inputs → same output) is
preserved.
This commit is contained in:
2026-05-07 18:33:06 -04:00
parent e4ebc71162
commit 042f9919fe
2 changed files with 34 additions and 16 deletions
+17 -8
View File
@@ -150,17 +150,24 @@ type bucketRequest struct {
// 1. Initial pass: allocate min(want, available) to each bucket;
// compute supply = available - allocated.
// 2. For each deficit bucket (allocated < want), split the deficit
// equally across peers that still have supply > 0.
// 3. Repeat until no further movement (deficit redistribution
// can create new deficits if the recipient was already at its
// `want`; we cap each bucket at its `available` after every pass).
// equally across peers that still have supply > 0. Track how
// much of that deficit each pass actually placed (some peers may
// not have enough supply for their full share).
// 3. Repeat until no movement happens — handles the case where a
// peer's supply ran out mid-distribution and the residual needs
// to roll to other peers in a later pass.
//
// For 3 buckets, two redistribution passes are sufficient in practice;
// we cap at 4 passes defensively.
// Bug history (May 2026): an earlier version recomputed
// `deficit = b.want - final[i]` every pass without subtracting
// previously-redistributed amounts, so a bucket with permanently
// 0 final and want=30 kept handing out 30 to peers each pass. The
// `redistributed` array below tracks per-source absorbed deficit so
// subsequent passes only move the residual.
func redistributeSlots(buckets []bucketRequest) []int {
n := len(buckets)
final := make([]int, n)
supply := make([]int, n)
redistributed := make([]int, n)
for i, b := range buckets {
final[i] = b.want
if final[i] > b.available {
@@ -172,11 +179,12 @@ func redistributeSlots(buckets []bucketRequest) []int {
for pass := 0; pass < 4; pass++ {
anyMoved := false
for i, b := range buckets {
deficit := b.want - final[i]
// Remaining deficit = original gap minus what we've already
// pushed out to peers in earlier passes.
deficit := b.want - final[i] - redistributed[i]
if deficit <= 0 {
continue
}
// Find peers with supply.
peers := make([]int, 0, n-1)
for j := 0; j < n; j++ {
if j == i {
@@ -202,6 +210,7 @@ func redistributeSlots(buckets []bucketRequest) []int {
if take > 0 {
final[j] += take
supply[j] -= take
redistributed[i] += take
anyMoved = true
}
}
+17 -8
View File
@@ -6,9 +6,10 @@ package playlists
import (
"context"
"crypto/sha256"
"encoding/binary"
"errors"
"fmt"
"hash/fnv"
"log/slog"
"sort"
"time"
@@ -42,19 +43,27 @@ func pickSeedArtistsFromRows(rows []seedArtistRow) []pgtype.UUID {
}
// tieBreakHash returns a deterministic 64-bit hash of (track_id, date_str).
// Used to break score ties in mix candidate ranking. Same inputs always
// produce the same output; different inputs almost always differ.
// Used to break score ties and to drive the For-You tail sample's
// daily-deterministic ordering. Same inputs always produce the same
// output; different inputs almost always differ.
//
// Uses FNV-1a 64-bit (stdlib hash/fnv) — fast, deterministic, no extra
// dependencies. The exact hash isn't load-bearing; any deterministic
// 64-bit hash works.
// Switched from FNV-1a (May 2026) because two date strings differing
// only in the last character (e.g. "2026-05-07" vs "2026-05-08") gave
// hash values whose RELATIVE ORDERING across our small candidate pools
// was identical — the dateStr-driven divergence concentrated in low
// bits that lost out to high-bit ordering during sort. SHA-256
// truncated to 8 bytes has full avalanche, so any single-byte input
// change roughly half-flips the output bits and reorders meaningfully.
// The hash isn't security-load-bearing; we just want strong avalanche
// for tiny input deltas.
func tieBreakHash(trackID pgtype.UUID, dateStr string) uint64 {
h := fnv.New64a()
h := sha256.New()
if trackID.Valid {
_, _ = h.Write(trackID.Bytes[:])
}
_, _ = h.Write([]byte(dateStr))
return h.Sum64()
sum := h.Sum(nil)
return binary.BigEndian.Uint64(sum[:8])
}
// rankedCandidate is a (track_id, score) pair used during in-memory