Recommendation relevance, the rollback unit, and a version that names what shipped #131
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: bvandeusen/minstrel#131
Reference in New Issue
Block a user
Eleven commits in three strands. CI green on
4ce47397across every lane, integration included.Recommendation relevance (milestone #398)
Operator report, 2026-09-10: "the songs like section algorithm is starting to feel weird", a radio session that came back entirely from one artist, and a hypothesis — "is the pool that we draw from somehow scaled to the amount of music in the library". All three were right.
Songs-like wandered (#3881) because it shared the
daily_mixweight profile with For-You, and the two surfaces want opposite things. Under the shared weights an unrelated track you'd liked and not played recently scored1.0 + 2.0 + 1.0 = 4.0, while a perfect similarity match you hadn't liked scored1.0 + 1.5 = 2.5. Liking something outranked sounding like the seed. It now has its own DB-backed profile where similarity outranges every seed-independent term combined: the same pair scores 5.00 vs 2.00. The seed artist's own music is eligible too — bounded at 3 of 25 by the diversity cap rather than excluded, since those are usually the seed track's nearest neighbours.Radio had no per-artist cap (#3882) while
discover.go,you_might_like.goandhome.goall did. On the fixture that reproduces the report, 10 tracks from 1 artist becomes 10 from 8. Two-pass selection, so the caps change which tracks are picked and never how many — a hard cap would have returned a six-track "radio" for a fifty-track request.The pool didn't scale (#3880).
DefaultCandidateSourceLimitsreturned what its own comment called "the v1 hardcoded constants per spec" — ~170 candidates for a 500-track library and a 100,000-track one alike, so the pool sampled a shrinking fraction of a growing collection. Now sqrt-scaled per arm, bounded at 4x, and only for arms that library size can actually help.Release machinery
Image tags follow family rules 145 and 147 —
dev→:dev,main→:latest + :<sha>, tag →:latest. No:<version>, no:main. The:<sha>is new and is the rollback unit; this repo previously had none.The version names what shipped.
ci/version.shderived from bareHEAD, so a CI-only or docs-only commit moved it. Now a denylist pathspec over the shipped file set, tests included — they don't ship, so they must not re-version an artifact..dockerignoreexcluded the wrong CI directory (.forgejo/and.github/, neither of which exists here) and had lost the pattern for the mobile client. Build context: 42.66 MB → 21.71 MB. An 18 MB binary was also tracked; it isn't now.What to expect on
mainThe rebundle path is exercised for the first time under the new tag map, and the tag build will publish the first
:<sha>rollback image. Scribe #3890 hasci-runner-f1watching this merge forandroid.yml's debug-APK artifact.Carrying caveats forward
The recommendation numbers are defensible starting points, not measured optima — spike #3879 still gates them, and
libraryScaleReference = 5000is an assumption about a library size nobody recorded. Two known defects stay open:likes_overlapassigns a flat0.6similarity it never computed (#3879), and four candidate arms order by unseededrandom()so same-day determinism is accidental (#3889).🤖 Generated with Claude Code
https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH
Three build-hygiene fixes that turned up while explaining the pathspec. **version.sh derives from what SHIPPED.** It read bare HEAD, so any commit moved the version — including one touching only CI or a README. Rules 148 and 149 both specify the pathspec form. Now a denylist, and the direction is the point: as an allowlist the list must be updated by whoever adds a directory and nothing fails if they don't, so the failure mode is a changed artifact keeping its old version silently on a green run. Inverted, new content counts by default. android/ is deliberately NOT excluded, and that is the subtle part. This repo ships TWO artifacts from ONE derivation: android/ is in no server image, but it is the APK's entire source, and excluding it would stop an Android-only commit from moving the APK's own version — the silent downgrade the versioning rework exists to prevent. So the list is the union: exclude only what ships in neither, and accept that an Android commit also nudges the server's reported version. Over-inclusion across the two, which is the harmless direction. roundtable/roundtable-android each keep tighter lists because they are one-artifact repos; don't copy theirs. **.dockerignore excluded the wrong CI directory.** It named .forgejo/ and .github/, neither of which this repo has. Gitea Actions reads .gitea/, so the one directory that exists was the one not excluded. The "Flutter mobile client" block had also lost its PATTERN when flutter_client/ was deleted, leaving a comment describing an exclusion that was not happening — android/ never took its place, so 4.1MB of Gradle project entered the context and busted the `COPY . .` layer on every Android-only change. bin/ excluded too. **bin/minstrel was tracked** — an 18MB binary last refreshed by a commit about web test mocks, and re-dirtied by every `make build` since. Untracked and ignored; the file stays on disk. Guards are behavioural rather than textual: they build throwaway repos with pinned commit timestamps and run version.sh against them, so they break when the derivation changes rather than when the wording does. Falsified — drop the .gitea exclusion and the CI-only commit moves the version; add an android exclusion and an Android commit stops moving it; exclude everything and a source commit refuses. One honest note on the refusal test: the script already refused an empty result via the downstream date check, so the new explicit check improves the diagnostic ("no commit touches the shipped file set — shallow clone?") and not the safety. The test pins the property, which is defended in depth. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLHOperator, 2026-09-10: "when I play it I'm expecting to get a consistent sound and style from the experience... I was getting a seeming wide variety of music from each one when I was hoping to stay in a certain neighborhood." Songs-like shared the `daily_mix` weight profile with For-You, and that sharing WAS the bug. The two surfaces want opposite things: For-You answers "what will they enjoy today" and is supposed to roam; Songs-like answers "what sounds like THIS". Under one profile the broad answer wins. The arithmetic, from the shared weights: unrelated track, liked, not played recently → 1.0 + 2.0 + 1.0 = 4.0 PERFECT similarity match, not liked → 1.0 + 1.5 = 2.5 Liking something outranked sounding like the seed, because LikeBoost (2.0) exceeded SimilarityWeight's whole range (1.5) and TasteWeight (1.5, and seed-INDEPENDENT) matched it outright. Under the new profile the same pair scores 5.00 vs 2.00. Two levers, because either alone leaves the other's failure intact: POOL. Songs-like now takes its own CandidateSourceLimits. The default gave ~29% of candidates a sim_score of literally zero — `taste_overlap` and `random_fill` are both `0.0::float8` in recommendation.sql, seed-independent by construction. Same total pool size; composition shifts to arms that measure distance from the seed, LBSimilar doubled. WEIGHTS. A third profile beside radio and daily_mix, DB-backed and live per rule 25, with the property that similarity's range exceeds the combined range of every seed-independent differentiator — so a closer match cannot be beaten on likes, freshness and taste alone, while tracks within ~0.39 similarity of each other still get ordered by what the user likes. Rule 131 changed the pool design mid-way and for the better. Zeroing the two seed-independent arms was the first instinct and is exactly the vanish-or-nothing shape that rule forbids: a seed with thin ListenBrainz coverage would yield a short mix or none. They are the tier-3 FLOOR — cut hard, never removed — and the weights keep them at the bottom of the ranking rather than out of the pool. "A few tracks further from the seed than we'd like" beats "no playlist". Caught while wiring it: switching only pickTopN's final Score would have been nearly INERT. scoreAndSortCandidates does the selection sort, and the caller caps and truncates in that order — so the playlist would still have been chosen by daily_mix and merely relabelled with songs_like numbers. It now takes the profile as a parameter, and each surface passes its own. Also corrects the daily_mix card's blurb, which claimed Songs-like as one of its surfaces and no longer is. Guards pin behaviour rather than the numbers, since numbers get retuned: that similarity beats an unrelated liked track, that daily_mix still DOESN'T (or the split buys nothing), that the tier-3 floor is non-zero, and that the UI card shows its own values rather than falling back. Each falsified against its named regression first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLHOperator, 2026-09-10: "is the pool that we draw from somehow scaled to the amount of music in the library... my earlier understanding of the tuning and work may have been skewed by what was in my library." It was not. DefaultCandidateSourceLimits returns what its own comment calls "the v1 hardcoded constants per spec" — ~170 candidates for a 500-track library and a 100,000-track one alike. The pool therefore samples a shrinking FRACTION of a growing collection: 17% of 1,000 tracks, 1.7% of 10,000, 0.17% of 100,000. RandomFill, whose whole job is exploration, becomes a thinner and noisier slice at exactly the moment a library gets more diverse — which is the "starting to feel weird" being reported. 1,000 tracks -> pool 170 (unchanged) 5,000 -> pool 170 (unchanged) 20,000 -> pool 280 80,000 -> pool 500 (ceiling) THE SCALING IS PER-ARM, and that is the substance rather than a refinement. A limit only matters if there are rows for it to cut off, so what an arm is BOUNDED BY decides whether library size can help it. LBSimilar, SimilarArtist, TagOverlap and RandomFill grow: they are bounded by similarity/tag data and by the library itself. LikesOverlap, UserCoplay and TasteOverlap do not: they are bounded by the user's likes, the instance's co-play graph and the taste profile, none of which grow when the library does. Raising those would sample more of a set that did not change — churn, not reach. It also keeps this from inflating the sim_score-0 share, since TasteOverlap is one of the two zero-similarity arms. sqrt, not linear: linear would put a 100,000-track library at a 3,400-candidate pool, long past where more candidates improve the answer. A 4x ceiling bounds it at ~500. Never shrinks an arm. The base limits are a floor, and #3889 makes that load-bearing rather than tidy — shrinking an arm ordered by unseeded random() changes pool membership between same-day rebuilds. Library size comes from a TTL-cached count reusing CountTracksMatching with an empty pattern (rule 28 — a new query would need sqlc regeneration, which is blocked). The ILIKE defeats every index, so it is a full scan and must not run per request. It degrades rather than fails: an error keeps the last known value, a never-counted cache returns 0, and 0 scales to the base limits — today's behaviour exactly. Nothing about sizing a pool justifies failing the request it is sizing. Bounded by a 3s deadline (rule 156), and a failed refresh does not stamp the clock, so a blip cannot pin a stale value for the whole TTL. THE REFERENCE IS ASSUMED, NOT MEASURED. libraryScaleReference = 5000 is where growth starts, and the size the v1 constants were really tuned against is unrecorded. #3879 should replace it; until then that constant is the one thing to change. Deliberately conservative: below it nothing scales at all, so no existing install changes behaviour. Falsification caught a weak guard: the sqrt-vs-linear assertion was written at SIXTEEN times the reference, where linear has already been clamped by the ceiling and both curves land on 4x. It proved nothing. Moved to four times the reference, below the ceiling for both, where sqrt gives 2x and linear would give 4x. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH