The worker kept only the similar recordings already in the library, at most
20 of ListenBrainz's 50, and judged freshness by the edges it had written.
Two failures followed, both measured on the operator's library (#3879):
- A seed whose answer matched nothing wrote nothing, so it was never fresh.
With the queue ordered by id, 25 such seeds held its head and were
re-asked every hour; 17 of 2,466 played seeds had any edges.
- A recording that reached the library after its seed was fetched (a
Lidarr import, an MBID from the AcoustID lookup) was never linked until
a refetch, which for the stuck seeds never came.
Now every answer is cached whole in listenbrainz_similar_recordings and
every answer, an empty one or a permanent 4xx included, is recorded in
track_similarity_fetches. The queue reads the fetch record: never-fetched
first, then the oldest, refreshed after 30 days. The listenbrainz edges are
derived in SQL from the cache, one present track per recording and no cap,
for the seed just fetched and for every seed once per tick, so new arrivals
link within the hour without asking ListenBrainz again.
Artists get the same queue fix via artist_similarity_fetches; their answer
was already kept in artist_similarity and artist_similarity_unmatched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Root cause of zero LB recommendations (every similar-recordings AND
similar-artists call returned HTTP 404, worker logs):
- Wrong host/path: client called
api.listenbrainz.org/1/explore/similar-{recordings,artists}/{mbid}.
/explore/... is a WEBSITE route, not an API endpoint — it 308s then
404s. Similarity datasets live on the separate Labs API.
- Invalid algorithm: the hardcoded
session_…_session_30_…_limit_100_filter_True_… is not a permitted
Labs enum member (400s) regardless of host.
Verified against the live Labs API:
GET labs.api.listenbrainz.org/similar-recordings/json
?recording_mbids=<mbid>&algorithm=<algo>
GET labs.api.listenbrainz.org/similar-artists/json
?artist_mbids=<mbid>&algorithm=<algo>
algorithm=session_based_days_9000_session_300_contribution_5_threshold_15_limit_50_skip_30
→ 200 for both. Response field names (recording_mbid/artist_mbid/
name/score) already match the existing structs — parsing unchanged.
- Add defaultLabsBaseURL + Client.LabsBaseURL (separate from the main
BaseURL; scrobble submission still uses api.listenbrainz.org).
- Drop the count/limit query param — result size is encoded in the
algorithm name (limit_50); caller still applies its own top-K.
- Tests: newTestClient sets LabsBaseURL; the two *_LimitParamSet tests
become *_MbidParamSet (assert the Labs path + mbid query param).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- similarity.Worker batch 5→25 (tracks AND artists per 1h tick). At
5/h a freshly-scrobbled library took days to build a usable
similarity pool; 25/h converges in hours, still well under
ListenBrainz rate limits (429 aborts the tick).
- scanrun Stage 2b track backfill now runs unbounded (-1) instead of
reusing the album backfill's 5000 staged cap. It's a one-time
whole-library heal with no progress UI; a cap just left tracks.mbid
partially NULL (3634/18056 after one pass) until several future
scans caught up, re-reading untagged files each time. One uncapped
pass converges; later scans only re-read the remaining NULL rows.
Album backfill keeps its 5000 cap (it has staged scan_runs UX).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
upsertArtistSimilar keeps the existing matched path (top-K rows into
artist_similarity) and adds a parallel unmatched-persist loop with the
same top-K cap. Empty-name rows are skipped — we can't render a
suggestion card without a name. Logs unmatched-side errors at WARN
without aborting the tick (mirrors the matched-path policy).
Replace stub with full tickOnce: drains played tracks/artists via
ListPlayedTracksNeedingSimilarity / ListPlayedArtistsNeedingSimilarity,
calls LB SimilarRecordings/SimilarArtists, filters to local library via
bulk MBID lookup, enforces top-K=20 by score, and upserts similarity
rows. 429 aborts the entire tick; other errors log-and-skip. Ten
integration tests covering no-op, library filtering, top-K cap, 7-day
freshness cap, stale refresh, 429 abort, transient skip, and artist
pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bvandeusenandClaude Opus 5.5