Similarity cache and round-robin artist arms (#5296, #5297) #142

Merged

bvandeusen merged 2 commits from dev into main 2026-10-07 20:03:00 -04:00

Conversation 0 Commits 2 Files Changed 12

+676 -207

bvandeusen commented 2026-10-07 20:02:55 -04:00

Owner

Copy Link

Copy Source

Two fixes from milestone 398, both measured on the live library in spike #3879.

• #5296: the similarity worker keeps ListenBrainz's whole similar-recordings answer (listenbrainz_similar_recordings) and records every fetch (track_similarity_fetches, artist_similarity_fetches).
  • The queue reads the fetch records: never-fetched first, then oldest, refreshed after 30 days. A seed that matches nothing no longer holds the head of the queue forever.
  • Edges are derived in SQL from the cache, for each seed after its fetch and for every seed once per tick. Recordings that arrive later, through Lidarr or AcoustID, link within the hour.
  • There is no longer a per-seed cap of 20.
  • Migration 0072.
• #5297: the similar_artists and coplay_artists arms take one track per related artist in turn, instead of filling from the top artist, and they skip missing tracks.

CI is green on e5dac9dd (run 8631): go, integration, web, android and govulncheck.

🤖 Generated with Claude Code

Two fixes from milestone 398, both measured on the live library in spike #3879. - **#5296**: the similarity worker keeps ListenBrainz's whole similar-recordings answer (`listenbrainz_similar_recordings`) and records every fetch (`track_similarity_fetches`, `artist_similarity_fetches`). - The queue reads the fetch records: never-fetched first, then oldest, refreshed after 30 days. A seed that matches nothing no longer holds the head of the queue forever. - Edges are derived in SQL from the cache, for each seed after its fetch and for every seed once per tick. Recordings that arrive later, through Lidarr or AcoustID, link within the hour. - There is no longer a per-seed cap of 20. - Migration 0072. - **#5297**: the similar_artists and coplay_artists arms take one track per related artist in turn, instead of filling from the top artist, and they skip missing tracks. CI is green on e5dac9dd (run 8631): go, integration, web, android and govulncheck. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 2 commits 2026-10-07 20:02:56 -04:00

fix(recommendation): artist-level arms take one track per related artist in turn (#5297) ... aa390c711a

The similar_artists and coplay_artists arms ordered by artist score, so the
closest related artist's catalogue filled the whole LIMIT. On the operator's
library the 30-row similar_artists arm held exactly one artist for all 17
seeds measured (#3879), though each seed had 7-33 similar artists in the
library. Both arms now rank tracks within each artist and take every
artist's first track, best artist first, before anyone's second.

Both also skip missing tracks: the outer select already dropped them, but
only after they had taken places in the arm's LIMIT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fix(similarity): keep ListenBrainz's whole answer and resolve it locally (#5296) ... e5dac9ddf0

The worker kept only the similar recordings already in the library, at most
20 of ListenBrainz's 50, and judged freshness by the edges it had written.
Two failures followed, both measured on the operator's library (#3879):

- A seed whose answer matched nothing wrote nothing, so it was never fresh.
  With the queue ordered by id, 25 such seeds held its head and were
  re-asked every hour; 17 of 2,466 played seeds had any edges.
- A recording that reached the library after its seed was fetched (a
  Lidarr import, an MBID from the AcoustID lookup) was never linked until
  a refetch, which for the stuck seeds never came.

Now every answer is cached whole in listenbrainz_similar_recordings and
every answer, an empty one or a permanent 4xx included, is recorded in
track_similarity_fetches. The queue reads the fetch record: never-fetched
first, then the oldest, refreshed after 30 days. The listenbrainz edges are
derived in SQL from the cache, one present track per recording and no cap,
for the seed just fetched and for every seed once per tick, so new arrivals
link within the hour without asking ListenBrainz again.

Artists get the same queue fix via artist_similarity_fetches; their answer
was already kept in artist_similarity and artist_similarity_unmatched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

bvandeusen merged commit a29ceaaeab into main 2026-10-07 20:03:00 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

deps

Dependency update (Renovate)

deps:major

Major dependency bump — held for manual review

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate-bot bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/minstrel#142