feat(discover): artist-tag cache for out-of-library candidates — #2376
test-go / test (push) Failing after 32s
test-go / integration (push) Successful in 4m50s

Migration 0050 adds candidate_artist_tags + candidate_artist_tag_state:
folksonomy tags for artists NOT in the library, which track_tags cannot
hold because it's FK'd to tracks(id) and a Discover candidate has no local
row. Slice 6 ranks against these; this slice only fills the cache.

The reuse the task claimed is real and verified: MusicBrainz's
fetchEntityTags(ctx, "artist", mbid, scale) already existed for the #1519
recording→artist fallback, so FetchArtistTags is a thin wrapper. Two
subtleties it does NOT inherit:

  - Weight scale is 1.0, not artistTagWeightFactor (0.6). That discount
    exists because FetchTrackTags uses artist tags as a *proxy* for a
    track's; here the artist IS the subject. Applying it would make these
    weights incomparable with track_tags — exactly the comparison slice 6
    depends on. Pinned by a test.
  - fetchEntityTags reports existing-but-untagged as (empty, nil) so the
    track path can fall through. There's no next level here, so empty
    becomes the terminal ErrNotFound; otherwise the enricher would settle
    a candidate as "enriched" with zero tags.

ArtistTagProvider is the split TrackTagProvider's own doc comment
anticipated ("e.g. artist-level tags"). Last.fm gains artist.getTopTags,
which returns the same toptags envelope, so the response type and
normalizer are reused unchanged.

Rather than write the merge-and-classify loop twice, extracted it from
EnrichTrack into runChain(). The ErrNotFound-vs-transient split is the
load-bearing part — those lead to opposite persistence decisions — so it
now has direct unit tests it never had while inlined.

Bookkeeping is a separate table, not columns, because the "providers had
nothing" outcome must be recordable for a candidate with zero tag rows,
and there is no per-candidate row to hang columns off (
artist_similarity_unmatched holds many rows per candidate). Absence of a
state row means "never processed", so a transient failure writes nothing
and stays eligible.

Two capacity realities are designed for, not papered over:
  - The pool is O(library artists x neighbours) and MusicBrainz allows
    ~1 req/s, so it can never drain in one pass. The eligibility query
    returns candidates in descending summed-similarity order, so the ones
    that can actually reach a deck are enriched first.
  - candidateBatch (50) is smaller than the track batch (200): tracks are
    finite and drain to completion, candidates are effectively unbounded
    and would otherwise starve the track arm forever.

GC sweeps both tables — the similarity feed churns, and a candidate that
joins the library has its tags in track_tags now. Tags swept before state
so a mid-sweep crash leaves a valid state, not a re-fetch loop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-02 20:04:15 -04:00
co-authored by Claude Opus 5
parent f17356560d
commit 4f9b083eec
16 changed files with 1387 additions and 41 deletions
@@ -0,0 +1,102 @@
-- Folksonomy tags for out-of-library Discover candidates (#2376). Parallel to
-- track_tags.sql, but keyed by MBID because the artist has no local row. See
-- 0050_candidate_artist_tags.up.sql for why the bookkeeping is its own table.
-- name: ListCandidateArtistsMissingTags :many
-- Candidates eligible for tag enrichment: never processed (no state row) or
-- settled 'none' under an older provider version.
--
-- artist_similarity_unmatched holds one row per (seed, candidate, source), so
-- this GROUPs to one row per candidate — enriching the same MBID once per seed
-- that pointed at it would multiply the API calls for no gain.
--
-- ORDER BY summed similarity DESC is the load-bearing part. The candidate pool
-- is O(library artists x neighbours per artist) — thousands — and MusicBrainz
-- allows ~1 req/s, so it can NEVER be fully enriched in one pass. Draining in
-- strength order means the candidates most likely to actually reach a user's
-- deck get tags first, and the long tail fills in over subsequent ticks
-- instead of starving behind it.
--
-- Already-in-library candidates are skipped: they have an artists row, so
-- their tags belong in track_tags, and the suggestion query filters them out
-- anyway. $1 = current tag_sources_version, $2 = limit.
-- candidate_name is coalesced to '' so it lands non-nullable in Go: the name is
-- only a Last.fm lookup key, and empty simply means "MBID-keyed providers only",
-- which the provider chain already handles. max() is an arbitrary-but-
-- deterministic pick when several seeds spell the same MBID differently.
SELECT u.candidate_mbid,
coalesce(max(u.candidate_name), '')::text AS candidate_name,
sum(u.score)::float8 AS total_score
FROM artist_similarity_unmatched u
LEFT JOIN candidate_artist_tag_state s ON s.candidate_mbid = u.candidate_mbid
WHERE NOT EXISTS (SELECT 1 FROM artists a WHERE a.mbid = u.candidate_mbid)
AND (
s.candidate_mbid IS NULL
OR (s.tag_source = 'none' AND s.tag_sources_version < $1)
)
GROUP BY u.candidate_mbid
ORDER BY total_score DESC, u.candidate_mbid
LIMIT $2;
-- name: DeleteCandidateArtistTags :exec
-- Clear a candidate's cached tags before rewriting (atomic replace by the
-- caller, same shape as DeleteTrackTags).
DELETE FROM candidate_artist_tags WHERE candidate_mbid = $1;
-- name: InsertCandidateArtistTag :exec
-- Upsert one (candidate, tag); keep the stronger weight when two providers
-- agree on a tag with different folksonomy strengths.
INSERT INTO candidate_artist_tags (candidate_mbid, tag, weight)
VALUES ($1, $2, $3)
ON CONFLICT (candidate_mbid, tag)
DO UPDATE SET weight = GREATEST(candidate_artist_tags.weight, EXCLUDED.weight);
-- name: SetCandidateArtistTagState :exec
-- Stamp the enrichment outcome so the drainer skips settled candidates.
-- $2 = 'musicbrainz' | 'lastfm' | 'mixed' | 'none', $3 = current version.
INSERT INTO candidate_artist_tag_state (candidate_mbid, tag_source, tag_sources_version)
VALUES ($1, $2, $3)
ON CONFLICT (candidate_mbid) DO UPDATE
SET tag_source = EXCLUDED.tag_source,
tag_sources_version = EXCLUDED.tag_sources_version,
updated_at = now();
-- name: ListCandidateArtistTagsForMbids :many
-- Cached tags for a set of candidates, for slice 6's taste-overlap ranking.
-- One row per (candidate, tag).
SELECT candidate_mbid, tag, weight
FROM candidate_artist_tags
WHERE candidate_mbid = ANY($1::text[]);
-- name: CountCandidateArtistTagCoverage :one
-- Operator-facing coverage: how many distinct candidates have been processed,
-- and how many of those actually yielded tags. The gap is the honest ceiling
-- from the task — obscure artists with no MBID presence or no upstream tags
-- stay thin no matter how long the worker runs, and that is worth being able
-- to see rather than inferring from a silent surface.
SELECT count(*)::bigint AS processed,
count(*) FILTER (WHERE tag_source <> 'none')::bigint AS with_tags
FROM candidate_artist_tag_state;
-- name: GcDeleteOrphanedCandidateArtistTags :execrows
-- Drops cached tags for candidates that no longer appear in the similarity
-- feed, or that have since been added to the library (their tags now live in
-- track_tags). The feed is refetched periodically and churns, so without this
-- the cache only ever grows.
DELETE FROM candidate_artist_tags t
WHERE NOT EXISTS (
SELECT 1 FROM artist_similarity_unmatched u
WHERE u.candidate_mbid = t.candidate_mbid
)
OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = t.candidate_mbid);
-- name: GcDeleteOrphanedCandidateArtistTagState :execrows
-- Same sweep for the bookkeeping rows. Kept as a separate statement rather
-- than a cascade: the two tables are independent by design (a 'none' outcome
-- has state but no tags), so neither can be the parent of the other.
DELETE FROM candidate_artist_tag_state s
WHERE NOT EXISTS (
SELECT 1 FROM artist_similarity_unmatched u
WHERE u.candidate_mbid = s.candidate_mbid
)
OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = s.candidate_mbid);