feat(discover): artist-tag cache for out-of-library candidates — #2376

Migration 0050 adds candidate_artist_tags + candidate_artist_tag_state:
folksonomy tags for artists NOT in the library, which track_tags cannot
hold because it's FK'd to tracks(id) and a Discover candidate has no local
row. Slice 6 ranks against these; this slice only fills the cache.

The reuse the task claimed is real and verified: MusicBrainz's
fetchEntityTags(ctx, "artist", mbid, scale) already existed for the #1519
recording→artist fallback, so FetchArtistTags is a thin wrapper. Two
subtleties it does NOT inherit:

  - Weight scale is 1.0, not artistTagWeightFactor (0.6). That discount
    exists because FetchTrackTags uses artist tags as a *proxy* for a
    track's; here the artist IS the subject. Applying it would make these
    weights incomparable with track_tags — exactly the comparison slice 6
    depends on. Pinned by a test.
  - fetchEntityTags reports existing-but-untagged as (empty, nil) so the
    track path can fall through. There's no next level here, so empty
    becomes the terminal ErrNotFound; otherwise the enricher would settle
    a candidate as "enriched" with zero tags.

ArtistTagProvider is the split TrackTagProvider's own doc comment
anticipated ("e.g. artist-level tags"). Last.fm gains artist.getTopTags,
which returns the same toptags envelope, so the response type and
normalizer are reused unchanged.

Rather than write the merge-and-classify loop twice, extracted it from
EnrichTrack into runChain(). The ErrNotFound-vs-transient split is the
load-bearing part — those lead to opposite persistence decisions — so it
now has direct unit tests it never had while inlined.

Bookkeeping is a separate table, not columns, because the "providers had
nothing" outcome must be recordable for a candidate with zero tag rows,
and there is no per-candidate row to hang columns off (
artist_similarity_unmatched holds many rows per candidate). Absence of a
state row means "never processed", so a transient failure writes nothing
and stays eligible.

Two capacity realities are designed for, not papered over:
  - The pool is O(library artists x neighbours) and MusicBrainz allows
    ~1 req/s, so it can never drain in one pass. The eligibility query
    returns candidates in descending summed-similarity order, so the ones
    that can actually reach a deck are enriched first.
  - candidateBatch (50) is smaller than the track batch (200): tracks are
    finite and drain to completion, candidates are effectively unbounded
    and would otherwise starve the track arm forever.

GC sweeps both tables — the similarity feed churns, and a candidate that
joins the library has its tags in track_tags now. Tags swept before state
so a mid-sweep crash leaves a valid state, not a re-fetch loop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

internal/db/dbq/candidate_artist_tags.sql.go

@@ -0,0 +1,233 @@
+// Code generated by sqlc. DO NOT EDIT.
+// versions:
+//   sqlc v1.31.1
+// source: candidate_artist_tags.sql
+
+package dbq
+
+import (
+	"context"
+)
+
+const countCandidateArtistTagCoverage = `-- name: CountCandidateArtistTagCoverage :one
+SELECT count(*)::bigint AS processed,
+       count(*) FILTER (WHERE tag_source <> 'none')::bigint AS with_tags
+  FROM candidate_artist_tag_state
+`
+
+type CountCandidateArtistTagCoverageRow struct {
+	Processed int64
+	WithTags  int64
+}
+
+// Operator-facing coverage: how many distinct candidates have been processed,
+// and how many of those actually yielded tags. The gap is the honest ceiling
+// from the task — obscure artists with no MBID presence or no upstream tags
+// stay thin no matter how long the worker runs, and that is worth being able
+// to see rather than inferring from a silent surface.
+func (q *Queries) CountCandidateArtistTagCoverage(ctx context.Context) (CountCandidateArtistTagCoverageRow, error) {
+	row := q.db.QueryRow(ctx, countCandidateArtistTagCoverage)
+	var i CountCandidateArtistTagCoverageRow
+	err := row.Scan(&i.Processed, &i.WithTags)
+	return i, err
+}
+
+const deleteCandidateArtistTags = `-- name: DeleteCandidateArtistTags :exec
+DELETE FROM candidate_artist_tags WHERE candidate_mbid = $1
+`
+
+// Clear a candidate's cached tags before rewriting (atomic replace by the
+// caller, same shape as DeleteTrackTags).
+func (q *Queries) DeleteCandidateArtistTags(ctx context.Context, candidateMbid string) error {
+	_, err := q.db.Exec(ctx, deleteCandidateArtistTags, candidateMbid)
+	return err
+}
+
+const gcDeleteOrphanedCandidateArtistTagState = `-- name: GcDeleteOrphanedCandidateArtistTagState :execrows
+DELETE FROM candidate_artist_tag_state s
+ WHERE NOT EXISTS (
+       SELECT 1 FROM artist_similarity_unmatched u
+        WHERE u.candidate_mbid = s.candidate_mbid
+   )
+    OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = s.candidate_mbid)
+`
+
+// Same sweep for the bookkeeping rows. Kept as a separate statement rather
+// than a cascade: the two tables are independent by design (a 'none' outcome
+// has state but no tags), so neither can be the parent of the other.
+func (q *Queries) GcDeleteOrphanedCandidateArtistTagState(ctx context.Context) (int64, error) {
+	result, err := q.db.Exec(ctx, gcDeleteOrphanedCandidateArtistTagState)
+	if err != nil {
+		return 0, err
+	}
+	return result.RowsAffected(), nil
+}
+
+const gcDeleteOrphanedCandidateArtistTags = `-- name: GcDeleteOrphanedCandidateArtistTags :execrows
+DELETE FROM candidate_artist_tags t
+ WHERE NOT EXISTS (
+       SELECT 1 FROM artist_similarity_unmatched u
+        WHERE u.candidate_mbid = t.candidate_mbid
+   )
+    OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = t.candidate_mbid)
+`
+
+// Drops cached tags for candidates that no longer appear in the similarity
+// feed, or that have since been added to the library (their tags now live in
+// track_tags). The feed is refetched periodically and churns, so without this
+// the cache only ever grows.
+func (q *Queries) GcDeleteOrphanedCandidateArtistTags(ctx context.Context) (int64, error) {
+	result, err := q.db.Exec(ctx, gcDeleteOrphanedCandidateArtistTags)
+	if err != nil {
+		return 0, err
+	}
+	return result.RowsAffected(), nil
+}
+
+const insertCandidateArtistTag = `-- name: InsertCandidateArtistTag :exec
+INSERT INTO candidate_artist_tags (candidate_mbid, tag, weight)
+VALUES ($1, $2, $3)
+ON CONFLICT (candidate_mbid, tag)
+    DO UPDATE SET weight = GREATEST(candidate_artist_tags.weight, EXCLUDED.weight)
+`
+
+type InsertCandidateArtistTagParams struct {
+	CandidateMbid string
+	Tag           string
+	Weight        float64
+}
+
+// Upsert one (candidate, tag); keep the stronger weight when two providers
+// agree on a tag with different folksonomy strengths.
+func (q *Queries) InsertCandidateArtistTag(ctx context.Context, arg InsertCandidateArtistTagParams) error {
+	_, err := q.db.Exec(ctx, insertCandidateArtistTag, arg.CandidateMbid, arg.Tag, arg.Weight)
+	return err
+}
+
+const listCandidateArtistTagsForMbids = `-- name: ListCandidateArtistTagsForMbids :many
+SELECT candidate_mbid, tag, weight
+  FROM candidate_artist_tags
+ WHERE candidate_mbid = ANY($1::text[])
+`
+
+type ListCandidateArtistTagsForMbidsRow struct {
+	CandidateMbid string
+	Tag           string
+	Weight        float64
+}
+
+// Cached tags for a set of candidates, for slice 6's taste-overlap ranking.
+// One row per (candidate, tag).
+func (q *Queries) ListCandidateArtistTagsForMbids(ctx context.Context, dollar_1 []string) ([]ListCandidateArtistTagsForMbidsRow, error) {
+	rows, err := q.db.Query(ctx, listCandidateArtistTagsForMbids, dollar_1)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+	var items []ListCandidateArtistTagsForMbidsRow
+	for rows.Next() {
+		var i ListCandidateArtistTagsForMbidsRow
+		if err := rows.Scan(&i.CandidateMbid, &i.Tag, &i.Weight); err != nil {
+			return nil, err
+		}
+		items = append(items, i)
+	}
+	if err := rows.Err(); err != nil {
+		return nil, err
+	}
+	return items, nil
+}
+
+const listCandidateArtistsMissingTags = `-- name: ListCandidateArtistsMissingTags :many
+
+SELECT u.candidate_mbid,
+       coalesce(max(u.candidate_name), '')::text AS candidate_name,
+       sum(u.score)::float8                      AS total_score
+  FROM artist_similarity_unmatched u
+  LEFT JOIN candidate_artist_tag_state s ON s.candidate_mbid = u.candidate_mbid
+ WHERE NOT EXISTS (SELECT 1 FROM artists a WHERE a.mbid = u.candidate_mbid)
+   AND (
+       s.candidate_mbid IS NULL
+       OR (s.tag_source = 'none' AND s.tag_sources_version < $1)
+   )
+ GROUP BY u.candidate_mbid
+ ORDER BY total_score DESC, u.candidate_mbid
+ LIMIT $2
+`
+
+type ListCandidateArtistsMissingTagsParams struct {
+	TagSourcesVersion int32
+	Limit             int32
+}
+
+type ListCandidateArtistsMissingTagsRow struct {
+	CandidateMbid string
+	CandidateName string
+	TotalScore    float64
+}
+
+// Folksonomy tags for out-of-library Discover candidates (#2376). Parallel to
+// track_tags.sql, but keyed by MBID because the artist has no local row. See
+// 0050_candidate_artist_tags.up.sql for why the bookkeeping is its own table.
+// Candidates eligible for tag enrichment: never processed (no state row) or
+// settled 'none' under an older provider version.
+//
+// artist_similarity_unmatched holds one row per (seed, candidate, source), so
+// this GROUPs to one row per candidate — enriching the same MBID once per seed
+// that pointed at it would multiply the API calls for no gain.
+//
+// ORDER BY summed similarity DESC is the load-bearing part. The candidate pool
+// is O(library artists x neighbours per artist) — thousands — and MusicBrainz
+// allows ~1 req/s, so it can NEVER be fully enriched in one pass. Draining in
+// strength order means the candidates most likely to actually reach a user's
+// deck get tags first, and the long tail fills in over subsequent ticks
+// instead of starving behind it.
+//
+// Already-in-library candidates are skipped: they have an artists row, so
+// their tags belong in track_tags, and the suggestion query filters them out
+// anyway. $1 = current tag_sources_version, $2 = limit.
+// candidate_name is coalesced to '' so it lands non-nullable in Go: the name is
+// only a Last.fm lookup key, and empty simply means "MBID-keyed providers only",
+// which the provider chain already handles. max() is an arbitrary-but-
+// deterministic pick when several seeds spell the same MBID differently.
+func (q *Queries) ListCandidateArtistsMissingTags(ctx context.Context, arg ListCandidateArtistsMissingTagsParams) ([]ListCandidateArtistsMissingTagsRow, error) {
+	rows, err := q.db.Query(ctx, listCandidateArtistsMissingTags, arg.TagSourcesVersion, arg.Limit)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+	var items []ListCandidateArtistsMissingTagsRow
+	for rows.Next() {
+		var i ListCandidateArtistsMissingTagsRow
+		if err := rows.Scan(&i.CandidateMbid, &i.CandidateName, &i.TotalScore); err != nil {
+			return nil, err
+		}
+		items = append(items, i)
+	}
+	if err := rows.Err(); err != nil {
+		return nil, err
+	}
+	return items, nil
+}
+
+const setCandidateArtistTagState = `-- name: SetCandidateArtistTagState :exec
+INSERT INTO candidate_artist_tag_state (candidate_mbid, tag_source, tag_sources_version)
+VALUES ($1, $2, $3)
+ON CONFLICT (candidate_mbid) DO UPDATE
+   SET tag_source          = EXCLUDED.tag_source,
+       tag_sources_version = EXCLUDED.tag_sources_version,
+       updated_at          = now()
+`
+
+type SetCandidateArtistTagStateParams struct {
+	CandidateMbid     string
+	TagSource         string
+	TagSourcesVersion int32
+}
+
+// Stamp the enrichment outcome so the drainer skips settled candidates.
+// $2 = 'musicbrainz' | 'lastfm' | 'mixed' | 'none', $3 = current version.
+func (q *Queries) SetCandidateArtistTagState(ctx context.Context, arg SetCandidateArtistTagStateParams) error {
+	_, err := q.db.Exec(ctx, setCandidateArtistTagState, arg.CandidateMbid, arg.TagSource, arg.TagSourcesVersion)
+	return err
+}

internal/db/dbq/models.go

@@ -240,6 +240,19 @@ type AuditLog struct {
 	CreatedAt pgtype.Timestamptz
 }
 
+type CandidateArtistTagState struct {
+	CandidateMbid     string
+	TagSource         string
+	TagSourcesVersion int32
+	UpdatedAt         pgtype.Timestamptz
+}
+
+type CandidateArtistTag struct {
+	CandidateMbid string
+	Tag           string
+	Weight        float64
+}
+
 type ContextualLike struct {
 	ID            pgtype.UUID
 	UserID        pgtype.UUID

internal/db/migrations/0050_candidate_artist_tags.down.sql

@@ -0,0 +1,3 @@
+DROP INDEX IF EXISTS candidate_artist_tag_state_source_idx;
+DROP TABLE IF EXISTS candidate_artist_tag_state;
+DROP TABLE IF EXISTS candidate_artist_tags;

internal/db/migrations/0050_candidate_artist_tags.up.sql

@@ -0,0 +1,54 @@
+-- 0050_candidate_artist_tags.up.sql — folksonomy tags for OUT-OF-LIBRARY
+-- artists (#2376, milestone #268 slice 5).
+--
+-- track_tags (0042) cannot hold these: it is FK'd to tracks(id), and a
+-- Discover candidate has no local row by definition. So this is a parallel
+-- cache keyed by the candidate's MusicBrainz MBID — the only stable identity
+-- available for an artist we don't have.
+--
+-- Purpose is slice 6: rank suggestions by overlap between these tags and the
+-- user's taste_profile_tags, turning "neighbour of an artist you play" into
+-- "matches the sound you like".
+--
+-- GLOBAL, not per-user: a candidate's tags are a property of the artist, not
+-- of anyone's taste. Nothing here is user-scoped, so rule #47 has nothing to
+-- scope — the per-user part lives entirely in slice 6's ranking.
+--
+-- weight is a normalized folksonomy strength in [0,1], same scale as
+-- track_tags, so the two can be compared without a conversion step.
+CREATE TABLE candidate_artist_tags (
+    candidate_mbid text             NOT NULL,
+    tag            text             NOT NULL,
+    weight         double precision NOT NULL DEFAULT 1,
+    PRIMARY KEY (candidate_mbid, tag)
+);
+
+-- Enrichment bookkeeping. This is a SEPARATE table rather than columns on the
+-- tags table, because the "providers had nothing" outcome must be recordable
+-- for a candidate with zero tag rows — otherwise every empty candidate stays
+-- eligible forever and the worker re-fetches it on every tick.
+--
+-- tracks solved the same problem with columns on `tracks` (0042), but there is
+-- no per-candidate row anywhere to hang them off: artist_similarity_unmatched
+-- is keyed (seed_artist_id, candidate_mbid, source) and holds MANY rows per
+-- candidate.
+--
+-- Absence of a row here means "never processed", so unlike tracks.tag_source
+-- this column can be NOT NULL — there is no null-means-pending state to model.
+--   'musicbrainz' | 'lastfm' | 'mixed' → found, cached
+--   'none'                             → providers confirmed nothing
+--   tag_sources_version                → bump to re-process settled 'none'
+--
+-- A transient failure writes NO row at all (rather than a row it would then
+-- have to distinguish), which leaves the candidate eligible for the next pass.
+CREATE TABLE candidate_artist_tag_state (
+    candidate_mbid      text        PRIMARY KEY,
+    tag_source          text        NOT NULL,
+    tag_sources_version integer     NOT NULL DEFAULT 0,
+    updated_at          timestamptz NOT NULL DEFAULT now()
+);
+
+-- Serves the eligibility scan's "settled 'none' under an older version"
+-- branch. The PK already covers the per-candidate lookups.
+CREATE INDEX candidate_artist_tag_state_source_idx
+    ON candidate_artist_tag_state (tag_source, tag_sources_version);

internal/db/queries/candidate_artist_tags.sql

@@ -0,0 +1,102 @@
+-- Folksonomy tags for out-of-library Discover candidates (#2376). Parallel to
+-- track_tags.sql, but keyed by MBID because the artist has no local row. See
+-- 0050_candidate_artist_tags.up.sql for why the bookkeeping is its own table.
+
+-- name: ListCandidateArtistsMissingTags :many
+-- Candidates eligible for tag enrichment: never processed (no state row) or
+-- settled 'none' under an older provider version.
+--
+-- artist_similarity_unmatched holds one row per (seed, candidate, source), so
+-- this GROUPs to one row per candidate — enriching the same MBID once per seed
+-- that pointed at it would multiply the API calls for no gain.
+--
+-- ORDER BY summed similarity DESC is the load-bearing part. The candidate pool
+-- is O(library artists x neighbours per artist) — thousands — and MusicBrainz
+-- allows ~1 req/s, so it can NEVER be fully enriched in one pass. Draining in
+-- strength order means the candidates most likely to actually reach a user's
+-- deck get tags first, and the long tail fills in over subsequent ticks
+-- instead of starving behind it.
+--
+-- Already-in-library candidates are skipped: they have an artists row, so
+-- their tags belong in track_tags, and the suggestion query filters them out
+-- anyway. $1 = current tag_sources_version, $2 = limit.
+-- candidate_name is coalesced to '' so it lands non-nullable in Go: the name is
+-- only a Last.fm lookup key, and empty simply means "MBID-keyed providers only",
+-- which the provider chain already handles. max() is an arbitrary-but-
+-- deterministic pick when several seeds spell the same MBID differently.
+SELECT u.candidate_mbid,
+       coalesce(max(u.candidate_name), '')::text AS candidate_name,
+       sum(u.score)::float8                      AS total_score
+  FROM artist_similarity_unmatched u
+  LEFT JOIN candidate_artist_tag_state s ON s.candidate_mbid = u.candidate_mbid
+ WHERE NOT EXISTS (SELECT 1 FROM artists a WHERE a.mbid = u.candidate_mbid)
+   AND (
+       s.candidate_mbid IS NULL
+       OR (s.tag_source = 'none' AND s.tag_sources_version < $1)
+   )
+ GROUP BY u.candidate_mbid
+ ORDER BY total_score DESC, u.candidate_mbid
+ LIMIT $2;
+
+-- name: DeleteCandidateArtistTags :exec
+-- Clear a candidate's cached tags before rewriting (atomic replace by the
+-- caller, same shape as DeleteTrackTags).
+DELETE FROM candidate_artist_tags WHERE candidate_mbid = $1;
+
+-- name: InsertCandidateArtistTag :exec
+-- Upsert one (candidate, tag); keep the stronger weight when two providers
+-- agree on a tag with different folksonomy strengths.
+INSERT INTO candidate_artist_tags (candidate_mbid, tag, weight)
+VALUES ($1, $2, $3)
+ON CONFLICT (candidate_mbid, tag)
+    DO UPDATE SET weight = GREATEST(candidate_artist_tags.weight, EXCLUDED.weight);
+
+-- name: SetCandidateArtistTagState :exec
+-- Stamp the enrichment outcome so the drainer skips settled candidates.
+-- $2 = 'musicbrainz' | 'lastfm' | 'mixed' | 'none', $3 = current version.
+INSERT INTO candidate_artist_tag_state (candidate_mbid, tag_source, tag_sources_version)
+VALUES ($1, $2, $3)
+ON CONFLICT (candidate_mbid) DO UPDATE
+   SET tag_source          = EXCLUDED.tag_source,
+       tag_sources_version = EXCLUDED.tag_sources_version,
+       updated_at          = now();
+
+-- name: ListCandidateArtistTagsForMbids :many
+-- Cached tags for a set of candidates, for slice 6's taste-overlap ranking.
+-- One row per (candidate, tag).
+SELECT candidate_mbid, tag, weight
+  FROM candidate_artist_tags
+ WHERE candidate_mbid = ANY($1::text[]);
+
+-- name: CountCandidateArtistTagCoverage :one
+-- Operator-facing coverage: how many distinct candidates have been processed,
+-- and how many of those actually yielded tags. The gap is the honest ceiling
+-- from the task — obscure artists with no MBID presence or no upstream tags
+-- stay thin no matter how long the worker runs, and that is worth being able
+-- to see rather than inferring from a silent surface.
+SELECT count(*)::bigint AS processed,
+       count(*) FILTER (WHERE tag_source <> 'none')::bigint AS with_tags
+  FROM candidate_artist_tag_state;
+
+-- name: GcDeleteOrphanedCandidateArtistTags :execrows
+-- Drops cached tags for candidates that no longer appear in the similarity
+-- feed, or that have since been added to the library (their tags now live in
+-- track_tags). The feed is refetched periodically and churns, so without this
+-- the cache only ever grows.
+DELETE FROM candidate_artist_tags t
+ WHERE NOT EXISTS (
+       SELECT 1 FROM artist_similarity_unmatched u
+        WHERE u.candidate_mbid = t.candidate_mbid
+   )
+    OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = t.candidate_mbid);
+
+-- name: GcDeleteOrphanedCandidateArtistTagState :execrows
+-- Same sweep for the bookkeeping rows. Kept as a separate statement rather
+-- than a cascade: the two tables are independent by design (a 'none' outcome
+-- has state but no tags), so neither can be the parent of the other.
+DELETE FROM candidate_artist_tag_state s
+ WHERE NOT EXISTS (
+       SELECT 1 FROM artist_similarity_unmatched u
+        WHERE u.candidate_mbid = s.candidate_mbid
+   )
+    OR EXISTS (SELECT 1 FROM artists a WHERE a.mbid = s.candidate_mbid);