release / govulncheck (push) Successful in 45s
release / web (push) Successful in 1m27s
release / go (push) Successful in 1m51s
release / integration (push) Successful in 5m36s
release / android (push) Successful in 7m47s
release / Build signed APK (releases and dev) (push) Successful in 8m29s
release / Attach APK to the Release (tag releases only) (push) Skipped
release / Build + push container image (push) Successful in 1m47s
release / Verify release artifacts (tag releases only) (push) Skipped
The worker kept only the similar recordings already in the library, at most 20 of ListenBrainz's 50, and judged freshness by the edges it had written. Two failures followed, both measured on the operator's library (#3879): - A seed whose answer matched nothing wrote nothing, so it was never fresh. With the queue ordered by id, 25 such seeds held its head and were re-asked every hour; 17 of 2,466 played seeds had any edges. - A recording that reached the library after its seed was fetched (a Lidarr import, an MBID from the AcoustID lookup) was never linked until a refetch, which for the stuck seeds never came. Now every answer is cached whole in listenbrainz_similar_recordings and every answer, an empty one or a permanent 4xx included, is recorded in track_similarity_fetches. The queue reads the fetch record: never-fetched first, then the oldest, refreshed after 30 days. The listenbrainz edges are derived in SQL from the cache, one present track per recording and no cap, for the seed just fetched and for every seed once per tick, so new arrivals link within the hour without asking ListenBrainz again. Artists get the same queue fix via artist_similarity_fetches; their answer was already kept in artist_similarity and artist_similarity_unmatched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
158 lines
6.5 KiB
Go
158 lines
6.5 KiB
Go
// Package dbtest provides shared helpers for integration tests that need
|
|
// a clean Postgres state without disturbing the operator's admin user.
|
|
//
|
|
// Background: when integration tests run against the same Postgres
|
|
// instance an operator uses for local dev (a common dev-laptop setup),
|
|
// a TRUNCATE that includes the users table wipes the operator's admin
|
|
// login between every test run. To avoid that, tests should:
|
|
//
|
|
// 1. Call ResetDB instead of issuing TRUNCATE statements directly. It
|
|
// truncates every data table EXCEPT users, then deletes only those
|
|
// users whose username starts with TestUserPrefix.
|
|
// 2. Always create test user rows with a username that begins with
|
|
// TestUserPrefix; otherwise those rows survive across runs and
|
|
// leak into other tests.
|
|
//
|
|
// One test legitimately needs to wipe the entire users table:
|
|
// internal/auth/bootstrap_test.go, which exercises first-time admin
|
|
// bootstrap. That file does its own TRUNCATE and does not use this
|
|
// helper.
|
|
package dbtest
|
|
|
|
import (
|
|
"context"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/jackc/pgx/v5/pgxpool"
|
|
)
|
|
|
|
// TestUserPrefix is the required username prefix for any user row
|
|
// created by an integration test. ResetDB removes every user matching
|
|
// this prefix and leaves all others intact.
|
|
const TestUserPrefix = "test-"
|
|
|
|
// dataTables is the union of every non-users table touched by any
|
|
// integration test in the repo. Truncating the union is harmless for
|
|
// callers that only care about a subset; truncating tables that don't
|
|
// exist would error, but every name here corresponds to a migration
|
|
// that has shipped.
|
|
var dataTables = []string{
|
|
"artist_similarity",
|
|
"track_similarity",
|
|
"artist_similarity_unmatched", // M5c
|
|
"listenbrainz_similar_recordings", // #5296
|
|
"track_similarity_fetches",
|
|
"artist_similarity_fetches",
|
|
"scrobble_queue",
|
|
"contextual_likes",
|
|
"general_likes_albums",
|
|
"general_likes_artists",
|
|
"general_likes",
|
|
"play_events",
|
|
"skip_events",
|
|
"play_sessions",
|
|
"sessions",
|
|
"lidarr_quarantine_actions",
|
|
"lidarr_quarantine",
|
|
// #2374. Keyed by (user_id, candidate_mbid) with no FK to artists —
|
|
// candidates are out-of-library — so the CASCADE from artists/users
|
|
// does NOT reach it for a leftover row whose user survived. Truncate
|
|
// explicitly or a stale snooze silently hides a candidate from the
|
|
// next test's suggestion assertions.
|
|
"suggestion_snoozes",
|
|
// #2376. Same reasoning: keyed by candidate MBID with no FK anywhere,
|
|
// so nothing cascades to them. A leftover tag row would make a
|
|
// candidate look enriched to the next test, and a leftover state row
|
|
// would make it look already-settled and thus ineligible.
|
|
"candidate_artist_tags",
|
|
"candidate_artist_tag_state",
|
|
"playlist_tracks",
|
|
"playlists",
|
|
"library_changes", // M7 #357 — must reset to keep cursor isolated per test
|
|
// SettingsService.reconcile() idempotently re-UpsertProviderSettings
|
|
// for every registered provider at boot, so truncating this is the
|
|
// correct per-test reset (clears test-modified enabled/api_key rows).
|
|
// cover_art_sources_meta is NOT truncated — boot only READS it
|
|
// (never recreates the singleton, seeded once by 0018); ResetDB
|
|
// resets its counter via UPDATE below instead.
|
|
"cover_art_provider_settings",
|
|
// Same reasoning for the tag-enrichment settings (#1490): truncate the
|
|
// per-provider rows (reconcile re-seeds registered ones), and reset the
|
|
// tag_sources_meta counter via UPDATE below rather than truncating it.
|
|
"tag_provider_settings",
|
|
// recsettings.New reconciles shipped defaults on every construction
|
|
// (#1250), so truncating gives each test pristine tuning values.
|
|
"recommendation_weight_profiles",
|
|
"taste_tuning",
|
|
// #2377. Same reasoning as taste_tuning above: recsettings.New re-seeds
|
|
// shipped defaults on every construction, so truncating gives each test
|
|
// pristine Discover knobs rather than whatever a previous test tuned.
|
|
"discover_tuning",
|
|
"recommendation_tuning_audit",
|
|
"duplicate_group_members", // M400
|
|
"duplicate_groups",
|
|
"duplicate_sweeps",
|
|
"track_fingerprints", // M400
|
|
"track_loudness", // M464
|
|
"album_loudness", // M464
|
|
"user_normalization_prefs", // M464
|
|
"tracks",
|
|
"albums",
|
|
"artists",
|
|
}
|
|
|
|
// ResetDB clears all data tables and removes any user whose username
|
|
// begins with TestUserPrefix. Users without that prefix (notably the
|
|
// operator's admin row) are left alone. Calls t.Fatalf on error.
|
|
func ResetDB(t *testing.T, pool *pgxpool.Pool) {
|
|
t.Helper()
|
|
ctx := context.Background()
|
|
stmt := "TRUNCATE " + strings.Join(dataTables, ", ") + " RESTART IDENTITY CASCADE"
|
|
if _, err := pool.Exec(ctx, stmt); err != nil {
|
|
t.Fatalf("dbtest.ResetDB truncate: %v", err)
|
|
}
|
|
if _, err := pool.Exec(ctx,
|
|
"DELETE FROM users WHERE username LIKE $1",
|
|
TestUserPrefix+"%",
|
|
); err != nil {
|
|
t.Fatalf("dbtest.ResetDB delete test users: %v", err)
|
|
}
|
|
// Reset the monotonic cover-art source-version counter to its
|
|
// post-migration seeded value. Truncating the row would break
|
|
// SettingsService boot, which reads (never recreates) this
|
|
// singleton; an UPDATE keeps the row while clearing cross-test
|
|
// version accumulation (CurrentVersion=4 want 1, key-only-bump).
|
|
if _, err := pool.Exec(ctx,
|
|
"UPDATE cover_art_sources_meta SET current_version = 1",
|
|
); err != nil {
|
|
t.Fatalf("dbtest.ResetDB reset cover-art version: %v", err)
|
|
}
|
|
// Same for the tag-sources counter (#1490) — clears cross-test version
|
|
// accumulation that would spuriously report version_bumped on a
|
|
// key-only change.
|
|
if _, err := pool.Exec(ctx,
|
|
"UPDATE tag_sources_meta SET current_version = 1, last_registered_providers_hash = ''",
|
|
); err != nil {
|
|
t.Fatalf("dbtest.ResetDB reset tag-sources version: %v", err)
|
|
}
|
|
// Fingerprinting settings (M400 #3913), a singleton like the counters above.
|
|
// Every column goes back to its migration default rather than to literals
|
|
// written here, so a test can pin the Go defaults to the migration's.
|
|
if _, err := pool.Exec(ctx, `
|
|
UPDATE fingerprint_settings
|
|
SET enabled = DEFAULT, chromaprint_length_sec = DEFAULT,
|
|
acoustic_max_bit_error_rate = DEFAULT, backfill_concurrency = DEFAULT,
|
|
sweep_interval_hours = DEFAULT, updated_at = DEFAULT`,
|
|
); err != nil {
|
|
t.Fatalf("dbtest.ResetDB reset fingerprint settings: %v", err)
|
|
}
|
|
// Loudness analysis settings (M464 #4995), reset the same way.
|
|
if _, err := pool.Exec(ctx, `
|
|
UPDATE loudness_settings
|
|
SET enabled = DEFAULT, backfill_concurrency = DEFAULT, updated_at = DEFAULT`,
|
|
); err != nil {
|
|
t.Fatalf("dbtest.ResetDB reset loudness settings: %v", err)
|
|
}
|
|
}
|