- similarity.Worker batch 5→25 (tracks AND artists per 1h tick). At
5/h a freshly-scrobbled library took days to build a usable
similarity pool; 25/h converges in hours, still well under
ListenBrainz rate limits (429 aborts the tick).
- scanrun Stage 2b track backfill now runs unbounded (-1) instead of
reusing the album backfill's 5000 staged cap. It's a one-time
whole-library heal with no progress UI; a cap just left tracks.mbid
partially NULL (3634/18056 after one pass) until several future
scans caught up, re-reading untagged files each time. One uncapped
pass converges; later scans only re-read the remaining NULL rows.
Album backfill keeps its 5000 cap (it has staged scan_runs UX).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tracks.mbid was 100% NULL: the scanner only extracted album + artist
MBIDs, never the recording MBID. The ListenBrainz similarity worker is
gated on tracks.mbid IS NOT NULL, so track_similarity could never
populate — starving For-You/radio's strongest candidate source
(lb_similar) on every library.
- mbids.go: add extractRecordingMBID (mbz.Recording / Picard
musicbrainz_recordingid). Separate fn so extractMBIDs' signature +
unit tests stay untouched.
- scanner.go: persist recording MBID via UpsertTrack (heals on the
file_path conflict, so re-scans backfill for free).
- BackfillTrackMBIDs: one-shot pass mirroring BackfillMBIDs, wired as
scan Stage 2b (idempotent via SetTrackMbidIfNull, gated by
BackfillCap, log-only progress).
- migration 0029: tracks_mbid_unique (0002, written when the column
was always NULL) wrongly assumes one MBID == one track. A recording
appears on multiple releases, so rows legitimately share a recording
MBID. Replace with a non-unique partial index. Zero-risk: column is
100% NULL at migration time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Slice 3c — completes the server-side producer set.
Publishes scan.run_started when InsertScanRun succeeds and
scan.run_finished after FinishScanRun completes (success or with an
error_message payload). Per-file progress events are intentionally NOT
emitted — they'd flood the stream during large library scans without
giving the admin scan card anything actionable. The two-beat signal
gives the UI enough to invalidate scanStatusProvider at the right
moments.
Bus access uses a package-level setter on internal/library because
threading bus through RunScan + TryStartScan + Scheduler + all their
callers would touch ~10 sites without changing behavior at the boundaries
that don't publish. Per-process singleton, matches the log.SetDefault
idiom. cmd/minstrel/main.go calls library.SetEventBus(bus) once at
startup; test contexts that never call it skip publishing safely
(publishScanEvent is a no-op when bus is nil).
This completes the server side of #392. Slice 4 wires the Flutter
consumer: live_events_provider.dart (StreamProvider) +
live_events_dispatcher.dart (event-kind → provider invalidation)
+ AppLifecycleState cold-start invalidation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each of the 4 stage workers (library walk, MBID backfill, cover
enrich, artist art enrich) gains a progressCb parameter that
receives a snapshot of the current running state per item processed.
The orchestrator stores the snapshot in a local var that the
publisher's marshal closure reads at flush time.
scanrun.go's 4 stage blocks are rewritten to construct one publisher
per stage, pass a Tick-firing closure to the worker as progressCb,
and call Flush() after the worker returns (routing any persist error
to captureErr — preserves today's diagnostic behavior).
The previous post-stage UpdateScanRun* calls are removed; the
publisher's Flush handles the final write. Cadence: every 500 items
OR every 1s, whichever fires first.
Outside-orchestrator callers (scanner_test.go, enricher_test.go)
pass nil for progressCb — no-op behavior preserved for tests that
don't need progress observability.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds an artist-art-enrich stage after cover-enrich in RunScan. The
stage drains EnrichArtistBatch with a configurable cap and the
operator's data_dir, persists processed/succeeded/failed tallies to
the scan_runs.artist_art_enrich jsonb column added in migration
0018.
The admin /admin Library Scan card grid expands from 3 to 4 columns
with the new "Artist art" card mirroring the cover-enrichment one
(Processed / Succeeded / Failed). The TS ScanStatus type gains an
optional artist_art_enrich field. The Go scanStatusResp gains the
matching ArtistArtEnrich json.RawMessage field.
cmd/minstrel/main.go (and admin scan-trigger handler) wire the new
RunScanConfig fields: ArtistEnrichCap reuses cfg.Library.CoverArtBackfillCap
for now (can be split if separate budgets are useful), DataDir is
cfg.Storage.DataDir.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Forward-fix from running the prior two commits in production. Backfill
on a real library exposed two further issues:
(1) gofmt -s flagged the column-aligned stubMeta methods in
mbids_test.go. Switched to the gofmt-canonical single-space form.
(2) Many albums failed to heal with SQLSTATE 23505 (unique constraint
albums_mbid_unique). Cause: the operator's library has duplicate
album rows in the DB — same MusicBrainz release, split into
multiple rows by the pre-MBID scanner because of subtle title or
artist disagreements between files. When backfill now correctly
extracts the MBID, two rows want to claim the same one and the
partial unique index rejects the second.
The conflict is correct DB behavior — we shouldn't have two rows
with the same MBID — but it's not a write failure to alarm the
operator about. Detect 23505 specifically:
- downgrade the log line from Warn to Info
- track these in a new BackfillMBIDsResult.Duplicates counter
(separate from Skipped, which retains its "no MBID in tag"
meaning)
- leave the duplicate row's mbid NULL; merging duplicates is a
separate (future) operator workflow
Same handling threaded through the scanner heal path so a regular
rescan doesn't generate the noise either.
JSON tally + admin UI gain a "Duplicates" line so the operator
can see how many duplicate rows their library carries.
Follow-up scope (separate task): a duplicate-album merge UX that
reparents the duplicate's tracks to the canonical row and deletes
the orphaned album row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds internal/library/scanrun.go with RunScan, which creates a scan_runs
row and sequences file-walk → MBID backfill → cover enrich, persisting
per-stage tallies as jsonb. Extends coverart.EnrichBatch to return
(processed, succeeded, failed, err) so the orchestrator can classify
outcomes. Replaces main.go's two separate boot goroutines with a single
RunScan call; passes nil scanner in the no-startup-scan branch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bvandeusen