Track identity was file_path, so a file that came back renamed or in a
different directory looked like a deletion plus an unrelated new track:
the old row kept the like and every play_event while a fresh zero-history
row appeared, and nothing connected them. A liked song read as unliked, its
play count reset, and Rediscover could offer it as a discovery — silently.
Renumbering an album was enough, which is what happened to the operator's
copy of Minutes to Midnight.
Adoption re-points the existing row's file_path at the new location and
clears its missing mark. The normal UpsertTrack then conflicts on file_path
and updates THAT row, so the track id survives and likes, plays and
playlist memberships travel with it — and clients see an update rather than
a delete-and-create, so no cache churn either.
Matching is MBID first (identifies the recording, so it survives a
re-encode), then file_size + duration_ms for untagged files. Both
fingerprint components must be non-zero: duration_ms is 0 when ffprobe
failed, and matching 0 against 0 would pair up unrelated broken files.
Only rows already marked missing are eligible — a row whose file is present
elsewhere is a duplicate, not a move, and re-pointing it would corrupt the
copy that still exists. An ambiguous match inserts fresh rather than
adopting one arbitrarily: a fork is recoverable later, a wrong merge isn't.
Scan is now three phases, and the order is the point. Adoption can only
claim a row that is ALREADY marked missing, but reconcile previously ran
after processing — so a rename performed while the server was down surfaced
the deletion and the addition in the same scan, the new path inserted first,
and the fork became permanent. Enumeration is therefore separated from
processing so reconcile can run between them: walk (paths only, no tag
reads or probes) -> reconcile -> process in walk order.
Consequence worth knowing: when reconcile refuses (an absent root, or a
reorganisation exceeding the 25% mark cap) adoption cannot fire and renamed
files fork as before. That's the pre-#2528 behaviour rather than a new
failure, and the warning now names it.
The old outer walk-error branch was unreachable — the callback always
returned nil, so WalkDir never surfaced an error — and verifyRootsPresent is
the real protection, so enumerate counts walk errors instead of pretending
to abort on them.
Nothing in Minstrel ever noticed a deleted file. The walk only visits
paths that exist, so a row whose file was gone was never scanned, never
errored, never counted — permanently invisible. classifyEvent ignores
fsnotify removals by design, and the safety-net scan is the same walk, so
it covers additions only. Rows accumulated forever.
Found on the operator's library: a completed scan reported
skipped=24185 errored=0 while the MBID backfill (which opens files by DB
path rather than walking) logged ~40 "no such file or directory" across
three reorganised albums. Those rows also kept their pre-#2499 welded
genre, which is how this surfaced — the version-stamped tag re-read can
only reach files the walk visits.
The harm is not cosmetic. tracks is the candidate universe for
recommendation.sql / discover.sql / system_mixes.sql and nothing filtered
on file existence, so a mix could spend a slot on a track that cannot
stream.
Marks rather than deletes. A missing file is a claim about the filesystem
and the filesystem lies transiently — an unmounted volume, a network
blip, a container that started before its media mount attached. Every
sweep in internal/gc resolves a truth INSIDE the database and is safe to
run blind; this one is not, so no deletion happens here. Three guards
refuse to act on ambiguous evidence: every scan root must resolve to a
non-empty directory, the walk must have seen at least one file, and one
reconcile may newly mark at most 25% of the library. Clearing a mark is
never the dangerous direction, so it runs unconditionally — otherwise a
library that tripped the cap could never recover once the mount returned.
Only a full Scan reconciles. The walk's set of seen paths is the
evidence, and ScanFiles has no basis for concluding anything about files
it did not look at.
Excludes marked tracks from all 13 track-emitting queries (radio x2,
system mixes x5, discover x4, most-played x2), the 6 play-history seed
picks, and the genre browse axis. Deliberately NOT filtered: the shared
ListPlaylistTracks read path, because it also serves user-curated
playlists where hiding a track the user added would be wrong — system
playlists shed orphans on their next daily rebuild instead. History and
the taste profile also keep them: those record the past, and a track you
played 200 times still says something about your taste.
Reconcile tallies land in scan_runs so a disappearance is visible rather
than discovered when a mix comes up short.
dhowden/tag's readTFrame splits ID3v2 null-separated multi-value text
frames and rejoins them with the EMPTY string, so a file tagged
"Alternative Rock" + "Rock" was stored as "Alternative RockRock". It also
leaves bare numeric ID3v1 references unresolved, which is why the
library showed genres like "4017" and "526617".
This corrupted more than the browse axis added in #367: taste_profile.sql
reads tracks.genre directly, so the welded tokens were entering the taste
profile's tag vocabulary, and recommendation.sql/discover.sql were
comparing them as single opaque tags. Genre counts were wrong everywhere.
ffprobe is not a fix — ffmpeg's read_ttag calls decode_str once with no
loop, keeping only the first value. Truncating multi-genre tags would
blunt the similarity signal genre mainly feeds. So the TCON frame is now
parsed directly (ID3v2.2/2.3/2.4, all four text encodings, per-frame and
tag-level unsynchronisation, numeric and parenthesised ID3v1 references);
everything else still comes from dhowden/tag. Values are stored
";"-delimited, which the read side already splits on, so no query changes.
Existing rows are repaired without an operator-run rebuild: migration
0054 adds tracks.tag_read_version DEFAULT 0, below the scanner's current
tagReadVersion, so the next scan re-reads tags it would otherwise skip on
mtime. Such a re-read reuses the stored duration instead of re-running
ffprobe, keeping a repair pass tag-read-bound rather than one fork+exec
per file. Bumping the constant is how a future extraction fix reaches an
existing library.
Only ID3v2 is in scope — dhowden welds nowhere else. The Vorbis/MP4
repeated-field question is #2500, unproven and deliberately not built.
tracks.mbid was 100% NULL: the scanner only extracted album + artist
MBIDs, never the recording MBID. The ListenBrainz similarity worker is
gated on tracks.mbid IS NOT NULL, so track_similarity could never
populate — starving For-You/radio's strongest candidate source
(lb_similar) on every library.
- mbids.go: add extractRecordingMBID (mbz.Recording / Picard
musicbrainz_recordingid). Separate fn so extractMBIDs' signature +
unit tests stay untouched.
- scanner.go: persist recording MBID via UpsertTrack (heals on the
file_path conflict, so re-scans backfill for free).
- BackfillTrackMBIDs: one-shot pass mirroring BackfillMBIDs, wired as
scan Stage 2b (idempotent via SetTrackMbidIfNull, gated by
BackfillCap, log-only progress).
- migration 0029: tracks_mbid_unique (0002, written when the column
was always NULL) wrongly assumes one MBID == one track. A recording
appears on multiple releases, so rows legitimately share a recording
MBID. Replace with a non-unique partial index. Zero-risk: column is
100% NULL at migration time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Headline of S4. "Shuffle all" is always present (home app-bar
shuffle icon); the pool degrades with reachability:
- online → GET /api/library/shuffle?limit=N (new): N random
library tracks server-side, per-user quarantine filtered
(ListRandomTracksForUser, ORDER BY random()).
- offline → ShuffleSource shuffles the whole local cache index
(audio_cache_index ∩ cached_tracks, names from cached_artists/
albums) — a UNION over liked AND recently-played, since the
two-bucket split is storage-only and never filters playback.
Offline gating: refreshable (singleton) system playlists need the
live build/shuffle endpoints, so their tile play is disabled when
offlineProvider is true — Shuffle all is the offline path. User
playlists still play from cache.
playlist_card now reads offlineProvider in build → wrapped the
direct-render widget test in ProviderScope (offlineProvider is
smoke-safe from S1: tracked timers, no connectivity mount).
S4b (offline Recently-played / Liked browsable surfaces over the
cache index) next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Returns batched upserts + deletes since the supplied cursor. Empty cursor
returns full snapshot; subsequent calls pull deltas. Per-user entities
(likes, playlists) are scoped to the authed user. Composite-key entities
(likes, playlist_tracks) use stable string ids encoded by sync.EncodeLikeID
/ sync.EncodePlaylistTrackID.
Behavior:
204 No Content - no changes since cursor
200 OK - JSON syncResponse {cursor, upserts, deletes}
410 Gone - cursor older than oldest log row; client must reset
401 / 500 - standard envelope errors
Adds sqlc queries GetAlbumsByIDs, GetTracksByIDs, GetPlaylistsByIDs to
mirror the existing GetArtistsByIDs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DeleteTrack returns album_id + artist_id so the calling service can
chain the album-empty / artist-empty cascade. DeleteAlbumIfEmpty and
DeleteArtistIfEmpty are no-ops when other rows still reference the
parent — the service treats pgx.ErrNoRows as "not orphaned, skip".
After Lidarr accepts a request the local reconciler imports albums and
tracks as they arrive — but until the request hit 'completed' the
operator had no way to see "is anything happening?" The /requests and
/admin/requests rows now surface a live progress line whenever the
request's matched entity has children in our library.
Backend:
- New CountAlbumsByArtist + CountTracksByArtist sqlc queries.
- requestView gains imported_album_count and imported_track_count.
- New fillProgress helper computes them from the matched entity:
- kind=artist → counts albums + tracks under matched_artist_id
- kind=album → counts tracks under matched_album_id
- kind=track → 1 once matched_track_id is set
N+1 in the list endpoints; acceptable at admin scale.
- handleListRequests, handleGetRequest, and handleListAdminRequests
populate the new fields on every response.
Frontend:
- LidarrRequest TS type extended with the two counters.
- Both the operator's /requests page and the admin /admin/requests
page render an accent-colored line under the row meta when at
least one counter is non-zero, e.g.:
Artist: "5 albums · 47 tracks ingested"
Album: "12 tracks ingested"
Track: "Track ingested"
- Updated test fixtures to include the new required fields.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusen
