72 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5.5	8bd48b4302	fix: a merge keeps the song link when the removed copy names the song (M498) ... MergeInheritSongLink took "linked" to mean song_key <> id, but the copy whose id became the song's key has no song_id of its own. Removing that copy dropped the survivor out of the song. It now asks whether another copy shares the key. The test covers both copies as the one removed, so it no longer passes or fails on which random id sorts first. The release-change test counted audit rows other tests left behind (ResetDB keeps audit_log); it now counts what its own pass adds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:53:35 -04:00
bvandeusenandClaude Opus 5.5	e1ad1c9ba2	fix: a release change Lidarr applied but did not answer is recorded (M498) ... On the first deploy pass Lidarr took longer than the client's 30s timeout to answer the release change for Cracker Island: its album update unlinks the files and queues the rescan before it replies. The change may well have landed, but the resolver logged a failure and wrote no audit row, so the album had no settle window and the report did not show it. An unanswered PUT (ErrUnreachable) is now read back: when the album's monitored release is the chosen one, the change is recorded as made. Any other error, or a read-back that disagrees, stands as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:42:50 -04:00
bvandeusenandClaude Opus 5.5	e17556dce9	feat: Lidarr searches for a better copy of a rip that has none (M498 #5447) ... The operator chose option 2: an album holding a video rip that is the only copy of its song is searched in Lidarr (AlbumSearch), at most once a week and ten albums a pass. Lidarr grabs only when the quality profile allows an upgrade, so nothing is removed and the rip stays held back either way. - lidarr.Client.SearchAlbums posts the AlbumSearch command. - ListAlbumsWithSoleCopyRips finds the albums: a held-back rip with no present clean copy sharing its song or its pending group, and not searched in the last week (audit action lidarr_rip_search). - Suspect sources shows when Lidarr last searched the track's album. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:32:30 -04:00
bvandeusenandClaude Opus 5.5	53eb954c86	feat: video rips and stray copies handle themselves (M498 #5439) ... - The resolver also merges cross-release and mismatch groups where Lidarr maps exactly one copy: the others fulfil nothing, so removing them opens no hole (D-a rule 1). That covers a rip beside the clean copy on another release and a wrong-file import Lidarr holds unmapped. With two or more mapped copies each fulfils its own release and nothing is removed. - A track whose file name carries a video-rip marker is held back from radio and the system mixes (tracks.source_verdict, migration 0077). It still plays when chosen. A renamed file is released; the operator's "fine" sticks. - Suspect sources shows what was done to each track, with "This one is fine" and "Hold back again" (PUT /api/admin/library/suspect-sources/{id}). - The Liked list prefers a copy that is not held back. Replacing a rip that has no clean copy is left for the operator to decide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:13:37 -04:00
bvandeusenandClaude Opus 5.5	5b372f61d7	feat: the same song on several releases counts as one song (M498 #5438) ... A single and the album it is on stay two files, since each fulfils its own release in Lidarr, but they are one song to the listener. - tracks.song_id links copies; the generated song_key (song_id, else the track's own id) is what they share (migration 0076). - The resolver links each cross-release group every pass (idempotent; only with auto-resolve on) and, when a link is new, shares existing likes across the song and logs them for sync. - A like or unlike (web and Subsonic) reaches every copy; each change is logged and published so clients update every heart. - The Liked list and its count show the song once; Shuffle and the mix writer take one copy per song. - A merge keeps the removed copy's song link; "Not the same song" on the Across releases tab dismisses the group and undoes the link. Shared plays ("heard via another copy") are left for a later step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:07:55 -04:00
bvandeusenandClaude Opus 5.5	4ecff52f19	feat: duplicates resolve themselves where Lidarr says it is safe (M498) ... The duplicate sweep proposed 4,197 groups and every one waited for the operator. Most are safe to settle, and Lidarr defines what safe means: it maps one file to each track of the release it monitors and downloads any mapped file that disappears. Deleting a mapped copy opens exactly the hole the operator saw Lidarr fill. Classify (#5435) - Migration 0075: duplicate_groups.class (same_release, cross_release, mismatch, review), resolve_note, resolved_automatically; duplicate_group_members.lidarr_state (tracked, unmapped); fingerprint_settings.auto_resolve; notification kind duplicates_resolved with both kind CHECKs swapped (rule 36). - library.ClassifyDuplicateGroup, with MatchTitleKey dropping featuring credits, remaster notes and video-rip markers, and keeping live, demo, remix and instrumental. The rip markers move from api to library. Choose the copy to keep (#5436) - ProposeSurvivor ranks the copy Lidarr maps first, then tag fit (a clash-free track number, no rip marker in the name, an MBID), then the quality rules. File size picked the wrong Humanz copy in 6 of 21 groups. Act (#5437) - An hourly resolver pass reads Lidarr's unmapped files, matched by the last three path components, and records each copy's state. - Same album, with at most one copy mapped: merged into the mapped copy. The merge is guarded, so a mapped copy can never be removed (MergeDuplicateGroupGuarded, ErrCopyTrackedByLidarr). - Same album, every copy mapped: the monitored release lists the song twice (Humanz's 14x12" box set). The pass moves Lidarr to the release that lists each song once and best covers what is on disk. It never picks one covering less, and is capped at 10 albums per pass. - Fixed point (lesson #4183): the chosen release no longer repeats. - The album is left alone for 24h while Lidarr rescans, so "every copy unmapped" mid-rescan is never read as licence to merge. - Both actions are audited with no actor and summarised to admins. The operator can switch them off in the Fingerprinting card (rule 25). - Manual merges use the same guard: 409 copy_tracked_by_lidarr, or 503 lidarr_unavailable when Lidarr cannot say. Web - Duplicates gets tabs: Needs review, Across releases, Resolved automatically. Each loads as you scroll (rule 172), replacing the pager. - Each copy says whether Lidarr uses it. - The resolver's note shows on each group. - The merge confirm blocks, before sending, a merge that would remove the copy Lidarr uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 21:49:37 -04:00
bvandeusenandClaude Opus 5.5	f11dba2336	feat(api): read-all takes an optional up_to cutoff (M489) ... POST /api/me/notifications/read-all accepts {"up_to": RFC3339}. Android queues "mark all read" for replay when offline, and a replay landing later must not mark notices that arrived in between, which the user never saw. A coalesced notice updated since then has a newer created_at, so it stays unread. An empty body still marks everything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 07:27:29 -04:00
bvandeusenandClaude Opus 5.5	1e9408c835	feat(notifications): library health reaches admins, coalesced (#5341) ... - A failed scan run sends scan_failed. Each failure adds to the count and the notice shows the latest error. A scan cut short by shutdown says nothing. - Marking tracks missing sends tracks_missing with a running count. - A duplicate sweep that proposes a group it had not proposed before sends duplicates_found, counting everything awaiting review. A sweep that only re-finds known groups stays quiet, so a read notice isn't repeated every sweep (CountDuplicateGroupsDetectedSince). - A playback-error report sends playback_errors, counting the unresolved errors (CountUnresolvedPlaybackErrors). The library package gets its notifier as a package-level SetNotifier beside SetEventBus, for the same reason the bus is package-level. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 07:20:33 -04:00
bvandeusenandClaude Opus 5.5	37d3a5bcd3	fix(library): read every genre of FLAC, Ogg, Opus and MP4 files (#2500) ... Vorbis comments repeat a field to give it several values (GENRE=Boom Bap, GENRE=Downtempo, ...). dhowden/tag keeps comments in a map keyed by field name, so each repeat overwrote the previous one and only the last genre was stored. MP4 has the same gap: several data atoms in one ©gen atom, or repeated ©gen atoms, collapse to one value. On the operator's library 3,658 of 4,092 FLACs declare more than one genre. Kupla's Life Forms carries eight and was stored as "Instrumental Hip Hop" alone, so browse and the taste profile never saw the other seven. - vorbisgenre.go reads the comment block directly: FLAC's metadata block (including FLACs behind an ID3v2 tag), and the comment packet of Ogg Vorbis and Opus, reassembled across pages when cover art makes it span several. - mp4genre.go walks moov > udta > meta > ilst and returns every ©gen text value. It handles ISO and QuickTime meta layouts and a moov placed after mdat. A file with only the numeric gnre atom still falls back to dhowden, which resolves it. - extractGenres routes VORBIS and MP4 through them, as #2499 did for ID3v2. - tagReadVersion 3 -> 4, so the next scan re-reads the tags of files already indexed. Unchanged files keep their duration and fingerprint, so the pass costs tag reads only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 23:40:56 -04:00
bvandeusenandClaude Opus 5.5	e5dac9ddf0	fix(similarity): keep ListenBrainz's whole answer and resolve it locally (#5296) ... The worker kept only the similar recordings already in the library, at most 20 of ListenBrainz's 50, and judged freshness by the edges it had written. Two failures followed, both measured on the operator's library (#3879): - A seed whose answer matched nothing wrote nothing, so it was never fresh. With the queue ordered by id, 25 such seeds held its head and were re-asked every hour; 17 of 2,466 played seeds had any edges. - A recording that reached the library after its seed was fetched (a Lidarr import, an MBID from the AcoustID lookup) was never linked until a refetch, which for the stuck seeds never came. Now every answer is cached whole in listenbrainz_similar_recordings and every answer, an empty one or a permanent 4xx included, is recorded in track_similarity_fetches. The queue reads the fetch record: never-fetched first, then the oldest, refreshed after 30 days. The listenbrainz edges are derived in SQL from the cache, one present track per recording and no cap, for the seed just fetched and for every seed once per tick, so new arrivals link within the hour without asking ListenBrainz again. Artists get the same queue fix via artist_similarity_fetches; their answer was already kept in artist_similarity and artist_similarity_unmatched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 19:56:22 -04:00
bvandeusenandClaude Opus 5.5	865a3176c9	feat(library): fold a missing track into its on-disk replacement (M485 #5286 #5287) ... A track marked missing whose replacement is already on disk under another row — on the operator's library 355 of 451 missing tracks, nearly all Lidarr mp3 -> flac upgrades — is folded into the replacement: likes, plays, playlist entries and tags move across and the missing row goes. Move adoption could not catch these: the replacements were re-encodes (no shared audio hash) with no recording MBID at import. Pairs (ListMissingTrackPairs): the same recording MBID within the album group, or the same album row and title ignoring case. Each side must have exactly one candidate; conflicting MBIDs refuse a pair. No duration or track position gate: on the 142 pairs known to be one recording, 18% differed by over 2s and the poorly tagged set is where numbering is broken (spike #5274). Each pair folds in its own transaction after locking both rows and checking the pair still holds. Runs automatically (operator, 2026-10-07) after a full scan, after a watcher batch that added or updated tracks, and after an AcoustID pass that matched any track. The first scan after deploy repairs the existing rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 17:23:32 -04:00
bvandeusenandClaude Opus 5.5	2df28345b4	refactor(library): the per-copy fold is a helper the duplicate merge calls (M485 #5285) ... MergeDuplicateGroup's loop body — repoint and copy every FK from a removed copy onto the survivor, inherit its MBID, delete its row, tidy an emptied album — moves into foldTrackInto, so the missing-pair pass can fold a stale missing row into its replacement with the same mechanics. The group lock, file removal and group bookkeeping stay in the merge. No behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 17:23:32 -04:00
bvandeusenandClaude Opus 5.5	5c19a916ba	feat(library): store each album's MusicBrainz release-group id (M483 #5242) ... albums.mbid is the release id (Picard's musicbrainz_albumid, one edition). Lidarr names albums by release group, so re-acquisition and request completion need that id too (#5241). Migration 0070 adds albums.release_group_mbid (nullable, non-unique index: several releases share a group). The scanner reads musicbrainz_releasegroupid through extractReleaseGroupMBID, writes it on insert and heals it onto existing rows when NULL. tagReadVersion goes to 3 so the next scan fills it for the library already indexed, bound by tag reads (no ffprobe, no decode). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 11:33:26 -04:00
bvandeusenandClaude Opus 5.5	3c575b137c	feat(library): AcoustID lookup worker fills the MBIDs tags leave empty (M401 #3920 #3921) ... Migration 0069 adds tracks.mbid_source (tag | acoustid), a lookup state per track (matched | ambiguous | no_match | failed) and the acoustid_settings row (off, no key, min score 0.85). The file's tag outranks a lookup (D4). UpsertTrack keeps a looked-up id through a re-read that finds no tag id and replaces it as soon as one appears. SetTrackMbidFromAcoustID refuses to write over a tag id. The worker fingerprints each untagged track with fpcalc's compressed print, looks it up and writes an id only when D5 settles it: one recording at or above the threshold, or one left after matching title and length. Ambiguous and no-match results write nothing. A key AcoustID refuses, or the service being unreachable, stops the pass and is reported in the worker's status. It never counts as a verdict on a track. A changed file drops its lookup in the scan. The re-lookup takes back an id that no longer matches. Admin API: GET /api/admin/library/acoustid (settings, status, coverage by source), PUT …/acoustid-settings (write-only key), POST …/acoustid/run, GET …/acoustid/unsettled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 23:26:03 -04:00
bvandeusenandClaude Opus 5.5	f13da62797	feat(library): AcoustID lookup client and the compressed fpcalc print (M401 #3919) ... internal/acoustid posts AcoustID's v2 lookup as a gzip form with meta=recordings, one request per 400ms (their limit is 3/s) and a 20s deadline. It returns every linked recording with its best score; choosing among them is the worker's job (D5). The server's error codes map to an invalid key (stop and say so), a rejected fingerprint (a verdict on the track) or unavailable (try again later). A cancelled caller stays a cancellation. fpcalcLookupArgs and parseFpcalcCompressed read fpcalc's default output, the compressed string the lookup takes (D1), always over the first 120s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 23:16:52 -04:00
bvandeusenandClaude Opus 5.5	13a7a3629a	fix(library): MBID backfills skip tracks whose files are missing (#5139) ... The track backfill listed every track with a NULL mbid, missing or not, so each scan tried to open every missing file and logged an "open failed" warning per track. Nothing ever healed. The album backfill could pick a missing track as the one to read, and since that pass is capped per scan, albums stuck that way were retried ahead of the rest every time. Both now read only tracks still on disk; an album with none left is skipped until a scan finds its files again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 21:22:05 -04:00
bvandeusenandClaude Opus 5.5	2e36e70268	feat(android): Sonos/UPnP queue plays leveled URLs, rendered a track ahead (M464 #5002) ... Every URL the Sonos queue loader sends is minted with level=true and the track's album-play verdict from its neighbours in the queue; the server returns the plain stream when leveling is off or changes nothing. The playing track and the one after it are rendered ahead, and each time the renderer moves on, the next is. Server: a mint no longer prerenders on its own. A queue load mints every track, which would have started an ffmpeg render per track at once. The request now carries prerender, and at most two prerenders run at a time; past that they are dropped, since a fetch renders on demand anyway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 19:38:21 -04:00
bvandeusenandClaude Opus 5.5	f34423a0e0	feat: leveled FLAC stream for Sonos/UPnP speakers (M464 #5001) ... Speakers fetch their own audio, so the phone cannot level it. A cast token minted with level=true (and the client's asAlbum, which only the queue holder knows) now returns GET /api/tracks/{id}/leveled.flac: the track rendered by ffmpeg at the user's gain (volume=XdB, plus alimiter at -1 dBFS for a limiter-mode boost), metadata stripped, FLAC at 16 or 24 bits and at most 48 kHz. The gain is computed server-side from the user's preference and the stored loudness, carried as ?g=<centi-dB>&lim=0|1 and signed into the token, so an edited URL does not verify. Unity gains get the plain stream. Renders are written beside the cache file and renamed in, keyed by the source's size and mtime, coalesced per file (singleflight, detached from the requesting speaker so a retry finds the render running), started at mint time so the fetch finds them ready, and evicted least recently used past leveled_cache_mb, a new admin setting (migration 0068, Loudness analysis card). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 19:31:07 -04:00
bvandeusenandClaude Opus 5.5	af36b2f24a	feat: per-user volume leveling preference, synced across devices (M464 #4998) ... Mode (off, auto, track, album), target (-18, -16, -14 LUFS) and boost (within headroom, or fully with a limiter), stored per user on the server so the web player, the Android app and casts apply the same one. - Server: user_normalization_prefs (migration 0067), GET/PUT /api/me/normalization; a whole-body PUT, validated, last write wins. - Web: Settings > Playback > Volume leveling. Saves at once, restores the old choice if the save fails, and caches the value for the player. - Android: Settings card. The device keeps a copy for offline playback (Room v9 with an explicit migration, so the upgrade wipes nothing). Writes are offline-first: shown at once, PUT best effort, queued on failure (NORMALIZATION_SET, collapsed to the newest). A refresh never overwrites a change still queued. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 14:29:40 -04:00
bvandeusenandClaude Opus 5.5	e3aa8629d3	feat(api): deliver loudness gains to every client (M464 #4997) ... ReplayGain 2.0 values (gain to -18 LUFS, linear peak) derived from the stored track and album loudness: - Web: GET /api/tracks/replay-gain?ids=... (up to 200), a lookup the player calls for its queue, rather than a field on every TrackRef surface. - Android: track_gain/track_peak and album_gain/album_peak on the sync views, so cached tracks level offline. Storing a measurement logs a track change, and an album's values moving logs an album change, both before the write (#2704), so caches pick the gains up. - OpenSubsonic: replayGain on every song (album, getSong, search3, starred), as a JSON object and an XML element. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:54:05 -04:00
bvandeusenandClaude Opus 5.5	c2f81bf8df	feat(library): album loudness from the tracks' summed block histograms (M464 #4996) ... Album-mode normalization plays a whole album at one gain. That gain comes from album_loudness (migration 0066): BS.1770's gated loudness over every block on the album, computed by summing the tracks' stored histograms and gating the sum. No audio is decoded again. Album true peak is the loudest track's. - Recomputed by the loudness worker each tick, after the track pass and whether or not analysis is switched on. ListAlbumsNeedingLoudness lists albums whose md5 over (present track id, measurement version and time) no longer matches the stored digest. One comparison covers every way membership changes (scan retag, duplicate merge, delete, missing and restored) without hooking each. - No album value until every present track has a settled measurement, so an album's gain doesn't shift mid-listen as the rest is measured. Silent and unreadable tracks count as settled. - Rows for albums with no present track left are dropped. - The parser and the merge share trimBins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:27:02 -04:00
bvandeusenandClaude Opus 5.5	f3196b3443	feat(library): measure every track's loudness in the background (M464 #4995) ... The first step of loudness normalization: the server measures each track with ffmpeg's EBU R128 filter (true peak, mono as dual mono) and stores the integrated loudness, true peak and loudness range in track_loudness (migration 0065). It also keeps a histogram of the 400 ms gating blocks at 0.1 LU, so album loudness can be computed exactly later with no second decode (#4996). The histogram reproduces ffmpeg's own figure (-10.68 against -10.7 on the captured fixture), and the analyzer logs a warning if the two ever drift. - A background worker, cloned from the fingerprint backfill, measures every track, new ones included. Measuring inline in the scan was dropped: the analysis decodes the whole file, and a large import could pass the scan's one-hour stuck threshold. The scan only deletes a changed file's measurement; the worker ticks every 10 minutes. - Timeouts, the cancel/missing-binary split and settled verdicts follow the fingerprint runner. Silence and undecodable files are stored as verdicts; stalls are retried. The deadline scales with track length. - loudness_settings (enabled, files at once) and an admin card with the coverage gauge, under GET/PUT /api/admin/library/loudness-settings and GET /api/admin/library/loudness. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:01:01 -04:00
bvandeusenandClaude Opus 5.5	327d49428f	feat(auth): store Subsonic API keys hashed; a new key is shown once (M462 #4983) ... users.api_token held each user's apiKey in plaintext and was looked up by equality, so a leaked row or backup handed out working keys. Migration 0063 replaces it with api_token_hash (sha256, hex), computed in place from the existing keys so every Subsonic client keeps working. The key can no longer be read back: GET /api/me/api-token is gone, and POST returns the new key once. Settings shows it right after Regenerate with a copy button and a "won't be shown again" note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 09:42:35 -04:00
bvandeusenandClaude Opus 5	077ae61235	feat(admin): fingerprinting settings — on/off, length, match threshold, concurrency, sweep interval (M400 #3913) ... Rule 25: the fingerprinting knobs move out of source into a DB-backed singleton (migration 0061), edited from a card on the Duplicates page and shared live with the scanner, the backfill and the duplicate sweep through one service instance, so a save needs no restart. The length is the knob that can silently break the library: prints taken at two lengths never match. Each track_fingerprints row now records the length it was taken at, and every reader filters on the current one — the backfill treats another length as stale, the gauge counts it pending, the sweep never streams it. Equivalent to a version bump, except that setting the length back makes rows not yet redone current again. The card warns before a length change re-fingerprints the library. Off stops every decode: the scan takes only the stream hash (a demux, and what recognises a moved file) and stores nothing, dropping a changed file's stale row; the backfill idles. A save also makes a sweep due, since a new threshold or length changes what the same prints group into, and the sweep interval gains slack so an hourly interval on an hourly tick doesn't skip every other tick. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:53:55 -04:00
bvandeusenandClaude Opus 5	c8bf9dc929	refactor(library): move detection matches on the audio hash, not size and duration (M400 #3914) ... A file that comes back renamed or moved keeps its track row, and with it its likes and play history, by being matched to the missing row it replaces (#2528). Untagged files were matched on (file_size, duration_ms), which was never a fingerprint. It could pair two unrelated files that happened to share a byte count and a duration, and it missed a file retagged in place, whose size changes. The only defence was requiring a unique match and otherwise giving up. Now there is a real identity. FindMissingTrackByAudioHash matches a missing track by the SHA-256 of its encoded audio (track_fingerprints, #3906). That survives a rename, a move and a retag, and only an identical recording can match it. adoptMovedTrack takes the new file's hash, which the scan already computes before adoption. The size and duration query and fallback are removed outright, with no second path (rule 22). Unchanged: - MBID first: it identifies the recording and survives a re-encode that even the hash does not - a unique match is still required - an absent hash is never looked up, so unhashable files cannot pair with each other The test fake answers the hash lookup only for the hash it holds, so the tests can tell adoption by identity apart from adoption by coincidence. That includes the case the old pair got wrong: different audio of equal size and duration is not adopted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:31:42 -04:00
bvandeusenandClaude Opus 5	11ef044ef6	feat(library): merge duplicates without losing history (M400 #3911) ... Merge keeps one copy of a duplicate group and removes the rest. Every table that references tracks does so ON DELETE CASCADE, so deleting a duplicate's row outright would silently destroy its likes, plays, playlist entries and tags. The merge moves all of that onto the kept copy first, then deletes the empty row. In one transaction, holding a lock on the group: - repoints play_events, skip_events, contextual_likes, playback_errors, lidarr_requests.matched_track_id and playlist_tracks. The last is keyed by position, so every entry stays where it was. - merges general_likes one per user, dated to the earlier like - takes the union of track_tags, keeping the kept copy's own weight on a shared tag - rewrites track_similarity onto the kept copy, dropping edges that would point a track at itself and keeping the kept copy's existing edge on a collision - lets the kept copy take a recording MBID only the removed copy had - deletes the removed copies' rows, tidies emptied albums and artists, marks the group merged - logs sync changes: track deletes, and like and playlist-track delete/upsert pairs The removed copies' files are deleted first, before any row changes, through the same helper as DeleteTrackFile (now shared, along with the album tidy-up). A merge that left the file behind would be undone by the next scan re-importing it. An unwritable library answers 409 library_not_writable and nothing changes. tracks.Service.MergeDuplicates wraps it with the opt-in Lidarr unmonitor from RemoveTrack, skipped when the removed copy is a second file of the kept copy's own album track: unmonitoring that would stop Lidarr managing the kept file. It writes a duplicate_merge audit row after commit, per the audit package's best-effort contract, naming both paths. POST /api/admin/library/duplicates/{id}/merge takes an optional survivor_track_id (the report's proposal otherwise) and unmonitor. On the report page: - each copy gets a Keep choice, defaulting to the proposed one - Merge needs a second click, on a button that says how many files it removes, with the consequence stated beside an opt-in Lidarr checkbox Integration tests cover: - every piece of history landing on the kept copy exactly: likes deduped at the earlier time, plays and skips counted, playlist position unchanged, tags unioned, similarity rewritten with no duplicate or self-edge, MBID inherited - the removed file gone, and a second merge refused - an unwritable file leaving likes, plays, row and group untouched - a survivor outside the group refused Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:25:06 -04:00
bvandeusenandClaude Opus 5	ff493a8c7d	feat(admin): the duplicates report — review proposed duplicate groups (M400 #3912) ... A new admin tab, Duplicates, beside Missing files: the proposals from the duplicate sweep, with a Sweep now trigger and a Not duplicates dismissal. Nothing on it merges or deletes; the merge is #3911. Each group shows: - whether it is identical audio or the same recording, with a match percentage from the weakest link between members - every copy's format, size, duration, path, and the likes and plays it carries (every user's; this is admin-only, and it is what decides which copy to keep) - the copy proposed to keep, and the rule that chose it The survivor rule is library.ProposeSurvivor, a pure function the merge will reuse: lossless over lossy, then the larger file, then the copy in the library longest, then lowest id. Bitrate is not in it because the scanner never fills tracks.bitrate, and for one recording at one duration a larger file is the higher bitrate. m4a is not counted as lossless: it may be AAC. The reason names the rule that separated first place from second, not every rule the winner passed. An empty report has three causes, and the page says which: still fingerprinting, the sweep has never run, or it ran and found nothing. The sweep's state and the backfill's progress come back with the groups for that reason. Groups left with fewer than two members since the sweep are not shown. GET /api/admin/library/duplicates, POST .../sweep (202, or 409 sweep_in_progress), POST .../{id}/dismiss (404 duplicate_group_not_pending when already resolved). Migration 0060 indexes play_events by track_id. Its only indexes led with user_id, so each copy's play count, and the merge's repointing of play history, would scan the whole table. Web only, like Missing files: Android has no library-health admin screens. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:11:11 -04:00
bvandeusenandClaude Opus 5	6379b6c31d	feat(library): the duplicate sweep — propose duplicate groups from fingerprints (M400 #3910) ... Reads fingerprints, runs them through the matcher, and records proposals in duplicate_groups (migration 0059). Nothing is merged or deleted: a group is a proposal for the admin report (#3912). Streaming. The whole library's fingerprints are hundreds of megabytes, but tracks are only compared within 3s of each other in duration. So candidates stream in (duration_ms, id) order, keyset-paged on a new tracks(duration_ms, id) index. The grouper holds only the tracks within 3s of the oldest one not yet settled. A seed is settled once a track arrives beyond its window, which gives the same result as grouping the whole sorted list. groupDuplicates is rebuilt on the same streamGrouper, so there is one grouping rule and the #3909 tests still cover it. Each fingerprint's alignment index and variety check are computed once instead of for every pair. Exact duplicates are grouped library-wide in SQL. The first member the stream meets stands in for the whole group in the acoustic pass. An exact group caught in an oversize acoustic cluster is still proposed: the acoustic evidence is discarded, identical bytes are not. Re-sweeping: - a group is identified by its sorted member ids, so finding it again refreshes the row in place - a proposal whose members all sat in one dismissed group is not proposed again (a subset repeats the verdict; a superset is new evidence) - a pending proposal no sweep has found again is retired, but only after a complete sweep, and only if an earlier sweep last confirmed it, so two overlapping sweeps cannot delete each other's findings - dismissals are kept DuplicateSweepWorker checks hourly and sweeps only when a fingerprint was written after the last sweep started. TryStartDuplicateSweep guards against two sweeps at once and reaps one stuck in flight for 2h. The sweep row is closed on a detached context with a deadline, so a sweep cancelled at shutdown still records that it ended. The integration test pages one row at a time and checks: - an acoustic pair and an exact pair are found - a track with no fingerprint, a missing track and a near-duration unrelated song are left out - a dismissed group is suppressed while the pending one refreshes without duplicating - a proposal that stops holding is retired and the dismissal survives Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:00:07 -04:00
bvandeusenandClaude Opus 5	c06af48cd6	feat(library): the duplicate matcher — a pure comparison over fingerprints (M400 #3909) ... Decides whether tracks are proposed as one recording. No database, no files, so every rule is falsifiable in a unit test. Two tiers: - exact: equal audio_stream_sha256 (identical encoded audio bytes). No threshold and no false positives. - acoustic: chromaprint fingerprints that agree once aligned. Two fingerprints can start at slightly different points in the audio (padding trimmed differently), so offsets within ±120 items (~15s) are voted on using items that share their high 14 bits. Bit-error rate is then measured over the overlap at the winning offset. The approach and both constants follow AcoustID's pg_acoustid; it was reimplemented from that description and no code was copied. No verdict below ~10s of overlap, or for low-information fingerprints (silence, a sustained tone). Two such tracks agree without being one recording. Grouping uses complete linkage: a track joins a group only if it matches every member. Otherwise A close to B and B close to C would merge A and C, which are not close, and it means any member can be the survivor. Other rules: - durations must be within 3s - acoustic groups are capped at 8, and larger clusters are reported and discarded as a likely shared jingle - an exact group absorbed into an acoustic one takes the acoustic tier - output does not depend on input order The acoustic threshold is 0.15 bit-error rate: deliberately conservative, since the operator's concern is different recordings of one song being merged, and an instrumental shares its vocal's harmony. It is unmeasured, and needs calibrating against real pairs once the backfill has populated fingerprints (#3913 exposes it). Nothing calls this yet; the sweep (#3910) does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 16:48:06 -04:00
bvandeusenandClaude Opus 5	b8855b480f	feat(library): backfill fingerprints for the existing library — M400 #3908 ... The scan fingerprints only bytes it has not seen, so everything imported before fingerprinting existed, and any row derived by an older fingerprintVersion, needs a pass of its own. That pass is a background worker, not a stage in RunScan. RunScan runs at boot and then every 12h, and an in-flight scan older than an hour is reaped and a second started beside it. A stage would have to stop inside the hour: a few hundred decodes a run, so about a month for a 50k-track library. It would also hold the run in flight and answer manual rescans with 409 while it worked. FingerprintBackfillWorker runs once at start, then hourly. Nothing a pass does (error or panic) can stop the next tick. A pass walks tracks with no fingerprint or a stale version, skipping missing tracks, keyset-paged on id. The cursor is what lets a pass end: an inconclusive attempt writes no row, so a file that keeps timing out would otherwise be re-listed and retried forever. Two decodes at a time, deliberately: they compete with transcoding for CPU and with streaming for the mount. storeFingerprint is now one package function shared by the scan and the worker, and reports whether the attempt was fingerprinted, rejected, inconclusive or failed to store. Progress is a live gauge on the Admin scan card, served by GET /api/admin/library/fingerprints: fingerprinted / rejected / pending of total, with missing tracks excluded so it can reach the end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 14:56:18 -04:00
bvandeusenandClaude Opus 5	d7a8e5f300	fix(library): a track delete that cannot remove its file deletes nothing — #3918 ... Two delete paths had opposite failure policies. tracks.RemoveTrack logged a failed os.Remove and deleted the row anyway, which CASCADEs likes, plays, playlist memberships and tags, while the file survived for the next scan to re-import as a stranger. library.DeleteTrackFile stopped correctly but reported it as a bare 500 nobody could read. One path now: library.DeleteTrackFile removes the file first and, on anything but ErrNotExist, returns *FileRemoveError with nothing deleted. Only then does it delete the row and tidy an emptied album and artist in one transaction, log the sync change and clear orphaned artist art. RemoveTrack calls it, which also fixes RemoveTrack never logging a sync change. Quarantine Delete file now tidies emptied albums and artists too. Both endpoints answer an unwritable library (EROFS, EACCES, EPERM) with 409 library_not_writable. The message names the directory (removal writes to the parent), the uid:gid the server runs as, and that nothing was deleted. Other remove errors are 500 file_delete_failed with the path. The reachable surface is quarantine Delete file, which failed silently: no copy for the code on either client, and Android swallowed the exception so the row just reappeared. Web and Android now have copy for both codes and append the server message for exactly those two. Android's quarantine screen shows it in a snackbar. DELETE /api/admin/tracks/{id} has had no client since f7278f24, which kept it on purpose for a safer admin surface, so its history loss was latent. Fixed rather than removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 14:23:01 -04:00
bvandeusenandClaude Opus 5	cba77a5187	feat(library): fingerprint every new or changed file — M400 #3905-#3907 ... Two identities per track, because they answer different questions: - audio_stream_sha256: SHA-256 of the ENCODED audio packets (ffmpeg -map 0:a -c:a copy -f hash). Equal means identical audio whatever the tags say. Measured against the #3885 pair: the two WWW files hash identically here and differently as whole files. Packets rather than decoded samples, so an ffmpeg upgrade cannot silently change every stored hash, and nothing is decoded. - chromaprint: fpcalc -raw -signed. The same recording at another bitrate or codec, for the acoustic tier. fpcalc ships in the image (libchromaprint-tools); shelled out because CGO_ENABLED=0 rules out bindings. Stored in a track_fingerprints table rather than on tracks: eight queries read tracks with SELECT *, including album pages, search and the Subsonic surface, and a ~4 KB array there would be de-TOASTed on every one of them. The scan fingerprints only bytes it has not seen (a new path, or mtime past the row's). A tag-repair pass leaves fingerprints alone, and unchanged files with no fingerprint are the backfill's job (#3908). Folding that into the skip check would re-decode the whole library on the first scan after upgrade and push a sync change per track. A failure that says nothing about the file (timeout, cancelled scan, tool not installed) is never stored, and on changed bytes it removes the old row. A tool that rejects the file stores NULL at the current version, so the backfill does not retry it every boot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 13:21:15 -04:00
bvandeusen	7ba673ed83	fix(library): tell clients when a file goes missing or comes back — #2704 ... The wire field shipped in 366692a1 was inert. MarkTracksMissing and ClearTracksMissing are plain UPDATEs, and /api/library/sync is a change-log feed: a row that never produces a change row is never re-sent. Clients would have kept their stale copy until an unrelated edit touched the track or the cursor fell out of the retention window and forced a full resync -- so the flag existed and nothing ever told anyone to read it. Found by checking the consumer set rather than the code: the field was threaded end to end and every test passed, because none of them asked the question "how does this reach a client?". Logged BEFORE the mutation, which is the opposite of the scanner's log-after-success pattern, and deliberately so. The failure modes are not symmetric. Log-then-fail-to-mark makes clients re-read a track that has not changed: one wasted fetch. Mark-then-fail-to-log leaves the mark with no change row -- and because both statements are idempotent (missing_since IS NULL / IS NOT NULL guards), the next scan will not retry the pair, so the client never learns. Permanently. A spurious re-read is much the cheaper mistake. Restoring logs too. A file coming back that nobody is told about stays greyed out on every device until something unrelated touches it, which would be a worse bug than the one being fixed. Op is upsert, not delete: the track still exists and keeps its history. Delete would tell clients to drop the row, which is precisely the design #2704 rejected when it chose to ship state instead of filtering the feed. Adds sync.LogChanges alongside LogChange, backed by an unnest batch insert. Every existing caller mutates one entity, so per-row was right for them; reconcile can mark a quarter of a library in one sweep, where a loop would be thousands of round-trips inside an already-slow scan.	2026-08-17 13:04:14 -04:00
bvandeusen	c3f3a17c6d	feat(library): a missing file stays in the playlist, greyed and unplayable — #2527 ... Every browse, discover and mix query filters missing_since, so a track whose file vanished disappears from the places Minstrel chooses music. A playlist is different: the entry is there because the user put it there, and silently dropping it rewrites their list behind their back. So playlists keep the row and mark it instead. ListPlaylistTracks now carries missing_since (still deliberately unfiltered), the service layer surfaces it as PlaylistTrack.Unavailable, and the wire gains "unavailable" on each entry. A missing entry also loses its stream_url. Refusing to hand out a URL that cannot serve is stronger than trusting every client to honour the flag, and "stream_url": null is a shape the clients already model -- PlaylistWire.streamUrl is documented nullable for the track-removed case -- so an older build degrades to "present but not playable" with no change. Nothing is deleted here and nothing should be: the row, its play history, its likes and its taste contribution all survive a file going missing, because the file may come back (and #2528 will adopt it if it comes back renamed). Also corrects two comments that had drifted into lying. delete.go still claimed the file-gone case was NOT auto-reconciled and told admins to delete rows by hand -- untrue since f6d1cf24, and that exact staleness is what produced drift #572. It now says what DeleteTrackFile really is: the destructive admin action, which CASCADEs play_events and likes, and is emphatically not the missing-file path. watcher.go claimed the safety-net scan "covers anything missed"; the walk only covers additions, and it is reconcile that covers removals.	2026-08-16 11:43:05 -04:00
bvandeusen	8e1d25a772	fix(scanner): repair acronym and apostrophe casing on genre tags — #2468 ... Operator decision: keep the ID3v1 table canonical, fix the casing. The operator's library carries "Edm", "Idm", "Aor", "Uk Garage", "Uk Hardcore", "Trap Edm", "Glitch Hop Edm" and "Children'S Music" — an external tag editor title-cased the whole genre field. The "'S" is the giveaway. Fixed at SCAN time, not in the display layer: taste_profile.sql reads tracks.genre directly, so a cosmetic-only fix would leave the taste vocabulary holding "Edm" while the UI showed "EDM", and any correctly tagged file would contribute a second, separate tag. trueUpCasing only ever changes case, never letters, so it cannot silently turn one genre into a different one — that is what separates it from the label-remapping idea this task rejected. Two narrow rules: - A short, evidence-led acronym list, matched case-insensitively so "edm", "Edm" and "EDM" all land on "EDM". This is a deliberate exception to the project's rule that genre case is exposed as the file says it: "Rock" and "rock" still stay separate rows, because folding those is a judgement about labels, whereas there is no genre named "Edm". - Apostrophe suffixes from a FIXED contraction list, so "Children'S" is repaired while "O'Brien" and "D'Angelo" keep their capital. A blanket "lowercase after an apostrophe" would have broken both. Matching uses the word's letter core rather than the raw word, so "(Edm)" and "Edm," are repaired and their punctuation re-attached. Interior punctuation stays in the core, so "Lo-Fi" and "R&B" are compared whole and cannot match a fragment by accident. My first version missed this and a test expecting "(Live EDM)" caught it. Names resolved from the ID3v1 table are deliberately NOT re-cased, per the operator's call — entry 40's "AlternRock" stays as the table spells it, with a test pinning that so a later tidy-up doesn't quietly "fix" it. tagReadVersion 1 -> 2, so this reaches the existing library on the next scan rather than new files only. That re-read reuses stored durations, so it costs tag reads and no ffprobe.	2026-08-07 14:43:36 -04:00
bvandeusen	481f906059	feat(metrics): publish margin of error on every delta — #2495, #2524 ... The metrics card had one volume threshold doing two jobs. recMetricsLowVolume = 20 is a DISPLAY floor — below that a skip rate is anecdote — but the card then presented deltas as though it were also a DECISION floor. Those differ by an order of magnitude: detecting the ~13pp differences that matter needs ~133 plays per arm for 80% power at a=0.05. So Discover's taste-matched (59 plays) and random-unheard (70) both rendered as full-confidence rows with a bold delta beside them, and that comparison sits at p ~ 0.06. The card said "signal"; the arithmetic said "maybe". It produced a recommendation the data didn't support, and any reader with the same numbers would have made the same call. Deltas now carry a 95% margin of error and a `distinguishable` flag, computed server-side so both clients read the same arithmetic instead of each re-deriving it. Skip rate is a two-proportion difference; completion is Welch, which needs a variance — hence completion_sqsum in the query. It is the sum of squares rather than stddev_samp on purpose: raw source rows are merged into surface families in Go, and sums of squares combine across groups exactly whereas standard deviations cannot. recMetricsLowVolume is untouched. "Too thin to show" and "too thin to act on" are different questions. Web renders an indistinguishable delta as dimmed and prefixed "≈", with the range on hover and a legend explaining the glyph. Colour is withheld unless the delta clears its margin — colouring noise red is what made the old card misleading. Breakdown rows go through the same path; those are the thinnest samples on screen and where the old card misled most. Also fixes the admin trends view, which had the same problem worse: its "Latest skip"/"Latest completion" columns are one WEEK while the adjacent Plays column is the whole window. I misread exactly that and briefly concluded Deep cuts was the worst surface, from ~17 plays in a single week — over 180 days it is one of the best. Headers now name their period and the skip cell carries that week's play count. #2524: resolveArtist now recognises a duplicate-MBID unique violation as the expected condition it is, matching resolveAlbum. Two rows mapping to one MusicBrainz artist is a merge candidate, not a fault; without the branch it logged a generic warning plus a Postgres ERROR line on every scan, which teaches an operator to ignore database errors.	2026-08-06 21:08:58 -04:00
bvandeusen	24d330424f	feat(library): adopt moved files instead of forking their history — #2528 ... Track identity was file_path, so a file that came back renamed or in a different directory looked like a deletion plus an unrelated new track: the old row kept the like and every play_event while a fresh zero-history row appeared, and nothing connected them. A liked song read as unliked, its play count reset, and Rediscover could offer it as a discovery — silently. Renumbering an album was enough, which is what happened to the operator's copy of Minutes to Midnight. Adoption re-points the existing row's file_path at the new location and clears its missing mark. The normal UpsertTrack then conflicts on file_path and updates THAT row, so the track id survives and likes, plays and playlist memberships travel with it — and clients see an update rather than a delete-and-create, so no cache churn either. Matching is MBID first (identifies the recording, so it survives a re-encode), then file_size + duration_ms for untagged files. Both fingerprint components must be non-zero: duration_ms is 0 when ffprobe failed, and matching 0 against 0 would pair up unrelated broken files. Only rows already marked missing are eligible — a row whose file is present elsewhere is a duplicate, not a move, and re-pointing it would corrupt the copy that still exists. An ambiguous match inserts fresh rather than adopting one arbitrarily: a fork is recoverable later, a wrong merge isn't. Scan is now three phases, and the order is the point. Adoption can only claim a row that is ALREADY marked missing, but reconcile previously ran after processing — so a rename performed while the server was down surfaced the deletion and the addition in the same scan, the new path inserted first, and the fork became permanent. Enumeration is therefore separated from processing so reconcile can run between them: walk (paths only, no tag reads or probes) -> reconcile -> process in walk order. Consequence worth knowing: when reconcile refuses (an absent root, or a reorganisation exceeding the 25% mark cap) adoption cannot fire and renamed files fork as before. That's the pre-#2528 behaviour rather than a new failure, and the warning now names it. The old outer walk-error branch was unreachable — the callback always returned nil, so WalkDir never surfaced an error — and verifyRootsPresent is the real protection, so enumerate counts walk errors instead of pretending to abort on them.	2026-08-06 15:56:07 -04:00
bvandeusen	f6d1cf24f0	feat(library): detect missing files and stop offering them — #2523 ... Nothing in Minstrel ever noticed a deleted file. The walk only visits paths that exist, so a row whose file was gone was never scanned, never errored, never counted — permanently invisible. classifyEvent ignores fsnotify removals by design, and the safety-net scan is the same walk, so it covers additions only. Rows accumulated forever. Found on the operator's library: a completed scan reported skipped=24185 errored=0 while the MBID backfill (which opens files by DB path rather than walking) logged ~40 "no such file or directory" across three reorganised albums. Those rows also kept their pre-#2499 welded genre, which is how this surfaced — the version-stamped tag re-read can only reach files the walk visits. The harm is not cosmetic. tracks is the candidate universe for recommendation.sql / discover.sql / system_mixes.sql and nothing filtered on file existence, so a mix could spend a slot on a track that cannot stream. Marks rather than deletes. A missing file is a claim about the filesystem and the filesystem lies transiently — an unmounted volume, a network blip, a container that started before its media mount attached. Every sweep in internal/gc resolves a truth INSIDE the database and is safe to run blind; this one is not, so no deletion happens here. Three guards refuse to act on ambiguous evidence: every scan root must resolve to a non-empty directory, the walk must have seen at least one file, and one reconcile may newly mark at most 25% of the library. Clearing a mark is never the dangerous direction, so it runs unconditionally — otherwise a library that tripped the cap could never recover once the mount returned. Only a full Scan reconciles. The walk's set of seen paths is the evidence, and ScanFiles has no basis for concluding anything about files it did not look at. Excludes marked tracks from all 13 track-emitting queries (radio x2, system mixes x5, discover x4, most-played x2), the 6 play-history seed picks, and the genre browse axis. Deliberately NOT filtered: the shared ListPlaylistTracks read path, because it also serves user-curated playlists where hiding a track the user added would be wrong — system playlists shed orphans on their next daily rebuild instead. History and the taste profile also keep them: those record the past, and a track you played 200 times still says something about your taste. Reconcile tallies land in scan_runs so a disappearance is visible rather than discovered when a mix comes up short.	2026-08-06 14:34:53 -04:00
bvandeusen	fd27819cdd	style(scanner): tagged switch on ID3 major version — #2499	2026-08-05 21:22:53 -04:00
bvandeusen	37b396a7e4	fix(scanner): read multi-value genre frames correctly — #2499 ... dhowden/tag's readTFrame splits ID3v2 null-separated multi-value text frames and rejoins them with the EMPTY string, so a file tagged "Alternative Rock" + "Rock" was stored as "Alternative RockRock". It also leaves bare numeric ID3v1 references unresolved, which is why the library showed genres like "4017" and "526617". This corrupted more than the browse axis added in #367: taste_profile.sql reads tracks.genre directly, so the welded tokens were entering the taste profile's tag vocabulary, and recommendation.sql/discover.sql were comparing them as single opaque tags. Genre counts were wrong everywhere. ffprobe is not a fix — ffmpeg's read_ttag calls decode_str once with no loop, keeping only the first value. Truncating multi-genre tags would blunt the similarity signal genre mainly feeds. So the TCON frame is now parsed directly (ID3v2.2/2.3/2.4, all four text encodings, per-frame and tag-level unsynchronisation, numeric and parenthesised ID3v1 references); everything else still comes from dhowden/tag. Values are stored ";"-delimited, which the read side already splits on, so no query changes. Existing rows are repaired without an operator-run rebuild: migration 0054 adds tracks.tag_read_version DEFAULT 0, below the scanner's current tagReadVersion, so the next scan re-reads tags it would otherwise skip on mtime. Such a re-read reuses the stored duration instead of re-running ffprobe, keeping a repair pass tag-read-bound rather than one fork+exec per file. Bumping the constant is how a future extraction fix reaches an existing library. Only ID3v2 is in scope — dhowden welds nowhere else. The Vorbis/MP4 repeated-field question is #2500, unproven and deliberately not built.	2026-08-05 21:17:59 -04:00
bvandeusen	e95138d412	style(server): gofmt watcher_test map alignment	2026-06-06 21:54:14 -04:00
bvandeusen	e994aae613	feat(server): retire scan scheduler for watcher + safety-net walk ... Wire the fsnotify watcher and a fixed 12h safety-net delta walk in main; remove the configurable scan scheduler (scheduler.go, scan_schedule table via migration 0033, GET/PATCH /api/admin/scan/schedule, and the server/api plumbing). Manual scan + scan status are unchanged.	2026-06-06 21:50:08 -04:00
bvandeusen	c39a9ca18f	feat(server): targeted ScanFiles + fsnotify library watcher ... Add Scanner.ScanFiles (watcher-driven targeted scan returning changed album IDs) and a recursive fsnotify Watcher that debounces filesystem events and enriches just the affected albums inline. Pure classifyEvent/drainPending seams unit-tested; ScanFiles covered in the scanner integration test.	2026-06-06 21:43:27 -04:00
bvandeusen	bda0896d82	docs(server): drift #572 — delete.go honest about missing reconcile ... The docstring claimed "the next library scan reconciles missing files by removing their tracks rows" — but scanner.go only does filepath.WalkDir + UpsertTrack; it never enumerates existing rows to check file_path presence, and it never DELETEs orphan rows. The audit verified this — repo-wide grep finds no orphan-sweep code. The lie is load-bearing: lidarrquarantine/service.go:270 leans on this guarantee, so downstream code thinks the orphan case heals itself. Fix the comment to state reality (admin re-trigger or manual cleanup) and reference the open follow-up for adding a real sweep. The actual reconcile pass is a separate piece of work (needs scanrun integration + retention semantics + tests) and stays in the Scribe audit queue.	2026-06-02 18:26:45 -04:00
bvandeusen	47d2f61161	fix: drift audit batch 1 — six small mechanical wins ... Six findings from the 2026-06-02 multi-system drift audit (Scribe parent task #552): - **#553 (web)** Tailwind class fix: web admin playback-errors Delete confirm button was using `bg-action-danger`, an undefined token — swap to `bg-action-destructive` to match every other destructive button. Restored the Oxblood signal that distinguishes Delete from Cancel. - **#555 (web)** Type the `source` field on `play_started` in the EventRequest discriminated union. Server's eventRequest accepts it; web's TS type was missing the slot, so a "drop extra properties" refactor could silently strip the source tag and break system- playlist rotation attribution. - **#556 + #557 (server)** Coverage rollup whitelist was pinned to ('sidecar','embedded','mbcaa','theaudiodb'); migration 0020 added 'deezer' and 'lastfm' as valid cover_art_source values but those never got wired in, so albums with art from those providers silently counted as MISSING in the admin Coverage dashboard. The rollup test was seeding only the pre-0020 sources, masking the gap in CI. Extend the query to include deezer + lastfm; seed the test with one row per valid source (regression-guards future additions). - **#558 (web)** Auth gate was blocking /forgot-password and /reset-password/<token> — both are entered without a session by definition, so the email-link reset flow was bouncing signed-out users to /login. Add /forgot-password to the public set and a /reset-password/ prefix matcher. New tests assert both routes reach their pages without redirect. - **#571 (server)** Library scanner was indexing only .mp3/.m4a/.flac /.ogg while the stream handler (media.go) had been extended to serve .opus, .aac, and .wav. A user with .opus files in their library never saw them in artist/album listings because the scanner skipped indexing — silent data loss. Aligned the scanner to match the media handler. Scribe statuses updated to in_progress; flipping to done after the push since these are mechanical and verified directly against the cited file:lines.	2026-06-02 18:11:25 -04:00
bvandeusenandClaude Opus 4.7	461c6bf514	fix(go): collapse struct-literal copies to type conversions (S1016) ... staticcheck S1016 in the new golangci-lint v2 flagged four sites copying field-by-field between two types with identical struct shapes. Direct type conversion is the canonical form: - internal/library/scanrun.go: - LibraryStageTallies copy from Stats → LibraryStageTallies(lastStats) - MBIDBackfillStageTallies copy from BackfillMBIDsResult → MBIDBackfillStageTallies(lastRes) - internal/recommendation/home.go: - dbq.ListRediscoverAlbumsForUserRow copy from the Fallback row type → dbq.ListRediscoverAlbumsForUserRow(r) - Same pattern for the Artists rediscover pair. No behavior change; the underlying struct shapes are identical (staticcheck verified the conversion is valid). Net -18 +4 lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 15:23:39 -04:00
bvandeusenandClaude Opus 4.7	d212621eaa	test: fix 4 integration-suite root causes surfaced by CI (C+A+B+F) ... Latent failures exposed now that integration tests run in CI (#339). All test/test-harness only — no production code changes. A (dbtest.ResetDB): also truncate cover_art_provider_settings + cover_art_sources_meta. The monotonic source-version counter (seeded 1 by 0018) accumulated across internal/coverart tests → CurrentVersion=4 want 1, key-only-bump assertions, enricher source skips. SettingsService re-seeds both at boot, so a truncated start is the correct fresh state. B (playlists system_test.seedPlayEvent): inserted play_events with a random session_id → play_events_session_id_fkey violation (7 tests). Create the parent play_sessions row in the same statement (CTE). C (similarity worker_integration_test.newTestWorker): built the client with only BaseURL. Post-4fca0e6 similarity hits the Labs API via LabsBaseURL, so an unset LabsBaseURL fell through to the real labs.api and the stub never ran → 0 similarity rows. Set LabsBaseURL to the stub. F (library scanner_test): NOT a flake — deterministic. Synthetic ID3-only MP3s have no decodable duration; the scanner intentionally won't skip duration_ms=0 rows (retries duration backfill), so every re-scan reported Updated not Skipped. Seed duration_ms>0 before the second scan so the mtime-based incremental-skip path under test is actually exercised. Production scanner behavior unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-17 13:32:42 -04:00
bvandeusenandClaude Opus 4.7	005965d6de	feat(coverart): #388 remove global cover-art backfill cap ... Operator feedback (2026-05-09): the 500-album cap meant cover art rolled in over many nightly runs even when local sources (sidecar/embedded) could finish in minutes. Remove the global cap; rely on the existing per-provider HTTP throttle (coverart httpClient MinInterval) so local art is disk-speed and remote providers stay TOS-friendly. - enricher.go / artist_enricher.go: EnrichBatch, EnrichArtistBatch, EnrichRetryMissing now treat limit<0 as unbounded (0 still = stage disabled). The cap was a SQL LIMIT; unbounded uses max int32. - main.go: RunScanConfig EnrichCap/ArtistEnrichCap = -1 (unbounded). - Drop LibraryConfig.CoverArtBackfillCap + the MINSTREL_LIBRARY_COVERART_BACKFILL_CAP env var. - Drop the now-dead coverBackfillCap param threaded through server.New + api.Mount + the handlers struct. - Admin bulk refetch (/api/admin/covers/refetch-missing) now drains unbounded; response {queued:int} → {started:bool} (the count is unknowable synchronously for a fire-and-forget drain). Web copy + client type + Go/web tests updated to match. No doc refs existed (config.example.yaml / docker-compose / README never documented the env var). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-16 18:41:21 -04:00
bvandeusenandClaude Opus 4.7	4fca0e66cb	fix(listenbrainz): similarity hits Labs API, not the 404'ing main API ... Root cause of zero LB recommendations (every similar-recordings AND similar-artists call returned HTTP 404, worker logs): - Wrong host/path: client called api.listenbrainz.org/1/explore/similar-{recordings,artists}/{mbid}. /explore/... is a WEBSITE route, not an API endpoint — it 308s then 404s. Similarity datasets live on the separate Labs API. - Invalid algorithm: the hardcoded session_…_session_30_…_limit_100_filter_True_… is not a permitted Labs enum member (400s) regardless of host. Verified against the live Labs API: GET labs.api.listenbrainz.org/similar-recordings/json ?recording_mbids=<mbid>&algorithm=<algo> GET labs.api.listenbrainz.org/similar-artists/json ?artist_mbids=<mbid>&algorithm=<algo> algorithm=session_based_days_9000_session_300_contribution_5_threshold_15_limit_50_skip_30 → 200 for both. Response field names (recording_mbid/artist_mbid/ name/score) already match the existing structs — parsing unchanged. - Add defaultLabsBaseURL + Client.LabsBaseURL (separate from the main BaseURL; scrobble submission still uses api.listenbrainz.org). - Drop the count/limit query param — result size is encoded in the algorithm name (limit_50); caller still applies its own top-K. - Tests: newTestClient sets LabsBaseURL; the two *_LimitParamSet tests become *_MbidParamSet (assert the Labs path + mbid query param). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-16 15:50:32 -04:00
bvandeusenandClaude Opus 4.7	ca1bc5af62	perf(similarity): worker batch 5→25; track-MBID backfill uncapped ... - similarity.Worker batch 5→25 (tracks AND artists per 1h tick). At 5/h a freshly-scrobbled library took days to build a usable similarity pool; 25/h converges in hours, still well under ListenBrainz rate limits (429 aborts the tick). - scanrun Stage 2b track backfill now runs unbounded (-1) instead of reusing the album backfill's 5000 staged cap. It's a one-time whole-library heal with no progress UI; a cap just left tracks.mbid partially NULL (3634/18056 after one pass) until several future scans caught up, re-reading untagged files each time. One uncapped pass converges; later scans only re-read the remaining NULL rows. Album backfill keeps its 5000 cap (it has staged scan_runs UX). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-16 14:24:45 -04:00