58 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5.5	f13da62797	feat(library): AcoustID lookup client and the compressed fpcalc print (M401 #3919) ... internal/acoustid posts AcoustID's v2 lookup as a gzip form with meta=recordings, one request per 400ms (their limit is 3/s) and a 20s deadline. It returns every linked recording with its best score; choosing among them is the worker's job (D5). The server's error codes map to an invalid key (stop and say so), a rejected fingerprint (a verdict on the track) or unavailable (try again later). A cancelled caller stays a cancellation. fpcalcLookupArgs and parseFpcalcCompressed read fpcalc's default output, the compressed string the lookup takes (D1), always over the first 120s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 23:16:52 -04:00
bvandeusenandClaude Opus 5.5	13a7a3629a	fix(library): MBID backfills skip tracks whose files are missing (#5139) ... The track backfill listed every track with a NULL mbid, missing or not, so each scan tried to open every missing file and logged an "open failed" warning per track. Nothing ever healed. The album backfill could pick a missing track as the one to read, and since that pass is capped per scan, albums stuck that way were retried ahead of the rest every time. Both now read only tracks still on disk; an album with none left is skipped until a scan finds its files again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 21:22:05 -04:00
bvandeusenandClaude Opus 5.5	2e36e70268	feat(android): Sonos/UPnP queue plays leveled URLs, rendered a track ahead (M464 #5002) ... Every URL the Sonos queue loader sends is minted with level=true and the track's album-play verdict from its neighbours in the queue; the server returns the plain stream when leveling is off or changes nothing. The playing track and the one after it are rendered ahead, and each time the renderer moves on, the next is. Server: a mint no longer prerenders on its own. A queue load mints every track, which would have started an ffmpeg render per track at once. The request now carries prerender, and at most two prerenders run at a time; past that they are dropped, since a fetch renders on demand anyway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 19:38:21 -04:00
bvandeusenandClaude Opus 5.5	f34423a0e0	feat: leveled FLAC stream for Sonos/UPnP speakers (M464 #5001) ... Speakers fetch their own audio, so the phone cannot level it. A cast token minted with level=true (and the client's asAlbum, which only the queue holder knows) now returns GET /api/tracks/{id}/leveled.flac: the track rendered by ffmpeg at the user's gain (volume=XdB, plus alimiter at -1 dBFS for a limiter-mode boost), metadata stripped, FLAC at 16 or 24 bits and at most 48 kHz. The gain is computed server-side from the user's preference and the stored loudness, carried as ?g=<centi-dB>&lim=0|1 and signed into the token, so an edited URL does not verify. Unity gains get the plain stream. Renders are written beside the cache file and renamed in, keyed by the source's size and mtime, coalesced per file (singleflight, detached from the requesting speaker so a retry finds the render running), started at mint time so the fetch finds them ready, and evicted least recently used past leveled_cache_mb, a new admin setting (migration 0068, Loudness analysis card). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 19:31:07 -04:00
bvandeusenandClaude Opus 5.5	af36b2f24a	feat: per-user volume leveling preference, synced across devices (M464 #4998) ... Mode (off, auto, track, album), target (-18, -16, -14 LUFS) and boost (within headroom, or fully with a limiter), stored per user on the server so the web player, the Android app and casts apply the same one. - Server: user_normalization_prefs (migration 0067), GET/PUT /api/me/normalization; a whole-body PUT, validated, last write wins. - Web: Settings > Playback > Volume leveling. Saves at once, restores the old choice if the save fails, and caches the value for the player. - Android: Settings card. The device keeps a copy for offline playback (Room v9 with an explicit migration, so the upgrade wipes nothing). Writes are offline-first: shown at once, PUT best effort, queued on failure (NORMALIZATION_SET, collapsed to the newest). A refresh never overwrites a change still queued. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 14:29:40 -04:00
bvandeusenandClaude Opus 5.5	e3aa8629d3	feat(api): deliver loudness gains to every client (M464 #4997) ... ReplayGain 2.0 values (gain to -18 LUFS, linear peak) derived from the stored track and album loudness: - Web: GET /api/tracks/replay-gain?ids=... (up to 200), a lookup the player calls for its queue, rather than a field on every TrackRef surface. - Android: track_gain/track_peak and album_gain/album_peak on the sync views, so cached tracks level offline. Storing a measurement logs a track change, and an album's values moving logs an album change, both before the write (#2704), so caches pick the gains up. - OpenSubsonic: replayGain on every song (album, getSong, search3, starred), as a JSON object and an XML element. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:54:05 -04:00
bvandeusenandClaude Opus 5.5	c2f81bf8df	feat(library): album loudness from the tracks' summed block histograms (M464 #4996) ... Album-mode normalization plays a whole album at one gain. That gain comes from album_loudness (migration 0066): BS.1770's gated loudness over every block on the album, computed by summing the tracks' stored histograms and gating the sum. No audio is decoded again. Album true peak is the loudest track's. - Recomputed by the loudness worker each tick, after the track pass and whether or not analysis is switched on. ListAlbumsNeedingLoudness lists albums whose md5 over (present track id, measurement version and time) no longer matches the stored digest. One comparison covers every way membership changes (scan retag, duplicate merge, delete, missing and restored) without hooking each. - No album value until every present track has a settled measurement, so an album's gain doesn't shift mid-listen as the rest is measured. Silent and unreadable tracks count as settled. - Rows for albums with no present track left are dropped. - The parser and the merge share trimBins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:27:02 -04:00
bvandeusenandClaude Opus 5.5	f3196b3443	feat(library): measure every track's loudness in the background (M464 #4995) ... The first step of loudness normalization: the server measures each track with ffmpeg's EBU R128 filter (true peak, mono as dual mono) and stores the integrated loudness, true peak and loudness range in track_loudness (migration 0065). It also keeps a histogram of the 400 ms gating blocks at 0.1 LU, so album loudness can be computed exactly later with no second decode (#4996). The histogram reproduces ffmpeg's own figure (-10.68 against -10.7 on the captured fixture), and the analyzer logs a warning if the two ever drift. - A background worker, cloned from the fingerprint backfill, measures every track, new ones included. Measuring inline in the scan was dropped: the analysis decodes the whole file, and a large import could pass the scan's one-hour stuck threshold. The scan only deletes a changed file's measurement; the worker ticks every 10 minutes. - Timeouts, the cancel/missing-binary split and settled verdicts follow the fingerprint runner. Silence and undecodable files are stored as verdicts; stalls are retried. The deadline scales with track length. - loudness_settings (enabled, files at once) and an admin card with the coverage gauge, under GET/PUT /api/admin/library/loudness-settings and GET /api/admin/library/loudness. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:01:01 -04:00
bvandeusenandClaude Opus 5.5	327d49428f	feat(auth): store Subsonic API keys hashed; a new key is shown once (M462 #4983) ... users.api_token held each user's apiKey in plaintext and was looked up by equality, so a leaked row or backup handed out working keys. Migration 0063 replaces it with api_token_hash (sha256, hex), computed in place from the existing keys so every Subsonic client keeps working. The key can no longer be read back: GET /api/me/api-token is gone, and POST returns the new key once. Settings shows it right after Regenerate with a copy button and a "won't be shown again" note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 09:42:35 -04:00
bvandeusenandClaude Opus 5	077ae61235	feat(admin): fingerprinting settings — on/off, length, match threshold, concurrency, sweep interval (M400 #3913) ... Rule 25: the fingerprinting knobs move out of source into a DB-backed singleton (migration 0061), edited from a card on the Duplicates page and shared live with the scanner, the backfill and the duplicate sweep through one service instance, so a save needs no restart. The length is the knob that can silently break the library: prints taken at two lengths never match. Each track_fingerprints row now records the length it was taken at, and every reader filters on the current one — the backfill treats another length as stale, the gauge counts it pending, the sweep never streams it. Equivalent to a version bump, except that setting the length back makes rows not yet redone current again. The card warns before a length change re-fingerprints the library. Off stops every decode: the scan takes only the stream hash (a demux, and what recognises a moved file) and stores nothing, dropping a changed file's stale row; the backfill idles. A save also makes a sweep due, since a new threshold or length changes what the same prints group into, and the sweep interval gains slack so an hourly interval on an hourly tick doesn't skip every other tick. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:53:55 -04:00
bvandeusenandClaude Opus 5	c8bf9dc929	refactor(library): move detection matches on the audio hash, not size and duration (M400 #3914) ... A file that comes back renamed or moved keeps its track row, and with it its likes and play history, by being matched to the missing row it replaces (#2528). Untagged files were matched on (file_size, duration_ms), which was never a fingerprint. It could pair two unrelated files that happened to share a byte count and a duration, and it missed a file retagged in place, whose size changes. The only defence was requiring a unique match and otherwise giving up. Now there is a real identity. FindMissingTrackByAudioHash matches a missing track by the SHA-256 of its encoded audio (track_fingerprints, #3906). That survives a rename, a move and a retag, and only an identical recording can match it. adoptMovedTrack takes the new file's hash, which the scan already computes before adoption. The size and duration query and fallback are removed outright, with no second path (rule 22). Unchanged: - MBID first: it identifies the recording and survives a re-encode that even the hash does not - a unique match is still required - an absent hash is never looked up, so unhashable files cannot pair with each other The test fake answers the hash lookup only for the hash it holds, so the tests can tell adoption by identity apart from adoption by coincidence. That includes the case the old pair got wrong: different audio of equal size and duration is not adopted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:31:42 -04:00
bvandeusenandClaude Opus 5	11ef044ef6	feat(library): merge duplicates without losing history (M400 #3911) ... Merge keeps one copy of a duplicate group and removes the rest. Every table that references tracks does so ON DELETE CASCADE, so deleting a duplicate's row outright would silently destroy its likes, plays, playlist entries and tags. The merge moves all of that onto the kept copy first, then deletes the empty row. In one transaction, holding a lock on the group: - repoints play_events, skip_events, contextual_likes, playback_errors, lidarr_requests.matched_track_id and playlist_tracks. The last is keyed by position, so every entry stays where it was. - merges general_likes one per user, dated to the earlier like - takes the union of track_tags, keeping the kept copy's own weight on a shared tag - rewrites track_similarity onto the kept copy, dropping edges that would point a track at itself and keeping the kept copy's existing edge on a collision - lets the kept copy take a recording MBID only the removed copy had - deletes the removed copies' rows, tidies emptied albums and artists, marks the group merged - logs sync changes: track deletes, and like and playlist-track delete/upsert pairs The removed copies' files are deleted first, before any row changes, through the same helper as DeleteTrackFile (now shared, along with the album tidy-up). A merge that left the file behind would be undone by the next scan re-importing it. An unwritable library answers 409 library_not_writable and nothing changes. tracks.Service.MergeDuplicates wraps it with the opt-in Lidarr unmonitor from RemoveTrack, skipped when the removed copy is a second file of the kept copy's own album track: unmonitoring that would stop Lidarr managing the kept file. It writes a duplicate_merge audit row after commit, per the audit package's best-effort contract, naming both paths. POST /api/admin/library/duplicates/{id}/merge takes an optional survivor_track_id (the report's proposal otherwise) and unmonitor. On the report page: - each copy gets a Keep choice, defaulting to the proposed one - Merge needs a second click, on a button that says how many files it removes, with the consequence stated beside an opt-in Lidarr checkbox Integration tests cover: - every piece of history landing on the kept copy exactly: likes deduped at the earlier time, plays and skips counted, playlist position unchanged, tags unioned, similarity rewritten with no duplicate or self-edge, MBID inherited - the removed file gone, and a second merge refused - an unwritable file leaving likes, plays, row and group untouched - a survivor outside the group refused Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:25:06 -04:00
bvandeusenandClaude Opus 5	ff493a8c7d	feat(admin): the duplicates report — review proposed duplicate groups (M400 #3912) ... A new admin tab, Duplicates, beside Missing files: the proposals from the duplicate sweep, with a Sweep now trigger and a Not duplicates dismissal. Nothing on it merges or deletes; the merge is #3911. Each group shows: - whether it is identical audio or the same recording, with a match percentage from the weakest link between members - every copy's format, size, duration, path, and the likes and plays it carries (every user's; this is admin-only, and it is what decides which copy to keep) - the copy proposed to keep, and the rule that chose it The survivor rule is library.ProposeSurvivor, a pure function the merge will reuse: lossless over lossy, then the larger file, then the copy in the library longest, then lowest id. Bitrate is not in it because the scanner never fills tracks.bitrate, and for one recording at one duration a larger file is the higher bitrate. m4a is not counted as lossless: it may be AAC. The reason names the rule that separated first place from second, not every rule the winner passed. An empty report has three causes, and the page says which: still fingerprinting, the sweep has never run, or it ran and found nothing. The sweep's state and the backfill's progress come back with the groups for that reason. Groups left with fewer than two members since the sweep are not shown. GET /api/admin/library/duplicates, POST .../sweep (202, or 409 sweep_in_progress), POST .../{id}/dismiss (404 duplicate_group_not_pending when already resolved). Migration 0060 indexes play_events by track_id. Its only indexes led with user_id, so each copy's play count, and the merge's repointing of play history, would scan the whole table. Web only, like Missing files: Android has no library-health admin screens. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:11:11 -04:00
bvandeusenandClaude Opus 5	6379b6c31d	feat(library): the duplicate sweep — propose duplicate groups from fingerprints (M400 #3910) ... Reads fingerprints, runs them through the matcher, and records proposals in duplicate_groups (migration 0059). Nothing is merged or deleted: a group is a proposal for the admin report (#3912). Streaming. The whole library's fingerprints are hundreds of megabytes, but tracks are only compared within 3s of each other in duration. So candidates stream in (duration_ms, id) order, keyset-paged on a new tracks(duration_ms, id) index. The grouper holds only the tracks within 3s of the oldest one not yet settled. A seed is settled once a track arrives beyond its window, which gives the same result as grouping the whole sorted list. groupDuplicates is rebuilt on the same streamGrouper, so there is one grouping rule and the #3909 tests still cover it. Each fingerprint's alignment index and variety check are computed once instead of for every pair. Exact duplicates are grouped library-wide in SQL. The first member the stream meets stands in for the whole group in the acoustic pass. An exact group caught in an oversize acoustic cluster is still proposed: the acoustic evidence is discarded, identical bytes are not. Re-sweeping: - a group is identified by its sorted member ids, so finding it again refreshes the row in place - a proposal whose members all sat in one dismissed group is not proposed again (a subset repeats the verdict; a superset is new evidence) - a pending proposal no sweep has found again is retired, but only after a complete sweep, and only if an earlier sweep last confirmed it, so two overlapping sweeps cannot delete each other's findings - dismissals are kept DuplicateSweepWorker checks hourly and sweeps only when a fingerprint was written after the last sweep started. TryStartDuplicateSweep guards against two sweeps at once and reaps one stuck in flight for 2h. The sweep row is closed on a detached context with a deadline, so a sweep cancelled at shutdown still records that it ended. The integration test pages one row at a time and checks: - an acoustic pair and an exact pair are found - a track with no fingerprint, a missing track and a near-duration unrelated song are left out - a dismissed group is suppressed while the pending one refreshes without duplicating - a proposal that stops holding is retired and the dismissal survives Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:00:07 -04:00
bvandeusenandClaude Opus 5	c06af48cd6	feat(library): the duplicate matcher — a pure comparison over fingerprints (M400 #3909) ... Decides whether tracks are proposed as one recording. No database, no files, so every rule is falsifiable in a unit test. Two tiers: - exact: equal audio_stream_sha256 (identical encoded audio bytes). No threshold and no false positives. - acoustic: chromaprint fingerprints that agree once aligned. Two fingerprints can start at slightly different points in the audio (padding trimmed differently), so offsets within ±120 items (~15s) are voted on using items that share their high 14 bits. Bit-error rate is then measured over the overlap at the winning offset. The approach and both constants follow AcoustID's pg_acoustid; it was reimplemented from that description and no code was copied. No verdict below ~10s of overlap, or for low-information fingerprints (silence, a sustained tone). Two such tracks agree without being one recording. Grouping uses complete linkage: a track joins a group only if it matches every member. Otherwise A close to B and B close to C would merge A and C, which are not close, and it means any member can be the survivor. Other rules: - durations must be within 3s - acoustic groups are capped at 8, and larger clusters are reported and discarded as a likely shared jingle - an exact group absorbed into an acoustic one takes the acoustic tier - output does not depend on input order The acoustic threshold is 0.15 bit-error rate: deliberately conservative, since the operator's concern is different recordings of one song being merged, and an instrumental shares its vocal's harmony. It is unmeasured, and needs calibrating against real pairs once the backfill has populated fingerprints (#3913 exposes it). Nothing calls this yet; the sweep (#3910) does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 16:48:06 -04:00
bvandeusenandClaude Opus 5	b8855b480f	feat(library): backfill fingerprints for the existing library — M400 #3908 ... The scan fingerprints only bytes it has not seen, so everything imported before fingerprinting existed, and any row derived by an older fingerprintVersion, needs a pass of its own. That pass is a background worker, not a stage in RunScan. RunScan runs at boot and then every 12h, and an in-flight scan older than an hour is reaped and a second started beside it. A stage would have to stop inside the hour: a few hundred decodes a run, so about a month for a 50k-track library. It would also hold the run in flight and answer manual rescans with 409 while it worked. FingerprintBackfillWorker runs once at start, then hourly. Nothing a pass does (error or panic) can stop the next tick. A pass walks tracks with no fingerprint or a stale version, skipping missing tracks, keyset-paged on id. The cursor is what lets a pass end: an inconclusive attempt writes no row, so a file that keeps timing out would otherwise be re-listed and retried forever. Two decodes at a time, deliberately: they compete with transcoding for CPU and with streaming for the mount. storeFingerprint is now one package function shared by the scan and the worker, and reports whether the attempt was fingerprinted, rejected, inconclusive or failed to store. Progress is a live gauge on the Admin scan card, served by GET /api/admin/library/fingerprints: fingerprinted / rejected / pending of total, with missing tracks excluded so it can reach the end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 14:56:18 -04:00
bvandeusenandClaude Opus 5	d7a8e5f300	fix(library): a track delete that cannot remove its file deletes nothing — #3918 ... Two delete paths had opposite failure policies. tracks.RemoveTrack logged a failed os.Remove and deleted the row anyway, which CASCADEs likes, plays, playlist memberships and tags, while the file survived for the next scan to re-import as a stranger. library.DeleteTrackFile stopped correctly but reported it as a bare 500 nobody could read. One path now: library.DeleteTrackFile removes the file first and, on anything but ErrNotExist, returns *FileRemoveError with nothing deleted. Only then does it delete the row and tidy an emptied album and artist in one transaction, log the sync change and clear orphaned artist art. RemoveTrack calls it, which also fixes RemoveTrack never logging a sync change. Quarantine Delete file now tidies emptied albums and artists too. Both endpoints answer an unwritable library (EROFS, EACCES, EPERM) with 409 library_not_writable. The message names the directory (removal writes to the parent), the uid:gid the server runs as, and that nothing was deleted. Other remove errors are 500 file_delete_failed with the path. The reachable surface is quarantine Delete file, which failed silently: no copy for the code on either client, and Android swallowed the exception so the row just reappeared. Web and Android now have copy for both codes and append the server message for exactly those two. Android's quarantine screen shows it in a snackbar. DELETE /api/admin/tracks/{id} has had no client since f7278f24, which kept it on purpose for a safer admin surface, so its history loss was latent. Fixed rather than removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 14:23:01 -04:00
bvandeusenandClaude Opus 5	cba77a5187	feat(library): fingerprint every new or changed file — M400 #3905-#3907 ... Two identities per track, because they answer different questions: - audio_stream_sha256: SHA-256 of the ENCODED audio packets (ffmpeg -map 0:a -c:a copy -f hash). Equal means identical audio whatever the tags say. Measured against the #3885 pair: the two WWW files hash identically here and differently as whole files. Packets rather than decoded samples, so an ffmpeg upgrade cannot silently change every stored hash, and nothing is decoded. - chromaprint: fpcalc -raw -signed. The same recording at another bitrate or codec, for the acoustic tier. fpcalc ships in the image (libchromaprint-tools); shelled out because CGO_ENABLED=0 rules out bindings. Stored in a track_fingerprints table rather than on tracks: eight queries read tracks with SELECT *, including album pages, search and the Subsonic surface, and a ~4 KB array there would be de-TOASTed on every one of them. The scan fingerprints only bytes it has not seen (a new path, or mtime past the row's). A tag-repair pass leaves fingerprints alone, and unchanged files with no fingerprint are the backfill's job (#3908). Folding that into the skip check would re-decode the whole library on the first scan after upgrade and push a sync change per track. A failure that says nothing about the file (timeout, cancelled scan, tool not installed) is never stored, and on changed bytes it removes the old row. A tool that rejects the file stores NULL at the current version, so the backfill does not retry it every boot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 13:21:15 -04:00
bvandeusen	7ba673ed83	fix(library): tell clients when a file goes missing or comes back — #2704 ... The wire field shipped in 366692a1 was inert. MarkTracksMissing and ClearTracksMissing are plain UPDATEs, and /api/library/sync is a change-log feed: a row that never produces a change row is never re-sent. Clients would have kept their stale copy until an unrelated edit touched the track or the cursor fell out of the retention window and forced a full resync -- so the flag existed and nothing ever told anyone to read it. Found by checking the consumer set rather than the code: the field was threaded end to end and every test passed, because none of them asked the question "how does this reach a client?". Logged BEFORE the mutation, which is the opposite of the scanner's log-after-success pattern, and deliberately so. The failure modes are not symmetric. Log-then-fail-to-mark makes clients re-read a track that has not changed: one wasted fetch. Mark-then-fail-to-log leaves the mark with no change row -- and because both statements are idempotent (missing_since IS NULL / IS NOT NULL guards), the next scan will not retry the pair, so the client never learns. Permanently. A spurious re-read is much the cheaper mistake. Restoring logs too. A file coming back that nobody is told about stays greyed out on every device until something unrelated touches it, which would be a worse bug than the one being fixed. Op is upsert, not delete: the track still exists and keeps its history. Delete would tell clients to drop the row, which is precisely the design #2704 rejected when it chose to ship state instead of filtering the feed. Adds sync.LogChanges alongside LogChange, backed by an unnest batch insert. Every existing caller mutates one entity, so per-row was right for them; reconcile can mark a quarter of a library in one sweep, where a loop would be thousands of round-trips inside an already-slow scan.	2026-08-17 13:04:14 -04:00
bvandeusen	c3f3a17c6d	feat(library): a missing file stays in the playlist, greyed and unplayable — #2527 ... Every browse, discover and mix query filters missing_since, so a track whose file vanished disappears from the places Minstrel chooses music. A playlist is different: the entry is there because the user put it there, and silently dropping it rewrites their list behind their back. So playlists keep the row and mark it instead. ListPlaylistTracks now carries missing_since (still deliberately unfiltered), the service layer surfaces it as PlaylistTrack.Unavailable, and the wire gains "unavailable" on each entry. A missing entry also loses its stream_url. Refusing to hand out a URL that cannot serve is stronger than trusting every client to honour the flag, and "stream_url": null is a shape the clients already model -- PlaylistWire.streamUrl is documented nullable for the track-removed case -- so an older build degrades to "present but not playable" with no change. Nothing is deleted here and nothing should be: the row, its play history, its likes and its taste contribution all survive a file going missing, because the file may come back (and #2528 will adopt it if it comes back renamed). Also corrects two comments that had drifted into lying. delete.go still claimed the file-gone case was NOT auto-reconciled and told admins to delete rows by hand -- untrue since f6d1cf24, and that exact staleness is what produced drift #572. It now says what DeleteTrackFile really is: the destructive admin action, which CASCADEs play_events and likes, and is emphatically not the missing-file path. watcher.go claimed the safety-net scan "covers anything missed"; the walk only covers additions, and it is reconcile that covers removals.	2026-08-16 11:43:05 -04:00
bvandeusen	8e1d25a772	fix(scanner): repair acronym and apostrophe casing on genre tags — #2468 ... Operator decision: keep the ID3v1 table canonical, fix the casing. The operator's library carries "Edm", "Idm", "Aor", "Uk Garage", "Uk Hardcore", "Trap Edm", "Glitch Hop Edm" and "Children'S Music" — an external tag editor title-cased the whole genre field. The "'S" is the giveaway. Fixed at SCAN time, not in the display layer: taste_profile.sql reads tracks.genre directly, so a cosmetic-only fix would leave the taste vocabulary holding "Edm" while the UI showed "EDM", and any correctly tagged file would contribute a second, separate tag. trueUpCasing only ever changes case, never letters, so it cannot silently turn one genre into a different one — that is what separates it from the label-remapping idea this task rejected. Two narrow rules: - A short, evidence-led acronym list, matched case-insensitively so "edm", "Edm" and "EDM" all land on "EDM". This is a deliberate exception to the project's rule that genre case is exposed as the file says it: "Rock" and "rock" still stay separate rows, because folding those is a judgement about labels, whereas there is no genre named "Edm". - Apostrophe suffixes from a FIXED contraction list, so "Children'S" is repaired while "O'Brien" and "D'Angelo" keep their capital. A blanket "lowercase after an apostrophe" would have broken both. Matching uses the word's letter core rather than the raw word, so "(Edm)" and "Edm," are repaired and their punctuation re-attached. Interior punctuation stays in the core, so "Lo-Fi" and "R&B" are compared whole and cannot match a fragment by accident. My first version missed this and a test expecting "(Live EDM)" caught it. Names resolved from the ID3v1 table are deliberately NOT re-cased, per the operator's call — entry 40's "AlternRock" stays as the table spells it, with a test pinning that so a later tidy-up doesn't quietly "fix" it. tagReadVersion 1 -> 2, so this reaches the existing library on the next scan rather than new files only. That re-read reuses stored durations, so it costs tag reads and no ffprobe.	2026-08-07 14:43:36 -04:00
bvandeusen	481f906059	feat(metrics): publish margin of error on every delta — #2495, #2524 ... The metrics card had one volume threshold doing two jobs. recMetricsLowVolume = 20 is a DISPLAY floor — below that a skip rate is anecdote — but the card then presented deltas as though it were also a DECISION floor. Those differ by an order of magnitude: detecting the ~13pp differences that matter needs ~133 plays per arm for 80% power at a=0.05. So Discover's taste-matched (59 plays) and random-unheard (70) both rendered as full-confidence rows with a bold delta beside them, and that comparison sits at p ~ 0.06. The card said "signal"; the arithmetic said "maybe". It produced a recommendation the data didn't support, and any reader with the same numbers would have made the same call. Deltas now carry a 95% margin of error and a `distinguishable` flag, computed server-side so both clients read the same arithmetic instead of each re-deriving it. Skip rate is a two-proportion difference; completion is Welch, which needs a variance — hence completion_sqsum in the query. It is the sum of squares rather than stddev_samp on purpose: raw source rows are merged into surface families in Go, and sums of squares combine across groups exactly whereas standard deviations cannot. recMetricsLowVolume is untouched. "Too thin to show" and "too thin to act on" are different questions. Web renders an indistinguishable delta as dimmed and prefixed "≈", with the range on hover and a legend explaining the glyph. Colour is withheld unless the delta clears its margin — colouring noise red is what made the old card misleading. Breakdown rows go through the same path; those are the thinnest samples on screen and where the old card misled most. Also fixes the admin trends view, which had the same problem worse: its "Latest skip"/"Latest completion" columns are one WEEK while the adjacent Plays column is the whole window. I misread exactly that and briefly concluded Deep cuts was the worst surface, from ~17 plays in a single week — over 180 days it is one of the best. Headers now name their period and the skip cell carries that week's play count. #2524: resolveArtist now recognises a duplicate-MBID unique violation as the expected condition it is, matching resolveAlbum. Two rows mapping to one MusicBrainz artist is a merge candidate, not a fault; without the branch it logged a generic warning plus a Postgres ERROR line on every scan, which teaches an operator to ignore database errors.	2026-08-06 21:08:58 -04:00
bvandeusen	24d330424f	feat(library): adopt moved files instead of forking their history — #2528 ... Track identity was file_path, so a file that came back renamed or in a different directory looked like a deletion plus an unrelated new track: the old row kept the like and every play_event while a fresh zero-history row appeared, and nothing connected them. A liked song read as unliked, its play count reset, and Rediscover could offer it as a discovery — silently. Renumbering an album was enough, which is what happened to the operator's copy of Minutes to Midnight. Adoption re-points the existing row's file_path at the new location and clears its missing mark. The normal UpsertTrack then conflicts on file_path and updates THAT row, so the track id survives and likes, plays and playlist memberships travel with it — and clients see an update rather than a delete-and-create, so no cache churn either. Matching is MBID first (identifies the recording, so it survives a re-encode), then file_size + duration_ms for untagged files. Both fingerprint components must be non-zero: duration_ms is 0 when ffprobe failed, and matching 0 against 0 would pair up unrelated broken files. Only rows already marked missing are eligible — a row whose file is present elsewhere is a duplicate, not a move, and re-pointing it would corrupt the copy that still exists. An ambiguous match inserts fresh rather than adopting one arbitrarily: a fork is recoverable later, a wrong merge isn't. Scan is now three phases, and the order is the point. Adoption can only claim a row that is ALREADY marked missing, but reconcile previously ran after processing — so a rename performed while the server was down surfaced the deletion and the addition in the same scan, the new path inserted first, and the fork became permanent. Enumeration is therefore separated from processing so reconcile can run between them: walk (paths only, no tag reads or probes) -> reconcile -> process in walk order. Consequence worth knowing: when reconcile refuses (an absent root, or a reorganisation exceeding the 25% mark cap) adoption cannot fire and renamed files fork as before. That's the pre-#2528 behaviour rather than a new failure, and the warning now names it. The old outer walk-error branch was unreachable — the callback always returned nil, so WalkDir never surfaced an error — and verifyRootsPresent is the real protection, so enumerate counts walk errors instead of pretending to abort on them.	2026-08-06 15:56:07 -04:00
bvandeusen	f6d1cf24f0	feat(library): detect missing files and stop offering them — #2523 ... Nothing in Minstrel ever noticed a deleted file. The walk only visits paths that exist, so a row whose file was gone was never scanned, never errored, never counted — permanently invisible. classifyEvent ignores fsnotify removals by design, and the safety-net scan is the same walk, so it covers additions only. Rows accumulated forever. Found on the operator's library: a completed scan reported skipped=24185 errored=0 while the MBID backfill (which opens files by DB path rather than walking) logged ~40 "no such file or directory" across three reorganised albums. Those rows also kept their pre-#2499 welded genre, which is how this surfaced — the version-stamped tag re-read can only reach files the walk visits. The harm is not cosmetic. tracks is the candidate universe for recommendation.sql / discover.sql / system_mixes.sql and nothing filtered on file existence, so a mix could spend a slot on a track that cannot stream. Marks rather than deletes. A missing file is a claim about the filesystem and the filesystem lies transiently — an unmounted volume, a network blip, a container that started before its media mount attached. Every sweep in internal/gc resolves a truth INSIDE the database and is safe to run blind; this one is not, so no deletion happens here. Three guards refuse to act on ambiguous evidence: every scan root must resolve to a non-empty directory, the walk must have seen at least one file, and one reconcile may newly mark at most 25% of the library. Clearing a mark is never the dangerous direction, so it runs unconditionally — otherwise a library that tripped the cap could never recover once the mount returned. Only a full Scan reconciles. The walk's set of seen paths is the evidence, and ScanFiles has no basis for concluding anything about files it did not look at. Excludes marked tracks from all 13 track-emitting queries (radio x2, system mixes x5, discover x4, most-played x2), the 6 play-history seed picks, and the genre browse axis. Deliberately NOT filtered: the shared ListPlaylistTracks read path, because it also serves user-curated playlists where hiding a track the user added would be wrong — system playlists shed orphans on their next daily rebuild instead. History and the taste profile also keep them: those record the past, and a track you played 200 times still says something about your taste. Reconcile tallies land in scan_runs so a disappearance is visible rather than discovered when a mix comes up short.	2026-08-06 14:34:53 -04:00
bvandeusen	fd27819cdd	style(scanner): tagged switch on ID3 major version — #2499	2026-08-05 21:22:53 -04:00
bvandeusen	37b396a7e4	fix(scanner): read multi-value genre frames correctly — #2499 ... dhowden/tag's readTFrame splits ID3v2 null-separated multi-value text frames and rejoins them with the EMPTY string, so a file tagged "Alternative Rock" + "Rock" was stored as "Alternative RockRock". It also leaves bare numeric ID3v1 references unresolved, which is why the library showed genres like "4017" and "526617". This corrupted more than the browse axis added in #367: taste_profile.sql reads tracks.genre directly, so the welded tokens were entering the taste profile's tag vocabulary, and recommendation.sql/discover.sql were comparing them as single opaque tags. Genre counts were wrong everywhere. ffprobe is not a fix — ffmpeg's read_ttag calls decode_str once with no loop, keeping only the first value. Truncating multi-genre tags would blunt the similarity signal genre mainly feeds. So the TCON frame is now parsed directly (ID3v2.2/2.3/2.4, all four text encodings, per-frame and tag-level unsynchronisation, numeric and parenthesised ID3v1 references); everything else still comes from dhowden/tag. Values are stored ";"-delimited, which the read side already splits on, so no query changes. Existing rows are repaired without an operator-run rebuild: migration 0054 adds tracks.tag_read_version DEFAULT 0, below the scanner's current tagReadVersion, so the next scan re-reads tags it would otherwise skip on mtime. Such a re-read reuses the stored duration instead of re-running ffprobe, keeping a repair pass tag-read-bound rather than one fork+exec per file. Bumping the constant is how a future extraction fix reaches an existing library. Only ID3v2 is in scope — dhowden welds nowhere else. The Vorbis/MP4 repeated-field question is #2500, unproven and deliberately not built.	2026-08-05 21:17:59 -04:00
bvandeusen	e95138d412	style(server): gofmt watcher_test map alignment	2026-06-06 21:54:14 -04:00
bvandeusen	e994aae613	feat(server): retire scan scheduler for watcher + safety-net walk ... Wire the fsnotify watcher and a fixed 12h safety-net delta walk in main; remove the configurable scan scheduler (scheduler.go, scan_schedule table via migration 0033, GET/PATCH /api/admin/scan/schedule, and the server/api plumbing). Manual scan + scan status are unchanged.	2026-06-06 21:50:08 -04:00
bvandeusen	c39a9ca18f	feat(server): targeted ScanFiles + fsnotify library watcher ... Add Scanner.ScanFiles (watcher-driven targeted scan returning changed album IDs) and a recursive fsnotify Watcher that debounces filesystem events and enriches just the affected albums inline. Pure classifyEvent/drainPending seams unit-tested; ScanFiles covered in the scanner integration test.	2026-06-06 21:43:27 -04:00
bvandeusen	bda0896d82	docs(server): drift #572 — delete.go honest about missing reconcile ... The docstring claimed "the next library scan reconciles missing files by removing their tracks rows" — but scanner.go only does filepath.WalkDir + UpsertTrack; it never enumerates existing rows to check file_path presence, and it never DELETEs orphan rows. The audit verified this — repo-wide grep finds no orphan-sweep code. The lie is load-bearing: lidarrquarantine/service.go:270 leans on this guarantee, so downstream code thinks the orphan case heals itself. Fix the comment to state reality (admin re-trigger or manual cleanup) and reference the open follow-up for adding a real sweep. The actual reconcile pass is a separate piece of work (needs scanrun integration + retention semantics + tests) and stays in the Scribe audit queue.	2026-06-02 18:26:45 -04:00
bvandeusen	47d2f61161	fix: drift audit batch 1 — six small mechanical wins ... Six findings from the 2026-06-02 multi-system drift audit (Scribe parent task #552): - **#553 (web)** Tailwind class fix: web admin playback-errors Delete confirm button was using `bg-action-danger`, an undefined token — swap to `bg-action-destructive` to match every other destructive button. Restored the Oxblood signal that distinguishes Delete from Cancel. - **#555 (web)** Type the `source` field on `play_started` in the EventRequest discriminated union. Server's eventRequest accepts it; web's TS type was missing the slot, so a "drop extra properties" refactor could silently strip the source tag and break system- playlist rotation attribution. - **#556 + #557 (server)** Coverage rollup whitelist was pinned to ('sidecar','embedded','mbcaa','theaudiodb'); migration 0020 added 'deezer' and 'lastfm' as valid cover_art_source values but those never got wired in, so albums with art from those providers silently counted as MISSING in the admin Coverage dashboard. The rollup test was seeding only the pre-0020 sources, masking the gap in CI. Extend the query to include deezer + lastfm; seed the test with one row per valid source (regression-guards future additions). - **#558 (web)** Auth gate was blocking /forgot-password and /reset-password/<token> — both are entered without a session by definition, so the email-link reset flow was bouncing signed-out users to /login. Add /forgot-password to the public set and a /reset-password/ prefix matcher. New tests assert both routes reach their pages without redirect. - **#571 (server)** Library scanner was indexing only .mp3/.m4a/.flac /.ogg while the stream handler (media.go) had been extended to serve .opus, .aac, and .wav. A user with .opus files in their library never saw them in artist/album listings because the scanner skipped indexing — silent data loss. Aligned the scanner to match the media handler. Scribe statuses updated to in_progress; flipping to done after the push since these are mechanical and verified directly against the cited file:lines.	2026-06-02 18:11:25 -04:00
bvandeusenandClaude Opus 4.7	461c6bf514	fix(go): collapse struct-literal copies to type conversions (S1016) ... staticcheck S1016 in the new golangci-lint v2 flagged four sites copying field-by-field between two types with identical struct shapes. Direct type conversion is the canonical form: - internal/library/scanrun.go: - LibraryStageTallies copy from Stats → LibraryStageTallies(lastStats) - MBIDBackfillStageTallies copy from BackfillMBIDsResult → MBIDBackfillStageTallies(lastRes) - internal/recommendation/home.go: - dbq.ListRediscoverAlbumsForUserRow copy from the Fallback row type → dbq.ListRediscoverAlbumsForUserRow(r) - Same pattern for the Artists rediscover pair. No behavior change; the underlying struct shapes are identical (staticcheck verified the conversion is valid). Net -18 +4 lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-20 15:23:39 -04:00
bvandeusenandClaude Opus 4.7	d212621eaa	test: fix 4 integration-suite root causes surfaced by CI (C+A+B+F) ... Latent failures exposed now that integration tests run in CI (#339). All test/test-harness only — no production code changes. A (dbtest.ResetDB): also truncate cover_art_provider_settings + cover_art_sources_meta. The monotonic source-version counter (seeded 1 by 0018) accumulated across internal/coverart tests → CurrentVersion=4 want 1, key-only-bump assertions, enricher source skips. SettingsService re-seeds both at boot, so a truncated start is the correct fresh state. B (playlists system_test.seedPlayEvent): inserted play_events with a random session_id → play_events_session_id_fkey violation (7 tests). Create the parent play_sessions row in the same statement (CTE). C (similarity worker_integration_test.newTestWorker): built the client with only BaseURL. Post-4fca0e6 similarity hits the Labs API via LabsBaseURL, so an unset LabsBaseURL fell through to the real labs.api and the stub never ran → 0 similarity rows. Set LabsBaseURL to the stub. F (library scanner_test): NOT a flake — deterministic. Synthetic ID3-only MP3s have no decodable duration; the scanner intentionally won't skip duration_ms=0 rows (retries duration backfill), so every re-scan reported Updated not Skipped. Seed duration_ms>0 before the second scan so the mtime-based incremental-skip path under test is actually exercised. Production scanner behavior unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-17 13:32:42 -04:00
bvandeusenandClaude Opus 4.7	005965d6de	feat(coverart): #388 remove global cover-art backfill cap ... Operator feedback (2026-05-09): the 500-album cap meant cover art rolled in over many nightly runs even when local sources (sidecar/embedded) could finish in minutes. Remove the global cap; rely on the existing per-provider HTTP throttle (coverart httpClient MinInterval) so local art is disk-speed and remote providers stay TOS-friendly. - enricher.go / artist_enricher.go: EnrichBatch, EnrichArtistBatch, EnrichRetryMissing now treat limit<0 as unbounded (0 still = stage disabled). The cap was a SQL LIMIT; unbounded uses max int32. - main.go: RunScanConfig EnrichCap/ArtistEnrichCap = -1 (unbounded). - Drop LibraryConfig.CoverArtBackfillCap + the MINSTREL_LIBRARY_COVERART_BACKFILL_CAP env var. - Drop the now-dead coverBackfillCap param threaded through server.New + api.Mount + the handlers struct. - Admin bulk refetch (/api/admin/covers/refetch-missing) now drains unbounded; response {queued:int} → {started:bool} (the count is unknowable synchronously for a fire-and-forget drain). Web copy + client type + Go/web tests updated to match. No doc refs existed (config.example.yaml / docker-compose / README never documented the env var). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-16 18:41:21 -04:00
bvandeusenandClaude Opus 4.7	4fca0e66cb	fix(listenbrainz): similarity hits Labs API, not the 404'ing main API ... Root cause of zero LB recommendations (every similar-recordings AND similar-artists call returned HTTP 404, worker logs): - Wrong host/path: client called api.listenbrainz.org/1/explore/similar-{recordings,artists}/{mbid}. /explore/... is a WEBSITE route, not an API endpoint — it 308s then 404s. Similarity datasets live on the separate Labs API. - Invalid algorithm: the hardcoded session_…_session_30_…_limit_100_filter_True_… is not a permitted Labs enum member (400s) regardless of host. Verified against the live Labs API: GET labs.api.listenbrainz.org/similar-recordings/json ?recording_mbids=<mbid>&algorithm=<algo> GET labs.api.listenbrainz.org/similar-artists/json ?artist_mbids=<mbid>&algorithm=<algo> algorithm=session_based_days_9000_session_300_contribution_5_threshold_15_limit_50_skip_30 → 200 for both. Response field names (recording_mbid/artist_mbid/ name/score) already match the existing structs — parsing unchanged. - Add defaultLabsBaseURL + Client.LabsBaseURL (separate from the main BaseURL; scrobble submission still uses api.listenbrainz.org). - Drop the count/limit query param — result size is encoded in the algorithm name (limit_50); caller still applies its own top-K. - Tests: newTestClient sets LabsBaseURL; the two *_LimitParamSet tests become *_MbidParamSet (assert the Labs path + mbid query param). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-16 15:50:32 -04:00
bvandeusenandClaude Opus 4.7	ca1bc5af62	perf(similarity): worker batch 5→25; track-MBID backfill uncapped ... - similarity.Worker batch 5→25 (tracks AND artists per 1h tick). At 5/h a freshly-scrobbled library took days to build a usable similarity pool; 25/h converges in hours, still well under ListenBrainz rate limits (429 aborts the tick). - scanrun Stage 2b track backfill now runs unbounded (-1) instead of reusing the album backfill's 5000 staged cap. It's a one-time whole-library heal with no progress UI; a cap just left tracks.mbid partially NULL (3634/18056 after one pass) until several future scans caught up, re-reading untagged files each time. One uncapped pass converges; later scans only re-read the remaining NULL rows. Album backfill keeps its 5000 cap (it has staged scan_runs UX). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-16 14:24:45 -04:00
bvandeusenandClaude Opus 4.7	e2432caa65	fix(library): extract recording MBID → tracks.mbid; unblock LB similarity ... tracks.mbid was 100% NULL: the scanner only extracted album + artist MBIDs, never the recording MBID. The ListenBrainz similarity worker is gated on tracks.mbid IS NOT NULL, so track_similarity could never populate — starving For-You/radio's strongest candidate source (lb_similar) on every library. - mbids.go: add extractRecordingMBID (mbz.Recording / Picard musicbrainz_recordingid). Separate fn so extractMBIDs' signature + unit tests stay untouched. - scanner.go: persist recording MBID via UpsertTrack (heals on the file_path conflict, so re-scans backfill for free). - BackfillTrackMBIDs: one-shot pass mirroring BackfillMBIDs, wired as scan Stage 2b (idempotent via SetTrackMbidIfNull, gated by BackfillCap, log-only progress). - migration 0029: tracks_mbid_unique (0002, written when the column was always NULL) wrongly assumes one MBID == one track. A recording appears on multiple releases, so rows legitimately share a recording MBID. Replace with a non-unique partial index. Zero-risk: column is 100% NULL at migration time. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-16 13:49:38 -04:00
bvandeusenandClaude Opus 4.7	f7dfeff256	feat(#392): scan run lifecycle events on the SSE bus ... Slice 3c — completes the server-side producer set. Publishes scan.run_started when InsertScanRun succeeds and scan.run_finished after FinishScanRun completes (success or with an error_message payload). Per-file progress events are intentionally NOT emitted — they'd flood the stream during large library scans without giving the admin scan card anything actionable. The two-beat signal gives the UI enough to invalidate scanStatusProvider at the right moments. Bus access uses a package-level setter on internal/library because threading bus through RunScan + TryStartScan + Scheduler + all their callers would touch ~10 sites without changing behavior at the boundaries that don't publish. Per-process singleton, matches the log.SetDefault idiom. cmd/minstrel/main.go calls library.SetEventBus(bus) once at startup; test contexts that never call it skip publishing safely (publishScanEvent is a no-op when bus is nil). This completes the server side of #392. Slice 4 wires the Flutter consumer: live_events_provider.dart (StreamProvider) + live_events_dispatcher.dart (event-kind → provider invalidation) + AppLifecycleState cold-start invalidation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-12 21:52:06 -04:00
bvandeusenandClaude Opus 4.7	0fa7dc7982	feat(server): scanner + DeleteTrackFile write library_changes ... Wires sync.LogChange into the library mutation sites so /api/library/sync reflects upserts and deletes. Architectural pivot: LogChange's signature is now (ctx, dbq.DBTX, ...) so it works with both *pgxpool.Pool and pgx.Tx. The scanner doesn't run mutations in explicit transactions, so it pool-binds; delete.go matches. Tx-bound callers (likes/playlists in subsequent commits) keep atomicity. Also: sync.FormatUUID centralizes the pgtype.UUID → canonical string conversion that both the scanner and the sync handler need; library_sync.go now uses it instead of a local copy. Best-effort logging on scanner failures (Warn, don't fail the scan): a LogChange error after a successful upsert is rare and self-healing — the next scan that touches the entity re-emits the change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-09 22:39:58 -04:00
bvandeusen	876f994904	fix(server/m7-recurring-scans): early-return in TryStartScan reap branch ... revive flagged the if/else where the else branch was a return — invert the condition and return the non-stale skip path early so the reap flow drops one indent level.	2026-05-06 23:03:21 -04:00
bvandeusenandClaude Sonnet 4.6	4607ce6db4	feat(server/m7-recurring-scans): Scheduler goroutine + boot wiring ... Adds ScheduleConfig + NextFire (per-mode logic for off/interval/ daily/weekly) and the Scheduler struct + loop goroutine. The loop reads scan_schedule on boot and on Refresh signals, computes next-fire, sleeps via time.Timer, and calls TryStartScan when the timer fires. Skip-on-conflict logging happens inside tryFire when TryStartScan reports started=false with a non-stale in-flight row. Boot wiring: cmd/minstrel/main.go constructs the scheduler after coverEnricher and calls Start with the server's long-lived ctx. The scheduler is not yet plumbed into the HTTP handlers (the next task does that — handlers gain a scheduler field, admin schedule endpoints land). Tests cover the NextFire truth table (off / interval / daily today + tomorrow / weekly today-before-time + today-after-time + upcoming day) plus invalid configs and parseHHMM edge cases. Refresh is verified non-blocking via the buffered-channel + default pattern. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-06 22:20:56 -04:00
bvandeusen	8784e938bd	feat(server/m7-recurring-scans): TryStartScan helper + reaper ... Extracts the in-flight check + scan-start logic from handleTriggerScan into a shared library.TryStartScan helper used by both manual and (future) scheduled paths. Folds in the lazy 1-hour stuck-scan reaper: in-flight rows older than StuckScanThreshold get FinishScanRun-ed with error_message="reaped (stale)" and the new scan proceeds. handleTriggerScan becomes a thin wrapper preserving the same external HTTP behavior (202/409/500). Operators no longer need to manually clear stuck rows after a server crash — the next manual trigger reaps it. Tests gated on MINSTREL_TEST_DATABASE_URL: no-in-flight starts; fresh-in-flight skips with row pointer; stale-in-flight reaps (verified via finished_at + error_message); DB error propagates.	2026-05-06 22:15:19 -04:00
bvandeusenandClaude Sonnet 4.6	3e9e46f190	feat(server/m7-scan-progress): wire stagePublisher into 4 scan stages ... Each of the 4 stage workers (library walk, MBID backfill, cover enrich, artist art enrich) gains a progressCb parameter that receives a snapshot of the current running state per item processed. The orchestrator stores the snapshot in a local var that the publisher's marshal closure reads at flush time. scanrun.go's 4 stage blocks are rewritten to construct one publisher per stage, pass a Tick-firing closure to the worker as progressCb, and call Flush() after the worker returns (routing any persist error to captureErr — preserves today's diagnostic behavior). The previous post-stage UpdateScanRun* calls are removed; the publisher's Flush handles the final write. Cadence: every 500 items OR every 1s, whichever fires first. Outside-orchestrator callers (scanner_test.go, enricher_test.go) pass nil for progressCb — no-op behavior preserved for tests that don't need progress observability. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-06 21:14:50 -04:00
bvandeusen	7b9ea131a2	feat(server/m7-scan-progress): stagePublisher helper ... Persists a stage tally to scan_runs at a bounded cadence — every flushEvery items OR every flushPeriod elapsed, whichever fires first. Caller supplies marshal + persist closures so each stage owns its own tally type. Tick swallows mid-stage persist errors silently (observability bug, not scan failure); Flush propagates errors so the orchestrator's captureErr can surface them. Tests cover: count-based flush, time-based flush, Flush always persists, Tick swallows errors and keeps firing, Flush propagates errors, counters reset after each flush.	2026-05-06 21:10:33 -04:00
bvandeusenandClaude Sonnet 4.6	00cd28e79b	feat(server,web/m7-cover-sources): scan-orchestrator 4th stage ... Adds an artist-art-enrich stage after cover-enrich in RunScan. The stage drains EnrichArtistBatch with a configurable cap and the operator's data_dir, persists processed/succeeded/failed tallies to the scan_runs.artist_art_enrich jsonb column added in migration 0018. The admin /admin Library Scan card grid expands from 3 to 4 columns with the new "Artist art" card mirroring the cover-enrichment one (Processed / Succeeded / Failed). The TS ScanStatus type gains an optional artist_art_enrich field. The Go scanStatusResp gains the matching ArtistArtEnrich json.RawMessage field. cmd/minstrel/main.go (and admin scan-trigger handler) wire the new RunScanConfig fields: ArtistEnrichCap reuses cfg.Library.CoverArtBackfillCap for now (can be split if separate budgets are useful), DataDir is cfg.Storage.DataDir. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-06 13:03:36 -04:00
bvandeusenandClaude Opus 4.7	4481874482	fix(server,web/m7-382): handle duplicate-MBID conflicts during backfill ... Forward-fix from running the prior two commits in production. Backfill on a real library exposed two further issues: (1) gofmt -s flagged the column-aligned stubMeta methods in mbids_test.go. Switched to the gofmt-canonical single-space form. (2) Many albums failed to heal with SQLSTATE 23505 (unique constraint albums_mbid_unique). Cause: the operator's library has duplicate album rows in the DB — same MusicBrainz release, split into multiple rows by the pre-MBID scanner because of subtle title or artist disagreements between files. When backfill now correctly extracts the MBID, two rows want to claim the same one and the partial unique index rejects the second. The conflict is correct DB behavior — we shouldn't have two rows with the same MBID — but it's not a write failure to alarm the operator about. Detect 23505 specifically: - downgrade the log line from Warn to Info - track these in a new BackfillMBIDsResult.Duplicates counter (separate from Skipped, which retains its "no MBID in tag" meaning) - leave the duplicate row's mbid NULL; merging duplicates is a separate (future) operator workflow Same handling threaded through the scanner heal path so a regular rescan doesn't generate the noise either. JSON tally + admin UI gain a "Duplicates" line so the operator can see how many duplicate rows their library carries. Follow-up scope (separate task): a duplicate-album merge UX that reparents the duplicate's tracks to the canonical row and deletes the orphaned album row. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-04 21:45:15 -04:00
bvandeusenandClaude Opus 4.7	cc12ab7e96	fix(server/m7-382): strip NUL bytes from MBIDs before persist ... Follow-up to 947f944. With the dhowden/tag/mbz extractor now correctly matching ID3v2 TXXX frames, mbid backfill started rejecting every row with "invalid byte sequence for encoding UTF8: 0x00 (SQLSTATE 22021)". Cause: dhowden's readTextWithDescrFrame (id3v2frames.go:454) — the path that decodes TXXX text — does NOT strip the trailing/embedded NUL bytes that ID3v2.4 uses as frame terminator and multi-value separator. Unlike readTFrame, which does (line 313). So a TXXX:MusicBrainz Album Id of "abc-123\x00" comes back to us verbatim, and Postgres refuses to store NULs in text columns. cleanMBID splits on NUL and returns the first non-empty segment, which also handles the multi-value case (collaboration artist IDs are NUL-separated; Minstrel uses the primary for MBCAA, so first wins). Tests cover the trailing-NUL, multi-value, and all-NUL paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-04 21:35:01 -04:00
bvandeusenandClaude Opus 4.7	947f944fe9	fix(server/m7-382): MBID extraction never matched dhowden/tag's Raw() keys ... The hand-rolled extractor in internal/library/mbids.go looked up keys that dhowden/tag never produces, for every supported audio format: Vorbis (FLAC/OGG): dhowden lowercases keys at parse time (vorbis.go:59), we looked up MUSICBRAINZ_ALBUMID. Always missed. ID3v2 (MP3): dhowden stores TXXX frames under bare TXXX/TXXX_N keys with *tag.Comm values; the Picard tag name is on the value's Description field, not part of the key. We looked up TXXX:MusicBrainz Album Id. Always missed. MP4 (M4A): dhowden stores freeform iTunes atoms under their bare sub-name. We looked up ----:com.apple.iTunes:MusicBrainz Album Id. Always missed. Net effect: every album in the m7-380 boot backfill reported processed=N, healed=0, skipped=N. The m7-379 enricher gate then short-circuited NULL on every album, so MBCAA was never queried. Symptom: "all albums skipped" during cover enrichment. The pre-existing unit tests passed because they hand-built Raw() maps with the intended-but-incorrect keys, never round-tripping through dhowden's parser. Replace the hand-rolled extractor with dhowden's own mbz sub-package, which knows the per-format Raw() conventions and uses lowercase canonical keys (mbz.Album, mbz.Artist). Update both call sites to pass the full tag.Metadata instead of just Raw(). Rewrite the test with a stubMeta implementing tag.Metadata, asserting against the actual per-format Raw() shape, plus a regression guard that pins "uppercase Vorbis keys must not match" so the v0 bug can't sneak back. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-04 21:15:51 -04:00
bvandeusenandClaude Sonnet 4.6	65a3c4892c	feat(server/m7-381): RunScan orchestrator chains library+backfill+enrich ... Adds internal/library/scanrun.go with RunScan, which creates a scan_runs row and sequences file-walk → MBID backfill → cover enrich, persisting per-stage tallies as jsonb. Extends coverart.EnrichBatch to return (processed, succeeded, failed, err) so the orchestrator can classify outcomes. Replaces main.go's two separate boot goroutines with a single RunScan call; passes nil scanner in the no-startup-scan branch. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-04 20:08:48 -04:00
bvandeusenandClaude Sonnet 4.6	34615fffbd	feat(server/m7-380): one-shot MBID backfill worker for existing libraries ... Adds a boot-time goroutine that walks albums with NULL mbid, re-reads tags from one track per album via dhowden/tag, and persists album + artist MBIDs. Healed albums also get their cover_art_source='none' cleared so the enricher's next batch retries the MBCAA fetch. Caps at 5000 albums per boot to avoid stalling startup on huge libraries. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-04 19:40:27 -04:00