181 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5.5	a4fbe44bd5	feat: the Wanted view lists artists A–Z with their albums inside, each album's tracks a click away (M502 #5572) ... The list runs like the library: artists by the library's sort name, each artist's albums oldest first, grouped under the artist (linked into the library when it holds them). A filter narrows it to an artist or album by name, on the server, so it finds what has not been scrolled to. Tracks opens an album's track list: the library's own for an album it holds, a lost file marked, otherwise MusicBrainz's standard edition, named. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 18:43:32 -04:00
bvandeusenandClaude Opus 5.5	bf409d5f3f	fix: a second copy of a track counts once, joining words don't part two titles, and a newer matcher retries an older one's refusals (M502 #5569) ... The second run's refusals were mostly albums holding a track twice (a two-track single with three files), which no edition can pair, and release names that differ only in "+" against "/". Tracks are now read once per disc, number and title, and "and", "&" and "+" count for nothing when titles are compared; an edition's own title also counts as the album's name. library_album_matches records which matcher version decided each album. Refusals by an older version are matched again, even after the listing is done, so an improvement reaches them without a migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 17:44:06 -04:00
bvandeusenandClaude Opus 5.5	b887ab759b	fix: every library album is matched to MusicBrainz before it is listed, by its tags, its artist's discography or a search, and checked against its tracks (M502 #5569) ... The one-time listing took only albums whose tags already named their release group and artist, and left 340 off the list. The rest are now matched: the release their tags name, else the artist's release groups called what the album is called, else a MusicBrainz search by artist and title. A group found by name is taken only when its tracks agree with the album's (by disc and number, then title; length and title, or recording), so a partial copy still matches and a same-titled other album does not. library_album_matches records each album the listing decided, listed or unmatched with why, so an album taken off the list is never re-listed. The albums no match was found for are named on the Held tab. Album rows with no tracks are no longer counted. The length and title checks move from the importer to release, shared by both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 17:16:05 -04:00
bvandeusenandClaude Opus 5.5	2c677b3184	feat: the library's albums go on the wanted list as held, once; after that the list is the truth (M502 #5567) ... A one-time job in the wanted worker lists every album in the library as a held want, one per release group, with a record that it happened (acquisition_settings.library_listed_at). Albums whose tags name only a release are named on MusicBrainz first, 120 a pass; what nothing names is counted and shown on the Held tab. Re-acquisition now wants back only albums on the list. Taking a held album off the list leaves its files, and it is never fetched back; albums put in the library folder by hand afterwards are not added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 16:13:46 -04:00
bvandeusenandClaude Opus 5.5	8245f4406e	feat: download lifecycle — stalled and failed downloads are given up and searched again; imported ones seed, then go (M502 #5523) ... Ported from Fabled Librarian's System 104. A new lifecycle poller, every ten minutes: - a download the client fails is given up at once; - one nobody shares, or still fetching its file list, past the stall setting (6 h) is asked for again three times, then given up; - giving up removes the download with its files, holds the release back for a week (release_blocks.expires_at), ends the grab and searches the album again, with no limit per album; - a running download the client stops listing for half an hour ends and is searched again, not held back; an empty list is not taken as gone; - an imported download seeds to ratio 2.0 or 14 days, then is removed; - a download the client would not remove is asked again every pass. Every condition and act is one open row per (client, download, condition) in download_interventions. Activity shows what is wrong with a download and how often it was retried, why one ended, its seeding, and a banner for a client that does not answer. A running download can be given up by hand (set aside for good). The seed ratio, seed days and stall hours are on the search settings card. The download client lookup moves to acquire.DownloadClientFor, shared by the importer and the lifecycle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 15:27:22 -04:00
bvandeusenandClaude Opus 5.5	ebbc49b902	feat: import with verification, and a Wanted view of its own (M502 #5522) ... A finished download is checked against MusicBrainz before anything is filed: the files are paired with an edition's tracks by position or path, and AcoustID, tag ids, lengths and titles say whether it is the album. What is the album is copied into the library as Artist/Album (Year)/Artist - Album - NN - Title, tagged with Picard's MusicBrainz ids, and scanned; the download keeps seeding. What is not is refused: set aside, removed from the client, and the album searched for again. A download the check cannot settle waits for a person, who imports it anyway or refuses it. Tracks the library already holds are skipped. The wanted list leaves Admin for a top-level Wanted view, admin-only and web-only, with tabs for Wanted, Activity (downloads and their imports, the ones needing a person first), Missing files, Held, Following (with the release-type profiles, moved off Integrations) and Blocked. The search settings gain "File new music into" when the library has several folders. The search pass logs why each failed search failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 10:36:21 -04:00
bvandeusenandClaude Opus 5.5	89b3b44779	feat: search and grab — music release names, a quality ladder, a paced search schedule and one live grab per want (M502 #5521) ... - internal/release: reads a release name (format, bitrate, bit depth and sample rate, source, year, video, discography) and judges it against the album wanted: the artist and title must be in it; video rips, discographies, other versions (drumless, instrumental, karaoke, remix, live unless the album is live…) and singles or EPs for an album are refused. Quality tiers on Lidarr's ladder; a .torrent's file list is read for format, video files and a track count short of the album. - internal/acquire: fan-out across every search source with a narrowed search kept, a process-wide pacer per source, every candidate judged and kept with its reasons, and the grab: one live grab per want, tie-broken (a person's grab beats the schedule's; a better release replaces a worse one unless it is nearly done). Each download lands in its own folder under the client's downloads folder. - internal/wanted: the search schedule, every ten minutes, never-searched first, then requests, missing files and followed artists, taking the best release when auto-grab is on (the default). - Migration 0082: acquisition_settings (auto-grab, interval, allowed qualities seeded from Lidarr's "Any"), want search columns, want_candidates, grabs, release_blocks. - Web: the Wanted page shows each want's last search, a Search button and every release found with its verdict (Grab, Grab anyway, Never this one, Allow again). Integrations gains "Searching and grabbing". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 08:56:47 -04:00
bvandeusenandClaude Opus 5.5	04515a977c	feat: a wanted list — follow artists on release-type profiles; requests and re-acquisition become wants (M502 #5520) ... Minstrel keeps its own list of what it is to get, keyed by MusicBrainz release group. A want comes from an approved request, a new release by a followed artist, or an album whose files went missing, and is held once the library has a full edition of it. - Migration 0081: release_type_profiles (Standard seeded: Album, EP, Single, studio only), follows, follow_seen_groups and wants. lidarr_requests becomes requests with plain *_mbid columns; approved album and track requests move onto the wanted list, approved artist requests become follows, and the library's album artists start out followed. missing_reacquisitions and the retry settings go. - internal/wanted: follows are checked against MusicBrainz discographies daily. The first check sets a baseline, and only releases after it are wanted unless the follow asked for what already exists. Profiles filter by primary and secondary type, so live recordings stay out unless a profile ticks Live. - internal/requests (was lidarrrequests): approving puts the request on the wanted list, and an artist request can be approved on a chosen profile. A request completes when its want is held. - The re-acquisition sweeper wants a missing album back instead of asking Lidarr; searching for it is the search schedule's (#5521). - Web: an admin Wanted page (Wanted, Missing files, Held, Following), a follow control on artist pages, a release-type profiles card on Integrations, "Approve on a profile" on Requests, and want state on Missing files. The artist detail API returns the artist's MBID. - Android: the request body sends artist_mbid, album_mbid and track_mbid. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 00:46:22 -04:00
bvandeusenandClaude Opus 5.5	77389f88bc	feat: a release catalogue from MusicBrainz; request search asks it directly (M502 #5519) ... Minstrel keeps its own record of what artists have released, read from MusicBrainz, so the wanted list, search and import no longer need Lidarr. - internal/musicbrainz: searches (artists, release groups, recordings) and browses (an artist's release groups; a group's releases with track lists). One process-wide pacer at 1.1s, shared with tag enrichment, because MusicBrainz's limit is per address. A 503 throttle is retried. - Migration 0080: mb_artists, mb_release_groups, mb_artist_release_groups (a collaboration belongs to each credited artist), mb_releases, mb_release_tracks. - internal/catalogue: Discography and Releases serve the held copy and re-read it when older than 7 or 30 days. An unreachable MusicBrainz serves the stale copy, marked. An hourly pass re-reads due artists. - Art for what isn't held comes through /api/catalogue/art from the enabled cover-art sources (Cover Art Archive by release group, then the name-based sources) and is kept on disk, so the browser never fetches a third-party host. A miss is a 404 and the card shows its icon. - Request search moves from /api/lidarr/search to /api/catalogue/search, with the same response shape. Albums are release groups, and "in library" matches albums.release_group_mbid. A 10-minute cache absorbs search-as-you-type. Discover suggestions point at the art endpoint instead of looking up Lidarr. - Android calls the new path and loads Discover art through ServerImage. Installed builds older than this one lose Discover search until they update (no compatibility path, rule 22). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 21:08:22 -04:00
bvandeusenandClaude Opus 5.5	a2e1890d4c	feat: indexers and download clients you add and configure (M502 #5518) ... Admin → Integrations gains "Indexers and download clients": two lists of connections the operator adds, Fabled Librarian's model. Nothing is seeded or copied from Lidarr. - Indexers: Prowlarr (its native /api/v1/search, which asks every indexer at once; Prowlarr has no aggregate Torznab feed) and any Torznab feed. - Download clients: qBittorrent (WebUI v2, with re-login on an expired session) and FabledExchange, one entry with a switch per job: downloads (/api/downloads), search (/api/search) and release details (/api/torrents/{infohash}). All three start on; its key is optional. - Each entry: enable, name, address, a write-only key or password, Test. A download client's Test lists its torrents and checks the downloads folder is visible and writable from Minstrel; an Exchange's Test reports each job switched on. - Download clients carry a path mapping (Minstrel's /downloads vs the client's own name for it); search sources carry searches per hour. Migration 0079 adds acquisition_services with its CHECKs. Ported from Fabled Librarian: internal/indexer (Torznab, 429 Retry-After, magnet redirects), internal/prowlarr, internal/exchange, internal/downloader, and internal/acquire's one builder per kind, which Test already uses so a passing Test vouches for the path a grab will take. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 20:34:51 -04:00
bvandeusenandClaude Opus 5.5	5306a261c1	feat: duplicates resolve themselves without Lidarr (M502 #5517) ... Lidarr is off, and with it the resolver had stopped merging: it only merged when Lidarr had said which copies it mapped. Minstrel is now the only program writing the library, so the rule is about the copies: - A song held twice on one album: the extra copies merge into the one whose tags fit best (then lossless, larger, older). - Across releases or under another title: when every copy but one is a video rip (held back, or named like one and not marked fine), the rips merge into the one that is not. - Clean copies on different releases are linked as one song, as before. - Anything else waits for the operator. Gone with Lidarr: release switches and their settle window, the unmapped-file read, the rip search, the merge guard and its 409/503, lidarr.Client's release methods, and duplicate_group_members.lidarr_state (migration 0078). The duplicates page drops the per-copy Lidarr badge and the blocked-merge notice; past release changes stay in its history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 20:12:49 -04:00
bvandeusenandClaude Opus 5.5	8bd48b4302	fix: a merge keeps the song link when the removed copy names the song (M498) ... MergeInheritSongLink took "linked" to mean song_key <> id, but the copy whose id became the song's key has no song_id of its own. Removing that copy dropped the survivor out of the song. It now asks whether another copy shares the key. The test covers both copies as the one removed, so it no longer passes or fails on which random id sorts first. The release-change test counted audit rows other tests left behind (ResetDB keeps audit_log); it now counts what its own pass adds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:53:35 -04:00
bvandeusenandClaude Opus 5.5	e17556dce9	feat: Lidarr searches for a better copy of a rip that has none (M498 #5447) ... The operator chose option 2: an album holding a video rip that is the only copy of its song is searched in Lidarr (AlbumSearch), at most once a week and ten albums a pass. Lidarr grabs only when the quality profile allows an upgrade, so nothing is removed and the rip stays held back either way. - lidarr.Client.SearchAlbums posts the AlbumSearch command. - ListAlbumsWithSoleCopyRips finds the albums: a held-back rip with no present clean copy sharing its song or its pending group, and not searched in the last week (audit action lidarr_rip_search). - Suspect sources shows when Lidarr last searched the track's album. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:32:30 -04:00
bvandeusenandClaude Opus 5.5	53eb954c86	feat: video rips and stray copies handle themselves (M498 #5439) ... - The resolver also merges cross-release and mismatch groups where Lidarr maps exactly one copy: the others fulfil nothing, so removing them opens no hole (D-a rule 1). That covers a rip beside the clean copy on another release and a wrong-file import Lidarr holds unmapped. With two or more mapped copies each fulfils its own release and nothing is removed. - A track whose file name carries a video-rip marker is held back from radio and the system mixes (tracks.source_verdict, migration 0077). It still plays when chosen. A renamed file is released; the operator's "fine" sticks. - Suspect sources shows what was done to each track, with "This one is fine" and "Hold back again" (PUT /api/admin/library/suspect-sources/{id}). - The Liked list prefers a copy that is not held back. Replacing a rip that has no clean copy is left for the operator to decide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:13:37 -04:00
bvandeusenandClaude Opus 5.5	5b372f61d7	feat: the same song on several releases counts as one song (M498 #5438) ... A single and the album it is on stay two files, since each fulfils its own release in Lidarr, but they are one song to the listener. - tracks.song_id links copies; the generated song_key (song_id, else the track's own id) is what they share (migration 0076). - The resolver links each cross-release group every pass (idempotent; only with auto-resolve on) and, when a link is new, shares existing likes across the song and logs them for sync. - A like or unlike (web and Subsonic) reaches every copy; each change is logged and published so clients update every heart. - The Liked list and its count show the song once; Shuffle and the mix writer take one copy per song. - A merge keeps the removed copy's song link; "Not the same song" on the Across releases tab dismisses the group and undoes the link. Shared plays ("heard via another copy") are left for a later step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:07:55 -04:00
bvandeusenandClaude Opus 5.5	4ecff52f19	feat: duplicates resolve themselves where Lidarr says it is safe (M498) ... The duplicate sweep proposed 4,197 groups and every one waited for the operator. Most are safe to settle, and Lidarr defines what safe means: it maps one file to each track of the release it monitors and downloads any mapped file that disappears. Deleting a mapped copy opens exactly the hole the operator saw Lidarr fill. Classify (#5435) - Migration 0075: duplicate_groups.class (same_release, cross_release, mismatch, review), resolve_note, resolved_automatically; duplicate_group_members.lidarr_state (tracked, unmapped); fingerprint_settings.auto_resolve; notification kind duplicates_resolved with both kind CHECKs swapped (rule 36). - library.ClassifyDuplicateGroup, with MatchTitleKey dropping featuring credits, remaster notes and video-rip markers, and keeping live, demo, remix and instrumental. The rip markers move from api to library. Choose the copy to keep (#5436) - ProposeSurvivor ranks the copy Lidarr maps first, then tag fit (a clash-free track number, no rip marker in the name, an MBID), then the quality rules. File size picked the wrong Humanz copy in 6 of 21 groups. Act (#5437) - An hourly resolver pass reads Lidarr's unmapped files, matched by the last three path components, and records each copy's state. - Same album, with at most one copy mapped: merged into the mapped copy. The merge is guarded, so a mapped copy can never be removed (MergeDuplicateGroupGuarded, ErrCopyTrackedByLidarr). - Same album, every copy mapped: the monitored release lists the song twice (Humanz's 14x12" box set). The pass moves Lidarr to the release that lists each song once and best covers what is on disk. It never picks one covering less, and is capped at 10 albums per pass. - Fixed point (lesson #4183): the chosen release no longer repeats. - The album is left alone for 24h while Lidarr rescans, so "every copy unmapped" mid-rescan is never read as licence to merge. - Both actions are audited with no actor and summarised to admins. The operator can switch them off in the Fingerprinting card (rule 25). - Manual merges use the same guard: 409 copy_tracked_by_lidarr, or 503 lidarr_unavailable when Lidarr cannot say. Web - Duplicates gets tabs: Needs review, Across releases, Resolved automatically. Each loads as you scroll (rule 172), replacing the pager. - Each copy says whether Lidarr uses it. - The resolver's note shows on each group. - The merge confirm blocks, before sending, a merge that would remove the copy Lidarr uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 21:49:37 -04:00
bvandeusenandClaude Opus 5.5	3f0540cb7a	feat: admin Suspect sources report, for files named like video rips (#5410) ... Humanz turned out to be YouTube rips: "(Official Video)", "Visualizer", a reaction video filed as a song, junk disc numbers (#5401). The library holds about 200 more files named the same way. This report lists them, grouped by folder like Missing files, with the markers each file name carries. - GET /api/admin/library/suspect-sources: one marker list in Go builds both the Postgres ~* filter and each row's labels, so they cannot drift. Basename only; missing files are left out. - Markers calibrated on the live library: "live in/at" dropped (real live albums), "reaction" narrowed (it caught "Chain Reaction"). - Admin tab "Suspect sources", read-only, loads as you scroll; a folder split across pages is joined back into one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 15:51:01 -04:00
bvandeusenandClaude Opus 5.5	9be5bbae3c	fix(discover): cap the taste-matched arm per album and artist before its LIMIT (#5356) ... Summed tag weight rewards a track for carrying many of the user's tags, so on the deploy two artists whose every track carries the whole lo-fi profile took all 120 rows of the taste-unheard query. capByAlbumAndArtist ran after the LIMIT and left 6, and the arm with the lowest skip rate (12% against ~23%) handed its slots to dormant and random. The query now ranks within album, then within artist over what the album cap kept, before the LIMIT: the same walk the Go cap makes, so the bucket fills from as many artists as match. The caps come from the Go constants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 09:24:20 -04:00
bvandeusenandClaude Opus 5.5	63709a433d	feat(notifications): grouped email digest, new music as a daily summary (#5346) ... Nothing is emailed per event. New music (request_completed) goes out at most once a day, at the summary hour in each user's own timezone, grouped by artist. Everything else is batched: one email a window after the first un-emailed item, holding whatever accumulated. - Migration 0074: notification_email_settings (summary hour, batch window, admin-configurable) and user_notification_email_state (batch start, last sent, failures and retry_after per user and group). Existing rows are stamped emailed so the upgrade sends no backlog. - The Notifier stamps emailed_at at write time when the recipient's email channel is off, so turning email on later doesn't send old items. - Read rows are never selected. A row is stamped only after the mailer accepts, in one transaction with the state, against the read's clock, so a coalesced row updated mid-send stays pending. - A failed send backs off 5m doubling to 6h; SMTP not configured just waits. - Links come from the public address; without one the email has none. - The mailer now RFC 2047-encodes subjects and strips line breaks from them. - Admin → Integrations gains a Notification emails card. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 07:48:22 -04:00
bvandeusenandClaude Opus 5.5	f11dba2336	feat(api): read-all takes an optional up_to cutoff (M489) ... POST /api/me/notifications/read-all accepts {"up_to": RFC3339}. Android queues "mark all read" for replay when offline, and a replay landing later must not mark notices that arrived in between, which the user never saw. A coalesced notice updated since then has a newer created_at, so it stays unread. An empty body still marks everything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 07:27:29 -04:00
bvandeusenandClaude Opus 5.5	1e9408c835	feat(notifications): library health reaches admins, coalesced (#5341) ... - A failed scan run sends scan_failed. Each failure adds to the count and the notice shows the latest error. A scan cut short by shutdown says nothing. - Marking tracks missing sends tracks_missing with a running count. - A duplicate sweep that proposes a group it had not proposed before sends duplicates_found, counting everything awaiting review. A sweep that only re-finds known groups stays quiet, so a read notice isn't repeated every sweep (CountDuplicateGroupsDetectedSince). - A playback-error report sends playback_errors, counting the unresolved errors (CountUnresolvedPlaybackErrors). The library package gets its notifier as a package-level SetNotifier beside SetEventBus, for the same reason the bus is package-level. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 07:20:33 -04:00
bvandeusenandClaude Opus 5.5	8bf333e748	feat(notifications): the inbox store, one writer, coalescing and retention (#5338) ... M489 step 1. The event bus is fire-and-forget, so a client that isn't connected never hears that a request completed or that tracks went missing. user_notifications is the durable record; the bus only nudges. - Migration 0073: user_notifications (kind CHECK-gated, payload jsonb, read_at, coalesce_key, emailed_at) and user_notification_prefs (per user, per kind: inbox, phone, email). A missing pref row means the kind's defaults, so nothing is seeded. - internal/notifications.Notifier is the only writer. It resolves recipients (admin kinds reach admins only, and never the excepted user), honours the inbox pref (phone and email ride on it), writes, and publishes a contentless notification.created nudge per recipient. - Burst-prone admin kinds coalesce into one unread row: tracks_missing and scan_failed add up their counts, duplicates_found and playback_errors take the latest total. Once read, the next event is a new row. - Retention: read rows go after 90 days, anything after a year, on the library_changes compactor's daily shape. Tests: unit (channel rules, kind table) and integration (recipients, nudge, coalescing both ways, prefs, owner-scoped idempotent mark-read, every kind against both schema CHECKs, retention cut-offs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 07:06:39 -04:00
bvandeusenandClaude Opus 5.5	efa3bf54ce	fix(playlists): a missing track never seeds For You or Songs-like (#2701) ... PickTopPlayedTracksForUser's liked tier read general_likes without joining tracks. With no plays to seed from, For You could pick a liked track whose file is gone. The play tiers were already safe: their play_events join filters missing_since. The same gap was in PickTopPlayedTrackForArtistByUser's fallback, which seeds Songs-like from the artist's newest album when there are no recent plays. It could pick a missing track, and since #5296 a missing track is never fetched for similarity, so that seed has no edges either. The caller already skips an empty seed, so an artist whose tracks are all missing gets no Songs-like mix instead of one aimed at nothing. Integration tests cover both cases: a liked-but-missing track is not a seed, and the Songs-like fallback moves to the next album once the newest one's track goes missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 23:10:17 -04:00
bvandeusenandClaude Opus 5.5	e5dac9ddf0	fix(similarity): keep ListenBrainz's whole answer and resolve it locally (#5296) ... The worker kept only the similar recordings already in the library, at most 20 of ListenBrainz's 50, and judged freshness by the edges it had written. Two failures followed, both measured on the operator's library (#3879): - A seed whose answer matched nothing wrote nothing, so it was never fresh. With the queue ordered by id, 25 such seeds held its head and were re-asked every hour; 17 of 2,466 played seeds had any edges. - A recording that reached the library after its seed was fetched (a Lidarr import, an MBID from the AcoustID lookup) was never linked until a refetch, which for the stuck seeds never came. Now every answer is cached whole in listenbrainz_similar_recordings and every answer, an empty one or a permanent 4xx included, is recorded in track_similarity_fetches. The queue reads the fetch record: never-fetched first, then the oldest, refreshed after 30 days. The listenbrainz edges are derived in SQL from the cache, one present track per recording and no cap, for the seed just fetched and for every seed once per tick, so new arrivals link within the hour without asking ListenBrainz again. Artists get the same queue fix via artist_similarity_fetches; their answer was already kept in artist_similarity and artist_similarity_unmatched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 19:56:22 -04:00
bvandeusenandClaude Opus 5.5	aa390c711a	fix(recommendation): artist-level arms take one track per related artist in turn (#5297) ... The similar_artists and coplay_artists arms ordered by artist score, so the closest related artist's catalogue filled the whole LIMIT. On the operator's library the 30-row similar_artists arm held exactly one artist for all 17 seeds measured (#3879), though each seed had 7-33 similar artists in the library. Both arms now rank tracks within each artist and take every artist's first track, best artist first, before anyone's second. Both also skip missing tracks: the outer select already dropped them, but only after they had taken places in the arm's LIMIT. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 19:56:21 -04:00
bvandeusenandClaude Opus 5.5	865a3176c9	feat(library): fold a missing track into its on-disk replacement (M485 #5286 #5287) ... A track marked missing whose replacement is already on disk under another row — on the operator's library 355 of 451 missing tracks, nearly all Lidarr mp3 -> flac upgrades — is folded into the replacement: likes, plays, playlist entries and tags move across and the missing row goes. Move adoption could not catch these: the replacements were re-encodes (no shared audio hash) with no recording MBID at import. Pairs (ListMissingTrackPairs): the same recording MBID within the album group, or the same album row and title ignoring case. Each side must have exactly one candidate; conflicting MBIDs refuse a pair. No duration or track position gate: on the 142 pairs known to be one recording, 18% differed by over 2s and the poorly tagged set is where numbering is broken (spike #5274). Each pair folds in its own transaction after locking both rows and checking the pair still holds. Runs automatically (operator, 2026-10-07) after a full scan, after a watcher batch that added or updated tracks, and after an AcoustID pass that matched any track. The first scan after deploy repairs the existing rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 17:23:32 -04:00
bvandeusenandClaude Opus 5.5	e509d7d5a9	feat(lidarr): ask Lidarr for release groups; repair stored release-id requests (M483 #5244) ... Lidarr's metadata is keyed by MusicBrainz release group, but re-acquisition requested albums by their release id, so every add came back "not found". - Sweeper requests an album by its tag-supplied release group, else the one MusicBrainz names (cached onto the album). An album MusicBrainz cannot name is skipped and counted, with no attempt spent. - Reconciler: an add refused as not found re-reads the request's album id as a release (library first, then MusicBrainz), rewrites the request to the group and adds again. This repairs the requests already stored. - Completion matches an album by release id or release group, and only once a track of it is on disk, so a re-acquisition request no longer completes against the row of the album it is trying to bring back. Closes #5241. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 11:38:45 -04:00
bvandeusenandClaude Opus 5.5	5c19a916ba	feat(library): store each album's MusicBrainz release-group id (M483 #5242) ... albums.mbid is the release id (Picard's musicbrainz_albumid, one edition). Lidarr names albums by release group, so re-acquisition and request completion need that id too (#5241). Migration 0070 adds albums.release_group_mbid (nullable, non-unique index: several releases share a group). The scanner reads musicbrainz_releasegroupid through extractReleaseGroupMBID, writes it on insert and heals it onto existing rows when NULL. tagReadVersion goes to 3 so the next scan fills it for the library already indexed, bound by tag reads (no ffprobe, no decode). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-07 11:33:26 -04:00
bvandeusenandClaude Opus 5.5	3c575b137c	feat(library): AcoustID lookup worker fills the MBIDs tags leave empty (M401 #3920 #3921) ... Migration 0069 adds tracks.mbid_source (tag | acoustid), a lookup state per track (matched | ambiguous | no_match | failed) and the acoustid_settings row (off, no key, min score 0.85). The file's tag outranks a lookup (D4). UpsertTrack keeps a looked-up id through a re-read that finds no tag id and replaces it as soon as one appears. SetTrackMbidFromAcoustID refuses to write over a tag id. The worker fingerprints each untagged track with fpcalc's compressed print, looks it up and writes an id only when D5 settles it: one recording at or above the threshold, or one left after matching title and length. Ambiguous and no-match results write nothing. A key AcoustID refuses, or the service being unreachable, stops the pass and is reported in the worker's status. It never counts as a verdict on a track. A changed file drops its lookup in the scan. The re-lookup takes back an id that no longer matches. Admin API: GET /api/admin/library/acoustid (settings, status, coverage by source), PUT …/acoustid-settings (write-only key), POST …/acoustid/run, GET …/acoustid/unsettled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 23:26:03 -04:00
bvandeusenandClaude Opus 5.5	13a7a3629a	fix(library): MBID backfills skip tracks whose files are missing (#5139) ... The track backfill listed every track with a NULL mbid, missing or not, so each scan tried to open every missing file and logged an "open failed" warning per track. Nothing ever healed. The album backfill could pick a missing track as the one to read, and since that pass is capped per scan, albums stuck that way were retried ahead of the rest every time. Both now read only tracks still on disk; an album with none left is skipped until a scan finds its files again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 21:22:05 -04:00
bvandeusenandClaude Opus 5.5	f34423a0e0	feat: leveled FLAC stream for Sonos/UPnP speakers (M464 #5001) ... Speakers fetch their own audio, so the phone cannot level it. A cast token minted with level=true (and the client's asAlbum, which only the queue holder knows) now returns GET /api/tracks/{id}/leveled.flac: the track rendered by ffmpeg at the user's gain (volume=XdB, plus alimiter at -1 dBFS for a limiter-mode boost), metadata stripped, FLAC at 16 or 24 bits and at most 48 kHz. The gain is computed server-side from the user's preference and the stored loudness, carried as ?g=<centi-dB>&lim=0|1 and signed into the token, so an edited URL does not verify. Unity gains get the plain stream. Renders are written beside the cache file and renamed in, keyed by the source's size and mtime, coalesced per file (singleflight, detached from the requesting speaker so a retry finds the render running), started at mint time so the fetch finds them ready, and evicted least recently used past leveled_cache_mb, a new admin setting (migration 0068, Loudness analysis card). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 19:31:07 -04:00
bvandeusenandClaude Opus 5.5	af36b2f24a	feat: per-user volume leveling preference, synced across devices (M464 #4998) ... Mode (off, auto, track, album), target (-18, -16, -14 LUFS) and boost (within headroom, or fully with a limiter), stored per user on the server so the web player, the Android app and casts apply the same one. - Server: user_normalization_prefs (migration 0067), GET/PUT /api/me/normalization; a whole-body PUT, validated, last write wins. - Web: Settings > Playback > Volume leveling. Saves at once, restores the old choice if the save fails, and caches the value for the player. - Android: Settings card. The device keeps a copy for offline playback (Room v9 with an explicit migration, so the upgrade wipes nothing). Writes are offline-first: shown at once, PUT best effort, queued on failure (NORMALIZATION_SET, collapsed to the newest). A refresh never overwrites a change still queued. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 14:29:40 -04:00
bvandeusenandClaude Opus 5.5	e3aa8629d3	feat(api): deliver loudness gains to every client (M464 #4997) ... ReplayGain 2.0 values (gain to -18 LUFS, linear peak) derived from the stored track and album loudness: - Web: GET /api/tracks/replay-gain?ids=... (up to 200), a lookup the player calls for its queue, rather than a field on every TrackRef surface. - Android: track_gain/track_peak and album_gain/album_peak on the sync views, so cached tracks level offline. Storing a measurement logs a track change, and an album's values moving logs an album change, both before the write (#2704), so caches pick the gains up. - OpenSubsonic: replayGain on every song (album, getSong, search3, starred), as a JSON object and an XML element. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:54:05 -04:00
bvandeusenandClaude Opus 5.5	c2f81bf8df	feat(library): album loudness from the tracks' summed block histograms (M464 #4996) ... Album-mode normalization plays a whole album at one gain. That gain comes from album_loudness (migration 0066): BS.1770's gated loudness over every block on the album, computed by summing the tracks' stored histograms and gating the sum. No audio is decoded again. Album true peak is the loudest track's. - Recomputed by the loudness worker each tick, after the track pass and whether or not analysis is switched on. ListAlbumsNeedingLoudness lists albums whose md5 over (present track id, measurement version and time) no longer matches the stored digest. One comparison covers every way membership changes (scan retag, duplicate merge, delete, missing and restored) without hooking each. - No album value until every present track has a settled measurement, so an album's gain doesn't shift mid-listen as the rest is measured. Silent and unreadable tracks count as settled. - Rows for albums with no present track left are dropped. - The parser and the merge share trimBins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:27:02 -04:00
bvandeusenandClaude Opus 5.5	f3196b3443	feat(library): measure every track's loudness in the background (M464 #4995) ... The first step of loudness normalization: the server measures each track with ffmpeg's EBU R128 filter (true peak, mono as dual mono) and stores the integrated loudness, true peak and loudness range in track_loudness (migration 0065). It also keeps a histogram of the 400 ms gating blocks at 0.1 LU, so album loudness can be computed exactly later with no second decode (#4996). The histogram reproduces ffmpeg's own figure (-10.68 against -10.7 on the captured fixture), and the analyzer logs a warning if the two ever drift. - A background worker, cloned from the fingerprint backfill, measures every track, new ones included. Measuring inline in the scan was dropped: the analysis decodes the whole file, and a large import could pass the scan's one-hour stuck threshold. The scan only deletes a changed file's measurement; the worker ticks every 10 minutes. - Timeouts, the cancel/missing-binary split and settled verdicts follow the fingerprint runner. Silence and undecodable files are stored as verdicts; stalls are retried. The deadline scales with track length. - loudness_settings (enabled, files at once) and an admin card with the coverage gauge, under GET/PUT /api/admin/library/loudness-settings and GET /api/admin/library/loudness. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 13:01:01 -04:00
bvandeusenandClaude Opus 5.5	edd9a3a6db	fix(auth): the Subsonic password is generated, never the login password (M462 #5026) ... `minstrel admin reset-password` copied the new login password into subsonic_password, which is stored in plain text because Subsonic t/s sign-in needs it. Every account recovered through the CLI had its login password readable in the database, and changing the password later left the copy behind. - reset-password now changes only password_hash. - Migration 0064 clears every subsonic_password, removing the copies. - Settings gets a Subsonic password card: the server generates a random password, shows it once, and it can be regenerated or turned off (GET/POST/DELETE /api/me/subsonic-password, audited). Generated rather than user-chosen so it can never be a reused password. - docs/security.md describes the separate password instead of the known issue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 10:54:30 -04:00
bvandeusenandClaude Opus 5.5	327d49428f	feat(auth): store Subsonic API keys hashed; a new key is shown once (M462 #4983) ... users.api_token held each user's apiKey in plaintext and was looked up by equality, so a leaked row or backup handed out working keys. Migration 0063 replaces it with api_token_hash (sha256, hex), computed in place from the existing keys so every Subsonic client keeps working. The key can no longer be read back: GET /api/me/api-token is gone, and POST returns the new key once. Settings shows it right after Regenerate with a copy button and a "won't be shown again" note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 09:42:35 -04:00
bvandeusenandClaude Opus 5.5	46194a609d	fix(auth): build password-reset links from an operator-set public address, never the Host header (M462 #4981) ... buildResetURL used r.Host and r.TLS, so a forgot-password request with a forged Host emailed the victim a real reset token on a link to the attacker's server. Links now come only from network_settings.public_url (migration 0062), and no reset email is sent while it is empty; the response stays the same opaque 200 and the log says why. The address is set on a new "Public address" card under Admin → Integrations, which offers the page's own origin and warns while unset. PUT /api/admin/network-settings takes either field alone, so the proxy card and this one can't overwrite each other. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 09:32:10 -04:00
bvandeusenandClaude Opus 5.5	755f997b0d	feat(auth): sessions expire server-side; password change and reset end other sessions (M462 #4978) ... Sessions had no server-side expiry: only the web cookie's 30-day Max-Age limited them, and a bearer token (Android) lived until revoked by hand. GetSessionByTokenHash and ListSessionsForUser now ignore sessions idle for 30 days or older than a year, and the GC worker deletes them hourly. A password change was a plain UPDATE, so a session opened with the old password survived it. Now: - self-service change signs out every other device and keeps this one; - reset by email ends every session the account has; - an admin reset ends the target's sessions (keeping the admin's own when they reset themselves). The success copy on web and Android says the other devices were signed out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-06 08:32:39 -04:00
bvandeusenandClaude Opus 5	077ae61235	feat(admin): fingerprinting settings — on/off, length, match threshold, concurrency, sweep interval (M400 #3913) ... Rule 25: the fingerprinting knobs move out of source into a DB-backed singleton (migration 0061), edited from a card on the Duplicates page and shared live with the scanner, the backfill and the duplicate sweep through one service instance, so a save needs no restart. The length is the knob that can silently break the library: prints taken at two lengths never match. Each track_fingerprints row now records the length it was taken at, and every reader filters on the current one — the backfill treats another length as stale, the gauge counts it pending, the sweep never streams it. Equivalent to a version bump, except that setting the length back makes rows not yet redone current again. The card warns before a length change re-fingerprints the library. Off stops every decode: the scan takes only the stream hash (a demux, and what recognises a moved file) and stores nothing, dropping a changed file's stale row; the backfill idles. A save also makes a sweep due, since a new threshold or length changes what the same prints group into, and the sweep interval gains slack so an hourly interval on an hourly tick doesn't skip every other tick. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:53:55 -04:00
bvandeusenandClaude Opus 5	c8bf9dc929	refactor(library): move detection matches on the audio hash, not size and duration (M400 #3914) ... A file that comes back renamed or moved keeps its track row, and with it its likes and play history, by being matched to the missing row it replaces (#2528). Untagged files were matched on (file_size, duration_ms), which was never a fingerprint. It could pair two unrelated files that happened to share a byte count and a duration, and it missed a file retagged in place, whose size changes. The only defence was requiring a unique match and otherwise giving up. Now there is a real identity. FindMissingTrackByAudioHash matches a missing track by the SHA-256 of its encoded audio (track_fingerprints, #3906). That survives a rename, a move and a retag, and only an identical recording can match it. adoptMovedTrack takes the new file's hash, which the scan already computes before adoption. The size and duration query and fallback are removed outright, with no second path (rule 22). Unchanged: - MBID first: it identifies the recording and survives a re-encode that even the hash does not - a unique match is still required - an absent hash is never looked up, so unhashable files cannot pair with each other The test fake answers the hash lookup only for the hash it holds, so the tests can tell adoption by identity apart from adoption by coincidence. That includes the case the old pair got wrong: different audio of equal size and duration is not adopted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:31:42 -04:00
bvandeusenandClaude Opus 5	11ef044ef6	feat(library): merge duplicates without losing history (M400 #3911) ... Merge keeps one copy of a duplicate group and removes the rest. Every table that references tracks does so ON DELETE CASCADE, so deleting a duplicate's row outright would silently destroy its likes, plays, playlist entries and tags. The merge moves all of that onto the kept copy first, then deletes the empty row. In one transaction, holding a lock on the group: - repoints play_events, skip_events, contextual_likes, playback_errors, lidarr_requests.matched_track_id and playlist_tracks. The last is keyed by position, so every entry stays where it was. - merges general_likes one per user, dated to the earlier like - takes the union of track_tags, keeping the kept copy's own weight on a shared tag - rewrites track_similarity onto the kept copy, dropping edges that would point a track at itself and keeping the kept copy's existing edge on a collision - lets the kept copy take a recording MBID only the removed copy had - deletes the removed copies' rows, tidies emptied albums and artists, marks the group merged - logs sync changes: track deletes, and like and playlist-track delete/upsert pairs The removed copies' files are deleted first, before any row changes, through the same helper as DeleteTrackFile (now shared, along with the album tidy-up). A merge that left the file behind would be undone by the next scan re-importing it. An unwritable library answers 409 library_not_writable and nothing changes. tracks.Service.MergeDuplicates wraps it with the opt-in Lidarr unmonitor from RemoveTrack, skipped when the removed copy is a second file of the kept copy's own album track: unmonitoring that would stop Lidarr managing the kept file. It writes a duplicate_merge audit row after commit, per the audit package's best-effort contract, naming both paths. POST /api/admin/library/duplicates/{id}/merge takes an optional survivor_track_id (the report's proposal otherwise) and unmonitor. On the report page: - each copy gets a Keep choice, defaulting to the proposed one - Merge needs a second click, on a button that says how many files it removes, with the consequence stated beside an opt-in Lidarr checkbox Integration tests cover: - every piece of history landing on the kept copy exactly: likes deduped at the earlier time, plays and skips counted, playlist position unchanged, tags unioned, similarity rewritten with no duplicate or self-edge, MBID inherited - the removed file gone, and a second merge refused - an unwritable file leaving likes, plays, row and group untouched - a survivor outside the group refused Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:25:06 -04:00
bvandeusenandClaude Opus 5	ff493a8c7d	feat(admin): the duplicates report — review proposed duplicate groups (M400 #3912) ... A new admin tab, Duplicates, beside Missing files: the proposals from the duplicate sweep, with a Sweep now trigger and a Not duplicates dismissal. Nothing on it merges or deletes; the merge is #3911. Each group shows: - whether it is identical audio or the same recording, with a match percentage from the weakest link between members - every copy's format, size, duration, path, and the likes and plays it carries (every user's; this is admin-only, and it is what decides which copy to keep) - the copy proposed to keep, and the rule that chose it The survivor rule is library.ProposeSurvivor, a pure function the merge will reuse: lossless over lossy, then the larger file, then the copy in the library longest, then lowest id. Bitrate is not in it because the scanner never fills tracks.bitrate, and for one recording at one duration a larger file is the higher bitrate. m4a is not counted as lossless: it may be AAC. The reason names the rule that separated first place from second, not every rule the winner passed. An empty report has three causes, and the page says which: still fingerprinting, the sweep has never run, or it ran and found nothing. The sweep's state and the backfill's progress come back with the groups for that reason. Groups left with fewer than two members since the sweep are not shown. GET /api/admin/library/duplicates, POST .../sweep (202, or 409 sweep_in_progress), POST .../{id}/dismiss (404 duplicate_group_not_pending when already resolved). Migration 0060 indexes play_events by track_id. Its only indexes led with user_id, so each copy's play count, and the merge's repointing of play history, would scan the whole table. Web only, like Missing files: Android has no library-health admin screens. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:11:11 -04:00
bvandeusenandClaude Opus 5	6379b6c31d	feat(library): the duplicate sweep — propose duplicate groups from fingerprints (M400 #3910) ... Reads fingerprints, runs them through the matcher, and records proposals in duplicate_groups (migration 0059). Nothing is merged or deleted: a group is a proposal for the admin report (#3912). Streaming. The whole library's fingerprints are hundreds of megabytes, but tracks are only compared within 3s of each other in duration. So candidates stream in (duration_ms, id) order, keyset-paged on a new tracks(duration_ms, id) index. The grouper holds only the tracks within 3s of the oldest one not yet settled. A seed is settled once a track arrives beyond its window, which gives the same result as grouping the whole sorted list. groupDuplicates is rebuilt on the same streamGrouper, so there is one grouping rule and the #3909 tests still cover it. Each fingerprint's alignment index and variety check are computed once instead of for every pair. Exact duplicates are grouped library-wide in SQL. The first member the stream meets stands in for the whole group in the acoustic pass. An exact group caught in an oversize acoustic cluster is still proposed: the acoustic evidence is discarded, identical bytes are not. Re-sweeping: - a group is identified by its sorted member ids, so finding it again refreshes the row in place - a proposal whose members all sat in one dismissed group is not proposed again (a subset repeats the verdict; a superset is new evidence) - a pending proposal no sweep has found again is retired, but only after a complete sweep, and only if an earlier sweep last confirmed it, so two overlapping sweeps cannot delete each other's findings - dismissals are kept DuplicateSweepWorker checks hourly and sweeps only when a fingerprint was written after the last sweep started. TryStartDuplicateSweep guards against two sweeps at once and reaps one stuck in flight for 2h. The sweep row is closed on a detached context with a deadline, so a sweep cancelled at shutdown still records that it ended. The integration test pages one row at a time and checks: - an acoustic pair and an exact pair are found - a track with no fingerprint, a missing track and a near-duration unrelated song are left out - a dismissed group is suppressed while the pending one refreshes without duplicating - a proposal that stops holding is retired and the dismissal survives Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 17:00:07 -04:00
bvandeusenandClaude Opus 5	b8855b480f	feat(library): backfill fingerprints for the existing library — M400 #3908 ... The scan fingerprints only bytes it has not seen, so everything imported before fingerprinting existed, and any row derived by an older fingerprintVersion, needs a pass of its own. That pass is a background worker, not a stage in RunScan. RunScan runs at boot and then every 12h, and an in-flight scan older than an hour is reaped and a second started beside it. A stage would have to stop inside the hour: a few hundred decodes a run, so about a month for a 50k-track library. It would also hold the run in flight and answer manual rescans with 409 while it worked. FingerprintBackfillWorker runs once at start, then hourly. Nothing a pass does (error or panic) can stop the next tick. A pass walks tracks with no fingerprint or a stale version, skipping missing tracks, keyset-paged on id. The cursor is what lets a pass end: an inconclusive attempt writes no row, so a file that keeps timing out would otherwise be re-listed and retried forever. Two decodes at a time, deliberately: they compete with transcoding for CPU and with streaming for the mount. storeFingerprint is now one package function shared by the scan and the worker, and reports whether the attempt was fingerprinted, rejected, inconclusive or failed to store. Progress is a live gauge on the Admin scan card, served by GET /api/admin/library/fingerprints: fingerprinted / rejected / pending of total, with missing tracks excluded so it can reach the end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 14:56:18 -04:00
bvandeusenandClaude Opus 5	cba77a5187	feat(library): fingerprint every new or changed file — M400 #3905-#3907 ... Two identities per track, because they answer different questions: - audio_stream_sha256: SHA-256 of the ENCODED audio packets (ffmpeg -map 0:a -c:a copy -f hash). Equal means identical audio whatever the tags say. Measured against the #3885 pair: the two WWW files hash identically here and differently as whole files. Packets rather than decoded samples, so an ffmpeg upgrade cannot silently change every stored hash, and nothing is decoded. - chromaprint: fpcalc -raw -signed. The same recording at another bitrate or codec, for the acoustic tier. fpcalc ships in the image (libchromaprint-tools); shelled out because CGO_ENABLED=0 rules out bindings. Stored in a track_fingerprints table rather than on tracks: eight queries read tracks with SELECT *, including album pages, search and the Subsonic surface, and a ~4 KB array there would be de-TOASTed on every one of them. The scan fingerprints only bytes it has not seen (a new path, or mtime past the row's). A tag-repair pass leaves fingerprints alone, and unchanged files with no fingerprint are the backfill's job (#3908). Folding that into the skip check would re-decode the whole library on the first scan after upgrade and push a sync change per track. A failure that says nothing about the file (timeout, cancelled scan, tool not installed) is never stored, and on changed bytes it removes the old row. A tool that rejects the file stores NULL at the current version, so the backfill does not retry it every boot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 13:21:15 -04:00
bvandeusenandClaude Opus 5	eff3d88931	fix(recommendation): make the candidate draw reproducible, not accidentally so ... Four arms of the candidate query ended in a bare `ORDER BY random()` with no seed: similar_artists, likes_overlap, coplay_artists and random_fill. Such an arm returns a STABLE set only while its LIMIT exceeds the rows eligible for it — at that point it returns all of them and the order stops mattering, because scoreAndSortCandidates sorts by track id before drawing jitter. Below that threshold it returns a random SUBSET, and two builds on the same day draw different ones. So daily determinism held BY ACCIDENT, and only for libraries smaller than the limits. Any real library is larger, which means same-day rebuilds have been producing different mixes since those arms were written — invisible, because a mix that changes after a refresh looks like a feature rather than a broken promise. Found by breaking it: cutting RandomFill to 10 while tuning Songs-like turned TestBuildSystemPlaylists_DailyNonceDeterminism red. That test seeds ~20 tracks against a default RandomFill of 30, so its determinism came from the limit exceeding the library, not from the code being right. It is now a real guard. The arms order by md5(id || $12) instead. The CALLER decides what that means, which is the point: system mixes pass a per-(user, day) seed and get the determinism they promise, radio passes a fresh value per request and keeps varying, which is what a radio should do. Same shape the browse queries in this file already use (`md5(id::text || current_date::text)`) — existing idiom, not a new one. This also unblocks the trim that #3881 wanted and could not have. Shrinking a randomly-ordered arm was what broke membership; a seeded one takes a smaller but REPRODUCIBLE slice. Songs-like's seed-independent share drops from 29% to 12%, which was the original intent before determinism forced it back to 20%. TestSongsLikeLimits_DoNotShrinkTheUnseededRandomArms is DELETED rather than kept passing. It existed to stop anyone trimming those arms while the ordering was broken; the ordering is fixed, so the constraint is gone and a guard enforcing it would now forbid correct code. Was filed as blocked on tooling. It was not: `make generate-go` runs sqlc as a pinned Go tool and is the same path CI takes. One thing worth knowing for next time: three files in internal/db/dbq are owned by root, left by `make generate` running sqlc in Docker. sqlc errored on the first it could not write. They are untouched by this change and the regeneration of recommendation.sql.go completed, but `make generate` will keep failing until they are chowned. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SQ31KQpYbStyK5y58UmPLH	2026-09-11 08:44:33 -04:00
bvandeusen	955a61194e	fix(library): a fully-missing album leaves the year axis too — #2702 ... Filed as a product decision, but the code had already made it: the genre queries filter tracks.missing_since inside their EXISTS, so an album whose every file had gone was ALREADY absent from genre while still listed under its year — where opening it found nothing playable. The two browse axes disagreed, and whichever answer won, one of them had to change. Hiding is the answer. Browsing is how you go looking for something to play, and the rule for that case is to take it out of view; the admin missing-files surface is where absence gets reported, with far more detail than a silent gap in a grid. It also means changing the axis that was inconsistent rather than the one that was already right. All three year queries move together — index, list and count. That is the invariant #367 needed care for at the genre level: if the index groups differently from the filter, a year leads to an empty page, and if the count disagrees with the list then "Load more" promises rows that never arrive. The predicate is "has at least one playable track", which also excludes an album carrying no tracks at all. Same answer for the same reason — nothing to play, nothing to browse to — and it is what genre has always done, since an album with no tracks contributes no genres either. That last part changed two existing tests, which had been seeding trackless albums as a convenience. Their intent (undated albums never appear in a range) is untouched; they now seed a track each, which is what a real album looks like anyway. Two new tests pin the actual behaviour: a fully-missing album leaves the axis while a half-missing one stays, and the count agrees with the filtered list.	2026-08-17 13:38:08 -04:00
bvandeusen	7ba673ed83	fix(library): tell clients when a file goes missing or comes back — #2704 ... The wire field shipped in 366692a1 was inert. MarkTracksMissing and ClearTracksMissing are plain UPDATEs, and /api/library/sync is a change-log feed: a row that never produces a change row is never re-sent. Clients would have kept their stale copy until an unrelated edit touched the track or the cursor fell out of the retention window and forced a full resync -- so the flag existed and nothing ever told anyone to read it. Found by checking the consumer set rather than the code: the field was threaded end to end and every test passed, because none of them asked the question "how does this reach a client?". Logged BEFORE the mutation, which is the opposite of the scanner's log-after-success pattern, and deliberately so. The failure modes are not symmetric. Log-then-fail-to-mark makes clients re-read a track that has not changed: one wasted fetch. Mark-then-fail-to-log leaves the mark with no change row -- and because both statements are idempotent (missing_since IS NULL / IS NOT NULL guards), the next scan will not retry the pair, so the client never learns. Permanently. A spurious re-read is much the cheaper mistake. Restoring logs too. A file coming back that nobody is told about stays greyed out on every device until something unrelated touches it, which would be a worse bug than the one being fixed. Op is upsert, not delete: the track still exists and keeps its history. Delete would tell clients to drop the row, which is precisely the design #2704 rejected when it chose to ship state instead of filtering the feed. Adds sync.LogChanges alongside LogChange, backed by an unnest batch insert. Every existing caller mutates one entity, so per-row was right for them; reconcile can mark a quarter of a library in one sweep, where a loop would be thousands of round-trips inside an already-slow scan.	2026-08-17 13:04:14 -04:00
bvandeusen	bab9b16831	feat(library): a missing file asks Lidarr for itself, on a backoff — #2527 ... Answers the open fork on #2527's last slice: automatic, not a button. Until now missing_since was a dead end -- reconcile marks it, every selection path skips it, the admin surface lists it, and there it sits. Two decisions carry most of the safety, both at the design level rather than as rate limits bolted on afterwards. The unit is the ALBUM, not the track. Lidarr acquires releases; there is no meaningful "fetch me one track", and a track-kind request needs a recording MBID plenty of files lack. Grouping means the loss that produced #2523 -- three reorganised albums, ~40 missing files -- becomes three requests instead of forty. The flood problem mostly dissolves. And nothing is requested until a file has been missing longer than the grace window (24h default). A filesystem lies transiently: an unmounted volume, a container that started before its media mount attached, a NAS mid-reboot. Every one of those resolves itself well inside a day at no cost. missing_since is never re-stamped (#2523), so it is a true "gone since" clock to measure against, not "when we last noticed". This is the difference between automatic and trigger-happy. Then the backoff proper: 6h -> 12h -> 24h -> 48h per album, clamped to a week, three attempts before giving up, and a per-pass ceiling so a genuinely large loss trickles instead of dumping hundreds of rows into the queue. Giving up is stamped as a timestamp rather than inferred from attempts >= max, so the verdict survives an operator later raising the maximum and the surface can say when. A sweeper, not a hook inside reconcile. Reconcile runs inside a scan and has no business deciding to talk to a third-party service; it also re-runs often, which would make "attempt once, then back off" awkward to express. A worker paces itself, survives a restart, and retries without needing another scan. Recovered albums have their state deleted rather than reset -- a future loss is a new problem, not a continuation. Requests are attributed to the oldest admin: lidarr_requests.user_id is NOT NULL and a re-acquisition has no requesting human, so this keeps the row auditable and in the same queue as everything else without inventing a synthetic principal the schema would have to understand. Auto-approve defaults ON. Requests are created pending and nothing reaches Lidarr until approval, so with it off this would be a notification rather than an attempt. Lidarr disabled leaves the request pending rather than counting a failure -- the record of intent is still right and becomes actionable the moment Lidarr is configured. Albums with no MBID are counted, not silently skipped: nothing can be asked of Lidarr for a release MusicBrainz cannot name, and quietly doing nothing would read as the feature being broken. Settings are DB-backed per rule #25 with CHECK-guarded ranges, validated in Go as well so the API answers 400 rather than surfacing a constraint violation. The admin card and the state on the missing-files page are next; this is the engine.	2026-08-16 23:53:21 -04:00