M1/#293: library scanner (walk + tag-parse + upsert incremental) + admin scan endpoint #6

2026-04-19T11:17:40-04:00

bvandeusen commented

2026-04-19 11:17:40 -04:00

(Migrated from git.fabledsword.com)

Summary

Implements Fable task #293 — library scanner (spec §4 #1, §12).

internal/library/scanner.go: filepath.WalkDir over library.scan_paths; per-file os.Stat → mtime vs tracks.updated_at skip; tag.ReadFrom (pure-Go, no cgo via github.com/dhowden/tag); resolve-or-create artist (GetArtistByName → UpsertArtist) and album (GetAlbumByArtistAndTitle → UpsertAlbum); UpsertTrack. Defaults: Unknown Artist / Unknown Album / filename fallback. DurationMs=0 pending ffprobe. Walk + per-file errors logged and counted; scan keeps going. Context-cancel via fs.SkipAll.
internal/auth/middleware.go: RequireAdmin(pool) checks X-API-Token → GetUserByAPIToken → IsAdmin; 401 missing/invalid, 403 not admin, 500 lookup. Exposes UserFromContext.
internal/server/server.go: ScanTrigger interface for testability; mounts /api/admin with RequireAdmin and POST /api/admin/scan (synchronous — small-library v1).
internal/config/config.go: LibraryConfig{ScanPaths, ScanOnStartup}; SMARTMUSIC_LIBRARY_SCAN_PATHS (colon-separated, empty entries dropped) + SMARTMUSIC_LIBRARY_SCAN_ON_STARTUP env.
cmd/minstrel/main.go: startup scan in goroutine when scan_on_startup + non-empty paths; scanner passed into server.New.
internal/db/queries/albums.sql: GetAlbumByArtistAndTitle for the no-mbid dedupe path (partial-unique mbid index doesn't dedupe NULL rows).
internal/library/scanner_test.go: integration test gated on MINSTREL_TEST_DATABASE_URL; generates ID3v2.3 fixtures at test time (no binaries in git); asserts walk counts, article-strip sort (The Artist Y → sort Artist Y), and incremental skip on re-scan.

Test plan

go build ./... clean, go vet ./... clean, gofmt -l clean
go test -short ./... passes
MINSTREL_TEST_DATABASE_URL integration run: scanner test green (first scan Added=4, second Skipped=4; artist sort [Artist X, Artist Y])
CI-runner image parity: golangci-lint + short tests green
E2E smoke on local pg: /healthz 200; /api/admin/scan — 401 (no token), 401 (bad token), 200 (admin token) with JSON stats
CI test.yml run on this PR green
Release :main image publishes after merge

## Summary Implements Fable task #293 — library scanner (spec §4 #1, §12). - **`internal/library/scanner.go`**: `filepath.WalkDir` over `library.scan_paths`; per-file `os.Stat` → mtime vs `tracks.updated_at` skip; `tag.ReadFrom` (pure-Go, no cgo via `github.com/dhowden/tag`); resolve-or-create artist (`GetArtistByName` → `UpsertArtist`) and album (`GetAlbumByArtistAndTitle` → `UpsertAlbum`); `UpsertTrack`. Defaults: `Unknown Artist` / `Unknown Album` / filename fallback. `DurationMs=0` pending ffprobe. Walk + per-file errors logged and counted; scan keeps going. Context-cancel via `fs.SkipAll`. - **`internal/auth/middleware.go`**: `RequireAdmin(pool)` checks `X-API-Token` → `GetUserByAPIToken` → `IsAdmin`; 401 missing/invalid, 403 not admin, 500 lookup. Exposes `UserFromContext`. - **`internal/server/server.go`**: `ScanTrigger` interface for testability; mounts `/api/admin` with `RequireAdmin` and `POST /api/admin/scan` (synchronous — small-library v1). - **`internal/config/config.go`**: `LibraryConfig{ScanPaths, ScanOnStartup}`; `SMARTMUSIC_LIBRARY_SCAN_PATHS` (colon-separated, empty entries dropped) + `SMARTMUSIC_LIBRARY_SCAN_ON_STARTUP` env. - **`cmd/minstrel/main.go`**: startup scan in goroutine when `scan_on_startup` + non-empty paths; scanner passed into `server.New`. - **`internal/db/queries/albums.sql`**: `GetAlbumByArtistAndTitle` for the no-mbid dedupe path (partial-unique mbid index doesn't dedupe NULL rows). - **`internal/library/scanner_test.go`**: integration test gated on `MINSTREL_TEST_DATABASE_URL`; generates ID3v2.3 fixtures at test time (no binaries in git); asserts walk counts, article-strip sort (`The Artist Y` → sort `Artist Y`), and incremental skip on re-scan. ## Test plan - [x] `go build ./...` clean, `go vet ./...` clean, `gofmt -l` clean - [x] `go test -short ./...` passes - [x] `MINSTREL_TEST_DATABASE_URL` integration run: scanner test green (first scan Added=4, second Skipped=4; artist sort `[Artist X, Artist Y]`) - [x] CI-runner image parity: golangci-lint + short tests green - [x] E2E smoke on local pg: `/healthz` 200; `/api/admin/scan` — 401 (no token), 401 (bad token), 200 (admin token) with JSON stats - [ ] CI `test.yml` run on this PR green - [ ] Release `:main` image publishes after merge

bvandeusen referenced this issue from a commit

2026-06-01 20:10:48 -04:00

feat(web): hero row at top of Home (For You + most recently added)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/minstrel#6