Answers the open fork on #2527's last slice: automatic, not a button. Until now missing_since was a dead end -- reconcile marks it, every selection path skips it, the admin surface lists it, and there it sits. Two decisions carry most of the safety, both at the design level rather than as rate limits bolted on afterwards. The unit is the ALBUM, not the track. Lidarr acquires releases; there is no meaningful "fetch me one track", and a track-kind request needs a recording MBID plenty of files lack. Grouping means the loss that produced #2523 -- three reorganised albums, ~40 missing files -- becomes three requests instead of forty. The flood problem mostly dissolves. And nothing is requested until a file has been missing longer than the grace window (24h default). A filesystem lies transiently: an unmounted volume, a container that started before its media mount attached, a NAS mid-reboot. Every one of those resolves itself well inside a day at no cost. missing_since is never re-stamped (#2523), so it is a true "gone since" clock to measure against, not "when we last noticed". This is the difference between automatic and trigger-happy. Then the backoff proper: 6h -> 12h -> 24h -> 48h per album, clamped to a week, three attempts before giving up, and a per-pass ceiling so a genuinely large loss trickles instead of dumping hundreds of rows into the queue. Giving up is stamped as a timestamp rather than inferred from attempts >= max, so the verdict survives an operator later raising the maximum and the surface can say when. A sweeper, not a hook inside reconcile. Reconcile runs inside a scan and has no business deciding to talk to a third-party service; it also re-runs often, which would make "attempt once, then back off" awkward to express. A worker paces itself, survives a restart, and retries without needing another scan. Recovered albums have their state deleted rather than reset -- a future loss is a new problem, not a continuation. Requests are attributed to the oldest admin: lidarr_requests.user_id is NOT NULL and a re-acquisition has no requesting human, so this keeps the row auditable and in the same queue as everything else without inventing a synthetic principal the schema would have to understand. Auto-approve defaults ON. Requests are created pending and nothing reaches Lidarr until approval, so with it off this would be a notification rather than an attempt. Lidarr disabled leaves the request pending rather than counting a failure -- the record of intent is still right and becomes actionable the moment Lidarr is configured. Albums with no MBID are counted, not silently skipped: nothing can be asked of Lidarr for a release MusicBrainz cannot name, and quietly doing nothing would read as the feature being broken. Settings are DB-backed per rule #25 with CHECK-guarded ranges, validated in Go as well so the API answers 400 rather than surfacing a constraint violation. The admin card and the state on the missing-files page are next; this is the engine.
99 lines
5.2 KiB
SQL
99 lines
5.2 KiB
SQL
-- Auto re-acquisition of missing files via Lidarr (#2527 slice 3, milestone
|
|
-- #290). A file going missing has been a dead end until now: reconcile marks
|
|
-- it (#2523), every selection path skips it, the admin surface lists it, and
|
|
-- there it sits.
|
|
--
|
|
-- The unit here is the ALBUM, not the track, and that is the design decision
|
|
-- carrying most of the safety. Lidarr acquires releases; there is no
|
|
-- meaningful "fetch me one track" operation, and a track-kind request needs a
|
|
-- recording MBID plenty of files simply don't have. Grouping by album means
|
|
-- the case that produced #2523 -- three reorganised albums, ~40 missing files
|
|
-- -- becomes three requests instead of forty.
|
|
|
|
CREATE TABLE missing_reacquisitions (
|
|
album_id uuid PRIMARY KEY REFERENCES albums (id) ON DELETE CASCADE,
|
|
attempts int NOT NULL DEFAULT 0,
|
|
last_attempt_at timestamptz,
|
|
-- The request this album's most recent attempt produced. SET NULL rather
|
|
-- than CASCADE: a purged request row must not erase the attempt history
|
|
-- that stops us asking again in a loop.
|
|
last_request_id uuid REFERENCES lidarr_requests (id) ON DELETE SET NULL,
|
|
-- Set when the attempt budget is spent. Distinct from "attempts = max"
|
|
-- so the reason survives a later change to the configured maximum, and so
|
|
-- the admin surface can say "gave up on the 3rd of August" rather than
|
|
-- inferring it.
|
|
gave_up_at timestamptz,
|
|
created_at timestamptz NOT NULL DEFAULT now(),
|
|
updated_at timestamptz NOT NULL DEFAULT now(),
|
|
CONSTRAINT missing_reacquisitions_attempts_nonneg CHECK (attempts >= 0)
|
|
);
|
|
|
|
-- The sweeper's own read: albums due another attempt, oldest attempt first.
|
|
-- Partial on the not-given-up rows because a spent album is never selected
|
|
-- again and would otherwise grow the index forever.
|
|
CREATE INDEX missing_reacquisitions_due_idx
|
|
ON missing_reacquisitions (last_attempt_at NULLS FIRST)
|
|
WHERE gave_up_at IS NULL;
|
|
|
|
-- Settings, singleton in the style of network_settings (0053). Every value an
|
|
-- operator might want to tune lives here rather than in YAML (rule #25).
|
|
CREATE TABLE reacquisition_settings (
|
|
id boolean PRIMARY KEY DEFAULT true,
|
|
|
|
-- Master switch. Default true: the operator asked for this to happen by
|
|
-- itself, and a feature that ships switched off is a feature nobody finds.
|
|
enabled boolean NOT NULL DEFAULT true,
|
|
|
|
-- How long a file must have been missing before the FIRST attempt. This
|
|
-- is what separates "automatic" from "trigger-happy": a filesystem lies
|
|
-- transiently -- an unmounted volume, a container that started before its
|
|
-- media mount attached, a NAS mid-reboot -- and every one of those
|
|
-- resolves itself well inside a day at no cost. tracks.missing_since is
|
|
-- never re-stamped (#2523), so it is a true "gone since" clock to measure
|
|
-- against.
|
|
grace_hours int NOT NULL DEFAULT 24,
|
|
|
|
-- Exponential spacing between attempts: base * 2^(attempts-1), clamped to
|
|
-- backoff_max_hours. 6h -> 12h -> 24h -> 48h by default. An album Lidarr
|
|
-- genuinely cannot find must get quieter, not keep pace.
|
|
backoff_base_hours int NOT NULL DEFAULT 6,
|
|
backoff_max_hours int NOT NULL DEFAULT 168, -- one week
|
|
|
|
-- Attempts before giving up. Three real tries spread over days is enough
|
|
-- to ride out a transient Lidarr/indexer outage; past that the answer is
|
|
-- "this release is not obtainable" and asking again is noise.
|
|
max_attempts int NOT NULL DEFAULT 3,
|
|
|
|
-- Ceiling on requests created per sweep. Album grouping already collapses
|
|
-- the common case, but a genuinely large loss (a whole drive slipping
|
|
-- under reconcile's 25% mark cap) should still trickle rather than dump
|
|
-- hundreds of requests into the queue at once.
|
|
max_per_pass int NOT NULL DEFAULT 20,
|
|
|
|
-- Whether the sweeper approves what it creates. Requests are created
|
|
-- pending and nothing reaches Lidarr until approval, so with this off the
|
|
-- feature is a notification rather than an attempt -- which is why the
|
|
-- default is on. Off is the review-first posture: rows appear in the
|
|
-- admin Requests queue for a human to release.
|
|
auto_approve boolean NOT NULL DEFAULT true,
|
|
|
|
CONSTRAINT reacquisition_settings_singleton CHECK (id = true),
|
|
-- Ranges exist to stop a typo becoming a behaviour change: a 0-hour grace
|
|
-- would fire on every transient unmount, and a 10000-per-pass cap would
|
|
-- defeat the point of having one.
|
|
CONSTRAINT reacquisition_settings_grace_range
|
|
CHECK (grace_hours >= 1 AND grace_hours <= 720),
|
|
CONSTRAINT reacquisition_settings_backoff_base_range
|
|
CHECK (backoff_base_hours >= 1 AND backoff_base_hours <= 168),
|
|
CONSTRAINT reacquisition_settings_backoff_max_range
|
|
CHECK (backoff_max_hours >= 1 AND backoff_max_hours <= 720),
|
|
CONSTRAINT reacquisition_settings_backoff_ordered
|
|
CHECK (backoff_max_hours >= backoff_base_hours),
|
|
CONSTRAINT reacquisition_settings_attempts_range
|
|
CHECK (max_attempts >= 1 AND max_attempts <= 10),
|
|
CONSTRAINT reacquisition_settings_per_pass_range
|
|
CHECK (max_per_pass >= 1 AND max_per_pass <= 200)
|
|
);
|
|
|
|
INSERT INTO reacquisition_settings (id) VALUES (true) ON CONFLICT (id) DO NOTHING;
|