feat: placement reconciler — plan, apply, revert (4246, slice 3a)
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 11s
CI / frontend-build (push) Successful in 29s
CI / backend-lint-and-test (push) Successful in 1m1s
Build images / build-web (push) Successful in 1m11s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m56s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m43s

Milestone #421 step 3, reframed on the operator's steer: not a one-off
migration but the system that keeps the tree true. The 33,789 misplaced rows
the survey found are just its first run.

The placement half was already done, verified by reading each writer rather
than assuming: downloads have always written `<root>/<slug>/<platform>/`
(gallery_dl.py:523), attach_in_place leaves files where the downloader put
them, and `_copy_to_library` / `_supersede` became canonical in #4244. So
nothing is written off-canon today; what remains is the backlog and a standing
check for future drift.

`LibraryPlacementRun` (migration 0099) holds the plan as JSONB, and that one
structure does three jobs: it is the PREVIEW the operator reads, the list the
APPLY executes (rather than re-deriving the set, so the two cannot disagree),
and — because `from` is retained — the UNDO.

The undo is the point. It makes a 33,789-file operation something to do one
artist at a time, look at in the gallery, and reverse if it reads wrong. That
settles whether artist_id or the folder held the truth (spike #4257) by doing
rather than by arguing it from a 50-row sample.

An applied run is therefore HISTORY, not state — lesson #4226's trap, since
it is the only record of where those files used to be. The model and the
migration both say so: any future retention here may prune ready/cancelled/
error runs, never an applied one.

Everything fails closed. The apply re-checks each row against what the plan
recorded — source still there, destination still free, row still pointing
where the plan said — because a download or a supersede can land in between.
A refusal is recorded with its reason and the run continues; one stale row is
not a reason to abandon the other 33,788. The row is updated only after its
rename lands, so a failed move can never leave `path` naming a file that is
not there.

Writing the collision test caught the code disagreeing with its own comment:
it claimed the first of two rows wanting one destination and skipped the
second, silently picking a winner by iteration order. Now it counts first and
filters after, so genuinely neither is planned.

Thumbnails are sha-addressed, not path-keyed, so they do not move — pinned by
a test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-21 12:46:06 -04:00
co-authored by Claude Opus 5
parent 9728407539
commit 9ccc460c69
5 changed files with 560 additions and 2 deletions
@@ -0,0 +1,99 @@
"""LibraryPlacementRun — one run of the placement reconciler (milestone #421).
The library is keyed on the Artist row's `slug`, one directory per artist.
Every writer agrees on that now (`utils.paths.canonical_subdir`, task #4244),
but ~33,789 rows were written under older rules and sit in some other
artist's directory. This row is a run of the sweep that trues them up.
State machine, mirroring LibraryAuditRun:
running -> ready -> applied -> reverted
\\-> cancelled
(any) -> error
## The `moves` column does three jobs
`moves` is the plan: `[{"image_id": 1, "from": "...", "to": "..."}, ...]`.
1. **Preview.** It is what the operator reads before agreeing.
2. **Apply.** The apply executes THIS list rather than re-deriving the set,
so the preview cannot describe a different set from the apply. That is
rule 93's guarantee reached the way LibraryAuditRun reaches it — the
plan is materialised, not recomputed.
3. **Revert.** `from` is retained, so a batch that looks wrong in the
gallery goes back where it came from.
## An applied run IS the undo ledger — it must never be pruned
This is the trap lesson #4226 names: a record that answers both "what is the
current plan" and "what happened" gets deleted by whatever forgets the first.
A `ready` run is disposable state. An `applied` run is HISTORY, and it is the
only record of where 33,789 files used to be — delete it and the moves become
irreversible.
No pruning exists for this table today, and that is deliberate. If retention
is ever added here, it may prune `ready`, `cancelled` and `error` runs; an
`applied` run is only safe to drop once someone decides the moves are settled
and undo is no longer wanted, which is an operator decision and not a
timer's.
"""
from datetime import datetime
from typing import Any
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, func, text
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from .base import Base
class LibraryPlacementRun(Base):
__tablename__ = "library_placement_run"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
status: Mapped[str] = mapped_column(
String(16), nullable=False, default="running", index=True,
server_default="running",
)
# running | ready | applied | reverted | cancelled | error
# Scope. NULL = the whole library; set = one artist, which is how this is
# meant to be used — do one artist, look at it in the gallery, continue or
# revert. ondelete SET NULL rather than CASCADE: deleting an artist must
# not destroy the record of where their files were moved.
artist_id: Mapped[int | None] = mapped_column(
ForeignKey("artist.id", ondelete="SET NULL"), nullable=True, index=True,
)
started_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now(),
)
finished_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True,
)
planned_count: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default="0",
)
moved_count: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default="0",
)
refused_count: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default="0",
)
# [{"image_id": int, "from": str, "to": str}, ...] — see the module
# docstring. This is the plan, the audit trail and the undo, in that order
# of appearance and in one place.
moves: Mapped[list[dict[str, Any]]] = mapped_column(
JSONB, nullable=False, default=list, server_default=text("'[]'::jsonb"),
)
# [{"image_id": int, "reason": str}, ...] — rows the apply declined to
# touch, with why. A refusal is an expected outcome, not an error: the
# world moves between plan and apply, and every gate fails closed.
refusals: Mapped[list[dict[str, Any]]] = mapped_column(
JSONB, nullable=False, default=list, server_default=text("'[]'::jsonb"),
)
error: Mapped[str | None] = mapped_column(Text, nullable=True)