revert: remove the placement reconciler — it manufactured the problem it solved
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 59s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m55s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m22s

Milestone #421 built a sweep that compared each image's `artist_id` to the
name of the directory holding its file, and called every mismatch a misplaced
image. It reported 33,789 of 63,605 as wrongly filed. That number described
the comparison, not the library.

What it actually was:

  32,475  (97.1%)  one artist's own folder, spelled differently
                   — Telepurte/ vs telepurte/. Same artist, same art.
     657  ( 2.0%)  loose at the images root
     328  ( 1.0%)  in a folder named after a different artist

And the 1% did not mean what the tool assumed either. `ImageProvenance`
records the post and source every file was downloaded from — the
authoritative answer, which the tool never consulted. Querying it for all 328:

    144  provenance agrees with the record  (move would be right)
     87  provenance agrees with the FOLDER  (the record is wrong; move wrong)
     53  provenance names SEVERAL artists   (no single correct folder)
     41  no provenance at all
      3  agrees with neither

So the sweep would have misfiled or arbitrarily picked for ~41% of the only
set it was really needed for. The system already knew where each file came
from; the tool inferred it from a column and a directory name instead.

Operator, 2026-09-21: *"the current system consistently records where items
are and where they came from this is just complicating something works and
doesn't need fixing."* Correct on both counts.

Removed: the service, the tasks, the model and migration 0099's table, the
/api/cleanup/layout and /placement/* endpoints, the Maintenance card and its
store actions, and the tests. 0100 drops the table (rule #22 — no legacy).

KEPT deliberately, per the operator:
- `utils.paths.canonical_subdir` — new filesystem imports derive their
  directory from the artist's slug, matching what the downloader always did.
  Not part of this tool; removing it would be churn that fixes nothing.
- The 327 files run 1 moved (InsoUwu/ -> insouwu/). Same artist either way,
  and the gallery renders them correctly.
- Everything from #4223 (three-gate dedup, 256-bit pHash) and #4234 (backup
  credential exclusion). Those fixed problems that were actually reported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-21 18:17:13 -04:00
co-authored by Claude Opus 5
parent 2dd9b956d5
commit 11a01a9686
14 changed files with 111 additions and 1850 deletions
-2
View File
@@ -22,7 +22,6 @@ from .import_batch import ImportBatch
from .import_settings import ImportSettings
from .import_task import ImportTask
from .library_audit_run import LibraryAuditRun
from .library_placement_run import LibraryPlacementRun
from .membership_sync import MembershipSync
from .ml_settings import MLSettings
from .patreon_failed_media import PatreonFailedMedia
@@ -86,7 +85,6 @@ __all__ = [
"ImportTask",
"ImportSettings",
"LibraryAuditRun",
"LibraryPlacementRun",
"MembershipSync",
"MLSettings",
"HeadAutoApplyRun",
@@ -1,99 +0,0 @@
"""LibraryPlacementRun — one run of the placement reconciler (milestone #421).
The library is keyed on the Artist row's `slug`, one directory per artist.
Every writer agrees on that now (`utils.paths.canonical_subdir`, task #4244),
but ~33,789 rows were written under older rules and sit in some other
artist's directory. This row is a run of the sweep that trues them up.
State machine, mirroring LibraryAuditRun:
running -> ready -> applied -> reverted
\\-> cancelled
(any) -> error
## The `moves` column does three jobs
`moves` is the plan: `[{"image_id": 1, "from": "...", "to": "..."}, ...]`.
1. **Preview.** It is what the operator reads before agreeing.
2. **Apply.** The apply executes THIS list rather than re-deriving the set,
so the preview cannot describe a different set from the apply. That is
rule 93's guarantee reached the way LibraryAuditRun reaches it — the
plan is materialised, not recomputed.
3. **Revert.** `from` is retained, so a batch that looks wrong in the
gallery goes back where it came from.
## An applied run IS the undo ledger — it must never be pruned
This is the trap lesson #4226 names: a record that answers both "what is the
current plan" and "what happened" gets deleted by whatever forgets the first.
A `ready` run is disposable state. An `applied` run is HISTORY, and it is the
only record of where 33,789 files used to be — delete it and the moves become
irreversible.
No pruning exists for this table today, and that is deliberate. If retention
is ever added here, it may prune `ready`, `cancelled` and `error` runs; an
`applied` run is only safe to drop once someone decides the moves are settled
and undo is no longer wanted, which is an operator decision and not a
timer's.
"""
from datetime import datetime
from typing import Any
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, func, text
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from .base import Base
class LibraryPlacementRun(Base):
__tablename__ = "library_placement_run"
id: Mapped[int] = mapped_column(Integer, primary_key=True)
status: Mapped[str] = mapped_column(
String(16), nullable=False, default="running", index=True,
server_default="running",
)
# running | ready | applied | reverted | cancelled | error
# Scope. NULL = the whole library; set = one artist, which is how this is
# meant to be used — do one artist, look at it in the gallery, continue or
# revert. ondelete SET NULL rather than CASCADE: deleting an artist must
# not destroy the record of where their files were moved.
artist_id: Mapped[int | None] = mapped_column(
ForeignKey("artist.id", ondelete="SET NULL"), nullable=True, index=True,
)
started_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now(),
)
finished_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True,
)
planned_count: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default="0",
)
moved_count: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default="0",
)
refused_count: Mapped[int] = mapped_column(
Integer, nullable=False, default=0, server_default="0",
)
# [{"image_id": int, "from": str, "to": str}, ...] — see the module
# docstring. This is the plan, the audit trail and the undo, in that order
# of appearance and in one place.
moves: Mapped[list[dict[str, Any]]] = mapped_column(
JSONB, nullable=False, default=list, server_default=text("'[]'::jsonb"),
)
# [{"image_id": int, "reason": str}, ...] — rows the apply declined to
# touch, with why. A refusal is an expected outcome, not an error: the
# world moves between plan and apply, and every gate fails closed.
refusals: Mapped[list[dict[str, Any]]] = mapped_column(
JSONB, nullable=False, default=list, server_default=text("'[]'::jsonb"),
)
error: Mapped[str | None] = mapped_column(Text, nullable=True)