revert: remove the placement reconciler — it manufactured the problem it solved
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 59s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m55s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m22s

Milestone #421 built a sweep that compared each image's `artist_id` to the
name of the directory holding its file, and called every mismatch a misplaced
image. It reported 33,789 of 63,605 as wrongly filed. That number described
the comparison, not the library.

What it actually was:

  32,475  (97.1%)  one artist's own folder, spelled differently
                   — Telepurte/ vs telepurte/. Same artist, same art.
     657  ( 2.0%)  loose at the images root
     328  ( 1.0%)  in a folder named after a different artist

And the 1% did not mean what the tool assumed either. `ImageProvenance`
records the post and source every file was downloaded from — the
authoritative answer, which the tool never consulted. Querying it for all 328:

    144  provenance agrees with the record  (move would be right)
     87  provenance agrees with the FOLDER  (the record is wrong; move wrong)
     53  provenance names SEVERAL artists   (no single correct folder)
     41  no provenance at all
      3  agrees with neither

So the sweep would have misfiled or arbitrarily picked for ~41% of the only
set it was really needed for. The system already knew where each file came
from; the tool inferred it from a column and a directory name instead.

Operator, 2026-09-21: *"the current system consistently records where items
are and where they came from this is just complicating something works and
doesn't need fixing."* Correct on both counts.

Removed: the service, the tasks, the model and migration 0099's table, the
/api/cleanup/layout and /placement/* endpoints, the Maintenance card and its
store actions, and the tests. 0100 drops the table (rule #22 — no legacy).

KEPT deliberately, per the operator:
- `utils.paths.canonical_subdir` — new filesystem imports derive their
  directory from the artist's slug, matching what the downloader always did.
  Not part of this tool; removing it would be churn that fixes nothing.
- The 327 files run 1 moved (InsoUwu/ -> insouwu/). Same artist either way,
  and the gallery renders them correctly.
- Everything from #4223 (three-gate dedup, 256-bit pHash) and #4234 (backup
  credential exclusion). Those fixed problems that were actually reported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-21 18:17:13 -04:00
co-authored by Claude Opus 5
parent 2dd9b956d5
commit 11a01a9686
14 changed files with 111 additions and 1850 deletions
+2 -112
View File
@@ -29,8 +29,8 @@ from quart import Blueprint, jsonify, request
from sqlalchemy import select
from ..extensions import get_session
from ..models import LibraryAuditRun, LibraryPlacementRun
from ..services import cleanup_service, library_layout
from ..models import LibraryAuditRun
from ..services import cleanup_service
from ._responses import error_response as _bad
cleanup_bp = Blueprint("cleanup", __name__, url_prefix="/api/cleanup")
@@ -196,113 +196,3 @@ async def audit_cancel(audit_id: int):
)
await session.commit()
return jsonify({"cancelled": True})
@cleanup_bp.route("/layout", methods=["GET"])
async def layout_survey():
"""Milestone #421 blast radius: which ImageRecord rows sit outside their
artist's canonical slug directory, per artist.
Read-only. `?check_disk=1` additionally stats every destination to find
collisions with a file already there and sources that have gone missing —
the numbers the apply refuses on, at the cost of one stat per misplaced
row over NFS. It is OFF by default because a count-only pass answers "how
big is this" in seconds where the disk pass can run for minutes and time
the request out.
"""
check_disk = request.args.get("check_disk", "").lower() in ("1", "true", "yes")
async with get_session() as session:
report = await session.run_sync(
lambda s: library_layout.survey_layout(
s, IMAGES_ROOT, check_disk=check_disk,
)
)
return jsonify({**report.as_dict(), "checked_disk": check_disk})
def _serialize_placement_run(run: LibraryPlacementRun, *, moves: bool = False) -> dict:
"""`moves` is opt-in: an applied whole-library run carries tens of
thousands of entries, which is a fine thing to hold in Postgres and a
poor thing to put in every list response."""
out = {
"id": run.id,
"status": run.status,
"artist_id": run.artist_id,
"started_at": run.started_at.isoformat() if run.started_at else None,
"finished_at": run.finished_at.isoformat() if run.finished_at else None,
"planned_count": run.planned_count,
"moved_count": run.moved_count,
"refused_count": run.refused_count,
"refusals": run.refusals or [],
"error": run.error,
}
if moves:
out["moves"] = run.moves or []
return out
@cleanup_bp.route("/placement/runs", methods=["GET"])
async def placement_runs():
"""Newest first. Without `moves`, so the list stays small."""
try:
limit = min(int(request.args.get("limit", "25")), 100)
except ValueError:
return _bad("invalid_limit")
async with get_session() as session:
rows = (await session.execute(
select(LibraryPlacementRun)
.order_by(LibraryPlacementRun.id.desc()).limit(limit)
)).scalars().all()
return jsonify({"runs": [_serialize_placement_run(r) for r in rows]})
@cleanup_bp.route("/placement/runs/<int:run_id>", methods=["GET"])
async def placement_run(run_id: int):
"""One run WITH its moves — this is the preview the operator reads before
agreeing, and the record of what happened afterwards."""
async with get_session() as session:
run = await session.get(LibraryPlacementRun, run_id)
if run is None:
return _bad("not_found", status=404)
return jsonify(_serialize_placement_run(run, moves=True))
@cleanup_bp.route("/placement/plan", methods=["POST"])
async def placement_plan():
"""Queue a planning run. `artist_id` scopes it to one artist, which is the
intended use: do one, look at the gallery, then continue or revert."""
body = await request.get_json(silent=True) or {}
artist_id = body.get("artist_id")
if artist_id is not None and not isinstance(artist_id, int):
return _bad("invalid_artist_id")
from ..tasks.library_placement import plan_placement
plan_placement.delay(artist_id)
return jsonify({"status": "dispatched"}), 202
@cleanup_bp.route("/placement/runs/<int:run_id>/apply", methods=["POST"])
async def placement_apply(run_id: int):
"""Execute a ready run. This renames files and rewrites rows."""
async with get_session() as session:
run = await session.get(LibraryPlacementRun, run_id)
if run is None:
return _bad("not_found", status=404)
if run.status != "ready":
return _bad("not_ready", detail=f"run is {run.status}")
from ..tasks.library_placement import apply_placement
apply_placement.delay(run_id)
return jsonify({"status": "dispatched"}), 202
@cleanup_bp.route("/placement/runs/<int:run_id>/revert", methods=["POST"])
async def placement_revert(run_id: int):
"""Put an applied run's files back. The reason the ledger is kept."""
async with get_session() as session:
run = await session.get(LibraryPlacementRun, run_id)
if run is None:
return _bad("not_found", status=404)
if run.status != "applied":
return _bad("not_applied", detail=f"run is {run.status}")
from ..tasks.library_placement import revert_placement
revert_placement.delay(run_id)
return jsonify({"status": "dispatched"}), 202