revert: remove the placement reconciler — it manufactured the problem it solved
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 59s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m55s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m22s
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 59s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m55s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m22s
Milestone #421 built a sweep that compared each image's `artist_id` to the name of the directory holding its file, and called every mismatch a misplaced image. It reported 33,789 of 63,605 as wrongly filed. That number described the comparison, not the library. What it actually was: 32,475 (97.1%) one artist's own folder, spelled differently — Telepurte/ vs telepurte/. Same artist, same art. 657 ( 2.0%) loose at the images root 328 ( 1.0%) in a folder named after a different artist And the 1% did not mean what the tool assumed either. `ImageProvenance` records the post and source every file was downloaded from — the authoritative answer, which the tool never consulted. Querying it for all 328: 144 provenance agrees with the record (move would be right) 87 provenance agrees with the FOLDER (the record is wrong; move wrong) 53 provenance names SEVERAL artists (no single correct folder) 41 no provenance at all 3 agrees with neither So the sweep would have misfiled or arbitrarily picked for ~41% of the only set it was really needed for. The system already knew where each file came from; the tool inferred it from a column and a directory name instead. Operator, 2026-09-21: *"the current system consistently records where items are and where they came from this is just complicating something works and doesn't need fixing."* Correct on both counts. Removed: the service, the tasks, the model and migration 0099's table, the /api/cleanup/layout and /placement/* endpoints, the Maintenance card and its store actions, and the tests. 0100 drops the table (rule #22 — no legacy). KEPT deliberately, per the operator: - `utils.paths.canonical_subdir` — new filesystem imports derive their directory from the artist's slug, matching what the downloader always did. Not part of this tool; removing it would be churn that fixes nothing. - The 327 files run 1 moved (InsoUwu/ -> insouwu/). Same artist either way, and the gallery renders them correctly. - Everything from #4223 (three-gate dedup, 256-bit pHash) and #4234 (backup credential exclusion). Those fixed problems that were actually reported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -22,7 +22,6 @@ from .import_batch import ImportBatch
|
||||
from .import_settings import ImportSettings
|
||||
from .import_task import ImportTask
|
||||
from .library_audit_run import LibraryAuditRun
|
||||
from .library_placement_run import LibraryPlacementRun
|
||||
from .membership_sync import MembershipSync
|
||||
from .ml_settings import MLSettings
|
||||
from .patreon_failed_media import PatreonFailedMedia
|
||||
@@ -86,7 +85,6 @@ __all__ = [
|
||||
"ImportTask",
|
||||
"ImportSettings",
|
||||
"LibraryAuditRun",
|
||||
"LibraryPlacementRun",
|
||||
"MembershipSync",
|
||||
"MLSettings",
|
||||
"HeadAutoApplyRun",
|
||||
|
||||
@@ -1,99 +0,0 @@
|
||||
"""LibraryPlacementRun — one run of the placement reconciler (milestone #421).
|
||||
|
||||
The library is keyed on the Artist row's `slug`, one directory per artist.
|
||||
Every writer agrees on that now (`utils.paths.canonical_subdir`, task #4244),
|
||||
but ~33,789 rows were written under older rules and sit in some other
|
||||
artist's directory. This row is a run of the sweep that trues them up.
|
||||
|
||||
State machine, mirroring LibraryAuditRun:
|
||||
|
||||
running -> ready -> applied -> reverted
|
||||
\\-> cancelled
|
||||
(any) -> error
|
||||
|
||||
## The `moves` column does three jobs
|
||||
|
||||
`moves` is the plan: `[{"image_id": 1, "from": "...", "to": "..."}, ...]`.
|
||||
|
||||
1. **Preview.** It is what the operator reads before agreeing.
|
||||
2. **Apply.** The apply executes THIS list rather than re-deriving the set,
|
||||
so the preview cannot describe a different set from the apply. That is
|
||||
rule 93's guarantee reached the way LibraryAuditRun reaches it — the
|
||||
plan is materialised, not recomputed.
|
||||
3. **Revert.** `from` is retained, so a batch that looks wrong in the
|
||||
gallery goes back where it came from.
|
||||
|
||||
## An applied run IS the undo ledger — it must never be pruned
|
||||
|
||||
This is the trap lesson #4226 names: a record that answers both "what is the
|
||||
current plan" and "what happened" gets deleted by whatever forgets the first.
|
||||
A `ready` run is disposable state. An `applied` run is HISTORY, and it is the
|
||||
only record of where 33,789 files used to be — delete it and the moves become
|
||||
irreversible.
|
||||
|
||||
No pruning exists for this table today, and that is deliberate. If retention
|
||||
is ever added here, it may prune `ready`, `cancelled` and `error` runs; an
|
||||
`applied` run is only safe to drop once someone decides the moves are settled
|
||||
and undo is no longer wanted, which is an operator decision and not a
|
||||
timer's.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, func, text
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from .base import Base
|
||||
|
||||
|
||||
class LibraryPlacementRun(Base):
|
||||
__tablename__ = "library_placement_run"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||
status: Mapped[str] = mapped_column(
|
||||
String(16), nullable=False, default="running", index=True,
|
||||
server_default="running",
|
||||
)
|
||||
# running | ready | applied | reverted | cancelled | error
|
||||
|
||||
# Scope. NULL = the whole library; set = one artist, which is how this is
|
||||
# meant to be used — do one artist, look at it in the gallery, continue or
|
||||
# revert. ondelete SET NULL rather than CASCADE: deleting an artist must
|
||||
# not destroy the record of where their files were moved.
|
||||
artist_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("artist.id", ondelete="SET NULL"), nullable=True, index=True,
|
||||
)
|
||||
|
||||
started_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, server_default=func.now(),
|
||||
)
|
||||
finished_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True,
|
||||
)
|
||||
|
||||
planned_count: Mapped[int] = mapped_column(
|
||||
Integer, nullable=False, default=0, server_default="0",
|
||||
)
|
||||
moved_count: Mapped[int] = mapped_column(
|
||||
Integer, nullable=False, default=0, server_default="0",
|
||||
)
|
||||
refused_count: Mapped[int] = mapped_column(
|
||||
Integer, nullable=False, default=0, server_default="0",
|
||||
)
|
||||
|
||||
# [{"image_id": int, "from": str, "to": str}, ...] — see the module
|
||||
# docstring. This is the plan, the audit trail and the undo, in that order
|
||||
# of appearance and in one place.
|
||||
moves: Mapped[list[dict[str, Any]]] = mapped_column(
|
||||
JSONB, nullable=False, default=list, server_default=text("'[]'::jsonb"),
|
||||
)
|
||||
# [{"image_id": int, "reason": str}, ...] — rows the apply declined to
|
||||
# touch, with why. A refusal is an expected outcome, not an error: the
|
||||
# world moves between plan and apply, and every gate fails closed.
|
||||
refusals: Mapped[list[dict[str, Any]]] = mapped_column(
|
||||
JSONB, nullable=False, default=list, server_default=text("'[]'::jsonb"),
|
||||
)
|
||||
|
||||
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
Reference in New Issue
Block a user