feat: placement reconciler tasks + API (4246, slice 3b)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 29s
CI / backend-lint-and-test (push) Successful in 1m1s
Build images / build-web (push) Successful in 1m34s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m26s
Build images / promote (push) Skipped
CI / integration (push) Failing after 3m5s
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 29s
CI / backend-lint-and-test (push) Successful in 1m1s
Build images / build-web (push) Successful in 1m34s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m26s
Build images / promote (push) Skipped
CI / integration (push) Failing after 3m5s
Three Celery tasks wrapping the 3a service, and the endpoints that drive them. Routed to `maintenance_long` alongside backups: 33k renames on NFS have no business in the quick lane where the self-healing sweeps live (the 2026-06-07 starvation). A durability bug in 3a, found by thinking about what a crash costs rather than by a failing test: `apply_run` wrote its ledger only at the end, so a worker dying at row 30,000 of 33,789 would have taken the undo information for the first 29,999 with it — and that ledger is the ONLY record of where those files came from. It now persists every 200 moves. Two things fell out of writing that: - `_persist` reassigns `run.moves`, so the loop had to snapshot the plan first rather than iterate the attribute it rewrites. - the reassignment is itself load-bearing: SQLAlchemy does not track in-place mutation of a JSONB list, so an `.append()` alone would never reach the database and the ledger would have stayed silently empty. Re-running a partially-applied plan is safe — the moved rows no longer match their `from` and refuse as "row moved since planning" — but `apply_placement` deliberately has NO autoretry: re-entering a half-applied plan should be the operator's call after reading what happened, not the queue's. Endpoints gate on run state as well as the service does, so a stray POST cannot re-apply an applied run. The list response omits `moves` (an applied whole-library run carries tens of thousands of entries); the detail endpoint includes them, because that detail IS the preview read before agreeing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -29,7 +29,7 @@ from quart import Blueprint, jsonify, request
|
||||
from sqlalchemy import select
|
||||
|
||||
from ..extensions import get_session
|
||||
from ..models import LibraryAuditRun
|
||||
from ..models import LibraryAuditRun, LibraryPlacementRun
|
||||
from ..services import cleanup_service, library_layout
|
||||
from ._responses import error_response as _bad
|
||||
|
||||
@@ -218,3 +218,91 @@ async def layout_survey():
|
||||
)
|
||||
)
|
||||
return jsonify({**report.as_dict(), "checked_disk": check_disk})
|
||||
|
||||
|
||||
def _serialize_placement_run(run: LibraryPlacementRun, *, moves: bool = False) -> dict:
|
||||
"""`moves` is opt-in: an applied whole-library run carries tens of
|
||||
thousands of entries, which is a fine thing to hold in Postgres and a
|
||||
poor thing to put in every list response."""
|
||||
out = {
|
||||
"id": run.id,
|
||||
"status": run.status,
|
||||
"artist_id": run.artist_id,
|
||||
"started_at": run.started_at.isoformat() if run.started_at else None,
|
||||
"finished_at": run.finished_at.isoformat() if run.finished_at else None,
|
||||
"planned_count": run.planned_count,
|
||||
"moved_count": run.moved_count,
|
||||
"refused_count": run.refused_count,
|
||||
"refusals": run.refusals or [],
|
||||
"error": run.error,
|
||||
}
|
||||
if moves:
|
||||
out["moves"] = run.moves or []
|
||||
return out
|
||||
|
||||
|
||||
@cleanup_bp.route("/placement/runs", methods=["GET"])
|
||||
async def placement_runs():
|
||||
"""Newest first. Without `moves`, so the list stays small."""
|
||||
try:
|
||||
limit = min(int(request.args.get("limit", "25")), 100)
|
||||
except ValueError:
|
||||
return _bad("invalid_limit")
|
||||
async with get_session() as session:
|
||||
rows = (await session.execute(
|
||||
select(LibraryPlacementRun)
|
||||
.order_by(LibraryPlacementRun.id.desc()).limit(limit)
|
||||
)).scalars().all()
|
||||
return jsonify({"runs": [_serialize_placement_run(r) for r in rows]})
|
||||
|
||||
|
||||
@cleanup_bp.route("/placement/runs/<int:run_id>", methods=["GET"])
|
||||
async def placement_run(run_id: int):
|
||||
"""One run WITH its moves — this is the preview the operator reads before
|
||||
agreeing, and the record of what happened afterwards."""
|
||||
async with get_session() as session:
|
||||
run = await session.get(LibraryPlacementRun, run_id)
|
||||
if run is None:
|
||||
return _bad("not_found", status=404)
|
||||
return jsonify(_serialize_placement_run(run, moves=True))
|
||||
|
||||
|
||||
@cleanup_bp.route("/placement/plan", methods=["POST"])
|
||||
async def placement_plan():
|
||||
"""Queue a planning run. `artist_id` scopes it to one artist, which is the
|
||||
intended use: do one, look at the gallery, then continue or revert."""
|
||||
body = await request.get_json(silent=True) or {}
|
||||
artist_id = body.get("artist_id")
|
||||
if artist_id is not None and not isinstance(artist_id, int):
|
||||
return _bad("invalid_artist_id")
|
||||
from ..tasks.library_placement import plan_placement
|
||||
plan_placement.delay(artist_id)
|
||||
return jsonify({"status": "dispatched"}), 202
|
||||
|
||||
|
||||
@cleanup_bp.route("/placement/runs/<int:run_id>/apply", methods=["POST"])
|
||||
async def placement_apply(run_id: int):
|
||||
"""Execute a ready run. This renames files and rewrites rows."""
|
||||
async with get_session() as session:
|
||||
run = await session.get(LibraryPlacementRun, run_id)
|
||||
if run is None:
|
||||
return _bad("not_found", status=404)
|
||||
if run.status != "ready":
|
||||
return _bad("not_ready", detail=f"run is {run.status}")
|
||||
from ..tasks.library_placement import apply_placement
|
||||
apply_placement.delay(run_id)
|
||||
return jsonify({"status": "dispatched"}), 202
|
||||
|
||||
|
||||
@cleanup_bp.route("/placement/runs/<int:run_id>/revert", methods=["POST"])
|
||||
async def placement_revert(run_id: int):
|
||||
"""Put an applied run's files back. The reason the ledger is kept."""
|
||||
async with get_session() as session:
|
||||
run = await session.get(LibraryPlacementRun, run_id)
|
||||
if run is None:
|
||||
return _bad("not_found", status=404)
|
||||
if run.status != "applied":
|
||||
return _bad("not_applied", detail=f"run is {run.status}")
|
||||
from ..tasks.library_placement import revert_placement
|
||||
revert_placement.delay(run_id)
|
||||
return jsonify({"status": "dispatched"}), 202
|
||||
|
||||
Reference in New Issue
Block a user