Provenance archive linkage, post reconciliation close-out, and the cleanup/admin DRY pass #127

Merged

bvandeusen merged 8 commits from dev into main 2026-06-22 20:22:56 -04:00

Conversation 0 Commits 8 Files Changed 18

+930 -352

bvandeusen commented 2026-06-22 20:22:48 -04:00

Owner

Copy Link

Copy Source

Batch of work on dev, all CI-green.

Provenance: archive → image linkage (milestone #87)

• Stopgap (068def2): scope the modal's Attachments to the originating post + scroll-cap the list, so a "High Resolution Files" bundle post stops ballooning the rail.
• Real fix (5269cd0, af7f007): capture which archive an extracted image came from — ImageProvenance.from_attachment_id (migration 0055), stamped at import for new/superseded/deduped members, idempotent so re-extraction backfills. for_image() shows only the containing archive when known. Guard for sidecar-less archives.

Post reconciliation close-out (milestone #73)

• Reconcile (gallery-dl↔native duplicate posts) was already shipped; closed it out by hardening the seam #87 opened (77d02f5): _repoint_post_links now carries a loser's from_attachment_id onto the keeper before dropping the colliding provenance row.

Cleanup/admin DRY pass (#753)

• A (6281cb1): _run_dry_run_op consolidates the 5 Tier-A dry-run/apply admin handlers.
• B (6599a07): _queued consolidates the 5 Celery-dispatch 202 responses (caught a self-recursion bug mid-edit).
• C (26589c3): _guard (all 13 admin-store actions) + _dryRunPost (the 6 maintenance ones).
• Frontend pattern (7c94d99): usePreviewCommit canonicalizes the sync preview→commit maintenance-tile flow (6 hand-rolled flows across TagMaintenanceCard + PostMaintenanceCard).
• Added route/store/composable test coverage for surfaces that previously had none; pattern catalog updated.

Post-deploy action

Run Settings → Maintenance → Re-extract Archives once to backfill from_attachment_id across the existing library.

🤖 Generated with Claude Code

Batch of work on `dev`, all CI-green. ## Provenance: archive → image linkage (milestone #87) - Stopgap (068def2): scope the modal's Attachments to the originating post + scroll-cap the list, so a "High Resolution Files" bundle post stops ballooning the rail. - Real fix (5269cd0, af7f007): capture **which archive an extracted image came from** — `ImageProvenance.from_attachment_id` (migration 0055), stamped at import for new/superseded/deduped members, idempotent so re-extraction backfills. `for_image()` shows only the containing archive when known. Guard for sidecar-less archives. ## Post reconciliation close-out (milestone #73) - Reconcile (gallery-dl↔native duplicate posts) was already shipped; closed it out by hardening the seam #87 opened (77d02f5): `_repoint_post_links` now carries a loser's `from_attachment_id` onto the keeper before dropping the colliding provenance row. ## Cleanup/admin DRY pass (#753) - A (6281cb1): `_run_dry_run_op` consolidates the 5 Tier-A dry-run/apply admin handlers. - B (6599a07): `_queued` consolidates the 5 Celery-dispatch 202 responses (caught a self-recursion bug mid-edit). - C (26589c3): `_guard` (all 13 admin-store actions) + `_dryRunPost` (the 6 maintenance ones). - Frontend pattern (7c94d99): `usePreviewCommit` canonicalizes the sync preview→commit maintenance-tile flow (6 hand-rolled flows across TagMaintenanceCard + PostMaintenanceCard). - Added route/store/composable test coverage for surfaces that previously had none; pattern catalog updated. ## Post-deploy action Run **Settings → Maintenance → Re-extract Archives** once to backfill `from_attachment_id` across the existing library. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 8 commits 2026-06-22 20:22:49 -04:00

fix(provenance): scope attachments to originating post + scroll-cap the list ... 068def2f24

The Attachments section aggregated PostAttachment rows across EVERY post an
image was pHash-linked to. When one of those was a 'High Resolution Files'
mega-bundle (dozens of unrelated archives), the list ballooned past the
viewport and overwhelmed the modal's right rail.

- for_image() now scopes attachments to ImageRecord.primary_post_id (the post
  the file was actually captured from), falling back to all linked posts only
  when primary_post_id is unset (older rows / filesystem imports).
- ProvenancePanel wraps the list in a max-height scroll container with a count
  in the heading, mirroring the cards' independent-scroll treatment.

Note: FC stores archives as opaque blobs and never records which archive an
extracted image came from, so attachments can't yet be scoped tighter than the
post. Capturing image->archive containment is tracked as separate work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

feat(provenance): capture which archive an extracted image came from (#87) ... 5269cd0709

Images pulled out of a .zip/.rar previously kept no record of WHICH archive
they came from — the member->archive link was computed during extraction and
discarded, leaving only image->post. So the provenance modal could only scope
attachments to the whole post, showing every archive a 'High Resolution Files'
bundle carried instead of the one a given file lives in.

- ImageProvenance.from_attachment_id: nullable FK -> post_attachment.id
  (SET NULL), migration 0055.
- importer: _import_archive stamps from_attachment_id on every member's
  provenance row for the post (new + superseded + deduped members), resolving
  the archive's own PostAttachment by (post, sha). Post-pass UPDATE, NULL-only
  and idempotent, so it doesn't touch the dedup/supersede branches and the
  backfill is safe to re-run. Nested members link to the outer stored archive.
- provenance_service.for_image: when the originating post's provenance row
  records from_attachment_id, return ONLY that archive; else fall back to the
  primary-post scoping from 068def2.
- ProvenancePanel: heading pluralizes ('Attachment' for a single file).
- Backfill: re-running reextract_archive_attachments (ArchiveReextractCard)
  routes through _import_archive and stamps existing rows — no new code.

Tests: capture stamps on fresh import, nested-archive attribution, per-post
archive on dedup; for_image filters to the containing archive; reextract
backfill stamps the link.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(importer): guard _stamp_member_archive against a sidecar-less archive ... af7f0078bc

A filesystem-imported archive with no adjacent sidecar has no Post, so
_post_for_sidecar returns None — and the milestone-#87 stamp call dereferenced
post.id. _stamp_member_archive already no-ops on a None post_id (no post → no
provenance to stamp); pass None instead of crashing. Caught by the existing
test_reimport_archive_is_idempotent (no-sidecar zip).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

fix(reconcile): preserve from_attachment_id when merging duplicate posts (#73/#87) ... 77d02f57ae

Milestone #73 (reconcile duplicate gallery-dl/native post rows) shipped in
eff6427; closing it out after today's #87 work, which added a seam it didn't
account for. _repoint_post_links drops a loser post's ImageProvenance row on the
(image, post) uniqueness collision — and that row may now carry from_attachment_id
(which archive the file was extracted from). For the exact gallery-dl->native
case this targets, the keeper is the native stub (no archive) and the loser is the
gallery-dl row that extracted the member, so a blind delete silently lost the
containing-archive linkage. Carry from_attachment_id onto the keeper's surviving
row (when NULL) before dropping the collision.

The rarer PostAttachment-collision case (both dup posts captured the same archive
blob) doesn't arise in the targeted scenario — the archive lives only on the
gallery-dl post, so it re-points straight to the keeper and the FK stays valid.

Test: collision merge preserves the loser's from_attachment_id on the keeper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

refactor(admin): consolidate Tier-A dry-run/apply handlers onto one helper (#753) ... 6281cb1e66

DRY pass on the cleanup/admin destructive-ops surface (task #753, hardened
process #594). Five Tier-A endpoints repeated the same get_json -> dry_run ->
run_sync(service_fn) -> jsonify block verbatim. Extract _run_dry_run_op(service_fn,
**kwargs); the five route handlers now delegate. reconcile keeps its source_id
validation and passes it through **kwargs.

The cleanup_service predicates were already shared between preview and apply
(find_*_conditions / find_duplicate_post_groups) — the post-data-loss fix — so no
backend-logic change; this is purely the HTTP-handler boilerplate.

Consumers (all routed through the helper, verified no copy left behind):
  prune_unused_tags, prune_bare_posts, reconcile_duplicate_posts (+source_id),
  purge_legacy_tags, reset_content_tagging.

Added route-level tests for prune-bare (apply) and reconcile (apply + source_id
passthrough + invalid-source_id 400) — the two helper consumers that previously
had only service-level coverage, so every consumer is exercised at the route.

Findings B (queued-response helper) and C (store dry-run POST helper) identified
but not applied this pass (operator scoped to A). The card preview->commit state
machine is deferred to a frontend pattern-consistency sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

refactor(admin): consolidate maintenance-trigger 202 responses onto _queued() (#753 Finding B) ... 6599a07468

DRY pass follow-up (note #1026). Five handlers returned the identical
jsonify({task_id, status:queued}), 202 shape; extract _queued(async_result).

Consumers routed through it: tags_normalize (live branch), trigger_reextract_archives,
trigger_prune_missing_files, trigger_dedup_videos, trigger_purge_gated_previews.
trigger_vacuum stays bespoke (returns no task_id — the UI doesn't poll it).

Added route-level tests for all five consumers (these trigger endpoints had no
route coverage before): 202 + task_id via _queued, and the dry_run flag threading
through to dedup/purge-gated. Behavior unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

refactor(admin-store): _guard + _dryRunPost consolidate the store actions (#753 Finding C) ... 26589c3d98

DRY pass follow-up (note #1026). All 13 admin-store actions repeated the same
lastError-capture/rethrow wrapper; the 6 Tier-A maintenance actions additionally
repeated the dry_run POST shape.

- _guard(fn): one copy of the lastError=null / try / catch(set lastError; rethrow)
  wrapper, used by all 13 actions.
- _dryRunPost(url, {dryRun, ...extra}): the dry_run POST shape on top of _guard,
  used by the 6 maintenance actions. reconcile maps sourceId -> source_id.

Public exports + every action signature unchanged (object-opts for Tier-A,
positional for cascade/bulk/tag ops), so no card/view changes. Behavior identical.

Added frontend spec (the admin store had none): _dryRunPost endpoint+body+default,
sourceId->source_id mapping, and _guard capturing lastError + clearing on success.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

refactor(settings): canonical usePreviewCommit for maintenance preview→commit tiles (#753) ... 7c94d99b9f

Frontend pattern-consistency sweep (note #1026, the last DRY-thread item).
TagMaintenanceCard (4 flows) + PostMaintenanceCard (2 flows) each hand-rolled the
same sync preview→commit state machine: a previewData/previewing/committing
triple + onPreview/onCommit that dry-run-previews, then applies and collapses the
projection (the apply shares the backend predicate, so afterward it's empty).

Extract usePreviewCommit({preview, commit, emptyPreview}) owning that lifecycle.
The 6 flows become declarative: supply the two thunks + the collapse shape. The
normalize flow (commit dispatches a self-resuming background task, not a sync
apply) omits emptyPreview so the projection stays and a truthy result = queued.

Composable returns are aliased to the cards' existing local names, so the
templates only change where they read the apply result (the success badges).
Long-Celery-task cards (GatedPurge/VideoDedup) keep useMaintenanceTask — a
different pattern (navigable-away task lifecycle), deliberately not merged.

Exhaustiveness: no card hand-rolls the refs anymore; the only dryRun:false
callers are these two cards, both via the composable. Added a vitest spec for
the primitive (collapse static + fn, dispatch-variant, re-preview clears result).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

bvandeusen merged commit bc69495a16 into main 2026-06-22 20:22:56 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#127