179c1a9dcc011ebdb9f034aa9a45cf2be438c9ef
336 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bvandeusen
|
179c1a9dcc |
feat(suggestions): visible, reversible rejection in the modal rail
A red-✗ dismissal no longer makes the suggestion vanish. The rejected tag stays in the rail — dimmed, struck-through, with a "rejected" pill and a one-click undo (↶) in place of the ✗ — so a misclick is recoverable and the operator can see what they've said no to (operator-asked 2026-06-27). Backend: SuggestionService.for_image now KEEPS rejected tags, flagged rejected=True, sorted to the bottom of their category, instead of dropping them. New AllowlistService.undismiss + POST /suggestions/undismiss clears the TagSuggestionRejection. Rejected items are still excluded from bulk consensus (for_selection) and the type-to-add dropdown, whose jobs are unchanged. Frontend: store.dismiss flags in place (canonical tags) rather than dropping; new store.undismiss reverts. SuggestionItem renders the rejected state and swaps ✗→↶; ✓ still accepts (which clears the rejection server-side). Tests: rejected-surfaced-flagged-then-reversible (service) + undismiss endpoint idempotency (API). Completes #1134's reversible-rejection half. Heads-as-suggestion-source is the remaining piece. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
1d39afa3b6 |
feat(modal): green ✓ / red ✗ verdict pair on suggestion rows
Replace the single "Accept" pill in the modal Suggestions rail with the eval card's green ✓ / red ✗ language: ✓ accepts the tag (positive), ✗ dismisses it for this image — which already persists a TagSuggestionRejection (hard negative the heads train on). The pair occupies ~the footprint of the old pill, so per-image rejection becomes a one-click peer of accepting instead of being buried in the kebab. Dismiss moves off the 3-dot menu, so the kebab now only carries alias actions and is hidden when none apply (centroid hits with no alias option). Toward #1134 (native per-image negatives in the rail). The bigger piece — heads as a suggestion source feeding this panel — is still ahead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
b69c70ab2b |
feat(tag-eval): "keep" records a confirmation so doubts stop resurfacing
"Keep" on a doubted positive was a no-op, so the same confirmed-correct images came back in "head doubts" every run (operator-flagged: reinforcement keeps surfacing the same images). Add tag_positive_confirmation (mirror of tag_suggestion_rejection): keep → POST /images/<id>/tags/<tag_id>/confirm, and the eval excludes confirmed positives from the doubts list — exactly as rejected items already drop out of the suggest list. The tag stays a positive either way (confirmation is a "reviewed" marker, not a training change). - model TagPositiveConfirmation + migration 0057; confirm endpoint (idempotent). - tag_eval: _confirmed_ids + exclude from head_doubts_positive examples. - store.confirmTag + card "keep" calls it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5143f4c34f |
feat(tag-eval): auto-apply operating point + server-side top-N concept discovery
Two additions driven by "what's the commit threshold?" + "find more tags":
1. High-precision operating point (Bar 4). Per concept, report the threshold that
maximizes recall while holding precision >= a target (default 0.97, configurable
via `precision_target`) — i.e. "could this fire without a human, and how much
would it catch?" `head.auto_apply` = {target, threshold, precision, recall} or
null if the target is unreachable. Surfaced on the card.
2. Server-side concept auto-discovery. `auto_top_n` param unions the explicit
concept list with the N most-tagged general tags (one fast DB query) so the
eval can broaden itself without hand-listing — replaces the slow HTTP directory
paging. Card gains "+ auto-add top-N" and precision-target inputs.
No migration; numpy/sklearn stay lazy. Existing _normalize_params test still
holds (new keys additive; None still falls back to DEFAULT_CONCEPTS).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
fc64f130b8 |
fix(tag-eval): thumbnail click opens the view modal, not Explore
Clicking an example in the maintenance card navigated to /explore/<id> — heavier than wanted (operator: just want a bigger look). Open the existing app-wide ImageViewer modal via modal.open(id) instead: bigger image + tags in place, no navigation away from Settings. The ✓/✗ actions are unaffected (separate overlay buttons). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
13d297b881 |
feat(tag-eval): inline confirm/reject actions on example thumbnails
Closes the learn-from-tags loop directly on the eval lists (operator-flagged: no surface to confirm/refine the head's suggestions). Each thumbnail gets a green ✓ / red ✗ that writes the SAME tables the head trains on: - suggest + ✓ → apply tag (new positive, POST /images/<id>/tags) - suggest + ✗ → record rejection (hard negative, suggestions/dismiss) - doubt + ✗ → remove tag + record rejection (kill bad positive, add negative) - doubt + ✓ → keep (stays a positive, no write) Acted thumbs grey out with a badge; re-run to see the head sharpen. Thumb still links to /explore/<id>. All endpoints already existed — no backend change. Inline is the starting point; longer-term the modal Suggestions rail gets the red "No" (negative) so per-image rejection is native there too (next slice). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4974b7cf77 |
feat(tag-eval): bigger, clickable example thumbnails (label-review queue)
The 56px example thumbs were too small to judge a label (operator-flagged). Bump to 120px and wrap each in a link to /explore/<id> (new tab) so the "head doubts / would suggest" galleries double as a review-and-fix queue — click a doubted positive, land on it in Explore, correct the tag, re-run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6cd7281af5 |
feat(settings): tag-eval admin card — trigger + persisted report (survives nav)
Frontend for #1130. A maintenance tile in Settings → Tagging: - Editable concept list + "Run eval" → POST /api/tag-eval (one running at a time). - Rehydrates on mount via the persisted run (getRun by latest id) and polls while running — so the report SURVIVES navigation (operator-flagged); the task runs backend-side regardless and the card reconnects to its row. - Renders the saved report: per-concept head-vs-centroid metrics table (AP/F1/ precision/recall) with Δ AP, the learning curve (AP @ N positives), and thumbnail galleries (head-would-suggest / head-doubts-positive) for eyeballing. Backend: _examples now stores thumbnail_urls (not just ids) so the report is a self-contained artifact that renders without per-id lookups on reload. No new top-level surface — slots into the existing maintenance area. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
958378312c |
fix(settings): sticky headers on the virtual data tables
Allowlist / Alias / ImportTask tables scroll their bodies (height=360/480) but the column headers scrolled away with the rows, so you lost the column labels (operator-flagged 2026-06-27). Add Vuetify `fixed-header` so the header row stays pinned while the body scrolls. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
d63dfa511a |
fix(explore): bound the 3-pane grid row so a tall rail can't scroll the page
The panes grid had no explicit row, so the implicit `auto` row sized to its tallest pane's content. With Provenance + Tags + a long Suggestions list, the rail outgrew the fixed-height workspace, spilled over and made the WHOLE page scrollable — showing as a weird empty gap at the top (operator-flagged 2026-06-26). grid-template-rows: minmax(0, 1fr) bounds the row to the container so each pane's own overflow-y:auto scrolls internally instead. Reset to `none` in the stacked (<=1100px) layout where the page is meant to scroll. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
e34f79fc56 |
feat(explore): show Provenance in the tag rail (post often names the character)
The post title/description frequently names the character, so surface it while tagging in Explore (operator-asked 2026-06-26). ProvenancePanel gains optional imageId/image props (default = modal store, so the modal is unchanged) since provenance is its own system loaded by id; ExploreView renders it above TagPanel in the right rail, hosted on the anchor. Self-collapses when the image has no provenance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
c8a8e23050 |
feat(explore/tags): return focus to the tag input after every action
Explore is a rapid walk-and-tag surface, so focus must keep returning to the tag input with no extra click (operator-asked 2026-06-26). Two gaps closed: - Navigation hardening: refocus on every focused-image change (neighbour click, breadcrumb, Random image, seed) now runs nextTick → requestAnimationFrame, so it lands AFTER the post-navigation re-render/paint instead of being stolen back by the neighbour-grid re-render. - All tag actions refocus, in both Explore and the modal: tag add (existing/new) and remove now hand focus back like accept-suggestion already did; and the rename + fandom-assignment dialogs refocus on @after-leave (fires after Vuetify's own focus-return to the activator, so ours wins). TagAutocomplete's mobile guard is preserved throughout (no soft-keyboard pop on touch). Modal behaviour gains the same stickier focus — consistent, low-risk. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5be6b9cada |
feat(explore): auto-focus the tag input on every image change
The workspace is built for rapid walk-and-tag, but the tag field was only focused once (TagAutocomplete's on-mount autofocus) — walking to a neighbour left focus behind, so the operator had to click the field each time (operator-asked 2026-06-26). TagPanel now exposes focusTagInput; ExploreView watches the focused image id and re-focuses the field on seed + every walk via nextTick. Reuses the existing focus path, so TagAutocomplete's mobile guard (no soft-keyboard pop on touch) is preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4a1f255164 |
fix(modal): place meta + save block under Provenance, above Tags
Operator-clarified 2026-06-26: the dimensions/size/type + save (floppy) block should sit DIRECTLY above the Tags section — i.e. just under Provenance — not at the very top of the rail. Reorder the rail's main scroll area to Provenance → ImageMetaBar → TagPanel (Related stays pinned at the bottom). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
1728b43167 |
fix(modal): pin Related to rail bottom, floppy-disk download, drop suggestions cap
Right-rail layout fixes (operator-flagged 2026-06-26 — the prior change wasn't
the intended improvement):
- Pin the Related strip to the BOTTOM of the rail: the side becomes a flex
column with a scrolling main area (meta + provenance + tags + suggestions)
and a pinned Related footer (capped at 45% of the rail, scrolls past that).
Related now stays reachable no matter how long Tags/Suggestions run, and
self-collapses (no footer space) when there's nothing to show.
- Remove the 320px suggestions scroll cap (
|
||
|
bvandeusen
|
2d1cddd9b7 |
feat(explore): 3-pane tagging workspace — gallery | viewer | tag rail
Reworks Explore from "anchor + neighbour grid + cluster tag-gap rail" into a persistent 3-pane workspace that unfolds the image modal so you can tag while rabbit-holing (operator concept 2026-06-26): - LEFT neighbour grid (larger thumbs), click = walk; breadcrumb retained. - CENTER light viewer — reuses ImageCanvas + ImageMetaBar(:image) for the focused image; "Open full viewer" still launches the overlay modal. - RIGHT the modal's TagPanel, hosted on the anchor for modal-parity tagging (chips, autocomplete, suggestions + Accept, fandom-on-chip, T/"/" focus). Reuse without destabilising the audited modal store: TagPanel and SuggestionsPanel gain an optional `host` prop (default = modal store, so the image modal is unchanged); the explore store implements the same small tag-CRUD surface (current/currentImageId + reloadTags/addExistingTag/ removeTag/createAndAdd) over the anchor. ImageMetaBar gains an optional `image` prop for the same reason. Drops the mass/cluster tagger (TagGapPanel deleted; clusterIds/thumbById removed) — per-image tagging feeds the per-tag reference-embedding centroid better than bulk ops. Nav: keep the Explore tab but bare /explore now SEEDS a random image (GET /api/showcase?limit=1 → /explore/:id) so the tab kick-starts a rabbit hole; explicit meta.navOrder pins nav order (Explore after Gallery) since router.getRoutes() doesn't preserve declaration order. Note: the backend cluster tag-gaps route/service (#94a) is now frontend-orphaned — left in place; flag for a separate cleanup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
3fcc4aeb43 |
fix(modal): scroll-cap the suggestions list so Related stays reachable
The right rail scrolls as a whole and ProvenancePanel already caps its cards + attachments, but SuggestionsPanel had no cap — a long suggestion set (the General bucket runs to dozens) stretched the rail and pushed the Related strip below the fold. Wrap the suggestion groups in a 320px max-height scroll box (hairline scrollbar matching the provenance regions), so suggestions scroll internally and Related stays visible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
7b712920a4 |
feat(explore): Explore view + tag-gap closing + modal meta/download (#94b–d, #4a/b)
Cluster C frontend, milestone #94. #94b Explore walk: new /explore/:imageId route + ExploreView + explore store. Anchor (reuse /api/gallery/image), neighbour grid (reuse /api/gallery/similar, 24), click a neighbour to re-anchor; in-memory breadcrumb that trims on backtrack (route is the source of truth). Empty/loading/error + no-embedding states. #94c tag-gap closing: components/explore/TagGapPanel — fetches /api/images/cluster/tag-gaps for the anchor+neighbours, a consensus-threshold slider (default 60%), per gap shows present/total + the missing thumbnails + 'Apply to N missing' → /api/tags/images/bulk/tags (source manual) → re-fetch. #94d entry points: 'Explore' button in the modal RelatedStrip; the TopNav entry comes free from the route's meta.title. #4a metadata HUD + #4b split Download: new modal ImageMetaBar (always-on, above ProvenancePanel) shows dimensions/size/type and a split Download button (default Download, chevron → Copy link via utils/clipboard — no clipboard-image, rule 95). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
e49cea3eba |
feat(tagging): allowlist tuning dashboard + post-accept toast + merge preview UI (#7c/#7d/#8b)
Cluster B frontend, milestone #99. #7c: AllowlistTable gains Applied + Covers columns and a live 'covers ~N at T' projection as the operator drags a row's threshold (debounced coverage call, then commits the threshold). allowlist store gains coverage(tagId, threshold) and refreshes coverage_count after a save. #7d: suggestions store surfaces a non-blocking toast when accept/alias newly allowlists a tag — '<verb>: <tag> — allowlisted, auto-applying to ~N images' (N is the projection; apply runs async). Falls back to the plain toast when the tag was already allowlisted. #8b: TagsView merge picker now previews the merge via usePreviewCommit before committing — shows images moving / already-on-target / series pages / alias-or- delete / a thumbnail sample, blocks the Merge button on an incompatible kind/fandom. adminStore.mergeTags gains a dryRun option. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
73dd301dbb |
feat(gallery): OR/exclude tag filtering — light chips + advanced builder (#6b/c/d)
Cluster A, milestone #97. Completes the frontend of the structured tag filter
(backend landed in
|
||
|
bvandeusen
|
23fab983a0 |
feat(gallery): tag→gallery nav from modal chips (#5) + OR/exclude tag scope (#6a)
Cluster A, milestone #97. #5: clicking an image-modal tag chip's body now closes the modal and opens the gallery filtered for that one tag (fresh filter); ✕/kebab stay as the explicit remove/rename controls. #6a (backend of OR/exclude filtering): gallery_service._apply_scope gains a structured tag model — tag_or_groups (AND-of-OR: one EXISTS(tag_id IN group) per group) + tag_exclude (NOT EXISTS(tag_id IN exclude)) — layered additively on the existing tag_ids AND path so cursors/facets/deep-links are untouched. Threaded through scroll/timeline/jump_cursor/facets/similar + facets common dict; _require_single_filter rejects post_id combined with OR/exclude. API parses tag_or (repeatable → one OR-group each) + tag_not (csv exclude). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
7c94d99b9f |
refactor(settings): canonical usePreviewCommit for maintenance preview→commit tiles (#753)
Frontend pattern-consistency sweep (note #1026, the last DRY-thread item). TagMaintenanceCard (4 flows) + PostMaintenanceCard (2 flows) each hand-rolled the same sync preview→commit state machine: a previewData/previewing/committing triple + onPreview/onCommit that dry-run-previews, then applies and collapses the projection (the apply shares the backend predicate, so afterward it's empty). Extract usePreviewCommit({preview, commit, emptyPreview}) owning that lifecycle. The 6 flows become declarative: supply the two thunks + the collapse shape. The normalize flow (commit dispatches a self-resuming background task, not a sync apply) omits emptyPreview so the projection stays and a truthy result = queued. Composable returns are aliased to the cards' existing local names, so the templates only change where they read the apply result (the success badges). Long-Celery-task cards (GatedPurge/VideoDedup) keep useMaintenanceTask — a different pattern (navigable-away task lifecycle), deliberately not merged. Exhaustiveness: no card hand-rolls the refs anymore; the only dryRun:false callers are these two cards, both via the composable. Added a vitest spec for the primitive (collapse static + fn, dispatch-variant, re-preview clears result). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
26589c3d98 |
refactor(admin-store): _guard + _dryRunPost consolidate the store actions (#753 Finding C)
DRY pass follow-up (note #1026). All 13 admin-store actions repeated the same lastError-capture/rethrow wrapper; the 6 Tier-A maintenance actions additionally repeated the dry_run POST shape. - _guard(fn): one copy of the lastError=null / try / catch(set lastError; rethrow) wrapper, used by all 13 actions. - _dryRunPost(url, {dryRun, ...extra}): the dry_run POST shape on top of _guard, used by the 6 maintenance actions. reconcile maps sourceId -> source_id. Public exports + every action signature unchanged (object-opts for Tier-A, positional for cascade/bulk/tag ops), so no card/view changes. Behavior identical. Added frontend spec (the admin store had none): _dryRunPost endpoint+body+default, sourceId->source_id mapping, and _guard capturing lastError + clearing on success. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5269cd0709 |
feat(provenance): capture which archive an extracted image came from (#87)
Images pulled out of a .zip/.rar previously kept no record of WHICH archive
they came from — the member->archive link was computed during extraction and
discarded, leaving only image->post. So the provenance modal could only scope
attachments to the whole post, showing every archive a 'High Resolution Files'
bundle carried instead of the one a given file lives in.
- ImageProvenance.from_attachment_id: nullable FK -> post_attachment.id
(SET NULL), migration 0055.
- importer: _import_archive stamps from_attachment_id on every member's
provenance row for the post (new + superseded + deduped members), resolving
the archive's own PostAttachment by (post, sha). Post-pass UPDATE, NULL-only
and idempotent, so it doesn't touch the dedup/supersede branches and the
backfill is safe to re-run. Nested members link to the outer stored archive.
- provenance_service.for_image: when the originating post's provenance row
records from_attachment_id, return ONLY that archive; else fall back to the
primary-post scoping from
|
||
|
bvandeusen
|
068def2f24 |
fix(provenance): scope attachments to originating post + scroll-cap the list
The Attachments section aggregated PostAttachment rows across EVERY post an image was pHash-linked to. When one of those was a 'High Resolution Files' mega-bundle (dozens of unrelated archives), the list ballooned past the viewport and overwhelmed the modal's right rail. - for_image() now scopes attachments to ImageRecord.primary_post_id (the post the file was actually captured from), falling back to all linked posts only when primary_post_id is unset (older rows / filesystem imports). - ProvenancePanel wraps the list in a max-height scroll container with a count in the heading, mirroring the cards' independent-scroll treatment. Note: FC stores archives as opaque blobs and never records which archive an extracted image came from, so attachments can't yet be scoped tighter than the post. Capturing image->archive containment is tracked as separate work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
311fe0ee9c |
feat(settings): tidy Maintenance tab into compact tiles + center the views (pass 2)
Goal (operator 2026-06-18): the overview of a Settings tab fits one unscrolled viewport; expanding a tile to read into it is the only reason to scroll. - Every Maintenance card converted to the collapsible MaintenanceTile (collapsed by default = icon + short title + one-line blurb). Task cards (ML backfill, centroids, thumbnails, archive re-extract, missing-file repair, DB maintenance) sit in a responsive grid; running tasks auto-expand. Tagging config (suggestion thresholds, allowlist, aliases) grouped in one Tagging section as collapsible tiles; Backup is its own collapsible tile. - Three labeled sections mirror the Cleanup tab: Backfills and reprocessing / Tagging / Storage. - Center the whole Settings surface: SettingsView is now a centered, width-capped (1140px) column so the tab strip and every panel sit in a tidy centered measure (was full-width). CleanupView drops its own left-aligned max-width to fill it. All card logic unchanged - only the chrome. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
3b435dc0ba |
feat(settings): tidy Cleanup tab into sectioned compact tiles (pass 1)
The Cleanup + Maintenance sections had ~17 full-width stacked cards with long descriptions — a hunt to scan. Operator wants compact, sectioned, scannable tiles (2026-06-18: keep both tabs, group inside, compact tiles in a grid). New common/MaintenanceTile.vue: a compact expandable tile (icon + short title + one-line blurb collapsed; click the header to expand the full controls/preview/ result inline; keyboard-accessible button + focus ring; tints the icon, keeps a running task expanded). Cleanup tab (this pass) restructured into 3 sections — Import-filter audits (Min dimensions, Transparency, Single-color) / Duplicates & posts (Bare posts, Duplicate posts, Deduplicate videos, Gated-post previews) / Tags (Unused, Legacy, Reset content tagging, Standardize casing) — each a responsive grid of tiles. PostMaintenanceCard split into 2 tiles, TagMaintenanceCard into 4. Moved VideoDedupCard + GatedPurgeCard from the Maintenance tab here (both are destructive content cleanup). All card logic unchanged — only the chrome. Maintenance tab tiling is pass 2 (TODO noted in MaintenancePanel). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
eff64275fc |
feat(maintenance): reconcile duplicate posts (gallery-dl→native unify)
An artist first downloaded by gallery-dl gets Post rows keyed by the per-
attachment id; a later native walk keys the SAME real post by the post id. They
never dedup (uq_post_source_external_id is on external_post_id) → duplicate post
rows (cheunart: 943→1109). The real post id is recoverable in-DB from
raw_metadata['post_id'] (both eras store the sidecar there).
reconcile_duplicate_posts (cleanup_service): group posts by (source_id, canonical
post_id = raw_metadata.post_id else external_post_id); for each group >1, keep the
row already keyed by the post id (the format the CURRENT native downloader
produces, so future walks dedup and this can't recur), re-point
ImageRecord.primary_post_id / ImageProvenance / PostAttachment / ExternalLink onto
it conflict-safe (drop the loser's row where the keeper already has the equivalent,
per each table's uniqueness), backfill the keeper's empty date/title/body/raw_meta
from a loser, set external_post_id=post_id + derive post_url, delete losers.
IMAGES ARE NOT TOUCHED (content-addressed/deduped already; operator-confirmed).
Preview/apply share find_duplicate_post_groups (rule 93). API
/api/admin/posts/reconcile-duplicates (dry_run→{groups,posts_to_merge,sample};
apply→{groups,merged,sample}; optional source_id). UI: a second section on
PostMaintenanceCard (preview groups+sample → confirm merge). Tests: merge +
metadata backfill + image move, no-op when unique, provenance-collision dedup.
Design: milestone #73. Forensics: note #917. Out of scope (flagged): cheunart vs
Cheunart case-variant artist dirs/rows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
bvandeusen
|
d8d8ecd78f |
feat(subscribestar): flip dispatch to the native ingester (#893, Step 5)
SubscribeStar now downloads + verifies through the native core ingester instead of gallery-dl — the go-live switch for milestone #71. - download_backends: subscribestar added to NATIVE_INGESTER_PLATFORMS; a _NATIVE_INGESTERS registry + _resolve_native_campaign_id make _run_native_ingester / preview_source / verify_source_credential platform-aware. SubscribeStar's campaign_id IS the creator URL (no resolver); Patreon still resolves the vanity. preview now catches the shared NativeIngestError (covers both platforms). - platform_lock: subscribestar serialized (one paced walk at a time). - gallery_dl: subscribestar entry removed from PLATFORM_DEFAULTS (rule 22 — no fallback once native works). - frontend SourceActions: isPatreon → isNative (patreon|subscribestar) so the recover/recapture actions show for subscribestar; download_service's cursor/mode/post_first + the preview endpoint already key on uses_native_ingester, so backfill/recovery/recapture/preview light up for free. - tests: download_backends (subscribestar native), platform_lock (serialized), and three gallery-dl-sample tests repointed to hentaifoundry (api_credentials verify, gallery_dl_service skip-value, api_sources arm-no-preflight). post_is_gated stays best-effort (can't cause junk downloads); not gating this. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
e287802ecb |
fix(maint): resurface dedup/gated-purge results after navigate-away (#877)
Long-running maintenance tasks must survive navigating away or reloading
the page. VideoDedupCard + GatedPurgeCard held the in-flight Celery task id
only in component refs and polled task-result inline, so leaving the page
mid-run lost the id and the result was never shown — even though the task
finished on the worker.
New shared composable useMaintenanceTask: persists {taskId, mode, startedAt}
to localStorage on dispatch, re-attaches on mount, and re-shows the result
when the task finishes (the celery result backend retains the summary well
under result_expires). Stale-guard skips resume past 3h. Both cards refactored
onto it; card-specific computeds + confirm dialog kept.
Also fixed the QueueStatusBar lane: both cards watched queue="maintenance"
but tasks.admin.* routes to maintenance_long, so the bar never reflected
their own task — now queue="maintenance_long".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
bvandeusen
|
540151290b |
feat(cleanup): purge misgrabbed gated-post blurred previews (#874 follow-up)
A one-shot Maintenance action to remove the blurred locked-preview images the ingester downloaded from tier-gated Patreon posts before #874. current_user_can_view was never persisted, so the cleanup re-walks each enabled Patreon source (read-only) to re-derive which posts are gated now and the blurred filehashes Patreon serves for them, then matches by CONTENT HASH against stored source_filehash. Because the hash is content-addressed, a real file downloaded when access existed has a different hash and can never match — regained-then-lost-access content is provably spared (operator's hard requirement). NULL source_filehash => unverifiable, kept + reported. On apply: delete matched ImageRecords + files (provenance cascades), clear seen/dead-letter ledger rows for those hashes so the real media re-ingests if access returns, and delete gated posts left bare. Shares one match predicate between preview and apply (rule 93). - cleanup_service: collect_gated_previews + purge_gated_previews - tasks.admin: purge_gated_previews_task (async re-walk bridge, timeboxed) - api.admin: POST /maintenance/purge-gated-previews - GatedPurgeCard.vue in Settings > Maintenance (preview -> confirm -> apply) - tests: collect predicate, hash-match delete/spare/unverifiable, ledger clear, bare-post removal, no-op Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
369e3de684 |
feat(ml): cadence-based video frame sampling + min-frame tag aggregation (#747)
Video tag noise root cause: frames were a FIXED count (6) max-pooled — a tag firing on one frame survived at peak confidence, and a fixed count under-samples long multi-scene videos so real scene-local tags looked like noise. Redesign (operator-steered): - Sample at a fixed CADENCE — one frame every `video_frame_interval_seconds` (default 4) across the 5–95% window — so a tag's frame-presence reflects real screen time independent of video length. Capped at `video_max_frames` (default 64): a long video stretches the spacing instead of exploding into hundreds of inferences, bounding per-video cost on the single ml-worker (per-frame ffmpeg timeout also cut 60s→30s). - Aggregate with `_aggregate_video_predictions`: keep a tag only if it appears in >= `video_min_tag_frames` sampled frames (≈ that many × interval seconds on screen — duration-independent noise rejection), with confidence = MEAN over the frames it appears in (not max). Clamps the threshold to the sample count so a 1–2-frame short video still tags. - All three knobs are DB-backed ml_settings (migration 0053), patchable via /api/ml/settings + sliders in the ML settings card — replaces the VIDEO_ML_FRAMES env var (product-not-project). Tests: aggregation drops one-frame noise + means corroborated tags + clamps on short videos; settings round-trip + min>max validation. Replaced the _maxpool_predictions unit test. NOTE: this is the QUALITY half of #747. The perf half — the ml-worker runs CPU-only — is GPU enablement, tracked separately in #872. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
41652db20f |
feat(maintenance): retroactive video-dedup action — preview + apply (#871)
Phase 2 of #871: clean up the duplicate videos already in the library (the #859 "same video from multiple sources" clutter). Import-time dedup (Phase 1) only prevents NEW dups; this is the operator-triggered cleanup of existing ones. cleanup_service.dedup_videos(dry_run): - backfill_video_durations: re-probe NULL-duration videos (pre-#871 rows) so the existing library participates; idempotent (only NULL rows), writes a negative sentinel for un-probeable files so they're neither re-probed forever nor matched. - find_video_dup_groups: cluster same-artist videos by duration (±tol) + aspect, anchored per cluster to bound the span (no chain drift); keeper = highest pixel area then bytes. Reuses the importer's _VIDEO_DUP_* tolerances. - apply: re-point each loser's post links to the keeper (so no post loses the video) THEN delete the redundant records + files via delete_images (cascade). dry_run shares the same discovery predicate and returns the projection only (rule 93). Tags on a loser are NOT merged (noted; videos rarely hand-curated). - dedup_videos_task (maintenance queue; summary → task_run.metadata). - POST /maintenance/dedup-videos {dry_run} + GET /maintenance/task-result/<id> so the card shows the dry-run projection before the destructive apply. - VideoDedupCard: Preview → shows groups/redundant/reclaimable, then Apply behind a confirm dialog. Mounted in the Maintenance panel. Tests: dedup collapses + re-links the loser's post to the keeper + removes the file; dry-run deletes nothing; distinct durations aren't grouped; task registered. (Migration 0052 for duration_seconds already shipped with Phase 1.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
949c9abcc6 |
fix(external): path-safe unlink + per-link staging + orphan repair (#859)
External downloads import IN PLACE, so the post-attach dedup-skip unlink could delete a file that IS an ImageRecord's backing file — orphaning the record and 404-ing on playback. Two sources of that: - Two links on the same post (same film from mega + gdrive) emitted the same filename into one external/<post_id>/ dir; the second overwrote the first. Stage per-LINK now (external/<post_id>/<link_id>/) so each file keeps its path. - The duplicate_hash/duplicate_phash branch unlinked `f` unconditionally. Make it path-safe: only unlink when `f` is NOT the existing record's canonical file. Plus an operator-triggered orphan-repair maintenance task (prune_missing_file_records_task) to clean up records already orphaned by the bug: scans ImageRecords, deletes those whose file is gone (cascade), with an NFS-stall guard that aborts without deleting if a large sample is mostly missing. Wired through POST /api/admin/maintenance/prune-missing-files and a MissingFileRepairCard in the Maintenance panel. Tests: refetch-same-link keeps the canonical file; orphan repair deletes only real orphans and aborts on the mostly-missing guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
f897e2534b |
feat(posts): full-width body for image-less posts (drop dead 'no images' box)
Text-only Patreon posts (WIP/announcement/poll — the bulk of a creator's feed) rendered a big empty 'No images attached to this post' placeholder taking half the card. Render the media column only when the post HAS images; image-less posts let the title + body span the full width. Removes the now-dead PostEmptyThumbs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
eb811e11f6 |
refactor(ingest): per-post handling into run stdout via a downloader outcome (#842)
Two corrections from operator review: 1. Reuse the existing 'Raw stdout' panel instead of a bespoke structured UI section — the native ingester now writes a per-post line into the run stdout (parity with gallery-dl's per-file stdout), so the per-post handling shows in the panel the operator already uses. 2. DRY: stop re-reading post['attributes'] inline in ingest_core. write_post_record now returns a PostRecordOutcome (path, post_type, title, body_chars) — mirroring the download_post -> MediaOutcome contract — and the downloader owns the read; ingest_core only formats the outcome into the log line. Reverts the post_diagnostics metadata field + DownloadDetailModal 'Post capture' section added earlier. Per-post line: 'post <id> [<post_type>] body: N chars' (+ ' — EMPTY' when 0), so an empty body is self-explanatory by post_type. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
bcc7266021 |
feat(downloads): per-post body-capture diagnostics in the event UI (#842)
Operator can't (and shouldn't have to) hunt worker logs to see why a recapture
left a post body empty. Surface per-post handling ON THE EVENT, in the UI.
The feed already requests post_type (in _FIELDS_POST), so ingest_core builds a
per-post diagnostic {post_id, title, post_type, body_chars} with zero extra
fetching — a 0-char body next to its post_type explains an empty post at a
glance (e.g. polls/embeds whose body the API never returns).
- ingest_core: accumulate post_diagnostics; thread via DownloadResult
- download_service: write to DownloadEvent.metadata_['post_diagnostics']
- DownloadDetailModal: 'Post capture' section — totals + empty-body table
(post_type + chars, flagged) + all-posts table; included in Copy-all
- tests: ingester diag (post_type + body_chars), download_service metadata
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
bvandeusen
|
65ec29ba9b |
feat(ingest): Recapture mode — re-grab post bodies/links + localize on-disk inline images (#830)
A plain backfill gates post-body capture on the seen-ledger, so a post whose media is already on disk AND whose post key is already seen never gets its body recaptured (operator-flagged: Industrial Lust description missing). Recovery recaptures unconditionally but re-downloads the whole source. New 'recapture' walk mode (4th beside tick/backfill/recovery): bypasses the post-record gate so EVERY post's body + external links are re-captured (detail-fetching empty bodies) WITHOUT re-downloading on-disk media; and surfaces already-present media via a separate non-deleting relink channel so the importer backfills ImageRecord.source_filehash for inline-image localization. - ingest_core: recapture mode + recapture_records gate bypass + relink collect - patreon_downloader: recapture surfaces seen-on-disk as skipped_disk(path), never refetches seen-missing media, still downloads genuinely-new - importer.relink_source_filehash: NULL-only sha256 backfill, never unlinks - download_service: mode derivation + phase-3 relink loop + lifecycle clear - source_service/api: start_recapture + backfill_recapture field + action - frontend: Recapture kebab action + 'Recapturing' badge across SourceActions/ Row/Card/SubscriptionsTab + sources store - tests across ingester/downloader/importer/source_service/api/download_service Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
8dbf29f803 |
feat(external): per-host enable toggles in Settings (Phase 4d)
Operator lever: disable a single file host (e.g. mega.nz when it's banning) without touching the others. Five booleans on import_settings (extdl_<host>_enabled, default true — works out of the box, rule #26); the worker already reads them via getattr so no worker change. Migration 0050 + model fields + settings GET/PATCH (uniform boolean validation) + a 'External file-host downloads' card in the subscriptions Settings tab. Completes Phase 4. Refs FC #830. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
d96918d777 |
feat(posts): extract + record external file-host links (Phase 3)
Capture off-platform links (mega/gdrive/mediafire/dropbox/pixeldrain) embedded in post bodies so they're never silently dropped, and surface them in the post view. The download worker (Phase 4) walks these rows. - link_extract.py: pure extractor — <a href> + bare URLs, unwraps Patreon redirect shims, PRESERVES the full url incl. #fragment (mega's key), dedups. Reusable by every platform (runs off Post.description). - external_link model + migration 0049: post_id/artist_id/host/url/label/status /attempts/last_error/attachment_id/timing; CHECK whitelists (full enum incl. worker statuses up front) + (post_id,url) unique. - importer._sync_external_links: insert-missing on both import paths (_apply_sidecar + upsert_post_record) so a re-import never resets a link's status; runs for all platforms. - post_feed_service.get_post: returns external_links (detail-only). - PostCard: renders the links (host chip + label + status) once expanded. - tests: extractor (5 hosts, fragment, shim unwrap, dedup), importer (record + no-dup on reimport), serializer. Refs FC #830. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
c342c73a25 |
feat(posts): faithful (semantic) HTML rendering of post bodies
Phase 1 of milestone #64. The body is captured (Phase 0) but was shown as plain text. Now: - html_sanitize.py: widen the allowlist to a faithful-but-safe set — headings, inline images, lists, blockquote, hr, code/pre, figure, links (div/span stay stripped; their text is preserved). Benefits the existing ProvenancePanel too. - post_feed_service.get_post: add sanitized `description_html` to the DETAIL response (the feed list stays lightweight plain text by design). - PostCard.vue: render description_html via v-html once expanded (fetched with detail); collapsed + no-detail fallback stay plain text. Styled close to the source (headings, images max-width, accent links, lists, quotes, code). Tests: sanitizer (headings/img/lists survive, img javascript: src dropped); get_post returns sanitized description_html. Refs FC #830. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
7fcef53d5b |
fix(series): sticky tabs + controls on the Series view
The Series tab strip and the Browse search/sort (and Suggestions controls) scrolled away on a long grid (operator-asked). Hoist the tabs + active-tab controls into one sticky header pinned under the 64px TopNav. The controls had to leave v-window — it clips sticky children — so they're driven by the tab from the header instead of living inside each window-item. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5c3f8ebd70 |
fix(aliases): store modal alias under raw model key + make aliases visible/manageable
The headline bug: aliases created from the modal NEVER resolved. Create
sent the normalized display name ('Sword', 'Uchiha Sasuke') while
resolution keys on the raw booru model key ('sword', 'uchiha_sasuke',
case-sensitive) — so the mapping was stored under a key nothing looks up,
and the prediction kept reappearing unaliased. The raw key wasn't even in
the /suggestions response, so the modal couldn't send it.
- Suggestion now carries raw_name (the model key an alias must use) and
via_alias (surfaced via an operator alias); both serialized by the API.
- Modal alias-create sends raw_name, not display_name (the fix). Aliased
suggestions show an 'alias' badge and a 'Remove alias' action; 'Treat as
alias for…' is hidden for centroid hits (no model key) and already-aliased
rows.
- Tag-side management: TagCard ⋮ → 'Aliases…' opens a dialog listing the
model keys that fold into a tag, with remove (GET /api/tags/<id>/aliases +
AliasService.list_for_tag). Creation stays in the modal suggestion flow.
Tests: full API round-trip locking the raw-key contract (raw_name exposed →
alias authored with it → resolves + via_alias on a later image);
list_for_tag (service + API); via_alias/raw_name on the existing service
suggestion tests. No migration.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
3e1303ea3c |
fix(browse): put tabs and search on one row
Operator-asked: the tab strip and search field were stacked; place them side-by-side in a single flex bar (tabs left, search + scope chips right), wrapping to two rows only on narrow viewports. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
2c544ad5af |
feat(browse): sticky tabs + per-tab search bar (server-side, scope-aware)
The Browse tab nav scrolled away (operator didn't know it existed) and Posts had no search. Roll the tab strip + a shared search field into one sticky block pinned under the 64px TopNav. - Posts gains server-side text search: PostFeedService.scroll()/around() + /api/posts accept q (ILIKE over post_title OR description), applied INSIDE the artist/platform WHERE so search stays scoped to the active filter. Scope shown as clearable chips next to the search field. - Artists/Tags search consolidates into the sticky bar: their inner search boxes are removed; they react to route.query.q (q is deep- linkable, e.g. /browse?tab=posts&q=foo). Platform/kind filters stay. - Posts empty state now distinguishes 'no matches' from 'no posts yet'. Tests: posts q-search matches title|description and stays artist-scoped (service); q passthrough (api). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
90c68f8b2a |
fix(series): round the kebab backing on series cards
The tinted backing was set on the square .fc-kebab wrapper span while the button is round, so a translucent square showed behind the round ⋮. border-radius:50% makes the backing a circle matching the button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
013b9d7f06 |
feat(series): operator-set sparse page numbers + gap blocks (#789 tweak)
Replaces the auto-renumbered 1..N position key with operator-OWNED page numbers: sparse, gaps allowed, editable, never auto-renumbered. Order follows the numbers; unnumbered pages sort to the tail. This is the fix for the model that clobbered hand-set numbers on the flatten — numbers are now data, not a derived sequence. - series_service: drop the renumber-on-reorder/remove; order by page_number NULLS LAST; new set_page_number(image_id, n|None); list_pages returns `gaps` (one entry per missing-number run) + each pending group's parsed `start_page`; set_cover renumbers below the current min; place_pending(image_ids, start_page) numbers placed pages sequentially from the start (drop junk first → numbers line up); add_post stamps the parsed start on staged pages. - api/tags: POST /series/<id>/pages/number (set one page's number); /pending/ place takes start_page; removed /reorder. - frontend: per-card editable number input; one gap block per gap with drop-on-edge to assign the adjacent number (middle → type); append drop zone; pending tray gets a "from page N" field + "Place from page N". - tests reworked: sparse numbers + gaps, place-from-start, set-page-number route. No migration; nothing destructive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
7bb765b6ed |
feat(series): pending staging for add-from-post (#789 Phase 2)
Add-from-post no longer appends straight into the run — it STAGES the post's
pages as pending (per-page status; page_number NULL), grouped by source post,
so the operator drops junk (text-free alts, bumpers) and places the keepers
into the sequence with clean series-global numbering.
- migration 0048: series_page.status ('placed' default | 'pending') + nullable
page_number.
- series_service: placed/pending split everywhere (list_pages returns the
placed run + a `pending` section grouped by source post; reorder/cover/
list_series operate on placed only); add_post stages pending; new
place_pending(image_ids, before_image_id=None) flips pending→placed spliced
before a page (or appended) and renumbers; junk removal reuses remove_images.
- api/tags: /add-post now returns staged count; new POST /series/<id>/pending/
place.
- frontend: PostSeriesMenu navigates to the series after staging; seriesManage
store surfaces `pending` + placePending; SeriesManageView gains a pending
tray (per-post groups, place-all / place-one / drop-junk).
- tests: pending staging, place (append + insert-before), ignore-already-
placed, drop-junk, route guard; updated add_post + match-accept expectations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
59746d213d |
feat(series): flat series sequence + cosmetic chapter dividers (#789 Phase 1)
Reframe a series from "ordered chapters that own pages" to ONE flat, series-global ordered run of pages with optional cosmetic chapter DIVIDERS over it. A chapter no longer wraps content — it's a labeled divider anchored to the page that begins it; a page's chapter is derived as the nearest preceding divider. This is what lets installments assembled from multiple sources sit in one continuous, correctly-numbered sequence (operator's Goblin Juice case). - migration 0047: flatten each series to a series-global page_number (preserving today's reading order); convert each existing chapter to a divider anchored at its first page (keeping title/stated_part); drop series_page.chapter_id; reshape series_chapter (anchor_page_id UNIQUE FK, drop chapter_number/is_placeholder/stated_page_start/end). Loss-safe for content; drops empty placeholder chapters + a redundant page-1 divider. - series_page: page_number is now the series-global order; no chapter_id. - series_chapter: anchored divider (anchor_page_id, title, stated_part). - series_service: flat list_pages (one run + derived dividers + per-page source_post + part_gaps), series-wide reorder/renumber, divider CRUD (create/update/move/delete); retired per-chapter reorder/merge/placement. - api/tags: drop chapter_id from add; /chapters endpoints are divider create/update/delete (removed chapter reorder/merge/page-reorder). - series_match_service: series "end" reads max(series_page.stated_page); accept appends via add_post. tag_service series-merge appends src's pages after tgt's max so the merged series stays one clean run. - frontend: seriesManage store + SeriesManageView → one continuous drag-reorder grid with inline divider bars + series-global page numbers; reader walks the flat run, headings from dividers; PostSeriesMenu copy. - tests reworked across the series suite for the divider model. Phase 2 (pending staging for add-from-post) is separate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
3610ba495f |
feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3)
Read cutover verified in prod (suggestions + allowlist read image_prediction; backfill complete at 908k rows / 51k images). Removes the old JSON column and everything that fed it: - ImageRecord.tagger_predictions column removed; migration 0046 DROPs it. tagger_model_version kept as the "tagged / current?" signal the backfill sweep reads (needs-tagging check switched to tagger_model_version IS NULL). - tag_and_embed no longer dual-writes the JSON — image_prediction is the only write path. - importer re-import reset drops the JSON line (image_prediction rows are already deleted on re-import). - Retired the one-time #768 backfill task + the #764 prune task, their admin endpoints, and their Maintenance cards (Backfill/PrunePredictionsCard). - Tests seed/assert via image_prediction; stale column refs removed. Disk reclaim is NOT automatic: DROP COLUMN is a catalog change. Run `VACUUM FULL image_record` off-hours afterward to return the ~100 GB to the OS so DB backups go small (#739). image_prediction (~90 MB) stays in pg_dump — it's the source of truth now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |