22c3b54746768ec800ecb9a995ac3b3adcfa9f9f
856 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bvandeusen
|
22c3b54746 |
feat(heads): production per-concept heads — train + score backend (#114 A)
The eval (#1130) proved the frozen-embedding + trained-head spine; this lands its production form (the first of three slices that make heads the suggestion source, replacing Camie + centroid). - tag_head: one logistic-regression head per general/character concept with enough labelled positives. Weights (pgvector), honest CV-derived suggest threshold + earned-auto-apply point, and per-concept quality metrics. - head_training_run: persisted batch lifecycle (mirrors tag_eval_run) so the admin card shows live + historical status across navigation. - services/ml/heads.py: TRAIN (sync, ml worker, reuses tag_eval's proven data loaders + metric math so production heads match measured eval numbers) and SCORE (async, API worker — numpy via pgvector, no scikit-learn): score one image's embedding against all heads → the rail's suggestions, cached on (count, max trained_at) so a retrain invalidates without per-request loads. - tasks.ml.train_heads (ml queue, commits per head so a kill leaves progress) + recover_stalled_head_training_runs sweep + retention(20) + 5-min beat (rule 89). - api/heads.py: POST /api/heads/train (one run at a time, 409 guard) + GET /api/heads (count, graduated, last-trained, running, per-concept table, recent runs). - ml_settings: head_min_positives + head_auto_apply_precision, tunable via /api/ml/settings. Scoring isn't wired into the rail yet (slice C) and the admin UI is slice B — this slice makes training + scoring exist and CI-verifiable. 'precision' column stored as precision_cv (SQL reserved word). Migration 0058. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
179c1a9dcc |
feat(suggestions): visible, reversible rejection in the modal rail
A red-✗ dismissal no longer makes the suggestion vanish. The rejected tag stays in the rail — dimmed, struck-through, with a "rejected" pill and a one-click undo (↶) in place of the ✗ — so a misclick is recoverable and the operator can see what they've said no to (operator-asked 2026-06-27). Backend: SuggestionService.for_image now KEEPS rejected tags, flagged rejected=True, sorted to the bottom of their category, instead of dropping them. New AllowlistService.undismiss + POST /suggestions/undismiss clears the TagSuggestionRejection. Rejected items are still excluded from bulk consensus (for_selection) and the type-to-add dropdown, whose jobs are unchanged. Frontend: store.dismiss flags in place (canonical tags) rather than dropping; new store.undismiss reverts. SuggestionItem renders the rejected state and swaps ✗→↶; ✓ still accepts (which clears the rejection server-side). Tests: rejected-surfaced-flagged-then-reversible (service) + undismiss endpoint idempotency (API). Completes #1134's reversible-rejection half. Heads-as-suggestion-source is the remaining piece. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
1d39afa3b6 |
feat(modal): green ✓ / red ✗ verdict pair on suggestion rows
Replace the single "Accept" pill in the modal Suggestions rail with the eval card's green ✓ / red ✗ language: ✓ accepts the tag (positive), ✗ dismisses it for this image — which already persists a TagSuggestionRejection (hard negative the heads train on). The pair occupies ~the footprint of the old pill, so per-image rejection becomes a one-click peer of accepting instead of being buried in the kebab. Dismiss moves off the 3-dot menu, so the kebab now only carries alias actions and is hidden when none apply (centroid hits with no alias option). Toward #1134 (native per-image negatives in the rail). The bigger piece — heads as a suggestion source feeding this panel — is still ahead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
b69c70ab2b |
feat(tag-eval): "keep" records a confirmation so doubts stop resurfacing
"Keep" on a doubted positive was a no-op, so the same confirmed-correct images came back in "head doubts" every run (operator-flagged: reinforcement keeps surfacing the same images). Add tag_positive_confirmation (mirror of tag_suggestion_rejection): keep → POST /images/<id>/tags/<tag_id>/confirm, and the eval excludes confirmed positives from the doubts list — exactly as rejected items already drop out of the suggest list. The tag stays a positive either way (confirmation is a "reviewed" marker, not a training change). - model TagPositiveConfirmation + migration 0057; confirm endpoint (idempotent). - tag_eval: _confirmed_ids + exclude from head_doubts_positive examples. - store.confirmTag + card "keep" calls it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4fd8790c85 |
fix(tag-eval): don't re-suggest already-rejected items every run
"head would suggest" drew from the whole negative pool, which INCLUDES the images the operator rejected. A rejected near-miss (e.g. an orc under "goblin") is a hard negative that still scores high, so it kept resurfacing as a fresh suggestion every run (operator-flagged: "same items keep appearing"). Exclude already-rejected ids from the suggest list — once you've said no, it's gone. (head doubts = lowest-scoring positives is unchanged; genuinely-hard true positives legitimately recur there.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5143f4c34f |
feat(tag-eval): auto-apply operating point + server-side top-N concept discovery
Two additions driven by "what's the commit threshold?" + "find more tags":
1. High-precision operating point (Bar 4). Per concept, report the threshold that
maximizes recall while holding precision >= a target (default 0.97, configurable
via `precision_target`) — i.e. "could this fire without a human, and how much
would it catch?" `head.auto_apply` = {target, threshold, precision, recall} or
null if the target is unreachable. Surfaced on the card.
2. Server-side concept auto-discovery. `auto_top_n` param unions the explicit
concept list with the N most-tagged general tags (one fast DB query) so the
eval can broaden itself without hand-listing — replaces the slow HTTP directory
paging. Card gains "+ auto-add top-N" and precision-target inputs.
No migration; numpy/sklearn stay lazy. Existing _normalize_params test still
holds (new keys additive; None still falls back to DEFAULT_CONCEPTS).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
fc64f130b8 |
fix(tag-eval): thumbnail click opens the view modal, not Explore
Clicking an example in the maintenance card navigated to /explore/<id> — heavier than wanted (operator: just want a bigger look). Open the existing app-wide ImageViewer modal via modal.open(id) instead: bigger image + tags in place, no navigation away from Settings. The ✓/✗ actions are unaffected (separate overlay buttons). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
13d297b881 |
feat(tag-eval): inline confirm/reject actions on example thumbnails
Closes the learn-from-tags loop directly on the eval lists (operator-flagged: no surface to confirm/refine the head's suggestions). Each thumbnail gets a green ✓ / red ✗ that writes the SAME tables the head trains on: - suggest + ✓ → apply tag (new positive, POST /images/<id>/tags) - suggest + ✗ → record rejection (hard negative, suggestions/dismiss) - doubt + ✗ → remove tag + record rejection (kill bad positive, add negative) - doubt + ✓ → keep (stays a positive, no write) Acted thumbs grey out with a badge; re-run to see the head sharpen. Thumb still links to /explore/<id>. All endpoints already existed — no backend change. Inline is the starting point; longer-term the modal Suggestions rail gets the red "No" (negative) so per-image rejection is native there too (next slice). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4974b7cf77 |
feat(tag-eval): bigger, clickable example thumbnails (label-review queue)
The 56px example thumbs were too small to judge a label (operator-flagged). Bump to 120px and wrap each in a link to /explore/<id> (new tab) so the "head doubts / would suggest" galleries double as a review-and-fix queue — click a doubted positive, land on it in Explore, correct the tag, re-run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6cd7281af5 |
feat(settings): tag-eval admin card — trigger + persisted report (survives nav)
Frontend for #1130. A maintenance tile in Settings → Tagging: - Editable concept list + "Run eval" → POST /api/tag-eval (one running at a time). - Rehydrates on mount via the persisted run (getRun by latest id) and polls while running — so the report SURVIVES navigation (operator-flagged); the task runs backend-side regardless and the card reconnects to its row. - Renders the saved report: per-concept head-vs-centroid metrics table (AP/F1/ precision/recall) with Δ AP, the learning curve (AP @ N positives), and thumbnail galleries (head-would-suggest / head-doubts-positive) for eyeballing. Backend: _examples now stores thumbnail_urls (not just ids) so the report is a self-contained artifact that renders without per-id lookups on reload. No new top-level surface — slots into the existing maintenance area. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6e3c5f697f |
feat(ml): tag-eval backend — head-vs-centroid learning-curve eval (persisted)
Slice 1 of milestone #114 (tagging v2). Proves the frozen-embedding + trained- head spine on the operator's own data, reusing the SigLIP embeddings already stored on image_record — no re-embedding, no GPU. Per concept: train a logistic-regression HEAD (positives + negatives = explicit rejections + sampled unlabeled) vs the old single-CENTROID baseline; report cross-validated precision/recall/AP for both, a LEARNING CURVE (AP/F1 as tagged positives grow 10→30→100→300), and example image ids (head-would-suggest / head-doubts-positive) to eyeball. Persisted so the report SURVIVES navigation (operator-flagged): the run + full report live in a new tag_eval_run row (mirrors library_audit_run); the admin card will rehydrate from GET on mount, not transient state. - models.TagEvalRun + migration 0056; runs on the ml queue (only worker with numpy/sklearn) — numpy/sklearn lazy-imported so the API can still enqueue. - services/ml/tag_eval (compute + start helper, one-running guard), tasks.ml .tag_eval_run, api/tag-eval (POST create, GET history light / detail w/ report). - recover_stalled_tag_eval_runs sweep + retention (keep last 20) + 5-min beat (rule 89). scikit-learn added to requirements-ml. - tests: param normalization + the rehydrate read-path + create/conflict. Frontend admin card (trigger + render persisted report) follows next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
958378312c |
fix(settings): sticky headers on the virtual data tables
Allowlist / Alias / ImportTask tables scroll their bodies (height=360/480) but the column headers scrolled away with the rows, so you lost the column labels (operator-flagged 2026-06-27). Add Vuetify `fixed-header` so the header row stays pinned while the body scrolls. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
d63dfa511a |
fix(explore): bound the 3-pane grid row so a tall rail can't scroll the page
The panes grid had no explicit row, so the implicit `auto` row sized to its tallest pane's content. With Provenance + Tags + a long Suggestions list, the rail outgrew the fixed-height workspace, spilled over and made the WHOLE page scrollable — showing as a weird empty gap at the top (operator-flagged 2026-06-26). grid-template-rows: minmax(0, 1fr) bounds the row to the container so each pane's own overflow-y:auto scrolls internally instead. Reset to `none` in the stacked (<=1100px) layout where the page is meant to scroll. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
e34f79fc56 |
feat(explore): show Provenance in the tag rail (post often names the character)
The post title/description frequently names the character, so surface it while tagging in Explore (operator-asked 2026-06-26). ProvenancePanel gains optional imageId/image props (default = modal store, so the modal is unchanged) since provenance is its own system loaded by id; ExploreView renders it above TagPanel in the right rail, hosted on the anchor. Self-collapses when the image has no provenance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
c8a8e23050 |
feat(explore/tags): return focus to the tag input after every action
Explore is a rapid walk-and-tag surface, so focus must keep returning to the tag input with no extra click (operator-asked 2026-06-26). Two gaps closed: - Navigation hardening: refocus on every focused-image change (neighbour click, breadcrumb, Random image, seed) now runs nextTick → requestAnimationFrame, so it lands AFTER the post-navigation re-render/paint instead of being stolen back by the neighbour-grid re-render. - All tag actions refocus, in both Explore and the modal: tag add (existing/new) and remove now hand focus back like accept-suggestion already did; and the rename + fandom-assignment dialogs refocus on @after-leave (fires after Vuetify's own focus-return to the activator, so ours wins). TagAutocomplete's mobile guard is preserved throughout (no soft-keyboard pop on touch). Modal behaviour gains the same stickier focus — consistent, low-risk. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
e3855a5ae0 |
chore(tags): remove orphaned cluster tag-gaps route + service method
The cluster tag-gap feature's only UI (Explore's TagGapPanel) was removed in the 3-pane rework, leaving the backend that fed it with no caller. Surgical removal: - drop the POST /api/images/cluster/tag-gaps route (cluster_tag_gaps) - drop BulkTagService.tag_gaps (+ the now-unused `import math`) - drop the tag_gaps tests (test_bulk_tag_service, test_api_bulk_tags) BulkTagService's common_tags / bulk_add / bulk_remove stay — they still back the gallery bulk editor. Pure deletion, no behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5be6b9cada |
feat(explore): auto-focus the tag input on every image change
The workspace is built for rapid walk-and-tag, but the tag field was only focused once (TagAutocomplete's on-mount autofocus) — walking to a neighbour left focus behind, so the operator had to click the field each time (operator-asked 2026-06-26). TagPanel now exposes focusTagInput; ExploreView watches the focused image id and re-focuses the field on seed + every walk via nextTick. Reuses the existing focus path, so TagAutocomplete's mobile guard (no soft-keyboard pop on touch) is preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4a1f255164 |
fix(modal): place meta + save block under Provenance, above Tags
Operator-clarified 2026-06-26: the dimensions/size/type + save (floppy) block should sit DIRECTLY above the Tags section — i.e. just under Provenance — not at the very top of the rail. Reorder the rail's main scroll area to Provenance → ImageMetaBar → TagPanel (Related stays pinned at the bottom). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
1728b43167 |
fix(modal): pin Related to rail bottom, floppy-disk download, drop suggestions cap
Right-rail layout fixes (operator-flagged 2026-06-26 — the prior change wasn't
the intended improvement):
- Pin the Related strip to the BOTTOM of the rail: the side becomes a flex
column with a scrolling main area (meta + provenance + tags + suggestions)
and a pinned Related footer (capped at 45% of the rail, scrolls past that).
Related now stays reachable no matter how long Tags/Suggestions run, and
self-collapses (no footer space) when there's nothing to show.
- Remove the 320px suggestions scroll cap (
|
||
|
bvandeusen
|
2d1cddd9b7 |
feat(explore): 3-pane tagging workspace — gallery | viewer | tag rail
Reworks Explore from "anchor + neighbour grid + cluster tag-gap rail" into a persistent 3-pane workspace that unfolds the image modal so you can tag while rabbit-holing (operator concept 2026-06-26): - LEFT neighbour grid (larger thumbs), click = walk; breadcrumb retained. - CENTER light viewer — reuses ImageCanvas + ImageMetaBar(:image) for the focused image; "Open full viewer" still launches the overlay modal. - RIGHT the modal's TagPanel, hosted on the anchor for modal-parity tagging (chips, autocomplete, suggestions + Accept, fandom-on-chip, T/"/" focus). Reuse without destabilising the audited modal store: TagPanel and SuggestionsPanel gain an optional `host` prop (default = modal store, so the image modal is unchanged); the explore store implements the same small tag-CRUD surface (current/currentImageId + reloadTags/addExistingTag/ removeTag/createAndAdd) over the anchor. ImageMetaBar gains an optional `image` prop for the same reason. Drops the mass/cluster tagger (TagGapPanel deleted; clusterIds/thumbById removed) — per-image tagging feeds the per-tag reference-embedding centroid better than bulk ops. Nav: keep the Explore tab but bare /explore now SEEDS a random image (GET /api/showcase?limit=1 → /explore/:id) so the tab kick-starts a rabbit hole; explicit meta.navOrder pins nav order (Explore after Gallery) since router.getRoutes() doesn't preserve declaration order. Note: the backend cluster tag-gaps route/service (#94a) is now frontend-orphaned — left in place; flag for a separate cleanup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
1aadf3267b |
fix(tags): correct directory image_count — fandom leg must correlate the outer tag
The directory card count regressed to a globally-inflated number (~every card showed the same ~469): the fandom leg used a doubly-nested correlated subquery — image_tag.tag_id IN (SELECT member.id WHERE member.fandom_id == Tag.id) — whose inner predicate did not correlate the outer Tag, so it matched EVERY character that has any fandom and counted all their images for every tag. The gallery scope and cleanup count were unaffected (they pass a literal tag id, a single-level subquery), which is why only the card diverged from the gallery. Rewrite the count as a single-level correlated scalar subquery: join `member` (the tag applied to the image) and match image_tag.tag_id == Tag.id (direct) OR member.fandom_id == Tag.id (a character of this fandom). Strengthen the directory test with a second unrelated fandom/character so a non-correlating fandom leg fails (count would read 4 instead of 3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
10434509d3 |
fix(tags): fandom views aggregate images via their characters
A fandom owns characters via Tag.fandom_id, but every image<->tag query went purely through direct image_tag rows, so a fandom only surfaced images literally tagged with it — images carrying one of its characters were invisible to its browse count, previews, and gallery filter. Derive membership at query time instead of materializing fandom rows (which would drift on every reassign/merge/remove). Add one shared predicate in tag_query.py — image_in_tag_scope / image_in_any_tag_scope: an image belongs to a tag if tagged with it directly OR (when the tag is a fandom) carrying a character whose fandom_id is that tag. The character leg is empty for non-fandom tags, so it applies uniformly with no kind branching. Route all read sites through it: - gallery _apply_scope: include, OR-groups, and symmetric exclude - directory image_count: correlated COUNT(DISTINCT) scalar subquery - directory previews: UNION direct + via-character, then ROW_NUMBER<=3 - cleanup count_tag_associations: Tier-B delete prompt now reports a fandom's true blast radius (was 0 for fandoms with no direct rows) find_unused_tags already protected fandoms via used_via_fandom; left as is. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
b85327a79d |
fix(celery): harden broker connection so workers ride out a Redis blip
A swarm overlay-network blip after the :latest redeploy left Redis healthy but transiently unreachable; a worker starting in that window crash-looped on the initial broker connect (kombu OperationalError) and needed a manual Redis reset to recover. Retry the broker forever on startup + at runtime (broker_connection_max_retries =None), add redis-transport socket options to the broker (short connect timeout, TCP keepalive, retry_on_timeout, periodic health check), and mirror the same on the Redis result backend. Now a transient outage self-heals when overlay routing returns instead of the worker exiting. Test pins the key resilience settings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
3fcc4aeb43 |
fix(modal): scroll-cap the suggestions list so Related stays reachable
The right rail scrolls as a whole and ProvenancePanel already caps its cards + attachments, but SuggestionsPanel had no cap — a long suggestion set (the General bucket runs to dozens) stretched the rail and pushed the Related strip below the fold. Wrap the suggestion groups in a 320px max-height scroll box (hairline scrollbar matching the provenance regions), so suggestions scroll internally and Related stays visible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
7b712920a4 |
feat(explore): Explore view + tag-gap closing + modal meta/download (#94b–d, #4a/b)
Cluster C frontend, milestone #94. #94b Explore walk: new /explore/:imageId route + ExploreView + explore store. Anchor (reuse /api/gallery/image), neighbour grid (reuse /api/gallery/similar, 24), click a neighbour to re-anchor; in-memory breadcrumb that trims on backtrack (route is the source of truth). Empty/loading/error + no-embedding states. #94c tag-gap closing: components/explore/TagGapPanel — fetches /api/images/cluster/tag-gaps for the anchor+neighbours, a consensus-threshold slider (default 60%), per gap shows present/total + the missing thumbnails + 'Apply to N missing' → /api/tags/images/bulk/tags (source manual) → re-fetch. #94d entry points: 'Explore' button in the modal RelatedStrip; the TopNav entry comes free from the route's meta.title. #4a metadata HUD + #4b split Download: new modal ImageMetaBar (always-on, above ProvenancePanel) shows dimensions/size/type and a split Download button (default Download, chevron → Copy link via utils/clipboard — no clipboard-image, rule 95). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
0ecd1ce4f1 |
feat(explore): cluster-consensus tag-gaps service + route (#94a)
Cluster C, milestone #94. BulkTagService.tag_gaps(image_ids, threshold) finds tags applied to >= threshold fraction of a visual neighbour set but not all of it (the '7 of 10 share Miku; these 3 don't' signal). Each gap carries the laggard image ids minus any TagSuggestionRejection rows, so apply-to-cluster never re-proposes a tag a neighbour dismissed. 100%-common tags and <2-image sets are excluded. New POST /api/images/cluster/tag-gaps. Tests: consensus found / common excluded / missing ids; rejected laggard excluded from missing; tag dropped when all laggards rejected; <2 images empty; route shape + bad input. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
0cd2f391ee |
test(allowlist): unique image paths in coverage tests (CI fix)
The new coverage tests' sequential shas (c{i:063d}) share their first 8 chars,
so deriving the image path from sha[:8] collided on uq_image_record_path. Use
the full sha in the path. Same hardening for test_api_suggestions._img.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX
|
||
|
bvandeusen
|
e49cea3eba |
feat(tagging): allowlist tuning dashboard + post-accept toast + merge preview UI (#7c/#7d/#8b)
Cluster B frontend, milestone #99. #7c: AllowlistTable gains Applied + Covers columns and a live 'covers ~N at T' projection as the operator drags a row's threshold (debounced coverage call, then commits the threshold). allowlist store gains coverage(tagId, threshold) and refreshes coverage_count after a save. #7d: suggestions store surfaces a non-blocking toast when accept/alias newly allowlists a tag — '<verb>: <tag> — allowlisted, auto-applying to ~N images' (N is the projection; apply runs async). Falls back to the plain toast when the tag was already allowlisted. #8b: TagsView merge picker now previews the merge via usePreviewCommit before committing — shows images moving / already-on-target / series pages / alias-or- delete / a thumbnail sample, blocks the Merge button on an incompatible kind/fandom. adminStore.mergeTags gains a dryRun option. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
7127714316 |
feat(tags): non-mutating merge preview + admin dry_run (#8a)
Cluster B, milestone #99. TagService.merge_preview(source, target) computes the same counts the apply produces (rule 93 parity) without mutating: images_moving (source links the apply UPDATEs), images_already_on_target (links it drops), source_total, series_pages, will_alias (_keep_as_alias), a kind/fandom compatible flag (surfaced, not raised, so the UI can warn), and up to 6 thumbnails of the moving images. The admin /tags/<dest>/merge route gains a dry_run flag returning the preview JSON. Tests: preview moving-count == apply merged_count (parity), incompatible flagged without raising, self/missing raise, admin dry_run returns preview + no mutation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
e206778a5c |
feat(allowlist): coverage projection + applied-count + post-accept projection (#7a/#7b)
Cluster B, milestone #99. Backend for the allowlist tuning dashboard. #7a: AllowlistService.coverage(tag_id, threshold) counts distinct images with a prediction resolving to the tag (raw_name==tag.name OR (raw_name,category) in the tag's aliases) scoring >= threshold — the gross candidate pool, mirroring tasks.ml._confidence_for_tag resolution. list_all now carries applied_count (grouped image_tag count) + coverage_count (at the row's threshold). New GET /api/tags/<id>/allowlist/coverage?threshold= for the live what-if number. #7b: /suggestions/accept + /alias return {allowlisted, tag_id, tag_name, projected_count} (projection at the tag's threshold) instead of 204, so the UI can show a non-blocking 'auto-applying to ~N images' toast. Apply still runs async via apply_allowlist_tags — projected_count is an estimate. Tests: coverage by threshold (direct + alias-with-category), list applied vs coverage, coverage route (explicit/default/bad threshold), accept/alias payload (newly-allowlisted vs already-on-list). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
73dd301dbb |
feat(gallery): OR/exclude tag filtering — light chips + advanced builder (#6b/c/d)
Cluster A, milestone #97. Completes the frontend of the structured tag filter
(backend landed in
|
||
|
bvandeusen
|
23fab983a0 |
feat(gallery): tag→gallery nav from modal chips (#5) + OR/exclude tag scope (#6a)
Cluster A, milestone #97. #5: clicking an image-modal tag chip's body now closes the modal and opens the gallery filtered for that one tag (fresh filter); ✕/kebab stay as the explicit remove/rename controls. #6a (backend of OR/exclude filtering): gallery_service._apply_scope gains a structured tag model — tag_or_groups (AND-of-OR: one EXISTS(tag_id IN group) per group) + tag_exclude (NOT EXISTS(tag_id IN exclude)) — layered additively on the existing tag_ids AND path so cursors/facets/deep-links are untouched. Threaded through scroll/timeline/jump_cursor/facets/similar + facets common dict; _require_single_filter rejects post_id combined with OR/exclude. API parses tag_or (repeatable → one OR-group each) + tag_not (csv exclude). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
7c94d99b9f |
refactor(settings): canonical usePreviewCommit for maintenance preview→commit tiles (#753)
Frontend pattern-consistency sweep (note #1026, the last DRY-thread item). TagMaintenanceCard (4 flows) + PostMaintenanceCard (2 flows) each hand-rolled the same sync preview→commit state machine: a previewData/previewing/committing triple + onPreview/onCommit that dry-run-previews, then applies and collapses the projection (the apply shares the backend predicate, so afterward it's empty). Extract usePreviewCommit({preview, commit, emptyPreview}) owning that lifecycle. The 6 flows become declarative: supply the two thunks + the collapse shape. The normalize flow (commit dispatches a self-resuming background task, not a sync apply) omits emptyPreview so the projection stays and a truthy result = queued. Composable returns are aliased to the cards' existing local names, so the templates only change where they read the apply result (the success badges). Long-Celery-task cards (GatedPurge/VideoDedup) keep useMaintenanceTask — a different pattern (navigable-away task lifecycle), deliberately not merged. Exhaustiveness: no card hand-rolls the refs anymore; the only dryRun:false callers are these two cards, both via the composable. Added a vitest spec for the primitive (collapse static + fn, dispatch-variant, re-preview clears result). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
26589c3d98 |
refactor(admin-store): _guard + _dryRunPost consolidate the store actions (#753 Finding C)
DRY pass follow-up (note #1026). All 13 admin-store actions repeated the same lastError-capture/rethrow wrapper; the 6 Tier-A maintenance actions additionally repeated the dry_run POST shape. - _guard(fn): one copy of the lastError=null / try / catch(set lastError; rethrow) wrapper, used by all 13 actions. - _dryRunPost(url, {dryRun, ...extra}): the dry_run POST shape on top of _guard, used by the 6 maintenance actions. reconcile maps sourceId -> source_id. Public exports + every action signature unchanged (object-opts for Tier-A, positional for cascade/bulk/tag ops), so no card/view changes. Behavior identical. Added frontend spec (the admin store had none): _dryRunPost endpoint+body+default, sourceId->source_id mapping, and _guard capturing lastError + clearing on success. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6599a07468 |
refactor(admin): consolidate maintenance-trigger 202 responses onto _queued() (#753 Finding B)
DRY pass follow-up (note #1026). Five handlers returned the identical jsonify({task_id, status:queued}), 202 shape; extract _queued(async_result). Consumers routed through it: tags_normalize (live branch), trigger_reextract_archives, trigger_prune_missing_files, trigger_dedup_videos, trigger_purge_gated_previews. trigger_vacuum stays bespoke (returns no task_id — the UI doesn't poll it). Added route-level tests for all five consumers (these trigger endpoints had no route coverage before): 202 + task_id via _queued, and the dry_run flag threading through to dedup/purge-gated. Behavior unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6281cb1e66 |
refactor(admin): consolidate Tier-A dry-run/apply handlers onto one helper (#753)
DRY pass on the cleanup/admin destructive-ops surface (task #753, hardened process #594). Five Tier-A endpoints repeated the same get_json -> dry_run -> run_sync(service_fn) -> jsonify block verbatim. Extract _run_dry_run_op(service_fn, **kwargs); the five route handlers now delegate. reconcile keeps its source_id validation and passes it through **kwargs. The cleanup_service predicates were already shared between preview and apply (find_*_conditions / find_duplicate_post_groups) — the post-data-loss fix — so no backend-logic change; this is purely the HTTP-handler boilerplate. Consumers (all routed through the helper, verified no copy left behind): prune_unused_tags, prune_bare_posts, reconcile_duplicate_posts (+source_id), purge_legacy_tags, reset_content_tagging. Added route-level tests for prune-bare (apply) and reconcile (apply + source_id passthrough + invalid-source_id 400) — the two helper consumers that previously had only service-level coverage, so every consumer is exercised at the route. Findings B (queued-response helper) and C (store dry-run POST helper) identified but not applied this pass (operator scoped to A). The card preview->commit state machine is deferred to a frontend pattern-consistency sweep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
77d02f57ae |
fix(reconcile): preserve from_attachment_id when merging duplicate posts (#73/#87)
Milestone #73 (reconcile duplicate gallery-dl/native post rows) shipped in eff6427; closing it out after today's #87 work, which added a seam it didn't account for. _repoint_post_links drops a loser post's ImageProvenance row on the (image, post) uniqueness collision — and that row may now carry from_attachment_id (which archive the file was extracted from). For the exact gallery-dl->native case this targets, the keeper is the native stub (no archive) and the loser is the gallery-dl row that extracted the member, so a blind delete silently lost the containing-archive linkage. Carry from_attachment_id onto the keeper's surviving row (when NULL) before dropping the collision. The rarer PostAttachment-collision case (both dup posts captured the same archive blob) doesn't arise in the targeted scenario — the archive lives only on the gallery-dl post, so it re-points straight to the keeper and the FK stays valid. Test: collision merge preserves the loser's from_attachment_id on the keeper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
af7f0078bc |
fix(importer): guard _stamp_member_archive against a sidecar-less archive
A filesystem-imported archive with no adjacent sidecar has no Post, so _post_for_sidecar returns None — and the milestone-#87 stamp call dereferenced post.id. _stamp_member_archive already no-ops on a None post_id (no post → no provenance to stamp); pass None instead of crashing. Caught by the existing test_reimport_archive_is_idempotent (no-sidecar zip). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5269cd0709 |
feat(provenance): capture which archive an extracted image came from (#87)
Images pulled out of a .zip/.rar previously kept no record of WHICH archive
they came from — the member->archive link was computed during extraction and
discarded, leaving only image->post. So the provenance modal could only scope
attachments to the whole post, showing every archive a 'High Resolution Files'
bundle carried instead of the one a given file lives in.
- ImageProvenance.from_attachment_id: nullable FK -> post_attachment.id
(SET NULL), migration 0055.
- importer: _import_archive stamps from_attachment_id on every member's
provenance row for the post (new + superseded + deduped members), resolving
the archive's own PostAttachment by (post, sha). Post-pass UPDATE, NULL-only
and idempotent, so it doesn't touch the dedup/supersede branches and the
backfill is safe to re-run. Nested members link to the outer stored archive.
- provenance_service.for_image: when the originating post's provenance row
records from_attachment_id, return ONLY that archive; else fall back to the
primary-post scoping from
|
||
|
bvandeusen
|
068def2f24 |
fix(provenance): scope attachments to originating post + scroll-cap the list
The Attachments section aggregated PostAttachment rows across EVERY post an image was pHash-linked to. When one of those was a 'High Resolution Files' mega-bundle (dozens of unrelated archives), the list ballooned past the viewport and overwhelmed the modal's right rail. - for_image() now scopes attachments to ImageRecord.primary_post_id (the post the file was actually captured from), falling back to all linked posts only when primary_post_id is unset (older rows / filesystem imports). - ProvenancePanel wraps the list in a max-height scroll container with a count in the heading, mirroring the cards' independent-scroll treatment. Note: FC stores archives as opaque blobs and never records which archive an extracted image came from, so attachments can't yet be scoped tighter than the post. Capturing image->archive containment is tracked as separate work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
311fe0ee9c |
feat(settings): tidy Maintenance tab into compact tiles + center the views (pass 2)
Goal (operator 2026-06-18): the overview of a Settings tab fits one unscrolled viewport; expanding a tile to read into it is the only reason to scroll. - Every Maintenance card converted to the collapsible MaintenanceTile (collapsed by default = icon + short title + one-line blurb). Task cards (ML backfill, centroids, thumbnails, archive re-extract, missing-file repair, DB maintenance) sit in a responsive grid; running tasks auto-expand. Tagging config (suggestion thresholds, allowlist, aliases) grouped in one Tagging section as collapsible tiles; Backup is its own collapsible tile. - Three labeled sections mirror the Cleanup tab: Backfills and reprocessing / Tagging / Storage. - Center the whole Settings surface: SettingsView is now a centered, width-capped (1140px) column so the tab strip and every panel sit in a tidy centered measure (was full-width). CleanupView drops its own left-aligned max-width to fill it. All card logic unchanged - only the chrome. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
3b435dc0ba |
feat(settings): tidy Cleanup tab into sectioned compact tiles (pass 1)
The Cleanup + Maintenance sections had ~17 full-width stacked cards with long descriptions — a hunt to scan. Operator wants compact, sectioned, scannable tiles (2026-06-18: keep both tabs, group inside, compact tiles in a grid). New common/MaintenanceTile.vue: a compact expandable tile (icon + short title + one-line blurb collapsed; click the header to expand the full controls/preview/ result inline; keyboard-accessible button + focus ring; tints the icon, keeps a running task expanded). Cleanup tab (this pass) restructured into 3 sections — Import-filter audits (Min dimensions, Transparency, Single-color) / Duplicates & posts (Bare posts, Duplicate posts, Deduplicate videos, Gated-post previews) / Tags (Unused, Legacy, Reset content tagging, Standardize casing) — each a responsive grid of tiles. PostMaintenanceCard split into 2 tiles, TagMaintenanceCard into 4. Moved VideoDedupCard + GatedPurgeCard from the Maintenance tab here (both are destructive content cleanup). All card logic unchanged — only the chrome. Maintenance tab tiling is pass 2 (TODO noted in MaintenancePanel). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
eff64275fc |
feat(maintenance): reconcile duplicate posts (gallery-dl→native unify)
An artist first downloaded by gallery-dl gets Post rows keyed by the per-
attachment id; a later native walk keys the SAME real post by the post id. They
never dedup (uq_post_source_external_id is on external_post_id) → duplicate post
rows (cheunart: 943→1109). The real post id is recoverable in-DB from
raw_metadata['post_id'] (both eras store the sidecar there).
reconcile_duplicate_posts (cleanup_service): group posts by (source_id, canonical
post_id = raw_metadata.post_id else external_post_id); for each group >1, keep the
row already keyed by the post id (the format the CURRENT native downloader
produces, so future walks dedup and this can't recur), re-point
ImageRecord.primary_post_id / ImageProvenance / PostAttachment / ExternalLink onto
it conflict-safe (drop the loser's row where the keeper already has the equivalent,
per each table's uniqueness), backfill the keeper's empty date/title/body/raw_meta
from a loser, set external_post_id=post_id + derive post_url, delete losers.
IMAGES ARE NOT TOUCHED (content-addressed/deduped already; operator-confirmed).
Preview/apply share find_duplicate_post_groups (rule 93). API
/api/admin/posts/reconcile-duplicates (dry_run→{groups,posts_to_merge,sample};
apply→{groups,merged,sample}; optional source_id). UI: a second section on
PostMaintenanceCard (preview groups+sample → confirm merge). Tests: merge +
metadata backfill + image move, no-op when unique, provenance-collision dedup.
Design: milestone #73. Forensics: note #917. Out of scope (flagged): cheunart vs
Cheunart case-variant artist dirs/rows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
bvandeusen
|
7f6345dccf |
fix(subscribestar): port gallery-dl date extraction (wrapped dates) + parse canary
Image posts wrap the post date in an <a> permalink (<div class="post-date"><a href="/posts/ID">DATE</a></div>); text-only posts don't. Our hand-written <div class="post-date">([^<]+)</div> regex matched ONLY the unwrapped case, so every image post got a null published_at and sorted to the top of the feed looking broken (cheunart 2026-06-17). Port gallery-dl's _data_from_post method: text up to the first </, then after the last > — handles both. Verified against the live raw feed (all 6 dates now parse). Robust logging (operator request): _parse_posts now logs per-page parse stats (posts / dated / with-body) and a WARNING canary when posts parse but NONE get a date or body while the raw markers are present — i.e. our extraction diverged from the live markup. Makes this failure class diagnosable from the worker log alone, no authed re-fetch needed. Test: a permalink-wrapped date parses to ISO. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
479b7b54da |
style(subscribestar): drop quoted forward-ref annotation (UP037)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
976f581aa2 |
feat(subscribestar): port gallery-dl doc + audio attachment extraction
Some SubscribeStar posts deliver content only through document/audio attachments, which live OUTSIDE data-gallery. Port gallery-dl's _media_from_post for them: - docs: scope uploads-docs..post-edit_form, split on doc_preview blocks, take the href URL + doc_preview-title + data-upload-id (kind=attachment). - audio: scope uploads-audios..post-edit_form, split on audio_preview-data blocks, take the src URL + audio_preview-title + data-upload-id (kind=audio). The existing downloader handles them unchanged (plain streaming GET; the file validator only inspects image/video extensions via is_validatable, so PDFs/zips/ audio pass straight through, no quarantine). Test covers doc + audio extraction (the cheunart sample has none, so this pins gallery-dl's documented markup shape). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
8771364cee |
fix(subscribestar): port gallery-dl's content + preview-skip extraction faithfully
Body rendered as a bogus '264 / 265' on every post: our balanced-</div> body regex either returned empty or over-captured into sibling upload divs and the 'View next posts (N / M)' pagination counter. Replace it with gallery-dl's exact _data_from_post rule — content between the post_content-text wrapper and the youtube-uploads div (literal markers), then strip the trix editor's <html><body>…</body></html> document wrapper to its inner. Verified against the live cheunart sample: clean per-post bodies, empty for genuinely text-less posts. Also port gallery-dl's _media_from_post preview guard: skip gallery items whose URL is under /previews (locked/blurred teasers) — the SubscribeStar analog of the Patreon gated-preview bug (#874); this is why a locked post yields no media. Tests: body must not bleed into the pagination counter; trix html-document wrapper stripped; /previews items skipped. Fixture now includes the youtube- uploads close marker present in real markup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
559d29fe1c |
fix(subscribestar): mirror gallery-dl's full request profile (verify_subscriber gate)
After the delimiter fix, the live cheunart fetch 302'd to /cheunart/verify_ subscriber even with valid .adult cookies (confirmed present: _personalization_id + _subscribestar_session on .subscribestar.adult, logged in). Walking gallery-dl's ENTIRE flow — including the base Extractor._init_session I'd not read — the divergence is the HTTP request profile, not the cookies or parser. gallery-dl's default (cookies-only) mode sends, on EVERY request including the first creator-page GET: a Firefox UA, Accept: */*, Accept-Language, and a same- site Referer (root/), with NO X-Requested-With anywhere (the load-more endpoint is a plain GET parsed as JSON). Our Chrome UA + missing Referer + XHR toggling looked unlike a browser → SubscribeStar gated the adult-creator page. Make our SubscribeStar session identical: Firefox UA + Accept */* + Accept- Language via make_session extra_headers; stamp Referer=<base>/ per walk; drop the per-request XHR headers (both feed and load-more now use the shared profile). Test updated to assert the gallery-dl-parity profile instead of the old navigation-vs-XHR split. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
204d341a99 |
fix(subscribestar): match gallery-dl's generic post delimiter (live-feed drift)
The native client split the feed on `<div class="post is-shown`, but `is-shown` is added by SubscribeStar's infinite-scroll JS when a post scrolls into view — present in a browser-SAVED page (what the Step-0 characterization used) but ABSENT from the raw server HTML we and gallery-dl actually fetch. So the live feed (cheunart) parsed to zero posts and raised a false SubscribeStarDriftError. Align with gallery-dl's proven `_pagination`: split on the generic `<div class="post ` (trailing space rules out the hyphenated post-content/ post-date/post-body siblings). Also mirror gallery-dl's redirect-based gating detection (/verify_subscriber, /age_confirmation_warning => auth, not drift). Regression tests: raw server markup without is-shown now parses; an age-wall redirect raises SubscribeStarAuthError. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
9201b7b539 |
diag(subscribestar): name the interstitial in the drift error (title + type)
The XHR fix worked (we now get a real 93KB HTML page, not JSON) but cheunart still drifts — we're being served a full HTML page that isn't the feed. Add _describe_page(): the drift error now reports the page <title> + which known interstitial it resembles (cloudflare/bot-challenge, age-gate, login, captcha), so the next run names the actual cause instead of "markup changed". Strong suspicion: a Cloudflare challenge (python-requests has no JS; cf_clearance is UA-locked and our hardcoded UA likely differs from the cookie-capturing browser). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |