Commit Graph

832 Commits

Author SHA1 Message Date
bvandeusen e75427b19a Merge pull request '#768 steps 1+2: normalized image_prediction table (read cutover)' (#92) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-web (push) Successful in 2m4s
Build images / build-ml (push) Successful in 2m42s
CI / integration (push) Successful in 3m8s
2026-06-10 20:15:26 -04:00
bvandeusen 75eab188c8 fix(migration): 0045 backfill filters to >= store floor (supersedes #764 prune)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m12s
The #764 in-place prune (rewrite tagger_predictions to >=0.70) is too slow on
100 GB of TOAST and fails at its soft limit (interrupts a query mid-flight ->
'another command is already in progress'). #768 supersedes it: extract only
the >=floor predictions into image_prediction via this set-based backfill,
then drop the column (step 3) — reading 100 GB once + writing ~840k small rows
beats rewriting 100 GB in place.

So this backfill no longer assumes the prune ran: it filters by
ml_settings.tagger_store_floor (default 0.70) itself, handling the full or
partially-pruned JSON identically.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 19:37:38 -04:00
bvandeusen 0319812b45 style: group tests._prediction_helpers import with backend (ruff I001)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m8s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 16:10:06 -04:00
bvandeusen 22cdf0f334 feat(ml): read suggestions + allowlist from image_prediction (#768 step 2)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m13s
Switch every prediction READER off the JSON column onto the normalized
image_prediction table. Parity by construction: each reader loads the same
{raw_name: {category, confidence}} dict it consumed before (via small
_load_predictions helpers), so all downstream threshold/alias/merge/consensus
logic is byte-identical — only the data source changed.

- suggestions.SuggestionService.for_image (and for_selection via it)
- ml.apply_allowlist_tags (iterates images that have prediction rows)
- importer re-import reset deletes the image's prediction rows
The tagger_predictions JSON column is still dual-written (step 1) so it stays
valid during transition; the backfill task's NULL check still works. Removing
the JSON write + DROP column + retiring the #764 prune is the cleanup
follow-up (needs a quiesced-worker window for the DROP lock).

Tests: shared tests/_prediction_helpers.seed_predictions seeds the table;
read-path tests (suggestions, bulk consensus, allowlist apply, API) seed there
instead of ImageRecord.tagger_predictions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 16:03:58 -04:00
bvandeusen 79089b50b0 feat(ml): image_prediction table + backfill + dual-write (#768 step 1)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m22s
Normalize tagger predictions out of the image_record.tagger_predictions JSON
blob into a queryable per-prediction table. Step 1 of the cutover (expand):
additive + low-risk — reads still use the JSON, this just adds the table and
keeps it populated.

- ImagePrediction(image_record_id, raw_name, category, score) — stores the
  RAW tagger vocab name (not tag_id) so read-time alias→canonical resolution
  is unchanged. Indexed for per-image reads + by (raw_name, score).
- Migration 0045: create table + set-based backfill from the JSON via
  json_each (fast post-#764-prune). The old column stays (vestigial) and is
  dropped in a later follow-up — DROP needs an ACCESS EXCLUSIVE lock on the
  hot image_record table, so it waits for a quiesced-worker window.
- tag_and_embed dual-writes the rows (delete-then-insert, idempotent);
  tagger_store_floor already applied in infer().

Next: switch suggestion + allowlist reads to the table, then drop the JSON
write. Plan-task #768.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 15:55:32 -04:00
bvandeusen 5447fab987 Merge pull request 'Activity search + RecoverySweep fix + tagger_predictions shrink (#762, #764) + backup polish' (#91) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 38s
Build images / build-web (push) Successful in 2m23s
Build images / build-ml (push) Successful in 2m51s
CI / integration (push) Successful in 3m10s
2026-06-10 14:33:35 -04:00
bvandeusen 7a40a50fe9 fix(backup): compressed -Fc dumps + pg_restore; reconcile subprocess timeouts (#739)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m14s
DB backup polish (plan-task #764 Q3):
- pg_dump now uses custom format (-Fc): compressed (much smaller on NFS) and
  restored via pg_restore. Artifact extension .sql → .dump; restore_db swaps
  psql -f for pg_restore -d. BackupRun.sql_path field name kept (it's just the
  db artifact path).
- Reconcile the subprocess guardrails: the DB timeout was 720s with a stale
  'Celery soft is 10 min' comment, but backup_db_task's soft limit is actually
  1800s — so the bounded-kill fired 18 min early. Set DB=1700s / images=21000s,
  each just under its task's Celery soft limit so _run_bounded stays the
  primary guard (an NFS D-state hang defeats Celery's own SIGKILL).

Real shrink of the DB is the #764 prune; this makes each dump smaller/faster
on top of that.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 14:29:17 -04:00
bvandeusen d55e52ae9b feat(admin): prune_low_confidence_predictions backfill task + UI (#764)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m13s
The one-time backfill that actually shrinks the DB: drops stored
tagger_predictions entries below ml_settings.tagger_store_floor from every
image_record row, and clamps any allowlist min_confidence below the floor up
to it. Keep predicate (confidence >= floor) mirrors Tagger.infer's store gate
so backfilled rows match new imports. Keyset by id ASC, idempotent,
self-resumes on the soft time limit; runs on the maintenance_long lane.

pg_dump copies live data only, so this alone fixes the #739 backup timeout —
the reclaim (VACUUM FULL / pg_repack on image_record) is a separate, optional
disk-return step, brief because post-prune the live data is tiny.

- admin.prune_low_confidence_predictions_task + POST /api/admin/maintenance/prune-predictions
- PrunePredictionsCard in the Maintenance panel (shows the current floor)
- tests: registration + prune-keeps->=floor/drops-<floor + allowlist clamp

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 13:57:39 -04:00
bvandeusen c8b815afe6 feat(ml): clamp allowlist min_confidence to the tagger store floor
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m12s
Consumer #4 of the store-floor change (#764). An allowlist tag can't
auto-apply more permissively than the ingest floor — predictions below
tagger_store_floor aren't stored, so a lower min_confidence behaves
identically to the floor. update_threshold now clamps to max(value, floor);
the AllowlistTable confidence input min-binds to the live floor and clamps
on edit. Keeps the stored threshold honest about actual apply behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 13:52:20 -04:00
bvandeusen 3f92669f12 feat(ml): DB-backed tagger_store_floor (default 0.70), the ingest confidence floor
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m18s
Promotes the prediction store-floor from the TAGGER_STORE_FLOOR env (default
0.05) to a DB-backed, Settings-UI-tunable ml_settings column (default 0.70).
Storing every tag down to 0.05 from a ~10k-tag tagger is what grew
image_record's TOAST to ~100 GB; the suggestion path already filters at 0.70
and the centroid/learned path covers lower-confidence preferred tags, so the
sub-0.70 tail is redundant. Foundation for plan-task #764 (backfill + reclaim
land next; this only changes the write gate for NEW imports).

- ml_settings.tagger_store_floor (migration 0044, default 0.70)
- tagger.Tagger.infer(store_floor=...); ml task passes settings.tagger_store_floor
- ML admin GET/PATCH expose it; PATCH rejects a category suggestion threshold
  below the floor (nothing below the floor is stored, so the gap surfaces
  nothing) — server backstop for the UI slider clamp
- Settings → ML: store-floor slider + caption; category sliders min-bound to it

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 13:50:30 -04:00
bvandeusen 9ba3db75fd fix(maintenance): download queue needs a sweep threshold above its 25-min time_limit
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m9s
recover_stalled_task_runs used the 5-min default for the download queue,
but download_source legitimately walks up to DOWNLOAD_HARD_TIME_LIMIT
(1500s = 25m). Healthy in-flight Patreon/gallery-dl walks were flagged as
phantom 'RecoverySweep' failures — visible in System Activity but absent
from the Subscriptions view (the download finished ok, reset the source's
consecutive_failures; only the orphaned task_run kept the stamp, since
_finalize only updates rows still 'running').

Add download:30 to QUEUE_STUCK_THRESHOLD_MINUTES — clears the 25-min hard
limit with buffer and matches DOWNLOAD_STALL_THRESHOLD_MINUTES so a real
hard kill is swept by the task-run and event sweeps together. Restores the
documented invariant (every override >= task time_limit). Regression test
pins the threshold above the hard limit so a future limit bump can't
silently re-break it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 11:57:31 -04:00
bvandeusen 70d4017cf6 feat(activity): search/filter on both Activity-tab panes
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 39s
CI / integration (push) Successful in 3m17s
Recent failures gains a client-side search over the already-loaded 24h
rows (task/queue/target/error), shown as a filtered/total count alongside
the existing error-type chips. All recent activity gains a debounced
server-side task-name search (new `task` ILIKE param on /runs) so it
spans the full history, not just the loaded page. LIKE wildcards are
escaped so task names' literal underscores match literally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 11:34:42 -04:00
bvandeusen bad37e07b2 Merge pull request 'Browse hub, series rename, full-prediction dropdown + a DRY pass (7 sweeps)' (#90) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 27s
CI / frontend-build (push) Successful in 44s
Build images / build-ml (push) Successful in 3m2s
CI / integration (push) Successful in 3m15s
Build images / build-web (push) Successful in 2m17s
2026-06-10 00:24:01 -04:00
bvandeusen 14c244bd3d refactor(tags): shared tag_query for fandom self-join + serialization (DRY sweep)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m17s
The fandom self-join (resolve a character's fandom NAME via Tag.fandom_id->Tag)
and the {id,name,kind,fandom_id,fandom_name} dict were hand-written in
TagService.autocomplete/.list_for_image, GalleryService.get_image_with_tags and
the api/tags handlers — the last few grown by this session's fandom-on-chip
feature. Consolidate to services/tag_query: fandom_join_alias() + tag_columns()
build the select; serialize_tag(row) builds the dict. Now a new tag field is
added in one place.

Over-DRY guard: TagDirectoryService selects the full Tag ORM + an image-count
aggregate (a different select shape) — left as its own variant. §8b: the
fandom_lookup alias lives only in tag_query; gallery + both api/tags handlers
serialize via serialize_tag. Test: serialize_tag handles enum + string kind.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 00:19:17 -04:00
bvandeusen 074c5868fb refactor(services): shared pagination cursor (DRY sweep)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 43s
CI / integration (push) Successful in 3m21s
encode_cursor/decode_cursor (base64 <iso8601>|<id>) were defined identically in
gallery_service AND post_feed_service, with artist_service importing gallery's
copy. Two implementations of one cursor format silently break pagination in
whichever feed drifts. Extract to services/pagination.py; gallery/post_feed/
artist all import it. Dropped now-unused base64/datetime imports.

§8b: encode_cursor/decode_cursor now defined only in pagination.py. Existing
cursor round-trip tests still cover it via the re-export. Catalog updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 00:10:57 -04:00
bvandeusen f1a664e5a7 fix(services): PEP 695 type params for get_or_create (ruff UP047)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m11s
CI lint flagged UP047 — use the native generic syntax def get_or_create[T](...)
instead of typing.TypeVar on Python 3.14.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:50:59 -04:00
bvandeusen 7b2a2051e9 refactor(services): shared race-safe get_or_create helper (DRY backend sweep)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m19s
The find-or-create dance — SELECT, then a SAVEPOINT INSERT that recovers (not a
full rollback) on IntegrityError when a concurrent worker inserted first — was
hand-rolled identically in 4 async sites: ArtistService.find_or_create,
TagService.find_or_create, ExtensionService._find_or_create_artist and
._find_or_create_source. Divergent copies of exactly this pattern are how the
duplicate-row/race bugs in reference_scalar_one_or_none_duplicates crept in, so
it now lives once in services/db_helpers.get_or_create (returns (row, created);
factory adds+flushes+returns the row; caller owns the outer commit).

Over-DRY guard: SourceService's IntegrityError sites RAISE DuplicateSourceError
(reject-on-conflict, a different concept) — left alone. Importer._get_or_create
is the lone SYNC consumer (already shared by 2 callers) — stays separate, can't
cross the sync/async boundary. §8b: no hand-rolled async find-or-create remains.
Test: get_or_create creates then returns existing without re-invoking the factory.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:46:01 -04:00
bvandeusen 9deebfa133 refactor(ui): CardHeading primitive for icon+title card/dialog headings (DRY pattern sweep)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m10s
The icon+title v-card-title heading (d-flex align-center + gap + <v-icon size=small> +
<span>) was hand-rolled identically in 13 cards/dialogs (15 heading instances).
Consolidate to <CardHeading icon title> (components/common) with an iconColor
prop (error headings) and a default slot for trailing content (spacer+actions,
inline status chip). Adopted everywhere the pattern appears — all-or-nothing per
the hardened DRY process.

Over-DRY guard: plain text-only <v-card-title> one-liners are NOT this pattern
and stay; DownloadDetailModal leads with a status CHIP (not an icon), a different
concept, left alone. §8b: the only remaining d-flex align-center v-card-title is
that intentional variant. Catalog updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:22:01 -04:00
bvandeusen 4854d74c5a refactor(ui): SampleNameGrid primitive for maintenance-card previews (DRY pattern sweep)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m9s
The preview sample-name grid (scrollable monospace chip grid) was hand-rolled
5 times with verbatim-duplicated markup + CSS — TagMaintenanceCard (×4) and
PostMaintenanceCard. Consolidate to <SampleNameGrid> (components/common): pass
:names for the plain case, default slot for the normalize from→to chips
(styled via :slotted .fc-name). Removed the duplicated .fc-name-grid/.fc-name
CSS from both cards.

Over-DRY guard: only the verbatim-duplicated grid is merged — each card's
preview/commit logic and result-count lines genuinely differ and stay put;
MinDimensionCard's typed-token confirm is a separate variant, untouched.
§8b: fc-name-grid now lives only in SampleNameGrid. Catalog updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:08:15 -04:00
bvandeusen 409bbd43db feat(series): rename a series from the management view
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m15s
The management view showed the series name but had no way to change it post-
creation (rename was only on the browse-card kebab). Add a pencil next to the
title that opens TagRenameDialog (reuses the canonical rename → PATCH
/api/tags/<id> with its collision→merge flow, since a series IS a
Tag(kind=series)); the new name reflects in place. Operator-asked 2026-06-09.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:05:01 -04:00
bvandeusen 4e83b4225a refactor(ui): single global .fc-muted token (DRY pattern sweep)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m21s
The muted-text token was redefined identically in 12 component <style scoped>
blocks. Consolidate to one global utility in styles/app.css; remove the 12
copies. Keeps the explicit on-surface-variant (vellum) token, NOT Vuetify's
opacity-based text-medium-emphasis (per the muted-text-token rule). Behavior-
preserving: every class=fc-muted usage now resolves to the single source.

§8b exhaustiveness caught (and I fixed) my own sed clobbering the new app.css
rule — now exactly one .fc-muted definition exists, zero component-local.
Catalog updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 22:42:26 -04:00
bvandeusen c774042a85 refactor(ui): consolidate 7 hand-rolled kebabs into one KebabMenu (DRY pattern sweep)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 40s
CI / integration (push) Successful in 3m18s
First pattern-consistency DRY pass (process #594). The overflow kebab was
hand-rolled 7 ways in two divergent activator strategies — Pattern A
(#activator + v-bind) which silently breaks inside the teleported image modal
(#711), and Pattern B (manual v-model + activator=parent + open-on-click=false +
z-index 2400) the modal kebabs needed as a workaround.

New <KebabMenu> (components/common) bakes in the modal-safe strategy
UNIVERSALLY, so every kebab works in modal and non-modal contexts — folding the
latent #711-class bug fix into all five Pattern-A sites. Menu items go in the
default slot; variations (size/variant/location/label/min-width) are props.

Adopted across all 7: TagChip, SuggestionItem, TagCard, SeriesView card,
SeriesManageView, BackupRunsTable, SourceActions. Exhaustiveness (§8b):
mdi-dots-vertical now lives only in KebabMenu. Labeled dropdowns / nav menus /
filter popovers are a different concept and left alone. Seeded the pattern
catalog so new code reuses the primitive. Test: KebabMenu renders slot items +
trigger label/glyph + presentational props.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 22:25:11 -04:00
bvandeusen d5d23a92f2 feat(nav): consolidate Posts/Artists/Tags into a Browse hub
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m13s
Posts, Artists, and Tags are the three 'browse the library by an axis'
surfaces; Subscriptions stays purely management (operator-asked 2026-06-09).
New BrowseView renders them as tabs (?tab=posts|artists|tags); only the active
tab mounts. The old standalone paths become redirects into the matching tab,
preserving deep-link query (/posts?post_id=N → /browse?tab=posts&post_id=N) and
keeping the route names so existing { name: 'posts'|'artists'|'tags' } links and
path pushes still resolve. Nav now reads Showcase · Gallery · Browse · Series ·
Subscriptions, with Settings pinned right.

Test: /browse resolves; /tags and /artists redirect into their tabs; a posts
deep link survives the redirect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:18:02 -04:00
bvandeusen 2bfc9936a1 Merge pull request 'UI batch: tagging flow, series browse, fandom chips, nav' (#89) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 39s
Build images / build-web (push) Successful in 2m21s
Build images / build-ml (push) Successful in 2m52s
CI / integration (push) Successful in 3m24s
2026-06-09 20:48:19 -04:00
bvandeusen a50902071a feat(nav): pin Settings to the right edge, separated from content nav
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 49s
CI / integration (push) Successful in 3m20s
Settings is configuration, not content, but sat mid-row (between Series and
Posts). Pull it out of the centered content links and pin it to the right as a
gear+label, matching the convention that config lives at the right edge. Mobile
is unchanged — Settings stays in the hamburger menu (navRoutes still includes
it). Operator-asked 2026-06-09.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 20:37:50 -04:00
bvandeusen c999c64cbe feat(suggestions): tag-input dropdown searches the full prediction set
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 40s
CI / integration (push) Successful in 3m18s
The typed dropdown sourced the threshold-filtered panel list (>= 0.70 general),
so low-confidence actions/features the model DID predict never appeared — forcing
hand-typed custom tags instead of accepting the model's canonical formatting.

Add a threshold override: SuggestionService.for_image(threshold_override=) and
GET /images/<id>/suggestions?min=<f> surface EVERY stored prediction (down to the
0.05 store floor), alias-resolved and normalized, still excluding applied/rejected
and unsurfaced categories. The suggestions store gains allByCategory + loadAll
(min=0); the dropdown searches that full set (cap 20), while the Suggestions panel
stays curated at the configured threshold. Accept/dismiss drop from both lists.

Operator-asked 2026-06-09. Test: a 0.30 general prediction is hidden by default
but surfaced with threshold_override=0.0; unsurfaced categories still excluded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 20:22:24 -04:00
bvandeusen 978f49adcc feat(tags): show a character's fandom on its chip (truncated)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 43s
CI / integration (push) Successful in 3m27s
A character chip with a fandom only rendered a bare arrow. Surface the fandom
NAME inline, truncated to 15 chars (full name in the tooltip). Resolve the name
via a Tag self-join in both tag paths the modal uses — list_for_image
(/api/images/<id>/tags) and gallery get_image_with_tags
(/api/gallery/image/<id>) — so chips show the fandom on first open and after any
reload. Falls back to the bare arrow when only fandom_id is known. Operator-asked
2026-06-09.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 19:35:59 -04:00
bvandeusen e4cebf70d1 feat(series): browse search + per-card kebab (rename/delete)
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 26s
CI / frontend-build (push) Successful in 49s
CI / integration (push) Successful in 3m13s
The Series browse tab had no way to find a series in a long grid and no
per-series actions. Add a search field (instant client-side name/artist filter
over the already-loaded list) and a kebab on each card with Rename (reuses
TagRenameDialog → PATCH /api/tags/<id>, with its collision-merge flow) and
Delete (confirm dialog → DELETE /api/admin/tags/<id>; series_page/chapter/
suggestion cascade, images kept). Gap badge moved to the cover's top-left so the
kebab can sit top-right. Operator-asked 2026-06-09.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 18:56:45 -04:00
bvandeusen 4958e8f7d4 feat(modal): return focus to tag input after accepting a suggestion
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 39s
CI / integration (push) Successful in 3m19s
Accepting an auto-suggested tag (Suggestions panel or the autocomplete
dropdown) left focus on <body>, so the operator had to re-click the tag field
to add the next one. Expose TagAutocomplete.focus (the existing mobile-aware
focusInput) and call it after accept from both paths; SuggestionsPanel emits
'accepted' for the parent to refocus. Operator-asked 2026-06-08.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 21:59:08 -04:00
bvandeusen 4c6406ee18 Merge pull request 'fix(posts): link duplicate items to every post + prune bare shells' (#88) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-web (push) Successful in 2m19s
Build images / build-ml (push) Successful in 2m41s
CI / integration (push) Successful in 3m5s
2026-06-08 19:42:31 -04:00
bvandeusen a8f624a0f1 fix(posts): link duplicate items to every post + prune bare shells
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m20s
The native Patreon backfill flooded the feed with bare 'Post <id>' shells
(1589 for Anduo). Root cause: PostAttachment.sha256 was GLOBALLY unique, so a
non-art file reused across posts only ever linked to the first one, and
_capture_attachment created the Post before that dedup check — leaving later
posts with no image and no attachment. Duplicate IMAGES had the mirror gap:
attach_in_place returned duplicate_hash/duplicate_phash before _apply_sidecar,
so the second post got no provenance row, and the feed only rendered via
primary_post_id (one post per image).

Operator requirement: a duplicate item must show on EVERY post it appears in.
Unify the fix as link-not-suppress:

- importer: on duplicate_hash / duplicate_phash(larger_exists), append an
  image_provenance row for the new post (keep primary on the first). Both the
  download path (attach_in_place) and the filesystem path (_import_media).
- post_feed_service: render thumbnails by image_provenance UNION primary_post_id,
  so a cross-posted image shows on every post (and legacy primary-only images
  still show).
- PostAttachment: per-post uniqueness — drop UNIQUE(sha256), add partial
  UNIQUE(post_id, sha256) + partial UNIQUE(sha256) WHERE post_id IS NULL
  (migration 0043); _capture_attachment dedups per-(post,sha) over the shared
  sha-addressed blob, so no post is left bare.
- cleanup: new prune-bare-posts maintenance action (cleanup_service
  _bare_post_conditions shared by preview/count/delete per preview/apply parity;
  admin endpoint; PostMaintenanceCard). Deletes posts with zero image links
  (primary or provenance) AND zero attachments. Run after the feed fix so a
  hidden provenance link spares the post instead of deleting it.

Tests: dup image shows on both posts; dup attachment shows on both posts; feed
renders provenance-linked duplicates; prune-bare delete-path == preview.

Operator redeploys (migration 0043) then runs the prune to clear the shells.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:28:33 -04:00
bvandeusen bb47e80b3e Merge pull request 'fix(cleanup): unused-tags delete must use the same predicate as the preview' (#87) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m13s
2026-06-08 18:17:49 -04:00
bvandeusen df76bc0f58 test(cleanup): fix prune-spares-fandom fixture — used character keeps fandom alive
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m8s
The character pointing at the fandom had no image associations, so it was
itself unused and inflated the dry-run count to 2. Tag it on a real image so
it is used (the real-world shape) — the fandom survives via a live character.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:09:27 -04:00
bvandeusen de4ef6ae74 fix(cleanup): live prune uses the same predicate as the preview (data loss)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Failing after 3m10s
The fandom/chapter exclusions added in fb05c5e only touched find_unused_tags
(the preview SAMPLE). prune_unused_tags re-implemented the predicate inline for
the dry-run COUNT and the live DELETE with only the image_tag + series_page
checks — so the preview showed a safe list of names while the delete removed
every fandom (and chaptered series). Operator-flagged 2026-06-08: real data loss
— assigned fandoms deleted, their characters SET-NULLed.

Extract _unused_tag_conditions() as the single source of truth and use it for
the preview, the count, AND the delete, so they can never diverge again. Added a
prune-commit test asserting the LIVE delete spares a character's fandom and a
chaptered series.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 18:04:02 -04:00
bvandeusen dc1083b5e0 Merge pull request 'Unused-tag fandom fix + ML-worker logging/tuning + unified dropdown Enter' (#86) from dev into main
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m16s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 2m21s
2026-06-08 17:27:55 -04:00
bvandeusen 408fcd488a refactor(ui): unify confirm-dropdown Enter behavior via useAcceptOnEnter
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 27s
CI / frontend-build (push) Successful in 1m44s
CI / integration (push) Successful in 3m8s
Operator-flagged (again) on the tag-merge picker: Enter on the dropdown re-opens
it instead of accepting the selection. I'd already patched this twice (fandom
picker + fandom set dialog) with copy-pasted capture-phase handlers, so DRY it.

New composable useAcceptOnEnter(accept): tracks the menu state and, on a
capture-phase Enter, lets Vuetify pick when the menu is open but calls accept()
(and blocks the re-open) when it's closed. Applied to every confirm-style picker:
- TagsView merge-into picker (the reported one)
- AliasPickerDialog
- PostSeriesMenu add-to-existing
- FandomPicker + FandomSetDialog (refactored off their bespoke handlers)

One behavior, one place to change it.
2026-06-08 08:59:55 -04:00
bvandeusen f2fbe2ae6e tweak(ml): default video frame samples 10 to 6
CI / backend-lint-and-test (push) Failing after 3s
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 23s
CI / integration (push) Successful in 3m6s
Operator: 10-frame max-pooled tagging on video produces a lot of noisy tags, and
the sampling burns time/GPU. Drop the VIDEO_ML_FRAMES default to 6 (still env-
overridable). Fewer frames = less per-frame noise into the max-pool and a smaller
frame-sampling budget. Quality/perf of the whole video path is being reviewed
separately.
2026-06-08 08:52:39 -04:00
bvandeusen b1778ca9f2 obs(ml): tag_and_embed logs file + phase + timing; failures name them
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m6s
The task logged nothing and SoftTimeLimitExceeded stringifies to empty, so a
timeout surfaced as a bare 'SoftTimeLimitExceeded()' with no clue which file or
why (operator-flagged 2026-06-08).

- Log start (id/path/mime/bytes/video?), per-phase timing (load_models, video
  probe/sample/infer, tag, embed, persist), and a success summary.
- Track a  + file ; on SoftTimeLimitExceeded log it and re-raise
  SoftTimeLimitExceeded WITH that context (keeps the 'timeout' task_run status
  but gives the activity a real error_message: which file, which phase, elapsed).
- On other exceptions, log context then re-raise the ORIGINAL (preserves
  autoretry for OSError/DBAPIError/OperationalError).

Now a stuck run names the culprit — most likely a slow video (frame sampling is
up to 10x60s ffmpeg) or a huge image; the phase log will say which.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 08:49:37 -04:00
bvandeusen fe0ed52595 test: drop unused binding in find_unused_tags test (ruff F841)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m6s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 08:45:36 -04:00
bvandeusen fb05c5eef7 fix(cleanup): don't flag a character's fandom (or a chaptered series) as unused
CI / lint (push) Failing after 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m7s
find_unused_tags only excluded tags with image_tag or series_page references, so
it flagged every fandom as 'unused' — fandoms are NEVER applied to images (a
character carries its fandom via tag.fandom_id), and the FK is ondelete=SET NULL,
so deleting one silently strips the fandom off all its characters
(operator-flagged 2026-06-08: artist-OC fandoms showing as unused).

Exclude tags referenced as a character's fandom_id, and (same class of gap) tags
referenced by a series_chapter (an all-placeholder series has chapters but no
pages yet). A genuinely orphaned fandom with no characters is still swept.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 08:41:27 -04:00
bvandeusen e46893fefd Merge pull request 'Migration lock safety + remove the merge's full-table scan (the real 0040-hang fix)' (#85) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 25s
Build images / build-web (push) Successful in 2m3s
Build images / build-ml (push) Successful in 2m33s
CI / integration (push) Successful in 3m7s
2026-06-08 01:03:06 -04:00
bvandeusen e90e6b2c34 perf(tags): protective-alias uses tag kind, drops the image_record full scan
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m7s
_create_protective_aliases scanned every image_record's tagger_predictions JSON
(unindexed full scan, ~59k rows) to find the categories a merged-away tag's name
was predicted under. That scan ran inside the merge transaction AFTER it had
locked series_page — on a large library it held that lock for minutes and is what
blocked migration 0040 (and starved the standardization task into its 40-min
timeout).

The scan was redundant: the tagger's tag_to_category map is one-to-one (a name has
exactly one category) and a tag's kind is set from that category when created, so
kind already IS the tagger's category for the name. The scan only ever rediscovered
the kind. Build the single protective alias from src_kind directly — no scan, no
lock-holding slow step in the merge.

Rewrote test_alias_per_observed_prediction_category (which encoded the
can't-actually-happen one-name-two-categories case) → test_protective_alias_uses_tag_kind.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 00:15:40 -04:00
bvandeusen 8e98e79968 fix(alembic): lock_timeout on migrations, drop the advisory lock
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m13s
Reverses the advisory-lock approach (7309d1d) — it treated a replica race that
wasn't the cause and added a new indefinite-hang mode (a sibling/stale migrator
holding the xact lock).

Real cause of the 0040 hang (operator-diagnosed 2026-06-07): web has always been
a single replica. The migration's ALTER series_page queued behind a concurrent
tag-merge that held a series_page lock for minutes — _do_merge repoints
series_page then runs _create_protective_aliases, an unindexed full scan of
image_record (JSON column, ~59k rows). Migrations ran with no lock_timeout, so
the DDL hung indefinitely and silently.

Fix: SET lock_timeout (default 30s, env-overridable) on the migration connection
before alembic's transaction. A blocked DDL now fails fast with 'canceling
statement due to lock timeout'; the entrypoint exits non-zero so the deploy
retries / surfaces loudly instead of wedging. General protection for every
future migration. (The slow _create_protective_aliases scan — the actual lock
holder — is the separate perf fix still under discussion.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 23:58:14 -04:00
bvandeusen 0666e15211 Merge pull request 'Series manage redesign (FC-6.4) + migration/normalize hardening + UX fixes' (#84) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 25s
Build images / build-web (push) Successful in 1m58s
Build images / build-ml (push) Successful in 2m28s
CI / integration (push) Successful in 3m6s
2026-06-07 21:17:57 -04:00
bvandeusen a00a2786e3 fix(tags): normalize task fails fast on lock + logs progress
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m5s
normalize_tags_task ran to the 40-min hard limit with zero logs (operator-
flagged 2026-06-07). Cause: a per-group merge repoints series_page (via
_repoint_series_pages); during the wedged 0040 migration that held ACCESS
EXCLUSIVE on series_page, the merge's UPDATE blocked on that lock. The time-box
check is at the top of the group loop, so a statement blocked mid-group never
yields back to it — the task sat until the Celery hard kill. No logs because the
only log fired per *finished* group.

- Set lock_timeout=30s on the normalize session (opt-in server_settings on the
  async factory). A blocked merge now raises, the per-group handler rolls back +
  counts an error, and the loop continues — one stuck group can't strand the
  chunk, and the budget checkpoint stays effective.
- Log group count at start + a heartbeat every 25 groups, so a long/slow run is
  diagnosable instead of silent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 21:02:39 -04:00
bvandeusen 9770dd3474 fix(tags): rename-onto-existing in the image modal now merges, not errors
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 26s
CI / frontend-build (push) Successful in 32s
CI / integration (push) Successful in 3m7s
The image-modal tag kebab's rename dialog still showed a leftover stub
('Merging two tags into one lands in FC-2c') on a name collision, dead-ending
the operator. The merge machinery has existed for a while — the Tags view
already resolves rename collisions this way. Wire TagRenameDialog to it: on the
409 collision hint, show the same merge confirmation FandomSetDialog uses
(target name, image associations moved, alias kept) and POST /api/tags/<id>/merge
into the existing tag.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:56:45 -04:00
bvandeusen 7daf90f41e fix(allowlist): lower default auto-apply threshold 0.95 → 0.90
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m7s
Operator evidence 2026-06-07: 0.95 was too strict, skipping confident-enough
auto-applications of accepted tags. Newly-accepted tags now allowlist at 0.90;
existing entries keep their stored value and per-tag thresholds stay tunable in
the allowlist table. No migration — min_confidence has no DB server_default, so
the Python insert default governs new rows only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:48:43 -04:00
bvandeusen aaa375654b fix(fandom): match change-fandom modal focus + Enter to FandomPicker
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m6s
The 'Fandom for <character>' dialog (FandomSetDialog) used plain autofocus and
had no Enter handling, so Enter re-opened the dropdown instead of submitting —
the same bug FandomPicker already fixed. Mirror that flow: parent v-dialogs
focus the field via @after-enter→focusSearch (reliable past the focus-trap);
capture-phase Enter Saves the changed selection instead of re-opening the menu;
Tab jumps to the new-fandom field; creating a fandom returns focus to the
dropdown so a single Enter saves it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:37:02 -04:00
bvandeusen 5bc8ef65ad chore: gitignore the .superpowers working dir
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 27s
CI / frontend-build (push) Successful in 40s
CI / integration (push) Successful in 3m5s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:29:10 -04:00
bvandeusen 978959bdc4 feat(series): manage-view redesign — big pages, editable Part #, slide-over picker (FC-6.4)
Operator feedback: thumbnails too small to judge order, no obvious way to mark
'this installment is Part 2', and the permanent two-pane picker was busy and
competed with the ordering work.

- Full-width parts, each a card with a big page grid (150px, contain so whole
  pages are visible) and drag-to-reorder; positional page number as a badge.
- Editable Part # (hero field) backed by new series_chapter.stated_part —
  separate from the auto-managed chapter_number, mirroring the page_number vs
  stated_page split so reorder/delete renumbering can't wipe a hand-set part.
  Missing-Part hints when consecutive parts' stated_part jump >1.
- Each part labels its source post (derived from pages' primary_post_id) and
  shows the printed-page range with clear labels.
- Picker demoted to an on-demand right slide-over ('Add pages') with a target-
  part selector; part actions (move/merge/delete) collapsed into an overflow ⋮.

alembic 0042 adds series_chapter.stated_part (nullable int).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:29:10 -04:00