1061 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
bvandeusen	ae569c0f9a	fix(gallery): ruff C408 (dict literal) + panel auto-open on deep-link ... Rewrite facets() common/plat_scope as dict literals (C408). Open the refine panel via a watch on hasRefineFilters rather than reading filter state at bar-setup time — the parent applies the URL query in its onMounted, after the bar child has set up, so the initial read was always the default (empty) state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:14:15 -04:00
bvandeusen	1adc47f59c	feat(gallery): faceted refine panel UI (Phase 2 frontend) ... Add a 'Refine ▾' toggle to the gallery filter bar that expands a full-width GalleryFacetPanel below it, inside the same sticky hazey chrome. The panel offers platform chips (with live counts + a 'No platform' unsourced bucket), two count-badged curation-flag toggles (Untagged / No artist), and a from/to date range bounded by the facet min/max. Store gains the platform/untagged/no_artist/date_from/date_to filter params (URL-mirrored, AND-composed) and a panel-gated, single-flighted loadFacets() that fetches /api/gallery/facets scoped to the active filter. Shared cloneFilter/filterToQuery helpers keep the bar and panel writing one URL format. The panel auto-opens on deep-link when refine filters are present and refetches counts (debounced) on every filter change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:06:51 -04:00
bvandeusen	9fe534139a	feat(gallery): faceted filter params + /facets counts endpoint (Phase 2 backend) ... Extend the composable gallery filter with platform / untagged / no_artist / date_from / date_to, AND-composed with the existing tag/artist/media/sort params and threaded through scroll, timeline, and jump_cursor. Add GalleryService.facets() + GET /api/gallery/facets returning live counts scoped to the current filter with per-group minus-self semantics: platform counts (COUNT(DISTINCT image) incl. a null unsourced bucket), curation-flag counts (untagged / no_artist), and effective_date min/max bounds. The UNSOURCED_PLATFORM sentinel makes filesystem-imported content reachable via the platform facet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:01:07 -04:00
bvandeusen	16e0268da7	fix(modal): next arrow clears metadata panel + arrows work in empty tag input ... Image viewer (#609): - The next (▶) arrow was offset from the viewport edge (right:16px) so it floated over the 320px metadata side panel. Offset it off a shared --fc-side-w var so it sits at the image's right edge instead; full-width again below 900px when the panel stacks under the image. - Arrow nav was fully disabled whenever a text field was focused. Now it yields to the caret ONLY when the field has text; an empty tag-entry field still navigates ←/→. Extracted to utils/textEntry.js (arrowNavAllowed). ESC behaviour unchanged (already closes the modal, overlay-aware). Test: arrowNavAllowed — empty/non-text → navigate, text present → don't. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 06:37:16 -04:00
bvandeusen	1f4ce8513b	style: ruff I001 — keep _sync_engine as-import on its own line (combine-as-imports=false) ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:59:26 -04:00
bvandeusen	5a116ca9d0	style: ruff I001 — aliased _sync_session_factory sorts before get_sync_engine ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:55:18 -04:00
bvandeusen	ef3ee5aceb	feat(maintenance): DB maintenance UI card + fix ruff I001 ... - Settings → Maintenance gains a "Database maintenance" card: a "Run VACUUM ANALYZE now" button (enqueues the maintenance task) plus a per-table bloat readout (live/dead/dead%/last vacuum) from /api/admin/maintenance/db-stats. - dbMaintenance store (loadStats / runVacuum) + test. - Fix ruff I001: combine the two _sync_engine imports onto one line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:52:36 -04:00
bvandeusen	914033db29	feat(maintenance): scheduled + manual DB VACUUM ANALYZE + bloat readout ... The TABLESAMPLE showcase reads physical blocks (bloat-sensitive), and the periodic prune/backfill/recovery tasks churn dead tuples faster than autovacuum always keeps up — so explicit maintenance earns its keep here. - tasks.maintenance.vacuum_analyze: VACUUM (ANALYZE) over high-churn tables (VACUUM_TABLES) on an AUTOCOMMIT connection (VACUUM can't run in a txn). Scheduled weekly via Beat; also operator-triggerable. - _sync_engine.get_sync_engine(): expose the process engine for the autocommit connection. - GET /api/admin/maintenance/db-stats: per-table n_live/n_dead/dead_pct + last (auto)vacuum/analyze from pg_stat_user_tables — visibility, not a black box. - POST /api/admin/maintenance/vacuum: enqueue the task on demand. Tests: vacuum task runs + reports tables; db-stats shape; trigger queues. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:49:39 -04:00
bvandeusen	d495605c12	style(gallery): hazey filter bar attached to the TopNav ... - Filter bar gets the same obsidian translucent + backdrop-blur as the TopNav so the two read as one piece of chrome. - margin-top:-8px cancels the v-container's pt-2 so the bar sits flush at 64px even at scroll 0 — fixes the gap/separation when scrolled to top. - Inputs/toggles get a more-opaque backing so they stay legible on the haze. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:40:32 -04:00
bvandeusen	76d8ad42a8	fix(showcase): buffered producer/consumer for a steady cascade ... The cascade "burped" — chunks appeared unevenly — because the old pipeline coupled display to fetch timing: it trickled each batch right after its fetch and assumed the next round-trip would land inside the ~240ms trickle window. When a fetch ran long (TABLESAMPLE hits random, sometimes-cold blocks; RTT jitter) the animation starved, then a clump burst in. Decouple the two: - Producer (_fill) races ahead fetching batches into a buffer up to a target depth, refilling when it dips below BUFFER_MIN. - Consumer (_drain) reveals one item every CADENCE_MS regardless of when fetches land; it only waits if the buffer genuinely starves. A small PRIME buffer precedes the drain so it doesn't starve at the front; the buffer (BUFFER_MIN×CADENCE runway) absorbs per-fetch jitter so images appear at an even pace. Public store API (loadInitial/shuffle/fetchPage/ images/loading/hasMore/isEmpty) unchanged — ShowcaseView/MasonryGrid need no change. Test (fake timers): fire-order + dedup, one-item-per-cadence rate limit, empty-library flag. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:35:25 -04:00
bvandeusen	711abea567	Merge pull request 'Gallery speed + fandom editing + filters + pinned filter bar' (#61) from dev into main	2026-06-04 00:07:19 -04:00
bvandeusen	6d630d13d6	feat(gallery): pinned filter bar (Phase 1) ... Gallery now has in-view filtering, styled like the app's sticky v-tabs chrome (pinned at top:64px under TopNav). - GalleryFilterBar: combined tag+artist autocomplete (searches /api/tags + /api/artists), closable filter chips (multi-tag AND), media toggle (All/Images/Videos), Newest/Oldest sort, Clear. Writes all state to the URL via router.push. - gallery store: filter is now { tag_ids, artist_id, media_type, sort, post_id }; applyFilterFromQuery makes the URL the single source of truth (deep-linkable, back-button works); chip labels resolved by id or pre-noted on pick. Replaces the standalone tag chip + setTag/PostFilter. - GalleryView: renders the bar (hidden in post-detail), syncs route.query → store on mount + every query change. Also untracks the transient .claude/scheduled_tasks.lock committed in 3f30327 and gitignores it. Tests: store parses query → composable scroll params, post_id exclusivity, newest-sort omitted, label pre-seed, single initial fetch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:57:11 -04:00
bvandeusen	3f30327fa5	feat(gallery): composable scroll filter (multi-tag AND, media, sort) ... Phase 1 backend for the gallery filter bar. Extends scroll/timeline/jump from a single mutually-exclusive filter to a composable one: - tag_ids: image must carry ALL of them (one correlated EXISTS per tag — AND, no row multiplication), replacing the single-tag JOIN. - artist_id composes with tags; media_type ('image'|'video') narrows by mime; post_id stays the exclusive post-detail path. - sort ('newest'|'oldest') flips the effective_date/id cursor comparison and ordering; the cursor value is unchanged (direction comes from the request). jump_cursor honors sort too. - Shared _apply_scope helper applied across scroll/timeline/jump so the timeline sidebar reflects the filtered set. API _parse_filters parses tag_id (comma list), artist_id, media, sort. Tests: multi-tag AND, media filter, sort reversal (service + API); post_id-excludes-others; single tag_id back-compat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:50:19 -04:00
bvandeusen	4f9464d215	feat(gallery,tags): clear active filters ... Two gaps where a filter couldn't be removed: - Gallery: a tag_id filter (from clicking a tag) had no indicator or clear control — only post_id did (PostInfoHeader). Add an "Tag: <name> ✕" chip that clears the filter by dropping tag_id from the URL. New lightweight GET /api/tags/<id> resolves the name; the store fetches it on filter set. - Tags view: the kind chip-group used mandatory="false" — a STRING ("false" is truthy in JS), which made the group mandatory so the active kind chip couldn't be deselected. Fixed to :mandatory="false" so the filter clears. Tests: GET /tags/<id> shape + 404; gallery store resolves filterTagName. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:26:04 -04:00
bvandeusen	e678d1dfdf	feat(tags): fandom-edit UI in tags directory + image modal ... Adds the missing UI to change a character tag's fandom, in both places: - FandomSetDialog (shared): pick an existing fandom, create a new one, or clear it; on a name collision in the target fandom it surfaces a merge confirmation and resolves via setFandom(merge:true). Reuses the tags store's fandom cache. - TagCard kebab gains "Set fandom…" for character tags (→ TagsView opens the dialog, reloads on success). - TagPanel chip kebab gains "Set fandom…" for character tags (→ reloads the modal's tag list on success). - tags store: setFandom(tagId, fandomId, {merge}) action + test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:21:25 -04:00
bvandeusen	d9ab6e15c6	feat(tags): edit a character tag's fandom (backend) ... No way existed to change which fandom a character tag belongs to after creation — PATCH /tags/<id> only renamed. - TagService.set_fandom(tag_id, fandom_id, merge=False): set / change / clear (fandom_id=None) a character's fandom, with the same validation as find_or_create. On a name collision in the target fandom it raises TagMergeConflict (→ 409, same shape as rename); merge=True resolves it by merging this tag INTO the existing character. - Extract _do_merge(source, target) from merge() so set_fandom can perform the deliberate CROSS-fandom merge the public merge() validation forbids. - PATCH /tags/<id> now accepts optional fandom_id (+ merge flag) alongside name, and returns fandom_id. Tests: set/change/clear, non-character + bad-ref rejection, collision raises, merge resolves; API set/clear + collision→merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:13:24 -04:00
bvandeusen	e05e0b9f37	perf(gallery): materialize indexed effective_date sort key ... The gallery cursored on COALESCE(post.post_date, image_record.created_at) across the Post outer join — an expression spanning two tables that no index can serve, so every /scroll sorted a large slice of the library (and the old frontend fired ten serially). Materialize it: - image_record.effective_date column + ix_image_record_effective_date (effective_date DESC, id DESC); alembic 0035 backfills COALESCE(primary post's post_date, created_at) for existing rows. - gallery_service._effective_date_col() now returns the column, so scroll / timeline / jump / neighbors all order off the index instead of re-deriving the COALESCE. _neighbors reads record.effective_date directly (drops an extra Post lookup). - importer._apply_sidecar maintains it: when a primary post with a date is linked, effective_date = post.post_date; plain inserts keep the created_at-equivalent server default. Tests: sidecar import asserts effective_date == post.post_date; gallery ordering/timeline/jump test seeds set effective_date alongside created_at. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 22:58:46 -04:00
bvandeusen	56cc253009	feat(gallery): reveal tiles on image load + single initial fetch ... The 5×10 metadata batching only staggered the cheap layer (JSON); thumbnails load as independent <img> requests and clustered, so tiles "popped in together" after a wait. Two changes: - GalleryItem reveals each tile when ITS OWN thumbnail fires @load (with an onMounted complete-check for cached thumbs), playing a showcase-style flip-up entrance. Tiles now cascade in natural load order instead of all at once. Honors prefers-reduced-motion. - gallery store does ONE initial fetch (limit=50) instead of 10 serial /scroll round-trips. Fewer RTTs, faster first paint; the reveal-on-load is what makes appearance progressive now. Infinite scroll pulls 25/trigger. Tests: GalleryItem gains is-loaded only after @load; loadInitial issues exactly one scroll request at the initial limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 22:50:03 -04:00
bvandeusen	844bb86802	Merge pull request 'fix(download): release DB connections across the gallery-dl subprocess' (#60) from dev into main	2026-06-03 22:11:36 -04:00
bvandeusen	576e16d14d	fix(download): release DB connections across the gallery-dl subprocess ... Backfill events were STILL stranding empty after the timeout-ladder fix. Worker logs showed the salvage path working ("Download timeout for anduo/patreon after 1170.0s (18 files written)") but then: Retry in 3s: DBAPIError(ConnectionDoesNotExistError: connection was closed in the middle of operation) ...succeeded in 0.149s <- in-flight guard no-op Root cause: DownloadService held the async + sync DB connections checked out across the entire (≤19.5-min backfill) gallery-dl subprocess. The server reaps the idle connection, so phase 3's first query hits a dead socket. That DBAPIError trips download_source's autoretry_for, the retry re-enters _phase1_setup, sees the event still 'running', returns in_flight and no-ops — leaving the event to be stranded empty by the recovery sweep. pool_pre_ping was already on both engines but can't help a *held* connection (it only validates on pool checkout). Fix: - DownloadService.download_source closes the async + sync sessions after phase 1, before the subprocess, so phase 3 re-acquires a live connection (matches the class's "Phase 2 — no DB connection" docstring). - The per-task async engine switches to NullPool so phase 3 always opens a fresh connection rather than a pooled one the server may have reaped. Tests: assert connections are released before gdl.download runs and the event still finalizes; assert the task engine uses NullPool. Also fixes a stale 1800s->1170s comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 21:49:52 -04:00
bvandeusen	a8f6a464aa	Merge pull request 'fix(download): salvage soft-time-limit kills + fix timeout ladder' (#59) from dev into main	2026-06-03 19:35:45 -04:00
bvandeusen	9cb24c9e1b	style(test): fix ruff I001 import order in download task test ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 17:04:29 -04:00
bvandeusen	6590dcdb39	fix(download): salvage soft-time-limit kills + fix timeout ladder ... Backfill downloads stranded with empty logs + a generic "stranded by recovery sweep" error. Root cause: the backfill gallery-dl subprocess timeout (1170s) exceeded download_source's Celery soft_time_limit (900s), so SoftTimeLimitExceeded preempted subprocess.TimeoutExpired. The TimeoutExpired path (which captures partial stdout/stderr and finalizes the event) never ran, the event was left 'running', and phase 3 never decremented backfill_runs_remaining — so the source re-ran and re-stranded every tick (Anduo #39912). Two layers: 1. Raise download_source limits (soft 900→1350, hard 1200→1500) so both subprocess budgets (870 tick / 1170 backfill) sit below the soft limit with phase-3 persist headroom. Promote to module constants and guard the invariant with a test. 2. Catch SoftTimeLimitExceeded in download_source and finalize the in-flight event with a real reason, mirror phase-3 source-health, and decrement backfill so a chronically-slow source self-heals to tick mode. The existing celery_signals handler only covered TaskRun, not DownloadEvent — that was the gap. Updates stale 900/1200 references in gallery_dl.py + maintenance.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 16:56:13 -04:00
bvandeusen	ab9922ad2e	Merge pull request 'feat(artist): "new since last visit" badge + banner' (#58) from dev into main	2026-06-03 16:20:54 -04:00
bvandeusen	3162cff96b	fix(artist): ruff UP017 + test_directory_card_shape pin ... Two CI bounces on b65e956: 1. ruff UP017 — Python 3.14's preferred form is `datetime.UTC`, not `timezone.utc`. Switch the test's two TZ literals. 2. test_directory_card_shape pinned the card key set to the pre-feature shape; `unseen_count` was added to the API payload but the pin wasn't updated. Same shape as the recurring 'plan-grep-pinned-tests' trap — should have grepped tests/ for card.keys() before pushing.	2026-06-03 15:45:59 -04:00
bvandeusen	b65e956ad2	feat(artist): "new since last visit" badge + banner ... Per-artist "+N" accent pill on the artists directory and a "N new since last visit" banner inside ArtistView. Counts new IMAGES (not posts) so multi-image posts increment correctly. - alembic 0034: artist_visit (artist_id PK, last_viewed_at NOT NULL). Seeds every existing artist with last_viewed_at=NOW() so the badge starts at 0 across the board — no noisy "5000 unseen images" on first deploy. - ArtistService.find_or_create autoseeds a visit row alongside new artists, so freshly imported content doesn't read as unseen. - ArtistService.overview reads pre-visit last_viewed_at, counts images created since, then atomically UPSERTs last_viewed_at=NOW() via postgres ON CONFLICT DO UPDATE (no SELECT-then-INSERT race per reference_scalar_one_or_none_duplicates). Returns the pre-update count as `unseen_count_at_visit` so the banner has data. - ArtistDirectoryService.list_artists adds an `unseen_count` aggregate to each card via LEFT JOIN artist_visit + conditional COUNT. NULL last_viewed_at (artist created before this code shipped) defensively counts as "never visited" → all images unseen. - Frontend: ArtistCard renders an accent pill in the preview-strip corner when unseen_count > 0 (capped at 99+); ArtistView shows a closable v-alert banner on initial load when unseen_count_at_visit > 0, re-arms on slug change. Single-row-per-artist (no user_id) — rule #47 multi-user ACL is aspirational; widens to (user_id, artist_id) PK when User lands, per rule #22. Scribe plan #597.	2026-06-03 15:27:11 -04:00
bvandeusen	0533807669	Merge pull request 'feat(ext): verify cookies in-browser before uploading (1.0.7)' (#57) from dev into main ext-1.0.7	2026-06-03 14:17:42 -04:00
bvandeusen	d3245f0c22	feat(ext): verify cookies in-browser before uploading (1.0.7) ... Pre-upload verify request: after capturing the live browser cookies, hit a known authenticated endpoint with credentials:'include' from the extension's background context. If the platform reports we're not logged in, abort the upload so we don't overwrite FC-side credentials with stale data. - platforms.js: add `verify` config per cookie-auth platform - hentaifoundry: HEAD /?enterAgree=1 (mirrors gallery-dl's HF _init_site_filters; same 401 path the operator hit 2026-06-03) - patreon: GET /api/current_user (clean 401 when logged out) - subscribestar, deviantart: no stable auth endpoint, skip verify - cookies.js: verifyCookiesForPlatform() returns {ok, status, reason}. ok=true/false/null tri-state — null = verify not configured, caller treats as "proceed". - background.js EXPORT_COOKIES + EXPORT_ALL_COOKIES: verify gates the upload; failures bubble up with the platform's name + reason. - popup.js: success message now appends "(verified ✓)" when applicable. - manifest + package.json: 1.0.6 → 1.0.7.	2026-06-03 14:04:45 -04:00
bvandeusen	279dff3fb6	Merge pull request 'feat(ml): normalize Camie suggestion names to human-readable' (#56) from dev into main	2026-06-03 13:18:44 -04:00
bvandeusen	e450145304	fix(ml): preserve digit-only tag names in normalize (year tags) ... Rule 8 'no letters -> drop' was over-eager: bare digit tags like '2005' returned None even though they're legitimate (booru year-tag shape). Widen the keep-condition to any alphanumeric. Emoticons (':/', '^_^', '+_+') still drop since they contain neither letters nor digits.	2026-06-03 13:09:35 -04:00
bvandeusen	a6e8d4b52e	feat(ml): normalize Camie suggestion names to human-readable ... Camie's booru-style vocab strings (`uchiha_sasuke_(naruto)`, `#unicus_(idolmaster)`, `1000-nen_ikiteru_(vocaloid)`, `:/`) were surfacing raw in SuggestionsPanel — and worse, the SAME raw string was written to tag.name on Accept, polluting the DB with `underscored_lowercase` names that don't match the operator's "Title Case" tag convention. Add backend/app/services/ml/tag_name.py with a single normalize() applying nine rules (strip leading junk #/./+/;/~/_/ws, drop trailing _(disambiguator) blocks iteratively, strip wrapping quotes, underscores to spaces, space after colon, title-case each word's first char, preserve hyphens/apostrophes/digits, drop entries with no letters). Wire into SuggestionService.for_image: - raw Camie key kept for alias_map lookup (alias rows are hand-curated against raw keys; don't disturb) - display_name = normalize(raw); None means drop the candidate - existing-tag lookup widened to case-insensitive match against BOTH raw and normalized forms so legacy underscore-named Tag rows accepted before this change still surface as "existing" not "+ new"	2026-06-03 13:00:08 -04:00
bvandeusen	37e66cddc4	Merge pull request 'chore(modal): drop ?image=N soft-compat — pure overlay' (#55) from dev into main	2026-06-02 19:35:04 -04:00
bvandeusen	f1860866de	chore(modal): drop the ?image=N soft-compat from G5.4 ... Operator confirmed they have no existing ?image=N bookmarks to preserve, so the soft-compat read on initial mount + router.replace strip is dead weight. The modal is now purely a Pinia overlay — no URL involvement on open OR initial mount. Drops 33 lines plus the now-unused vue-router imports in both ArtistGalleryTab (entire onMounted gone) and GalleryView (just the ?image=N block; the post_id/tag_id filter handling stays).	2026-06-02 19:24:09 -04:00
bvandeusen	9cf6b2d363	Merge pull request 'audit-g5 final + ML threshold default + kebab menu fix' (#54) from dev into main	2026-06-02 19:09:49 -04:00
bvandeusen	b181d779fe	fix(test): default suggestion_threshold_general now 0.70 (alembic 0033)	2026-06-02 18:49:24 -04:00
bvandeusen	0fbb19dc24	fix(modal): TagPanel kebab — apply the wrapping-span fix that the prior commit missed ... The previous commit (8326e54) updated the CSS but the structural edit to the v-menu wrapper didn't take. Re-applying so the chip kebab actually opens.	2026-06-02 18:48:53 -04:00
bvandeusen	8326e5447a	fix(modal): kebab menus weren't opening (chip + suggestion rows) ... Operator-flagged 2026-06-02. Both kebab activators were broken: - TagPanel chip kebab had `@click.stop` directly on the v-icon activator. In Vue 3, an explicit @click on the same element as `v-bind="props"` overrides the spread onClick — so Vuetify's activator handler never fired. Menu never opened. - SuggestionItem kebab didn't have @click.stop, but for consistency and to make both kebabs follow the same shape, wrap it too. The fix: each kebab is now wrapped in a `<span @click.stop>`. The v-icon / v-btn receives Vuetify's onClick cleanly and opens the menu; the bubbling click then reaches the span where stopPropagation absorbs it before it can affect a parent (the v-chip's close button in TagPanel's case).	2026-06-02 18:48:32 -04:00
bvandeusen	1fd594baaf	chore(ml): suggestion_threshold default 0.50 → 0.70 ... Operator-flagged 2026-06-02 — the 0.50 default (set on 2026-06-01) surfaces too many low-confidence picks in the modal's Suggestions rail. 0.70 keeps the rail signal-rich while still showing more than the original 0.95 (which hid almost everything). Alembic 0033 updates the singleton row conditionally — only rows still at the old 0.50 default flip to 0.70. Operators who tuned to some other value via Settings → ML keep their pick. Settings UI already exposes both sliders (MLThresholdSliders.vue), so further tuning continues to work without a deploy.	2026-06-02 18:38:12 -04:00
bvandeusen	ecac6c4bda	fix(audit-g5): centroid version DB-as-truth + modal as overlay ... Closes the last two findings from the 2026-06-02 audit (G5.1 + G5.4). G5.1 — Centroid version no longer drifts: CentroidService now reads MLSettings.embedder_model_version (the DB row tag_and_embed already writes from) for both the centroid model- version stamp and the drift-detection comparison. Previously the centroid sites imported MODEL_VERSION from env, so the version stamped on centroids could disagree with the version stamped on the embeddings they were built from. By construction those now match, so list_drifted won't silently miss the env-vs-DB drift case. embedder.py keeps MODEL_VERSION as an env-driven constant for the actual model loader — that's a different concern (which weights are loaded) from the version-stamp that gets persisted alongside data. G5.4 — Modal is a Pinia-only overlay: The previous URL↔modal sync in GalleryView and ArtistGalleryTab leaked the modal across route changes (RouterLink to /artist/<slug> left the modal mounted on top of the new route) and re-opened it on history back/forward with stale ?image=N entries. Now: openImage() just calls modal.open(id) — no URL push. GalleryView's dead closeImage helper is deleted. A route.name watcher in App.vue closes the modal whenever the route changes, which auto-fixes RouterLink-in-modal and back/forward. Backward-compat: ?image=N is still honored on initial mount as a one-shot deep-link opener, then router.replace strips the query so the URL doesn't re-trigger and no extra history entry is added. Existing bookmarks / shared URLs keep working; new opens stay Pinia-only.	2026-06-02 18:28:57 -04:00
bvandeusen	6ef0fed41f	Merge pull request 'audit-g5: architectural debt — 4 bundles (A/B/C/D)' (#53) from dev into main	2026-06-02 18:07:25 -04:00
bvandeusen	9f7261b9c0	fix(audit-g5c): set CURATOR_BOOTSTRAP_NEW_KEY=1 in conftest ... CredentialCrypto's safety check fires on create_app() instantiation because the test environment has no pre-seeded Fernet key file. Set the bootstrap env var before any test imports so the auto-create path is allowed during tests.	2026-06-02 17:55:25 -04:00
bvandeusen	f05aaa707b	fix(audit-g5d): surface ErrorType taxonomy on FailingSourcesCard ... Alembic 0032 adds Source.error_type (varchar(32), indexed). _update_source_health stamps it alongside last_error on status='error' and clears it on 'ok'. SourceRecord/to_dict exposes it. FailingSourcesCard renders a colored chip next to the consecutive- failures count, with a tooltip explaining the suggested operator action. Color reflects intent: - warning (yellow) — operator action needed (auth_error) - info (blue) — backend-paced (rate_limited / timeout / network_error / partial / tier_limited) - error (red) — likely terminal without intervention (not_found / access_denied / validation_failed / unsupported_url / http_error / unknown_error) Audit 2026-06-02: the backend computed 13 ErrorType categories but only the free-text last_error reached the operator. Bulk-triage by class ("all auth_error → rotate cookies", "12 rate_limited → just wait") required opening Logs per row.	2026-06-02 17:54:44 -04:00
bvandeusen	4df98171ab	fix(audit-g5c): refuse silent Fernet key regeneration on partial restore ... Audit 2026-06-02: `_load_or_create_key` silently minted a new Fernet key whenever the key file was missing — no log, no warning. The failure mode the audit flagged: a partial disaster restore where the DB was restored but `/images/secrets/` was lost would produce a working-looking system in which every authenticated download fails AUTH_ERROR until the operator re-uploads every credential by hand. Two opt-ins now needed for auto-creation: 1. Explicit `bootstrap_ok=True` kwarg (tests, scripts), OR 2. `CURATOR_BOOTSTRAP_NEW_KEY=1` env var (operator first-time setup) Otherwise the constructor raises `MissingCredentialKey` so the app fails fast at startup and the operator can restore the key file from backup before encrypted_blob rows go undecryptable. Also: docstring path was wrong (said "images/data root" but actual location is `/images/secrets/credential_key.b64`) — corrected. Tests updated to pass `bootstrap_ok=True` explicitly, and two new tests cover the safety behavior (missing-key-raises, env-var-bootstraps).	2026-06-02 17:04:27 -04:00
bvandeusen	8d75ade1d5	fix(audit-g5b): race-poisoning in three find-or-create sites ... Audit 2026-06-02 flagged three SELECT-then-INSERT sites that lost the race under concurrent writers / recovery-sweep replays, poisoning the outer transaction with an unrecoverable IntegrityError and crashing the calling task. Same shape as the banked 2026-05-26 ImageProvenance incident (see reference_scalar_one_or_none_duplicates memory). Mirrors the importer._get_or_create pattern in all three: savepoint via begin_nested + IntegrityError rollback to that savepoint + re- SELECT to grab the row the other worker committed first. - importer._capture_attachment: PostAttachment.sha256 UNIQUE. attachments.store is sha-addressed so both workers race to write the same on-disk path (shutil.copy2 + rename is idempotent), so no extra cleanup needed. - TagService.find_or_create: partial uniqueness index on (name, kind, COALESCE(fandom_id, -1)). The previous docstring claimed "INSERT ... ON CONFLICT" but the implementation was SELECT-then-INSERT with no recovery. - ArtistService.find_or_create: already had IntegrityError handling but did session.rollback() (unwinds the WHOLE transaction); now uses begin_nested + sp.rollback() so the surrounding request's progress isn't lost.	2026-06-02 17:02:12 -04:00
bvandeusen	75c63e1511	fix(audit-g5a): ruff isort — platforms after patreon_resolver	2026-06-02 16:55:56 -04:00
bvandeusen	98673d4dca	fix(audit-g5a): small architectural cleanups bundle ... Five small G5 findings from the 2026-06-02 audit. Each is local and follows an established FC pattern. - download_service: replace hardcoded ('discord','pixiv') tuple with auth_type_for(platform) == 'token'. A 7th token-platform now picks up the right credential path without touching this site. - /api/tags/<source_id>/merge enqueues recompute_centroid.delay after merge so the target's centroid reflects its new image set immediately. Daily list_drifted catches it within 24h, but eager recompute closes the suggestion-quality dip in the meantime. - backfill_thumbnails added to beat_schedule (daily). The task docstring claimed periodic Beat but the entry was never registered, so the library got no self-healing thumbnail repair; only the manual admin-UI button fired it. - modal.createAndAdd pushes a kind='fandom' tag into tagsStore.fandomCache so FandomPicker sees the new fandom on next open. Was: cache-gated load (length===0) skipped refetch, new fandom invisible until full page reload. - cleanup cluster: - Drop .webp from cleanup_service.unlink — thumbnailer only writes .jpg/.png; the third tuple member was dead code. - Drop effective_date from /api/gallery/scroll response — no FE consumer reads it. Service still computes the attribute for timeline ordering; this just trims the JSON. - Rename store.recentMinute → store.recentRuns across the systemActivity store + three consumers (SystemActivitySummary, QueuesTable, SystemActivityTab). The data is the last 200 runs (not actually "last minute"), so the name lied. NOT in this bundle: the duplicate tag-merge endpoint (/api/tags vs /api/admin/tags) is harder — has 1 FE caller and 3 tests on the admin variant; consolidation is its own change.	2026-06-02 16:46:46 -04:00
bvandeusen	89b48f8f35	Merge pull request 'audit-g4: status-enum miss batch' (#52) from dev into main	2026-06-02 16:15:00 -04:00
bvandeusen	4bff1d8558	fix(audit-g4): status-enum miss batch ... Five extension-miss findings from the 2026-06-02 audit, where a status value was added on one side but a downstream consumer didn't pick it up. - download_service._phase3_persist: explicit branches for ImportResult.status in ('failed','refreshed'). For 'failed' (archive probe crash from _import_archive), unlink the source file so the filesystem scanner doesn't re-import and re-crash on the same archive forever. 'refreshed' is currently unreachable from the download path (no deep=True) but matches the importer's documented contract; treat as 'attached'. - gallery-dl backfill auto-complete now gates on dl_result.success + no error_type, not just return_code==0 + files_downloaded==0. VALIDATION_FAILED exits the subprocess with returncode=0 and files_downloaded=0 when every file was quarantined, matching the prior predicate exactly and zeroing the operator's armed backfill budget on the FIRST quarantine run instead of decrementing. - attach_in_place archive dispatch now threads artist + source_row through _import_archive (and _import_media for archive members) and _supersede. The path-walk fallback (_resolve_artist) is still used by filesystem-import; the download path now binds ImageProvenance to the explicit subscription Source instead of rediscovering by (artist_id, platform). - Three FE handlers now recognize status:'deferred' from /api/sources/<id>/check: SubscriptionsTab.onCheck (was toasting "event #undefined"), SubscriptionsTab.checkAll (was counting deferred as queued), DownloadEventRow.onRetry (was saying "re-queued" when nothing was). Pattern matches DownloadsTab.onRetryAll which already had it. - celery_signals._queue_for now maps backup/admin/library_audit prefixes to 'maintenance' (matching task_routes). TaskRun.queue was returning 'default' for those rows, so per-queue dashboard filters and per-queue threshold overrides (added in G3) silently missed them.	2026-06-02 16:04:59 -04:00
bvandeusen	d60e0b9494	Merge pull request 'audit-g3: lifecycle batch — recovery sweeps, retention, timeouts' (#51) from dev into main	2026-06-02 14:49:28 -04:00
bvandeusen	e30f50e6fe	fix(audit-g3): lifecycle batch — recovery sweeps, retention, timeouts ... Plugs the FC long-running-entity discipline gaps the 2026-06-02 audit flagged: every entity that can get stuck now has recovery + retention + timeout, and the long-runners no longer collide with the FC-3i sweep. Recovery sweeps (every 5 min): - recover_stalled_backup_runs — flips BackupRun stuck in running/restoring past 7h (covers the 6.5h images-backup hard limit) to error. prune_backups docstring corrected — the FC-3i TaskRun sweep never touched BackupRun rows. - recover_stalled_library_audit_runs — flips LibraryAuditRun stuck past 135 min (10-min buffer above scan_library_for_rule's 2h5m hard limit) to error. Previously a SIGKILL'd row blocked all future audits until manual DB surgery. - recover_stalled_import_batches — finalizes ImportBatch rows stuck running >2h whose child tasks are all terminal (orphan case where the orchestrator crashed before the closing UPDATE). Uses the same EXISTS predicate /api/system/stats already had. Retention (daily): - prune_library_audit_runs — 30-day window. Audit rows carry matched_ids JSONB blobs that can hold tens of thousands of ids. - prune_import_batches — 30-day window. Cascades to ImportTask via the model relationship. time_limits on five long-runners that previously had none (the audit's headline finding — every one of these collided with the recover_stalled_task_runs 5-min default and could be marked 'error' mid-flight): - scan_directory: 60m soft / 70m hard - verify_integrity: 60m / 70m - backfill_phash: 30m / 35m - apply_allowlist_tags: 30m / 35m - recompute_centroids: 30m / 35m QUEUE_STUCK_THRESHOLD_MINUTES now covers maintenance (75) and scan (75) — above the longest task on each — with per-task overrides for the outliers (backup_images_task 420, restore_images_task 420, scan_library_for_rule 130). start_audit_run guard is now age-aware: a 'running' row older than the audit hard limit doesn't block a new run (the sweep will catch it within 5 min). Previously a SIGKILL'd row blocked forever. /api/import/status now uses the same EXISTS predicate /api/system/stats does, so the two endpoints no longer disagree on the active-batch question. DownloadEvent.started_at resets on pending→running so a freshly- promoted event from a busy queue isn't measured against its original enqueue time (was racing recover_stalled_download_events on heavy-queue days).	2026-06-02 14:30:46 -04:00