bvandeusen/FabledCurator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 33 Wiki Activity
426 Commits 3 Branches 33 Tags
992f38ec20fec7a9d04010f03799a2e2acc05208
Commit Graph

426 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
bvandeusen 992f38ec20 fix(test): drop unused Post binding in test_importer_provenance_race (ruff F841) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 14:39:57 -04:00
bvandeusen 0bc5767a2b fix(importer): Source = one per (artist, platform), not one per post — filesystem importer's sidecar paths now reuse the artist's existing subscription Source (or create one synthetic anchor with enabled=False) instead of fabricating a new Source per post URL. Alembic 0022 consolidates existing per-post Sources to canonical (prefers campaign URL; falls back to sidecar:<platform>:<slug>) and re-parents Posts + ImageProvenance, merging Post collisions. 
Operator-flagged 2026-05-26: Atole artist detail page showed 406 Sources where 1 was right.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 14:19:29 -04:00
bvandeusen 397021dcbd fix(importer): ImageProvenance (image_record_id, post_id) race-safe via savepoint + alembic 0021 UNIQUE — closes the SELECT-then-INSERT window that planted duplicates and broke .scalar_one_or_none() on every later deep-scan rederive (MultipleResultsFound). Migration dedupes existing rows (min(id) per pair); model gains __table_args__; gallery-filter test that seeded duplicates dropped. 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 13:47:55 -04:00
bvandeusen b0bfbc585a fix(import-admin): retry-failed + clear-stuck — same UPDATE…WHERE pattern as the maintenance sweep, so neither endpoint can hit psycopg's 65535-parameter ceiling once accumulated row counts exceed ~65k 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 11:47:40 -04:00
bvandeusen 110c1c0e51 fix(maintenance): recover_interrupted_tasks — fold SELECT into UPDATE…WHERE…RETURNING so the IN-list no longer blows past psycopg's 65535-parameter ceiling (operator-hit 2026-05-26 after deep scan orphan pile) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 11:46:06 -04:00
bvandeusen 6de84d0d60 feat(ui): ErrorDetailModal — context panel (task/queue/target/duration/started/retries/worker/celery-id/args) + contrast fix (background-token bg vs surface-variant pale-on-pale) + copyText helper for Copy button 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 11:16:37 -04:00
bvandeusen 4e1f208a9f fix(ui-copy): copyText utility with execCommand fallback — navigator.clipboard is gated by Secure Context (HTTPS-only) and is undefined on plain-HTTP self-hosted deployments. Apply to ExtensionKeyBar + BrowserExtensionCard. 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 11:16:32 -04:00
bvandeusen 06913eba8e feat(thumb-backfill): MaintenancePanel — wire ThumbnailBackfillCard into grid, broaden intro to cover non-ML backfills 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 10:37:09 -04:00
bvandeusen b7f693b15e feat(thumb-backfill): ThumbnailBackfillCard — 'Run backfill now' button, mirrors MLBackfillCard pattern 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 10:36:55 -04:00
bvandeusen ecd0199799 feat(thumb-backfill): Pinia store — triggerBackfill() POSTs /api/thumbnails/backfill 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 10:36:45 -04:00
bvandeusen 983da9e5b1 feat(thumb-backfill): /api/thumbnails/backfill endpoint — POST → 202 + celery_task_id 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 10:36:38 -04:00
bvandeusen a41eddae3f feat(thumb-backfill): backfill_thumbnails planner task — keyset-paginates ImageRecord, NULLs bad thumb paths, enqueues generate_thumbnail for NULL/missing/corrupt 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 10:36:21 -04:00
bvandeusen 7aa7f5a3d6 feat(thumb-backfill): _thumb_is_valid helper — JPEG/PNG magic-byte check on the on-disk thumbnail file 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 10:35:06 -04:00
bvandeusen 2505b197ae feat(fc-cleanup): Pinia store + 3 cards + CleanupView + SettingsView tab + TagMaintenanceCard moved from Maintenance + ruff lint fixes — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 08:16:46 -04:00
bvandeusen 0d0b236ac3 feat(fc-cleanup): api/cleanup.py blueprint (9 endpoints) + register + delete-audit-<id> token (matches modal convention) — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 08:13:53 -04:00
bvandeusen a06ada4c9b fix(ext-ui): direct :href install button (Firefox needs anchor click, not programmatic navigation) + manifest version detection ignores -latest.xpi alias — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 08:07:22 -04:00
bvandeusen ebd985990c feat(ui): ErrorDetailModal — click error → flat-text modal with copy button (replaces unusable :title tooltip for multi-line SQLAlchemy tracebacks) — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 07:53:14 -04:00
bvandeusen 4da8d1d774 fix(importer): race-safe savepoint-based find-or-create for Source + Post (uq_source_artist_platform_url UniqueViolation operator-flagged 2026-05-26) — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 07:51:12 -04:00
bvandeusen 2d4bfa4375 fix(fc-cleanup): test sha256 fixtures stay within varchar(64) + isort the registration import — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 01:32:50 -04:00
bvandeusen 6ed2021ad6 feat(fc-cleanup): scan_library_for_rule Celery task + maintenance-queue registration — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 01:21:37 -04:00
bvandeusen 4f2ceaaf31 feat(fc-cleanup): audit lifecycle service functions (start/apply/cancel) + tests — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 01:20:41 -04:00
bvandeusen 8a5b337a53 feat(fc-cleanup): min-dimension service functions + tests — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 01:20:04 -04:00
bvandeusen 900d878d27 feat(fc-cleanup): audits/single_color.py + tests — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 01:18:38 -04:00
bvandeusen fd80d40a34 feat(fc-cleanup): audits/transparency.py + tests — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 01:18:07 -04:00
bvandeusen 929d3fc092 feat(fc-cleanup): migration 0020 + LibraryAuditRun model — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 01:17:42 -04:00
bvandeusen c0c9e56fb9 fix(importer): skip transparency check on animated images (operator-flagged 2026-05-26: animated WebP triggered 5+ min PIL multi-frame decode → Celery hard-timeout SIGKILL); compute_phash seeks frame 0 defensively — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 00:45:08 -04:00
bvandeusen 06a2f60c08 fix(ext-ci): use browser_download_url not /releases/assets/<id> + add -f to curl + magic-byte sanity check (operator-flagged 2026-05-26: prior build silently wrote '404 page not found' into the XPI file, Firefox rejected as corrupt) — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 00:42:35 -04:00
bvandeusen 0978fbac66 fix(sidecar): strip gallery-dl 'NN_' numbering prefix when locating sidecars — fixes 'deep scan refresh count high but 0 Posts created' (operator-flagged 2026-05-26) — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 00:27:43 -04:00
bvandeusen efb142239d fix(ext-ci): drop actions/upload-artifact (Forgejo Actions doesn't support v4+ GHES) — build-web reads XPI directly from the ext-<version> Forgejo release asset — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 23:32:58 -04:00
bvandeusen 5587a76606 fix(ext-ci): replace jq with python3 (jq not in ci-python image) + bump ext 1.0.2→1.0.3 (escape AMO 'version already exists' from prior partial-failure run) + add rollback to prevent empty cache-release tombstones — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 23:14:11 -04:00
bvandeusen 17e19081a2 fix(ext-ci): drop web-ext-config.cjs (v8 mis-parses .cjs configs as if module.exports were a config option) — inline ignore-files on CLI + --no-config-discovery — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 22:46:58 -04:00
bvandeusen 770bcf3aa6 feat(artist): tab split (Overview/Settings) so DangerZone is reachable without exhausting the infinite-scroll image grid — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 22:32:07 -04:00
bvandeusen 52d7905c43 perf(importer): cache phash candidates on Importer to fix archive-import soft-timeout (was O(M×N) per-member SELECTs) — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 22:25:32 -04:00
bvandeusen e6ededbe8e feat(deep-scan): IR-parity port — refreshed status + counter, re-queue completed paths in deep mode, honest UX — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 22:16:26 -04:00
bvandeusen c06cbc0abe feat(ci): inline extension sign into build.yml + Forgejo Release Assets as XPI cache (v26.05.25.5) — bump ext to 1.0.2 — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 21:30:11 -04:00
bvandeusen ac39509a74 fix(ci): rename shard jobs to no-separator names (intapi/intimp/intcore) + add diagnostic docker ps dump so next bounce surfaces the real act_runner naming convention — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 20:59:17 -04:00
bvandeusen 3531f373ee feat(settings): move BrowserExtensionCard from Maintenance to Overview tab — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 20:33:44 -04:00
bvandeusen 36cc0622cb fix(importer): sanitize PostAttachment.ext to skip mangled gallery-dl URL-encoded basenames (varchar(32) overrun) — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 20:33:44 -04:00
bvandeusen e50f92d900 perf(ci): shard integration suite into 3 parallel jobs (int_api, int_imp, int_core) — newly feasible after act_runner capacity 2→6 — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 20:05:15 -04:00
bvandeusen ba8d9b112d fix(ext-ci): add diagnostic tracing to commit step to surface why run #309 reported success without producing the XPI side-commit — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 20:05:15 -04:00
bvandeusen c451061ca5 fix(ext-ci): self-retrigger workflow on its own edits (path filter includes workflow file) — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 18:24:31 -04:00
bvandeusen 47d760550d fix(ext-ci): glob AMO-renamed signed XPI + canonicalize to fabledcurator-<version>.xpi — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 18:22:01 -04:00
bvandeusen dc3bce7fc1 chore(ext): bump to 1.0.1 to trigger initial sign-and-publish — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 17:37:10 -04:00
bvandeusen f657582f30 feat(import-ui): deep scan button, sticky settings tabs, tasks-above-filters, fix Scanning-undefined source_path — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 17:37:10 -04:00
bvandeusen 111b952535 fix(ml): load SigLIP image-only processor to avoid SentencePiece dep — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 17:31:06 -04:00
bvandeusen a0470b5f60 feat(importer): _supersede() now applies the new (larger) file's sidecar — operator wanted to scan GS download dir to supersede smaller IR-migrated images AND wire up gallery-dl Post metadata, but supersede was file-only and silently dropped the sidecar. 
_apply_sidecar is additive: it find-or-creates Post/Source/ImageProvenance
and sets primary_post_id NULL-only, so any IR-migration provenance on the
existing row survives untouched and the new GS sidecar adds a second
ImageProvenance pointing at the freshly-created Post.

Wrapped in try/except so a malformed sidecar can't unwind the file-swap
commit — the file replacement is the critical operation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 13:51:51 -04:00
bvandeusen b0bb7ae6cc ci: pip wheel cache (actions/cache on requirements.txt hash) + uv-when-available — ~2 min saved on warm runs, no risk 
uv falls back to pip install on runners without uv binary, so this
change is forward-compatible with the current ci-python image. When
the runner image gets uv pre-installed in a future bump, the warm
install path drops from ~2 min to ~10 seconds.

pytest-xdist parallelization is OUT OF SCOPE for this commit:
tests/conftest.py uses a TRUNCATE ALL TABLES RESTART IDENTITY CASCADE
fixture after every integration test against a single shared
database; xdist workers running in parallel would nuke each other's
mid-test state. A future refactor to per-worker databases or
per-worker schema isolation is the prerequisite.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 13:47:27 -04:00
bvandeusen 1bbe478fd0 ci: report slowest 25 integration tests via pytest --durations=25 — instrumentation pass before deciding parallelization vs targeted slow-test fixes 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 13:09:25 -04:00
bvandeusen 5666fd5ca5 fix(ui): scan trigger immediate-feedback toast + delayed status re-poll — operator-flagged 'click does nothing' was actually scan_directory's skip-set finalizing the batch in <100ms when every file already had an ImportTask row, before refreshStatus could ever see the active state. Now the click always produces visible feedback (immediate 'Scan triggered' + 2s 'no new files' if it quick-finalizes). 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 13:06:11 -04:00
bvandeusen 3a359f6c5e fix(ui): Quick scan button always visible (disabled when active batch present) + inline Clear stuck action — operator was clicking a spinner area thinking it was the button because activeBatch hid the Quick scan button entirely 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 12:43:37 -04:00