bvandeusen/FabledCurator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 33 Wiki Activity
374 Commits 3 Branches 33 Tags
9f54efdedfc81645525ee79ee61ff31dfb16e469
Commit Graph

374 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
bvandeusen 9f54efdedf fix(migrators): tag_apply phase 4 now covers deviantart + pixiv (was silently dropping IR PostMetadata from those platforms) 
_PLATFORM_PROFILE_URL had only patreon/subscribestar/hentaifoundry but
FC's extension_service.py recognizes 5 platforms. Any IR PostMetadata
with platform=deviantart or pixiv fell through _profile_url returning
None and the entry was silently skipped — explaining operator's
2026-05-25 finding that IR-migrated images had tags but no provenance
for the deviantart + pixiv subscriptions.

Pixiv caveat noted in comment: real profile URL takes numeric user_id
(https://www.pixiv.net/users/12345) but IR's PostMetadata.artist
stores display name. We slug the name and use it as if it were the id
so the artist->post->image linkage survives migration; the resulting
Source.url won't resolve in a browser and operator can fix via
Settings -> Subscriptions later if they want.

To recover existing IR-migrated state: re-run /api/migrate/tag_apply
on the existing manifest. Phase 4 is idempotent; new posts get
inserted only for the previously-skipped platforms.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 11:25:45 -04:00
bvandeusen 6b1bb87647 fix(tests): update test_ensure_camie_skips_when_present to v2 filenames (camie-tagger-v2.onnx + camie-tagger-v2-metadata.json) — pinned-test bounce from 3b3e756 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 11:12:55 -04:00
bvandeusen 68cffce322 fix(importer): catch PIL OSError during transparency + phash blocks, skip as invalid_image instead of letting Celery autoretry loop forever 
Operator hit a corrupt JPEG in the IR set 2026-05-25: PIL.verify() only
validates header structure but doesn't catch truncated/broken pixel
data. The error surfaces later in _transparency_pct (via getchannel
'A' -> load) or compute_phash (load) — both blow up with OSError
'broken data stream when reading image file'. Celery's autoretry_for
then bounces the same file forever instead of marking it skipped.

Wrap both PIL.load-triggering call sites with try/except OSError ->
ImportResult(status=skipped, skip_reason=invalid_image).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 02:34:36 -04:00
bvandeusen 52445eb501 fix(ui): double directory card width (220px -> 440px) + bound preview slot height (min 150 / max 220 / overflow hidden / explicit display+object-position) so tall source images can't escape on browsers that don't compute aspect-ratio 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 02:32:05 -04:00
bvandeusen 3b3e7565fb fix(ml): align tagger + downloader with Camie v2 actual layout (model.onnx -> camie-tagger-v2.onnx + JSON metadata + ImageNet preprocessing + sigmoid on refined output) 
The HF repo Camais03/camie-tagger-v2 has camie-tagger-v2.onnx (789 MB)
+ camie-tagger-v2-metadata.json (7.77 MB) at root, NOT model.onnx +
selected_tags.csv. Tags ship as nested JSON (dataset_info.tag_mapping)
not CSV. Per the published onnx_inference.py reference: input is NCHW
not NHWC, normalize with ImageNet mean/std, pad-square color (124,116,
104), sigmoid the second output (refined predictions) not the first.

Operator hit this during the IR migration ML backfill — download_models
silently fetched only 3 json files (allow_patterns matched nothing
useful), tagger.load() then raised RuntimeError. Fetched the actual
v2 layout via WebFetch, rewrote tagger to match published reference.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 02:25:30 -04:00
bvandeusen 9d5abb09f6 fix(maintenance): recover_interrupted_tasks also sweeps pending/queued orphans (>30 min) to failed 
scan_directory creates ImportTask rows with status='pending' (commit) then
in a second pass transitions to 'queued' + .delay() (commit). Crashes in
that window leave rows orphaned with no recovery path. Operator hit 5490
such rows 2026-05-25; the existing sweep only handled 'processing'.
Flipping to 'failed' (not re-enqueue) lets the operator drain via the
existing /api/import/retry-failed endpoint at their own pace.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 02:00:29 -04:00
bvandeusen 832345a245 fix(fc3k): add origin=imported_filesystem to test ImageRecord ctors (second NOT NULL column after mime) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 01:28:54 -04:00
bvandeusen a0136fa30d fix(fc3k): add mime=image/jpeg to test ImageRecord ctors (NOT NULL) + reorder admin import after stdlib/3rd-party (ruff I001) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 01:12:52 -04:00
bvandeusen de1a4b64b7 fc3k(ui): TagMaintenanceCard — preview-then-commit prune-unused under Settings → Maintenance 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:52:30 -04:00
bvandeusen 3f500e592e fc3k(ui): per-tag dots-menu with Merge + Delete actions; Tier-B count-surfacing modal 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:51:51 -04:00
bvandeusen d97e3f9b59 fc3k(ui): bulk-delete action in BulkEditorPanel with sha8 confirm token + projected counts modal 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:49:45 -04:00
bvandeusen 035c49f675 fc3k(ui): ArtistDangerZone card + slot at bottom of ArtistView with cascade-delete flow 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:48:17 -04:00
bvandeusen e41ab1cca5 fc3k(ui): Pinia admin store — six endpoints + task_run polling for Tier-C dispatched ops 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:47:09 -04:00
bvandeusen 42c6b642c2 fc3k(ui): rename + relocate BackupConfirmModal → modal/DestructiveConfirmModal; add tier + projectedCounts props 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:46:37 -04:00
bvandeusen ad3d34a1fc fc3k: /api/admin endpoint integration tests — dry-run, confirm-mismatch, dispatch, Tier-A/B/C paths 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:45:31 -04:00
bvandeusen b5289ed372 fc3k: admin Celery task tests — registration, success, failure, missing-id idempotency 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:44:33 -04:00
bvandeusen f6aa805725 fc3k: cleanup_service unit tests — projections + mutations + file unlinks against real Postgres 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:43:51 -04:00
bvandeusen f096c9a5fb fc3k: register admin_bp in api/__init__ all_blueprints 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:42:40 -04:00
bvandeusen 676a86b514 fc3k: /api/admin tags prune-unused endpoint — Tier-A preview-then-commit flow 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:42:20 -04:00
bvandeusen 44cc625d4a fc3k: /api/admin tag endpoints — Tier-B delete + merge + usage-count helper 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:42:03 -04:00
bvandeusen f7ee122243 fc3k: /api/admin blueprint — Tier-C artist cascade + bulk image delete with sha8-keyed confirm tokens 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:40:53 -04:00
bvandeusen 7c6f11964a fc3k: celery_app — register admin tasks on maintenance queue 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:40:23 -04:00
bvandeusen 94c60c0af2 fc3k: admin Celery tasks — delete_artist_cascade_task + bulk_delete_images_task on maintenance queue 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:40:03 -04:00
bvandeusen 6df102b83d fc3k: cleanup_service mutations — unlink primitive, artist cascade, bulk image delete, tag delete, prune unused 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:39:45 -04:00
bvandeusen 2ae01d27e3 fc3k: cleanup_service projections — artist cascade, bulk delete, tag usage, unused tags 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 00:38:57 -04:00
bvandeusen 718cc79905 fix(fc3h): split semicolon-stacked statements (E702) and bridge v-dialog v-model to avoid prop write 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:37:06 -04:00
bvandeusen e78a35d333 fc3h: collapse multi-line sqlalchemy import in backup_run.py — fits under line-length=100, ruff I001 would bounce 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:12:06 -04:00
bvandeusen 83bd3b4b2d fc3h(ui): slot BackupCard into Maintenance panel above migration card 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:06:22 -04:00
bvandeusen aecedd9fe4 fc3h(ui): BackupCard.vue + BackupRunsTable.vue — combined card with DB + Images sub-sections 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:06:00 -04:00
bvandeusen 1e34b1b428 fc3h(ui): BackupConfirmModal.vue — typed-token confirmation for restore + delete 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:05:04 -04:00
bvandeusen 102c21feaa fc3h(ui): Pinia backup store — runs, triggers, restore, delete, tag, settings 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:04:37 -04:00
bvandeusen 57a338f7e6 fc3h(tests): drop pinned migration-backup tests (retired surface; coverage moved to test_backup_service.py + test_api_system_backup.py) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:03:57 -04:00
bvandeusen d04983138a fc3h: /api/system/backup endpoint integration tests 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:02:32 -04:00
bvandeusen 9ec6fdb596 fc3h: Celery task integration tests (backup, restore, prune, nightly) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:01:39 -04:00
bvandeusen 86ad9b80e9 fc3h: backup_service unit tests (subprocess monkeypatched) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:00:40 -04:00
bvandeusen 2b05f147f4 fc3h: migrators docstring — note backup/rollback retired to backup_service.py 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 23:00:01 -04:00
bvandeusen 70e1e010d1 fc3h: remove backend/app/services/migrators/backup.py + rollback.py (relocated to backup_service.py) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:59:18 -04:00
bvandeusen d3d4320ed5 fc3h: retire backup + rollback from migrate API/task — moved to /api/system/backup/* per FC-3h 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:59:11 -04:00
bvandeusen 7d42cddb11 fc3h: /api/system/backup blueprint — trigger, list, get, patch, restore, delete, settings 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:57:25 -04:00
bvandeusen 1f01c4819a fc3h: celery_app — register backup tasks (include + maintenance route + Beat hourly tick + daily prune) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:56:06 -04:00
bvandeusen 06d527cb92 fc3h: prune_backups (daily retention) + backup_db_nightly (hourly tick, settings-gated) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:55:19 -04:00
bvandeusen e9ea376aed fc3h: restore_db_task + restore_images_task — restore creates 'restoring' marker row linked via restored_from_id 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:53:50 -04:00
bvandeusen 882cb491ba fc3h: backup_db_task + backup_images_task Celery tasks 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:53:26 -04:00
bvandeusen 319e7de547 fc3h: backup_service.py — DB + images backup/restore + unlink helpers (relocated, split per-kind) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:52:35 -04:00
bvandeusen e43312a129 fc3h: ImportSettings backup_* knobs + alembic 0018 (nightly-enabled, hour, keep-N per kind) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:52:00 -04:00
bvandeusen 8f2732a56f fc3h: alembic 0017 — backup_run table with indexes + partial-tag 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:51:22 -04:00
bvandeusen c3e855bd9b fc3h: BackupRun model — artifact record for backup/restore runs 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:50:59 -04:00
bvandeusen 37fcc74954 fix(fc3i): single-line celery.signals import + INT32 bounds on target_id + dict.fromkeys + rewrite retry test as direct _finalize unit 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 21:22:42 -04:00
bvandeusen 6a532c1497 fc3i(ui): Settings → Activity tab + Overview summary card 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 21:10:14 -04:00
bvandeusen 368613068a fc3i(ui): SystemActivityTab.vue — queues + failures + all-activity panes 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 21:09:20 -04:00