FabledCurator

bvandeusen/FabledCurator

Fork 0

RSS Feed

v26.05.25.3 ac55d0e8d8

Compare
v26.05.25.3 — ML embedder SigLIP fix, import-UX, extension publish wiring Stable

bvandeusen released this 2026-05-25 19:37:09 -04:00 | 644 commits to dev since this release
What's new

ML worker no longer crashes on SentencePiece import
- embedder.py now uses SiglipImageProcessor.from_pretrained() (image-only) instead of AutoProcessor.from_pretrained() — the full processor pulled in SiglipTokenizer which required sentencepiece at import time and killed the ml-worker before any tag_and_embed could run. FC's embedder is image-only by design.
Import-tab UX overhaul
- Deep scan button alongside Quick scan — backend already accepted mode: 'deep' (chains a backfill_phash after the walk); now exposed in the UI.
- /api/import/status now returns source_path + scan_mode — fixes the "Scanning undefined —" banner.
- Settings tabs strip is now position: sticky; top: 64px — no more scrolling to the top of the page to switch tabs.
- Import tab reorders to Trigger → Tasks → Filters so hit/miss feedback sits directly under the buttons.
- Toast wording adapts per mode; task list re-fetches 2s after trigger.
Browser-extension publish workflow corrected
- Extension bumped to 1.0.1 to trigger initial sign-and-publish.
- Workflow's commit step previously globbed fabledcurator-*.xpi but AMO renames signed XPIs to use its addon-id-safe-string (e.g. 997017ca3e104e30a75a-1.0.1.xpi) — now globs *.xpi and canonicalizes the copy to fabledcurator-<version>.xpi so the backend's whitelist keeps working.
- Workflow now self-retriggers on edits to its own file (path filter includes .forgejo/workflows/extension.yml) — no more manual dispatch on workflow fixes.
Known follow-ups (next release)
- The signed XPI side-commit didn't actually land on main in this cycle despite the workflow showing success on run #309 — investigating; expect a v26.05.25.4 with the bundled extension once root-caused.
- CI speedup via pre-baked requirements.txt in ci-python:3.14 image + job-shard of the integration suite across the 6-capacity runner (operator just expanded act_runner capacity 2→6).
Verification post-deploy
- Restart ml-worker → fire any tag_and_embed — should no longer ImportError on sentencepiece.
- Settings → Import: Deep scan button present, tabs stay pinned to top of viewport while scrolling Maintenance panel.
🤖 Generated with Claude Code
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.25.2 4e9aac2c05

Compare
v26.05.25.2 — supersede + sidecar enrichment, scan UI feedback, CI uv + cache Stable

bvandeusen released this 2026-05-25 14:30:50 -04:00 | 855 commits to main since this release
Patch release on top of v26.05.25.1. See PR #12 for the change list.

Operator-facing win

Scanning the GS download dir now enriches superseded images with the GS sidecar's post metadata. Previously _supersede() was file-only — bigger version of an IR-migrated image would replace the file but lose the gallery-dl JSON sidecar's Post/Source/ImageProvenance entirely. The behavior is additive: any existing IR-migration provenance survives, and the new GS-sidecar provenance gets added as a second ImageProvenance row that the modal surfaces.

UI
- Scan trigger button now produces visible feedback (immediate "Scan triggered" toast + 2s status refresh + "Scan complete — no new files" toast when the scan quick-finalizes against an already-imported library).
CI
- pip wheel cache + uv pip install --system when available — ~2 min saved on warm runs.
- pytest --durations=25 on integration job to expose slowest tests for future targeted optimization.
Schema changes

None.

Next step for the operator

Now that this is on main + deployed, run a deep scan (Settings → Import → Quick scan with deep mode, or curl -X POST /api/import/trigger -d '{"mode":"deep"}'). For each GS file at /import/...:
- Same SHA256 as an existing FC row → _deep_rederive backfills phash + sidecar
- Phash matches a smaller existing IR-migrated image → _supersede swaps file AND applies sidecar
- Brand new → fresh import with sidecar
🤖 Generated with Claude Code
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.25.1 2879ac6f2b

Compare
v26.05.25.1 — maintenance sweeps, Camie v2, corrupt-file handling, post-date gallery Stable

bvandeusen released this 2026-05-25 13:04:39 -04:00 | 860 commits to main since this release
Patch release on top of v26.05.25.0. See PR #11 for the full change list.

Operator-facing wins
- Gallery now sorts by original post date when known — IR-migrated content surfaces at its actual publish year instead of piling up in one FC-scan-week bucket. Re-run /api/migrate/tag_apply to backfill primary_post_id on existing migrated images.
- DeviantArt + Pixiv PostMetadata is no longer silently dropped during tag_apply phase 4 — one of two root causes of the missing-provenance complaint.
- Corrupt files in /import no longer Celery-autoretry-loop forever — PIL OSError now skips as invalid_image cleanly.
- Stuck batches are recoverable from the UI: new "Clear stuck..." button on the import task list, plus an inline action next to the scanning spinner. No more SQL needed.
- Quick scan button is always visible (was hidden by a wedged active batch in the old UI).
- Camie v2 ML tagger actually works — download_models.py was fetching the wrong filenames; tagger.py had wrong preprocessing + wrong output handling. Aligned with the published onnx_inference.py reference.
- Directory card UI: 2× width, bounded preview slot heights so tall source images can't escape.
Maintenance / recovery
- recover_interrupted_tasks Beat sweep now flips pending/queued orphans (>30 min) to failed in addition to recycling processing rows. Fixes the 5,490-row pile a scan_directory crash can leave behind.
- /api/import/clear-stuck manual escape hatch for the autoretry-loop case.
Schema changes

None. All fixes are logic-only.

Recovery steps

After deploying:
1. Re-run /api/migrate/tag_apply on the existing manifest — phase 4 is idempotent; backfills primary_post_id and picks up previously-skipped DeviantArt + Pixiv entries.
2. Gallery + timeline + jump immediately reflect publish dates.
🤖 Generated with Claude Code
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.25.0 b8dce6c483

Compare
v26.05.25.0 — Backup first-class + admin destructive actions Stable

bvandeusen released this 2026-05-25 01:47:52 -04:00 | 870 commits to main since this release
Two roadmap stages cut together. See PR #10 for the full change list.

FC-3h — Backup as first-class service

Backup promoted out of the migration tooling into a permanent FC feature, split into independent DB + image components.
- BackupRun model + alembic 0017/0018
- backend/app/services/backup_service.py (pg_dump / tar+zstd / restore / unlink)
- 6 Celery tasks on maintenance queue (autoretry, per-kind soft/hard time limits)
- Beat: hourly nightly tick (settings-gated, default OFF) + daily retention prune
- /api/system/backup/* blueprint with typed-token confirmation on restore + delete
- New Settings → Maintenance card with DB + Images sub-sections
FC-3k — Admin destructive actions

Five first-class destructive actions, surviving FC-3j cleanup of the migration tooling.
- cleanup_service.py (artist cascade + image unlink + tag delete + prune-unused, lifted from migrators/cleanup.py)
- 2 Celery tasks for long-running ops (delete-artist-cascade, bulk-delete-images)
- /api/admin/* blueprint (6 endpoints) with graduated confirmation friction (Tier A single-click / Tier B yes/no with count / Tier C typed-token)
- ArtistDangerZone.vue, BulkEditorPanel "Delete selected", per-tag dots-menu (Merge + Delete), TagMaintenanceCard.vue (prune-unused preview)
- BackupConfirmModal generalized to DestructiveConfirmModal and shared between FC-3h + FC-3k
Inheritance

Both stages inherit FC-3i's task_run lifecycle (recovery sweep, daily prune, soft/hard time limits, duration tracking) for the long-running Celery tasks. No new lifecycle plumbing.

Schema
- alembic 0017 — backup_run table
- alembic 0018 — backup_* knobs on import_settings
Run alembic upgrade head after pulling.

What's not in this release
- IR migration verification (operator-driven manual check before FC-3j)
- FC-3j (remove migration tooling) — depends on verification
- FC-3g extension verification
🤖 Generated with Claude Code
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.24.3 d1c0b82a22

Compare
v26.05.24.3 — FC-3i System Activity dashboard Stable

bvandeusen released this 2026-05-24 21:48:19 -04:00 | 707 commits to dev since this release
FC-3i ships: IR-style operator visibility into every Celery lane, plus two dogfood fixes that unblock the IR migration UI.

FC-3i — System Activity Overview
- New task_run table populated automatically by Celery signal handlers; no per-task instrumentation.
- Recovery sweep + retention sweep on Beat (5 min / daily).
- /api/system/activity/{queues,workers,runs,failures} endpoints.
- Settings → Activity tab with queues+workers pane, recent failures pane (24h, clickable error-type pills), full activity table with queue/status filters.
- Quick summary card on Settings → Overview.
Migration UX unblock
- Backup-gate retired on the migration API; IR/GS ingest from the UI now works without a recent backup. FC-3h will rewrite backup as a first-class feature.
Modal Escape
- Document-level keydown listener so Escape closes the viewer modal even when a <video> element has focus.
Operator workflow
1. Pull the new image; force-update app, worker, ml-worker, scheduler, web.
2. Alembic auto-applies migration 0016 (creates task_run).
3. Within 5 min, task_run rows accumulate for every task that fires.
4. Settings → Activity tab live-tracks queue depths, worker counts, recent failures across all lanes.
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.24.2 5526b8dc78

Compare
v26.05.24.2 — IR Post/Provenance restore + modal artist fallback Stable

bvandeusen released this 2026-05-24 14:30:26 -04:00 | 724 commits to dev since this release
Extends the IR migration so post-level data (title, description, source URL, publish date, attachment count) survives IR shutdown, and adds a modal-side artist fallback so IR-imported images without ImageProvenance still show a "by " line.

IR Post/Provenance restore
- IR export script bumped to schema v2; dumps PostMetadata as image_posts with per-post sha256 lists.
- ir_ingest.py accepts schema 1 or 2; carries image_posts into the manifest.
- tag_apply.py new phase 4: find-or-creates Source + Post + ImageProvenance per IR PostMetadata entry. Idempotent, dry-run safe.
Modal artist fallback
- /api/gallery/image/<id> now returns artist: {id, name, slug} when image_record.artist_id is set.
- ProvenancePanel.vue shows a "by " line when ImageProvenance is empty but a direct artist FK exists.
Operator workflow
1. Pull this image; force-update services.
2. Deploy a one-shot ir-export service inside the IR stack (see chat for snippet) → runs scripts/export_for_fabledcurator.py --output /staging/ir-export-v2.json.
3. Settings → Maintenance → Legacy migration → upload the new JSON to the IR slot.
4. Click "3. Ingest IR" (works now thanks to the FormData fix in v26.05.24.1).
5. Click "5. Apply IR tags" — phase 4 fires alongside phases 1-3.
6. Open any image modal: ImageProvenance entries render for images-in-posts; artist fallback line renders for everything else.
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.24.1 16eb7075c4

Compare
v26.05.24.1 — FC-3g Firefox extension + worker resilience Stable

bvandeusen released this 2026-05-24 12:52:52 -04:00 | 727 commits to dev since this release
Big release rolling up FC-3g (Firefox extension), worker-level recovery hardening, and several dogfood fixes surfaced during the IR migration.

Headline — FC-3g: Firefox extension

Plain-JS MV3 Firefox extension under extension/. Six platforms (Patreon, SubscribeStar, Hentai-Foundry, Discord, Pixiv, DeviantArt). Popup with per-platform Export + Export All + Sources tab. Floating "Add to FabledCurator" button on artist pages. New backend /api/extension/quick-add-source + /api/extension/manifest + /extension/<filename> (serves signed XPI with application/x-xpinstall MIME). New BrowserExtensionCard.vue on Settings → Maintenance. CI workflow signs via Mozilla AMO (unlisted) on main pushes + commits XPI back so next build bundles it into the Docker image.

Worker resilience
- Recovery sweep tightened 30min → 5min — flips stuck processing rows automatically.
- import_media_file body-wrapped so no exit leaves rows in processing.
- Celery autoretry_for (OperationalError, DBAPIError, OSError) + exponential backoff on import_media_file, generate_thumbnail, tag_and_embed, download_source.
- Per-task soft_time_limit + time_limit safety nets.
Migration UX
- FormData passthrough in useApi — IR/GS export-file UI uploads now actually work.
- Backup subprocess 30-min wall-clock timeout (FC-3h queued for the full promotion).
UI
- Global .fc-content padding-top so view content doesn't start hidden behind the sticky TopNav.
CI / lint
- extension.yml YAML schema fix.
- ASYNC240 — /api/extension/manifest filesystem work moved to asyncio.to_thread.
Test hygiene
- test_recover_interrupted_only_old unpinned from the threshold value.
Operator workflow
- After pulling: force-update app, worker, ml-worker, scheduler, web.
- 2224 stuck-processing rows from the IR migration will recover within 5 min of scheduler coming up.
- First signed XPI lands ~5-10 min after this release (Mozilla signing + bot commit triggers a second build that bundles the XPI). After that, the Browser Extension card on Settings → Maintenance goes from "No bundled extension found" to the "Install Firefox extension" CTA.
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.24.0 885dcf64f3

Compare
v26.05.24.0 — TopNav re-fix Stable

bvandeusen released this 2026-05-23 22:49:44 -04:00 | 747 commits to dev since this release
Single-commit follow-up to v26.05.23.3's TopNav layout attempt.

Fix

The 1fr auto 1fr grid layout shipped in v26.05.23.3 wasn't actually symmetric. CSS Grid treats 1fr as minmax(auto, 1fr), so a side cell whose intrinsic content (brand on the left, action button on the right) exceeded its 1fr share quietly grew past it. On narrower viewports the centered link block still shifted when Gallery or Showcase rendered their teleport-slot action buttons.

The replacement is a flex layout with explicit equal-weight side cells (flex: 1 1 0 on both .fc-nav-left and .fc-nav-actions), keeping the middle anchored regardless of side content widths.

Operator notes
- Pull and force-update web (frontend) — the only change is in frontend/src/components/TopNav.vue.
- Hard-refresh the browser (Ctrl+Shift+R) to pick up the new content-hashed CSS bundle.
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.23.3 f2f6b6d25e

Compare
v26.05.23.3 — dogfood UX polish + accurate active-batch stats Stable

bvandeusen released this 2026-05-23 22:06:22 -04:00 | 749 commits to dev since this release
Patch release rolling up five dogfood-driven fixes from the live IR migration.

UI fixes
- Poll task list during scan: Recent Import Tasks list now refreshes every 5 s while a batch is in flight, so the table tracks the counter that's already polling.
- TopNav grid layout: switching to Gallery or Showcase no longer shifts the centered links — the nav now uses a 1fr / auto / 1fr grid with mirrored side columns so contextual action buttons (Select / Shuffle) don't rebalance the layout.
- Viewer modal haze + fit-to-view: the modal background is now mid-opacity obsidian + backdrop-filter: blur(8px) (matches TopNav haze) and large images fit the viewport at zoom 1× instead of overflowing.
- Viewer modal close-on-click-outside: IR-parity. Click on the image still toggles zoom; click on the surrounding haze closes the modal.
Backend fix
- Auto-finalize empty scans + accurate active-batch stats: a re-triggered scan that finds every file already on a non-failed ImportTask no longer leaves the batch row stuck in running forever. scan_directory flips the empty batch to complete immediately; /api/system/stats filters its active-batch query to running batches that still have outstanding ImportTask work, so the UI tracks the batch actually doing the import instead of the most recently-triggered empty one.
Operator notes
- If you have a previous empty batch stuck in running (counters all zero), one-shot SQL fix: UPDATE import_batch SET status='complete', finished_at=NOW() WHERE total_files=0 AND status='running';
- Force-update app to pick up the /api/system/stats query change.
- Force-update web (frontend) for the four UI fixes.
- Force-update scheduler to pick up the scan_directory auto-finalize behavior.
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)
v26.05.23.2 0822240fde

Compare
v26.05.23.2 — serve /images + artist cleanup migrator Stable

bvandeusen released this 2026-05-23 12:19:31 -04:00 | 755 commits to dev since this release
Patch release rolling up two post-migration fixes.

Fixes
- Serve /images/<path> from disk: without this route, the SPA catch-all swallowed every thumbnail/original URL and returned index.html, causing the aspect-ratio-shaped grey placeholders in Showcase and Gallery. Added a guarded send_from_directory route registered before the SPA fallback.
Features
- /api/migrate/cleanup migrator: targeted destructive cleanup — deletes every image_record attributed to one artist (CASCADE handles image_tag, image_provenance, series_page, tag_suggestion_rejection), unlinks originals + thumbnails on disk, optionally sweeps import_task rows under a given source_path_prefix, removes empty import_batch rows, and finally deletes the artist row. Built for the IR-migration rescue case where the scan derived a bogus imagerepo artist from a mismatched bind-mount layout. Honors dry_run for a safe preview; requires force: true to bypass the pre-migration backup guard.
Operator workflow for the IR re-do

After pulling this image and force-updating services:
```
# 1. Dry-run to verify scope (no writes)
curl -sX POST http://curator.traefik.internal/api/migrate/cleanup \
  -H 'Content-Type: application/json' \
  -d '{"slug":"imagerepo","dry_run":true,"force":true}'

# 2. Actually wipe the IR import
curl -sX POST http://curator.traefik.internal/api/migrate/cleanup \
  -H 'Content-Type: application/json' \
  -d '{"slug":"imagerepo","force":true,"source_path_prefix":"/import/imagerepo"}'

# 3. Poll until complete
curl -s http://curator.traefik.internal/api/migrate/runs/<id> | jq
```
Then change the IR bind mount in docker-compose.yml from /var/lib/imagerepo/images:/import/imagerepo:ro → /var/lib/imagerepo/images:/import:ro so the scan sees <artist>/<file> as the top level. Re-trigger scan, then re-run tag_apply (the manifest at /images/_migration_state/ir_tag_manifest.json survives the cleanup).
Downloads
- Source Code (ZIP)
- Source Code (TAR.GZ)

1 2 3 4

33 Releases 33 Tags

v26.05.25.3 — ML embedder SigLIP fix, import-UX, extension publish wiring Stable

What's new

Known follow-ups (next release)

Verification post-deploy

v26.05.25.2 — supersede + sidecar enrichment, scan UI feedback, CI uv + cache Stable

Operator-facing win

UI

CI

Schema changes

Next step for the operator

v26.05.25.1 — maintenance sweeps, Camie v2, corrupt-file handling, post-date gallery Stable

Operator-facing wins

Maintenance / recovery

Schema changes

Recovery steps

v26.05.25.0 — Backup first-class + admin destructive actions Stable

FC-3h — Backup as first-class service

FC-3k — Admin destructive actions

Inheritance

Schema

What's not in this release

v26.05.24.3 — FC-3i System Activity dashboard Stable

FC-3i — System Activity Overview

Migration UX unblock

Modal Escape

Operator workflow

v26.05.24.2 — IR Post/Provenance restore + modal artist fallback Stable

IR Post/Provenance restore

Modal artist fallback

Operator workflow

v26.05.24.1 — FC-3g Firefox extension + worker resilience Stable

Headline — FC-3g: Firefox extension

Worker resilience

Migration UX

UI

CI / lint

Test hygiene

Operator workflow

v26.05.24.0 — TopNav re-fix Stable

Fix

Operator notes

v26.05.23.3 — dogfood UX polish + accurate active-batch stats Stable

UI fixes

Backend fix

Operator notes

v26.05.23.2 — serve /images + artist cleanup migrator Stable

Fixes

Features

Operator workflow for the IR re-do