Compare commits

..
Author SHA1 Message Date
bvandeusen ca6d26d236 Merge pull request 'feat(translation): acceptance gate on Interpreter detection confidence (#1376)' (#208) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m43s
2026-07-09 17:58:22 -04:00
bvandeusen 5366047d55 Merge pull request 'feat(translation): 8h sweep + drain button + detection probe (#1376)' (#207) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m50s
2026-07-08 20:19:33 -04:00
bvandeusen 2923257529 Merge pull request 'fix(ui): tag-chip left-edge clipping + kind-coloured border' (#206) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m45s
2026-07-08 10:37:09 -04:00
bvandeusen 934731e9ef Merge pull request 'feat(translation): per-field language detection for mixed-language posts' (#205) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m48s
2026-07-08 09:25:23 -04:00
bvandeusen 4c67c116b2 Merge pull request 'Bolder auto-tag accept/reject + larger tag chips' (#204) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m50s
2026-07-07 23:21:26 -04:00
bvandeusen f8b667604f Merge pull request 'Graceful shutdown + resilience & Explore polish (m147)' (#203) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m48s
2026-07-07 22:53:34 -04:00
bvandeusen 11572f469a Merge pull request 'Re-translate on Interpreter model change — artist-scoped + global (m146)' (#202) from dev into main
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 11s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m41s
2026-07-07 20:22:40 -04:00
bvandeusen ba3fd2a118 Merge pull request 'Post translation via Interpreter (milestone 143)' (#201) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m44s
2026-07-07 13:05:18 -04:00
bvandeusen 9bf095179a Merge pull request 'Auto-tag accept/reject as an in-pill yes/no pair; confirm refocuses the tag input' (#200) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m44s
2026-07-07 01:13:02 -04:00
bvandeusen cefc064e0f Merge pull request 'Surface auto-hide misfires proactively (review strip no longer gated on Show-hidden)' (#199) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m42s
2026-07-07 00:43:45 -04:00
bvandeusen 06757daf80 Merge pull request 'Presentation-chrome auto-hide (milestone 141) + tagging-UX fixes' (#198) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m46s
2026-07-07 00:24:51 -04:00
bvandeusen a723ef436b Merge pull request 'Soft auto-apply (retract + confirm, no self-training) + tagging UX (reject-rest, tag-input race, modal playlist)' (#197) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-agent (push) Successful in 9s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m46s
2026-07-06 21:03:30 -04:00
bvandeusen d68f39ca50 Merge pull request 'Suggestions perf + UX polish: incremental CCIP + head training, hover label, scroll/toast fixes, dead-preview removal' (#196) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m43s
2026-07-06 16:41:08 -04:00
bvandeusen a66a695977 Merge pull request 'Tag-hover → crop-region grounding overlay (#1206, milestone #133)' (#195) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 17s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 40s
CI / integration (push) Successful in 3m51s
2026-07-06 13:29:31 -04:00
bvandeusen 5289fa3879 Merge pull request 'Crop-proposer detector config → Settings → Tagging (default-on, lease-announced) (#134)' (#194) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m40s
2026-07-05 20:10:02 -04:00
bvandeusen ebac34e17b Merge pull request 'Artist identity/storage/display decoupling — rename, move-source, add-time name resolution (#130)' (#193) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m40s
2026-07-04 22:43:12 -04:00
bvandeusen 4c3ba20198 Merge pull request 'Recapture image→post linking (#1288) + ugoira timings + system suggestion category' (#192) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 41s
CI / integration (push) Successful in 3m36s
2026-07-04 20:36:58 -04:00
bvandeusen d2acec61ae Merge pull request 'Pixiv native ingester (#129) + SubscribeStar .art fix, disable-clears-failure, system-tag suggestion floor' (#191) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 39s
CI / integration (push) Successful in 3m35s
2026-07-03 16:36:56 -04:00
bvandeusen 15ae5ef4fa Merge pull request 'SubscribeStar .art age fix, modal tag-flow sync, system tags / training hygiene (#128)' (#190) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 9s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m34s
2026-07-03 09:16:16 -04:00
bvandeusen e3ceefc820 Merge pull request 'Surgical re-fetch for deep posts (ExternalLink reset) — recovery no longer needs deep scans' (#189) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 9s
Build images / build-agent (push) Successful in 14s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 38s
CI / backend-lint-and-test (push) Successful in 2m10s
CI / integration (push) Successful in 3m40s
2026-07-02 21:14:45 -04:00
bvandeusen e52090a4cf Merge pull request 'Agent sleep mode: empty queue sheds to one downloader, lease poll backs off to 15 min' (#188) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 8s
CI / frontend-build (push) Successful in 21s
Build images / build-web (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 42s
CI / integration (push) Successful in 3m50s
2026-07-02 20:37:48 -04:00
bvandeusen bfba8045e4 Merge pull request 'Admin IA overhaul, cap-aware autoscaler, optional ml-worker, hardened tag reset' (#187) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 40s
CI / integration (push) Successful in 3m29s
2026-07-02 18:01:56 -04:00
bvandeusen a708357436 Merge pull request 'Settings defaults, GPU error tombstones + failure triage/recovery, agent bandwidth cap, approved retirements' (#186) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-agent (push) Successful in 9s
Build images / build-ml (push) Successful in 10s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m33s
2026-07-02 15:48:48 -04:00
bvandeusen d9354ac1e1 Merge pull request 'Agent: short-video sampling fix + "Retry errored jobs" recovery action' (#185) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m26s
2026-07-01 21:13:48 -04:00
bvandeusen 1d48770793 Merge pull request 'Agent: fix Status freeze — conchint.textContent destroyed #capn (root cause)' (#184) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
2026-07-01 20:46:28 -04:00
bvandeusen 489e6aaaee Merge pull request 'Agent: server-side throughput rates + killable-on-stop ffmpeg' (#183) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 8s
Build images / build-agent (push) Successful in 9s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m26s
2026-07-01 20:01:00 -04:00
bvandeusen ed20df905b Merge pull request 'Agent: DRY pass + Status rate-metrics + real start/stop state machine' (#182) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
2026-07-01 16:30:32 -04:00
bvandeusen 5ba9871ef0 Merge pull request 'fix(agent): stream videos via ffmpeg-from-URL (no full download) — env-agnostic' (#181) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m27s
2026-07-01 14:20:53 -04:00
bvandeusen 2a820d0848 Merge pull request 'fix(agent): log the real fetch/submit failure reason (diagnose #1225)' (#180) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m28s
2026-07-01 12:37:45 -04:00
bvandeusen 2f9aa3d86c Merge pull request 'perf(web): 4 MiB file streaming + 4 hypercorn workers (fix 40s downloads)' (#179) from dev into main
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 29s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 6s
CI / integration (push) Successful in 3m26s
2026-07-01 11:55:35 -04:00
bvandeusen 560a5000a2 Merge pull request 'gallery: sort by earliest post date across all posts (new default)' (#178) from dev into main
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Successful in 4s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m26s
2026-07-01 10:54:41 -04:00
bvandeusen 7ddad231f8 Merge pull request 'agent: stop downloader pool stampeding a slow curator (congestion collapse)' (#177) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 8s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 7s
CI / integration (push) Successful in 3m22s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 35s
2026-07-01 10:45:48 -04:00
bvandeusen a78f7eaace Merge pull request 'explore: more variance in the related rail (stronger MMR diversification)' (#176) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-agent (push) Successful in 8s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m25s
2026-07-01 00:50:59 -04:00
bvandeusen 71337b0ba4 Merge pull request 'agent: temporal video dedup — drop near-duplicate frames before the GPU' (#175) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
2026-07-01 00:38:53 -04:00
bvandeusen 3996205f3b Merge pull request 'agent: dedupe near-duplicate crops before the SigLIP embed' (#174) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 5s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m26s
2026-07-01 00:24:39 -04:00
bvandeusen 9f9db01456 Merge pull request 'agent: download/GPU producer-consumer pipeline + detector fuse fix' (#173) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m25s
2026-06-30 23:39:25 -04:00
bvandeusen 216b7fc743 Merge pull request 'Agent: bound video GPU work (early-exit frame loop at max_regions)' (#172) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m26s
2026-06-30 22:57:33 -04:00
bvandeusen fbb76e6f36 Merge pull request 'Agent: huge-image load + figure/region caps + stale-active fix' (#171) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
2026-06-30 22:47:48 -04:00
bvandeusen 5ef1478ade Merge pull request 'Modernise agent base → Ubuntu 24.04 / Python 3.12 / CUDA 12.9' (#170) from dev into main
Build images / build-ml (push) Failing after 3s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 41s
CI / integration (push) Successful in 3m22s
Build images / build-agent (push) Successful in 8m44s
2026-06-30 22:30:18 -04:00
bvandeusen 88ff4147e1 Merge pull request 'Fix: agent py3.10 startup crash + submit-path retry + pin agent ruff to py310' (#169) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m25s
2026-06-30 22:03:57 -04:00
bvandeusen 1ea02ad44c Merge pull request 'Release: prompt agent stop + lazy curator polling + build marker + agent in CI' (#168) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
Build images / build-web (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m23s
2026-06-30 21:42:55 -04:00
bvandeusen 937cfb65b4 Merge pull request 'Release: agent per-stage timing breakdown' (#167) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
CI / backend-lint-and-test (push) Successful in 25s
Build images / build-web (push) Successful in 5s
CI / frontend-build (push) Successful in 18s
CI / integration (push) Successful in 3m23s
2026-06-30 21:28:57 -04:00
bvandeusen baac851220 Merge pull request 'perf: GPU-job leasing stays O(batch) — partial indexes + two-phase lease' (#166) from dev into main
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 9s
Build images / build-ml (push) Successful in 9s
CI / integration (push) Successful in 3m27s
CI / lint (push) Successful in 4s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
2026-06-30 21:18:23 -04:00
bvandeusen a28c33281a Merge pull request 'Release: unfreeze agent status view + smoothed throughput autoscaler + log pane' (#165) from dev into main
CI / lint (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 9s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 16s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m23s
2026-06-30 20:20:34 -04:00
bvandeusen 40be0a9323 Merge pull request 'Release: agent page no-store cache header' (#164) from dev into main
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Failing after 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 16s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m23s
2026-06-30 20:00:36 -04:00
bvandeusen 2c6bf26bfc Merge pull request 'Release: graceful Start/Stop (starting/stopping states) + cached GPU reads' (#163) from dev into main
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 16s
CI / backend-lint-and-test (push) Successful in 26s
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
CI / integration (push) Successful in 3m24s
2026-06-30 19:38:50 -04:00
bvandeusen 3c1e76bd44 Merge pull request 'Release: stable util-band autoscaler + live GPU meters' (#162) from dev into main
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 5s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 16s
Build images / build-web (push) Successful in 6s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m44s
2026-06-30 19:16:29 -04:00
bvandeusen 831c5b1c10 Merge pull request 'Release: agent page — centered column + logs fill to bottom' (#161) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
2026-06-30 19:10:50 -04:00
bvandeusen 76b6af4903 Merge pull request 'Release: agent perf (batched crop embeds) + UI (full-width, copy logs, quiet noise) + truncated-image fix' (#160) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 17s
Build images / build-web (push) Successful in 6s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m32s
2026-06-30 18:47:55 -04:00
bvandeusen 92494ec4ed Merge pull request 'Release: agent control-page logs console + styling pass' (#159) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 7s
CI / integration (push) Successful in 3m26s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
2026-06-30 18:37:35 -04:00
bvandeusen bdfc17477c Merge pull request 'Release: GPU agent autoscaler (throughput hill-climb, Auto default-on)' (#158) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m24s
2026-06-30 18:23:15 -04:00
bvandeusen 6cd3153bf4 Merge pull request 'Release: SigLIP 2 default (new installs) + embedder model dropdown' (#157) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m26s
2026-06-30 18:01:01 -04:00
bvandeusen 5f2853168a Merge pull request 'Release: GPU re-process trigger (apply new crop detectors to the existing library)' (#156) from dev into main
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m22s
Build images / build-agent (push) Successful in 3m35s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
2026-06-30 16:13:36 -04:00
bvandeusen 9a979ee808 Merge pull request 'Release: Explore diversification, model-swap, Camie/centroid retirement, tag-API, crop proposers' (#155) from dev into main
Build images / build-agent (push) Successful in 10m56s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 29s
Build images / build-web (push) Successful in 2m21s
Build images / build-ml (push) Successful in 2m42s
CI / integration (push) Successful in 3m26s
2026-06-30 16:06:13 -04:00
bvandeusen 3138f912fd Merge pull request 'Release: SigLIP concept crops + max-over-bag + agent redeploy-survival' (#154) from dev into main
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
Build images / sign-extension (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 26s
Build images / build-web (push) Successful in 2m18s
Build images / build-ml (push) Successful in 2m44s
CI / integration (push) Successful in 3m35s
Build images / build-agent (push) Successful in 7m50s
2026-06-30 08:46:20 -04:00
bvandeusen 9df874e396 Merge pull request 'CCIP automation + reference quality (auto-enqueue, auto-apply, contamination fix, cache)' (#152) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 28s
CI / frontend-build (push) Successful in 20s
Build images / build-web (push) Successful in 2m20s
Build images / build-ml (push) Successful in 2m40s
Build images / build-agent (push) Successful in 3m17s
CI / integration (push) Successful in 3m27s
2026-06-29 22:29:30 -04:00
bvandeusen 6915a7590a Merge pull request 'fix(tags): refocus tag input after reject/un-reject too' (#151) from dev into main
CI / integration (push) Successful in 3m26s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 16s
CI / backend-lint-and-test (push) Successful in 27s
2026-06-29 21:07:21 -04:00
bvandeusen 5e8c28236a Merge pull request 'CCIP match threshold (tunable, 0.85) + Explore → variance' (#150) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m29s
2026-06-29 20:48:14 -04:00
bvandeusen 7e3c0f0b74 Merge pull request 'agent: raise worker cap to 32 + size HTTP pool' (#149) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 1s
CI / backend-lint-and-test (push) Successful in 26s
CI / frontend-build (push) Successful in 20s
Build images / build-web (push) Successful in 2m17s
Build images / build-ml (push) Successful in 2m47s
Build images / build-agent (push) Successful in 3m11s
CI / integration (push) Successful in 3m31s
2026-06-29 19:42:11 -04:00
bvandeusen 5d0c7ba706 Merge pull request 'fix(agent): flatten transparency onto white before RGB' (#148) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m27s
2026-06-29 19:18:48 -04:00
bvandeusen 18300e1f8a Merge pull request 'Agent: GPU load + live worker tuning + fast orphan recovery' (#147) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-agent (push) Successful in 8s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m27s
2026-06-29 19:11:22 -04:00
bvandeusen d52ac0a0e2 Merge pull request 'fix(agent): cuDNN base image so onnxruntime-gpu loads' (#146) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 16s
CI / backend-lint-and-test (push) Successful in 26s
Build images / build-web (push) Successful in 2m19s
Build images / build-ml (push) Successful in 2m50s
Build images / build-agent (push) Successful in 3m29s
CI / integration (push) Successful in 3m28s
2026-06-29 18:48:24 -04:00
bvandeusen 401fe8213e Merge pull request 'ci: publish the GPU agent image' (#145) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-agent (push) Successful in 3m15s
CI / integration (push) Successful in 3m27s
2026-06-29 14:26:51 -04:00
bvandeusen e8774d7953 Merge pull request 'CCIP characters + crop/region pipeline + desktop GPU agent (#114)' (#144) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 26s
CI / lint (push) Successful in 3s
Build images / build-web (push) Successful in 2m8s
Build images / build-ml (push) Successful in 2m37s
CI / integration (push) Successful in 3m26s
2026-06-29 14:18:57 -04:00
bvandeusen bc0f00c51b Merge pull request 'Earned auto-apply (fire + observability + UI), retrain cadences, Explore arrow-nav' (#143) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-web (push) Successful in 2m2s
Build images / build-ml (push) Successful in 2m30s
CI / integration (push) Successful in 3m24s
2026-06-29 07:30:41 -04:00
bvandeusen 1bef68aa29 Merge pull request 'Tagging-v2: heads are the suggestion source (learn-from-tags) + rail accept/reject' (#142) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 25s
Build images / build-web (push) Successful in 2m4s
Build images / build-ml (push) Successful in 2m44s
CI / integration (push) Successful in 3m25s
2026-06-28 11:55:49 -04:00
bvandeusen 1a4bc2f981 Merge pull request 'tag-eval: "keep" records a confirmation so doubts stop resurfacing' (#141) from dev into main
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-web (push) Successful in 2m5s
CI / integration (push) Successful in 3m20s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 2m30s
2026-06-28 01:36:26 -04:00
bvandeusen 862ace69d6 Merge pull request 'fix(tag-eval): stop re-suggesting already-rejected items' (#140) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m19s
2026-06-28 01:10:26 -04:00
bvandeusen abf88b1a15 Merge pull request 'tag-eval: auto-apply operating point + server-side top-N concept discovery' (#139) from dev into main
CI / backend-lint-and-test (push) Failing after 2s
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
Build images / build-web (push) Successful in 2m3s
Build images / build-ml (push) Successful in 2m37s
CI / integration (push) Successful in 3m19s
2026-06-28 00:54:14 -04:00
bvandeusen c05dcafbea Merge pull request 'fix(tag-eval): example thumbnail opens the view modal instead of Explore' (#138) from dev into main
Build images / build-ml (push) Successful in 7s
CI / frontend-build (push) Successful in 18s
CI / integration (push) Successful in 3m20s
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-web (push) Successful in 9s
CI / backend-lint-and-test (push) Successful in 27s
2026-06-28 00:11:20 -04:00
bvandeusen 55e8632dab Merge pull request 'Tag-eval review actions: bigger clickable thumbnails + inline confirm/reject' (#137) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-web (push) Successful in 2m2s
Build images / build-ml (push) Successful in 2m36s
CI / integration (push) Successful in 3m18s
2026-06-27 23:49:12 -04:00
bvandeusen 825e6b90bf Merge pull request 'Tag-eval (heads vs centroid) + focus/provenance/layout/sticky-header fixes + tag-gaps cleanup' (#136) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-web (push) Successful in 2m5s
Build images / build-ml (push) Successful in 2m31s
CI / integration (push) Successful in 3m21s
2026-06-27 23:03:04 -04:00
bvandeusen fb012c557c Merge pull request 'fix(explore): bound 3-pane grid row so a tall rail can't scroll the page' (#135) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-web (push) Successful in 2m5s
Build images / build-ml (push) Successful in 2m37s
CI / integration (push) Successful in 3m20s
2026-06-26 23:37:43 -04:00
bvandeusen 66593ab895 Merge pull request 'Explore: focus-everywhere + provenance in rail; tag-gaps cleanup' (#134) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m17s
2026-06-26 21:40:56 -04:00
bvandeusen b266a54ad3 Merge pull request 'feat(explore): auto-focus tag input on every image change' (#133) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m25s
2026-06-26 11:28:32 -04:00
bvandeusen ad803b646f Merge pull request 'fix(modal): place meta + save block under Provenance, above Tags' (#132) from dev into main
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m20s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 11s
2026-06-26 10:14:15 -04:00
bvandeusen 1f5da3d283 Merge pull request 'Fandom count fix + Explore 3-pane workspace + modal rail layout' (#131) from dev into main
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m21s
2026-06-26 08:24:26 -04:00
bvandeusen 93034f580d Merge pull request 'fix(tags): fandom views aggregate images via their characters' (#130) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-web (push) Successful in 2m6s
Build images / build-ml (push) Successful in 2m34s
CI / integration (push) Successful in 3m19s
2026-06-26 00:28:21 -04:00
bvandeusen 9b9b12f410 Merge pull request 'Modal suggestions scroll-cap + Celery broker resilience' (#129) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-web (push) Successful in 2m3s
Build images / build-ml (push) Failing after 2m34s
CI / integration (push) Successful in 3m30s
2026-06-24 08:34:17 -04:00
bvandeusen 376d310693 Merge pull request 'Tagging & viewing roadmap: tag query surface + hygiene projections + Explore view (Clusters A/B/C)' (#128) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-web (push) Successful in 2m6s
Build images / build-ml (push) Successful in 2m39s
CI / integration (push) Successful in 3m24s
2026-06-23 08:57:34 -04:00
bvandeusen bc69495a16 Merge pull request 'Provenance archive linkage, post reconciliation close-out, and the cleanup/admin DRY pass' (#127) from dev into main
Build images / build-web (push) Successful in 2m2s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-ml (push) Successful in 2m33s
CI / integration (push) Successful in 3m22s
2026-06-22 20:22:56 -04:00
bvandeusen 478f898e72 Merge pull request 'feat(settings): Maintenance tab compact tiles + centered Settings (pass 2)' (#126) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
Build images / build-web (push) Successful in 2m18s
Build images / build-ml (push) Successful in 3m3s
CI / integration (push) Successful in 3m26s
2026-06-18 00:11:54 -04:00
bvandeusen 38a5e7f332 Merge pull request 'feat(settings): tidy Cleanup tab into sectioned compact tiles (pass 1)' (#125) from dev into main
CI / backend-lint-and-test (push) Failing after 2s
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 19s
CI / integration (push) Successful in 3m19s
2026-06-17 23:49:21 -04:00
bvandeusen 57fe15c267 Merge pull request 'feat(maintenance): reconcile duplicate posts (gallery-dl→native unify)' (#124) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 9s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m22s
2026-06-17 22:01:12 -04:00
bvandeusen eb3231ef10 Merge pull request 'fix(subscribestar): port gallery-dl date extraction (wrapped dates) + parse canary' (#123) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m19s
2026-06-17 21:15:43 -04:00
bvandeusen e9af459c0d Merge pull request 'feat(subscribestar): port gallery-dl doc + audio attachment extraction' (#122) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 9s
CI / frontend-build (push) Successful in 17s
Build images / build-web (push) Successful in 7s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m15s
2026-06-17 16:20:12 -04:00
bvandeusen 6f02806aec Merge pull request 'fix(subscribestar): port gallery-dl content + preview-skip extraction (body bug)' (#121) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 12s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m25s
2026-06-17 16:10:56 -04:00
bvandeusen a1d19bd96a Merge pull request 'fix(subscribestar): mirror gallery-dl's full request profile (verify_subscriber gate)' (#120) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m24s
2026-06-17 15:32:28 -04:00
bvandeusen 26827ff38f Merge pull request 'fix(subscribestar): match gallery-dl's generic post delimiter (live-feed drift)' (#119) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m15s
2026-06-17 15:07:59 -04:00
bvandeusen 26dcfaf6c2 Merge pull request 'fix(subscribestar): initial feed GET is a navigation, not XHR (first-run drift)' (#118) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m14s
2026-06-17 14:30:02 -04:00
bvandeusen 9b1b0369cc Merge pull request 'SubscribeStar → native core ingester + native-ingest DRY pass (milestone #71)' (#117) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m20s
2026-06-17 12:48:29 -04:00
bvandeusen 18123fb9cb Merge pull request 'fix(external): recovery-sweep threshold + queue recording + split fetch timeouts (#883)' (#116) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m14s
2026-06-16 21:24:31 -04:00
bvandeusen 2e806f202f Merge pull request 'test(artist-dir): fix flaky uq_image_record_sha256 collision' (#115) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m17s
2026-06-16 20:34:21 -04:00
bvandeusen 18d5c05639 Merge pull request 'fix(ml): per-task async engine for recompute_centroid (#881)' (#114) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Failing after 3m19s
2026-06-16 20:24:37 -04:00
bvandeusen 11ddfc3876 Merge pull request 'fix(maint): resurface dedup/gated-purge results after navigate-away (#877)' (#113) from dev into main
Build images / sign-extension (push) Successful in 2s
Build images / build-ml (push) Successful in 6s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
Build images / build-web (push) Successful in 15s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m20s
2026-06-16 16:48:52 -04:00
bvandeusen 2b8ce86622 Merge pull request 'Gated Patreon posts: skip on ingest + cleanup tool (#874)' (#112) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m27s
2026-06-16 15:25:34 -04:00
bvandeusen 49bee77cdc Merge pull request 'Video tag quality: cadence sampling + min-frame aggregation + ML thread cap (#747)' (#111) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m15s
2026-06-16 14:08:40 -04:00
bvandeusen c209e3b37e Merge pull request 'Tier-1 video dedup: import-time + retroactive cleanup (#871)' (#110) from dev into main
CI / integration (push) Successful in 3m16s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-web (push) Successful in 26s
2026-06-16 08:55:38 -04:00
bvandeusen cffdd93418 Merge pull request 'Nested-archive extraction (#718) + post-first ingest (#67) + post-body canary (#862)' (#109) from dev into main
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m19s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
2026-06-15 21:37:39 -04:00
bvandeusen fd84be40dd Merge pull request 'External-attach orphan fix (#859) + image-less post display' (#108) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 40s
Build images / build-web (push) Successful in 10s
CI / integration (push) Successful in 3m19s
2026-06-15 01:55:36 -04:00
bvandeusen 79f510d7f8 Merge pull request 'Merge dev → main: read post body from content_json_string (the empty-body fix) (#842)' (#107) from dev into main
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 10s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m24s
Build images / sign-extension (push) Successful in 3s
2026-06-15 00:19:45 -04:00
bvandeusen 59181069da Merge pull request 'Merge dev → main: per-post stdout diagnostics + post-field write DRY (#842/#753)' (#106) from dev into main
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 41s
Build images / sign-extension (push) Successful in 3s
Build images / build-web (push) Successful in 2m15s
Build images / build-ml (push) Successful in 2m55s
CI / integration (push) Successful in 3m11s
2026-06-14 23:45:39 -04:00
bvandeusen 428ecd8642 Merge pull request 'Merge dev → main: per-post body-capture diagnostics in the event UI (#842)' (#105) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / integration (push) Successful in 3m11s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 37s
2026-06-14 23:14:14 -04:00
bvandeusen ed1e04b831 Merge pull request 'Merge dev → main: recapture body-fetch fix + diagnostics (#842)' (#104) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m15s
2026-06-14 22:31:41 -04:00
bvandeusen f5156bd847 Merge pull request 'Merge dev → main: Recapture mode (#842) — re-grab post bodies/links + localize on-disk inline images' (#103) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 32s
Build images / build-web (push) Successful in 2m30s
CI / integration (push) Successful in 3m29s
Build images / build-ml (push) Successful in 3m27s
2026-06-14 21:21:49 -04:00
bvandeusen dfc3922d24 Merge pull request 'Merge dev → main: #830 rich post capture + external-host downloads (+ #768/#789/#739)' (#102) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 48s
Build images / build-web (push) Successful in 2m21s
Build images / build-ml (push) Successful in 2m47s
CI / integration (push) Successful in 3m20s
2026-06-14 19:16:09 -04:00
bvandeusen 3eb08e926b Merge pull request 'fix(aliases): modal raw-key bug + alias visibility/management' (#101) from dev into main
CI / lint (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m13s
2026-06-12 14:02:00 -04:00
bvandeusen 9e81ced359 Merge pull request 'fix(images): percent-encode original-image URLs ('#' in paths 404'd)' (#100) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m15s
2026-06-12 00:44:56 -04:00
bvandeusen 11e9f5af60 Merge pull request 'fix(browse): tabs and search on one row' (#99) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 7s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 29s
Build images / build-web (push) Successful in 11s
CI / integration (push) Successful in 3m13s
2026-06-12 00:28:38 -04:00
bvandeusen 909fa37b15 Merge pull request 'Browse search + series numbering rework + kebab fix' (#98) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-web (push) Successful in 2m17s
Build images / build-ml (push) Successful in 2m50s
CI / integration (push) Successful in 3m16s
2026-06-12 00:14:31 -04:00
bvandeusen dfab8f65ff Merge pull request 'feat(series): flat sequence + cosmetic dividers + pending staging (#789)' (#97) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
CI / frontend-build (push) Successful in 29s
CI / backend-lint-and-test (push) Successful in 38s
Build images / build-web (push) Successful in 2m13s
Build images / build-ml (push) Successful in 2m49s
CI / integration (push) Successful in 3m9s
2026-06-11 22:07:57 -04:00
bvandeusen 618f7cdc36 Merge pull request 'feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3)' (#96) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 25s
Build images / build-web (push) Successful in 2m29s
Build images / build-ml (push) Successful in 3m0s
CI / integration (push) Successful in 3m13s
2026-06-11 19:32:05 -04:00
bvandeusen 028ea33a7c Merge pull request 'fix(migration): make 0045 DDL-only; backfill image_prediction via batched task (#768)' (#95) from dev into main
CI / backend-lint-and-test (push) Successful in 38s
Build images / build-ml (push) Successful in 2m58s
CI / integration (push) Successful in 3m15s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
Build images / build-web (push) Successful in 2m50s
2026-06-11 09:22:22 -04:00
bvandeusen 444c1fb075 Merge pull request 'perf(migration): 0045 streams json_each (no materialize / no temp blowup)' (#94) from dev into main
Build images / sign-extension (push) Successful in 3s
Build images / build-web (push) Successful in 2m14s
Build images / build-ml (push) Successful in 2m36s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m8s
2026-06-10 22:07:16 -04:00
bvandeusen 26c68b0a75 Merge pull request 'fix(migration): 0045 guards json_each against scalar tagger_predictions' (#93) from dev into main
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m13s
Build images / sign-extension (push) Successful in 2s
Build images / build-web (push) Successful in 6s
Build images / build-ml (push) Successful in 2m40s
2026-06-10 20:33:27 -04:00
bvandeusen e75427b19a Merge pull request '#768 steps 1+2: normalized image_prediction table (read cutover)' (#92) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-web (push) Successful in 2m4s
Build images / build-ml (push) Successful in 2m42s
CI / integration (push) Successful in 3m8s
2026-06-10 20:15:26 -04:00
bvandeusen 5447fab987 Merge pull request 'Activity search + RecoverySweep fix + tagger_predictions shrink (#762, #764) + backup polish' (#91) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 38s
Build images / build-web (push) Successful in 2m23s
Build images / build-ml (push) Successful in 2m51s
CI / integration (push) Successful in 3m10s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
2026-06-10 14:33:35 -04:00
bvandeusen bad37e07b2 Merge pull request 'Browse hub, series rename, full-prediction dropdown + a DRY pass (7 sweeps)' (#90) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 27s
CI / frontend-build (push) Successful in 44s
Build images / build-ml (push) Successful in 3m2s
CI / integration (push) Successful in 3m15s
Build images / build-web (push) Successful in 2m17s
2026-06-10 00:24:01 -04:00
bvandeusen 2bfc9936a1 Merge pull request 'UI batch: tagging flow, series browse, fandom chips, nav' (#89) from dev into main
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 39s
Build images / sign-extension (push) Successful in 3s
CI / frontend-build (push) Successful in 26s
Build images / build-web (push) Successful in 2m21s
Build images / build-ml (push) Successful in 2m52s
CI / integration (push) Successful in 3m24s
2026-06-09 20:48:19 -04:00
bvandeusen 4c6406ee18 Merge pull request 'fix(posts): link duplicate items to every post + prune bare shells' (#88) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-web (push) Successful in 2m19s
Build images / build-ml (push) Successful in 2m41s
CI / integration (push) Successful in 3m5s
2026-06-08 19:42:31 -04:00
bvandeusen bb47e80b3e Merge pull request 'fix(cleanup): unused-tags delete must use the same predicate as the preview' (#87) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m13s
2026-06-08 18:17:49 -04:00
bvandeusen dc1083b5e0 Merge pull request 'Unused-tag fandom fix + ML-worker logging/tuning + unified dropdown Enter' (#86) from dev into main
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m16s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 2m21s
2026-06-08 17:27:55 -04:00
bvandeusen e46893fefd Merge pull request 'Migration lock safety + remove the merge's full-table scan (the real 0040-hang fix)' (#85) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 25s
Build images / build-web (push) Successful in 2m3s
Build images / build-ml (push) Successful in 2m33s
CI / integration (push) Successful in 3m7s
2026-06-08 01:03:06 -04:00
bvandeusen 0666e15211 Merge pull request 'Series manage redesign (FC-6.4) + migration/normalize hardening + UX fixes' (#84) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 25s
Build images / build-web (push) Successful in 1m58s
Build images / build-ml (push) Successful in 2m28s
CI / integration (push) Successful in 3m6s
2026-06-07 21:17:57 -04:00
bvandeusen 747390631d Merge pull request 'FC-6 series authoring + backup/NFS hardening + UX fixes' (#83) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
Build images / build-web (push) Successful in 2m5s
Build images / build-ml (push) Successful in 2m43s
CI / integration (push) Successful in 3m6s
2026-06-07 19:31:04 -04:00
bvandeusen e0d2a20588 Merge pull request 'Modal focus/keyboard polish, Camie-in-autocomplete, re-extract self-resume' (#82) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 28s
Build images / build-web (push) Successful in 2m2s
Build images / build-ml (push) Successful in 2m32s
CI / integration (push) Successful in 3m9s
2026-06-07 12:17:43 -04:00
bvandeusen 1d84f67418 Merge pull request 'Modal: large centered spinner + kebab z-index fix' (#81) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 27s
CI / backend-lint-and-test (push) Successful in 29s
Build images / build-web (push) Successful in 2m3s
Build images / build-ml (push) Successful in 2m35s
CI / integration (push) Successful in 3m7s
2026-06-07 10:57:00 -04:00
bvandeusen 91265df3d6 Merge pull request 'Maintenance-queue health + modal/tagging keyboard pass' (#80) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 25s
Build images / build-web (push) Successful in 2m5s
Build images / build-ml (push) Successful in 2m26s
CI / integration (push) Successful in 3m1s
2026-06-07 10:31:01 -04:00
bvandeusen 11acdb0322 Merge pull request 'Patreon: a missing media file_name is a fallback, not API drift' (#79) from dev into main
CI / lint (push) Successful in 2s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 20s
Build images / sign-extension (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m2s
2026-06-06 23:14:25 -04:00
bvandeusen 2eb9fd5dd0 Merge pull request 'Patreon: enforce the backfill time-box mid-post (stop soft-limit overruns)' (#78) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 8s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m3s
2026-06-06 23:00:29 -04:00
bvandeusen 01e5ce1410 Merge pull request 'Patreon: resolve creator campaign from a single-post URL' (#77) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m4s
2026-06-06 21:38:33 -04:00
bvandeusen 3bb94674cf Merge pull request 'Patreon download concurrency cap + immediate backfill kickoff on create' (#76) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m1s
2026-06-06 21:27:22 -04:00
bvandeusen a75c602175 Merge pull request 'Subscriptions UX overhaul + Patreon /cw/ vanity fix' (#75) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m2s
2026-06-06 21:12:02 -04:00
bvandeusen ef8f4f7193 Merge pull request 'Tag-casing acronym fix, Patreon resolver hardening, archive diagnostics, post-card strip' (#74) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m1s
2026-06-06 19:59:32 -04:00
bvandeusen ec3d27b219 Merge pull request 'Tag-maintenance sweep + bug-fix batch: #699 #700 #701 #709 #711 #712 #713 #714' (#73) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m2s
2026-06-06 16:49:00 -04:00
bvandeusen 03bd3b2eda Merge pull request 'Native Patreon ingester + download-engine ownership (plans #697, #703–#708)' (#72) from dev into main
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 10s
CI / backend-lint-and-test (push) Successful in 25s
CI / frontend-build (push) Successful in 32s
CI / integration (push) Successful in 3m1s
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
2026-06-06 12:57:31 -04:00
bvandeusen 7395e77d75 Merge pull request 'Smarter backfill: time-boxed chunks, run-until-done (plan #693)' (#71) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
CI / backend-lint-and-test (push) Successful in 13s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 18s
CI / integration (push) Successful in 2m58s
2026-06-05 16:33:06 -04:00
bvandeusen 575d817919 Merge pull request '#70 dev→main: cursor-paged backfill + mobile modal fixes' from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
CI / backend-lint-and-test (push) Successful in 13s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 18s
CI / integration (push) Successful in 2m59s
2026-06-05 11:10:02 -04:00
bvandeusen 2a8f7cd8b6 Merge pull request '#69 dev→main: release v26.06.04.0' from dev into main
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 12s
CI / frontend-build (push) Successful in 19s
CI / integration (push) Successful in 3m2s
Build images / sign-extension (push) Has been skipped
Build images / build-ml (push) Successful in 6s
Build images / build-web (push) Successful in 6s
2026-06-04 23:16:12 -04:00
bvandeusen 83f8af8090 Merge pull request 'dev→main: surface near-duplicate (pHash) control + reorder import tab' (#68) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 6s
CI / backend-lint-and-test (push) Successful in 11s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 20s
CI / integration (push) Successful in 2m56s
2026-06-04 21:39:44 -04:00
bvandeusen 9a2617c1a2 Merge pull request 'dev→main: post-card redesign (images→modal, in-place text expand)' (#67) from dev into main
CI / lint (push) Failing after 2s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
CI / backend-lint-and-test (push) Successful in 11s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 24s
CI / integration (push) Successful in 2m57s
2026-06-04 17:32:50 -04:00
bvandeusen 81688815a0 Merge pull request 'dev→main: similar-search render fix + reset-content-tagging + scan persistence' (#66) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 9s
CI / backend-lint-and-test (push) Successful in 13s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 22s
CI / integration (push) Successful in 2m58s
2026-06-04 16:59:52 -04:00
bvandeusen 773128c3bf Merge pull request 'dev→main: purpose-built mobile layout for subscriptions hub' (#65) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
CI / backend-lint-and-test (push) Successful in 14s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 38s
CI / integration (push) Successful in 2m56s
2026-06-04 13:43:56 -04:00
bvandeusen ce7b154ae9 Merge pull request 'dev→main: subscriptions table mobile card layout' (#64) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 8s
CI / backend-lint-and-test (push) Successful in 14s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 22s
CI / integration (push) Successful in 2m57s
2026-06-04 12:56:58 -04:00
bvandeusen 9430a9d9c3 Merge pull request 'dev→main: gallery similarity search (Phase 3) + UI/mobile polish' (#63) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
CI / backend-lint-and-test (push) Successful in 12s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 21s
CI / integration (push) Successful in 2m56s
2026-06-04 11:21:26 -04:00
bvandeusen 23aee56ce3 Merge pull request 'dev→main: showcase cascade + filter styling + DB maintenance + gallery filter Phase 2 + showcase decode-gate + CI perf' (#62) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 9s
CI / backend-lint-and-test (push) Successful in 12s
Build images / build-web (push) Successful in 11s
CI / frontend-build (push) Successful in 20s
CI / integration (push) Successful in 3m0s
2026-06-04 08:27:47 -04:00
bvandeusen 711abea567 Merge pull request 'Gallery speed + fandom editing + filters + pinned filter bar' (#61) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 8s
CI / backend-lint-and-test (push) Successful in 18s
CI / frontend-build (push) Successful in 19s
Build images / build-web (push) Successful in 11s
CI / intimp (push) Successful in 3m48s
CI / intapi (push) Successful in 7m45s
CI / intcore (push) Successful in 8m29s
2026-06-04 00:07:19 -04:00
bvandeusen 844bb86802 Merge pull request 'fix(download): release DB connections across the gallery-dl subprocess' (#60) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 10s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 25s
CI / intimp (push) Successful in 3m46s
CI / intapi (push) Successful in 7m41s
CI / intcore (push) Successful in 8m24s
2026-06-03 22:11:36 -04:00
bvandeusen a8f6a464aa Merge pull request 'fix(download): salvage soft-time-limit kills + fix timeout ladder' (#59) from dev into main
CI / lint (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 21s
CI / frontend-build (push) Successful in 24s
Build images / build-web (push) Successful in 2m37s
Build images / build-ml (push) Successful in 3m21s
CI / intimp (push) Successful in 3m40s
CI / intapi (push) Successful in 7m47s
CI / intcore (push) Successful in 8m15s
2026-06-03 19:35:45 -04:00
bvandeusen ab9922ad2e Merge pull request 'feat(artist): "new since last visit" badge + banner' (#58) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
CI / backend-lint-and-test (push) Successful in 23s
Build images / build-ml (push) Successful in 2m53s
CI / frontend-build (push) Successful in 21s
Build images / build-web (push) Successful in 2m9s
CI / intimp (push) Successful in 3m36s
CI / intapi (push) Successful in 7m32s
CI / intcore (push) Successful in 8m9s
2026-06-03 16:20:54 -04:00
bvandeusen 0533807669 Merge pull request 'feat(ext): verify cookies in-browser before uploading (1.0.7)' (#57) from dev into main
CI / backend-lint-and-test (push) Successful in 32s
Build images / build-ml (push) Successful in 3m28s
Build images / build-web (push) Successful in 3m42s
CI / intapi (push) Successful in 9m32s
CI / frontend-build (push) Successful in 20s
extension / lint (push) Successful in 14s
Build images / sign-extension (push) Successful in 3m17s
CI / intimp (push) Successful in 4m26s
CI / lint (push) Successful in 3s
CI / intcore (push) Successful in 10m18s
2026-06-03 14:17:42 -04:00
bvandeusen 279dff3fb6 Merge pull request 'feat(ml): normalize Camie suggestion names to human-readable' (#56) from dev into main
Build images / build-ml (push) Successful in 2m55s
CI / intimp (push) Successful in 3m36s
CI / intapi (push) Successful in 7m38s
CI / intcore (push) Successful in 8m24s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 15s
CI / frontend-build (push) Successful in 17s
Build images / build-web (push) Successful in 2m17s
2026-06-03 13:18:44 -04:00
bvandeusen 37e66cddc4 Merge pull request 'chore(modal): drop ?image=N soft-compat — pure overlay' (#55) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 27s
CI / backend-lint-and-test (push) Successful in 38s
Build images / build-ml (push) Successful in 2m46s
Build images / build-web (push) Successful in 2m36s
CI / intimp (push) Successful in 3m42s
CI / intapi (push) Successful in 7m25s
CI / intcore (push) Successful in 8m12s
2026-06-02 19:35:04 -04:00
bvandeusen 9cf6b2d363 Merge pull request 'audit-g5 final + ML threshold default + kebab menu fix' (#54) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 4s
CI / backend-lint-and-test (push) Successful in 23s
CI / frontend-build (push) Successful in 27s
Build images / build-web (push) Successful in 3m0s
Build images / build-ml (push) Successful in 3m45s
CI / intimp (push) Successful in 3m46s
CI / intapi (push) Successful in 8m8s
CI / intcore (push) Successful in 8m34s
2026-06-02 19:09:49 -04:00
bvandeusen 6ef0fed41f Merge pull request 'audit-g5: architectural debt — 4 bundles (A/B/C/D)' (#53) from dev into main
CI / lint (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 9s
CI / backend-lint-and-test (push) Successful in 18s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 26s
CI / intimp (push) Successful in 3m51s
CI / intapi (push) Successful in 7m38s
CI / intcore (push) Successful in 8m52s
2026-06-02 18:07:25 -04:00
bvandeusen 89b48f8f35 Merge pull request 'audit-g4: status-enum miss batch' (#52) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 11s
CI / backend-lint-and-test (push) Successful in 13s
Build images / build-web (push) Successful in 10s
CI / frontend-build (push) Successful in 35s
CI / intimp (push) Successful in 3m41s
CI / intapi (push) Successful in 7m25s
CI / intcore (push) Successful in 8m10s
2026-06-02 16:15:00 -04:00
bvandeusen d60e0b9494 Merge pull request 'audit-g3: lifecycle batch — recovery sweeps, retention, timeouts' (#51) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 9s
CI / backend-lint-and-test (push) Successful in 12s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 36s
CI / intimp (push) Successful in 3m39s
CI / intapi (push) Successful in 7m39s
CI / intcore (push) Successful in 8m10s
2026-06-02 14:49:28 -04:00
bvandeusen 9c27a2d3c7 Merge pull request 'audit-g2: async race / state-leak fixes across eight stores' (#50) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
CI / backend-lint-and-test (push) Successful in 23s
CI / frontend-build (push) Successful in 37s
Build images / build-web (push) Successful in 13s
CI / intimp (push) Successful in 3m52s
CI / intapi (push) Successful in 8m15s
CI / intcore (push) Successful in 8m50s
2026-06-02 14:17:12 -04:00
bvandeusen 93e37681b7 Merge pull request 'audit-g1: six one-liner drift fixes from 2026-06-02 audit' (#49) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 11s
CI / backend-lint-and-test (push) Successful in 11s
CI / frontend-build (push) Successful in 19s
Build images / build-web (push) Successful in 10s
CI / intapi (push) Successful in 7m28s
CI / intcore (push) Successful in 8m3s
CI / intimp (push) Successful in 3m42s
2026-06-02 13:29:17 -04:00
bvandeusen 64ca858574 Merge pull request 'UX fixes: suggestion-accept chip refresh, showcase endless feed, non-media downloads' (#48) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Successful in 9s
CI / backend-lint-and-test (push) Successful in 12s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 20s
CI / intimp (push) Successful in 3m52s
CI / intapi (push) Successful in 7m45s
CI / intcore (push) Successful in 8m20s
2026-06-02 08:26:58 -04:00
bvandeusen 9d0c0b7da8 Merge pull request 'fix(thumbnails): surface backfill counts + tighten validity check' (#47) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 9s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 18s
CI / intimp (push) Successful in 3m45s
CI / intapi (push) Successful in 7m32s
CI / intcore (push) Successful in 8m4s
2026-06-01 22:34:38 -04:00
bvandeusen 8e4d252ae4 Merge pull request 'fix(downloads): enqueue thumbnail + ML per attached image' (#46) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / intimp (push) Successful in 3m40s
CI / lint (push) Successful in 3s
CI / intapi (push) Successful in 7m51s
CI / intcore (push) Successful in 8m46s
Build images / build-ml (push) Successful in 8s
Build images / build-web (push) Successful in 8s
CI / backend-lint-and-test (push) Successful in 19s
CI / frontend-build (push) Successful in 26s
2026-06-01 21:52:42 -04:00
bvandeusen fdd3e01f56 Merge pull request 'ux(failing-sources): visible row separators + clearer hover' (#45) from dev into main
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-ml (push) Successful in 10s
Build images / build-web (push) Successful in 10s
CI / backend-lint-and-test (push) Successful in 20s
CI / frontend-build (push) Successful in 33s
CI / intimp (push) Successful in 3m44s
CI / intapi (push) Successful in 7m48s
CI / intcore (push) Successful in 8m16s
2026-06-01 21:09:28 -04:00
bvandeusen c82fb308b6 Merge pull request 'Post.source_id refactor + tick/backfill modes + PARTIAL classifier + Mux fix + UX' (#44) from dev into main
Build images / build-ml (push) Failing after 2s
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 3s
CI / intimp (push) Successful in 3m47s
CI / intapi (push) Successful in 7m52s
CI / intcore (push) Successful in 8m16s
Build images / build-web (push) Successful in 12s
CI / backend-lint-and-test (push) Successful in 19s
CI / frontend-build (push) Successful in 31s
2026-06-01 20:44:23 -04:00
bvandeusen 8cf8d2ca4d Merge pull request 'Modal Esc/overflow polish, artist-scoped post scroll, failing-sources Logs button' (#43) from dev into main
Build images / sign-extension (push) Successful in 3s
CI / lint (push) Successful in 8s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 2m45s
Build images / build-ml (push) Successful in 3m31s
CI / intimp (push) Successful in 3m48s
CI / intapi (push) Successful in 7m11s
CI / intcore (push) Successful in 7m47s
2026-06-01 12:10:11 -04:00
bvandeusen b1d58bc3b8 Merge pull request 'fix(ci): POSIX-safe SHORT_SHA in build.yml' (#42) from dev into main
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 15s
CI / frontend-build (push) Successful in 20s
Build images / sign-extension (push) Successful in 3s
Build images / build-web (push) Successful in 2m17s
Build images / build-ml (push) Successful in 3m24s
CI / intimp (push) Successful in 3m22s
CI / intapi (push) Successful in 7m51s
CI / intcore (push) Successful in 8m5s
2026-06-01 08:03:17 -04:00
bvandeusen 65386f02a0 Merge pull request 'View modal batch: autofocus, suggestions UX, post-title click, retire copyright/artist' (#41) from dev into main
Build images / sign-extension (push) Successful in 2s
CI / lint (push) Successful in 2s
Build images / build-ml (push) Failing after 4s
Build images / build-web (push) Failing after 3s
CI / backend-lint-and-test (push) Successful in 15s
CI / frontend-build (push) Successful in 17s
CI / intimp (push) Successful in 3m46s
CI / intapi (push) Successful in 7m17s
CI / intcore (push) Successful in 8m2s
2026-06-01 07:01:42 -04:00
bvandeusen 667b05f14e Merge pull request 'Extension probe-and-add (v1.0.6) + per-commit image tags' (#40) from dev into main
CI / lint (push) Successful in 2s
Build images / build-ml (push) Failing after 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 22s
extension / lint (push) Successful in 8s
Build images / sign-extension (push) Successful in 1m51s
Build images / build-web (push) Failing after 4s
CI / intimp (push) Successful in 3m38s
CI / intapi (push) Successful in 7m20s
CI / intcore (push) Successful in 7m45s
2026-06-01 01:44:03 -04:00
bvandeusen 856e9104b4 Merge pull request 'Sidecar synthetic anchor cleanup + tier-gated classifier fix' (#39) from dev into main
CI / lint (push) Successful in 5s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 28s
CI / intimp (push) Successful in 3m54s
Build images / sign-extension (push) Has been skipped
Build images / build-web (push) Successful in 1m1s
Build images / build-ml (push) Successful in 1m21s
CI / intapi (push) Successful in 7m31s
CI / intcore (push) Successful in 8m0s
2026-06-01 00:16:58 -04:00
bvandeusen 0397642b21 Merge pull request 'Showcase cadence tuning + cooldown-aware bulk retry' (#38) from dev into main 2026-05-30 23:50:36 -04:00
bvandeusen 237575447d Merge pull request 'Thumbnail URL fix + archive daemon fix + batched initial loads' (#37) from dev into main 2026-05-30 22:01:43 -04:00
bvandeusen ed358757dc Merge pull request 'Most-overdue-first scheduling + rich timeout diagnostics' (#36) from dev into main 2026-05-30 14:30:41 -04:00
bvandeusen d181f4afb8 Merge pull request 'Downloads burst-prevention + maintenance-menu fix + gdl timeout' (#35) from dev into main 2026-05-30 11:43:18 -04:00
bvandeusen 2886fa4997 Merge pull request 'Tooltip !important fix — 104cac5 follow-up after Vite CSS reorder' (#34) from dev into main 2026-05-30 00:02:24 -04:00
bvandeusen f256f587ee Merge pull request 'UI batch + I1–I6 service passes + download-event recovery sweep' (#33) from dev into main 2026-05-29 22:46:16 -04:00
bvandeusen 384d8d5e50 Merge pull request 'Dashboard insights + project-wide DRY pass' (#32) from dev into main 2026-05-28 15:38:26 -04:00
bvandeusen 319e8c1d18 Merge pull request 'v26.05.28.0: downloads dashboard + task-resilience overhaul (timeouts, archive split, 3-layer poison-pill defense)' (#31) from dev into main 2026-05-28 00:45:00 -04:00
bvandeusen 9075d8eadd Merge pull request 'v26.05.27.2: subscribestar + HF cookie quirks, platforms package refactor, showcase IR-parity, secure-context audit' (#30) from dev into main 2026-05-27 21:34:02 -04:00
bvandeusen 88e53e5b86 Merge pull request 'v26.05.27.1: subscriptions hub + post-card merge + sidecar audit' (#29) from dev into main 2026-05-27 17:12:48 -04:00
bvandeusen 37e8b796a1 Merge pull request 'v26.05.27.0: PostCard redesign + IR-style tag suffix + drop meta/rating + extension v1.0.4 CSP fix' (#28) from dev into main 2026-05-27 11:31:18 -04:00
bvandeusen 4e82208926 Merge pull request 'v26.05.26.5 — extension CORS unblock + UI gap closes + CI workflow cleanup' (#27) from dev into main 2026-05-26 20:15:07 -04:00
bvandeusen 52fff00353 Merge pull request 'v26.05.26.4 — hotfix: migration 0022 pre-DELETE colliding ImageProvenance before UPDATE' (#26) from dev into main 2026-05-26 18:06:20 -04:00
bvandeusen c14338cbce Merge pull request 'v26.05.26.3 — hotfix: migration 0022 pre-merge across ENTIRE (canonical+others) group' (#25) from dev into main 2026-05-26 17:52:59 -04:00
bvandeusen 8c36dd28b0 Merge pull request 'v26.05.26.2 — hotfix: alembic 0022 Post-collision pre-merge + ci.yml cache continue-on-error' (#24) from dev into main 2026-05-26 16:50:43 -04:00
bvandeusen 88cfb3dd02 Merge pull request 'v26.05.26.1 — thumb backfill, modal redesign, recovery sweep race-safety, artist view redesign, extension fixes' (#23) from dev into main 2026-05-26 16:32:00 -04:00
bvandeusen 5d4f223b71 Merge pull request 'Release v26.05.25.7 — FC-Cleanup tab + UniqueViolation fix + error modal + extension install fix' (#22) from dev into main 2026-05-26 08:26:46 -04:00
bvandeusen 05090c6e85 Merge pull request 'Release v26.05.25.7 — animated-WebP worker fix + FC-Cleanup backend' (#21) from dev into main 2026-05-26 01:48:13 -04:00
bvandeusen 3a577d5ade Merge pull request 'fix(ext-ci): use browser_download_url + curl -f + ZIP magic check (XPI silently corrupt)' (#20) from dev into main 2026-05-26 00:43:02 -04:00
bvandeusen f4fe02e346 Merge pull request 'fix(ext-ci): drop actions/upload-artifact (Forgejo doesn't support v4+ GHES)' (#19) from dev into main 2026-05-25 23:33:40 -04:00
bvandeusen e766197d99 Merge pull request 'fix(ext-ci): jq→python + bump ext to 1.0.3 + rollback-on-upload-failure' (#18) from dev into main 2026-05-25 23:14:51 -04:00
bvandeusen 3872e1dda9 Merge pull request 'fix(ext-ci): web-ext v8 .cjs config workaround' (#17) from dev into main 2026-05-25 22:49:14 -04:00
bvandeusen 9814f3dbaf Merge pull request 'Release v26.05.25.5 — Extension publish refactor, deep-scan IR-parity, archive-import perf, artist Settings tab' (#16) from dev into main 2026-05-25 22:44:59 -04:00
bvandeusen b214460fdb Merge pull request 'Release v26.05.25.4 — importer ext sanitize fix, CI shard split, BrowserExtensionCard on Overview' (#15) from dev into main 2026-05-25 21:11:50 -04:00
bvandeusen ac55d0e8d8 Merge pull request 'fix(ext-ci): match AMO-renamed signed XPI' (#14) from dev into main 2026-05-25 18:22:50 -04:00
bvandeusen 89a89e0ded Merge pull request 'Release v26.05.25.3 — ML embedder SigLIP fix, import-UX, extension publish' (#13) from dev into main 2026-05-25 17:56:50 -04:00
bvandeusen 4e9aac2c05 Merge pull request 'v26.05.25.2: supersede + sidecar enrichment, scan toast feedback, CI uv + pip cache + durations' (#12) from dev into main 2026-05-25 14:30:25 -04:00
bvandeusen 2879ac6f2b Merge pull request 'v26.05.25.1: maintenance sweep + Camie v2 + corrupt-file handling + post-date gallery + clear-stuck escape hatch' (#11) from dev into main 2026-05-25 12:57:46 -04:00
bvandeusen b8dce6c483 Merge pull request 'FC-3h + FC-3k: backup first-class + admin destructive actions' (#10) from dev into main 2026-05-25 01:41:53 -04:00
bvandeusen d1c0b82a22 Merge pull request 'v26.05.24.3: FC-3i System Activity dashboard + migration backup-gate retired + modal Escape' (#9) from dev into main 2026-05-24 21:47:53 -04:00
bvandeusen 5526b8dc78 Merge pull request 'v26.05.24.2: IR Post/Provenance restore + modal artist fallback' (#8) from dev into main 2026-05-24 14:30:06 -04:00
bvandeusen 16eb7075c4 Merge pull request 'v26.05.24.1: FC-3g Firefox extension + worker resilience + UI/migration fixes' (#7) from dev into main 2026-05-24 12:52:31 -04:00
bvandeusen 885dcf64f3 Merge pull request 'v26.05.24.0: TopNav re-fix (flex 1 1 0 side cells)' (#6) from dev into main 2026-05-23 22:49:29 -04:00
bvandeusen f2f6b6d25e Merge pull request 'v26.05.23.3: dogfood UX polish + accurate active-batch stats' (#5) from dev into main 2026-05-23 22:05:59 -04:00
bvandeusen 0822240fde Merge pull request 'v26.05.23.2: serve /images + artist cleanup migrator' (#4) from dev into main 2026-05-23 12:19:16 -04:00
bvandeusen 27f7f3fd01 Merge pull request 'v26.05.23.1: migration durability + dogfood UX' (#3) from dev into main 2026-05-23 11:21:33 -04:00
bvandeusen c5bf564f53 Merge dev: v26.05.23.0 migration follow-ups (#2)
pg_dump + zstd in runtime image, lift Quart body cap to 1 GiB. See PR #2.
2026-05-22 22:37:06 -04:00
bvandeusen 602c7d275d Merge dev: FC-1 → FC-5 v1 build (#1)
First merge of `dev` into `main` for FabledCurator. Brings FC-1 (Foundation) through FC-5 (Migration tooling) onto `main`. See PR #1 body for the full stage rollup.
2026-05-22 14:15:45 -04:00
162 changed files with 1486 additions and 5931 deletions
+25 -188
View File
@@ -2,18 +2,10 @@ name: Build images
on: on:
push: push:
# `:dev` builds were dropped 2026-05-26 to save a docker build per dev # `:dev` builds dropped 2026-05-26 — operator tests from `:latest` after
# push, on the reasoning that "operator tests from `:latest` after # merge-to-main, not from the dev branch image. Saves one full docker
# merge-to-main". Restored 2026-08-27: that is testing by shipping, and # build per dev push.
# family rules 146/147 now name it directly — `main` IS production, and a branches: [main]
# channel that can only be refreshed by shipping is not a channel. The
# pressure to merge in order to try something does not come from
# carelessness; it comes from `:dev` being unable to carry the build.
#
# All three images build on dev, deliberately: a `:dev` web image paired
# with a stale `:dev` ml or agent is a worse trap than no dev channel at
# all, since the mismatch only shows up as a runtime failure.
branches: [main, dev]
# Tag-push triggers an immutable per-version image build (e.g. # Tag-push triggers an immutable per-version image build (e.g.
# `:v26.05.26.5`) — gives a real rollback story alongside the floating # `:v26.05.26.5`) — gives a real rollback story alongside the floating
# `:main` / `:latest`. Layer reuse keeps the registry-storage cost # `:main` / `:latest`. Layer reuse keeps the registry-storage cost
@@ -33,135 +25,28 @@ jobs:
# Forgejo release exists yet, otherwise downloads the cached signed XPI. # Forgejo release exists yet, otherwise downloads the cached signed XPI.
# Result is uploaded as an Actions artifact for build-web to consume. # Result is uploaded as an Actions artifact for build-web to consume.
# #
# Why this lives in build.yml (not a separate workflow): the image a push # Why this lives in build.yml (not a separate workflow): the merge-commit's
# publishes MUST carry the XPI. A separate sign workflow racing build.yml # docker image tagged `:latest` MUST carry the XPI. A separate sign workflow
# leaves that image without one for ~5min (until the commit-back triggers # racing build.yml leaves `:latest` without the XPI for ~5min (until the
# another build). Inline ordering eliminates the race. # commit-back triggers another build). Inline ordering eliminates the race.
# Cache strategy: Forgejo Release Assets — picked 2026-05-25 over Generic # Cache strategy: Forgejo Release Assets — picked 2026-05-25 over Generic
# Packages (cleaner API surface) and commit-back-to-side-branch (no extra # Packages (cleaner API surface) and commit-back-to-side-branch (no extra
# branch to manage). AMO blocks re-signing the same version (returns 409), # branch to manage). AMO blocks re-signing the same version (returns 409),
# so signing is intentionally one-shot per version. # so signing is intentionally one-shot per version bump.
#
# BOTH branches sign (milestone 271 step 6, 2026-08-27). Not two signatures:
# the version is the commit TIME of the newest packaged-extension change, so
# dev and main derive the SAME number for the same extension source. A dev
# push that changes the extension signs it; the merge to main then finds the
# ext-<version> release already there, hits the cache, and bundles the
# byte-identical XPI into `:latest` with no second AMO call. One signature
# per extension CHANGE, shared by both channels — that is what makes two
# channels affordable, and it is why step 4 (derived version) had to land
# first. Ungating this while the version was still the hand-set 1.0.11 would
# have hit the existing ext-1.0.11 cache and bundled MAIN's stale XPI into
# `:dev` — a dev channel confidently serving old code.
#
# Tags stay excluded: the tag path deliberately skips signing and polls for
# the release instead (see build-web's race note, 2026-05-27).
sign-extension: sign-extension:
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev' if: github.ref == 'refs/heads/main'
runs-on: python-ci runs-on: python-ci
container: container:
image: git.fabledsword.com/bvandeusen/ci-python:3.14 image: git.fabledsword.com/bvandeusen/ci-python:3.14
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with:
# Full history is load-bearing, not a convenience: the version this
# job signs is derived from the commit TIME of the newest packaged
# extension change. A depth-1 clone sees one commit and derives a
# wrong, too-low value rather than failing (ci-requirements.md).
fetch-depth: 0
# The version is DERIVED, not read from the repo (milestone 271 step 4, - name: Resolve extension version
# cut over 2026-08-27). `packaging.sh version` returns MAJOR.MINOR from
# manifest.json plus a patch component that is the commit TIME of the
# newest change to a PACKAGED extension file, in minutes since
# 2020-01-01 — family rule 149, never a commit count, which orders by
# branch rather than by recency.
#
# The committed "version" in manifest.json / package.json no longer
# decides anything: the stamp step below overwrites it in the working
# tree before web-ext ever reads it. It is deliberately NOT committed
# back — the commit carrying the bump would itself be a change to the
# extension and would move the version again. The repo holds the source;
# the build derives the label.
- name: Derive extension version
id: extver id: extver
run: | run: |
set -eu VERSION=$(grep -E '"version"' extension/package.json | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/')
VERSION=$(sh extension/scripts/packaging.sh version)
echo "version=$VERSION" >> "$GITHUB_OUTPUT" echo "version=$VERSION" >> "$GITHUB_OUTPUT"
echo "Derived extension version: $VERSION" echo "Resolved extension version: $VERSION"
# Firefox refuses a downgrade and AMO never releases a burned version,
# so a version that moves BACKWARDS is unrecoverable: it strands every
# install that already took the higher one. Two ways it could happen —
# a checkout without full history (derives too low), or a rewritten
# history that drops the newest packaged commit.
#
# The test is `derived < highest already signed`, strictly. Equality is
# the ORDINARY case, not a fault: an unchanged extension derives the same
# version it did last build, which is exactly what lets the ext-<version>
# cache hit and holds AMO to one call per extension CHANGE. Only moving
# backwards is a failure, so this runs on every path — cache hit
# included — rather than only before a sign.
- name: Guard — the derived version must never go backwards
env:
TOKEN: ${{ secrets.RELEASE_TOKEN }}
DERIVED: ${{ steps.extver.outputs.version }}
run: |
python3 - <<'PY'
import json, os, sys, urllib.request
API = ("https://git.fabledsword.com/api/v1/repos/"
"bvandeusen/FabledCurator/releases")
headers = {"Authorization": "token " + os.environ["TOKEN"]}
# Paginated rather than first-page-only: ext-* releases share this
# list with the v* release tags, so one page would start missing them
# as those accumulate. The bound FAILS rather than silently scanning
# part of the list and calling the highest it saw the highest there is.
tags = []
for page in range(1, 21):
req = urllib.request.Request(
f"{API}?limit=50&page={page}", headers=headers)
with urllib.request.urlopen(req, timeout=30) as resp:
batch = json.load(resp)
if not batch:
break
tags += [r.get("tag_name", "") for r in batch]
else:
sys.exit("guard: >1000 releases — pagination bound reached")
def parse(v):
try:
return tuple(int(part) for part in v.split("."))
except ValueError:
return None
derived_s = os.environ["DERIVED"]
derived = parse(derived_s)
if derived is None:
sys.exit(f"guard: derived version {derived_s!r} is not numeric")
signed = sorted(
(v, t) for t in tags if t.startswith("ext-")
for v in [parse(t[4:])] if v
)
if not signed:
print("guard: no ext-* release yet — nothing to go backwards from")
raise SystemExit(0)
hi, hi_tag = signed[-1]
print(f"guard: derived={derived_s} highest already signed={hi_tag}")
if derived < hi:
sys.exit(
f"REFUSING TO SIGN: derived {derived_s} is OLDER than the "
f"already-signed {hi_tag}. Firefox would reject it as a "
f"downgrade, and AMO will not release the burned version. "
f"First thing to check: did this job check out with "
f"fetch-depth: 0?"
)
print("guard: ok")
PY
- name: Check Forgejo release-asset cache - name: Check Forgejo release-asset cache
id: cache id: cache
@@ -199,29 +84,6 @@ jobs:
# removal — sign-extension's job is just to ensure the cache # removal — sign-extension's job is just to ensure the cache
# exists on Forgejo; the build-web side reads it independently). # exists on Forgejo; the build-web side reads it independently).
# web-ext signs whatever manifest.json says, so the derived value has to
# reach the tree before signing. package.json is written too: the two are
# required to agree (ci.yml's guard), and a local `npm run build` reads
# it. Working tree only — never committed, per the note on the derive
# step.
- name: Stamp the derived version into manifest.json + package.json
env:
DERIVED: ${{ steps.extver.outputs.version }}
run: |
python3 - <<'PY'
import json, os
version = os.environ["DERIVED"]
for path in ("extension/manifest.json", "extension/package.json"):
with open(path) as fh:
doc = json.load(fh)
doc["version"] = version
with open(path, "w") as fh:
json.dump(doc, fh, indent=2)
fh.write("\n")
print(f"{path}: version -> {version}")
PY
- name: Sign via AMO (cache miss) - name: Sign via AMO (cache miss)
if: steps.cache.outputs.cached != 'true' if: steps.cache.outputs.cached != 'true'
run: | run: |
@@ -248,11 +110,6 @@ jobs:
# created it so an upload failure below can roll back (don't # created it so an upload failure below can roll back (don't
# leave an empty release tombstone that the next run's # leave an empty release tombstone that the next run's
# cache-check mistakes for a partial-failure state). # cache-check mistakes for a partial-failure state).
#
# target_commitish is the signing commit, not a branch name: since
# step 6 either branch can create this release, and hard-coding
# `main` would tag a dev-signed XPI against a main commit that may
# not even contain the extension source it was built from.
STATUS=$(curl -s -o release.json -w "%{http_code}" \ STATUS=$(curl -s -o release.json -w "%{http_code}" \
-H "Authorization: token $TOKEN" \ -H "Authorization: token $TOKEN" \
"https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases/tags/ext-$VERSION" || echo 000) "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases/tags/ext-$VERSION" || echo 000)
@@ -260,7 +117,7 @@ jobs:
CREATED_BY_US=false CREATED_BY_US=false
else else
curl -s -X POST -H "Authorization: token $TOKEN" -H "Content-Type: application/json" \ curl -s -X POST -H "Authorization: token $TOKEN" -H "Content-Type: application/json" \
-d "{\"tag_name\":\"ext-$VERSION\",\"name\":\"Extension $VERSION (signed XPI cache)\",\"body\":\"Internal cache for the signed XPI consumed by build.yml's build-web job. Not a user-facing FC release.\",\"target_commitish\":\"$GITHUB_SHA\"}" \ -d "{\"tag_name\":\"ext-$VERSION\",\"name\":\"Extension $VERSION (signed XPI cache)\",\"body\":\"Internal cache for the signed XPI consumed by build.yml's build-web job. Not a user-facing FC release.\",\"target_commitish\":\"main\"}" \
-o release.json \ -o release.json \
"https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases" "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator/releases"
CREATED_BY_US=true CREATED_BY_US=true
@@ -303,31 +160,19 @@ jobs:
build-web: build-web:
needs: [sign-extension] needs: [sign-extension]
# sign-extension runs on main and dev, and is skipped on a tag push (which # sign-extension is main-only; on dev it's skipped, build-web still runs.
# polls for the release instead). Either is fine to build on; a FAILED sign
# is not — this condition lets success and skipped through, so a failure
# skips build-web rather than shipping an image without the XPI.
if: always() && (needs.sign-extension.result == 'success' || needs.sign-extension.result == 'skipped') if: always() && (needs.sign-extension.result == 'success' || needs.sign-extension.result == 'skipped')
runs-on: python-ci runs-on: python-ci
container: container:
image: git.fabledsword.com/bvandeusen/ci-python:3.14 image: git.fabledsword.com/bvandeusen/ci-python:3.14
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with:
# Full history: this job RE-DERIVES the extension version rather than
# being handed it, and a depth-1 clone derives a wrong, too-low value
# rather than failing — which would 404 the download of a release
# that exists perfectly well under its real name.
fetch-depth: 0
- name: Download signed XPI from Forgejo release asset - name: Download signed XPI from Forgejo release asset (main + tags)
# Fires on every trigger shape. dev and main each bundle the XPI their # Fires on main-push AND on tag-push. Tag-push builds re-package the
# own sign-extension just published — that is the whole point of the # same source code as the preceding main-push build but with an
# channel work (milestone 271 step 6): the dev image carries the # immutable version tag — they need the XPI too, otherwise the
# extension being developed, rather than requiring a merge to try it. # versioned image ships without the signed extension.
# Tag-push builds re-package the same source as the preceding main-push
# build but with an immutable version tag — they need the XPI too,
# otherwise the versioned image ships without the signed extension.
# #
# Tag-push vs main-push race (operator-flagged 2026-05-27 after # Tag-push vs main-push race (operator-flagged 2026-05-27 after
# v26.05.27.0 hit it): a release cut fires BOTH workflows almost # v26.05.27.0 hit it): a release cut fires BOTH workflows almost
@@ -339,18 +184,12 @@ jobs:
# for up to 10min total) before giving up. Main-push's signing # for up to 10min total) before giving up. Main-push's signing
# eventually wins and tag-push picks the release up on a later # eventually wins and tag-push picks the release up on a later
# iteration. # iteration.
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/') if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')
env: env:
TOKEN: ${{ secrets.RELEASE_TOKEN }} TOKEN: ${{ secrets.RELEASE_TOKEN }}
run: | run: |
set -eux set -eux
# Re-derived, not read from the repo: sign-extension published VERSION=$(grep -E '"version"' extension/package.json | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/')
# ext-<derived>, and the committed version has been inert since
# milestone 271 step 4. Both jobs run `packaging.sh version` over the
# same commit, so they agree by construction — and if they ever
# didn't, this download 404s and the build fails loudly instead of
# shipping a stale XPI.
VERSION=$(sh extension/scripts/packaging.sh version)
# Poll for the ext-<version> release. main-push's sign-extension # Poll for the ext-<version> release. main-push's sign-extension
# step (AMO round-trip, 1-5min) needs to finish + upload before # step (AMO round-trip, 1-5min) needs to finish + upload before
# tag-push can fetch. 30s * 20 = up to 10min wait, then hard-fail. # tag-push can fetch. 30s * 20 = up to 10min wait, then hard-fail.
@@ -416,11 +255,9 @@ jobs:
# rollback unit"). Rollback to any commit # rollback unit"). Rollback to any commit
# becomes `docker pull …:c-<sha>` without a # becomes `docker pull …:c-<sha>` without a
# release ceremony. # release ceremony.
# refs/heads/dev → push to dev: publish :dev, the rolling test # anything else → safety net; shouldn't fire given the `on:`
# channel (family rule 146). Rolling means it may # config above. Tag :dev to surface the
# carry newer contents than the :c-<sha> of the # unexpected run in the registry.
# same commit; it never writes :c-<sha> itself,
# because that is the rollback unit (rule 145).
# POSIX-safe substring (the runner shell is dash/BusyBox sh, not # POSIX-safe substring (the runner shell is dash/BusyBox sh, not
# bash — `${var:0:7}` errors with "Bad substitution"; cut works # bash — `${var:0:7}` errors with "Bad substitution"; cut works
# everywhere). Operator-flagged 2026-06-01 after first :c-<sha> # everywhere). Operator-flagged 2026-06-01 after first :c-<sha>
+8 -125
View File
@@ -2,7 +2,6 @@ name: CI
# CI lanes per FabledRulebook/forgejo.md "CI philosophy": # CI lanes per FabledRulebook/forgejo.md "CI philosophy":
# - lint: ruff only, no dep install — fast-fail for the common lint bounce. # - lint: ruff only, no dep install — fast-fail for the common lint bounce.
# - extension-version: guards the extension publish path (see the job).
# - backend-lint-and-test: `pytest -m "not integration"`, no service containers. # - backend-lint-and-test: `pytest -m "not integration"`, no service containers.
# - frontend-build: vitest unit + vite build. # - frontend-build: vitest unit + vite build.
# - integration: pgvector + redis service containers; alembic + `pytest -m integration`. # - integration: pgvector + redis service containers; alembic + `pytest -m integration`.
@@ -10,15 +9,10 @@ name: CI
on: on:
push: push:
branches: [dev, main] branches: [dev, main]
# Renovate opens PRs from `renovate/*` branches into `dev`. Those branches # pull_request trigger intentionally absent — with branches: [dev, main]
# never push to dev/main, so the push trigger above gives them NO pre-merge # above, every PR commit already fires CI via the push event on dev. Adding
# CI — a bump could only be validated after it was already merged. This # pull_request would duplicate runs on dev→main PRs. FC has no fork PRs
# pull_request trigger (base `dev` only) validates Renovate PRs before merge. # (single-operator Forgejo repo) so push coverage is complete.
# It deliberately does NOT fire on dev→main PRs (base `main`), which still
# rely on the dev push run — so no duplicate runs. FC has no fork PRs
# (single-operator Forgejo repo), so secrets-on-PR is not a concern.
pull_request:
branches: [dev]
jobs: jobs:
# Fast-fail lint lane. ruff is pre-installed in the ci-python image, so # Fast-fail lint lane. ruff is pre-installed in the ci-python image, so
@@ -42,117 +36,6 @@ jobs:
# catching syntax errors before the image build. # catching syntax errors before the image build.
run: python -m compileall -q agent/fc_agent run: python -m compileall -q agent/fc_agent
# Guards the extension publish path, which has no self-correcting behavior.
#
# build.yml's sign-extension job keys its AMO-signing cache purely on the
# version string in extension/package.json: if an `ext-<version>` Forgejo
# release already carries an XPI, signing is SKIPPED and that old signed XPI
# is what build-web bakes into `:latest`. Nothing in that path inspects
# whether extension/ actually changed — so a forgotten version bump ships a
# stale extension on a fully green build, silently. (AMO can't help: it 409s
# on re-signing a version, which is exactly why the cache exists.)
#
# This job makes that case loud, on the dev push, instead of invisible at
# merge-to-main. It is pure git + text work — no deps, no services.
extension-version:
runs-on: python-ci
container:
image: git.fabledsword.com/bvandeusen/ci-python:3.14
steps:
- uses: actions/checkout@v4
with:
# Full history: the check diffs against the push's `before` SHA (or
# the PR base), which a depth-1 clone wouldn't contain.
fetch-depth: 0
- name: Extension version guard
env:
BEFORE: ${{ github.event.before }}
PR_BASE: ${{ github.event.pull_request.base.sha }}
run: |
set -eu
# busybox sh on the act_runner — no bashisms (family rule).
ver() { grep -E '"version"' "$1" | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/'; }
PKG=$(ver extension/package.json)
MAN=$(ver extension/manifest.json)
test -n "$PKG" || { echo "ERROR: no version found in extension/package.json"; exit 1; }
test -n "$MAN" || { echo "ERROR: no version found in extension/manifest.json"; exit 1; }
# (1) Unconditional: the two version strings must agree. `web-ext sign`
# reads manifest.json (package.json sits in --ignore-files and isn't
# even inside the XPI), so AMO signs MAN and Firefox installs MAN.
# build.yml keys its cache, release tag, XPI filename — and therefore
# the version /api/extension/manifest reports to the update prompt —
# on PKG. Divergence either hard-fails at AMO or ships a mislabelled
# XPI whose update prompt lies about what's installed.
if [ "$MAN" != "$PKG" ]; then
echo "ERROR: extension version mismatch."
echo " extension/manifest.json = $MAN <- what AMO signs / Firefox installs"
echo " extension/package.json = $PKG <- what CI caches, names, and reports"
echo "Set both to the same value."
exit 1
fi
# (2) If the SHIPPED extension changed, the version must have moved.
#
# Compare against MAIN, not against the previous push. The publish
# decision is made at merge-to-main against whatever ext-<version>
# already exists, so "differs from main" is the question that matters.
# Diffing against the previous dev push instead would demand a fresh
# bump on every iteration — push, tweak the extension again, and CI
# would insist on a second bump that buys nothing, inflating the
# version for no reason. On a main push there is no "main to compare
# to" yet, so fall back to that push's own before-SHA.
if [ "${GITHUB_REF##*/}" = "main" ]; then
BASE="${BEFORE:-}"
else
BASE=$(git rev-parse --verify -q origin/main 2>/dev/null || git rev-parse --verify -q main 2>/dev/null || echo "")
# PR base is the fallback when main isn't in the clone at all.
[ -n "$BASE" ] || BASE="${PR_BASE:-}"
fi
case "$BASE" in
''|0000000000000000000000000000000000000000)
echo "No usable base ref (no main in clone / first push) — skipping the bump check."
echo "OK: extension version $PKG"
exit 0
;;
esac
if ! git cat-file -e "$BASE^{commit}" 2>/dev/null; then
echo "Base commit $BASE not in this clone — skipping the bump check."
echo "OK: extension version $PKG"
exit 0
fi
# The exclusion list is NOT written out here — it comes from
# extension/scripts/packaging.sh, the one definition of what ships,
# shared with web-ext's --ignore-files and the derived-version patch
# count. Three hand-kept copies of that fact is how #2397 happened.
#
# `set -f` is required around the substitution: without it the shell
# globs `test/**` against the working tree and silently narrows it.
set -f
CHANGED=$(git diff --name-only "$BASE" HEAD -- extension/ $(sh extension/scripts/packaging.sh pathspec))
set +f
if [ -z "$CHANGED" ]; then
echo "No packaged extension files changed since $BASE — nothing to guard."
echo "OK: extension version $PKG"
exit 0
fi
echo "Packaged extension files changed since $BASE:"
echo "$CHANGED" | sed 's/^/ /'
PKG_OLD=$(git show "$BASE:extension/package.json" 2>/dev/null | grep -E '"version"' | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/')
if [ -z "$PKG_OLD" ]; then
echo "Could not read the base version — skipping the bump check."
echo "OK: extension version $PKG"
exit 0
fi
if [ "$PKG_OLD" = "$PKG" ]; then
echo "ERROR: packaged extension files changed but the version is still $PKG."
echo "build.yml would find the existing ext-$PKG release, skip AMO signing,"
echo "and bake the OLD signed XPI into :latest — a green build shipping stale code."
echo "Bump the version in BOTH extension/package.json and extension/manifest.json."
exit 1
fi
echo "OK: extension version $PKG_OLD -> $PKG"
backend-lint-and-test: backend-lint-and-test:
runs-on: python-ci runs-on: python-ci
container: container:
@@ -209,10 +92,10 @@ jobs:
# If we want strict lockfile-based reproducibility later, commit a # If we want strict lockfile-based reproducibility later, commit a
# package-lock.json and flip this back to `npm ci`. # package-lock.json and flip this back to `npm ci`.
- run: npm install --no-audit --no-fund - run: npm install --no-audit --no-fund
# No type-check step: the frontend is pure JS (no .ts files, no JSDoc), # `npm run check` (vue-tsc --noEmit) skipped: the frontend is pure JS
# so a type-checker has nothing to do. The vue-tsc devDep + its `check` # with no .ts files and no JSDoc annotations, so vue-tsc has nothing
# script were dropped 2026-07-11 rather than bumped to v3. If we add # to type-check. Re-enable once we add a tsconfig.json and either
# TS/JSDoc later, re-add a tsconfig.json + vue-tsc + a type-check step. # convert to TS or add JSDoc.
- run: npm run test:unit - run: npm run test:unit
- run: npm run build - run: npm run build
+4 -57
View File
@@ -1,5 +1,5 @@
name: extension name: extension
# Lint + unit tests. The sign-and-publish dance moved into build.yml's # Lint-only workflow. The sign-and-publish dance moved into build.yml's
# `sign-extension` job (2026-05-25) — `:latest` now always bundles the XPI # `sign-extension` job (2026-05-25) — `:latest` now always bundles the XPI
# because sign-extension runs as a build-web dependency in the SAME workflow, # because sign-extension runs as a build-web dependency in the SAME workflow,
# eliminating the prior race between build.yml and a separate extension.yml. # eliminating the prior race between build.yml and a separate extension.yml.
@@ -10,73 +10,20 @@ on:
paths: paths:
- 'extension/**' - 'extension/**'
- '.forgejo/workflows/extension.yml' - '.forgejo/workflows/extension.yml'
# test/version.spec.js asserts ci.yml's extension-version guard never
# ignores a file web-ext actually packages, so a ci.yml-only edit can
# break this suite and must trigger it.
- '.forgejo/workflows/ci.yml'
pull_request: pull_request:
branches: [main] branches: [main]
paths: paths:
- 'extension/**' - 'extension/**'
- '.forgejo/workflows/ci.yml'
workflow_dispatch: workflow_dispatch:
jobs: jobs:
lint: lint:
runs-on: python-ci runs-on: python-ci
container: container:
image: node:24-bookworm-slim image: node:22-bookworm-slim
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
# Not --no-save: vitest and web-ext are both real devDependencies now, - name: Install web-ext
# and the suite needs vitest resolvable from node_modules. run: cd extension && npm install --no-save --no-audit --no-fund
- name: Install dev dependencies
run: cd extension && npm install --no-audit --no-fund
- name: Lint - name: Lint
run: cd extension && npm run lint run: cd extension && npm run lint
# Pure-logic specs over lib/url.js and lib/platforms.js plus manifest /
# package version-consistency checks. No browser, no network.
- name: Unit tests
run: cd extension && npm run test:unit
# Everything else about packaging is asserted against our own declaration
# of what ships. This is the only check that asks web-ext what it ACTUALLY
# put in the archive. Until now that was an unverified assumption about
# glob semantics — and a fragile one: `test/**` reaches web-ext intact
# only because callers `set -f` first, so losing that quoting would
# silently start shipping dev files with no other signal.
- name: Verify XPI contents
run: |
set -eu
command -v unzip >/dev/null 2>&1 || { apt-get update -qq && apt-get install -y -qq unzip; }
cd extension
npm run build
ZIP=$(ls web-ext-artifacts/*.zip | head -1)
echo "=== packaged entries in $ZIP ==="
unzip -Z1 "$ZIP" | sort
echo "=== end ==="
ENTRIES=$(unzip -Z1 "$ZIP")
fail=0
# Must NOT ship: repo infrastructure with no business in a user's browser.
for pat in 'test/' 'scripts/' 'vitest.config.js' 'package.json' 'package-lock.json' 'README.md' 'node_modules/' 'web-ext-artifacts/'; do
if echo "$ENTRIES" | grep -q "^$pat"; then
echo "ERROR: '$pat' was packaged into the XPI but must not be"
fail=1
fi
done
# Must ship: if an exclusion pattern ever over-matches, the extension
# breaks at runtime rather than at build time, so assert presence too.
for req in 'manifest.json' 'lib/url.js' 'lib/api.js' 'lib/platforms.js' 'lib/cookies.js'; do
if ! echo "$ENTRIES" | grep -q "^$req$"; then
echo "ERROR: '$req' is missing from the XPI"
fail=1
fi
done
for dir in 'background/' 'popup/' 'options/' 'content/' 'icons/'; do
if ! echo "$ENTRIES" | grep -q "^$dir"; then
echo "ERROR: nothing from '$dir' was packaged"
fail=1
fi
done
[ "$fail" -eq 0 ] || exit 1
echo "XPI contents verified."
+2 -2
View File
@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:1.25 # syntax=docker/dockerfile:1.7
FROM node:24-alpine AS frontend-builder FROM node:22-alpine AS frontend-builder
WORKDIR /build WORKDIR /build
COPY frontend/package.json frontend/package-lock.json* ./ COPY frontend/package.json frontend/package-lock.json* ./
# No package-lock.json is tracked yet (we don't run npm locally per # No package-lock.json is tracked yet (we don't run npm locally per
+1 -1
View File
@@ -1,4 +1,4 @@
# syntax=docker/dockerfile:1.25 # syntax=docker/dockerfile:1.7
FROM python:3.14-slim FROM python:3.14-slim
ENV PYTHONUNBUFFERED=1 \ ENV PYTHONUNBUFFERED=1 \
+6 -35
View File
@@ -6,21 +6,7 @@ Combines what was [ImageRepo](https://git.fabledsword.com/bvandeusen/ImageRepo)
## Status ## Status
In production. `main` is continuously deployed — every merge to `main` builds Pre-v1. Not yet functional.
and publishes `:latest` images, so whatever is on `main` is what is running.
Day-to-day work happens on `dev`, which publishes `:dev` images.
## What's in here
Five deployable pieces, built by `.forgejo/workflows/build.yml`:
| Piece | Built from | Image | Role |
| --- | --- | --- | --- |
| **Web / workers** | `Dockerfile` | `fabledcurator` | Quart API + the built Vue SPA in one image. `entrypoint.sh` picks the role: `web`, `worker`, `scheduler`. The `maintenance-long` service is a second `worker` pinned to the long-running maintenance queue. |
| **ML worker** | `Dockerfile.ml` | `fabledcurator-ml` | Same app, plus `requirements-ml.txt` — tagging and embedding models that run in-container. |
| **GPU agent** | `agent/Dockerfile` | `fabledcurator-agent` | Optional desktop-GPU worker (`agent/`). Leases jobs over **HTTP only** — never touches the database or Redis. Run it for a burst, stop it to reclaim the card. See `agent/README.md`. |
| **Firefox extension** | `extension/` | signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on `main` only, then bundled into the web image and served from Settings → Maintenance. See `extension/README.md`. |
| **Data** | — | `pgvector/pgvector:pg16`, `redis:7-alpine` | Postgres with pgvector for embeddings; Redis as the Celery broker. |
## Quick start ## Quick start
@@ -43,37 +29,22 @@ docker compose -f docker-compose.yml up -d
# (skips the override so containers pull registry images) # (skips the override so containers pull registry images)
``` ```
The GPU agent is deployed separately, on the machine with the card —
`agent/docker-compose.yml`, not this stack.
## Deployment posture ## Deployment posture
FabledCurator is designed to run inside a self-hosted homelab environment over plain HTTP. If you want TLS, terminate it at your reverse proxy. The app does not generate certificates, redirect to HTTPS, or set HSTS. FabledCurator is designed to run inside a self-hosted homelab environment over plain HTTP. If you want TLS, terminate it at your reverse proxy. The app does not generate certificates, redirect to HTTPS, or set HSTS.
## CI / Forgejo setup ## CI / Forgejo setup
Three workflows: `ci.yml` (lint, extension-version guard, backend unit tests, The repo's workflows expect:
frontend build, integration), `extension.yml` (extension lint, vitest, XPI
content verification), and `build.yml` (sign + publish).
**The toolchain each job runs in is its `container.image`, not its `runs-on` - **Runner label `python-ci`** — a Forgejo runner with Python 3.14, ruff, and Node 22 pre-installed. Both `ci.yml` and `build.yml` use this label. The runner image (`runner-base:python-ci`) is built from `CI-Runner/CI-python/` in the operator's workspace; `make push` from that directory builds and pushes a new image when toolchain pins change.
label.** `runs-on: python-ci` only schedules the job onto a runner; every job - **Repo secret `RELEASE_TOKEN`** — a Forgejo PAT with the following scopes:
then names the image it actually wants. `ci-requirements.md` is the current,
authoritative list of images and per-job installs — read that rather than a
copy here, so the two can't drift.
The repo expects one secret:
- **`RELEASE_TOKEN`** — a Forgejo PAT with:
- `write:package` + `read:package` — for `docker push` to `git.fabledsword.com` - `write:package` + `read:package` — for `docker push` to `git.fabledsword.com`
- `write:release` — for the `ext-<version>` releases that cache the signed XPI - `write:release` — for future release-cutting workflows
- `write:issue` — for issue-management automation - `write:issue` — for future issue-management automation
Generate at https://git.fabledsword.com/user/settings/applications. The injected `GITHUB_TOKEN` cannot be used because it lacks `write:package`. Generate at https://git.fabledsword.com/user/settings/applications. The injected `GITHUB_TOKEN` cannot be used because it lacks `write:package`.
AMO signing additionally needs `MOZILLA_AMO_JWT_KEY` / `MOZILLA_AMO_JWT_SECRET`; it runs on
`main` only and is cached per version, since AMO rejects a re-signed version.
## License ## License
Personal project; use at your own discretion. Personal project; use at your own discretion.
+1 -4
View File
@@ -21,7 +21,7 @@ log = logging.getLogger("fc_agent.app")
# Bump on every agent change. The page embeds this and /status reports it; the UI # Bump on every agent change. The page embeds this and /status reports it; the UI
# warns to reload when they differ — so a stale browser-cached page can't be # warns to reload when they differ — so a stale browser-cached page can't be
# mistaken for "the new image didn't deploy". (Belt-and-braces with no-store.) # mistaken for "the new image didn't deploy". (Belt-and-braces with no-store.)
VERSION = "2026-07-17.1 · idle model-unload: after ~5 min idle the GPU models release their VRAM and reload on the next job (env IDLE_UNLOAD_SECONDS, 0=off) · sleep mode sheds to one downloader" VERSION = "2026-07-02.6 · sleep mode: an empty queue sheds to one downloader and backs the lease poll off to 15 min"
logbuf.install() logbuf.install()
cfg = Config.from_env() cfg = Config.from_env()
@@ -334,12 +334,9 @@ _PAGE = """<!doctype html><html><head><meta charset=utf-8>
waited.textContent=s.transient||0 waited.textContent=s.transient||0
// Instantaneous pool state → demoted to the sub-line, where its jumpiness reads // Instantaneous pool state → demoted to the sub-line, where its jumpiness reads
// as live churn rather than a "broken" headline metric. // as live churn rather than a "broken" headline metric.
// '=== false' (not falsy) so a stale page that doesn't send models_loaded shows
// nothing; when the idle monitor unloads, the VRAM meter drops alongside this.
pipe.textContent='downloaders '+(s.downloaders!=null?s.downloaders:'')+' · consumers '+(s.consumers!=null?s.consumers:'')+' · on GPU '+(s.active||0) pipe.textContent='downloaders '+(s.downloaders!=null?s.downloaders:'')+' · consumers '+(s.consumers!=null?s.consumers:'')+' · on GPU '+(s.active||0)
+' · net '+(s.net_mb_s!=null?s.net_mb_s.toFixed(1):'')+' MB/s' +' · net '+(s.net_mb_s!=null?s.net_mb_s.toFixed(1):'')+' MB/s'
+(s.bandwidth_limit_mb_s>0?(' / cap '+s.bandwidth_limit_mb_s):'') +(s.bandwidth_limit_mb_s>0?(' / cap '+s.bandwidth_limit_mb_s):'')
+(s.models_loaded===false?' · GPU models unloaded (idle — reload on next job)':'')
if(document.activeElement!==bw && s.bandwidth_limit_mb_s!=null) bw.value=s.bandwidth_limit_mb_s if(document.activeElement!==bw && s.bandwidth_limit_mb_s!=null) bw.value=s.bandwidth_limit_mb_s
// Buffer occupancy bar (also driven here so it tracks the /status cadence). // Buffer occupancy bar (also driven here so it tracks the /status cadence).
if(s.buffer!=null && s.buffer_max){ const p=Math.round(100*s.buffer/s.buffer_max) if(s.buffer!=null && s.buffer_max){ const p=Math.round(100*s.buffer/s.buffer_max)
-10
View File
@@ -51,12 +51,6 @@ class Config:
bandwidth_limit_mb_s: float # aggregate download cap in MEGABYTES/s across bandwidth_limit_mb_s: float # aggregate download cap in MEGABYTES/s across
# all downloaders + video streams (0 = unlimited); # all downloaders + video streams (0 = unlimited);
# tunable live from the agent UI # tunable live from the agent UI
idle_unload_seconds: float # after this long with the GPU idle (nothing in
# flight, queue empty or Stopped), unload the
# SigLIP embedder + YOLO proposers to free their
# VRAM; they reload lazily on the next job. A
# 24/7 agent otherwise squats on ~5GB doing
# nothing. 0 disables (keep models warm forever).
@classmethod @classmethod
def from_env(cls) -> Config: def from_env(cls) -> Config:
@@ -93,8 +87,4 @@ class Config:
# link to ~1-1.5 MB/s per stream, browser included). Raise it (or 0) # link to ~1-1.5 MB/s per stream, browser included). Raise it (or 0)
# from the agent UI on wired/faster networks. # from the agent UI on wired/faster networks.
bandwidth_limit_mb_s=float(os.environ.get("BANDWIDTH_LIMIT_MB_S", "8")), bandwidth_limit_mb_s=float(os.environ.get("BANDWIDTH_LIMIT_MB_S", "8")),
# 5 min: long enough that a lull between job bursts doesn't thrash the
# (few-second) reload, short enough that an agent left running with an
# empty queue hands its VRAM back promptly.
idle_unload_seconds=float(os.environ.get("IDLE_UNLOAD_SECONDS", "300")),
) )
-15
View File
@@ -170,13 +170,6 @@ class YoloProposer:
)) ))
return out return out
def unload(self) -> None:
"""Drop the loaded YOLO so its VRAM can be reclaimed; detect() reloads it
lazily on the next job. Leaves _ok untouched — a healthy proposer comes
back, but one that self-disabled on a fault stays off."""
with self._lock:
self._model = None
class Proposers: class Proposers:
"""The agent's proposer set, built from config. Each detector is optional — """The agent's proposer set, built from config. Each detector is optional —
@@ -223,11 +216,3 @@ class Proposers:
def panels(self, image): def panels(self, image):
return self._top(self._panel, image, self.cfg.max_panels) return self._top(self._panel, image, self.cfg.max_panels)
def unload(self) -> None:
"""Release every loaded proposer's YOLO (idle VRAM reclaim). The worker
also drops its reference to this Proposers and rebuilds a fresh one via
_proposers_for on the next job, so this is belt-and-braces."""
for p in (self._person, self._anatomy, self._panel):
if p is not None:
p.unload()
-15
View File
@@ -75,18 +75,3 @@ class CropEmbedder:
pooled = out.pooler_output if hasattr(out, "pooler_output") else out pooled = out.pooler_output if hasattr(out, "pooler_output") else out
arr = pooled.float().cpu().numpy().astype(np.float32) arr = pooled.float().cpu().numpy().astype(np.float32)
return [row.reshape(-1).tolist() for row in arr] return [row.reshape(-1).tolist() for row in arr]
def unload(self) -> bool:
"""Drop the loaded model so its VRAM can be reclaimed — the idle monitor
calls this after a spell with no work so an idle agent doesn't squat on
the card; the next embed() reloads it lazily (a few seconds). Held under
BOTH the load and inference locks so it can never race a concurrent load
or an in-flight forward pass. Returns True if a model was actually
released (the caller then runs one empty_cache() to hand the freed blocks
back to the driver)."""
with self._load_lock, self._infer_lock:
if self._model is None:
return False
self._model = None
self._processor = None
return True
-75
View File
@@ -57,15 +57,6 @@ MAX_BACKOFF_SECONDS = 60.0
# up on their own. # up on their own.
IDLE_POLL_MAX_SECONDS = 900.0 IDLE_POLL_MAX_SECONDS = 900.0
# Idle VRAM reclaim (operator 2026-07-17): the SigLIP embedder + YOLO proposers
# load lazily and then stay warm for fast job bursts — but a 24/7 agent with an
# empty queue would otherwise squat on that VRAM (~5GB on the operator's card)
# indefinitely while doing nothing. So a monitor unloads them after
# cfg.idle_unload_seconds with the GPU genuinely idle (nothing in flight, buffer
# drained); they reload lazily on the next job. This is just how often the
# monitor wakes to check — it bounds how soon past the threshold the unload fires.
IDLE_UNLOAD_CHECK_INTERVAL = 30.0
# A job whose fetch dies transiently this many times IN ONE SESSION stops being # A job whose fetch dies transiently this many times IN ONE SESSION stops being
# handed back and is failed instead. Transient handbacks (release) burn no # handed back and is failed instead. Transient handbacks (release) burn no
# attempts on the server, so a poisoned transfer — an original that stalls the # attempts on the server, so a poisoned transfer — an original that stalls the
@@ -277,11 +268,6 @@ class Worker:
self._proposers_sig = None # detector-config signature the current self._proposers_sig = None # detector-config signature the current
# proposers were built for (#134) # proposers were built for (#134)
self._proposers_lock = threading.Lock() self._proposers_lock = threading.Lock()
# Monotonic time of the last GPU activity (a consumer finishing a job).
# The idle monitor unloads the warm models once this goes stale by
# cfg.idle_unload_seconds — see _idle_unload_loop.
self._last_gpu_activity = time.monotonic()
threading.Thread(target=self._idle_unload_loop, daemon=True).start()
# --- held-lease bookkeeping -------------------------------------------- # --- held-lease bookkeeping --------------------------------------------
def _hold(self, job_ids) -> None: def _hold(self, job_ids) -> None:
@@ -622,9 +608,6 @@ class Worker:
"net_mb_s": round(self._net_mb_s, 1), # observed aggregate rate "net_mb_s": round(self._net_mb_s, 1), # observed aggregate rate
"bw_capped": self._bw_capped, # autoscaler holding at the cap (UI hint) "bw_capped": self._bw_capped, # autoscaler holding at the cap (UI hint)
"idle": self._idle, # queue empty → poll backed off (UI hint) "idle": self._idle, # queue empty → poll backed off (UI hint)
# Whether the GPU models are currently resident (False after an idle
# unload freed their VRAM) — a plain bool read, UI hint only.
"models_loaded": self._embedder is not None or self._proposers is not None,
} }
def _bump(self, *, processed=0, downloaded=0, errors=0, active=0, transient=0): def _bump(self, *, processed=0, downloaded=0, errors=0, active=0, transient=0):
@@ -805,9 +788,6 @@ class Worker:
self._bump(processed=1) self._bump(processed=1)
finally: finally:
self._bump(active=-1) self._bump(active=-1)
# Mark the GPU busy-until-now so the idle monitor starts its
# unload countdown from when work actually stopped, not before.
self._last_gpu_activity = time.monotonic()
def _ensure_embedder(self, model_name: str): def _ensure_embedder(self, model_name: str):
if self._embedder is not None: if self._embedder is not None:
@@ -865,61 +845,6 @@ class Worker:
self._proposers_sig = sig self._proposers_sig = sig
return self._proposers return self._proposers
def _unload_models(self) -> bool:
"""Release the GPU-resident models (SigLIP embedder + YOLO proposers) so an
idle agent hands their VRAM back instead of squatting on the card. They
reload lazily on the next job (_ensure_embedder / _proposers_for) — a
few seconds' cost paid only when work actually resumes. Dropping the
shared instances under their build locks means a concurrent job either
sees the old instance (before) or rebuilds a fresh one (after); the idle
monitor only calls this with nothing in flight, so no inference is using
them. Returns True if anything was released."""
released = False
with self._embedder_lock:
if self._embedder is not None:
self._embedder.unload()
self._embedder = None
released = True
with self._proposers_lock:
if self._proposers is not None:
self._proposers.unload()
self._proposers = None
self._proposers_sig = None
released = True
if released:
try:
import torch
if torch.cuda.is_available():
# torch's caching allocator holds freed blocks; hand them back
# to the driver so nvidia-smi actually reflects the drop.
torch.cuda.empty_cache()
except Exception: # noqa: BLE001 — torch absent / CPU-only → nothing to free
pass
return released
def _idle_unload_loop(self) -> None:
"""Unload the warm GPU models after a stretch of inactivity so a 24/7
agent with an empty queue doesn't hold ~5GB of VRAM doing nothing. Fires
only when nothing is in flight (active == 0 AND the buffer is drained) and
no job has completed for cfg.idle_unload_seconds — a window long enough
that a brief lull between bursts doesn't thrash reload/unload. Covers BOTH
sleep mode (queue empty, pipeline still running) and a full Stop; the
models reload lazily on the next job. idle_unload_seconds <= 0 disables it."""
idle_after = self.cfg.idle_unload_seconds
if idle_after <= 0:
return
while True:
time.sleep(IDLE_UNLOAD_CHECK_INTERVAL)
if self._embedder is None and self._proposers is None:
continue # nothing loaded → nothing to free
if self._active != 0 or not self._buffer.empty():
continue # work in flight → keep them warm
if time.monotonic() - self._last_gpu_activity < idle_after:
continue # not idle long enough yet
if self._unload_models():
log.info("idle %.0fs — unloaded GPU models, freed VRAM "
"(reload on next job)", idle_after)
def _consume(self, job: dict, frames: list, stop_evt: threading.Event) -> bool: def _consume(self, job: dict, frames: list, stop_evt: threading.Event) -> bool:
"""Detect + embed the decoded frames and submit the result. Returns True """Detect + embed the decoded frames and submit the result. Returns True
when the job was completed (→ count it processed), False otherwise: a when the job was completed (→ count it processed), False otherwise: a
@@ -1,51 +0,0 @@
"""translation strictness setting + per-post translation override (milestone 155)
ImportSettings gains ``translation_min_confidence`` (the latin-script acceptance
floor, now operator-tunable in the UI; default 0.9 — stricter than the old
hardcoded 0.8, since Interpreter confidently mis-detects short ASCII English at
~0.86). Post gains ``translation_override`` — a sticky per-post choice of
auto / force / original so the operator can force a skipped translation on, or
knock a wrongly-translated one back to the original, and have it survive a
Re-translate-all.
Revision ID: 0084
Revises: 0083
Create Date: 2026-07-10
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0084"
down_revision: Union[str, None] = "0083"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"import_settings",
sa.Column(
"translation_min_confidence", sa.Float(), nullable=False,
server_default=sa.text("0.9"),
),
)
op.add_column(
"post",
sa.Column(
"translation_override", sa.String(16), nullable=False,
server_default="auto",
),
)
op.create_check_constraint(
"ck_post_translation_override",
"post",
"translation_override IN ('auto', 'force', 'original')",
)
def downgrade() -> None:
op.drop_constraint("ck_post_translation_override", "post", type_="check")
op.drop_column("post", "translation_override")
op.drop_column("import_settings", "translation_min_confidence")
@@ -1,35 +0,0 @@
"""title-based WIP auto-tagging (task #1458) — ImportSettings toggle
ImportSettings gains wip_title_tagging_enabled (ON by default): when a freshly
imported post's title explicitly declares work-in-progress ("WIP" / "work in
progress"), the importer applies the `wip` system tag to its images. No new
table — the tag itself is the seeded `wip` system tag (migration 0075) and the
application reuses image_tag with source='wip_title'.
Revision ID: 0085
Revises: 0084
Create Date: 2026-07-12
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0085"
down_revision: Union[str, None] = "0084"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"import_settings",
sa.Column(
"wip_title_tagging_enabled", sa.Boolean(), nullable=False,
server_default=sa.text("true"),
),
)
def downgrade() -> None:
op.drop_column("import_settings", "wip_title_tagging_enabled")
@@ -1,61 +0,0 @@
"""process auto-apply settings + review mode (#1464) — system-tag refactor
The system-tag behavior refactor gives `wip` / `editor screenshot` (the PROCESS
group) their own provisional auto-apply, parallel to the presentation (chrome)
sweep. MLSettings gains three knobs: enabled (OFF by default — a new whole-library
auto-tagger is opt-in), the flat apply threshold, and the ring-loud conflict
threshold. presentation_review gains a `mode` column so one review surface serves
both chrome and process flags (existing rows backfill 'chrome'). server_defaults
so the existing rows fill cleanly.
Revision ID: 0086
Revises: 0085
Create Date: 2026-07-13
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0086"
down_revision: Union[str, None] = "0085"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"ml_settings",
sa.Column(
"process_auto_apply_enabled", sa.Boolean(), nullable=False,
server_default=sa.text("false"),
),
)
op.add_column(
"ml_settings",
sa.Column(
"process_auto_apply_threshold", sa.Float(), nullable=False,
server_default="0.90",
),
)
op.add_column(
"ml_settings",
sa.Column(
"process_conflict_threshold", sa.Float(), nullable=False,
server_default="0.50",
),
)
op.add_column(
"presentation_review",
sa.Column(
"mode", sa.String(16), nullable=False,
server_default="chrome",
),
)
def downgrade() -> None:
op.drop_column("presentation_review", "mode")
op.drop_column("ml_settings", "process_conflict_threshold")
op.drop_column("ml_settings", "process_auto_apply_threshold")
op.drop_column("ml_settings", "process_auto_apply_enabled")
@@ -1,33 +0,0 @@
"""soft WIP title tier toggle (#1474) — ImportSettings.wip_soft_title_tagging_enabled
The soft tier also tags sketch/doodle/scribble titles, but with a provisional source
that never trains the head. OFF by default (a lower-precision tier is opt-in).
server_default so the existing singleton row (id=1) fills cleanly.
Revision ID: 0087
Revises: 0086
Create Date: 2026-07-13
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0087"
down_revision: Union[str, None] = "0086"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"import_settings",
sa.Column(
"wip_soft_title_tagging_enabled", sa.Boolean(), nullable=False,
server_default=sa.text("false"),
),
)
def downgrade() -> None:
op.drop_column("import_settings", "wip_soft_title_tagging_enabled")
-16
View File
@@ -459,22 +459,6 @@ async def trigger_prune_missing_files():
return _queued(async_result) return _queued(async_result)
@admin_bp.route("/maintenance/reclaim-attachments", methods=["POST"])
async def trigger_reclaim_attachments():
"""Reclaim orphaned attachments (#3068). Body {"dry_run": bool}: dry_run
(the DEFAULT here) projects the orphan rows and unreferenced store blobs
without touching either; dry_run=false deletes the rows then unlinks every
blob no surviving row references. Maintenance queue; operator-triggered
only — never an unattended sweep, since the apply unlinks files. Returns the
Celery task id — poll /maintenance/task-result/<id> for the summary."""
from ..tasks.admin import reclaim_orphaned_attachments_task
body = await request.get_json(silent=True) or {}
dry_run = bool(body.get("dry_run", True)) # default to the SAFE preview
async_result = reclaim_orphaned_attachments_task.delay(dry_run=dry_run)
return _queued(async_result)
@admin_bp.route("/maintenance/dedup-videos", methods=["POST"]) @admin_bp.route("/maintenance/dedup-videos", methods=["POST"])
async def trigger_dedup_videos(): async def trigger_dedup_videos():
"""Tier-1 video dedup (#871). Body {"dry_run": bool}: dry_run=true previews """Tier-1 video dedup (#871). Body {"dry_run": bool}: dry_run=true previews
+1 -10
View File
@@ -6,7 +6,6 @@ from __future__ import annotations
import asyncio import asyncio
import hashlib import hashlib
import hmac
import re import re
from pathlib import Path from pathlib import Path
@@ -42,15 +41,7 @@ async def _ext_key_required(session) -> bool:
stored = (await session.execute( stored = (await session.execute(
select(AppSetting.value).where(AppSetting.key == "extension_api_key") select(AppSetting.value).where(AppSetting.key == "extension_api_key")
)).scalar_one_or_none() )).scalar_one_or_none()
if stored is None: return stored is not None and supplied == stored
return False
# compare_digest, not `==`: the stored key is a shared secret, and a
# short-circuiting compare leaks its prefix through timing. Costs nothing
# here — it is not that this route is exposed (#3072). Compared as BYTES:
# compare_digest's str form rejects non-ASCII with TypeError, and this
# header is attacker-supplied, so a str compare would turn a junk key into
# a 500 instead of a 403.
return hmac.compare_digest(supplied.encode("utf-8"), stored.encode("utf-8"))
def _extract_version(xpi_name: str) -> str: def _extract_version(xpi_name: str) -> str:
+3 -19
View File
@@ -148,17 +148,6 @@ async def similar():
# Explore passes exclude_wip=1 to also drop work-in-progress from the # Explore passes exclude_wip=1 to also drop work-in-progress from the
# rabbit-hole; the gallery's own "similar" button omits it (keeps wip, #1274). # rabbit-hole; the gallery's own "similar" button omits it (keeps wip, #1274).
exclude_wip = request.args.get("exclude_wip") in ("1", "true", "True") exclude_wip = request.args.get("exclude_wip") in ("1", "true", "True")
# Explore reach (#1476): 0 = nearest (gallery default), →1 reaches into farther
# distance bands so the walk can escape a dense cluster. exclude_ids = the
# breadcrumb, so already-walked images aren't re-served as neighbours.
try:
reach = max(0.0, min(1.0, float(request.args.get("reach", "0"))))
except ValueError:
reach = 0.0
exclude_ids = [
int(x) for x in request.args.get("exclude_ids", "").split(",")
if x.strip().isdigit()
] or None
# post_id is the exclusive post-detail view — not a similarity scope. # post_id is the exclusive post-detail view — not a similarity scope.
# include_hidden is a gallery-browse flag; similar() has its OWN presentation # include_hidden is a gallery-browse flag; similar() has its OWN presentation
# exclusion (a similarity-quality concern, #1274), so drop it here (#141). # exclusion (a similarity-quality concern, #1274), so drop it here (#141).
@@ -169,8 +158,7 @@ async def similar():
svc = GalleryService(session) svc = GalleryService(session)
try: try:
images = await svc.similar( images = await svc.similar(
image_id=similar_to, limit=limit, exclude_wip=exclude_wip, image_id=similar_to, limit=limit, exclude_wip=exclude_wip, **scope)
reach=reach, exclude_ids=exclude_ids, **scope)
except ValueError as exc: except ValueError as exc:
return jsonify({"error": str(exc)}), 400 return jsonify({"error": str(exc)}), 400
if images is None: if images is None:
@@ -248,10 +236,8 @@ async def jump():
# content", surfaced in the gallery's Show-hidden review strip. ----------- # content", surfaced in the gallery's Show-hidden review strip. -----------
@gallery_bp.route("/hidden-review", methods=["GET"]) @gallery_bp.route("/hidden-review", methods=["GET"])
async def hidden_review(): async def hidden_review():
"""Unresolved system-tag auto-apply review flags (chrome + process, #1464), """Unresolved presentation auto-hide flags, most-concerning first (highest
most-concerning first (highest content score) — for the review strip. `mode` content score) — for the gallery's Hidden-view review strip."""
tells the client whether the flagged tag hid the image ('chrome') or left it
visible ('process'), which decides the resolve labels (un-hide vs remove-tag)."""
ptag = aliased(Tag) ptag = aliased(Tag)
ctag = aliased(Tag) ctag = aliased(Tag)
async with get_session() as session: async with get_session() as session:
@@ -261,7 +247,6 @@ async def hidden_review():
PresentationReview.tag_id, PresentationReview.tag_id,
PresentationReview.conflict_tag_id, PresentationReview.conflict_tag_id,
PresentationReview.conflict_score, PresentationReview.conflict_score,
PresentationReview.mode,
ImageRecord.path, ImageRecord.thumbnail_path, ImageRecord.path, ImageRecord.thumbnail_path,
ImageRecord.sha256, ImageRecord.mime, ImageRecord.sha256, ImageRecord.mime,
ptag.name.label("tag_name"), ptag.name.label("tag_name"),
@@ -281,7 +266,6 @@ async def hidden_review():
"conflict_tag_id": r.conflict_tag_id, "conflict_tag_id": r.conflict_tag_id,
"conflict_name": r.conflict_name, "conflict_name": r.conflict_name,
"conflict_score": r.conflict_score, "conflict_score": r.conflict_score,
"mode": r.mode,
"thumbnail_url": thumbnail_url(r.thumbnail_path, r.sha256, r.mime), "thumbnail_url": thumbnail_url(r.thumbnail_path, r.sha256, r.mime),
"image_url": image_url(r.path), "image_url": image_url(r.path),
} }
+3 -1
View File
@@ -256,7 +256,9 @@ async def lease():
if not await _agent_authed(session): if not await _agent_authed(session):
return jsonify({"error": "unauthorized"}), 401 return jsonify({"error": "unauthorized"}), 401
jobs = await GpuJobService(session).lease(agent_id, batch_size=batch) jobs = await GpuJobService(session).lease(agent_id, batch_size=batch)
ml = await MLSettings.load(session) ml = (
await session.execute(select(MLSettings).where(MLSettings.id == 1))
).scalar_one()
# image rows for url/mime in one shot # image rows for url/mime in one shot
ids = [j.image_record_id for j in jobs] ids = [j.image_record_id for j in jobs]
imgs = { imgs = {
+38 -24
View File
@@ -4,7 +4,6 @@ from quart import Blueprint, jsonify, request
from ..extensions import get_session from ..extensions import get_session
from ..models import MLSettings from ..models import MLSettings
from ..services.ml.heads import AUTO_APPLY_THRESHOLD_MAX, AUTO_APPLY_THRESHOLD_MIN
ml_admin_bp = Blueprint("ml_admin", __name__, url_prefix="/api/ml") ml_admin_bp = Blueprint("ml_admin", __name__, url_prefix="/api/ml")
@@ -43,9 +42,6 @@ _EDITABLE = (
"presentation_auto_apply_enabled", "presentation_auto_apply_enabled",
"presentation_auto_apply_threshold", "presentation_auto_apply_threshold",
"presentation_conflict_threshold", "presentation_conflict_threshold",
"process_auto_apply_enabled",
"process_auto_apply_threshold",
"process_conflict_threshold",
"embedder_model_name", "embedder_model_name",
"embedder_model_version", "embedder_model_version",
*_DETECTOR_FIELDS, *_DETECTOR_FIELDS,
@@ -84,21 +80,45 @@ async def embedder_models():
@ml_admin_bp.route("/settings", methods=["GET"]) @ml_admin_bp.route("/settings", methods=["GET"])
async def get_settings(): async def get_settings():
from sqlalchemy import select
async with get_session() as session: async with get_session() as session:
s = await MLSettings.load(session) s = (
# Table-driven off _EDITABLE (which PATCH also writes) so a new settings field await session.execute(select(MLSettings).where(MLSettings.id == 1))
# can never be silently absent from GET — the split that historically dropped ).scalar_one()
# fields. _EDITABLE already includes *_DETECTOR_FIELDS. return jsonify(
return jsonify({f: getattr(s, f) for f in _EDITABLE}) {
"cpu_embed_enabled": s.cpu_embed_enabled,
"video_frame_interval_seconds": s.video_frame_interval_seconds,
"video_max_frames": s.video_max_frames,
"embedder_model_version": s.embedder_model_version,
"head_min_positives": s.head_min_positives,
"head_auto_apply_precision": s.head_auto_apply_precision,
"head_auto_apply_enabled": s.head_auto_apply_enabled,
"head_auto_apply_min_positives": s.head_auto_apply_min_positives,
"ccip_match_threshold": s.ccip_match_threshold,
"ccip_auto_apply_enabled": s.ccip_auto_apply_enabled,
"ccip_auto_apply_threshold": s.ccip_auto_apply_threshold,
"presentation_auto_apply_enabled": s.presentation_auto_apply_enabled,
"presentation_auto_apply_threshold": s.presentation_auto_apply_threshold,
"presentation_conflict_threshold": s.presentation_conflict_threshold,
"embedder_model_name": s.embedder_model_name,
**{f: getattr(s, f) for f in _DETECTOR_FIELDS},
}
)
@ml_admin_bp.route("/settings", methods=["PATCH"]) @ml_admin_bp.route("/settings", methods=["PATCH"])
async def patch_settings(): async def patch_settings():
from sqlalchemy import select
body = await request.get_json() body = await request.get_json()
if not isinstance(body, dict): if not isinstance(body, dict):
return jsonify({"error": "body must be an object"}), 400 return jsonify({"error": "body must be an object"}), 400
async with get_session() as session: async with get_session() as session:
s = await MLSettings.load(session) s = (
await session.execute(select(MLSettings).where(MLSettings.id == 1))
).scalar_one()
# Merge the patch over current values, then validate the result as a # Merge the patch over current values, then validate the result as a
# whole — the store-floor invariant couples three fields, so they # whole — the store-floor invariant couples three fields, so they
@@ -128,26 +148,20 @@ def _validate(p: dict) -> str | None:
# Head training (#114). # Head training (#114).
if int(p["head_min_positives"]) < 1: if int(p["head_min_positives"]) < 1:
return "head_min_positives must be >= 1" return "head_min_positives must be >= 1"
if not (AUTO_APPLY_THRESHOLD_MIN <= float(p["head_auto_apply_precision"]) <= AUTO_APPLY_THRESHOLD_MAX): if not (0.5 <= float(p["head_auto_apply_precision"]) <= 0.999):
return f"head_auto_apply_precision must be between {AUTO_APPLY_THRESHOLD_MIN} and {AUTO_APPLY_THRESHOLD_MAX}" return "head_auto_apply_precision must be between 0.5 and 0.999"
if int(p["head_auto_apply_min_positives"]) < 1: if int(p["head_auto_apply_min_positives"]) < 1:
return "head_auto_apply_min_positives must be >= 1" return "head_auto_apply_min_positives must be >= 1"
if not (AUTO_APPLY_THRESHOLD_MIN <= float(p["ccip_match_threshold"]) <= AUTO_APPLY_THRESHOLD_MAX): if not (0.5 <= float(p["ccip_match_threshold"]) <= 0.999):
return f"ccip_match_threshold must be between {AUTO_APPLY_THRESHOLD_MIN} and {AUTO_APPLY_THRESHOLD_MAX}" return "ccip_match_threshold must be between 0.5 and 0.999"
if not (AUTO_APPLY_THRESHOLD_MIN <= float(p["ccip_auto_apply_threshold"]) <= AUTO_APPLY_THRESHOLD_MAX): if not (0.5 <= float(p["ccip_auto_apply_threshold"]) <= 0.999):
return f"ccip_auto_apply_threshold must be between {AUTO_APPLY_THRESHOLD_MIN} and {AUTO_APPLY_THRESHOLD_MAX}" return "ccip_auto_apply_threshold must be between 0.5 and 0.999"
# Presentation chrome auto-hide (#141). Auto-apply runs high (hiding is # Presentation chrome auto-hide (#141). Auto-apply runs high (hiding is
# consequential); the conflict cut is a plain probability [0,1]. # consequential); the conflict cut is a plain probability [0,1].
if not (AUTO_APPLY_THRESHOLD_MIN <= float(p["presentation_auto_apply_threshold"]) <= AUTO_APPLY_THRESHOLD_MAX): if not (0.5 <= float(p["presentation_auto_apply_threshold"]) <= 0.999):
return f"presentation_auto_apply_threshold must be between {AUTO_APPLY_THRESHOLD_MIN} and {AUTO_APPLY_THRESHOLD_MAX}" return "presentation_auto_apply_threshold must be between 0.5 and 0.999"
if not (0.0 <= float(p["presentation_conflict_threshold"]) <= 1.0): if not (0.0 <= float(p["presentation_conflict_threshold"]) <= 1.0):
return "presentation_conflict_threshold must be between 0 and 1" return "presentation_conflict_threshold must be between 0 and 1"
# Process auto-apply (#1464). wip/editor stay VISIBLE so a false apply is
# low-harm (excludes-from-training + a review flag), but keep the same bar.
if not (AUTO_APPLY_THRESHOLD_MIN <= float(p["process_auto_apply_threshold"]) <= AUTO_APPLY_THRESHOLD_MAX):
return f"process_auto_apply_threshold must be between {AUTO_APPLY_THRESHOLD_MIN} and {AUTO_APPLY_THRESHOLD_MAX}"
if not (0.0 <= float(p["process_conflict_threshold"]) <= 1.0):
return "process_conflict_threshold must be between 0 and 1"
# Embedder model swap (#1190): both must be non-empty. Changing them means a # Embedder model swap (#1190): both must be non-empty. Changing them means a
# different embedding space — the operator must re-embed + retrain after. # different embedding space — the operator must re-embed + retrain after.
for key in ("embedder_model_name", "embedder_model_version"): for key in ("embedder_model_name", "embedder_model_version"):
-83
View File
@@ -3,28 +3,12 @@
from quart import Blueprint, jsonify, request from quart import Blueprint, jsonify, request
from ..extensions import get_session from ..extensions import get_session
from ..models import ImportSettings, Post
from ..services import interpreter_client as ic
from ..services.post_feed_service import PostFeedService from ..services.post_feed_service import PostFeedService
from ..services.source_service import KNOWN_PLATFORMS from ..services.source_service import KNOWN_PLATFORMS
from ..utils.text import html_to_plain
from ._responses import error_response as _bad from ._responses import error_response as _bad
posts_bp = Blueprint("posts", __name__, url_prefix="/api/posts") posts_bp = Blueprint("posts", __name__, url_prefix="/api/posts")
_TRANSLATION_OVERRIDES = ("auto", "force", "original")
def _queue_for_sweep(post: Post) -> None:
"""Mark a post untranslated (all translation columns NULL) so the periodic
sweep re-runs it under its new override — used when Interpreter is down and we
can't translate inline."""
post.post_title_translated = None
post.description_translated = None
post.translated_source_lang = None
post.translation_engine_version = None
post.translated_at = None
@posts_bp.route("", methods=["GET"]) @posts_bp.route("", methods=["GET"])
async def list_posts(): async def list_posts():
@@ -98,70 +82,3 @@ async def get_post(post_id: int):
if item is None: if item is None:
return _bad("not_found", status=404, detail=f"post id={post_id}") return _bad("not_found", status=404, detail=f"post id={post_id}")
return jsonify(item) return jsonify(item)
@posts_bp.route("/<int:post_id>/translation-override", methods=["POST"])
async def set_translation_override(post_id: int):
"""Sticky per-post translation override (milestone 155). Body:
``{"override": "auto" | "force" | "original"}``.
'original' keeps the original (clears any stored translation now — no
Interpreter needed). 'force'/'auto' translate the post immediately if the
service is up (force bypasses the acceptance floor; auto re-runs the gate);
if it's down we save the flag and mark the post untranslated so the next sweep
applies it. The override persists, so the sweep + Re-translate-all keep
honoring it. Returns the updated translation fields + an ``applied`` status."""
body = await request.get_json(silent=True) or {}
override = body.get("override")
if override not in _TRANSLATION_OVERRIDES:
return _bad(
"invalid_override",
detail=f"override must be one of {list(_TRANSLATION_OVERRIDES)}",
)
# Lazy import (mirrors settings.py) so the API module doesn't pull the celery
# task graph at import time.
from ..tasks.translation import _store_translation, _translate_field
async with get_session() as session:
post = await session.get(Post, post_id)
if post is None:
return _bad("not_found", status=404, detail=f"post id={post_id}")
post.translation_override = override
cfg = await ImportSettings.load(session)
target = (cfg.translation_target_lang or "en").strip() or "en"
if override == "original":
_store_translation(post, (None, None, None), (None, None, None), target)
applied = "cleared"
else:
base_url = (cfg.interpreter_base_url or "").strip()
if cfg.translation_enabled and base_url and ic.health(base_url):
force = override == "force"
title = (post.post_title or "").strip()
desc = (html_to_plain(post.description) if post.description else "") or ""
desc = desc.strip()
mc = cfg.translation_min_confidence
try:
title_res = _translate_field(title, base_url, target, mc, force=force)
desc_res = _translate_field(desc, base_url, target, mc, force=force)
except ic.InterpreterUnavailable:
_queue_for_sweep(post)
applied = "queued"
else:
_store_translation(post, title_res, desc_res, target)
applied = "translated"
else:
# Disabled / no URL / unhealthy → let the sweep apply it later.
_queue_for_sweep(post)
applied = "queued"
await session.commit()
return jsonify({
"id": post.id,
"translation_override": post.translation_override,
"post_title_translated": post.post_title_translated,
"description_translated": post.description_translated,
"translated_source_lang": post.translated_source_lang,
"applied": applied,
})
+25 -38
View File
@@ -47,9 +47,6 @@ _EDITABLE_FIELDS = (
"translation_enabled", "translation_enabled",
"interpreter_base_url", "interpreter_base_url",
"translation_target_lang", "translation_target_lang",
"translation_min_confidence",
"wip_title_tagging_enabled",
"wip_soft_title_tagging_enabled",
) )
# Per-host external-download toggles — all plain booleans, validated uniformly. # Per-host external-download toggles — all plain booleans, validated uniformly.
@@ -66,9 +63,31 @@ _EXTDL_TOGGLE_FIELDS = (
async def get_import_settings(): async def get_import_settings():
async with get_session() as session: async with get_session() as session:
row = await ImportSettings.load(session) row = await ImportSettings.load(session)
# Table-driven off _EDITABLE_FIELDS (which PATCH also writes) so a new field return jsonify({
# can't be silently absent from GET. "min_width": row.min_width,
return jsonify({f: getattr(row, f) for f in _EDITABLE_FIELDS}) "min_height": row.min_height,
"skip_transparent": row.skip_transparent,
"transparency_threshold": row.transparency_threshold,
"skip_single_color": row.skip_single_color,
"single_color_threshold": row.single_color_threshold,
"single_color_tolerance": row.single_color_tolerance,
"phash_threshold": row.phash_threshold,
"download_rate_limit_seconds": row.download_rate_limit_seconds,
"download_validate_files": row.download_validate_files,
"download_schedule_default_seconds": row.download_schedule_default_seconds,
"download_event_retention_days": row.download_event_retention_days,
"download_failure_warning_threshold": row.download_failure_warning_threshold,
"series_suggest_enabled": row.series_suggest_enabled,
"series_suggest_threshold": row.series_suggest_threshold,
"extdl_mega_enabled": row.extdl_mega_enabled,
"extdl_gdrive_enabled": row.extdl_gdrive_enabled,
"extdl_mediafire_enabled": row.extdl_mediafire_enabled,
"extdl_dropbox_enabled": row.extdl_dropbox_enabled,
"extdl_pixeldrain_enabled": row.extdl_pixeldrain_enabled,
"translation_enabled": row.translation_enabled,
"interpreter_base_url": row.interpreter_base_url,
"translation_target_lang": row.translation_target_lang,
})
@settings_bp.route("/settings/import", methods=["PATCH"]) @settings_bp.route("/settings/import", methods=["PATCH"])
@@ -136,32 +155,12 @@ async def update_import_settings():
for key in ("interpreter_base_url", "translation_target_lang"): for key in ("interpreter_base_url", "translation_target_lang"):
if key in body and not isinstance(body[key], str): if key in body and not isinstance(body[key], str):
return jsonify({"error": f"{key} must be a string"}), 400 return jsonify({"error": f"{key} must be a string"}), 400
# Acceptance floor (milestone 155): latin-script translations below this
# Interpreter confidence are kept as the original.
if "translation_min_confidence" in body:
v = body["translation_min_confidence"]
if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1:
return jsonify(
{"error": "translation_min_confidence must be a number in [0, 1]"}
), 400
if "series_suggest_threshold" in body: if "series_suggest_threshold" in body:
v = body["series_suggest_threshold"] v = body["series_suggest_threshold"]
if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1: if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1:
return jsonify( return jsonify(
{"error": "series_suggest_threshold must be a number in [0, 1]"} {"error": "series_suggest_threshold must be a number in [0, 1]"}
), 400 ), 400
if "wip_title_tagging_enabled" in body and not isinstance(
body["wip_title_tagging_enabled"], bool
):
return jsonify(
{"error": "wip_title_tagging_enabled must be a boolean"}
), 400
if "wip_soft_title_tagging_enabled" in body and not isinstance(
body["wip_soft_title_tagging_enabled"], bool
):
return jsonify(
{"error": "wip_soft_title_tagging_enabled must be a boolean"}
), 400
async with get_session() as session: async with get_session() as session:
row = await ImportSettings.load(session) row = await ImportSettings.load(session)
@@ -173,18 +172,6 @@ async def update_import_settings():
return await get_import_settings() return await get_import_settings()
@settings_bp.route("/settings/wip-title/scan", methods=["POST"])
async def wip_title_scan():
"""Enqueue the back-catalogue WIP-title scan (task #1458 Settings button):
apply the `wip` system tag to EXISTING posts whose title declares
work-in-progress. New imports are tagged live by the importer; this catches
the existing library. Returns the Celery task id (202)."""
from ..tasks.maintenance import backfill_wip_title_tags
r = backfill_wip_title_tags.delay()
return jsonify({"celery_task_id": r.id}), 202
@settings_bp.route("/system/stats", methods=["GET"]) @settings_bp.route("/system/stats", methods=["GET"])
async def system_stats(): async def system_stats():
async with get_session() as session: async with get_session() as session:
+30 -10
View File
@@ -11,11 +11,10 @@ suggestions_bp = Blueprint("suggestions", __name__, url_prefix="/api")
@suggestions_bp.route("/images/<int:image_id>/suggestions", methods=["GET"]) @suggestions_bp.route("/images/<int:image_id>/suggestions", methods=["GET"])
async def get_suggestions(image_id: int): async def get_suggestions(image_id: int):
# ?min=<float> overrides the per-head suggest thresholds for INCLUSION. The # ?min=<float> overrides the configured per-category thresholds so the typed
# rail sends min=0 in its single per-image fetch to get EVERY head (each row # tag-input dropdown can surface EVERY stored prediction (min=0), including
# still carries above_threshold vs its natural cut), then derives the panel # low-confidence actions/features, in canonical formatting. Omitted → the
# (above_threshold) and the typed dropdown (all, filtered by text) client-side # curated above-threshold list the Suggestions panel uses.
# — no second request. Omitted → only above-threshold rows.
override = None override = None
raw_min = request.args.get("min") raw_min = request.args.get("min")
if raw_min is not None: if raw_min is not None:
@@ -37,11 +36,12 @@ async def get_suggestions(image_id: int):
"category": s.category, "category": s.category,
"score": round(s.score, 4), "score": round(s.score, 4),
"source": s.source, "source": s.source,
# whether the score cleared the head's own suggest cut. "creates_new_tag": s.creates_new_tag,
# The single min=0 fetch returns every head; the panel # raw model key (alias is stored under this) + whether an
# shows above_threshold, the typed dropdown shows all and # operator alias produced this suggestion — drive the
# annotates each match with its score. # modal's "Treat as alias"/"Remove alias" affordances.
"above_threshold": s.above_threshold, "raw_name": s.raw_name,
"via_alias": s.via_alias,
# operator dismissed this tag for this image — surfaced # operator dismissed this tag for this image — surfaced
# (not dropped) so the rail can show it rejected + offer # (not dropped) so the rail can show it rejected + offer
# one-click un-reject. # one-click un-reject.
@@ -73,6 +73,26 @@ async def accept_suggestion(image_id: int):
return jsonify({"accepted": True, "tag_id": tag_id}) return jsonify({"accepted": True, "tag_id": tag_id})
@suggestions_bp.route(
"/images/<int:image_id>/suggestions/alias", methods=["POST"]
)
async def alias_suggestion(image_id: int):
body = await request.get_json()
required = {"alias_string", "alias_category", "canonical_tag_id"}
if not body or not required.issubset(body):
return jsonify({"error": f"required: {sorted(required)}"}), 400
canonical_tag_id = body["canonical_tag_id"]
async with get_session() as session:
await AllowlistService(session).add_alias_and_accept(
image_id,
body["alias_string"],
body["alias_category"],
canonical_tag_id,
)
await session.commit()
return jsonify({"accepted": True, "tag_id": canonical_tag_id})
@suggestions_bp.route( @suggestions_bp.route(
"/images/<int:image_id>/suggestions/dismiss", methods=["POST"] "/images/<int:image_id>/suggestions/dismiss", methods=["POST"]
) )
+1 -11
View File
@@ -171,19 +171,9 @@ def make_celery() -> Celery:
}, },
"presentation-auto-apply-daily": { "presentation-auto-apply-daily": {
"task": "backend.app.tasks.ml.scheduled_presentation_auto_apply", "task": "backend.app.tasks.ml.scheduled_presentation_auto_apply",
"schedule": 86400.0, # auto-hide banner chrome (#141); "schedule": 86400.0, # auto-hide banner/editor chrome (#141);
# no-op unless presentation_auto_apply_enabled # no-op unless presentation_auto_apply_enabled
}, },
"process-auto-apply-daily": {
"task": "backend.app.tasks.ml.scheduled_process_auto_apply",
"schedule": 86400.0, # auto-tag wip/editor process art (#1464);
# no-op unless process_auto_apply_enabled (opt-in)
},
"soft-wip-conflict-audit-daily": {
"task": "backend.app.tasks.ml.scheduled_soft_wip_conflict_audit",
"schedule": 86400.0, # flag ring-loud soft-WIP (sketch/doodle) tags
# for review (#1474); no-op with no content heads
},
"prune-presentation-reviews-daily": { "prune-presentation-reviews-daily": {
"task": "backend.app.tasks.ml.prune_presentation_reviews", "task": "backend.app.tasks.ml.prune_presentation_reviews",
"schedule": 86400.0, # retention: drop resolved review flags >30d "schedule": 86400.0, # retention: drop resolved review flags >30d
+1 -2
View File
@@ -27,7 +27,7 @@ from .patreon_seen_media import PatreonSeenMedia
from .pixiv_failed_media import PixivFailedMedia from .pixiv_failed_media import PixivFailedMedia
from .pixiv_seen_media import PixivSeenMedia from .pixiv_seen_media import PixivSeenMedia
from .post import Post from .post import Post
from .post_attachment import PostAttachment, attachment_download_url from .post_attachment import PostAttachment
from .presentation_review import PresentationReview from .presentation_review import PresentationReview
from .series_chapter import SeriesChapter from .series_chapter import SeriesChapter
from .series_page import SeriesPage from .series_page import SeriesPage
@@ -58,7 +58,6 @@ __all__ = [
"SubscribeStarSeenMedia", "SubscribeStarSeenMedia",
"Post", "Post",
"PostAttachment", "PostAttachment",
"attachment_download_url",
"PresentationReview", "PresentationReview",
"SeriesChapter", "SeriesChapter",
"SeriesPage", "SeriesPage",
-27
View File
@@ -106,33 +106,6 @@ class ImportSettings(Base):
translation_target_lang: Mapped[str] = mapped_column( translation_target_lang: Mapped[str] = mapped_column(
Text, nullable=False, default="en", server_default="en", Text, nullable=False, default="en", server_default="en",
) )
# The latin-script acceptance floor for the translation gate: a translation
# whose Interpreter-reported confidence is below this is kept as the original
# (operator-tunable, milestone 155). Default 0.9 — stricter than the old
# hardcoded 0.8, because Interpreter confidently mis-detects short ASCII
# English (e.g. "… WIP Part 1") as a European language at ~0.86. CJK stays
# trusted regardless (script-detected). Per-post overrides handle the misses.
translation_min_confidence: Mapped[float] = mapped_column(
Float, nullable=False, default=0.9, server_default="0.9",
)
# Title-based WIP auto-tagging (task #1458). When a freshly-imported post's
# TITLE explicitly declares work-in-progress ("WIP" / "work in progress"),
# the importer applies the `wip` system tag to its images — the artist's own
# label, used to keep unfinished pieces out of the Explore/gallery browse. ON
# by default (rule 26 — the feature works out of the box). Gates only the
# LIVE import hook; the existing catalogue is caught by the operator-triggered
# "Scan existing posts" backfill (which runs regardless of this flag).
wip_title_tagging_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default="true",
)
# Soft WIP title tier (#1474): also tag sketch/doodle/scribble titles, but with
# a PROVISIONAL source (`wip_title_soft`) that never trains the head, since these
# are lower-precision (a finished "sketch" isn't WIP). OFF by default — a lower-
# precision tier is opt-in (the ring-loud audit surfaces false positives).
wip_soft_title_tagging_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, server_default="false",
)
@classmethod @classmethod
async def load(cls, session) -> ImportSettings: async def load(cls, session) -> ImportSettings:
+6 -40
View File
@@ -10,7 +10,6 @@ from sqlalchemy import (
Integer, Integer,
String, String,
func, func,
select,
) )
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
@@ -86,14 +85,12 @@ class MLSettings(Base):
Float, nullable=False, default=0.95 Float, nullable=False, default=0.95
) )
# -- Presentation chrome auto-hide (#141) ------------------------------- # -- Presentation chrome auto-hide (#141) -------------------------------
# `banner` (chrome — clusters on UI, not content) auto-applies on the sweep # banner / editor screenshot auto-apply on the sweep with their OWN flat
# with its OWN flat threshold (decoupled from content-head graduation) and is # threshold (decoupled from content-head graduation). Hiding is consequential
# HIDDEN from the gallery. Hiding is consequential so it runs HIGH. When an # so it runs HIGH. `wip` is never auto-applied. When an image would be
# image would be auto-hidden but ALSO scores >= presentation_conflict_threshold # auto-hidden but ALSO scores >= presentation_conflict_threshold on a content
# on a content head, it's still hidden but flagged for review # head, it's still hidden but flagged for review (PresentationReview) instead
# (PresentationReview, mode='chrome') instead of buried silently. ON by default # of buried silently. ON by default (opt-out); every auto-tag is reversible.
# (opt-out); every auto-tag is reversible. NOTE (#1464): `wip` + `editor
# screenshot` are no longer chrome — they went to the PROCESS path below.
presentation_auto_apply_enabled: Mapped[bool] = mapped_column( presentation_auto_apply_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True Boolean, nullable=False, default=True
) )
@@ -103,26 +100,6 @@ class MLSettings(Base):
presentation_conflict_threshold: Mapped[float] = mapped_column( presentation_conflict_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.50 Float, nullable=False, default=0.50
) )
# -- Process auto-apply (#1464) ----------------------------------------
# `wip` / `editor screenshot` are PROCESS art — unfinished pieces + program
# screenshots that must stay OUT of head/CCIP training but, unlike chrome,
# remain VISIBLE in the gallery (operator 2026-07-12). They auto-apply on the
# sweep with their OWN flat threshold and a PROVISIONAL source (`process_auto`,
# in training_data._AUTO_SOURCES) so the head NEVER trains on its own output —
# it learns only from title (`wip_title`) + manual labels, which breaks the
# runaway loop. When a process tag would be applied but the image ALSO scores
# >= process_conflict_threshold on a content head, it's flagged for review
# (PresentationReview, mode='process') rather than silently marked. OFF by
# default — a new whole-library auto-tagger is opt-in; every auto-tag reversible.
process_auto_apply_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False
)
process_auto_apply_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.90
)
process_conflict_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.50
)
# Default = SigLIP 2 (so400m, 512px) for new installs (migration 0069); # Default = SigLIP 2 (so400m, 512px) for new installs (migration 0069);
# existing libraries keep their stored value until the operator re-embeds. # existing libraries keep their stored value until the operator re-embeds.
embedder_model_version: Mapped[str] = mapped_column( embedder_model_version: Mapped[str] = mapped_column(
@@ -213,14 +190,3 @@ class MLSettings(Base):
updated_at: Mapped[datetime] = mapped_column( updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now() DateTime(timezone=True), nullable=False, server_default=func.now()
) )
@classmethod
async def load(cls, session) -> MLSettings:
"""The singleton settings row (id=1), via an async session. Mirrors
ImportSettings.load — the shared singleton-loader pattern."""
return (await session.execute(select(cls).where(cls.id == 1))).scalar_one()
@classmethod
def load_sync(cls, session) -> MLSettings:
"""The singleton settings row (id=1), via a sync session."""
return session.execute(select(cls).where(cls.id == 1)).scalar_one()
+1 -25
View File
@@ -8,17 +8,7 @@ artist-filter queries don't depend on the Source detour).
from datetime import datetime from datetime import datetime
from sqlalchemy import ( from sqlalchemy import JSON, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
JSON,
CheckConstraint,
DateTime,
ForeignKey,
Integer,
String,
Text,
UniqueConstraint,
func,
)
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from .base import Base from .base import Base
@@ -33,10 +23,6 @@ class Post(Base):
# (created in alembic 0030) covers that case via # (created in alembic 0030) covers that case via
# (artist_id, external_post_id). # (artist_id, external_post_id).
UniqueConstraint("source_id", "external_post_id", name="uq_post_source_external_id"), UniqueConstraint("source_id", "external_post_id", name="uq_post_source_external_id"),
CheckConstraint(
"translation_override IN ('auto', 'force', 'original')",
name="ck_post_translation_override",
),
) )
id: Mapped[int] = mapped_column(Integer, primary_key=True) id: Mapped[int] = mapped_column(Integer, primary_key=True)
@@ -78,16 +64,6 @@ class Post(Base):
translated_at: Mapped[datetime | None] = mapped_column( translated_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True DateTime(timezone=True), nullable=True
) )
# Sticky per-post override of the translation decision (milestone 155):
# 'auto' = the acceptance gate decides; 'force' = always store Interpreter's
# translation even below the confidence floor (rescue a skipped legit-foreign
# title); 'original' = never translate, keep the original (kill a confidently
# mis-flagged one the floor can't catch). The sweep reads this on every run,
# and re-translate leaves 'original' posts alone, so the choice survives a
# Re-translate-all.
translation_override: Mapped[str] = mapped_column(
String(16), nullable=False, default="auto", server_default="auto",
)
downloaded_at: Mapped[datetime] = mapped_column( downloaded_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now() DateTime(timezone=True), nullable=False, server_default=func.now()
-12
View File
@@ -65,15 +65,3 @@ class PostAttachment(Base):
captured_at: Mapped[datetime] = mapped_column( captured_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now() DateTime(timezone=True), nullable=False, server_default=func.now()
) )
def attachment_download_url(attachment_id: int) -> str:
"""The path that streams this attachment's bytes.
Both serializers that expose an attachment to the frontend
(`provenance_service`, `post_feed_service`) built this literal themselves,
so changing the route in `api/attachments.py` meant two edits and only one
would be remembered (#3072). `test_attachment_download_url` pins it against
the app's registered rule, so the drift is caught rather than trusted to.
"""
return f"/api/attachments/{attachment_id}/download"
+7 -15
View File
@@ -1,17 +1,15 @@
"""PresentationReview — a system-tag the auto-apply sweep applied that ALSO looked """PresentationReview — an auto-hidden presentation tag that ALSO looked like
like real content, flagged for operator review (milestone 141 + #1464). real content, flagged for operator review (milestone 141).
When a sweep applies a system tag but the image ALSO scores highly on a content When the auto-apply sweep hides an image as chrome (banner / editor screenshot)
head, it still applies the tag but records this row so a review strip can surface but the image ALSO scores highly on a content head, it still hides it but records
it ("⚠ also looks like <conflict tag>"). Two modes (#1464): 'chrome' (banner — this row so the Hidden view can surface it ("⚠ also looks like <conflict tag>")
image is HIDDEN, review is keep-hidden / un-hide) and 'process' (wip / editor for a keep-hidden / un-hide decision. Resolved rows are pruned by retention.
screenshot — image stays VISIBLE, review is confirm / remove-tag). Resolved rows
are pruned by retention.
""" """
from datetime import datetime from datetime import datetime
from sqlalchemy import DateTime, Float, ForeignKey, String, func from sqlalchemy import DateTime, Float, ForeignKey, func
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from .base import Base from .base import Base
@@ -33,12 +31,6 @@ class PresentationReview(Base):
ForeignKey("tag.id", ondelete="SET NULL"), nullable=True ForeignKey("tag.id", ondelete="SET NULL"), nullable=True
) )
conflict_score: Mapped[float] = mapped_column(Float, nullable=False) conflict_score: Mapped[float] = mapped_column(Float, nullable=False)
# Which sweep flagged this (#1464): 'chrome' (banner, hidden) or 'process'
# (wip / editor screenshot, shown). Drives which review strip surfaces it and
# what "resolve" means (un-hide vs remove-tag). Existing rows backfill 'chrome'.
mode: Mapped[str] = mapped_column(
String(16), nullable=False, default="chrome", server_default="chrome"
)
created_at: Mapped[datetime] = mapped_column( created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now() DateTime(timezone=True), nullable=False, server_default=func.now()
) )
+7 -12
View File
@@ -43,19 +43,14 @@ class TagKind(StrEnum):
# to keep historic tag rows queryable. # to keep historic tag rows queryable.
# The seeded system tags (migration 0075). Two behavior groups (#1464): # The seeded system tags (migration 0075). PRESENTATION tags additionally
# CHROME (banner): clusters on UI chrome, not content → HIDDEN from the default # hide from whole-image similarity results — they cluster on UI chrome, not
# gallery + from similarity; auto-applied via the sweep's chrome mode. # content. `wip` is real art: only the training pipelines exclude it.
# PROCESS (wip, editor screenshot): real-but-unfinished art / program screenshots
# → SHOWN in the gallery (operator 2026-07-12), but excluded from the Explore
# rabbit-hole; auto-applied via the sweep's process mode (provisional source,
# ring-loud review guard).
# ALL three are excluded from OTHER concepts' head/CCIP training (training-hygiene,
# keyed on is_system); a system tag's OWN head trains on them — that's what makes
# auto-flagging work.
SYSTEM_TAG_NAMES = ("wip", "banner", "editor screenshot") SYSTEM_TAG_NAMES = ("wip", "banner", "editor screenshot")
CHROME_SYSTEM_TAGS = ("banner",) PRESENTATION_SYSTEM_TAGS = ("banner", "editor screenshot")
PROCESS_SYSTEM_TAGS = ("wip", "editor screenshot") # `wip` marks real-but-unfinished art. It's kept in the gallery's own "similar"
# results (#1274), but the Explore rabbit-hole opts to hide it (exclude_wip) so a
# browse doesn't keep surfacing work-in-progress (operator, 2026-07-08).
WIP_SYSTEM_TAG = "wip" WIP_SYSTEM_TAG = "wip"
image_tag = Table( image_tag = Table(
+6 -256
View File
@@ -48,47 +48,6 @@ log = logging.getLogger(__name__)
_VIDEO_DURATION_UNKNOWN = -1.0 _VIDEO_DURATION_UNKNOWN = -1.0
# -- artist-cascade predicates (rule 93: ONE definition, preview + apply) ---
# project_artist_cascade (preview) and delete_artist_cascade (apply) both build
# their queries from these. The preview used to re-derive its own — which is how
# it came to count images and stay silent about posts and attachments while the
# apply destroyed both. Same failure shape as the 2026-06-08 fandom-tag
# deletion, where a re-implemented delete predicate diverged from the preview's.
# Returned as condition LISTS spread into `.where(*conds)`, matching
# _unused_tag_conditions / _bare_post_conditions below.
def _artist_images_conditions(artist_id: int) -> list:
"""Images the cascade deletes (rows AND their on-disk files)."""
return [ImageRecord.artist_id == artist_id]
def _artist_posts_conditions(artist_id: int) -> list:
"""Posts the cascade destroys. The apply never names these — post.artist_id
is ondelete=CASCADE, so Postgres takes them when the artist row goes — which
is exactly why the preview has to name them: an artist whose posts are
body-only (no images) otherwise previews as `images: 0` and reads as an
empty artist, while every captured body/description/external-link set is
destroyed."""
return [Post.artist_id == artist_id]
def _artist_attachments_conditions(artist_id: int) -> list:
"""Attachments the cascade deletes. Matched by artist_id OR by the owning
post's artist: artist_id is nullable (_capture_attachment leaves it NULL
when no artist resolved), so neither arm alone covers every row. The
sha-addressed blobs are NOT unlinked (one blob backs many rows) — these are
row counts, and the bytes are not part of this operation's footprint."""
return [
or_(
PostAttachment.artist_id == artist_id,
PostAttachment.post_id.in_(
select(Post.id).where(*_artist_posts_conditions(artist_id))
),
)
]
def project_artist_cascade(session: Session, *, slug: str) -> dict: def project_artist_cascade(session: Session, *, slug: str) -> dict:
"""Read-only projection of what delete_artist_cascade would touch. """Read-only projection of what delete_artist_cascade would touch.
@@ -97,17 +56,12 @@ def project_artist_cascade(session: Session, *, slug: str) -> dict:
"artist": {"id": int, "name": str, "slug": str}, "artist": {"id": int, "name": str, "slug": str},
"projected": { "projected": {
"images": int, "images": int,
"posts": int, # hard-deleted by the post.artist_id CASCADE
"attachments": int, # rows deleted; the sha-addressed blobs stay
"sources": int, "sources": int,
"thumbs": int, # images with a thumbnail_path set "thumbs": int, # images with a thumbnail_path set
"import_tasks": int, # ImportTask rows referencing the artist's images "import_tasks": int, # ImportTask rows referencing the artist's images
"bytes_on_disk": int, # SUM(image_record.size_bytes) — column is NOT NULL "bytes_on_disk": int, # SUM(image_record.size_bytes) — column is NOT NULL
}, },
} }
Every count is built from the shared `_artist_*_conditions` predicates the
apply uses, so the two halves cannot drift (rule 93).
Raises LookupError if slug not found. No mutations. Raises LookupError if slug not found. No mutations.
""" """
from ..models.import_task import ImportTask from ..models.import_task import ImportTask
@@ -119,49 +73,36 @@ def project_artist_cascade(session: Session, *, slug: str) -> dict:
if artist is None: if artist is None:
raise LookupError(f"artist slug not found: {slug!r}") raise LookupError(f"artist slug not found: {slug!r}")
images_conds = _artist_images_conditions(artist.id)
images_count = session.execute( images_count = session.execute(
select(func.count(ImageRecord.id)).where(*images_conds) select(func.count(ImageRecord.id))
.where(ImageRecord.artist_id == artist.id)
).scalar_one() ).scalar_one()
posts_count = session.execute(
select(func.count(Post.id))
.where(*_artist_posts_conditions(artist.id))
).scalar_one()
attachments_count = session.execute(
select(func.count(PostAttachment.id))
.where(*_artist_attachments_conditions(artist.id))
).scalar_one()
# Sources have no shared predicate: the apply never queries them either, it
# gets them from the Artist.sources ORM cascade. Counted directly here.
sources_count = session.execute( sources_count = session.execute(
select(func.count(Source.id)) select(func.count(Source.id))
.where(Source.artist_id == artist.id) .where(Source.artist_id == artist.id)
).scalar_one() ).scalar_one()
thumbs_count = session.execute( thumbs_count = session.execute(
select(func.count(ImageRecord.id)) select(func.count(ImageRecord.id))
.where(*images_conds) .where(ImageRecord.artist_id == artist.id)
.where(ImageRecord.thumbnail_path.is_not(None)) .where(ImageRecord.thumbnail_path.is_not(None))
).scalar_one() ).scalar_one()
import_tasks_count = session.execute( import_tasks_count = session.execute(
select(func.count(ImportTask.id)) select(func.count(ImportTask.id))
.where( .where(
ImportTask.result_image_id.in_( ImportTask.result_image_id.in_(
select(ImageRecord.id).where(*images_conds) select(ImageRecord.id).where(ImageRecord.artist_id == artist.id)
) )
) )
).scalar_one() ).scalar_one()
bytes_on_disk = session.execute( bytes_on_disk = session.execute(
select(func.coalesce(func.sum(ImageRecord.size_bytes), 0)) select(func.coalesce(func.sum(ImageRecord.size_bytes), 0))
.where(*images_conds) .where(ImageRecord.artist_id == artist.id)
).scalar_one() ).scalar_one()
return { return {
"artist": {"id": artist.id, "name": artist.name, "slug": artist.slug}, "artist": {"id": artist.id, "name": artist.name, "slug": artist.slug},
"projected": { "projected": {
"images": images_count, "images": images_count,
"posts": posts_count,
"attachments": attachments_count,
"sources": sources_count, "sources": sources_count,
"thumbs": thumbs_count, "thumbs": thumbs_count,
"import_tasks": import_tasks_count, "import_tasks": import_tasks_count,
@@ -336,10 +277,6 @@ def delete_artist_cascade(
series_page / tag_suggestion_rejection from ImageRecord delete, series_page / tag_suggestion_rejection from ImageRecord delete,
and source / post / download_event / etc. from Artist delete and source / post / download_event / etc. from Artist delete
(via Artist.sources cascade="all, delete-orphan"). (via Artist.sources cascade="all, delete-orphan").
The artist's post_attachment rows are cleared EXPLICITLY before the
artist row goes — see the comment at that step; leaving them to the
cascade aborts the whole delete on a unique violation.
""" """
artist = session.get(Artist, artist_id) artist = session.get(Artist, artist_id)
if artist is None: if artist is None:
@@ -350,22 +287,11 @@ def delete_artist_cascade(
"files_deleted": 0, "files_deleted": 0,
"thumbs_deleted": 0, "thumbs_deleted": 0,
"import_tasks_nulled": 0, "import_tasks_nulled": 0,
"posts_deleted": 0,
"attachments_deleted": 0,
"files_failed": 0, "files_failed": 0,
}, },
} }
artist_info = {"id": artist.id, "name": artist.name, "slug": artist.slug} artist_info = {"id": artist.id, "name": artist.name, "slug": artist.slug}
# Counted BEFORE the delete: Postgres takes these via the post.artist_id
# CASCADE when the artist row goes, so afterwards there is nothing left to
# count. Reported so the summary can be checked against the preview's
# `posts` — the parity rule 93 asks for is only testable if both halves
# actually state the number.
posts_deleted = session.execute(
select(func.count(Post.id)).where(*_artist_posts_conditions(artist.id))
).scalar_one()
images_deleted = 0 images_deleted = 0
files_deleted = 0 files_deleted = 0
thumbs_deleted = 0 thumbs_deleted = 0
@@ -374,7 +300,7 @@ def delete_artist_cascade(
while True: while True:
rows = session.execute( rows = session.execute(
select(ImageRecord) select(ImageRecord)
.where(*_artist_images_conditions(artist.id)) .where(ImageRecord.artist_id == artist.id)
.limit(500) .limit(500)
).scalars().all() ).scalars().all()
if not rows: if not rows:
@@ -397,28 +323,6 @@ def delete_artist_cascade(
# source_path_prefix matching that's out of scope here. # source_path_prefix matching that's out of scope here.
import_tasks_nulled = 0 import_tasks_nulled = 0
# Clear the artist's attachments BEFORE the artist row, or the delete below
# aborts. Deleting an artist CASCADEs to Post (post.artist_id is
# ondelete=CASCADE), which SET NULLs post_attachment.post_id — and
# `uq_post_attachment_null_post_sha` is a partial UNIQUE on sha256 ALONE
# WHERE post_id IS NULL, so any two of this artist's attachments sharing a
# sha collapse onto one another and raise. That is an ORDINARY shape, not a
# corrupt one: _capture_attachment deliberately writes one row per post over
# a single sha-addressed blob (a creator who attaches the same pdf to two
# posts has two rows), and a pre-existing filesystem-import row with the same
# sha and a NULL post_id collides on its own. Migration 0043 reasoned only
# about upgrade-time safety and never about this later SET NULL.
# _repoint_post_links guards the identical collision class in the reconcile
# path; this is its artist-cascade counterpart.
#
# Which rows count as the artist's — and why the blobs are left on disk —
# is _artist_attachments_conditions, shared with the preview.
attachments_deleted = session.execute(
delete(PostAttachment)
.where(*_artist_attachments_conditions(artist.id))
).rowcount or 0
session.commit()
session.delete(artist) session.delete(artist)
session.commit() session.commit()
@@ -429,8 +333,6 @@ def delete_artist_cascade(
"files_deleted": files_deleted, "files_deleted": files_deleted,
"thumbs_deleted": thumbs_deleted, "thumbs_deleted": thumbs_deleted,
"import_tasks_nulled": import_tasks_nulled, "import_tasks_nulled": import_tasks_nulled,
"posts_deleted": posts_deleted,
"attachments_deleted": attachments_deleted,
"files_failed": files_failed, "files_failed": files_failed,
}, },
} }
@@ -1592,155 +1494,3 @@ def purge_gated_previews(
"ledger_cleared": ledger_cleared, "ledger_cleared": ledger_cleared,
"posts_deleted": posts_deleted, "posts_deleted": posts_deleted,
} }
# -- orphaned attachment reclamation ---------------------------------------
# PostAttachment's two FKs are both ON DELETE SET NULL, so a deleted post or
# artist leaves the row behind rather than taking it. Nothing ever pruned those
# rows, and nothing has ever unlinked a file under the attachment store — so
# both rows and bytes accumulated permanently and were invisible to every
# existing diagnostic.
#
# Why this is a DISK->DB reconciliation rather than a row sweep: the store is
# sha-addressed and idempotent (attachment_store.store), so ONE blob backs MANY
# rows. Deleting a row therefore does not free its blob, and — since the artist
# cascade now deletes its attachment rows outright — a freed blob has no DB
# pointer left to find it by. Walking the store and asking "does any row still
# reference this sha?" catches orphans from every cause, including ones no
# future delete path will think to report.
# A blob is written by attachment_store.store BEFORE its row is inserted and
# committed, so a just-stored file legitimately has no referencing row for a
# moment. Same guard, same reasoning as ORPHAN_TEMP_MIN_AGE_HOURS in
# tasks/maintenance.py: never judge a file younger than this.
_ATTACHMENT_ORPHAN_MIN_AGE_HOURS = 6
# Wall-clock budget for the store walk (rule 89). A library with a large
# attachment store shouldn't be able to run this past its soft time limit; on
# exhaustion it reports partial=True and the operator re-runs to finish.
_ATTACHMENT_RECLAIM_BUDGET_SECONDS = 900
# The store names files `<sha256><ext>`. Parse the sha as the first 64 chars
# rather than via Path.stem: store() takes the extension straight from the
# source filename, and a URL-encoded basename yields a multi-dot "suffix"
# (see [[reference_url_encoded_basename_suffix]]) that would make stem eat part
# of the sha. Validating the 64 chars as hex also skips anything else in the
# tree that isn't a stored blob.
_SHA256_HEX_LEN = 64
def _orphan_attachment_conditions() -> list:
"""PostAttachment rows belonging to nothing: both FKs nulled by a deleted
post AND a deleted artist. A row with post_id NULL but an artist_id is the
deliberate filesystem-import case (importer._capture_attachment writes it
that way) and is NOT an orphan — it is still attributed."""
return [
PostAttachment.post_id.is_(None),
PostAttachment.artist_id.is_(None),
]
def _is_sha_named(name: str) -> bool:
"""True when `name` starts with a 64-char lowercase-hex sha256."""
if len(name) < _SHA256_HEX_LEN:
return False
head = name[:_SHA256_HEX_LEN]
return all(c in "0123456789abcdef" for c in head)
def reclaim_orphaned_attachments(
session: Session, *, images_root: Path, dry_run: bool = False,
) -> dict:
"""Prune unattributed PostAttachment rows, then unlink store blobs that no
surviving row references.
Returns (same discovery keys either way, so the UI renders one shape):
{"rows": int, # orphan rows found / deleted
"files": int, # unreferenced blobs found / unlinked
"bytes": int, # their total size
"scanned": int, # blobs examined
"skipped_recent": int, # blobs under the min-age guard
"files_failed": int, # unlink raised (apply only)
"partial": bool} # walk hit the time budget
dry_run computes exactly what the apply would do and mutates nothing — the
surviving-sha set is derived by NEGATING the same orphan predicate the
delete uses, so the preview cannot disagree with the apply (rule 93).
"""
started = time.monotonic()
orphan_conds = _orphan_attachment_conditions()
if dry_run:
rows = session.execute(
select(func.count(PostAttachment.id)).where(*orphan_conds)
).scalar_one()
else:
rows = session.execute(
delete(PostAttachment).where(*orphan_conds)
).rowcount or 0
session.commit()
# Shas that still have a home. In the apply path the orphan rows are already
# gone, so `NOT orphan` is redundant but harmless; in the dry-run path it is
# what makes the projection honest about blobs the delete would free. One
# predicate, one query, both modes.
surviving_shas = set(session.execute(
select(PostAttachment.sha256).where(~and_(*orphan_conds)).distinct()
).scalars())
root = Path(images_root) / "attachments"
cutoff = (
datetime.now(UTC).timestamp()
- _ATTACHMENT_ORPHAN_MIN_AGE_HOURS * 3600
)
files = 0
freed_bytes = 0
scanned = 0
skipped_recent = 0
files_failed = 0
partial = False
if root.is_dir():
for path in root.rglob("*"):
if time.monotonic() - started >= _ATTACHMENT_RECLAIM_BUDGET_SECONDS:
partial = True
break
# .partial staging files belong to cleanup_orphaned_temp_files —
# leave them alone rather than racing an in-flight store().
if path.suffix in (".part", ".partial") or not path.is_file():
continue
if not _is_sha_named(path.name):
continue
scanned += 1
sha = path.name[:_SHA256_HEX_LEN]
if sha in surviving_shas:
continue
try:
st = path.stat()
if st.st_mtime >= cutoff:
skipped_recent += 1
continue
size = st.st_size
if not dry_run:
path.unlink()
files += 1
freed_bytes += size
except OSError as exc:
files_failed += 1
log.warning("reclaim_orphaned_attachments: %s: %s", path, exc)
if not dry_run and (rows or files):
log.info(
"attachment reclaim: %d orphan row(s) deleted, %d blob(s) unlinked "
"(%d bytes), %d failed, partial=%s",
rows, files, freed_bytes, files_failed, partial,
)
return {
"rows": rows,
"files": files,
"bytes": freed_bytes,
"scanned": scanned,
"skipped_recent": skipped_recent,
"files_failed": files_failed,
"partial": partial,
}
+1 -1
View File
@@ -181,7 +181,7 @@ def _augment_cookies(platform: str, netscape: str) -> str:
"""Delegate to the platform's `augment_cookies` hook if one is """Delegate to the platform's `augment_cookies` hook if one is
registered (subscribestar, hentaifoundry, etc. — see registered (subscribestar, hentaifoundry, etc. — see
`services/platforms/<name>.py`). No-op when the platform doesn't `services/platforms/<name>.py`). No-op when the platform doesn't
register a hook (Patreon, Discord). Centralizing the register a hook (Patreon, DeviantArt). Centralizing the
quirks-per-platform in the platforms package means adding a new quirks-per-platform in the platforms package means adding a new
platform's cookie quirks doesn't require touching this file.""" platform's cookie quirks doesn't require touching this file."""
info = PLATFORMS.get(platform) info = PLATFORMS.get(platform)
+3 -2
View File
@@ -31,8 +31,9 @@ from .pixiv_ingester import PixivIngester
from .subscribestar_ingester import SubscribeStarIngester from .subscribestar_ingester import SubscribeStarIngester
# Platforms whose download + verify go through the native ingester rather than # Platforms whose download + verify go through the native ingester rather than
# gallery-dl. gallery-dl still serves the rest (hentaifoundry, discord) until # gallery-dl. gallery-dl still serves the rest (hentaifoundry, discord,
# they migrate too. # deviantart — the latter slated for retirement, not migration) until they
# migrate too.
NATIVE_INGESTER_PLATFORMS = frozenset({"patreon", "subscribestar", "pixiv"}) NATIVE_INGESTER_PLATFORMS = frozenset({"patreon", "subscribestar", "pixiv"})
# Mirrors patreon_resolver._CAMPAIGNS_URL — surfaced in resolution-failure # Mirrors patreon_resolver._CAMPAIGNS_URL — surfaced in resolution-failure
+8 -7
View File
@@ -35,14 +35,9 @@ class InvalidUrlError(Exception):
# reviewers catch drift. # reviewers catch drift.
_PLATFORM_PATTERNS: list[tuple[str, re.Pattern[str]]] = [ _PLATFORM_PATTERNS: list[tuple[str, re.Pattern[str]]] = [
("patreon", re.compile( ("patreon", re.compile(
# Three creator URL shapes — bare (patreon.com/Atole), `c/`, and `cw/`
# (the "creator workspace" URL served once subscribed, see
# patreon_resolver._VANITY_RE). A trailing sub-path is allowed so a
# creator's inner page still derives the slug. Nav pages stay excluded.
r"^https?://(?:www\.)?patreon\.com/" r"^https?://(?:www\.)?patreon\.com/"
r"(?:cw/|c/)?" r"(?!home$|search\b|messages\b|notifications\b|library\b|settings\b|posts\b|c/)"
r"(?!(?:home|search|messages|notifications|library|settings|posts)(?:[/?#]|$))" r"(?P<slug>[^/?#]+)/?$",
r"(?P<slug>[^/?#]+)",
re.IGNORECASE, re.IGNORECASE,
)), )),
("subscribestar", re.compile( ("subscribestar", re.compile(
@@ -55,6 +50,12 @@ _PLATFORM_PATTERNS: list[tuple[str, re.Pattern[str]]] = [
r"^https?://(?:www\.)?hentai-foundry\.com/user/(?P<slug>[^/?#]+)", r"^https?://(?:www\.)?hentai-foundry\.com/user/(?P<slug>[^/?#]+)",
re.IGNORECASE, re.IGNORECASE,
)), )),
("deviantart", re.compile(
r"^https?://(?:www\.)?deviantart\.com/"
r"(?!home$|watch\b|tag\b|browse\b)"
r"(?P<slug>[^/?#]+)/?$",
re.IGNORECASE,
)),
("pixiv", re.compile( ("pixiv", re.compile(
r"^https?://(?:www\.)?pixiv\.net/(?:en/)?users/(?P<slug>\d+)", r"^https?://(?:www\.)?pixiv\.net/(?:en/)?users/(?P<slug>\d+)",
re.IGNORECASE, re.IGNORECASE,
+11 -3
View File
@@ -299,9 +299,8 @@ class GalleryDLService:
# (services/patreon_ingester.py), not gallery-dl. # (services/patreon_ingester.py), not gallery-dl.
PLATFORM_DEFAULTS = { PLATFORM_DEFAULTS = {
# subscribestar removed — native-ingester platform now (#71); pixiv # subscribestar removed — native-ingester platform now (#71); pixiv
# removed likewise (#129); deviantart removed at #3069 as a dropped # removed likewise (#129). The remaining entries are the gallery-dl
# platform, not a migrated one. The remaining entries are the # platforms not yet migrated.
# gallery-dl platforms not yet migrated.
"hentaifoundry": { "hentaifoundry": {
"content_types": ["all"], "content_types": ["all"],
"directory": [], "directory": [],
@@ -317,6 +316,15 @@ class GalleryDLService:
"reactions": False, "reactions": False,
"threads": True, "threads": True,
}, },
"deviantart": {
"content_types": ["all"],
"directory": [],
"filename": "{index:>03}_{title[:50]}.{extension}",
"flat": True,
"original": True,
"mature": True,
"metadata": True,
},
} }
def __init__( def __init__(
+15 -51
View File
@@ -31,7 +31,7 @@ from ..models import (
Tag, Tag,
TagPositiveConfirmation, TagPositiveConfirmation,
) )
from ..models.tag import CHROME_SYSTEM_TAGS, PROCESS_SYSTEM_TAGS, image_tag from ..models.tag import PRESENTATION_SYSTEM_TAGS, WIP_SYSTEM_TAG, image_tag
from .pagination import decode_cursor, encode_cursor from .pagination import decode_cursor, encode_cursor
from .tag_query import ( from .tag_query import (
fandom_join_alias, fandom_join_alias,
@@ -396,25 +396,6 @@ def _diversify_similar(src, rows, limit, *, dup_threshold=8, lam=0.40):
return [kept[i] for i in order] return [kept[i] for i in order]
def _reach_sample(rows, limit, reach):
"""From a distance-sorted candidate pool (nearest first), pick a spread of ranks
that MIXES near (tag the current cluster) and mid-far (escape it) BEFORE dedup +
MMR — the Explore "reach" dial (#1476).
reach in (0, 1]: the sampled span grows outward from the anchor (0.25→1.0 of the
pool), evenly strided from rank 0 so the nearest are still represented. In a
dense signature the nearest ranks are near-identical, so reaching farther is the
only way to hand MMR genuinely different content — MMR alone can't escape a pool
that's already all-near. reach<=0 or a small pool passes through unchanged."""
n = len(rows)
want = max(limit * 8, 100)
if reach <= 0 or n <= want:
return rows
span = int(min(1.0, 0.25 + 0.75 * reach) * n)
idx = sorted({min(int(i * span / want), n - 1) for i in range(want)})
return [rows[i] for i in idx]
async def _artists_for(session, image_ids: list[int]) -> dict[int, dict]: async def _artists_for(session, image_ids: list[int]) -> dict[int, dict]:
"""Map image_id -> {"name","slug"} via the canonical """Map image_id -> {"name","slug"} via the canonical
image_record.artist_id (FC-2d-vii-c). Bounded by page size.""" image_record.artist_id (FC-2d-vii-c). Bounded by page size."""
@@ -438,17 +419,16 @@ class GalleryService:
async def _hidden_tag_ids( async def _hidden_tag_ids(
self, include_hidden, tag_ids, tag_or_groups, self, include_hidden, tag_ids, tag_or_groups,
) -> list[int] | None: ) -> list[int] | None:
"""Chrome (banner) tag ids to implicitly exclude from a gallery query, or """Presentation-chrome tag ids to implicitly exclude from a gallery query,
None. None when the caller asked to include hidden, when the operator is or None. None when the caller asked to include hidden, when the operator
explicitly filtering FOR a chrome tag (they clearly want to see it), or when is explicitly filtering FOR a presentation tag (they clearly want to see
no chrome tags exist. (milestone 141; #1464: editor screenshot is now PROCESS it), or when no presentation tags exist. (milestone 141)"""
— shown — so only `banner` hides here.)"""
if include_hidden: if include_hidden:
return None return None
rows = await self.session.execute( rows = await self.session.execute(
select(Tag.id).where( select(Tag.id).where(
Tag.is_system.is_(True), Tag.is_system.is_(True),
Tag.name.in_(CHROME_SYSTEM_TAGS), Tag.name.in_(PRESENTATION_SYSTEM_TAGS),
) )
) )
pres = [r[0] for r in rows] pres = [r[0] for r in rows]
@@ -736,7 +716,6 @@ class GalleryService:
untagged: bool = False, no_artist: bool = False, untagged: bool = False, no_artist: bool = False,
date_from: datetime | None = None, date_to: datetime | None = None, date_from: datetime | None = None, date_to: datetime | None = None,
exclude_wip: bool = False, exclude_wip: bool = False,
reach: float = 0.0, exclude_ids: list[int] | None = None,
) -> list[GalleryImage] | None: ) -> list[GalleryImage] | None:
"""Visual "more like this": images near `image_id`'s SigLIP embedding """Visual "more like this": images near `image_id`'s SigLIP embedding
(pgvector, HNSW-indexed — alembic 0036), then DIVERSIFIED so the result (pgvector, HNSW-indexed — alembic 0036), then DIVERSIFIED so the result
@@ -765,27 +744,20 @@ class GalleryService:
# wide pool there's nothing but the near-dupes to choose from. Widened # wide pool there's nothing but the near-dupes to choose from. Widened
# (5×→8×, cap 200→400) so the stronger MMR has genuinely distinct # (5×→8×, cap 200→400) so the stronger MMR has genuinely distinct
# neighbourhoods to reach into for more variance (operator, 2026-07-01). # neighbourhoods to reach into for more variance (operator, 2026-07-01).
# Explore's reach>0 (#1476) widens it a LOT more: in a dense signature the pool_n = min(400, max(limit * 8, 100))
# nearest few hundred are all near-identical, so far-enough candidates only
# exist deeper in the ranked pool. _reach_sample then strides across them.
if reach > 0:
pool_n = min(1000, max(limit * 25, 100))
else:
pool_n = min(400, max(limit * 8, 100))
distance = ImageRecord.siglip_embedding.cosine_distance(src.siglip_embedding) distance = ImageRecord.siglip_embedding.cosine_distance(src.siglip_embedding)
eff = _effective_date_col() eff = _effective_date_col()
stmt = select(ImageRecord, Post.post_date, eff.label("eff")) stmt = select(ImageRecord, Post.post_date, eff.label("eff"))
stmt = _outer_join_primary_post(stmt) stmt = _outer_join_primary_post(stmt)
# Chrome (banner, #128) clusters on UI rather than content, so near any one # Presentation images (banner / editor-screenshot system tags, #128)
# of them they'd fill the grid → excluded from CANDIDATES always (the anchor # cluster on UI chrome rather than content, so near any one of them
# itself may be a banner). PROCESS art (wip / editor screenshot) stays # they'd fill the grid. Excluded from CANDIDATES only — the anchor
# surfaced here by default (real content; only the training pipelines exclude # itself may be a banner. `wip` stays surfaced here by default (real art;
# it), but the Explore rabbit-hole passes exclude_wip to also drop the whole # only the training pipelines exclude it), but the Explore rabbit-hole
# process group so a browse doesn't keep surfacing work-in-progress # passes exclude_wip to also drop work-in-progress (operator, 2026-07-08).
# (operator, 2026-07-08; #1464 — editor now rides with wip here). excluded_system_tags = PRESENTATION_SYSTEM_TAGS
excluded_system_tags = CHROME_SYSTEM_TAGS
if exclude_wip: if exclude_wip:
excluded_system_tags = (*CHROME_SYSTEM_TAGS, *PROCESS_SYSTEM_TAGS) excluded_system_tags = (*PRESENTATION_SYSTEM_TAGS, WIP_SYSTEM_TAG)
presentation = ( presentation = (
select(image_tag.c.image_record_id) select(image_tag.c.image_record_id)
.join(Tag, Tag.id == image_tag.c.tag_id) .join(Tag, Tag.id == image_tag.c.tag_id)
@@ -799,10 +771,6 @@ class GalleryService:
ImageRecord.id != image_id, ImageRecord.id != image_id,
ImageRecord.id.not_in(presentation), ImageRecord.id.not_in(presentation),
) )
# Anti-revisit (#1476): the Explore walk passes its breadcrumb so already-
# walked images aren't re-served as neighbours — → can't loop you back in.
if exclude_ids:
stmt = stmt.where(ImageRecord.id.not_in(exclude_ids))
stmt = _apply_scope( stmt = _apply_scope(
stmt, tag_ids=tag_ids, post_id=None, stmt, tag_ids=tag_ids, post_id=None,
artist_id=artist_id, media_type=media_type, artist_id=artist_id, media_type=media_type,
@@ -812,10 +780,6 @@ class GalleryService:
) )
stmt = stmt.order_by(distance.asc()).limit(pool_n) stmt = stmt.order_by(distance.asc()).limit(pool_n)
rows = (await self.session.execute(stmt)).all() rows = (await self.session.execute(stmt)).all()
# Explore reach: stride across an outward-growing distance span so the pool
# handed to MMR spans near→mid-far, not just the tight cluster (#1476).
if reach > 0:
rows = _reach_sample(rows, limit, reach)
rows = _diversify_similar(src, rows, limit) rows = _diversify_similar(src, rows, limit)
artists = await _artists_for(self.session, [r[0].id for r in rows]) artists = await _artists_for(self.session, [r[0].id for r in rows])
return _gallery_images(rows, artists) return _gallery_images(rows, artists)
-51
View File
@@ -1,51 +0,0 @@
"""Bulk, idempotent writes to the ``image_tag`` association table.
Three writers attach tags to images in bulk: the WIP-title backfill
(`wip_title.apply_wip_image_tags`), the concept-head auto-apply sweep and the
system-tag auto-apply sweep (both in `ml/heads.py`). The two sweeps used to
issue ONE INSERT PER ROW from inside their per-image loop — fine in steady
state, but a first pass over a back-catalogue is tens of thousands of
individual round-trips (#3072). All three share this one chunked multi-row
insert now.
Sync only: every caller runs on a sync ``Session`` (the Celery task path). No
async service writes image_tag in bulk, so there is no async sibling to keep in
step — unlike `db_helpers.get_or_create`, which does have one.
"""
from __future__ import annotations
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.orm import Session
from ..models.tag import image_tag
# 5000 rows x 3 bound params = 15000, comfortably inside Postgres' 65535-param
# ceiling for a single statement. Raising this past ~21000 rows would exceed it.
INSERT_CHUNK = 5000
def insert_image_tags(
session: Session, rows: list[dict], *, chunk: int = INSERT_CHUNK
) -> None:
"""Attach ``rows`` to their images, skipping any tag already on one.
Each row is ``{"image_record_id": int, "tag_id": int, "source": str}``.
Does NOT commit — the caller owns the transaction.
ON CONFLICT DO NOTHING against the (image_record_id, tag_id) primary key,
so an existing tag keeps its ORIGINAL ``source``: re-running a sweep can
never re-stamp a tag the operator applied by hand as machine-applied.
Returns nothing on purpose. psycopg reports ``rowcount`` -1 for a multi-row
ON CONFLICT DO NOTHING insert (it runs via an executemany path), so a count
taken from the statement would be a lie rather than an approximation.
Callers that need an accurate count derive it themselves — see
`wip_title.apply_wip_image_tags`' pre-SELECT, and the sweeps' `skip` sets.
"""
for start in range(0, len(rows), chunk):
session.execute(
pg_insert(image_tag)
.values(rows[start:start + chunk])
.on_conflict_do_nothing(index_elements=["image_record_id", "tag_id"])
)
-65
View File
@@ -47,21 +47,9 @@ from .attachment_store import AttachmentStore
from .audits import single_color from .audits import single_color
from .link_extract import extract_external_links from .link_extract import extract_external_links
from .thumbnailer import Thumbnailer from .thumbnailer import Thumbnailer
from .wip_title import (
WIP_TITLE_SOFT_SOURCE,
WIP_TITLE_SOURCE,
apply_wip_image_tags,
matches_soft_wip_title,
matches_wip_title,
resolve_wip_tag_id,
)
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
# Sentinel for the lazily-resolved wip tag id (distinguishes "not resolved yet"
# from a genuine None = tag absent, so absence is cached and not re-queried).
_UNSET = object()
class SkipReason(StrEnum): class SkipReason(StrEnum):
too_small = "too_small" too_small = "too_small"
@@ -195,10 +183,6 @@ class Importer:
# invalidated mid-Importer (Importer instances are per-task / # invalidated mid-Importer (Importer instances are per-task /
# per-archive-import so cross-instance staleness is harmless). # per-archive-import so cross-instance staleness is harmless).
self._phash_candidates: list[tuple] | None = None self._phash_candidates: list[tuple] | None = None
# Lazily-resolved `wip` system tag id for title-based WIP auto-tagging
# (task #1458). Sentinel _UNSET so a genuine None (tag absent) is cached
# and not re-queried per media. Importer is per-task, so this can't stale.
self._wip_tag_id: int | None = _UNSET
def _phash_candidates_cache(self) -> list[tuple]: def _phash_candidates_cache(self) -> list[tuple]:
"""Cached `(phash, width, height, id)` rows from image_record. """Cached `(phash, width, height, id)` rows from image_record.
@@ -949,10 +933,6 @@ class Importer:
# Thumbnail is queued separately by the calling task; the importer # Thumbnail is queued separately by the calling task; the importer
# does not generate thumbnails inline so the import queue stays moving. # does not generate thumbnails inline so the import queue stays moving.
# Title-based WIP auto-tag (task #1458): fresh import only, after the
# sidecar has linked the post so record.primary_post_id / its title exist.
self._maybe_apply_wip_title(record)
self.session.commit() self.session.commit()
return ImportResult(status="imported", image_id=record.id) return ImportResult(status="imported", image_id=record.id)
@@ -996,47 +976,6 @@ class Importer:
self.session.commit() self.session.commit()
return ImportResult(status="refreshed", image_id=existing.id) return ImportResult(status="refreshed", image_id=existing.id)
def _maybe_apply_wip_title(self, record: ImageRecord) -> None:
"""Auto-apply the `wip` system tag to a FRESHLY-imported image when its
primary post's TITLE explicitly declares work-in-progress (task #1458 —
the artist's own "WIP" / "work in progress" label).
Called ONLY from the two new-record paths (never deep-scan / supersede),
so a manually-removed WIP tag is never re-applied by a routine re-scan —
removal sticks. The existing catalogue is covered separately by the
operator-triggered backfill sweep. Gated by the settings toggle, and
best-effort: any failure is logged, never allowed to fail the import."""
hard_on = self.settings.wip_title_tagging_enabled
soft_on = self.settings.wip_soft_title_tagging_enabled
if not (hard_on or soft_on):
return
if record.primary_post_id is None:
return
try:
title = self.session.execute(
select(Post.post_title).where(Post.id == record.primary_post_id)
).scalar_one_or_none()
# HARD tier ("WIP"/"work in progress") wins — higher precision, and it
# trains the head; SOFT (sketch/doodle, #1474) is the provisional fallback
# that never trains (source wip_title_soft).
if hard_on and matches_wip_title(title):
source = WIP_TITLE_SOURCE
elif soft_on and matches_soft_wip_title(title):
source = WIP_TITLE_SOFT_SOURCE
else:
return
if self._wip_tag_id is _UNSET:
self._wip_tag_id = resolve_wip_tag_id(self.session)
if self._wip_tag_id is None:
return
apply_wip_image_tags(
self.session, [record.id], self._wip_tag_id, source=source
)
except Exception as exc: # noqa: BLE001 — a tag must never fail an import
log.warning(
"wip-title auto-tag failed for image %s: %s", record.id, exc
)
def _apply_post_fields(self, post: Post, sd) -> None: def _apply_post_fields(self, post: Post, sd) -> None:
"""Write a parsed sidecar's post-level fields onto a Post — the SINGLE """Write a parsed sidecar's post-level fields onto a Post — the SINGLE
predicate shared by BOTH ingest paths: the per-media path (_apply_sidecar) predicate shared by BOTH ingest paths: the per-media path (_apply_sidecar)
@@ -1314,10 +1253,6 @@ class Importer:
# per-post Source row. # per-post Source row.
self._apply_sidecar(record, path, artist, explicit_source=source) self._apply_sidecar(record, path, artist, explicit_source=source)
# Title-based WIP auto-tag (task #1458): fresh import only, see the
# matching call in _import_media.
self._maybe_apply_wip_title(record)
self.session.commit() self.session.commit()
return ImportResult(status="imported", image_id=record.id) return ImportResult(status="imported", image_id=record.id)
+14
View File
@@ -12,11 +12,13 @@ from sqlalchemy.ext.asyncio import AsyncSession
from ...models import TagSuggestionRejection from ...models import TagSuggestionRejection
from ...models.tag import image_tag from ...models.tag import image_tag
from .aliases import AliasService
class AllowlistService: class AllowlistService:
def __init__(self, session: AsyncSession): def __init__(self, session: AsyncSession):
self.session = session self.session = session
self.aliases = AliasService(session)
async def _apply_image_tag(self, image_id: int, tag_id: int, source: str): async def _apply_image_tag(self, image_id: int, tag_id: int, source: str):
stmt = insert(image_tag).values( stmt = insert(image_tag).values(
@@ -39,6 +41,18 @@ class AllowlistService:
await self._apply_image_tag(image_id, tag_id, source="ml_accepted") await self._apply_image_tag(image_id, tag_id, source="ml_accepted")
await self._clear_rejection(image_id, tag_id) await self._clear_rejection(image_id, tag_id)
async def add_alias_and_accept(
self,
image_id: int,
alias_string: str,
alias_category: str,
canonical_tag_id: int,
) -> None:
await self.aliases.create(
alias_string, alias_category, canonical_tag_id
)
await self.accept(image_id, canonical_tag_id)
async def dismiss(self, image_id: int, tag_id: int) -> None: async def dismiss(self, image_id: int, tag_id: int) -> None:
stmt = insert(TagSuggestionRejection).values( stmt = insert(TagSuggestionRejection).values(
image_record_id=image_id, tag_id=tag_id image_record_id=image_id, tag_id=tag_id
@@ -150,7 +150,9 @@ def refresh_character_prototypes(
"""Incrementally refresh the prototype store. `full=True` rebuilds every """Incrementally refresh the prototype store. `full=True` rebuilds every
character regardless of the gate/fingerprints (nightly reconcile). Returns character regardless of the gate/fingerprints (nightly reconcile). Returns
{skipped, rebuilt, removed}; commits.""" {skipped, rebuilt, removed}; commits."""
settings = MLSettings.load_sync(session) settings = session.execute(
select(MLSettings).where(MLSettings.id == 1)
).scalar_one()
sig = _global_signature(session) sig = _global_signature(session)
if not full and settings.ccip_ref_signature == sig: if not full and settings.ccip_ref_signature == sig:
return {"skipped": True, "rebuilt": 0, "removed": 0} return {"skipped": True, "rebuilt": 0, "removed": 0}
@@ -202,7 +204,9 @@ def retract_auto_applied_ccip(session: Session) -> int:
n_retracted.""" n_retracted."""
import numpy as np import numpy as np
settings = MLSettings.load_sync(session) settings = session.execute(
select(MLSettings).where(MLSettings.id == 1)
).scalar_one()
if not settings.ccip_auto_apply_enabled: if not settings.ccip_auto_apply_enabled:
return 0 return 0
thr = float(settings.ccip_auto_apply_threshold) thr = float(settings.ccip_auto_apply_threshold)
+97 -218
View File
@@ -23,7 +23,6 @@ from datetime import UTC, datetime
from typing import Any from typing import Any
from sqlalchemy import delete, exists, func, select from sqlalchemy import delete, exists, func, select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -40,11 +39,9 @@ from ...models import (
TagPositiveConfirmation, TagPositiveConfirmation,
TagSuggestionRejection, TagSuggestionRejection,
) )
from ...models.tag import CHROME_SYSTEM_TAGS, PROCESS_SYSTEM_TAGS, image_tag from ...models.tag import PRESENTATION_SYSTEM_TAGS, image_tag
from ..image_tag_apply import insert_image_tags
from .training_data import ( from .training_data import (
_AUTO_SOURCES, _AUTO_SOURCES,
_applied_or_rejected,
_auto_apply_point, _auto_apply_point,
_hygiene_excluded_ids, _hygiene_excluded_ids,
_ids_with_tag, _ids_with_tag,
@@ -64,14 +61,6 @@ MIN_POSITIVES_FLOOR = 8 # hard floor; settings.head_min_positives can raise
_UNLABELED_POOL = 4000 _UNLABELED_POOL = 4000
_EXAMPLES_MIN = 8 # need at least this many embedded +/- to fit a head _EXAMPLES_MIN = 8 # need at least this many embedded +/- to fit a head
# Auto-apply / match confidence operating range. Every graduated auto-apply or
# CCIP-match threshold the operator can set lives in this band, and the head
# precision target is clamped to it: below 0.5 "auto-apply" is meaningless, and
# 1.0 is unachievable so 0.999 is the ceiling. One source shared by the service
# clamp (_normalize_params) and the API validator (ml_admin._validate).
AUTO_APPLY_THRESHOLD_MIN = 0.5
AUTO_APPLY_THRESHOLD_MAX = 0.999
# Only these tag kinds get heads (the surfaced suggestion categories). # Only these tag kinds get heads (the surfaced suggestion categories).
_HEAD_KINDS = (TagKind.general, TagKind.character) _HEAD_KINDS = (TagKind.general, TagKind.character)
# tag.kind -> the suggestion category the rail groups under. # tag.kind -> the suggestion category the rail groups under.
@@ -89,38 +78,6 @@ _CATEGORY = {TagKind.general: "general", TagKind.character: "character"}
_SYSTEM_TAG_SUGGEST_FLOOR = 0.65 _SYSTEM_TAG_SUGGEST_FLOOR = 0.65
def _sigmoid(z, np):
"""Logistic sigmoid 1/(1+e^-z): the head score→probability transform. One home
for what was inlined at every scoring site (suggest, both sweeps, retract)."""
return 1.0 / (1.0 + np.exp(-z))
def _conflict_scores(Xn, Wc, bc, np):
"""The presentation conflict signal (#141): per row, the MAX content-head
probability and WHICH head produced it. Shared by the system-tag sweep's guard-2
and the soft-wip audit — both ask "does this ALSO look like real content?"."""
cprobs = _sigmoid(Xn @ Wc.T + bc, np)
return cprobs.max(axis=1), cprobs.argmax(axis=1)
def _insert_presentation_review(
session, *, image_record_id, tag_id, conflict_tag_id, conflict_score, mode,
):
"""Single-source the ring-loud PresentationReview row shape so the two writers
(system-tag sweep guard-2 + soft-wip audit) can't drift on columns or `mode` —
they share the (image_record_id, tag_id) composite PK, so a divergent `mode`
would be a silent first-writer-wins bug."""
session.execute(
pg_insert(PresentationReview)
.values(
image_record_id=image_record_id, tag_id=tag_id,
conflict_tag_id=conflict_tag_id, conflict_score=conflict_score,
mode=mode,
)
.on_conflict_do_nothing()
)
class HeadTrainingAlreadyRunning(Exception): class HeadTrainingAlreadyRunning(Exception):
"""Raised by start_head_training_run when a run is already in flight.""" """Raised by start_head_training_run when a run is already in flight."""
@@ -146,7 +103,9 @@ def start_head_training_run(session: Session, params: dict[str, Any]) -> int:
def _settings(session: Session) -> MLSettings: def _settings(session: Session) -> MLSettings:
return MLSettings.load_sync(session) return session.execute(
select(MLSettings).where(MLSettings.id == 1)
).scalar_one()
def _normalize_params(session: Session, params: dict[str, Any] | None) -> dict[str, Any]: def _normalize_params(session: Session, params: dict[str, Any] | None) -> dict[str, Any]:
@@ -165,7 +124,7 @@ def _normalize_params(session: Session, params: dict[str, Any] | None) -> dict[s
except (TypeError, ValueError): except (TypeError, ValueError):
cv_folds = DEFAULT_CV_FOLDS cv_folds = DEFAULT_CV_FOLDS
try: try:
precision_target = min(max(float(params.get("precision_target", s.head_auto_apply_precision)), AUTO_APPLY_THRESHOLD_MIN), AUTO_APPLY_THRESHOLD_MAX) precision_target = min(max(float(params.get("precision_target", s.head_auto_apply_precision)), 0.5), 0.999)
except (TypeError, ValueError): except (TypeError, ValueError):
precision_target = s.head_auto_apply_precision precision_target = s.head_auto_apply_precision
return { return {
@@ -541,19 +500,13 @@ async def score_image(
session: AsyncSession, image_id: int, threshold_override: float | None = None, session: AsyncSession, image_id: int, threshold_override: float | None = None,
) -> list[dict]: ) -> list[dict]:
"""Suggestions for one image from the trained heads: [{tag_id, name, """Suggestions for one image from the trained heads: [{tag_id, name,
category, score, above_threshold, grounding}], ranked. A concept is INCLUDED category, score}], ranked. A concept surfaces when its score clears the
when its score clears the head's own suggest_threshold — or, when head's own suggest_threshold — or, when threshold_override is given (the
threshold_override is given (the typed-dropdown "show everything" mode), that typed-dropdown "show everything" mode), that flat floor instead (0 → every
flat floor instead (0 → every head). System-tag heads (wip/banner/editor) head). System-tag heads (wip/banner/editor) instead use a flat
instead use a flat _SYSTEM_TAG_SUGGEST_FLOOR so their false positives surface _SYSTEM_TAG_SUGGEST_FLOOR so their false positives surface for rejection
for rejection (still overridden by threshold_override). Empty if the image has (still overridden by threshold_override). Empty if the image has no
no embedding or no heads exist yet. embedding or no heads exist yet.
``above_threshold`` is reported SEPARATELY from inclusion: it's always whether
the score cleared the head's NATURAL cut (suggest_threshold, or the system
floor), regardless of any override. So the single min=0 fetch returns every
head, and the caller can split panel (above_threshold) from dropdown (all)
without a second request.
MAX-OVER-BAG: the image is scored as a BAG of embeddings — the whole-image MAX-OVER-BAG: the image is scored as a BAG of embeddings — the whole-image
vector PLUS every concept-region crop the agent embedded (same model vector PLUS every concept-region crop the agent embedded (same model
@@ -577,32 +530,28 @@ async def score_image(
norms[norms == 0] = 1.0 norms[norms == 0] = 1.0
Xn = X / norms Xn = X / norms
Z = Xn @ heads["W"].T + heads["b"] # (B, H) Z = Xn @ heads["W"].T + heads["b"] # (B, H)
probs_bag = _sigmoid(Z, np) # (B, H) probs_bag = 1.0 / (1.0 + np.exp(-Z)) # (B, H)
probs = probs_bag.max(axis=0) # (H,) best over the bag probs = probs_bag.max(axis=0) # (H,) best over the bag
# ARGMAX beside the max: WHICH bag row won each head → the region that grounds # ARGMAX beside the max: WHICH bag row won each head → the region that grounds
# the tag (bag_meta[win]); None when the whole-image vector won (#1206). # the tag (bag_meta[win]); None when the whole-image vector won (#1206).
winners = probs_bag.argmax(axis=0) # (H,) winners = probs_bag.argmax(axis=0) # (H,)
out = [] out = []
for i, p in enumerate(probs): for i, p in enumerate(probs):
m = heads["meta"][i] if threshold_override is not None:
# The head's NATURAL suggest cut — system tags use the flat floor (see cut = threshold_override
# _SYSTEM_TAG_SUGGEST_FLOOR) so their false positives show up for the elif heads["meta"][i]["is_system"]:
# operator to reject; content heads use their own precision-tuned # System tags surface at the flat floor (see _SYSTEM_TAG_SUGGEST_FLOOR)
# threshold. This is what "above threshold" means (drives the panel). # so their false positives show up for the operator to reject.
natural = ( cut = _SYSTEM_TAG_SUGGEST_FLOOR
_SYSTEM_TAG_SUGGEST_FLOOR if m["is_system"] else float(heads["thr"][i]) else:
) cut = heads["thr"][i]
# INCLUSION is looser under threshold_override (dropdown show-all,
# override=0): every head comes back so a low-confidence concept can still
# be typed + picked, each carrying its own above_threshold flag.
cut = threshold_override if threshold_override is not None else natural
if p >= cut: if p >= cut:
m = heads["meta"][i]
out.append({ out.append({
"tag_id": m["tag_id"], "tag_id": m["tag_id"],
"name": m["name"], "name": m["name"],
"category": m["category"], "category": m["category"],
"score": float(p), "score": float(p),
"above_threshold": bool(p >= natural),
"grounding": bag_meta[int(winners[i])], "grounding": bag_meta[int(winners[i])],
}) })
out.sort(key=lambda d: d["score"], reverse=True) out.sort(key=lambda d: d["score"], reverse=True)
@@ -655,7 +604,9 @@ async def ground_applied_tag(
async def _settings_async(session: AsyncSession) -> MLSettings: async def _settings_async(session: AsyncSession) -> MLSettings:
return await MLSettings.load(session) return (
await session.execute(select(MLSettings).where(MLSettings.id == 1))
).scalar_one()
# --- Earned auto-apply (sync, ml worker) --------------------------------- # --- Earned auto-apply (sync, ml worker) ---------------------------------
@@ -726,6 +677,7 @@ def auto_apply_sweep(
embeddings in chunks; commits per chunk on a real run. Returns embeddings in chunks; commits per chunk on a real run. Returns
{n_applied, concepts:[{tag_id,name,applied,scanned,threshold}]}.""" {n_applied, concepts:[{tag_id,name,applied,scanned,threshold}]}."""
import numpy as np import numpy as np
from sqlalchemy.dialects.postgresql import insert as pg_insert
settings = _settings(session) settings = _settings(session)
rows = _auto_apply_heads( rows = _auto_apply_heads(
@@ -742,7 +694,18 @@ def auto_apply_sweep(
names = [r.name for r in rows] names = [r.name for r in rows]
# Skip images that already carry, or have rejected, each tag. # Skip images that already carry, or have rejected, each tag.
skip = _applied_or_rejected(session, tag_ids) skip = {tid: set() for tid in tag_ids}
for tid in tag_ids:
for (iid,) in session.execute(
select(image_tag.c.image_record_id).where(image_tag.c.tag_id == tid)
):
skip[tid].add(iid)
for (iid,) in session.execute(
select(TagSuggestionRejection.image_record_id).where(
TagSuggestionRejection.tag_id == tid
)
):
skip[tid].add(iid)
applied = [0] * len(rows) applied = [0] * len(rows)
scanned = 0 scanned = 0
@@ -756,12 +719,8 @@ def auto_apply_sweep(
if not cids: if not cids:
continue continue
Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np) Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np)
probs = _sigmoid(Xn @ W.T + b, np) # (N, H) probs = 1.0 / (1.0 + np.exp(-(Xn @ W.T + b))) # (N, H)
scanned += len(cids) scanned += len(cids)
# Collected across every head, then written as ONE insert below. Was an
# insert per applied tag from inside this loop, which on a first sweep
# over a back-catalogue is tens of thousands of round-trips (#3072).
pending: list[dict] = []
for h in range(len(rows)): for h in range(len(rows)):
tid = tag_ids[h] tid = tag_ids[h]
for idx in np.where(probs[:, h] >= thr[h])[0]: for idx in np.where(probs[:, h] >= thr[h])[0]:
@@ -771,12 +730,12 @@ def auto_apply_sweep(
skip[tid].add(iid) skip[tid].add(iid)
applied[h] += 1 applied[h] += 1
if not dry_run: if not dry_run:
pending.append({ session.execute(
"image_record_id": iid, "tag_id": tid, pg_insert(image_tag)
"source": "head_auto", .values(image_record_id=iid, tag_id=tid, source="head_auto")
}) .on_conflict_do_nothing()
)
if not dry_run: if not dry_run:
insert_image_tags(session, pending)
session.commit() session.commit()
run.last_progress_at = datetime.now(UTC) run.last_progress_at = datetime.now(UTC)
session.commit() session.commit()
@@ -790,42 +749,18 @@ def auto_apply_sweep(
_PRESENTATION_SOURCE = "presentation_auto" _PRESENTATION_SOURCE = "presentation_auto"
_PROCESS_SOURCE = "process_auto"
# System-tag auto-apply modes (#1464). Both modes run the identical sweep — apply
# a system tag at a flat threshold with a PROVISIONAL source + a ring-loud review
# guard — and differ ONLY in which tags, which settings knobs, and which
# source/review-mode. 'chrome' (banner) is HIDDEN from the gallery; 'process'
# (wip / editor screenshot) stays VISIBLE (the hide is a gallery-query effect of
# the tag's group membership, not of this sweep).
_SWEEP_MODES = {
"chrome": {
"names": CHROME_SYSTEM_TAGS,
"enabled": "presentation_auto_apply_enabled",
"threshold": "presentation_auto_apply_threshold",
"conflict": "presentation_conflict_threshold",
"source": _PRESENTATION_SOURCE,
},
"process": {
"names": PROCESS_SYSTEM_TAGS,
"enabled": "process_auto_apply_enabled",
"threshold": "process_auto_apply_threshold",
"conflict": "process_conflict_threshold",
"source": _PROCESS_SOURCE,
},
}
def _system_tag_heads(session: Session, embedding_version: str, names): def _presentation_heads(session: Session, embedding_version: str):
"""Trained heads for a system-tag group (chrome banner / process wip+editor). """Trained heads for the presentation chrome tags (banner / editor screenshot).
They fire at the group's FLAT threshold regardless of graduation — a head They fire at the FLAT presentation threshold regardless of graduation — a head
exists once the operator has labelled enough (head_min_positives).""" exists once the operator has labelled enough chrome (head_min_positives)."""
return session.execute( return session.execute(
select(TagHead.tag_id, Tag.name, TagHead.weights, TagHead.bias) select(TagHead.tag_id, Tag.name, TagHead.weights, TagHead.bias)
.join(Tag, Tag.id == TagHead.tag_id) .join(Tag, Tag.id == TagHead.tag_id)
.where(TagHead.embedding_version == embedding_version) .where(TagHead.embedding_version == embedding_version)
.where(Tag.is_system.is_(True)) .where(Tag.is_system.is_(True))
.where(Tag.name.in_(names)) .where(Tag.name.in_(PRESENTATION_SYSTEM_TAGS))
).all() ).all()
@@ -857,32 +792,27 @@ def _valued_image_ids(session: Session) -> set[int]:
return {r[0] for r in rows} return {r[0] for r in rows}
def system_tag_auto_apply_sweep( def presentation_auto_apply_sweep(session: Session, dry_run: bool = False) -> dict:
session: Session, *, mode: str, dry_run: bool = False """Auto-hide presentation chrome (banner / editor screenshot) at the FLAT
) -> dict: presentation threshold (#141) — NOT the per-head graduated threshold. Two
"""Auto-apply a system-tag group at its FLAT threshold. mode='chrome' (banner, guards keep it safe: (1) never hide an image carrying a human/confirmed content
#141) hides the image; mode='process' (wip / editor screenshot, #1464) keeps it tag; (2) if an image about to be hidden ALSO scores >= the conflict threshold
VISIBLE — the ONLY difference is the tag group's gallery membership, not this on a content head, still hide it but flag it (PresentationReview) so the Hidden
sweep. Two guards keep it safe: (1) never touch an image carrying a view surfaces "also looks like <X>" for review. No-op unless
human/confirmed content tag; (2) if the image ALSO scores >= the conflict presentation_auto_apply_enabled. numpy-only (no sklearn). Returns
threshold on a content head, still apply but flag it (PresentationReview, {n_applied, n_flagged, concepts}."""
mode=<mode>) so the review strip surfaces "also looks like <X>". The source is
PROVISIONAL so the head never trains on its own output. No-op unless the mode's
enabled flag is set. numpy-only (no sklearn). Returns {n_applied, n_flagged,
concepts}."""
import numpy as np import numpy as np
from sqlalchemy.dialects.postgresql import insert as pg_insert
cfg = _SWEEP_MODES[mode]
settings = _settings(session) settings = _settings(session)
if not dry_run and not getattr(settings, cfg["enabled"]): if not dry_run and not settings.presentation_auto_apply_enabled:
return {"n_applied": 0, "n_flagged": 0, "concepts": []} return {"n_applied": 0, "n_flagged": 0, "concepts": []}
ver = settings.embedder_model_version ver = settings.embedder_model_version
pres = _system_tag_heads(session, ver, cfg["names"]) pres = _presentation_heads(session, ver)
if not pres: if not pres:
return {"n_applied": 0, "n_flagged": 0, "concepts": []} return {"n_applied": 0, "n_flagged": 0, "concepts": []}
thr = float(getattr(settings, cfg["threshold"])) thr = float(settings.presentation_auto_apply_threshold)
conflict_thr = float(getattr(settings, cfg["conflict"])) conflict_thr = float(settings.presentation_conflict_threshold)
source = cfg["source"]
Wp = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in pres]) Wp = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in pres])
bp = np.asarray([r.bias for r in pres], dtype=np.float32) bp = np.asarray([r.bias for r in pres], dtype=np.float32)
@@ -899,7 +829,18 @@ def system_tag_auto_apply_sweep(
valued = _valued_image_ids(session) valued = _valued_image_ids(session)
# Skip images that already carry, or have rejected, each presentation tag. # Skip images that already carry, or have rejected, each presentation tag.
skip = _applied_or_rejected(session, pres_tag_ids) skip = {tid: set() for tid in pres_tag_ids}
for tid in pres_tag_ids:
for (iid,) in session.execute(
select(image_tag.c.image_record_id).where(image_tag.c.tag_id == tid)
):
skip[tid].add(iid)
for (iid,) in session.execute(
select(TagSuggestionRejection.image_record_id).where(
TagSuggestionRejection.tag_id == tid
)
):
skip[tid].add(iid)
applied = [0] * len(pres) applied = [0] * len(pres)
n_flagged = 0 n_flagged = 0
@@ -914,15 +855,12 @@ def system_tag_auto_apply_sweep(
if not cids: if not cids:
continue continue
Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np) Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np)
probs = _sigmoid(Xn @ Wp.T + bp, np) # (N, P) probs = 1.0 / (1.0 + np.exp(-(Xn @ Wp.T + bp))) # (N, P)
if Wc is not None: if Wc is not None:
max_c, arg_c = _conflict_scores(Xn, Wc, bc, np) # (N,), (N,) cprobs = 1.0 / (1.0 + np.exp(-(Xn @ Wc.T + bc))) # (N, C)
max_c = cprobs.max(axis=1)
arg_c = cprobs.argmax(axis=1)
scanned += len(cids) scanned += len(cids)
# Same batching as auto_apply_sweep (#3072): collect the chunk's rows
# and write them once, below. The PresentationReview rows stay per-row —
# they FK to image_record/tag, not to image_tag, so writing the tags
# after them is safe, and a flagged conflict is rare by construction.
pending: list[dict] = []
for p in range(len(pres)): for p in range(len(pres)):
tid = pres_tag_ids[p] tid = pres_tag_ids[p]
for idx in np.where(probs[:, p] >= thr)[0]: for idx in np.where(probs[:, p] >= thr)[0]:
@@ -932,25 +870,28 @@ def system_tag_auto_apply_sweep(
skip[tid].add(iid) skip[tid].add(iid)
applied[p] += 1 applied[p] += 1
if not dry_run: if not dry_run:
pending.append({ session.execute(
"image_record_id": iid, "tag_id": tid, pg_insert(image_tag)
"source": source, .values(
}) image_record_id=iid, tag_id=tid,
# Guard 2: also looks like real content → still apply, but flag it source=_PRESENTATION_SOURCE,
# for the review strip instead of silently marking (chrome hides, )
# process stays visible — either way the operator gets a heads-up). .on_conflict_do_nothing()
)
# Guard 2: also looks like content → hide but flag for review.
if Wc is not None and float(max_c[idx]) >= conflict_thr: if Wc is not None and float(max_c[idx]) >= conflict_thr:
n_flagged += 1 n_flagged += 1
if not dry_run: if not dry_run:
_insert_presentation_review( session.execute(
session, pg_insert(PresentationReview)
image_record_id=iid, tag_id=tid, .values(
conflict_tag_id=conf_tag_ids[int(arg_c[idx])], image_record_id=iid, tag_id=tid,
conflict_score=float(max_c[idx]), conflict_tag_id=conf_tag_ids[int(arg_c[idx])],
mode=mode, conflict_score=float(max_c[idx]),
)
.on_conflict_do_nothing()
) )
if not dry_run: if not dry_run:
insert_image_tags(session, pending)
session.commit() session.commit()
concepts = [ concepts = [
@@ -963,68 +904,6 @@ def system_tag_auto_apply_sweep(
} }
def soft_wip_conflict_audit(session: Session, dry_run: bool = False) -> dict:
"""Ring-loud audit for the SOFT WIP-title cohort (#1474). Images auto-tagged
`wip` from a low-precision sketch/doodle title (source='wip_title_soft') that ALSO
score >= the process conflict threshold on a content head are probably FINISHED
art mis-tagged as process — flag them (PresentationReview, mode='process') so the
review strip surfaces them ("also looks like <X>", Keep tag / Remove tag). Does
NOT remove the tag; the operator decides. No-op when there are no content heads.
numpy-only. Returns {n_scanned, n_flagged}."""
import numpy as np
from ..wip_title import WIP_TITLE_SOFT_SOURCE, resolve_wip_tag_id
settings = _settings(session)
ver = settings.embedder_model_version
conflict_thr = float(settings.process_conflict_threshold)
conf = _conflict_heads(session, ver)
wip_id = resolve_wip_tag_id(session)
if not conf or wip_id is None:
return {"n_scanned": 0, "n_flagged": 0}
Wc = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in conf])
bc = np.asarray([r.bias for r in conf], dtype=np.float32)
conf_tag_ids = [r.tag_id for r in conf]
soft_ids = [iid for (iid,) in session.execute(
select(image_tag.c.image_record_id)
.where(image_tag.c.tag_id == wip_id)
.where(image_tag.c.source == WIP_TITLE_SOFT_SOURCE)
)]
# Skip images already flagged for this tag (idempotent re-runs).
flagged = {iid for (iid,) in session.execute(
select(PresentationReview.image_record_id)
.where(PresentationReview.tag_id == wip_id)
)}
soft_ids = [i for i in soft_ids if i not in flagged]
n_flagged = 0
scanned = 0
for start in range(0, len(soft_ids), _AUTO_APPLY_CHUNK):
chunk = soft_ids[start:start + _AUTO_APPLY_CHUNK]
emb = _load_embeddings(session, chunk)
cids = [i for i in chunk if i in emb]
if not cids:
continue
scanned += len(cids)
Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np)
max_c, arg_c = _conflict_scores(Xn, Wc, bc, np)
for k in range(len(cids)):
if float(max_c[k]) >= conflict_thr:
n_flagged += 1
if not dry_run:
_insert_presentation_review(
session,
image_record_id=cids[k], tag_id=wip_id,
conflict_tag_id=conf_tag_ids[int(arg_c[k])],
conflict_score=float(max_c[k]),
mode="process",
)
if not dry_run:
session.commit()
return {"n_scanned": scanned, "n_flagged": n_flagged}
def retract_auto_applied_heads(session: Session) -> int: def retract_auto_applied_heads(session: Session) -> int:
"""Soft auto-apply (milestone 139): re-score every standing source='head_auto' """Soft auto-apply (milestone 139): re-score every standing source='head_auto'
tag against its CURRENT head and REMOVE the ones now BELOW the head's tag against its CURRENT head and REMOVE the ones now BELOW the head's
@@ -1072,7 +951,7 @@ def retract_auto_applied_heads(session: Session) -> int:
continue continue
Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np) Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np)
w = np.asarray(weights, dtype=np.float32) w = np.asarray(weights, dtype=np.float32)
probs = _sigmoid(Xn @ w + float(bias), np) probs = 1.0 / (1.0 + np.exp(-(Xn @ w + float(bias))))
below = [cids[k] for k in np.where(probs < float(thr))[0]] below = [cids[k] for k in np.where(probs < float(thr))[0]]
for iid in below: for iid in below:
session.execute( session.execute(
+17 -17
View File
@@ -22,20 +22,22 @@ from .heads import score_image
@dataclass(frozen=True) @dataclass(frozen=True)
class Suggestion: class Suggestion:
# Every suggestion is a canonical Tag: heads/CCIP only score EXISTING concept # canonical_tag_id is None when this is a raw Camie tag with no alias and
# tags (tagging-v2, #114). The old raw-model-key / creates-new / alias-remap # no existing Tag row — accepting it will create the tag.
# cases are gone — a suggestion always maps to a real tag id. canonical_tag_id: int | None
canonical_tag_id: int
display_name: str display_name: str
category: str category: str
score: float score: float
source: str # 'head' | 'ccip' | 'both' (Camie tagger/centroid removed in v2) source: str # 'head' | 'ccip' | 'both' (Camie tagger/centroid removed in v2)
# above_threshold = the score cleared the head's own suggest cut (or the creates_new_tag: bool
# system floor). The Suggestions PANEL shows only these; the typed-tag # raw_name = the booru model vocab key behind this suggestion. It's the key
# dropdown fetches ALL suggestions (every head, min=0) and just annotates each # an alias MUST be stored under (resolution looks up the raw key), so the
# matching row with its score, so a low-confidence concept can still be typed # modal needs it to author an alias correctly. None for centroid-only hits
# and picked. CCIP character matches are always above their match threshold. # (no underlying prediction → nothing to alias).
above_threshold: bool raw_name: str | None = None
# via_alias = this suggestion was surfaced because an operator alias remapped
# the raw prediction to this canonical tag. Lets the UI mark it + offer undo.
via_alias: bool = False
# rejected = the operator dismissed this tag for this image (a stored # rejected = the operator dismissed this tag for this image (a stored
# TagSuggestionRejection). It stays in the list — flagged, not dropped — so # TagSuggestionRejection). It stays in the list — flagged, not dropped — so
# the rejection is VISIBLE and REVERSIBLE in the rail (misclick recovery, # the rejection is VISIBLE and REVERSIBLE in the rail (misclick recovery,
@@ -106,7 +108,6 @@ class SuggestionService:
for h in hits: for h in hits:
merged[(h["category"], h["tag_id"])] = { merged[(h["category"], h["tag_id"])] = {
"name": h["name"], "score": h["score"], "source": "head", "name": h["name"], "score": h["score"], "source": "head",
"above_threshold": h["above_threshold"],
"grounding": h.get("grounding"), "grounding": h.get("grounding"),
} }
for c in ccip_hits: for c in ccip_hits:
@@ -115,16 +116,12 @@ class SuggestionService:
if ex is not None: if ex is not None:
ex["source"] = "both" ex["source"] = "both"
ex["score"] = max(ex["score"], c["score"]) ex["score"] = max(ex["score"], c["score"])
# CCIP only returns matches above its own threshold, so a CCIP
# corroboration always makes the merged suggestion above-threshold.
ex["above_threshold"] = True
# Keep the head's localized crop if it had one; else fall back to # Keep the head's localized crop if it had one; else fall back to
# the CCIP figure so a corroborated character still grounds (#1206). # the CCIP figure so a corroborated character still grounds (#1206).
ex["grounding"] = ex.get("grounding") or c.get("grounding") ex["grounding"] = ex.get("grounding") or c.get("grounding")
else: else:
merged[key] = { merged[key] = {
"name": c["name"], "score": c["score"], "source": "ccip", "name": c["name"], "score": c["score"], "source": "ccip",
"above_threshold": True,
"grounding": c.get("grounding"), "grounding": c.get("grounding"),
} }
@@ -139,7 +136,7 @@ class SuggestionService:
category=cat, category=cat,
score=m["score"], score=m["score"],
source=m["source"], source=m["source"],
above_threshold=m["above_threshold"], creates_new_tag=False,
rejected=tag_id in rejected, rejected=tag_id in rejected,
grounding=m.get("grounding"), grounding=m.get("grounding"),
) )
@@ -160,7 +157,8 @@ class SuggestionService:
was suggested for (or already applied to) >= threshold fraction of was suggested for (or already applied to) >= threshold fraction of
the selection AND was acceptable on >= 1 image. Confidence is the the selection AND was acceptable on >= 1 image. Confidence is the
mean over images where it was suggested. Aggregated by mean over images where it was suggested. Aggregated by
canonical_tag_id (every suggestion is a canonical tag now).""" canonical_tag_id; creates-new (no canonical id) suggestions are
skipped (bulk Accept applies by tag id)."""
if not image_ids: if not image_ids:
return {} return {}
threshold = min(1.0, max(0.0, threshold)) threshold = min(1.0, max(0.0, threshold))
@@ -171,6 +169,8 @@ class SuggestionService:
sl = await self.for_image(image_id) sl = await self.for_image(image_id)
for category, items in sl.by_category.items(): for category, items in sl.by_category.items():
for s in items: for s in items:
if s.canonical_tag_id is None or s.creates_new_tag:
continue
# for_image keeps rejected tags (flagged) for the rail; # for_image keeps rejected tags (flagged) for the rail;
# bulk consensus must still ignore them — a tag dismissed on # bulk consensus must still ignore them — a tag dismissed on
# an image isn't a suggestion for that image. # an image isn't a suggestion for that image.
+1 -27
View File
@@ -29,15 +29,7 @@ from ...models.tag import image_tag
# a CCIP reference) unless the operator confirms them (milestone 139). Keeping # a CCIP reference) unless the operator confirms them (milestone 139). Keeping
# auto-applied predictions out of training is what makes them "soft" — a misfire # auto-applied predictions out of training is what makes them "soft" — a misfire
# can't reinforce itself, so the retraction sweep can actually drop it. # can't reinforce itself, so the retraction sweep can actually drop it.
# `process_auto` (#1464): wip/editor screenshot applied by the process sweep are _AUTO_SOURCES = ("head_auto", "ccip_auto", "ml_auto", "presentation_auto")
# ALSO provisional — the head must learn only from title (`wip_title`) + manual
# labels, never its own auto-applied output, or it would runaway (operator 2026-07-12).
# `wip_title_soft` (#1474): the soft title tier (sketch/doodle) is LOW-precision, so
# it's provisional too — a finished piece titled "sketch" must not train the wip head.
_AUTO_SOURCES = (
"head_auto", "ccip_auto", "ml_auto", "presentation_auto", "process_auto",
"wip_title_soft",
)
def _hygiene_excluded_ids(session: Session) -> set[int]: def _hygiene_excluded_ids(session: Session) -> set[int]:
@@ -94,24 +86,6 @@ def _rejected_ids(session: Session, tag_id: int) -> list[int]:
] ]
def _applied_or_rejected(session: Session, tag_ids) -> dict[int, set[int]]:
"""Per-tag skip set for the auto-apply sweeps: every image that ALREADY carries
the tag (ANY source — not just training positives) OR has rejected it. A sweep
never re-applies to these. Shared by auto_apply_sweep + system_tag_auto_apply_sweep
(heads.py) and scheduled_ccip_auto_apply (tasks/ml.py). Callers mutate the returned
sets in-place to also dedupe within a single run."""
skip: dict[int, set[int]] = {}
for tid in tag_ids:
ids = {
r[0] for r in session.execute(
select(image_tag.c.image_record_id).where(image_tag.c.tag_id == tid)
).all()
}
ids.update(_rejected_ids(session, tid))
skip[tid] = ids
return skip
def _sample_unlabeled(session: Session, exclude: set[int], limit: int) -> list[int]: def _sample_unlabeled(session: Session, exclude: set[int], limit: int) -> list[int]:
"""Random image ids (with an embedding) NOT carrying the tag. Concepts are """Random image ids (with an embedding) NOT carrying the tag. Concepts are
sparse, so an untagged image is almost always a true negative.""" sparse, so an untagged image is almost always a true negative."""
+36 -20
View File
@@ -91,46 +91,48 @@ def _sync_lookup(vanity: str, cookies_path: str | None) -> str | None:
) )
def _campaigns_api_first(vanity: str, cookies_path: str | None) -> dict | None: def _lookup_via_api(vanity: str, cookies_path: str | None) -> str | None:
"""The first `data` object from Patreon's campaigns API filtered by vanity
(`?filter[vanity]=<vanity>&fields[campaign]=name`), or None on any failure
(network / non-200 / non-JSON / empty). The single request shape shared by
_lookup_via_api (plucks the campaign id) and resolve_display_name (plucks the
display name)."""
jar = _load_cookie_jar(cookies_path) jar = _load_cookie_jar(cookies_path)
headers = {
"User-Agent": _USER_AGENT,
"Accept": "application/vnd.api+json",
}
params = {
"filter[vanity]": vanity,
"fields[campaign]": "name",
}
try: try:
resp = requests.get( resp = requests.get(
_CAMPAIGNS_URL, _CAMPAIGNS_URL,
params={"filter[vanity]": vanity, "fields[campaign]": "name"}, params=params,
headers={"User-Agent": _USER_AGENT, "Accept": "application/vnd.api+json"}, headers=headers,
cookies=jar, cookies=jar,
timeout=_TIMEOUT_SECONDS, timeout=_TIMEOUT_SECONDS,
) )
except requests.RequestException as exc: except requests.RequestException as exc:
log.warning("Patreon campaigns API request failed for vanity=%s: %s", vanity, exc) log.warning("Patreon campaigns API request failed for vanity=%s: %s", vanity, exc)
return None return None
if resp.status_code != 200: if resp.status_code != 200:
log.warning( log.warning(
"Patreon campaigns API returned HTTP %d for vanity=%s", "Patreon campaigns API returned HTTP %d for vanity=%s",
resp.status_code, vanity, resp.status_code, vanity,
) )
return None return None
try: try:
payload = resp.json() payload = resp.json()
except ValueError as exc: except ValueError as exc:
log.warning("Patreon campaigns API returned non-JSON for vanity=%s: %s", vanity, exc) log.warning("Patreon campaigns API returned non-JSON for vanity=%s: %s", vanity, exc)
return None return None
data = payload.get("data") if isinstance(payload, dict) else None
if not isinstance(data, list) or not data or not isinstance(data[0], dict):
return None
return data[0]
if not isinstance(payload, dict):
def _lookup_via_api(vanity: str, cookies_path: str | None) -> str | None:
first = _campaigns_api_first(vanity, cookies_path)
if first is None:
return None return None
campaign_id = first.get("id") data = payload.get("data")
if not isinstance(data, list) or not data:
return None
first = data[0] if isinstance(data[0], dict) else None
campaign_id = first.get("id") if first else None
if not isinstance(campaign_id, str) or not campaign_id: if not isinstance(campaign_id, str) or not campaign_id:
return None return None
log.info("Resolved Patreon vanity=%s → campaign_id=%s", vanity, campaign_id) log.info("Resolved Patreon vanity=%s → campaign_id=%s", vanity, campaign_id)
@@ -142,10 +144,24 @@ def resolve_display_name(vanity: str, cookies_path: str | None) -> str | None:
(`fields[campaign]=name`), used to name the Artist at add-time (#130). None (`fields[campaign]=name`), used to name the Artist at add-time (#130). None
on any failure — the caller falls back to the vanity handle. Sync: call from on any failure — the caller falls back to the vanity handle. Sync: call from
an executor.""" an executor."""
first = _campaigns_api_first(vanity, cookies_path) jar = _load_cookie_jar(cookies_path)
if first is None: try:
resp = requests.get(
_CAMPAIGNS_URL,
params={"filter[vanity]": vanity, "fields[campaign]": "name"},
headers={"User-Agent": _USER_AGENT, "Accept": "application/vnd.api+json"},
cookies=jar,
timeout=_TIMEOUT_SECONDS,
)
if resp.status_code != 200:
return None
data = resp.json().get("data")
except (requests.RequestException, ValueError) as exc:
log.warning("Patreon name lookup failed for vanity=%s: %s", vanity, exc)
return None return None
name = (first.get("attributes") or {}).get("name") if not isinstance(data, list) or not data or not isinstance(data[0], dict):
return None
name = (data[0].get("attributes") or {}).get("name")
return name.strip() if isinstance(name, str) and name.strip() else None return name.strip() if isinstance(name, str) and name.strip() else None
+4 -3
View File
@@ -8,10 +8,9 @@ PLATFORMS below. Sidecar parsing, cookie materialization, and
Lifted from GallerySubscriber's Lifted from GallerySubscriber's
~/Nextcloud/Projects/GallerySubscriber/backend/app/api/platforms.py ~/Nextcloud/Projects/GallerySubscriber/backend/app/api/platforms.py
and ~/.../extension/lib/platforms.js. Five platforms; auth_type and and ~/.../extension/lib/platforms.js. Six platforms; auth_type and
URL patterns match GS exactly so the existing browser extension URL patterns match GS exactly so the existing browser extension
hits FC unmodified. deviantart was dropped at #3069 (2026-08-27) — hits FC unmodified.
FC downloaders are art-dedicated services only.
""" """
from .base import ( from .base import (
@@ -19,6 +18,7 @@ from .base import (
DEFAULT_EXTERNAL_POST_ID_KEYS, DEFAULT_EXTERNAL_POST_ID_KEYS,
PlatformInfo, PlatformInfo,
) )
from .deviantart import INFO as _DEVIANTART
from .discord import INFO as _DISCORD from .discord import INFO as _DISCORD
from .hentaifoundry import INFO as _HENTAIFOUNDRY from .hentaifoundry import INFO as _HENTAIFOUNDRY
from .patreon import INFO as _PATREON from .patreon import INFO as _PATREON
@@ -33,6 +33,7 @@ PLATFORMS: dict[str, PlatformInfo] = {
_HENTAIFOUNDRY, _HENTAIFOUNDRY,
_DISCORD, _DISCORD,
_PIXIV, _PIXIV,
_DEVIANTART,
) )
} }
+1 -1
View File
@@ -63,7 +63,7 @@ class PlatformInfo:
# Synthesize a post permalink from sidecar data. Required when # Synthesize a post permalink from sidecar data. Required when
# gallery-dl's `url` field is the file/CDN URL rather than the post # gallery-dl's `url` field is the file/CDN URL rather than the post
# permalink (subscribestar/pixiv/hf/discord). None = trust the bare # permalink (subscribestar/pixiv/hf/discord). None = trust the bare
# `url` field (patreon). # `url` field (patreon, deviantart).
derive_post_url: Callable[[dict], str | None] | None = None derive_post_url: Callable[[dict], str | None] | None = None
# Post-process the materialized cookies.txt for gallery-dl. Used by # Post-process the materialized cookies.txt for gallery-dl. Used by
@@ -0,0 +1,23 @@
"""DeviantArt — no exercised quirks yet.
No operator-owned DeviantArt archive existed at the 2026-05-27 sidecar
audit, so we don't know yet whether DA's gallery-dl sidecars are
well-behaved or have their own quirks. When DA gets exercised for the
first time, add `derive_post_url` / `augment_cookies` here as needed.
"""
from .base import GD_DEFAULTS, PlatformInfo
INFO = PlatformInfo(
key="deviantart",
name="DeviantArt",
description="Download artwork from DeviantArt artists",
auth_type="cookies",
requires_auth=False,
url_pattern=r"^https?://(www\.)?deviantart\.com/",
url_examples=[
"https://www.deviantart.com/example-artist",
"https://www.deviantart.com/example-artist/gallery",
],
default_config={**GD_DEFAULTS, "content_types": ["gallery"]},
)
+1 -4
View File
@@ -24,7 +24,6 @@ from ..models import (
Post, Post,
PostAttachment, PostAttachment,
Source, Source,
attachment_download_url,
) )
from ..utils.html_sanitize import ( from ..utils.html_sanitize import (
extract_img_srcs, extract_img_srcs,
@@ -361,7 +360,7 @@ class PostFeedService:
"ext": att.ext, "ext": att.ext,
"mime": att.mime, "mime": att.mime,
"size_bytes": att.size_bytes, "size_bytes": att.size_bytes,
"download_url": attachment_download_url(att.id), "download_url": f"/api/attachments/{att.id}/download",
}) })
return out return out
@@ -398,8 +397,6 @@ class PostFeedService:
"post_title_translated": post.post_title_translated, "post_title_translated": post.post_title_translated,
"description_translated": desc_trans_short, "description_translated": desc_trans_short,
"translated_source_lang": post.translated_source_lang, "translated_source_lang": post.translated_source_lang,
# Sticky per-post translation choice (auto/force/original, #155).
"translation_override": post.translation_override,
"artist": {"id": artist.id, "name": artist.name, "slug": artist.slug}, "artist": {"id": artist.id, "name": artist.name, "slug": artist.slug},
"source": ( "source": (
{"id": source.id, "platform": source.platform} {"id": source.id, "platform": source.platform}
+1 -3
View File
@@ -16,7 +16,6 @@ from ..models import (
Post, Post,
PostAttachment, PostAttachment,
Source, Source,
attachment_download_url,
) )
from ..utils.html_sanitize import sanitize_post_html from ..utils.html_sanitize import sanitize_post_html
@@ -36,7 +35,6 @@ def _post_dict(p: Post) -> dict:
"title_translated": p.post_title_translated, "title_translated": p.post_title_translated,
"description_translated": p.description_translated, "description_translated": p.description_translated,
"translated_source_lang": p.translated_source_lang, "translated_source_lang": p.translated_source_lang,
"translation_override": p.translation_override,
} }
@@ -54,7 +52,7 @@ def _attachment_dict(a: PostAttachment) -> dict:
"original_filename": a.original_filename, "original_filename": a.original_filename,
"size_bytes": a.size_bytes, "size_bytes": a.size_bytes,
"ext": a.ext, "ext": a.ext,
"download_url": attachment_download_url(a.id), "download_url": f"/api/attachments/{a.id}/download",
} }
-121
View File
@@ -1,121 +0,0 @@
"""Title-based WIP auto-tagging (task #1458).
Deterministic heuristic: when a post's TITLE explicitly declares work-in-progress
(the artist's own "WIP" / "work in progress" label), the ``wip`` system tag is
applied to that post's images — a cheap, high-precision complement to the
image-based ML ``wip`` head. WIP images are excluded from the Explore/gallery
browse (see gallery_service ``excluded_system_tags``), so honouring the artist's
own label keeps unfinished pieces out of the main browse right at import.
Precision over recall — a false WIP tag HIDES a finished post — so matching is
token-anchored: ``swipe`` / ``wiped`` / ``wiping`` never trip it (a letter on the
boundary blocks the match).
Sync-only: both consumers (the importer and the backfill Celery task) run on a
sync Session. Application is idempotent-additive (ON CONFLICT DO NOTHING) and
stamps a distinct ``image_tag.source`` so a later pass can tell where a wip tag
came from — the "manual" / "head_auto" / "ccip_auto" / "ml_accepted" provenance
family gains one member.
"""
import re
from sqlalchemy import select
from sqlalchemy.orm import Session
from ..models.tag import WIP_SYSTEM_TAG, Tag, image_tag
from .image_tag_apply import insert_image_tags
# image_tag.source stamped on title-heuristic WIP tags — distinct from the other
# apply sources so provenance stays legible and a future undo can target only these.
# HARD tier ("WIP"/"work in progress") is high-precision → trains the wip head.
WIP_TITLE_SOURCE = "wip_title"
# SOFT tier (sketch/doodle/scribble, #1474) is LOWER-precision — a finished "sketch"
# is often not WIP. This source is PROVISIONAL (in training_data._AUTO_SOURCES) so it
# NEVER trains the wip head; a soft-tagged image that also looks like real content is
# surfaced by the ring-loud audit for review.
WIP_TITLE_SOFT_SOURCE = "wip_title_soft"
# A standalone "WIP" / "W.I.P" token, or the phrase "work in progress"
# (space/underscore/hyphen separated). The letter-boundary lookarounds are what
# make this precision-first: `s|wip|e`, `|wip|ed`, `|wip|ing` all have a letter
# abutting the token, so they're rejected. A trailing digit is allowed so
# "WIP2" (= WIP part 2) still matches.
_WIP_RE = re.compile(
r"(?<![A-Za-z])(?:w\.?i\.?p\.?|work[\s_-]+in[\s_-]+progress)(?![A-Za-z])",
re.IGNORECASE,
)
# Soft tier: sketch / doodle / scribble (+ plurals), letter-boundary anchored so
# "sketchbook" / "kadoodle" don't trip it. Deliberately conservative — recall is
# secondary because the soft source doesn't train the head and the ring-loud audit
# catches false positives.
_SOFT_WIP_RE = re.compile(
r"(?<![A-Za-z])(?:sketch|sketches|doodle|doodles|scribble|scribbles)(?![A-Za-z])",
re.IGNORECASE,
)
# Coarse SQL prefilters for the backfill sweep — narrow the post scan to rows that
# COULD match before the precise regex confirms. Case-insensitive ILIKE patterns.
# Each MUST stay a SUPERSET of its regex or the sweep would silently miss posts.
WIP_TITLE_SQL_PREFILTER = ("%wip%", "%work%progress%")
SOFT_WIP_TITLE_SQL_PREFILTER = ("%sketch%", "%doodle%", "%scribble%")
# Chunk bulk inserts so a large sweep can't blow past psycopg's 65535-parameter
# ceiling (3 params/row → ~21k rows max; 5k stays comfortably under).
_INSERT_CHUNK = 5000
def matches_wip_title(title: str | None) -> bool:
"""True when a post title explicitly marks it work-in-progress (HARD tier)."""
if not title:
return False
return _WIP_RE.search(title) is not None
def matches_soft_wip_title(title: str | None) -> bool:
"""True when a title carries a SOFT WIP cue (sketch/doodle/scribble, #1474)."""
if not title:
return False
return _SOFT_WIP_RE.search(title) is not None
def resolve_wip_tag_id(session: Session) -> int | None:
"""The seeded ``wip`` system tag's id (migration 0075), or None if absent."""
return session.execute(
select(Tag.id).where(Tag.name == WIP_SYSTEM_TAG, Tag.is_system.is_(True))
).scalar_one_or_none()
def apply_wip_image_tags(
session: Session, image_ids, tag_id: int, *, source: str = WIP_TITLE_SOURCE
) -> int:
"""Attach ``tag_id`` (stamped with ``source``) to each image id, idempotently —
never disturbs an existing tag or its source. Returns the number of image_tag
rows newly inserted. Does NOT commit.
The insert count is computed from a pre-SELECT of already-tagged ids rather
than the statement's ``rowcount``: psycopg reports -1 for a multi-row
ON CONFLICT DO NOTHING insert (it runs via an executemany path), so rowcount
is unusable here. The SELECT is accurate within this single transaction (no
concurrent writer touches these (image, wip) rows); ON CONFLICT DO NOTHING
stays as a race-safety belt so a rare concurrent insert can't error."""
ids = list({int(i) for i in image_ids})
if not ids:
return 0
inserted = 0
for start in range(0, len(ids), _INSERT_CHUNK):
chunk = ids[start:start + _INSERT_CHUNK]
already = set(session.execute(
select(image_tag.c.image_record_id)
.where(image_tag.c.tag_id == tag_id)
.where(image_tag.c.image_record_id.in_(chunk))
).scalars())
to_insert = [iid for iid in chunk if iid not in already]
if not to_insert:
continue
insert_image_tags(session, [
{"image_record_id": iid, "tag_id": tag_id, "source": source}
for iid in to_insert
])
inserted += len(to_insert)
return inserted
-28
View File
@@ -409,31 +409,3 @@ def rescan_series_suggestions_task(self, after_post_id: int = 0) -> dict:
) )
rescan_series_suggestions_task.delay(summary["resume_after_id"]) rescan_series_suggestions_task.delay(summary["resume_after_id"])
return summary return summary
@celery.task(
name="backend.app.tasks.admin.reclaim_orphaned_attachments_task",
bind=True,
autoretry_for=(OperationalError, DBAPIError),
retry_backoff=15, retry_backoff_max=180, max_retries=1,
# The service stops walking at its own 900s budget and reports partial, so
# these limits are the backstop for a wedged filesystem (NFS stall), not the
# expected exit. Comfortably above the budget so a normal run always returns
# its summary rather than being killed mid-walk.
soft_time_limit=1200, time_limit=1500, # 20 min / 25 min
)
def reclaim_orphaned_attachments_task(self, dry_run: bool = True) -> dict:
"""Reclaim unattributed PostAttachment rows and the store blobs nothing
references any more (#3068). dry_run (the default) returns the projection
without touching rows or files; apply deletes the orphan rows, then unlinks
every blob no surviving row references.
Defaults to the SAFE preview — unlike the other tasks here, whose apply is
reversible-ish or scoped; this one deletes files. Operator-triggered only,
never on a beat: an unattended sweep that unlinks blobs is not something to
run without someone reading the projection first."""
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
return cleanup_service.reclaim_orphaned_attachments(
session, images_root=IMAGES_ROOT, dry_run=dry_run,
)
+72 -143
View File
@@ -76,8 +76,6 @@ DOWNLOAD_STALL_THRESHOLD_MINUTES = 30
OLD_TASK_DAYS = 7 OLD_TASK_DAYS = 7
PHASH_PAGE = 500 PHASH_PAGE = 500
VERIFY_PAGE = 200 VERIFY_PAGE = 200
# Title-based WIP backfill (task #1458): posts scanned per keyset page.
WIP_BACKFILL_PAGE = 500
FFPROBE_TIMEOUT_SECONDS = 10 FFPROBE_TIMEOUT_SECONDS = 10
TASK_RUN_KEEP_OK_SECONDS = 24 * 3600 # 24 h TASK_RUN_KEEP_OK_SECONDS = 24 * 3600 # 24 h
TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600 # 7 days TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600 # 7 days
@@ -173,12 +171,6 @@ TASK_STUCK_THRESHOLD_MINUTES: dict[str, int] = {
# task-name override beats the queue threshold whatever queue the row records # task-name override beats the queue threshold whatever queue the row records
# (it recorded 'default' before the celery_signals fix → download). 65 = 60+5. # (it recorded 'default' before the celery_signals fix → download). 65 = 60+5.
"backend.app.tasks.external.fetch_external_link": 65, "backend.app.tasks.external.fetch_external_link": 65,
# Attachment reclaim walks the whole sha-addressed store; the service caps
# itself at a 900s budget and reports partial, but the task's hard limit is
# 25 min for a wedged filesystem (NFS stall). Same phantom-flag class as the
# external-fetch entry above — without an override a healthy in-flight walk
# is swept 'RecoverySweep' at the bare 5-min default. 30 = 25 + 5.
"backend.app.tasks.admin.reclaim_orphaned_attachments_task": 30,
} }
@@ -782,62 +774,89 @@ def recover_stalled_library_audit_runs() -> int:
return recovered return recovered
def _recover_stalled_runs(model, *, stall_minutes: int, keep_runs: int, label: str) -> int:
"""Shared recovery + retention sweep for the head run-tracking tables
(HeadTrainingRun / HeadAutoApplyRun, which share the
status/last_progress_at/started_at/finished_at/error/id columns): flip 'running'
rows with no progress past `stall_minutes` to 'error', then prune to the last
`keep_runs` (rule 89). Returns the number recovered. NOTE the two other recover
tasks are deliberately NOT folded in — library-audit has no prune tail and
backup uses a single started_at cutoff."""
SessionLocal = _sync_session_factory()
now = datetime.now(UTC)
cutoff = now - timedelta(minutes=stall_minutes)
with SessionLocal() as session:
result = session.execute(
update(model)
.where(model.status == "running")
.where(func.coalesce(model.last_progress_at, model.started_at) < cutoff)
.values(
status="error", finished_at=now,
error=f"stranded by recovery sweep (no progress for {stall_minutes} min)",
)
)
keep = session.execute(
select(model.id).order_by(model.id.desc()).limit(keep_runs)
).scalars().all()
if keep:
session.execute(delete(model).where(model.id.not_in(keep)))
session.commit()
recovered = result.rowcount or 0
if recovered:
log.info("%s: recovered %d rows", label, recovered)
return recovered
@celery.task(name="backend.app.tasks.maintenance.recover_stalled_head_training_runs") @celery.task(name="backend.app.tasks.maintenance.recover_stalled_head_training_runs")
def recover_stalled_head_training_runs() -> int: def recover_stalled_head_training_runs() -> int:
"""Flip HeadTrainingRun rows stuck in 'running' past the stall threshold to """Flip HeadTrainingRun rows stuck in 'running' past the stall threshold to
'error', and prune old runs to the last HEAD_TRAINING_KEEP_RUNS (retention, 'error', and prune old runs to the last HEAD_TRAINING_KEEP_RUNS (retention,
rule 89). Runs every 5 min on the maintenance lane; no-op when idle.""" rule 89). Runs every 5 min on the maintenance lane; no-op when idle."""
return _recover_stalled_runs( SessionLocal = _sync_session_factory()
HeadTrainingRun, now = datetime.now(UTC)
stall_minutes=HEAD_TRAINING_STALL_THRESHOLD_MINUTES, cutoff = now - timedelta(minutes=HEAD_TRAINING_STALL_THRESHOLD_MINUTES)
keep_runs=HEAD_TRAINING_KEEP_RUNS, with SessionLocal() as session:
label="recover_stalled_head_training_runs", result = session.execute(
) update(HeadTrainingRun)
.where(HeadTrainingRun.status == "running")
.where(
func.coalesce(
HeadTrainingRun.last_progress_at, HeadTrainingRun.started_at
)
< cutoff
)
.values(
status="error", finished_at=now,
error=(
f"stranded by recovery sweep (no progress for "
f"{HEAD_TRAINING_STALL_THRESHOLD_MINUTES} min)"
),
)
)
keep = session.execute(
select(HeadTrainingRun.id).order_by(HeadTrainingRun.id.desc())
.limit(HEAD_TRAINING_KEEP_RUNS)
).scalars().all()
if keep:
session.execute(
delete(HeadTrainingRun).where(HeadTrainingRun.id.not_in(keep))
)
session.commit()
recovered = result.rowcount or 0
if recovered:
log.info(
"recover_stalled_head_training_runs: recovered %d rows", recovered
)
return recovered
@celery.task(name="backend.app.tasks.maintenance.recover_stalled_head_auto_apply_runs") @celery.task(name="backend.app.tasks.maintenance.recover_stalled_head_auto_apply_runs")
def recover_stalled_head_auto_apply_runs() -> int: def recover_stalled_head_auto_apply_runs() -> int:
"""Flip stalled HeadAutoApplyRun 'running' rows to 'error' + prune to the """Flip stalled HeadAutoApplyRun 'running' rows to 'error' + prune to the
last HEAD_AUTO_APPLY_KEEP_RUNS (retention, rule 89). 5-min maintenance lane.""" last HEAD_AUTO_APPLY_KEEP_RUNS (retention, rule 89). 5-min maintenance lane."""
return _recover_stalled_runs( SessionLocal = _sync_session_factory()
HeadAutoApplyRun, now = datetime.now(UTC)
stall_minutes=HEAD_AUTO_APPLY_STALL_THRESHOLD_MINUTES, cutoff = now - timedelta(minutes=HEAD_AUTO_APPLY_STALL_THRESHOLD_MINUTES)
keep_runs=HEAD_AUTO_APPLY_KEEP_RUNS, with SessionLocal() as session:
label="recover_stalled_head_auto_apply_runs", result = session.execute(
) update(HeadAutoApplyRun)
.where(HeadAutoApplyRun.status == "running")
.where(
func.coalesce(
HeadAutoApplyRun.last_progress_at, HeadAutoApplyRun.started_at
)
< cutoff
)
.values(
status="error", finished_at=now,
error=(
f"stranded by recovery sweep (no progress for "
f"{HEAD_AUTO_APPLY_STALL_THRESHOLD_MINUTES} min)"
),
)
)
keep = session.execute(
select(HeadAutoApplyRun.id).order_by(HeadAutoApplyRun.id.desc())
.limit(HEAD_AUTO_APPLY_KEEP_RUNS)
).scalars().all()
if keep:
session.execute(
delete(HeadAutoApplyRun).where(HeadAutoApplyRun.id.not_in(keep))
)
session.commit()
recovered = result.rowcount or 0
if recovered:
log.info(
"recover_stalled_head_auto_apply_runs: recovered %d rows", recovered
)
return recovered
# Keep ~6 months of daily head-metric snapshots (enough to see tuning trends). # Keep ~6 months of daily head-metric snapshots (enough to see tuning trends).
@@ -1026,96 +1045,6 @@ def cleanup_old_download_events() -> int:
return result.rowcount or 0 return result.rowcount or 0
def _backfill_wip_tier(session, tag_id, prefilter, matcher, source) -> int:
"""One keyset-paginated pass over posts whose title matches a WIP tier, applying
`tag_id` (stamped `source`) to their images. Shared by the hard + soft tiers
(#1458 / #1474). Coarse `prefilter` (ILIKE superset) narrows the scan; the precise
`matcher` confirms. Idempotent-additive (ON CONFLICT DO NOTHING). Returns the row
count newly applied."""
from ..models import Post
from ..models.image_provenance import ImageProvenance
from ..services.wip_title import apply_wip_image_tags
applied = 0
last_id = 0
while True:
rows = session.execute(
select(Post.id, Post.post_title)
.where(Post.id > last_id)
.where(Post.post_title.is_not(None))
.where(or_(*[Post.post_title.ilike(p) for p in prefilter]))
.order_by(Post.id.asc())
.limit(WIP_BACKFILL_PAGE)
).all()
if not rows:
break
last_id = rows[-1][0]
match_ids = [pid for pid, title in rows if matcher(title)]
if match_ids:
image_ids = session.execute(
select(ImageProvenance.image_record_id)
.where(ImageProvenance.post_id.in_(match_ids))
).scalars().all()
applied += apply_wip_image_tags(session, image_ids, tag_id, source=source)
session.commit()
return applied
@celery.task(
name="backend.app.tasks.maintenance.backfill_wip_title_tags",
# Coarse-prefiltered scan over posts; the candidate set is small on a typical
# library, but bound it like the other full-library sweeps.
soft_time_limit=1800, time_limit=2100,
)
def backfill_wip_title_tags() -> int:
"""Scan EXISTING posts for WIP titles and apply the `wip` system tag to their
images — the operator-triggered back-catalogue catch-up (task #1458 hard tier +
#1474 soft tier). New imports are tagged live by the importer; this covers the
existing library.
HARD tier ("WIP"/"work in progress") always runs (the operator triggered the
scan); the SOFT tier (sketch/doodle, provisional source) runs only when
wip_soft_title_tagging_enabled, AFTER hard so a title matching both keeps the
trained hard tag (ON CONFLICT DO NOTHING). Keyset-paginated, restart-safe.
Deliberately NOT scheduled as a beat: a periodic re-run would re-apply to matching
posts and silently undo a manual WIP removal, so it stays an explicit operator
action (Settings → "Scan existing posts for WIP titles"). Returns rows applied.
"""
from ..models import ImportSettings
from ..services.wip_title import (
SOFT_WIP_TITLE_SQL_PREFILTER,
WIP_TITLE_SOFT_SOURCE,
WIP_TITLE_SOURCE,
WIP_TITLE_SQL_PREFILTER,
matches_soft_wip_title,
matches_wip_title,
resolve_wip_tag_id,
)
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
tag_id = resolve_wip_tag_id(session)
if tag_id is None:
log.warning(
"backfill_wip_title_tags: no `wip` system tag present; nothing to do"
)
return 0
settings = ImportSettings.load_sync(session)
applied = _backfill_wip_tier(
session, tag_id, WIP_TITLE_SQL_PREFILTER, matches_wip_title,
WIP_TITLE_SOURCE,
)
if settings.wip_soft_title_tagging_enabled:
applied += _backfill_wip_tier(
session, tag_id, SOFT_WIP_TITLE_SQL_PREFILTER, matches_soft_wip_title,
WIP_TITLE_SOFT_SOURCE,
)
if applied:
log.info("backfill_wip_title_tags: applied wip to %d image(s)", applied)
return applied
@celery.task(name="backend.app.tasks.maintenance.vacuum_analyze") @celery.task(name="backend.app.tasks.maintenance.vacuum_analyze")
def vacuum_analyze() -> dict: def vacuum_analyze() -> dict:
"""Periodic VACUUM (ANALYZE) over the high-churn tables (VACUUM_TABLES) to """Periodic VACUUM (ANALYZE) over the high-churn tables (VACUUM_TABLES) to
+36 -52
View File
@@ -105,7 +105,9 @@ def embed_image(self, image_id: int) -> dict:
record = session.get(ImageRecord, image_id) record = session.get(ImageRecord, image_id)
if record is None: if record is None:
return {"status": "missing", "image_id": image_id} return {"status": "missing", "image_id": image_id}
settings = MLSettings.load_sync(session) settings = session.execute(
select(MLSettings).where(MLSettings.id == 1)
).scalar_one()
src = Path(record.path) src = Path(record.path)
is_vid = _is_video(src) is_vid = _is_video(src)
@@ -486,10 +488,15 @@ def scheduled_ccip_auto_apply() -> str:
from sqlalchemy import select as sa_select from sqlalchemy import select as sa_select
from sqlalchemy.dialects.postgresql import insert as pg_insert from sqlalchemy.dialects.postgresql import insert as pg_insert
from ..models import ImageRegion, MLSettings, Tag, TagKind from ..models import ImageRegion, MLSettings, Tag, TagKind, TagSuggestionRejection
from ..models.tag import image_tag from ..models.tag import image_tag
from ..services.ml.ccip import _FIGURE_KINDS
from ..services.ml.training_data import _applied_or_rejected, _l2norm fig = ("face", "figure")
def _l2(m):
n = np.linalg.norm(m, axis=1, keepdims=True)
n[n == 0] = 1.0
return m / n
SessionLocal = _sync_session_factory() SessionLocal = _sync_session_factory()
with SessionLocal() as session: with SessionLocal() as session:
@@ -514,7 +521,7 @@ def scheduled_ccip_auto_apply() -> str:
) )
.join(Tag, Tag.id == image_tag.c.tag_id) .join(Tag, Tag.id == image_tag.c.tag_id)
.where(Tag.kind == TagKind.character) .where(Tag.kind == TagKind.character)
.where(ImageRegion.kind.in_(_FIGURE_KINDS)) .where(ImageRegion.kind.in_(fig))
.where(ImageRegion.ccip_embedding.is_not(None)) .where(ImageRegion.ccip_embedding.is_not(None))
.where(ImageRegion.image_record_id.in_(single)) .where(ImageRegion.image_record_id.in_(single))
).all() ).all()
@@ -525,16 +532,29 @@ def scheduled_ccip_auto_apply() -> str:
for tid, vec in ref_rows: for tid, vec in ref_rows:
by_char.setdefault(tid, []).append(vec) by_char.setdefault(tid, []).append(vec)
ref_tags = list(by_char) ref_tags = list(by_char)
mats = [_l2norm(np.asarray(by_char[t], dtype=np.float32), np) for t in ref_tags] mats = [_l2(np.asarray(by_char[t], dtype=np.float32)) for t in ref_tags]
allref = np.vstack(mats) # (total, 768) allref = np.vstack(mats) # (total, 768)
seg = np.cumsum([0] + [len(m) for m in mats])[:-1] # per-char start seg = np.cumsum([0] + [len(m) for m in mats])[:-1] # per-char start
# Per character: images that already carry OR rejected the tag — skip. # Per character: images that already carry OR rejected the tag — skip.
skip = _applied_or_rejected(session, ref_tags) skip = {t: set() for t in ref_tags}
for t in ref_tags:
for (iid,) in session.execute(
sa_select(image_tag.c.image_record_id).where(
image_tag.c.tag_id == t
)
):
skip[t].add(iid)
for (iid,) in session.execute(
sa_select(TagSuggestionRejection.image_record_id).where(
TagSuggestionRejection.tag_id == t
)
):
skip[t].add(iid)
img_ids = list(session.execute( img_ids = list(session.execute(
sa_select(ImageRegion.image_record_id) sa_select(ImageRegion.image_record_id)
.where(ImageRegion.kind.in_(_FIGURE_KINDS), ImageRegion.ccip_embedding.is_not(None)) .where(ImageRegion.kind.in_(fig), ImageRegion.ccip_embedding.is_not(None))
.distinct() .distinct()
).scalars()) ).scalars())
@@ -546,7 +566,7 @@ def scheduled_ccip_auto_apply() -> str:
sa_select(ImageRegion.image_record_id, ImageRegion.ccip_embedding) sa_select(ImageRegion.image_record_id, ImageRegion.ccip_embedding)
.where( .where(
ImageRegion.image_record_id.in_(chunk), ImageRegion.image_record_id.in_(chunk),
ImageRegion.kind.in_(_FIGURE_KINDS), ImageRegion.kind.in_(fig),
ImageRegion.ccip_embedding.is_not(None), ImageRegion.ccip_embedding.is_not(None),
) )
).all() ).all()
@@ -554,7 +574,7 @@ def scheduled_ccip_auto_apply() -> str:
for iid, vec in rows: for iid, vec in rows:
by_img.setdefault(iid, []).append(vec) by_img.setdefault(iid, []).append(vec)
for iid, vecs in by_img.items(): for iid, vecs in by_img.items():
q = _l2norm(np.asarray(vecs, dtype=np.float32), np) # (nq, 768) q = _l2(np.asarray(vecs, dtype=np.float32)) # (nq, 768)
colmax = (q @ allref.T).max(axis=0) # (total,) colmax = (q @ allref.T).max(axis=0) # (total,)
charmax = np.maximum.reduceat(colmax, seg) # (n_chars,) charmax = np.maximum.reduceat(colmax, seg) # (n_chars,)
for ci in np.where(charmax >= thr)[0]: for ci in np.where(charmax >= thr)[0]:
@@ -579,54 +599,18 @@ def scheduled_ccip_auto_apply() -> str:
soft_time_limit=1800, time_limit=2100, soft_time_limit=1800, time_limit=2100,
) )
def scheduled_presentation_auto_apply() -> str: def scheduled_presentation_auto_apply() -> str:
"""Auto-hide presentation chrome (banner) on a daily passive sweep (#141). """Auto-hide presentation chrome (banner / editor screenshot) on a daily
No-op unless presentation_auto_apply_enabled. Idempotent — already-tagged images passive sweep (#141). No-op unless presentation_auto_apply_enabled. Idempotent
are skipped — so an interrupted run simply re-runs next cycle (that IS the — already-hidden images are skipped — so an interrupted run simply re-runs next
recovery). Wall-clock bounded by the task time limits.""" cycle (that IS the recovery). Wall-clock bounded by the task time limits."""
from ..services.ml.heads import system_tag_auto_apply_sweep from ..services.ml.heads import presentation_auto_apply_sweep
SessionLocal = _sync_session_factory() SessionLocal = _sync_session_factory()
with SessionLocal() as session: with SessionLocal() as session:
result = system_tag_auto_apply_sweep(session, mode="chrome") result = presentation_auto_apply_sweep(session)
return f"applied={result['n_applied']} flagged={result['n_flagged']}" return f"applied={result['n_applied']} flagged={result['n_flagged']}"
@celery.task(
name="backend.app.tasks.ml.scheduled_process_auto_apply",
soft_time_limit=1800, time_limit=2100,
)
def scheduled_process_auto_apply() -> str:
"""Auto-apply the PROCESS system tags (wip / editor screenshot) on a daily
passive sweep (#1464) — provisional source, ring-loud review guard, image stays
VISIBLE. No-op unless process_auto_apply_enabled (opt-in). Idempotent —
already-tagged/rejected images are skipped — so an interrupted run just re-runs
next cycle (the recovery). Wall-clock bounded by the task time limits."""
from ..services.ml.heads import system_tag_auto_apply_sweep
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
result = system_tag_auto_apply_sweep(session, mode="process")
return f"applied={result['n_applied']} flagged={result['n_flagged']}"
@celery.task(
name="backend.app.tasks.ml.scheduled_soft_wip_conflict_audit",
soft_time_limit=1800, time_limit=2100,
)
def scheduled_soft_wip_conflict_audit() -> str:
"""Ring-loud audit over the SOFT WIP-title cohort (#1474) — flag sketch/doodle
auto-tags that ALSO look like real content for review. No-op when there are no
content heads; idempotent (already-flagged images skipped). Runs regardless of
the process-sweep toggle, since soft-title tags come from the importer, not that
sweep. Wall-clock bounded by the task time limits."""
from ..services.ml.heads import soft_wip_conflict_audit
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
result = soft_wip_conflict_audit(session)
return f"scanned={result['n_scanned']} flagged={result['n_flagged']}"
@celery.task(name="backend.app.tasks.ml.prune_presentation_reviews") @celery.task(name="backend.app.tasks.ml.prune_presentation_reviews")
def prune_presentation_reviews() -> str: def prune_presentation_reviews() -> str:
"""Retention (rule 89): drop RESOLVED presentation-review flags older than 30 """Retention (rule 89): drop RESOLVED presentation-review flags older than 30
+36 -79
View File
@@ -60,17 +60,14 @@ _INTERRUPT_BACKOFF_MAX = 900
# language scores low; require it to clear this floor, else keep the original. # language scores low; require it to clear this floor, else keep the original.
# Calibrated 2026-07-09 against fresh (uncached) probes once Interpreter returned # Calibrated 2026-07-09 against fresh (uncached) probes once Interpreter returned
# real langdetect confidence: genuine German detected at 1.0, while a correctly- # real langdetect confidence: genuine German detected at 1.0, while a correctly-
# detected but ambiguous latin string dipped to 0.86 — so a single constant can't win. # detected but ambiguous latin string dipped to 0.86 — so the floor sits at 0.80,
# The floor is now operator-tunable (ImportSettings.translation_min_confidence, # below that band, to keep legitimate ambiguous non-English while still rejecting
# milestone 155) with a stricter 0.90 default; per-post overrides # genuinely-unsure guesses. Real langdetect also fixed the original mis-flag at
# (Post.translation_override) rescue the false negatives it skips. Genuine German # the source (short English titles now detect as English → passthrough), so this
# and mis-flagged short English both land ~0.86, and single-word mis-flags sit at
# a confident 1.0 no floor catches. Real langdetect fixed some cases at the
# source (some short English titles now detect as English → passthrough), so this
# floor is a safety net, not the primary fix. Re-tune via the "Test translation" # floor is a safety net, not the primary fix. Re-tune via the "Test translation"
# box (send fresh text — cache hits report 1.0). # box (send fresh text — cache hits report 1.0).
_CJK_LANGS = frozenset({"ja", "ko", "zh"}) _CJK_LANGS = frozenset({"ja", "ko", "zh"})
_DEFAULT_MIN_LATIN_CONFIDENCE = 0.90 _MIN_LATIN_CONFIDENCE = 0.80
def _interrupt_backoff(retry_after) -> int: def _interrupt_backoff(retry_after) -> int:
@@ -103,10 +100,10 @@ def translate_posts(drain: bool = False) -> str:
ready = _translation_config(session) ready = _translation_config(session)
if isinstance(ready, str): if isinstance(ready, str):
return ready return ready
base_url, target, min_confidence = ready base_url, target = ready
posts = _select_untranslated(session, None, _MAX_POSTS_PER_RUN) posts = _select_untranslated(session, None, _MAX_POSTS_PER_RUN)
status, translated, retry_after = _translate_batch( status, translated, retry_after = _translate_batch(
session, posts, base_url, target, min_confidence, session, posts, base_url, target,
) )
# Drain mode (manual "Translate now"): chase the tail until the backlog is # Drain mode (manual "Translate now"): chase the tail until the backlog is
# zero, so one press clears the whole pile instead of one 300-chunk. # zero, so one press clears the whole pile instead of one 300-chunk.
@@ -155,7 +152,7 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
ready = _translation_config(session) ready = _translation_config(session)
if isinstance(ready, str): if isinstance(ready, str):
return ready return ready
base_url, target, min_confidence = ready base_url, target = ready
if not _reset_done: if not _reset_done:
n = _reset_translations(session, artist_ids) n = _reset_translations(session, artist_ids)
@@ -166,7 +163,7 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
posts = _select_untranslated(session, artist_ids, _MAX_POSTS_PER_RUN) posts = _select_untranslated(session, artist_ids, _MAX_POSTS_PER_RUN)
status, translated, retry_after = _translate_batch( status, translated, retry_after = _translate_batch(
session, posts, base_url, target, min_confidence, session, posts, base_url, target,
) )
# run-until-done: chase the tail when the chunk finished cleanly. # run-until-done: chase the tail when the chunk finished cleanly.
@@ -204,19 +201,17 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
def _translation_config(session): def _translation_config(session):
"""Resolve (base_url, target, min_confidence) if translation is enabled + """Resolve (base_url, target) if translation is enabled + healthy, else a
healthy, else a short status string ("disabled" / "interpreter unavailable") short status string ("disabled" / "interpreter unavailable") for the caller
for the caller to return. Centralises the guard so translate/retranslate agree to return. Centralises the guard so translate/retranslate agree exactly."""
exactly, including the operator-tunable acceptance floor."""
cfg = ImportSettings.load_sync(session) cfg = ImportSettings.load_sync(session)
if not cfg.translation_enabled or not cfg.interpreter_base_url.strip(): if not cfg.translation_enabled or not cfg.interpreter_base_url.strip():
return "disabled" return "disabled"
base_url = cfg.interpreter_base_url.strip() base_url = cfg.interpreter_base_url.strip()
target = (cfg.translation_target_lang or "en").strip() or "en" target = (cfg.translation_target_lang or "en").strip() or "en"
min_confidence = cfg.translation_min_confidence
if not ic.health(base_url): if not ic.health(base_url):
return "interpreter unavailable" return "interpreter unavailable"
return base_url, target, min_confidence return base_url, target
def _untranslated_filter(stmt, artist_ids): def _untranslated_filter(stmt, artist_ids):
@@ -249,13 +244,10 @@ def _reset_translations(session, artist_ids) -> int:
"""Clear the stored translation columns for scoped, already-handled posts so """Clear the stored translation columns for scoped, already-handled posts so
the untranslated sweep re-runs them. Only touches rows that were translated the untranslated sweep re-runs them. Only touches rows that were translated
(translated_source_lang IS NOT NULL) — untranslated rows are already NULL. (translated_source_lang IS NOT NULL) — untranslated rows are already NULL.
Skips 'keep original' posts (translation_override = 'original') so that choice Returns the row count reset (commits it)."""
survives a Re-translate-all (milestone 155). Returns the row count reset
(commits it)."""
stmt = ( stmt = (
update(Post) update(Post)
.where(Post.translated_source_lang.is_not(None)) .where(Post.translated_source_lang.is_not(None))
.where(Post.translation_override != "original")
.values( .values(
post_title_translated=None, post_title_translated=None,
description_translated=None, description_translated=None,
@@ -271,7 +263,7 @@ def _reset_translations(session, artist_ids) -> int:
return n return n
def _translate_batch(session, posts, base_url: str, target: str, min_confidence: float): def _translate_batch(session, posts, base_url: str, target: str):
"""Translate each post in the chunk with a per-post commit (an interrupted """Translate each post in the chunk with a per-post commit (an interrupted
run keeps progress). Returns (status, translated, retry_after) where status is run keeps progress). Returns (status, translated, retry_after) where status is
one of "ok" / "interrupted" (unavailable/drain) / "stopped" (400) / "timeout"; one of "ok" / "interrupted" (unavailable/drain) / "stopped" (400) / "timeout";
@@ -280,7 +272,7 @@ def _translate_batch(session, posts, base_url: str, target: str, min_confidence:
translated = 0 translated = 0
for post in posts: for post in posts:
try: try:
translated += _translate_one(session, post, base_url, target, min_confidence) translated += _translate_one(session, post, base_url, target)
session.commit() session.commit()
except ic.InterpreterUnavailable as e: except ic.InterpreterUnavailable as e:
session.rollback() session.rollback()
@@ -304,36 +296,28 @@ def _summary(status: str, translated: int, scanned: int) -> str:
return f"translated={translated} scanned={scanned}" return f"translated={translated} scanned={scanned}"
def _accept( def _accept(language: str, confidence) -> bool:
language: str, confidence, min_confidence: float = _DEFAULT_MIN_LATIN_CONFIDENCE,
) -> bool:
"""Should curator store Interpreter's translation, or keep the original? """Should curator store Interpreter's translation, or keep the original?
Consumes ONLY Interpreter's own detection (curator does none, Scribe rule Consumes ONLY Interpreter's own detection (curator does none, Scribe rule
133): CJK languages are trusted regardless of the reported number; a 133): CJK languages are trusted regardless of the reported number; a
latin-script detection must clear ``min_confidence`` (the operator-tunable latin-script detection must clear ``_MIN_LATIN_CONFIDENCE``. A missing
ImportSettings.translation_min_confidence). A missing confidence fails OPEN confidence fails OPEN (accept), so a service that omits the field never causes
(accept), so a service that omits the field never silently drops a translations to be silently dropped."""
translation. Per-post ``force`` overrides bypass this entirely (see
``_translate_one``)."""
if (language or "").split("-", 1)[0].lower() in _CJK_LANGS: if (language or "").split("-", 1)[0].lower() in _CJK_LANGS:
return True return True
if confidence is None: if confidence is None:
return True return True
return confidence >= min_confidence return confidence >= _MIN_LATIN_CONFIDENCE
def _translate_field( def _translate_field(text: str, base_url: str, target: str):
text: str, base_url: str, target: str, min_confidence: float, force: bool = False,
):
"""Translate ONE field independently. Returns (translated, source_lang, """Translate ONE field independently. Returns (translated, source_lang,
engine_version), or (None, None, None) when there's nothing to store — empty engine_version), or (None, None, None) when there's nothing to store — empty
text, already the target language, a passthrough (engine "none"), or a text, already the target language, a passthrough (engine "none"), or a
latin-script detection the acceptance gate rejects as too low-confidence (kept latin-script detection the acceptance gate rejects as too low-confidence (kept
as the original). ``force`` (a per-post 'force' override) bypasses the as the original). Per-field (not aggregate first-item) detection so a
confidence gate so a legitimately-foreign title Interpreter is only ~0.86 sure non-English description still gets translated when the title is already
about is still stored; a genuine passthrough stores nothing regardless (there English (mixed-language posts)."""
is no translation to force). Per-field (not aggregate first-item) detection so
a non-English description still translates when the title is already English."""
if not text: if not text:
return None, None, None return None, None, None
res = ic.translate([text], base_url=base_url, target=target) res = ic.translate([text], base_url=base_url, target=target)
@@ -343,30 +327,23 @@ def _translate_field(
# Trust only Interpreter's own detection (Scribe rule 133): keep the original # Trust only Interpreter's own detection (Scribe rule 133): keep the original
# when a latin-script detection doesn't clear the confidence floor, so a short # when a latin-script detection doesn't clear the confidence floor, so a short
# English title mis-labelled as e.g. German isn't rewritten into the archive. # English title mis-labelled as e.g. German isn't rewritten into the archive.
# A 'force' override skips the gate — the operator is overriding a rejection. if not _accept(detected, res.get("detected_confidence")):
if not force and not _accept(
detected, res.get("detected_confidence"), min_confidence,
):
return None, None, None return None, None, None
return res["translations"][0], detected, res["engine_version"] return res["translations"][0], detected, res["engine_version"]
def _store_translation(post, title_res, desc_res, target: str) -> int: def _translate_one(session, post, base_url: str, target: str) -> int:
"""Apply per-field (translated, source_lang, engine_version) results to a post """Translate a post's title/description in place, each field independently.
in place. Returns 1 if a translation was stored, 0 when the post is all Returns 1 if it stored at least one translation, 0 when the post is all
passthrough/empty — in which case the translated columns are CLEARED and the passthrough/empty (still marks it handled so the sweep won't revisit it)."""
post is marked handled (translated_source_lang = target). Clearing (not just title = (post.post_title or "").strip()
leaving) matters on re-apply: a 'keep original' override, or a re-translate desc = (html_to_plain(post.description) if post.description else "") or ""
that now rejects a previously-accepted mis-flag, must remove the stale desc = desc.strip()
translation, not keep it.""" title_tr, title_lang, title_ev = _translate_field(title, base_url, target)
title_tr, title_lang, title_ev = title_res desc_tr, desc_lang, desc_ev = _translate_field(desc, base_url, target)
desc_tr, desc_lang, desc_ev = desc_res
if title_tr is None and desc_tr is None: if title_tr is None and desc_tr is None:
post.post_title_translated = None # Nothing non-target (or nothing to translate) → handled, store nothing.
post.description_translated = None
post.translated_source_lang = target post.translated_source_lang = target
post.translation_engine_version = None
post.translated_at = None
return 0 return 0
post.post_title_translated = title_tr post.post_title_translated = title_tr
post.description_translated = desc_tr post.description_translated = desc_tr
@@ -376,23 +353,3 @@ def _store_translation(post, title_res, desc_res, target: str) -> int:
post.translation_engine_version = title_ev or desc_ev post.translation_engine_version = title_ev or desc_ev
post.translated_at = datetime.now(UTC) post.translated_at = datetime.now(UTC)
return 1 return 1
def _translate_one(
session, post, base_url: str, target: str, min_confidence: float,
) -> int:
"""Translate a post's title/description in place, each field independently,
honoring the per-post override (milestone 155): 'original' keeps the original
(clears any stored translation, no Interpreter call); 'force' translates even
below the confidence floor; 'auto' (default) runs the acceptance gate. Returns
1 if it stored a translation, else 0 (still marks the post handled so the sweep
won't revisit it)."""
if post.translation_override == "original":
return _store_translation(post, (None, None, None), (None, None, None), target)
force = post.translation_override == "force"
title = (post.post_title or "").strip()
desc = (html_to_plain(post.description) if post.description else "") or ""
desc = desc.strip()
title_res = _translate_field(title, base_url, target, min_confidence, force=force)
desc_res = _translate_field(desc, base_url, target, min_confidence, force=force)
return _store_translation(post, title_res, desc_res, target)
+3 -39
View File
@@ -11,26 +11,12 @@ git.fabledsword.com/bvandeusen/ci-python:3.14
- python 3.14 - python 3.14
- ruff (analyzer for `backend/`, `tests/`, `alembic/`) - ruff (analyzer for `backend/`, `tests/`, `alembic/`)
- node (frontend job: `npm install` + vitest + vite build) - node (frontend job: `npm install` + vitest + vite build)
- docker CLI + buildx (`.forgejo/workflows/build.yml`: build-web, build-ml — Fabled-Git registry push) - docker CLI + buildx (`.forgejo/workflows/build.yml`: build-web, build-ml — Forgejo registry push)
## Secondary runtime image
node:24-bookworm-slim — `.forgejo/workflows/extension.yml` only.
The extension lane is the one job that does NOT run on `ci-python:3.14`: it
needs a current Node for `web-ext` and vitest and nothing Python at all. Kept
on the upstream slim image rather than adding a Node toolchain to `ci-python`,
per `docs/process.md`'s "add deps to the image when used by >1 project".
## Per-job tool installs ## Per-job tool installs
- `pip install -r requirements.txt pytest pytest-asyncio` — in `backend-lint-and-test` and `integration` jobs - `pip install -r requirements.txt pytest pytest-asyncio` — in `backend-lint-and-test` and `integration` jobs
- `npm install --no-audit --no-fund` — in `frontend-build` job - `npm install --no-audit --no-fund` — in `frontend-build` job
- `npm install --no-audit --no-fund` — in `extension.yml`'s `lint` job (web-ext + vitest)
- `unzip` — in `extension.yml`'s "Verify XPI contents" step, installed via apt
only when absent (`node:24-bookworm-slim` may or may not carry it). Debian
package, ~2s. Not worth baking into a shared image for a single consumer, per
`docs/process.md`'s ">1 project" rule.
## Notes ## Notes
@@ -40,34 +26,12 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
"add deps to image when used by >1 project" rule: FC alone is one Python "add deps to image when used by >1 project" rule: FC alone is one Python
project, so the deps live in `requirements.txt` and install per-job. project, so the deps live in `requirements.txt` and install per-job.
Reconsider when a second Fabled-family Python backend lands. Reconsider when a second Fabled-family Python backend lands.
- Integration uses Fabled-Git Actions `services:` + socket-discovered bridge IPs - Integration uses Forgejo Actions `services:` + socket-discovered bridge IPs
because `act_runner` (swarm-runner v0.6+) puts services on the default because `act_runner` (swarm-runner v0.6+) puts services on the default
bridge with no embedded DNS. The pattern is documented in the rulebook's bridge with no embedded DNS. The pattern is documented in the rulebook's
`fabled-git.md` "CI philosophy" section and FC's `ci.yml` is the canonical `forgejo.md` "CI philosophy" section and FC's `ci.yml` is the canonical
example. example.
- No `package-lock.json` is tracked yet (FC's `feedback_no_local_runs` - No `package-lock.json` is tracked yet (FC's `feedback_no_local_runs`
memory bans `npm install` locally). Using `npm install` rather than memory bans `npm install` locally). Using `npm install` rather than
`npm ci` until a lockfile lands. `npm ci` until a lockfile lands.
- No `imagemagick` / `pandoc` per-job installs needed. - No `imagemagick` / `pandoc` per-job installs needed.
- `extension/`'s vitest specs load `lib/*.js` by evaluating the real file as a
classic script (`test/helpers/loadLib.js`) rather than adding `module.exports`
shims to production code — the libs ship as `background.scripts`, not ES
modules, so the specs exercise exactly the bytes packaged into the XPI.
- **`extension/scripts/packaging.sh` is the single definition of what ships
inside the XPI.** Three consumers read from it rather than keeping their own
copy: web-ext's `--ignore-files` (`extension/package.json`), the `:(exclude)`
pathspec in `ci.yml`'s `extension-version` guard, and the `git log` pathspec
that derives the extension version. Three hand-kept copies of that one fact
is what allowed issue #2397.
- Jobs that derive the extension version check out with `fetch-depth: 0`. The
version is the commit TIME of the newest packaged-extension change (minutes
since 2020-01-01, per family rule 149 — never a commit count, which orders
by branch rather than by recency). A depth-1 clone sees one commit and
derives a wrong, too-low value rather than failing, so the full-history
checkout is load-bearing wherever `packaging.sh version` is called.
- Callers MUST `set -f` before substituting the script's output. Without it the
shell expands `test/**` against the working tree and silently narrows the
pattern to whatever files exist at that moment — a failure that looks like
nothing until dev files start appearing in the XPI. `test/version.spec.js`
asserts every `--ignore-files` consumer sets it, and that no consumer has
quietly reinstated a hardcoded list.
+3 -3
View File
@@ -1,9 +1,9 @@
# FabledCurator Firefox Extension # FabledCurator Firefox Extension
Self-hosted Firefox extension that pushes session cookies from supported Self-hosted Firefox extension that pushes session cookies from supported
platforms (Patreon, SubscribeStar, Hentai-Foundry, Discord, Pixiv) platforms (Patreon, SubscribeStar, Hentai-Foundry, Discord, Pixiv,
into FabledCurator, and lets you add a creator as a Source from their DeviantArt) into FabledCurator, and lets you add a creator as a Source
page in one click. from their page in one click.
## Install (operator) ## Install (operator)
+30 -92
View File
@@ -31,68 +31,6 @@ browser.runtime.onInstalled.addListener(() => ensureInitialized());
browser.runtime.onStartup.addListener(() => ensureInitialized()); browser.runtime.onStartup.addListener(() => ensureInitialized());
ensureInitialized().catch(e => console.error('init failed:', e)); ensureInitialized().catch(e => console.error('init failed:', e));
// ---- Extension self-update check (#1489) ----
// Installed per-instance from the operator's FC host, so Firefox's static
// update_url can't apply (each instance has a different host). Instead ask the
// configured backend for the latest published version and nudge the operator to
// reinstall the freshly-signed XPI — surfaced as a popup banner (on demand) and
// a toolbar badge (daily). /api/extension/manifest is public and returns
// {version, latest_url, sha256}; the XPI is served from the web root (not /api).
function versionIsNewer(candidate, current) {
// Dotted numeric compare so 1.0.10 > 1.0.9 (a plain string compare wouldn't).
const a = String(candidate).split('.').map(n => parseInt(n, 10) || 0);
const b = String(current).split('.').map(n => parseInt(n, 10) || 0);
for (let i = 0; i < Math.max(a.length, b.length); i++) {
if ((a[i] || 0) !== (b[i] || 0)) return (a[i] || 0) > (b[i] || 0);
}
return false;
}
async function checkForUpdateInfo() {
await ensureInitialized();
if (!api.isConfigured()) return { updateAvailable: false, configured: false };
let info;
try {
info = await api.getExtensionManifest();
} catch (e) {
return { updateAvailable: false, error: e.message };
}
const currentVersion = browser.runtime.getManifest().version;
const latestVersion = info && info.version ? info.version : null;
// latest_url is served from the web root, not the JSON API.
const base = api.webRoot();
return {
updateAvailable: !!latestVersion && versionIsNewer(latestVersion, currentVersion),
currentVersion,
latestVersion,
xpiUrl: info && info.latest_url ? `${base}${info.latest_url}` : null,
};
}
async function refreshUpdateBadge() {
let r;
try { r = await checkForUpdateInfo(); } catch { return; }
try {
await browser.action.setBadgeText({ text: r.updateAvailable ? '↑' : '' });
if (r.updateAvailable) {
await browser.action.setBadgeBackgroundColor({ color: '#F4BA7A' });
await browser.action.setTitle({ title: `FabledCurator — update available (v${r.latestVersion})` });
} else {
await browser.action.setTitle({ title: 'FabledCurator' });
}
} catch { /* action API unavailable — non-fatal */ }
}
// Daily proactive check (needs the "alarms" permission). create() is idempotent
// by name, so re-running it on each event-page load is safe.
browser.alarms.create('fc-update-check', { periodInMinutes: 24 * 60, delayInMinutes: 1 });
browser.alarms.onAlarm.addListener((alarm) => {
if (alarm.name === 'fc-update-check') refreshUpdateBadge();
});
browser.runtime.onStartup.addListener(() => refreshUpdateBadge());
browser.runtime.onInstalled.addListener(() => refreshUpdateBadge());
// ---- Discord token capture via webRequest ---- // ---- Discord token capture via webRequest ----
browser.webRequest.onBeforeSendHeaders.addListener( browser.webRequest.onBeforeSendHeaders.addListener(
@@ -210,21 +148,6 @@ browser.webRequest.onBeforeRedirect.addListener(
{ urls: ['https://app-api.pixiv.net/web/v1/users/auth/pixiv/callback*'] }, { urls: ['https://app-api.pixiv.net/web/v1/users/auth/pixiv/callback*'] },
); );
// Extract → verify → upload one cookie-auth platform. Returns a structured
// outcome so the two callers (EXPORT_COOKIES single, EXPORT_ALL_COOKIES) shape
// their own response + skip semantics. Verifies the captured cookies are
// actually live BEFORE uploading, so a confirmed-stale session doesn't overwrite
// good FC-side credentials; platforms with no verify config (v.ok === null) fall
// through to upload.
async function exportPlatformCookies(key) {
const cookies = await extractCookiesForPlatform(key);
if (cookies.length === 0) return { status: 'empty' };
const v = await verifyCookiesForPlatform(key);
if (v.ok === false) return { status: 'stale', reason: v.reason, cookieCount: cookies.length };
await api.uploadCredentials(key, 'cookies', toNetscapeFormat(cookies));
return { status: 'ok', cookieCount: cookies.length, verified: v.ok === true };
}
// ---- Message router ---- // ---- Message router ----
browser.runtime.onMessage.addListener(async (msg) => { browser.runtime.onMessage.addListener(async (msg) => {
@@ -269,14 +192,22 @@ browser.runtime.onMessage.addListener(async (msg) => {
if (!platform) return { error: `Unknown platform: ${key}` }; if (!platform) return { error: `Unknown platform: ${key}` };
try { try {
if (platform.authType === 'cookies') { if (platform.authType === 'cookies') {
const r = await exportPlatformCookies(key); const cookies = await extractCookiesForPlatform(key);
if (r.status === 'empty') return { error: 'No cookies found — log in first.' }; if (cookies.length === 0) return { error: 'No cookies found — log in first.' };
if (r.status === 'stale') { // Verify the captured cookies are actually live BEFORE
// uploading. Skips upload on confirmed-stale sessions so we
// don't overwrite FC-side credentials with garbage. Platforms
// without a verify config (verify.ok === null) fall through
// to upload as before.
const v = await verifyCookiesForPlatform(key);
if (v.ok === false) {
return { return {
error: `Captured ${r.cookieCount} ${platform.name} cookies but they don't appear authenticated (${r.reason}). Log in again in this browser, then retry.`, error: `Captured ${cookies.length} ${platform.name} cookies but they don't appear authenticated (${v.reason}). Log in again in this browser, then retry.`,
}; };
} }
return { success: true, cookieCount: r.cookieCount, verified: r.verified }; const data = toNetscapeFormat(cookies);
await api.uploadCredentials(key, 'cookies', data);
return { success: true, cookieCount: cookies.length, verified: v.ok === true };
} }
if (key === 'discord') { if (key === 'discord') {
if (!discordToken) return { error: 'Open discord.com to capture a token first.' }; if (!discordToken) return { error: 'Open discord.com to capture a token first.' };
@@ -304,10 +235,18 @@ browser.runtime.onMessage.addListener(async (msg) => {
continue; continue;
} }
try { try {
const r = await exportPlatformCookies(key); const cookies = await extractCookiesForPlatform(key);
if (r.status === 'empty') results[key] = { skipped: true, reason: 'no cookies' }; if (cookies.length === 0) {
else if (r.status === 'stale') results[key] = { error: `verify failed: ${r.reason}` }; results[key] = { skipped: true, reason: 'no cookies' };
else results[key] = { success: true, cookieCount: r.cookieCount, verified: r.verified }; continue;
}
const v = await verifyCookiesForPlatform(key);
if (v.ok === false) {
results[key] = { error: `verify failed: ${v.reason}` };
continue;
}
await api.uploadCredentials(key, 'cookies', toNetscapeFormat(cookies));
results[key] = { success: true, cookieCount: cookies.length, verified: v.ok === true };
} catch (e) { } catch (e) {
results[key] = { error: e.message }; results[key] = { error: e.message };
} }
@@ -344,9 +283,11 @@ browser.runtime.onMessage.addListener(async (msg) => {
} }
case 'OPEN_ARTIST_PAGE': { case 'OPEN_ARTIST_PAGE': {
// The SPA artist route (/artist/:slug) is served from the web root, not // apiUrl is configured with the /api suffix (see
// the JSON API — see api.webRoot(). // options/options.html placeholder); the SPA artist route is
const base = api.webRoot(); // /artist/:slug, served from the same origin. Strip /api so the
// browser-level URL hits the Vue router, not the JSON API.
const base = (api.baseUrl || '').replace(/\/+$/, '').replace(/\/api$/, '');
const slug = encodeURIComponent(msg.slug || ''); const slug = encodeURIComponent(msg.slug || '');
if (!base || !slug) return { error: 'apiUrl or slug missing' }; if (!base || !slug) return { error: 'apiUrl or slug missing' };
try { try {
@@ -357,9 +298,6 @@ browser.runtime.onMessage.addListener(async (msg) => {
} }
} }
case 'CHECK_UPDATE':
return await checkForUpdateInfo();
default: default:
return { error: `Unknown message type: ${msg.type}` }; return { error: `Unknown message type: ${msg.type}` };
} }
+1 -23
View File
@@ -11,10 +11,7 @@ class FabledCuratorAPI {
async init() { async init() {
const cfg = await browser.storage.local.get(['apiUrl', 'apiKey']); const cfg = await browser.storage.local.get(['apiUrl', 'apiKey']);
// Normalize on READ, not just on save: configs stored before the options this.baseUrl = cfg.apiUrl || null;
// page started normalizing are missing the `/api` suffix, and this heals
// them without the operator having to reopen Settings.
this.baseUrl = normalizeApiUrl(cfg.apiUrl) || null;
this.apiKey = cfg.apiKey || null; this.apiKey = cfg.apiKey || null;
return this.isConfigured(); return this.isConfigured();
} }
@@ -53,13 +50,6 @@ class FabledCuratorAPI {
} catch { } catch {
message = `HTTP ${response.status}: ${response.statusText}`; message = `HTTP ${response.status}: ${response.statusText}`;
} }
// 404/405 from FC almost always means the request never reached the JSON
// API — it fell through to the SPA catch-all, which serves HTML on GET
// and rejects everything else. Say so, rather than making the operator
// decode "Method Not Allowed" on an endpoint that plainly allows POST.
if (response.status === 404 || response.status === 405) {
message += `${url} isn't the FC API. Check the FC URL in settings.`;
}
const err = new Error(message); const err = new Error(message);
err.status = response.status; err.status = response.status;
throw err; throw err;
@@ -99,18 +89,6 @@ class FabledCuratorAPI {
const qs = new URLSearchParams({ url }).toString(); const qs = new URLSearchParams({ url }).toString();
return this.request('GET', `/extension/probe?${qs}`); return this.request('GET', `/extension/probe?${qs}`);
} }
// Latest published extension version on this instance — drives the in-app
// update prompt. Public endpoint (no key needed, but request() sends it
// harmlessly). Returns {version, xpi_url, latest_url, sha256}.
getExtensionManifest() {
return this.request('GET', '/extension/manifest');
}
// The web/SPA root: where the Vue router (artist pages) and the served XPI
// live, NOT the JSON API. Used by OPEN_ARTIST_PAGE + the self-update check.
webRoot() {
return webRootFromApiUrl(this.baseUrl);
}
// Connection test = the cheapest read with auth. // Connection test = the cheapest read with auth.
testConnection() { testConnection() {
+12 -10
View File
@@ -68,6 +68,16 @@ const PLATFORMS = {
urlPattern: /^https?:\/\/(www\.)?pixiv\.net/, urlPattern: /^https?:\/\/(www\.)?pixiv\.net/,
note: 'Click to authenticate via OAuth', note: 'Click to authenticate via OAuth',
}, },
deviantart: {
name: 'DeviantArt',
domains: ['.deviantart.com', 'www.deviantart.com', 'deviantart.com'],
authType: 'cookies',
color: '#05CC47',
urlPattern: /^https?:\/\/(www\.)?deviantart\.com/,
// DA's logged-in-only endpoints sit behind their internal _napi
// namespace which shifts; skipping verify until a stable check
// surfaces. Same posture as SubscribeStar.
},
}; };
/** /**
@@ -76,18 +86,10 @@ const PLATFORMS = {
* script to decide whether to show the floating "Add as source" button. * script to decide whether to show the floating "Add as source" button.
*/ */
const PLATFORM_ARTIST_PATTERNS = { const PLATFORM_ARTIST_PATTERNS = {
// Patreon serves the same creator under three URL shapes (see backend patreon: /^https?:\/\/(www\.)?patreon\.com\/(?!home$|search\b|messages\b|notifications\b|library\b|settings\b|posts\b|c\/)[^/?#]+\/?$/i,
// patreon_resolver._VANITY_RE): bare `patreon.com/Atole`, `c/` prefix, and
// `cw/` "creator workspace" — the last is the URL you land on once you're
// SUBSCRIBED, which is exactly when the button matters. Match all three, and
// drop the single-segment end-anchor so a creator's inner page
// (…/cw/Atole/posts, …/Atole/membership) also injects the button. Nav pages
// (home/search/…/posts permalink) stay excluded. Mirrors extension_service
// ._PLATFORM_PATTERNS — keep in sync (operator-flagged 2026-07-13: button
// vanished once subscribed because the old pattern only matched the bare root).
patreon: /^https?:\/\/(www\.)?patreon\.com\/(?:cw\/|c\/)?(?!(?:home|search|messages|notifications|library|settings|posts)(?:[\/?#]|$))[^/?#]+/i,
subscribestar: /^https?:\/\/(www\.)?subscribestar\.(com|adult)\/(?!feed$|messages$|library$)[^/?#]+\/?$/i, subscribestar: /^https?:\/\/(www\.)?subscribestar\.(com|adult)\/(?!feed$|messages$|library$)[^/?#]+\/?$/i,
hentaifoundry: /^https?:\/\/(www\.)?hentai-foundry\.com\/user\/[^/?#]+/i, hentaifoundry: /^https?:\/\/(www\.)?hentai-foundry\.com\/user\/[^/?#]+/i,
deviantart: /^https?:\/\/(www\.)?deviantart\.com\/(?!home$|watch\b|tag\b|browse\b)[^/?#]+\/?$/i,
pixiv: /^https?:\/\/(www\.)?pixiv\.net\/(en\/)?users\/\d+/i, pixiv: /^https?:\/\/(www\.)?pixiv\.net\/(en\/)?users\/\d+/i,
}; };
-32
View File
@@ -1,32 +0,0 @@
/**
* Canonical FC endpoint derivation, shared by the background client and the
* options page so a URL entered either way behaves identically.
*
* FC serves two things on one origin: the JSON API under `/api`, and the Vue
* SPA from the root. `api.js` builds requests as `${baseUrl}/credentials`, so
* the stored base URL has to carry the `/api` suffix.
*/
/**
* Accept what an operator would naturally type — the instance root
* (`http://curator.example.com`) or the API root (`.../api`) — and return the
* API root either way.
*
* Worth normalizing rather than validating: a root-form URL doesn't fail
* loudly, it lands on the SPA catch-all, which answers `GET /credentials` with
* 200 HTML and rejects `POST /credentials` with 405. The operator sees a
* working Test Connection and a broken export.
*/
function normalizeApiUrl(raw) {
const trimmed = (raw || '').trim().replace(/\/+$/, '');
if (!trimmed) return '';
return /\/api$/i.test(trimmed) ? trimmed : `${trimmed}/api`;
}
/**
* The SPA root — where the Vue router (artist pages) and the served XPI live,
* NOT the JSON API. Accepts either input form, same as normalizeApiUrl.
*/
function webRootFromApiUrl(raw) {
return normalizeApiUrl(raw).replace(/\/api$/i, '');
}
+5 -4
View File
@@ -1,7 +1,7 @@
{ {
"manifest_version": 3, "manifest_version": 3,
"name": "FabledCurator", "name": "FabledCurator",
"version": "1.0.11", "version": "1.0.7",
"description": "Export cookies from supported platforms to FabledCurator and add creators as sources in one click.", "description": "Export cookies from supported platforms to FabledCurator and add creators as sources in one click.",
"browser_specific_settings": { "browser_specific_settings": {
@@ -22,8 +22,7 @@
"tabs", "tabs",
"activeTab", "activeTab",
"webRequest", "webRequest",
"webRequestBlocking", "webRequestBlocking"
"alarms"
], ],
"host_permissions": [ "host_permissions": [
@@ -33,6 +32,7 @@
"*://*.hentai-foundry.com/*", "*://*.hentai-foundry.com/*",
"*://*.discord.com/*", "*://*.discord.com/*",
"*://*.pixiv.net/*", "*://*.pixiv.net/*",
"*://*.deviantart.com/*",
"*://app-api.pixiv.net/*", "*://app-api.pixiv.net/*",
"*://oauth.secure.pixiv.net/*", "*://oauth.secure.pixiv.net/*",
"*://*/*" "*://*/*"
@@ -45,7 +45,7 @@
}, },
"background": { "background": {
"scripts": ["lib/platforms.js", "lib/cookies.js", "lib/url.js", "lib/api.js", "background/background.js"] "scripts": ["lib/platforms.js", "lib/cookies.js", "lib/api.js", "background/background.js"]
}, },
"options_ui": { "options_ui": {
@@ -60,6 +60,7 @@
"*://*.subscribestar.com/*", "*://*.subscribestar.com/*",
"*://*.subscribestar.adult/*", "*://*.subscribestar.adult/*",
"*://*.hentai-foundry.com/*", "*://*.hentai-foundry.com/*",
"*://*.deviantart.com/*",
"*://*.pixiv.net/*" "*://*.pixiv.net/*"
], ],
"js": ["lib/platforms.js", "content/content-script.js"], "js": ["lib/platforms.js", "content/content-script.js"],
+3 -7
View File
@@ -21,12 +21,9 @@
<body> <body>
<h1>FabledCurator extension</h1> <h1>FabledCurator extension</h1>
<label for="api-url">FC instance URL</label> <label for="api-url">FC base URL</label>
<input id="api-url" type="url" placeholder="http://curator.example.com" /> <input id="api-url" type="url" placeholder="http://curator.example.com/api" />
<div class="hint"> <div class="hint">Find this on FC → Settings → Maintenance → Browser extension.</div>
Your FabledCurator address — with or without the trailing <code>/api</code>; both work.
Find it on FC → Settings → Maintenance → Browser extension.
</div>
<label for="api-key">Extension API key</label> <label for="api-key">Extension API key</label>
<input id="api-key" type="password" placeholder="paste from FC Settings card" /> <input id="api-key" type="password" placeholder="paste from FC Settings card" />
@@ -39,7 +36,6 @@
<div id="status" class="status" style="display:none;"></div> <div id="status" class="status" style="display:none;"></div>
<script src="../lib/url.js"></script>
<script src="options.js"></script> <script src="options.js"></script>
</body> </body>
</html> </html>
+5 -23
View File
@@ -8,7 +8,7 @@ document.addEventListener('DOMContentLoaded', async () => {
}); });
async function save() { async function save() {
const apiUrl = normalizeApiUrl(document.getElementById('api-url').value); const apiUrl = document.getElementById('api-url').value.trim().replace(/\/+$/, '');
const apiKey = document.getElementById('api-key').value.trim(); const apiKey = document.getElementById('api-key').value.trim();
if (!apiUrl || !apiKey) { if (!apiUrl || !apiKey) {
showStatus('Both fields are required.', 'err'); showStatus('Both fields are required.', 'err');
@@ -16,14 +16,11 @@ async function save() {
} }
await browser.storage.local.set({ apiUrl, apiKey }); await browser.storage.local.set({ apiUrl, apiKey });
await browser.storage.local.remove(['lastConnectionTest', 'lastConnectionStatus']); await browser.storage.local.remove(['lastConnectionTest', 'lastConnectionStatus']);
// Show what was actually stored — the operator may have typed the instance showStatus('Saved.', 'ok');
// root and it was normalized to the API root.
document.getElementById('api-url').value = apiUrl;
showStatus(`Saved — using ${apiUrl}`, 'ok');
} }
async function test() { async function test() {
const apiUrl = normalizeApiUrl(document.getElementById('api-url').value); const apiUrl = document.getElementById('api-url').value.trim().replace(/\/+$/, '');
const apiKey = document.getElementById('api-key').value.trim(); const apiKey = document.getElementById('api-key').value.trim();
if (!apiUrl || !apiKey) { if (!apiUrl || !apiKey) {
showStatus('Fill both fields first.', 'err'); showStatus('Fill both fields first.', 'err');
@@ -34,23 +31,8 @@ async function test() {
method: 'GET', method: 'GET',
headers: { 'X-Extension-Key': apiKey }, headers: { 'X-Extension-Key': apiKey },
}); });
if (!r.ok) { if (r.ok) showStatus(`Connected — HTTP ${r.status}.`, 'ok');
showStatus(`HTTP ${r.status}: ${r.statusText}`, 'err'); else showStatus(`HTTP ${r.status}: ${r.statusText}`, 'err');
return;
}
// A 200 is NOT sufficient. If the URL resolves to the Vue SPA instead of
// the JSON API, the catch-all route returns 200 with an HTML document —
// which used to report "Connected" on a config that could not POST at all.
const contentType = r.headers.get('content-type') || '';
if (!contentType.includes('json')) {
showStatus(
`${apiUrl} answered with ${contentType || 'no content-type'}, not JSON `
+ '— that looks like the FC web UI rather than its API.',
'err',
);
return;
}
showStatus(`Connected to ${apiUrl} — HTTP ${r.status}.`, 'ok');
} catch (e) { } catch (e) {
showStatus(`Cannot reach ${apiUrl}: ${e.message}`, 'err'); showStatus(`Cannot reach ${apiUrl}: ${e.message}`, 'err');
} }
+6 -9
View File
@@ -1,18 +1,15 @@
{ {
"name": "fabledcurator-extension", "name": "fabledcurator-extension",
"version": "1.0.11", "version": "1.0.7",
"private": true, "private": true,
"description": "Firefox extension for FabledCurator", "description": "Firefox extension for FabledCurator",
"comment_ignore_files": "The --ignore-files list comes from scripts/packaging.sh, the single source of truth shared with ci.yml's guard and the derived-version patch count. `set -f` is REQUIRED before the substitution: without it the shell globs `test/**` against the working tree and silently narrows the pattern to whatever files happen to exist.",
"scripts": { "scripts": {
"lint": "set -f; web-ext lint --source-dir=. --no-config-discovery --ignore-files $(sh scripts/packaging.sh ignore)", "lint": "web-ext lint --source-dir=. --no-config-discovery --ignore-files package.json package-lock.json web-ext-artifacts node_modules README.md .gitignore",
"start": "set -f; web-ext run --source-dir=. --no-config-discovery --ignore-files $(sh scripts/packaging.sh ignore) --firefox=firefox", "start": "web-ext run --source-dir=. --no-config-discovery --ignore-files package.json package-lock.json web-ext-artifacts node_modules README.md .gitignore --firefox=firefox",
"build": "set -f; web-ext build --source-dir=. --no-config-discovery --ignore-files $(sh scripts/packaging.sh ignore) --overwrite-dest", "build": "web-ext build --source-dir=. --no-config-discovery --ignore-files package.json package-lock.json web-ext-artifacts node_modules README.md .gitignore --overwrite-dest",
"sign": "set -f; web-ext sign --source-dir=. --no-config-discovery --ignore-files $(sh scripts/packaging.sh ignore) --channel=unlisted --api-key=$WEB_EXT_API_KEY --api-secret=$WEB_EXT_API_SECRET", "sign": "web-ext sign --source-dir=. --no-config-discovery --ignore-files package.json package-lock.json web-ext-artifacts node_modules README.md .gitignore --channel=unlisted --api-key=$WEB_EXT_API_KEY --api-secret=$WEB_EXT_API_SECRET"
"test:unit": "vitest run"
}, },
"devDependencies": { "devDependencies": {
"vitest": "^4.0.0", "web-ext": "^8.0.0"
"web-ext": "^10.0.0"
} }
} }
-11
View File
@@ -72,17 +72,6 @@ body {
.btn.block { display: block; width: 100%; margin-top: 8px; } .btn.block { display: block; width: 100%; margin-top: 8px; }
.btn.link { background: none; color: var(--on-surface-variant); padding: 4px; } .btn.link { background: none; color: var(--on-surface-variant); padding: 4px; }
.btn.link:hover { color: var(--accent); } .btn.link:hover { color: var(--accent); }
.btn.small { padding: 6px 12px; font-size: 13px; }
/* In-app update prompt (accent-tinted so it reads as an actionable notice). */
.update-banner {
display: flex; align-items: center; gap: 10px;
margin: 10px 10px 0; padding: 10px 12px;
background: rgba(244, 186, 122, 0.12);
border: 1px solid rgba(244, 186, 122, 0.4);
border-radius: 6px;
}
#update-text { flex: 1; font-size: 13px; }
.source-row .play { .source-row .play {
background: none; border: none; color: var(--on-surface-variant); background: none; border: none; color: var(--on-surface-variant);
-5
View File
@@ -20,11 +20,6 @@
</section> </section>
<section id="main-content" class="main hidden"> <section id="main-content" class="main hidden">
<div id="update-banner" class="update-banner hidden">
<span id="update-text"></span>
<button id="update-btn" class="btn primary small">Update</button>
</div>
<nav class="tabs"> <nav class="tabs">
<button class="tab active" data-tab="platforms">Platforms</button> <button class="tab active" data-tab="platforms">Platforms</button>
<button class="tab" data-tab="sources">Sources</button> <button class="tab" data-tab="sources">Sources</button>
+12 -33
View File
@@ -2,15 +2,6 @@ document.addEventListener('DOMContentLoaded', init);
const CONNECTION_TEST_INTERVAL = 2 * 60 * 1000; const CONNECTION_TEST_INTERVAL = 2 * 60 * 1000;
// A centered muted note div — the loading / empty state shared by the platform
// and sources lists.
function mutedNote(text) {
const d = document.createElement('div');
d.style.cssText = 'text-align:center;padding:18px;color:var(--on-surface-variant);';
d.textContent = text;
return d;
}
async function init() { async function init() {
try { try {
const cfg = await browser.runtime.sendMessage({ type: 'GET_CONFIG' }); const cfg = await browser.runtime.sendMessage({ type: 'GET_CONFIG' });
@@ -23,7 +14,6 @@ async function init() {
setupEventListeners(); setupEventListeners();
showPlatformsLoading(); showPlatformsLoading();
testConnectionIfNeeded(); testConnectionIfNeeded();
checkForUpdate();
loadPlatformStatus().catch(e => showError(`Failed to load platforms: ${e.message}`)); loadPlatformStatus().catch(e => showError(`Failed to load platforms: ${e.message}`));
} catch (e) { } catch (e) {
showSetupRequired(); showSetupRequired();
@@ -47,7 +37,10 @@ function showSetupRequired() {
function showPlatformsLoading() { function showPlatformsLoading() {
const c = document.getElementById('platforms-list'); const c = document.getElementById('platforms-list');
c.textContent = ''; c.textContent = '';
c.appendChild(mutedNote('Loading platforms…')); const d = document.createElement('div');
d.style.cssText = 'text-align:center;padding:18px;color:var(--on-surface-variant);';
d.textContent = 'Loading platforms…';
c.appendChild(d);
} }
async function testConnectionIfNeeded() { async function testConnectionIfNeeded() {
@@ -70,26 +63,6 @@ function updateConnectionDot(connected) {
d.title = connected ? 'Connected to FabledCurator' : 'Disconnected'; d.title = connected ? 'Connected to FabledCurator' : 'Disconnected';
} }
// Nudge to reinstall when the configured instance publishes a newer signed XPI
// (the extension is self-hosted, so there's no Firefox auto-update). Never
// blocks the popup — a failed check just leaves the banner hidden.
async function checkForUpdate() {
try {
const r = await browser.runtime.sendMessage({ type: 'CHECK_UPDATE' });
if (r && r.updateAvailable && r.xpiUrl) showUpdateBanner(r);
} catch { /* non-fatal */ }
}
function showUpdateBanner(r) {
document.getElementById('update-text').textContent =
`Update available — v${r.latestVersion} (installed v${r.currentVersion})`;
// Opening the signed XPI triggers Firefox's native install prompt.
document.getElementById('update-btn').addEventListener('click', () => {
browser.tabs.create({ url: r.xpiUrl });
});
document.getElementById('update-banner').classList.remove('hidden');
}
async function loadPlatformStatus() { async function loadPlatformStatus() {
const status = await browser.runtime.sendMessage({ type: 'GET_PLATFORM_STATUS' }); const status = await browser.runtime.sendMessage({ type: 'GET_PLATFORM_STATUS' });
const c = document.getElementById('platforms-list'); const c = document.getElementById('platforms-list');
@@ -189,7 +162,10 @@ async function exportAllCookies() {
async function loadSources() { async function loadSources() {
const c = document.getElementById('sources-list'); const c = document.getElementById('sources-list');
c.textContent = ''; c.textContent = '';
c.appendChild(mutedNote('Loading sources…')); const d = document.createElement('div');
d.style.cssText = 'text-align:center;padding:18px;color:var(--on-surface-variant);';
d.textContent = 'Loading sources…';
c.appendChild(d);
const r = await browser.runtime.sendMessage({ type: 'LIST_SOURCES' }); const r = await browser.runtime.sendMessage({ type: 'LIST_SOURCES' });
c.textContent = ''; c.textContent = '';
if (r.error) { if (r.error) {
@@ -200,7 +176,10 @@ async function loadSources() {
return; return;
} }
if (!r.sources || r.sources.length === 0) { if (!r.sources || r.sources.length === 0) {
c.appendChild(mutedNote('No sources yet.')); const empty = document.createElement('div');
empty.style.cssText = 'text-align:center;padding:18px;color:var(--on-surface-variant);';
empty.textContent = 'No sources yet.';
c.appendChild(empty);
return; return;
} }
for (const src of r.sources) c.appendChild(createSourceRow(src)); for (const src of r.sources) c.appendChild(createSourceRow(src));
-130
View File
@@ -1,130 +0,0 @@
#!/bin/sh
# Single source of truth for "what ships inside the XPI", plus the version
# derived from it.
#
# Three consumers used to hand-maintain their own copy of this list, and
# keeping three copies of one fact in sync by hand is how issue #2397 happened:
#
# 1. web-ext's --ignore-files (extension/package.json's four scripts)
# 2. the :(exclude) pathspec (ci.yml's extension-version guard)
# 3. the git-log pathspec (the derived version, below)
#
# They now all read from here. POSIX sh only — CI's run shell is busybox.
#
# -f (no pathname expansion) is set for the whole script and is load-bearing:
# the lists below are iterated with deliberate word-splitting, and without -f
# the shell would also GLOB them, expanding `test/**` into whatever files
# happen to exist and corrupting the output. A caller's own `set -f` does not
# help here — this runs as a separate sh process and does not inherit it.
# Callers still need their own `set -f` for the substituted result; the two
# guards protect different expansions.
set -euf
# Paths under extension/ that are NOT packaged into the XPI.
#
# Split by whether git tracks them: node_modules and web-ext-artifacts are
# build/dependency output that never appears in a commit, so they belong in
# web-ext's ignore list but would be meaningless in a git pathspec.
#
# Directories need BOTH forms. `test/**` matches the files inside, but not the
# directory entry itself — web-ext writes an entry for the directory too, so
# with only the glob the XPI ends up carrying empty `test/` and `scripts/`
# entries (caught by the XPI-content check on 2026-08-03). The bare name alone
# is not enough either: minimatch's `test` does not match `test/url.spec.js`,
# so dropping the glob would ship the contents. Keep both.
NOT_PACKAGED_TRACKED='package.json package-lock.json README.md .gitignore vitest.config.js scripts scripts/** test test/**'
NOT_PACKAGED_BUILD='web-ext-artifacts node_modules'
usage() {
echo "usage: packaging.sh {ignore|pathspec|version|major-minor|patch}" >&2
exit 2
}
# web-ext --ignore-files values, space-separated.
#
# Callers MUST disable pathname expansion first (`set -f`), or the shell will
# glob `test/**` against the working tree before web-ext ever sees the pattern
# and silently narrow it to whatever happens to exist right now.
cmd_ignore() {
echo "$NOT_PACKAGED_TRACKED $NOT_PACKAGED_BUILD"
}
# git pathspec excluding the non-packaged tracked files, e.g.
# :(exclude)extension/package.json :(exclude)extension/test/**
# Same `set -f` requirement as above.
cmd_pathspec() {
for entry in $NOT_PACKAGED_TRACKED; do
printf ':(exclude)extension/%s ' "$entry"
done
echo
}
# MAJOR.MINOR stays hand-set in manifest.json — it's the part that carries
# deliberate meaning. Only the patch component is derived.
cmd_major_minor() {
root=$(git rev-parse --show-toplevel)
grep -E '"version"' "$root/extension/manifest.json" \
| head -1 \
| sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([0-9]+)\.([0-9]+).*/\1.\2/'
}
# 2020-01-01T00:00:00Z — the anchor for the derived patch component. Fixed
# forever; moving it would renumber every version downwards.
VERSION_EPOCH=1577836800
# Minutes since VERSION_EPOCH of the LATEST commit that touched a PACKAGED
# extension file.
#
# Time-derived, per family rule 149: an artifact's ordering key must never be a
# commit count. A count is per-branch — `dev` and `main` count different
# histories of the same code — so the moment BOTH channels publish, their
# versions order by which branch accumulated more commits rather than by which
# is newer. A squash-merge makes that permanent: main gains one commit where dev
# gained five, so dev climbs away from main and a dev install can never cross
# back. That is Roundtable's 2026-08-24 incident (`versionCode` was the branch's
# commit count) in a different repo. Measured here on 2026-08-27: main=23,
# dev=24 under the old formula — one apart, which is exactly how the inversion
# stays invisible until it strands somebody.
#
# Why the commit's time and not the build's:
# * MONOTONIC — max() over a set that only ever gains members. Verified
# across all 24 extension-touching commits: zero non-monotonic steps.
# * STABLE while the extension is unchanged, so an unchanged extension keeps
# its version, the ext-<version> signature cache still hits, and AMO is
# called once per extension CHANGE rather than once per push. Build-time
# minutes would re-sign on every push and never let two channels share a
# signature.
# * SHARED ACROSS CHANNELS — after a merge, `main` sees the same commit and
# derives the same number, so `:latest` reuses the signature `:dev` already
# produced for byte-identical code. Same code, same version, one signing.
# * REPRODUCIBLE — any checkout of a commit yields that commit's version.
#
# Requires real history: a depth-1 clone sees one commit and will derive a wrong
# (too low) value. Every consumer must check out with fetch-depth: 0.
cmd_patch() {
root=$(git rev-parse --show-toplevel)
# Unquoted on purpose: the pathspec must word-split into separate args.
# Globbing is already off script-wide (set -euf above).
# shellcheck disable=SC2046
ts=$(cd "$root" && git log --format=%ct HEAD -- extension/ $(cmd_pathspec) \
| sort -n | tail -1)
if [ -z "$ts" ]; then
echo "packaging.sh: no commit touches a packaged extension file" >&2
exit 1
fi
echo $(( (ts - VERSION_EPOCH) / 60 ))
}
cmd_version() {
echo "$(cmd_major_minor).$(cmd_patch)"
}
[ $# -ge 1 ] || usage
case "$1" in
ignore) cmd_ignore ;;
pathspec) cmd_pathspec ;;
version) cmd_version ;;
major-minor) cmd_major_minor ;;
patch) cmd_patch ;;
*) usage ;;
esac
-29
View File
@@ -1,29 +0,0 @@
import { readFileSync } from 'node:fs'
import { fileURLToPath } from 'node:url'
import path from 'node:path'
const LIB_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', '..', 'lib')
/**
* Load an extension lib and hand back the globals it declares.
*
* The files under lib/ are CLASSIC scripts, not ES modules: manifest.json
* lists them in `background.scripts` and options.html pulls them in with a
* plain <script> tag, so they declare bare functions into a shared scope and
* export nothing. Rather than bolt a `module.exports` shim onto production
* code that would never run in the browser, evaluate the real file the same
* way the browser does — as a script body — and pick the declarations back out.
*
* This means the specs exercise the exact bytes that get packaged into the
* XPI. Only usable for libs that touch no browser APIs at load time
* (url.js, platforms.js); cookies.js and api.js reference `browser.*` and
* would need stubbing, which is why they aren't loaded this way.
*
* @param {string} filename e.g. 'url.js'
* @param {string[]} names declarations to return, e.g. ['normalizeApiUrl']
*/
export function loadLib(filename, names) {
const source = readFileSync(path.join(LIB_DIR, filename), 'utf8')
const factory = new Function(`${source}\nreturn { ${names.join(', ')} }`)
return factory()
}
-190
View File
@@ -1,190 +0,0 @@
import { describe, it, expect } from 'vitest'
import { readFileSync } from 'node:fs'
import { fileURLToPath } from 'node:url'
import path from 'node:path'
import { loadLib } from './helpers/loadLib.js'
const EXT_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..')
const manifest = JSON.parse(readFileSync(path.join(EXT_DIR, 'manifest.json'), 'utf8'))
const { getPlatformFromUrl, isArtistPage, PLATFORMS, PLATFORM_ARTIST_PATTERNS } = loadLib(
'platforms.js',
['getPlatformFromUrl', 'isArtistPage', 'PLATFORMS', 'PLATFORM_ARTIST_PATTERNS']
)
describe('getPlatformFromUrl', () => {
it('identifies each platform from a domain URL', () => {
expect(getPlatformFromUrl('https://www.patreon.com/Atole')).toBe('patreon')
expect(getPlatformFromUrl('https://subscribestar.adult/someone')).toBe('subscribestar')
expect(getPlatformFromUrl('https://www.hentai-foundry.com/user/someone')).toBe('hentaifoundry')
expect(getPlatformFromUrl('https://discord.com/channels/@me')).toBe('discord')
expect(getPlatformFromUrl('https://www.pixiv.net/en/users/123')).toBe('pixiv')
})
it('accepts http as well as https, with or without www', () => {
expect(getPlatformFromUrl('http://patreon.com/Atole')).toBe('patreon')
expect(getPlatformFromUrl('https://www.patreon.com/Atole')).toBe('patreon')
})
it('returns null for unrelated hosts', () => {
expect(getPlatformFromUrl('https://example.com/patreon.com')).toBe(null)
expect(getPlatformFromUrl('https://not-patreon.com/Atole')).toBe(null)
expect(getPlatformFromUrl('')).toBe(null)
})
it('returns null for deviantart, retired at #3069', () => {
// The 2026-07-05 product decision (FC downloaders = art-dedicated services
// only) left deviantart wired for seven weeks. Asserting the negative is
// what keeps a partial retirement from being re-completed by accident.
expect(getPlatformFromUrl('https://www.deviantart.com/someone')).toBe(null)
expect(PLATFORMS.deviantart).toBeUndefined()
expect(PLATFORM_ARTIST_PATTERNS.deviantart).toBeUndefined()
})
})
describe('isArtistPage', () => {
// Regression cases from issue #1485: the Add-to-FC button vanished once the
// operator SUBSCRIBED to a creator, because Patreon serves subscribed users
// the /cw/ ("creator workspace") URL and the pattern only matched the bare
// root. All three creator URL shapes must match, plus inner pages — the
// button matters most exactly when you're subscribed.
it('matches all three Patreon creator URL shapes', () => {
expect(isArtistPage('https://www.patreon.com/Atole', 'patreon')).toBe(true)
expect(isArtistPage('https://www.patreon.com/c/Atole', 'patreon')).toBe(true)
expect(isArtistPage('https://www.patreon.com/cw/Atole', 'patreon')).toBe(true)
})
it('matches Patreon creator inner pages', () => {
expect(isArtistPage('https://www.patreon.com/cw/Atole/posts', 'patreon')).toBe(true)
expect(isArtistPage('https://www.patreon.com/Atole/membership', 'patreon')).toBe(true)
})
it('excludes Patreon navigation pages that are not creators', () => {
for (const nav of ['home', 'search', 'messages', 'notifications', 'library', 'settings']) {
expect(isArtistPage(`https://www.patreon.com/${nav}`, 'patreon')).toBe(false)
expect(isArtistPage(`https://www.patreon.com/${nav}/anything`, 'patreon')).toBe(false)
}
})
it('matches SubscribeStar creator roots on both TLDs but not feed pages', () => {
expect(isArtistPage('https://subscribestar.adult/someone', 'subscribestar')).toBe(true)
expect(isArtistPage('https://subscribestar.com/someone', 'subscribestar')).toBe(true)
expect(isArtistPage('https://subscribestar.adult/feed', 'subscribestar')).toBe(false)
expect(isArtistPage('https://subscribestar.adult/messages', 'subscribestar')).toBe(false)
})
it('matches Hentai Foundry user pages only', () => {
expect(isArtistPage('https://www.hentai-foundry.com/user/someone', 'hentaifoundry')).toBe(true)
expect(isArtistPage('https://www.hentai-foundry.com/pictures/popular', 'hentaifoundry')).toBe(
false
)
})
it('matches Pixiv numeric user pages, with or without the /en/ prefix', () => {
expect(isArtistPage('https://www.pixiv.net/users/12345', 'pixiv')).toBe(true)
expect(isArtistPage('https://www.pixiv.net/en/users/12345', 'pixiv')).toBe(true)
expect(isArtistPage('https://www.pixiv.net/en/artworks/999', 'pixiv')).toBe(false)
})
it('returns false for a platform with no artist pattern (discord)', () => {
expect(isArtistPage('https://discord.com/channels/@me', 'discord')).toBe(false)
})
it('returns false for an unknown platform key', () => {
expect(isArtistPage('https://www.patreon.com/Atole', 'nope')).toBe(false)
})
})
describe('platform table integrity', () => {
it('gives every artist pattern a corresponding platform entry', () => {
// A pattern keyed to a platform that no longer exists is dead code that
// silently never fires; the reverse (a platform with no pattern) is the
// legitimate discord case, so only this direction is an error.
for (const key of Object.keys(PLATFORM_ARTIST_PATTERNS)) {
expect(Object.keys(PLATFORMS)).toContain(key)
}
})
it('gives every platform the fields the popup renders', () => {
for (const [key, platform] of Object.entries(PLATFORMS)) {
expect(platform.name, `${key}.name`).toBeTruthy()
expect(platform.color, `${key}.color`).toMatch(/^#[0-9A-Fa-f]{6}$/)
expect(['cookies', 'token'], `${key}.authType`).toContain(platform.authType)
expect(platform.urlPattern, `${key}.urlPattern`).toBeInstanceOf(RegExp)
expect(Array.isArray(platform.domains), `${key}.domains`).toBe(true)
expect(platform.domains.length, `${key}.domains`).toBeGreaterThan(0)
}
})
it('keeps every artist URL matched by its own platform pattern too', () => {
// isArtistPage is only ever consulted after getPlatformFromUrl resolves a
// key, so an artist pattern matching a URL its platform's urlPattern
// rejects would be unreachable.
const samples = {
patreon: 'https://www.patreon.com/cw/Atole',
subscribestar: 'https://subscribestar.adult/someone',
hentaifoundry: 'https://www.hentai-foundry.com/user/someone',
pixiv: 'https://www.pixiv.net/en/users/12345'
}
for (const [key, url] of Object.entries(samples)) {
expect(isArtistPage(url, key), `${key} artist pattern`).toBe(true)
expect(getPlatformFromUrl(url), `${key} urlPattern`).toBe(key)
}
})
})
describe('manifest.json agrees with the platform table', () => {
// #3069: deviantart was dropped from the product in July but survived in
// manifest.json until late August, because NOTHING tied the manifest's
// domain lists back to PLATFORMS. These two specs are that tie. Both
// directions matter: a stale match ships host access the product decided
// not to use, and a missing one silently kills the Add-to-FC button.
const matches = manifest.content_scripts[0].matches
// '*://*.patreon.com/*' -> '.patreon.com', the form PLATFORMS.domains uses.
const hostOf = (m) => m.replace(/^\*:\/\/\*/, '').replace(/\/\*$/, '')
it('injects the content script only on domains a platform claims', () => {
for (const m of matches) {
const host = hostOf(m)
const owner = Object.entries(PLATFORMS).find(
([, p]) => p.domains.includes(host)
)
expect(owner, `no platform claims content-script match "${m}"`).toBeTruthy()
// The content script exists to draw the Add-as-source button, so a
// platform with no artist pattern (discord) has no business here.
expect(
PLATFORM_ARTIST_PATTERNS[owner[0]],
`"${m}" injects for ${owner[0]}, which has no artist pattern`
).toBeTruthy()
}
})
it('injects on every platform that has an artist pattern', () => {
const covered = new Set(
matches
.map(hostOf)
.map((h) => Object.entries(PLATFORMS).find(([, p]) => p.domains.includes(h)))
.filter(Boolean)
.map(([key]) => key)
)
for (const key of Object.keys(PLATFORM_ARTIST_PATTERNS)) {
expect(covered, `${key} has an artist pattern but no content-script match`).toContain(key)
}
})
it('requests no host permission for a domain no platform claims', () => {
// '*://*/*' is the deliberate exception: FC is self-hosted at an arbitrary
// operator-chosen URL, so the extension cannot enumerate its own backend.
// Every OTHER entry is a platform domain and must still have an owner.
for (const h of manifest.host_permissions) {
if (h === '*://*/*') continue
const host = hostOf(h)
// pixiv's OAuth/API hosts are pixiv infrastructure, not creator pages,
// so they are matched by suffix rather than by the domains list.
const claimed = Object.values(PLATFORMS).some(
(p) => p.domains.includes(host) || p.domains.some((d) => host.endsWith(d))
)
expect(claimed, `host permission "${h}" belongs to no platform`).toBe(true)
}
})
})
-93
View File
@@ -1,93 +0,0 @@
import { describe, it, expect } from 'vitest'
import { loadLib } from './helpers/loadLib.js'
const { normalizeApiUrl, webRootFromApiUrl } = loadLib('url.js', [
'normalizeApiUrl',
'webRootFromApiUrl'
])
describe('normalizeApiUrl', () => {
// The bug this exists for (issue #2393): the instance root was accepted and
// stored verbatim, so every request went to /credentials instead of
// /api/credentials. That path is a Vue router route, so the SPA catch-all
// answered GET with 200 HTML and rejected POST with 405 — which read as a
// backend bug rather than a URL one.
it('appends /api to an instance root', () => {
expect(normalizeApiUrl('http://curator.traefik.internal')).toBe(
'http://curator.traefik.internal/api'
)
})
it('leaves an API root alone rather than doubling the suffix', () => {
expect(normalizeApiUrl('http://curator.traefik.internal/api')).toBe(
'http://curator.traefik.internal/api'
)
})
it('is idempotent', () => {
const once = normalizeApiUrl('http://curator.example.com')
expect(normalizeApiUrl(once)).toBe(once)
})
it('strips trailing slashes before deciding', () => {
expect(normalizeApiUrl('http://curator.example.com/')).toBe('http://curator.example.com/api')
expect(normalizeApiUrl('http://curator.example.com///')).toBe('http://curator.example.com/api')
expect(normalizeApiUrl('http://curator.example.com/api/')).toBe('http://curator.example.com/api')
})
it('trims surrounding whitespace (paste artifacts)', () => {
expect(normalizeApiUrl(' http://curator.example.com ')).toBe(
'http://curator.example.com/api'
)
})
it('matches the /api suffix case-insensitively', () => {
expect(normalizeApiUrl('http://curator.example.com/API')).toBe('http://curator.example.com/API')
})
it('returns empty string for empty/nullish input, never a bare "/api"', () => {
// isConfigured() gates on truthiness, so a bogus '/api' here would read as
// "configured" and produce a request against the options page's own origin.
expect(normalizeApiUrl('')).toBe('')
expect(normalizeApiUrl(' ')).toBe('')
expect(normalizeApiUrl(null)).toBe('')
expect(normalizeApiUrl(undefined)).toBe('')
})
it('does not treat a path merely containing "api" as the suffix', () => {
expect(normalizeApiUrl('http://curator.example.com/apiary')).toBe(
'http://curator.example.com/apiary/api'
)
})
it('preserves a subpath deployment', () => {
expect(normalizeApiUrl('http://host.internal/curator')).toBe('http://host.internal/curator/api')
})
})
describe('webRootFromApiUrl', () => {
// The SPA root, where the Vue router and the served XPI live. Used by
// OPEN_ARTIST_PAGE and the self-update check — NOT the JSON API.
it('strips the /api suffix', () => {
expect(webRootFromApiUrl('http://curator.example.com/api')).toBe('http://curator.example.com')
})
it('accepts an instance root unchanged', () => {
expect(webRootFromApiUrl('http://curator.example.com')).toBe('http://curator.example.com')
})
it('agrees with normalizeApiUrl in both directions', () => {
for (const input of ['http://curator.example.com', 'http://curator.example.com/api']) {
expect(normalizeApiUrl(webRootFromApiUrl(input))).toBe(normalizeApiUrl(input))
}
})
it('preserves a subpath deployment', () => {
expect(webRootFromApiUrl('http://host.internal/curator/api')).toBe('http://host.internal/curator')
})
it('returns empty string for empty/nullish input', () => {
expect(webRootFromApiUrl('')).toBe('')
expect(webRootFromApiUrl(null)).toBe('')
})
})
-112
View File
@@ -1,112 +0,0 @@
import { describe, it, expect } from 'vitest'
import { readFileSync } from 'node:fs'
import { execFileSync } from 'node:child_process'
import { fileURLToPath } from 'node:url'
import path from 'node:path'
const EXT_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..')
const read = (name) => JSON.parse(readFileSync(path.join(EXT_DIR, name), 'utf8'))
const readText = (...seg) => readFileSync(path.join(EXT_DIR, ...seg), 'utf8')
// Only the git-free subcommands are exercised here: `version`/`patch` shell out
// to git, and the extension lane runs on node:24-bookworm-slim which may not
// ship it. Those two are covered where git is guaranteed — ci.yml and build.yml
// run on ci-python.
const packaging = (cmd) =>
execFileSync('sh', [path.join(EXT_DIR, 'scripts', 'packaging.sh'), cmd], {
cwd: EXT_DIR,
encoding: 'utf8'
})
.trim()
.split(/\s+/)
.filter(Boolean)
describe('packaging.sh — the single definition of what ships', () => {
it('emits an ignore list and a pathspec that agree on the tracked files', () => {
const ignore = packaging('ignore')
const pathspec = packaging('pathspec').map((e) => e.replace(':(exclude)extension/', ''))
// Every git-excluded path must also be hidden from web-ext. The reverse is
// not required: node_modules and web-ext-artifacts are build output git
// never tracks, so they appear only in the ignore list.
for (const entry of pathspec) {
expect(ignore, `pathspec has "${entry}" but --ignore-files does not`).toContain(entry)
}
expect(pathspec.length).toBeGreaterThan(0)
expect(ignore).toContain('node_modules')
})
it('emits glob patterns literally, never expanded against the working tree', () => {
// The script iterates its lists with deliberate word-splitting, so it must
// run with pathname expansion off. Without that, invoking it from a cwd
// where test/ exists (exactly how ci.yml and vitest call it) expands
// `test/**` into the individual spec files, and the pathspec silently stops
// covering anything added later.
const pathspec = packaging('pathspec')
expect(pathspec).toContain(':(exclude)extension/test/**')
expect(pathspec).toContain(':(exclude)extension/scripts/**')
expect(pathspec.some((e) => e.includes('.spec.js'))).toBe(false)
expect(pathspec.some((e) => e.includes('helpers'))).toBe(false)
})
it('keeps its own scripts and specs out of the XPI', () => {
// Both are repo infrastructure. web-ext packages everything not ignored, so
// omitting either would ship dev tooling to users -- and `test/**` in
// particular only survives because callers `set -f` before substituting it.
const ignore = packaging('ignore')
expect(ignore).toContain('vitest.config.js')
// Both forms per directory. The glob covers the contents; the bare name
// covers the directory ENTRY, which web-ext writes separately — with only
// the glob, the XPI carries an empty `test/` and `scripts/`.
for (const dir of ['test', 'scripts']) {
expect(ignore, `${dir} contents`).toContain(`${dir}/**`)
expect(ignore, `${dir} directory entry`).toContain(dir)
}
})
})
describe('consumers delegate rather than keeping their own copy', () => {
// These assertions are the actual anti-regression value: it is easy for a
// future edit to "simplify" by inlining a literal list again, which silently
// reintroduces the drift that issue #2397 was about.
it('package.json derives --ignore-files from the script', () => {
for (const [name, script] of Object.entries(read('package.json').scripts)) {
if (!script.includes('--ignore-files')) continue
expect(script, `${name} should call packaging.sh`).toContain('scripts/packaging.sh ignore')
expect(script, `${name} must set -f before the substitution`).toMatch(/set -f;/)
}
})
it('ci.yml derives its pathspec from the script and hardcodes none', () => {
const ci = readText('..', '.forgejo', 'workflows', 'ci.yml')
expect(ci).toContain('extension/scripts/packaging.sh pathspec')
// A literal :(exclude)extension/... in the workflow means someone bypassed
// the shared definition.
expect(ci).not.toMatch(/:\(exclude\)extension\//)
})
})
describe('extension version', () => {
it('keeps manifest.json and package.json in lockstep', () => {
expect(read('manifest.json').version).toBe(read('package.json').version)
})
it('uses a plain dotted numeric version AMO will accept', () => {
expect(read('package.json').version).toMatch(/^\d+(\.\d+)*$/)
})
it('declares manifest v3', () => {
expect(read('manifest.json').manifest_version).toBe(3)
})
it('lists every background script that exists, in dependency order', () => {
// url.js must load BEFORE api.js: api.js calls normalizeApiUrl at
// init()-time, and these are classic scripts sharing one scope, so a
// reordering here is a runtime ReferenceError with no build-time signal.
const scripts = read('manifest.json').background.scripts
for (const rel of scripts) {
expect(() => readFileSync(path.join(EXT_DIR, rel)), `missing ${rel}`).not.toThrow()
}
expect(scripts.indexOf('lib/url.js')).toBeLessThan(scripts.indexOf('lib/api.js'))
})
})
-13
View File
@@ -1,13 +0,0 @@
import { defineConfig } from 'vitest/config'
// Mirrors frontend/vitest.config.js, minus the Vue plugin — the extension has
// no SFCs and mounts nothing. Pure-logic specs only, so `node` is enough; the
// libs under test are deliberately the ones with no browser-API surface (see
// test/helpers/loadLib.js).
export default defineConfig({
test: {
environment: 'node',
include: ['test/**/*.spec.js'],
passWithNoTests: true
}
})
+13 -11
View File
@@ -4,28 +4,30 @@
"private": true, "private": true,
"type": "module", "type": "module",
"engines": { "engines": {
"node": ">=24" "node": ">=22"
}, },
"scripts": { "scripts": {
"dev": "vite", "dev": "vite",
"build": "vite build", "build": "vite build",
"preview": "vite preview", "preview": "vite preview",
"test:unit": "vitest run" "test:unit": "vitest run",
"check": "vue-tsc --noEmit"
}, },
"dependencies": { "dependencies": {
"vue": "^3.5.0", "vue": "^3.4.0",
"vue-router": "^5.0.0", "vue-router": "^4.3.0",
"pinia": "^3.0.0", "pinia": "^2.1.0",
"vuetify": "^4.0.0", "vuetify": "^3.5.0",
"@mdi/font": "^7.4.0" "@mdi/font": "^7.4.0"
}, },
"devDependencies": { "devDependencies": {
"@vitejs/plugin-vue": "^6.0.0", "@vitejs/plugin-vue": "^5.0.0",
"vite": "^8.0.0", "vite": "^5.2.0",
"vite-plugin-vuetify": "^2.1.0", "vue-tsc": "^2.0.0",
"vite-plugin-vuetify": "^2.0.0",
"sass": "^1.71.0", "sass": "^1.71.0",
"vitest": "^4.0.0", "vitest": "^2.1.0",
"@vue/test-utils": "^2.4.0", "@vue/test-utils": "^2.4.0",
"happy-dom": "^20.0.0" "happy-dom": "^15.0.0"
} }
} }
+6 -6
View File
@@ -18,11 +18,11 @@ const route = useRoute()
<style scoped> <style scoped>
.fc-content { .fc-content {
min-height: 100vh; min-height: 100vh;
/* NO padding-top: the TopNav is position:sticky, so it already reserves its /* Push initial viewport content below the sticky TopNav. Without
own space in the v-app flex column — content flows directly below it. The this, some views' first rows / form fields / table headers can
old 64px padding-top was a leftover from a FIXED navbar and double-counted end up obscured by the navbar (depending on parent overflow
that space, leaving a large empty band at the top of EVERY view (and pushing context interacting with position: sticky). Scrolled-down content
the full-height calc(100vh - 64px) views down so they overflowed). Removed still slides under the nav — the gradient-fade design is intact. */
2026-07-13. Scrolled content still slides under the sticky nav as before. */ padding-top: 64px;
} }
</style> </style>
+1 -1
View File
@@ -1,7 +1,7 @@
<template> <template>
<v-snackbar <v-snackbar
v-model="show" :color="color" location="bottom right" timeout="4000" v-model="show" :color="color" location="bottom right" timeout="4000"
min-height="68" elevation="4" multi-line elevation="4"
> >
{{ message }} {{ message }}
<template #actions> <template #actions>
+7 -52
View File
@@ -1,5 +1,5 @@
<template> <template>
<header ref="navEl" class="fc-topnav" :class="{ 'fc-topnav--chrome': hasStickyChrome }"> <header class="fc-topnav">
<div class="fc-nav-left"> <div class="fc-nav-left">
<RouterLink :to="FRONT_DOOR" class="fc-brand" aria-label="FabledCurator home"> <RouterLink :to="FRONT_DOOR" class="fc-brand" aria-label="FabledCurator home">
<img src="/favicon.svg" alt="" class="fc-brand__glyph" width="22" height="22" /> <img src="/favicon.svg" alt="" class="fc-brand__glyph" width="22" height="22" />
@@ -64,39 +64,13 @@
</template> </template>
<script setup> <script setup>
import { computed, onBeforeUnmount, onMounted, ref } from 'vue' import { computed, onMounted } from 'vue'
import { useRoute } from 'vue-router'
import router, { FRONT_DOOR } from '../router.js' import router, { FRONT_DOOR } from '../router.js'
import { useSystemStore } from '../stores/system.js' import { useSystemStore } from '../stores/system.js'
import PipelineStatusChip from './PipelineStatusChip.vue' import PipelineStatusChip from './PipelineStatusChip.vue'
const system = useSystemStore() const system = useSystemStore()
onMounted(() => system.refreshHealth())
// Publish the nav's REAL height as --fc-nav-h so full-height workspaces
// (Explore/Subscriptions) and sticky sub-headers pin to it exactly instead of a
// hardcoded 64px that Vuetify 4's MD3 sizing broke — the Explore breadcrumb was
// tucking under a taller nav (#1481). ResizeObserver keeps it live as the nav
// reflows (per-view teleported actions, mobile breakpoint, chip state changes).
const navEl = ref(null)
let navRO = null
onMounted(() => {
system.refreshHealth()
if (navEl.value && 'ResizeObserver' in window) {
navRO = new ResizeObserver(() => {
const h = navEl.value?.offsetHeight
if (h) document.documentElement.style.setProperty('--fc-nav-h', `${h}px`)
})
navRO.observe(navEl.value)
}
})
onBeforeUnmount(() => { navRO?.disconnect() })
// Views that pin a sticky sub-header (filter bar / tabs) directly under the nav
// declare `meta.stickyChrome`. On those, the nav doesn't fade to transparent at
// its bottom — it hands off at the shared seam alpha so the sub-header can
// continue the SAME fade (see .fc-chrome-continues in app.css). One gradient.
const route = useRoute()
const hasStickyChrome = computed(() => !!route.meta?.stickyChrome)
// Every route with a meta.title is a nav entry. Order by meta.navOrder — // Every route with a meta.title is a nav entry. Order by meta.navOrder —
// router.getRoutes() does NOT guarantee declaration order, so explicit numbers // router.getRoutes() does NOT guarantee declaration order, so explicit numbers
@@ -145,35 +119,16 @@ const health = computed(() => {
align-items: center; align-items: center;
gap: 1rem; gap: 1rem;
padding: 0.75rem 1rem; padding: 0.75rem 1rem;
/* Obsidian (#14171A) fade — content scrolls under it. Holds high (0.92 → /* Obsidian (#14171A = 20,23,26) gradient fade — content scrolls under it. */
0.84) through the top half, then eases to transparent over the bottom
quarter so it tails off softly instead of a straight line to a hard edge
(operator 2026-07-13). Shared --fc-chrome-rgb keeps it in sync with the
sub-header continuation. */
background: linear-gradient( background: linear-gradient(
to bottom, to bottom,
rgba(var(--fc-chrome-rgb), 0.92) 0%, rgba(20, 23, 26, 0.92) 0%,
rgba(var(--fc-chrome-rgb), 0.84) 50%, rgba(20, 23, 26, 0.65) 60%,
rgba(var(--fc-chrome-rgb), 0.55) 75%, rgba(20, 23, 26, 0) 100%
rgba(var(--fc-chrome-rgb), 0) 100%
); );
backdrop-filter: blur(2px); backdrop-filter: blur(2px);
-webkit-backdrop-filter: blur(2px); -webkit-backdrop-filter: blur(2px);
} }
/* On a view with a sticky sub-header pinned beneath (meta.stickyChrome), the nav
stops fading at the shared seam alpha instead of going fully transparent — the
sub-header (.fc-chrome-continues) picks the fade up from there, so the two read
as one continuous gradient. Compound selector out-specifies .fc-topnav so it
wins regardless of Vite's production CSS ordering. --fc-chrome-* come from the
global :root in app.css (custom props inherit into scoped styles). */
.fc-topnav.fc-topnav--chrome {
background: linear-gradient(
to bottom,
rgba(var(--fc-chrome-rgb), 0.92) 0%,
rgba(var(--fc-chrome-rgb), 0.84) 60%,
rgba(var(--fc-chrome-rgb), var(--fc-chrome-seam)) 100%
);
}
.fc-brand { .fc-brand {
display: flex; display: flex;
@@ -50,19 +50,14 @@ const projected = ref(null)
const projectedCounts = computed(() => projected.value?.projected || null) const projectedCounts = computed(() => projected.value?.projected || null)
// `posts` is named here, not left to the counts grid below it: an artist whose const modalDescription = computed(
// posts are body-only previews as `images: 0`, and a summary line that says () => projected.value
// only "0 images" reads as "this artist is empty" while the apply destroys ? `Artist “${props.artistName}” — `
// every captured post body (#3067). Attachments stay in the grid — the grid + `${projected.value.projected.images} images, `
// renders every key, so this line carries only what changes the read. + `${projected.value.projected.sources} sources, `
const modalDescription = computed(() => { + `${Math.round(projected.value.projected.bytes_on_disk / 1_048_576)} MiB on disk`
const p = projectedCounts.value : '',
return p )
? `Artist “${props.artistName}” — ${p.images} images, `
+ `${p.posts} posts, ${p.sources} sources, `
+ `${Math.round(p.bytes_on_disk / 1_048_576)} MiB on disk`
: ''
})
async function onClick() { async function onClick() {
loading.value = true loading.value = true
@@ -10,7 +10,7 @@
filter, applied retroactively to the existing library. filter, applied retroactively to the existing library.
</p> </p>
<v-row density="compact"> <v-row dense>
<v-col cols="6"> <v-col cols="6">
<v-text-field <v-text-field
v-model.number="minW" label="Min width (px)" type="number" v-model.number="minW" label="Min width (px)" type="number"
@@ -13,7 +13,7 @@
cadence as the transparency audit. cadence as the transparency audit.
</p> </p>
<v-row density="compact"> <v-row dense>
<v-col cols="6"> <v-col cols="6">
<v-text-field <v-text-field
v-model.number="threshold" label="Threshold (01)" v-model.number="threshold" label="Threshold (01)"
@@ -1,56 +0,0 @@
<!--
Canonical settings number field (DRY pass #161): a compact numeric v-text-field
with a built-in clamp to [min,max] on commit. Hand-rolled identically across the
ML settings cards (HeadsCard x6, CropProposersCard, VideoEmbeddingCard).
The clamp is the point: the cards previously sent Number(raw) straight to the
API, so an out-of-range value bounced off the API's 400 validator (only
TranslationCard clamped). This is now the single home for that clamp.
Binds `modelValue` (v-model) and emits `change` on blur/enter AFTER clamping, so
the parent's save reads the already-clamped value same as the prior
`v-model.number` + `@change=save` pattern.
-->
<template>
<v-text-field
:model-value="modelValue"
:label="label"
type="number"
:min="min"
:max="max"
:step="step"
:disabled="disabled"
:density="density" hide-details
:style="{ maxWidth }"
@update:model-value="v => emit('update:modelValue', v)"
@change="onCommit"
/>
</template>
<script setup>
const props = defineProps({
modelValue: { type: [Number, String], default: null },
label: { type: String, default: '' },
min: { type: [Number, String], default: null },
max: { type: [Number, String], default: null },
step: { type: [Number, String], default: 1 },
maxWidth: { type: String, default: '200px' },
density: { type: String, default: 'compact' },
disabled: { type: Boolean, default: false },
})
const emit = defineEmits(['update:modelValue', 'change'])
function onCommit() {
// On blur/enter: coerce to a number and clamp to [min,max] so an out-of-range
// value never reaches the API. props.modelValue reflects the latest keystroke
// (kept in sync by the passthrough above); re-emit the clamped number, then let
// the parent persist.
let n = Number(props.modelValue)
if (!Number.isNaN(n)) {
if (props.min !== null && props.min !== '') n = Math.max(Number(props.min), n)
if (props.max !== null && props.max !== '') n = Math.min(Number(props.max), n)
if (n !== Number(props.modelValue)) emit('update:modelValue', n)
}
emit('change')
}
</script>
@@ -1,42 +0,0 @@
<!--
Canonical settings toggle row (DRY pass #161): an accent icon + an uppercase
.fc-section-h label + a right-aligned switch. Hand-rolled identically in the
ML settings cards (HeadsCard x3, CropProposersCard, MLBackfillCard).
Two-way binds `modelValue` (so the parent switch state stays optimistic) AND
emits `change` with the new boolean, so the parent can persist + revert on
failure matching the prior `v-model` + `@update:model-value=handler` pattern.
-->
<template>
<div class="d-flex align-center mb-1" style="gap: 10px;">
<v-icon v-if="icon" size="18" :color="iconColor">{{ icon }}</v-icon>
<span class="fc-section-h">{{ label }}</span>
<v-switch
:model-value="modelValue"
:loading="loading"
:disabled="disabled"
hide-details density="compact" color="success" class="ml-auto"
@update:model-value="onSwitch"
/>
</div>
</template>
<script setup>
defineProps({
modelValue: { type: Boolean, default: false },
label: { type: String, default: '' },
icon: { type: String, default: '' },
// Icon tint. Default accent; pass null for the theme default (e.g. when a row
// is off). null (not undefined) so the default doesn't override it.
iconColor: { type: String, default: 'accent' },
loading: { type: Boolean, default: false },
disabled: { type: Boolean, default: false },
})
const emit = defineEmits(['update:modelValue', 'change'])
function onSwitch(v) {
const b = !!v
emit('update:modelValue', b)
emit('change', b)
}
</script>
+3 -3
View File
@@ -14,10 +14,10 @@
@update:search="onSearch" @update:search="onSearch"
@update:model-value="onPick" @update:model-value="onPick"
> >
<template #item="{ props: itemProps, internalItem }"> <template #item="{ props: itemProps, item }">
<v-list-item v-bind="itemProps" :title="internalItem.raw.name"> <v-list-item v-bind="itemProps" :title="item.raw.name">
<template #subtitle> <template #subtitle>
{{ internalItem.raw.fandom_name ? `character · ${internalItem.raw.fandom_name}` : internalItem.raw.kind }} {{ item.raw.fandom_name ? `character · ${item.raw.fandom_name}` : item.raw.kind }}
</template> </template>
</v-list-item> </v-list-item>
</template> </template>
@@ -1,5 +1,5 @@
<template> <template>
<div class="fc-filterbar-wrap fc-chrome-continues"> <div class="fc-filterbar-wrap">
<div class="fc-filterbar"> <div class="fc-filterbar">
<v-autocomplete <v-autocomplete
v-model="selected" v-model="selected"
@@ -13,14 +13,14 @@
@update:search="onSearch" @update:search="onSearch"
@update:model-value="onPick" @update:model-value="onPick"
> >
<template #item="{ props: itemProps, internalItem }"> <template #item="{ props: itemProps, item }">
<v-list-item v-bind="itemProps" :title="internalItem.raw.name"> <v-list-item v-bind="itemProps" :title="item.raw.name">
<template #prepend> <template #prepend>
<v-icon size="small">{{ iconFor(internalItem.raw) }}</v-icon> <v-icon size="small">{{ iconFor(item.raw) }}</v-icon>
</template> </template>
<template #subtitle> <template #subtitle>
{{ internalItem.raw.kind === 'artist' ? 'artist' {{ item.raw.kind === 'artist' ? 'artist'
: (internalItem.raw.fandom_name ? `character · ${internalItem.raw.fandom_name}` : internalItem.raw.kind) }} : (item.raw.fandom_name ? `character · ${item.raw.fandom_name}` : item.raw.kind) }}
</template> </template>
</v-list-item> </v-list-item>
</template> </template>
@@ -306,17 +306,27 @@ function pushFilter(mutate) {
frosted block pinned directly under the 64px TopNav and continuous with it. */ frosted block pinned directly under the 64px TopNav and continuous with it. */
.fc-filterbar-wrap { .fc-filterbar-wrap {
position: sticky; position: sticky;
top: var(--fc-nav-h, 64px); /* pins at the nav's real measured bottom (#1481) */ top: 64px;
z-index: 5; z-index: 5;
/* Attach to the TopNav: cancel the v-container's top padding (pt-2 = 8px) /* Attach to the TopNav: cancel the v-container's top padding (pt-2 = 8px)
so the bar sits flush at 64px even at scroll 0 — without this it detaches so the bar sits flush at 64px even at scroll 0 — without this it detaches
and a gap shows through when scrolled to the top. */ and a gap shows through when scrolled to the top. */
margin-top: -8px; margin-top: -8px;
margin-bottom: 12px; margin-bottom: 12px;
/* The frost itself (obsidian fade + blur) is the shared .fc-chrome-continues /* EXACT same gradiated obsidian (#14171A = 20,23,26) frost as the TopNav so
primitive: it CONTINUES the TopNav's fade from the seam alpha to transparent the two read as one continuous piece of chrome — images scroll visibly
rather than re-darkening, so the nav + bar read as one gradient (operator under both. The nav's gradient fades to transparent at ITS bottom; this
2026-07-13). This block only owns the sticky positioning now. */ bar re-darkens at its top, so a faint seam (the page/image showing through
the nav's transparent edge) separates them when scrolled to the very top,
while under-scroll they frost as one. */
background: linear-gradient(
to bottom,
rgba(20, 23, 26, 0.92) 0%,
rgba(20, 23, 26, 0.65) 60%,
rgba(20, 23, 26, 0) 100%
);
backdrop-filter: blur(2px);
-webkit-backdrop-filter: blur(2px);
} }
.fc-filterbar { .fc-filterbar {
display: flex; display: flex;
@@ -331,20 +341,6 @@ function pushFilter(mutate) {
.fc-filterbar-wrap :deep(.v-btn-group) { .fc-filterbar-wrap :deep(.v-btn-group) {
background-color: rgba(20, 23, 26, 0.72); background-color: rgba(20, 23, 26, 0.72);
} }
/* Media toggle (All / Images / Videos) as ONE cohesive segmented control.
FC's global VBtn { rounded: 'pill' } default made Vuetify 4 pill-round each
SEGMENT individually, so the rounded ends collided at the joins — the shapes
landed awkwardly on the button edges (operator 2026-07-13). Square the inner
segments (over the pill utility's !important) and clip the group to a single
8px outline (matches the chips/tiles rounding elsewhere in the app). Radius
only — no height change, so the bar height and nav offset are untouched. */
.fc-filterbar-wrap :deep(.v-btn-toggle) {
border-radius: 8px;
overflow: hidden;
}
.fc-filterbar-wrap :deep(.v-btn-toggle .v-btn) {
border-radius: 0 !important;
}
.fc-filterbar__search { max-width: 320px; min-width: 200px; } .fc-filterbar__search { max-width: 320px; min-width: 200px; }
.fc-filterbar__chips { display: flex; align-items: center; gap: 6px; flex-wrap: wrap; } .fc-filterbar__chips { display: flex; align-items: center; gap: 6px; flex-wrap: wrap; }
/* The tag chips' bodies toggle include/exclude — signal they're clickable. */ /* The tag chips' bodies toggle include/exclude — signal they're clickable. */
@@ -139,9 +139,9 @@ function onThumbError() { thumbError.value = true }
position: absolute; top: 8px; left: 8px; position: absolute; top: 8px; left: 8px;
width: 22px; height: 22px; border-radius: 4px; width: 22px; height: 22px; border-radius: 4px;
border: 2px solid rgba(232, 228, 216, 0.8); border: 2px solid rgba(232, 228, 216, 0.8);
background: rgba(var(--v-theme-background), 0.45); background: rgba(20, 23, 26, 0.45);
display: grid; place-items: center; display: grid; place-items: center;
color: rgb(var(--v-theme-background)); z-index: 11; color: #14171A; z-index: 11;
} }
.fc-gallery-item__checkbox.on { .fc-gallery-item__checkbox.on {
background: rgb(var(--v-theme-accent)); background: rgb(var(--v-theme-accent));
@@ -152,7 +152,7 @@ function onThumbError() { thumbError.value = true }
min-width: 22px; height: 22px; padding: 0 5px; min-width: 22px; height: 22px; padding: 0 5px;
border-radius: 11px; border-radius: 11px;
background: rgb(var(--v-theme-accent)); background: rgb(var(--v-theme-accent));
color: rgb(var(--v-theme-background)); font-size: 12px; font-weight: 700; color: #14171A; font-size: 12px; font-weight: 700;
display: grid; place-items: center; z-index: 11; display: grid; place-items: center; z-index: 11;
pointer-events: none; pointer-events: none;
} }
@@ -160,8 +160,7 @@ function onThumbError() { thumbError.value = true }
position: absolute; left: 0; right: 0; bottom: 0; position: absolute; left: 0; right: 0; bottom: 0;
padding: 14px 8px 6px; padding: 14px 8px 6px;
background: linear-gradient( background: linear-gradient(
to top, rgba(var(--v-theme-background), 0.78), to top, rgba(20, 23, 26, 0.78), rgba(20, 23, 26, 0)
rgba(var(--v-theme-background), 0)
); );
font-size: 12px; line-height: 1.2; font-size: 12px; line-height: 1.2;
white-space: nowrap; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; overflow: hidden; text-overflow: ellipsis;
@@ -1,15 +1,14 @@
<template> <template>
<!-- System-tag auto-applies (chrome hides / process WIP tags) that ALSO looked <!-- Auto-hidden chrome that ALSO looked like real content surfaced PROACTIVELY
like real content surfaced PROACTIVELY atop the gallery whenever there's atop the gallery whenever there's something to review (NOT gated on the
something to review (NOT gated on the Show-hidden toggle, so misfires can't Show-hidden toggle, so misfires can't go unnoticed), most-concerning first,
go unnoticed), most-concerning first, with keep / remove (#141, #1464). with keep / un-hide (#141). Renders nothing when there's nothing to review. -->
Renders nothing when there's nothing to review. --> <section v-if="items.length" class="fc-review" aria-label="Hidden images to review">
<section v-if="items.length" class="fc-review" aria-label="Auto-tagged images to review">
<div class="fc-review__head"> <div class="fc-review__head">
<v-icon size="18" color="warning">mdi-alert-outline</v-icon> <v-icon size="18" color="warning">mdi-alert-outline</v-icon>
<span class="fc-review__title"> <span class="fc-review__title">
{{ items.length }} auto-tagged {{ items.length === 1 ? 'image' : 'images' }} {{ items.length }} auto-hidden {{ items.length === 1 ? 'image' : 'images' }}
may be real content — review may be real content — review before they stay hidden
</span> </span>
</div> </div>
<div class="fc-review__cards"> <div class="fc-review__cards">
@@ -27,16 +26,16 @@
> >
also looks like <strong>{{ it.conflict_name || 'content' }}</strong> also looks like <strong>{{ it.conflict_name || 'content' }}</strong>
</div> </div>
<div class="fc-review-card__tag">{{ tagLine(it) }}</div> <div class="fc-review-card__tag">hidden as {{ it.tag_name }}</div>
<div class="fc-review-card__acts"> <div class="fc-review-card__acts">
<button <button
type="button" class="fc-review-btn fc-review-btn--keep" type="button" class="fc-review-btn fc-review-btn--keep"
:disabled="busy.includes(keyOf(it))" @click="resolve(it, 'keep')" :disabled="busy.includes(keyOf(it))" @click="resolve(it, 'keep')"
>{{ keepLabel(it) }}</button> >Keep hidden</button>
<button <button
type="button" class="fc-review-btn fc-review-btn--unhide" type="button" class="fc-review-btn fc-review-btn--unhide"
:disabled="busy.includes(keyOf(it))" @click="resolve(it, 'unhide')" :disabled="busy.includes(keyOf(it))" @click="resolve(it, 'unhide')"
>{{ removeLabel(it) }}</button> >Un-hide</button>
</div> </div>
</div> </div>
</div> </div>
@@ -55,11 +54,6 @@ const items = ref([])
const busy = ref([]) const busy = ref([])
function keyOf(it) { return `${it.image_id}:${it.tag_id}` } function keyOf(it) { return `${it.image_id}:${it.tag_id}` }
// Chrome flags hide the image (keep-hidden / un-hide); process flags leave it
// visible and just tagged (keep-tag / remove-tag). Same endpoints, different words.
function tagLine(it) { return (it.mode === 'process' ? 'auto-tagged ' : 'hidden as ') + it.tag_name }
function keepLabel(it) { return it.mode === 'process' ? 'Keep tag' : 'Keep hidden' }
function removeLabel(it) { return it.mode === 'process' ? 'Remove tag' : 'Un-hide' }
async function load() { async function load() {
// Fetched unconditionally on mount — the strip prompts for pending misfires // Fetched unconditionally on mount — the strip prompts for pending misfires
@@ -77,8 +71,7 @@ async function resolve(it, action) {
await api.post(`/api/gallery/hidden-review/${it.image_id}/${it.tag_id}/${action}`) await api.post(`/api/gallery/hidden-review/${it.image_id}/${it.tag_id}/${action}`)
items.value = items.value.filter((x) => keyOf(x) !== k) items.value = items.value.filter((x) => keyOf(x) !== k)
if (action === 'unhide') { if (action === 'unhide') {
const verb = it.mode === 'process' ? 'Removed' : 'Un-hidden' toast({ text: `Un-hidden — “${it.tag_name}” removed; it'll train the head`, type: 'success' })
toast({ text: `${verb} — “${it.tag_name}” removed; it'll train the head`, type: 'success' })
} }
} catch (e) { } catch (e) {
toast({ toast({
@@ -0,0 +1,72 @@
<template>
<v-card>
<v-card-title>Treat as alias for</v-card-title>
<v-card-text>
<p class="text-caption mb-2">
Pick the existing tag the model's prediction should map to. All
future predictions of this name will resolve to your tag.
</p>
<v-autocomplete
v-model="selectedId"
v-model:menu="menuOpen"
:items="results"
:item-title="(t) => t.fandom_name ? `${t.name} — ${t.fandom_name}` : t.name"
:item-value="(t) => t.id"
:loading="loading"
label="Canonical tag"
no-filter clearable density="compact"
@update:search="onSearch"
@keydown.enter.capture="onEnter"
/>
</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn @click="$emit('cancel')">Cancel</v-btn>
<v-btn
color="primary" rounded="pill" :disabled="!selectedId"
@click="$emit('confirm', selectedId)"
>Use this tag</v-btn>
</v-card-actions>
</v-card>
</template>
<script setup>
import { ref } from 'vue'
import { useApi } from '../../composables/useApi.js'
import { useAcceptOnEnter } from '../../composables/useAcceptOnEnter.js'
const props = defineProps({ category: { type: String, required: true } })
const emit = defineEmits(['confirm', 'cancel'])
const api = useApi()
const results = ref([])
const loading = ref(false)
const selectedId = ref(null)
let debounce = null
// Enter on the closed dropdown confirms the selection instead of re-opening it.
const { menuOpen, onEnter } = useAcceptOnEnter(() => {
if (selectedId.value != null) emit('confirm', selectedId.value)
})
function onSearch(q) {
if (debounce) clearTimeout(debounce)
debounce = setTimeout(async () => {
const query = (q || '').trim()
if (!query) { results.value = []; return }
loading.value = true
try {
// Scope the autocomplete to the prediction's category where it
// maps to a tag kind. Only 'character' surfaces as both a
// suggestion category and a tag kind now ('artist' + 'copyright'
// retired); other categories search unscoped.
const kind = props.category === 'character' ? 'character' : null
const params = { q: query, limit: 20 }
if (kind) params.kind = kind
results.value = await api.get('/api/tags/autocomplete', { params })
} finally {
loading.value = false
}
}, 200)
}
</script>
@@ -11,6 +11,10 @@
{{ suggestion.display_name }} {{ suggestion.display_name }}
<span v-if="suggestion.rejected" class="fc-suggestion__rejected-tag" <span v-if="suggestion.rejected" class="fc-suggestion__rejected-tag"
title="You rejected this for this image — un-reject to recover">rejected</span> title="You rejected this for this image — un-reject to recover">rejected</span>
<span v-else-if="suggestion.creates_new_tag" class="fc-suggestion__new"
title="No matching tag yet — accepting creates it">+ new</span>
<span v-else-if="suggestion.via_alias" class="fc-suggestion__alias"
:title="`Mapped from the tagger's “${suggestion.raw_name}” via an alias`">alias</span>
</span> </span>
<span class="fc-suggestion__score">{{ scorePct }}</span> <span class="fc-suggestion__score">{{ scorePct }}</span>
<!-- Green / red pair (operator-asked 2026-06-28) mirrors the eval <!-- Green / red pair (operator-asked 2026-06-28) mirrors the eval
@@ -42,14 +46,40 @@
@click="$emit('dismiss', suggestion)" @click="$emit('dismiss', suggestion)"
><v-icon size="16">mdi-close</v-icon></button> ><v-icon size="16">mdi-close</v-icon></button>
</div> </div>
<!-- Modal-safe kebab is baked into KebabMenu (this row lives in the
teleported image modal — #711). Only rendered when an alias action
applies — dismiss now lives on the red ✗, so a centroid hit with no
alias option has no menu. -->
<KebabMenu
v-if="hasMenu"
class="fc-suggestion__menu" size="small" variant="outlined"
:label="`More actions for ${suggestion.display_name}`"
>
<!-- Alias is a tagger-prediction remap, so only offer it for tagger
suggestions with a raw model key that aren't already aliased.
Centroid hits (raw_name null) have nothing to alias. -->
<v-list-item
v-if="suggestion.raw_name && !suggestion.via_alias"
@click="$emit('alias', suggestion)"
>
<v-list-item-title>Treat as alias for</v-list-item-title>
</v-list-item>
<v-list-item
v-if="suggestion.via_alias"
@click="$emit('remove-alias', suggestion)"
>
<v-list-item-title>Remove alias</v-list-item-title>
</v-list-item>
</KebabMenu>
</div> </div>
</template> </template>
<script setup> <script setup>
import { computed, inject } from 'vue' import { computed, inject } from 'vue'
import KebabMenu from '../common/KebabMenu.vue'
const props = defineProps({ suggestion: { type: Object, required: true } }) const props = defineProps({ suggestion: { type: Object, required: true } })
defineEmits(['accept', 'dismiss', 'undismiss']) defineEmits(['accept', 'alias', 'remove-alias', 'dismiss', 'undismiss'])
// #1206: on hover, tell the image viewer which crop produced this suggestion so // #1206: on hover, tell the image viewer which crop produced this suggestion so
// it highlights that region. Provided by ImageViewer/Explore; a no-op elsewhere. // it highlights that region. Provided by ImageViewer/Explore; a no-op elsewhere.
@@ -62,6 +92,12 @@ function onLeave () {
} }
const scorePct = computed(() => `${Math.round(props.suggestion.score * 100)}%`) const scorePct = computed(() => `${Math.round(props.suggestion.score * 100)}%`)
// Kebab now only carries alias actions: show it when this suggestion can be
// aliased (raw model key, not yet aliased) or is already aliased (so it can be
// un-aliased). Centroid hits (no raw_name, no alias) have an empty menu → hide.
const hasMenu = computed(() =>
Boolean(props.suggestion.raw_name) || Boolean(props.suggestion.via_alias)
)
</script> </script>
<style scoped> <style scoped>
@@ -83,6 +119,26 @@ const scorePct = computed(() => `${Math.round(props.suggestion.score * 100)}%`)
color: rgb(var(--v-theme-on-surface)); color: rgb(var(--v-theme-on-surface));
overflow: hidden; text-overflow: ellipsis; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
} }
.fc-suggestion__new {
display: inline-block;
font-size: 10px; font-weight: 600;
color: rgb(var(--v-theme-accent));
background: rgba(var(--v-theme-accent), 0.12);
border: 1px solid rgb(var(--v-theme-accent), 0.4);
padding: 1px 6px; border-radius: 999px;
margin-left: 6px;
text-transform: uppercase; letter-spacing: 0.04em;
}
.fc-suggestion__alias {
display: inline-block;
font-size: 10px; font-weight: 600;
color: rgb(var(--v-theme-on-surface-variant));
background: rgb(var(--v-theme-surface-light));
border: 1px solid rgb(var(--v-theme-surface-light));
padding: 1px 6px; border-radius: 999px;
margin-left: 6px;
text-transform: uppercase; letter-spacing: 0.04em;
}
.fc-suggestion__score { .fc-suggestion__score {
flex: 0 0 auto; min-width: 38px; text-align: right; flex: 0 0 auto; min-width: 38px; text-align: right;
font-size: 11px; font-size: 11px;
@@ -110,6 +166,10 @@ const scorePct = computed(() => `${Math.round(props.suggestion.score * 100)}%`)
background: transparent; color: rgb(var(--v-theme-on-surface-variant)); background: transparent; color: rgb(var(--v-theme-on-surface-variant));
border: 1px solid rgb(var(--v-theme-on-surface-variant), 0.5); border: 1px solid rgb(var(--v-theme-on-surface-variant), 0.5);
} }
.fc-suggestion__menu {
flex: 0 0 auto;
}
/* Rejected state: the row stays put (recovery), dimmed + red-edged so it /* Rejected state: the row stays put (recovery), dimmed + red-edged so it
reads as "handled, negative" without shouting over live suggestions. */ reads as "handled, negative" without shouting over live suggestions. */
.fc-suggestion--rejected { .fc-suggestion--rejected {
@@ -26,6 +26,8 @@
v-for="(s, i) in items" :key="`${s.display_name}-${i}`" v-for="(s, i) in items" :key="`${s.display_name}-${i}`"
:suggestion="s" :suggestion="s"
@accept="$emit('accept', $event)" @accept="$emit('accept', $event)"
@alias="$emit('alias', $event)"
@remove-alias="$emit('remove-alias', $event)"
@dismiss="$emit('dismiss', $event)" @dismiss="$emit('dismiss', $event)"
@undismiss="$emit('undismiss', $event)" @undismiss="$emit('undismiss', $event)"
/> />
@@ -43,7 +45,7 @@ const props = defineProps({
collapsible: { type: Boolean, default: false }, collapsible: { type: Boolean, default: false },
defaultOpen: { type: Boolean, default: true } defaultOpen: { type: Boolean, default: true }
}) })
defineEmits(['accept', 'dismiss', 'undismiss', 'reject-all']) defineEmits(['accept', 'alias', 'remove-alias', 'dismiss', 'undismiss', 'reject-all'])
// Still-unhandled suggestions (not yet rejected) — how many "Reject rest" clears. // Still-unhandled suggestions (not yet rejected) — how many "Reject rest" clears.
const rejectableCount = computed(() => props.items.filter((s) => !s.rejected).length) const rejectableCount = computed(() => props.items.filter((s) => !s.rejected).length)
@@ -20,39 +20,48 @@
first, so false positives are easy to spot and reject (reject-to-train first, so false positives are easy to spot and reject (reject-to-train
for these heads). Small set collapsible, open by default. --> for these heads). Small set collapsible, open by default. -->
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-if="store.aboveByCategory.system && store.aboveByCategory.system.length" v-if="store.byCategory.system && store.byCategory.system.length"
label="System" :items="store.aboveByCategory.system" label="System" :items="store.byCategory.system"
collapsible :default-open="true" collapsible :default-open="true"
@accept="onAccept" @accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll('system')" @reject-all="onRejectAll('system')"
/> />
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-for="cat in peopleCats" :key="cat" v-for="cat in peopleCats" :key="cat"
v-show="store.aboveByCategory[cat] && store.aboveByCategory[cat].length" v-show="store.byCategory[cat] && store.byCategory[cat].length"
:label="labelFor(cat)" :items="store.aboveByCategory[cat] || []" :label="labelFor(cat)" :items="store.byCategory[cat] || []"
@accept="onAccept" @accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll(cat)" @reject-all="onRejectAll(cat)"
/> />
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-if="store.aboveByCategory.general && store.aboveByCategory.general.length" v-if="store.byCategory.general && store.byCategory.general.length"
label="General" :items="store.aboveByCategory.general" label="General" :items="store.byCategory.general"
collapsible :default-open="true" collapsible :default-open="true"
@accept="onAccept" @accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll('general')" @reject-all="onRejectAll('general')"
/> />
</div> </div>
<v-dialog v-model="aliasDialog" max-width="480">
<AliasPickerDialog
v-if="aliasTarget"
:category="aliasTarget.category"
@confirm="onAliasConfirm" @cancel="aliasDialog = false"
/>
</v-dialog>
</section> </section>
</template> </template>
<script setup> <script setup>
import { toast } from '../../utils/toast.js' import { toast } from '../../utils/toast.js'
import { computed, watch } from 'vue' import { computed, ref, watch } from 'vue'
import { useSuggestionsStore, CATEGORY_LABELS } from '../../stores/suggestions.js' import { useSuggestionsStore, CATEGORY_LABELS } from '../../stores/suggestions.js'
import { useModalStore } from '../../stores/modal.js' import { useModalStore } from '../../stores/modal.js'
import SuggestionsCategoryGroup from './SuggestionsCategoryGroup.vue' import SuggestionsCategoryGroup from './SuggestionsCategoryGroup.vue'
import AliasPickerDialog from './AliasPickerDialog.vue'
const props = defineProps({ const props = defineProps({
imageId: { type: Number, required: true }, imageId: { type: Number, required: true },
@@ -91,14 +100,13 @@ const peopleCats = ['character']
function labelFor(c) { return CATEGORY_LABELS[c] || c } function labelFor(c) { return CATEGORY_LABELS[c] || c }
const isEmpty = computed(() => const isEmpty = computed(() =>
Object.values(store.aboveByCategory).every(list => !list || list.length === 0) Object.values(store.byCategory).every(list => !list || list.length === 0)
) )
watch(() => props.imageId, (id) => { watch(() => props.imageId, (id) => {
if (id == null) return if (id == null) return
// One fetch (min=0) backs both surfaces: the panel reads aboveByCategory, the store.load(id) // panel: curated, ≥ threshold
// typed tag-input dropdown reads the full set. No second request. store.loadAll(id) // dropdown: full prediction set (low-confidence included)
store.load(id)
}, { immediate: true }) }, { immediate: true })
// After a successful accept/alias-accept, refresh the modal's current // After a successful accept/alias-accept, refresh the modal's current
@@ -117,6 +125,31 @@ async function onAccept(s) {
toast({ text: `Accept failed: ${e.message}`, type: 'error' }) toast({ text: `Accept failed: ${e.message}`, type: 'error' })
} }
} }
const aliasDialog = ref(false)
const aliasTarget = ref(null)
function onAlias(s) { aliasTarget.value = s; aliasDialog.value = true }
async function onAliasConfirm(canonicalTagId) {
try {
await store.aliasAccept(aliasTarget.value, canonicalTagId)
aliasDialog.value = false
await host.reloadTags()
emit('accepted')
} catch (e) {
toast({ text: `Alias failed: ${e.message}`, type: 'error' })
}
}
// Undo the model-key→tag mapping behind an aliased suggestion. The store
// reloads suggestions so the prediction reverts to its raw form; the applied
// canonical tag (if any) stays, so no tag-rail reload is needed.
async function onRemoveAlias(s) {
try {
await store.removeAlias(s)
} catch (e) {
toast({ text: `Remove alias failed: ${e.message}`, type: 'error' })
}
}
</script> </script>
<style scoped> <style scoped>
@@ -17,14 +17,7 @@
density="compact" class="fc-tag-autocomplete__list" density="compact" class="fc-tag-autocomplete__list"
> >
<template v-for="(row, idx) in rows" :key="row.key"> <template v-for="(row, idx) in rows" :key="row.key">
<!-- One unified row per DB tag (server autocomplete). Every suggestion is <!-- Existing tag match (server autocomplete). -->
a canonical tag now, so when the model ALSO scored this tag for the
image the row just carries its confidence (🔧 %) in place of the
redundant kind label the coloured leading icon already shows kind.
That means a searched tag that's also a suggestion shows its % on ONE
row, with no dedup and no flicker. Picking it emits pick-existing;
TagPanel.findPending routes a matching suggestion through accept() so
the acceptance is still recorded + it drops from the panel. -->
<v-list-item <v-list-item
v-if="row.type === 'hit'" v-if="row.type === 'hit'"
:active="idx === highlight" @click="onPickRow(row)" :active="idx === highlight" @click="onPickRow(row)"
@@ -39,18 +32,32 @@
<span v-if="row.hit.fandom_name" class="text-caption"> {{ row.hit.fandom_name }}</span> <span v-if="row.hit.fandom_name" class="text-caption"> {{ row.hit.fandom_name }}</span>
</v-list-item-title> </v-list-item-title>
<template #append> <template #append>
<span <span class="text-caption">{{ row.hit.kind }}</span>
v-if="row.sugg" </template>
class="fc-tag-autocomplete__sugg-tag" </v-list-item>
:class="{ 'fc-tag-autocomplete__sugg-tag--below': !row.sugg.above_threshold }"
:title="row.sugg.above_threshold <!-- ML suggestion for THIS image that matches the typed query. Picking
? 'The model suggests this tag for this image' it routes through the same accept path as the Suggestions panel, so
: 'The model scored this tag below its suggest threshold'" it's recorded + drops out of the panel (operator-asked 2026-06-07). -->
> <v-list-item
v-else-if="row.type === 'suggestion'"
:active="idx === highlight" @click="onPickRow(row)"
class="fc-tag-autocomplete__sugg"
>
<template #prepend>
<v-icon size="small" :color="store.colorFor(row.sugg.category)">
{{ iconFor(row.sugg.category) }}
</v-icon>
</template>
<v-list-item-title>
{{ row.sugg.display_name }}
<span v-if="row.sugg.creates_new_tag" class="text-caption">— new</span>
</v-list-item-title>
<template #append>
<span class="fc-tag-autocomplete__sugg-tag">
<v-icon size="x-small">mdi-auto-fix</v-icon> <v-icon size="x-small">mdi-auto-fix</v-icon>
{{ scorePct(row.sugg) }} {{ scorePct(row.sugg) }}
</span> </span>
<span v-else class="text-caption">{{ row.hit.kind }}</span>
</template> </template>
</v-list-item> </v-list-item>
@@ -93,10 +100,11 @@ import { useSuggestionsStore } from '../../stores/suggestions.js'
import { useInflightToken } from '../../composables/useInflightToken.js' import { useInflightToken } from '../../composables/useInflightToken.js'
import FandomPicker from './FandomPicker.vue' import FandomPicker from './FandomPicker.vue'
const emit = defineEmits(['pick-existing', 'pick-new', 'cancel']) const emit = defineEmits(['pick-existing', 'pick-new', 'accept-suggestion', 'cancel'])
// The host surface's applied tags (host.current?.tags). The dropdown filters // The host surface's applied tags (host.current?.tags). Both dropdown sections
// against this so a just-added tag drops out of the search the moment the chip // filter against this so a just-added tag drops out of the search the moment
// rail updates, not after the next modal refresh (operator-asked 2026-07-03). // the chip rail updates, not after the next modal refresh (operator-asked
// 2026-07-03).
const props = defineProps({ const props = defineProps({
appliedTags: { type: Array, default: () => [] }, appliedTags: { type: Array, default: () => [] },
}) })
@@ -230,6 +238,9 @@ const createLabel = computed(() =>
function scorePct (s) { return `${Math.round(s.score * 100)}%` } function scorePct (s) { return `${Math.round(s.score * 100)}%` }
const appliedIds = computed(() => new Set(props.appliedTags.map(t => t.id))) const appliedIds = computed(() => new Set(props.appliedTags.map(t => t.id)))
const appliedNames = computed(() =>
new Set(props.appliedTags.map(t => `${t.kind}:${(t.name || '').toLowerCase()}`)),
)
// Server matches minus the image's applied tags. allowCreate/sameNameCharExists // Server matches minus the image's applied tags. allowCreate/sameNameCharExists
// keep reading the RAW hits — they reason about the tag universe (does this tag // keep reading the RAW hits — they reason about the tag universe (does this tag
@@ -238,29 +249,49 @@ const visibleHits = computed(() =>
hits.value.filter(h => !appliedIds.value.has(h.id)), hits.value.filter(h => !appliedIds.value.has(h.id)),
) )
// This image's suggestions keyed by canonical tag id, so a matching DB-tag row // This image's suggestions that match the typed query, minus any the server
// can show the model's confidence inline. Every suggestion is a canonical tag // autocomplete already returned (same name+kind) so a tag never shows twice.
// now (tagging-v2), so the id is the join key — no name/kind matching, no dedup, // Sources the FULL prediction set (allByCategory, down to the store floor) — NOT
// no flicker. Rejected suggestions are excluded: a dismissed tag shouldn't // the threshold-filtered panel list — so a low-confidence action/feature the
// advertise a score in the type-to-add dropdown (un-reject lives in the panel). // model saw can be typed and accepted in canonical formatting instead of being
const suggByTagId = computed(() => { // hand-entered as a custom tag (operator-asked 2026-06-09). The typed query is
const m = new Map() // the only filter; the threshold no longer hides anything here.
for (const list of Object.values(suggestions.byCategory)) { const suggestionHits = computed(() => {
const q = parsedName.value.toLowerCase()
if (!q) return []
const seen = new Set(visibleHits.value.map(h => `${h.kind}:${h.name.toLowerCase()}`))
const out = []
for (const list of Object.values(suggestions.allByCategory)) {
for (const s of list || []) { for (const s of list || []) {
// Rejected suggestions now stay in allByCategory (flagged) so the panel
// can show + un-reject them; keep them OUT of the type-to-add dropdown,
// whose job is finding a tag to ADD (un-reject lives in the panel).
if (s.rejected) continue if (s.rejected) continue
if (s.canonical_tag_id != null) m.set(s.canonical_tag_id, s) // Already on the image (matched by canonical id or name+kind): nothing
// to add. Covers the window between an add and the next suggestions
// fetch, where the stale row would otherwise still be pickable.
if (s.canonical_tag_id != null && appliedIds.value.has(s.canonical_tag_id)) continue
const key = `${s.category}:${s.display_name.toLowerCase()}`
if (appliedNames.value.has(key)) continue
if (!s.display_name.toLowerCase().includes(q)) continue
if (seen.has(key)) continue
seen.add(key)
out.push(s)
} }
} }
return m // Best matches first; cap generously so a specific typed query surfaces its
// matches even when many predictions exist, while the list stays scrollable.
out.sort((a, b) => b.score - a.score)
return out.slice(0, 20)
}) })
// One ordered list backing both the rendered dropdown and keyboard nav, so the // One ordered list backing both the rendered dropdown and keyboard nav, so the
// highlight index maps 1:1 to a row. Each hit is annotated with its suggestion // highlight index maps 1:1 to a row regardless of which section it's in.
// (score) when the model scored that tag for this image.
const rows = computed(() => { const rows = computed(() => {
const r = visibleHits.value.map(h => ({ const r = visibleHits.value.map(h => ({ type: 'hit', key: `h${h.id}`, hit: h }))
type: 'hit', key: `h${h.id}`, hit: h, sugg: suggByTagId.value.get(h.id) || null, for (const s of suggestionHits.value) {
})) r.push({ type: 'suggestion', key: `s${s.category}:${s.display_name}`, sugg: s })
}
if (allowCreate.value) r.push({ type: 'create', key: 'create' }) if (allowCreate.value) r.push({ type: 'create', key: 'create' })
return r return r
}) })
@@ -282,9 +313,16 @@ function moveHighlight (delta) {
// Dispatch a chosen dropdown row by its type. // Dispatch a chosen dropdown row by its type.
function onPickRow (row) { function onPickRow (row) {
if (row.type === 'hit') { emit('pick-existing', row.hit); reset() } if (row.type === 'hit') { emit('pick-existing', row.hit); reset() }
else if (row.type === 'suggestion') { onPickSuggestion(row.sugg) }
else { onCreate() } else { onCreate() }
} }
// Picking an ML suggestion hands it to the parent, which runs the same accept
// flow as the Suggestions panel (creates the tag if raw, records acceptance,
// drops it from the panel). reset() clears the query; the panel-drop makes it
// fall out of suggestionHits reactively.
function onPickSuggestion (s) { emit('accept-suggestion', s); reset() }
function onCreate () { function onCreate () {
const name = parsedName.value const name = parsedName.value
const kind = parsedKind.value const kind = parsedKind.value
@@ -351,16 +389,14 @@ function reset () { query.value = ''; hits.value = []; highlight.value = 0 }
border: 1px solid rgb(var(--v-theme-surface-light)); border: 1px solid rgb(var(--v-theme-surface-light));
border-radius: 6px; border-radius: 6px;
} }
/* The model-confidence badge on a row whose tag the model also scored. Accent /* Mark ML-suggestion rows so they read as distinct from typed/known matches. */
when above the head's suggest threshold; muted when below (a low-confidence .fc-tag-autocomplete__sugg {
match you can still type + pick). */ border-left: 2px solid rgb(var(--v-theme-accent), 0.5);
}
.fc-tag-autocomplete__sugg-tag { .fc-tag-autocomplete__sugg-tag {
display: inline-flex; align-items: center; gap: 2px; display: inline-flex; align-items: center; gap: 2px;
font-size: 11px; font-size: 11px;
font-family: 'JetBrains Mono', monospace; font-family: 'JetBrains Mono', monospace;
color: rgb(var(--v-theme-accent)); color: rgb(var(--v-theme-accent));
} }
.fc-tag-autocomplete__sugg-tag--below {
color: rgb(var(--v-theme-on-surface-variant));
}
</style> </style>
@@ -17,6 +17,7 @@
ref="tagInputRef" ref="tagInputRef"
:applied-tags="host.current?.tags || []" :applied-tags="host.current?.tags || []"
@pick-existing="onPickExisting" @pick-new="onPickNew" @pick-existing="onPickExisting" @pick-new="onPickNew"
@accept-suggestion="onAcceptSuggestion"
/> />
<v-alert v-if="errorMsg" type="error" variant="tonal" class="mt-2" closable> <v-alert v-if="errorMsg" type="error" variant="tonal" class="mt-2" closable>
@@ -133,6 +134,7 @@ async function onRemove(tagId) {
// until the next modal open (operator-asked 2026-07-03). // until the next modal open (operator-asked 2026-07-03).
if (host.currentImageId != null) { if (host.currentImageId != null) {
suggestions.load(host.currentImageId) suggestions.load(host.currentImageId)
suggestions.loadAll(host.currentImageId)
} }
focusTagInput() focusTagInput()
} }
@@ -176,6 +178,18 @@ async function onPickNew(payload) {
} }
catch (e) { errorMsg.value = e.message } catch (e) { errorMsg.value = e.message }
} }
// A suggestion picked from the autocomplete dropdown runs the SAME path as the
// Suggestions panel's Accept: the store creates the tag if it's raw, records the
// acceptance, and drops it from the panel; then we refresh the chip rail.
async function onAcceptSuggestion(s) {
errorMsg.value = null
try {
await suggestions.accept(s)
await host.reloadTags()
focusTagInput()
} catch (e) { errorMsg.value = e.message }
}
const renameDialog = ref(false) const renameDialog = ref(false)
const renameTarget = ref(null) const renameTarget = ref(null)
@@ -70,14 +70,6 @@
@click.stop="showOriginal = !showOriginal" @click.stop="showOriginal = !showOriginal"
>{{ showOriginal ? 'Show translation' : `Show original${sourceLangLabel}` }}</button> >{{ showOriginal ? 'Show translation' : `Show original${sourceLangLabel}` }}</button>
<!-- Per-post translation override (#155): force a skipped translation on,
or keep the original for a confidently mis-flagged title. Only where
there's text to translate. -->
<PostTranslationControl
v-if="post.post_title || post.description_plain"
:post="post"
/>
<!-- Faithful (semantic) body render once expanded: backend-sanitized <!-- Faithful (semantic) body render once expanded: backend-sanitized
HTML (headings, lists, links, inline images). Collapsed and the HTML (headings, lists, links, inline images). Collapsed and the
no-detail fallback stay plain text. --> no-detail fallback stay plain text. -->
@@ -144,7 +136,6 @@ import { useModalStore } from '../../stores/modal.js'
import { usePostsStore } from '../../stores/posts.js' import { usePostsStore } from '../../stores/posts.js'
import { toPlainText } from '../../utils/htmlSanitize.js' import { toPlainText } from '../../utils/htmlSanitize.js'
import PostSeriesMenu from './PostSeriesMenu.vue' import PostSeriesMenu from './PostSeriesMenu.vue'
import PostTranslationControl from './PostTranslationControl.vue'
const props = defineProps({ const props = defineProps({
post: { type: Object, required: true }, post: { type: Object, required: true },
@@ -1,102 +0,0 @@
<template>
<!-- Sticky per-post translation override (#155). Quiet inline control under the
post title: force a skipped legit-foreign title on, or knock a wrongly
mis-translated one back to the original. The choice sticks through a
Re-translate-all. -->
<div class="fc-post-tx">
<span class="fc-post-tx__label">Translation:</span>
<v-menu :disabled="busy">
<template #activator="{ props: menuProps }">
<button
type="button" class="fc-post-tx__btn" v-bind="menuProps" :disabled="busy"
:aria-label="`Translation handling: ${currentLabel}`"
>
{{ currentLabel }}
<v-icon size="14">mdi-menu-down</v-icon>
</button>
</template>
<v-list density="compact" min-width="240" class="fc-post-tx__list">
<v-list-item
v-for="opt in OPTIONS" :key="opt.value"
:active="opt.value === current" @click="choose(opt.value)"
>
<template #prepend>
<v-icon size="small">{{ opt.icon }}</v-icon>
</template>
<v-list-item-title>{{ opt.label }}</v-list-item-title>
<v-list-item-subtitle>{{ opt.help }}</v-list-item-subtitle>
</v-list-item>
</v-list>
</v-menu>
<v-progress-circular v-if="busy" indeterminate size="13" width="2" />
</div>
</template>
<script setup>
import { computed, ref } from 'vue'
import { usePostsStore } from '../../stores/posts.js'
import { toast } from '../../utils/toast.js'
const props = defineProps({ post: { type: Object, required: true } })
const posts = usePostsStore()
const busy = ref(false)
const OPTIONS = [
{ value: 'auto', label: 'Auto', icon: 'mdi-cog-outline',
help: 'Translate only when confident enough' },
{ value: 'force', label: 'Force translate', icon: 'mdi-translate',
help: 'Always translate, even at low confidence' },
{ value: 'original', label: 'Keep original', icon: 'mdi-translate-off',
help: 'Never translate — keep the original text' },
]
const current = computed(() => props.post.translation_override || 'auto')
const currentLabel = computed(
() => (OPTIONS.find((o) => o.value === current.value) || OPTIONS[0]).label,
)
async function choose (value) {
if (value === current.value || busy.value) return
busy.value = true
try {
const res = await posts.applyTranslationOverride(props.post.id, value)
if (res.applied === 'queued') {
toast({
text: 'Saved — this post will translate on the next sweep (service offline).',
type: 'info',
})
}
} catch (e) {
toast({ text: `Couldn't update translation: ${e.message}`, type: 'error' })
} finally {
busy.value = false
}
}
</script>
<style scoped>
.fc-post-tx {
display: inline-flex; align-items: center; gap: 6px;
margin: 2px 0 6px;
}
.fc-post-tx__label {
font-size: 0.72rem;
color: rgb(var(--v-theme-on-surface-variant));
}
.fc-post-tx__btn {
display: inline-flex; align-items: center; gap: 1px;
padding: 0; background: none; border: 0; cursor: pointer;
font-size: 0.72rem; font-weight: 700;
color: rgb(var(--v-theme-on-surface-variant));
}
.fc-post-tx__btn:hover { color: rgb(var(--v-theme-accent)); }
.fc-post-tx__btn:focus-visible {
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 1px;
border-radius: 2px;
}
.fc-post-tx__btn:disabled { cursor: default; opacity: 0.6; }
.fc-post-tx__list :deep(.v-list-item-subtitle) {
font-size: 0.7rem;
}
</style>
@@ -1,127 +0,0 @@
<template>
<!-- #3068: attachment reclamation. PostAttachment's FKs are both SET NULL, so
a deleted post or artist leaves the row behind; and the store is
sha-addressed, so one blob backs many rows and deleting a row never freed
its file. Nothing swept either. Preview first, then apply (destructive:
unlinks files). -->
<MaintenanceTile
icon="mdi-paperclip-off"
title="Reclaim orphaned attachments"
blurb="Remove attachment records belonging to nothing, and the files nothing references."
destructive
:open="applying || previewing"
>
<p class="text-body-2 mb-3">
Attachment records survive the post and artist they belonged to, and the
files behind them are shared between records — so a deleted record never
freed its file on its own. This finds records attributed to
<strong>neither</strong> a post nor an artist, and files in the attachment
store that <strong>no remaining record</strong> points at.
<strong>Preview</strong> first; <strong>Apply</strong> deletes those
records and unlinks those files. Files written in the last few hours are
always left alone, so an in-progress download is never caught mid-write.
</p>
<div class="d-flex align-center flex-wrap" style="gap: 12px;">
<v-btn
color="primary" variant="tonal" rounded="pill"
:loading="previewing" :disabled="applying" @click="preview"
>
<v-icon start>mdi-magnify</v-icon> Preview
</v-btn>
<v-btn
color="error" rounded="pill"
:loading="applying"
:disabled="previewing || !canApply"
@click="confirmOpen = true"
>
<v-icon start>mdi-paperclip-off</v-icon> Apply
</v-btn>
</div>
<v-alert
v-if="summary" :type="summaryType" variant="tonal" class="mt-4"
density="comfortable"
>
<span v-if="applied">
Deleted {{ summary.rows }} orphaned record(s) and unlinked
{{ summary.files }} file(s), reclaiming {{ humanBytes(summary.bytes) }}.
</span>
<span v-else-if="hasWork">
{{ summary.rows }} orphaned record(s) and {{ summary.files }}
unreferenced file(s) — {{ humanBytes(summary.bytes) }} reclaimable.
Click <strong>Apply</strong> to remove them.
</span>
<span v-else>Nothing to reclaim — every attachment is accounted for.</span>
<!-- Both of these change what the numbers MEAN, so they are stated
whenever they are non-zero rather than hidden in a tooltip. -->
<div v-if="summary.files_failed" class="mt-1 text-caption">
{{ summary.files_failed }} file(s) could not be read or removed — see
the worker log.
</div>
<div v-if="summary.partial" class="mt-1 text-caption">
Stopped early at the time limit; some of the store was not examined.
Run it again to continue.
</div>
</v-alert>
<QueueStatusBar queue="maintenance_long" queue-label="Maintenance" />
<v-dialog v-model="confirmOpen" max-width="440">
<v-card>
<v-card-title>Reclaim orphaned attachments?</v-card-title>
<v-card-text class="text-body-2">
This permanently deletes
<strong>{{ summary?.rows ?? 0 }}</strong> attachment record(s) and
unlinks <strong>{{ summary?.files ?? 0 }}</strong> file(s)
({{ humanBytes(summary?.bytes) }}). Only files that no remaining
record points at are removed, so nothing still attached to a post
is affected.
</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn variant="text" @click="confirmOpen = false">Cancel</v-btn>
<v-btn color="error" @click="apply">Reclaim</v-btn>
</v-card-actions>
</v-card>
</v-dialog>
</MaintenanceTile>
</template>
<script setup>
import { computed, ref } from 'vue'
import { useMaintenanceTask } from '../../composables/useMaintenanceTask.js'
import { humanBytes } from '../../utils/bytes.js'
import MaintenanceTile from '../common/MaintenanceTile.vue'
import QueueStatusBar from './QueueStatusBar.vue'
const confirmOpen = ref(false)
// Walks the whole attachment store, so it can run for minutes on a large
// library — the service caps itself at 900s and reports `partial`. 150 polls
// × 2s ≈ 5m of foreground waiting; past that the composable hands off to the
// task dashboard rather than spinning forever.
const { previewing, applying, summary, applied, preview, apply: applyTask } = useMaintenanceTask({
endpoint: '/api/admin/maintenance/reclaim-attachments',
storageKey: 'fc.maint.reclaimAttachments',
appliedToast: 'Orphaned attachments reclaimed',
maxPolls: 150,
})
const hasWork = computed(
() => !!summary.value && (summary.value.rows > 0 || summary.value.files > 0),
)
const canApply = computed(() => hasWork.value && !applied.value)
const summaryType = computed(() => {
if (applied.value) return 'success'
return hasWork.value ? 'info' : 'success'
})
// The confirm dialog gates the destructive apply; close it, then run.
function apply () {
confirmOpen.value = false
applyTask()
}
</script>
@@ -9,7 +9,7 @@
<v-card-text> <v-card-text>
<p class="fc-muted text-body-2"> <p class="fc-muted text-body-2">
Pushes session cookies from supported platforms Pushes session cookies from supported platforms
(patreon, subscribestar, hentaifoundry, discord, pixiv) (patreon, subscribestar, hentaifoundry, discord, pixiv, deviantart)
into FabledCurator, and lets you add a creator as a source from into FabledCurator, and lets you add a creator as a source from
their page in one click. their page in one click.
</p> </p>
@@ -15,23 +15,28 @@
</p> </p>
<div v-for="p in proposers" :key="p.key" class="fc-proposer"> <div v-for="p in proposers" :key="p.key" class="fc-proposer">
<SettingToggleRow <div class="d-flex align-center mb-1" style="gap: 10px;">
v-model="p.on" :loading="busy" :icon="p.icon" <v-icon size="18" :color="p.on ? 'accent' : undefined">{{ p.icon }}</v-icon>
:icon-color="p.on ? 'accent' : null" :label="p.label" <span class="fc-section-h">{{ p.label }}</span>
@change="v => saveToggle(p, v)" <v-switch
/> v-model="p.on" :loading="busy" hide-details density="compact"
color="success" class="ml-auto"
@update:model-value="v => saveToggle(p, v)"
/>
</div>
<p class="fc-muted text-body-2 mb-2">{{ p.help }}</p> <p class="fc-muted text-body-2 mb-2">{{ p.help }}</p>
<div class="d-flex flex-wrap mb-4" style="gap: 12px;"> <div class="d-flex flex-wrap mb-4" style="gap: 12px;">
<v-text-field <v-text-field
v-model="p.weights" label="Weights" density="compact" hide-details v-model="p.weights" label="Weights" density="compact" hide-details
style="min-width: 300px; flex: 1;" :disabled="busy || !p.on" style="min-width: 300px; flex: 1;" :disabled="busy || !p.on"
placeholder="name | URL | hf_repo::file" placeholder="name | URL | hf_repo::file"
@change="saveField({ [`detector_${p.key}_weights`]: p.weights })" @change="save({ [`detector_${p.key}_weights`]: p.weights })"
/> />
<SettingNumberField <v-text-field
v-model="p.conf" label="Confidence" :min="0" :max="1" :step="0.05" v-model.number="p.conf" label="Confidence" type="number"
max-width="140px" :disabled="busy || !p.on" min="0" max="1" step="0.05" density="compact" hide-details
@change="saveField({ [`detector_${p.key}_conf`]: Number(p.conf) })" style="max-width: 140px;" :disabled="busy || !p.on"
@change="save({ [`detector_${p.key}_conf`]: Number(p.conf) })"
/> />
</div> </div>
</div> </div>
@@ -43,12 +48,12 @@
storage. Dedupe IoU drops near-duplicate crops before embedding. storage. Dedupe IoU drops near-duplicate crops before embedding.
</p> </p>
<div class="d-flex flex-wrap" style="gap: 12px;"> <div class="d-flex flex-wrap" style="gap: 12px;">
<SettingNumberField <v-text-field
v-for="c in caps" :key="c.key" v-for="c in caps" :key="c.key"
v-model="c.val" :label="c.label" v-model.number="c.val" :label="c.label" type="number"
:min="c.min" :max="c.max" :step="c.step || 1" :min="c.min" :max="c.max" :step="c.step || 1" density="compact"
max-width="165px" :disabled="busy" hide-details style="max-width: 165px;" :disabled="busy"
@change="saveField({ [c.key]: Number(c.val) })" @change="save({ [c.key]: Number(c.val) })"
/> />
</div> </div>
</div> </div>
@@ -56,16 +61,14 @@
</template> </template>
<script setup> <script setup>
import { toast } from '../../utils/toast.js'
import { onMounted, ref } from 'vue' import { onMounted, ref } from 'vue'
import MaintenanceTile from '../common/MaintenanceTile.vue' import MaintenanceTile from '../common/MaintenanceTile.vue'
import SettingNumberField from '../common/SettingNumberField.vue'
import SettingToggleRow from '../common/SettingToggleRow.vue'
import { useSettingSave } from '../../composables/useSettingSave.js'
import { useMLStore } from '../../stores/ml.js' import { useMLStore } from '../../stores/ml.js'
const mlSettings = useMLStore() const mlSettings = useMLStore()
const { busy, save } = useSettingSave(mlSettings.patchSettings) const busy = ref(false)
const proposers = ref([]) const proposers = ref([])
const caps = ref([]) const caps = ref([])
@@ -108,20 +111,31 @@ onMounted(async () => {
caps.value = CAP_DEFS.map(c => ({ ...c, val: s[c.key] ?? 0 })) caps.value = CAP_DEFS.map(c => ({ ...c, val: s[c.key] ?? 0 }))
}) })
// Field @change → persist with a "Saved" confirmation. SettingNumberField has async function save(patch, revert) {
// already clamped numeric values to their [min,max] before this fires. busy.value = true
function saveField(patch) { try {
save(patch, { successMessage: 'Saved' }) await mlSettings.patchSettings(patch)
toast({ text: 'Saved', type: 'success' })
} catch (e) {
if (revert) revert()
toast({ text: `Could not save: ${e.message}`, type: 'error' })
} finally {
busy.value = false
}
} }
async function saveToggle(p, v) { function saveToggle (p, v) {
// Revert the switch on failure so it never lies about the persisted state. // Revert the switch on failure so it never lies about the persisted state.
const ok = await save({ [`detector_${p.key}_enabled`]: !!v }, { successMessage: 'Saved' }) save({ [`detector_${p.key}_enabled`]: !!v }, () => { p.on = !v })
if (!ok) p.on = !v
} }
</script> </script>
<style scoped> <style scoped>
.fc-muted { color: rgb(var(--v-theme-on-surface-variant)); }
.fc-section-h {
font-size: 13px; font-weight: 700; letter-spacing: 0.03em;
text-transform: uppercase; color: rgb(var(--v-theme-on-surface));
}
.fc-proposer { .fc-proposer {
border-top: 1px solid rgb(var(--v-theme-surface-light)); padding-top: 14px; border-top: 1px solid rgb(var(--v-theme-surface-light)); padding-top: 14px;
} }
@@ -112,6 +112,7 @@ async function onCommit() {
</script> </script>
<style scoped> <style scoped>
.fc-muted { color: rgb(var(--v-theme-on-surface-variant)); }
.fc-code { .fc-code {
background: rgb(var(--v-theme-surface-light)); background: rgb(var(--v-theme-surface-light));
border-radius: 4px; padding: 2px 8px; border-radius: 4px; padding: 2px 8px;
@@ -42,7 +42,7 @@
</tr> </tr>
</tbody> </tbody>
</v-table> </v-table>
<p v-else class="text-caption mt-3 fc-muted"> <p v-else class="text-caption mt-3" style="opacity: 0.6;">
No table statistics yet. No table statistics yet.
</p> </p>
</MaintenanceTile> </MaintenanceTile>

Some files were not shown because too many files have changed in this diff Show More