Commit Graph

73 Commits

Author SHA1 Message Date
bvandeusen bb3fb5145a feat(provenance): /api/provenance image + post endpoints
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:07:06 -04:00
bvandeusen 0de8eaa4c6 feat(provenance): ProvenanceService.for_post (post-header payload)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:06:36 -04:00
bvandeusen 491bdcb943 feat(provenance): ProvenanceService.for_image (peer provenance rows)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:06:15 -04:00
bvandeusen 3b5f894435 feat(provenance): nh3 HTML sanitizer for scraped post descriptions
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 18:05:45 -04:00
bvandeusen 9d433fc1ab fix(test): sidecar idempotency needs phash_threshold=0 (orthogonal splits collapse at default)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 15:13:48 -04:00
bvandeusen 9b79b66b76 feat(sidecar): importer creates Source/Post/ImageProvenance from sidecars (idempotent, fail-soft)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 14:46:17 -04:00
bvandeusen ef7e4d92a9 feat(sidecar): minimal gallery-dl sidecar find + parse util
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 14:44:37 -04:00
bvandeusen 15dac50367 feat(post): add nullable description + attachment_count (migration 0007)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 09:38:43 -04:00
bvandeusen 3f2f50cdb9 fix(test): threshold test needs structured images (solid colors phash-collapse)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:25:48 -04:00
bvandeusen 7f55f360a3 fix(test): phash 'far' case needs structured images — solid colors phash-collapse
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:18:07 -04:00
bvandeusen 4d5564c363 feat(phash): import task maps 'superseded' to complete + re-queues ML/thumbnail
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:06:46 -04:00
bvandeusen a510378603 feat(phash): importer near-dup dispatch + in-place supersession (clear ML)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:05:43 -04:00
bvandeusen 9ef5e5047d feat(phash): import_settings.phash_threshold (migration 0006) exposed + validated in API
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:04:13 -04:00
bvandeusen d2e8dd3a3b feat(phash): compute_phash + find_similar util (hash_size=8, fits existing column)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:02:42 -04:00
bvandeusen 129af2e594 feat(series): list_pages returns full image_url for the reader
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 16:28:20 -04:00
bvandeusen a9e7ee9ba9 feat(series): TagService.merge repoints series_page.series_tag_id
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:36:04 -04:00
bvandeusen d8e585a76e feat(series): /api/series pages/add/remove/reorder/cover routes
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:35:12 -04:00
bvandeusen 0db76c70e8 feat(series): SeriesService list/add(move)/remove/reorder/cover with guards
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:34:17 -04:00
bvandeusen b83baa7d7f feat(series): SeriesPage model + 0005 migration (unique image, ordered pages)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:33:12 -04:00
bvandeusen 35e26d6226 fix(bulk): node-safe toast (globalThis.window) + common-tags test matches non-empty spec guard
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 18:15:23 -04:00
bvandeusen f946918cd2 feat(bulk): SuggestionService.for_selection consensus + POST /api/suggestions/bulk
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 18:03:16 -04:00
bvandeusen e1c8400c87 feat(bulk): BulkTagService (common_tags/bulk_add/bulk_remove) + 3 image bulk-tag routes
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 18:02:01 -04:00
bvandeusen 93b6fc21c8 fix(tags): invalid 'copyright' kind 500 — KINDS uses real enum 'fandom'; directory 400s on bad kind
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 15:51:27 -04:00
bvandeusen 98c9fb9402 fix(tests): read column values not stale ORM entities after Core-DML merge (MissingGreenlet)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 13:48:50 -04:00
bvandeusen 305687dc33 feat(api): rich 409 merge hint and POST tags merge endpoint with backfill enqueue
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 13:40:37 -04:00
bvandeusen 8fbeabe9ba feat(tags): protective alias creation per observed tagger category with kind fallback
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 13:39:16 -04:00
bvandeusen 33efba8bf5 feat(tags): merge repoints allowlist, embedding, aliases, fandom children
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 13:38:27 -04:00
bvandeusen 003181ed41 feat(tags): merge repoints image_tag and tag_suggestion_rejection with dedup
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 13:36:13 -04:00
bvandeusen e0a455466d feat(tags): TagService.merge guards + MergeResult + repoint stubs
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 13:35:26 -04:00
bvandeusen 96e6003b66 feat(tags): TagMergeConflict + provenance helper, enrich rename collision
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 13:34:25 -04:00
bvandeusen dfa6a5c895 fix(integration): lazy seed-snapshot (unblock unit job); isolate recover test from eager import 2026-05-15 22:43:46 -04:00
bvandeusen 7a896605e0 fix(integration): snapshot/restore seeded singletons, dispose app engine per test 2026-05-15 22:35:37 -04:00
bvandeusen 13be9085b5 fix(integration): asyncio auto-mode, per-test truncation, artist images IN-subquery 2026-05-15 22:23:32 -04:00
bvandeusen a4c7d44780 fix(fc2c-i): ruff I001 one-per-line import in test_artist_service 2026-05-15 16:27:55 -04:00
bvandeusen e43c2a0dd0 feat(fc2c-i): artist overview + paged images endpoints 2026-05-15 15:52:02 -04:00
bvandeusen 8484cb9aaa feat(fc2c-i): tag directory service and /api/tags/directory 2026-05-15 15:50:41 -04:00
bvandeusen a74b313596 feat(fc2c-i): showcase random-sample service and endpoint 2026-05-15 15:49:25 -04:00
bvandeusen a638c469e0 feat(fc2c-i): migration 0004 enables tsm_system_rows 2026-05-15 15:48:29 -04:00
bvandeusen fc33c23dd5 refactor(fc2c-i): single shared async engine with lifecycle dispose 2026-05-15 13:53:42 -04:00
bvandeusen 4623551bf6 fix(fc2b): ruff import sort in test_ml_tagger.py
ruff isort orders Tagger before TagPrediction (its case-insensitive
tiebreak). One-char autofix; ruff check now clean across backend/tests/alembic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 10:07:38 -04:00
bvandeusen aa4cc7c629 feat(fc2b): add tag rename (PATCH /api/tags/<id>) + /api/ml admin
Rename returns 409 on collision (frontend shows FC-2c merge hint), 400
on empty. ml_admin: GET/PATCH settings (thresholds + min_reference;
model versions read-only here), POST backfill / recompute-centroids
returning 202 + celery task id. Tests integration-marked, eager Celery.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:54:12 -04:00
bvandeusen 46338a1f2e feat(fc2b): add /api/allowlist and /api/aliases endpoints
Allowlist: list-all, get-one (404 if not listed), PATCH threshold
(range-validated), DELETE. Aliases: list-all (with canonical name),
create (idempotent, 201), DELETE by (string, category). Tests
integration-marked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:46:38 -04:00
bvandeusen 07961dfe04 feat(fc2b): add /api/images/<id>/suggestions endpoints
GET (ranked, category-grouped), accept (→ apply + allowlist; enqueues
retro-apply when newly added), alias (create alias + accept canonical),
dismiss (per-image rejection). Thin blueprint over SuggestionService +
AllowlistService. Tests marked integration, eager Celery.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:45:55 -04:00
bvandeusen b68a382b60 feat(fc2b): importer enqueues tag_and_embed + ml-worker model self-heal
import_media_file now enqueues tag_and_embed alongside generate_thumbnail
after a successful import. scripts/download_models.py snapshots Camie +
SigLIP into /models, idempotent (skips when present). The ml-worker
entrypoint runs it before starting the Celery worker so a fresh /models
volume self-heals on first boot. Downloader tests are pure-logic (no
network in CI).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:45:09 -04:00
bvandeusen 3e6cc8fffa feat(fc2b): add allowlist-apply + centroid recompute tasks + beat
apply_allowlist_tags: 4 modes (tag-only / image-only / both / full sweep),
matches a tag to a prediction either by direct name or via alias
(name, category) resolution, gates on per-tag min_confidence, skips
applied/rejected, applies source='ml_auto'. recompute_centroid /
recompute_centroids: async-bridged calls into CentroidService, delta-gated.
Beat: daily backfill, daily centroid recompute, daily allowlist sweep.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:43:39 -04:00
bvandeusen ac7e0d13bc feat(fc2b): add tag_and_embed + backfill Celery tasks
tag_and_embed: Camie + SigLIP on one image (video → 10-frame sample,
max-pool tags, mean-pool embeddings), stores predictions/embedding with
model versions, then enqueues per-image allowlist apply. backfill:
keyset-paginated discovery of images missing predictions/embeddings for
the current model versions (restart-safe). apply_allowlist_tags stub
included so .delay() resolves between commits (filled in Task 9).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:40:27 -04:00
bvandeusen ed92548c0f feat(fc2b): add AllowlistService + TagService.rename
AllowlistService: accept (apply ml_accepted + add to allowlist + clear
rejection; returns whether newly-added so API can kick retro-apply),
add_alias_and_accept, dismiss, reject_applied_tag (remove + record
rejection so the allowlist won't re-apply), threshold update, remove,
list_all.

TagService.rename: refuses on (name, kind, fandom_id) collision with a
message pointing at FC-2c merge. Tests marked integration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:39:33 -04:00
bvandeusen 7860b86a13 feat(fc2b): add SuggestionService — alias-resolved, threshold-filtered, ranked
The read path: load tagger_predictions, drop unsurfaced categories
(rating/meta/year), apply per-category thresholds, batch-resolve aliases,
skip applied + rejected, augment with centroid hits above the similarity
threshold, merge duplicate signals (take max score, mark source 'both'),
group by category, sort by score DESC. Tests marked integration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:38:33 -04:00
bvandeusen dfa67d6437 feat(fc2b): add CentroidService — per-tag SigLIP centroids + similarity
recompute_for_tag (mean of member embeddings, eligible-kind + min-refs
gated, upsert), list_drifted (the delta-gate: member-count mismatch OR
missing OR wrong model version), find_similar_tags (pgvector cosine
distance, similarity = 1 - distance). Tests marked integration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:37:53 -04:00
bvandeusen 03c6a61673 feat(fc2b): add AliasService — (name, category) -> canonical tag
resolve() / resolve_many() (batch, used by the suggestion read path),
idempotent create, remove, list_all. Category-scoped so 'naruto' as
character vs copyright map to different canonicals. Tests marked
integration (real DB).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:37:13 -04:00