bvandeusen/FabledCurator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 33 Wiki Activity
436 Commits 3 Branches 33 Tags
1803a09306d22fb089517bde3377ad7bcba6500a
Commit Graph

24 Commits

Author SHA1 Message Date
bvandeusen 0f7cd3cb76 fix(migration-0022): pre-merge colliding Posts before the bulk reparent — Postgres fires uq_post_source_external_id row-by-row during UPDATE, so the post-reparent merge-collisions step never ran (operator's v26.05.26.1 deploy hit it: 'duplicate key (source_id, external_post_id)=(42, 6166997)'). Detect (keep, drop) Post pairs whose external_post_id already exists under canonical, merge the drop into keep, then bulk-reparent the rest cleanly. 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 16:48:49 -04:00
bvandeusen 0bc5767a2b fix(importer): Source = one per (artist, platform), not one per post — filesystem importer's sidecar paths now reuse the artist's existing subscription Source (or create one synthetic anchor with enabled=False) instead of fabricating a new Source per post URL. Alembic 0022 consolidates existing per-post Sources to canonical (prefers campaign URL; falls back to sidecar:<platform>:<slug>) and re-parents Posts + ImageProvenance, merging Post collisions. 
Operator-flagged 2026-05-26: Atole artist detail page showed 406 Sources where 1 was right.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 14:19:29 -04:00
bvandeusen 397021dcbd fix(importer): ImageProvenance (image_record_id, post_id) race-safe via savepoint + alembic 0021 UNIQUE — closes the SELECT-then-INSERT window that planted duplicates and broke .scalar_one_or_none() on every later deep-scan rederive (MultipleResultsFound). Migration dedupes existing rows (min(id) per pair); model gains __table_args__; gallery-filter test that seeded duplicates dropped. 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 13:47:55 -04:00
bvandeusen 929d3fc092 feat(fc-cleanup): migration 0020 + LibraryAuditRun model — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-26 01:17:42 -04:00
bvandeusen e6ededbe8e feat(deep-scan): IR-parity port — refreshed status + counter, re-queue completed paths in deep mode, honest UX — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 2026-05-25 22:16:26 -04:00
bvandeusen e43312a129 fc3h: ImportSettings backup_* knobs + alembic 0018 (nightly-enabled, hour, keep-N per kind) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:52:00 -04:00
bvandeusen 8f2732a56f fc3h: alembic 0017 — backup_run table with indexes + partial-tag 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 22:51:22 -04:00
bvandeusen 0cda46fcdb fc3i: alembic 0016 — task_run table with single + composite indexes 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-24 21:00:46 -04:00
bvandeusen da9c19b2dc fc5: migration 0015 + MigrationRun model (kind/status as String(32), API-validated) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-22 08:11:42 -04:00
bvandeusen 5f2ebd66b7 fc3d: migration 0014 — source.consecutive_failures + import_settings scheduling knobs 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-21 15:58:30 -04:00
bvandeusen 159d4cb046 feat(fc3c): migration 0013 — download_event.metadata + import_settings downloader fields 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 20:36:19 -04:00
bvandeusen 2a18016b69 feat(fc3b): migration 0012 + AppSetting model 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 18:33:14 -04:00
bvandeusen e623e97be2 feat(fc3b): migration 0011 — credential schema aligned with GS wire fields 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 18:32:52 -04:00
bvandeusen a13a5878eb feat(fc3a): migration 0010 — unique(source.artist_id, platform, url) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 12:53:34 -04:00
bvandeusen 8f69478227 feat(attachments): PostAttachment model + 0009 (table + batch counter) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-19 11:10:53 -04:00
bvandeusen 63e7185811 feat(provenance): image_record.artist_id + 0008 backfill/drop-artist-tags 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 21:33:11 -04:00
bvandeusen 15dac50367 feat(post): add nullable description + attachment_count (migration 0007) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 09:38:43 -04:00
bvandeusen 9ef5e5047d feat(phash): import_settings.phash_threshold (migration 0006) exposed + validated in API 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-17 22:04:13 -04:00
bvandeusen b83baa7d7f feat(series): SeriesPage model + 0005 migration (unique image, ordered pages) 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 23:33:12 -04:00
bvandeusen a638c469e0 feat(fc2c-i): migration 0004 enables tsm_system_rows 2026-05-15 15:48:29 -04:00
bvandeusen 906804140c feat(fc2b): schema migration 0003 — ML pipeline tables 
Renames image_record.wd14_* -> tagger_* (we're on Camie now, not WD14).
Adds tag_allowlist (auto-apply opt-in, per-tag confidence),
tag_suggestion_rejection (per-image dismissals), tag_alias (composite
(string, category) -> canonical tag, resolved at read time),
tag_reference_embedding (per-tag SigLIP centroids), and the ml_settings
singleton (per-category + centroid thresholds, model version pins).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-15 07:35:22 -04:00
bvandeusen 9eb1614fbf feat(fc2a): schema migration 0002 — tag kinds + fandom hierarchy + import lifecycle 
Adds Tag.kind enum (artist/character/fandom/general/series/archive/post/meta/rating),
Tag.fandom_id FK with CHECK constraint (only valid for kind='character'), and a
kind-aware uniqueness index so the same name can exist across kinds and the same
character name can exist in different fandoms.

Adds ImportBatch + ImportTask state-machine tables for scan tracking, plus a
single-row ImportSettings table (CHECK id=1) holding the importer's filter knobs.

Adds image_record.integrity_status column (defaults to 'unknown'); FC-2e
populates this via the integrity verifier.

Drops the unused tag.namespace column from FC-1 — superseded by kind.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 12:03:41 -04:00
bvandeusen d6a156dcd2 feat: define unified schema (Artist, Source, Post, ImageRecord, etc.) and initial migration 
Implements the data model from spec §3 in one go so FC-2/FC-3 don't need
schema-adding migrations of their own. Artist is the unified entity for
both gallery 'artist:' tags and GallerySubscriber Subscriptions
(is_subscription flag). ImageProvenance is many-to-one, enabling the
enrich-on-duplicate rule for downloaded content that pHash-matches an
existing record.

The SigLIP embedding column uses pgvector(1152) for SigLIP-so400m;
swapping models in FC-2 will require a column-width migration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 07:34:29 -04:00
bvandeusen a03655039c feat: add SQLAlchemy declarative base, Alembic environment, and gitignore fix 
Configures stable constraint naming so autogeneration produces clean diffs.
Alembic uses the sync psycopg driver while the runtime app uses asyncpg.

Also fixes a .gitignore bug caught during this task: the bare 'models/'
rule for the ML weights volume was matching backend/app/models/ (Python
package). Anchored all volume rules to repo root (/images/, /import/,
/downloads/, /models/, /postgres_data/, /redis_data/).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 07:32:58 -04:00