b14818303cf4af1f92d6fff2bdcaf09267ce488c
375
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bvandeusen
|
5a0e1bbd03 |
perf(ml): batch the auto-apply sweeps' image_tag inserts (#3072)
Item 1 of #3072. Both sweeps issued a single-row pg_insert(image_tag) from inside their per-image loop. Steady state that is nothing; a first sweep over a back-catalogue is one round-trip per applied tag, tens of thousands of them. Each chunk now collects its rows and writes them in one statement. The ticket suggested one insert per chunk PER TAG. A single multi-row VALUES carries every tag at once, so it is one statement per chunk full stop — and the sweeps already accumulate across all heads before they commit, so nothing had to be restructured to allow it. Not a new helper: wip_title.apply_wip_image_tags was already doing the chunked ON CONFLICT DO NOTHING insert, so that shape is extracted to services/image_tag_apply.insert_image_tags and all three writers share it. The extraction deliberately leaves wip_title's pre-SELECT behind rather than pulling it into the shared function — the sweeps don't need it (their `skip` sets already exclude applied and rejected images) and it exists only to produce an accurate count, which the sweeps also compute themselves. So the shared primitive returns nothing: psycopg reports rowcount -1 for a multi-row ON CONFLICT DO NOTHING insert, and a count taken from the statement would be a lie rather than an approximation. Ordering note for the system-tag sweep: tag rows are now written after that chunk's PresentationReview rows rather than interleaved before them. Safe — PresentationReview FKs to image_record and tag, not to image_tag. Chunk size stays 5000: 5000 rows x 3 bound params = 15000, inside Postgres' 65535-parameter ceiling with room to spare. tests/test_image_tag_apply.py covers the primitive directly, since it is now the single place three writers can be wrong at once — most importantly that a re-run never restamps a hand-applied tag's source, which would silently poison head training (it excludes the auto sources). Left alone: _insert_presentation_review is still per-row, and the retract path still deletes per-row. Both operate on sets that are small by construction, unlike the apply path. Refs #3072 |
||
|
bvandeusen
|
1ac448d881 |
refactor: four small cleanups from the review pass (#3072)
Items 2-5 of #3072. Item 1 (the per-row sweep inserts) is separate. 2. .fc-bad was not merely duplicated — it is .fc-weak under a second name. Both local definitions were `color: rgb(var(--v-theme-error))`, identical to the global .fc-weak, and GpuAgentCard was already using .fc-weak to colour exactly what GpuActivityPanel coloured .fc-bad (an errored count, red when non-zero). So rather than promoting a synonym to app.css, both call sites now use .fc-weak and the local defs are gone. app.css's status-colour comment records why there is no .fc-bad, next to the existing note on why .fc-ok is deliberately NOT global. 3. GalleryItem.vue's obsidian literals now use --v-theme-background, which IS obsidian (vuetify-theme.js maps background -> surfaces. obsidian). Preferred over --fc-chrome-rgb: same value, but that variable is named for the nav fade, not for the palette entry. The ticket said these were the only three real uses in the tree. They are not — GalleryItem itself had two more in the artist-label gradient (fixed here, so the file is now consistent), and ~13 more live in SeriesView, SeriesReaderView, ImageViewer, ArtistHeader, ExploreView and GalleryFilterBar. Those are a separate sweep, filed rather than folded in here. 4. The attachment download path had two hand-formatted copies. One definition now, `attachment_download_url`, next to the model both serializers already import. The test pins it by MATCHING the built path against the app's real URL map rather than comparing to a literal — a string-equality test would still pass after someone renamed the route, which is the drift the helper exists to prevent. 5. Extension API key now compares with hmac.compare_digest. Compared as BYTES, not str: compare_digest's str form raises TypeError on non-ASCII, and this value comes straight from an attacker-controlled header, so the str form would turn a junk key into a 500 instead of a 403. Low stakes either way — the API is unauthenticated-by-design on a LAN — but it costs nothing. Refs #3072 |
||
 bvandeusenandClaude Opus 5
|
ddf896078c |
refactor(platforms): retire deviantart end-to-end (#3069)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
CI / frontend-build (push) Successful in 23s
extension / lint (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m43s
Executes the 2026-07-05 product decision (FC downloaders = art-dedicated
services only), which removed Twitter/X and Bluesky but left deviantart
fully wired for seven weeks — the half-retired state rule 22 exists to
prevent.
Removed: the PlatformInfo module and its registry entry, the gallery-dl
extractor block, extension_service's artist-page pattern, the extension's
PLATFORMS + PLATFORM_ARTIST_PATTERNS entries, its manifest host permission
and content-script match, the frontend icon/colour/label, and the operator-
facing "supported platforms" list that still advertised it.
Two judgment calls, both recorded in migration 0088:
* existing `source` rows are DISABLED, not deleted. The row is the only
record of the artist's DeviantArt URL. Disabling is also required for
correctness rather than tidiness: with the platform unregistered the
download path falls through to gallery-dl, which carries its OWN
deviantart extractor, so an enabled row would have kept downloading
from a dropped platform.
* the `credential` row IS deleted — a live session cookie for a site FC
will never call again.
Adds the invariant whose absence is why manifest.json drifted in the first
place: nothing tied its domain lists back to the platform table. The
extension suite now asserts both directions, plus that no host permission
belongs to an unclaimed domain (`*://*/*` exempted — FC is self-hosted at
an operator-chosen URL the extension cannot enumerate).
Extension version 1.0.10 -> 1.0.11: ci.yml's guard hard-fails a packaged
extension change without a bump. No release is cut — build.yml's
sign-extension job only runs on main.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
bvandeusenandClaude Opus 5
|
2e0f8f8c61 |
feat(cleanup): reclaim orphaned attachments — rows and store blobs (#3068)
PostAttachment's two FKs are both ON DELETE SET NULL, so a deleted post or artist left the row behind rather than taking it. Nothing ever pruned those rows, and nothing in the repo had ever unlinked a file under the attachment store — so both rows and bytes accumulated permanently, invisible to every existing diagnostic. Why a disk->DB reconciliation rather than a row sweep: the store is sha-addressed and idempotent, so ONE blob backs MANY rows. Deleting a row does not free its blob, and since the artist cascade (#3066) now deletes its attachment rows outright, a freed blob has no DB pointer left to find it by. Walking the store and asking "does any row still reference this sha?" catches orphans from every cause, including ones no future delete path will think to report. Preview and apply share `_orphan_attachment_conditions` (rule 93). The dry-run derives its surviving-sha set by NEGATING that same predicate, so it is honest about blobs the delete would free rather than counting them as still-referenced — the one place this was easy to get backwards, so it has its own parity test. Guards, each with a reason: - A blob is written before its row commits, so a just-stored file legitimately has no referencing row. Files under 6h are never judged — same guard and reasoning as ORPHAN_TEMP_MIN_AGE_HOURS. - `.partial` staging files belong to cleanup_orphaned_temp_files; skipped rather than raced. - The sha is parsed as the first 64 chars, not via Path.stem: store() takes the extension from the source filename, and a URL-encoded basename yields a multi-dot suffix that would make stem eat part of the sha. - A 900s walk budget reports partial=True instead of running to the task's hard limit (rule 89). - TASK_STUCK_THRESHOLD_MINUTES override at 30 (= time_limit 25 + 5). Without it a healthy 20-minute walk is phantom-flagged 'RecoverySweep' at the bare 5-min default — the #883 failure class; its invariant test is mirrored here. Defaults to the safe preview at both the task and the route, unlike the other maintenance triggers: this apply unlinks files. Operator-triggered only, never on a beat. Ships with its UI (rule 27): AttachmentReclaimCard in Cleanup → Duplicates & leftovers, built on the existing useMaintenanceTask/MaintenanceTile shapes, so a run survives navigating away. Surfaces files_failed and partial explicitly, since both change what the numbers mean. Also promotes humanBytes to utils/bytes.js — it was byte-identical in VideoDedupCard and GatedPurgeCard and this card would have been the third copy. The three divergent `formatBytes` helpers are deliberately left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5
|
2ce467e347 |
fix(cleanup): artist cascade preview counts posts and attachments (#3067)
`project_artist_cascade` is documented as "a read-only projection of what delete_artist_cascade would touch" and drives the Tier-C confirm dialog, but it counted only images, sources, thumbs, import_tasks and bytes. It never counted posts or attachments, both of which the apply destroys. That is silent in the worst case. `Post.artist_id` is ondelete=CASCADE, so every post goes whether or not it carried an image — and FC has a large body-only post population (#1288 measured 694 pixiv posts with text and no images). Such an artist previewed as `images: 0`, reading as "empty, safe to remove", while the apply destroyed every captured body, description, external-link set and raw_metadata snapshot. The danger-zone card already promised "every image, source, post, and attachment" — the copy was honest and the numbers were not. Root of the drift: the preview re-derived its own predicates instead of sharing the apply's, the same shape as the 2026-06-08 fandom-tag deletion that rule 93 exists for. So rather than bolt on two counts, both halves now build from shared `_artist_{images,posts,attachments}_conditions` helpers, following the `_unused_tag_conditions` / `_bare_post_conditions` style already in the file. Sources keep no helper — the apply doesn't query them either, it gets them from the Artist.sources ORM cascade. The apply also now reports `posts_deleted` (counted before the delete, since the CASCADE leaves nothing to count after). Rule 93's second half asks for the apply to be tested, and parity is only assertable if both halves state the number. Adds a preview/apply parity test that runs both against one artist and asserts the three pairs agree AND that the rows actually went, plus a body-only-artist test covering the case that motivated this. The confirm dialog's counts grid renders every key, so posts and attachments surface there automatically; the prose summary line names posts explicitly, since that is the number that changes how an artist reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 5
|
39cf81aea6 |
fix(cleanup): clear an artist's attachments before the cascade delete (#3066)
`delete_artist_cascade` could abort partway through, and it aborted after the irreversible half. Deleting an artist CASCADEs to Post (post.artist_id is ondelete=CASCADE), which SET NULLs post_attachment.post_id — and `uq_post_attachment_null_post_sha` is a partial UNIQUE on sha256 ALONE WHERE post_id IS NULL. So any two of that artist's attachments sharing a sha collapse onto one another and raise. That shape is ordinary, not corrupt: `_capture_attachment` deliberately writes one row per post over a single sha-addressed blob, so a creator who attaches the same pdf to two posts already has two such rows. A pre-existing filesystem-import row (post_id NULL) with the same sha collides on its own. The images and their on-disk files are deleted and committed in 500-row batches BEFORE the artist row is touched, so the failure landed after them: images gone, artist and posts alive, files unrecoverable. Fix: delete the artist's post_attachment rows explicitly first, matched by artist_id OR by the owning post's artist (artist_id is nullable, so neither arm alone covers every row). `_repoint_post_links` already guards the identical collision class in the reconcile path; this is its artist-cascade counterpart. Migration 0043 reasoned only about upgrade-time safety and never about this later SET NULL. The sha-addressed blobs are deliberately left on disk: one blob backs many rows, so unlinking needs a refcount pass, and this Tier-C op must not delete bytes its own preview never disclosed. Adds `attachments_deleted` to the summary, and two regression tests — the same sha on two posts, and an unrelated NULL-post row that must survive. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
099e1e664c |
refactor(patreon): DRY the campaigns-API request (#161)
_lookup_via_api and resolve_display_name shared ~90% of their body (same endpoint, params, headers, error handling — differing only in which field they pluck from data[0]). Extract _campaigns_api_first(vanity, cookies_path) -> dict|None; callers pluck the campaign id vs the display name. Return-value behavior preserved (the display-name path additionally gains the helper's more granular warning logs). Covered by the existing test_patreon_resolver.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi |
||
|
bvandeusenandClaude Opus 4.8
|
666b3a2ec8 |
refactor(ml): DRY pass — shared sweep helpers + table-driven settings (#161)
Consolidate duplication accrued across the ML tagging + settings backend, behavior-preserving (over-DRY guard applied — the three auto-apply sweep BODIES stay separate; only their shared inner helpers are extracted). - _sigmoid / _conflict_scores / _insert_presentation_review (heads.py): the score→prob transform (6 inlined sites), the presentation conflict signal (2 sites), and the ring-loud PresentationReview insert (2 sites, single- sourced so the mode column can't drift on the shared composite PK). - _applied_or_rejected (training_data.py): the per-tag "applied ∪ rejected" skip-set, byte-identical at 3 sweep sites (heads.py x2, tasks/ml.py ccip). - ccip sweep divergence fixes: import ccip._FIGURE_KINDS + training_data._l2norm instead of local copies that silently drift when the canonical changes. - MLSettings.load / .load_sync classmethods (mirror ImportSettings); route all 8 scalar_one singleton reads through them (the session.get None-path stays). - GET serializers for MLSettings + ImportSettings are now table-driven off the same _EDITABLE tuples PATCH writes, so a new field can't be silently absent from GET (the split that historically dropped fields). - AUTO_APPLY_THRESHOLD_MIN/MAX constant single-sources the [0.5,0.999] operating range across the service clamp + the 5 API validators. - test_ml_dry_helpers.py pins _applied_or_rejected + _sigmoid. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi |
||
|
bvandeusenandClaude Opus 4.8
|
69b5637bd6 |
fix(extension): show Add-to-FC button on subscribed Patreon creators (#1485)
CI / lint (push) Successful in 3s
extension / lint (push) Successful in 11s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m47s
extension / lint (pull_request) Successful in 9s
Symptom (operator-flagged): the extension injected the Add-as-source button on a Patreon creator you had NOT subscribed to, but it disappeared once you were subscribed — the opposite of when it's useful. Root cause (extension logic, not Patreon security): Patreon serves a creator under three URL shapes — bare patreon.com/Atole, patreon.com/c/Atole, and patreon.com/cw/Atole (the 'creator workspace' URL you land on once subscribed; documented in patreon_resolver._VANITY_RE). The button's artist-page gate (PLATFORM_ARTIST_PATTERNS.patreon in platforms.js) and its byte-mirror probe pattern (_PLATFORM_PATTERNS in extension_service._derive) only matched the bare single-segment form and explicitly excluded c/. So the subscribed-view URL failed the gate → no button. The ingestion resolver already handled all three; only these two gates were too narrow. Fix: both regexes now accept optional cw/ and c/ prefixes and drop the strict single-segment end-anchor, so a creator's inner page (/cw/Atole/posts, /Atole/membership) also matches — robust to whatever exact shape the subscribed view uses. Nav-page exclusions (home/search/messages/notifications/library/ settings/posts + post permalinks) preserved. New unit test covers all three prefixes, sub-paths, and nav-page rejection (both regexes validated identically). Bump extension 1.0.7→1.0.8 so a fresh signed XPI ships the fix (also exercises batch-5 web-ext-10's AMO sign path end-to-end on the main build). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
fac5ae6ce5 |
feat(explore): reach dial to escape dense clusters + anti-revisit (#1476)
The Explore walk got stuck in dense signatures — neighbours all too similar, so forward-arrow couldn't escape and Random was the only exit. Root cause: MMR only diversifies WITHIN the nearest ~400 pool; in a dense cluster that whole pool is near-identical, so there's no escape route in it. - gallery_service.similar(reach=0.0, exclude_ids=None): reach>0 widens the pool (cap 400→1000) and _reach_sample strides across an outward-growing distance span so the set handed to MMR spans near→mid-far (guaranteed escape routes), not just the tight cluster. exclude_ids drops already-walked images. Gallery 'more like this' (reach=0) is unchanged. - api/gallery similar: parse reach + exclude_ids. - explore store: default reach 0.4 (auto-diversifies without touching the dial), pass the breadcrumb as exclude_ids, setReach action. - ExploreView: a Near↔Far reach slider in the trail. - tests: _reach_sample math (deeper ranks with higher reach, near kept); similar exclude_ids drops walked + reach path runs clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
af0d39ed52 |
feat(wip): soft title tier — sketch/doodle vocab + ring-loud audit (#1474)
Extends WIP title-tagging to lower-precision cues (sketch/doodle/scribble) safely. - wip_title.py: soft matcher (word-anchored; sketchbook/kadoodle don't trip it); WIP_TITLE_SOFT_SOURCE + soft SQL prefilter; apply_wip_image_tags takes a source arg. - training_data._AUTO_SOURCES += 'wip_title_soft' → the soft tier is PROVISIONAL and never trains the wip head (a finished "sketch" can't pollute it). Only the hard tier (wip_title) + manual train. - ImportSettings.wip_soft_title_tagging_enabled (OFF by default, opt-in). Migration 0087. - importer: hard tier wins, soft is the fallback (source wip_title_soft). - backfill: refactored into a shared _backfill_wip_tier; hard always, soft when enabled. - heads.soft_wip_conflict_audit + daily beat: score soft-tagged images against content heads, flag ring-loud ones (PresentationReview mode=process) for the review strip — the operator's "measure if they got falsely tagged" safety. - api settings toggle; ImportFiltersForm soft toggle. - tests: soft matcher pos/neg; soft source not a training positive; audit flags ring-loud + spares quiet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
ad2a5fc5fe |
feat(system-tags): process vs chrome groups + WIP provisional auto-apply (#1464)
Backend for the system-tag behavior refactor (milestone #157). editor screenshot moves from chrome (hidden) to the PROCESS group (shown, like wip); wip+editor gain provisional auto-apply so they stop needing endless manual identification — without a runaway loop. - tag.py: split PRESENTATION_SYSTEM_TAGS → CHROME_SYSTEM_TAGS (banner) + PROCESS_SYSTEM_TAGS (wip, editor screenshot). - heads.py: generalize presentation_auto_apply_sweep → system_tag_auto_apply_sweep (mode chrome|process). Same Guard 1 (skip human/confirmed) + Guard 2 (ring-loud conflict → PresentationReview). process mode uses source 'process_auto' and does NOT hide (hide is a gallery-query effect of group membership). - training_data._AUTO_SOURCES += 'process_auto' → the head never trains on its own auto-applied output; only wip_title/manual train it (the runaway break). - ml_settings: process_auto_apply_enabled (OFF, opt-in) + threshold + conflict threshold. presentation_review.mode ('chrome'|'process'). Migration 0086. - gallery_service: default-hide reads CHROME only (editor now shows); Explore neighbors exclude the whole PROCESS group. - tasks/ml + celery beat: scheduled_process_auto_apply (daily, opt-in); prune covers both modes. - api: ml_admin process_* CRUD+validation; hidden-review returns mode. - tests: rename chrome sweep calls; new test_process_auto_apply (apply, guards, mode flag, no-self-train); gallery test asserts editor now visible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
0da0e47784 |
fix(wip-title): count inserts via pre-SELECT, not driver rowcount
apply_wip_image_tags relied on result.rowcount, but psycopg reports -1 for a multi-row INSERT ... ON CONFLICT DO NOTHING (executemany path), so the return count (and the backfill's reported total) was wrong. Compute the count from a pre-SELECT of already-tagged ids within the same transaction; keep ON CONFLICT DO NOTHING as a race-safety belt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
503c8854bc |
style(importer): sort wip_title import into the local-import block
Ruff I001 fixup for
|
||
|
bvandeusenandClaude Opus 4.8
|
571938781a |
feat(tagging): title-based WIP auto-tagging (#1458)
Auto-apply the `wip` system tag to posts whose TITLE explicitly declares
work-in-progress ("WIP" / "work in progress") — a deterministic, high-precision
complement to the image-based ML `wip` head. WIP images are excluded from the
Explore/gallery browse, so honouring the artist's own label keeps unfinished
pieces out of the main browse.
- services/wip_title.py: precision-first token-anchored matcher (swipe/wiped
never trip it) + sync apply helpers (source='wip_title', ON CONFLICT DO
NOTHING, chunked under the psycopg param ceiling).
- importer: live hook on FRESH import only (never on deep-scan/supersede), so a
manually-removed WIP tag is never re-applied by a routine re-scan.
- maintenance.backfill_wip_title_tags: operator-triggered back-catalogue sweep
(coarse SQL prefilter + regex confirm, keyset-paginated). Deliberately NOT a
beat — a periodic re-run would silently undo manual removals.
- ImportSettings.wip_title_tagging_enabled (default ON, migration 0085) gating
the live hook; GET/PATCH + POST /settings/wip-title/scan.
- Settings UI: toggle + "Scan existing posts" button.
- Tests: pure matcher unit tests + integration (apply idempotency, backfill
precision).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusenandClaude Opus 4.8
|
aea2701c28 |
feat(translation): tunable acceptance floor (0.90) + per-post sticky override (#155)
The gate at a fixed 0.80 couldn't catch the real pain: Interpreter (fresh == cached, verified by probe) confidently mis-detects short ASCII English like "... WIP Part 1" as German at 0.86 — above the floor — so it was accepted and a re-translate reproduced it. Confidence alone can't separate the 0.86 collision (genuine German lands there too), and single-word mis-flags sit at a confident 1.0 no floor catches. Two operator-approved levers: - Acceptance floor is now a live Settings value (ImportSettings. translation_min_confidence, default 0.90; surfaced in the Translation card), so it's tunable without a redeploy. _accept takes the threshold as a parameter. - Per-post sticky override (Post.translation_override: auto/force/original). 'force' stores a translation even below the floor (rescue a skipped legit-foreign title); 'original' keeps the original and clears any stored translation (kill a confident mis-flag no floor catches). The sweep honors it on every run and _reset_translations skips 'original', so the choice survives a Re-translate-all. POST /api/posts/<id>/translation-override applies it immediately (translate now when the service is up, else queue for the sweep). UI: PostTranslationControl on the posts-feed card. Migration 0084 (both columns + a CHECK on the override). The feed + provenance serializers expose translation_override. With a stricter floor the rollback finally works: raise it -> Re-translate all -> the 0.86 mis-flags are rejected and restored to the original; force / keep-original handle the residual either way. Tests: gate thresholds against the param (0.86 rejected at 0.90, explicit-floor cases); sweep force/original + re-translate-skips-original; override endpoint (validation, original clears, force queues when disabled, feed exposes it); settings min_confidence default/save/validate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy |
||
|
bvandeusenandClaude Opus 4.8
|
a444cf82d1 |
refactor(tags): unify suggestion source — one canonical DB-tag dropdown, drop dead raw/alias machinery (#154)
Every tag suggestion is a canonical DB tag now (tagging-v2 #114: heads + CCIP score EXISTING concept tags). The pre-heads apparatus for model-predicted tags that didn't exist in the DB — creates_new_tag / raw_name / via_alias, the /suggestions/alias endpoint + add_alias_and_accept, AliasPickerDialog, and the store's aliasAccept/removeAlias — was dead and is removed. The type-to-add dropdown was TWO row sources (server autocomplete + the image's ML suggestions) merged with a dedup that dropped the %-bearing suggestion row when the debounced server hit landed — the operator's "confidence % flickers then vanishes". Now it's ONE list of DB-tag matches, each annotated with the model's confidence (join by canonical_tag_id) when the tag was scored for this image. No dedup, no flicker; picking a suggested tag still records acceptance via TagPanel.findPending. Single per-image fetch: score_image now reports above_threshold per row (computed vs the head's own suggest cut, separate from the inclusion floor), so the rail makes ONE min=0 request and derives the panel (above_threshold) and the dropdown (all, text-filtered) client-side — the two /suggestions calls collapse to one. Manual "Create 'X' as <kind>" (novel typed names) is unchanged; the alias table + tag-side alias admin + auto-apply alias matching are untouched. Tests: gate/serializer assertions updated (above_threshold; dropped dead-field + alias-endpoint checks); frontend spec seeds via the single load and covers the byCategory/aboveByCategory split. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy |
||
|
bvandeusenandClaude Opus 4.8
|
ffdbdbaf07 |
feat(translation): 8h sweep + drain button + detection probe (#1376)
Throughput: translate_posts now runs every 8h (was daily) as the steady-state cadence for newly-imported posts, and the Settings "Translate now" button runs it in drain mode (run-until-done, no reset) so one press clears the whole untranslated backlog instead of a single 300-post chunk. The interrupt/backoff re-enqueue now preserves the drain flag so a bulk drain resumes cleanly after an Interpreter restart. Misdetection groundwork: surface the detector's confidence from the Interpreter client (it was in the detectedLanguage payload but discarded) and add a read-only "Test translation" box — POST /settings/translation/ probe + TranslationCard UI — that shows detected language + confidence + engine + result for pasted text, without saving. Lets the operator see why a short/abbreviation-heavy English title gets mis-detected so the detection guard (min-length + confidence floor) can be tuned from real numbers. The guard itself follows once the mis-detected cases are probed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy |
||
|
bvandeusenandClaude Opus 4.8
|
9eae636047 |
feat(translation): pooled Interpreter session + manual sweep resumes after drain
- interpreter_client: shared requests.Session with a connect-only retry (connect=2, no status retries — we map 429/5xx ourselves) so a proxy reload is smoothed and the keep-alive connection is pooled across the sweep. - translate_posts: on an interrupt (drain), re-enqueue after the Retry-After hint / default backoff instead of waiting for the daily beat; self-terminates via the health gate. Steady-state one-chunk-per-run on success is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
f8105046dc |
feat(explore): exclude WIP-tagged work from the Explore rabbit-hole
Explore's neighbour grid (/api/gallery/similar → gallery_service.similar) now takes an Explore-only exclude_wip flag that drops `wip` system-tagged images from the candidates, alongside the banner/editor presentation tags. The gallery's own "similar" button is unchanged (keeps wip, #1274) — only the Explore store passes exclude_wip=1. The anchor itself may still be a WIP; only neighbours are filtered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
d631ed023c |
style(translation): use datetime.UTC alias (ruff UP017)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
c64261593d |
feat(ops): graceful shutdown — worker stop-grace + Interpreter drain resilience
Deploys (docker SIGTERM→SIGKILL, default 10s) were killing Celery jobs mid-flight. Give in-flight work room to drain and make interrupted work resume cleanly instead of stalling. - docker-compose.yml: stop_grace_period per lane (web 30s / worker 90s / scheduler 60s / maintenance-long 180s / ml-worker 120s) so warm shutdown can actually drain before SIGKILL. - celery_app.py: task_reject_on_worker_lost=True — a task killed past the grace window is re-queued (safe: idempotent + chunked, recovery sweeps re-drive stragglers). - interpreter_client.py: map 429/5xx (502/503/504) → InterpreterUnavailable and parse Retry-After (delta-seconds or HTTP-date); a draining Interpreter behind a reverse proxy no longer raises an opaque HTTPError. - translation.py: thread retry_after out of _translate_batch; retranslate_posts resumes after the Retry-After hint (or 60s default, capped 900s) on an interrupt with _reset_done=True, self-terminating via the health gate. - tests: 429/5xx mapping + Retry-After parse; interrupt-resume + default backoff. No migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusenandClaude Opus 4.8
|
ead60978e3 |
feat(translation): expose translated fields in post feed + provenance payloads (#143 step 4)
post_feed_service (card + detail) and provenance_service._post_dict now include post_title_translated, description_translated (card-truncated / detail-uncapped) and translated_source_lang, keeping the originals for the toggle. Feed serialization test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
7f8073c4c8 |
feat(translation): Interpreter client — batch translate + health (#143 step 2)
services/interpreter_client.py: sync (requests) client for the LibreTranslate- compatible /v1/translate — no new dep, mirrors the platform clients. translate() maps translatedText[]↔texts (order + length), returns detected_lang + engine_version (aggregate = first item, fine for a per-post [title, description] batch); passthrough items come back unchanged in their slot. InterpreterUnavailable on 503 / connection error (retry later), InterpreterBadRequest on 400. health() checks /v1/health engines.llm. 10 unit tests with mocked HTTP. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
eedf8d109a |
feat(ml): presentation-chrome auto-hide sweep + hard-skip + conflict flagging (#141 step 4)
presentation_auto_apply_sweep fires banner/editor-screenshot heads at the FLAT presentation threshold (source=presentation_auto). Two guards: (1) hard-skip any image already carrying a human/confirmed content tag — you valued it, so the model can't bury it; (2) if an auto-hide ALSO scores >= presentation_conflict_threshold on a content head, hide it but record a PresentationReview row (conflict tag + score) for the Hidden view. _auto_apply_heads now excludes system tags, so a graduated wip/banner can't fire via the content path (and wip never auto-applies at all). presentation_auto added to _AUTO_SOURCES so auto-hidden chrome never self-trains. Tests: applies, hard-skip valued, conflict-flag, disabled no-op, ignores wip, content-path excludes system. Settings UI + scheduling land next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
e86b91dfe2 |
feat(gallery): default-hide presentation chrome (banner/editor screenshot) (#141 step 1)
The default gallery + facets now implicitly exclude images carrying a presentation system tag (banner / editor screenshot), reusing the tag-scope EXISTS machinery. Suppressed when the operator explicitly filters FOR a presentation tag OR passes include_hidden (the Hidden view — step 2). `wip` is NOT hidden (real, in-progress art). include_hidden threaded through scroll/timeline/jump_cursor/facets + the gallery API _parse_filters. Test covers default-hide, include_hidden, explicit-filter-shows, and wip-stays-visible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
18bb25f140 |
fix: ruff C416 (dict() over comprehension) + frontend test playlistIds rename
- heads.py: conf_map = dict(conf) instead of a dict comprehension (ruff C416). - postCard.spec.js: the modal-playlist rename (postImageIds→playlistIds) missed this frontend test (grep was src-only); update the expected call args. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
bae077e323 |
feat(ml): CCIP references exclude unconfirmed auto character tags + confirm trips detectors (m139)
Completes "no self-training": unconfirmed auto-applied character tags no longer seed CCIP references — character_references + the prototype builder (_current_fingerprints/_rebuild_one) gain a shared _positive_char_tag filter (human-applied OR operator-confirmed), mirroring the head-positive exclusion. Confirming a tag also has to move the change-detectors, or an incremental refresh/Retrain right after a confirm wouldn't fold the tag in (only the nightly full pass would): the CCIP global gate now counts character confirmations, and the head training fingerprint counts confirmations. Test for the CCIP path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
775941609d |
feat(ui): confirm/keep auto-applied tags (milestone 139)
Auto-applied tags are provisional (they don't train the model + can be retracted until confirmed), so surface and confirm them: - Backend: list_for_image + get_image_with_tags now include `source` + a `confirmed` flag on each applied tag (via serialize_tag, image-scoped; defaulted for autocomplete/directory callers). - Frontend: TagChip badges an unconfirmed auto-tag with an "auto" pill + a one-click Keep/confirm (✓) → POST /images/<id>/tags/<id>/confirm, which promotes it to a training positive and shields it from the retraction sweep; TagPanel reloads so the badge + button drop once confirmed. Contract test for the source/confirmed payload. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
2d44a26bdf |
feat(ml): auto-applied tags don't train a head unless confirmed (milestone 139)
Makes auto-apply truly "soft" for heads: _ids_with_tag (head positives) and _eligible_tag_ids (graduation count) now count human-applied + operator-confirmed tags only, via a shared _AUTO_SOURCES (head_auto/ccip_auto/ml_auto) exclusion. Unconfirmed auto-applied tags no longer train the head that judges them, so a misfire can't reinforce itself and the retraction sweep can actually drop it. Confirming a tag (TagPositiveConfirmation) promotes it to a positive AND protects it from retraction. sklearn-free tests. CCIP reference exclusion is the companion piece, next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
3006e84cc0 |
feat(ml): soft auto-apply — retract auto-tags now below threshold (milestone 139)
Daily scheduled_retract_auto_tags re-scores standing auto-applied tags and drops the ones the model no longer supports: - retract_auto_applied_heads: per graduated head, re-score its source='head_auto' images (bounded — only the images already carrying the auto-tag, not the whole library) and remove ones now < auto_apply_threshold. - retract_auto_applied_ccip: per source='ccip_auto' character tag, max-cosine the image's figure vectors vs that character's prototypes; remove ones now below the ccip auto-apply threshold. Both SKIP operator-confirmed tags (TagPositiveConfirmation) and are SILENT — a low score isn't proof the tag was wrong, so no hard negative is recorded (that's reserved for an operator removal). No-op unless the relevant auto-apply switch is on. New daily beat. sklearn-free tests for both paths + the disabled no-op. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
2cfbb284d5 |
feat(heads): incremental retraining — refit only changed tags (#1317 phase 2, m138)
train_all_heads is now incremental by default: a per-tag training-data fingerprint (positive + rejection count/latest-timestamp, stored on tag_head.train_fingerprint) means a manual Retrain refits ONLY the tags whose data changed — O(what you touched), not O(all heads). The nightly scheduled_train_heads passes full=True to reconcile sampled-negative + hygiene drift across every head. First incremental run after deploy still refits everyone (NULL fingerprints), stamping them, then it's incremental. The refit decision + fingerprint are split into sklearn-free helpers (_head_fingerprints, _heads_needing_retrain) so the incremental logic is unit-tested directly (train_head itself needs scikit-learn). Migration 0080. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
a94f6a2789 |
feat(ccip): matcher reads the incremental prototype store (#1317, m138 step 4)
match_image now sources character references from character_prototype via a per-character in-process cache (_load_prototypes) that reloads ONLY the characters whose ccip_prototype_state.updated_at advanced — no request-path rebuild, so the per-accept ~4s stall is gone once the store is populated. Cold start (store empty pre-first-refresh) falls back to the legacy on-the-fly reference build, so character suggestions work immediately post-deploy and the background refresh populates the store within ~15 min. Match math + grounding are unchanged; existing tests exercise the legacy fallback, and a new test covers matching from the populated prototype store. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
9504870c9a |
feat(ccip): incremental character-prototype builder (#1317, m138 step 2)
refresh_character_prototypes (sync, celery ml worker): - Cheap GLOBAL gate (a few COUNTs) → no-op when nothing that affects references changed since the last refresh (the operator's "only recompute if something was tagged" trigger). - Else a per-character fingerprint diff (one GROUP BY: ref count + max region id) rebuilds ONLY the characters whose references moved — each capped to MLSettings.ccip_prototype_cap — and drops characters that lost all refs. Cost scales with WHAT changed, not library size. Reuses ccip's reference predicate (single-character, non-hygiene, figure CCIP) so prototypes match the legacy matcher exactly. The async matcher (next step) will READ the table. Tests: gate no-op when idle, only-changed-character rebuild, capping, single-character exclusion, lost-reference cleanup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
8f6547f8d7 |
refactor(subscriptions): remove the dead backfill dry-run preview (#1281)
PreviewDialog.vue was orphaned — nothing mounted it — so the dry-run preview was unreachable. Rather than wire it up, remove the whole chain: its unique value (a capped 3-page count of what a backfill would grab) is low, and its adjacent needs are already covered — auth validation by verify_source_credential, and actually fetching recent posts by "Check now". Operator decision 2026-07-06. Removed: - frontend: PreviewDialog.vue + sources store previewSource() - backend: POST /api/sources/<id>/preview route, download_backends.preview_source, IngestCore.preview() + its now-unused NativeIngestError import - tests: the 3 ingester preview tests Nothing else referenced the chain (verified). Shared campaign-resolution and ledger helpers stay — they're used by run()/verify_source_credential. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
9bb4211722 |
feat(ui): hover an applied tag chip → highlight its grounding crop (#133 step 4)
Applied tags aren't scored live, so compute the grounding on demand: run the
tag's head over the image's max-over-bag (whole-image + concept crops), argmax
→ the region that best explains the tag on this image, mirroring what
score_image records for live suggestions.
- heads.py: extract _image_bag (now shared by score_image) + ground_applied_tag.
Returns (grounding, has_head): has_head False = no head to localize with →
no overlay; grounding None = the whole-image vector won → whole-image frame.
- tags.py: GET /api/images/<id>/tags/<id>/grounding → {grounding, has_head}.
- TagChip/TagPanel: applied chips inject fcSuggestionHover and fetch grounding
on hover (cached per image+tag, race-guarded), reusing Step 3's overlay in
both the modal and Explore. No new frontend overlay code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
|
||
|
bvandeusenandClaude Opus 4.8
|
dfe2fda564 |
feat(ml): CCIP character matches ground to the matched figure region (#133 step 2)
match_image now tracks WHICH query figure produced the winning cosine per
character (argmax over the per-figure best-reference sim) and attaches its bbox as
grounding {bbox,kind:'figure',detector}. SuggestionService carries it: a CCIP-only
character hit grounds to its figure; a 'both' hit keeps the head's localized crop
if it had one, else falls back to the CCIP figure — so corroborated characters
stay grounded. Test: a character match carries the matched figure's bbox+kind.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
|
||
|
bvandeusenandClaude Opus 4.8
|
409724b981 |
feat(ml): argmax grounding in score_image → suggestions carry the winning crop (#133 step 1)
score_image now keeps the ARGMAX beside the max-over-bag: which bag row won each
head. The region query also selects bbox/kind/detector_version, a parallel
bag_meta maps each row → its region (None for the whole-image vector), and every
hit gains grounding {bbox,kind,detector} (null when the global vector won). Threaded
through SuggestionService (new Suggestion.grounding field) → /api/.../suggestions
payload. This is the data the #1206 hover-overlay draws. CCIP-only hits ground null
for now (figure grounding = step 2). Tests: winning crop grounds the tag with its
bbox+kind; whole-image win → grounding None.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
|
||
|
bvandeusenandClaude Opus 4.8
|
0963bf0db3 |
feat(artist): resolve patreon + subscribestar display names at add-time (#130 step 5)
Parity with pixiv (operator ask): the extension add now resolves the real display name for our other native platforms too, not just the URL handle. patreon_resolver.resolve_display_name reads the campaigns API's attributes.name; SubscribeStarClient.resolve_display_name pulls the creator name off the profile page (og:title, else the <title> stripped of the SubscribeStar suffix). extension_service._resolve_artist_name dispatches per platform (pixiv=token, patreon/subscribestar=cookies via get_cookies_path), best-effort in an executor, falling back to the readable URL handle on any failure. Still all curator core — the extension is unchanged (sends only the URL). gallery-dl platforms keep the handle (readable, no native client). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
0c4b8aef8c |
feat(artist): pixiv display-name at add-time + identity-by-source (#130 steps 2+3)
Final piece of the artist decoupling. (1) Identity-by-source: quick_add_source resolves the artist by an existing (platform, url) Source first, so a re-add reuses the artist even after it was renamed (its frozen slug no longer matches the name) — a slug-based lookup would have duplicated it. (2) Pixiv naming: a new pixiv source resolves the real display name via the app API (PixivClient .resolve_display_name → /v1/user/detail) using the stored token, so the artist is 'Kurotsuchi Machi' not '12345678' — and its name-derived slug matches what a native download produces, unifying them. Falls back to the numeric id when no token/crypto. ExtensionService gains the crypto seam; the endpoint passes it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
a69bd1baa8 |
feat(artist): move a source into another artist (#130 step 4)
Operator ask: a surface to merge new sources into existing artists (consolidate the singleton artist a fresh add spins up). Enabled by the #130 slug decoupling — the storage path is immutable, so re-attribution moves NO files. SourceService .reassign moves the source, re-points its posts (Post.source_id==S) and the images it contributed (ImageProvenance via S, scoped to the old artist so shared images aren't stolen), and deletes the old artist if it's left fully empty (else clears its subscription flag). POST /api/sources/<id>/reassign. Frontend: a 'Move…' action per source on the artist Management tab → artist-autocomplete picker → confirm → routes to the target (whose slug is stable). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
87d53db0cb |
feat(artist): editable display name + rename surface; drop name-uniqueness (#130 step 1)
First step of decoupling artist identity/storage/display. migration 0077 drops uq_artist_name so the display name is free text (two genuinely different creators can share a name); the slug stays the immutable, unique storage/identity key (the on-disk path component — untouched, so nothing moves). ArtistService.rename + PATCH /api/artists/<id> change the name ONLY. Frontend: inline pencil-edit on the artist header (mirrors TagCard), slug/route unaffected so no navigation. Fixes the operator's 'no surface to rename an artist' + the name-collision fragility. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
8838b325fb |
fix(recapture): link on-disk images to their post (#1288)
Recapture disk-skips already-downloaded media, and upsert_post_record only writes Post fields — so a pre-existing image (e.g. one pulled under the old gallery-dl path, imported bare with no post) stays orphaned even after its post record is (re)written. Confirmed on the operator's instance: 329 pixiv images with primary_post_id NULL, 694 pixiv posts with content but no linked images, 0 duplicate posts. Fix: the recapture relink channel now carries the media's post_id (2- → 3-tuple path/url/post_id), and phase 3 calls importer.link_existing_image_to_post — match the on-disk image by path, find its Post by (source, external_post_id), upsert image_provenance + primary_post_id. Factored the provenance-linking out of _apply_sidecar into a shared _attach_provenance so the fresh-import and recapture-backlink paths can't diverge. Idempotent; generic across native platforms (no-op for already-linked Patreon/SubscribeStar). Re-running recapture now repairs orphaned images; future walks never orphan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
437bf4d37a |
feat(suggestions): group wip/banner/editor under a separate 'system' category
System tags are kind=general, so their suggestions previously landed in the General group. Give them their own 'system' suggestion category so the operator reviews them apart from content tags: _current_heads maps is_system heads to category 'system' (still trained as general heads, still gated by the 0.65 floor). Frontend: CATEGORY_ORDER/LABELS gain 'system'; SuggestionsPanel renders a 'System' group first (small, collapsible, open — false positives easy to spot and reject); the typed-dropdown shows the shield icon for system entries. Safe: system-tag suggestions always carry a canonical_tag_id, so the create-by-kind path (which would send 'system' as a TagKind) is never hit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
f33808b977 |
fix(pixiv): capture ugoira frame timings in the post record (ordering bug)
The core writes the post record BEFORE extract_media, but the ugoira frame delays were only memoized DURING extract_media — so write_post_record never saw them and ugoira_frames was always empty in the record. Extract a memoized _ugoira_meta (frames + zip url share ONE /v1/ugoira/metadata call regardless of order) and inject client.fetch_ugoira_frames into the downloader (mirrors Patreon's content_fetcher) so write_post_record populates the frames itself. Zero extra API calls — the fetch is shared/memoized with extract_media. A recapture now backfills the timings onto existing ugoira posts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
6c6e8bdb6d |
feat(heads): surface system-tag suggestions at a flat 0.65 confidence floor
System tags (wip/banner/editor) already get heads (kind=general) and aren't filtered from suggestions, but they surfaced only at each head's precision-tuned suggest_threshold — high enough to hide the borderline/false-positive guesses the operator wants to SEE and REJECT (hard-negative mining: 'negatively reinforce what isn't a system tag'). score_image now uses a flat _SYSTEM_TAG_SUGGEST_FLOOR (0.65, operator-set) for system-tag heads instead of their auto threshold; content-tag heads keep their own, and the typed-dropdown threshold_override still overrides everything. _current_heads carries Tag.is_system into the head meta to drive it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
c9696a2faf |
fix(subscribestar): route .art creators to .adult; clear source failure on disable
Two pre-merge fixes: 1. SubscribeStar .art age wall: the 18+ cookie doesn't clear the age gate on the .art domain (keeps 302'ing to /age_confirmation_warning even with the cookie — Elasid #54116), but the same creator is reachable on .adult where the cookie works. _normalize_ss_host rewrites subscribestar.art → subscribestar.adult at request time (stored Source.url untouched), logged so it's visible in walk logs. .com/.adult pass through. 2. Disabling a source now clears its failure state (last_error, error_type, consecutive_failures) so subs you pause (not paying for) stop lingering as 'failing'. Only the explicit disable clears — an unrelated edit to an already-disabled source leaves state alone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Opus 4.8
|
544e30bfb9 |
fix(pixiv): match gallery-dl's exact on-disk filename to avoid re-download at cutover
The native downloader used the Windows-safe sanitize_segment, but gallery-dl on Linux (path-restrict auto→'/', path-remove default control chars, path-strip auto→'') replaces ONLY '/' and deletes control chars — the Windows-forbidden set (<>:"|?*) and trailing dots/spaces stay RAW in on-disk titles. Any pixiv title with those chars would therefore miss the tier-2 disk-skip and re-download the whole work at cutover (seen-ledger starts empty). Replace sanitize_segment with gdl_clean_filename, a byte-exact mirror of gallery-dl 1.32.5 build_filename (verified against path.py). Directory + template already matched; this closes the last parity gap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Fable 5
|
0bbcdee3bd |
feat(pixiv): flip dispatch to the native ingester (#129 step 4)
pixiv joins NATIVE_INGESTER_PLATFORMS: download/verify/preview and the recover/recapture UI actions now route through PixivIngester. Campaign id is parsed straight from the source URL (numeric user id — no network resolver), with a platform-aware resolution-failure message. auth_token now rides the uniform adapter construction (token platforms use it, cookie platforms accept-and-ignore), and the preview endpoint fetches/threads it. The legacy gallery-dl pixiv path is fully removed (PLATFORM_DEFAULTS entry + the refresh-token config branches in download/verify) per no-legacy policy; gallery-dl keeps hentaifoundry/discord/deviantart until they migrate/retire. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusenandClaude Fable 5
|
0563b2d750 |
feat(pixiv): ledger models + migration 0076 + PixivIngester adapter (#129 step 3)
pixiv_seen_media / pixiv_failed_media mirror the Patreon/SubscribeStar ledgers (keys are always synthesized <illust_id>:p<num> / <illust_id>:ugoira — pximg URLs carry no content hash). PixivIngester wires client/downloader/ ledgers into ingest_core with drift label 'Pixiv app API' and the new body_canary=False opt-out: caption-less pixiv artists are common, so the zero-bodies #862 alarm would false-positive here — the client's response-shape drift checks cover that failure class instead. auth_token joins the uniform adapter constructor (pixiv is the first token-auth native platform). verify_pixiv_credential = one OAuth refresh, no feed walk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |