Compare commits

...

35 Commits

Author SHA1 Message Date
bvandeusen 4e82208926 Merge pull request 'v26.05.26.5 — extension CORS unblock + UI gap closes + CI workflow cleanup' (#27) from dev into main 2026-05-26 20:15:07 -04:00
bvandeusen 85b640f32e fix(views): close the 24-32px gap below TopNav across all views — every v-container had py-6 (or py-8 on PlaceholderView) which pushed the first content item well below where the TopNav's fade-to-transparent gradient bottoms out. Switch to pt-2 pb-6 (8px top, 24px bottom) so content sits comfortably right below the nav, matches the ArtistHeader's 'continuous with TopNav' feel. PlaceholderView uses pt-3 pb-8 keeping its larger bottom padding.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 19:47:23 -04:00
bvandeusen c7001f4aed fix(extension): CORS preflight for moz-extension:// + chrome-extension:// origins — operator-flagged 2026-05-26 that the extension's Test connection returned NetworkError because /api/credentials POSTs with X-Extension-Key trigger a browser preflight OPTIONS that hit a 405 (no OPTIONS method registered) with no Access-Control-Allow-* headers. Adds two app-level hooks: before_request short-circuits OPTIONS from extension origins with 204, after_request stamps the necessary ACL headers on responses to extension-origin requests. Whitelist is intentionally narrow (extension schemes only) so normal browser usage doesn't get permissive CORS. Five integration tests pin the contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 19:43:31 -04:00
bvandeusen f827612930 fix(artist-header): close gap below TopNav (top:64px → 48px to match TopNav's actual ~48px height) + center the tab strip via 1fr|auto|1fr layout with a right-side spacer cell
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 18:36:19 -04:00
bvandeusen 3f0153cba5 ci(workflows): dedupe + versioned image tags
ci.yml: drop pull_request: trigger — push: branches: [dev, main] already covers it; pull_request was duplicating ci.yml runs on every dev push with an open PR. (No fork PRs in this repo.)

build.yml: drop dev from push triggers — operator doesn't use the :dev image. Add tags: ['v*'] trigger + tag-push branch in the Determine-tag logic so cutting a release tag publishes an immutable :v26.05.26.X image (rollback story) without re-publishing :latest. Extend the XPI-download step to fire on tag pushes too so the versioned image carries the signed extension.

Net per hotfix cycle: 5 runs → 3 (no tag) / 4 (with tag).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 18:26:56 -04:00
bvandeusen 52fff00353 Merge pull request 'v26.05.26.4 — hotfix: migration 0022 pre-DELETE colliding ImageProvenance before UPDATE' (#26) from dev into main 2026-05-26 18:06:20 -04:00
bvandeusen f3e8f30a8f fix(migration-0022): pre-DELETE colliding image_provenance rows before the UPDATE post_id — same row-by-row UNIQUE pattern as the post-collision case, just one level deeper. When image X has provenance under both keep and drop, UPDATE drop→keep would fire uq_image_provenance_image_post on the row that'd collide with the existing (X, keep). Pre-delete those rows (their info is already represented by the keep-side provenance) before the UPDATE moves the rest.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 18:05:49 -04:00
bvandeusen eee107766e fix(migration-0022): rename unused _epid loop var (ruff B007)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 18:03:11 -04:00
bvandeusen c14338cbce Merge pull request 'v26.05.26.3 — hotfix: migration 0022 pre-merge across ENTIRE (canonical+others) group' (#25) from dev into main 2026-05-26 17:52:59 -04:00
bvandeusen 7a64730bd2 fix(migration-0022): pre-merge ALL duplicate-external_post_id Posts across the (canonical+others) group, not just canonical-vs-others — operator's v26.05.26.2 deploy still tripped uq_post_source_external_id because two non-canonical Sources both had Posts with epid=6166997. Bulk UPDATE moved the first cleanly then collided on the second. New pre-merge groups all Posts in the (artist, platform) by external_post_id; for any group with count>1, picks the keep (prefer one under canonical; else lowest id) and merges the rest before the bulk reparent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 17:52:29 -04:00
bvandeusen 1803a09306 ci(workflow): remove the 4 Cache pip wheels steps entirely — act_runner's cache backend has been broken for 11+ days and the cached path (~/.cache/pip) wasn't even the primary install tool's cache anyway (uv uses ~/.cache/uv). Net cost ~30s/job of wheel downloads. Long-term: mount ~/.cache/uv as a docker volume at the runner level (skips actions/cache entirely) or fix the runner-side cache backend.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 16:54:05 -04:00
bvandeusen 8c36dd28b0 Merge pull request 'v26.05.26.2 — hotfix: alembic 0022 Post-collision pre-merge + ci.yml cache continue-on-error' (#24) from dev into main 2026-05-26 16:50:43 -04:00
bvandeusen 0f7cd3cb76 fix(migration-0022): pre-merge colliding Posts before the bulk reparent — Postgres fires uq_post_source_external_id row-by-row during UPDATE, so the post-reparent merge-collisions step never ran (operator's v26.05.26.1 deploy hit it: 'duplicate key (source_id, external_post_id)=(42, 6166997)'). Detect (keep, drop) Post pairs whose external_post_id already exists under canonical, merge the drop into keep, then bulk-reparent the rest cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 16:48:49 -04:00
bvandeusen 7b0dd4182c ci(workflow): continue-on-error on Cache pip wheels — act_runner's cache backend has been broken since 2026-05-15 and now hard-fails ('Cannot find module .../dist/restore/index.js') instead of warning. Install step handles cold caches natively; ~30s wheel-download cost per job until the runner-side cache backend is fixed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 16:35:26 -04:00
bvandeusen 88cfb3dd02 Merge pull request 'v26.05.26.1 — thumb backfill, modal redesign, recovery sweep race-safety, artist view redesign, extension fixes' (#23) from dev into main 2026-05-26 16:32:00 -04:00
bvandeusen fb41b90110 fix(extension): _find_or_create_artist + _find_or_create_source race-safe via savepoint + IntegrityError recovery — same pattern as importer's helpers. Two concurrent quick-add-source calls on the same artist/url would have 500'd on uq_artist_slug / uq_source_artist_platform_url; now the second one rolls the savepoint back and returns the existing row.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 16:27:51 -04:00
bvandeusen 7d84990f6d feat(artist-view): ArtistView rewrite — sticky frosted ArtistHeader (name + stats + tabs) replaces the in-body h1; three lazy tabs (Posts default, Gallery fallback, Management); ?tab= URL state; cross-artist store reset; document.title set on slug change
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 16:01:14 -04:00
bvandeusen ca55d92c68 feat(artist-view): ArtistManagementTab — Overview chips (Subscription badge + subscription count) + Frequent tags + Activity sparkline + Subscriptions table + Danger zone. 'View posts' chip and 'Credential health · FC-3b' placeholder chip dropped; 'Sources' section renamed to 'Subscriptions'
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 16:00:47 -04:00
bvandeusen cce014be3a feat(artist-view): ArtistGalleryTab — MasonryGrid wired to the artist store's existing loadMoreImages; modal-open preserves ?tab=. No global gallery-store coupling (avoids cross-pollution into /gallery)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 16:00:14 -04:00
bvandeusen c07effb593 feat(artist-view): ArtistPostsTab — PostCard infinite-scroll list, artist_id pinned, no filter bar; mirrors PostsView mechanics
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 16:00:02 -04:00
bvandeusen a36f72b383 feat(artist-view): ArtistHeader — sticky frosted bar (top:64px) matching TopNav recipe, hosts name + image/post stats + tab strip
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 15:59:46 -04:00
bvandeusen 2e8d7c960c feat(artist): post_count on the artist overview response — drives the Posts/Gallery default-tab fallback in the upcoming ArtistView redesign
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 15:59:28 -04:00
bvandeusen 992f38ec20 fix(test): drop unused Post binding in test_importer_provenance_race (ruff F841)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 14:39:57 -04:00
bvandeusen 0bc5767a2b fix(importer): Source = one per (artist, platform), not one per post — filesystem importer's sidecar paths now reuse the artist's existing subscription Source (or create one synthetic anchor with enabled=False) instead of fabricating a new Source per post URL. Alembic 0022 consolidates existing per-post Sources to canonical (prefers campaign URL; falls back to sidecar:<platform>:<slug>) and re-parents Posts + ImageProvenance, merging Post collisions.
Operator-flagged 2026-05-26: Atole artist detail page showed 406 Sources where 1 was right.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 14:19:29 -04:00
bvandeusen 397021dcbd fix(importer): ImageProvenance (image_record_id, post_id) race-safe via savepoint + alembic 0021 UNIQUE — closes the SELECT-then-INSERT window that planted duplicates and broke .scalar_one_or_none() on every later deep-scan rederive (MultipleResultsFound). Migration dedupes existing rows (min(id) per pair); model gains __table_args__; gallery-filter test that seeded duplicates dropped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 13:47:55 -04:00
bvandeusen b0bfbc585a fix(import-admin): retry-failed + clear-stuck — same UPDATE…WHERE pattern as the maintenance sweep, so neither endpoint can hit psycopg's 65535-parameter ceiling once accumulated row counts exceed ~65k
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 11:47:40 -04:00
bvandeusen 110c1c0e51 fix(maintenance): recover_interrupted_tasks — fold SELECT into UPDATE…WHERE…RETURNING so the IN-list no longer blows past psycopg's 65535-parameter ceiling (operator-hit 2026-05-26 after deep scan orphan pile)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 11:46:06 -04:00
bvandeusen 6de84d0d60 feat(ui): ErrorDetailModal — context panel (task/queue/target/duration/started/retries/worker/celery-id/args) + contrast fix (background-token bg vs surface-variant pale-on-pale) + copyText helper for Copy button
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 11:16:37 -04:00
bvandeusen 4e1f208a9f fix(ui-copy): copyText utility with execCommand fallback — navigator.clipboard is gated by Secure Context (HTTPS-only) and is undefined on plain-HTTP self-hosted deployments. Apply to ExtensionKeyBar + BrowserExtensionCard.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 11:16:32 -04:00
bvandeusen 06913eba8e feat(thumb-backfill): MaintenancePanel — wire ThumbnailBackfillCard into grid, broaden intro to cover non-ML backfills
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 10:37:09 -04:00
bvandeusen b7f693b15e feat(thumb-backfill): ThumbnailBackfillCard — 'Run backfill now' button, mirrors MLBackfillCard pattern
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 10:36:55 -04:00
bvandeusen ecd0199799 feat(thumb-backfill): Pinia store — triggerBackfill() POSTs /api/thumbnails/backfill
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 10:36:45 -04:00
bvandeusen 983da9e5b1 feat(thumb-backfill): /api/thumbnails/backfill endpoint — POST → 202 + celery_task_id
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 10:36:38 -04:00
bvandeusen a41eddae3f feat(thumb-backfill): backfill_thumbnails planner task — keyset-paginates ImageRecord, NULLs bad thumb paths, enqueues generate_thumbnail for NULL/missing/corrupt
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 10:36:21 -04:00
bvandeusen 7aa7f5a3d6 feat(thumb-backfill): _thumb_is_valid helper — JPEG/PNG magic-byte check on the on-disk thumbnail file
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 10:35:06 -04:00
47 changed files with 1998 additions and 364 deletions
+38 -5
View File
@@ -2,7 +2,17 @@ name: Build images
on:
push:
branches: [dev, main]
# `:dev` builds dropped 2026-05-26 — operator tests from `:latest` after
# merge-to-main, not from the dev branch image. Saves one full docker
# build per dev push.
branches: [main]
# Tag-push triggers an immutable per-version image build (e.g.
# `:v26.05.26.5`) — gives a real rollback story alongside the floating
# `:main` / `:latest`. Layer reuse keeps the registry-storage cost
# negligible per tag. Doesn't overlap with the push-to-main build (that
# one publishes `:main` + `:latest`; the tag-push build publishes only
# `:<tag>`).
tags: ['v*']
# Requires repo secret RELEASE_TOKEN — a Forgejo PAT with scopes:
# - write:package, read:package (for docker push to git.fabledsword.com)
@@ -158,8 +168,12 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Download signed XPI from Forgejo release asset (main only)
if: github.ref == 'refs/heads/main'
- name: Download signed XPI from Forgejo release asset (main + tags)
# Fires on main-push AND on tag-push. Tag-push builds re-package the
# same source code as the preceding main-push build but with an
# immutable version tag — they need the XPI too, otherwise the
# versioned image ships without the signed extension.
if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')
env:
TOKEN: ${{ secrets.RELEASE_TOKEN }}
run: |
@@ -200,7 +214,20 @@ jobs:
- name: Determine tag
id: tag
run: |
if [ "${GITHUB_REF##*/}" = "main" ]; then
# Three trigger shapes:
# refs/tags/v… → tag-push: publish ONLY the immutable version
# tag (e.g. :v26.05.26.5). Don't touch :latest;
# that already got published by the main-push
# build for the merge commit.
# refs/heads/main → push to main (incl. PR merge commits):
# publish :main + :latest (floating).
# anything else → safety net; shouldn't fire given the `on:`
# config above (dev was dropped). Tag :dev to
# surface the unexpected run in the registry.
if [ "${GITHUB_REF#refs/tags/}" != "${GITHUB_REF}" ]; then
TAG_NAME="${GITHUB_REF#refs/tags/}"
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:${TAG_NAME}" >> "$GITHUB_OUTPUT"
elif [ "${GITHUB_REF##*/}" = "main" ]; then
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:main,git.fabledsword.com/bvandeusen/fabledcurator:latest" >> "$GITHUB_OUTPUT"
else
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:dev" >> "$GITHUB_OUTPUT"
@@ -231,7 +258,13 @@ jobs:
- name: Determine tag
id: tag
run: |
if [ "${GITHUB_REF##*/}" = "main" ]; then
# Mirrors build-web's three-shape logic (tag-push / main-push /
# safety-net dev). The -ml image follows the same release cadence
# as the web image.
if [ "${GITHUB_REF#refs/tags/}" != "${GITHUB_REF}" ]; then
TAG_NAME="${GITHUB_REF#refs/tags/}"
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:${TAG_NAME}" >> "$GITHUB_OUTPUT"
elif [ "${GITHUB_REF##*/}" = "main" ]; then
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:main,git.fabledsword.com/bvandeusen/fabledcurator-ml:latest" >> "$GITHUB_OUTPUT"
else
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:dev" >> "$GITHUB_OUTPUT"
+15 -30
View File
@@ -8,8 +8,10 @@ name: CI
on:
push:
branches: [dev, main]
pull_request:
branches: [main]
# pull_request trigger intentionally absent — with branches: [dev, main]
# above, every PR commit already fires CI via the push event on dev. Adding
# pull_request would duplicate runs on dev→main PRs. FC has no fork PRs
# (single-operator Forgejo repo) so push coverage is complete.
jobs:
backend-lint-and-test:
@@ -24,13 +26,17 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Cache pip wheels
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: pip-${{ runner.os }}-py314-${{ hashFiles('requirements.txt') }}
restore-keys: |
pip-${{ runner.os }}-py314-
# Cache step removed 2026-05-26: act_runner's cache backend has been
# broken on this homelab runner since 2026-05-15 (first as request-
# timeout warnings, then as hard "Cannot find module .../dist/restore/
# index.js" failures that tank the whole job). The cache step targeted
# ~/.cache/pip but the install below uses `uv pip install` primarily,
# whose own cache lives at ~/.cache/uv — so the cache step's real
# benefit was marginal even when working. Cost of removal: ~30s of
# wheel downloads per job. Future re-enable: mount ~/.cache/uv as a
# docker volume at the runner level (skips actions/cache entirely),
# or fix the runner-side cache backend (clear /var/run/act/actions/*,
# pin act_runner version, etc.).
- name: Install Python deps
# ruff is pre-installed in the ci-python image (see CI-Runner/CI-python/
@@ -124,13 +130,6 @@ jobs:
--health-retries 10
steps:
- uses: actions/checkout@v4
- name: Cache pip wheels
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: pip-${{ runner.os }}-py314-${{ hashFiles('requirements.txt') }}
restore-keys: |
pip-${{ runner.os }}-py314-
- name: API integration shard (resolve service IPs, migrate, test)
run: |
set -eux
@@ -189,13 +188,6 @@ jobs:
--health-retries 10
steps:
- uses: actions/checkout@v4
- name: Cache pip wheels
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: pip-${{ runner.os }}-py314-${{ hashFiles('requirements.txt') }}
restore-keys: |
pip-${{ runner.os }}-py314-
- name: Importer integration shard (resolve service IPs, migrate, test)
run: |
set -eux
@@ -254,13 +246,6 @@ jobs:
--health-retries 10
steps:
- uses: actions/checkout@v4
- name: Cache pip wheels
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: pip-${{ runner.os }}-py314-${{ hashFiles('requirements.txt') }}
restore-keys: |
pip-${{ runner.os }}-py314-
- name: Core integration shard (everything not api / importer / migration / phash / sidecar / scan / archive / backfill)
run: |
set -eux
@@ -0,0 +1,54 @@
"""provenance-race: dedupe + UNIQUE(image_record_id, post_id) on image_provenance
Revision ID: 0021
Revises: 0020
Create Date: 2026-05-26
Closes the race in Importer._apply_sidecar's existence-check + INSERT pattern.
Two workers writing for the same (image, post) pair both saw no existing row
and both inserted, leaving duplicates that then broke .scalar_one_or_none()
on every subsequent deep-scan rederive against those images
(MultipleResultsFound). Most plausibly seeded when the 5-min recovery sweep
re-enqueued a still-running long-import task and the second worker collided
with the first inside _apply_sidecar.
Migration steps:
1. DELETE all but min(id) per (image_record_id, post_id) pair. Operator's
DB had 2 affected pairs at write-time; harmless no-op if zero.
2. Add UNIQUE constraint so the importer's new savepoint+IntegrityError
recovery path can trip on collision and re-select, mirroring
uq_source_artist_platform_url and uq_post_source_external_id.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0021"
down_revision: Union[str, None] = "0020"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.execute(
"""
DELETE FROM image_provenance ip1
USING image_provenance ip2
WHERE ip1.image_record_id = ip2.image_record_id
AND ip1.post_id = ip2.post_id
AND ip1.id > ip2.id
"""
)
op.create_unique_constraint(
"uq_image_provenance_image_post",
"image_provenance",
["image_record_id", "post_id"],
)
def downgrade() -> None:
op.drop_constraint(
"uq_image_provenance_image_post",
"image_provenance",
type_="unique",
)
@@ -0,0 +1,223 @@
"""source-collapse: one Source per (artist, platform) — consolidate junk per-post Sources
Revision ID: 0022
Revises: 0021
Create Date: 2026-05-26
Closes the operator-flagged 2026-05-26 issue where the filesystem importer
called _find_or_create_source(url=sd.post_url), creating one Source row per
imported post URL. Operator's Atole artist had 406 Source rows where there
should have been 1 (the /cw/Atole subscription Source).
Source represents a subscription feed (one per artist+platform — the
gallery-dl URL polled by the FC-3 downloader). Posts hang off it. The
filesystem importer was misusing Source as a per-post key.
Migration steps per (artist_id, platform) group with >1 Source:
1. Pick canonical — prefer a URL NOT matching '/posts/<id>$' (real
campaign URL like /cw/Atole); else min(id).
2. PRE-merge any Posts under non-canonical sources whose
external_post_id ALREADY exists under the canonical source. (Same
gallery-dl post imported via two different sidecar paths can plant
two Post rows with identical external_post_id under different
Sources for the same artist.) Repoint ImageProvenance +
ImageRecord.primary_post_id to the canonical-side Post, dedupe
ImageProvenance against alembic 0021's uq, then delete the
non-canonical-side Post. This MUST happen before step 3 — Postgres
fires uq_post_source_external_id row-by-row during the bulk UPDATE
and the merge-after-reparent ordering 500s on first collision
(operator-hit during v26.05.26.1 deploy, 2026-05-26).
3. Reparent remaining Posts onto canonical (no collisions possible now).
4. Reparent ImageProvenance.source_id off the non-canonical sources.
5. Delete the orphan Source rows.
6. If the canonical Source's URL still looks like a per-post URL (no
campaign URL existed among candidates), rewrite it to
'sidecar:<platform>:<artist_slug>' so the artist detail page shows
something readable.
"""
from typing import Sequence, Union
from alembic import op
from sqlalchemy import text
revision: str = "0022"
down_revision: Union[str, None] = "0021"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_POST_URL_RE = r"/posts/[^/]+$"
def upgrade() -> None:
conn = op.get_bind()
# Find (artist_id, platform) groups with > 1 Source row.
groups = conn.execute(text("""
SELECT artist_id, platform
FROM source
GROUP BY artist_id, platform
HAVING COUNT(*) > 1
""")).fetchall()
for artist_id, platform in groups:
rows = conn.execute(
text("""
SELECT id, url FROM source
WHERE artist_id = :a AND platform = :p
ORDER BY id ASC
"""),
{"a": artist_id, "p": platform},
).fetchall()
# Canonical: first row whose URL doesn't look like a per-post URL;
# else min(id).
canonical_id = None
for sid, url in rows:
if not _matches_post_url(url):
canonical_id = sid
break
if canonical_id is None:
canonical_id = rows[0][0]
other_ids = [sid for sid, _ in rows if sid != canonical_id]
if not other_ids:
continue
# STEP 2: PRE-merge ALL Posts with duplicate external_post_id
# across the entire (canonical + others) group, BEFORE the bulk
# reparent. Two cases must both be handled:
# (A) canonical has Post X with epid=N; an "other" source has
# Post Y with epid=N → after bulk UPDATE, (canonical, N)
# collides with itself.
# (B) two different "other" sources each have a Post with
# epid=N; canonical has none → after bulk UPDATE, both
# are repointed to (canonical, N) and the second collides.
# The earlier version of this migration only handled (A); the
# operator's deploy 2026-05-26 tripped (B) at line 139.
# Fix: group ALL Posts in the (artist, platform) by epid; for
# any group with count>1, pick the keep (prefer one already
# under canonical; else lowest id) and merge the rest into it.
all_posts = conn.execute(
text("""
SELECT external_post_id, id, source_id
FROM post
WHERE source_id = :canonical OR source_id = ANY(:others)
ORDER BY external_post_id, id
"""),
{"canonical": canonical_id, "others": other_ids},
).fetchall()
by_epid: dict = {}
for epid, post_id, src_id in all_posts:
by_epid.setdefault(epid, []).append((post_id, src_id))
for _epid, posts in by_epid.items():
if len(posts) <= 1:
continue
# Prefer a Post already under canonical as the keep.
canonical_posts = [p for p in posts if p[1] == canonical_id]
if canonical_posts:
keep_id = canonical_posts[0][0]
else:
keep_id = posts[0][0] # already sorted by id ASC
drop_ids = [p[0] for p in posts if p[0] != keep_id]
for drop_id in drop_ids:
# Pre-delete image_provenance rows under drop_ whose
# image_record_id ALREADY has a provenance under keep —
# the UPDATE below would otherwise repoint them and
# trip uq_image_provenance_image_post (alembic 0021)
# row-by-row before any after-the-fact dedupe could
# run. Operator's v26.05.26.3 deploy 2026-05-26 tripped
# this at line 123.
conn.execute(
text("""
DELETE FROM image_provenance
WHERE post_id = :drop_
AND image_record_id IN (
SELECT image_record_id FROM image_provenance
WHERE post_id = :keep
)
"""),
{"keep": keep_id, "drop_": drop_id},
)
# Now safe to repoint the survivors.
conn.execute(
text("""
UPDATE image_provenance SET post_id = :keep
WHERE post_id = :drop_
"""),
{"keep": keep_id, "drop_": drop_id},
)
conn.execute(
text("""
UPDATE image_record SET primary_post_id = :keep
WHERE primary_post_id = :drop_
"""),
{"keep": keep_id, "drop_": drop_id},
)
conn.execute(
text("DELETE FROM post WHERE id = :drop_"),
{"drop_": drop_id},
)
# STEP 3: Bulk reparent the remaining Posts off the other
# Sources. After step 2, no collisions on
# (canonical, external_post_id) are possible.
conn.execute(
text("""
UPDATE post SET source_id = :canonical
WHERE source_id = ANY(:others)
"""),
{"canonical": canonical_id, "others": other_ids},
)
# STEP 4: Reparent ImageProvenance.source_id (denormalized FK).
# No UNIQUE on source_id; safe bulk update.
conn.execute(
text("""
UPDATE image_provenance SET source_id = :canonical
WHERE source_id = ANY(:others)
"""),
{"canonical": canonical_id, "others": other_ids},
)
# STEP 5: Drop the orphan Sources.
conn.execute(
text("DELETE FROM source WHERE id = ANY(:others)"),
{"others": other_ids},
)
# If the canonical's URL still looks per-post (no campaign URL
# existed among the candidates), rewrite to a synthetic anchor so
# the artist detail page renders something readable.
canonical_url = conn.execute(
text("SELECT url FROM source WHERE id = :id"),
{"id": canonical_id},
).scalar_one()
if _matches_post_url(canonical_url):
slug = conn.execute(
text("SELECT slug FROM artist WHERE id = :id"),
{"id": artist_id},
).scalar_one()
conn.execute(
text("""
UPDATE source
SET url = :new_url, enabled = false
WHERE id = :id
"""),
{
"id": canonical_id,
"new_url": f"sidecar:{platform}:{slug}",
},
)
def downgrade() -> None:
# Lossy migration — orphan Sources deleted, Posts reparented, Posts
# merged. No safe downgrade. If you need to roll back the schema
# invariant, fork from 0021 and re-run filesystem imports.
pass
def _matches_post_url(url: str) -> bool:
"""True if url ends with /posts/<token> (gallery-dl-style per-post URL)."""
import re
return bool(re.search(_POST_URL_RE, url or ""))
+38 -1
View File
@@ -3,13 +3,23 @@
import logging
from pathlib import Path
from quart import Quart
from quart import Quart, request
from .api import all_blueprints
from .config import get_config
from .frontend import frontend_bp
from .services.credential_crypto import CredentialCrypto
# Browser-extension origins. The FabledCurator extension fetches from
# moz-extension://<uuid>/ on Firefox and chrome-extension://<uuid>/ on
# Chromium-based browsers. Operator-flagged 2026-05-26: extension's
# 'Test connection' returned `NetworkError` because the X-Extension-Key
# header on /api/credentials triggers a CORS preflight that our routes
# don't handle. Whitelisting only these two schemes (not opening CORS
# up generally) lets the extension talk to a plain-HTTP self-hosted FC
# without weakening the no-CORS posture for normal browser usage.
_EXTENSION_ORIGIN_SCHEMES = ("moz-extension://", "chrome-extension://")
_CREDENTIAL_KEY_PATH = Path("/images/secrets/credential_key.b64")
@@ -35,6 +45,33 @@ def create_app() -> Quart:
# Registered last so /api/* routes win over the SPA catch-all.
app.register_blueprint(frontend_bp)
@app.before_request
async def _extension_cors_preflight():
# Short-circuit OPTIONS preflight from the browser extension with a
# 204 + CORS headers (the after_request hook below adds them).
# Without this, OPTIONS lands on routes that only declared POST/GET
# methods and 405s before the after_request gets a chance.
if request.method != "OPTIONS":
return None
origin = request.headers.get("Origin", "")
if any(origin.startswith(s) for s in _EXTENSION_ORIGIN_SCHEMES):
return "", 204
return None
@app.after_request
async def _extension_cors_headers(response):
origin = request.headers.get("Origin", "")
if any(origin.startswith(s) for s in _EXTENSION_ORIGIN_SCHEMES):
response.headers["Access-Control-Allow-Origin"] = origin
response.headers["Access-Control-Allow-Methods"] = (
"GET, POST, PATCH, DELETE, OPTIONS"
)
response.headers["Access-Control-Allow-Headers"] = (
"Content-Type, X-Extension-Key"
)
response.headers["Access-Control-Max-Age"] = "86400"
return response
@app.after_serving
async def _dispose_db_engine() -> None:
from .extensions import dispose_engine
+2
View File
@@ -38,6 +38,7 @@ def all_blueprints() -> list[Blueprint]:
from .system_activity import system_activity_bp
from .system_backup import system_backup_bp
from .tags import tags_bp
from .thumbnails import thumbnails_bp
return [
api_bp,
attachments_bp,
@@ -58,6 +59,7 @@ def all_blueprints() -> list[Blueprint]:
allowlist_bp,
aliases_bp,
ml_admin_bp,
thumbnails_bp,
sources_bp,
platforms_bp,
posts_bp,
+32 -29
View File
@@ -103,17 +103,22 @@ async def list_tasks():
@import_admin_bp.route("/retry-failed", methods=["POST"])
async def retry_failed():
# Fold SELECT into UPDATE…WHERE…RETURNING — the prior SELECT-then-
# UPDATE-WHERE-id-IN pattern blew past psycopg's 65535-parameter
# ceiling once failed_ids exceeded ~65k rows.
async with get_session() as session:
failed_ids = (
await session.execute(select(ImportTask.id).where(ImportTask.status == "failed"))
).scalars().all()
result = await session.execute(
update(ImportTask)
.where(ImportTask.status == "failed")
.values(
status="queued", error=None,
started_at=None, finished_at=None,
)
.returning(ImportTask.id)
)
failed_ids = [row[0] for row in result.all()]
if not failed_ids:
return jsonify({"retried": 0})
await session.execute(
update(ImportTask)
.where(ImportTask.id.in_(failed_ids))
.values(status="queued", error=None, started_at=None, finished_at=None)
)
await session.commit()
from ..tasks.import_file import import_media_file
@@ -138,28 +143,26 @@ async def clear_stuck():
autoretry-looped for 2 days after a corrupt-data PIL OSError.
"""
async with get_session() as session:
stuck_ids = (
await session.execute(
select(ImportTask.id).where(
ImportTask.status.in_(["pending", "queued", "processing"])
)
# Fold SELECT into UPDATE…WHERE — see /retry-failed for the
# 65535-parameter ceiling rationale. rowcount is enough here
# because we don't need the ids afterward (no .delay()).
clear_result = await session.execute(
update(ImportTask)
.where(
ImportTask.status.in_(["pending", "queued", "processing"])
)
).scalars().all()
if stuck_ids:
await session.execute(
update(ImportTask)
.where(ImportTask.id.in_(stuck_ids))
.values(
status="failed",
finished_at=datetime.now(UTC),
error=(
"manually cleared via /api/import/clear-stuck "
"— stuck in non-terminal state; retry once "
"underlying cause (corrupt file, missing model, "
"etc.) is resolved"
),
)
.values(
status="failed",
finished_at=datetime.now(UTC),
error=(
"manually cleared via /api/import/clear-stuck "
"— stuck in non-terminal state; retry once "
"underlying cause (corrupt file, missing model, "
"etc.) is resolved"
),
)
)
tasks_failed = clear_result.rowcount or 0
# Finalize any 'running' ImportBatch that no longer has any
# active children. The "Scanning..." banner is driven by
@@ -195,7 +198,7 @@ async def clear_stuck():
await session.commit()
return jsonify({
"tasks_failed": len(stuck_ids),
"tasks_failed": tasks_failed,
"batches_finalized": finalized_batches,
})
+13
View File
@@ -0,0 +1,13 @@
"""Thumbnail admin API: backfill trigger."""
from quart import Blueprint, jsonify
thumbnails_bp = Blueprint("thumbnails", __name__, url_prefix="/api/thumbnails")
@thumbnails_bp.route("/backfill", methods=["POST"])
async def trigger_backfill():
from ..tasks.thumbnail import backfill_thumbnails
r = backfill_thumbnails.delay()
return jsonify({"celery_task_id": r.id}), 202
+15 -5
View File
@@ -1,14 +1,18 @@
"""ImageProvenance — links an ImageRecord to a Post.
Many-to-one (one image, many provenance rows) enables the enrich-on-duplicate
rule (spec §3): when a downloaded image is a pHash dupe of an existing
record, we append a new provenance row to the existing record rather than
dropping the metadata.
One image can have many provenance rows — different posts each contribute
metadata (enrich-on-duplicate rule, spec §3: a downloaded image that is a
pHash dupe of an existing record gets a NEW provenance row for the new post
appended, rather than the metadata being dropped). But the (image, post)
pair is unique — alembic 0021 enforces uq_image_provenance_image_post
after operator-flagged 2026-05-26 saw _apply_sidecar's existence-check +
INSERT race plant duplicates that then broke .scalar_one_or_none() on
every later deep-scan rederive (MultipleResultsFound).
"""
from datetime import datetime
from sqlalchemy import JSON, DateTime, ForeignKey, Integer, func
from sqlalchemy import JSON, DateTime, ForeignKey, Integer, UniqueConstraint, func
from sqlalchemy.orm import Mapped, mapped_column
from .base import Base
@@ -16,6 +20,12 @@ from .base import Base
class ImageProvenance(Base):
__tablename__ = "image_provenance"
__table_args__ = (
UniqueConstraint(
"image_record_id", "post_id",
name="uq_image_provenance_image_post",
),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True)
image_record_id: Mapped[int] = mapped_column(
+10
View File
@@ -111,12 +111,22 @@ class ArtistService:
)
).all()
post_count = (
await self.session.execute(
select(func.count(func.distinct(Post.id)))
.select_from(Post)
.join(Source, Source.id == Post.source_id)
.where(Source.artist_id == aid)
)
).scalar_one()
return {
"id": artist.id,
"name": artist.name,
"slug": artist.slug,
"is_subscription": bool(artist.is_subscription),
"image_count": int(image_count),
"post_count": int(post_count),
"date_range": {
"min": dmin.isoformat() if dmin else None,
"max": dmax.isoformat() if dmax else None,
+44 -7
View File
@@ -11,6 +11,7 @@ from __future__ import annotations
import re
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession
from ..models import Artist, Source
@@ -97,20 +98,40 @@ class ExtensionService:
raise UnknownPlatformError(f"no platform pattern matched {url!r}")
async def _find_or_create_artist(self, raw_name: str) -> tuple[Artist, bool]:
"""Race-safe find-or-create on Artist by slug. Mirrors the
savepoint + IntegrityError recovery pattern used in
Importer._find_or_create_source/post (see
reference_scalar_one_or_none_duplicates memory). Without this,
two concurrent quick-add-source calls hitting the same artist
would both miss the existence check and the second INSERT would
500 against uq_artist_slug.
"""
slug = slugify(raw_name)
existing = (await self.session.execute(
select(Artist).where(Artist.slug == slug)
)).scalar_one_or_none()
if existing is not None:
return existing, False
artist = Artist(name=raw_name, slug=slug, is_subscription=True)
self.session.add(artist)
await self.session.flush()
return artist, True
sp = await self.session.begin_nested()
try:
artist = Artist(name=raw_name, slug=slug, is_subscription=True)
self.session.add(artist)
await self.session.flush()
await sp.commit()
return artist, True
except IntegrityError:
await sp.rollback()
recovered = (await self.session.execute(
select(Artist).where(Artist.slug == slug)
)).scalar_one()
return recovered, False
async def _find_or_create_source(
self, *, artist_id: int, platform: str, url: str,
) -> tuple[Source, bool]:
"""Race-safe — same pattern as _find_or_create_artist above. The
uq_source_artist_platform_url constraint catches the duplicate
insert; we roll the savepoint back and re-select."""
existing = (await self.session.execute(
select(Source).where(
Source.artist_id == artist_id,
@@ -120,8 +141,24 @@ class ExtensionService:
)).scalar_one_or_none()
if existing is not None:
return existing, False
src = Source(artist_id=artist_id, platform=platform, url=url, enabled=True)
self.session.add(src)
await self.session.flush()
sp = await self.session.begin_nested()
try:
src = Source(
artist_id=artist_id, platform=platform,
url=url, enabled=True,
)
self.session.add(src)
await self.session.flush()
await sp.commit()
except IntegrityError:
await sp.rollback()
recovered = (await self.session.execute(
select(Source).where(
Source.artist_id == artist_id,
Source.platform == platform,
Source.url == url,
)
)).scalar_one()
return recovered, False
await self.session.commit()
return src, True
+83 -13
View File
@@ -247,6 +247,62 @@ class Importer:
)
).scalar_one()
def _source_for_sidecar(
self, *, artist_id: int, platform: str, artist_slug: str,
) -> Source:
"""Filesystem-import sidecar Source resolver.
Source represents a subscription feed (one per artist+platform — the
gallery-dl URL polled by the FC-3 downloader). The filesystem importer
used to call _find_or_create_source(url=sd.post_url), which created
one Source row per post URL — 100s of junk Sources per artist, all
with enabled=True, polluting the artist detail page and tricking the
subscription checker into trying to poll patreon post URLs as feeds.
Operator-flagged 2026-05-26.
New behaviour: if any Source row exists for (artist_id, platform),
reuse it regardless of its URL — the artist's real subscription Source
(created by the downloader / extension / UI) is the canonical
attachment point for filesystem-imported posts. If none exists, create
ONE synthetic anchor with url='sidecar:<platform>:<artist_slug>' and
enabled=False (so the subscription checker doesn't poll it).
"""
existing = self.session.execute(
select(Source)
.where(
Source.artist_id == artist_id,
Source.platform == platform,
)
.order_by(Source.id.asc())
.limit(1)
).scalar_one_or_none()
if existing is not None:
return existing
synthetic_url = f"sidecar:{platform}:{artist_slug}"
sp = self.session.begin_nested()
try:
row = Source(
artist_id=artist_id,
platform=platform,
url=synthetic_url,
enabled=False,
)
self.session.add(row)
self.session.flush()
sp.commit()
return row
except IntegrityError:
sp.rollback()
return self.session.execute(
select(Source)
.where(
Source.artist_id == artist_id,
Source.platform == platform,
)
.order_by(Source.id.asc())
.limit(1)
).scalar_one()
def _find_or_create_post(
self, *, source_id: int, external_post_id: str,
) -> Post:
@@ -315,9 +371,8 @@ class Importer:
return None
sd = parse_sidecar(data)
platform = sd.platform or "unknown"
url = sd.post_url or f"sidecar:{platform}"
src = self._find_or_create_source(
artist_id=artist.id, platform=platform, url=url,
src = self._source_for_sidecar(
artist_id=artist.id, platform=platform, artist_slug=artist.slug,
)
epid = sd.external_post_id or sc.stem
return self._find_or_create_post(
@@ -763,9 +818,9 @@ class Importer:
src = explicit_source
else:
platform = sd.platform or "unknown"
url = sd.post_url or f"sidecar:{platform}"
src = self._find_or_create_source(
artist_id=artist.id, platform=platform, url=url,
src = self._source_for_sidecar(
artist_id=artist.id, platform=platform,
artist_slug=artist.slug,
)
epid = sd.external_post_id or sc.stem
@@ -784,6 +839,15 @@ class Importer:
post.attachment_count = sd.attachment_count
post.raw_metadata = sd.raw
# Race-safe (image_record_id, post_id) upsert — mirrors the
# _find_or_create_source/post savepoint pattern. The plain
# SELECT-then-INSERT pattern lost a race when two workers ran
# _apply_sidecar on the same (image, post) pair (e.g. the 5-min
# recovery sweep re-enqueued a still-running long import), planting
# duplicates that then broke .scalar_one_or_none() on every later
# deep-scan rederive (MultipleResultsFound). Alembic 0021 adds the
# uq_image_provenance_image_post UNIQUE so this savepoint actually
# trips on collision.
exists = self.session.execute(
select(ImageProvenance.id).where(
ImageProvenance.image_record_id == record.id,
@@ -791,14 +855,20 @@ class Importer:
)
).scalar_one_or_none()
if exists is None:
self.session.add(
ImageProvenance(
image_record_id=record.id,
post_id=post.id,
source_id=src.id,
captured_metadata=sd.raw,
sp = self.session.begin_nested()
try:
self.session.add(
ImageProvenance(
image_record_id=record.id,
post_id=post.id,
source_id=src.id,
captured_metadata=sd.raw,
)
)
)
self.session.flush()
sp.commit()
except IntegrityError:
sp.rollback()
if record.primary_post_id is None:
record.primary_post_id = post.id
self.session.flush()
+29 -30
View File
@@ -54,41 +54,40 @@ def recover_interrupted_tasks() -> int:
processing_cutoff = now - timedelta(minutes=STUCK_THRESHOLD_MINUTES)
orphan_cutoff = now - timedelta(minutes=ORPHAN_PENDING_THRESHOLD_MINUTES)
with SessionLocal() as session:
stuck_ids = session.execute(
select(ImportTask.id)
# Both sweeps used to be SELECT ids → UPDATE WHERE id IN (...) which
# blew past psycopg's 65535-parameter ceiling once a sweep covered
# tens of thousands of rows (operator hit it 2026-05-26 after the
# /import deep scan piled up orphans). Folding the SELECT into the
# UPDATE eliminates the IN-list entirely. RETURNING gives us back
# exactly the ids that flipped so the stuck sweep can still
# .delay() each one.
stuck_result = session.execute(
update(ImportTask)
.where(ImportTask.status == "processing")
.where(ImportTask.started_at < processing_cutoff)
).scalars().all()
.values(
status="queued",
started_at=None,
error="recovered from stuck state",
)
.returning(ImportTask.id)
)
stuck_ids = [row[0] for row in stuck_result.all()]
orphan_ids = session.execute(
select(ImportTask.id)
orphan_result = session.execute(
update(ImportTask)
.where(ImportTask.status.in_(["pending", "queued"]))
.where(ImportTask.created_at < orphan_cutoff)
).scalars().all()
if not stuck_ids and not orphan_ids:
return 0
if stuck_ids:
session.execute(
update(ImportTask)
.where(ImportTask.id.in_(stuck_ids))
.values(status="queued", started_at=None, error="recovered from stuck state")
)
if orphan_ids:
session.execute(
update(ImportTask)
.where(ImportTask.id.in_(orphan_ids))
.values(
status="failed",
error=(
"orphan pending/queued swept by recover_interrupted_tasks "
"(scanner likely crashed mid-enqueue); retry via "
"/api/import/retry-failed"
),
)
.values(
status="failed",
error=(
"orphan pending/queued swept by recover_interrupted_tasks "
"(scanner likely crashed mid-enqueue); retry via "
"/api/import/retry-failed"
),
)
)
orphan_count = orphan_result.rowcount or 0
session.commit()
@@ -97,7 +96,7 @@ def recover_interrupted_tasks() -> int:
for tid in stuck_ids:
import_media_file.delay(tid)
return len(stuck_ids) + len(orphan_ids)
return len(stuck_ids) + orphan_count
@celery.task(name="backend.app.tasks.maintenance.cleanup_old_tasks")
+79
View File
@@ -17,6 +17,30 @@ from ._sync_engine import sync_session_factory as _sync_session_factory
IMAGES_ROOT = Path("/images")
THUMB_MAGIC_JPEG = b"\xff\xd8\xff"
THUMB_MAGIC_PNG = b"\x89PNG\r\n\x1a\n"
def _thumb_is_valid(path: Path) -> bool:
"""Return True iff `path` exists and starts with a JPEG or PNG magic header.
The on-disk thumbnail format is set by services/thumbnailer.py — JPEG for
opaque sources, PNG for alpha sources. Anything else (missing file, OSError,
truncated, wrong magic) is invalid.
"""
try:
with path.open("rb") as f:
head = f.read(12)
except OSError:
return False
if len(head) < 8:
return False
if head[:3] == THUMB_MAGIC_JPEG:
return True
if head[:8] == THUMB_MAGIC_PNG:
return True
return False
@celery.task(
name="backend.app.tasks.thumbnail.generate_thumbnail",
@@ -50,3 +74,58 @@ def generate_thumbnail(self, image_id: int) -> dict:
session.add(record)
session.commit()
return {"status": "ok", "image_id": image_id, "path": str(result.path)}
@celery.task(
name="backend.app.tasks.thumbnail.backfill_thumbnails",
bind=True,
)
def backfill_thumbnails(self) -> dict:
"""Scan ImageRecord and enqueue generate_thumbnail for rows whose
thumbnail is missing, gone from disk, or has wrong magic bytes.
Keyset paginates by id ASC, page size 500. NULLs out thumbnail_path for
rows that point at a missing or corrupt file before enqueueing — keeps
the DB self-consistent on partial runs and makes re-runs safe.
Returns {"enqueued": N, "ok": M, "regenerated": K} where:
- enqueued = total generate_thumbnail.delay() calls
- ok = rows whose existing thumbnail file is valid (skipped)
- regenerated = subset of enqueued that had a non-NULL thumbnail_path
cleared (i.e. missing + corrupt)
"""
from sqlalchemy import select, update
SessionLocal = _sync_session_factory()
enqueued = 0
ok = 0
regenerated = 0
last_id = 0
with SessionLocal() as session:
while True:
rows = session.execute(
select(ImageRecord.id, ImageRecord.thumbnail_path)
.where(ImageRecord.id > last_id)
.order_by(ImageRecord.id.asc())
.limit(500)
).all()
if not rows:
break
for image_id, thumb_path in rows:
if thumb_path is None:
generate_thumbnail.delay(image_id)
enqueued += 1
elif _thumb_is_valid(Path(thumb_path)):
ok += 1
else:
session.execute(
update(ImageRecord)
.where(ImageRecord.id == image_id)
.values(thumbnail_path=None)
)
generate_thumbnail.delay(image_id)
enqueued += 1
regenerated += 1
session.commit()
last_id = rows[-1][0]
return {"enqueued": enqueued, "ok": ok, "regenerated": regenerated}
@@ -0,0 +1,48 @@
<template>
<div class="fc-artist-gallery">
<MasonryGrid
:items="store.images"
:loading="store.imagesLoading"
:has-more="store.hasMoreImages"
@load-more="store.loadMoreImages(props.slug)"
@open="openImage"
/>
</div>
</template>
<script setup>
import { onMounted, watch } from 'vue'
import { useRoute, useRouter } from 'vue-router'
import { useArtistStore } from '../../stores/artist.js'
import { useModalStore } from '../../stores/modal.js'
import MasonryGrid from '../discovery/MasonryGrid.vue'
const props = defineProps({
slug: { type: String, required: true },
})
const store = useArtistStore()
const modal = useModalStore()
const route = useRoute()
const router = useRouter()
onMounted(() => {
const initial = parseInt(route.query.image, 10)
if (!isNaN(initial)) modal.open(initial)
})
watch(() => route.query.image, (q) => {
const id = parseInt(q, 10)
if (!isNaN(id) && id !== modal.currentImageId) modal.open(id)
else if (isNaN(id) && modal.currentImageId !== null) modal.close()
})
function openImage (id) {
router.push({ query: { ...route.query, image: id } })
}
</script>
<style scoped>
.fc-artist-gallery { min-width: 0; }
</style>
@@ -0,0 +1,128 @@
<template>
<header class="fc-artist-header">
<div class="fc-artist-header__left">
<h1 class="fc-artist-header__name">{{ name }}</h1>
<span v-if="stats" class="fc-artist-header__stats">{{ stats }}</span>
</div>
<v-tabs
:model-value="modelValue"
color="accent"
density="compact"
class="fc-artist-header__tabs"
@update:model-value="$emit('update:modelValue', $event)"
>
<v-tab value="posts">
Posts
<span v-if="postCount != null" class="fc-artist-header__tab-count">
({{ postCount }})
</span>
</v-tab>
<v-tab value="gallery">
Gallery
<span v-if="imageCount != null" class="fc-artist-header__tab-count">
({{ imageCount }})
</span>
</v-tab>
<v-tab value="management">Management</v-tab>
</v-tabs>
<!-- Right-side spacer: balances the left cell's flex weight so the
centered tabs stay geometrically centered regardless of the
artist-name length. Mirrors TopNav's 1fr | auto | 1fr layout. -->
<div class="fc-artist-header__right" />
</header>
</template>
<script setup>
import { computed } from 'vue'
const props = defineProps({
name: { type: String, required: true },
imageCount: { type: Number, default: null },
postCount: { type: Number, default: null },
lastAdded: { type: String, default: null },
modelValue: { type: String, required: true },
})
defineEmits(['update:modelValue'])
const stats = computed(() => {
const parts = []
if (props.imageCount != null) {
parts.push(`${props.imageCount} image${props.imageCount === 1 ? '' : 's'}`)
}
if (props.lastAdded) {
parts.push(`last added ${props.lastAdded.slice(0, 10)}`)
}
return parts.join(' · ')
})
</script>
<style scoped>
/* Matches TopNav.vue's frosted recipe exactly. top:48px parks it flush
against TopNav's bottom edge (TopNav is 0.75rem padding + ~24px content
= ~48px tall; operator-flagged 2026-05-26 that top:64px left a visible
gap). The two bars now read as one continuous frosted strip. */
.fc-artist-header {
position: sticky;
top: 48px;
z-index: 4;
display: flex;
align-items: center;
gap: 1rem;
padding: 0.5rem 1rem;
background: linear-gradient(
to bottom,
rgba(20, 23, 26, 0.92) 0%,
rgba(20, 23, 26, 0.65) 60%,
rgba(20, 23, 26, 0) 100%
);
backdrop-filter: blur(2px);
-webkit-backdrop-filter: blur(2px);
}
.fc-artist-header__left {
flex: 1 1 0;
min-width: 0;
display: flex;
align-items: baseline;
gap: 12px;
overflow: hidden;
}
.fc-artist-header__name {
font-family: 'Fraunces', Georgia, serif;
font-size: 24px;
font-weight: 500;
margin: 0;
color: rgb(var(--v-theme-on-surface));
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.fc-artist-header__stats {
font-size: 13px;
color: rgb(var(--v-theme-on-surface-variant));
font-variant-numeric: tabular-nums;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.fc-artist-header__tabs {
flex: 0 0 auto;
}
.fc-artist-header__right {
flex: 1 1 0;
min-width: 0;
}
.fc-artist-header__tab-count {
margin-left: 4px;
font-size: 12px;
color: rgb(var(--v-theme-on-surface-variant));
font-variant-numeric: tabular-nums;
}
</style>
@@ -0,0 +1,133 @@
<template>
<div class="fc-artist-mgmt">
<section class="fc-artist-mgmt__sec">
<h2 class="fc-h2">Overview</h2>
<div class="fc-artist-mgmt__chips">
<v-chip
size="small"
:variant="overview.is_subscription ? 'flat' : 'outlined'"
:color="overview.is_subscription ? 'accent' : undefined"
prepend-icon="mdi-rss"
>{{ overview.is_subscription ? 'Subscription' : 'One-off' }}</v-chip>
<v-chip
size="small" variant="outlined" prepend-icon="mdi-link-variant"
:to="`/subscriptions?artist_id=${overview.id}`"
>{{ overview.sources.length }} subscription{{ overview.sources.length === 1 ? '' : 's' }}</v-chip>
</div>
</section>
<section v-if="overview.cooccurring_tags.length" class="fc-artist-mgmt__sec">
<h2 class="fc-h2">Frequent tags</h2>
<div class="fc-artist-mgmt__tags">
<v-chip
v-for="t in overview.cooccurring_tags" :key="t.id"
size="small" @click="openTag(t.id)"
>{{ t.name }} <span class="fc-artist-mgmt__tagc">{{ t.count }}</span></v-chip>
</div>
</section>
<section v-if="overview.activity.length" class="fc-artist-mgmt__sec">
<h2 class="fc-h2">Activity</h2>
<svg
class="fc-artist-mgmt__spark" :viewBox="`0 0 ${sparkW} ${sparkH}`"
preserveAspectRatio="none" role="img" aria-label="posts over time"
>
<polyline :points="sparkPoints" fill="none"
stroke="rgb(var(--v-theme-accent))" stroke-width="2" />
</svg>
</section>
<section v-if="overview.sources.length" class="fc-artist-mgmt__sec">
<div class="fc-artist-mgmt__sec-head">
<h2 class="fc-h2">Subscriptions</h2>
<RouterLink
:to="`/subscriptions?artist_id=${overview.id}`"
class="fc-artist-mgmt__manage"
>Manage subscriptions </RouterLink>
</div>
<v-table density="compact">
<thead>
<tr><th>Platform</th><th>URL</th><th class="text-right">Images</th></tr>
</thead>
<tbody>
<tr v-for="s in overview.sources" :key="s.id">
<td>{{ s.platform }}</td>
<td class="fc-artist-mgmt__url">{{ s.url }}</td>
<td class="text-right">{{ s.image_count }}</td>
</tr>
</tbody>
</v-table>
</section>
<section class="fc-artist-mgmt__sec">
<h2 class="fc-h2">Danger zone</h2>
<ArtistDangerZone
:slug="overview.slug"
:artist-id="overview.id"
:artist-name="overview.name"
/>
</section>
</div>
</template>
<script setup>
import { computed } from 'vue'
import { useRouter, RouterLink } from 'vue-router'
import ArtistDangerZone from './ArtistDangerZone.vue'
const props = defineProps({
overview: { type: Object, required: true },
})
const router = useRouter()
const sparkW = 600
const sparkH = 80
const sparkPoints = computed(() => {
const a = props.overview.activity ?? []
if (a.length === 0) return ''
const max = Math.max(...a.map(p => p.count), 1)
const stepX = a.length > 1 ? sparkW / (a.length - 1) : 0
return a.map((p, i) => {
const x = i * stepX
const y = sparkH - (p.count / max) * (sparkH - 4) - 2
return `${x.toFixed(1)},${y.toFixed(1)}`
}).join(' ')
})
function openTag (tagId) {
router.push({ name: 'gallery', query: { tag_id: tagId } })
}
</script>
<style scoped>
.fc-artist-mgmt { padding-top: 1rem; }
.fc-h2 {
font-family: 'Fraunces', Georgia, serif;
font-size: 20px; font-weight: 500; margin-bottom: 8px;
}
.fc-artist-mgmt__sec { margin-bottom: 28px; }
.fc-artist-mgmt__chips { display: flex; gap: 8px; flex-wrap: wrap; }
.fc-artist-mgmt__tags { display: flex; flex-wrap: wrap; gap: 6px; }
.fc-artist-mgmt__tagc {
color: rgb(var(--v-theme-on-surface-variant));
margin-left: 4px;
font-variant-numeric: tabular-nums;
}
.fc-artist-mgmt__spark { width: 100%; height: 80px; }
.fc-artist-mgmt__url {
max-width: 380px; overflow: hidden; text-overflow: ellipsis;
white-space: nowrap;
}
.fc-artist-mgmt__sec-head {
display: flex; align-items: baseline; justify-content: space-between;
margin-bottom: 0.25rem;
}
.fc-artist-mgmt__manage {
font-size: 0.85rem;
color: rgb(var(--v-theme-accent));
text-decoration: none;
}
.fc-artist-mgmt__manage:hover { text-decoration: underline; }
</style>
@@ -0,0 +1,90 @@
<template>
<div class="fc-artist-posts">
<v-alert v-if="store.error" type="error" variant="tonal" closable class="mb-3">
{{ String(store.error) }}
</v-alert>
<div v-if="store.loading && store.items.length === 0" class="fc-artist-posts__loading">
<v-progress-circular indeterminate color="accent" size="36" />
</div>
<div v-else-if="store.items.length === 0 && store.done" class="fc-artist-posts__empty">
<p>No posts for this artist yet. Switch to
<a href="#" @click.prevent="$emit('switch-tab', 'gallery')">Gallery</a>
to see imported images, or visit
<RouterLink to="/subscriptions">Subscriptions</RouterLink>
to start capturing posts.
</p>
</div>
<div v-else>
<PostCard v-for="p in store.items" :key="p.id" :post="p" />
<div ref="sentinel" class="fc-artist-posts__sentinel">
<v-progress-circular v-if="store.loading" indeterminate color="accent" size="24" />
<span v-else-if="store.done" class="fc-artist-posts__end">End of stream</span>
</div>
</div>
</div>
</template>
<script setup>
import { onMounted, onUnmounted, ref, watch } from 'vue'
import { RouterLink } from 'vue-router'
import { usePostsStore } from '../../stores/posts.js'
import PostCard from '../posts/PostCard.vue'
const props = defineProps({
artistId: { type: Number, required: true },
})
defineEmits(['switch-tab'])
const store = usePostsStore()
const sentinel = ref(null)
let observer = null
async function reload () {
await store.loadInitial({ artist_id: props.artistId, platform: null })
}
watch(() => props.artistId, reload)
onMounted(async () => {
await reload()
observer = new IntersectionObserver((entries) => {
if (entries.some(e => e.isIntersecting)) {
store.loadMore()
}
}, { rootMargin: '400px 0px' })
if (sentinel.value) observer.observe(sentinel.value)
})
onUnmounted(() => {
if (observer) observer.disconnect()
})
</script>
<style scoped>
.fc-artist-posts {
max-width: 900px;
margin: 0 auto;
}
.fc-artist-posts__loading,
.fc-artist-posts__empty {
display: flex;
justify-content: center;
padding: 2rem;
color: rgb(var(--v-theme-on-surface-variant));
}
.fc-artist-posts__sentinel {
display: flex;
justify-content: center;
padding: 1.5rem 0;
min-height: 2rem;
}
.fc-artist-posts__end {
color: rgb(var(--v-theme-on-surface-variant));
font-size: 0.85rem;
}
</style>
@@ -4,14 +4,20 @@
<v-card>
<v-card-title class="d-flex align-center" style="gap: 12px;">
<v-icon icon="mdi-alert-circle-outline" color="error" />
<span>{{ title }}</span>
<span>{{ displayTitle }}</span>
<v-spacer />
<v-btn icon variant="text" size="small" @click="close">
<v-icon>mdi-close</v-icon>
</v-btn>
</v-card-title>
<v-card-text>
<pre class="fc-err-pre">{{ message || '(no error message)' }}</pre>
<dl v-if="contextRows.length" class="fc-err-context">
<template v-for="(row, idx) in contextRows" :key="idx">
<dt>{{ row[0] }}</dt>
<dd>{{ row[1] }}</dd>
</template>
</dl>
<pre class="fc-err-pre">{{ displayMessage || '(no error message)' }}</pre>
</v-card-text>
<v-card-actions>
<v-btn
@@ -27,12 +33,21 @@
</template>
<script setup>
import { ref, watch } from 'vue'
import { computed, ref, watch } from 'vue'
import { copyText } from '../../utils/clipboard.js'
const props = defineProps({
modelValue: { type: Boolean, default: false },
// Legacy mode: pass title + message strings. Used by callers whose row
// shape lacks structured context (e.g. ImportTaskList where `error` is
// a plain string on the import_task row, not a TaskRun).
title: { type: String, default: 'Error details' },
message: { type: String, default: '' },
// Row mode: pass the full TaskRun-shaped dict from /api/system_activity.
// When set, displayTitle/displayMessage derive from the row and a context
// panel of task_name/queue/target/duration/etc. renders above the error.
row: { type: Object, default: null },
})
const emit = defineEmits(['update:modelValue'])
@@ -40,6 +55,47 @@ const emit = defineEmits(['update:modelValue'])
const copied = ref(false)
let copiedTimer = null
const displayTitle = computed(() => {
if (props.row) return props.row.error_type || 'Error details'
return props.title
})
const displayMessage = computed(() => {
if (props.row) return props.row.error_message || ''
return props.message
})
function _shortTaskName (name) {
if (!name) return ''
const parts = String(name).split('.')
return parts[parts.length - 1]
}
function _formatDuration (ms) {
if (ms == null) return null
if (ms < 1000) return `${ms} ms`
if (ms < 60_000) return `${(ms / 1000).toFixed(1)} s`
return `${(ms / 60_000).toFixed(1)} min`
}
const contextRows = computed(() => {
const r = props.row
if (!r) return []
const rows = []
if (r.task_name) rows.push(['Task', _shortTaskName(r.task_name)])
if (r.queue) rows.push(['Queue', r.queue])
if (r.target_id != null) rows.push(['Target', r.target_id])
const dur = _formatDuration(r.duration_ms)
if (dur != null) rows.push(['Duration', dur])
if (r.started_at) rows.push(['Started', r.started_at])
if (r.finished_at) rows.push(['Finished', r.finished_at])
if (r.retry_count) rows.push(['Retries', r.retry_count])
if (r.worker_hostname) rows.push(['Worker', r.worker_hostname])
if (r.celery_task_id) rows.push(['Celery ID', r.celery_task_id])
if (r.args_summary) rows.push(['Args', r.args_summary])
return rows
})
watch(() => props.modelValue, (open) => {
if (!open) {
copied.value = false
@@ -47,13 +103,18 @@ watch(() => props.modelValue, (open) => {
}
})
function close() {
function close () {
emit('update:modelValue', false)
}
async function onCopy() {
async function onCopy () {
let text = displayMessage.value || ''
if (contextRows.value.length) {
const header = contextRows.value.map(([k, v]) => `${k}: ${v}`).join('\n')
text = `${header}\n\nError: ${displayTitle.value}\n${text}`
}
try {
await navigator.clipboard.writeText(props.message || '')
await copyText(text)
copied.value = true
if (copiedTimer) clearTimeout(copiedTimer)
copiedTimer = setTimeout(() => { copied.value = false }, 1500)
@@ -64,19 +125,43 @@ async function onCopy() {
</script>
<style scoped>
/* The full error often contains a SQLAlchemy statement + parameters
block + multi-line traceback. Pre-wrap keeps long lines readable;
monospace + tabular layout keeps the structure scannable. The
max-height + overflow-auto prevents a 50-line traceback from
pushing the Close button off-screen. Operator-flagged 2026-05-26:
the prior :title="..." tooltip was unusable for content this long. */
/* Context panel: muted labels in vellum, crisp values in parchment. The
2-column key/value grid keeps rows visually scannable when there are
many fields (Task, Queue, Target, Duration, Started, Finished, Retries,
Worker, Celery ID, Args). */
.fc-err-context {
display: grid;
grid-template-columns: max-content 1fr;
column-gap: 14px;
row-gap: 4px;
margin: 0 0 14px 0;
font-size: 13px;
}
.fc-err-context dt {
color: rgb(var(--v-theme-on-surface-variant));
font-weight: 500;
white-space: nowrap;
}
.fc-err-context dd {
color: rgb(var(--v-theme-on-surface));
margin: 0;
font-variant-numeric: tabular-nums;
word-break: break-all;
}
/* Error pre block: high-contrast pairing. The page's `background` token
(obsidian #14171A) is darker than the modal card's `surface` (iron
#1E2228), so parchment text reads crisply against it. The prior pairing
used `surface-variant` which Vuetify auto-derives to a near-parchment
light value in this theme — pale-on-pale and unreadable. Operator-
flagged 2026-05-26 ("ui contrast is poor"). */
.fc-err-pre {
font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
font-family: 'JetBrains Mono', ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
font-size: 12px;
line-height: 1.5;
white-space: pre-wrap;
word-break: break-word;
background: rgb(var(--v-theme-surface-variant, 38 36 41));
background: rgb(var(--v-theme-background));
color: rgb(var(--v-theme-on-surface));
padding: 12px 14px;
border-radius: 6px;
@@ -28,6 +28,7 @@
<script setup>
import { onMounted, ref } from 'vue'
import { useCredentialsStore } from '../../stores/credentials.js'
import { copyText } from '../../utils/clipboard.js'
const store = useCredentialsStore()
const showRotateConfirm = ref(false)
@@ -37,7 +38,7 @@ onMounted(() => store.loadKey())
async function copyKey() {
if (!store.extensionKey) return
try {
await navigator.clipboard.writeText(store.extensionKey)
await copyText(store.extensionKey)
globalThis.window?.__fcToast?.({ text: 'Copied', type: 'success' })
} catch {
globalThis.window?.__fcToast?.({ text: 'Copy failed', type: 'error' })
@@ -110,6 +110,7 @@
<script setup>
import { computed, onMounted, ref } from 'vue'
import { useApi } from '../../composables/useApi.js'
import { copyText } from '../../utils/clipboard.js'
const api = useApi()
@@ -172,7 +173,7 @@ async function rotateKey() {
async function copy(text, label) {
try {
await navigator.clipboard.writeText(text)
await copyText(text)
window.__fcToast?.({ text: `${label} copied.`, type: 'success' })
} catch (e) {
window.__fcToast?.({ text: `Copy failed: ${e.message}`, type: 'error' })
@@ -1,13 +1,15 @@
<template>
<div class="fc-maint">
<p class="text-body-2 mb-4">
Machine-assisted tagging controls. Backfill and centroid recompute run
nightly automatically; the allowlist auto-applies accepted tags to new
and existing images.
Operational backfills and tagging controls. The ML backfill and centroid
recompute run nightly automatically; the allowlist auto-applies accepted
tags to new and existing images. Use the cards below to trigger a
one-off pass.
</p>
<div class="fc-maint__grid">
<MLBackfillCard />
<CentroidRecomputeCard />
<ThumbnailBackfillCard />
</div>
<MLThresholdSliders class="mt-4" />
<AllowlistTable class="mt-4" />
@@ -23,6 +25,7 @@
<script setup>
import MLBackfillCard from './MLBackfillCard.vue'
import CentroidRecomputeCard from './CentroidRecomputeCard.vue'
import ThumbnailBackfillCard from './ThumbnailBackfillCard.vue'
import MLThresholdSliders from './MLThresholdSliders.vue'
import AllowlistTable from './AllowlistTable.vue'
import AliasTable from './AliasTable.vue'
@@ -62,7 +62,7 @@
<td>
<button
type="button" class="fc-err-link"
@click="openError(r.error_type, r.error_message)"
@click="openError(r)"
:title="'Click for full error'"
>{{ r.error_type }}</button>
</td>
@@ -145,8 +145,7 @@
<ErrorDetailModal
v-model="showErrorModal"
:title="errorModalTitle"
:message="errorModalMessage"
:row="errorModalRow"
/>
</div>
</template>
@@ -163,12 +162,10 @@ import QueuesTable from './QueuesTable.vue'
// rollback + traceback content rendered as a cramped browser tooltip
// you couldn't copy from or scroll within).
const showErrorModal = ref(false)
const errorModalTitle = ref('')
const errorModalMessage = ref('')
const errorModalRow = ref(null)
function openError(title, message) {
errorModalTitle.value = title || 'Error details'
errorModalMessage.value = message || ''
function openError(row) {
errorModalRow.value = row
showErrorModal.value = true
}
@@ -0,0 +1,30 @@
<template>
<v-card>
<v-card-title>Thumbnail backfill</v-card-title>
<v-card-text>
<p class="text-body-2 mb-3">
Scan the library for images with no thumbnail, or whose thumbnail file
is missing or corrupt on disk. Repair candidates are re-enqueued for
thumbnail generation. Safe to re-run.
</p>
<v-btn color="primary" rounded="pill" :loading="busy" @click="run">
<v-icon start>mdi-image-refresh</v-icon> Run backfill now
</v-btn>
<span v-if="done" class="ml-3 text-caption">Enqueued.</span>
</v-card-text>
</v-card>
</template>
<script setup>
import { ref } from 'vue'
import { useThumbnailsStore } from '../../stores/thumbnails.js'
const store = useThumbnailsStore()
const busy = ref(false)
const done = ref(false)
async function run () {
busy.value = true
try { await store.triggerBackfill(); done.value = true }
catch (e) { window.__fcToast?.({ text: e.message, type: 'error' }) }
finally { busy.value = false }
}
</script>
+13 -3
View File
@@ -1,6 +1,7 @@
import { defineStore } from 'pinia'
import { ref, computed } from 'vue'
import { useApi } from '../composables/useApi.js'
import { usePostsStore } from './posts.js'
const PAGE = 60
@@ -15,7 +16,11 @@ export const useArtistStore = defineStore('artist', () => {
const notFound = ref(false)
let started = false
async function load(slug) {
async function load (slug) {
// Cross-artist reset: clear this store AND the posts store so the new
// artist doesn't briefly render with the previous artist's content
// when the user is on the Posts tab. (Gallery tab uses this artist
// store's own images list — cleared above.)
overview.value = null
images.value = []
nextCursor.value = null
@@ -23,6 +28,7 @@ export const useArtistStore = defineStore('artist', () => {
started = false
error.value = null
loading.value = true
usePostsStore().$reset?.()
try {
overview.value = await api.get(`/api/artist/${encodeURIComponent(slug)}`)
await loadMoreImages(slug)
@@ -34,7 +40,7 @@ export const useArtistStore = defineStore('artist', () => {
}
}
async function loadMoreImages(slug) {
async function loadMoreImages (slug) {
if (imagesLoading.value) return
if (started && nextCursor.value === null) return
imagesLoading.value = true
@@ -55,9 +61,13 @@ export const useArtistStore = defineStore('artist', () => {
}
const hasMoreImages = computed(() => !started || nextCursor.value !== null)
const postCount = computed(() => overview.value?.post_count ?? null)
const imageCount = computed(() => overview.value?.image_count ?? null)
const lastAdded = computed(() => overview.value?.date_range?.max ?? null)
return {
overview, images, loading, imagesLoading, error, notFound,
hasMoreImages, load, loadMoreImages
hasMoreImages, postCount, imageCount, lastAdded,
load, loadMoreImages,
}
})
+12
View File
@@ -0,0 +1,12 @@
import { defineStore } from 'pinia'
import { useApi } from '../composables/useApi.js'
export const useThumbnailsStore = defineStore('thumbnails', () => {
const api = useApi()
async function triggerBackfill () {
await api.post('/api/thumbnails/backfill')
}
return { triggerBackfill }
})
+35
View File
@@ -0,0 +1,35 @@
// Clipboard write that works on plain-HTTP self-hosted deployments.
//
// navigator.clipboard is gated by the browser's Secure Context restriction
// (HTTPS or localhost only). FabledCurator runs over plain HTTP per the
// homelab posture, so the modern API is undefined in production. We fall
// back to the legacy execCommand('copy') path via a temporary off-screen
// textarea — wide browser support, no HTTPS requirement.
export async function copyText (text) {
const str = text == null ? '' : String(text)
if (typeof navigator !== 'undefined' && navigator.clipboard?.writeText) {
try {
await navigator.clipboard.writeText(str)
return
} catch {
// Fall through to the legacy path. Some browsers throw even when
// the API exists (permission denied, focus loss, etc.).
}
}
const ta = document.createElement('textarea')
ta.value = str
ta.setAttribute('readonly', '')
ta.style.position = 'fixed'
ta.style.top = '0'
ta.style.left = '0'
ta.style.opacity = '0'
ta.style.pointerEvents = 'none'
document.body.appendChild(ta)
ta.select()
ta.setSelectionRange(0, str.length)
let ok = false
try { ok = document.execCommand('copy') } catch { ok = false }
document.body.removeChild(ta)
if (!ok) throw new Error('clipboard copy not supported')
}
+65 -189
View File
@@ -1,220 +1,96 @@
<template>
<v-container fluid class="py-6">
<div v-if="store.loading && !store.overview" class="fc-artist__loading">
<v-progress-circular indeterminate color="accent" size="36" />
</div>
<div v-if="store.loading && !store.overview" class="fc-artist__loading">
<v-progress-circular indeterminate color="accent" size="36" />
</div>
<v-alert v-else-if="store.notFound" type="warning" variant="tonal">
Artist not found.
</v-alert>
<v-container v-else-if="store.notFound" class="pt-2 pb-6">
<v-alert type="warning" variant="tonal">Artist not found.</v-alert>
</v-container>
<v-alert v-else-if="store.error" type="error" variant="tonal" closable>
{{ store.error }}
</v-alert>
<template v-else-if="store.overview">
<header class="fc-artist__head">
<h1 class="fc-h1">{{ store.overview.name }}</h1>
<div class="fc-artist__stats">
<span>{{ store.overview.image_count }} images</span>
<span v-if="dateRange">· {{ dateRange }}</span>
</div>
</header>
<div class="fc-artist__fc4">
<v-chip
size="small"
:variant="store.overview.is_subscription ? 'flat' : 'outlined'"
:color="store.overview.is_subscription ? 'accent' : undefined"
prepend-icon="mdi-rss"
>{{ store.overview.is_subscription ? 'Subscription' : 'One-off' }}</v-chip>
<v-chip
size="small" variant="outlined" prepend-icon="mdi-link-variant"
:to="`/subscriptions?artist_id=${store.overview.id}`"
>{{ store.overview.sources.length }} source{{ store.overview.sources.length === 1 ? '' : 's' }}</v-chip>
<v-chip
size="small" variant="outlined" prepend-icon="mdi-rss"
:to="`/posts?artist_id=${store.overview.id}`"
>View posts</v-chip>
<v-chip
size="small" variant="outlined" disabled
prepend-icon="mdi-clock-outline"
>Credential health · FC-3b</v-chip>
</div>
<!-- Tabs split (2026-05-25): Settings was previously slotted at the
bottom of the page after the infinite-scroll image grid, which
made it effectively unreachable for any artist with more than
a couple of pages of content. The Settings tab now hosts
destructive admin actions (artist+content cascade-delete) and
any future per-artist management UI. v-tabs is `position:
sticky; top: 64px` (under the 64px AppShell TopNav) so it
stays parked while the gallery scrolls. -->
<v-tabs
v-model="tab" color="accent" class="mb-4"
style="position: sticky; top: 64px; z-index: 4;
background: rgb(var(--v-theme-surface));"
>
<v-tab value="overview">Overview</v-tab>
<v-tab value="settings">Settings</v-tab>
</v-tabs>
<v-container v-else-if="store.error && !store.overview" class="pt-2 pb-6">
<v-alert type="error" variant="tonal" closable>{{ store.error }}</v-alert>
</v-container>
<template v-else-if="store.overview">
<ArtistHeader
v-model="tab"
:name="store.overview.name"
:image-count="store.imageCount"
:post-count="store.postCount"
:last-added="store.lastAdded"
/>
<v-container fluid class="pt-2 pb-4">
<v-window v-model="tab">
<v-window-item value="overview">
<section v-if="store.overview.cooccurring_tags.length" class="fc-artist__sec">
<h2 class="fc-h2">Frequent tags</h2>
<div class="fc-artist__tags">
<v-chip
v-for="t in store.overview.cooccurring_tags" :key="t.id"
size="small" @click="openTag(t.id)"
>{{ t.name }} <span class="fc-artist__tagc">{{ t.count }}</span></v-chip>
</div>
</section>
<section v-if="store.overview.activity.length" class="fc-artist__sec">
<h2 class="fc-h2">Activity</h2>
<svg class="fc-artist__spark" :viewBox="`0 0 ${sparkW} ${sparkH}`"
preserveAspectRatio="none" role="img" aria-label="posts over time">
<polyline :points="sparkPoints" fill="none"
stroke="rgb(var(--v-theme-accent))" stroke-width="2" />
</svg>
</section>
<section v-if="store.overview.sources.length" class="fc-artist__sec">
<div class="fc-artist__sec-head">
<h2 class="fc-h2">Sources</h2>
<RouterLink
:to="`/subscriptions?artist_id=${store.overview.id}`"
class="fc-artist__manage"
>Manage subscriptions </RouterLink>
</div>
<v-table density="compact">
<thead>
<tr><th>Platform</th><th>URL</th><th class="text-right">Images</th></tr>
</thead>
<tbody>
<tr v-for="s in store.overview.sources" :key="s.id">
<td>{{ s.platform }}</td>
<td class="fc-artist__url">{{ s.url }}</td>
<td class="text-right">{{ s.image_count }}</td>
</tr>
</tbody>
</v-table>
</section>
<section class="fc-artist__sec">
<h2 class="fc-h2">Images</h2>
<MasonryGrid
:items="store.images"
:loading="store.imagesLoading"
:has-more="store.hasMoreImages"
@load-more="store.loadMoreImages(slug)"
@open="openImage"
/>
</section>
</v-window-item>
<v-window-item value="settings">
<ArtistDangerZone
:slug="slug"
<v-window-item value="posts">
<ArtistPostsTab
:artist-id="store.overview.id"
:artist-name="store.overview.name"
@switch-tab="(t) => tab = t"
/>
</v-window-item>
<v-window-item value="gallery">
<ArtistGalleryTab :slug="slug" />
</v-window-item>
<v-window-item value="management">
<ArtistManagementTab :overview="store.overview" />
</v-window-item>
</v-window>
</template>
</v-container>
</v-container>
</template>
</template>
<script setup>
import { computed, ref, watch } from 'vue'
import { useRoute, useRouter, RouterLink } from 'vue-router'
import { useRoute, useRouter } from 'vue-router'
import { useArtistStore } from '../stores/artist.js'
import { useModalStore } from '../stores/modal.js'
import MasonryGrid from '../components/discovery/MasonryGrid.vue'
import ArtistDangerZone from '../components/artist/ArtistDangerZone.vue'
import ArtistHeader from '../components/artist/ArtistHeader.vue'
import ArtistPostsTab from '../components/artist/ArtistPostsTab.vue'
import ArtistGalleryTab from '../components/artist/ArtistGalleryTab.vue'
import ArtistManagementTab from '../components/artist/ArtistManagementTab.vue'
const VALID_TABS = ['posts', 'gallery', 'management']
const route = useRoute()
const router = useRouter()
const store = useArtistStore()
const modal = useModalStore()
const slug = computed(() => route.params.slug)
// Per-artist tab — defaults to Overview. Settings tab hosts destructive
// admin actions (DangerZone). Switching artists resets to Overview so the
// destructive surface isn't re-shown by accident when navigating between
// artists.
const tab = ref('overview')
const tab = ref('posts')
watch(slug, (s) => {
if (s) {
store.load(s)
tab.value = 'overview'
}
function resolveDefaultTab () {
const fromUrl = route.query.tab
if (VALID_TABS.includes(fromUrl)) return fromUrl
if ((store.postCount ?? 0) > 0) return 'posts'
return 'gallery'
}
watch(slug, async (s) => {
if (!s) return
await store.load(s)
document.title = store.overview
? `${store.overview.name} — FabledCurator`
: 'FabledCurator'
tab.value = resolveDefaultTab()
}, { immediate: true })
const dateRange = computed(() => {
const r = store.overview?.date_range
if (!r || !r.min) return null
const fmt = (iso) => iso.slice(0, 10)
return r.min === r.max ? fmt(r.min) : `${fmt(r.min)}${fmt(r.max)}`
// Reflect tab changes back into the URL so refresh/back/forward work.
watch(tab, (newTab) => {
if (route.query.tab === newTab) return
router.replace({
query: { ...route.query, tab: newTab },
})
})
const sparkW = 600
const sparkH = 80
const sparkPoints = computed(() => {
const a = store.overview?.activity ?? []
if (a.length === 0) return ''
const max = Math.max(...a.map(p => p.count), 1)
const stepX = a.length > 1 ? sparkW / (a.length - 1) : 0
return a.map((p, i) => {
const x = i * stepX
const y = sparkH - (p.count / max) * (sparkH - 4) - 2
return `${x.toFixed(1)},${y.toFixed(1)}`
}).join(' ')
// React to URL-tab changes (e.g., back/forward).
watch(() => route.query.tab, (q) => {
if (q && VALID_TABS.includes(q) && tab.value !== q) {
tab.value = q
}
})
function openImage(id) {
modal.open(id)
}
function openTag(tagId) {
router.push({ name: 'gallery', query: { tag_id: tagId } })
}
</script>
<style scoped>
.fc-h1 {
font-family: 'Fraunces', Georgia, serif;
font-size: 32px; font-weight: 500;
color: rgb(var(--v-theme-on-surface));
.fc-artist__loading {
display: flex; justify-content: center; padding: 64px 0;
}
.fc-h2 {
font-family: 'Fraunces', Georgia, serif;
font-size: 20px; font-weight: 500; margin-bottom: 8px;
}
.fc-artist__loading { display: flex; justify-content: center; padding: 64px 0; }
.fc-artist__head { margin-bottom: 12px; }
.fc-artist__stats { opacity: 0.75; margin-top: 4px; }
.fc-artist__fc4 { display: flex; gap: 8px; flex-wrap: wrap; margin-bottom: 24px; }
.fc-artist__sec { margin-bottom: 28px; }
.fc-artist__tags { display: flex; flex-wrap: wrap; gap: 6px; }
.fc-artist__tagc { opacity: 0.6; margin-left: 4px; font-variant-numeric: tabular-nums; }
.fc-artist__spark { width: 100%; height: 80px; }
.fc-artist__url {
max-width: 380px; overflow: hidden; text-overflow: ellipsis;
white-space: nowrap;
}
.fc-artist__sec-head {
display: flex; align-items: baseline; justify-content: space-between;
margin-bottom: 0.25rem;
}
.fc-artist__manage {
font-size: 0.85rem;
color: rgb(var(--v-theme-accent));
text-decoration: none;
}
.fc-artist__manage:hover { text-decoration: underline; }
</style>
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<div class="fc-artists__controls">
<v-text-field
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<ExtensionKeyBar />
<v-alert v-if="credentialsStore.error" type="error" variant="tonal" closable class="mb-4">
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<FilterPills v-model="pill" />
<v-alert v-if="store.error" type="error" variant="tonal" closable class="my-4">
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<Teleport to="#fc-nav-actions">
<v-btn
:color="sel.isSelectMode ? 'accent' : undefined"
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container class="py-8">
<v-container class="pt-3 pb-8">
<h1 class="fc-h1 mb-4">{{ title }}</h1>
<v-alert type="info" variant="tonal" icon="mdi-toolbox">
This surface is a placeholder. It will be implemented in
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container class="py-6" max-width="900">
<v-container class="pt-2 pb-6" max-width="900">
<PostsFilterBar
:artist-id="artistFilter"
:platform="platformFilter"
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<div class="fc-series__head">
<span class="fc-series__name">{{ store.series?.name || 'Series' }}</span>
<span class="fc-series__count">{{ store.pages.length }} page(s)</span>
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<!-- Sticky tabs: operator-flagged 2026-05-25 long Import / Maintenance
panels pushed the tab strip out of the viewport, forcing a scroll-
to-top just to change tab. AppShell's TopNav is 64px sticky, so the
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<Teleport to="#fc-nav-actions">
<v-btn
prepend-icon="mdi-shuffle-variant" variant="tonal" color="accent"
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<div class="fc-subs__bar">
<v-btn color="accent" prepend-icon="mdi-plus" @click="openAddSource(null)">
Add subscription
+1 -1
View File
@@ -1,5 +1,5 @@
<template>
<v-container fluid class="py-6">
<v-container fluid class="pt-2 pb-6">
<div class="fc-tags__controls">
<v-text-field
+24
View File
@@ -29,6 +29,30 @@ async def test_artist_overview_ok(client, db):
body = await resp.get_json()
assert body["name"] == "Mira"
assert body["image_count"] == 0
assert body["post_count"] == 0
@pytest.mark.asyncio
async def test_artist_overview_post_count(client, db):
from backend.app.models import Post, Source
a = Artist(name="Lyra", slug="lyra")
db.add(a)
await db.flush()
s = Source(
artist_id=a.id, platform="patreon",
url="https://patreon.com/cw/lyra", enabled=True,
)
db.add(s)
await db.flush()
db.add(Post(source_id=s.id, external_post_id="p1"))
db.add(Post(source_id=s.id, external_post_id="p2"))
await db.flush()
await db.commit()
resp = await client.get("/api/artist/lyra")
assert resp.status_code == 200
body = await resp.get_json()
assert body["post_count"] == 2
@pytest.mark.asyncio
+32
View File
@@ -0,0 +1,32 @@
import pytest
from backend.app import create_app
from backend.app.celery_app import celery
pytestmark = pytest.mark.integration
@pytest.fixture(autouse=True)
def eager():
celery.conf.task_always_eager = True
yield
celery.conf.task_always_eager = False
@pytest.fixture
async def app():
return create_app()
@pytest.fixture
async def client(app):
async with app.test_client() as c:
yield c
@pytest.mark.asyncio
async def test_trigger_thumbnail_backfill(client):
r = await client.post("/api/thumbnails/backfill")
assert r.status_code == 202
body = await r.get_json()
assert "celery_task_id" in body
+242
View File
@@ -0,0 +1,242 @@
"""Thumbnail backfill: _thumb_is_valid helper + backfill_thumbnails planner."""
from pathlib import Path
import pytest
from backend.app.models import ImageRecord
from backend.app.tasks.thumbnail import _thumb_is_valid
pytestmark = pytest.mark.integration
def test_thumb_is_valid_jpeg(tmp_path):
p = tmp_path / "good.jpg"
p.write_bytes(b"\xff\xd8\xff\xe0" + b"\x00" * 100)
assert _thumb_is_valid(p) is True
def test_thumb_is_valid_png(tmp_path):
p = tmp_path / "good.png"
p.write_bytes(b"\x89PNG\r\n\x1a\n" + b"\x00" * 100)
assert _thumb_is_valid(p) is True
def test_thumb_is_valid_too_short(tmp_path):
p = tmp_path / "tiny"
p.write_bytes(b"\xff\xd8")
assert _thumb_is_valid(p) is False
def test_thumb_is_valid_wrong_magic(tmp_path):
p = tmp_path / "garbage"
p.write_bytes(b"\x00" * 12)
assert _thumb_is_valid(p) is False
def test_thumb_is_valid_missing_file(tmp_path):
assert _thumb_is_valid(tmp_path / "nope") is False
# --- backfill_thumbnails planner tests ------------------------------------
class _Ctx:
def __init__(self, s):
self.s = s
def __enter__(self):
return self.s
def __exit__(self, *a):
return False
def _sf(db_sync):
"""sessionmaker-like returning the test's bound session, matching the
pattern in tests/test_backfill_phash.py."""
class _SM:
def __call__(self):
return _Ctx(db_sync)
return _SM()
def _sha(prefix: str) -> str:
return f"{prefix}".ljust(64, "0")[:64]
def _rec(db_sync, path, *, sha, thumb_path=None, mime="image/jpeg"):
rec = ImageRecord(
path=str(path), sha256=sha, size_bytes=1, mime=mime,
width=64, height=64, origin="imported_filesystem",
integrity_status="unknown",
thumbnail_path=str(thumb_path) if thumb_path is not None else None,
)
db_sync.add(rec)
db_sync.flush()
return rec
def _write_jpeg(p: Path) -> Path:
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(b"\xff\xd8\xff\xe0" + b"\x00" * 100)
return p
def _write_png(p: Path) -> Path:
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(b"\x89PNG\r\n\x1a\n" + b"\x00" * 100)
return p
def _write_garbage(p: Path) -> Path:
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(b"\x00" * 12)
return p
def test_backfill_null_path_enqueued(db_sync, tmp_path, monkeypatch):
from backend.app.tasks import thumbnail as m
src = tmp_path / "a.bin"
src.write_bytes(b"x")
rec = _rec(db_sync, src, sha=_sha("a"), thumb_path=None)
db_sync.commit()
monkeypatch.setattr(m, "_sync_session_factory", lambda: _sf(db_sync))
delayed: list[int] = []
monkeypatch.setattr(
m.generate_thumbnail, "delay", lambda image_id: delayed.append(image_id)
)
result = m.backfill_thumbnails()
assert result == {"enqueued": 1, "ok": 0, "regenerated": 0}
assert delayed == [rec.id]
def test_backfill_missing_file_clears_and_enqueues(db_sync, tmp_path, monkeypatch):
from backend.app.tasks import thumbnail as m
src = tmp_path / "b.bin"
src.write_bytes(b"x")
rec = _rec(
db_sync, src, sha=_sha("b"),
thumb_path=tmp_path / "thumbs" / "missing.jpg",
)
db_sync.commit()
monkeypatch.setattr(m, "_sync_session_factory", lambda: _sf(db_sync))
delayed: list[int] = []
monkeypatch.setattr(
m.generate_thumbnail, "delay", lambda image_id: delayed.append(image_id)
)
result = m.backfill_thumbnails()
db_sync.expire_all()
assert result == {"enqueued": 1, "ok": 0, "regenerated": 1}
assert delayed == [rec.id]
assert db_sync.get(ImageRecord, rec.id).thumbnail_path is None
def test_backfill_valid_jpeg_skipped(db_sync, tmp_path, monkeypatch):
from backend.app.tasks import thumbnail as m
src = tmp_path / "c.bin"
src.write_bytes(b"x")
thumb = _write_jpeg(tmp_path / "thumbs" / "c.jpg")
rec = _rec(db_sync, src, sha=_sha("c"), thumb_path=thumb)
db_sync.commit()
monkeypatch.setattr(m, "_sync_session_factory", lambda: _sf(db_sync))
delayed: list[int] = []
monkeypatch.setattr(
m.generate_thumbnail, "delay", lambda image_id: delayed.append(image_id)
)
result = m.backfill_thumbnails()
db_sync.expire_all()
assert result == {"enqueued": 0, "ok": 1, "regenerated": 0}
assert delayed == []
assert db_sync.get(ImageRecord, rec.id).thumbnail_path == str(thumb)
def test_backfill_valid_png_skipped(db_sync, tmp_path, monkeypatch):
from backend.app.tasks import thumbnail as m
src = tmp_path / "d.bin"
src.write_bytes(b"x")
thumb = _write_png(tmp_path / "thumbs" / "d.png")
_rec(db_sync, src, sha=_sha("d"), thumb_path=thumb)
db_sync.commit()
monkeypatch.setattr(m, "_sync_session_factory", lambda: _sf(db_sync))
delayed: list[int] = []
monkeypatch.setattr(
m.generate_thumbnail, "delay", lambda image_id: delayed.append(image_id)
)
result = m.backfill_thumbnails()
assert result == {"enqueued": 0, "ok": 1, "regenerated": 0}
assert delayed == []
def test_backfill_corrupt_magic_clears_and_enqueues(db_sync, tmp_path, monkeypatch):
from backend.app.tasks import thumbnail as m
src = tmp_path / "e.bin"
src.write_bytes(b"x")
thumb = _write_garbage(tmp_path / "thumbs" / "e.jpg")
rec = _rec(db_sync, src, sha=_sha("e"), thumb_path=thumb)
db_sync.commit()
monkeypatch.setattr(m, "_sync_session_factory", lambda: _sf(db_sync))
delayed: list[int] = []
monkeypatch.setattr(
m.generate_thumbnail, "delay", lambda image_id: delayed.append(image_id)
)
result = m.backfill_thumbnails()
db_sync.expire_all()
assert result == {"enqueued": 1, "ok": 0, "regenerated": 1}
assert delayed == [rec.id]
assert db_sync.get(ImageRecord, rec.id).thumbnail_path is None
def test_backfill_mixed_aggregate(db_sync, tmp_path, monkeypatch):
from backend.app.tasks import thumbnail as m
src_null = tmp_path / "src_null.bin"
src_null.write_bytes(b"x")
src_jpeg = tmp_path / "src_jpeg.bin"
src_jpeg.write_bytes(b"x")
src_png = tmp_path / "src_png.bin"
src_png.write_bytes(b"x")
src_missing = tmp_path / "src_missing.bin"
src_missing.write_bytes(b"x")
src_bad = tmp_path / "src_bad.bin"
src_bad.write_bytes(b"x")
jpeg = _write_jpeg(tmp_path / "thumbs" / "ok.jpg")
png = _write_png(tmp_path / "thumbs" / "ok.png")
bad = _write_garbage(tmp_path / "thumbs" / "bad.jpg")
r_null = _rec(db_sync, src_null, sha=_sha("aa"), thumb_path=None)
_rec(db_sync, src_jpeg, sha=_sha("bb"), thumb_path=jpeg)
_rec(db_sync, src_png, sha=_sha("cc"), thumb_path=png)
r_missing = _rec(
db_sync, src_missing, sha=_sha("dd"),
thumb_path=tmp_path / "thumbs" / "missing.jpg",
)
r_bad = _rec(db_sync, src_bad, sha=_sha("ee"), thumb_path=bad)
db_sync.commit()
monkeypatch.setattr(m, "_sync_session_factory", lambda: _sf(db_sync))
delayed: list[int] = []
monkeypatch.setattr(
m.generate_thumbnail, "delay", lambda image_id: delayed.append(image_id)
)
result = m.backfill_thumbnails()
assert result == {"enqueued": 3, "ok": 2, "regenerated": 2}
assert sorted(delayed) == sorted([r_null.id, r_missing.id, r_bad.id])
+86
View File
@@ -0,0 +1,86 @@
"""CORS preflight + response headers for moz-extension:// + chrome-extension://.
Operator-flagged 2026-05-26: extension's first 'Test connection' tap
returned `NetworkError` because /api/credentials had no OPTIONS handler
and no Access-Control-Allow-Origin response header. Browser preflight
failed → fetch blocked.
These tests pin the contract: any request from a moz-extension:// or
chrome-extension:// origin gets the right ACL headers. Plain browser
requests (no Origin header, or a regular https:// Origin) get nothing
— we don't want to open CORS up generally.
"""
import pytest
from backend.app import create_app
pytestmark = pytest.mark.integration
@pytest.fixture
async def client():
app = create_app()
async with app.test_client() as c:
yield c
@pytest.mark.asyncio
async def test_extension_preflight_returns_204_with_acl_headers(client):
resp = await client.options(
"/api/credentials",
headers={
"Origin": "moz-extension://abcd1234-uuid-fake",
"Access-Control-Request-Method": "GET",
"Access-Control-Request-Headers": "X-Extension-Key",
},
)
assert resp.status_code == 204
assert resp.headers["Access-Control-Allow-Origin"] == "moz-extension://abcd1234-uuid-fake"
assert "OPTIONS" in resp.headers["Access-Control-Allow-Methods"]
assert "X-Extension-Key" in resp.headers["Access-Control-Allow-Headers"]
@pytest.mark.asyncio
async def test_extension_get_carries_acl_headers(client):
# The actual response (post-preflight) also needs ACL headers — the
# browser checks them again before exposing the response body.
resp = await client.get(
"/api/credentials",
headers={"Origin": "moz-extension://abcd1234-uuid-fake"},
)
# 200 with empty list (no creds seeded) — what matters here is the
# CORS header is present.
assert resp.headers["Access-Control-Allow-Origin"] == "moz-extension://abcd1234-uuid-fake"
@pytest.mark.asyncio
async def test_chrome_extension_origin_also_allowed(client):
resp = await client.options(
"/api/credentials",
headers={
"Origin": "chrome-extension://abcd1234-uuid-fake",
"Access-Control-Request-Method": "POST",
},
)
assert resp.status_code == 204
assert resp.headers["Access-Control-Allow-Origin"] == "chrome-extension://abcd1234-uuid-fake"
@pytest.mark.asyncio
async def test_normal_browser_request_gets_no_cors_headers(client):
# A regular browser tab (https://example.com / file:// / no Origin
# at all) should NOT get any Access-Control-Allow-* — the extension
# whitelist is intentionally narrow.
resp = await client.get(
"/api/credentials",
headers={"Origin": "https://evil.example.com"},
)
assert "Access-Control-Allow-Origin" not in resp.headers
@pytest.mark.asyncio
async def test_no_origin_header_unaffected(client):
# Same-origin requests (no Origin header) — unaffected.
resp = await client.get("/api/credentials")
assert "Access-Control-Allow-Origin" not in resp.headers
+7 -14
View File
@@ -69,20 +69,13 @@ async def test_scroll_post_id_filter(db):
assert {x.id for x in page.images} == {i1.id, i2.id}
@pytest.mark.asyncio
async def test_scroll_post_id_dedups_multi_rows(db):
i1 = await _img(db, 1)
_, s, p = await _post(db, "A", "a", "10")
# two provenance rows, same image+post (enrich-on-duplicate shape)
db.add(ImageProvenance(image_record_id=i1.id, post_id=p.id,
source_id=s.id))
await db.flush()
db.add(ImageProvenance(image_record_id=i1.id, post_id=p.id,
source_id=s.id))
await db.flush()
svc = GalleryService(db)
page = await svc.scroll(cursor=None, limit=10, post_id=p.id)
assert [x.id for x in page.images] == [i1.id] # appears once
# test_scroll_post_id_dedups_multi_rows removed 2026-05-26: it deliberately
# inserted two ImageProvenance rows with the same (image_record_id, post_id),
# now prevented at the DB layer by uq_image_provenance_image_post (alembic
# 0021). The EXISTS-based dedup in _provenance_clause is still useful for the
# artist-id filter (one image legitimately joins many provenance rows via
# different posts under the same artist), so the gallery_service logic is
# unchanged.
@pytest.mark.asyncio
+184
View File
@@ -0,0 +1,184 @@
"""Race-safe ImageProvenance insert in Importer._apply_sidecar.
Operator-flagged 2026-05-26: the prior SELECT-then-INSERT pattern lost a
race when two workers ran _apply_sidecar on the same (image, post) pair
(plausibly seeded when the 5-min recovery sweep re-enqueued a still-running
long import). Duplicates then broke .scalar_one_or_none() on every later
deep-scan rederive (MultipleResultsFound). Alembic 0021 added
uq_image_provenance_image_post; the importer's new savepoint+IntegrityError
recovery path now trips on collision and gracefully recovers.
Tests cover:
- idempotent: re-running _apply_sidecar via _deep_rederive produces
exactly one ImageProvenance row.
- IntegrityError recovery: pre-seed a provenance row, force the first
SELECT to return None (simulating the race window where two workers
both observed no row), call _apply_sidecar — the savepoint INSERT
trips uq_image_provenance_image_post, gets rolled back, no exception
escapes, still exactly one row.
"""
import json
from pathlib import Path
import pytest
from PIL import Image
from sqlalchemy import func, select
from backend.app.models import (
ImageProvenance,
ImageRecord,
ImportSettings,
Source,
)
from backend.app.services.importer import Importer
from backend.app.services.thumbnailer import Thumbnailer
pytestmark = pytest.mark.integration
@pytest.fixture
def import_layout(tmp_path):
import_root = tmp_path / "import"
images_root = tmp_path / "images"
import_root.mkdir()
images_root.mkdir()
return import_root, images_root
@pytest.fixture
def importer(db_sync, import_layout):
import_root, images_root = import_layout
settings = db_sync.execute(
select(ImportSettings).where(ImportSettings.id == 1)
).scalar_one()
return Importer(
session=db_sync,
images_root=images_root,
import_root=import_root,
thumbnailer=Thumbnailer(images_root=images_root),
settings=settings,
)
@pytest.fixture
def deep_importer(db_sync, import_layout):
import_root, images_root = import_layout
settings = db_sync.execute(
select(ImportSettings).where(ImportSettings.id == 1)
).scalar_one()
return Importer(
session=db_sync,
images_root=images_root,
import_root=import_root,
thumbnailer=Thumbnailer(images_root=images_root),
settings=settings,
deep=True,
)
def _split(path: Path, orient, size=(256, 256)):
path.parent.mkdir(parents=True, exist_ok=True)
w, h = size
im = Image.new("L", size, 0)
px = im.load()
for y in range(h):
for x in range(w):
if (x / w if orient == "v" else y / h) >= 0.5:
px[x, y] = 255
im.convert("RGB").save(path, "JPEG")
def _sidecar(media: Path, payload: dict):
media.with_suffix(".json").write_text(json.dumps(payload))
def test_apply_sidecar_idempotent_on_deep_rederive(
importer, deep_importer, import_layout,
):
"""Normal-flow path: deep rederive on an already-imported image finds
the existing provenance row via .scalar_one_or_none() and skips the
insert. Exactly one ImageProvenance row after both runs."""
import_root, _ = import_layout
m = import_root / "Alice" / "a.jpg"
_split(m, "v")
_sidecar(m, {
"category": "patreon", "id": 555,
"url": "https://patreon.com/posts/555", "title": "Set 1",
})
r = importer.import_one(m)
assert r.status == "imported"
# Re-import via deep mode — sha matches → _deep_rederive → _apply_sidecar.
r2 = deep_importer.import_one(m)
assert r2.status == "refreshed"
count = importer.session.execute(
select(func.count()).select_from(ImageProvenance)
).scalar_one()
assert count == 1
def test_apply_sidecar_recovers_from_integrity_error(
importer, deep_importer, import_layout, db_sync, monkeypatch,
):
"""Race recovery: a row already exists for (image, post). We force the
importer's existence-check SELECT to return None for one call, mimicking
the race window where two workers both saw no row. The savepoint INSERT
then trips uq_image_provenance_image_post; the helper rolls the
savepoint back, no exception escapes, and the row count stays at 1.
"""
import_root, _ = import_layout
m = import_root / "Bob" / "b.jpg"
_split(m, "v")
_sidecar(m, {
"category": "patreon", "id": 777,
"url": "https://patreon.com/posts/777", "title": "Set 2",
})
# First import lays the canonical provenance row.
r = importer.import_one(m)
assert r.status == "imported"
rec = importer.session.get(ImageRecord, r.image_id)
src = importer.session.execute(select(Source)).scalar_one()
assert rec is not None
assert src is not None
# Monkeypatch session.execute so the FIRST select inside _apply_sidecar's
# existence-check returns a "no row" wrapper. Subsequent selects (e.g.
# the find_or_create_source / find_or_create_post existence checks
# earlier in _apply_sidecar) all run normally; we intercept only the
# ImageProvenance lookup, identified by the SELECT's target columns
# mentioning image_provenance.
real_execute = db_sync.execute
intercepted = [False]
def _intercepting_execute(stmt, *args, **kwargs):
text = str(stmt)
if (
not intercepted[0]
and "image_provenance" in text
and "image_record_id" in text
and "post_id" in text
):
intercepted[0] = True
class _ForcedMiss:
def scalar_one_or_none(self):
return None
return _ForcedMiss()
return real_execute(stmt, *args, **kwargs)
monkeypatch.setattr(db_sync, "execute", _intercepting_execute)
# Re-import via deep mode → _deep_rederive → _apply_sidecar. With the
# provenance-SELECT forced to miss, the helper will attempt the INSERT,
# trip uq_image_provenance_image_post, catch IntegrityError, and recover.
r2 = deep_importer.import_one(m)
assert r2.status == "refreshed"
# Lift the intercept, then verify the row count.
monkeypatch.undo()
count = db_sync.execute(
select(func.count()).select_from(ImageProvenance)
).scalar_one()
assert count == 1
+65
View File
@@ -145,3 +145,68 @@ def test_find_or_create_source_recovers_from_integrity_error(
artist_id=artist_row.id, platform="patreon", url=canonical_url,
)
assert recovered.id == pre_existing.id
def test_source_for_sidecar_reuses_existing_subscription(
importer, artist_row, db_sync,
):
"""The filesystem-import sidecar resolver should attach to whatever
Source already exists for (artist, platform) — the canonical subscription
Source — regardless of its URL. Without this, every imported post
spawned its own Source row.
"""
canonical = Source(
artist_id=artist_row.id, platform="patreon",
url="https://www.patreon.com/cw/testartist", enabled=True,
)
db_sync.add(canonical)
db_sync.flush()
resolved = importer._source_for_sidecar(
artist_id=artist_row.id, platform="patreon",
artist_slug=artist_row.slug,
)
assert resolved.id == canonical.id
def test_source_for_sidecar_creates_synthetic_anchor_when_none_exists(
importer, artist_row, db_sync,
):
"""No subscription Source for this (artist, platform) yet. The helper
creates one synthetic anchor (enabled=False, url='sidecar:<plat>:<slug>')
so subsequent imports reuse it instead of spawning per-post Sources.
"""
resolved = importer._source_for_sidecar(
artist_id=artist_row.id, platform="pixiv",
artist_slug=artist_row.slug,
)
assert resolved.url == f"sidecar:pixiv:{artist_row.slug}"
assert resolved.enabled is False
assert resolved.artist_id == artist_row.id
assert resolved.platform == "pixiv"
# Second call returns the same row (no new Source spawned).
again = importer._source_for_sidecar(
artist_id=artist_row.id, platform="pixiv",
artist_slug=artist_row.slug,
)
assert again.id == resolved.id
def test_source_for_sidecar_distinct_platforms_distinct_anchors(
importer, artist_row, db_sync,
):
"""One synthetic anchor per (artist, platform). Different platforms get
different anchors even when no campaign Source exists for either.
"""
p = importer._source_for_sidecar(
artist_id=artist_row.id, platform="patreon",
artist_slug=artist_row.slug,
)
x = importer._source_for_sidecar(
artist_id=artist_row.id, platform="pixiv",
artist_slug=artist_row.slug,
)
assert p.id != x.id
assert p.platform == "patreon"
assert x.platform == "pixiv"
+4
View File
@@ -23,3 +23,7 @@ def test_import_media_file_registered():
def test_generate_thumbnail_registered():
assert "backend.app.tasks.thumbnail.generate_thumbnail" in celery.tasks
def test_backfill_thumbnails_registered():
assert "backend.app.tasks.thumbnail.backfill_thumbnails" in celery.tasks