The gallery's newest/oldest sort keys off image_record.effective_date =
COALESCE(primary post's post_date, created_at). The primary post is often the
repost/download the file came from, so the grid led with download dates rather
than when content was first posted (operator-flagged).
Add a second materialized sort key, earliest_post_date = MIN(post_date) across
ALL of an image's provenance posts (every post it appears in), else created_at —
the original publish date. Mirrors the effective_date pattern so the sort stays a
forward index scan.
- alembic 0071: add earliest_post_date + index (DESC, id DESC); backfill
created_at baseline then MIN over image_provenance ⋈ post.
- importer: recompute earliest_post_date whenever a dated post is linked (MIN over
the image's provenance, which now includes the just-added row).
- gallery_service: new sorts posted_new / posted_old key off earliest_post_date;
cursor + year/month grouping follow the active column transparently.
- api: accept posted_new|posted_old; DEFAULT is now posted_new so the grid leads
with original publish date. newest/oldest (effective_date) still available.
- frontend: sort dropdown gains "Newest/Oldest post date" (default Newest post
date); existing effective-date sorts relabelled "Newest/Oldest added".
- tests: service test asserts posted_new/posted_old key off earliest_post_date;
frontend default-sort omission test updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
A fandom owns characters via Tag.fandom_id, but every image<->tag query
went purely through direct image_tag rows, so a fandom only surfaced
images literally tagged with it — images carrying one of its characters
were invisible to its browse count, previews, and gallery filter.
Derive membership at query time instead of materializing fandom rows
(which would drift on every reassign/merge/remove). Add one shared
predicate in tag_query.py — image_in_tag_scope / image_in_any_tag_scope:
an image belongs to a tag if tagged with it directly OR (when the tag is
a fandom) carrying a character whose fandom_id is that tag. The character
leg is empty for non-fandom tags, so it applies uniformly with no kind
branching. Route all read sites through it:
- gallery _apply_scope: include, OR-groups, and symmetric exclude
- directory image_count: correlated COUNT(DISTINCT) scalar subquery
- directory previews: UNION direct + via-character, then ROW_NUMBER<=3
- cleanup count_tag_associations: Tier-B delete prompt now reports a
fandom's true blast radius (was 0 for fandoms with no direct rows)
find_unused_tags already protected fandoms via used_via_fandom; left as is.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cluster A, milestone #97. #5: clicking an image-modal tag chip's body now
closes the modal and opens the gallery filtered for that one tag (fresh
filter); ✕/kebab stay as the explicit remove/rename controls.
#6a (backend of OR/exclude filtering): gallery_service._apply_scope gains a
structured tag model — tag_or_groups (AND-of-OR: one EXISTS(tag_id IN group)
per group) + tag_exclude (NOT EXISTS(tag_id IN exclude)) — layered additively
on the existing tag_ids AND path so cursors/facets/deep-links are untouched.
Threaded through scroll/timeline/jump_cursor/facets/similar + facets common
dict; _require_single_filter rejects post_id combined with OR/exclude. API
parses tag_or (repeatable → one OR-group each) + tag_not (csv exclude).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX
An image whose on-disk path contains '#' (post folders like 'BLUE#59')
served its hash-named thumbnail fine but 404'd the original: the unencoded
'#' in image_url was parsed by the browser as a URL fragment, so
'#59/01_timelapse.jpg' never reached the /images route. Add a shared
image_url(path) helper that percent-encodes the path (safe='/') and route
the 3 raw builders (gallery detail + 2 in series) through it. Not a
cleanup-tool deletion — the file is on disk; only the URL was wrong.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 backend for the gallery filter bar. Extends scroll/timeline/jump
from a single mutually-exclusive filter to a composable one:
- tag_ids: image must carry ALL of them (one correlated EXISTS per tag —
AND, no row multiplication), replacing the single-tag JOIN.
- artist_id composes with tags; media_type ('image'|'video') narrows by
mime; post_id stays the exclusive post-detail path.
- sort ('newest'|'oldest') flips the effective_date/id cursor comparison
and ordering; the cursor value is unchanged (direction comes from the
request). jump_cursor honors sort too.
- Shared _apply_scope helper applied across scroll/timeline/jump so the
timeline sidebar reflects the filtered set. API _parse_filters parses
tag_id (comma list), artist_id, media, sort.
Tests: multi-tag AND, media filter, sort reversal (service + API);
post_id-excludes-others; single tag_id back-compat.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gallery cursored on COALESCE(post.post_date, image_record.created_at)
across the Post outer join — an expression spanning two tables that no
index can serve, so every /scroll sorted a large slice of the library
(and the old frontend fired ten serially). Materialize it:
- image_record.effective_date column + ix_image_record_effective_date
(effective_date DESC, id DESC); alembic 0035 backfills
COALESCE(primary post's post_date, created_at) for existing rows.
- gallery_service._effective_date_col() now returns the column, so scroll
/ timeline / jump / neighbors all order off the index instead of
re-deriving the COALESCE. _neighbors reads record.effective_date
directly (drops an extra Post lookup).
- importer._apply_sidecar maintains it: when a primary post with a date is
linked, effective_date = post.post_date; plain inserts keep the
created_at-equivalent server default.
Tests: sidecar import asserts effective_date == post.post_date; gallery
ordering/timeline/jump test seeds set effective_date alongside created_at.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Operator-asked 2026-06-01 after the Dymkens orphan investigation
(Scribe plan #540). The pre-2030 sidecar-synthetic Source pattern
(`sidecar:<platform>:<slug>` enabled=false rows) existed solely to
satisfy `Post.source_id NOT NULL`, and leaked into the Subscriptions
UI as phantom subscriptions. Now the data model says what's true:
filesystem-imported content with no live subscription has NULL
source_id, full stop.
## Schema (alembic 0030)
- `post.artist_id` — NEW NOT NULL FK to artist (CASCADE). Backfilled
from source.artist_id in the migration. Indexed for the artist-filter
queries.
- `post.source_id` — NOT NULL → nullable; FK ondelete CASCADE → SET
NULL. Deleting a Source detaches its Posts instead of destroying
archived content (subscription ends, archive stays).
- `image_provenance.source_id` — same nullable + SET NULL.
- Partial unique index `uq_post_artist_external_id_null_source` on
(artist_id, external_post_id) WHERE source_id IS NULL — guards
filesystem-import dedup since the existing source-bound unique
ignores NULLs (Postgres treats NULL != NULL).
- Sidecar synthetic Sources deleted: NULL out FKs in post,
image_provenance first, then DELETE FROM source WHERE url LIKE
'sidecar:%'. The Dymkens cleanup.
## Model + service changes
- `Post.source_id` → `Mapped[int | None]`; new `Post.artist_id`
denormalized.
- `ImageProvenance.source_id` → `Mapped[int | None]`.
- Importer: `_source_for_sidecar` (synthetic-creating) →
`_lookup_source_for_sidecar` (returns None when no subscription).
`_find_or_create_post` takes required `artist_id`; matches on
(source_id, external_post_id) for source-bound posts or
(artist_id, external_post_id) for NULL-source posts.
- Service queries switched off the Source detour to use Post.artist_id
directly: post_feed_service.scroll/around/get_post (LEFT JOIN to
Source so NULL-source posts surface); artist_service date_row/
activity/post_count; provenance_service.for_image/for_post (LEFT
JOIN); gallery_service._provenance_exists_where_artist via
Post.artist_id instead of ImageProvenance.source_id → Source.
- `_to_dict` and provenance dict-builders emit `"source": null` for
NULL-source rows.
## Frontend
- `ProvenancePanel.vue` + `PostCard.vue`: render `e.source?.platform
?? 'filesystem import'` so NULL-source posts get a clear
"filesystem import" affordance instead of a NaN crash.
## Tests
- `test_importer_upsert_helpers`: removed the four synthetic-anchor
tests; added `_find_or_create_post_idempotent_with_null_source`
(dedup via the partial unique index) and
`_lookup_source_for_sidecar_returns_*` (existing-subscription +
none cases). The existing `_find_or_create_post_idempotent` now
also passes `artist_id` and asserts it.
- 8 other test files updated: every direct `Post(...)` construction
gains `artist_id=<artist>.id`. The `_seed_post` helper in
`test_post_feed_service` looks up artist_id from the source row so
callsites stay one-arg.
## Verification on deploy
After alembic 0030 runs:
- `SELECT COUNT(*) FROM source WHERE url LIKE 'sidecar:%'` → 0.
- `SELECT COUNT(*) FROM post WHERE source_id IS NULL` → count of
filesystem-imported posts (Dymkens + any other historical).
- Every `post.artist_id` non-null; consistent with source.artist_id
for source-bound rows.
- Subscriptions tab: no Dymkens phantom row.
- Artist detail → Posts/Gallery: Dymkens's content still reachable
via Post.artist_id.
- Provenance panel renders "filesystem import" chip for NULL-source
posts; PostCard same.
## Out of scope
- UI to manage/delete orphan NULL-source Posts. Data model is right;
UI follows if operator wants it.
Operator hit this 2026-05-25 after the IR tag_apply landed: ~57k images
all scanned into FC in the same week share image_record.created_at, so
the gallery timeline collapses them into a single month bucket and
scroll orders them all together at the top. Their actual publish dates
(spread over years) were already available in Post.post_date but the
gallery never read it.
Backend wire-up:
- tag_apply phase 4 now sets ImageRecord.primary_post_id when creating
ImageProvenance (only if currently NULL — preserves the canonical
download-time linkage set by the importer for new FC ingests).
- gallery_service.py introduces _effective_date_col() =
COALESCE(post.post_date, image_record.created_at), used in:
* scroll() ORDER BY + cursor WHERE clauses
* timeline() year/month group-by
* jump_cursor() year/month filter
* _neighbors() prev/next ordering
- Each method LEFT OUTER JOIN Post on primary_post_id so the COALESCE
works for images without a post (NULL on the Post side, fall back
to created_at).
- GalleryImage gains posted_at + effective_date fields; API /gallery
/scroll exposes both alongside the existing created_at so the UI
can render 'Posted on X (imported Y)' if desired.
- get_image_with_tags() returns posted_at for the modal.
Cursor format unchanged — the encoded datetime is now the effective_
date (whichever column won the COALESCE) and pagination remains
consistent.
To pick up new behavior for an already-migrated IR set: re-run
/api/migrate/tag_apply on the existing manifest (phase 4 is
idempotent; the new primary_post_id assignment backfills).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CI failure resolving 'postgres' hostname was the symptom; the cause is
that the workflow violated FabledRulebook/forgejo.md's "CI philosophy —
lint + short unit tests only" rule. Integration tests against a real
Postgres are supposed to run locally via docker-compose, not in CI.
Changes:
- Marked 8 DB-dependent test files with @pytest.mark.integration:
test_tag_service, test_importer, test_gallery_service, test_api_gallery,
test_api_tags, test_api_settings, test_api_import_admin, test_maintenance.
- CI workflow drops the postgres/redis service containers and the alembic
upgrade smoke step entirely.
- Pytest invocation in CI changes to `pytest -v -m "not integration"`.
- Added pytest marker registration to pyproject.toml.
- DB_PASSWORD and SECRET_KEY env vars retained because config.py reads
them at import time even though unit tests don't actually use them
(set to placeholder values).
What CI now runs:
- ruff check
- pytest on the 6 unit test files: test_slug, test_paths,
test_migration_0002, test_thumbnailer, test_celery_smoke,
test_tasks_register.
- npm install + npm run build
What CI no longer runs:
- alembic upgrade (no live DB)
- the 8 integration test files (these run locally via docker-compose)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ruff lint surfaced 23 violations across three rules; all addressed:
UP017 (Use datetime.UTC alias):
Replaced 13 sites of datetime.now(timezone.utc) with datetime.now(UTC),
also adjusted from-imports accordingly. UTC is a Python 3.11+ alias for
timezone.utc that ruff's pyupgrade rules prefer.
UP042 (StrEnum):
Replaced `class TagKind(str, Enum)` and `class SkipReason(str, Enum)`
with `class Foo(StrEnum)`. StrEnum was added in Python 3.11 stdlib and
is the modern idiom. Behavior is equivalent for our usage (the .value
attribute, str(member) semantics).
I001 (Import sorting):
Added `known-first-party = ["backend"]` to ruff.toml's [lint.isort] so
ruff groups `backend.*` imports correctly. Without it, ruff treated
them as third-party and demanded a different grouping. The existing
import order is stdlib → third-party → first-party → local relative,
which ruff now accepts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cursor format: base64(iso8601_created_at|image_id). Pagination key is
(created_at DESC, id DESC) so we don't drift when new imports land between
page loads. Timeline groups by date_part(year, month) so the sidebar can
render year-month jump buckets. get_image_with_tags returns full image
detail plus prev/next ids so the modal viewer can navigate without an
extra round-trip per arrow press.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusen