Commit Graph
17 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 235393c08b feat: link the Patreon teaser to the Discord drop it announced (388 step E5)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 32s
Build images / build-web (push) Successful in 1m3s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m53s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m12s
The point of the milestone rather than its tail. Two of the operator's artists
post a deliberately cropped fragment on Patreon to signal that the real thing
has landed in their Discord; this proposes those pairs.

Confirm-only, following the FC-6.3 series matcher. A wrongly-asserted
association tells the operator two different pieces are one, which is strictly
worse than no link: no link leaves them where they already were, a wrong one
actively misinforms and then propagates into whatever reads it. So the
matcher's job is a SHORT list worth reading, not a long list worth trusting.

**The threshold sits above every single signal weight, and that is the
design.** Proximity is 0.55, declaration 0.45, the cut 0.60 — so neither
signal can carry a pair alone. That makes "time proximity alone is never
sufficient" an arithmetic property rather than an aspiration: on a busy day an
artist posts several times, and a matcher that could pair on proximity alone
would turn every one of those days into false pairs until the review queue got
abandoned. A guard test asserts the relationship against WEIGHTS directly, so
it survives any refactor of the scorer, and says in its own failure message not
to fix it by lowering the assertion.

**Crop-to-source matching is HELD, on the plan's instruction** — real work with
real false-positive risk, worth building only once signals 1 and 2 are shown
insufficient against the operator's actual artists. Worth stating: a naive
whole-image SigLIP similarity is NOT that signal. A cropped teaser and its full
version are precisely the pair a whole-image comparison handles worst, so
adding one as a "bonus" would mostly add noise while looking like progress.

Two premises in the plan corrected in the building:

* **E4 is not actually a prerequisite.** A Patreon Source and a Discord Source
  the operator has added under one Artist already share `Post.artist_id`, and
  the synthetic grouping inherits it. E4 EXTENDS this to creators FC has to
  learn the association for; it is not needed to represent one FC was told.
  Same-artist is then a hard filter, not a scored signal — two different
  creators posting minutes apart is a coincidence, not evidence.
* **`link_extract` cannot supply the declaration signal.** It exists, but
  `SUPPORTED_HOSTS` is file hosts only and `host_for()` returns None for a
  Discord URL, so no ExternalLink row is ever written for one. The signal
  reads the post body directly instead.

And a bug my own test would have caught: `declared_signal` stripped the HTML
before looking for an invite, but `html_to_plain` discards attributes and
these creators put the invite in an anchor's `href` — so the strongest form of
the signal was being thrown away, leaving only whatever the link text said.
The invite now matches the raw body; the bare mention still matches stripped
text, so `\bdiscord\b` is tested against prose rather than against markup.

Dismissed rows are kept, not deleted: the row is what remembers the rejection,
and re-proposing a rejected pair on every scan is the one behaviour that makes
a review queue get ignored. Both FKs CASCADE, so E3's one-DELETE reversal
cannot leave a proposal pointing at a post that no longer exists.

Only ACCEPTED links reach the post payload. A pending proposal is a question
for the review queue, not a claim to render beside the artwork.

UI (rule 27) follows in the next commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
2026-09-10 11:47:28 -04:00
bvandeusenandClaude Opus 5 1e45e2c56c feat: an open grouping — a later drop joins its post (milestone 388 step E3)
CI / extension-version (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m6s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m59s
Build images / promote (push) Skipped
CI / integration (push) Failing after 2m7s
A synthetic post is no longer sealed at creation. A creator who adds two more
variants the next day extends the existing post, its body grows with the new
messages, and no rival post appears. That is what makes chat capture read as
content trickling in rather than as a stream of separate arrivals.

The sweep now runs two passes per source and the ORDER is load-bearing: offer
new messages to still-open groups BEFORE founding new ones, because whichever
runs first claims a message.

E3's three named problems, each answered rather than discovered later:

**Bridging.** A candidate near two groups joins NEITHER. Nearest-wins would
silently make an arbitrary choice between two posts the operator may already
have seen; merging them is worse still, because a merge rewrites history and
anything pointing at the absorbed post dangles. Leaving it to found its own
group is the recoverable failure. AMBIGUITY_MARGIN is a module constant and
deliberately not a setting — it is not a quality dial anyone would tune toward
a better feed, and exposing it would invite turning it to zero, which is
exactly the silent arbitrary choice it prevents.

**Re-surfacing without thrashing.** A grouping has two dates, and which one
orders the feed is a real decision, so the feed orders by neither directly.
Ordering by when the drop STARTED buries a group that grows a week later under
a week of other posts — defeating the point of keeping it open. Ordering by
every growth lets a group gaining one image a day live permanently at the top,
so chat out-competes authored posts for the front page — the opposite of "post
pacing stays front and centre". Instead `resurfaced_at` moves only when growth
clears BOTH a minimum-images bar and a cooldown, so a drip-feed updates in
place and a genuine second wave resurfaces exactly once. It is NULL on every
ordinary post, so the sort key COALESCEs through it without moving anything
that is not a grouping.

**Reopening forever.** Groups close after a quiet period — artists reuse
characters for years, and a group left open indefinitely will eventually
absorb something it shouldn't. Openness is DERIVED, not stored: a group is
open if it grew (or started) within the window. Lowering the setting closes
old groups and raising it reopens them, with nothing to repair either way; a
stored closed_at would have needed a sweep to set it and a repair path to ever
change the policy.

Rule 89 is satisfied structurally rather than by a parallel mechanism:
celery_signals writes a TaskRun for every task, which already supplies
duration, the 5-minute stalled-run recovery, and retention pruning. What this
step owed on top of that was a wall-clock limit (present) and idempotence —
re-running the joiner adds nothing, asserted directly rather than left to the
unique (image, post) constraint to catch.

Two bugs fixed in the writing, one of which my own test would have hit:

* `assign_to_group` sorted bare (distance, Post) tuples, which falls through
  to comparing Posts when two distances tie — and a perfectly symmetric
  bridge, the exact case the function exists for, would have raised TypeError
  instead of declining to choose. Now keyed on the distance alone.
* The cursor was still built from `post_date or downloaded_at` while the
  ORDER BY had gained `resurfaced_at`. Two expressions that disagree at a page
  boundary don't error, they silently skip or repeat rows; both sites now go
  through one `_post_sort_value`, and a test pages through one row at a time
  to prove the walk matches the whole list.

Image linking is now one shared helper rather than written twice, because
creation and joining would otherwise be free to drift on exactly the detail
(which post owns the image) that makes a grouping reversible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
2026-09-10 11:30:17 -04:00
bvandeusenandClaude Opus 5 73eeb7a377 feat: FC authors the post that Discord never wrote (milestone 388 step E2)
CI / lint (push) Failing after 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m53s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m5s
Discord is a delivery channel, not a publisher. One message is not one post,
and today every message lands as its own `post` row, so chat lines compete
with authored work for the same surface. Rather than demote them into a
second-class feed, FC now writes the post itself: one row per DROP, its
images the drop's images, its body the messages' text in arrival order.

Synthesising a `Post` (rather than inventing a parallel entity) is the whole
point — the result is post-shaped by construction, so feed, provenance,
translation, attachments and series keep working on it unchanged.

The predicate is three axes ANDed, and the time one does the real work:

    same source  AND  cosine distance <= threshold  AND  no gap > window

Similarity alone over-groups, and that is the failure that would make this
useless: any two pieces of the same character by the same artist sit close in
SigLIP space, so a cosine-only rule collapses a month of one character into a
single "post". Two details inside the predicate are load-bearing —

* distance is measured to the group's SEED, never to the previous member,
  because chaining lets a group DRIFT: twenty small steps walk from one piece
  to a completely different one, each hop individually within threshold;
* the window is measured between CONSECUTIVE messages, not from the first, so
  an artist trickling variants out over an evening stays one drop.

Why a post-import sweep and not part of ingest. The obvious alternative was to
migrate Discord to the native post-first ingester (#1266) and group at capture
time. That cannot work: the grouping signal is `siglip_embedding`, which is
produced asynchronously AFTER import (tasks/ml.py, the GPU backfill), so at
capture time there is nothing to group on. Grouping is necessarily something
that happens once the vectors catch up — hence a re-runnable sweep that skips
what it cannot yet place, and an hourly (not daily) cadence.

The honesty rule, enforced in the schema. `post.synthesized_by` names the
grouper; `synthesis_details` records the members, the count, and the
thresholds AS THEY WERE (they are operator-tunable, so without that "why did
it group these" is unanswerable a month later). Member posts are absorbed, not
destroyed — they remain the images' true origin and the audit trail — and
`absorbed_by_post_id` is ON DELETE SET NULL, so deleting a synthetic post
releases its members back into the feed in one DELETE with no repair step.
`post_title` stays NULL deliberately: a synthesised title is the one place
this could put words in a creator's mouth.

Two guards the first draft would have failed:

* the per-run cap took the lowest post IDs, not the oldest posts — DISTINCT ON
  forces its own ORDER BY, so the sort now happens outside the subquery;
* a cap landing mid-drop would have published a truncated group claiming to be
  a whole drop, so the last group is left for the next run.

And one vacuous test caught before it shipped: the support vector perturbed a
single component of an all-ones vector, moving it ~1e-6, so every distance
assertion passed regardless of what the predicate did. `_vec` now builds a
unit vector at a stated angle, where distance is exactly 1 - cos(delta) —
rule 167, a guard has to be able to fail.

UI (rule 27) follows in the next commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
2026-09-10 11:15:14 -04:00
bvandeusen 1ac448d881 refactor: four small cleanups from the review pass (#3072)
CI / extension-version (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 4m57s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
Items 2-5 of #3072. Item 1 (the per-row sweep inserts) is separate.

2. .fc-bad was not merely duplicated — it is .fc-weak under a second
   name. Both local definitions were `color: rgb(var(--v-theme-error))`,
   identical to the global .fc-weak, and GpuAgentCard was already using
   .fc-weak to colour exactly what GpuActivityPanel coloured .fc-bad (an
   errored count, red when non-zero). So rather than promoting a synonym
   to app.css, both call sites now use .fc-weak and the local defs are
   gone. app.css's status-colour comment records why there is no .fc-bad,
   next to the existing note on why .fc-ok is deliberately NOT global.

3. GalleryItem.vue's obsidian literals now use --v-theme-background,
   which IS obsidian (vuetify-theme.js maps background -> surfaces.
   obsidian). Preferred over --fc-chrome-rgb: same value, but that
   variable is named for the nav fade, not for the palette entry.

   The ticket said these were the only three real uses in the tree. They
   are not — GalleryItem itself had two more in the artist-label
   gradient (fixed here, so the file is now consistent), and ~13 more
   live in SeriesView, SeriesReaderView, ImageViewer, ArtistHeader,
   ExploreView and GalleryFilterBar. Those are a separate sweep, filed
   rather than folded in here.

4. The attachment download path had two hand-formatted copies. One
   definition now, `attachment_download_url`, next to the model both
   serializers already import. The test pins it by MATCHING the built
   path against the app's real URL map rather than comparing to a
   literal — a string-equality test would still pass after someone
   renamed the route, which is the drift the helper exists to prevent.

5. Extension API key now compares with hmac.compare_digest. Compared as
   BYTES, not str: compare_digest's str form raises TypeError on
   non-ASCII, and this value comes straight from an attacker-controlled
   header, so the str form would turn a junk key into a 500 instead of a
   403. Low stakes either way — the API is unauthenticated-by-design on
   a LAN — but it costs nothing.

Refs #3072
2026-08-27 07:48:18 -04:00
bvandeusenandClaude Opus 4.8 aea2701c28 feat(translation): tunable acceptance floor (0.90) + per-post sticky override (#155)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m52s
The gate at a fixed 0.80 couldn't catch the real pain: Interpreter (fresh ==
cached, verified by probe) confidently mis-detects short ASCII English like
"... WIP Part 1" as German at 0.86 — above the floor — so it was accepted and a
re-translate reproduced it. Confidence alone can't separate the 0.86 collision
(genuine German lands there too), and single-word mis-flags sit at a confident
1.0 no floor catches.

Two operator-approved levers:

- Acceptance floor is now a live Settings value (ImportSettings.
  translation_min_confidence, default 0.90; surfaced in the Translation card), so
  it's tunable without a redeploy. _accept takes the threshold as a parameter.

- Per-post sticky override (Post.translation_override: auto/force/original).
  'force' stores a translation even below the floor (rescue a skipped
  legit-foreign title); 'original' keeps the original and clears any stored
  translation (kill a confident mis-flag no floor catches). The sweep honors it
  on every run and _reset_translations skips 'original', so the choice survives a
  Re-translate-all. POST /api/posts/<id>/translation-override applies it
  immediately (translate now when the service is up, else queue for the sweep).
  UI: PostTranslationControl on the posts-feed card.

Migration 0084 (both columns + a CHECK on the override). The feed + provenance
serializers expose translation_override.

With a stricter floor the rollback finally works: raise it -> Re-translate all ->
the 0.86 mis-flags are rejected and restored to the original; force /
keep-original handle the residual either way.

Tests: gate thresholds against the param (0.86 rejected at 0.90, explicit-floor
cases); sweep force/original + re-translate-skips-original; override endpoint
(validation, original clears, force queues when disabled, feed exposes it);
settings min_confidence default/save/validate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-10 22:38:25 -04:00
bvandeusenandClaude Opus 4.8 ead60978e3 feat(translation): expose translated fields in post feed + provenance payloads (#143 step 4)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m44s
post_feed_service (card + detail) and provenance_service._post_dict now include
post_title_translated, description_translated (card-truncated / detail-uncapped)
and translated_source_lang, keeping the originals for the toggle. Feed
serialization test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:31:46 -04:00
bvandeusenandClaude Opus 4.8 96c29c370b feat(ingest): localize inline post-body images to local copies (Phase 2)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m14s
Render a post body faithfully by serving our stored copies of inline
images instead of hotlinking the public CDN. The join key is the CDN
filehash (32-hex MD5) shared between a body <img src> and the media URL
we downloaded (the same identity extract_media dedups by):

- utils.paths.filehash_from_url — one source of truth for the extractor;
  patreon_client._filehash now delegates so capture- and render-time
  hashing cannot drift.
- ImageRecord gains source_url (provenance) + source_filehash (indexed
  match key); migration 0051.
- the per-media sidecar carries the file's source_url; the importer
  persists it (NULL-only) on the ImageRecord via _apply_sidecar.
- post_feed_service.get_post remaps body <img src> -> /images/<path> for
  every inline image whose filehash maps to a stored image of THIS
  artist; unmatched / pre-Phase-2 images keep hotlinking.

Pre-existing on-disk images have no filehash yet, so they fall back to
hotlinking until re-downloaded; localization is forward-looking.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 16:39:58 -04:00
bvandeusenandClaude Opus 4.8 d96918d777 feat(posts): extract + record external file-host links (Phase 3)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m20s
Capture off-platform links (mega/gdrive/mediafire/dropbox/pixeldrain) embedded
in post bodies so they're never silently dropped, and surface them in the post
view. The download worker (Phase 4) walks these rows.

- link_extract.py: pure extractor — <a href> + bare URLs, unwraps Patreon
  redirect shims, PRESERVES the full url incl. #fragment (mega's key), dedups.
  Reusable by every platform (runs off Post.description).
- external_link model + migration 0049: post_id/artist_id/host/url/label/status
  /attempts/last_error/attachment_id/timing; CHECK whitelists (full enum incl.
  worker statuses up front) + (post_id,url) unique.
- importer._sync_external_links: insert-missing on both import paths
  (_apply_sidecar + upsert_post_record) so a re-import never resets a link's
  status; runs for all platforms.
- post_feed_service.get_post: returns external_links (detail-only).
- PostCard: renders the links (host chip + label + status) once expanded.
- tests: extractor (5 hosts, fragment, shim unwrap, dedup), importer (record +
  no-dup on reimport), serializer.

Refs FC #830.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 13:15:36 -04:00
bvandeusenandClaude Opus 4.8 c342c73a25 feat(posts): faithful (semantic) HTML rendering of post bodies
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 3m20s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
Phase 1 of milestone #64. The body is captured (Phase 0) but was shown as
plain text. Now:
- html_sanitize.py: widen the allowlist to a faithful-but-safe set — headings,
  inline images, lists, blockquote, hr, code/pre, figure, links (div/span stay
  stripped; their text is preserved). Benefits the existing ProvenancePanel too.
- post_feed_service.get_post: add sanitized `description_html` to the DETAIL
  response (the feed list stays lightweight plain text by design).
- PostCard.vue: render description_html via v-html once expanded (fetched with
  detail); collapsed + no-detail fallback stay plain text. Styled close to the
  source (headings, images max-width, accent links, lists, quotes, code).

Tests: sanitizer (headings/img/lists survive, img javascript: src dropped);
get_post returns sanitized description_html.

Refs FC #830.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 13:02:21 -04:00
bvandeusenandClaude Opus 4.8 2c544ad5af feat(browse): sticky tabs + per-tab search bar (server-side, scope-aware)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m10s
The Browse tab nav scrolled away (operator didn't know it existed) and
Posts had no search. Roll the tab strip + a shared search field into one
sticky block pinned under the 64px TopNav.

- Posts gains server-side text search: PostFeedService.scroll()/around()
  + /api/posts accept q (ILIKE over post_title OR description), applied
  INSIDE the artist/platform WHERE so search stays scoped to the active
  filter. Scope shown as clearable chips next to the search field.
- Artists/Tags search consolidates into the sticky bar: their inner
  search boxes are removed; they react to route.query.q (q is deep-
  linkable, e.g. /browse?tab=posts&q=foo). Platform/kind filters stay.
- Posts empty state now distinguishes 'no matches' from 'no posts yet'.

Tests: posts q-search matches title|description and stays artist-scoped
(service); q passthrough (api).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 00:04:06 -04:00
bvandeusenandClaude Opus 4.8 074c5868fb refactor(services): shared pagination cursor (DRY sweep)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 43s
CI / integration (push) Successful in 3m21s
encode_cursor/decode_cursor (base64 <iso8601>|<id>) were defined identically in
gallery_service AND post_feed_service, with artist_service importing gallery's
copy. Two implementations of one cursor format silently break pagination in
whichever feed drifts. Extract to services/pagination.py; gallery/post_feed/
artist all import it. Dropped now-unused base64/datetime imports.

§8b: encode_cursor/decode_cursor now defined only in pagination.py. Existing
cursor round-trip tests still cover it via the re-export. Catalog updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 00:10:57 -04:00
bvandeusenandClaude Opus 4.8 a8f624a0f1 fix(posts): link duplicate items to every post + prune bare shells
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m20s
The native Patreon backfill flooded the feed with bare 'Post <id>' shells
(1589 for Anduo). Root cause: PostAttachment.sha256 was GLOBALLY unique, so a
non-art file reused across posts only ever linked to the first one, and
_capture_attachment created the Post before that dedup check — leaving later
posts with no image and no attachment. Duplicate IMAGES had the mirror gap:
attach_in_place returned duplicate_hash/duplicate_phash before _apply_sidecar,
so the second post got no provenance row, and the feed only rendered via
primary_post_id (one post per image).

Operator requirement: a duplicate item must show on EVERY post it appears in.
Unify the fix as link-not-suppress:

- importer: on duplicate_hash / duplicate_phash(larger_exists), append an
  image_provenance row for the new post (keep primary on the first). Both the
  download path (attach_in_place) and the filesystem path (_import_media).
- post_feed_service: render thumbnails by image_provenance UNION primary_post_id,
  so a cross-posted image shows on every post (and legacy primary-only images
  still show).
- PostAttachment: per-post uniqueness — drop UNIQUE(sha256), add partial
  UNIQUE(post_id, sha256) + partial UNIQUE(sha256) WHERE post_id IS NULL
  (migration 0043); _capture_attachment dedups per-(post,sha) over the shared
  sha-addressed blob, so no post is left bare.
- cleanup: new prune-bare-posts maintenance action (cleanup_service
  _bare_post_conditions shared by preview/count/delete per preview/apply parity;
  admin endpoint; PostMaintenanceCard). Deletes posts with zero image links
  (primary or provenance) AND zero attachments. Run after the feed fix so a
  hidden provenance link spares the post instead of deleting it.

Tests: dup image shows on both posts; dup attachment shows on both posts; feed
renders provenance-linked duplicates; prune-bare delete-path == preview.

Operator redeploys (migration 0043) then runs the prune to clear the shells.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 19:28:33 -04:00
bvandeusen 2f66de2928 feat(model): nullable Post.source_id + denormalized Post.artist_id; retire sidecar synthetics
CI / backend-lint-and-test (push) Successful in 11s
CI / frontend-build (push) Successful in 17s
CI / lint (push) Failing after 3s
CI / intimp (push) Failing after 2m15s
CI / intapi (push) Failing after 2m18s
CI / intcore (push) Failing after 2m17s
Operator-asked 2026-06-01 after the Dymkens orphan investigation
(Scribe plan #540). The pre-2030 sidecar-synthetic Source pattern
(`sidecar:<platform>:<slug>` enabled=false rows) existed solely to
satisfy `Post.source_id NOT NULL`, and leaked into the Subscriptions
UI as phantom subscriptions. Now the data model says what's true:
filesystem-imported content with no live subscription has NULL
source_id, full stop.

## Schema (alembic 0030)

- `post.artist_id` — NEW NOT NULL FK to artist (CASCADE). Backfilled
  from source.artist_id in the migration. Indexed for the artist-filter
  queries.
- `post.source_id` — NOT NULL → nullable; FK ondelete CASCADE → SET
  NULL. Deleting a Source detaches its Posts instead of destroying
  archived content (subscription ends, archive stays).
- `image_provenance.source_id` — same nullable + SET NULL.
- Partial unique index `uq_post_artist_external_id_null_source` on
  (artist_id, external_post_id) WHERE source_id IS NULL — guards
  filesystem-import dedup since the existing source-bound unique
  ignores NULLs (Postgres treats NULL != NULL).
- Sidecar synthetic Sources deleted: NULL out FKs in post,
  image_provenance first, then DELETE FROM source WHERE url LIKE
  'sidecar:%'. The Dymkens cleanup.

## Model + service changes

- `Post.source_id` → `Mapped[int | None]`; new `Post.artist_id`
  denormalized.
- `ImageProvenance.source_id` → `Mapped[int | None]`.
- Importer: `_source_for_sidecar` (synthetic-creating) →
  `_lookup_source_for_sidecar` (returns None when no subscription).
  `_find_or_create_post` takes required `artist_id`; matches on
  (source_id, external_post_id) for source-bound posts or
  (artist_id, external_post_id) for NULL-source posts.
- Service queries switched off the Source detour to use Post.artist_id
  directly: post_feed_service.scroll/around/get_post (LEFT JOIN to
  Source so NULL-source posts surface); artist_service date_row/
  activity/post_count; provenance_service.for_image/for_post (LEFT
  JOIN); gallery_service._provenance_exists_where_artist via
  Post.artist_id instead of ImageProvenance.source_id → Source.
- `_to_dict` and provenance dict-builders emit `"source": null` for
  NULL-source rows.

## Frontend

- `ProvenancePanel.vue` + `PostCard.vue`: render `e.source?.platform
  ?? 'filesystem import'` so NULL-source posts get a clear
  "filesystem import" affordance instead of a NaN crash.

## Tests

- `test_importer_upsert_helpers`: removed the four synthetic-anchor
  tests; added `_find_or_create_post_idempotent_with_null_source`
  (dedup via the partial unique index) and
  `_lookup_source_for_sidecar_returns_*` (existing-subscription +
  none cases). The existing `_find_or_create_post_idempotent` now
  also passes `artist_id` and asserts it.
- 8 other test files updated: every direct `Post(...)` construction
  gains `artist_id=<artist>.id`. The `_seed_post` helper in
  `test_post_feed_service` looks up artist_id from the source row so
  callsites stay one-arg.

## Verification on deploy

After alembic 0030 runs:
- `SELECT COUNT(*) FROM source WHERE url LIKE 'sidecar:%'` → 0.
- `SELECT COUNT(*) FROM post WHERE source_id IS NULL` → count of
  filesystem-imported posts (Dymkens + any other historical).
- Every `post.artist_id` non-null; consistent with source.artist_id
  for source-bound rows.
- Subscriptions tab: no Dymkens phantom row.
- Artist detail → Posts/Gallery: Dymkens's content still reachable
  via Post.artist_id.
- Provenance panel renders "filesystem import" chip for NULL-source
  posts; PostCard same.

## Out of scope

- UI to manage/delete orphan NULL-source Posts. Data model is right;
  UI follows if operator wants it.
2026-06-01 14:17:52 -04:00
bvandeusenandClaude Opus 4.7 44bb12a93d fix(thumbnails): derive URL from stored thumbnail_path, not (sha256, mime)
The showcase/gallery/artist/series/post-feed APIs were constructing
thumbnail URLs from (sha256, mime). The MIME-based extension predicate
("png if image/png or image/gif else jpg") DISAGREED with the
thumbnailer's actual on-disk extension predicate ("png if alpha else
jpg"). Result: every PNG source without transparency 404'd (URL asked
.png, disk had .jpg); every WebP/AVIF source with transparency 404'd
(URL asked .jpg, disk had .png) — despite the thumbnail file existing
on disk.

The backfill task couldn't catch these because backfill checks the
ACTUAL thumbnail_path stored on the record (correct), not the URL the
browser fetches (broken derivation). So records with valid on-disk
thumbnails kept showing as broken in the UI no matter how many times
backfill ran.

Operator-flagged 2026-05-30: "the generate thumbnails function appears
to not catch all of the failed thumbnail cases" — turned out to not be
a backfill bug at all.

Fix: thumbnail_url now takes (thumbnail_path, sha256, mime) and returns
the stored path verbatim — Quart serves /images/* 1:1 from the volume
(frontend.py:20-36), so the URL IS the disk path. Falls back to the old
sha256+mime derivation only when thumbnail_path is NULL (thumbnailer
hasn't run yet); that URL will 404 in the browser until backfill catches
it, same as before the path was tracked.

All 8 callers updated: showcase_service, gallery_service (2 sites),
artist_service, series_service, post_feed_service, tag_directory_service,
artist_directory_service. The four sites whose query was raw-tuple now
also SELECT ImageRecord.thumbnail_path.

Net effect: every record that has a valid on-disk thumbnail will now
render correctly, regardless of which extension the thumbnailer chose,
without any DB migration or backfill rerun needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-30 16:55:38 -04:00
bvandeusenandClaude Opus 4.7 c95b760294 feat(posts): in-context anchored feed with bidirectional infinite scroll
Provenance "View post" deep-links to /posts?post_id=X, which now opens the
feed centered on that post with infinite load in BOTH directions.

Backend: PostFeedService.scroll gains a direction (older|newer); new
around(post_id) returns a window of newer + the post + older with a cursor
for each end. /api/posts accepts ?around= and ?direction=. + API tests.

Frontend: posts store gains loadAround/loadOlder/loadNewer (older appends,
newer prepends) with per-end cursors; PostsView's anchored mode scrolls to
the post, observes top + bottom sentinels, and preserves scroll position on
upward prepend so the page doesn't jump. Normal feed mode unchanged.

Closes the remaining half of the post-navigation work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 21:12:18 -04:00
bvandeusenandClaude Opus 4.7 42c33e44f9 feat(post-api): get_post returns uncapped thumbnails — PostModal masonry needs full image list; feed query unchanged (still capped at 6 for previews). _thumbnails_for gains a limit kwarg; get_post passes limit=None.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 20:47:18 -04:00
bvandeusenandClaude Opus 4.7 5925ebe1f1 fc3e: PostFeedService — cursor-paginated unified posts feed mirroring GalleryService.scroll
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 18:54:49 -04:00