Compare commits

...
11 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 3fe9d0a612 fix: the Latest feed's filter dropdowns opened empty
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m2s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m54s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m20s
Operator: "the filters in the latest feed, drop down but don't have values". Two separate causes:

- Platform: PostsFilterBar built its items from `platformsStore.platforms`. The platforms store has never had that property; it exposes `list` and `byKey`. The read returned undefined, `|| []` turned that into an empty list, and nothing failed. ArtistsView had copied the same read, so the Browse → Artists platform filter was empty too. Both now read `list` and show platform names rather than raw keys.

- Artist: the autocomplete searched the server only after something was typed (autocomplete returns [] for an empty query by design, which its tests pin). Opening the dropdown therefore showed an empty menu. PostsFilterBar now loads every artist once from a new lightweight `GET /api/artists/names` (id, name, slug; alphabetical; no joins) and filters client-side, so the list is there on open. A deep-linked artist_id now also shows the artist's real name instead of "Artist #id".

Guard: frontend/test/storeUsage.spec.js scans src for `platformsStore.<name>` and fails on any name the store doesn't define, since the frontend CI has no type-checker to catch this. A positive control shows the shipped `platformsStore.platforms` read is flagged, and a vacuity check confirms the scan really walks the tree.

tests/test_api_artists_create.py covers /names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 22:38:27 -04:00
bvandeusenandClaude Opus 5 7ff8915147 feat: a source stops pulling once its membership ends, and resumes on resubscribe (3995)
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 27s
CI / backend-lint-and-test (push) Successful in 34s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m52s
Build images / promote (push) Skipped
CI / integration (push) Successful in 3m1s
Operator, 2026-09-13: "if I kill a subscription on patreon I would like the pulling to stop on curator as well", with auto-resume chosen. This reverses the 2026-09-11 "report only" decision for lapsed sources.

membership_reconcile.apply_membership_lapses runs in sync_memberships right after each platform's successful sync, so it only ever acts on the roster just written.

It stops a source (enabled=false, with the same failure-state reset as a manual disable, #1285) only when all of these hold:
- the roster is fresh
- the source's matched membership says has_paid_access is False (lapsed, or a free follow)
- the paid-through date has passed, where the platform gives one (Patreon's member.access_expires_at; SubscribeStar gives none, so it stops at once)
- the source is enabled
- the operator hasn't chosen to keep it

It never acts on absence. A source with no matched membership keeps pulling, because a rename or a never-walked source produces the same absence. An unrecognised status is never a lapse either.

It resumes only sources carrying its own `_membership_stopped` marker, once the membership is paid again.

The operator outranks the sweep both ways (SourceService.update):
- turning a stopped source back on marks it `_membership_kept`, so the next sweep leaves it alone until it's paid again
- turning a source off by hand drops the marker, so the sweep never switches it back on

Both are `_`-prefixed app-managed config keys, which operator edits already preserve. No migration.

The roster/fetch line holds. This is a source-level action by the sweep. No download path reads the roster, and the scheduler still selects on `enabled` alone. test_no_fetch_path_can_read_the_roster is unchanged.

UI: SourceRow shows a neutral "Membership ended" chip, with the status and the resume/keep explanation, ahead of the other chips. The sweep's task summary reports stopped/resumed counts.

Tests (tests/test_membership_lapses.py):
- a lapse stops the source with a clean slate and keeps the id cache
- paid-through is honoured
- absence, an unknown status and a stale roster never stop anything
- a resume touches only what the sweep stopped
- a manual on sticks, a manual off drops the marker, and a kept source is released once paid

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 22:08:06 -04:00
bvandeusenandClaude Opus 5 4d84ab2816 fix: the filter rail goes, and the subscriptions card stops listing what you don't pay for — and can be dismissed
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
Build images / build-ml (push) Successful in 7s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m8s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m29s
Operator live review of #253 on the wide window. The images landed well, but:

- "the left rail feels like wasted space". The filters and check status move back into one row above the feed (PostsView). On a wide window the row lines up with the feed column, clear of the day gutter, and the container recentres without the 280px rail. The narrow layout is as before, with the status still first. This retires #407 option E; A and D stay.

- The "you can't see" link landed on the Subscriptions reconcile card, which listed 9 sources as `former_patron` under "sources your roster doesn't account for". The operator doesn't want sources they no longer pay for listed there, and wants the card dismissable "unless something changes":
  - MembershipReconcileCard renders only "creators you subscribe to but don't follow here" and the stale-roster warning.
  - Stopping pulls on lapsed sources is the membership sweep's job instead (#3995, next).
  - A close button dismisses the card. The dismissal is keyed to a fingerprint of what the card says (offered membership ids plus stale-roster flags), so a new subscription or a roster going stale brings it back. It's stored per browser, which is enough for a single-operator instance.

The backend still computes tracked_not_subscribed. #3995 acts on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 22:04:02 -04:00
bvandeusenandClaude Opus 5 9b82a95b7e feat: Settings can re-download the Discord images the None naming broke (3999)
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m9s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m3s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m35s
The operator chose a clean re-download over relinking in place. The ~1,600 broken files can't be relinked reliably: their message ids are gone, and their sidecars collided.

Settings → Maintenance → "Repair Discord downloads" previews, then applies:
- Deletes every image whose path is `…/discord/None/<8 digits>_None_…`. Both the folder and the name are required, because that pair is only what the bug produced. It reuses cleanup_service.delete_images for the record and file deletes.
- Sweeps the leftover collided sidecars from those folders and removes the emptied folders.
- Only then clears gallery-dl's archive rows `discord%`, excluding `discordasset_%`. Upstream keys message attachments as `discord{message_id}_{num}`. Since the broken files lost their message ids, per-source forgetting is impossible. Every pre-fix Discord download is broken, and files fetched after the fix still exist on disk, so gallery-dl's `skip` won't re-fetch them.
- Arms a fresh backfill on every Discord source.

The apply defaults to preview at both the route and the task, runs on maintenance_long, and is never on a beat. The card uses the confirm-dialog pattern of AttachmentReclaimCard.

Supporting refactors, with no behaviour change:
- gallery_dl.archive_path() is the single definition of the archive location.
- source_service.arm_backfill() is the mutation start_backfill already did, now shared with the sync repair.

Tests (tests/test_discord_repair.py):
- The archive clear leaves other platforms and Discord assets alone, and counting mutates nothing.
- Case-twin artist folders are both found.
- The folder sweep works.
- An integration run shows only the broken image goes. A correctly named Discord file and a `None` folder under Patreon survive, and only Discord sources are re-armed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 20:52:32 -04:00
bvandeusenandClaude Opus 5 dc840feec7 fix: Discord downloads land as None/<date>_None_<name> with no post — name them from the keys gallery-dl really emits
CI / lint (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 5s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 35s
Build images / build-web (push) Successful in 1m17s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m15s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
Operator report: a Discord source showed the wrong post time. Listing the real downloads found every Discord folder on the instance (7 artist folders, ~1,600 media) in a directory named `None`, as `<date>_None_<original name>`, next to sidecars named `<original name>.json`.

The real sidecar settles why:
- gallery-dl's discord extractor emits `channel` as a plain string, the message as `message_id`, and the attachment position as `num`. It has no `id` key.
- The patterns asked for `{channel[name]}` and `{id}`. gallery-dl renders a missing field as "None" and carries on.
- The sidecar was named `{filename}.json`, the attachment's ORIGINAL name. find_sidecar never pairs that with `<date>_None_<name>.png`, so no Discord file ever got a Post or a post date. The card fell back to downloaded_at.
- Every `image.png` in a channel also overwrote the same `image.json`.

Fix (gallery_dl.py):
- The directory is `{channel}`.
- The filename is `{date:%Y%m%d}_{message_id}_{num:>02}_{filename}.{extension}`, unique per attachment.
- A discord-scoped metadata postprocessor names the sidecar exactly like the media minus its extension, so find_sidecar's first candidate matches. A per-source filename override re-derives the sidecar name. save_metadata=False drops it.

Guard (tests/test_gallery_dl_naming.py) renders the patterns through Python's formatter against a sanitized copy of the real sidecar (same keys and types, invented values). A missing key or a subscript into a string raises, which is the loud failure gallery-dl doesn't give. A positive control shows both shipped patterns fail it.

Existing broken downloads are NOT repaired by this. gallery-dl's archive already records them, so a re-run skips them, and their collided sidecars no longer describe them. That repair is a separate, destructive step for the operator to decide on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 20:01:11 -04:00
bvandeusenandClaude Opus 5 98c3b74260 feat: the Latest feed uses a wide window — filmstrip cards, a day gutter and a filter rail (407 A, D, E)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 6s
Build images / build-ml (push) Successful in 6s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m1s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m10s
On the operator's 3432px window the feed was a 900px column, 26% of the width. They picked three options from a to-scale layout study.

A — filmstrip card (PostCard.vue):
- A card measures itself with a ResizeObserver. At 1100px or wider its hero gets a fixed height, clamp(260px, 34vh, 460px), and the extra images move into a 2-column grid of squares beside it. The grid cells are sized from the hero height, so the grid ends flush with the hero.
- The rail cap is 4 cells in this layout (2×2, the last becoming "+N") and 5 in the narrow layout, which is unchanged.
- The description clamp drops to 4 lines, because long reads happen in the expanded view.
- The hero has a height, not a width, so a wide card can't grow into a full-screen post. That was the operator's constraint.

D — day gutter (PostsView.vue):
- The normal feed groups consecutive posts by local day (Today, Yesterday, a weekday, or a date), with post and artist counts for what has loaded.
- Runs rather than date buckets, because the sort key includes resurfaced_at, which the payload doesn't carry. A resurfaced grouping gets its own heading where it actually appears, instead of being pulled out of order.

E — filter rail (PostsView.vue):
- At 1600px and wider, the filters and status ribbon stack in a sticky 280px left rail, and each day's heading sits in a sticky 150px gutter beside its posts.
- Below 1600px the layout is exactly the old one, including the 900px column.

The in-context (post_id) view gets the wide column but no rail or day grouping, so anchor scrolling is untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 19:48:46 -04:00
bvandeusenandClaude Opus 5 4b4e532c56 fix: the membership status vocabulary moves beside Membership, so the Patreon client never imports the roster
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 35s
Build images / build-web (push) Successful in 1m20s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m6s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m33s
240f11c made PatreonClient._membership import has_paid_access from membership_roster. test_gated_reason::test_no_fetch_path_can_read_the_roster failed on it, correctly: native_ingest_common is a fetch root, patreon_client is reachable from it, and no fetch path may be able to reach the roster. The roster is allowed to explain a skip, never to cause one.

MEMBERSHIP_STATUS and has_paid_access are pure platform knowledge with no database behind them. They move to native_ingest_common, next to the Membership type they interpret (the same move C7 made for Membership itself). membership_roster, membership_reconcile, patreon_client and the tests import them from there. There is no re-export from membership_roster.

The guard is unchanged. The lapsed-orphan skip from 240f11c stays as it was.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 16:07:26 -04:00
bvandeusenandClaude Opus 5 240f11c5aa fix: one lapsed membership to a deleted creator no longer fails the Patreon roster
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 7s
CI / backend-lint-and-test (push) Failing after 35s
CI / frontend-build (push) Successful in 32s
Build images / build-web (push) Successful in 1m20s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m17s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m26s
The first live sweep failed with "Patreon member resource has no campaign relationship". The operator ran FC's exact /api/members request in a browser and shared the response.
- It has 104 rows. The browser's settings page shows 43, because FC sends no membership-type filter and so also gets lapses back to 2016.
- Exactly one row has no `campaign` key at all: a former_patron whose membership ended in 2017. Its included reward has no campaign link either, so the creator's page is gone.
- Every other row, including the 4 is_active=false campaigns, has a campaign.

_membership returns None for a campaign-less row only when has_paid_access says it is definitely not paying (a known lapsed status, or a free member), and iter_memberships skips it. That changes no conclusion. A lapsed membership and an absent one both mean "not paying", and no Source can match a campaign with no id.

An active or unrecognised membership without a campaign still raises, because dropping a membership that might be paid would read downstream as a cancellation.

Paging still counts the rows the server sent, not the rows kept. A test pins that, so a skip can't re-read an offset or stop a page short.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 16:01:50 -04:00
bvandeusenandClaude Opus 5 57c880a623 feat: the full brand mark as a faint backdrop behind every page
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 5s
Build images / build-ml (push) Successful in 6s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m9s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m36s
The operator asked for one large version of the logo as the site background.
It is pinned to the viewport behind .fc-content at about 6% strength, so it
shows in the gutters and on bare page ground while cards and the nav cover it.
The series reader is immersive, skips the shell, and never draws over it.

It is a layered background (the page colour at 94% over logo.svg), not an
overlay element with opacity. An overlay needs the content z-indexed above it,
which turns every page into one stacking context under the nav's z-index 1000
and can trap an in-page overlay beneath the nav. A background changes nothing
about stacking.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 15:58:40 -04:00
bvandeusenandClaude Opus 5 dd766eb976 fix: the favicon and nav brand mark rendered blank — "--" inside an XML comment
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
Build images / build-agent (push) Successful in 5s
Build images / build-ml (push) Successful in 7s
Build images / build-web (push) Successful in 1m8s
CI / integration (push) Successful in 2m36s
7ca6ee0's favicon.svg had a comment naming the `--fc-chrome-rgb` custom property. `--` is illegal inside an XML comment, so the file is not well-formed XML, and a browser renders an SVG-as-image only if it parses. It fails without any error, just a blank image. Because TopNav's brand glyph is the same file, the tab icon and the nav mark both went missing on the #251 deploy. logo.svg parses and was unaffected.

The comment now names the property without the hyphens, and says why it has to.

tests/test_public_svgs.py parses every SVG under frontend/public. It includes a vacuity guard (the directory really contains the two known files) and a positive control (the exact defect raises ParseError), so the check can actually fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 15:57:19 -04:00
bvandeusenandClaude Opus 5 e3c516d6be fix: a retired platform leaves the chip maps, and a fixture stops creating pixiv sources (406)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 6s
Build images / build-ml (push) Successful in 6s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 34s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m25s
extension / lint (pull_request) Successful in 49s
e3fd8c6 failed two lanes, both on things pixiv's retirement correctly started refusing.

Unit - test_fe_be_contract pins platformColor.js's ICONS keys to known_platform_keys(). I had kept pixiv's icon, colour and label "so existing pixiv posts don't look broken". That was wrong, and the file's own header already said why: unknown platforms fall back to a grey chip with the raw key, and that fallback is deliberately what a retired platform hits - it is how a pre-3069 deviantart row already renders. So pixiv leaves all three maps, and its posts show as a grey "pixiv" chip rather than a broken one. The header now says outright that a retired platform is removed, never kept, since the milestone plan itself got this backwards.

Integration - four reassign tests built their fixture source through SourceService.create with platform="pixiv", which the validator now rejects. Reassign never reads the platform and never moves files, so any registered platform serves; the fixture uses hentaifoundry. 1294 other integration tests passed on e3fd8c6, so nothing else used pixiv through a validating path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
2026-09-13 11:52:06 -04:00
35 changed files with 1773 additions and 240 deletions
+14
View File
@@ -475,6 +475,20 @@ async def trigger_reclaim_attachments():
return _queued(async_result) return _queued(async_result)
@admin_bp.route("/maintenance/repair-discord-downloads", methods=["POST"])
async def trigger_repair_discord_downloads():
"""Clean re-download of the Discord files broken by the `None` naming
(#3999). Body {"dry_run": bool}; dry_run is the DEFAULT, because the apply
deletes files and makes gallery-dl forget every Discord download. Returns the
Celery task id — poll /maintenance/task-result/<id> for the summary."""
from ..tasks.admin import repair_discord_downloads_task
body = await request.get_json(silent=True) or {}
dry_run = bool(body.get("dry_run", True))
async_result = repair_discord_downloads_task.delay(dry_run=dry_run)
return _queued(async_result)
@admin_bp.route("/maintenance/dedup-videos", methods=["POST"]) @admin_bp.route("/maintenance/dedup-videos", methods=["POST"])
async def trigger_dedup_videos(): async def trigger_dedup_videos():
"""Tier-1 video dedup (#871). Body {"dry_run": bool}: dry_run=true previews """Tier-1 video dedup (#871). Body {"dry_run": bool}: dry_run=true previews
+10
View File
@@ -65,6 +65,16 @@ async def autocomplete():
]) ])
@artists_bp.route("/names", methods=["GET"])
async def names():
"""Every artist, id + name + slug, alphabetical. For filter pickers that
list artists before anything is typed; `autocomplete` deliberately returns
nothing for an empty query."""
async with get_session() as session:
rows = await ArtistService(session).all_names()
return jsonify([{"id": i, "name": n, "slug": s} for i, n, s in rows])
@artists_bp.route("/directory", methods=["GET"]) @artists_bp.route("/directory", methods=["GET"])
async def directory(): async def directory():
"""FC-3f: cursor-paginated artists directory. """FC-3f: cursor-paginated artists directory.
+12
View File
@@ -300,6 +300,18 @@ class ArtistService:
await self.session.commit() await self.session.commit()
return artist return artist
async def all_names(self) -> list[tuple[int, str, str]]:
"""Every artist as (id, name, slug), alphabetical.
For pickers that should show a full list before anything is typed (the
Latest feed's artist filter). Three columns and no joins, so it stays
cheap on a library of thousands of artists.
"""
rows = (await self.session.execute(
select(Artist.id, Artist.name, Artist.slug).order_by(func.lower(Artist.name))
)).all()
return [(r.id, r.name, r.slug) for r in rows]
async def autocomplete(self, prefix: str, limit: int = 20) -> list[Artist]: async def autocomplete(self, prefix: str, limit: int = 20) -> list[Artist]:
cleaned = (prefix or "").strip() cleaned = (prefix or "").strip()
if not cleaned: if not cleaned:
+183
View File
@@ -0,0 +1,183 @@
"""Repair the Discord downloads made before the naming fix (issue #3999).
Until dc840fe, gallery-dl's Discord patterns asked for keys the extractor never
emits, so every Discord download landed as
`<artist>/discord/None/<date>_None_<original name>`, next to a sidecar named
after the attachment's ORIGINAL name. That broke two things:
* **No Post, no date.** `find_sidecar` can never pair those names, so these
files were imported as loose images with no Post, and a card shows the
download time.
* **No trustworthy metadata to relink from.** Every `image.png` in a channel
wrote the same `image.json`, so the surviving sidecar describes whichever
message was written last. The message id is gone from the filename too.
The operator chose a clean re-download (2026-09-13) over relinking in place:
delete the broken files and their records, make gallery-dl forget it fetched
them, and backfill every Discord source again under the fixed naming.
## Why the archive is cleared for ALL of Discord
gallery-dl records a download as `discord{message_id}_{num}` (upstream
`DiscordExtractor.archive_fmt`, prefixed with the category). The broken files
lost their message ids, so there is no way to forget one source's entries and
not another's. Every Discord download made before the fix is broken, so
forgetting all of them is exactly right. Anything downloaded AFTER the fix still
exists on disk under its correct name, and gallery-dl's `skip` sees the file and
does not fetch it again.
## What it does not touch
Discord posts FC grouped itself (#388 E2) are built from Posts, and these files
never had one, so there is nothing grouped to unwind. Images outside a
`discord/None/` directory are never selected: the path filter requires both the
`None` directory and the `_None_` filename, the pair only the bug produced.
Operator-triggered only (Settings, preview first). Never on a beat.
"""
from __future__ import annotations
import logging
import sqlite3
from pathlib import Path
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from ..models import ImageRecord, Source
from .cleanup_service import delete_images
from .gallery_dl import archive_path
from .source_service import arm_backfill
log = logging.getLogger(__name__)
# `%` and `_` are LIKE wildcards, so the literal underscores around None are
# escaped. `________` is the eight-digit date prefix the old pattern wrote.
_BROKEN_PATH_LIKE = r"%/discord/None/________\_None\_%"
# Upstream keys asset downloads as `asset_{server_id}_{id}`. FC never fetches
# server assets, but excluding them keeps this to exactly the message
# attachments the bug mangled.
_ARCHIVE_SQL_MATCH = r"entry LIKE 'discord%' AND entry NOT LIKE 'discordasset\_%' ESCAPE '\'"
_COUNT_SQL = "SELECT COUNT(*) FROM archive WHERE " + _ARCHIVE_SQL_MATCH
_DELETE_SQL = "DELETE FROM archive WHERE " + _ARCHIVE_SQL_MATCH
def broken_directories(images_root: Path) -> list[Path]:
"""Every `<artist>/discord/None` directory. Artist folders that differ only by
case (`Conto` and `conto`) are separate directories and both are found."""
return sorted(d for d in Path(images_root).glob("*/discord/None") if d.is_dir())
def count_archive_entries(archive: Path) -> int:
return _archive(archive, delete=False)
def forget_archive_entries(archive: Path) -> int:
return _archive(archive, delete=True)
def _archive(archive: Path, *, delete: bool) -> int:
if not archive.is_file():
return 0
# A download running at the same moment holds this file briefly. Waiting
# 30s for its lock beats failing the repair over a transient contention.
conn = sqlite3.connect(str(archive), timeout=30)
try:
has_table = conn.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='archive'"
).fetchone()
if not has_table:
return 0
if not delete:
return conn.execute(_COUNT_SQL).fetchone()[0]
cur = conn.execute(_DELETE_SQL)
conn.commit()
return cur.rowcount
finally:
conn.close()
def _sweep_directory(directory: Path) -> tuple[int, bool]:
"""Remove what the record deletes left behind: the collided sidecars, plus any
file that never became a record (a quarantined or rejected download). Then
the directory itself, if it is empty. Returns (files removed, dir removed)."""
removed = 0
for f in directory.iterdir():
if f.is_file():
try:
f.unlink()
removed += 1
except OSError as exc:
log.warning("discord repair: could not remove %s: %s", f, exc)
try:
directory.rmdir()
return removed, True
except OSError:
return removed, False
def repair_discord_downloads(
session: Session, *, images_root: Path, dry_run: bool,
) -> dict:
images_root = Path(images_root)
archive = archive_path(images_root)
broken = select(ImageRecord.id, ImageRecord.size_bytes).where(
ImageRecord.path.like(_BROKEN_PATH_LIKE, escape="\\")
)
rows = session.execute(broken).all()
image_ids = [r.id for r in rows]
directories = broken_directories(images_root)
sources = session.execute(
select(Source).where(Source.platform == "discord")
).scalars().all()
summary = {
"images": len(image_ids),
"bytes": sum(r.size_bytes or 0 for r in rows),
"directories": len(directories),
"sources": len(sources),
"enabled_sources": sum(1 for s in sources if s.enabled),
}
if dry_run:
summary["archive_entries"] = count_archive_entries(archive)
return summary
deleted = delete_images(session, image_ids=image_ids, images_root=images_root)
swept = 0
directories_removed = 0
for d in directories:
n, gone = _sweep_directory(d)
swept += n
directories_removed += int(gone)
# Only after the files are gone. Forgetting first and failing half way would
# leave gallery-dl free to re-fetch into a directory still full of the old
# copies.
forgotten = forget_archive_entries(archive)
for source in sources:
arm_backfill(source)
session.commit()
remaining = session.execute(
select(func.count(ImageRecord.id)).where(
ImageRecord.path.like(_BROKEN_PATH_LIKE, escape="\\")
)
).scalar_one()
summary.update(
images_deleted=deleted["images_deleted"],
files_failed=deleted["files_failed"],
leftover_files_removed=swept,
directories_removed=directories_removed,
archive_entries=forgotten,
backfills_started=len(sources),
remaining=remaining,
)
log.info("discord repair applied: %s", summary)
return summary
+69 -3
View File
@@ -94,6 +94,58 @@ BACKFILL_CHUNK_SECONDS = 600
_DEFAULT_GDL_TIMEOUT_SECONDS = 870 _DEFAULT_GDL_TIMEOUT_SECONDS = 870
# --- Discord naming ---------------------------------------------------------
#
# Derived from a REAL sidecar (operator's instance, 2026-09-13), not from memory
# of gallery-dl's extractor. What gallery-dl's discord extractor actually emits
# for an attachment: `channel` is a plain STRING (the channel's name), the
# message is `message_id`, the attachment's position in it is `num`, and there
# is NO `id` key at all.
#
# The previous patterns asked for `{channel[name]}` and `{id}`. Both render as
# "None", so every Discord download since the platform was added landed in a
# directory called `None` as `<date>_None_<original name>`. Worse, the sidecar was
# named `{filename}.json` — the attachment's ORIGINAL name — which (a) `find_
# sidecar` can never pair with `<date>_None_<name>.png`, so no Discord file ever
# got a Post or a post date, and (b) collides: every `image.png` in a channel
# overwrote the same `image.json`, so the one sidecar that survived described
# whichever message happened to be written last.
#
# The fix names the sidecar EXACTLY like the media minus its extension, so
# `find_sidecar`'s first candidate (`media.with_suffix(".json")`) is the match
# and the name is unique per attachment. tests/test_gallery_dl_naming.py renders
# these patterns against a sanitized copy of the real sidecar, so a key that
# does not exist fails CI instead of silently becoming "None".
DISCORD_FILENAME = "{date:%Y%m%d}_{message_id}_{num:>02}_{filename}.{extension}"
DISCORD_DIRECTORY = ["{channel}"]
def sidecar_name_for(media_pattern: str) -> str | None:
"""The metadata filename pattern that names a sidecar exactly like its media.
Returns None for a pattern that does not end in `.{extension}`, since then
there is no media stem to mirror and the caller must fall back.
"""
suffix = ".{extension}"
if not media_pattern.endswith(suffix):
return None
return media_pattern[: -len(suffix)] + ".json"
def metadata_postprocessor(filename: str) -> dict:
return {"name": "metadata", "mode": "json", "directory": ".", "filename": filename}
def archive_path(images_root: Path) -> Path:
"""gallery-dl's download archive: the record of what it has already fetched.
One definition, because the Discord repair (services/discord_repair.py) has
to find the same file the downloader writes, without constructing a service
whose __init__ creates directories.
"""
return Path(images_root) / ".gallery-dl" / "archive.sqlite3"
@dataclass @dataclass
class SourceConfig: class SourceConfig:
"""Per-source overrides loaded from Source.config_overrides JSON. """Per-source overrides loaded from Source.config_overrides JSON.
@@ -376,8 +428,11 @@ class GalleryDLService:
}, },
"discord": { "discord": {
"content_types": ["all"], "content_types": ["all"],
"directory": ["{channel[name]}"], "directory": DISCORD_DIRECTORY,
"filename": "{date:%Y%m%d}_{id}_{filename}.{extension}", "filename": DISCORD_FILENAME,
# Overrides the global `{filename}.json` sidecar for this extractor
# only — see the Discord naming note above.
"postprocessors": [metadata_postprocessor(sidecar_name_for(DISCORD_FILENAME))],
"embeds": "all", "embeds": "all",
"stickers": True, "stickers": True,
"reactions": False, "reactions": False,
@@ -402,7 +457,7 @@ class GalleryDLService:
config = { config = {
"extractor": { "extractor": {
"base-directory": str(self.images_root), "base-directory": str(self.images_root),
"archive": str(self._config_dir / "archive.sqlite3"), "archive": str(archive_path(self.images_root)),
"skip": True, "skip": True,
"sleep": self._rate_limit, "sleep": self._rate_limit,
"sleep-request": max(0.5, self._rate_limit / 4), "sleep-request": max(0.5, self._rate_limit / 4),
@@ -497,6 +552,17 @@ class GalleryDLService:
if source_config.filename_pattern: if source_config.filename_pattern:
platform_section["filename"] = source_config.filename_pattern platform_section["filename"] = source_config.filename_pattern
# A platform that names its sidecar after its media must keep doing so
# under a per-source filename override, or the pairing breaks exactly the
# way Discord's did. No metadata wanted means no platform postprocessor
# either — the global list was already dropped above.
if "postprocessors" in platform_section:
mirrored = sidecar_name_for(platform_section.get("filename") or "")
if not source_config.save_metadata or mirrored is None:
platform_section.pop("postprocessors")
else:
platform_section["postprocessors"] = [metadata_postprocessor(mirrored)]
platform_section["metadata"] = source_config.save_metadata platform_section["metadata"] = source_config.save_metadata
return config return config
+129 -5
View File
@@ -8,9 +8,10 @@ trustworthy enough to act on.
1. `subscribed_not_tracked` — you pay for this and FC does not follow it. The 1. `subscribed_not_tracked` — you pay for this and FC does not follow it. The
adoption win, and the only bucket carrying an action. adoption win, and the only bucket carrying an action.
2. `tracked_not_subscribed` — FC follows this and the roster does not show you 2. `tracked_not_subscribed` — FC follows this and the roster does not show you
paying for it. REPORT ONLY, by the operator's decision (2026-09-11): it says paying for it. No longer shown on the card: the operator reversed the
what it sees and links to the existing Subscriptions row, and offers no 2026-09-11 "report only" call on 2026-09-13. The lapsed half of it now ACTS,
one-click disable. in `apply_membership_lapses` below (#3995). The absent half still only
reports, because absence proves nothing.
3. `matched` — the healthy set. Counted, not listed loudly. 3. `matched` — the healthy set. Counted, not listed loudly.
4. `unidentified` — sources this join cannot speak to at all. Reported as 4. `unidentified` — sources this join cannot speak to at all. Reported as
exactly that, because the alternative is filing them under a verdict. exactly that, because the alternative is filing them under a verdict.
@@ -39,7 +40,7 @@ rendered as lapsed. That is the whole reason it returns a tri-state.
from __future__ import annotations from __future__ import annotations
from datetime import datetime from datetime import UTC, datetime
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
@@ -47,12 +48,12 @@ from sqlalchemy.ext.asyncio import AsyncSession
from ..models import Artist, MembershipSync, PlatformMembership, Source from ..models import Artist, MembershipSync, PlatformMembership, Source
from .membership_roster import ( from .membership_roster import (
get_sync_state, get_sync_state,
has_paid_access,
identity_keys_for_source, identity_keys_for_source,
pair_sources_with_memberships, pair_sources_with_memberships,
roster_is_fresh, roster_is_fresh,
url_tail, url_tail,
) )
from .native_ingest_common import has_paid_access
# Why a source appears in `tracked_not_subscribed`. Ordered strongest first — # Why a source appears in `tracked_not_subscribed`. Ordered strongest first —
# the UI renders a different sentence per basis, because collapsing them into # the UI renders a different sentence per basis, because collapsing them into
@@ -233,3 +234,126 @@ async def reconcile_all(session: AsyncSession, now: datetime | None = None) -> d
await reconcile(session, platform=p, now=now) for p in sorted(platforms) await reconcile(session, platform=p, now=now) for p in sorted(platforms)
] ]
} }
# ---------------------------------------------------------------------------
# Stop pulling what the account no longer pays for (#3995)
# ---------------------------------------------------------------------------
#
# Operator decision, 2026-09-13, reversing the 2026-09-11 "report only" call
# for this direction: "if I kill a subscription on patreon I would like the
# pulling to stop on curator as well", with automatic resume on resubscribing.
#
# This is a SOURCE-level action taken by the daily sweep, visible on the source
# row and reversible there. It is not a fetch-path decision. The line C5 draws,
# that the roster never decides a POST is inaccessible, still holds: nothing
# here reads per-post access, and no download path reads the roster
# (`test_no_fetch_path_can_read_the_roster`). The scheduler keeps selecting on
# `enabled` alone.
#
# Acts ONLY on positive evidence. A source whose matched membership says access
# has ended is stopped. A source with NO matched membership is left alone,
# because absence has innocent causes: a creator rename, a source never walked
# so no id is cached, a membership the platform stopped listing. Stopping on
# absence would switch off things the operator still pays for.
#
# Two app-managed config_overrides keys carry the state. The `_` prefix is
# already the "FC writes this, an operator edit preserves it" family.
# _membership_stopped set when the sweep stops a source; the sweep resumes
# ONLY sources carrying it, so a source the operator
# switched off by hand is never switched back on
# _membership_kept set by SourceService.update when the operator turns a
# stopped source back ON: a deliberate choice to keep
# pulling a lapsed creator, which the next sweep must
# not undo. Cleared when the membership is paid again.
STOPPED_KEY = "_membership_stopped"
KEPT_KEY = "_membership_kept"
def _access_expires_at(m: PlatformMembership) -> datetime | None:
"""When paid access actually ends, if the platform says.
Patreon keeps a cancelled membership's access until the end of the billing
period and reports that date (`member.access_expires_at`, note #3992).
SubscribeStar's page gives no such date, so a cancelled SubscribeStar
membership stops at once. Returns None when there is no usable date.
"""
details = m.details or {}
raw = details.get("access_expires_at") or (details.get("member") or {}).get("access_expires_at")
if not isinstance(raw, str) or not raw:
return None
try:
parsed = datetime.fromisoformat(raw.replace("Z", "+00:00"))
except ValueError:
return None
return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
async def apply_membership_lapses(
session: AsyncSession, *, platform: str, now: datetime | None = None,
) -> dict:
"""Stop sources whose paid access has ended; resume the ones this stopped.
Refuses to act on a roster that isn't fresh, for the same reason C4 refuses
to draw conclusions from one.
"""
now = now or datetime.now(UTC)
state = await get_sync_state(session, platform)
if not roster_is_fresh(state, now=now):
return {"platform": platform, "skipped": "roster not fresh", "stopped": 0, "resumed": 0}
memberships = (await session.execute(
select(PlatformMembership).where(PlatformMembership.platform == platform)
)).scalars().all()
sources = (await session.execute(
select(Source).where(Source.platform == platform)
)).scalars().all()
pairs = pair_sources_with_memberships(list(sources), list(memberships))
stopped: list[int] = []
resumed: list[int] = []
for source in sources:
pair = pairs.get(source.id)
if pair is None:
continue # absence is never acted on, see above
m, _kind = pair
paid = has_paid_access(
m.platform, m.status,
is_free_member=bool((m.details or {}).get("is_free_member")),
)
co = dict(source.config_overrides or {})
if paid is True:
changed = co.pop(KEPT_KEY, None) is not None
if STOPPED_KEY in co:
co.pop(STOPPED_KEY)
source.enabled = True
resumed.append(source.id)
changed = True
if changed:
source.config_overrides = co
continue
# Unknown status: never a reason to stop something (has_paid_access's
# tri-state exists for exactly this).
if paid is None:
continue
if not source.enabled or co.get(KEPT_KEY):
continue
expires = _access_expires_at(m)
if expires is not None and expires > now:
continue # still inside the paid-through period
co[STOPPED_KEY] = {"at": now.isoformat(), "status": m.status}
source.config_overrides = co
source.enabled = False
# The same clean slate a manual disable gives (SourceService.update,
# #1285), so a stopped source doesn't linger as failing or gated.
source.last_error = None
source.error_type = None
source.consecutive_failures = 0
stopped.append(source.id)
await session.commit()
return {"platform": platform, "stopped": len(stopped), "resumed": len(resumed)}
+5 -71
View File
@@ -20,8 +20,10 @@ own word — `active_patron`, not some normalised FC value. The mapping from
those words to FC's meaning is a read-site concern and belongs in code that can those words to FC's meaning is a read-site concern and belongs in code that can
be corrected without a migration, because the vocabulary comes from whatever be corrected without a migration, because the vocabulary comes from whatever
each platform says and will be discovered per platform rather than designed up each platform says and will be discovered per platform rather than designed up
front. `MEMBERSHIP_STATUS` below is a place for that knowledge to accumulate as front. `native_ingest_common.MEMBERSHIP_STATUS` is where that knowledge
platforms are characterised; it is deliberately empty of guesses today. accumulates as platforms are characterised, and it holds no guesses. It lives
there rather than here because platform clients need it, and a client may not
import this module (test_gated_reason.py).
""" """
from __future__ import annotations from __future__ import annotations
@@ -35,78 +37,10 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from ..models import MembershipSync, PlatformMembership, Source from ..models import MembershipSync, PlatformMembership, Source
from .native_ingest_common import has_paid_access
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
# Platform word -> whether the account currently has paid access.
#
# Every entry here must come from a CHARACTERISED response, never from API docs
# or a plausible guess — project rule 130, and inventing a status before seeing
# it in a real payload is exactly the failure it names.
#
# patreon: from a live capture of the operator's own session, 2026-09-10
# (Scribe note #3886). Only two values were OBSERVED in `patron_status` and
# only those two are here.
#
# `declined_patron` is deliberately ABSENT even though it looks obviously
# right. It appears in the request's `filter[membership_type]`, and the capture
# proved that filter is NOT the same vocabulary as the attribute — a row
# selected by the filter as `free_member` came back with
# `patron_status: former_patron`, a word the filter does not contain. Reading
# the filter as an enum is the specific mistake the capture caught; adding
# `declined_patron` on the strength of it would be repeating that mistake one
# step later.
#
# Unknown words are NOT an error: an unrecognised status means the roster
# records evidence it cannot yet interpret, which is a better state than
# dropping the row or asserting a meaning for it.
#
# subscribestar: from a live capture of the account's /subscriptions page,
# 2026-09-13 (Scribe note #3989). SubscribeStar gives NO per-row status word —
# a membership's state is which of two tables it sits in — so the "word" stored
# is the table card's own `data-identifier`, verbatim. Those two identifiers are
# the whole vocabulary; there is nothing further to characterise later.
MEMBERSHIP_STATUS: dict[str, dict[str, bool]] = {
"patreon": {
"active_patron": True,
"former_patron": False,
},
"subscribestar": {
"active_subscriptions": True,
"cancelled_subscriptions": False,
},
}
def has_paid_access(
platform: str, status: str | None, *, is_free_member: bool = False,
) -> bool | None:
"""Does this membership mean the account currently PAYS for access?
Returns None for a status this code has not been taught, which callers must
treat as "unknown" rather than as False. The difference matters: False says
the operator has lost access, and asserting that from an unrecognised word
would tell them to cancel a source they are still paying for.
`is_free_member` is a second axis, not a status, and that is Patreon's
design rather than ours: the capture shows a free follow expressed as a
boolean alongside `patron_status`, so a "current" membership can still be
one nobody is paying for. Taking status alone would report a free follower
as a paying patron, and C4 would then never offer to clean it up.
(Honest limit: the capture contains no ACTIVE free member, so it cannot
demonstrate the two axes coming apart. The separation is what the payload's
shape says; the sample only shows it is possible, not that it happens.)
"""
if status is None:
return None
known = MEMBERSHIP_STATUS.get(platform, {}).get(status)
if known is None:
return None
if not known:
return False
return not is_free_member
async def touch_membership( async def touch_membership(
session: AsyncSession, session: AsyncSession,
+79 -1
View File
@@ -231,7 +231,7 @@ class Membership:
`status` carries the PLATFORM's own word, verbatim and unmapped `status` carries the PLATFORM's own word, verbatim and unmapped
(`active_patron`, `former_patron`, ...). Deciding what it means is the read (`active_patron`, `former_patron`, ...). Deciding what it means is the read
site's job — `membership_roster.has_paid_access` — precisely so an site's job — `has_paid_access`, below — precisely so an
unrecognised word records as evidence rather than as a decision. unrecognised word records as evidence rather than as a decision.
`is_free_member` is SEPARATE from status and must stay that way. Patreon `is_free_member` is SEPARATE from status and must stay that way. Patreon
@@ -396,3 +396,81 @@ class BaseNativeDownloader:
sidecar_path = media_path.with_suffix(".json") sidecar_path = media_path.with_suffix(".json")
sidecar_path.write_text(json.dumps(data, indent=2)) sidecar_path.write_text(json.dumps(data, indent=2))
return sidecar_path return sidecar_path
# --- membership status vocabulary (#387) ------------------------------------
#
# Lives here, beside `Membership`, rather than in `membership_roster`. It is
# pure platform knowledge with no database behind it, and the platform clients
# need it too. Patreon's must tell a lapsed membership to a deleted creator
# (skippable) from a paid one it cannot attribute (drift), and a client may not
# import `membership_roster`: test_gated_reason.py forbids any fetch path from
# reaching the roster, so the roster can explain a skip but never cause one.
#
# Platform word -> whether the account currently has paid access.
#
# Every entry here must come from a CHARACTERISED response, never from API docs
# or a plausible guess — project rule 130, and inventing a status before seeing
# it in a real payload is exactly the failure it names.
#
# patreon: from a live capture of the operator's own session, 2026-09-10
# (Scribe note #3886). Only two values were OBSERVED in `patron_status` and
# only those two are here.
#
# `declined_patron` is deliberately ABSENT even though it looks obviously
# right. It appears in the request's `filter[membership_type]`, and the capture
# proved that filter is NOT the same vocabulary as the attribute — a row
# selected by the filter as `free_member` came back with
# `patron_status: former_patron`, a word the filter does not contain. Reading
# the filter as an enum is the specific mistake the capture caught; adding
# `declined_patron` on the strength of it would be repeating that mistake one
# step later.
#
# Unknown words are NOT an error: an unrecognised status means the roster
# records evidence it cannot yet interpret, which is a better state than
# dropping the row or asserting a meaning for it.
#
# subscribestar: from a live capture of the account's /subscriptions page,
# 2026-09-13 (Scribe note #3989). SubscribeStar gives NO per-row status word —
# a membership's state is which of two tables it sits in — so the "word" stored
# is the table card's own `data-identifier`, verbatim. Those two identifiers are
# the whole vocabulary; there is nothing further to characterise later.
MEMBERSHIP_STATUS: dict[str, dict[str, bool]] = {
"patreon": {
"active_patron": True,
"former_patron": False,
},
"subscribestar": {
"active_subscriptions": True,
"cancelled_subscriptions": False,
},
}
def has_paid_access(
platform: str, status: str | None, *, is_free_member: bool = False,
) -> bool | None:
"""Does this membership mean the account currently PAYS for access?
Returns None for a status this code has not been taught, which callers must
treat as "unknown" rather than as False. The difference matters: False says
the operator has lost access, and asserting that from an unrecognised word
would tell them to cancel a source they are still paying for.
`is_free_member` is a second axis, not a status, and that is Patreon's
design rather than ours: the capture shows a free follow expressed as a
boolean alongside `patron_status`, so a "current" membership can still be
one nobody is paying for. Taking status alone would report a free follower
as a paying patron, and C4 would then never offer to clean it up.
(Honest limit: the capture contains no ACTIVE free member, so it cannot
demonstrate the two axes coming apart. The separation is what the payload's
shape says; the sample only shows it is possible, not that it happens.)
"""
if status is None:
return None
known = MEMBERSHIP_STATUS.get(platform, {}).get(status)
if known is None:
return None
if not known:
return False
return not is_free_member
+34 -2
View File
@@ -58,6 +58,7 @@ from .native_ingest_common import (
NativeDriftError, NativeDriftError,
NativeIngestError, NativeIngestError,
basename_from_url, basename_from_url,
has_paid_access,
make_session, make_session,
retry_after_seconds, retry_after_seconds,
) )
@@ -631,7 +632,25 @@ class PatreonClient:
"cannot tell a complete roster from a truncated one" "cannot tell a complete roster from a truncated one"
) )
def _membership(self, member: dict, index: dict) -> Membership: def _membership(self, member: dict, index: dict) -> Membership | None:
"""One member row as a Membership, or None for a row the roster can skip.
The one skippable row is a LAPSED membership whose creator no longer
exists. The live roster (note #3886, CORRECTION 3) returned 104 rows,
because FC sends no membership-type filter and so gets lapses going back
years. One of them, a membership that ended in 2017, carried no
`campaign` relationship at all: the key is absent, not null, and its
reward names no campaign either. The creator's page is gone.
Raising on that row made the whole roster unusable over one membership
nobody can act on. Skipping it changes no conclusion. A lapsed
membership already means "not paying", absence means the same, and no
Source can be matched to a campaign that no longer has an id.
The refusal stays for every other row. An active or unrecognised
membership without a creator is something FC cannot vouch for, and
dropping it would read downstream as a cancellation.
"""
attrs = member.get("attributes") or {} attrs = member.get("attributes") or {}
if "patron_status" not in attrs: if "patron_status" not in attrs:
raise PatreonDriftError( raise PatreonDriftError(
@@ -640,6 +659,17 @@ class PatreonClient:
campaign_ids = self._related_ids(member, "campaign") campaign_ids = self._related_ids(member, "campaign")
if not campaign_ids: if not campaign_ids:
paid = has_paid_access(
"patreon", attrs.get("patron_status"),
is_free_member=bool(attrs.get("is_free_member")),
)
if paid is False:
log.info(
"Patreon roster: skipping a lapsed membership with no campaign "
"(creator deleted); status=%s access_expires_at=%s",
attrs.get("patron_status"), attrs.get("access_expires_at"),
)
return None
raise PatreonDriftError( raise PatreonDriftError(
"Patreon member resource has no campaign relationship — a " "Patreon member resource has no campaign relationship — a "
"membership we cannot attribute to a creator is not usable" "membership we cannot attribute to a creator is not usable"
@@ -700,7 +730,9 @@ class PatreonClient:
index = self._transform(response) index = self._transform(response)
rows = [m for m in (response.get("data") or []) if isinstance(m, dict)] rows = [m for m in (response.get("data") or []) if isinstance(m, dict)]
for member in rows: for member in rows:
yield self._membership(member, index) membership = self._membership(member, index)
if membership is not None:
yield membership
seen += len(rows) seen += len(rows)
total = int(response["meta"]["pagination"]["total"] or 0) total = int(response["meta"]["pagination"]["total"] or 0)
+37 -7
View File
@@ -19,6 +19,7 @@ from ..models import (
) )
from .db_helpers import failing_sources_clause from .db_helpers import failing_sources_clause
from .gallery_dl import ErrorType from .gallery_dl import ErrorType
from .membership_reconcile import KEPT_KEY, STOPPED_KEY
from .membership_roster import gated_reasons_for_sources from .membership_roster import gated_reasons_for_sources
from .platforms import known_platform_keys from .platforms import known_platform_keys
from .scheduler_service import compute_next_check_at from .scheduler_service import compute_next_check_at
@@ -158,6 +159,38 @@ def _is_app_managed(key: str) -> bool:
BACKFILL_MAX_CHUNKS = 200 BACKFILL_MAX_CHUNKS = 200
def arm_backfill(source: Source) -> None:
"""Arm a fresh run-until-done backfill on `source` (plan #693). Mutation
only — the caller commits. Shared by `SourceService.start_backfill` and the
sync Discord repair, so both clear exactly the same resume state."""
co = dict(source.config_overrides or {})
co["_backfill_state"] = "running"
for k in ("_backfill_cursor", "_backfill_cursor_stalls", "_backfill_chunks",
"_backfill_posts"):
co.pop(k, None)
source.config_overrides = co
source.backfill_runs_remaining = BACKFILL_MAX_CHUNKS
def _record_manual_enable_choice(source: Source, *, enabled: bool) -> None:
"""Keep the membership sweep (#3995) from overriding the operator.
Turning a source the sweep STOPPED back on is a deliberate choice to keep
pulling a lapsed creator, so it is marked kept and the next sweep leaves it
alone. Turning a source off by hand drops any sweep marker, so the sweep
never switches back on something the operator switched off themselves.
"""
co = dict(source.config_overrides or {})
if enabled and STOPPED_KEY in co:
co.pop(STOPPED_KEY)
co[KEPT_KEY] = True
elif not enabled:
co.pop(STOPPED_KEY, None)
else:
return
source.config_overrides = co
class SourceService: class SourceService:
def __init__(self, session: AsyncSession): def __init__(self, session: AsyncSession):
self.session = session self.session = session
@@ -415,6 +448,9 @@ class SourceService:
for key, value in fields.items(): for key, value in fields.items():
setattr(source, key, value) setattr(source, key, value)
if "enabled" in fields:
_record_manual_enable_choice(source, enabled=bool(fields["enabled"]))
if url_changed: if url_changed:
# Repointing a source at a different creator makes a cached campaign # Repointing a source at a different creator makes a cached campaign
# id WRONG, not merely stale, and `patreon_resolver` consults that # id WRONG, not merely stale, and `patreon_resolver` consults that
@@ -473,13 +509,7 @@ class SourceService:
)).scalar_one_or_none() )).scalar_one_or_none()
if source is None: if source is None:
raise LookupError(f"source id={source_id} not found") raise LookupError(f"source id={source_id} not found")
co = dict(source.config_overrides or {}) arm_backfill(source)
co["_backfill_state"] = "running"
for k in ("_backfill_cursor", "_backfill_cursor_stalls", "_backfill_chunks",
"_backfill_posts"):
co.pop(k, None)
source.config_overrides = co
source.backfill_runs_remaining = BACKFILL_MAX_CHUNKS
await self.session.commit() await self.session.commit()
return await self._row_to_record(source) return await self._row_to_record(source)
+1 -1
View File
@@ -316,7 +316,7 @@ _ROSTER_URL = f"{_ROSTER_BASE}/subscriptions"
# `data-identifier`, the one vocabulary that names a state: the table class # `data-identifier`, the one vocabulary that names a state: the table class
# inside the cancelled card says `for-unsubscribed_users`, a different word for # inside the cancelled card says `for-unsubscribed_users`, a different word for
# the same list (note #3989, CORRECTION 1). The identifier is stored verbatim as # the same list (note #3989, CORRECTION 1). The identifier is stored verbatim as
# Membership.status and mapped in membership_roster.MEMBERSHIP_STATUS. # Membership.status and mapped in native_ingest_common.MEMBERSHIP_STATUS.
_ROSTER_ACTIVE = "active_subscriptions" _ROSTER_ACTIVE = "active_subscriptions"
_ROSTER_CANCELLED = "cancelled_subscriptions" _ROSTER_CANCELLED = "cancelled_subscriptions"
+22
View File
@@ -122,6 +122,28 @@ def prune_missing_file_records_task(self) -> dict:
return {"checked": checked, "missing": len(missing_ids), "deleted": deleted} return {"checked": checked, "missing": len(missing_ids), "deleted": deleted}
@celery.task(
name="backend.app.tasks.admin.repair_discord_downloads_task",
bind=True,
autoretry_for=(OperationalError, DBAPIError),
retry_backoff=15, retry_backoff_max=180, max_retries=1,
soft_time_limit=1800, time_limit=2400, # 30 min / 40 min
)
def repair_discord_downloads_task(self, dry_run: bool = True) -> dict:
"""Clean re-download of the Discord files broken by the `None` naming
(#3999). dry_run (the default) returns the projection; apply deletes the
broken images and their files, makes gallery-dl forget every Discord
download, and restarts every Discord source's backfill. Defaults to the SAFE
preview because the apply deletes files. Operator-triggered only."""
from ..services.discord_repair import repair_discord_downloads
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
return repair_discord_downloads(
session, images_root=IMAGES_ROOT, dry_run=dry_run,
)
@celery.task( @celery.task(
name="backend.app.tasks.admin.dedup_videos_task", name="backend.app.tasks.admin.dedup_videos_task",
bind=True, bind=True,
+18 -3
View File
@@ -1247,6 +1247,7 @@ def sync_memberships() -> str:
from ..services.artist_membership_service import rescan as membership_rescan from ..services.artist_membership_service import rescan as membership_rescan
from ..services.credential_crypto import CredentialCrypto from ..services.credential_crypto import CredentialCrypto
from ..services.credential_service import CredentialService from ..services.credential_service import CredentialService
from ..services.membership_reconcile import apply_membership_lapses
from ..services.membership_roster import roster_user_id, sync_platform from ..services.membership_roster import roster_user_id, sync_platform
from ..services.patreon_client import PatreonClient from ..services.patreon_client import PatreonClient
from ..services.subscribestar_client import SubscribeStarClient from ..services.subscribestar_client import SubscribeStarClient
@@ -1298,8 +1299,17 @@ def sync_memberships() -> str:
) )
async with async_factory() as session: async with async_factory() as session:
results.append( result = await sync_platform(session, platform=platform, fetch=fetch)
await sync_platform(session, platform=platform, fetch=fetch) results.append(result)
# #3995: stop pulling sources whose paid access has ended, and
# resume the ones this stopped once they are paid again. Only
# right after a successful sync, so it always acts on the roster
# just written, never on a stale one.
if result.get("ok"):
async with async_factory() as session:
result["lapses"] = await apply_membership_lapses(
session, platform=platform,
) )
# #388 E4: offer the freshly-synced roster to the artists FC already # #388 E4: offer the freshly-synced roster to the artists FC already
@@ -1321,7 +1331,12 @@ def sync_memberships() -> str:
if "skipped" in r: if "skipped" in r:
parts.append(f"{r['platform']}=skipped({r['skipped']})") parts.append(f"{r['platform']}=skipped({r['skipped']})")
elif r.get("ok"): elif r.get("ok"):
parts.append(f"{r['platform']}={r['count']}") lapses = r.get("lapses") or {}
detail = (
f"(stopped={lapses['stopped']},resumed={lapses['resumed']})"
if lapses.get("stopped") or lapses.get("resumed") else ""
)
parts.append(f"{r['platform']}={r['count']}{detail}")
else: else:
parts.append(f"{r['platform']}=FAILED({r['error']})") parts.append(f"{r['platform']}=FAILED({r['error']})")
if res.get("suggested") is not None: if res.get("suggested") is not None:
+4 -1
View File
@@ -15,7 +15,10 @@
Colours are theme tokens (frontend/src/theme/fabled-tokens.js): obsidian Colours are theme tokens (frontend/src/theme/fabled-tokens.js): obsidian
plate, accent gold. The plate is kept here (unlike logo.svg) so the tab plate, accent gold. The plate is kept here (unlike logo.svg) so the tab
icon is self-contained against any browser chrome; on the nav it is icon is self-contained against any browser chrome; on the nav it is
invisible because it matches --fc-chrome-rgb exactly. --> invisible because it matches the fc-chrome-rgb custom property exactly.
No double hyphen may appear inside this comment: XML forbids it, and a
browser refuses to render an SVG that does not parse (it happened once —
tests/test_public_svgs.py). -->
<rect width="32" height="32" rx="6" fill="#14171A"/> <rect width="32" height="32" rx="6" fill="#14171A"/>
<rect x="6.2" y="4.2" width="19.6" height="23.6" rx="1.4" <rect x="6.2" y="4.2" width="19.6" height="23.6" rx="1.4"
fill="none" stroke="#A87338" stroke-width="2.4"/> fill="none" stroke="#A87338" stroke-width="2.4"/>

Before

Width:  |  Height:  |  Size: 1.4 KiB

After

Width:  |  Height:  |  Size: 1.7 KiB

+16
View File
@@ -17,6 +17,22 @@ const route = useRoute()
<style scoped> <style scoped>
.fc-content { .fc-content {
/* The full brand mark as one large, faint backdrop behind every page,
pinned to the viewport so content scrolls over it. Opaque surfaces
(cards, the nav) cover it; it shows in the gutters and on bare page
ground. The series reader is immersive and skips the shell, so reading
is never drawn over it.
Faded by laying the page colour over it at 94%, NOT with `opacity` on an
overlay element: an overlay needs this element to be z-indexed above it,
which makes all page content one stacking context under the nav and can
trap an in-page overlay beneath it. A background changes no stacking.
The mark is gold and parchment, close to the text colours, so it has to
stay this faint to keep text over it readable. */
background:
linear-gradient(rgba(var(--v-theme-background), 0.94), rgba(var(--v-theme-background), 0.94)),
url('/logo.svg') center / min(88vmin, 1100px) no-repeat;
background-attachment: fixed;
min-height: 100vh; min-height: 100vh;
/* NO padding-top: the TopNav is position:sticky, so it already reserves its /* NO padding-top: the TopNav is position:sticky, so it already reserves its
own space in the v-app flex column — content flows directly below it. The own space in the v-app flex column — content flows directly below it. The
+73 -11
View File
@@ -1,5 +1,8 @@
<template> <template>
<v-card class="fc-post-card" variant="outlined"> <v-card
ref="cardEl" class="fc-post-card" variant="outlined"
:class="{ 'fc-post-card--wide': wide }"
>
<div class="fc-post-card__head"> <div class="fc-post-card__head">
<!-- Posts with no live subscription have source=null (alembic 0030); <!-- Posts with no live subscription have source=null (alembic 0030);
show a "filesystem import" affordance instead of a platform chip. --> show a "filesystem import" affordance instead of a platform chip. -->
@@ -58,7 +61,7 @@
</button> </button>
<div <div
v-if="rail.length || moreCount" class="fc-post-card__rail" v-if="rail.length || moreCount" class="fc-post-card__rail"
:style="{ '--fc-rail-cols': railCols }" :style="{ '--fc-rail-cols': railCols, '--fc-grid-cols': Math.min(2, railCols) }"
> >
<button <button
v-for="t in rail" :key="t.image_id" type="button" v-for="t in rail" :key="t.image_id" type="button"
@@ -221,20 +224,33 @@ const synthesisTitle = computed(() => {
}) })
const hero = computed(() => images.value[0]) const hero = computed(() => images.value[0])
// The thumbnail strip spans the hero's full width (CSS grid, equal columns),
// rather than a fixed 3-cell cap. Show up to RAIL_MAX cells; when there are // Filmstrip layout (milestone #407, option A). On a very wide window a card
// more images than fit, the last cell becomes a "+N" overflow tile so the // that just grew would give one post the whole screen, so a WIDE card instead
// count stays accurate. // pins the hero to a fixed height and moves the extra images into a 2-column
const RAIL_MAX = 5 // grid BESIDE it. Measured on the card rather than the viewport because the
// same card renders in the Latest feed, Browse, and the in-context view, each
// at a different width.
const WIDE_CARD_PX = 1100
const cardEl = ref(null)
const wide = ref(false)
// The narrow layout's strip spans the hero's full width (CSS grid, equal
// columns); the wide layout's grid is 2×2. Show up to that many cells; when
// there are more images than fit, the last cell becomes a "+N" overflow tile so
// the count stays accurate.
const RAIL_MAX_NARROW = 5
const RAIL_MAX_WIDE = 4
const serverMore = computed(() => props.post.thumbnails_more || 0) const serverMore = computed(() => props.post.thumbnails_more || 0)
const afterHero = computed(() => images.value.slice(1)) const afterHero = computed(() => images.value.slice(1))
const railMax = computed(() => (wide.value ? RAIL_MAX_WIDE : RAIL_MAX_NARROW))
const hasOverflow = computed( const hasOverflow = computed(
() => serverMore.value > 0 || afterHero.value.length > RAIL_MAX, () => serverMore.value > 0 || afterHero.value.length > railMax.value,
) )
const rail = computed(() => const rail = computed(() =>
hasOverflow.value hasOverflow.value
? afterHero.value.slice(0, RAIL_MAX - 1) ? afterHero.value.slice(0, railMax.value - 1)
: afterHero.value.slice(0, RAIL_MAX), : afterHero.value.slice(0, railMax.value),
) )
const visibleCount = computed(() => (images.value.length ? 1 + rail.value.length : 0)) const visibleCount = computed(() => (images.value.length ? 1 + rail.value.length : 0))
const moreCount = computed(() => { const moreCount = computed(() => {
@@ -349,8 +365,17 @@ function measureOverflow () {
} }
let ro = null let ro = null
let cardRo = null
onMounted(() => { onMounted(() => {
nextTick(measureOverflow) nextTick(measureOverflow)
const root = cardEl.value?.$el
if (typeof ResizeObserver !== 'undefined' && root) {
cardRo = new ResizeObserver((entries) => {
const w = entries[0]?.contentRect?.width ?? 0
wide.value = w >= WIDE_CARD_PX
})
cardRo.observe(root)
}
// Re-measure when the card resizes (the container-query clamp differs by // Re-measure when the card resizes (the container-query clamp differs by
// width). Guarded for happy-dom / older runtimes without ResizeObserver. // width). Guarded for happy-dom / older runtimes without ResizeObserver.
if (typeof ResizeObserver !== 'undefined' && descEl.value) { if (typeof ResizeObserver !== 'undefined' && descEl.value) {
@@ -358,7 +383,10 @@ onMounted(() => {
ro.observe(descEl.value) ro.observe(descEl.value)
} }
}) })
onBeforeUnmount(() => { if (ro) { ro.disconnect(); ro = null } }) onBeforeUnmount(() => {
if (ro) { ro.disconnect(); ro = null }
if (cardRo) { cardRo.disconnect(); cardRo = null }
})
async function toggleDesc () { async function toggleDesc () {
if (!descExpanded.value) { if (!descExpanded.value) {
@@ -495,6 +523,40 @@ function formatBytes (n) {
color: rgb(var(--v-theme-accent)); color: rgb(var(--v-theme-accent));
} }
/* --- Filmstrip layout (wide cards, #407 A) ---------------------------------
The hero has a HEIGHT, not a width: a wide card must not turn into a
full-screen post, so its height stays roughly a third of the viewport
whatever the window's width. The extra images sit beside it as square cells
whose size derives from that same height, so the grid always ends flush with
the hero's bottom edge. */
.fc-post-card--wide {
--fc-hero-h: clamp(260px, 34vh, 460px);
--fc-grid-gap: 8px;
--fc-cell: calc((var(--fc-hero-h) - var(--fc-grid-gap)) / 2);
}
.fc-post-card--wide .fc-post-card__body { flex-direction: row; gap: 24px; }
.fc-post-card--wide .fc-post-card__media {
flex: 0 0 auto;
display: flex;
gap: var(--fc-grid-gap);
}
.fc-post-card--wide .fc-post-card__hero {
width: auto;
height: var(--fc-hero-h);
}
.fc-post-card--wide .fc-post-card__rail {
margin-top: 0;
gap: var(--fc-grid-gap);
grid-template-columns: repeat(var(--fc-grid-cols, 2), var(--fc-cell));
grid-auto-rows: var(--fc-cell);
}
.fc-post-card--wide .fc-post-card__rail-cell,
.fc-post-card--wide .fc-post-card__rail-more { height: 100%; }
.fc-post-card--wide .fc-post-card__text { flex: 1 1 0; min-width: 0; }
/* Text is secondary here — long reads happen in the expanded view — so the
clamp keeps the text column no taller than the images beside it. */
.fc-post-card--wide .fc-post-card__desc--clamped { -webkit-line-clamp: 4; }
.fc-post-card__title { .fc-post-card__title {
font-family: 'Fraunces', Georgia, serif; font-family: 'Fraunces', Georgia, serif;
font-size: 18px; font-weight: 700; font-size: 18px; font-weight: 700;
@@ -1,21 +1,22 @@
<template> <template>
<div class="fc-posts-filters"> <div class="fc-posts-filters">
<!-- The full artist list, filtered client-side as you type. This used to
search the server only once something was typed, so opening the
dropdown showed an empty menu. -->
<v-autocomplete <v-autocomplete
v-model="artistModel" v-model="artistModel"
:items="artistOptions" :items="artistOptions"
:loading="artistLoading" :loading="artistLoading"
:search="artistQuery"
item-title="name" item-title="name"
item-value="id" item-value="id"
label="Artist" label="Artist"
density="compact" density="compact"
hide-details hide-details
clearable clearable
no-filter
return-object return-object
no-data-text="No matching artists"
class="fc-posts-filters__artist" class="fc-posts-filters__artist"
@update:search="onArtistSearch" @update:model-value="emitFilters"
@update:model-value="onArtistPicked"
/> />
<v-select <v-select
@@ -39,7 +40,8 @@
<script setup> <script setup>
import { computed, onMounted, ref, watch } from 'vue' import { computed, onMounted, ref, watch } from 'vue'
import { useSourcesStore } from '../../stores/sources.js'
import { useApi } from '../../composables/useApi.js'
import { usePlatformsStore } from '../../stores/platforms.js' import { usePlatformsStore } from '../../stores/platforms.js'
const props = defineProps({ const props = defineProps({
@@ -48,18 +50,19 @@ const props = defineProps({
}) })
const emit = defineEmits(['update:filters']) const emit = defineEmits(['update:filters'])
const sourcesStore = useSourcesStore() const api = useApi()
const platformsStore = usePlatformsStore() const platformsStore = usePlatformsStore()
// Artist autocomplete state
const artistQuery = ref('')
const artistOptions = ref([]) const artistOptions = ref([])
const artistLoading = ref(false) const artistLoading = ref(false)
const artistModel = ref(null) const artistModel = ref(null)
// Platform v-select state // `list`, not `platforms`: the store has never had a `platforms` property, so
// this read undefined and the dropdown opened empty.
// test/storeUsage.spec.js now fails on any read of a property the store does
// not define.
const platformItems = computed(() => const platformItems = computed(() =>
(platformsStore.platforms || []).map(p => ({ title: p.key, value: p.key })) platformsStore.list.map(p => ({ title: p.name || p.key, value: p.key }))
) )
const platformModel = ref(props.platform) const platformModel = ref(props.platform)
@@ -67,29 +70,6 @@ const hasFilters = computed(
() => artistModel.value != null || platformModel.value != null () => artistModel.value != null || platformModel.value != null
) )
let _searchTimer = null
function onArtistSearch(q) {
artistQuery.value = q
clearTimeout(_searchTimer)
if (!q || !q.trim()) {
artistOptions.value = artistModel.value ? [artistModel.value] : []
return
}
_searchTimer = setTimeout(async () => {
artistLoading.value = true
try {
const results = await sourcesStore.autocompleteArtist(q, 20)
artistOptions.value = results
} finally {
artistLoading.value = false
}
}, 200)
}
function onArtistPicked() {
emitFilters()
}
function emitFilters() { function emitFilters() {
emit('update:filters', { emit('update:filters', {
artist_id: artistModel.value?.id ?? null, artist_id: artistModel.value?.id ?? null,
@@ -100,30 +80,38 @@ function emitFilters() {
function clearAll() { function clearAll() {
artistModel.value = null artistModel.value = null
platformModel.value = null platformModel.value = null
artistOptions.value = []
artistQuery.value = ''
emitFilters() emitFilters()
} }
onMounted(async () => { // A deep link carries only the id. Once the list is loaded, show the real name
await platformsStore.loadAll() // rather than a placeholder.
// If a deep-link arrives with an artist_id, hydrate the selected display. function selectArtistById(id) {
if (props.artistId != null) { if (id == null) {
const seed = { id: props.artistId, name: `Artist #${props.artistId}` } artistModel.value = null
artistModel.value = seed return
artistOptions.value = [seed]
} }
if (artistModel.value?.id === id) return
artistModel.value = artistOptions.value.find(a => a.id === id)
|| { id, name: `Artist #${id}` }
}
async function loadArtists() {
artistLoading.value = true
try {
artistOptions.value = await api.get('/api/artists/names')
} catch {
artistOptions.value = []
} finally {
artistLoading.value = false
}
}
onMounted(async () => {
await Promise.all([platformsStore.loadAll(), loadArtists()])
selectArtistById(props.artistId)
}) })
watch(() => props.artistId, (val) => { watch(() => props.artistId, (val) => selectArtistById(val))
if (val == null) {
artistModel.value = null
} else if (artistModel.value?.id !== val) {
const seed = { id: val, name: `Artist #${val}` }
artistModel.value = seed
artistOptions.value = [seed]
}
})
watch(() => props.platform, (val) => { watch(() => props.platform, (val) => {
if (platformModel.value !== val) platformModel.value = val || null if (platformModel.value !== val) platformModel.value = val || null
@@ -0,0 +1,118 @@
<template>
<!-- #3999: Discord downloads made before the naming fix landed in a `None`
folder with no post and a download-time date. This re-downloads them
cleanly. Preview first; the apply deletes files. -->
<MaintenanceTile
icon="mdi-download-off-outline"
title="Repair Discord downloads"
blurb="Re-download Discord images that were saved with no post and the wrong date."
destructive
:open="applying || previewing"
>
<p class="text-body-2 mb-3">
Before the naming fix, every Discord download was saved into a folder
called <code>None</code>. Those images never got a post, so they show the
time Curator downloaded them rather than when they were posted.
<strong>Apply</strong> deletes those images, clears Discord from the
download history, and starts a fresh backfill of every Discord source so
they come back with their posts and dates. Tags you added by hand to those
images are lost. Nothing outside a Discord <code>None</code> folder is
touched.
</p>
<div class="d-flex align-center flex-wrap" style="gap: 12px;">
<v-btn
color="primary" variant="tonal" rounded="pill"
:loading="previewing" :disabled="applying" @click="preview"
>
<v-icon start>mdi-magnify</v-icon> Preview
</v-btn>
<v-btn
color="error" rounded="pill"
:loading="applying"
:disabled="previewing || !canApply"
@click="confirmOpen = true"
>
<v-icon start>mdi-download-off-outline</v-icon> Apply
</v-btn>
</div>
<v-alert
v-if="summary" :type="summaryType" variant="tonal" class="mt-4"
density="comfortable"
>
<span v-if="applied">
Deleted {{ summary.images_deleted }} image(s) and cleared
{{ summary.archive_entries }} Discord download record(s). Backfills
started on {{ summary.backfills_started }} Discord source(s).
</span>
<span v-else-if="hasWork">
{{ summary.images }} broken image(s) ({{ humanBytes(summary.bytes) }})
across {{ summary.directories }} folder(s), and
{{ summary.archive_entries }} Discord download record(s) to clear.
{{ summary.sources }} Discord source(s) will backfill again.
</span>
<span v-else>Nothing to repair no broken Discord downloads found.</span>
<div v-if="applied && summary.files_failed" class="mt-1 text-caption">
{{ summary.files_failed }} file(s) could not be removed see the worker log.
</div>
<div v-if="applied && summary.remaining" class="mt-1 text-caption">
{{ summary.remaining }} broken image(s) are still present. Run it again.
</div>
</v-alert>
<QueueStatusBar queue="maintenance_long" queue-label="Maintenance" />
<v-dialog v-model="confirmOpen" max-width="440">
<v-card>
<v-card-title>Repair Discord downloads?</v-card-title>
<v-card-text class="text-body-2">
This permanently deletes <strong>{{ summary?.images ?? 0 }}</strong>
Discord image(s) ({{ humanBytes(summary?.bytes) }}) and re-downloads
them from Discord with their posts and dates. Any tags you added to
those images by hand will not come back.
</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn variant="text" @click="confirmOpen = false">Cancel</v-btn>
<v-btn color="error" @click="apply">Repair</v-btn>
</v-card-actions>
</v-card>
</v-dialog>
</MaintenanceTile>
</template>
<script setup>
import { computed, ref } from 'vue'
import { useMaintenanceTask } from '../../composables/useMaintenanceTask.js'
import { humanBytes } from '../../utils/bytes.js'
import MaintenanceTile from '../common/MaintenanceTile.vue'
import QueueStatusBar from './QueueStatusBar.vue'
const confirmOpen = ref(false)
const { previewing, applying, summary, applied, preview, apply: applyTask } = useMaintenanceTask({
endpoint: '/api/admin/maintenance/repair-discord-downloads',
storageKey: 'fc.maint.repairDiscordDownloads',
appliedToast: 'Discord downloads cleared — backfills started',
})
// The archive count matters too: files could already be gone while gallery-dl
// still believes it has them, which would stop the backfill re-fetching.
const hasWork = computed(
() => !!summary.value && (summary.value.images > 0 || summary.value.archive_entries > 0),
)
const canApply = computed(() => hasWork.value && !applied.value)
const summaryType = computed(() => {
if (applied.value) return 'success'
return hasWork.value ? 'info' : 'success'
})
// The confirm dialog gates the destructive apply; close it, then run.
function apply () {
confirmOpen.value = false
applyTask()
}
</script>
@@ -15,6 +15,7 @@
<ImportFiltersForm /> <ImportFiltersForm />
<TranslationCard /> <TranslationCard />
<DiscordGroupingCard /> <DiscordGroupingCard />
<DiscordRepairCard />
<PostAssociationsCard /> <PostAssociationsCard />
<MembershipRosterCard /> <MembershipRosterCard />
<MembershipSuggestionsCard /> <MembershipSuggestionsCard />
@@ -85,6 +86,7 @@ import VideoEmbeddingCard from './VideoEmbeddingCard.vue'
import CropProposersCard from './CropProposersCard.vue' import CropProposersCard from './CropProposersCard.vue'
import HeadsCard from './HeadsCard.vue' import HeadsCard from './HeadsCard.vue'
import DiscordGroupingCard from './DiscordGroupingCard.vue' import DiscordGroupingCard from './DiscordGroupingCard.vue'
import DiscordRepairCard from './DiscordRepairCard.vue'
import MembershipRosterCard from './MembershipRosterCard.vue' import MembershipRosterCard from './MembershipRosterCard.vue'
import MembershipSuggestionsCard from './MembershipSuggestionsCard.vue' import MembershipSuggestionsCard from './MembershipSuggestionsCard.vue'
import PostAssociationsCard from './PostAssociationsCard.vue' import PostAssociationsCard from './PostAssociationsCard.vue'
@@ -1,12 +1,20 @@
<template> <template>
<!-- Renders nothing when there is nothing to say same posture as <!-- Renders nothing when there is nothing to say same posture as
NeedsAttentionCard. A reconciliation card that always shows would train NeedsAttentionCard. A reconciliation card that always shows would train
the operator to scroll past it. --> the operator to scroll past it. It can also be DISMISSED, and stays
<v-card v-if="anythingToSay" variant="tonal" class="mb-4 fc-recon"> dismissed until what it would say changes. -->
<v-card v-if="visible" variant="tonal" class="mb-4 fc-recon">
<v-card-text> <v-card-text>
<v-btn
icon="mdi-close" size="small" variant="text" class="fc-recon__dismiss"
title="Hide until something changes" aria-label="Hide until something changes"
@click="dismiss"
/>
<div v-for="p in interesting" :key="p.platform" class="fc-recon__platform"> <div v-for="p in interesting" :key="p.platform" class="fc-recon__platform">
<!-- Bucket 1: the adoption win. The only direction with an action, <!-- The adoption win: subscriptions FC doesn't follow yet. The only
because adding a source is the reversible half. --> bucket shown. Sources you follow but no longer pay for are not
listed here (operator, 2026-09-13) the membership sweep stops
pulling those instead (#3995). -->
<template v-if="p.subscribed_not_tracked.length"> <template v-if="p.subscribed_not_tracked.length">
<div class="fc-recon__head"> <div class="fc-recon__head">
<v-icon icon="mdi-account-plus-outline" size="small" class="me-2" /> <v-icon icon="mdi-account-plus-outline" size="small" class="me-2" />
@@ -36,37 +44,13 @@
</div> </div>
</template> </template>
<!-- Bucket 2: REPORT ONLY. No disable control here by design — the <!-- The roster could not be trusted. Said in words: an empty list here
source list on this same page is where that decision belongs. --> must never read as "all clear". -->
<template v-if="p.tracked_not_subscribed.length">
<div class="fc-recon__head">
<v-icon icon="mdi-help-circle-outline" size="small" class="me-2" />
<strong>
{{ p.tracked_not_subscribed.length }}
{{ p.platform }}
{{ p.tracked_not_subscribed.length === 1 ? 'source' : 'sources' }}
your roster doesn't account for
</strong>
</div>
<div v-for="s in p.tracked_not_subscribed" :key="s.id" class="fc-recon__row">
<div class="fc-recon__body">
<strong>{{ s.artist.name }}</strong>
<div class="fc-recon__dim">{{ reasonFor(s) }}</div>
</div>
</div>
<div class="fc-recon__dim fc-recon__note">
Nothing has been changed. If you want one of these to stop checking,
disable it in the list below.
</div>
</template>
<!-- The roster could not be trusted, so bucket 2 was not computed. Said
in words: an empty list here must never read as "all clear". -->
<div v-if="!p.fresh" class="fc-recon__dim fc-recon__note"> <div v-if="!p.fresh" class="fc-recon__dim fc-recon__note">
<v-icon icon="mdi-clock-alert-outline" size="small" class="me-1" /> <v-icon icon="mdi-clock-alert-outline" size="small" class="me-1" />
<template v-if="!p.last_success_at"> <template v-if="!p.last_success_at">
Your {{ p.platform }} roster has never synced, so FC can't tell which Your {{ p.platform }} roster has never synced, so FC can't tell which
sources you still subscribe to. creators you subscribe to.
</template> </template>
<template v-else> <template v-else>
Your {{ p.platform }} roster last synced Your {{ p.platform }} roster last synced
@@ -81,44 +65,62 @@
</template> </template>
<script setup> <script setup>
import { computed, onMounted } from 'vue' import { computed, onMounted, ref } from 'vue'
import { formatRelative } from '../../utils/date.js' import { formatRelative } from '../../utils/date.js'
import { useMembershipReconcileStore } from '../../stores/membershipReconcile.js' import { useMembershipReconcileStore } from '../../stores/membershipReconcile.js'
// #387 C4. Two directions, deliberately unequal: subscriptions FC doesn't // #387 C4. Offers the subscriptions FC doesn't follow yet. The other direction
// follow get a one-click add, while sources the roster doesn't account for are // (sources FC follows that the roster says you no longer pay for) used to be
// REPORTED ONLY (operator decision, 2026-09-11). The asymmetry is the point — // listed here as a report. The operator didn't want it listed (2026-09-13); the
// adding a source is trivially undone, and "you no longer subscribe to this" is // membership sweep stops pulling those sources instead (#3995).
// computed from an absence that has three possible causes.
const store = useMembershipReconcileStore() const store = useMembershipReconcileStore()
// An untrustworthy roster is only worth mentioning when there is something it // An untrustworthy roster is only worth mentioning when there is something it
// would have reconciled — otherwise a failed sweep on a platform with no // would have reconciled — otherwise a failed sweep on a platform with no
// sources yet would put a warning on a page with nothing to warn about. // sources yet would put a warning on a page with nothing to warn about.
const interesting = computed(() => store.platforms.filter( const interesting = computed(() => store.platforms.filter(
p => p.subscribed_not_tracked.length p => p.subscribed_not_tracked.length || (!p.fresh && p.tracked_total)
|| p.tracked_not_subscribed.length
|| (!p.fresh && p.tracked_total)
)) ))
const anythingToSay = computed(() => interesting.value.length > 0)
function reasonFor (s) { // Dismissal is keyed to WHAT the card says, not to the card: a fingerprint of
if (s.basis === 'lapsed') { // the offered memberships and any stale-roster warnings. Dismissing hides this
return `Your membership reads "${s.membership?.status}" — you're not a paying supporter of this creator right now.` // exact set; a new subscription (or a roster going stale) changes the
} // fingerprint and brings the card back. Per-browser, which is enough for a
if (s.basis === 'absent_exact') { // single-operator instance.
return "FC knows this creator's id on the platform, and it isn't in your roster." const DISMISS_KEY = 'fc.recon.dismissed'
} const fingerprint = computed(() => interesting.value
// absent_handle — the weakest claim, and it says so. A renamed creator looks .flatMap(p => [
// exactly like this, which is why it is not phrased as a conclusion. ...p.subscribed_not_tracked.map(m => `${p.platform}:${m.id}`),
return "No membership matched this source's address. It may simply have been renamed." ...(p.fresh ? [] : [`${p.platform}:stale`]),
])
.sort()
.join('|'))
function readDismissed () {
try { return localStorage.getItem(DISMISS_KEY) } catch { return null }
} }
const dismissed = ref(readDismissed())
function dismiss () {
dismissed.value = fingerprint.value
try { localStorage.setItem(DISMISS_KEY, fingerprint.value) } catch { /* private mode */ }
}
const visible = computed(
() => interesting.value.length > 0 && fingerprint.value !== dismissed.value,
)
onMounted(() => { store.load() }) onMounted(() => { store.load() })
</script> </script>
<style scoped> <style scoped>
.fc-recon { position: relative; }
.fc-recon__dismiss {
position: absolute;
top: 6px;
right: 6px;
}
.fc-recon__platform + .fc-recon__platform { .fc-recon__platform + .fc-recon__platform {
margin-top: 16px; margin-top: 16px;
} }
@@ -39,8 +39,25 @@
{{ formatRelative(source.next_check_at, { future: true }) }} {{ formatRelative(source.next_check_at, { future: true }) }}
</td> </td>
<td> <td>
<!-- #3995: the membership sweep stopped this source because paid access
ended. First in the chain it explains why an otherwise healthy
source is off. Neutral, never error: nothing is broken. -->
<v-chip <v-chip
v-if="(source.consecutive_failures || 0) > 0" v-if="membershipStopped"
size="x-small" variant="tonal" label
prepend-icon="mdi-account-cancel-outline"
>Membership ended
<v-tooltip activator="parent" location="top" max-width="420">
<span>
Curator stopped checking this source because your membership
{{ membershipStopped.status ? `reads "${membershipStopped.status}"` : 'ended' }}.
It starts again on its own if you resubscribe. Turn it back on to
keep checking anyway.
</span>
</v-tooltip>
</v-chip>
<v-chip
v-else-if="(source.consecutive_failures || 0) > 0"
size="x-small" color="error" variant="tonal" label size="x-small" color="error" variant="tonal" label
>{{ source.consecutive_failures }} >{{ source.consecutive_failures }}
<!-- #1: show the actual failure reason on hover instead of a bare count. --> <!-- #1: show the actual failure reason on hover instead of a bare count. -->
@@ -109,6 +126,12 @@ const props = defineProps({
// operator's subscription, not anything FC can retry. The count is only joined // operator's subscription, not anything FC can retry. The count is only joined
// in by the list endpoint, so phrase it without one when it's absent rather // in by the list endpoint, so phrase it without one when it's absent rather
// than rendering a fabricated zero. // than rendering a fabricated zero.
// #3995: set by the membership sweep when it stops a source whose paid access
// ended; cleared when it resumes, or when the operator toggles the source.
const membershipStopped = computed(
() => props.source.config_overrides?._membership_stopped || null,
)
const noAccessTip = computed(() => { const noAccessTip = computed(() => {
const n = props.source.tier_gated_count const n = props.source.tier_gated_count
const what = n const what = n
+7 -6
View File
@@ -1,9 +1,13 @@
// Single source of truth for platform → color + icon mapping. Used by // Single source of truth for platform → color + icon mapping. Used by
// PlatformChip and any other GS-style platform-tagged surface. The five // PlatformChip and any other GS-style platform-tagged surface. The four
// platforms FC supports map 1:1 to the GS palette; unknown platforms fall // platforms FC supports map 1:1 to the GS palette; unknown platforms fall
// back to grey + mdi-web — which is deliberately what a retired platform // back to grey + mdi-web — which is deliberately what a retired platform
// hits: a pre-#3069 deviantart source row still renders, as its raw key on // hits: a pre-#3069 deviantart source row, or a post from pixiv (retired at
// a grey chip. Operator-confirmed scope 2026-05-27. The ICONS key set is // milestone #406), still renders, as its raw key on a grey chip. So a
// retired platform is REMOVED from these maps, never kept "so old rows
// look right" — the fallback is what makes old rows look right, and keeping
// the entry would break the contract pin below. Operator-confirmed scope
// 2026-05-27. The ICONS key set is
// pinned against backend known_platform_keys() by // pinned against backend known_platform_keys() by
// tests/test_fe_be_contract.py. // tests/test_fe_be_contract.py.
@@ -12,7 +16,6 @@ const ICONS = {
subscribestar: 'mdi-star', subscribestar: 'mdi-star',
hentaifoundry: 'mdi-palette', hentaifoundry: 'mdi-palette',
discord: 'mdi-discord', discord: 'mdi-discord',
pixiv: 'mdi-alpha-p-box',
} }
const COLORS = { const COLORS = {
@@ -20,7 +23,6 @@ const COLORS = {
subscribestar: 'amber', subscribestar: 'amber',
hentaifoundry: 'purple', hentaifoundry: 'purple',
discord: 'indigo', discord: 'indigo',
pixiv: 'blue',
} }
const LABELS = { const LABELS = {
@@ -28,7 +30,6 @@ const LABELS = {
subscribestar: 'SubscribeStar', subscribestar: 'SubscribeStar',
hentaifoundry: 'HentaiFoundry', hentaifoundry: 'HentaiFoundry',
discord: 'Discord', discord: 'Discord',
pixiv: 'Pixiv',
} }
export function platformIcon(platform) { export function platformIcon(platform) {
+3 -2
View File
@@ -58,8 +58,9 @@ useInfiniteScroll(sentinelEl, () => {
onMounted(async () => { onMounted(async () => {
await platformsStore.loadAll() await platformsStore.loadAll()
platformItems.value = (platformsStore.platforms || []).map(p => ({ // `list` — the store has no `platforms` property (test/storeUsage.spec.js).
title: p.key, value: p.key, platformItems.value = platformsStore.list.map(p => ({
title: p.name || p.key, value: p.key,
})) }))
// Apply any deep-linked q (and load). setQuery resets + fetches. // Apply any deep-linked q (and load). setQuery resets + fetches.
store.setQuery(searchTerm.value || '') store.setQuery(searchTerm.value || '')
+124 -6
View File
@@ -1,5 +1,8 @@
<template> <template>
<v-container class="pt-2 pb-6" max-width="900"> <!-- Width is set in CSS, not with `max-width` here: below 1600px it is
today's 900px column, and above it the feed widens and gains the day
gutter (milestone #407). -->
<v-container fluid class="pt-2 pb-6 fc-posts">
<!-- In-context view: deep-linked to one post, with bidirectional infinite <!-- In-context view: deep-linked to one post, with bidirectional infinite
scroll — newer posts load above, older posts below. --> scroll — newer posts load above, older posts below. -->
<template v-if="postIdFilter != null"> <template v-if="postIdFilter != null">
@@ -48,14 +51,19 @@
</template> </template>
<!-- Normal feed --> <!-- Normal feed -->
<template v-else> <div v-else class="fc-posts__main">
<FeedStatusRibbon v-if="statusRibbon" /> <!-- Filters and status sit in one row above the feed. #407 option E put
them in a sticky left rail; the operator found it wasted space
(2026-09-13), so on a wide window this row lines up with the feed
column instead, clear of the day gutter. -->
<div class="fc-posts__toolbar">
<FeedStatusRibbon v-if="statusRibbon" class="fc-posts__status" />
<PostsFilterBar <PostsFilterBar
:artist-id="artistFilter" :artist-id="artistFilter"
:platform="platformFilter" :platform="platformFilter"
@update:filters="onFilters" @update:filters="onFilters"
/> />
</div>
<v-alert v-if="store.error" type="error" variant="tonal" closable class="mb-3"> <v-alert v-if="store.error" type="error" variant="tonal" closable class="mb-3">
{{ String(store.error) }} {{ String(store.error) }}
@@ -74,14 +82,27 @@
</div> </div>
<div v-else> <div v-else>
<PostCard v-for="p in store.items" :key="p.id" :post="p" /> <!-- Day groups (#407 D). The heading sits above its posts on a narrow
window and in a sticky left gutter on a wide one. -->
<section v-for="d in days" :key="d.key" class="fc-posts__day">
<header class="fc-posts__day-head">
<span class="fc-posts__day-label">{{ d.label }}</span>
<span class="fc-posts__day-count">
{{ d.posts.length }} post{{ d.posts.length === 1 ? '' : 's' }}
· {{ d.artistCount }} artist{{ d.artistCount === 1 ? '' : 's' }}
</span>
</header>
<div class="fc-posts__day-posts">
<PostCard v-for="p in d.posts" :key="p.id" :post="p" />
</div>
</section>
<div ref="sentinel" class="fc-posts__sentinel"> <div ref="sentinel" class="fc-posts__sentinel">
<v-progress-circular v-if="store.loading" indeterminate color="accent" size="24" /> <v-progress-circular v-if="store.loading" indeterminate color="accent" size="24" />
<span v-else-if="store.done" class="fc-posts__end">End of stream</span> <span v-else-if="store.done" class="fc-posts__end">End of stream</span>
</div> </div>
</div> </div>
</template> </div>
</v-container> </v-container>
</template> </template>
@@ -121,6 +142,48 @@ const hasActiveFilter = computed(() =>
artistFilter.value != null || platformFilter.value != null || searchFilter.value != null artistFilter.value != null || platformFilter.value != null || searchFilter.value != null
) )
// --- day groups (#407 D) ---
// CONSECUTIVE runs, not a bucket per date. The feed's sort key includes
// `resurfaced_at` (a Discord grouping that grew moves back to the top), which
// the payload does not carry, so a resurfaced post can sit above newer ones.
// Bucketing by date would pull it out of order; a run gives it its own heading
// where it actually appears. Counts cover what has LOADED, and grow as the
// infinite scroll fetches more of the same day.
function dayKey (iso) {
const d = new Date(iso)
return `${d.getFullYear()}-${d.getMonth()}-${d.getDate()}`
}
function dayLabel (iso) {
const d = new Date(iso)
const today = new Date()
const startOf = (x) => new Date(x.getFullYear(), x.getMonth(), x.getDate()).getTime()
const days = Math.round((startOf(today) - startOf(d)) / 86400000)
if (days === 0) return 'Today'
if (days === 1) return 'Yesterday'
if (days > 1 && days < 7) return d.toLocaleDateString(undefined, { weekday: 'long' })
const sameYear = d.getFullYear() === today.getFullYear()
return d.toLocaleDateString(undefined, {
month: 'short', day: 'numeric', ...(sameYear ? {} : { year: 'numeric' }),
})
}
const days = computed(() => {
const groups = []
for (const p of store.items) {
const iso = p.post_date || p.downloaded_at
const key = dayKey(iso)
let g = groups[groups.length - 1]
if (!g || g.dayKey !== key) {
// Suffix with the run index so a day that appears twice (see above)
// still has a unique v-for key.
g = { key: `${key}#${groups.length}`, dayKey: key, label: dayLabel(iso), posts: [], artists: new Set() }
groups.push(g)
}
g.posts.push(p)
if (p.artist?.id != null) g.artists.add(p.artist.id)
}
return groups.map((g) => ({ ...g, artistCount: g.artists.size }))
})
// Drop only `post_id` and stay where we are — keeps Browse's `tab=posts` (and // Drop only `post_id` and stay where we are — keeps Browse's `tab=posts` (and
// any active artist/platform scope) intact instead of resetting the surface. // any active artist/platform scope) intact instead of resetting the surface.
const allPostsTarget = computed(() => { const allPostsTarget = computed(() => {
@@ -232,6 +295,61 @@ onUnmounted(() => { teardownFeed(); teardownAround() })
</script> </script>
<style scoped> <style scoped>
/* Below the breakpoint: today's layout exactly — a 900px column with the
filters and ribbon inline above the feed. */
.fc-posts { max-width: 900px; }
.fc-posts__day-head {
display: flex;
align-items: baseline;
gap: 10px;
padding: 4px 0 8px;
color: rgb(var(--v-theme-on-surface-variant));
}
.fc-posts__day-label {
font-family: 'Fraunces', Georgia, serif;
font-weight: 600;
font-size: 1rem;
color: rgb(var(--v-theme-accent));
}
.fc-posts__day-count { font-size: 0.78rem; }
/* Wide window (#407 A + D). Each day's heading moves into a sticky gutter
beside its posts; the column widens and the cards switch to their filmstrip
layout on their own (PostCard measures itself). The toolbar lines up with the
feed column, clear of the gutter. */
@media (min-width: 1600px) {
.fc-posts { max-width: 2080px; }
.fc-posts__main { max-width: 2080px; margin: 0 auto; }
.fc-posts__toolbar {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 8px 24px;
margin-left: calc(150px + 24px);
margin-bottom: 8px;
}
/* Status reads after the filters on a wide row, but stays first when the
narrow layout stacks them (it is the front door's headline). */
.fc-posts__status { order: 2; padding: 0; }
.fc-posts__toolbar :deep(.fc-posts-filters) { padding-bottom: 0; }
.fc-posts__day {
display: grid;
grid-template-columns: 150px minmax(0, 1fr);
gap: 0 24px;
}
.fc-posts__day-head {
position: sticky;
top: calc(var(--fc-nav-h, 64px) + 16px);
align-self: start;
flex-direction: column;
align-items: flex-end;
gap: 2px;
padding-top: 12px;
text-align: right;
}
.fc-posts__day-label { font-size: 1.1rem; }
}
.fc-posts__loading, .fc-posts__loading,
.fc-posts__empty { .fc-posts__empty {
display: flex; display: flex;
+63
View File
@@ -0,0 +1,63 @@
// Reads of a store property that does not exist.
//
// JavaScript returns `undefined` for a missing property instead of failing, and
// the frontend CI has no type-checker to catch it. `platformsStore.platforms`
// was read for months by the Latest feed's filter bar and the Artists view. The
// store exposes `list` and `byKey`, so both platform dropdowns opened empty and
// only someone clicking them noticed.
//
// This scans the source for `platformsStore.<name>` and checks each name
// against what the store really defines.
import { readdirSync, readFileSync, statSync } from 'node:fs'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { createPinia, setActivePinia } from 'pinia'
import { describe, expect, it } from 'vitest'
import { usePlatformsStore } from '../src/stores/platforms.js'
const SRC = fileURLToPath(new URL('../src', import.meta.url))
const READ = /\bplatformsStore\.([A-Za-z_$][\w$]*)/g
function sourceFiles (dir) {
return readdirSync(dir).flatMap((name) => {
const path = join(dir, name)
if (statSync(path).isDirectory()) return sourceFiles(path)
return /\.(vue|js)$/.test(name) ? [path] : []
})
}
function unknownReads (text, known) {
return [...text.matchAll(READ)].map(m => m[1]).filter(name => !known.has(name))
}
function storeKeys () {
setActivePinia(createPinia())
return new Set(Object.keys(usePlatformsStore()))
}
describe('platforms store usage', () => {
it('every platformsStore.<name> read in src names something the store defines', () => {
const known = storeKeys()
const offenders = sourceFiles(SRC).flatMap((file) =>
unknownReads(readFileSync(file, 'utf8'), known)
.map(name => `${file.slice(SRC.length + 1)}: platformsStore.${name}`),
)
expect(offenders).toEqual([])
})
it('the scan finds the read that shipped broken (positive control)', () => {
const known = storeKeys()
expect(known.has('list')).toBe(true)
expect(unknownReads('(platformsStore.platforms || []).map(p => p)', known))
.toEqual(['platforms'])
})
it('the scan actually walks the source tree', () => {
const hasRead = /\bplatformsStore\./
const withReads = sourceFiles(SRC).filter(f => hasRead.test(readFileSync(f, 'utf8')))
expect(withReads.length).toBeGreaterThan(2)
})
})
+28
View File
@@ -0,0 +1,28 @@
{
"author": "example-artist",
"author_files": [],
"author_id": "100000000000000001",
"category": "discord",
"channel": "nsfw-drops",
"channel_id": "200000000000000002",
"channel_topic": "",
"channel_type": 0,
"date": "2024-07-16 16:43:23",
"extension": "png",
"filename": "image",
"files": [],
"is_thread": false,
"message": "",
"message_id": "300000000000000003",
"num": 1,
"owner_id": "100000000000000001",
"parent": "",
"parent_id": "",
"parent_type": 0,
"server": "Example Server",
"server_files": [],
"server_id": "400000000000000004",
"subcategory": "channel",
"type": "attachment",
"url": "https://cdn.discordapp.com/attachments/200000000000000002/500000000000000005/image.png"
}
+13
View File
@@ -72,3 +72,16 @@ async def test_patch_rename_validation(client):
f"/api/artists/{created['id']}", json={})).status_code == 400 f"/api/artists/{created['id']}", json={})).status_code == 400
assert (await client.patch( assert (await client.patch(
"/api/artists/999999", json={"name": "Ghost"})).status_code == 404 "/api/artists/999999", json={"name": "Ghost"})).status_code == 404
@pytest.mark.asyncio
async def test_names_lists_every_artist_alphabetically(client):
"""The Latest feed's artist filter shows this list before anything is typed.
`autocomplete` stays empty for an empty query; this is the full list."""
for name in ("zed", "Alice", "bob"):
await client.post("/api/artists", json={"name": name})
resp = await client.get("/api/artists/names")
assert resp.status_code == 200
body = await resp.get_json()
assert [a["name"] for a in body] == ["Alice", "bob", "zed"]
assert set(body[0]) == {"id", "name", "slug"}
+170
View File
@@ -0,0 +1,170 @@
"""The Discord repair (#3999): forget the broken downloads, delete them, backfill again.
The danger in a repair like this is reach. It deletes files and clears
gallery-dl's memory of what it fetched, so most of these tests pin what it must
NOT touch: other platforms' archive entries, Discord server assets, images
outside a `discord/None/` folder, and a correctly named Discord file that
happens to share a folder name.
"""
import sqlite3
import pytest
from sqlalchemy import func, select
from backend.app.models import Artist, ImageRecord, Source
from backend.app.services import discord_repair
from backend.app.services.gallery_dl import archive_path
from backend.app.services.source_service import BACKFILL_MAX_CHUNKS
def _archive(tmp_path, entries):
path = archive_path(tmp_path)
path.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(path)
# The table gallery-dl itself creates (gallery_dl/archive.py).
conn.execute("CREATE TABLE archive (entry TEXT PRIMARY KEY) WITHOUT ROWID")
conn.executemany("INSERT INTO archive (entry) VALUES (?)", [(e,) for e in entries])
conn.commit()
conn.close()
return path
def _entries(path):
conn = sqlite3.connect(path)
try:
return sorted(r[0] for r in conn.execute("SELECT entry FROM archive"))
finally:
conn.close()
# --- the archive ------------------------------------------------------------
def test_only_discord_message_entries_are_forgotten(tmp_path):
path = _archive(tmp_path, [
"discord300000000000000003_1",
"discord300000000000000004_2",
"discordasset_400000000000000004_9",
"hentaifoundry12345",
])
assert discord_repair.count_archive_entries(path) == 2
assert discord_repair.forget_archive_entries(path) == 2
assert _entries(path) == ["discordasset_400000000000000004_9", "hentaifoundry12345"]
def test_counting_changes_nothing(tmp_path):
path = _archive(tmp_path, ["discord1_1", "discord2_1"])
discord_repair.count_archive_entries(path)
assert _entries(path) == ["discord1_1", "discord2_1"]
def test_a_missing_archive_or_table_is_zero_not_an_error(tmp_path):
assert discord_repair.count_archive_entries(tmp_path / "nope.sqlite3") == 0
empty = tmp_path / "empty.sqlite3"
sqlite3.connect(empty).close()
assert discord_repair.forget_archive_entries(empty) == 0
# --- the folders ------------------------------------------------------------
def test_broken_directories_finds_every_artist_folder_including_case_twins(tmp_path):
for artist in ("Conto", "conto", "knuxy"):
(tmp_path / artist / "discord" / "None").mkdir(parents=True)
(tmp_path / "conto" / "discord" / "general").mkdir(parents=True)
(tmp_path / "conto" / "patreon" / "None").mkdir(parents=True)
found = [
p.relative_to(tmp_path).as_posix()
for p in discord_repair.broken_directories(tmp_path)
]
assert found == ["Conto/discord/None", "conto/discord/None", "knuxy/discord/None"]
def test_sweep_removes_leftover_sidecars_and_the_empty_folder(tmp_path):
d = tmp_path / "conto" / "discord" / "None"
d.mkdir(parents=True)
(d / "image.json").write_text("{}")
(d / "20240716_None_rejected.png").write_bytes(b"x")
removed, gone = discord_repair._sweep_directory(d)
assert (removed, gone) == (2, True)
assert not d.exists()
# --- the whole repair, against the database -----------------------------------
def _image(db_sync, artist, path, sha):
img = ImageRecord(
artist_id=artist.id, path=str(path), sha256=sha * 64, size_bytes=100,
mime="image/png", origin="downloaded",
)
db_sync.add(img)
return img
@pytest.mark.integration
def test_repair_deletes_only_broken_discord_images_and_rearms_discord_backfills(
db_sync, tmp_path,
):
artist = Artist(name="Conto", slug="conto")
db_sync.add(artist)
db_sync.flush()
discord = Source(
artist_id=artist.id, platform="discord", url="https://discord.com/channels/1/2",
enabled=True, config_overrides={"_backfill_state": "complete", "_backfill_cursor": "x"},
)
patreon = Source(
artist_id=artist.id, platform="patreon", url="https://www.patreon.com/conto",
enabled=True, config_overrides={},
)
db_sync.add_all([discord, patreon])
broken_dir = tmp_path / "conto" / "discord" / "None"
good_dir = tmp_path / "conto" / "discord" / "general"
patreon_dir = tmp_path / "conto" / "patreon" / "None"
for d in (broken_dir, good_dir, patreon_dir):
d.mkdir(parents=True)
broken = broken_dir / "20240716_None_image.png"
good = good_dir / "20240716_300000000000000003_01_image.png"
# A `None` folder under another platform, and a `_None_` name outside one.
# Neither is the bug's pair, so neither may be touched.
elsewhere = patreon_dir / "20240716_None_image.png"
for f in (broken, good, elsewhere):
f.write_bytes(b"x")
(broken_dir / "image.json").write_text("{}")
_image(db_sync, artist, broken, "a")
_image(db_sync, artist, good, "b")
_image(db_sync, artist, elsewhere, "c")
db_sync.commit()
_archive(tmp_path, ["discord300000000000000003_1", "hentaifoundry1"])
preview = discord_repair.repair_discord_downloads(db_sync, images_root=tmp_path, dry_run=True)
assert preview["images"] == 1
assert preview["archive_entries"] == 1
assert broken.exists()
result = discord_repair.repair_discord_downloads(db_sync, images_root=tmp_path, dry_run=False)
assert result["images_deleted"] == 1
assert result["remaining"] == 0
assert result["backfills_started"] == 1
assert not broken.exists() and not broken_dir.exists()
assert good.exists() and elsewhere.exists()
assert _entries(archive_path(tmp_path)) == ["hentaifoundry1"]
kept = db_sync.execute(select(func.count(ImageRecord.id))).scalar_one()
assert kept == 2
rows = {
platform: (runs, overrides)
for platform, runs, overrides in db_sync.execute(
select(Source.platform, Source.backfill_runs_remaining, Source.config_overrides)
).all()
}
runs, overrides = rows["discord"]
assert runs == BACKFILL_MAX_CHUNKS
assert overrides["_backfill_state"] == "running"
assert "_backfill_cursor" not in overrides
# Only Discord sources are re-armed.
assert rows["patreon"][1] == {}
+117
View File
@@ -0,0 +1,117 @@
"""gallery-dl naming patterns, rendered against the metadata gallery-dl really emits.
gallery-dl does not fail on a format field that names a key the extractor never
sets. It renders "None" and carries on. That is how every Discord download
landed in a `None/` directory as `<date>_None_<name>`: the patterns asked for
`{channel[name]}` and `{id}`, and the real metadata has a string `channel` and
`message_id`, with no `id` at all. The mismatch also broke sidecar pairing, so
no Discord file ever got a Post or a post date.
The fixture keeps the key set and value TYPES of a real attachment sidecar from
the operator's instance (2026-09-13), with every value invented. Rendering
through Python's own formatter raises on a missing key or a subscript into a
string, which is the loud failure gallery-dl does not give.
"""
import json
import string
from datetime import datetime
from pathlib import Path
import pytest
from backend.app.services.gallery_dl import (
DISCORD_DIRECTORY,
DISCORD_FILENAME,
GalleryDLService,
SourceConfig,
sidecar_name_for,
)
from backend.app.utils.sidecar import find_sidecar
_FIXTURE = Path(__file__).parent / "fixtures" / "discord_attachment_sidecar.json"
@pytest.fixture
def kwdict():
data = json.loads(_FIXTURE.read_text())
# gallery-dl hands the formatter a datetime; the JSON sidecar stores it as text.
data["date"] = datetime.strptime(data["date"], "%Y-%m-%d %H:%M:%S")
return data
def _render(pattern, kwdict):
return string.Formatter().vformat(pattern, (), kwdict)
def test_the_discord_filename_renders_from_real_keys(kwdict):
name = _render(DISCORD_FILENAME, kwdict)
assert "None" not in name
assert name == "20240716_300000000000000003_01_image.png"
def test_the_discord_directory_renders_from_real_keys(kwdict):
assert [_render(p, kwdict) for p in DISCORD_DIRECTORY] == ["nsfw-drops"]
@pytest.mark.parametrize(
"broken", ["{channel[name]}", "{date:%Y%m%d}_{id}_{filename}.{extension}"],
)
def test_the_patterns_that_shipped_would_have_failed_here(kwdict, broken):
"""Positive control: the two patterns that produced `None/..._None_...`
must fail this renderer, or the tests above prove nothing."""
with pytest.raises((KeyError, TypeError)):
_render(broken, kwdict)
def test_the_sidecar_is_named_exactly_like_its_media(kwdict, tmp_path):
"""So `find_sidecar` pairs them on its first candidate, and two attachments
that share an original name (`image.png`) can never share a sidecar."""
media = tmp_path / _render(DISCORD_FILENAME, kwdict)
sidecar = tmp_path / _render(sidecar_name_for(DISCORD_FILENAME), kwdict)
media.write_bytes(b"x")
sidecar.write_text("{}")
assert find_sidecar(media) == sidecar
def test_two_attachments_with_the_same_original_name_get_distinct_sidecars(kwdict):
other = {**kwdict, "message_id": "300000000000000099"}
pattern = sidecar_name_for(DISCORD_FILENAME)
assert _render(pattern, kwdict) != _render(pattern, other)
def test_sidecar_name_for_needs_an_extension_suffix():
assert sidecar_name_for("{a}_{b}.{extension}") == "{a}_{b}.json"
assert sidecar_name_for("{a}_{b}") is None
# --- the config gallery-dl is actually given -----------------------------------
@pytest.fixture
def gdl(tmp_path):
return GalleryDLService(images_root=tmp_path / "images", validate_files=False)
def _discord_section(gdl, **overrides):
cfg = gdl._build_config_for_source(
platform="discord", source_config=SourceConfig(**overrides), artist_slug="a",
)
return cfg["extractor"]["discord"]
def test_discord_config_carries_the_mirrored_sidecar(gdl):
section = _discord_section(gdl)
assert section["filename"] == DISCORD_FILENAME
assert section["directory"] == DISCORD_DIRECTORY
assert section["postprocessors"][0]["filename"] == sidecar_name_for(DISCORD_FILENAME)
def test_a_filename_override_keeps_the_sidecar_mirrored(gdl):
section = _discord_section(gdl, filename_pattern="{message_id}_{num}.{extension}")
assert section["postprocessors"][0]["filename"] == "{message_id}_{num}.json"
def test_no_metadata_means_no_discord_postprocessor(gdl):
section = _discord_section(gdl, save_metadata=False)
assert "postprocessors" not in section
+190
View File
@@ -0,0 +1,190 @@
"""#3995: stop pulling a source once the account stops paying, resume on resubscribe.
Operator, 2026-09-13: "if I kill a subscription on patreon I would like the
pulling to stop on curator as well", with automatic resume when they resubscribe.
Stopping a source is easy. Not stopping the wrong one is the work, so most of
these pin refusals: an absent membership is never a lapse, an unknown status is
never a lapse, a stale roster decides nothing, a paid-through period is honoured,
and the operator's own on/off choice always outranks the sweep in both directions.
"""
from datetime import UTC, datetime, timedelta
import pytest
from sqlalchemy import select
from backend.app.models import Artist, Source
from backend.app.services.membership_reconcile import (
KEPT_KEY,
STOPPED_KEY,
apply_membership_lapses,
)
from backend.app.services.membership_roster import ROSTER_STALE_AFTER
from backend.app.services.source_service import SourceService
from tests.roster_builders import membership as _membership
from tests.roster_builders import synced as _synced
pytestmark = pytest.mark.integration
async def _source(
db, *, overrides=None, enabled=True, url="https://www.patreon.com/an-old-handle",
):
artist = Artist(name="Maewix", slug="maewix")
db.add(artist)
await db.flush()
s = Source(
artist_id=artist.id, platform="patreon", url=url, enabled=enabled,
config_overrides={"patreon_campaign_id": "c1", **(overrides or {})},
last_error="boom", error_type="tier_limited", consecutive_failures=3,
)
db.add(s)
await db.flush()
return s.id
async def _state(db, source_id):
row = (await db.execute(
select(Source.enabled, Source.config_overrides, Source.error_type,
Source.consecutive_failures).where(Source.id == source_id)
)).one()
return row
@pytest.mark.asyncio
async def test_a_lapsed_membership_stops_its_source_with_a_clean_slate(db):
sid = await _source(db)
await _membership(db, campaign="c1", status="former_patron")
await _synced(db)
await db.commit()
out = await apply_membership_lapses(db, platform="patreon")
assert out["stopped"] == 1
enabled, co, error_type, failures = await _state(db, sid)
assert enabled is False
assert co[STOPPED_KEY]["status"] == "former_patron"
assert co["patreon_campaign_id"] == "c1" # the identity cache survives
assert (error_type, failures) == (None, 0)
@pytest.mark.asyncio
async def test_the_paid_through_period_is_honoured(db):
"""Patreon keeps access until the billing period ends, and says when."""
sid = await _source(db)
later = (datetime.now(UTC) + timedelta(days=10)).isoformat()
await _membership(
db, campaign="c1", status="former_patron",
details={"member": {"access_expires_at": later}},
)
await _synced(db)
await db.commit()
assert (await apply_membership_lapses(db, platform="patreon"))["stopped"] == 0
assert (await _state(db, sid)).enabled is True
@pytest.mark.asyncio
async def test_an_absent_membership_is_never_a_lapse(db):
"""No match has innocent causes (a rename, a never-walked source), so
absence alone must never switch a source off."""
sid = await _source(db, overrides={"patreon_campaign_id": "nobody-has-this"})
await _membership(db, campaign="c1", status="former_patron", details={})
await _synced(db)
await db.commit()
assert (await apply_membership_lapses(db, platform="patreon"))["stopped"] == 0
assert (await _state(db, sid)).enabled is True
@pytest.mark.asyncio
async def test_an_unrecognised_status_is_never_a_lapse(db):
sid = await _source(db)
await _membership(db, campaign="c1", status="some_word_nobody_characterised")
await _synced(db)
await db.commit()
assert (await apply_membership_lapses(db, platform="patreon"))["stopped"] == 0
assert (await _state(db, sid)).enabled is True
@pytest.mark.asyncio
async def test_a_stale_roster_decides_nothing(db):
sid = await _source(db)
await _membership(db, campaign="c1", status="former_patron")
await _synced(db, ago=ROSTER_STALE_AFTER + timedelta(hours=1))
await db.commit()
out = await apply_membership_lapses(db, platform="patreon")
assert out["skipped"] == "roster not fresh"
assert (await _state(db, sid)).enabled is True
@pytest.mark.asyncio
async def test_resubscribing_resumes_only_what_the_sweep_stopped(db):
stopped = await _source(
db, enabled=False, overrides={STOPPED_KEY: {"status": "former_patron"}},
)
await _membership(db, campaign="c1", status="active_patron")
await _synced(db)
await db.commit()
assert (await apply_membership_lapses(db, platform="patreon"))["resumed"] == 1
enabled, co, _e, _f = await _state(db, stopped)
assert enabled is True
assert STOPPED_KEY not in co
@pytest.mark.asyncio
async def test_a_source_the_operator_switched_off_is_never_switched_back_on(db):
sid = await _source(db, enabled=False)
await _membership(db, campaign="c1", status="active_patron")
await _synced(db)
await db.commit()
assert (await apply_membership_lapses(db, platform="patreon"))["resumed"] == 0
assert (await _state(db, sid)).enabled is False
@pytest.mark.asyncio
async def test_turning_a_stopped_source_back_on_keeps_it_on(db):
"""Re-enabling a stopped source by hand is a choice to keep pulling a lapsed
creator. The next sweep must not undo it."""
sid = await _source(
db, enabled=False, overrides={STOPPED_KEY: {"status": "former_patron"}},
)
await _membership(db, campaign="c1", status="former_patron")
await _synced(db)
await db.commit()
await SourceService(db).update(sid, enabled=True)
enabled, co, _e, _f = await _state(db, sid)
assert enabled is True
assert co.get(KEPT_KEY) is True and STOPPED_KEY not in co
assert (await apply_membership_lapses(db, platform="patreon"))["stopped"] == 0
assert (await _state(db, sid)).enabled is True
@pytest.mark.asyncio
async def test_switching_a_stopped_source_off_by_hand_drops_the_resume_marker(db):
sid = await _source(
db, enabled=False, overrides={STOPPED_KEY: {"status": "former_patron"}},
)
await db.commit()
await SourceService(db).update(sid, enabled=False)
_enabled, co, _e, _f = await _state(db, sid)
assert STOPPED_KEY not in co
@pytest.mark.asyncio
async def test_a_kept_source_is_released_once_paid_again(db):
sid = await _source(db, overrides={KEPT_KEY: True})
await _membership(db, campaign="c1", status="active_patron")
await _synced(db)
await db.commit()
await apply_membership_lapses(db, platform="patreon")
_enabled, co, _e, _f = await _state(db, sid)
assert KEPT_KEY not in co
+2 -5
View File
@@ -10,11 +10,8 @@ import pytest
from sqlalchemy import select from sqlalchemy import select
from backend.app.models import PlatformMembership from backend.app.models import PlatformMembership
from backend.app.services.membership_roster import ( from backend.app.services.membership_roster import touch_membership
MEMBERSHIP_STATUS, from backend.app.services.native_ingest_common import MEMBERSHIP_STATUS, has_paid_access
has_paid_access,
touch_membership,
)
pytestmark = pytest.mark.integration pytestmark = pytest.mark.integration
+60 -2
View File
@@ -239,14 +239,72 @@ def test_a_missing_data_list_is_drift(client, payload):
list(client.iter_memberships(user_id="1")) list(client.iter_memberships(user_id="1"))
def test_a_member_with_no_campaign_is_drift(client, payload): def _index_of(payload, status):
return next(
i for i, m in enumerate(payload["data"])
if m["attributes"]["patron_status"] == status
)
@pytest.mark.parametrize("shape", ["null", "absent"])
def test_an_active_member_with_no_campaign_is_drift(client, payload, shape):
"""A PAID membership FC cannot attribute must refuse the roster. Dropping it
would read downstream as the operator having cancelled it."""
mangled = json.loads(json.dumps(payload)) mangled = json.loads(json.dumps(payload))
mangled["data"][0]["relationships"]["campaign"] = {"data": None} rels = mangled["data"][_index_of(mangled, "active_patron")]["relationships"]
if shape == "null":
rels["campaign"] = {"data": None}
else:
del rels["campaign"]
client._request = lambda *a, **k: mangled client._request = lambda *a, **k: mangled
with pytest.raises(PatreonDriftError, match="campaign"): with pytest.raises(PatreonDriftError, match="campaign"):
list(client.iter_memberships(user_id="1")) list(client.iter_memberships(user_id="1"))
def test_an_unrecognised_status_with_no_campaign_is_drift(client, payload):
"""Only a status KNOWN to mean "not paying" may be skipped. An unknown word
could be a paid membership (has_paid_access returns None for it)."""
mangled = json.loads(json.dumps(payload))
row = mangled["data"][_index_of(mangled, "active_patron")]
row["attributes"]["patron_status"] = "some_new_status"
del row["relationships"]["campaign"]
client._request = lambda *a, **k: mangled
with pytest.raises(PatreonDriftError, match="campaign"):
list(client.iter_memberships(user_id="1"))
def test_a_lapsed_member_whose_creator_is_gone_is_skipped(client, payload):
"""The live roster's shape (note #3886, CORRECTION 3). A membership that
lapsed in 2017 came back with no `campaign` key at all, because the creator's
page no longer exists. It used to fail the whole roster. Now it is the only
row missing, and every other row still arrives."""
mangled = json.loads(json.dumps(payload))
lapsed = _index_of(mangled, "former_patron")
del mangled["data"][lapsed]["relationships"]["campaign"]
client._request = lambda *a, **k: mangled
rows = list(client.iter_memberships(user_id="1"))
assert len(rows) == len(payload["data"]) - 1
assert all(r.campaign_id for r in rows)
assert all(r.status == "active_patron" for r in rows)
def test_skipping_does_not_end_paging_early(client, payload):
"""Paging counts the rows the SERVER sent, not the ones kept. Counting kept
rows would re-request an offset that was already read, or stop one page
short, whenever a row is skipped."""
first = json.loads(json.dumps(payload))
del first["data"][_index_of(first, "former_patron")]["relationships"]["campaign"]
total = 2 * len(payload["data"])
first["meta"]["pagination"]["total"] = total
second = json.loads(json.dumps(payload))
second["meta"]["pagination"]["total"] = total
calls = _serve(client, first, second)
rows = list(client.iter_memberships(user_id="1"))
assert len(calls) == 2
assert calls[1][1]["page[offset]"] == str(len(payload["data"]))
assert len(rows) == total - 1
def test_a_member_with_no_patron_status_is_drift(client, payload): def test_a_member_with_no_patron_status_is_drift(client, payload):
mangled = json.loads(json.dumps(payload)) mangled = json.loads(json.dumps(payload))
del mangled["data"][0]["attributes"]["patron_status"] del mangled["data"][0]["attributes"]["patron_status"]
+40
View File
@@ -0,0 +1,40 @@
"""Every SVG the browser loads from `frontend/public/` must parse as XML.
A browser renders an SVG used as an image only if it is well-formed XML, and
when it is not, nothing reports it: no console error in most browsers, no
failed build, no failed request — the icon is simply blank. `favicon.svg`
shipped that way (merge #251) because its comment contained a CSS custom
property name, and `--` is illegal inside an XML comment. Both the tab icon and
the nav brand mark went missing, and only a person looking at the page noticed.
"""
from __future__ import annotations
import xml.etree.ElementTree as ET
from pathlib import Path
import pytest
PUBLIC = Path(__file__).resolve().parent.parent / "frontend" / "public"
SVGS = sorted(PUBLIC.rglob("*.svg"))
def test_the_public_dir_has_svgs_to_check():
"""Guards the guard: a moved directory would otherwise pass vacuously."""
assert {p.name for p in SVGS} >= {"favicon.svg", "logo.svg"}
@pytest.mark.parametrize("svg", SVGS, ids=lambda p: p.name)
def test_svg_is_well_formed_xml(svg):
root = ET.parse(svg).getroot()
assert root.tag == "{http://www.w3.org/2000/svg}svg"
def test_the_parser_rejects_the_shape_that_broke_the_favicon():
"""Positive control: the exact defect must fail this parser, or the
parametrized test above proves nothing."""
broken = (
'<svg xmlns="http://www.w3.org/2000/svg">'
"<!-- matches --fc-chrome-rgb exactly --></svg>"
)
with pytest.raises(ET.ParseError):
ET.fromstring(broken)
+6 -3
View File
@@ -388,14 +388,17 @@ async def test_update_while_enabled_keeps_failure_state(db):
async def _source_with_content(db, svc, artist): async def _source_with_content(db, svc, artist):
"""A source under `artist` with one post + one image it contributed.""" """A source under `artist` with one post + one image it contributed."""
from backend.app.models import ImageProvenance, ImageRecord, Post from backend.app.models import ImageProvenance, ImageRecord, Post
# Any registered platform will do — reassign never reads the platform, and
# never moves files (the storage path is immutable). This used pixiv until
# pixiv was retired (milestone #406) and `create` began refusing it.
rec = await svc.create( rec = await svc.create(
artist_id=artist.id, platform="pixiv", artist_id=artist.id, platform="hentaifoundry",
url=f"https://www.pixiv.net/users/{artist.id}", url=f"https://www.hentai-foundry.com/user/{artist.slug}/profile",
) )
post = Post(source_id=rec.id, artist_id=artist.id, external_post_id="p1") post = Post(source_id=rec.id, artist_id=artist.id, external_post_id="p1")
db.add(post) db.add(post)
img = ImageRecord( img = ImageRecord(
path=f"/images/{artist.slug}/pixiv/pixiv/1_a_00.jpg", path=f"/images/{artist.slug}/hentaifoundry/hentaifoundry/1_a_00.jpg",
sha256=str(artist.id).rjust(64, "0"), size_bytes=1, mime="image/jpeg", sha256=str(artist.id).rjust(64, "0"), size_bytes=1, mime="image/jpeg",
width=1, height=1, origin="imported_filesystem", width=1, height=1, origin="imported_filesystem",
integrity_status="unknown", artist_id=artist.id, integrity_status="unknown", artist_id=artist.id,
+2 -2
View File
@@ -22,8 +22,8 @@ from types import SimpleNamespace
import pytest import pytest
from backend.app.services.membership_roster import has_paid_access, roster_user_id from backend.app.services.membership_roster import roster_user_id
from backend.app.services.native_ingest_common import Membership from backend.app.services.native_ingest_common import Membership, has_paid_access
from backend.app.services.subscribestar_client import ( from backend.app.services.subscribestar_client import (
SubscribeStarAuthError, SubscribeStarAuthError,
SubscribeStarClient, SubscribeStarClient,