v26.05.24.1: FC-3g Firefox extension + worker resilience + UI/migration fixes #7

2026-05-24T12:52:26-04:00

bvandeusen commented

2026-05-24 12:52:26 -04:00

(Migrated from git.fabledsword.com)

Big release rolling up FC-3g (Firefox extension), worker-level recovery hardening, and several dogfood fixes surfaced during the IR migration.

Headline — FC-3g: Firefox extension

Plain-JS MV3 Firefox extension that pushes session cookies from the six supported platforms into FC's /api/credentials, and lets you add a creator as a Source from their page in one click.

Lives in extension/ at the repo root; built with Mozilla web-ext.
Six platforms: Patreon, SubscribeStar, Hentai-Foundry (cookies); Discord (token via webRequest), Pixiv (PKCE OAuth); DeviantArt (cookies).
Popup UI with per-platform Export + Export All + Sources tab (lists + check-now).
Content script injects a floating "Add to FabledCurator" button on artist pages.
New backend: /api/extension/quick-add-source (derives platform + artist slug from URL, find-or-creates Artist + Source), /api/extension/manifest (install metadata), /extension/<filename> static route serving the signed XPI with application/x-xpinstall MIME so Firefox installs in one click.
New BrowserExtensionCard.vue on Settings → Maintenance: install button + API URL/key copy/rotate.
CI workflow .forgejo/workflows/extension.yml lints on dev, signs via Mozilla AMO (unlisted) + commits XPI back on main → next build bundles it into the Docker image.
Sequencing: first signed XPI lands when this PR merges; the bot's commit on main triggers a second build that includes the XPI.

Worker resilience (operator-mandated 2026-05-24)

Operator hit a 2224-row pile of ImportTask.status='processing' zombies during the IR migration. Built proper recovery into the worker code itself:

Tightened recovery sweep: STUCK_THRESHOLD_MINUTES 30 → 5. The sweep flips the existing 2224 zombies retroactively within 5 min of the new image being deployed.
import_media_file body-wrap: every exit path now flips the row to failed (with a real error message) or relies on Celery autoretry. No path can leave it stuck in processing.
Celery autoretry on OperationalError / DBAPIError / OSError for import_media_file, generate_thumbnail, tag_and_embed, download_source. Exponential backoff with jitter, max 3 attempts.
Per-task soft_time_limit + time_limit safety nets so a runaway task can't sit forever.

Migration UX fixes

FormData passthrough in useApi — IR/GS export-file uploads from the Legacy Migration card were JSON-stringifying FormData into "{}" (backend saw an empty multipart payload). Now FormData passes through unaltered.
Backup subprocess 30-min wall-clock timeout — short-term mitigation for the NFS-pg_dump hang; FC-3h will promote backup out of the migrators package and add a proper recovery sweep + retention.

UI

Global padding-top on .fc-content so views don't start hidden behind the sticky TopNav (depending on parent overflow interactions). Scrolled-down content still slides under the nav — gradient-fade design intact.

CI

extension.yml YAML schema fix — single-line bot commit message; the multi-line trailer was tripping Forgejo's YAML validator.
ASYNC240 fix — /api/extension/manifest filesystem work moved into asyncio.to_thread per ruff.

Test hygiene

Unpin test_recover_interrupted_only_old from the recovery threshold value so future tightening doesn't break it.

Operator notes

After pulling: force-update app, worker, ml-worker, scheduler, and web services.
The 2224 stuck processing rows will recover within 5 min of scheduler coming up on the new image.
First signed XPI for the extension lands ~5-10 min after merge (Mozilla signing + bot commit + second build cycle). After that, Settings → Maintenance → Browser extension card goes from "No bundled extension found" warning to the "Install Firefox extension" CTA.

Big release rolling up FC-3g (Firefox extension), worker-level recovery hardening, and several dogfood fixes surfaced during the IR migration. ## Headline — FC-3g: Firefox extension Plain-JS MV3 Firefox extension that pushes session cookies from the six supported platforms into FC's `/api/credentials`, and lets you add a creator as a Source from their page in one click. - Lives in `extension/` at the repo root; built with Mozilla `web-ext`. - Six platforms: Patreon, SubscribeStar, Hentai-Foundry (cookies); Discord (token via webRequest), Pixiv (PKCE OAuth); DeviantArt (cookies). - Popup UI with per-platform Export + Export All + Sources tab (lists + check-now). - Content script injects a floating "Add to FabledCurator" button on artist pages. - New backend: `/api/extension/quick-add-source` (derives platform + artist slug from URL, find-or-creates Artist + Source), `/api/extension/manifest` (install metadata), `/extension/<filename>` static route serving the signed XPI with `application/x-xpinstall` MIME so Firefox installs in one click. - New `BrowserExtensionCard.vue` on Settings → Maintenance: install button + API URL/key copy/rotate. - CI workflow `.forgejo/workflows/extension.yml` lints on dev, signs via Mozilla AMO (unlisted) + commits XPI back on main → next build bundles it into the Docker image. - Sequencing: first signed XPI lands when this PR merges; the bot's commit on main triggers a second build that includes the XPI. ## Worker resilience (operator-mandated 2026-05-24) Operator hit a 2224-row pile of `ImportTask.status='processing'` zombies during the IR migration. Built proper recovery into the worker code itself: - **Tightened recovery sweep:** `STUCK_THRESHOLD_MINUTES` 30 → 5. The sweep flips the existing 2224 zombies retroactively within 5 min of the new image being deployed. - **import_media_file body-wrap:** every exit path now flips the row to `failed` (with a real error message) or relies on Celery autoretry. No path can leave it stuck in `processing`. - **Celery autoretry** on `OperationalError` / `DBAPIError` / `OSError` for `import_media_file`, `generate_thumbnail`, `tag_and_embed`, `download_source`. Exponential backoff with jitter, max 3 attempts. - **Per-task `soft_time_limit` + `time_limit`** safety nets so a runaway task can't sit forever. ## Migration UX fixes - **FormData passthrough in useApi** — IR/GS export-file uploads from the Legacy Migration card were JSON-stringifying FormData into `"{}"` (backend saw an empty multipart payload). Now FormData passes through unaltered. - **Backup subprocess 30-min wall-clock timeout** — short-term mitigation for the NFS-pg_dump hang; FC-3h will promote backup out of the migrators package and add a proper recovery sweep + retention. ## UI - **Global padding-top on `.fc-content`** so views don't start hidden behind the sticky TopNav (depending on parent overflow interactions). Scrolled-down content still slides under the nav — gradient-fade design intact. ## CI - **extension.yml YAML schema fix** — single-line bot commit message; the multi-line trailer was tripping Forgejo's YAML validator. - **ASYNC240 fix** — `/api/extension/manifest` filesystem work moved into `asyncio.to_thread` per ruff. ## Test hygiene - Unpin `test_recover_interrupted_only_old` from the recovery threshold value so future tightening doesn't break it. ## Operator notes - After pulling: force-update `app`, `worker`, `ml-worker`, `scheduler`, and `web` services. - The 2224 stuck `processing` rows will recover within 5 min of `scheduler` coming up on the new image. - First signed XPI for the extension lands ~5-10 min after merge (Mozilla signing + bot commit + second build cycle). After that, Settings → Maintenance → Browser extension card goes from "No bundled extension found" warning to the "Install Firefox extension" CTA.

bvandeusen referenced this issue from a commit

2026-06-06 00:04:25 -04:00

feat(patreon): dead-letter ledger for permanently-failing media — #705 step 2 (#7)

bvandeusen referenced this issue from a commit

2026-06-06 00:15:50 -04:00

feat(patreon): within-pass transient retry for media GETs — #8 (was overstated as done)

bvandeusen referenced this pull request

2026-06-23 08:57:22 -04:00

Tagging & viewing roadmap: tag query surface + hygiene projections + Explore view (Clusters A/B/C) #128

Sign in to join this conversation.