Files
FabledCurator/backend/app/services/credential_service.py
T
bvandeusen abafc3265e refactor(platforms): promote services/platforms.py → services/platforms/ package with per-platform quirk colocation
Operator-requested 2026-05-27: centralize the per-platform quirks that
had been accumulating across credential_service, sidecar, and platforms
into a single per-platform module so adding/updating quirks becomes
"edit one file."

**Layout**

  services/platforms/
    base.py            PlatformInfo dataclass + module-default key
                       chains + shared helpers (str_id_value, str_field)
    __init__.py        PLATFORMS dict + public API (auth_type_for,
                       known_platform_keys, to_dict,
                       external_post_id_keys_for, description_keys_for)
    patreon.py         metadata only — the reference platform, no quirks
    subscribestar.py   metadata + augment_cookies (18+ agreement) +
                       derive_post_url (synthetic /posts/<post_id>)
    hentaifoundry.py   metadata + augment_cookies (host-only PHPSESSID
                       duplicate) + derive_post_url (/pictures/user/...)
    pixiv.py           metadata + derive_post_url (/artworks/<id>)
    discord.py         metadata + derive_post_url
                       (channels/<server>/<channel>/<message>)
    deviantart.py      metadata only — un-audited; quirks to be added
                       when an operator first exercises DA

**PlatformInfo extensions**

Existing fields preserved. Four new optional fields:

  external_post_id_keys: tuple[str, ...] | None
      Override the sidecar external_post_id lookup chain. None falls
      back to DEFAULT_EXTERNAL_POST_ID_KEYS in base.py
      ("post_id", "id", "index", "message_id") — covers every current
      platform.

  description_keys: tuple[str, ...] | None
      Override the description body lookup chain. None falls back to
      DEFAULT_DESCRIPTION_KEYS ("content", "description", "caption",
      "message") — Discord's "message" body field is covered by the
      default's trailing entry.

  derive_post_url: Callable[[dict], str | None] | None
      Synthesize the post permalink from sidecar metadata. None = trust
      the bare `url` / `post_url` field (patreon, deviantart).
      subscribestar/pixiv/hf/discord override this because their `url`
      is the file CDN URL.

  augment_cookies: Callable[[str], str] | None
      Post-process the materialized cookies.txt before gallery-dl
      consumes it. None = no-op. Used by subscribestar (age cookie) and
      hentaifoundry (host-only PHPSESSID duplicate).

**Consumer changes**

- credential_service._augment_cookies(platform, netscape) shrunk from a
  per-platform-conditional dispatcher (~80 lines of inlined helpers) to
  a 5-line lookup: `info.augment_cookies(netscape) if info and
  info.augment_cookies else netscape`. The platform-specific helper
  bodies moved verbatim into the per-platform modules.

- sidecar.parse_sidecar similarly delegates: external_post_id chain via
  external_post_id_keys_for(category), description chain via
  description_keys_for(category), post_url via
  PLATFORMS[category].derive_post_url. The _DERIVED_URL_PLATFORMS set
  and inline _derive_post_url body both gone. Added a shared `_first_id`
  helper for bool-safe id coercion.

**Public API preserved**

PLATFORMS, PlatformInfo, auth_type_for, known_platform_keys, to_dict
are all re-exported from the package's __init__.py. test_platforms_registry
test_credential_service, and test_sidecar_util pass without changes
because the behavior is identical; only the implementation moved.

**Adding a new platform**

1. Create services/platforms/<name>.py with `INFO = PlatformInfo(...)`
   and any of the four optional hooks.
2. Import it in services/platforms/__init__.py + add to the PLATFORMS
   tuple-comprehension.
3. Done. sidecar parsing, cookie materialization, /api/platforms all
   pick it up automatically.
2026-05-27 19:46:05 -04:00

214 lines
7.4 KiB
Python

"""FC-3b: CRUD over Credential rows + on-demand cookies-file
materialisation for gallery-dl (consumed by FC-3c).
"""
from __future__ import annotations
import json
import os
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from ..models import Credential
from .credential_crypto import CredentialCrypto
from .platforms import PLATFORMS, auth_type_for
class CredentialServiceError(Exception):
"""Base."""
class UnknownPlatformError(CredentialServiceError):
pass
class WrongAuthTypeError(CredentialServiceError):
def __init__(self, expected: str):
super().__init__(f"wrong credential_type for platform; expected {expected!r}")
self.expected = expected
class EmptyDataError(CredentialServiceError):
pass
@dataclass(frozen=True)
class CredentialRecord:
platform: str
credential_type: str
captured_at: str
expires_at: str | None
last_verified: str | None
def to_dict(self) -> dict:
return {
"platform": self.platform,
"credential_type": self.credential_type,
"captured_at": self.captured_at,
"expires_at": self.expires_at,
"last_verified": self.last_verified,
}
def _to_record(row: Credential) -> CredentialRecord:
return CredentialRecord(
platform=row.platform,
credential_type=row.credential_type,
captured_at=row.captured_at.isoformat() if row.captured_at else "",
expires_at=row.expires_at.isoformat() if row.expires_at else None,
last_verified=row.last_verified.isoformat() if row.last_verified else None,
)
# Default cookies dir under the image root; tests override.
_DEFAULT_COOKIES_DIR = Path("/images/cookies")
class CredentialService:
def __init__(
self,
session: AsyncSession,
crypto: CredentialCrypto,
cookies_dir: Path | None = None,
):
self.session = session
self.crypto = crypto
self.cookies_dir = Path(cookies_dir) if cookies_dir else _DEFAULT_COOKIES_DIR
async def list(self) -> list[CredentialRecord]:
rows = (await self.session.execute(
select(Credential).order_by(Credential.platform.asc())
)).scalars().all()
return [_to_record(r) for r in rows]
async def get(self, platform: str) -> CredentialRecord | None:
row = (await self.session.execute(
select(Credential).where(Credential.platform == platform)
)).scalar_one_or_none()
return _to_record(row) if row else None
async def upsert(
self,
*,
platform: str,
credential_type: str,
data: str,
expires_at: datetime | None = None,
) -> CredentialRecord:
if platform not in PLATFORMS:
raise UnknownPlatformError(f"unknown platform: {platform!r}")
expected = auth_type_for(platform)
if credential_type != expected:
raise WrongAuthTypeError(expected=expected or "")
cleaned = (data or "").strip()
if not cleaned:
raise EmptyDataError("data must be a non-empty string")
blob = self.crypto.encrypt(cleaned)
row = (await self.session.execute(
select(Credential).where(Credential.platform == platform)
)).scalar_one_or_none()
if row is None:
row = Credential(
platform=platform,
credential_type=credential_type,
encrypted_blob=blob,
expires_at=expires_at,
)
self.session.add(row)
else:
row.credential_type = credential_type
row.encrypted_blob = blob
row.expires_at = expires_at
row.last_verified = None # invalidate prior verification
await self.session.commit()
await self.session.refresh(row)
return _to_record(row)
async def delete(self, platform: str) -> None:
row = (await self.session.execute(
select(Credential).where(Credential.platform == platform)
)).scalar_one_or_none()
if row is None:
raise LookupError(f"no credential for platform {platform!r}")
await self.session.delete(row)
await self.session.commit()
async def get_cookies_path(self, platform: str) -> Path | None:
"""Decrypt the credential and write a Netscape cookies.txt for
gallery-dl. Returns None if no credential or wrong kind."""
row = (await self.session.execute(
select(Credential).where(Credential.platform == platform)
)).scalar_one_or_none()
if row is None or row.credential_type != "cookies":
return None
plaintext = self.crypto.decrypt(row.encrypted_blob)
netscape = _to_netscape(plaintext)
netscape = _augment_cookies(platform, netscape)
self.cookies_dir.mkdir(parents=True, exist_ok=True)
out = self.cookies_dir / f"{platform}_cookies.txt"
out.write_text(netscape)
os.chmod(out, 0o600)
return out
async def get_token(self, platform: str) -> str | None:
row = (await self.session.execute(
select(Credential).where(Credential.platform == platform)
)).scalar_one_or_none()
if row is None or row.credential_type != "token":
return None
return self.crypto.decrypt(row.encrypted_blob)
def _augment_cookies(platform: str, netscape: str) -> str:
"""Delegate to the platform's `augment_cookies` hook if one is
registered (subscribestar, hentaifoundry, etc. — see
`services/platforms/<name>.py`). No-op when the platform doesn't
register a hook (Patreon, DeviantArt). Centralizing the
quirks-per-platform in the platforms package means adding a new
platform's cookie quirks doesn't require touching this file."""
info = PLATFORMS.get(platform)
if info is None or info.augment_cookies is None:
return netscape
return info.augment_cookies(netscape)
def _to_netscape(plaintext: str) -> str:
"""Accept either Netscape-format text (the extension's output) or a
JSON array of cookie dicts (a manual-paste edge case); produce
Netscape-format text suitable for gallery-dl --cookies."""
stripped = plaintext.strip()
if not stripped:
return ""
if stripped.startswith("#") or "\t" in stripped:
return plaintext # already Netscape
try:
loaded = json.loads(stripped)
except json.JSONDecodeError:
return plaintext # write as-is and hope; gallery-dl will complain if invalid
if isinstance(loaded, dict):
loaded = [loaded]
if not isinstance(loaded, list):
return plaintext
lines = ["# Netscape HTTP Cookie File"]
for c in loaded:
domain = str(c.get("domain", ""))
if domain and not domain.startswith("."):
domain = "." + domain
flag = "TRUE"
path = str(c.get("path", "/"))
secure = "TRUE" if c.get("secure", False) else "FALSE"
expiration = c.get("expiration") or c.get("expirationDate") or 0
try:
expiration_str = str(int(float(expiration)))
except (TypeError, ValueError):
expiration_str = "0"
name = str(c.get("name", ""))
value = str(c.get("value", ""))
lines.append("\t".join([domain, flag, path, secure, expiration_str, name, value]))
return "\n".join(lines) + "\n"