Files
FabledCurator/backend/app/utils/paths.py
T
bvandeusenandClaude Opus 5 30337a6c11
CI / lint (push) Failing after 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m11s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m4s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m50s
fix: library paths follow the artist's slug, not the import folder's name (4244)
Step 1 of milestone #421. The images tree has 57 directory families for what
the database says are single artists — `Conto`/`conto`, `InCaseArt`/`incaseart`,
`StickySpoodge`/`Stickyspoodge`/`stickyspoodge`, and so on down to a four-way
split for Pocket Ace Games.

There was never a duplicate Artist row. `/api/artists/names` returns exactly
one per artist. The files simply get written to two places for one row:
`_copy_to_library` built its destination from `derive_subdir`, which mirrors
the IMPORT tree's folder name verbatim, while the download path leaves files
where the ingester wrote them — under the slug. Two writers, two conventions,
one artist.

This is the half that stops it re-growing, and it has to land before anything
moves existing files: consolidate first and the next filesystem import out of
a capitalised folder re-creates the directory that was just emptied.

`canonical_subdir` replaces the top-level segment with the artist's slug and
leaves everything below it alone — the post hierarchy is the downloader's
business. Two deliberate pass-throughs: no resolved artist (nothing
authoritative to canonicalise against) and an empty subdir (a file at the
images root, whose fate is task #4247, not a side effect of this helper).

`_supersede` resolves the KEPT row's artist for the same reason — a supersede
rewrites `existing.path`, so writing it anywhere else would move a row back
out of the tree being consolidated. ImageRecord carries `artist_id` with no
relationship attribute, so that is a session lookup rather than an attribute.

test_import_one_happy_path pinned the old `Alice/` destination and now pins
`alice/` (rule 90).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
2026-09-21 08:48:46 -04:00

112 lines
4.5 KiB
Python

"""Filesystem path helpers — destination derivation, hash-suffixed names."""
import re
from pathlib import Path
_MAX_EXT_LEN = 16
# A Patreon/gallery CDN URL embeds a 32-char hex (MD5) path segment that is the
# file's stable per-file identity — the same role gallery-dl's `_filehash`
# plays. It is the join key between a post body `<img src=CDN>` and the local
# copy we downloaded (extract_media dedups content vs gallery images by it), so
# this ONE extractor must be used for both capture-time persistence and
# render-time matching — they cannot be allowed to drift. Match the FIRST 32-hex
# run anywhere in the URL (path or query); real CDN URLs carry exactly one.
_FILEHASH_RE = re.compile(r"([0-9a-fA-F]{32})")
def filehash_from_url(url: str | None) -> str | None:
"""The 32-char hex (MD5) CDN identity segment of `url`, lowercased, or None
when the URL is empty / carries no such segment."""
if not url:
return None
match = _FILEHASH_RE.search(url)
return match.group(1).lower() if match else None
def safe_ext(name: str | Path) -> str:
"""Conservatively extract a short, alphanumeric file extension.
gallery-dl and Patreon CDN URLs produce basenames with URL-encoded
query-string artifacts, so `Path.suffix` can return 50+ chars of base64-ish
junk that blows bounded VARCHAR columns (e.g. PostAttachment.ext varchar(32)).
Accept only a suffix ≤16 chars whose post-dot characters are all alphanumeric;
otherwise return "" (no known extension). Operator-flagged 2026-05-25 — ONE
impl for the importer and the native Patreon client.
"""
suffix = Path(name).suffix.lower()
if not suffix or len(suffix) > _MAX_EXT_LEN:
return ""
if not all(c.isalnum() for c in suffix[1:]):
return ""
return suffix
def derive_subdir(source_path: Path, import_root: Path) -> str:
"""Returns the relative subdirectory of source_path under import_root.
The top-level folder name is treated as the 'artist' bucket. Nested
paths preserve hierarchy.
import_root=/import
source_path=/import/Alice/sub/x.png -> "Alice/sub"
source_path=/import/Alice/x.png -> "Alice"
source_path=/import/x.png -> ""
"""
try:
rel = source_path.parent.relative_to(import_root)
except ValueError:
return ""
return str(rel) if str(rel) != "." else ""
def canonical_subdir(subdir: str, artist_slug: str | None) -> str:
"""`subdir` with its TOP-LEVEL segment replaced by the artist's slug.
canonical_subdir("Conto/patreon", "conto") -> "conto/patreon"
canonical_subdir("Conto", "conto") -> "conto"
canonical_subdir("Conto/patreon", None) -> "Conto/patreon"
canonical_subdir("", "conto") -> ""
`derive_subdir` mirrors the IMPORT tree's folder names verbatim, so a
filesystem import out of `/import/Conto/...` used to write
`<images_root>/Conto/...` while the download path wrote `<root>/conto/...`
for the very same Artist row. One artist, two directories, forever — 57
such families had accumulated by 2026-09-21, and the database never had
duplicate artists at all (milestone #421).
The slug is the canonical name because it is the Artist row's own
identifier: it is what `/api/artists` reports, what the ingesters already
write, and the one spelling that cannot vary with how a folder happened to
be capitalised on the way in.
Two deliberate pass-throughs. NO artist resolved means there is nothing
authoritative to canonicalise against, and an EMPTY subdir is a file
landing at the images root — those have no artist folder to correct, and
what becomes of them is its own decision (task #4247), not a side effect
of this helper.
"""
if not artist_slug or not subdir:
return subdir
return str(Path(artist_slug, *Path(subdir).parts[1:]))
def hash_suffixed_name(stem: str, sha256_hex: str, ext: str) -> str:
"""Builds 'stem__<first10ofhash><ext>'.
Examples:
hash_suffixed_name("photo", "abcdef1234567890...", ".png")
-> "photo__abcdef1234.png"
"""
return f"{stem}__{sha256_hex[:10]}{ext}"
def derive_top_level_artist(source_path: Path, import_root: Path) -> str | None:
"""Returns the top-level folder name under import_root, or None if the
file is directly in import_root.
"""
subdir = derive_subdir(source_path, import_root)
if not subdir:
return None
return subdir.split("/", 1)[0]