feat(forge): adapter seam + Gitea implementation — optional read access to the operator's forge (#2689)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 36s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 36s
Step 4 of milestone 288 (decision #2686). services/forge.py defines the contract steps 5-7 consume — read_file (content + last_commit_sha, the provenance stamp), default_branch, resolve_repo, check — with GiteaForge as the first implementation over the REST contents/repo/version/user endpoints. Repo identity reuses normalize_repo_key: the host segment selects whether this forge serves a recorded repo, the remainder is the API path, so no new identity scheme exists. Read-only by construction; errors never carry the token; first outbound-HTTP timeout convention (5s total, no retries — the consumer's fallback is the retry policy). OPTIONAL per instance (rule #115): get_forge() returns None when unconfigured and every consumer treats None as today's behavior. Config lives in admin settings (Settings → Config → Git Forge: kind/base URL/token, save + test-connection probe reporting version + identity), with FORGE_* env / Docker-secret fallbacks; DB wins so a UI edit can't silently lose to an env var. Token treatment follows the smtp_password convention (masked on read, mask-sentinel skipped on write, absent from audit details) — and wiring it surfaced that the generic GET/PUT /api/settings dump bypassed that masking for the owning admin's raw KV rows, so secret keys are now masked there too (fixes the same exposure for smtp_password). Contract tests run against httpx.MockTransport as the fake forge — the reference behaviors the GitHub adapter (step 8) must reproduce — plus the off-by-default gate, partial-config-is-off, env-vs-DB precedence, and route/mask structural checks. Also: the step-2 definition detector learned to skip dunders after flagging __init__ as 'already defined in 4 files' on this step's own build — guaranteed noise for a hint that must stay trustworthy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,237 @@
|
||||
"""Forge adapter — optional server-side READ access to the operator's git forge.
|
||||
|
||||
Step 4 of milestone 288 (#2689, decision #2686). The recorded location of a
|
||||
snippet is the source of truth for its code and the stored body is a cache;
|
||||
this module is the seam that lets the SERVER read that source of truth, so the
|
||||
cache can be refreshed at pull time (step 5), drift can be flagged from push
|
||||
webhooks (step 6), and coverage can be measured (step 7).
|
||||
|
||||
Design constraints, in force everywhere below:
|
||||
|
||||
- OPTIONAL per instance (rule #115). `get_forge()` returns None when nothing
|
||||
is configured, and every consumer must treat None as "keep today's
|
||||
behavior". An install that never configures a forge is not degraded — it
|
||||
is the baseline.
|
||||
- READ-ONLY by construction. The adapter exposes reads; there is no write
|
||||
method to misuse. The token an operator mints for it only ever needs read
|
||||
scope, and the docs say so.
|
||||
- The contract stays as small as its consumers (steps 5-7): read_file /
|
||||
default_branch / resolve_repo / check. GitHub later implements this same
|
||||
contract (step 8); resist widening it speculatively.
|
||||
- Repo identity is the repo-binding key — `normalize_repo_key`'s
|
||||
host/owner/repo — so the join between a snippet's recorded repo and the
|
||||
forge needs no new identity scheme. The host segment selects whether THIS
|
||||
forge can serve the repo; the remainder is the API path.
|
||||
- Errors carry no token, ever, and failures are exceptions the caller
|
||||
handles — a consumer decides whether to fall back (pull-time fetch) or
|
||||
surface (settings test button); this module never silently swallows.
|
||||
|
||||
This is also the codebase's first outbound-HTTP client with a real timeout
|
||||
convention (oauth.py predates it): short total timeout, no retries — every
|
||||
consumer has a fallback, so a slow forge must cost bounded time.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from urllib.parse import quote, urlsplit
|
||||
|
||||
import httpx
|
||||
|
||||
from scribe.config import Config
|
||||
from scribe.services.repo_bindings import normalize_repo_key
|
||||
from scribe.services.settings import get_admin_setting
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
FORGE_KIND_KEY = "forge_kind"
|
||||
FORGE_BASE_URL_KEY = "forge_base_url"
|
||||
FORGE_TOKEN_KEY = "forge_token"
|
||||
|
||||
# Kinds an instance can configure. GitHub joins in step 8 of milestone 288.
|
||||
FORGE_KINDS = ("gitea",)
|
||||
|
||||
# Total budget per forge call. Consumers either have a cache to fall back to
|
||||
# (step 5) or a user watching a button (the test probe) — neither tolerates a
|
||||
# hung socket, and there is no retry: the fallback IS the retry policy.
|
||||
_TIMEOUT = httpx.Timeout(5.0)
|
||||
|
||||
|
||||
class ForgeError(RuntimeError):
|
||||
"""A forge call failed (network, auth, unexpected payload). Token-free."""
|
||||
|
||||
|
||||
class ForgeNotFound(ForgeError):
|
||||
"""The repo, path, or ref does not exist on the forge — the one failure
|
||||
consumers treat differently, because for a recorded snippet location it is
|
||||
itself a finding (the recorded path is gone)."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ForgeFile:
|
||||
"""One file read from the forge at a specific point in history."""
|
||||
|
||||
content: str
|
||||
# The commit the content was served at — what provenance stores (#2688).
|
||||
commit_sha: str
|
||||
path: str
|
||||
|
||||
|
||||
def _host_of(url: str) -> str:
|
||||
return (urlsplit(url).hostname or "").lower()
|
||||
|
||||
|
||||
class GiteaForge:
|
||||
"""The Gitea implementation of the forge contract, over its REST API.
|
||||
|
||||
`transport` exists for tests: httpx.MockTransport makes the contract
|
||||
testable without a live server or a new dependency. Production callers
|
||||
never pass it.
|
||||
"""
|
||||
|
||||
kind = "gitea"
|
||||
|
||||
def __init__(self, base_url: str, token: str, *, transport=None) -> None:
|
||||
self.base_url = (base_url or "").rstrip("/")
|
||||
self._token = token or ""
|
||||
self._transport = transport
|
||||
|
||||
@property
|
||||
def host(self) -> str:
|
||||
return _host_of(self.base_url)
|
||||
|
||||
def resolve_repo(self, repo_or_url: str) -> str | None:
|
||||
"""The forge-API repo path for a recorded repo — or None if this forge
|
||||
does not serve it.
|
||||
|
||||
Accepts anything `normalize_repo_key` accepts (a raw remote URL or an
|
||||
already-normalized key). None is a NORMAL answer, not an error: a
|
||||
snippet recorded against github.com on an instance whose forge is a
|
||||
self-hosted Gitea is simply out of this forge's reach.
|
||||
"""
|
||||
key = normalize_repo_key(repo_or_url or "")
|
||||
if not key or "/" not in key:
|
||||
return None
|
||||
host, _, rest = key.partition("/")
|
||||
if host != self.host or "/" not in rest:
|
||||
return None
|
||||
return rest
|
||||
|
||||
def _client(self) -> httpx.AsyncClient:
|
||||
kwargs: dict = {
|
||||
"base_url": f"{self.base_url}/api/v1",
|
||||
"headers": {"Authorization": f"token {self._token}"},
|
||||
"timeout": _TIMEOUT,
|
||||
}
|
||||
if self._transport is not None:
|
||||
kwargs["transport"] = self._transport
|
||||
return httpx.AsyncClient(**kwargs)
|
||||
|
||||
async def _get(self, client: httpx.AsyncClient, url: str, **kw) -> httpx.Response:
|
||||
try:
|
||||
resp = await client.get(url, **kw)
|
||||
except httpx.HTTPError as exc:
|
||||
# str(exc) on transport errors names hosts and timeouts, never
|
||||
# headers — safe, and the detail is what makes the test button useful.
|
||||
raise ForgeError(f"forge unreachable: {exc}") from exc
|
||||
if resp.status_code == 404:
|
||||
raise ForgeNotFound(f"not found on forge: {url}")
|
||||
if resp.status_code in (401, 403):
|
||||
raise ForgeError("forge rejected the token (check its read scope)")
|
||||
if resp.status_code >= 400:
|
||||
raise ForgeError(f"forge returned HTTP {resp.status_code} for {url}")
|
||||
return resp
|
||||
|
||||
async def read_file(self, repo: str, path: str, ref: str = "") -> ForgeFile:
|
||||
"""Read one file's current content, with the commit it was served at.
|
||||
|
||||
`repo` is the API path from resolve_repo ("owner/repo"); `ref` is a
|
||||
branch, tag, or commit — empty means the default branch.
|
||||
"""
|
||||
params = {"ref": ref} if ref else None
|
||||
async with self._client() as client:
|
||||
resp = await self._get(
|
||||
client,
|
||||
f"/repos/{repo}/contents/{quote(path, safe='/')}",
|
||||
params=params,
|
||||
)
|
||||
payload = resp.json()
|
||||
if isinstance(payload, list):
|
||||
raise ForgeNotFound(f"{path} is a directory on the forge, not a file")
|
||||
if payload.get("type") != "file":
|
||||
raise ForgeNotFound(
|
||||
f"{path} is a {payload.get('type', 'non-file')} on the forge"
|
||||
)
|
||||
if payload.get("encoding") != "base64" or payload.get("content") is None:
|
||||
raise ForgeError(f"forge returned no readable content for {path}")
|
||||
try:
|
||||
content = base64.b64decode(payload["content"]).decode("utf-8")
|
||||
except (binascii.Error, UnicodeDecodeError) as exc:
|
||||
raise ForgeError(f"forge content for {path} is not utf-8 text") from exc
|
||||
return ForgeFile(
|
||||
content=content,
|
||||
# last_commit_sha is the commit that last touched the file — the
|
||||
# honest provenance stamp. The blob sha is a content address, not
|
||||
# a point in history, so it is deliberately not surfaced.
|
||||
commit_sha=payload.get("last_commit_sha") or "",
|
||||
path=payload.get("path") or path,
|
||||
)
|
||||
|
||||
async def default_branch(self, repo: str) -> str:
|
||||
async with self._client() as client:
|
||||
resp = await self._get(client, f"/repos/{repo}")
|
||||
branch = (resp.json() or {}).get("default_branch") or ""
|
||||
if not branch:
|
||||
raise ForgeError(f"forge reported no default branch for {repo}")
|
||||
return branch
|
||||
|
||||
async def check(self) -> dict:
|
||||
"""Health probe for the settings test button: reach the forge AND
|
||||
prove the token is accepted. Returns {"ok", "version", "username"}."""
|
||||
async with self._client() as client:
|
||||
version = (await self._get(client, "/version")).json() or {}
|
||||
user = (await self._get(client, "/user")).json() or {}
|
||||
return {
|
||||
"ok": True,
|
||||
"version": version.get("version") or "",
|
||||
"username": user.get("login") or user.get("username") or "",
|
||||
}
|
||||
|
||||
|
||||
async def forge_config() -> dict:
|
||||
"""The instance's forge configuration, DB-first with env fallback.
|
||||
|
||||
The env channel exists so a deployment can keep the token out of the
|
||||
database entirely (Docker secret via FORGE_TOKEN_FILE) — the DB value wins
|
||||
when both are present because the admin UI writes there, and a UI edit
|
||||
that silently loses to an env var would look exactly like a broken form.
|
||||
"""
|
||||
return {
|
||||
"kind": (await get_admin_setting(FORGE_KIND_KEY, "") or Config.FORGE_KIND)
|
||||
.strip()
|
||||
.lower(),
|
||||
"base_url": (
|
||||
await get_admin_setting(FORGE_BASE_URL_KEY, "") or Config.FORGE_BASE_URL
|
||||
).rstrip("/"),
|
||||
"token": await get_admin_setting(FORGE_TOKEN_KEY, "") or Config.FORGE_TOKEN,
|
||||
}
|
||||
|
||||
|
||||
async def get_forge(*, transport=None) -> GiteaForge | None:
|
||||
"""The configured forge adapter, or None — and None means "behave exactly
|
||||
as if this module did not exist", which every consumer must honor."""
|
||||
cfg = await forge_config()
|
||||
if cfg["kind"] not in FORGE_KINDS:
|
||||
if cfg["kind"]:
|
||||
# A kind we don't implement is a misconfiguration, not "off" —
|
||||
# say so once per lookup rather than silently reading as absent.
|
||||
logger.warning("unknown forge kind %r configured — forge disabled", cfg["kind"])
|
||||
return None
|
||||
if not cfg["base_url"] or not cfg["token"]:
|
||||
return None
|
||||
if not cfg["base_url"].startswith(("http://", "https://")):
|
||||
logger.warning("forge base URL %r has no http(s) scheme — forge disabled", cfg["base_url"])
|
||||
return None
|
||||
return GiteaForge(cfg["base_url"], cfg["token"], transport=transport)
|
||||
Reference in New Issue
Block a user