CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / integration (push) Successful in 22s
CI & Build / Python tests (push) Successful in 53s
CI & Build / Build & push image (push) Successful in 39s
Second adapter consumer. POST /api/webhooks/forge validates Gitea's
X-Gitea-Signature (HMAC-SHA256, constant-time; no secret configured =
the endpoint 404s out of existence), extracts changed/removed paths,
and flags matched snippets by writing verification.invalidated_by
{commit_sha, at, path, removed} — the existing attention vocabulary
extended, not a new flag: needs_attention includes it, both filter
dialects (Python + jsonpath SQL) include it in 'attention' and exclude
it from 'ok', and recording ANY fresh verdict clears it by construction
because compose_verification builds a new dict. Unverified snippets are
skipped (already in their own bucket); replayed deliveries at the same
head commit are no-ops; processing failures return 200 with a WARNING +
AppLog canary so the forge never marks deliveries failed and operators
never disable the hook over a transient (#2663's lesson).
Matching goes through repo BINDINGS: recorded location repos are
free-form names ('Scribe') that cannot address a forge, so a snippet
reaches its forge repo through its project's binding — which also fixes
step 5's pull-time resolution for every real record via the same
fallback. O(bindings + snippets-in-project + changed files).
Settings: webhook secret beside the forge config (masked, sentinel-
skipped, Docker-secret env channel, endpoint documented in the UI).
Tests: signature gate, payload parsing, path semantics, both filter
dialects extended in the drift-check guard file, and real-Postgres
end-to-end (flag lands, attention lists it, replay quiet, re-verify
clears, unbound repo untouched).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
108 lines
4.6 KiB
Python
108 lines
4.6 KiB
Python
"""Forge push webhook — drift flagging at the moment the repo moves (#2691).
|
|
|
|
The forge POSTs here on every push; changed paths are matched against recorded
|
|
snippet locations (through repo bindings) and matched verdicts get an
|
|
``invalidated_by`` marker that surfaces in the ``verification="attention"``
|
|
listing. This is what makes verification scale past tens of records: sessions
|
|
recheck what pushes flagged instead of sweeping everything.
|
|
|
|
Registering the webhook on the forge is per-instance setup (Settings → Config
|
|
→ Git Forge shows the endpoint and holds the secret) — the server never
|
|
self-registers on the forge.
|
|
|
|
Contract with the forge's delivery loop:
|
|
- No secret configured → 404: the endpoint doesn't exist until an operator
|
|
creates it. Bad signature → 401: that's a caller problem worth signaling.
|
|
- A PROCESSING failure returns 200 with ``{"ok": false}`` and drops a
|
|
WARNING + AppLog row (the #2663 canary pattern): repeated 5xx responses
|
|
make forges mark deliveries failed and operators disable the hook, which
|
|
would silently turn the feature off — the exact failure mode this
|
|
milestone exists to prevent.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import hashlib
|
|
import hmac
|
|
import logging
|
|
import traceback
|
|
|
|
from quart import Blueprint, jsonify, request
|
|
|
|
from scribe.config import Config
|
|
from scribe.services.repo_bindings import normalize_repo_key
|
|
from scribe.services.settings import get_admin_setting
|
|
from scribe.services.snippets import invalidate_for_push
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
webhooks_bp = Blueprint("webhooks", __name__, url_prefix="/api/webhooks")
|
|
|
|
FORGE_WEBHOOK_SECRET_KEY = "forge_webhook_secret"
|
|
|
|
|
|
def signature_ok(secret: str, body: bytes, signature: str) -> bool:
|
|
"""Validate Gitea's push signature: X-Gitea-Signature is the hex HMAC-SHA256
|
|
of the raw body under the webhook secret. Constant-time compare."""
|
|
if not secret or not signature:
|
|
return False
|
|
expected = hmac.new(secret.encode("utf-8"), body, hashlib.sha256).hexdigest()
|
|
return hmac.compare_digest(expected, signature.strip().lower())
|
|
|
|
|
|
def push_facts(payload: dict) -> tuple[str, list[str], list[str], str]:
|
|
"""(repo identity, changed paths, removed paths, head commit) from a Gitea
|
|
push payload. Tolerant: absent fields read as empty, never raise."""
|
|
repo = payload.get("repository") or {}
|
|
raw_repo = repo.get("clone_url") or repo.get("html_url") or repo.get("full_name") or ""
|
|
changed: list[str] = []
|
|
removed: list[str] = []
|
|
for commit in payload.get("commits") or []:
|
|
changed.extend(commit.get("added") or [])
|
|
changed.extend(commit.get("modified") or [])
|
|
removed.extend(commit.get("removed") or [])
|
|
# De-dup while keeping order stable for logs.
|
|
changed = list(dict.fromkeys(changed))
|
|
removed = list(dict.fromkeys(removed))
|
|
return raw_repo, changed, removed, str(payload.get("after") or "")
|
|
|
|
|
|
@webhooks_bp.route("/forge", methods=["POST"])
|
|
async def forge_push():
|
|
secret = await get_admin_setting(FORGE_WEBHOOK_SECRET_KEY, "") or Config.FORGE_WEBHOOK_SECRET
|
|
if not secret:
|
|
# Not "forbidden" — the endpoint is not a thing on this instance.
|
|
return jsonify({"error": "Not found"}), 404
|
|
|
|
body = await request.get_data()
|
|
if not signature_ok(secret, body, request.headers.get("X-Gitea-Signature", "")):
|
|
return jsonify({"error": "Invalid signature"}), 401
|
|
|
|
try:
|
|
payload = await request.get_json(force=True) or {}
|
|
raw_repo, changed, removed, head = push_facts(payload)
|
|
repo_key = normalize_repo_key(raw_repo)
|
|
if not repo_key:
|
|
return jsonify({"ok": True, "flagged": 0, "reason": "no repository in payload"})
|
|
flagged = await invalidate_for_push(repo_key, changed, removed, head)
|
|
if flagged:
|
|
logger.info(
|
|
"forge push %s flagged %d snippet(s) for recheck", head[:12], flagged
|
|
)
|
|
return jsonify({"ok": True, "flagged": flagged})
|
|
except Exception:
|
|
logger.warning("forge webhook processing failed", exc_info=True)
|
|
try:
|
|
from scribe.services.logging import log_error
|
|
|
|
await log_error(
|
|
endpoint="webhooks/forge",
|
|
error_type="forge_webhook_failed",
|
|
error_message="push received but drift flagging failed — "
|
|
"snippets touched by this push were not marked for recheck",
|
|
traceback=traceback.format_exc(),
|
|
)
|
|
except Exception:
|
|
logger.debug("forge webhook canary write failed", exc_info=True)
|
|
# 200 on purpose — see the module docstring's delivery contract.
|
|
return jsonify({"ok": False})
|