bvandeusen
a47e1b9c4e
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 50s
CI & Build / Build & push image (push) Successful in 17s
Completes the previous two commits. `--frozen` failed run 3007 with `ModuleNotFoundError: No module named 'pgvector'` — the lock wasn't merely stale in its recorded metadata, it was missing a real dependency. pgvector was added to pyproject for the vector-search work and the lock was never regenerated, and nothing noticed because CI resolved from pyproject and never read the lock. The lock has been dead weight for some time. Regenerated with `uv lock` inside a throwaway `ci-python:3.14` container — this workstation has no uv and no pip, and the CI image already carries the right toolchain, so nothing was installed to do it. uv was conservative as promised: pgvector 0.5.0 added, and NOT ONE existing pin moved (verified by diffing name=version pairs across all 106 packages). Both lanes now run `uv sync --locked`, so a dependency edit without a re-lock fails loudly at install rather than resolving around the lock. The check paid for itself on its first run by surfacing the missing pgvector. Also added uv.lock to the workflow's `paths:` filter. It was absent, so a lock-only change — exactly what a dependency bump looks like now — would not have triggered CI at all. Closes the CI half of #2194. The Dockerfile still resolves independently and is tracked there.
310 lines
13 KiB
YAML
310 lines
13 KiB
YAML
# CI runs first; build only proceeds if all checks pass.
|
|
#
|
|
# Push to dev: typecheck + lint + test + build :dev + :<sha>
|
|
# Push to main: typecheck + lint + test + build :latest + :<sha>
|
|
# Tag v* (release): typecheck + lint + test + build :latest + :<version> + :<sha>
|
|
#
|
|
# Both dev and main are gated AND built. dev pushes move :dev; main pushes move
|
|
# :latest — main IS the production line, so :latest tracks main's tip and there
|
|
# is no separate :main tag. Every push also gets an immutable :<sha> (the
|
|
# rollback point). A v* release tag additionally publishes the dated :<version>;
|
|
# since main already moved :latest, the release tag's distinct job is that
|
|
# :<version> marker (it refreshes :latest too, harmlessly).
|
|
#
|
|
# Successive pushes to the SAME ref supersede each other (see concurrency
|
|
# below), so rapid pushes don't stack identical work; dev and main runs are
|
|
# independent refs and never cancel one another.
|
|
#
|
|
# To cut a release:
|
|
# Create a release via the Forgejo UI on main with a v* tag name.
|
|
# The tag push triggers this workflow; build job pushes :latest + :<version>.
|
|
#
|
|
# PRs aren't triggered on purpose — this is a solo dev→main flow, so
|
|
# gating on branch push is already enough.
|
|
#
|
|
# NOTE on the `if:` guards below: Forgejo Actions does not consistently
|
|
# honor `on.push.branches` as a filter, so every job repeats the ref check
|
|
# explicitly — permitting dev, main, and v* tags, rejecting anything else.
|
|
#
|
|
# Required secrets (repo → Settings → Secrets → Actions):
|
|
# REGISTRY_USER — your Forgejo username
|
|
# REGISTRY_TOKEN — Forgejo PAT with write:packages scope
|
|
name: CI & Build
|
|
|
|
on:
|
|
push:
|
|
branches: [dev, main]
|
|
tags: ["v*"]
|
|
paths:
|
|
- "src/**"
|
|
- "frontend/**"
|
|
- "tests/**"
|
|
- "pyproject.toml"
|
|
# The lock now determines what gets installed, so a lock-only change has
|
|
# to trigger a run — otherwise a dependency bump lands untested.
|
|
- "uv.lock"
|
|
- "alembic/**"
|
|
- "alembic.ini"
|
|
- "Dockerfile"
|
|
- "assets/**"
|
|
- "fable-mcp/**"
|
|
- ".forgejo/workflows/ci.yml"
|
|
# Manual trigger from the Forgejo Actions UI. Useful when an image has
|
|
# been built but the deployment didn't pick it up, or when re-running
|
|
# against the same source produces different upstream behaviour
|
|
# (e.g. a transient HF download flake during the voice-bundle step).
|
|
workflow_dispatch: {}
|
|
|
|
# Cancel older runs on the same branch when a newer push lands. Tag runs
|
|
# get their own group implicitly (refs/tags/v1.2.3 ≠ refs/heads/dev) and
|
|
# are never cancelled, so a release build can't kill itself mid-flight.
|
|
concurrency:
|
|
group: ci-${{ github.ref }}
|
|
cancel-in-progress: ${{ !startsWith(github.ref, 'refs/tags/') }}
|
|
|
|
# Least-privilege default. Jobs that need more (build pushes to the
|
|
# registry) upgrade explicitly.
|
|
permissions:
|
|
contents: read
|
|
|
|
env:
|
|
REGISTRY: git.fabledsword.com
|
|
IMAGE: git.fabledsword.com/bvandeusen/fabledscribe
|
|
|
|
jobs:
|
|
typecheck:
|
|
name: TypeScript typecheck
|
|
# Gate dev, main, and v* tags; reject any other ref (see header note).
|
|
if: github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
|
|
runs-on: python-ci
|
|
container:
|
|
image: git.fabledsword.com/bvandeusen/ci-python:3.14
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Cache npm download cache
|
|
uses: actions/cache@v4
|
|
# Non-fatal: a transient cache-backend hiccup must NOT fail the whole
|
|
# typecheck job (it was skipping install + type check and reporting red
|
|
# on backend-only pushes — see issue task #828). On cache miss/error the
|
|
# job just installs without the cache.
|
|
continue-on-error: true
|
|
with:
|
|
path: ~/.npm
|
|
key: npm-cache-${{ hashFiles('frontend/package-lock.json') }}
|
|
restore-keys: npm-cache-
|
|
|
|
- name: Install dependencies
|
|
run: npm ci
|
|
working-directory: frontend
|
|
|
|
- name: Type check
|
|
run: npx vue-tsc --noEmit
|
|
working-directory: frontend
|
|
|
|
lint:
|
|
name: Python lint
|
|
if: github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
|
|
runs-on: python-ci
|
|
container:
|
|
image: git.fabledsword.com/bvandeusen/ci-python:3.14
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
# ruff is pre-installed in the ci-python image — no install
|
|
# step needed, lint runs in ~2s.
|
|
- name: Lint
|
|
run: ruff check src/
|
|
|
|
test:
|
|
name: Python tests
|
|
if: github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
|
|
runs-on: python-ci
|
|
container:
|
|
image: git.fabledsword.com/bvandeusen/ci-python:3.14
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Cache uv packages
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: ~/.cache/uv
|
|
# Keyed on the LOCK, not pyproject: the lock is what determines the
|
|
# installed set now, and a pyproject edit that doesn't change
|
|
# resolution shouldn't throw the cache away.
|
|
key: uv-${{ hashFiles('uv.lock') }}
|
|
restore-keys: uv-
|
|
|
|
# Installs exactly what uv.lock pins, and resolves nothing itself.
|
|
#
|
|
# This replaced `uv pip install -e ".[dev]"`, which resolved from the
|
|
# pyproject constraints and ignored the lock entirely. Every dependency
|
|
# floated: on 2026-07-28 mcp 2.0.0 shipped mid-session and turned `main`
|
|
# red with no repo change (issue #2194). Green CI has to mean "these exact
|
|
# versions passed", or it isn't evidence of anything.
|
|
#
|
|
# `--locked` also FAILS when uv.lock is stale against pyproject, so a
|
|
# dependency edit has to go through a deliberate `uv lock` — it can't
|
|
# arrive on its own. That check earned its place immediately: it caught
|
|
# that the lock had been missing `pgvector` entirely (added to pyproject,
|
|
# never re-locked), which the old install path had been silently papering
|
|
# over by resolving from pyproject instead.
|
|
- name: Install locked dependencies
|
|
env:
|
|
UV_PROJECT_ENVIRONMENT: /opt/venv
|
|
run: uv sync --locked --extra dev
|
|
|
|
- name: Run tests
|
|
# Integration tests (real Postgres) run in the `integration` job below.
|
|
run: /opt/venv/bin/python -m pytest tests/ -q -m "not integration"
|
|
|
|
# Real-Postgres lane (family rule 6). Exercises the async SQLAlchemy connection
|
|
# path the unit stubs can't reach — the un-awaited execution_options regression
|
|
# that made every VACUUM report 0/6 lived here. Like `test`, it runs for
|
|
# visibility and does NOT gate the build.
|
|
#
|
|
# Job key stays separator-free ("integration"): act_runner derives the service-
|
|
# container name from the (truncated) job display name and the discovery step
|
|
# filters `docker ps` by it. Service hostnames aren't routable on this runner,
|
|
# so the step resolves the Postgres container's bridge IP. No `name:` on purpose.
|
|
integration:
|
|
if: github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
|
|
runs-on: python-ci
|
|
container:
|
|
image: git.fabledsword.com/bvandeusen/ci-python:3.14
|
|
env:
|
|
# Config + the module engine read these at import time. DATABASE_URL itself
|
|
# is built from the discovered service IP in the run step.
|
|
SECRET_KEY: ci_integration_placeholder
|
|
services:
|
|
postgres:
|
|
# pgvector image so `alembic upgrade head` can run migration 0067
|
|
# (CREATE EXTENSION vector). PG17 — matches the prod/quickstart image.
|
|
image: pgvector/pgvector:pg17
|
|
env:
|
|
POSTGRES_USER: scribe
|
|
POSTGRES_PASSWORD: ci_integration
|
|
POSTGRES_DB: scribe_test
|
|
options: >-
|
|
--health-cmd "pg_isready -U scribe"
|
|
--health-interval 10s
|
|
--health-timeout 5s
|
|
--health-retries 10
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
# Same locked install as the unit lane — the two must agree on versions,
|
|
# or "unit green, integration red" stops being a signal about the code.
|
|
- name: Install locked dependencies
|
|
env:
|
|
UV_PROJECT_ENVIRONMENT: /opt/venv
|
|
run: uv sync --locked --extra dev
|
|
- name: Integration suite (resolve service IP, migrate, test)
|
|
run: |
|
|
set -eux
|
|
echo "=== container landscape (diagnostic for the name filter) ==="
|
|
docker ps -a --format '{{.ID}} {{.Image}} -> {{.Names}}'
|
|
PG=$(docker ps --filter "name=integration" --filter "ancestor=pgvector/pgvector:pg17" -q | head -n1)
|
|
test -n "$PG"
|
|
PG_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$PG")
|
|
test -n "$PG_IP"
|
|
export DATABASE_URL="postgresql+asyncpg://scribe:ci_integration@${PG_IP}:5432/scribe_test"
|
|
# Wait for Postgres to accept connections (busybox sh — the runner
|
|
# default — has no bash /dev/tcp, so use Python).
|
|
/opt/venv/bin/python - "$PG_IP" <<'PY'
|
|
import socket, sys, time
|
|
for _ in range(30):
|
|
try:
|
|
socket.create_connection((sys.argv[1], 5432), timeout=2).close()
|
|
break
|
|
except OSError:
|
|
time.sleep(1)
|
|
else:
|
|
sys.exit("postgres did not become reachable")
|
|
PY
|
|
# Real migrations build the schema; the maintenance tests then run
|
|
# VACUUM (ANALYZE) and read pg_stat_user_tables against it.
|
|
/opt/venv/bin/alembic upgrade head
|
|
/opt/venv/bin/python -m pytest tests/ -v -m integration
|
|
|
|
build:
|
|
name: Build & push image
|
|
needs: [typecheck, lint, test]
|
|
# Build on dev, main, and v* tag pushes. dev → :dev, main → :latest,
|
|
# tag → :latest + :<version>; every build also gets an immutable :<sha>.
|
|
if: github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
|
|
runs-on: python-ci
|
|
container:
|
|
image: git.fabledsword.com/bvandeusen/ci-python:3.14
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Generate image tags and version
|
|
id: tags
|
|
# POSIX `case` instead of bash `[[ ]]` because act_runner invokes
|
|
# `sh -e` (dash on the ci-python:3.14 image, which has no bash on
|
|
# the default PATH for /bin/sh). Previous `[[ ]]` form failed
|
|
# silently — only the SHA tag got appended, so :dev / :latest
|
|
# never updated in the registry and the deployed stack kept
|
|
# pulling stale images. Verified via `[[: not found` lines in
|
|
# the runner log on commit 2a374d9.
|
|
run: |
|
|
TAGS="${{ env.IMAGE }}:${{ github.sha }}"
|
|
BUILD_VERSION="dev"
|
|
case "${{ github.ref }}" in
|
|
refs/heads/dev)
|
|
TAGS="$TAGS,${{ env.IMAGE }}:dev"
|
|
;;
|
|
refs/heads/main)
|
|
# main IS the production line: publish :latest (plus the :<sha>
|
|
# set above). No separate :main tag.
|
|
TAGS="$TAGS,${{ env.IMAGE }}:latest"
|
|
BUILD_VERSION="main"
|
|
;;
|
|
refs/tags/*)
|
|
TAGS="$TAGS,${{ env.IMAGE }}:latest,${{ env.IMAGE }}:${{ github.ref_name }}"
|
|
BUILD_VERSION="${{ github.ref_name }}"
|
|
;;
|
|
esac
|
|
echo "value=$TAGS" >> $GITHUB_OUTPUT
|
|
echo "build_version=$BUILD_VERSION" >> $GITHUB_OUTPUT
|
|
|
|
- name: Free disk space
|
|
# Self-hosted runner housekeeping. Two-step cleanup:
|
|
# 1. Prune dangling containers/images globally (stops the runner
|
|
# from accumulating cruft from past failed builds).
|
|
# 2. Trim the BuildKit layer cache to a 5GB ceiling so the pip
|
|
# mount cache survives but old intermediate layers don't
|
|
# accumulate indefinitely.
|
|
run: |
|
|
docker system prune -af || true
|
|
docker builder prune --keep-storage 5g -f || true
|
|
|
|
- name: Set up Docker Buildx
|
|
uses: docker/setup-buildx-action@v4
|
|
|
|
- name: Log in to Forgejo registry
|
|
uses: docker/login-action@v4
|
|
with:
|
|
registry: ${{ env.REGISTRY }}
|
|
username: ${{ secrets.REGISTRY_USER }}
|
|
password: ${{ secrets.REGISTRY_TOKEN }}
|
|
|
|
- name: Build and push
|
|
uses: docker/build-push-action@v7
|
|
with:
|
|
context: .
|
|
push: true
|
|
provenance: false
|
|
tags: ${{ steps.tags.outputs.value }}
|
|
build-args: BUILD_VERSION=${{ steps.tags.outputs.build_version }}
|
|
# Registry-backed layer cache. Pull from :cache to prime
|
|
# BuildKit, push updated layers back to :cache so the next
|
|
# build starts warm even if the runner's local cache was
|
|
# pruned. `mode=max` exports all intermediate layers, not
|
|
# just the final image, which is what gives the ~80% speedup.
|
|
cache-from: type=registry,ref=${{ env.IMAGE }}:cache
|
|
cache-to: type=registry,ref=${{ env.IMAGE }}:cache,mode=max,ignore-error=true
|