bvandeusen/FabledScribe
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 50 Wiki Activity

bench_ollama: PEP 723 inline script metadata for uv-run

Browse Source 
Was failing with ModuleNotFoundError for httpx when run via system
python — httpx is a project dep but isn't on the system interpreter's
path. Adding PEP 723 script metadata + uv-run shebang means the script
auto-resolves its deps in an ephemeral venv on every invocation, no
project-venv setup required.

Run with `uv run scripts/bench_ollama.py …` or directly via the shebang
`./scripts/bench_ollama.py …`. `python scripts/bench_ollama.py …` still
works only when httpx happens to be on the active interpreter.

Docstring updated to reflect the running options.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Bryan Van Deusen
2026-05-21 14:53:18 -04:00
parent cf986b5097
commit 2d5d3ffdff
1 changed files with 20 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/bench_ollama.py
+20 -5
View File
@@ -1,4 +1,8 @@
#!/usr/bin/env python3 #!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = ["httpx>=0.27"]
# ///
"""Benchmark Ollama models for FabledScribe chat / curator workloads. """Benchmark Ollama models for FabledScribe chat / curator workloads.
Measures time-to-first-token, total wall time, and tokens/sec for each Measures time-to-first-token, total wall time, and tokens/sec for each
@@ -26,27 +30,38 @@ Think modes:
contributes vs `off` on the same model). contributes vs `off` on the same model).
Prerequisites: Prerequisites:
- `uv` installed (https://docs.astral.sh/uv/). The script's shebang
uses `uv run --script`, which auto-creates an ephemeral venv with
httpx — no project install required.
- Ollama running and reachable at --server (default http://localhost:11434). - Ollama running and reachable at --server (default http://localhost:11434).
- The models named in --models must already be pulled - The models named in --models must already be pulled
(`ollama pull qwen3:30b-a3b` etc). (`ollama pull qwen3:30b-a3b` etc).
Running:
# Preferred — uv handles the venv:
uv run scripts/bench_ollama.py --models qwen3:30b-a3b --scenario curator
# Or via the shebang (the file is executable):
./scripts/bench_ollama.py --models qwen3:30b-a3b --scenario curator
# `python scripts/bench_ollama.py …` works ONLY if httpx is on the
# interpreter's path — usually it isn't outside the project venv.
Usage examples: Usage examples:
# Curator candidate on CPU, 3 runs each (default), one model: # Curator candidate on CPU, 3 runs each (default), one model:
python scripts/bench_ollama.py --models qwen3:30b-a3b --scenario curator uv run scripts/bench_ollama.py --models qwen3:30b-a3b --scenario curator
# Chat candidate on GPU, against a remote server: # Chat candidate on GPU, against a remote server:
python scripts/bench_ollama.py \\ uv run scripts/bench_ollama.py \\
--server http://dock02:11434 \\ --server http://dock02:11434 \\
--models llama3.2:3b \\ --models llama3.2:3b \\
--scenario chat --num-gpu 99 --scenario chat --num-gpu 99
# Compare two qwen curator candidates on CPU, 5 runs each: # Compare two qwen curator candidates on CPU, 5 runs each:
python scripts/bench_ollama.py \\ uv run scripts/bench_ollama.py \\
--models qwen3:30b-a3b,qwen3:32b \\ --models qwen3:30b-a3b,qwen3:32b \\
--scenario curator --runs 5 --out bench-qwen-curator.md --scenario curator --runs 5 --out bench-qwen-curator.md
# Cross-family curator comparison with think forced OFF (apples-to-apples): # Cross-family curator comparison with think forced OFF (apples-to-apples):
python scripts/bench_ollama.py \\ uv run scripts/bench_ollama.py \\
--models qwen3:30b-a3b,gemma2:27b,mistral-small:22b \\ --models qwen3:30b-a3b,gemma2:27b,mistral-small:22b \\
--scenario curator --think off --runs 3 \\ --scenario curator --think off --runs 3 \\
--out bench-curator-no-think.md --out bench-curator-no-think.md