bench_ollama: PEP 723 inline script metadata for uv-run
Was failing with ModuleNotFoundError for httpx when run via system python — httpx is a project dep but isn't on the system interpreter's path. Adding PEP 723 script metadata + uv-run shebang means the script auto-resolves its deps in an ephemeral venv on every invocation, no project-venv setup required. Run with `uv run scripts/bench_ollama.py …` or directly via the shebang `./scripts/bench_ollama.py …`. `python scripts/bench_ollama.py …` still works only when httpx happens to be on the active interpreter. Docstring updated to reflect the running options. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
+20
-5
@@ -1,4 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# dependencies = ["httpx>=0.27"]
|
||||
# ///
|
||||
"""Benchmark Ollama models for FabledScribe chat / curator workloads.
|
||||
|
||||
Measures time-to-first-token, total wall time, and tokens/sec for each
|
||||
@@ -26,27 +30,38 @@ Think modes:
|
||||
contributes vs `off` on the same model).
|
||||
|
||||
Prerequisites:
|
||||
- `uv` installed (https://docs.astral.sh/uv/). The script's shebang
|
||||
uses `uv run --script`, which auto-creates an ephemeral venv with
|
||||
httpx — no project install required.
|
||||
- Ollama running and reachable at --server (default http://localhost:11434).
|
||||
- The models named in --models must already be pulled
|
||||
(`ollama pull qwen3:30b-a3b` etc).
|
||||
|
||||
Running:
|
||||
# Preferred — uv handles the venv:
|
||||
uv run scripts/bench_ollama.py --models qwen3:30b-a3b --scenario curator
|
||||
# Or via the shebang (the file is executable):
|
||||
./scripts/bench_ollama.py --models qwen3:30b-a3b --scenario curator
|
||||
# `python scripts/bench_ollama.py …` works ONLY if httpx is on the
|
||||
# interpreter's path — usually it isn't outside the project venv.
|
||||
|
||||
Usage examples:
|
||||
# Curator candidate on CPU, 3 runs each (default), one model:
|
||||
python scripts/bench_ollama.py --models qwen3:30b-a3b --scenario curator
|
||||
uv run scripts/bench_ollama.py --models qwen3:30b-a3b --scenario curator
|
||||
|
||||
# Chat candidate on GPU, against a remote server:
|
||||
python scripts/bench_ollama.py \\
|
||||
uv run scripts/bench_ollama.py \\
|
||||
--server http://dock02:11434 \\
|
||||
--models llama3.2:3b \\
|
||||
--scenario chat --num-gpu 99
|
||||
|
||||
# Compare two qwen curator candidates on CPU, 5 runs each:
|
||||
python scripts/bench_ollama.py \\
|
||||
uv run scripts/bench_ollama.py \\
|
||||
--models qwen3:30b-a3b,qwen3:32b \\
|
||||
--scenario curator --runs 5 --out bench-qwen-curator.md
|
||||
|
||||
# Cross-family curator comparison with think forced OFF (apples-to-apples):
|
||||
python scripts/bench_ollama.py \\
|
||||
uv run scripts/bench_ollama.py \\
|
||||
--models qwen3:30b-a3b,gemma2:27b,mistral-small:22b \\
|
||||
--scenario curator --think off --runs 3 \\
|
||||
--out bench-curator-no-think.md
|
||||
|
||||
Reference in New Issue
Block a user