The curator scenario hardcoded think=true, which is qwen3-family-specific.
Non-qwen3 models silently ignore the field, so cross-family curator
comparisons were apples-to-oranges (qwen thinks, others don't).
New --think flag:
- auto (default): scenario-driven — chat=off, curator=on. Matches the
prior behaviour and the most common case.
- off: force disabled across all runs. Use for fair cross-family
comparison; aligns behaviour explicitly even though non-qwen models
would ignore think anyway.
- on: force enabled across all runs. Use to measure what think
contributes on the same model (paired runs: --think off then on).
Output markdown table now records the think mode used, so saved results
are self-documenting when you diff cross-server or cross-config.
Docstring + usage examples updated to reflect the qwen3 candidate set
the bench was originally tuned for.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusen
cf986b5097