fix(llm): default generate_completion num_ctx to Config.OLLAMA_NUM_CTX
Non-streaming generate_completion was the only LLM entry point that didn't default num_ctx — stream_chat and stream_chat_with_tools both fall back to Config.OLLAMA_NUM_CTX (16384). When a caller omitted the argument, Ollama silently used the model's default window (~4k on qwen3) and truncated the prompt. That footgun was masked by fallback paths in the research pipeline: _generate_outline's prompt carries ~12 sources × 2000 chars (~6k tokens) of source material plus a system prompt, so the prompt got chopped, the model never saw the sources, JSON parsing failed twice, and run_research_pipeline dropped into the single-note "monolith" fallback (research.py:251). The user reported chat 215 producing such a monolith note for a multi-source research topic. Two-layer fix: - Default num_ctx to Config.OLLAMA_NUM_CTX inside generate_completion, matching the streaming entry points. Any current or future caller that forgets the argument stops silently losing input. - Pin num_ctx=16384 explicitly in _generate_outline and _generate_executive_summary with comments pointing at the failure mode, so a refactor of the generate_completion default can't silently regress the research pipeline. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -317,9 +317,17 @@ async def generate_completion(
|
||||
num_ctx overrides the model's context window for this call only.
|
||||
"""
|
||||
last_exc: Exception | None = None
|
||||
options: dict = {"num_predict": max_tokens}
|
||||
if num_ctx is not None:
|
||||
options["num_ctx"] = num_ctx
|
||||
# Default num_ctx to Config.OLLAMA_NUM_CTX (matching stream_chat /
|
||||
# stream_chat_with_tools). Without this, Ollama silently uses the model's
|
||||
# default window (~4k on qwen3) and truncates anything longer. That is
|
||||
# how the research pipeline's outline step kept falling back to a single
|
||||
# monolith note: its 12-source prompt is ~6k tokens and was being chopped
|
||||
# before the model ever saw it. Non-streaming callers must not inherit
|
||||
# that footgun — if you truly want the model default, pass num_ctx=0.
|
||||
options: dict = {
|
||||
"num_predict": max_tokens,
|
||||
"num_ctx": num_ctx if num_ctx is not None else Config.OLLAMA_NUM_CTX,
|
||||
}
|
||||
for attempt in range(3):
|
||||
if attempt > 0:
|
||||
delay = 3.0 * attempt
|
||||
|
||||
Reference in New Issue
Block a user