fix: set keep_alive to 2h on all Ollama requests
Prevents models from sitting in VRAM indefinitely. Applies to both streaming chat calls and the non-streaming generate_completion path, as well as the startup warm-up request. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -169,7 +169,7 @@ def create_app() -> Quart:
|
||||
async with httpx.AsyncClient(timeout=300.0) as client:
|
||||
await client.post(
|
||||
f"{Config.OLLAMA_URL}/api/generate",
|
||||
json={"model": model, "prompt": "", "keep_alive": "30m"},
|
||||
json={"model": model, "prompt": "", "keep_alive": "2h"},
|
||||
)
|
||||
logger.info("Warmed model '%s' into VRAM", model)
|
||||
except Exception:
|
||||
|
||||
Reference in New Issue
Block a user