fix(projects): route auto-summary through main model
Project auto-summaries were using the 3B background model, but the
task — synthesizing a coherent paragraph over ~10 notes — is well past
what 3B can do reliably. Evidence on dev: "doging conversation
hygiene", "MCPview). MCP).", trailing stray quotes, and hallucinated
topics ("AI regulation").
Route through the user's default chat model instead. Project summary
regeneration is rare (only when a project changes) so the KV cache
eviction cost on the main model is negligible.
Title generation, tag suggestions, and RSS classification continue to
use the background model — those tasks are within what 3B handles.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -119,12 +119,17 @@ async def generate_project_summary(user_id: int, project_id: int) -> None:
|
|||||||
f"Recent notes:\n{note_snippets}"
|
f"Recent notes:\n{note_snippets}"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Project summarization is synthesis over ~10 notes — too hard for the
|
||||||
|
# 3B background model, which produces incoherent, repetitive output
|
||||||
|
# (misspellings, broken parens, cliché filler). Route through the main
|
||||||
|
# chat model instead. These are rare events (only when a project
|
||||||
|
# changes), so the main model's KV cache eviction cost is negligible.
|
||||||
from fabledassistant.services.llm import generate_completion
|
from fabledassistant.services.llm import generate_completion
|
||||||
from fabledassistant.config import Config
|
from fabledassistant.config import Config
|
||||||
from fabledassistant.services.settings import get_setting
|
from fabledassistant.services.settings import get_setting
|
||||||
messages = [{"role": "user", "content": prompt}]
|
messages = [{"role": "user", "content": prompt}]
|
||||||
bg_model = await get_setting(user_id, "background_model", Config.OLLAMA_BACKGROUND_MODEL)
|
main_model = await get_setting(user_id, "default_model", Config.OLLAMA_MODEL)
|
||||||
summary = await generate_completion(messages, model=bg_model, max_tokens=400, num_ctx=2048)
|
summary = await generate_completion(messages, model=main_model, max_tokens=400, num_ctx=2048)
|
||||||
if not summary:
|
if not summary:
|
||||||
return
|
return
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user