fix(chat): surface silent generations instead of empty bubbles
Qwen3:14b sometimes burns output tokens on tool-calling attempts whose emission doesn't parse into any field we read — eval_count > 0 but no thinking/content/tool_calls ever stream to the caller. Generation completes "successfully," the user sees an empty assistant bubble, and no error is logged. Seen in conv 220 today. Two safety rails: - stream_chat_with_tools now tracks whether it yielded anything; when Ollama's done frame reports eval_count > 0 with zero yields, log a warning including the last ~5 raw frames so the next occurrence leaves breadcrumbs for diagnosis. - run_generation checks the same post-condition after the tool loop exits and, if content is empty with no tool calls but output_tokens > 0, substitutes a visible fallback message and streams it as a chunk so the user gets something readable instead of a blank bubble. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -542,6 +542,29 @@ async def run_generation(
|
||||
# Strip model artifacts from final content
|
||||
buf.content_so_far = _TOOL_CALL_MARKER.sub("", buf.content_so_far)
|
||||
|
||||
# Silent-generation safety net: the model burned output tokens but
|
||||
# nothing landed in content or tool_calls (seen with qwen3:14b when
|
||||
# its tool-call emission doesn't parse). Show a visible fallback so
|
||||
# the user isn't staring at an empty bubble.
|
||||
if (
|
||||
not cancelled
|
||||
and not buf.content_so_far.strip()
|
||||
and not all_tool_calls
|
||||
and (timing.get("output_tokens") or 0) > 0
|
||||
):
|
||||
logger.warning(
|
||||
"Silent generation for conv %d: output_tokens=%s but empty content "
|
||||
"and no tool calls (model=%s)",
|
||||
conv_id, timing.get("output_tokens"), model,
|
||||
)
|
||||
fallback = (
|
||||
"I wasn't able to produce a usable response — the model generated "
|
||||
"tokens that couldn't be parsed as content or a tool call. "
|
||||
"Please try rephrasing, or try again."
|
||||
)
|
||||
buf.content_so_far = fallback
|
||||
buf.append_event("chunk", {"chunk": fallback})
|
||||
|
||||
# Final save
|
||||
logger.info("Generation complete for conv %d: content_length=%d, tool_calls=%d",
|
||||
conv_id, len(buf.content_so_far), len(all_tool_calls))
|
||||
|
||||
Reference in New Issue
Block a user