Refactor AI Assist to background-task + buffer architecture
The assist flow previously tied the entire LLM generation to a single
POST request with no keepalives, causing NS_ERROR_NET_PARTIAL_TRANSFER
in Firefox when Hypercorn closed the connection during gaps between
chunks. This refactor decouples generation into a background task with
a buffer and a separate SSE stream — the same pattern used by chat.
- generation_buffer.py: Widen _buffers to support string keys, add
create/get/remove_assist_buffer() using "assist:{user_id}" keys,
fix cleanup log format for string keys
- generation_task.py: Add run_assist_generation() — lightweight
background task with no DB persistence or title generation
- notes.py: Replace single POST SSE route with POST /api/notes/assist
(returns 202) + GET /api/notes/assist/stream (SSE with 15s keepalives
and Last-Event-ID reconnection); 409 if already running
- useAssist.ts: Switch from apiStreamPost to apiPost + apiSSEStream
two-step pattern with named event mapping and stream handle cleanup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -134,3 +134,25 @@ async def run_generation(
|
||||
buf.state = GenerationState.ERRORED
|
||||
buf.finished_at = time.monotonic()
|
||||
buf.append_event("error", {"error": str(e)})
|
||||
|
||||
|
||||
async def run_assist_generation(
|
||||
buf: GenerationBuffer,
|
||||
messages: list[dict],
|
||||
model: str,
|
||||
) -> None:
|
||||
"""Stream LLM response for assist into buffer. No DB persistence."""
|
||||
try:
|
||||
async for chunk in stream_chat(messages, model, options={"num_predict": 4096}):
|
||||
buf.content_so_far += chunk
|
||||
buf.append_event("chunk", {"chunk": chunk})
|
||||
|
||||
buf.state = GenerationState.COMPLETED
|
||||
buf.finished_at = time.monotonic()
|
||||
buf.append_event("done", {"done": True, "full_text": buf.content_so_far})
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("Error in assist generation task")
|
||||
buf.state = GenerationState.ERRORED
|
||||
buf.finished_at = time.monotonic()
|
||||
buf.append_event("error", {"error": str(e)})
|
||||
|
||||
Reference in New Issue
Block a user