Project-aware assist, link suggestions, project-scoped RAG, semantic search tool, SSE race fix

- Writing assistant: inject project notes as context (definition-tagged first), wikilink suggestions
- Link suggestions: server-side endpoint finds unlinked term occurrences, NoteEditorView sidebar panel
- Project-scoped RAG: ChatView ProjectSelector filters semantic+keyword search to selected project
- Semantic search tool: LLM search_notes upgraded to hybrid semantic (0.40 threshold) + keyword merge
- SSE race condition fix: drain remaining events after stream loop exits in chat.py and notes.py
- RAG_AUTO_SNIPPET raised 800→4000; sidebar include uses full note body; MAX_BODY_CHARS 8000→24000
- Enter-to-submit on writing assistant instruction textareas (note and task editors)
- DiffView: equal-line collapsing with 3-line context around changes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-06 14:02:54 -05:00
parent 9036dfd931
commit 48f070f773
22 changed files with 767 additions and 113 deletions
+47 -1
View File
@@ -12,7 +12,53 @@
> Include file-level details in the commit body when the change is non-trivial.
## Last Updated
2026-03-05Note editor sidebar layout, full-document writing assist, persistent drafts, version history, context window bump.
2026-03-06Project-aware writing assist, link suggestions, project-scoped RAG, semantic note search tool, SSE done-event race fix, RAG snippet increase, sidebar include full body, Enter-to-submit assist.
**SSE done-event race fix:** `routes/notes.py` and `routes/chat.py` — The SSE stream generator had a race condition where `yield buf.format_sse(event)` hands control back to the event loop, allowing the generation task to set `buf.state = COMPLETED` and append the `done` event between the last yield and the state check. The loop would then break without delivering the `done` event (containing `full_text`). Fixed by adding a final drain loop after the `while` exits in both stream generators. Added INFO-level logging to `run_assist_generation` showing `input_chars`, `output_chars`, event count, and done-event index.
**RAG snippet + sidebar include:** `services/llm.py``RAG_AUTO_SNIPPET` raised from 800 to 4000 chars. Sidebar-included notes (`include_note_ids`) now deliver full body (`n.body or ""`) instead of the previous 2000-char truncation, matching the paperclip/context_note behaviour.
**Assist `MAX_BODY_CHARS`:** `services/assist.py` — raised from 8000 to 24000 chars, safely within the 16384-token Ollama context (input + output headroom considered).
**DiffView equal-line collapsing:** `DiffView.vue` — unchanged lines are now collapsed into `⋯ N unchanged lines` markers when a run of 6+ equal lines exists, showing only 3 lines of context around each change (git-style). Makes reviewing large-document diffs practical.
**Enter-to-submit assist textarea:** Both `NoteEditorView.vue` and `TaskEditorView.vue` — writing assistant instruction textareas now submit on plain Enter (Shift+Enter inserts newline). Uses `@keydown.enter.exact.prevent` with `canSubmit` guard.
**Project-aware writing assist:** `services/assist.py``build_assist_messages()` accepts `context_notes: list[dict] | None`; when provided, injects a `--- Project Context ---` block into the user message (before the document) listing related note titles, tags, and body previews. Instructs the model to use `[[wikilinks]]` for referenced concepts. `routes/notes.py` `assist_route` — accepts `project_id` and `note_id`; fetches up to 5 project notes as context: `definition`-tagged notes first (up to 3, always injected for canonical term definitions), then recently-updated notes fill remaining slots. `composables/useAssist.ts` — accepts `projectId?: Ref<number | null>` third param; includes `note_id` and `project_id` in the POST body. Both `NoteEditorView.vue` and `TaskEditorView.vue` pass their project ref to `useAssist`.
**Link suggestions:** `routes/notes.py` — new `POST /api/notes/link-suggestions` endpoint. Accepts `{body, project_id, exclude_note_id}`. Fetches all project note titles (up to 200), runs regex to find titles appearing as plain text outside `[[...]]` spans, returns `[{note_id, title, count}]` sorted by occurrence count. `NoteEditorView.vue` — new "Link Suggestions" sidebar section: fetches suggestions on mount and debounced 2s after each body edit (when project is set). Per-term "Link" button applies `[[Title]]` wrapping to all unlinked occurrences; "Link all" applies all at once. Text replacement splits on `[[...]]` spans to avoid double-linking.
**Project-scoped RAG:** `services/embeddings.py` `semantic_search_notes()` — new `project_id` param filters the SQL join to `Note.project_id == project_id`. `services/notes.py` `search_notes_for_context()` — same `project_id` filter. `services/llm.py` `build_context()` — new `rag_project_id` param threaded to both search functions. `services/generation_task.py` `run_generation()` + `routes/chat.py` `send_message_route``rag_project_id` accepted from POST body and passed through. `stores/chat.ts` `sendMessage()` — new `ragProjectId` param included in POST. `ChatView.vue``ProjectSelector` added to chat sidebar under "Scope notes to project" label; when set, all RAG (semantic + keyword fallback) restricts to that project's notes.
**Semantic note search tool:** `services/tools.py` `search_notes` — upgraded from pure keyword to hybrid semantic+keyword. Runs `semantic_search_notes()` first (threshold 0.40, broader than auto-inject 0.60 since this is an intentional query); merges keyword results for any not already found. Results include a `relevance` field (percentage for semantic, `"keyword"` for text matches). Added `project` parameter (resolves by name, same pattern as `list_notes`). Preview extended to 300 chars. Tool description updated to explain semantic capability. `services/llm.py` tool guidance updated to direct the model to use `search_notes` for conceptual/thematic queries. `generation_task.py` status label updated to "Searching notes (semantic)".
**Previous session (2026-03-06):** Code review fixes: SSRF guard, config validation, race condition handling, tag autocomplete SQL, note versions now store tags, unmount cleanup, stale log string, PATCH due_date validation.
**SSRF protection:** `services/llm.py` — new `_is_private_url(url)` helper uses `socket.getaddrinfo` + `ipaddress` to block requests to loopback, private, link-local, reserved, and multicast addresses before fetching. `follow_redirects` set to `False` (redirects to internal IPs would bypass the check). All web research and search_web tool calls go through `fetch_url_content`, so this covers the full pipeline.
**Config validation:** `config.py` — new `Config.validate()` classmethod checks `OLLAMA_NUM_CTX`, `LOG_RETENTION_DAYS`, `IMAGE_MAX_BYTES`, `SMTP_PORT` bounds, and `BASE_URL` format when OIDC is enabled. `app.py``Config.validate()` called at the top of `create_app()` so misconfigurations surface immediately with clear messages instead of cryptic runtime errors.
**create_buffer race:** `routes/chat.py` — wrapped `create_buffer(conv_id, assistant_msg.id)` in try/except RuntimeError → returns 409 instead of 500 if two concurrent requests race past the buffer-running check (which is possible because there are await points between the GET check and the create call).
**Stale log message:** `services/generation_task.py` — "90s" → "180s" in the model-load timeout warning.
**Tag autocomplete SQL:** `services/notes.py``get_all_tags()` now pushes the `q` filter into the SQL query (`ILIKE`) and adds `LIMIT 100` on both filtered and unfiltered paths, replacing the previous Python-side filter over a full table scan.
**Note versions now store tags:** Migration `0023_add_tags_to_note_versions.py` adds `tags TEXT[] NOT NULL DEFAULT '{}'` to `note_versions`. `models/note_version.py` — added `tags` field; `to_dict()` includes it. `services/note_versions.py``create_version()` accepts optional `tags` param. `services/notes.py` — snapshots `old_tags` before update, passes them to `create_version`. Frontend: `NoteVersion` interface (in `HistoryPanel.vue` and `types/task.ts`) updated with `tags: string[]`. `HistoryPanel.vue``restore` emit updated to include tags as second argument. `NoteEditorView.vue` — restore handler now restores both body and tags.
**useAssist unmount cleanup:** `NoteEditorView.vue` — added `onUnmounted(() => assist.clearSelection())` which closes any in-flight SSE stream handle, preventing orphaned server connections when navigating away mid-generate.
**PATCH due_date validation:** `routes/notes.py``date.fromisoformat()` in the PATCH handler now wrapped in try/except `(ValueError, TypeError)`, returning 400 with a clear message instead of a 500.
**Previous session (2026-03-06 earlier):** SSE reconnection for chat and writing assist.
**SSE reconnection — chat:** `stores/chat.ts` — new `reconnectIfGenerating(convId)` function exported from the store. Checks `isStreamingConv` first (skips if stream already active). Finds the last assistant message with `status: 'generating'`, removes it from the messages array (the `done` event re-adds the final version), initialises stream state with `status: "Reconnecting..."`, then calls `_streamGeneration` which replays all buffered events from `last_event_id=-1`. If the buffer is >60s stale (already cleaned up), `_streamGeneration` exhausts its retries and falls back to `fetchConversation` to show the DB-saved final content. `ChatView.vue``reconnectIfGenerating` called (fire-and-forget, no await) in both `onMounted` and `watch(convId)` after the fetch step, covering both the page-refresh and navigate-away-and-back scenarios.
**SSE reconnection — writing assistant:** `composables/useAssist.ts` — extracted `_buildProposedFullBody()` helper. `submit()` now wraps the SSE connection in a retry loop (up to 3 retries, exponential back-off 1 s / 2 s / 4 s, capped at 5 s). Tracks `lastEventId`; reconnects use `?last_event_id=N` so only new chunks are received (no duplication). Loop breaks early on `done`, `error`, or if state left `'streaming'` (user cancelled). Falls back to accumulated text as draft on exhausted retries.
**Model load timeout:** `generation_task.py``wait_for_model_loaded` timeout raised from 90 s to 180 s.
**Previous session (2026-03-05):** Note editor sidebar layout, full-document writing assist, persistent drafts, version history, context window bump.
**Note editor sidebar layout:** `NoteEditorView.vue` fully refactored into two-column layout matching `TaskEditorView`. Right sidebar (280px) contains Project, Milestone, Tags + tag suggestions, and the Writing Assistant panel (always visible, no toggle). Mobile collapses to accordion. Removed `assistOpen` ref and `✨ Assist` toggle button. `InlineAssistPanel` removed from note editor.