bvandeusen 4792e6459b perf(startup): prime Ollama KV cache with system prompt on warm-up ...

After loading each user's chat model into VRAM, send a minimal chat
request with the real system prompt (num_predict=1) to populate the
KV cache. The first real user message then only needs to process its
own tokens rather than the full ~5,600-token system prompt, reducing
cold-start TTFT from ~25s to <1s.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-04-03 10:47:59 -04:00

..

models

fix: rename Note.metadata → entity_meta (reserved by SQLAlchemy Declarative API)

2026-03-31 18:13:47 -04:00

routes

fix: briefing refresh — weather not populated + workspace conv bleed

2026-04-01 12:46:53 -04:00

services

feat(settings): add background model picker with KV cache performance warning

2026-04-03 10:44:08 -04:00

static

Initial commit: note-taking/task-tracking app with LLM integration scaffold

2026-02-09 23:35:44 -05:00

utils

Fix tag handling for multi-word tags

2026-02-26 06:05:58 -05:00

__init__.py

Initial commit: note-taking/task-tracking app with LLM integration scaffold

2026-02-09 23:35:44 -05:00

app.py

perf(startup): prime Ollama KV cache with system prompt on warm-up

2026-04-03 10:47:59 -04:00

auth.py

feat: add bearer token auth to _check_auth, falls back to session

2026-03-23 20:57:26 -04:00

config.py

perf(llm): route background tasks to dedicated model to preserve KV cache

2026-04-03 01:33:54 -04:00

rate_limit.py

security: fix 10 vulnerabilities from security audit

2026-03-29 00:37:13 -04:00