Compare commits

...

9 Commits

Author SHA1 Message Date
bvandeusen dcbe018297 Merge pull request 'Release v26.04.15.1 — Dynamic voice silence threshold + filled mic halo' (#35) from dev into main 2026-04-15 04:38:27 +00:00
bvandeusen 36fb71699b feat(voice): dynamic silence threshold + filled red mic halo
The web silence detector previously ran RMS over getByteFrequencyData
bytes (which are already dB-scaled) and re-log'd the result, producing
numbers that didn't line up with real dBFS. Combined with a static
-40 dB threshold that sat right on top of room ambient, silence
detection rarely fired and the user had to click stop manually.

- Rewrite useSilenceDetector to use getFloatTimeDomainData for honest
  linear RMS → dBFS.
- Silence threshold is now dynamic: track session peak dBFS and treat
  "silent" as 15 dB below peak. Auto-calibrates per mic/room.
- Grace period (1500 ms) at start so the user can begin speaking
  before checks arm; static -45 dB fallback until peak clears -25 dB
  so dead-silent sessions don't spin forever.
- Silence duration bumped 1500 → 2000 ms for breathing room.

Visual: detached red radial-gradient disc sits behind the mic in a
wrapper; the button no longer scales, so the white mic icon stays
legible on top while the halo grows from 1x to ~2.6x with live
amplitude. Much more obvious "you are live" signal than the prior
subtle box-shadow pulse.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 00:26:52 -04:00
bvandeusen 027fbec606 Merge pull request 'Release v26.04.14.3 — Live amplitude mic pulse' (#34) from dev into main 2026-04-15 02:54:42 +00:00
bvandeusen 730dbfaf7b feat(voice): pulse mic button with live amplitude
The mic button now scales and glows proportional to the silence
detector's RMS amplitude instead of sitting static. Gives obvious
feedback that audio is actually being picked up — silence still
breathes via a 0.1 floor, loud input caps at ~1.18x scale so it
doesn't punch through the input bar.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 22:46:37 -04:00
bvandeusen 267a975455 Merge pull request 'Release v26.04.14.2 — Discuss failure mode fixes' (#33) from dev into main 2026-04-15 01:45:36 +00:00
bvandeusen 7bd1548f71 fix(discuss): hard-fail empty articles and skip RAG on seed turn
Discuss flow was hallucinating unrelated content when article
extraction returned empty or RAG pulled in orphan notes that looked
more relevant than the generic seed prompt.

- seed_article_discussion raises EmptyArticleError on empty body;
  briefing and /news routes return 422 instead of staging an empty
  synthetic tool result.
- build_context skips RAG auto-injection when user_message matches
  ARTICLE_DISCUSS_SEED so the article IS the context on turn one;
  follow-up turns keep RAG on.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 18:13:17 -04:00
bvandeusen 9f3b3450fa Merge pull request 'Release v26.04.14.1' (#32) from dev into main 2026-04-14 12:02:13 +00:00
bvandeusen ba90ad8132 feat(article-discuss): unify /news + briefing entry points, persist summaries to RAG
Both the /news discuss button and the briefing discuss button now call a
shared seed_article_discussion() helper that stages the synthetic
read_article tool exchange and the conversational seed prompt — behavior
stays byte-identical across entry points. /news also auto-starts
generation so the chat screen lands on an in-flight stream.

First assistant reply in a seeded article conversation is persisted as a
Note (tags: article-summary + article topics) and backlinked via
rss_items.discussion_note_id, so the knowledge base stops being amnesiac
about articles the user has engaged with.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 07:54:24 -04:00
bvandeusen 9157740069 ci: gate typecheck/lint/test on ref so main merge commits skip work
Forgejo Actions doesn't consistently honor `on.push.branches` as a
filter — the merge commit landing on main after a release PR was still
triggering the full CI workflow on a SHA that had already passed on
dev, burning ~1 minute of runner time for no reason.

The `build` job already had a ref guard, so no duplicate images were
being pushed, but typecheck/lint/test ran unconditionally. Add the
same guard (`refs/heads/dev` or `refs/tags/v*`) to those three so main
pushes trigger the workflow but every job skips immediately with zero
runner time.

Also tightened the build job's tag filter from `refs/tags/` to
`refs/tags/v` for consistency with the new guards and to avoid ever
building from a non-version tag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 23:40:41 -04:00
12 changed files with 504 additions and 124 deletions
+23 -9
View File
@@ -1,8 +1,12 @@
# CI runs first; build only proceeds if all checks pass.
#
# Push to any branch: typecheck + lint + test
# Push to dev: gates + build :dev + :<sha>
# Tag v* (release): gates + build :latest + :<sha> + :<version>
# Push to dev: typecheck + lint + test + build :dev + :<sha>
# Tag v* (release): typecheck + lint + test + build :latest + :<sha> + :<version>
#
# main pushes are NOT gated here: a merge to main only happens after
# dev has already passed CI, and the release tag is the sole trigger
# for a production image. Re-running CI on the merge commit just burns
# runner time without changing the outcome.
#
# To cut a release:
# Create a release via the Forgejo UI on main with a v* tag name.
@@ -11,6 +15,13 @@
# PRs aren't triggered on purpose — this is a solo dev→main flow, so
# gating on branch push is already enough.
#
# NOTE on the `if:` guards below: Forgejo Actions does not consistently
# honor `on.push.branches` as a filter — merge commits landing on main
# still trigger the workflow, producing redundant runs on the same SHA
# that was already gated on dev. Every job therefore repeats the ref
# check so main pushes trigger the workflow but every job skips
# immediately (no runner time, no duplicate work).
#
# Required secrets (repo → Settings → Secrets → Actions):
# REGISTRY_USER — your Forgejo username
# REGISTRY_TOKEN — Forgejo PAT with write:packages scope
@@ -18,7 +29,7 @@ name: CI & Build
on:
push:
branches: [dev, main]
branches: [dev]
tags: ["v*"]
paths:
- "src/**"
@@ -51,6 +62,8 @@ env:
jobs:
typecheck:
name: TypeScript typecheck
# Skip on main merge-commit pushes — see workflow header comment.
if: github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/v')
runs-on: ci-runner
steps:
- uses: actions/checkout@v6
@@ -77,6 +90,7 @@ jobs:
lint:
name: Python lint
if: github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/v')
runs-on: ci-runner
steps:
- uses: actions/checkout@v6
@@ -88,6 +102,7 @@ jobs:
test:
name: Python tests
if: github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/v')
runs-on: ci-runner
steps:
- uses: actions/checkout@v6
@@ -118,11 +133,10 @@ jobs:
name: Build & push image
needs: [typecheck, lint, test]
# Build on dev branch pushes and version tag pushes only.
# main branch pushes run the gates for safety but do not build —
# the release tag (v*) is the sole trigger for a production image.
if: |
github.event_name == 'push' &&
(github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/'))
# Mirrors the ref guard on the gate jobs above — main merge-commit
# pushes skip here too, so no production image is ever built from a
# raw main push (only from the v* tag the release creates).
if: github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/v')
runs-on: ci-runner
permissions:
contents: read
@@ -0,0 +1,38 @@
"""Link rss_items to their discussion-summary note.
Revision ID: 0039
Revises: 0038
Create Date: 2026-04-14
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
revision = "0039"
down_revision = "0038"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"rss_items",
sa.Column("discussion_note_id", sa.BigInteger(), nullable=True),
)
op.create_foreign_key(
"fk_rss_items_discussion_note_id",
"rss_items",
"notes",
["discussion_note_id"],
["id"],
ondelete="SET NULL",
)
def downgrade() -> None:
op.drop_constraint(
"fk_rss_items_discussion_note_id", "rss_items", type_="foreignkey"
)
op.drop_column("rss_items", "discussion_note_id")
+3 -1
View File
@@ -426,7 +426,9 @@ export async function deleteRssReaction(rssItemId: number): Promise<void> {
return apiDelete(`/api/briefing/rss-reactions/${rssItemId}`);
}
export async function openArticleInChat(itemId: number): Promise<{ conversation_id: number }> {
export async function openArticleInChat(
itemId: number,
): Promise<{ conversation_id: number; assistant_message_id?: number; status?: string }> {
return apiPost(`/api/chat/from-article/${itemId}`, {});
}
+69 -12
View File
@@ -114,6 +114,20 @@ const transcribingVoice = ref(false)
const recorder = useVoiceRecorder()
const silenceDetector = useSilenceDetector()
// Live mic halo. A solid red disc sits behind the mic button and scales
// with `silenceDetector.amplitude` (0..1 RMS, already amplified for
// visibility). The button itself stays put so the mic icon is always
// legible on top. 0.2 baseline breathes on silence; loud speech drives
// the disc out to ~2.6× the button size with a softer radial fade.
const micGlowStyle = computed(() => {
if (!recorder.recording.value) return { display: 'none' }
const pulse = 0.2 + Math.min(silenceDetector.amplitude.value, 1) * 0.8
return {
transform: `translate(-50%, -50%) scale(${1 + pulse * 1.6})`,
opacity: String(0.45 + pulse * 0.4),
}
})
async function toggleVoice() {
if (transcribingVoice.value) return
if (recorder.recording.value) {
@@ -230,23 +244,25 @@ defineExpose({ focus, prefill })
class="input-textarea"
></textarea>
<!-- PTT mic -->
<button
v-if="voiceEnabled"
class="btn-icon btn-mic"
:class="{ 'mic-recording': recorder.recording.value, 'mic-transcribing': transcribingVoice }"
@click.prevent="toggleVoice"
:disabled="transcribingVoice || !store.chatReady"
:title="recorder.recording.value ? 'Click to stop (or wait for silence)' : 'Click to speak'"
:aria-label="recorder.recording.value ? 'Stop recording' : 'Start recording'"
>
<!-- PTT mic (with live amplitude halo behind) -->
<div v-if="voiceEnabled" class="mic-wrapper">
<div class="mic-glow" :style="micGlowStyle" aria-hidden="true"></div>
<button
class="btn-icon btn-mic"
:class="{ 'mic-recording': recorder.recording.value, 'mic-transcribing': transcribingVoice }"
@click.prevent="toggleVoice"
:disabled="transcribingVoice || !store.chatReady"
:title="recorder.recording.value ? 'Click to stop (or wait for silence)' : 'Click to speak'"
:aria-label="recorder.recording.value ? 'Stop recording' : 'Start recording'"
>
<svg v-if="!transcribingVoice" width="17" height="17" viewBox="0 0 24 24" fill="currentColor">
<path d="M12 14c1.66 0 3-1.34 3-3V5c0-1.66-1.34-3-3-3S9 3.34 9 5v6c0 1.66 1.34 3 3 3zm-1-9c0-.55.45-1 1-1s1 .45 1 1v6c0 .55-.45 1-1 1s-1-.45-1-1V5zm6 6c0 2.76-2.24 5-5 5s-5-2.24-5-5H5c0 3.53 2.61 6.43 6 6.92V21h2v-3.08c3.39-.49 6-3.39 6-6.92h-2z"/>
</svg>
<svg v-else width="17" height="17" viewBox="0 0 24 24" fill="currentColor">
<circle cx="12" cy="12" r="8" opacity="0.35"/><circle cx="12" cy="12" r="4"/>
</svg>
</button>
</button>
</div>
<!-- Abort (streaming) or Send -->
<button
@@ -343,9 +359,50 @@ defineExpose({ focus, prefill })
.btn-icon:hover { opacity: 1; }
.btn-icon:disabled { opacity: 0.3; cursor: default; }
.btn-mic.mic-recording { opacity: 1; color: #ef4444; }
.btn-mic.mic-recording {
opacity: 1;
/* White icon sits on top of the red halo so it stays legible at any
pulse size. */
color: #fff;
position: relative;
z-index: 1;
}
.btn-mic.mic-transcribing { opacity: 0.5; }
/* Mic wrapper + live amplitude halo. The glow is a filled red disc
absolutely positioned behind the button, scaled/faded by the
computed `micGlowStyle` so the user gets unmistakable feedback
that their voice is being picked up while the mic icon itself
stays put and readable on top. */
.mic-wrapper {
position: relative;
display: inline-flex;
align-items: center;
justify-content: center;
flex-shrink: 0;
}
.mic-glow {
position: absolute;
top: 50%;
left: 50%;
width: 28px;
height: 28px;
border-radius: 50%;
background: radial-gradient(
circle,
rgba(239, 68, 68, 0.9) 0%,
rgba(239, 68, 68, 0.6) 55%,
rgba(239, 68, 68, 0) 100%
);
transform: translate(-50%, -50%) scale(1);
transform-origin: center;
pointer-events: none;
/* Smooth between amplitude samples (~100ms) so the pulse feels continuous
rather than stepping. */
transition: transform 0.12s ease-out, opacity 0.12s ease-out;
z-index: 0;
}
.note-picker-wrapper { position: relative; }
.note-picker-dropdown {
position: absolute;
+66 -14
View File
@@ -1,15 +1,54 @@
import { ref, readonly } from 'vue'
export interface SilenceDetectorOptions {
thresholdDb?: number // default -40
silenceDurationMs?: number // default 1500
minRecordingMs?: number // default 500
/**
* Absolute fallback silence threshold in dBFS, used during the first
* moments before any real speech has been observed. Once the session peak
* clears `dynamicArmDb` we switch to the dynamic threshold instead.
*/
fallbackThresholdDb?: number // default -45
/** How many dB below the session peak counts as "silent" once armed. */
dropFromPeakDb?: number // default 15
/**
* Session peak must reach this level (dBFS) before dynamic thresholding
* kicks in. Until then the fallback threshold is used so we don't lock
* onto a noise-floor peak.
*/
dynamicArmDb?: number // default -25
/** How long to wait at the start of recording before running silence checks. */
graceMs?: number // default 1500
silenceDurationMs?: number // default 2000
minRecordingMs?: number // default 500
}
/**
* Mic silence detector + live amplitude signal.
*
* Uses `getFloatTimeDomainData()` for honest linear RMS in [-1, 1] space,
* then derives dBFS for the silence threshold. The previous implementation
* ran RMS over `getByteFrequencyData` bytes — those bytes are already a
* dB-scaled quantity, so taking their RMS and re-log'ing it produced
* numbers that didn't line up with real dBFS and made any static threshold
* unpredictable.
*
* Silence threshold is dynamic: we track the session peak dBFS and treat
* "silent" as "current level is at least [dropFromPeakDb] below peak."
* This auto-calibrates to whatever mic and room the user is on. Until the
* peak climbs above [dynamicArmDb] we fall back to a conservative static
* threshold so a quiet room doesn't spin forever. A grace period at the
* start of the recording gives the user time to begin speaking before
* silence checks arm.
*
* `amplitude` is the raw linear RMS scaled ×5 and clamped to 1 so quiet
* speech still visibly moves UI that binds to it.
*/
export function useSilenceDetector(options: SilenceDetectorOptions = {}) {
const {
thresholdDb = -40,
silenceDurationMs = 1500,
fallbackThresholdDb = -45,
dropFromPeakDb = 15,
dynamicArmDb = -25,
graceMs = 1500,
silenceDurationMs = 2000,
minRecordingMs = 500,
} = options
@@ -18,31 +57,43 @@ export function useSilenceDetector(options: SilenceDetectorOptions = {}) {
let intervalId: ReturnType<typeof setInterval> | null = null
let silenceMs = 0
let startedAt = 0
let peakDb = -100
function start(stream: MediaStream, onSilence: () => void): void {
stop()
audioCtx = new AudioContext()
// Some browsers start AudioContext in "suspended" state — resume so
// getByteFrequencyData returns real values instead of all zeros.
// the analyser returns real values instead of all zeros.
audioCtx.resume().catch(() => {})
const source = audioCtx.createMediaStreamSource(stream)
const analyser = audioCtx.createAnalyser()
analyser.fftSize = 256
analyser.fftSize = 1024
source.connect(analyser)
const data = new Uint8Array(analyser.frequencyBinCount)
const samples = new Float32Array(analyser.fftSize)
silenceMs = 0
startedAt = Date.now()
peakDb = -100
intervalId = setInterval(() => {
analyser.getByteFrequencyData(data)
const rms = Math.sqrt(data.reduce((s, v) => s + v * v, 0) / data.length) / 255
amplitude.value = rms
analyser.getFloatTimeDomainData(samples)
let sumSq = 0
for (let i = 0; i < samples.length; i++) sumSq += samples[i] * samples[i]
const rms = Math.sqrt(sumSq / samples.length)
amplitude.value = Math.min(rms * 5, 1)
const db = rms > 0 ? 20 * Math.log10(rms) : -100
if (db < thresholdDb) {
const db = rms > 1e-7 ? 20 * Math.log10(rms) : -100
if (db > peakDb) peakDb = db
const elapsed = Date.now() - startedAt
if (elapsed < graceMs) return
const threshold =
peakDb > dynamicArmDb ? peakDb - dropFromPeakDb : fallbackThresholdDb
if (db < threshold) {
silenceMs += 100
if (silenceMs >= silenceDurationMs && Date.now() - startedAt >= minRecordingMs) {
if (silenceMs >= silenceDurationMs && elapsed >= minRecordingMs) {
stop()
onSilence()
}
@@ -63,6 +114,7 @@ export function useSilenceDetector(options: SilenceDetectorOptions = {}) {
}
amplitude.value = 0
silenceMs = 0
peakDb = -100
}
return { amplitude: readonly(amplitude), start, stop }
+8 -1
View File
@@ -1,6 +1,6 @@
from datetime import datetime, timezone
from sqlalchemy import ARRAY, DateTime, ForeignKey, Index, Integer, Text, UniqueConstraint
from sqlalchemy import ARRAY, BigInteger, DateTime, ForeignKey, Index, Integer, Text, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column, relationship
from fabledassistant.models import Base
@@ -66,6 +66,13 @@ class RssItem(Base):
classified_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
# Note persisting the first-click discussion summary. Set by the article
# discussion pipeline once the seeded chat completes its first assistant
# reply; links back into RAG so re-discussing the same article lands the
# prior summary in context.
discussion_note_id: Mapped[int | None] = mapped_column(
BigInteger, ForeignKey("notes.id", ondelete="SET NULL"), nullable=True
)
feed: Mapped["RssFeed"] = relationship(back_populates="items")
+13 -46
View File
@@ -532,54 +532,21 @@ async def discuss_article(item_id: int):
if get_buffer(conv_id) is not None:
return jsonify({"error": "Generation already in progress"}), 409
# Three-layer cache: context_prepared (post-map-reduce) → content_full
# (raw trafilatura) → fresh fetch. Only the first miss pays the fetch
# cost; only a large uncached article pays the map-reduce cost. Repeat
# clicks on the same article skip straight to the chat turn.
from fabledassistant.services.article_context import prepare_article_context
from fabledassistant.services.rss import get_or_fetch_full_article
# Shared helper handles the three-layer cache (context_prepared
# content_full → fresh fetch), writes the synthetic read_article tool
# exchange and the conversational seed user prompt into the conversation.
# The /news from-article route calls the same helper so behavior stays
# byte-identical across entry points.
from fabledassistant.services.article_context import (
EmptyArticleError,
seed_article_discussion,
)
model = await get_setting(uid, "default_model", "") or ""
if item.context_prepared:
article_content = item.context_prepared
else:
raw_body = await get_or_fetch_full_article(item) or item.content or ""
article_content = await prepare_article_context(
item.title or "", item.url, raw_body, model,
)
if article_content:
async with async_session() as session:
fresh = await session.get(RssItem, item.id)
if fresh is not None:
fresh.context_prepared = article_content
await session.commit()
# Store synthetic assistant message with read_article tool result
synthetic_tool_calls = [{
"function": "read_article",
"arguments": {"url": item.url},
"result": {
"success": True,
"type": "article_content",
"url": item.url,
"content": article_content,
"truncated": False,
},
}]
await add_message(conv_id, "assistant", "", status="complete", tool_calls=synthetic_tool_calls)
# Conversational seed — invites a real discussion rather than asking for
# a one-shot summary. The model sees the article context in the tool
# result above and responds to this user turn as the start of an ongoing
# conversation the user will steer with follow-ups.
discuss_prompt = (
"I want to talk about this article. Start with a substantive summary "
"of what it's arguing and the key evidence it uses, then tell me what "
"stood out to you or seems worth pushing back on. I'll ask follow-ups "
"from there."
)
await add_message(conv_id, "user", discuss_prompt)
try:
discuss_prompt = await seed_article_discussion(conv_id, item, model)
except EmptyArticleError as e:
return jsonify({"error": str(e)}), 422
# Reload conversation with fresh messages to build history
conv = await get_conversation(uid, conv_id)
+56 -25
View File
@@ -510,47 +510,78 @@ async def delete_model_route():
@chat_bp.route("/from-article/<int:item_id>", methods=["POST"])
@login_required
async def create_conversation_from_article(item_id: int):
"""Create a chat conversation seeded with an RSS article's content."""
"""Create a chat conversation seeded for article discussion and auto-run.
Mirrors the briefing ``discuss_article`` route: creates a fresh
conversation, stages the shared synthetic read_article exchange + seed
prompt, then kicks off generation so the client lands on an in-flight
stream. The Flutter and web chat screens reconnect to the running buffer
on mount.
"""
from sqlalchemy import select as _select
from fabledassistant.models import async_session as _async_session
from fabledassistant.models.conversation import Message
from fabledassistant.models.rss_feed import RssItem, RssFeed
from fabledassistant.services.article_context import (
EmptyArticleError,
seed_article_discussion,
)
uid = get_current_user_id()
async with _async_session() as session:
result = await session.execute(
_select(RssItem, RssFeed.title.label("feed_title"))
_select(RssItem)
.join(RssFeed, RssItem.feed_id == RssFeed.id)
.where(RssItem.id == item_id, RssFeed.user_id == uid)
)
row = result.first()
item = result.scalars().first()
if row is None:
if item is None:
return jsonify({"error": "Article not found"}), 404
item, feed_title = row
conv_title = (item.title or "Article discussion")[:80]
conv = await create_conversation(uid, title=conv_title, conversation_type="chat")
from fabledassistant.services.rss import _fetch_full_article
source = feed_title or "News"
content_body = (await _fetch_full_article(item.url) if item.url else None) or (item.content or "").strip()
seeded_text = f"**{source}**\n\n**{item.title}**"
if content_body:
seeded_text += f"\n\n{content_body}"
if item.url:
seeded_text += f"\n\nSource: {item.url}"
model = await get_setting(uid, "default_model", "") or Config.OLLAMA_MODEL
try:
discuss_prompt = await seed_article_discussion(conv.id, item, model)
except EmptyArticleError as e:
# Roll back the empty conversation so the user doesn't end up with a
# phantom entry in their chat list.
try:
await delete_conversation(uid, conv.id)
except Exception:
logger.warning("Failed to clean up empty article conversation %s", conv.id)
return jsonify({"error": str(e)}), 422
async with _async_session() as session:
msg = Message(
conversation_id=conv.id,
role="assistant",
content=seeded_text,
msg_metadata={"rss_item_ids": [item_id]},
)
session.add(msg)
await session.commit()
# Reload conversation so we see the two messages the helper just added.
conv = await get_conversation(uid, conv.id)
assert conv is not None
return jsonify({"conversation_id": conv.id}), 201
history: list[dict] = []
for msg in conv.messages:
if msg.role == "system":
continue
msg_dict: dict = {"role": msg.role, "content": msg.content or ""}
if msg.tool_calls:
msg_dict["tool_calls"] = [
{"function": {"name": tc["function"], "arguments": tc["arguments"]}}
for tc in msg.tool_calls
]
history.append(msg_dict)
for tc in msg.tool_calls:
history.append({"role": "tool", "content": json.dumps(tc.get("result", {}))})
else:
history.append(msg_dict)
assistant_msg = await add_message(conv.id, "assistant", "", status="generating")
buf = create_buffer(conv.id, assistant_msg.id)
asyncio.create_task(run_generation(
buf, history, model, uid, conv.id, conv.title or "", discuss_prompt,
))
return jsonify({
"conversation_id": conv.id,
"assistant_message_id": assistant_msg.id,
"status": "generating",
}), 202
+114 -5
View File
@@ -1,15 +1,21 @@
"""Prepare article bodies as conversation-ready context.
Used by the briefing ``discuss-article`` flow. A raw trafilatura extraction
is often too large to drop whole into a chat history without eating the
context window, so this module runs a map-reduce step over oversized
articles and returns a compact, structured context that still preserves the
article's meaning across sections.
Used by the briefing ``discuss-article`` flow and the ``/news`` discuss button.
A raw trafilatura extraction is often too large to drop whole into a chat
history without eating the context window, so this module runs a map-reduce
step over oversized articles and returns a compact, structured context that
still preserves the article's meaning across sections.
Small articles pass through unchanged — map-reduce only fires when the raw
body exceeds CHAR_BUDGET. The output is cached on ``rss_items.context_prepared``
by the caller, so repeat discuss-clicks on the same article skip this work
entirely.
The module also owns ``seed_article_discussion``, the shared routine that
stages a synthetic ``read_article`` tool exchange plus a conversational seed
prompt into a conversation. Both the briefing and ``/news`` entry points call
it so the two flows stay byte-identical — the only thing that differs between
them is whether the conversation already existed or was freshly created.
"""
from __future__ import annotations
@@ -18,6 +24,9 @@ import asyncio
import logging
import re
from fabledassistant.models import async_session
from fabledassistant.models.rss_feed import RssItem
from fabledassistant.services.chat import add_message
from fabledassistant.services.llm import generate_completion
logger = logging.getLogger(__name__)
@@ -159,3 +168,103 @@ async def prepare_article_context(
for i, summary in enumerate(summaries)
]
return header + "\n\n".join(parts)
# Conversational seed prompt for article discussions. Kept here so both the
# briefing and /news entry points use the exact same wording. See
# feedback_discuss_prompt_style memory: numbered checklists produce
# assignment-completion responses; this conversational seed opens a dialogue.
ARTICLE_DISCUSS_SEED = (
"I want to talk about this article. Start with a substantive summary "
"of what it's arguing and the key evidence it uses, then tell me what "
"stood out to you or seems worth pushing back on. I'll ask follow-ups "
"from there."
)
class EmptyArticleError(Exception):
"""Raised when an article has no extractable body text.
Callers (the briefing and /news discuss routes) map this to a 422 so the
user sees a clear error instead of a hallucinated summary built from an
empty synthetic tool result.
"""
async def seed_article_discussion(
conv_id: int,
item: RssItem,
model: str,
) -> str:
"""Stage the synthetic read_article tool exchange + conversational seed.
Used by both the briefing ``discuss_article`` route and the ``/news``
``from-article`` conversation creator. Handles the three-layer cache
(``context_prepared`` → ``content_full`` → fresh fetch) and inserts two
messages into ``conv_id``:
1. An assistant message with a synthetic ``read_article`` tool_call whose
``result.content`` carries the prepared article context. The message
also carries ``msg_metadata={"rss_item_id": ...}`` so the post-generation
hook in ``generation_task.py`` can locate it and persist the first
reply as a discussion-summary Note.
2. A user message with the shared conversational seed prompt.
Returns the seed prompt string so callers can pass it to ``run_generation``
as ``user_content``.
"""
# Avoid circulars: rss helper imports article_context indirectly nowhere,
# but keep this local for symmetry with the route-level imports it
# replaces.
from fabledassistant.services.rss import get_or_fetch_full_article
if item.context_prepared:
article_content = item.context_prepared
else:
raw_body = await get_or_fetch_full_article(item) or item.content or ""
if not raw_body.strip():
# Hard-fail rather than stage an empty synthetic tool result.
# An empty `content` field silently tells the model "the article
# has nothing in it" and it confabulates from RAG/history. Better
# to surface a clean error to the user.
logger.warning(
"Article discussion aborted: empty body for rss_item %s (%s)",
item.id, item.url,
)
raise EmptyArticleError(
"Couldn't extract any readable text from this article."
)
article_content = await prepare_article_context(
item.title or "", item.url, raw_body, model,
)
if not article_content.strip():
raise EmptyArticleError(
"Couldn't extract any readable text from this article."
)
async with async_session() as session:
fresh = await session.get(RssItem, item.id)
if fresh is not None:
fresh.context_prepared = article_content
await session.commit()
synthetic_tool_calls = [{
"function": "read_article",
"arguments": {"url": item.url},
"result": {
"success": True,
"type": "article_content",
"url": item.url,
"content": article_content,
"truncated": False,
},
}]
await add_message(
conv_id,
"assistant",
"",
status="complete",
tool_calls=synthetic_tool_calls,
msg_metadata={"rss_item_id": item.id, "article_seed": True},
)
await add_message(conv_id, "user", ARTICLE_DISCUSS_SEED)
return ARTICLE_DISCUSS_SEED
+3
View File
@@ -187,6 +187,7 @@ async def add_message(
context_note_id: int | None = None,
status: str | None = None,
tool_calls: list | None = None,
msg_metadata: dict | None = None,
) -> Message:
async with async_session() as session:
kwargs: dict = dict(
@@ -199,6 +200,8 @@ async def add_message(
kwargs["status"] = status
if tool_calls is not None:
kwargs["tool_calls"] = tool_calls
if msg_metadata is not None:
kwargs["msg_metadata"] = msg_metadata
msg = Message(**kwargs)
session.add(msg)
# Touch conversation updated_at
@@ -36,6 +36,85 @@ _TOOL_CALL_MARKER = re.compile(r"^\s*\[TOOL_CALLS\]\s*", re.IGNORECASE)
DB_FLUSH_INTERVAL = 5.0 # seconds between partial DB flushes
async def _maybe_save_article_discussion_note(
user_id: int, conv_id: int, reply_content: str,
) -> None:
"""Persist a seeded article-discussion's first reply as a Note.
Fires after ``run_generation`` completes. Looks for a synthetic
read_article seed message on the conversation; if found AND the linked
``rss_items`` row has no ``discussion_note_id`` yet, saves ``reply_content``
as a Note, tags it, and writes the backlink. Subsequent discuss clicks on
the same article are a no-op (already linked).
Failures are logged and swallowed — the chat UI should never break because
Note persistence hit a snag.
"""
try:
if not reply_content or not reply_content.strip():
return
from sqlalchemy import select as _select
from fabledassistant.models.conversation import Message as _Message
from fabledassistant.models.rss_feed import RssItem as _RssItem
from fabledassistant.services.notes import create_note
async with async_session() as session:
result = await session.execute(
_select(_Message)
.where(_Message.conversation_id == conv_id)
.order_by(_Message.id.asc())
)
messages = result.scalars().all()
seed_meta = None
for m in messages:
meta = m.msg_metadata or {}
if meta.get("article_seed") and meta.get("rss_item_id"):
seed_meta = meta
break
if seed_meta is None:
return
item_id = int(seed_meta["rss_item_id"])
item = await session.get(_RssItem, item_id)
if item is None or item.discussion_note_id is not None:
return
article_title = (item.title or "Untitled article").strip()
article_url = item.url
article_topics = list(item.topics or [])
note_title = f"Article: {article_title}"[:200]
body_parts = [f"**Source:** {article_url}"] if article_url else []
body_parts.append(reply_content.strip())
note_body = "\n\n".join(body_parts)
tags = ["article-summary"] + [t for t in article_topics if t]
note = await create_note(
user_id=user_id,
title=note_title,
body=note_body,
tags=tags,
entity_meta={
"source": "article_discussion",
"rss_item_id": item_id,
"url": article_url,
"conversation_id": conv_id,
},
)
async with async_session() as session:
fresh = await session.get(_RssItem, item_id)
if fresh is not None and fresh.discussion_note_id is None:
fresh.discussion_note_id = note.id
await session.commit()
logger.info(
"Saved article-discussion summary as note %d for rss_item %d (conv %d)",
note.id, item_id, conv_id,
)
except Exception:
logger.warning(
"Failed to persist article-discussion note for conv %d",
conv_id, exc_info=True,
)
# ---------------------------------------------------------------------------
# Thinking decision
# ---------------------------------------------------------------------------
@@ -526,6 +605,18 @@ async def run_generation(
msg_count = len(non_system)
should_gen_title = not conv_title or (msg_count > 0 and msg_count % 10 == 0)
# Persist article-discussion seed conversations as a Note on their
# first assistant reply. This makes "Discuss" summaries part of RAG
# so the knowledge base stops being amnesiac about articles the user
# has already engaged with. The hook detects a seeded conversation by
# finding a synthetic read_article assistant message whose
# msg_metadata carries ``article_seed: True`` and whose rss_items row
# has no discussion_note_id yet. Fire-and-forget so the done event
# lands immediately.
asyncio.create_task(_maybe_save_article_discussion_note(
user_id, conv_id, buf.content_so_far,
))
if should_gen_title:
# Feed the title model the *raw* conversation turns only — never
# the post-build_context ``messages`` list. ``build_context``
+20 -11
View File
@@ -724,18 +724,27 @@ async def build_context(
orphan_only = rag_project_id is None
effective_project_id = rag_project_id if (rag_project_id is not None and rag_project_id != -1) else None
try:
from fabledassistant.services.embeddings import semantic_search_notes
for score, note in await semantic_search_notes(
user_id, user_message, exclude_ids=search_exclude or None, limit=8,
project_id=effective_project_id,
orphan_only=orphan_only,
):
found_scored.append((score, note))
except Exception:
logger.warning("Semantic note search failed, falling back to keyword search", exc_info=True)
# Skip RAG auto-injection on the first turn of a seeded article discussion.
# The article body is already the sole context the user wants — pulling in
# unrelated orphan notes tricks the model into summarizing those instead.
# Follow-up turns keep RAG on because by then the user's own messages drive
# the query rather than the generic seed prompt.
from fabledassistant.services.article_context import ARTICLE_DISCUSS_SEED
_skip_rag_for_article_seed = user_message.strip() == ARTICLE_DISCUSS_SEED
if not found_scored:
if not _skip_rag_for_article_seed:
try:
from fabledassistant.services.embeddings import semantic_search_notes
for score, note in await semantic_search_notes(
user_id, user_message, exclude_ids=search_exclude or None, limit=8,
project_id=effective_project_id,
orphan_only=orphan_only,
):
found_scored.append((score, note))
except Exception:
logger.warning("Semantic note search failed, falling back to keyword search", exc_info=True)
if not found_scored and not _skip_rag_for_article_seed:
keywords = _extract_keywords(user_message)
if keywords:
try: