Previously removed all voice deps from the runtime image because of the
numpy<2 / cp314 wheel chain. Actual upstream check (PyPI 2026-05-21)
shows the chain has resolved for the STT half:
- ctranslate2 v4.7.2 (2026-05-19) ships cp314 wheels
- faster-whisper v1.2.1 is pure Python and works on any supported runtime
- onnxruntime v1.26.0 has cp314 wheels (not used here but shared with
the upcoming piper-tts install)
The blocker was kokoro, not the whole stack. Kokoro has been stale
upstream since April 2025 with a `requires_python='<3.13'` pin; that's
being replaced separately with piper-tts.
This commit restores ONLY STT — faster-whisper + soundfile. No torch
(ctranslate2 does its own CPU inference), no kokoro, no spacy. Image
add: ~150 MB.
Voice code is lazily guarded; STT now works when VOICE_ENABLED=true.
TTS still fails gracefully (kokoro import error logged, voice degrades)
until the piper-tts swap lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI broke on the build job: kokoro's resolver walks back to a version
that pins numpy<2, which has no cp314 wheel; pip falls back to compiling
numpy from source; python:3.14-slim has no compiler; build fails.
Removing the voice deps install (torch + faster-whisper + kokoro +
soundfile + spacy) from the runtime image:
- unblocks the 3.14 build immediately
- shrinks the image by ~2 GB (torch alone)
- aligns with the explicit operator preference (voice/TTS doesn't pay
off in their workflow; conversational chat will get smaller/faster
with the new no-tools chat model on GPU, so transcription matters
even less)
Voice paths in code are already lazily guarded — TYPE_CHECKING-only
imports plus try/except inside load_stt_model. With VOICE_ENABLED=false
(default), the app starts cleanly with no voice deps installed. With
voice enabled, the import error is caught and logged; the feature
degrades gracefully rather than crashing.
To re-enable voice in a future build, `pyproject.toml` already has the
`voice` extra ready: install it with `pip install .[voice]` plus the
torch index pin, and download spacy en_core_web_sm. Dockerfile comment
documents the path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an empirical surface for evaluating model swaps. One row per
assistant turn captures: model, think_enabled, tools_available,
tools_attempted, tools_succeeded, tools_failed (with error details
as JSONB). Without this, judging whether a new model "actually fires
record_moment when it should" relies on anecdote across user-reported
sessions. With it, the data is queryable directly.
Pieces:
- Migration 0046: generation_tool_log table with user_created and
per-conversation indexes.
- Model: SQLAlchemy GenerationToolLog with to_dict() for plain-dict
consumption outside session scope.
- Service: log_tool_outcomes() normalizes the in-app tool-call shape
(function/result/status) into the split buckets and persists. It
catches its own exceptions — telemetry failure must NEVER affect
the user-facing generation flow. recent_logs() helper for read.
- Integration in run_generation: called once per turn right after
log_generation, fire-and-forget.
- Tests: pure-normalization unit tests using a stub session — no DB
needed in CI. Cover the success/error split, the empty-tool-calls
case, the exception-swallowing contract, and the success=False
edge case where status incorrectly says "success".
No UI for the telemetry yet — internal infrastructure (the operator
is the consumer, not the journal user), which the FabledRulebook
"no UI no ship" explicitly excepts. Query via psql or extend the
Fable MCP later if direct shell access gets tiresome.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The chat generation pipeline previously forced think=True unconditionally
to match qwen3's combined think+tools template, locking the system into
that model family. Bench data (2026-05-21, qwen3:30b-a3b/qwen3:32b on
CPU) showed thinking adds 1-2 minutes per turn for unclear quality
benefit — qwen3:30b-a3b even produced more rambling with think on.
This decouples think from the model family by reading a per-user
`think_enabled` setting (default `false`). Non-qwen3 models can now run
through the same pipeline without the silent-generation failure mode
that content-gated thinking would have caused — they just don't think.
qwen3 users who still want thinking can opt in via the Settings UI.
Settings UI:
- New "Enable model thinking" checkbox in General → Assistant section.
- Help text explains the default-off rationale and when to opt in.
- Persists via the existing settings API; no schema migration needed
(Setting is key/value text).
Telemetry to confirm whether this regresses tool-call reliability on
qwen3 (the current model) is in a follow-up commit (generation_tool_log).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Was failing with ModuleNotFoundError for httpx when run via system
python — httpx is a project dep but isn't on the system interpreter's
path. Adding PEP 723 script metadata + uv-run shebang means the script
auto-resolves its deps in an ephemeral venv on every invocation, no
project-venv setup required.
Run with `uv run scripts/bench_ollama.py …` or directly via the shebang
`./scripts/bench_ollama.py …`. `python scripts/bench_ollama.py …` still
works only when httpx happens to be on the active interpreter.
Docstring updated to reflect the running options.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The curator scenario hardcoded think=true, which is qwen3-family-specific.
Non-qwen3 models silently ignore the field, so cross-family curator
comparisons were apples-to-oranges (qwen thinks, others don't).
New --think flag:
- auto (default): scenario-driven — chat=off, curator=on. Matches the
prior behaviour and the most common case.
- off: force disabled across all runs. Use for fair cross-family
comparison; aligns behaviour explicitly even though non-qwen models
would ignore think anyway.
- on: force enabled across all runs. Use to measure what think
contributes on the same model (paired runs: --think off then on).
Output markdown table now records the think mode used, so saved results
are self-documenting when you diff cross-server or cross-config.
Docstring + usage examples updated to reflect the qwen3 candidate set
the bench was originally tuned for.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Standalone tool to measure Ollama model performance under the two
workload shapes the chat+curator architecture would impose:
- chat scenario: short user message, short reply, no thinking. Mirrors
the no-tools chat companion's expected load.
- curator scenario: ~700-token journal transcript with an extraction
prompt, thinking enabled. Mirrors the curator's expected load.
Defaults to CPU-only inference (num_gpu=0). Streams responses; reports
TTFT, total wall time, tokens/sec (from Ollama's eval_count/eval_duration
so it excludes client-side stream overhead), and prompt token count.
First request per (model, num_gpu) is a warm-up to load the model into
memory; not counted in the measured runs.
Designed for cross-server comparison: --server points at any Ollama
instance, --out writes a markdown table. Comparing the two CPU servers
becomes a matter of running the same command on each and diffing the
output.
Lives outside the chat/curator architecture commitment — measurement
tool only. Tells us "is qwen2.5:32b on CPU fast enough for a 10-20 min
curator cadence?" without writing any of the architecture code yet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Match CI + runtime target exactly — both run Python 3.14, so the
package metadata signals consumers that we don't test against 3.12/3.13.
uv.lock is tracked so the test job's `uv venv` resolution is
reproducible (currently the test job installs the editable package
without consulting the lockfile; future work could wire `uv sync` in).
Lockfile resolves 179 packages against Python 3.14.4.
ci-requirements.md updated to drop the prior "permissive lower bound"
caveat.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mitigation for the nightly fabledscribe Postgres outage on the
vdnt-docker02 Swarm node (incidents 2026-05-15/16/17 around 03:50 UTC).
Confirmed kill chain (not the trigger): a brief host-level setns/exec
stall makes the Docker healthcheck exec fail with exit 1 → unhealthy →
SIGKILL → fast-shutdown can't finish on NFS in 10s → exit 137 → swarm
restart_policy.max_attempts: 5 burns out → DB stays dead.
Hardens the `db` service so a transient host blip can't escalate to
killing the database:
- stop_grace_period: 120s (gives PG room to fsync on shutdown)
- healthcheck: interval 30s / timeout 10s / retries 10 / start_period 180s
(only gates app startup order — not authoritative liveness)
- prod: restart_policy condition=on-failure, max_attempts=0, window=120s
- quickstart/dev: restart: unless-stopped
Host-side trigger (what stalls runc/exec at ~03:50 UTC) is still under
investigation — see project_pg_nightly_outage.md.
Note: the Portainer prod stack differs from docker-compose.prod.yml
here (NFS bind, traefik labels, no ollama). The same `db:` block needs
to be pasted into Portainer for the prod mitigation to apply.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrate to the FabledRulebook CI-Runner contract:
- .forgejo/workflows/ci.yml: all four jobs (typecheck/lint/test/build)
now schedule on the `python-ci` runner label and run inside
container.image: git.fabledsword.com/bvandeusen/ci-python:3.14
(Python 3.14 + Node 24 + ruff + uv + Docker CLI). Dropped the inline
uv install in the test job — uv is now baked into the image.
- Dockerfile: production runtime bumped to python:3.14-slim so test
results stay representative against what we ship.
- ci-requirements.md: new file at repo root declaring image deps and
per-job installs (per FabledRulebook ci-runners.md).
- infra/Dockerfile.runner-base: deleted. The in-repo runner base
(Ubuntu 24.04 + Python 3.12 + Node 22) is superseded by the shared
ci-python image. The runner-host deployment files
(runner-compose.yml + act-runner-config.yml) stay as deployment-shape
documentation; source of truth is the deployed config.
- docs/development.md: CI/CD + Runner sections refreshed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prep prose (services/journal_prep.py):
- Emit explicit "WEATHER: none available — do NOT mention weather"
absent-marker so a small model can't invent partly-cloudy/temperature
prose when both configured locations have empty addresses.
- Replace negative-only system rule with positive-anchored guidance
forbidding weather/temp/precip mentions unless a numeric WEATHER
section is present; also bans echoing parenthetical labels verbatim.
- Reword overdue header to "(past their due date, still open — backlog,
not today's work)" and render lines as "was due <date>, N day(s)
overdue" with correct singular/plural. Supersedes the wording noted
in Fable task #159.
- Deterministic fabricated-weather reconciler: low-false-positive regex
detects fabricated weather phrasing; on trip with an empty section,
regenerate once with a corrective. Persistent fabrication logs ERROR
rather than mangling prose.
Journal route (routes/journal.py):
- Override message_count with len(messages) in _day_payload. The chat
path already does this; the journal path was hitting the
Conversation.to_dict() fallback to 0 because messages aren't
eager-loaded on that instance.
Tests:
- tests/test_journal_message_count.py — pins the model-level trap and
the override contract (3 cases).
- tests/test_journal_prep_hardening.py — 11 cases covering the
fabricated-weather reconciler and absent-marker rendering.
- tests/test_journal_prep_filtering.py — updated one stale assertion.
Tracks Fable task #171.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Version list rows now render a kind-aware badge: filled circle for
manual pins (with the label inline), half-filled circle for auto-pinned
versions. The right pane gains a control row above the diff:
- Unpinned: 'Pin version' button → label input → Save creates a manual
pin with that label.
- Manual: 'Edit label' + 'Unpin' buttons.
- Auto: 'Pin permanently' (promotes auto → manual with editable label).
Local state is patched from the API response so the UI updates without
reloading the panel.
Both export paths emit pin_kind and pin_label per note_version row.
Restore reads them via .get() so backups predating the schema still
import cleanly (defaults to None → rolling).
BackgroundScheduler with a single CronTrigger fires scan_all_users_for
_auto_pins via asyncio.run_coroutine_threadsafe (mirrors the journal-
scheduler pattern). Wired into app startup/shutdown alongside the other
schedulers.
_promote_stable_versions_for_note is the pure-function core: walks
versions chronologically and pins any with a >= AUTO_PIN_STABILITY_DAYS
(2-day) gap to the next version (or to now, for the latest). Auto-
generated label describes the stability window.
_scan_one_note loads versions for one note, runs the promotion, commits
mutations to the attached rows, then calls prune_auto_pins to cap the
auto bucket. scan_user_for_auto_pins fans out across the user's notes;
scan_all_users_for_auto_pins is the top-level entrypoint for the cron.
Per-note and per-user errors are caught and logged.
Auto-pinned versions live in their own bucket with MAX_AUTO_PINS=25 cap.
The scan job calls this after each note's promotions complete; the
oldest auto-pinned rows are dropped past the cap. Manual pins and
rolling rows are untouched.
pin_version sets pin_kind='manual' and pin_label on the target row.
Accepts already-pinned rows (promotes auto→manual, updates label).
Labels are capped at PIN_LABEL_MAX_LEN=500 chars; longer values raise
ValueError before any DB access.
unpin_version clears both fields, downgrading the row to rolling. Does
NOT delete — if the row is past the rolling FIFO depth, the next
autosave's prune will drop it.
The DELETE inside create_version now filters pin_kind IS NULL so pinned
rows (auto or manual) aren't counted toward MAX_VERSIONS=50 and aren't
candidates for deletion. Pinned versions live indefinitely regardless
of how heavy rolling autosave traffic gets on the same note.
Spec: docs/superpowers/specs/2026-05-13-note-version-pinning-design.md
- pin_kind: NULL=rolling, 'auto'=stability-scan, 'manual'=user-declared.
- pin_label: NULL for rolling; auto-generated for 'auto'; user-supplied
string for 'manual' (may be NULL).
No backfill — every existing row stays rolling. The daily auto-pin scan
will catch up on the first run after deploy.
The knowledge-note return path in create_note_tool reads note.project_id;
the SimpleNamespace fake didn't define it, so the tool crashed with
AttributeError instead of returning. The task-branch test already
included project_id; mirror that here.
New Tasks section in the General tab with a single checkbox controlling
whether the consolidation pipeline fires automatically. Persists to the
auto_consolidate_tasks user setting (string 'true'/'false'). Manual
'Re-consolidate' in the task editor bypasses the gate.
When consolidated_at is set on a task, the editor:
- shows a banner above the body indicating the body is auto-summarized
- hides the Write tab; locks the body view to read-only preview
- exposes a Re-consolidate button that calls POST /api/tasks/:id/consolidate
and refreshes the body from the response
Pre-consolidation behavior is unchanged — the Write tab and TiptapEditor
remain available.
Note type gains description and consolidated_at fields. TaskEditorView
adds a Goal textarea above the body editor (wired through dirty/save/
autosave paths). TaskViewerView renders Goal as a subordinate block
above the body, plus a subtle 'Auto-summarized from work logs' banner
when consolidated_at is set.
Also adds a consolidateTask client function for the upcoming
re-consolidate button (Task 11).
New endpoint manually triggers a consolidation pass for a single task.
Bypasses the auto_consolidate_tasks setting since the user is asking
explicitly. Returns the task with the freshly-written body and
consolidated_at timestamp.
Also un-aliases description and body in the create/update task routes
(was: description folded into body as legacy fallback). With separate
fields under the task-as-durable-record design, both flow through as
distinct kwargs to create_note / update_note.
log_work description now mentions that logs feed the task's auto-summary,
nudging the LLM toward specific log content (commands, decisions, failures)
rather than vague entries.
create_note description gains a runbook-shape clause: code blocks, numbered
procedures, and explicit 'save this as a note/runbook' signals should
spawn standalone notes. Task-specific work-in-progress routes to log_work
instead.
create_note tool:
- New 'description' parameter accepted and forwarded to the service.
- When status is set (creating a task), 'body' is dropped before the
service call. Task bodies are owned by the consolidation pipeline.
update_note tool:
- New 'description' parameter; routed through update_fields.
- When the resolved target has is_task=True and 'body' is in the
arguments, the call errors with a message nudging toward log_work or
description. Knowledge notes are unaffected.
HTTP routes (POST/PATCH/PUT /api/notes) accept body freely — the
restriction is only at the LLM tool layer.
log_work tool now invokes maybe_consolidate(reason='log_added') after a
successful create_log. The gate inside the consolidation service handles
threshold + setting checks.
update_note service snapshots old_status before mutation and fires
maybe_consolidate(reason='task_closed') when the status transitions into
'done' or 'cancelled'. Re-saving an already-terminal status doesn't
retrigger — only transitions count.
consolidate_task reads the task title, description (read-only context),
and chronological work logs; builds a prompt via _build_consolidation_prompt;
calls generate_completion with the user's background_model setting; on a
non-empty result, writes back to Note.body, stamps consolidated_at, and
re-runs the embedding pipeline.
Errors are caught and logged. LLM failures leave body untouched so the
next trigger retries cleanly. Per-task asyncio lock prevents simultaneous
passes for the same task.
New services/consolidation.py module with maybe_consolidate() — the
debounced trigger gate. Two reasons:
- log_added: gated by DEFAULT_LOG_THRESHOLD (3) counted since the task's
consolidated_at timestamp.
- task_closed: bypasses the count gate; fires whenever status flips to
done/cancelled.
Both reasons gated by the auto_consolidate_tasks user setting (default
on). Per-task asyncio.Lock prevents two simultaneous passes for the same
task. consolidate_task is a stub here — full implementation in the next
commit.
create_note service accepts a new description kwarg and forwards it to the
Note constructor. PUT/PATCH/POST routes include description in the field
whitelist. update_note already passed **fields through setattr, so the new
column is reachable without touching that signature.
Spec: docs/superpowers/specs/2026-05-13-task-as-durable-record-design.md
- description: user-stated goal / initial context for tasks (NULL for
knowledge notes).
- consolidated_at: timestamp of the most recent auto-summary pass (NULL
until first consolidation).
- Migration 0044 backfills description from body for existing rows where
status IS NOT NULL (i.e. tasks). Body left in place; first consolidation
pass will overwrite it.
CI surfaced three issues:
- 'famous supply project' didn't substring-match 'Famous-Supply Work topics'
because the trailing filler word 'project' blocked the substring tier.
Strip {project, projects} from the query before the substring check.
- SequenceMatcher fallback against `combined` (title + description +
summary) diluted ratios to ~0.5 for plausible matches. Use title
directly; the 0.70 tier already handles description/summary mentions.
- Test patches used patch.object on a consumer module where
list_projects is imported locally — patch the source module instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bvandeusen