Commit Graph

774 Commits

Author SHA1 Message Date
bvandeusen 41d252e9d1 deps: pin requires-python = ">=3.14"; commit uv.lock
Match CI + runtime target exactly — both run Python 3.14, so the
package metadata signals consumers that we don't test against 3.12/3.13.

uv.lock is tracked so the test job's `uv venv` resolution is
reproducible (currently the test job installs the editable package
without consulting the lockfile; future work could wire `uv sync` in).
Lockfile resolves 179 packages against Python 3.14.4.

ci-requirements.md updated to drop the prior "permissive lower bound"
caveat.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:25:01 -04:00
bvandeusen 6cf70e22db compose(db): lenient healthcheck + stop_grace_period to survive host stalls
Mitigation for the nightly fabledscribe Postgres outage on the
vdnt-docker02 Swarm node (incidents 2026-05-15/16/17 around 03:50 UTC).
Confirmed kill chain (not the trigger): a brief host-level setns/exec
stall makes the Docker healthcheck exec fail with exit 1 → unhealthy →
SIGKILL → fast-shutdown can't finish on NFS in 10s → exit 137 → swarm
restart_policy.max_attempts: 5 burns out → DB stays dead.

Hardens the `db` service so a transient host blip can't escalate to
killing the database:
- stop_grace_period: 120s (gives PG room to fsync on shutdown)
- healthcheck: interval 30s / timeout 10s / retries 10 / start_period 180s
  (only gates app startup order — not authoritative liveness)
- prod: restart_policy condition=on-failure, max_attempts=0, window=120s
- quickstart/dev: restart: unless-stopped

Host-side trigger (what stalls runc/exec at ~03:50 UTC) is still under
investigation — see project_pg_nightly_outage.md.

Note: the Portainer prod stack differs from docker-compose.prod.yml
here (NFS bind, traefik labels, no ollama). The same `db:` block needs
to be pasted into Portainer for the prod mitigation to apply.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:18:19 -04:00
bvandeusen 3f1bcc3360 ci: consume shared ci-python:3.14 image; bump runtime to 3.14
Migrate to the FabledRulebook CI-Runner contract:

- .forgejo/workflows/ci.yml: all four jobs (typecheck/lint/test/build)
  now schedule on the `python-ci` runner label and run inside
  container.image: git.fabledsword.com/bvandeusen/ci-python:3.14
  (Python 3.14 + Node 24 + ruff + uv + Docker CLI). Dropped the inline
  uv install in the test job — uv is now baked into the image.
- Dockerfile: production runtime bumped to python:3.14-slim so test
  results stay representative against what we ship.
- ci-requirements.md: new file at repo root declaring image deps and
  per-job installs (per FabledRulebook ci-runners.md).
- infra/Dockerfile.runner-base: deleted. The in-repo runner base
  (Ubuntu 24.04 + Python 3.12 + Node 22) is superseded by the shared
  ci-python image. The runner-host deployment files
  (runner-compose.yml + act-runner-config.yml) stay as deployment-shape
  documentation; source of truth is the deployed config.
- docs/development.md: CI/CD + Runner sections refreshed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:16:26 -04:00
bvandeusen 5d2d27c499 fix(journal): anti-hallucination hardening + message_count fix
Prep prose (services/journal_prep.py):
- Emit explicit "WEATHER: none available — do NOT mention weather"
  absent-marker so a small model can't invent partly-cloudy/temperature
  prose when both configured locations have empty addresses.
- Replace negative-only system rule with positive-anchored guidance
  forbidding weather/temp/precip mentions unless a numeric WEATHER
  section is present; also bans echoing parenthetical labels verbatim.
- Reword overdue header to "(past their due date, still open — backlog,
  not today's work)" and render lines as "was due <date>, N day(s)
  overdue" with correct singular/plural. Supersedes the wording noted
  in Fable task #159.
- Deterministic fabricated-weather reconciler: low-false-positive regex
  detects fabricated weather phrasing; on trip with an empty section,
  regenerate once with a corrective. Persistent fabrication logs ERROR
  rather than mangling prose.

Journal route (routes/journal.py):
- Override message_count with len(messages) in _day_payload. The chat
  path already does this; the journal path was hitting the
  Conversation.to_dict() fallback to 0 because messages aren't
  eager-loaded on that instance.

Tests:
- tests/test_journal_message_count.py — pins the model-level trap and
  the override contract (3 cases).
- tests/test_journal_prep_hardening.py — 11 cases covering the
  fabricated-weather reconciler and absent-marker rendering.
- tests/test_journal_prep_filtering.py — updated one stale assertion.

Tracks Fable task #171.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 18:54:35 -04:00
bvandeusen e6f2ee2b94 feat(frontend): pin/unpin/auto-pin UI in HistoryPanel
Version list rows now render a kind-aware badge: filled circle for
manual pins (with the label inline), half-filled circle for auto-pinned
versions. The right pane gains a control row above the diff:

- Unpinned: 'Pin version' button → label input → Save creates a manual
  pin with that label.
- Manual: 'Edit label' + 'Unpin' buttons.
- Auto: 'Pin permanently' (promotes auto → manual with editable label).

Local state is patched from the API response so the UI updates without
reloading the panel.
2026-05-13 14:00:57 -04:00
bvandeusen 59dee3a19f feat(frontend): NoteVersion pin fields + pin/unpin client helpers 2026-05-13 13:59:47 -04:00
bvandeusen ce41f2a3ee feat(versions): include pin_kind/pin_label in backup export+restore
Both export paths emit pin_kind and pin_label per note_version row.
Restore reads them via .get() so backups predating the schema still
import cleanly (defaults to None → rolling).
2026-05-13 13:59:20 -04:00
bvandeusen b1226d4e16 feat(versions): daily 03:00 UTC auto-pin scan scheduler
BackgroundScheduler with a single CronTrigger fires scan_all_users_for
_auto_pins via asyncio.run_coroutine_threadsafe (mirrors the journal-
scheduler pattern). Wired into app startup/shutdown alongside the other
schedulers.
2026-05-13 13:58:34 -04:00
bvandeusen 37c704e875 feat(versions): auto-pin scan promotes stable versions
_promote_stable_versions_for_note is the pure-function core: walks
versions chronologically and pins any with a >= AUTO_PIN_STABILITY_DAYS
(2-day) gap to the next version (or to now, for the latest). Auto-
generated label describes the stability window.

_scan_one_note loads versions for one note, runs the promotion, commits
mutations to the attached rows, then calls prune_auto_pins to cap the
auto bucket. scan_user_for_auto_pins fans out across the user's notes;
scan_all_users_for_auto_pins is the top-level entrypoint for the cron.
Per-note and per-user errors are caught and logged.
2026-05-13 13:57:56 -04:00
bvandeusen bb6249e00e feat(versions): prune_auto_pins FIFO-trims auto-pinned bucket
Auto-pinned versions live in their own bucket with MAX_AUTO_PINS=25 cap.
The scan job calls this after each note's promotions complete; the
oldest auto-pinned rows are dropped past the cap. Manual pins and
rolling rows are untouched.
2026-05-13 13:57:09 -04:00
bvandeusen 9c0308dfee feat(versions): POST/DELETE /api/notes/:id/versions/:vid/pin 2026-05-13 13:56:42 -04:00
bvandeusen 925a53e0f7 feat(versions): pin_version and unpin_version services
pin_version sets pin_kind='manual' and pin_label on the target row.
Accepts already-pinned rows (promotes auto→manual, updates label).
Labels are capped at PIN_LABEL_MAX_LEN=500 chars; longer values raise
ValueError before any DB access.

unpin_version clears both fields, downgrading the row to rolling. Does
NOT delete — if the row is past the rolling FIFO depth, the next
autosave's prune will drop it.
2026-05-13 13:56:21 -04:00
bvandeusen b65d736869 feat(versions): rolling-cap prune ignores pinned versions
The DELETE inside create_version now filters pin_kind IS NULL so pinned
rows (auto or manual) aren't counted toward MAX_VERSIONS=50 and aren't
candidates for deletion. Pinned versions live indefinitely regardless
of how heavy rolling autosave traffic gets on the same note.
2026-05-13 13:55:44 -04:00
bvandeusen 17211c6e82 feat(schema): add note_version.pin_kind and pin_label
Spec: docs/superpowers/specs/2026-05-13-note-version-pinning-design.md

- pin_kind: NULL=rolling, 'auto'=stability-scan, 'manual'=user-declared.
- pin_label: NULL for rolling; auto-generated for 'auto'; user-supplied
  string for 'manual' (may be NULL).

No backfill — every existing row stays rolling. The daily auto-pin scan
will catch up on the first run after deploy.
2026-05-13 13:55:18 -04:00
bvandeusen 90aa1f2fdb fix(tests): preserves_body test \$Note stub needs project_id
The knowledge-note return path in create_note_tool reads note.project_id;
the SimpleNamespace fake didn't define it, so the tool crashed with
AttributeError instead of returning. The task-branch test already
included project_id; mirror that here.
2026-05-13 12:30:09 -04:00
bvandeusen b519a1c140 feat(frontend): auto-consolidate tasks toggle in General settings
New Tasks section in the General tab with a single checkbox controlling
whether the consolidation pipeline fires automatically. Persists to the
auto_consolidate_tasks user setting (string 'true'/'false'). Manual
'Re-consolidate' in the task editor bypasses the gate.
2026-05-13 12:23:06 -04:00
bvandeusen 257b306a27 feat(frontend): gate body editor when task body is auto-maintained
When consolidated_at is set on a task, the editor:
- shows a banner above the body indicating the body is auto-summarized
- hides the Write tab; locks the body view to read-only preview
- exposes a Re-consolidate button that calls POST /api/tasks/:id/consolidate
  and refreshes the body from the response

Pre-consolidation behavior is unchanged — the Write tab and TiptapEditor
remain available.
2026-05-13 12:21:51 -04:00
bvandeusen 8b0878f227 feat(frontend): description field in task editor + Goal block in viewer
Note type gains description and consolidated_at fields. TaskEditorView
adds a Goal textarea above the body editor (wired through dirty/save/
autosave paths). TaskViewerView renders Goal as a subordinate block
above the body, plus a subtle 'Auto-summarized from work logs' banner
when consolidated_at is set.

Also adds a consolidateTask client function for the upcoming
re-consolidate button (Task 11).
2026-05-13 12:20:42 -04:00
bvandeusen 9191ab5b27 feat(tasks): POST /api/tasks/:id/consolidate + separate body/description fields
New endpoint manually triggers a consolidation pass for a single task.
Bypasses the auto_consolidate_tasks setting since the user is asking
explicitly. Returns the task with the freshly-written body and
consolidated_at timestamp.

Also un-aliases description and body in the create/update task routes
(was: description folded into body as legacy fallback). With separate
fields under the task-as-durable-record design, both flow through as
distinct kwargs to create_note / update_note.
2026-05-13 12:16:43 -04:00
bvandeusen fd25d2e436 docs(tools): clarify log_work feeds summary; create_note runbook guidance
log_work description now mentions that logs feed the task's auto-summary,
nudging the LLM toward specific log content (commands, decisions, failures)
rather than vague entries.

create_note description gains a runbook-shape clause: code blocks, numbered
procedures, and explicit 'save this as a note/runbook' signals should
spawn standalone notes. Task-specific work-in-progress routes to log_work
instead.
2026-05-13 12:15:36 -04:00
bvandeusen 103db883ad feat(tools): tasks accept description; reject body writes via tools
create_note tool:
- New 'description' parameter accepted and forwarded to the service.
- When status is set (creating a task), 'body' is dropped before the
  service call. Task bodies are owned by the consolidation pipeline.

update_note tool:
- New 'description' parameter; routed through update_fields.
- When the resolved target has is_task=True and 'body' is in the
  arguments, the call errors with a message nudging toward log_work or
  description. Knowledge notes are unaffected.

HTTP routes (POST/PATCH/PUT /api/notes) accept body freely — the
restriction is only at the LLM tool layer.
2026-05-13 12:15:09 -04:00
bvandeusen 5fa203019a feat(consolidation): trigger from log_work and status terminal transitions
log_work tool now invokes maybe_consolidate(reason='log_added') after a
successful create_log. The gate inside the consolidation service handles
threshold + setting checks.

update_note service snapshots old_status before mutation and fires
maybe_consolidate(reason='task_closed') when the status transitions into
'done' or 'cancelled'. Re-saving an already-terminal status doesn't
retrigger — only transitions count.
2026-05-13 12:13:31 -04:00
bvandeusen bda6e6c80f feat(consolidation): full consolidate_task with background model
consolidate_task reads the task title, description (read-only context),
and chronological work logs; builds a prompt via _build_consolidation_prompt;
calls generate_completion with the user's background_model setting; on a
non-empty result, writes back to Note.body, stamps consolidated_at, and
re-runs the embedding pipeline.

Errors are caught and logged. LLM failures leave body untouched so the
next trigger retries cleanly. Per-task asyncio lock prevents simultaneous
passes for the same task.
2026-05-13 12:12:37 -04:00
bvandeusen 5419330633 feat(consolidation): debounced gate for task body consolidation
New services/consolidation.py module with maybe_consolidate() — the
debounced trigger gate. Two reasons:

- log_added: gated by DEFAULT_LOG_THRESHOLD (3) counted since the task's
  consolidated_at timestamp.
- task_closed: bypasses the count gate; fires whenever status flips to
  done/cancelled.

Both reasons gated by the auto_consolidate_tasks user setting (default
on). Per-task asyncio.Lock prevents two simultaneous passes for the same
task. consolidate_task is a stub here — full implementation in the next
commit.
2026-05-13 12:11:41 -04:00
bvandeusen 362ead7f0d feat(notes): accept and return description field through service and routes
create_note service accepts a new description kwarg and forwards it to the
Note constructor. PUT/PATCH/POST routes include description in the field
whitelist. update_note already passed **fields through setattr, so the new
column is reachable without touching that signature.
2026-05-13 12:11:03 -04:00
bvandeusen 8a3bba4eb8 feat(schema): add note.description and note.consolidated_at
Spec: docs/superpowers/specs/2026-05-13-task-as-durable-record-design.md

- description: user-stated goal / initial context for tasks (NULL for
  knowledge notes).
- consolidated_at: timestamp of the most recent auto-summary pass (NULL
  until first consolidation).
- Migration 0044 backfills description from body for existing rows where
  status IS NOT NULL (i.e. tasks). Body left in place; first consolidation
  pass will overwrite it.
2026-05-13 12:09:45 -04:00
bvandeusen a551f52682 fix(tools): score_project_match strips 'project' filler + uses title for SequenceMatcher
CI surfaced three issues:
- 'famous supply project' didn't substring-match 'Famous-Supply Work topics'
  because the trailing filler word 'project' blocked the substring tier.
  Strip {project, projects} from the query before the substring check.
- SequenceMatcher fallback against `combined` (title + description +
  summary) diluted ratios to ~0.5 for plausible matches. Use title
  directly; the 0.70 tier already handles description/summary mentions.
- Test patches used patch.object on a consumer module where
  list_projects is imported locally — patch the source module instead.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 20:30:37 -04:00
bvandeusen 6de855e226 feat(llm): search-first heuristic in chat tool_lines static block
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:44:00 -04:00
bvandeusen c6357e52d9 feat(journal): search-first heuristic for existing work in calibration prompt
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:43:42 -04:00
bvandeusen 5d40f2113f feat(tools): record_moment requires task_titles from prior search_notes
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:43:21 -04:00
bvandeusen 9a96fdb3c0 refactor(tools): resolve_project uses shared score_project_match helper
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:43:05 -04:00
bvandeusen 460959f0d4 refactor(tools): search_projects uses shared score_project_match helper
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:42:34 -04:00
bvandeusen 0dbbb98cf5 feat(tools): score_project_match helper with substring-first tiering
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:42:01 -04:00
bvandeusen 4b7ca1b17e feat(tools): search_notes description nudges away from type-nouns in query
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:41:28 -04:00
bvandeusen 65a3689aaa fix(notes): list_notes strips type-nouns from q before ILIKE filter
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:41:15 -04:00
bvandeusen 42c11dedae feat(notes): _strip_type_nouns helper for search query sanitization
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:40:54 -04:00
bvandeusen 0fbb1fbd92 fix(tools): search_projects returns success:True so result isn't mislabeled error
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 16:40:27 -04:00
bvandeusen bb650ba563 feat(profile): Settings panel with recent journal observations
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:44:51 -04:00
bvandeusen c663532fd4 feat(profile): Settings toggle for nightly journal closeout
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:43:59 -04:00
bvandeusen 090b7d83dd feat(frontend): API client for profile observations + closeout flag
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:43:27 -04:00
bvandeusen 4e9eead3ab feat(profile): GET /api/profile/observations returns recent raw entries
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:36:42 -04:00
bvandeusen fc6ebf81eb feat(journal): closeout catch-up on startup when slot already passed
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:36:29 -04:00
bvandeusen b88d5ee6b3 feat(journal): register per-user closeout job at day_rollover_hour
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:35:57 -04:00
bvandeusen 020bd6614b test(journal): cover closeout skip paths (no conv / prep-only / sentinel)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:34:58 -04:00
bvandeusen 4403026797 feat(journal): run_for_user orchestrates closeout extraction
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:34:42 -04:00
bvandeusen 552943d6c0 feat(journal): closeout system prompt with structured-field guard
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:33:57 -04:00
bvandeusen 2576be9e49 feat(journal): _build_transcript caps content and message window
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:33:27 -04:00
bvandeusen e17fc088b2 feat(journal): journal_closeout._filter_messages excludes daily_prep
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:33:03 -04:00
bvandeusen c5b0344240 feat(journal): add closeout_enabled to default config
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 14:32:40 -04:00
bvandeusen c33cab7020 fix(journal): wire weather refresh on config save; drop orphaned cache rows
Two related gaps in the journal weather panel:

1. Saving locations via PUT /journal/config didn't trigger a weather
   fetch, so newly-entered sites had no cache row (or a stale one) until
   the user manually clicked the panel's refresh button. The panel
   rendered "two sites with empty values" against pre-existing cache
   rows that no longer matched what the user had configured.

2. get_cached_weather_rows returned every WeatherCache row for the user
   regardless of whether the location was still in journal_config.
   Briefing-era rows survived migration 0040 (which only deleted the
   briefing_config setting, not the cache table) and showed up as
   ghost tabs in the UI.

Changes:
- get_cached_weather_rows accepts an optional valid_keys filter; rows
  whose location_key is not in the set are excluded.
- routes/journal.py:
  - put_config kicks off a background refresh_location_cache for any
    saved location with valid lat/lon.
  - GET /weather and POST /weather/refresh both pass valid_keys derived
    from the current config so orphaned rows don't surface.
- services/journal_prep.py filters the weather section to currently-
  configured locations as well; uses a lazy import of get_journal_config
  to avoid a cycle (journal_scheduler imports journal_prep).

153 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:37:23 -04:00