Commit Graph
10 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 5c6175ad97 feat(placement): a record you only cite carries its status (#4154)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 50s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / Python tests (push) Failing after 1m4s
CI & Build / Build & push image (push) Skipped
Step 1 made placement cheap for a task whose status CHANGES: create_task
and update_task return where it sits, and the report is written from
that. It did nothing for a task a reply merely cites.

This milestone's own step-6 review reported "#4014 is the open step of
milestone 409". #4014 had been done for four days; the open step was
#4015. The id did not come from a read — it came from a retrieval hint,
which carries an id, a kind and a title and says nothing about status,
while list_milestones said "8 of 9" and would not say which one. The
gap was there to be filled and the nearest-looking id filled it.

Two surfaces, one principle: the status arrives with the id.

1. get_project_milestone_summaries gains next_step — the earliest open
   step, {id, title, status} or None — carried through _BRIEF_FIELDS to
   enter_project, get_project and list_milestones. One extra flat query
   for the whole batch, so #2384's fan-out does not come back.

   OPEN_STEP_STATUSES moves to services/milestones.py and placement.py
   imports it; both surfaces now answer "what is next" and must not
   drift on what counts as open. Both step queries take the same
   readable_notes_clause (rule 78), so a row cannot name a step its own
   progress numbers exclude.

2. _record_kind renders a task's status: [task (done)], [issue (todo)].
   A finished step and an open one read identically before, which is
   exactly the line the misreport was taken from. Only tasks — is_task
   IS status-is-not-None on the model, so there is no fallback branch.

reporting-back gains the practice, owned and registered in the guidance
ownership table: a record you only mention is a record to read.

The guards are structural and each fails on the regression it names:
the query count is asserted rather than the payload shape, and the two
surfaces' agreement is pinned on the rendered ORDER BY, since a mocked
session hands back whatever order the test chose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
2026-09-18 12:08:00 -04:00
bvandeusenandClaude Opus 5 5e4fd017ae fix(mcp): list tools return rows that say what a record is, not what it says (#4061)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 49s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m31s
CI & Build / Build & push image (push) Successful in 22s
list_tasks returned every row's to_dict(), body included: a project's todo list
came to 93-165k characters, past what an MCP client accepts inline, so the list
arrived as a file to page through (the #4045 failure, one call over).

- notes.brief_row: id, title, type, project, tags, updated_at; for tasks,
  status, kind, priority, milestone id and title; description, parent and due
  date only when set.
- milestones.titles_for: one query for the milestone titles a page of rows
  names.
- Brief rows on list_tasks, list_notes, get_milestone's steps, get_system and
  list_system_records. get_task / get_note / get_snippet read a record in full,
  and each docstring says so.
- tests/test_list_rows_brief.py pins the ceiling: 100 rows of ~5k-character
  step plans stay under 40k characters.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
2026-09-15 16:07:00 -04:00
bvandeusenandClaude Opus 5 3a501c2cac feat(search): milestones are searchable by meaning — "is there already a plan for this?" (#4078)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / integration (push) Successful in 56s
CI & Build / Python tests (push) Successful in 1m37s
CI & Build / Build & push image (push) Successful in 28s
`search` covered notes, tasks and rules, and a milestone — the record a plan
lives in — could not be found. A project whose roadmap was written as
milestones had every later plan opened beside the one that already described
it, because nothing could have told the session it existed.

- milestone_embeddings (migration 0102): the third sibling of note_ and
  rule_embeddings, for note 3163's reason — the search is milestone-specific.
  The document is title — description, then description and the plan body,
  so a roadmap milestone with no description is still found by its design.
- Written on create, on a title/description/body update, and for a plan made
  through start_planning / create_records, fire-and-forget with the parent-row
  claim (#3262); a startup backfill covers every existing milestone. Derived,
  so it joins _NOT_INCLUDED beside the other embeddings.
- semantic_search_milestones: a project's milestones when the caller can read
  it (access.can_read_project), otherwise the caller's own; optional status.
- search(content_type="milestone"): id, title, description, status, project
  and progress. Its own shape, and not part of "all", whose results are
  note-shaped. The docstring says what it is for: ask before start_planning.
- Integration test on real Postgres: found in its project and not another,
  status narrows, an unreadable project returns nothing.

Milestone 415 step 3.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
2026-09-15 13:34:03 -04:00
bvandeusenandClaude Opus 5 184a3e026d feat(mcp): enter_project names active milestones with no steps as open work (#4076)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / integration (push) Canceled after 42s
CI & Build / TypeScript typecheck (push) Canceled after 42s
CI & Build / Python tests (push) Canceled after 47s
A plan written as a milestone with a description and no steps was invisible to
the session handshake: it lists the 5 most recently touched milestones (#4045),
and touching is a step changing, so a step-less milestone can never qualify.
FabledLibrarian's roadmap (nine such milestones) sat unseen while later plans
were opened as new milestones beside the ones that already described them.

enter_project adds `unplanned_milestones`: active milestones with no steps, in
roadmap order, id/title/description, up to 10 with an omitted count, none
repeated from the recent list, and absent when there are none. The docstring
says what they are for: check them before starting a new milestone, and add
steps to a match with create_records(milestone_id=...).

Milestone 415 step 1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
2026-09-15 13:28:05 -04:00
bvandeusenandClaude Opus 5 7f974d9749 feat(mcp): enter_project becomes a small primer: goal, recent work, open work, vocabulary (#4045)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 46s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m44s
CI & Build / Build & push image (push) Successful in 23s
The handshake carried the whole project record, every milestone's plan, full
rule text, the notes most recently edited and ~9k of design guidance. For
project 2 that was ~222k characters, past what an MCP client accepts as a tool
result. Each category was walked through with the operator and sized to what a
session needs on arrival; each names the call that has the rest.

- project: id, title, status and the full goal (session start's "full goal"
  pointer still lands here). get_project keeps the whole record.
- milestone_summary: the 5 most recently touched milestones, any status, most
  recent first, without plans. Summaries gain last_touched_at: the later of
  the milestone's own edit and its newest step update, from the query that
  already counts steps. milestone_summary_omitted counts the rest and points
  to list_milestones. get_project and list_milestones list every milestone,
  also without plans.
- open_tasks: the 10 most recently touched open tasks, with or without a
  milestone, each naming its milestone. list_notes gains sort="touched"
  (the later of updated_at and the newest work-log), because a log doesn't
  bump updated_at.
- recent_notes: dropped. Retrieval surfaces notes by relevance, and
  get_recent covers recency.
- systems: id and name.
- design_system: summary plus guidance_call. get_design_system gains
  resolved_guidance, the chain-merged prose; its own guidance field is only
  the departures, so session start's old pointer to it led to a fragment.
  The session start pointer and using-scribe's "Building UI" section now
  name resolved_guidance.
- rules: rules_payload(brief=True) gives project_rules as id and title plus
  subscribed_rulebooks, and records only what it shows. Retrieval delivers
  rules in full and ignores subscriptions (#4052). Other callers unchanged.
- pattern_coverage, inception and systems_bootstrap: unchanged.

Clients: the plugin's using-scribe skill, the compaction notice and session
start are updated here; the REST project summary only gains last_touched_at.
Plugin version minted.

Tests: a size ceiling on the handshake for a large project; milestone and
task selection and naming; brief rules; resolved_guidance; the session
start pointer; and a real-Postgres test that a work-log touches its task and
a step update touches its milestone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
2026-09-14 22:04:38 -04:00
bvandeusenandClaude Opus 5 9b2de3552f fix(mcp): project reads list milestones without their plans, and cap done ones (#4045)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m0s
CI & Build / Python tests (push) Successful in 1m32s
CI & Build / Build & push image (push) Successful in 30s
enter_project returned every milestone's full plan body. On a project with 39
milestones the handshake came to ~222k characters, 168k of them bodies (110k
from done milestones). That is past what an MCP client accepts as a tool
result, so the call meant to orient a session arrived as a file to page
through. It grows with a project's history, so any long-lived project on any
install gets there.

- brief_milestone_summary (services/milestones.py) trims summary rows to the
  listing fields: id, title, description, status, order_index and progress.
  The plan is get_milestone's job. user_id, project_id and timestamps repeat
  what the caller knows.
- enter_project and get_project share one block: every open milestone plus
  the 5 most recently updated done ones, in order. milestone_summary_omitted
  is attached only when older done ones were left out, and names
  list_milestones and get_milestone.
- list_milestones lists every milestone, done included, without bodies. It is
  the call the omitted line points to, and it had the same size problem.
- The REST project summary is unchanged; the web UI reads it.

Tests: trimming, the done cap and its order, the omitted key present and
absent, get_project and list_milestones, and a size ceiling on enter_project's
milestone block for a 200-milestone history.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
2026-09-14 19:48:24 -04:00
bvandeusenandClaude Fable 5 7d48eb0b1b refactor(services): one periodic-task shape, one APScheduler job shape, one token hash, one summary rule — the services pass of the shape audit (#2830, milestone 296)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / TypeScript typecheck (push) Successful in 12s
CI & Build / integration (push) Successful in 24s
CI & Build / Python tests (push) Successful in 55s
CI & Build / Build & push image (push) Successful in 24s
- background.start_periodic(interval, work, label=) replaces the three hand-rolled
  while-True/sleep/try loops in logging, auth and notifications.
- services/scheduler.ScheduledJob replaces the four private BackgroundScheduler
  copies in recurrence/version_pinning/trash/db_maintenance schedulers; public
  start_/stop_/reschedule_ surfaces unchanged.
- api_keys.hash_token is the one sha256 helper; auth.py used to inline it 5x.
- auth.is_registration_open reads via settings.get_admin_setting; notification
  prefs read via settings.get_setting; _fire_share_email uses _get_user_email.
- projects.get_project_summary / milestones.get_project_milestone_summary are
  now the one-id view of their batch siblings instead of a second copy of the
  queries; sharing.best_permission_by (was _deduplicate_by_permission) is the
  one rank-dedup, now also used by list_projects_for_user.
- backup: the row builders for every section both exporters carry are named
  functions, so a column added to one export cannot silently miss the other.
- iso() from models.base replaces the attr.isoformat()-if-attr-else-None idiom
  and db_maintenance._iso across services; backup keeps its explicit shape.
- trash.py hoists the sql_delete/timedelta imports it re-imported per function.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 12:34:28 -04:00
bvandeusenandClaude Opus 5 be3a0ffaf9 fix(projects): batch the summary queries — the fan-out was exhausting the pool
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 43s
CI & Build / integration (push) Successful in 2m33s
CI & Build / Python tests (push) Successful in 3m1s
CI & Build / Build & push image (push) Successful in 44s
Reported live: Projects and Snippets showed skeletons that never resolved,
/knowledge worked intermittently. The logs named it exactly:

    QueuePool limit of size 5 overflow 10 reached, connection timed out, 30.00
    GET /api/settings  500  30584.0ms
    GET /api/projects  200  30882.9ms

/api/projects was not hanging — it was waiting out the 30-second checkout
timeout and then returning 200 with summaries silently missing, because
_attach swallowed the TimeoutError. Nobody waits 31 seconds, so it read as a
hang.

THE SHAPE: routes/projects.py ran asyncio.gather over every project. Each
_attach called get_project_summary, which opened its own session for three
queries and then called get_project_milestone_summary — which opened one more
session PER MILESTONE. So 25 projects asked for roughly 250 concurrent
checkouts against a pool of 15 (SQLAlchemy's default 5 + 10 overflow).

That is why unrelated routes failed too. Snippets and /knowledge were never
broken; they queued behind the burst and inherited its timeout. /api/settings
returning 500 while /api/projects returned 200 is the same cause wearing two
faces.

The comment above the gather said "one backend pass instead of N+1 frontend
calls". It did remove the N+1 from the network — and recreated it against the
connection pool, where it is worse, because the browser had at least been
serialising those calls.

Now: get_project_summaries() does all projects in four queries and one session,
and get_project_milestone_summaries() does all milestones in two. Two sessions
total for the whole page, independent of how many projects exist.

The progress calculation is extracted to _progress_from_counts and shared by
both the batch and single paths, so the cancelled-exclusion rule cannot drift
into two versions that disagree about whether a milestone is finished.

Tests assert the SESSION COUNT, not just the values. An implementation that
returned identical output while opening a session per project would pass a
correctness test and reproduce the outage.

Deliberately NOT done: raising pool_size. It would move the cliff rather than
remove it, and this endpoint now needs two connections regardless of scale.

Closes #2384.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs
2026-08-02 19:09:36 -04:00
bvandeusenandClaude Opus 4.8 1f6c592226 feat(plans): milestone-as-plan-container; retire kind=plan (T3)
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Python tests (push) Successful in 42s
CI & Build / Build & push image (push) Successful in 59s
The milestone becomes the plan container: a new nullable milestones.body
holds the design/intent (Goal/Approach/Verification) and individual steps
live as first-class child tasks (milestone_id) instead of checkboxes crammed
into one kind=plan task body. start_planning now creates a MILESTONE seeded
with the body template (not a kind=plan task) and returns it with applicable
rules; a new get_milestone MCP tool reads the plan back (body + steps + rules).

kind=plan is hard-retired going forward — start_planning never creates one.
The 'plan' task_kind enum value stays valid so the 11 historical plan-tasks
remain readable in place; no body-shredding backfill (corpus review showed
auto-splitting their checklists into tasks would be lossy: embedded code
blocks, a non-binary [~] state, tables, ID-encoded hierarchy).

- migration 0066: add milestones.body
- model/service/route/MCP: body passthrough on create+update; get_milestone
- server _INSTRUCTIONS: "plan" = milestone w/ body + child step-tasks
- UI: ProjectView shows/edits a milestone's plan body; start_planning expands
  the new milestone and opens its plan editor
- tests updated to the milestone contract + new body/get_milestone coverage

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:22:22 -04:00
bvandeusenandClaude Opus 4.8 b255a0f90e refactor: rename package fabledassistant -> scribe (code-only)
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python tests (push) Successful in 54s
CI & Build / Build & push image (push) Successful in 1m14s
Renames src/fabledassistant -> src/scribe and all imports, plus the
default DB name and DB user/password (fabled -> scribe) in config +
compose. 952 refs / 154 files. Reverses the old 'internal name stays
fabledassistant' convention.

Code-only: live databases are still physically named 'fabledassistant'.
Deployed environments must set POSTGRES_DB / POSTGRES_USER (or rename the
DB) since the defaults now resolve to 'scribe'. Repo (FabledScribe), git
host (fabledsword), MCP (fabled-git) and the image name (fabledscribe)
are intentionally unchanged.

ruff check src/ clean locally; CI (typecheck + pytest) is the gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 15:48:35 -04:00