Fix the projects-page pool exhaustion, and cap milestone bars at 10 #95

Merged

bvandeusen merged 2 commits from dev into main 2026-08-02 19:43:00 -04:00

Conversation 0 Commits 2 Files Changed 5

+366 -31

bvandeusen commented 2026-08-02 19:42:44 -04:00

Owner

Copy Link

Copy Source

Two commits. The first is the fix for the loading failure reported after the last deploy; the second is the card-height change.

What to test

1. /projects loads promptly. It was taking 30.9 seconds and arriving with summaries silently missing.
2. Snippets and /knowledge load reliably. They were never broken — they queued behind the projects burst and inherited its timeout.
3. Settings loads. It was returning 500 for the same reason.
4. Roundtable's card — should now show 10 bars, open work first, with "+N more milestones" beneath.

be3a0ff — the pool exhaustion

Your log named it exactly:

QueuePool limit of size 5 overflow 10 reached, connection timed out, 30.00
GET /api/settings  500  30584.0ms
GET /api/projects  200  30882.9ms

/api/projects was not hanging. It waited out the full 30-second checkout timeout and returned 200 with summaries absent, because the error was swallowed. Nobody waits 31 seconds, so it read as a hang.

asyncio.gather fanned out over every project → each opened its own session for three queries → then called the milestone summary, which opened one more session per milestone. Roundtable alone has ~35. So ~250 concurrent checkouts against a pool of 15, and every other route on the instance queued behind them.

The comment above it read "one backend pass instead of N+1 frontend calls". It did remove the N+1 from the network — and recreated it against the connection pool, where it is worse, because the browser had at least been serialising those calls.

Now two sessions for the whole page regardless of project count: four queries for all summaries, two for all milestones. The progress calculation is shared by the batch and single paths so the cancelled-exclusion rule cannot drift into two versions that disagree about whether a milestone is done.

Tests assert session count, not just values — an implementation returning identical output while opening a session per project would pass a correctness test and reproduce the outage.

Not done: raising pool_size. That moves the cliff rather than removing it.

5795fa9 — milestone bars capped

10 bars, open work first, newest first within each group, with a +N more milestones note.

Recency alone would have been wrong: a long-running project's oldest milestones are usually its finished ones, so the ten most recent could easily have been ten completed bars while the three in flight were hidden.

The palette index is captured before slicing, so bars don't recolour when a milestone closes. Computed once per load rather than called from the template. The notice is plain text, not a link — the card already navigates there, and a nested link inside a clickable region is an accessibility trap.

Payload unchanged; the API still returns every milestone.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs

Two commits. The first is the fix for the loading failure reported after the last deploy; the second is the card-height change. ## What to test 1. **`/projects` loads promptly.** It was taking 30.9 seconds and arriving with summaries silently missing. 2. **Snippets and `/knowledge` load reliably.** They were never broken — they queued behind the projects burst and inherited its timeout. 3. **Settings loads.** It was returning 500 for the same reason. 4. **Roundtable's card** — should now show 10 bars, open work first, with "+N more milestones" beneath. ## `be3a0ff` — the pool exhaustion Your log named it exactly: QueuePool limit of size 5 overflow 10 reached, connection timed out, 30.00 GET /api/settings 500 30584.0ms GET /api/projects 200 30882.9ms `/api/projects` was not hanging. It waited out the full 30-second checkout timeout and returned **200 with summaries absent**, because the error was swallowed. Nobody waits 31 seconds, so it read as a hang. `asyncio.gather` fanned out over every project → each opened its own session for three queries → then called the milestone summary, which opened **one more session per milestone**. Roundtable alone has ~35. So ~250 concurrent checkouts against a pool of 15, and every other route on the instance queued behind them. The comment above it read *"one backend pass instead of N+1 frontend calls"*. It did remove the N+1 from the network — and recreated it against the connection pool, where it is worse, because the browser had at least been serialising those calls. Now two sessions for the whole page regardless of project count: four queries for all summaries, two for all milestones. The progress calculation is shared by the batch and single paths so the cancelled-exclusion rule cannot drift into two versions that disagree about whether a milestone is done. **Tests assert session count, not just values** — an implementation returning identical output while opening a session per project would pass a correctness test and reproduce the outage. Not done: raising `pool_size`. That moves the cliff rather than removing it. ## `5795fa9` — milestone bars capped 10 bars, **open work first**, newest first within each group, with a `+N more milestones` note. Recency alone would have been wrong: a long-running project's oldest milestones are usually its finished ones, so the ten most recent could easily have been ten completed bars while the three in flight were hidden. The palette index is captured before slicing, so bars don't recolour when a milestone closes. Computed once per load rather than called from the template. The notice is plain text, not a link — the card already navigates there, and a nested link inside a clickable region is an accessibility trap. Payload unchanged; the API still returns every milestone. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs

bvandeusen added 2 commits 2026-08-02 19:42:44 -04:00

fix(projects): batch the summary queries — the fan-out was exhausting the pool ... be3a0ffaf9

Reported live: Projects and Snippets showed skeletons that never resolved,
/knowledge worked intermittently. The logs named it exactly:

    QueuePool limit of size 5 overflow 10 reached, connection timed out, 30.00
    GET /api/settings  500  30584.0ms
    GET /api/projects  200  30882.9ms

/api/projects was not hanging — it was waiting out the 30-second checkout
timeout and then returning 200 with summaries silently missing, because
_attach swallowed the TimeoutError. Nobody waits 31 seconds, so it read as a
hang.

THE SHAPE: routes/projects.py ran asyncio.gather over every project. Each
_attach called get_project_summary, which opened its own session for three
queries and then called get_project_milestone_summary — which opened one more
session PER MILESTONE. So 25 projects asked for roughly 250 concurrent
checkouts against a pool of 15 (SQLAlchemy's default 5 + 10 overflow).

That is why unrelated routes failed too. Snippets and /knowledge were never
broken; they queued behind the burst and inherited its timeout. /api/settings
returning 500 while /api/projects returned 200 is the same cause wearing two
faces.

The comment above the gather said "one backend pass instead of N+1 frontend
calls". It did remove the N+1 from the network — and recreated it against the
connection pool, where it is worse, because the browser had at least been
serialising those calls.

Now: get_project_summaries() does all projects in four queries and one session,
and get_project_milestone_summaries() does all milestones in two. Two sessions
total for the whole page, independent of how many projects exist.

The progress calculation is extracted to _progress_from_counts and shared by
both the batch and single paths, so the cancelled-exclusion rule cannot drift
into two versions that disagree about whether a milestone is finished.

Tests assert the SESSION COUNT, not just the values. An implementation that
returned identical output while opening a session per project would pass a
correctness test and reproduce the outage.

Deliberately NOT done: raising pool_size. It would move the cliff rather than
remove it, and this endpoint now needs two connections regardless of scale.

Closes #2384.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs

feat(projects): cap milestone bars at 10, open work first ... 5795fa908a

Roundtable's card rendered ~35 milestone bars and ran several viewport-heights
tall, so one tile dwarfed the grid and stopped being scannable — which is the
whole job of a card (#2391).

Now 10 bars, ordered OPEN WORK FIRST and newest first within each group, with
"+25 more milestones" beneath.

Ordering by recency alone would have been wrong, and the operator's call was to
lead with open work: a long-running project's oldest milestones are usually its
finished ones, so the ten most recent could easily have been ten completed bars
while the three in flight were the ones hidden. A card answers "what is
happening", not "what happened".

Three details that are the actual work:

- The palette index is captured from the FULL list before slicing. Colour keyed
  to visible position would have recoloured every bar on the card each time a
  milestone closed or was added.
- Computed once per load into a Map rather than called from the template. A
  helper invoked inside v-for re-runs on every render, and this one sorts.
- The overflow notice is plain text, not a link. The whole card already
  navigates to the project, and a link nested inside a clickable region is a
  trap for keyboard and screen-reader users.

Saying the count matters more than the cap: a list that simply stops reads as a
rendering bug, while a count reads as a summary.

Payload is unchanged — the API still returns every milestone. Capping
server-side would also need the total to travel with it, or the "+N" has
nothing to count from; not worth it while the response is two queries (#2384).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs

bvandeusen merged commit fefae606ed into main 2026-08-02 19:43:00 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#95