bvandeusen/FabledCurator
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases 33 Wiki Activity

fix(agent): sleep mode — an empty queue sheds to one downloader and backs the lease poll off to 15 min
CI / lint (push) Successful in 2s
Details
CI / frontend-build (push) Successful in 20s
Details
CI / backend-lint-and-test (push) Successful in 32s
Details
CI / integration (push) Successful in 3m29s
Details

Browse Source 
Operator-flagged on the deployed .5 build: the autoscaler grew the pool 1→8
against an EMPTY queue (an empty buffer read as 'GPU starving' regardless of
WHY), and every downloader kept polling lease every 10s all night.

- New idle signal straight from the lease results: an empty lease sets _idle,
  any jobs clear it. The occupancy-low branch now distinguishes three cases:
  queue empty → shed to ONE polling downloader; pinned at the bandwidth cap →
  shed toward 3; cap headroom + work flowing → grow.
- Idle lease polls back off exponentially per downloader to
  IDLE_POLL_MAX_SECONDS (15 min) and reset the moment work appears — so an
  idle night costs one HTTP call per 15 min, and new work is noticed within
  at most ~15 min (operator-accepted trade-off).
- UI hint: 'idle — queue empty, lease poll backed off'; /status gains idle.
  Agent build 2026-07-02.6.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
This commit is contained in:
Bryan Van Deusen
2026-07-02 20:32:11 -04:00
parent f08a64f7ae
commit b1cfbcc06a
2 changed files with 36 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
agent/fc_agent/app.py
+3 -2
View File
@@ -21,7 +21,7 @@ log = logging.getLogger("fc_agent.app")
# Bump on every agent change. The page embeds this and /status reports it; the UI
# warns to reload when they differ — so a stale browser-cached page can't be
# mistaken for "the new image didn't deploy". (Belt-and-braces with no-store.)
VERSION = "2026-07-02.5 · cap-aware autoscaler: downloaders stop growing (and shed) when the bandwidth cap — not concurrency — is the bottleneck"
VERSION = "2026-07-02.6 · sleep mode: an empty queue sheds to one downloader and backs the lease poll off to 15 min"
logbuf.install()
cfg = Config.from_env()
@@ -345,7 +345,8 @@ _PAGE = """<!doctype html><html><head><meta charset=utf-8>
if(document.activeElement!==autochk) autochk.checked=!!s.auto
conc.disabled=!!s.auto; conc.style.opacity=s.auto?0.55:1
conchint.textContent=(s.auto?('auto-tuning downloaders to keep the GPU fed · max '+CAP):('manual downloaders · max '+CAP))
+(s.bw_capped?' · holding at the bandwidth cap (more downloaders would not go faster)':'')
+(s.idle?' · idle — queue empty, lease poll backed off (new work noticed within ~15 min)'
:(s.bw_capped?' · holding at the bandwidth cap (more downloaders would not go faster)':''))
if(document.activeElement!==conc) conc.value=s.concurrency
conc.max=CAP
// Connection pill + queue come only from the /status poll (the Start/Stop POST