Agent: DRY pass + Status rate-metrics + real start/stop state machine #182

Merged
bvandeusen merged 3 commits from dev into main 2026-07-01 16:30:33 -04:00

3 Commits

Author SHA1 Message Date
bvandeusen 6282e753a9 fix(agent): real start/stop state machine — kill the stuck "stopping" pill
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m27s
The Status pill hung on "stopping" forever (operator-flagged 2026-07-01).
Root cause: the backend had no lifecycle state — status() only returned
running/stopped — so the UI FABRICATED "stopping" in JS as `!running &&
active>0`. That pill only cleared when the backend's `active` counter hit 0,
but stop() (a) blocked the HTTP handler on lease-release calls to curator and
(b) left `active>0` whenever a consumer wedged mid-submit/release to an
overloaded curator → "stopping" that never resolved.

Give the backend a real, truthful state it drives itself:
  stopped → starting → running → stopping → stopped
- start(): → starting; a downloader flips it to running on its FIRST
  successful lease (so "running" means curator is actually answering, not
  just "Start was clicked"). If curator's down it honestly stays "starting".
- stop(): → stopping; returns immediately (no handler block). A background
  monitor waits for the worker threads to actually exit, releases leases,
  then → stopped — bounded by STOPPING_TIMEOUT (20s) so a wedged submit can
  NEVER hold the UI in "stopping" again. In-flight work is handed back safely.
- Buttons follow the real state (Start only from stopped; both disabled
  through the transition), so you can't fight a transition.
- Log every Start/Stop button press (routes) and every transition (worker),
  so the Logs panel shows exactly what each button did.

Frontend now trusts s.state (drops the active>0 hack); VERSION → .8.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 16:20:12 -04:00
bvandeusen 91ea06be79 feat(agent): Status shows smoothed jobs/min + downloads/min (replace jumpy gauges)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m26s
The buffer / on-GPU / downloader counts flip many times a second, so a 3s
status poll only ever samples noise — the tiles looked frozen (same value
twice) or random (wildly different), reading as "the Status section doesn't
update" when the backend was in fact live (operator-flagged 2026-07-01).

Replace the three instantaneous gauge tiles with two derived RATE tiles:
  - jobs / min      — GPU throughput, from the monotonic `processed` counter
  - downloads / min — fetch throughput, from a new monotonic `downloaded`
                      counter (bumped when a job is decoded into the buffer)
Together they also show pipeline balance (dl/min > j/min ⇒ GPU-bound; the
reverse ⇒ GPU starved). Both are EWMA-smoothed over the poll deltas, clamped
at 0 (agent restart resets the counters), and skip a backgrounded-tab gap.

The still-useful instantaneous state is demoted, not lost: buffer stays as
the occupancy bar; downloaders/consumers/on-GPU move to the sub-line. `waited
out` (transient) gets promoted to a tile.

backend: worker.status() gains `downloaded`; `_bump(downloaded=)`.
frontend: retiled Status + rate math in applyStatus; VERSION → .7.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 16:01:29 -04:00
bvandeusen 98b2ac90dd refactor(agent): DRY pass on the GPU agent worker package
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
Consolidate genuine duplication in agent/fc_agent into single-source
helpers (behavior-preserving; DRY Pass process #594):

worker.py
- _fail(jid, image_id, exc, verb) — 4 terminal "fail this job" blocks
  (downloader HTTP-fault + decode, consumer non-transient + generic).
- _release(job_ids) (was _release_owned) — the one lease hand-back path;
  6 inline release([jid])+unhold sites now route through it.
- _stopped(stop_evt) + _abort_if_stopped(jid, stop_evt) — 4 stop-check
  -and-release blocks and every bare stop-check.
- _timed(stage) contextmanager — ~8 monotonic()/_record() timing pairs;
  records only on clean exit, matching the old skip-on-raise behavior.
- _ewma(prev, x, alpha) module fn — 3 EWMA updates in the autoscaler.

client.py
- _submit(path, payload) — submit / submit_embedding (retrying session).
- _post_quiet(path, payload) — heartbeat / fail / release fire-and-forget.

detectors.py
- Proposers._top(detector, image, cap) — merges components() and panels().

config.py
- _bool_env(name, default) — auto_start / auto_scale env parsing.

Left alone (recorded): the xyxy→norm-xywh conversion duplicated across
models.py/detectors.py (2 copies, independent wrapper modules — sharing
would couple them), and the _ensure_embedder/_ensure_proposers pair (same
lock shape, different concepts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:53:58 -04:00