Commit Graph
4 Commits
Author SHA1 Message Date
bvandeusenandClaude Fable 5 a7abcc41ca feat(triage): failed-processing triage — probe errored files, flag defects, recover (#125 C1-C3)
An errored GPU job's stored reason is a suspicion; the file probe is the
verdict. A 15-min beat sweep (triage_gpu_errors) runs verify_integrity's own
probe (sha256 + decode) on each errored image ONCE and writes both verdicts:
ImageRecord.integrity_status and the new GpuJob.triage_status ('defect' |
'file_ok', migration 0072). Every classification logs at WARNING so it
surfaces in Logs/System Activity.

- 'defect' rows are excluded from /retry_errors (re-running a known-bad file
  burns agent time re-minting the tombstone); response now reports
  defects_kept and the GpuAgentCard toast says so.
- GET /api/gpu/errors: triage view — reason buckets (classify_reason),
  probe verdicts, per-job detail. POST /errors/triage runs the sweep now.
- POST /api/gpu/errors/<id>/recover: reuses the Layer-2 refetch pattern —
  delete the defective copy + record (full cascade takes the tombstones too)
  and re-poll its subscription Source so a fresh copy re-imports and re-enters
  the pipeline; 'no_source' when nothing pollable resolves.
- New 'Failed processing' card (GpuTriageCard) in Maintenance: verdict counts,
  reason summary, probe-now, defect list with thumbnails + per-image Recover.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 12:36:02 -04:00
bvandeusenandClaude Fable 5 686808d3f3 feat(gpu): "Retry errored jobs" — scoped requeue of errors only
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m26s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
After an agent-side fix (e.g. the short-video sampler), the errored jobs
(~2.8k) have exhausted their 3 attempts and stay parked: backfill skips
images that already have a job, and /reprocess is the nuclear option (it
resets the 179k DONE jobs too). There was no way to re-run just the errors.

POST /api/gpu/retry_errors resets every status='error' job (all task types)
to pending with attempts=0 and the stored error cleared — a small inline
UPDATE that returns {requeued: n} so the UI toast can show the count.

UI: a "Retry errored jobs" button on the GPU-agent card, right under the
queue tiles; disabled when errored==0. With the agent now logging ffmpeg's
stderr on failure, retrying also reveals which errors were real vs victims
of the fps-filter bug.

Test: retry_errors requeues the errored job (fresh attempts, error cleared)
and leaves done work untouched; asserts via column selects (Core-DML
gotcha), not ORM refresh.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-01 21:09:07 -04:00
bvandeusenandClaude Opus 4.8 80f8eb4756 feat(gpu): re-process trigger to apply new crop detectors to the existing library (#1202)
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
CI / lint (push) Successful in 3s
The siglip/ccip backfills skip images that already have current-version regions,
so adding crop detectors only affected NEW images — the back-catalogue would
never be re-cropped. Add a reprocess trigger that resets every done/error job of
a task back to pending, so the agent re-runs the FULL pipeline (figure detection
+ CCIP + concept/panel crops) over the whole library under the current detectors.

- reprocess_gpu_jobs(task='ccip') task + POST /api/gpu/reprocess.
- gpu store reprocess() + GpuAgentCard "Re-process library (re-detect + re-crop)"
  button with a confirm (it's heavy).
- Test: a done job resets to pending (attempts cleared).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-30 16:09:37 -04:00
bvandeusenandClaude Opus 4.8 d91eef7a4b feat(gpu): GPU agent admin card — token, queue, backfill (#114)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m25s
The FC-side control surface the operator asked for: Settings → Tagging → "GPU
agent". Generate/reveal/copy/rotate the agent bearer token (with the FC URL to
point the agent at), see the live job-queue depth (pending/in-flight/done/
errored, polled), and a "Queue character embedding (CCIP)" button that triggers
the library backfill. Plain-HTTP-safe copy (copyText resolves on success,
throws on fail). Closes the "how do I get the token in the UI" gap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-29 11:53:46 -04:00