GPU enablement (#872) cancelled — not worth the Pascal-specific build for a
modest CPU→GPU win on an old P4. Remove the dead GPU code (device.py, the CUDA
provider branch in tagger, the .to('cuda') path in embedder) so nothing carries
it forward.
Instead, bound CPU inference threads by default so the ml-worker is a predictable
core consumer on a SHARED node — the intended scaling model is multiple worker
replicas (each --concurrency=1, each its own cgroup limit), not one big
container. ONNX Runtime and torch otherwise size their thread pools to ALL host
cores, so each replica would grab every core and oversubscribe / starve the
co-located DB+web. Cap both to _INTRA_OP_THREADS=4 (matches the prior per-worker
cpus:4 unit): run N replicas where N×4 stays within the cores allotted to ML.
- tagger: ort.SessionOptions().intra_op_num_threads = 4 (CPUExecutionProvider).
- embedder: torch.set_num_threads(4).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bvandeusen
60a9c9e6ef