Inference Optimization ML Engineer

Own inference performance end-to-end — diagnose and improve latency, throughput, and efficiency of large foundation models in production

Build systematic performance attribution: latency decomposition (compute vs. memory bandwidth vs. I/O), bottleneck identification, and prioritization across model families

Apply and develop optimization techniques including quantization, pruning, distillation, operator fusion, and model compilation (e.g., TensorRT, torch.compile, XLA)

Optimize attention mechanisms, KV caching, and memory layouts for large multimodal models (vision, video, language, proprioception)

Work with kernel-level tooling (e.g., CUDA, Triton) to identify hotspots and implement or tune custom kernels where needed

Build benchmarking and regression detection infrastructure: latency baselines, throughput curves, and automated detection of performance regressions across model versions

Collaborate closely with research engineers to translate model innovations into optimized, deployment-ready implementations

3+ years of experience in inference optimization, ML systems, or a closely related field

Deep hands-on experience with modern ML stacks (PyTorch required; JAX a plus)

Strong understanding of compute, memory bandwidth, and I/O bottlenecks in large model inference

Experience with model optimization techniques: quantization (INT8/FP8/AWQ), distillation, pruning, and compilation

Familiarity with inference serving frameworks (e.g., Triton, TensorRT, vLLM, TorchServe)

Exceptional debugging and measurement ability: turn "inference is slow" into clear bottlenecks, experiments, and validated improvements

High ownership mindset and comfort in a fast-moving environment