Staff Software Engineer, GPU Inference

Minimum Qualifications

• 8+ years of software engineering experience, including substantial individual-contributor ownership of complex production systems.
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
• Strong programming ability in C++ and Python, including experience with multithreading, concurrency, memory management, and performance-sensitive software.
• Hands-on experience with a high-performance model-serving framework such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or an equivalent internally developed system.
• Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling methodology.
• Experience debugging distributed systems across multiple layers rather than treating the serving framework or accelerator runtime as a black box.
• Experience with Linux, containers, Kubernetes or comparable orchestration systems, observability, CI/CD, and operating latency-sensitive services in production.
• Ability to design rigorous benchmarks, interpret noisy performance results, identify bottlenecks, and translate findings into production improvements.
• Strong communication and technical leadership skills, with a demonstrated ability to drive ambiguous cross-functional projects to completion.
• Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

Preferred Qualifications

• Experience with AMD Instinct accelerators and the ROCm ecosystem, including HIP, RCCL, rocprofiler, AMD SMI, AITER, hipBLASLt, Composable Kernel, or related libraries and tools.
• Deep CUDA experience that demonstrates an ability to transfer GPU systems knowledge across accelerator platforms.
• Experience modifying or contributing to vLLM, SGLang, PyTorch, Triton, TensorRT-LLM, or another open-source ML systems project.
• Experience optimizing prefill-heavy or disaggregated prefill/decode inference architectures.
• Understanding of KV-cache transfer, prefix caching, continuous batching, chunked prefill, request scheduling, and memory-aware admission control.
• Experience with multi-GPU and multi-node inference, including tensor parallelism, pipeline parallelism, expert parallelism, RDMA, collective communication, and failure handling.
• Experience optimizing Mixture-of-Experts or multimodal models.
• Knowledge of GPU kernel optimization, operator fusion, graph capture, attention kernels, GEMM tuning, and communication/computation overlap.
• Experience with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4, including validation of their numerical and model-quality effects.
• Experience building numerical-comparison, determinism, model-validation, or performance-regression test systems.
• Experience collaborating directly with accelerator vendors, framework maintainers, or open-source communities.

Why Join Cerebras

Apply today and become part of the forefront of groundbreaking advancements in AI!