Performance Engineer (Inference, Training & GPU)

What You Will Do

• Optimize inference and serving end to end — latency, throughput, batching, caching, and scheduling — to serve our models efficiently at production scale.
• Write and tune GPU kernels (CUDA, Triton) for hot paths; drive kernel fusion, memory- and bandwidth-bound optimization, and low-precision (FP8/INT8) execution.
• Optimize training throughput and GPU utilization: parallelism strategies, communication/compute overlap, mixed precision, and eliminating pipeline stalls.
• Build performance models, profiling workflows, and observability that make throughput, latency, cost, utilization, and their tradeoffs legible across the stack.
• Own numerical correctness across precision, kernel, and hardware changes — treating correctness as part of performance, not separate from it.
• Partner with researchers to productionize models for serving and to make experiments run faster and more reliably.
• Where needed, work on the distributed systems that training and inference run on — but the core of the job is squeezing the most out of every GPU.

Key Qualifications

• You should excel at the fundamentals below — we index on inference, serving, GPU optimization, and training performance. Distributed-systems breadth is welcome, but secondary.
• Strong performance-engineering foundations: profiling, roofline analysis, latency/throughput optimization, and disciplined root-cause investigation.
• Deep GPU programming and optimization experience (CUDA and/or Triton) — kernel-level tuning, memory hierarchy, and bandwidth optimization at scale.
• Hands-on experience optimizing inference and serving for large models: batching, KV/prompt caching, quantization, and low-latency, high-throughput sampling.
• Hands-on experience optimizing training performance: parallelism, distributed communication, mixed/low precision, and utilization.
• Working knowledge of ML framework internals (PyTorch and/or JAX; torch.compile, XLA, or similar compiler paths).
• Strong proficiency in Python, with the ability to drop into C++/CUDA (and Rust or Go) as the work demands.
• High-ownership mindset — you measure yourself by throughput shipped and latency cut, not tickets closed.

Preferred Qualifications

• Experience at an AI lab or ML-native company, optimizing systems used directly by researchers and productionizing research code.
• Low-precision and numerics depth: FP8/INT8 quantization, mixed-precision, and detecting numerical regressions across hardware platforms.
• Distributed systems for large-scale training and inference — collective communication (NCCL), interconnects (NVLink), model and tensor parallelism, and fault tolerance. A strong plus, but not a substitute for the core skills above.
• Experience serving generative, diffusion, video, or 3D/spatial models — not just text LLMs.
• Multi-accelerator experience (GPU plus TPU or Trainium) and partnering with hardware vendors on accelerator capabilities.
• Building performance-modeling and observability frameworks for GPU utilization and cost.

Who You Are

Equal Employment Opportunity

California Pay Transparency

Pay Range

• $200-$300k base salary (good-faith estimate for San Francisco Bay Area upon hire; actual offer based on experience, skills, and qualifications)

Total Compensation

• Base salary plus equity awards

Salary History

• We do not request or consider prior compensation in making offers
• Compliance: Cal. Lab. Code §432.3 (pay scale disclosure & salary history ban); Cal. Lab. Code §1197.5 (Equal Pay Act); Cal. Gov. Code §12940 (FEHA); 42 U.S.C. §2000e (Title VII); 29 U.S.C. §621 (ADEA); 42 U.S.C. §12101 (ADA)