Senior Software Engineer - AI Inference Performance

• More than 6 years of experience in full-stack LLM/VLM inference performance involving models, serving, distributed runtimes, kernels, and hardware. Your efforts result in measurable gains in production or production-representative environments.
• Strong programming skills in Python, Rust and/or C++, plus hands-on experience with CUDA or another GPU programming environment.
• Demonstrated expertise in speed-of-light analysis, roofline models, microbenchmarks, and tools including NVIDIA Nsight Systems and Nsight Compute. You convert profiles into testable hypotheses and validated progress.
• Deep understanding of GPU architecture, including Tensor Cores, memory hierarchy, caches, occupancy, synchronization, and numerical formats across hardware generations.
• Practical experience optimizing inference servers and model execution. You can choose techniques for the workload, including batching, scheduling, KV-cache management, quantization, speculative decoding, and various parallelism strategies
• Understanding of distributed systems and networking for accelerated computing. You can reason about collectives, topology, and scale-up versus scale-out performance.
• BS or MS in Computer Science, Computer Engineering, or a related field, or equivalent experience.

Ways to stand out from the crowd:

• Contributions to one or more high-performance AI projects. Examples include TensorRT-LLM, vLLM, SGLang, PyTorch, CUDA, Triton, or NCCL.
• Experience developing AI-agent-supported performance workflows that automatically gather and analyze profiles, identify bottlenecks, explore serving configurations, or produce optimized runtime and kernel code. You validate generated changes through reproducible, human-reviewed tests for performance, model quality, and correctness.
• Published research, conference presentations, technical talks, or blog posts that clearly explain inference performance methods and results.
• Delivered advancements for new LLM or VLM architectures, long-context inference, mixture-of-experts models, multimodal pipelines, or large-scale distributed serving.Successfully carrying these out will shape the future of AI inference performance!

With competitive salaries and a generous benefits package ( www.nvidiabenefits.com ), we are widely considered to be one of the technology world’s most desirable employers.

We have some of the most forward-thinking and hardworking people in the world working for us and, due to outstanding growth, our best-in-class engineering teams are rapidly growing.

If you're a creative and autonomous engineer with a real passion for technology, we want to hear from you!

You will also be eligible for equity and benefits .

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.