Staff Software Engineer, Inference Performance Optimization, GenAI, DeepMind

Responsibilities

• Analyze and optimize AI inference workloads across the application, model, and distributed fleet infrastructure layers to methodically increase throughput-per-GPU and reduce latency.
• Design and implement inference optimization techniques.
• Investigate and resolve complex model inference performance bottlenecks across the stack.
• Model the latency-to-cost impacts of system variables (such as batch-sizing and utilization goals) and translate these insights into actionable signals that drive production systems.
• Develop investigative tools and metrics (e.g., compute/FLOPs funnels) that track where compute is spent across the fleet.
• - Analyze and optimize AI inference workloads across the application, model, and distributed fleet infrastructure layers to methodically increase throughput-per-GPU and reduce latency. - Design and implement inference optimization techniques. - Investigate and resolve complex model inference performance bottlenecks across the stack. - Model the latency-to-cost impacts of system variables (such as batch-sizing and utilization goals) and translate these insights into actionable signals that drive production systems. - Develop investigative tools and metrics (e.g., compute/FLOPs funnels) that track where compute is spent across the fleet.

Minimum qualifications:

Preferred qualifications: