Principal Software Engineer - LLM Optimization

Job Responsibilities

• Own systematic benchmarking and performance characterization across all production LLM workloads. Establish reproducible baselines, catch regressions early, and quantify the impact of every configuration change before it touches production
• Design and execute quantization experiments — FP8, INT8/INT4 (GPTQ/AWQ), next-generation precision formats on current hardware — measuring accuracy delta, throughput improvement, memory reduction, and cost-per-token impact
• Drive speculative decoding strategy across the model portfolio: draft model, n-gram, and multi-token prediction approaches. Own acceptance rate measurement and per-workload configuration recommendations
• Build and maintain a GPU efficiency scorecard: utilization, memory headroom, cost per 1K tokens, and waste identified — giving leadership a data-driven view of platform efficiency at all times
• Benchmark our platform against external providers and published industry numbers — know what good looks like, and close the gap
• Lead inference engine upgrade evaluations: new scheduler architectures, async tensor parallelism, disaggregated prefill/decode, advanced speculative decoding — systematic validation before production promotion
• Collaborate with the EKS and disaggregated serving teams on KV-cache optimization, prefix caching strategies, and multi-node serving architecture
• Design and run GPU chaos engineering: induced failure scenarios, hardware diagnostic monitoring, detection and recovery measurement
• Architect and govern agentic AI-enabled engineering workflows (using enterprise-authorized tools within the work environment) to improve delivery speed, code quality, and operational outcomes at scale (e.g., AI-driven PR review assistance, test generation/maintenance, release readiness checks, incident triage and root-cause acceleration), while defining guardrails for validation, security, resiliency, and reuse across teams.
• Apply knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation at scale.

Required qualifications, capabilities, and skills