ashby
Software Engineer, Model Runtime
OpenAI
Remote, USA
Full-time
Remote
$266K – $445K • Offers Equity
Discovered 3 weeks ago
C++RustPythonLLM inferenceInference runtime developmentDistributed systems
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
C++RustPython
Role Overview
Build the model runtime within an inference engine for complex frontier models running at scale on custom silicon.
Develop a production-grade runtime that translates demanding inference workloads into efficient execution while optimizing throughput, latency, utilization, and reliability.
Work across model architecture, distributed systems, compilers, kernels, and silicon.
Key Skills for This Role
C++RustPythonLLM inferenceInference runtime developmentDistributed systems
Full Job Posting
About the Role
Build the model runtime within an inference engine for complex frontier models running at scale on custom silicon.
Develop a production-grade runtime that translates demanding inference workloads into efficient execution while optimizing throughput, latency, utilization, and reliability.
Work across model architecture, distributed systems, compilers, kernels, and silicon.
Responsibilities
- Design and implement the LLM inference runtime for frontier models on custom silicon.
- Build scheduling, continuous batching, memory management, KV-cache management, and execution orchestration.
- Develop distributed execution across chips, hosts, and racks, including partitioning, communication, and synchronization.
- Optimize latency, throughput, memory efficiency, and hardware utilization across models and serving workloads.
- Partner with kernel, compiler, architecture, and silicon teams to remove performance bottlenecks.
- Enable new model features, execution patterns, numerical formats, and hardware capabilities.
- Create profiling, observability, benchmarking, and performance-modeling tools.
- Debug correctness, performance, and reliability issues across software, communication layers, and hardware.
- Turn workload insights into requirements for future silicon and system architecture.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at OpenAI
Creative Operations Lead, Business Marketing
San Francisco, USA
Discovered TodayFull-time
Software Engineer, Manufacturing Infrastructure
San Francisco, USA
Discovered TodayFull-time
Staff PLM & Engineering Applications Engineer, Consumer Devices
San Francisco, USA
Discovered TodayFull-time
Development Lead
, USA
Discovered TodayFull-time
Data Center Shell Planning Lead, Industrial Compute
, USA
Discovered TodayFull-time
Strategic Finance, Compute
San Francisco, USA
Discovered TodayFull-time
Senior Counsel, Litigation & Regulatory
San Francisco, USA
Discovered TodayFull-time
Business Systems Lead, Procure-to-Pay
San Francisco, USA
Discovered TodayFull-time