Staff AI Runtime Engineer at FlexAI — Santa Clara, USA | Base Career

Full Job Posting

Role Overview

At FlexAI , we’re building a high-performance, cloud-agnostic AI compute platform designed for next-generation training and inference workloads.

As a Staff AI Runtime Engineer , you’ll play a pivotal role in the design, development, and optimization of the core runtime infrastructure that powers distributed training and deployment of large AI models (LLMs and beyond).

This is a hands-on leadership role - perfect for a systems-minded software engineer who thrives at the intersection of AI workloads, runtimes, and performance-critical infrastructure .

You’ll own critical components of our PyTorch-based stack, lead technical direction, and collaborate across engineering, research, and product to push the boundaries of elastic, fault-tolerant, high-performance model execution.

Lead Runtime Design & Development

Own the core runtime architecture supporting AI training and inference at scale.
Design resilient and elastic runtime features (e.g. dynamic node scaling, job recovery) within our custom PyTorch stack.
Optimize distributed training reliability, orchestration, and job-level fault tolerance.

Drive Performance At Scale

Profile and enhance low-level system performance across training and inference pipelines.
Improve packaging, deployment, and integration of customer models in production environments.
Ensure consistent throughput, latency, and reliability metrics across multi-node, multi-GPU setups.

Build Internal Tooling & Frameworks

Design and maintain libraries and services that support model lifecycle: training, checkpointing, fault recovery, packaging, and deployment.

8+ years of experience in systems/software engineering, with deep exposure to AI runtime, distributed systems, or compiler/runtime interaction.
Experience in delivering PaaS services.
Proven experience optimizing and scaling deep learning runtimes (e.g. PyTorch, TensorFlow, JAX) for large-scale training and/or inference.
Strong programming skills in Python and C++ (Go or Rust is a plus).
Familiarity with distributed training frameworks , low-level performance tuning , and resource orchestration .
Experience working with multi-GPU , multi-node , or cloud-native AI workloads .
Solid understanding of containerized workloads, job scheduling, and failure recovery in production environments.

Nice to Have

Contributions to PyTorch internals or open-source DL infrastructure projects.
Familiarity with LLM training pipelines , checkpointing , or elastic training orchestration .
Experience with Kubernetes , Ray , TorchElastic , or custom AI job orchestrators .
Background in systems research , compilers , or runtime architecture for HPC or ML.
Start up previous experience

This position is In-Person and located at our Santa Clara, CA Office.

Staff AI Runtime Engineer

Job Fit Check

Full Job Posting

Role Overview

Lead Runtime Design & Development

Drive Performance At Scale

Build Internal Tooling & Frameworks

Apply for this job in 1 click

More jobs at FlexAI

Senior Backend Engineer

Senior Full Stack Engineer

Software Engineer, New Grad

Senior Backend Engineer

Staff DevOps Engineer/SRE

Senior FullStack Engineer

Senior DevOps Engineer/SRE

Senior Backend Engineer

Senior Full Stack Engineer

Software Engineer, New Grad

Senior Backend Engineer

Staff DevOps Engineer/SRE

Senior FullStack Engineer

Senior DevOps Engineer/SRE

Collaborate & Mentor

What You’ll Need to Be Successful

What We Offer

About FlexAI

Autofill Plugin