Inference Infrastructure Engineer

Design and operate large-scale infrastructure to run model workloads across cloud and on-prem environments

Build and maintain Kubernetes-based deployment pipelines for managing distributed ML workloads

Own resource scheduling and orchestration across GPU clusters — optimizing utilization, workload balancing, and cost-performance tradeoffs

Integrate and manage ML frameworks and model serving systems (e.g., Triton, Ray Serve, TorchServe) across research and production use cases

Build tooling for model deployment, versioning, and observability to support fast iteration cycles

Contribute to the reliability and scalability of the infrastructure stack as model complexity and deployment footprint grow

3+ years of experience in ML infrastructure, MLOps, or distributed systems

Strong proficiency with Kubernetes and containerized deployment pipelines

Experience with GPU orchestration and resource scheduling across large distributed jobs

Experience with cloud providers (e.g., AWS, GCP) and hybrid cloud/on-prem infrastructure

Familiarity with ML frameworks (e.g., PyTorch, JAX) and model serving tools (e.g., Triton, Ray Serve, TorchServe)

Strong debugging instincts and ownership mentality — comfortable driving issues to resolution across the stack

Experience with streaming systems or high-throughput data transport (e.g., Kafka, gRPC, NATS)

Background in networking, low-latency systems, or network-aware scheduling