Software Engineer, ML Infrastructure

Expected Qualifications

• 5+ years of professional experience in ML infrastructure, systems engineering, or production ML roles.
• Strong software engineering fundamentals; ability to write robust, maintainable production code .
• Deep hands-on experience with LLM inference infrastructure , including: PyTorch (required) TensorFlow (working knowledge)
• PyTorch (required)
• TensorFlow (working knowledge)
• Proven experience with GPU inference optimization , including: TensorRT / TensorRT-LLM vLLM Triton Inference Server SGLang or similar serving runtimes
• TensorRT / TensorRT-LLM
• vLLM
• Triton Inference Server
• SGLang or similar serving runtimes
• Strong understanding of LLM internals , such as: Transformer architectures Attention and KV caching Batching, streaming, and token-level generation
• Transformer architectures
• Attention and KV caching
• Batching, streaming, and token-level generation
• Experience running ML systems in production with high traffic and SLAs
• Comfortable working in Linux-based, cloud production environments

Preferred Qualifications

• Experience deploying LLMs on Kubernetes and GPU clusters.
• Familiarity with CUDA, NCCL , or low-level GPU performance concepts.
• Experience with: Model sharding and parallelism strategies Multi-GPU inference Streaming inference systems
• Model sharding and parallelism strategies
• Multi-GPU inference
• Streaming inference systems
• Knowledge of observability for ML systems (metrics, latency breakdowns, GPU monitoring).
• Experience working at startups or owning systems with minimal abstraction layers.

Additional Information

Compensation & Benefits

• Market aligned compensation and benefits
• Founding engineer equity ( Equity is a significant component of this role and will be discussed)
• Medical, Dental, Vision, Life insurance, 401-K, In-office lunch etc.