ashby
ML Model Serving Engineer
Sesame
San Francisco, USA
Full-time
Mid
Onsite
Discovered 1 weeks ago
awsazuregcpkubernetespytorch
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
awsazuregcp
Key Skills for This Role
awsazuregcpkubernetespytorch
Full Job Posting
Responsibilities:
- Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.
- Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer to power a new consumer product category.
- Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest techniques in high-performance model serving.
- Work with the training team to identify opportunities to produce faster models without sacrificing quality.
- Use techniques like in-flight batching, caching, and custom kernels to speed up inference.
- Find ways to reduce model initialization times without sacrificing quality.
Required Qualifications:
Expert in some differentiable array computing framework, preferably PyTorch.
Expert in optimizing machine learning models for serving reliably at high throughput, with low latency.
Significant systems programming experience; ex. Experience working on high-performance server systems—you’d be just as comfortable with the internals of VLLM as you would with a complex PyTorch codebase.
Significant performance engineering experience; ex. Bottleneck analysis in high-scale server systems or profiling low-level systems code.
Always up to date on the latest techniques for model serving optimization.
Preferred Qualifications:
Familiarity with high-performance LLM serving; ex. experience with VLLM, SGlang deployment, and internals.
Experience with a public cloud platform such as GCP, AWS, or Azure.
Experience deploying and scaling inference workloads in the cloud using Kubernetes, Ray, etc.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Sesame
Electrical Engineering Technician (Contract)
San Francisco, USA
Discovered 2 days agoContract
Research Scientist
San Francisco, USA
Discovered 1 weeks agoFull-time
Staff Software Engineer, Backend Infrastructure
San Francisco, USA
Discovered 1 weeks agoFull-time
Fabrication Technician
San Francisco, USA
Discovered 1 weeks agoFull-time
Product Designer
San Francisco, USA
Discovered 1 weeks agoFull-time
Research Engineer
San Francisco, USA
Discovered 1 weeks agoFull-time
Product Manager, Hardware
San Francisco, USA
Discovered 1 weeks agoFull-time
iOS Engineer
San Francisco, USA
Discovered 1 weeks agoFull-time
