gem
Cloud Inference Engineer
Modular
Remote, USA
Full-time
Senior · 5+ years experience
Remote
USD 166500-273000 yearly / year
Discovered 1 weeks ago
KubernetesHelmGoGolangMachine learning
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
KubernetesHelmGo
Key Skills for This Role
KubernetesHelmGoGolangMachine learning
Full Job Posting
What you will do:
- Build & ship a LLM focused inference platform using best in class inference techniques (disaggregated inference, multi-node deployment of large models, high performance networking, distributed kv-cache management, high throughput batch processing, etc)
- Push the envelope for operational excellence with request-to-kernel observability, multi-cloud deployments, clever autoscaling, cold-start optimizations, and more.
- Collaborate with our kernels and genAI teams to achieve SOTA application performance by integrating SOTA kernel & serving optimizations with SOTA cluster optimizations.
- Build helm charts, kubernetes operators, and more to make a create simple, effective, maintainable deployments.
What you bring to the table:
5+ years of experience working in backend engineering
Experience with kubernetes and operating your own services
Ability to create durable, reusable software tools and libraries that are leveraged across teams and functions
Experience in machine learning technologies and use cases
Creativity and curiosity for solving complex problems, a team-oriented attitude that enables you to work well with others, and alignment with our culture
Strongly identifies with our core company cultural values.
Helpful but not required:
Experience with high performance computing / networking
Experience working on high scale ML inference infrastructure (traditional AI or genAI)
Familiarity with golang
What Modular brings to the table:
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Modular
Developer Advocate, MAX Inference & Serving (Copy)
Los Altos, USA
MidFull-time
Discovered 3 weeks agoView →
AI Inference Tools Engineer
, USA
Discovered 2 weeks agoFull-time
Inference Optimization Manager
Los Altos, USA
Discovered 3 weeks agoFull-time
Senior Community Engineer
Los Altos, USA
Discovered 3 weeks agoFull-time
Inference Optimization Engineer
Los Altos, USA
Discovered 3 weeks agoFull-time
Developer Advocate, MAX Inference & Serving
Los Altos, USA
Discovered 3 weeks agoFull-time
Developer Advocate, MAX Inference & Serving (Copy)
Los Altos, USA
Discovered 3 weeks agoFull-time
Product Manager, Cloud
, USA
Discovered 3 weeks agoFull-time