ashby
Member of Technical Staff, Inference
Inferact
Remote, USA
Full-time
Senior
Remote
Discovered 1 weeks ago
PythonPyTorchvLLMTensorRT-LLMSGLangTGI
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonPyTorchvLLM
Role Overview
We're looking for an inference runtime engineer to push the boundaries of what's possible in LLM and diffusion model serving. Models grow larger. Architectures shift: mixture-of-experts, multimodal, agentic. Every breakthrough demands innovations on the inference engine itself. You'll work at the core of vLLM, optimizing how models execute across diverse hardware and architectures. Your work will directly impact how the world runs AI inference.
Key Skills for This Role
PythonPyTorchvLLMTensorRT-LLMSGLangTGI
Full Job Posting
About the Role
We're looking for an inference runtime engineer to push the boundaries of what's possible in LLM and diffusion model serving. Models grow larger. Architectures shift: mixture-of-experts, multimodal, agentic. Every breakthrough demands innovations on the inference engine itself. You'll work at the core of vLLM, optimizing how models execute across diverse hardware and architectures. Your work will directly impact how the world runs AI inference.
Skills and Qualifications
- Minimum qualifications:
- Bachelor's degree or equivalent experience in computer science, engineering, or similar.
- Deep understanding of transformer architectures and their variants.
- Strong programming skills in Python with experience in PyTorch internals.
- Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI).
- Ability to read and implement model architectures and inference techniques from research papers.
- Demonstrate the ability to contribute performant and maintainable code and debug in complex ML codebases.
- Preferred qualifications:
- Deep understanding of KV-cache memory management, prefix caching, and hybrid model serving.
- Familiarity with RL frameworks and algorithms for LLMs.
- Experience with multimodal inference (audio/image/video/text).
- Contributions to open-source ML or system infrastructure projects.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Inferact
Member of Technical Staff, Kernel Engineering
San Francisco, USA
SeniorFull-time
Discovered 2 days agoView →
Member of Technical Staff, Cloud Orchestration
San Francisco, USA
SeniorFull-time
Discovered 2 days agoView →
Member of Technical Staff, Performance and Scale
San Francisco, USA
SeniorFull-time
Discovered 2 days agoView →
Member of Technical Staff, TPU & AMD GPU Performance Engineering
San Francisco, USA
SeniorFull-time
Discovered 1 weeks agoView →
HR / People Lead
San Francisco, USA
Discovered 2 days agoFull-time
IT Support & Operations Engineer
San Francisco, USA
Discovered 2 days agoFull-time
Member of Technical Staff, Kernel Engineering
San Francisco, USA
Discovered 2 days agoFull-time
Member of Technical Staff, Cloud Orchestration
San Francisco, USA
Discovered 2 days agoFull-time
Member of Technical Staff, Performance and Scale
San Francisco, USA
Discovered 2 days agoFull-time
Inference Engineering, Co-op
San Francisco, USA
Discovered 2 days agoInternship
Head of Legal
San Francisco, USA
Discovered 2 days agoFull-time
Member of Technical Staff, TPU & AMD GPU Performance Engineering
San Francisco, USA
Discovered 1 weeks agoFull-time