ashby
Software Engineer – AI Inference Engine
FriendliAI
San Francisco, USA
Full-time
Mid · 5+ years experience
Hybrid
Discovered 1 weeks ago
PythonC++GPUGPU kernelsCompiler infrastructure
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonC++GPU
Role Overview
We are seeking a highly technical Inference Engine Engineer to optimize the performance and efficiency of our core inference engine.
In this role, you will focus on designing, implementing, and optimizing GPU kernels and supporting infrastructure for next-generation generative and agentic AI workloads.
Your work will directly power the most latency-critical and compute-intensive systems deployed by our customers.
We are looking for an exceptional engineer with a strong foundation in GPU programming and compiler infrastructure.
The ideal candidate enjoys pushing performance boundaries and has experience supporting production-scale machine learning applications.
Key Skills for This Role
PythonC++GPUGPU kernelsCompiler infrastructure
Full Job Posting
About the Job
We are seeking a highly technical Inference Engine Engineer to optimize the performance and efficiency of our core inference engine. In this role, you will focus on designing, implementing, and optimizing GPU kernels and supporting infrastructure for next-generation generative and agentic AI workloads. Your work will directly power the most latency-critical and compute-intensive systems deployed by our customers.
We are looking for an exceptional engineer with a strong foundation in GPU programming and compiler infrastructure. The ideal candidate enjoys pushing performance boundaries and has experience supporting production-scale machine learning applications.
Key Responsibilities
- Design and optimize custom GPU kernels for AI (e.g., transformer and diffusion) workloads
- Contribute to the development of FriendliAI’s kernel compiler, memory planner, runtime, and other core components.
- Collaborate with cloud and infrastructure engineers to ensure end-to-end inference performance
- Analyze performance bottlenecks across the software and hardware stack, and implement targeted optimizations
- Drive support for new model architectures and tensor compute patterns
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at FriendliAI
Solutions Architect - AI Inference Specialist
San Francisco, USA
MidFull-time
Discovered 1 weeks agoView →
Solutions Architect - AI Inference Specialist
San Francisco, USA
Discovered 1 weeks agoFull-time
Software Engineer – GPU Kernel
San Francisco, USA
Discovered 1 weeks agoFull-time
Solutions Architect - AI Model Specialist
San Francisco, USA
Discovered 1 weeks agoFull-time
Software Engineer – Platform Security
San Francisco, USA
Discovered 1 weeks agoFull-time
Software Engineer - Full Stack
San Francisco, USA
Discovered 1 weeks agoFull-time
Software Engineer – AI Agents
San Francisco, USA
Discovered 1 weeks agoFull-time
Director of Product Management
San Francisco, USA
Discovered 1 weeks agoFull-time
Software Engineer - Senior Backend
San Francisco, USA
Discovered 1 weeks agoFull-time