ashby
ML Engineer, Inference Optimization
build-ai
San Francisco, USA
Full-time
Onsite
$200K – $320K • Offers Equity
Discovered 2 weeks ago
Machine learning systemsInference optimizationModel servingCompilersCUDA and GPU kernelsQuantization
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
Machine learning systemsInference optimizationModel serving
Role Overview
Inference represents approximately 90% of compute spend, so the role focuses on reducing inference cost while improving speed and maintaining sufficient quality.
The ML engineer will help scale the data engine and product without GPU costs becoming prohibitive.
Key Skills for This Role
Machine learning systemsInference optimizationModel servingCompilersCUDA and GPU kernelsQuantization
Full Job Posting
Job Summary
Inference represents approximately 90% of compute spend, so the role focuses on reducing inference cost while improving speed and maintaining sufficient quality.
The ML engineer will help scale the data engine and product without GPU costs becoming prohibitive.
Key Responsibilities
- Own inference latency, throughput, and cost per unit of work such as tokens, frames, or jobs.
- Optimize kernels, batching, quantization, compilation, serving, and hardware utilization.
- Profile pipelines with Nsight, PyTorch Profiler, or equivalent tools and ship fixes.
- Partner with research and product so accurate models are affordable to run at scale.
- Build serving and evaluation paths that make inference costs visible during experiments.
- Treat cost as a first-class metric alongside quality.
Must-Have Qualifications
- Real inference optimization experience in serving, compilers, CUDA or kernels, quantization, or a similar area.
- Comfortable with Python and C++ or Rust for performance-critical paths.
- Ability to reason about dollars and tokens or frames per second, not only accuracy tables.
- Familiarity with PyTorch or JAX and profiling tools.
- Comfort working in a small research team shipping under cost pressure.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at build-ai
Events & Field Marketing Manager
San Francisco, USA
ManagerFull-time
Build AI is seeking an Events & Field Marketing Manager to establish and lead its events function across AI and robotics conferences, customer experiences, recruiting events, and community gatherings. The role owns strat
Discovered 2 weeks agoView →
Controller
San Francisco, USA
ManagerFull-time
Build AI is hiring a Controller to build and own the accounting infrastructure for global hardware, manufacturing, data-collection, and payment operations. The role covers close, consolidation, international entities, in
Discovered 2 weeks agoView →
Head of Design & Taste
San Francisco, USA
LeadFull-time
Build AI is seeking a design leader to define and execute the visual direction across its product, brand, website, job board, internal tools, merchandise, and physical objects. The role requires a strong portfolio, exper
Discovered 2 weeks agoView →
Data Scientist, Marketplace Incentives
San Francisco, USA
Full-time
Build AI is seeking a Data Scientist to optimize marketplace incentives, payouts, and collection behavior against a dataset quality objective. The role requires quantitative problem-solving, experiment design and analysi
Discovered 2 weeks agoView →
Lead Engineer, Data Platform
San Francisco, USA
LeadFull-time
Build AI is seeking a lead engineer to own a petabyte-scale data platform from on-device video capture through upload, storage, dataset packaging, and delivery. The role requires production media or sensor-data experienc
Discovered 2 weeks agoView →
Software Engineer, Data Visualization
San Francisco, USA
Full-time
Build AI is hiring a Software Engineer, Data Visualization to build analysis and visualization products for datasets, collection systems, quality metrics, and evaluations. The role requires strong software engineering ex
Discovered 2 weeks agoView →
Software Engineer, Scaling
San Francisco, USA
Full-time
Build AI is hiring a Software Engineer, Scaling, to build internal products for field collection operations across tasking, devices, sites, shifts, and local tooling. The role suits a generalist engineer who ships practi
Discovered 2 weeks agoView →
Software Engineer, Generalist
San Francisco, USA
Full-time
Build AI is hiring a generalist software engineer to identify high-impact technical gaps and ship production software across data, infrastructure, hardware integrations, and customer products. Candidates need strong full
Discovered 2 weeks agoView →
Events & Field Marketing Manager
San Francisco, USA
Discovered 2 weeks agoFull-time
Controller
San Francisco, USA
Discovered 2 weeks agoFull-time
Head of Design & Taste
San Francisco, USA
Discovered 2 weeks agoFull-time
Data Scientist, Marketplace Incentives
San Francisco, USA
Discovered 2 weeks agoFull-time
Lead Engineer, Data Platform
San Francisco, USA
Discovered 2 weeks agoFull-time
Software Engineer, Data Visualization
San Francisco, USA
Discovered 2 weeks agoFull-time
Software Engineer, Scaling
San Francisco, USA
Discovered 2 weeks agoFull-time
Software Engineer, Generalist
San Francisco, USA
Discovered 2 weeks agoFull-time