ashby
Pre-training Research Engineer
Sciforium
San Francisco, USA
Full-time
Senior · 5+ years experience
Onsite
USD 165000-225000 yearly / year
Discovered 1 weeks ago
JAXFlaxXLAFSDPZeROMegatron
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
JAXFlaxXLA
Role Overview
As a Pre-training Research Engineer, you’ll focus on model implementation, pertaining and scaling, and improving the quality of our byte-native and multimodal foundation models. You’ll build and iterate quickly on research ideas, contribute production-grade training code and infrastructure, and help deliver high-quality base models that can serve real-world use cases at scale.
Key Skills for This Role
JAXFlaxXLAFSDPZeROMegatron
Full Job Posting
About the role
As a Pre-training Research Engineer, you’ll focus on model implementation, pertaining and scaling, and improving the quality of our byte-native and multimodal foundation models. You’ll build and iterate quickly on research ideas, contribute production-grade training code and infrastructure, and help deliver high-quality base models that can serve real-world use cases at scale.
Pre-training & Scaling
Train large byte-native and multimodal foundation models across massive, heterogeneous corpora.
Implement and evaluate new model architectures, training objectives, and optimization methods.
Develop stable pre-training recipes and run scaling experiments for novel architectures.
Conduct ablations and analyze training dynamics, model behavior, and base-model quality.
Work with data and distributed training engineers to improve training efficiency, reliability, and scalability.
Must-Haves
- 5+ years of experience in machine learning research or engineering, with a proven track record of developing and pre-training large language or multimodal foundation models.
- Software Engineering: Strong general software engineering skills, with the ability to write robust and performant training code.
- ML Foundations: Solid understanding of deep learning fundamentals and modern pre-training methods and literature.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Sciforium
GPU Kernel Engineer
San Francisco, USA
Discovered YesterdayFull-time
Data Center Real Estate & Development Specialist
San Francisco, USA
Discovered 3 days agoFull-time
Software Engineer, Fullstack
San Francisco, USA
Discovered 1 weeks agoFull-time
Senior Research Scientist
San Francisco, USA
Discovered 1 weeks agoFull-time
Software Engineer, Backend
San Francisco, USA
Discovered 1 weeks agoFull-time
Lead Software Engineer, Model Serving Platform
San Francisco, USA
Discovered 1 weeks agoFull-time
GPU Kernel Engineer
San Francisco, USA
Discovered 1 weeks agoFull-time
Distributed Training Engineer
San Francisco, USA
Discovered 1 weeks agoFull-time