Research Engineer - Pre-training
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
Key Skills for This Role
Full Job Posting
Key Responsibilities
- Distributed pretraining : Implement and optimize model-parallel training. Data, pipeline, and tensor parallelism for large models on heterogeneous GPUs under low-bandwidth, high-latency links.
- Performance optimization : Implement techniques that reduce communication overhead while maintaining model convergence in challenging network environments.
- Elasticity and fault tolerance : Make runs survive node churn. Robust checkpointing, state synchronization, and recovery as participants join and leave.
- Run instrumentation : Build the monitoring that shows throughput, bottlenecks, and model quality across hundreds of devices.
What We're Looking For
Hands-on distributed training (required) : You've trained models across many devices in PyTorch with FSDP, DeepSpeed, Megatron, or your own implementation. You understand data, tensor, and pipeline parallelism.
Strong engineering : Production-quality Python. Concurrency, failure handling, profiling before optimizing.
Evidence of execution : Shipped systems, research code, open-source work, or serious personal projects.
Mission alignment : You believe Protocol Learning is the viable third path for collective, trustless, and sovereign AI.
Nice to Have
Hands-on experience training or serving large language models such as Nemotron, Qwen or OLMo.
Experience with P2P networking and NAT traversal.
Experience with post-training and RL.
Experience with inference and serving systems.
Experience at proprietary, open-weight and open-source AI labs
Compensation & Benefits
- Equity-Heavy Package : We offer significant ownership for key technical contributors in addition to a high base salary.
- Remote-First Culture : Flexible work environment with team members distributed globally.
- Visa Sponsorship : Optional full visa sponsorship and relocation support to either Australia or the US.
- Open Problems : Training and serving frontier models on hardware you don't control, over networks you don't own, mostly has no published answers yet. You'll write some of the first ones.
FYI's
We work remotely across the world, with the main teams in Australia and North America. You'll need to be comfortable working across timezones.
Applicants must have professional-level English proficiency (written and spoken).
Recruiters: we aren't looking for agency support at this time. We'll reach out if we need help.
We are backed by Union Square Ventures and other tier-1 investors, and we are a world-class, deeply technical team of ML researchers. Pluralis is unapologetically ideological. We believe AI, and the world, end up on a better path if we succeed in implementing the protocol for intelligence. If this resonates, please apply.
About Pluralis Research
AI research lab developing collectively owned foundation models through decentralized collaborative training across many participants.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Pluralis Research
Research Engineer - Decentralized Training and Inference Verification
, USA
Operations Associate
, AUS
Executive Assistant
, AUS
Research Scientist
, USA
Research Engineer Intern
, AUS
Machine Learning Engineer - ML Training Platform
, USA
Research Scientist Intern
, USA
Research Engineer - Decentralized Training and Inference Verification
, USA
Research Engineer - Geo-Distributed Inference
, USA