Helix AI Engineer, Pretraining

Responsibilities

• Design and train large-scale foundation models across multimodal data (e.g., text, vision, and robot data)
• Develop pretraining strategies that improve generalization, reasoning, and transfer to downstream embodied tasks
• Explore and implement architectures including transformer-based and emerging foundation model paradigms
• Work on scaling laws, dataset mixture design, and training dynamics for frontier models
• Build and optimize large-scale distributed training pipelines across multi-node GPU clusters
• Collaborate closely with video, generative, agent, and robot learning teams to integrate pretrained models into the autonomy stack
• Design evaluation frameworks to measure reasoning ability, robustness, and cross-domain generalization
• Contribute to post-training approaches including fine-tuning, alignment, and model adaptation

Requirements

• Experience training large-scale foundation models or working on pretraining for LLMs or multimodal systems
• Strong understanding of modern deep learning architectures, especially transformers
• Experience with large-scale distributed training and optimization
• Proficiency in Python and deep learning frameworks such as PyTorch
• Strong experimental rigor and ability to iterate on model design and training strategies
• Solid software engineering skills and ability to build scalable, reliable systems
• Ability to operate independently and drive ambiguous, high-impact technical problems

Bonus Qualifications