greenhouse
Helix AI Engineer, Training Performance
Figure
Napi Headquarters, USA
Mid · 3+ years experience
USD 200000-400000 / year
Discovered 1 weeks ago
cplusplusjaxpythonpytorch
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
cplusplusjaxpython
Key Skills for This Role
cplusplusjaxpythonpytorch
Full Job Posting
Responsibilities
- Optimize training performance for a 100B+ parameter models across 100k+ GPUs.
- Collaborate with the broader team on accelerator choice, cluster topology, scheduling, and hardware procurement decisions to inform future scaling.
- Write and optimize custom kernels (Triton/CUDA)
- Build tooling and dashboards for continuous performance monitoring, regression detection, and root-cause analysis across training jobs
- Optimize data loading and preprocessing pipelines so I/O never gates the accelerators
- Improve checkpointing, fault tolerance, and elastic restart so large jobs recover quickly from node failures without losing significant wall-clock time
- Partner with researchers to co-design model architectures and training recipes that are performant at scale (e.g., activation checkpointing strategies, mixed precision, sequence packing)
- Extend and contribute to kernel compilers (e.g., Triton, Gluon) to improve iteration speed and enable targeting of custom/non-NVIDIA accelerators
- Build and extend agentic systems that automatically generate, benchmark, and iterate on custom kernels
- Evaluate emerging accelerator architectures (AMD, TPU, SRAM-based ASICs, and other novel hardware) for fit with our training workloads, and lead proof-of-concept ports/benchmarks
- Explore different model/data parallelisms (FSDP, context parallel, expert parallel, etc.) to determine optimal configuration per model size.
Requirements
- Bachelor's or Master's degree in Computer Science, Computer/Electrical Engineering, or a related field
- 3+ years in AI performance engineering, with significant time leading large-scale performance improvement projects
- Deep understanding of GPU architecture and performance characteristics (memory bandwidth, compute-bound vs. memory-bound ops, occupancy)
- Proficiency with profiling tools (Nsight Systems/Compute, PyTorch Profiler, HTA, or similar) and ability to translate traces into concrete optimizations
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Figure
Software Engineer, Service & Operations
San Jose, USA
Discovered Today
Production Lead (Swing Shift)
San Jose, USA
Discovered Yesterday
Special Projects Intern [Fall 2026]
San Jose, USA
Discovered Yesterday
Senior Marketing Manager, ABM
, USA
Discovered 1 weeks agoFull-time
Deployment Logistics Coordinator
San Jose, USA
Discovered 1 weeks agoFull-time
Logistics & Dispatch Coordinator
San Jose, USA
Discovered 1 weeks agoFull-time
Helix AI Engineer, Data Infrastructure
Napi Headquarters, USA
Discovered 1 weeks ago
Project Lead, Data Creators (CDMX)
Mexico, USA
Discovered 1 weeks ago
