ashby
Software Engineer, Evaluation Platform / Infra
Thinking Machines
San Francisco, USA
Full-time
Entry · 2+ years experience
Onsite
USD 300000-475000 yearly / year
Discovered 1 weeks ago
PythonRustReactTypeScript
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonRustReact
Role Overview
Evaluation is one of the most important pillars of building frontier AI systems. It guides research direction, powers experimentation, and helps us understand whether changes to data and training are improving the capabilities and behaviors we care about.
To support this work, researchers need a powerful, self-serve platform that makes it easy to author evaluations, run them or reproduce them reliably at scale, and extract insight from the results. The platform must support both standardized external benchmarks and fast-moving internal evaluations, many kinds of tasks and graders, and inspection from aggregate metrics down to individual model trajectories.
In this role, you will design and build this platform end to end. You will work across Python frameworks, data pipelines, APIs, and user-facing applications, and collaborate closely with pre-training, post-training, and applied teams to improve how we evaluate models and turn results into research decisions.
Key Skills for This Role
PythonRustReactTypeScript
Full Job Posting
About Thinking Machines
The mission of Thinking Machines is to build AI that extends human will and judgment. We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication. We believe the future worth building is human, and we're hiring people who want to build it.
About the Role
Evaluation is one of the most important pillars of building frontier AI systems. It guides research direction, powers experimentation, and helps us understand whether changes to data and training are improving the capabilities and behaviors we care about.
To support this work, researchers need a powerful, self-serve platform that makes it easy to author evaluations, run them or reproduce them reliably at scale, and extract insight from the results. The platform must support both standardized external benchmarks and fast-moving internal evaluations, many kinds of tasks and graders, and inspection from aggregate metrics down to individual model trajectories.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Thinking Machines
Research Engineer, Infrastructure, Numerics
San Francisco, USA
SeniorFull-time
Discovered 2 days agoView →
Software Engineer, Security
San Francisco, USA
Discovered 2 days agoFull-time
Research Engineer, Infrastructure, Numerics
San Francisco, USA
Discovered 2 days agoFull-time
Engineering Manager
San Francisco, USA
Discovered 2 days agoFull-time
Endpoint Engineer, IT
San Francisco, USA
Discovered 2 days agoFull-time
IT Engineer
New York City, USA
Discovered 2 days agoFull-time
Research, Audio Expertise
San Francisco, USA
Discovered 2 days agoFull-time
Technical Sourcer
San Francisco, USA
Discovered 2 days agoFull-time
Software Engineer, Systems Generalist
San Francisco, USA
Discovered 2 days agoFull-time