Research Engineer, Model Evaluation and Improvement
Job Fit Check
Base Career helps you apply smarter for this job.
Full Job Posting
Role Overview
We’re a team focused on making frontier AI models better at science. LLMs know an extraordinary amount of biology, but there’s still a large gap in reasoning for the real-world problems scientists face every day. We recently published some of our work here.
You’ll build the datasets, evaluations, and systems that help close that gap. You’ll work with scientists to turn complex scientific work into rigorous tasks that models can learn from and be evaluated against. You’ll partner with leading AI labs to understand where models fail and how to improve them.
This is an early and rapidly evolving area. You’ll work at the intersection of software engineering, biology, and frontier AI: finding tasks that are challenging for LLMs and valuable to scientists, designing evaluations that capture real scientific judgment, and building systems to create these tasks at scale.
RESPONSIBILITIES
- Build datasets for evaluating and improving frontier models, turning complex scientific data into high-quality tasks and environments for LLMs.
- Analyze model failure modes, running experiments across frontier models to understand where they struggle and identify opportunities for improvement.
- Build scalable data infrastructure, creating pipelines that curate, transform, and validate large volumes of scientific data into tasks for model evaluation and improvement.
- Collaborate with frontier AI labs, helping develop and evaluate new approaches for improving models on challenging scientific tasks.
- Work closely with scientists, translating expert judgment into problems and evaluation criteria that can reliably distinguish strong model behavior.
QUALIFICATIONS
- 2+ years at the intersection of biology and AI, with experience evaluating and improving scientific models or LLMs for biological applications.
- Experience building with LLMs, with an intuition for where current models excel, where they struggle, and how to design systems around their capabilities.
- Curiosity and excitement about frontier AI, with a desire to understand and push the capabilities of rapidly improving models.
- Comfort working on ambiguous problems, where the playing field is rapidly shifting and the right technical approaches are still being discovered.
- Collaborative mindset, able to work closely with engineers, scientists, and external research partners.
- Desire to work in a fast-paced environment, where priorities can shift and rapid experimentation is encouraged.
HOW WE WORK
This is an in-person team in San Francisco built around collaborating in the office in a fast-paced environment. We’re in the office Monday through Friday. #LI-KW1
Benchling welcomes everyone.
We believe diversity enriches our team so we hire people with a wide range of identities, backgrounds, and experiences. We are an equal opportunity employer. That means we don’t discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We also consider for employment qualified applicants with arrest and conviction records, consistent with applicable federal, state and local law, including but not limited to the San Francisco Fair Chance Ordinance.
About Benchling
Benchling offers a cloud-based R&D platform purpose-built for the life sciences, combining electronic notebooks, molecular biology tools, and data management in a single system.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Benchling
People Systems Analyst
San Francisco, USA
Executive Assistant to Sales Leadership (Contract)
San Francisco, USA
Pricing & Packaging Ops Manager
San Francisco, USA
Product Support Analyst
San Francisco, USA
Software Engineer, Tools and Automation
San Francisco, USA
Software Engineer, Agents
, CAN
Leader, Professional Services
Boston, USA
Software Engineer, Applications (App Foundations) (High Seniority)
, CAN
