ashby
Evaluations Engineer
Vals AI
San Francisco, USA
Full-time
Mid
Onsite
USD 130000-165000 yearly / year
Discovered 1 weeks ago
DocentPythonDjangoReactAWSCDK
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
DocentPythonDjango
Role Overview
We are looking for strong engineers to join our team and own the leaderboards that appear on Vals AI.
You will be responsible for testing and benchmarking new models as they are released on tasks in law, tax, coding, finance, and more. You will analyze error modes of models, evaluate their strengths and weaknesses, and work with our communications team to release results.
Our results are used by startups, enterprises, and research labs alike. We work with all the major foundation model labs, some of the largest financial institutions, and hospital systems in the world. Our work has been featured by the Wall Street Journal, Washington Post, and Bloomberg.
We are building the standard for evaluating the ability of LLMs to perform real-world tasks. You will contribute directly to the leaderboards that make this possible.
Key Skills for This Role
DocentPythonDjangoReactAWSCDK
Full Job Posting
About the Role
We are looking for strong engineers to join our team and own the leaderboards that appear on Vals AI.
You will be responsible for testing and benchmarking new models as they are released on tasks in law, tax, coding, finance, and more. You will analyze error modes of models, evaluate their strengths and weaknesses, and work with our communications team to release results.
Our results are used by startups, enterprises, and research labs alike. We work with all the major foundation model labs, some of the largest financial institutions, and hospital systems in the world. Our work has been featured by the Wall Street Journal, Washington Post, and Bloomberg.
We are building the standard for evaluating the ability of LLMs to perform real-world tasks. You will contribute directly to the leaderboards that make this possible.
What You’ll Do
Evaluate new LLM model releases across the Vals AI suite of benchmarks
Work directly with both open-source and closed-source foundation model labs in evaluating model performance
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Vals AI
Founding Head of Partnerships (Vals Smith)
San Francisco, USA
MidFull-time
Discovered 1 weeks agoView →
Marketing Designer
San Francisco, USA
Discovered TodayFull-time
Member of Technical Staff - Research
San Francisco, USA
Discovered 1 weeks agoFull-time
Member of Technical Staff - Platform
San Francisco, USA
Discovered 1 weeks agoFull-time
Head of Research
San Francisco, USA
Discovered 1 weeks agoFull-time
Founding Head of Partnerships (Vals Smith)
San Francisco, USA
Discovered 1 weeks agoFull-time
Founding Product Manager (Vals Smith)
San Francisco, USA
Discovered 1 weeks agoFull-time