bayt
Software Engineering Evaluation Specialist
Unknown
Baladiyat ad Dawhah, QAT
Mid-Senior
Onsite
$35 per hour
Discovered 2 weeks ago
Backend software developmentPythonpytestDockerLinux and BashAI coding agents
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
Backend software developmentPythonpytest
Role Overview
Mindrift connects specialists with project-based AI opportunities for leading technology companies.
The work focuses on testing, evaluating, and improving AI systems.
Participation is project based and is not permanent employment.
The role is Software Engineering Evaluation Specialist for the Terminal Bench program.
Key Skills for This Role
Backend software developmentPythonpytestDockerLinux and BashAI coding agents
Full Job Posting
About the Opportunity
Mindrift connects specialists with project-based AI opportunities for leading technology companies.
The work focuses on testing, evaluating, and improving AI systems.
Participation is project based and is not permanent employment.
The role is Software Engineering Evaluation Specialist for the Terminal Bench program.
Responsibilities
- Design coding tasks that challenge frontier AI coding agents.
- Create realistic scenarios involving bugs, broken ETL processes, or missing features.
- Build reproducible Docker environments with pinned dependencies.
- Write deterministic pytest verification that checks outcomes without leaking the fix.
- Write instruction.md files that resemble Jira tickets.
- Write reference solve.sh scripts proving tasks are solvable.
- Calibrate task difficulty so agents solve tasks 20–60% of the time.
- Iterate based on expert QA feedback.
- Later, review other authors' tasks as a QA reviewer.
Required Qualifications
- At least 3 years of production software development in one backend stack: Python, Go, Node.js, Java, or Rust.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Unknown
Executive Secretary
, KSA
Senior
In the dynamic field of construction, the Executive Secretary plays a pivotal role in providing essential administrative and organizational support. This position acts as the primary link between the company's headquarte
Discovered TodayView →
Administrative Assistant (Enthusiasts)
, KSA
Mid-SeniorPart-time
This role as an Administrative Assistant is pivotal for supporting daily operations in a dynamic environment. The ideal candidate will be responsible for organizing documents, updating databases, and assisting in report
Discovered TodayView →
Receptionist
, KSA
Entry
A high-impact opportunity exists for a skilled Receptionist in the vibrant market research sector. This role demands a commitment to exceptional customer service and the ability to create a welcoming atmosphere for clien
Discovered TodayView →
Receptionist (Saudi National)
, KSA
A hotel is seeking a Receptionist / Front Desk Agent to welcome guests, manage check-ins and check-outs, handle inquiries and payments, and support smooth front office operations. The role requires Arabic and English com
Discovered TodayView →
Receptionist - The Jeddah EDITION
, KSA
As a Receptionist at The Jeddah EDITION, you will be the first point of contact for guests, addressing their requests and concerns through various communication channels. Your role is vital in ensuring guest satisfaction
Discovered TodayView →
Executive Secretary
, KSA
Discovered Today
Administrative Assistant (Enthusiasts)
, KSA
Discovered TodayPart-time
Receptionist
, KSA
Discovered Today
Senior HR Coordinator
, UAE
Discovered TodayFull-time
Document Controller
Riyadh, KSA
Discovered TodayFull-time
Receptionist and Showroom Sales Coordinator
, UAE
Discovered TodayFull-time
Receptionist (Saudi National)
, KSA
Discovered Today
Receptionist - The Jeddah EDITION
, KSA
Discovered Today