Senior Manager, Data Quality & Evaluation
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
A partner company is looking for a Senior Manager, Data Quality & Evaluation to build and lead quality frameworks for AI data programs. The role involves defining evaluation standards, managing quality operations, and collaborating with cross-functional teams.
Key Skills for This Role
Responsibilities
- Design and manage quality frameworks for AI data and evaluation programs, including quality standards, acceptance criteria, review processes, and performance metrics.
- Translate customer requirements into scalable quality workflows that support data collection, annotation, evaluation, review, calibration, and reporting activities.
- Identify quality risks early and collaborate with delivery teams to resolve issues affecting timelines, customer confidence, or program outcomes.
- Build repeatable quality processes for calibration, QA sampling, adjudication, reviewer performance tracking, and customer reporting.
- Lead quality operations across multilingual evaluation, speech and audio review, transcription, annotation, human preference evaluation, expert review, coding evaluation, and AI model assessment programs.
- Develop and improve rubrics, task instructions, reviewer guidelines, calibration exercises, golden datasets, scorecards, and reporting templates.
- Monitor quality indicators such as reviewer agreement, error trends, performance metrics, and root causes of quality variation.
- Turn quality insights into actionable improvements across instructions, training programs, tooling, staffing models, and operational workflows.
- Support strategic customer programs through quality reviews, business discussions, escalations, retrospectives, and improvement plans.
- Partner with internal teams to ensure programs are designed for quality success from initial planning through production delivery.
- Build reusable quality assets and standardized approaches that improve scalability across multiple programs.
- Lead, coach, and develop Quality Managers, Quality Leads, Quality Specialists, reviewers, and other contributors.
Requirements
- 5+ years of experience in quality operations, data operations, AI data services, localization quality, annotation quality, evaluation operations, trust and safety quality, or a related field.
- Proven experience managing quality programs for complex customer accounts or large scale operational delivery environments.
- Strong understanding of QA methodologies, calibration processes, sampling strategies, adjudication, error analysis, and performance reporting.
- Experience collaborating with cross functional teams including operations, delivery, supply chain, sales, and customer facing stakeholders.
- Strong analytical skills with the ability to transform quality data into operational improvements and strategic recommendations.
- Excellent written and verbal communication skills.
- Experience building processes in fast moving, ambiguous environments.
- Strong leadership and people management skills.
Full Job Posting
Position Overview
- This role offers the opportunity to build and lead the quality foundation behind advanced AI data programs supporting the next generation of intelligent systems.
- You will define evaluation frameworks, quality standards, and operational processes that ensure reliable and scalable human data workflows.
- The position combines strategic leadership, analytical problem solving, and hands on execution across AI evaluation, annotation, and data quality initiatives.
- You will collaborate with cross functional teams and customers to transform complex requirements into effective quality solutions.
Accountabilities
- Design and manage quality frameworks for AI data and evaluation programs, including quality standards, acceptance criteria, review processes, and performance metrics.
- Translate customer requirements into scalable quality workflows that support data collection, annotation, evaluation, review, calibration, and reporting activities.
- Identify quality risks early and collaborate with delivery teams to resolve issues affecting timelines, customer confidence, or program outcomes.
- Build repeatable quality processes for calibration, QA sampling, adjudication, reviewer performance tracking, and customer reporting.
- Lead quality operations across multilingual evaluation, speech and audio review, transcription, annotation, human preference evaluation, expert review, coding evaluation, and AI model assessment programs.
- Develop and improve rubrics, task instructions, reviewer guidelines, calibration exercises, golden datasets, scorecards, and reporting templates.
- Monitor quality indicators such as reviewer agreement, error trends, performance metrics, and root causes of quality variation.
- Turn quality insights into actionable improvements across instructions, training programs, tooling, staffing models, and operational workflows.
- Support strategic customer programs through quality reviews, business discussions, escalations, retrospectives, and improvement plans.
- Partner with internal teams to ensure programs are designed for quality success from initial planning through production delivery.
- Build reusable quality assets and standardized approaches that improve scalability across multiple programs.
- Lead, coach, and develop Quality Managers, Quality Leads, Quality Specialists, reviewers, and other contributors involved in data quality operations.
Requirements
- 5+ years of experience in quality operations, data operations, AI data services, localization quality, annotation quality, evaluation operations, trust and safety quality, or a related field.
- Proven experience managing quality programs for complex customer accounts or large scale operational delivery environments.
- Strong understanding of QA methodologies, calibration processes, sampling strategies, adjudication, error analysis, and performance reporting.
- Experience collaborating with cross functional teams including operations, delivery, supply chain, sales, and customer facing stakeholders.
- Strong analytical skills with the ability to transform quality data into operational improvements and strategic recommendations.
- Excellent written and verbal communication skills, with the ability to explain quality insights clearly to customers and senior stakeholders.
- Experience building processes in fast moving, ambiguous environments where systems and workflows are continuously evolving.
- Strong leadership and people management skills, including coaching and developing quality specialists, reviewers, annotators, or operational teams.
- Experience with AI data workflows, RLHF, LLM evaluation, speech/audio evaluation, transcription, coding evaluation, multilingual evaluation, or expert review programs is highly valued.
- Experience designing evaluation rubrics, annotation guidelines, reviewer training materials, calibration workflows, or quality scorecards is a plus.
- Familiarity with human in the loop data processes, annotation platforms, QA tools, dashboards, and distributed contributor networks is desirable.
- Knowledge of multilingual evaluation, cultural considerations, language quality risks, or specialized domain review processes is beneficial.
Benefits
- Fully remote position with flexibility to work from anywhere.
- Opportunity to contribute to the development of AI systems through high impact data quality initiatives.
- Leadership role with significant influence over quality strategy, processes, and operational standards.
- Collaboration with international teams working on advanced AI programs.
- Opportunity to build scalable systems and shape the future of AI evaluation operations.
- Professional growth opportunities within a fast evolving technology environment.
- Dynamic workplace focused on innovation, continuous improvement, and operational excellence.
How Jobgether Works
- We use an AI powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements.
- Our system identifies the top fitting candidates, and this shortlist is then shared directly with the hiring company.
- The final decision and next steps (interviews, assessments) are managed by their internal team.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Jobgether
Technical Recruiter / Talent Acquisition Partner
Abu Dhabi, UAE
Jobgether is looking for an experienced Technical Recruiter / Talent Acquisition Partner based in the UAE to manage end-to-end recruitment across technology, engineering, and business functions. The role offers remote wo
Userplane Protocol Stack Engineer
Abu Dhabi, UAE
Jobgether is hiring a Userplane Protocol Stack Engineer for a partner company in the UAE. The role involves designing, developing, and supporting network communication solutions using TCP/IP, Kubernetes, cloud platforms,
DOTNetCore developers
Abu Dhabi, UAE
Jobgether is hiring a .NET Core Developer for a partner company in the UAE. The role involves designing, building, and integrating secure, scalable platforms using .NET Core, REST, gRPC, and event-driven architectures. R
Freelance Recruiter (Global Recruitment Partner)
Abu Dhabi, UAE
We are seeking an experienced Freelance Recruiter to manage end-to-end sourcing and placement across multiple industries. You will source, screen, and engage candidates using LinkedIn, Boolean search, and professional ne
Talent Sourcer (Freelance)
Abu Dhabi, UAE
We are looking for a freelance Talent Sourcer based in the UAE to identify and engage exceptional candidates using innovative sourcing strategies. You will build talent pipelines, conduct market mapping, and screen candi
International Creative Writer with French
Abu Dhabi, UAE
We are looking for a freelance International Creative Writer with native French skills to craft engaging dialogues for a virtual assistant. You will write, edit, and localize conversational content, ensuring cultural rel
Head of Product (Core/PAM)
Abu Dhabi, UAE
Jobgether seeks a Head of Product to define product vision and strategy for a Player Account Management (PAM) platform in the iGaming industry. This remote leadership role requires 7+ years of product management experien
Software Craftsperson/Node.js/React.js/AI
Abu Dhabi, UAE
A remote opportunity for a software engineer passionate about building high-quality, scalable systems using Node.js, React.js, and AI-powered tools. You will take full-stack ownership, apply software craftsmanship practi
Technical Recruiter / Talent Acquisition Partner
Abu Dhabi, UAE
Userplane Protocol Stack Engineer
Abu Dhabi, UAE
DOTNetCore developers
Abu Dhabi, UAE
Freelance Recruiter (Global Recruitment Partner)
Abu Dhabi, UAE
Talent Sourcer (Freelance)
Abu Dhabi, UAE
International Creative Writer with French
Abu Dhabi, UAE
Head of Product (Core/PAM)
Abu Dhabi, UAE
Software Craftsperson/Node.js/React.js/AI
Abu Dhabi, UAE