AI QA Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
We are looking for a Senior QA Automation Engineer to validate LLMs, RAG pipelines, AI agents, and predictive ML systems. You will build automated frameworks using Python, PyTest, Playwright, and integrate testing into CI/CD.
Key Skills for This Role
Responsibilities
- Build automation frameworks for AI, ML and LLM applications
- Validate RAG pipelines and AI generated outputs
- Develop UI, API and backend automation using Python
- Integrate automated testing into CI/CD pipelines
- Design regression testing for AI models and prompt changes
- Test enterprise integrations with large scale platforms
- Work closely with AI Engineers, Data Scientists and Product teams to improve product quality throughout the SDLC
Requirements
- 5+ years in QA Automation
- Strong Python programming skills
- Experience with PyTest, Playwright and/or Selenium
- API testing using Requests or similar frameworks
- Experience testing AI, Machine Learning or data driven applications
- Knowledge of LLM evaluation, RAG or Generative AI is highly desirable
- Experience with SQL, CI/CD and GitLab pipelines
- Comfortable working in fast paced product engineering environments
Full Job Posting
Role Overview
- We're partnering with an innovative AI company and are looking for a Senior QA Automation Engineer to help shape the quality of production AI systems.
- This is not traditional software testing. You'll be validating LLMs, RAG pipelines, AI agents and predictive ML systems, building automated frameworks that ensure AI products are reliable, scalable and production ready.
What you'll be working on
- Build automation frameworks for AI, ML and LLM applications
- Validate RAG pipelines and AI generated outputs
- Develop UI, API and backend automation using Python
- Integrate automated testing into CI/CD pipelines
- Design regression testing for AI models and prompt changes
- Test enterprise integrations with large scale platforms
- Work closely with AI Engineers, Data Scientists and Product teams to improve product quality throughout the SDLC
What we're looking for
- 5+ years in QA Automation
- Strong Python programming skills
- Experience with PyTest, Playwright and/or Selenium
- API testing using Requests or similar frameworks
- Experience testing AI, Machine Learning or data driven applications
- Knowledge of LLM evaluation, RAG or Generative AI is highly desirable
- Experience with SQL, CI/CD and GitLab pipelines
- Comfortable working in fast paced product engineering environments
Why apply?
- Fully remote – work from anywhere in the world
- Build cutting edge AI products using the latest LLM technologies
- Work alongside world class AI engineers and researchers
- Long term contract with exciting greenfield projects
- High impact role working on enterprise scale AI platforms
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Quantum Talent Group
Digital Marketing & Campaign Lead
Jeddah, KSA
Quantum Talent Group is seeking an experienced Digital Marketing & Campaign Lead with 7-10 years of experience to drive customer acquisition and product growth through data-driven digital marketing campaigns in the banki
ServiceNow Integration Engineer
Abu Dhabi Emirate, UAE
Quantum Talent Group is seeking an experienced ServiceNow Integration Engineer for a 12-month contract in Abu Dhabi. The role involves designing and delivering enterprise integrations between ServiceNow Employee Center,
Partnership Manager (Merchant Partnerships & Loyalty)
Doha, QAT
A leading payment solutions company in Doha, Qatar seeks a Partnership Manager for an 18-month contract to drive merchant acquisition, offer design, and loyalty program management. The role requires 5-8 years of experien
Head of Operations & Service Delivery
Abu Dhabi Emirate, UAE
We are partnering with a leading organization to hire a Head of Operations & Service Delivery to lead enterprise IT operations, infrastructure, cybersecurity, and IT service delivery in Abu Dhabi. The role requires 10+ y
Senior SAP Systems Engineer
Abu Dhabi Emirate, UAE
Quantum Talent Group is seeking a Senior SAP Systems Engineer for a leading AI organization in Abu Dhabi. The role involves architecting how AI agents interact with SAP, transforming enterprise systems into AI-ready plat
UI/UX Designer (Enterprise SaaS Products)
Dubai, UAE
Quantum Talent Group is seeking a UI/UX Designer to shape the user experience of a sophisticated enterprise SaaS platform in Dubai. The role involves designing intuitive interfaces, creating wireframes and prototypes, an
BIM Software Engineer – Autodesk Platform Services (APS)
Abu Dhabi Emirate, UAE
Quantum Talent Group is seeking a BIM Software Engineer to develop cloud-based BIM data processing solutions using Autodesk Platform Services (APS). The role involves building workflows with Data Management API and Model
Scrum Master
Abu Dhabi Emirate, UAE
Quantum Talent Group seeks an experienced Scrum Master for a 12+ month contract in Abu Dhabi to support a major transformation program. The role involves facilitating Agile ceremonies, driving technical delivery pods, an
Digital Marketing & Campaign Lead
Jeddah, KSA
ServiceNow Integration Engineer
Abu Dhabi Emirate, UAE
Partnership Manager (Merchant Partnerships & Loyalty)
Doha, QAT
Head of Operations & Service Delivery
Abu Dhabi Emirate, UAE
Senior SAP Systems Engineer
Abu Dhabi Emirate, UAE
UI/UX Designer (Enterprise SaaS Products)
Dubai, UAE
BIM Software Engineer – Autodesk Platform Services (APS)
Abu Dhabi Emirate, UAE
Scrum Master
Abu Dhabi Emirate, UAE