AI Tester
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
DeepLight AI is seeking a senior-level AI Tester to drive end-to-end quality assurance, test automation strategy, and advanced validation frameworks for web interfaces, microservice APIs, and AI-driven systems.
Key Skills for This Role
Responsibilities
- Designing, building, and maintaining scalable, robust end to end automation frameworks from scratch, utilizing Playwright as the primary automation engine across web interfaces
- Authoring and executing comprehensive API testing suites to validate distributed microservices, ensuring strict data integrity, state consistency, and schema compliance
- Designing validation strategies for asynchronous, event driven data architectures, tracking messages and auditing system behaviors across Kafka based streaming pipelines
- Establishing specialized testing methodologies to evaluate Generative AI and Large Language Model (LLM) outputs, assessing models for hallucination, bias, semantic accuracy, and safety constraints
- Managing, curating, and versioning baseline prompt validation datasets and ground truth test collections to ensure consistent benchmarking of AI system performance
- Partnering closely with AI research engineers, product owners, and DevOps squads to integrate automated testing gates directly into modern CI/CD deployment pipelines
Requirements
- Advanced technical capability in building automated test suites using Playwright, combined with deep proficiency in testing RESTful and gRPC APIs
- Practical knowledge of utilizing specialized AI quality tools and observability platforms such as Ragas, LangSmith, or TruLens
- Strong technical comprehension of microservices communication patterns, database transactions, and data integrity verification across distributed environments
- Advanced coding proficiency in TypeScript, JavaScript, or Python
- Competence in interacting with event streaming platforms (Kafka or Azure Event Hubs) to produce, consume, and validate asynchronous message payloads
- A minimum of 6 years of experience in dedicated software quality engineering, test automation, or SDET roles
- Documented history of validating complex enterprise workflows that rely heavily on Kafka message queues, event sourcing, or real time data pipelines
- Hands on experience executing tests and navigating application workloads containerized via Docker and orchestrated within Kubernetes clusters
- Practical experience integrating automated test definitions, smoke suites, and regression testing gates directly into enterprise delivery setups (e.g., GitHub Actions, Azure DevOps)
Full Job Posting
Company Overview
- DeepLight AI is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services and banking.
- Our team combines deep expertise in data science, statistical modeling, AI/ML technologies, workflow automation, and systems integration with a practical understanding of complex business operations.
- We deliver tailored AI solutions designed to integrate seamlessly into existing enterprise architectures, ensuring that innovation is both scalable and secure.
Role Overview
- The AI Tester is a specialized, senior level quality engineering position within the Testing work pillar.
- This role is responsible for driving end to end quality assurance, test automation strategy, and advanced validation frameworks across complex web interfaces, microservice APIs, and cutting edge, AI driven systems.
- Operating at the intersection of traditional software testing and advanced machine learning engineering, this position focuses heavily on validating distributed backend architectures, streaming data workflows, and Generative AI/LLM outputs.
Key Responsibilities
- Designing, building, and maintaining scalable, robust end to end automation frameworks from scratch, utilizing Playwright as the primary automation engine across web interfaces.
- Authoring and executing comprehensive API testing suites to validate distributed microservices, ensuring strict data integrity, state consistency, and schema compliance.
- Designing validation strategies for asynchronous, event driven data architectures, tracking messages and auditing system behaviors across Kafka based streaming pipelines.
- Establishing specialized testing methodologies to evaluate Generative AI and Large Language Model (LLM) outputs, assessing models for hallucination, bias, semantic accuracy, and safety constraints.
- Managing, curating, and versioning baseline prompt validation datasets and ground truth test collections to ensure consistent benchmarking of AI system performance.
- Partnering closely with AI research engineers, product owners, and DevOps squads to integrate automated testing gates directly into modern CI/CD deployment pipelines.
Requirements
- Advanced technical capability in building automated test suites using Playwright, combined with deep proficiency in testing RESTful and gRPC APIs.
- Practical knowledge of utilizing specialized AI quality tools and observability platforms such as Ragas, LangSmith, or TruLens to score and evaluate model responses.
- A strong technical comprehension of microservices communication patterns, database transactions, and data integrity verification across distributed environments.
- Advanced coding proficiency in TypeScript, JavaScript, or Python to write clean, modular, and maintainable test scripts.
- Competence in interacting with event streaming platforms (Kafka or Azure Event Hubs) to produce, consume, and validate asynchronous message payloads.
- A minimum of 6 years of experience in dedicated software quality engineering, test automation, or SDET roles, with a proven focus on modern automated architectures.
- A documented history of validating complex enterprise workflows that rely heavily on Kafka message queues, event sourcing, or real time data pipelines.
- Hands on experience executing tests and navigating application workloads containerized via Docker and orchestrated within Kubernetes clusters.
- Practical experience integrating automated test definitions, smoke suites, and regression testing gates directly into enterprise delivery setups (e.g., GitHub Actions, Azure DevOps).
Preferred Skills
- Conceptual or practical familiarity with the unique data privacy, regulatory security compliance requirements, and risk environments of banking applications.
- Experience utilizing performance testing utilities (such as k6, JMeter, or Locust) to evaluate API latency and system threshold capacities under stress.
- A basic understanding of the broader Machine Learning Lifecycle, model registry operations, and automated dataset versioning practices (e.g., DVC).
Benefits
- Competitive salary
- Comprehensive personal health insurance
- Visa Sponsorship for the successful individual
- Professional development and certification support
- Subscription reimbursement relating to your role
- Opportunity to work on cutting edge AI projects
- Monthly Employee Incentive program
- Career advancement opportunities in a rapidly growing AI company
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Deeplight
QA Engineer
Dubai, UAE
DeepLight AI seeks a QA Engineer to ensure data platform correctness through automated testing, data validation, and API testing. You will design test suites, perform source-to-target reconciliation, and collaborate with
Senior AI Engineer
Abu Dhabi, UAE
DeepLight AI seeks a Senior AI Engineer to design, develop, and deploy production-grade AI and Generative AI solutions for clients in financial services. The role involves building agentic systems, RAG pipelines, and opt
Senior Full Stack Engineer
Abu Dhabi, UAE
DeepLight AI seeks a Senior Full Stack AI Engineer to bridge data science and enterprise software architecture. The role involves designing AI systems, deploying models, and managing MLOps pipelines for clients in financ
UI Engineer
, UAE
DeepLight AI is seeking a UI Engineer to build responsive, accessible, and performant web interfaces for banking platforms and customer-facing products. The role requires 6+ years of React JS experience, expertise in fro
DevOps Engineer
, UAE
DeepLight AI is seeking a DevOps Engineer to own the deployment architecture and infrastructure for an Agentic Workflow Automation platform in Abu Dhabi. The role requires expertise in Terraform/Bicep, Kubernetes (AKS),
Senior AI DevOps Engineer
, UAE
DeepLight AI is seeking a Senior AI DevOps Engineer to design and implement secure, scalable cloud architectures on Azure and AWS for core banking and AI platforms. The role requires expertise in hybrid-cloud strategy, F
AI Tester
, UAE
DeepLight AI is seeking a senior-level AI Tester to drive end-to-end quality assurance, test automation strategy, and advanced validation frameworks for web interfaces, microservice APIs, and AI-driven systems. The role
DevOps Engineer
, UAE
DeepLight AI is seeking a DevOps Engineer to own the deployment architecture and infrastructure for an Agentic Workflow Automation platform in Dubai. The role requires expertise in Terraform/Bicep, Kubernetes (AKS), CI/C
QA Engineer
Dubai, UAE
Senior AI Engineer
Abu Dhabi, UAE
Senior Full Stack Engineer
Abu Dhabi, UAE
UI Engineer
, UAE
DevOps Engineer
, UAE
Senior AI DevOps Engineer
, UAE
AI Tester
, UAE
DevOps Engineer
, UAE