Quality Assurance Automation Engineer

Role Overview

Location and Work Arrangement

• The role is based in Abu Dhabi and is onsite.
• Immediate joiners and candidates serving their notice period are preferred.
• Candidates outside the UAE may be considered if they are willing to relocate.

Key Skills

• Python, Generative AI, LLMs, AI testing, LLM evaluation, AI agent testing, prompt testing, hallucination testing, AI agents, agentic AI, QA automation, and responsible AI.
• Relevant domain experience includes airline, aviation, travel, or transportation.

Key Responsibilities

• Develop and support AI applications using Python, Generative AI, and LLMs.
• Design, execute, and automate comprehensive AI test strategies.
• Validate models, LLMs, AI agents, and agentic workflows for accuracy, reliability, consistency, and safety.
• Perform functional, integration, regression, performance, security, and user acceptance testing.
• Conduct LLM evaluation, prompt testing, hallucination testing, bias and fairness assessments, and guardrail validation.
• Build automated frameworks for AI agents, chatbots, copilots, and autonomous workflows.
• Define quality metrics, benchmarks, and acceptance criteria.
• Analyze and resolve defects in AI outputs, workflows, and model behavior.
• Collaborate with business and development teams on production-ready airline AI solutions.

Required Skills and Experience

• 8–9+ years of relevant experience.
• Strong Python experience.
• Hands-on experience with LLMs, Generative AI, RAG architectures, vector databases, and AI models.
• Experience with LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, Semantic Kernel, or an equivalent AI framework.
• Experience developing and testing AI agents, agentic AI solutions, chatbots, and conversational AI platforms.
• Experience creating AI quality frameworks, test cases, evaluation datasets, and benchmarking methodologies.
• Familiarity with Azure OpenAI, AWS AI, Google AI services, or equivalent platforms.
• Airline, aviation, travel, or transportation domain experience is required.

Preferred Qualifications

• Knowledge of airline operations, reservations, customer service, crew management, airport operations, or passenger experience solutions.
• A bachelor's or master's degree in computer science, artificial intelligence, data science, engineering, or a related field.
• Certifications in AI, machine learning, software testing, or cloud technologies.