Quality Assurance Engineer

The Mission

Senior Qa Automation Engineer

Test Readiness Reviews (Trr)

• AI & LLM Validation

• Non-Deterministic Testing: Architect automated frameworks to evaluate Generative AI outputs for hallucination, consistency, and factual accuracy against "Gold Standard" datasets.
• RAG Evaluation: Implement automated metrics (e.g., RAGAS, faithfulness, answer relevance) to verify that Retrieval-Augmented Generation pipelines accurately cite technical and regulatory documentation.
• Prompt Regression: Design regression suites to monitor "prompt drift," ensuring model updates do not degrade the quality of AI-generated engineering documents.

• Integration & System Verification

• Enterprise Integration: Build robust tests to validate data consistency between AI agents and critical systems (e.g., SAP S/4HANA, Ariba), ensuring the integrity of Bill of Materials (BOM) and financial data.
• Performance Benchmarking: Design tests to validate latency and throughput for forecasting models and risk-scoring engines using tools like Locust, JMeter, or K6.
• API & Security Validation: Automate testing of secure API gateways, verifying Role-Based Access Control (RBAC) and PII redaction logic before data reaches AI models.

• Governance & Traceability

• V-Model Alignment: Map automated test cases to "System Requirements" to create digital evidence for formal Verification and Validation (V&V) reports.
• Stage Gate Compliance: Prepare "Test Readiness" packages for formal reviews, providing quantitative evidence that systems are stable enough to move from MVP to Production.
• Defect Lifecycle Management: Manage the feedback loop between Requirements Quality Assistants and development teams, tracing AI logic defects back to specific model versions.

Technical Requirements

• Core Automation: Expert proficiency in Python (Pytest) and standard libraries (Selenium/Playwright, Requests).
• AI Evaluation: Hands-on experience with LLM evaluation frameworks (e.g., DeepEval, TruLens) and "Ground Truth" dataset management.
• Performance Engineering: Proficiency in crafting Performance Test Plans and implementations (Locust, K6, etc.).
• Data Validation: Expertise in SQL and data quality tools (e.g., Great Expectations) for Data Lakehouses and Vector Databases.
• CI/CD & DevOps: Strong experience integrating quality gates into GitLab CI/CD pipelines.
• Engineering Practices: Deep understanding of modern QE practices, including Shift Left, Test Pyramid, and Mono-repo architectures.

Professional Qualifications

• Experience: 5+ years in QA Automation, with 2+ years focused on complex data-driven applications, ML models, or AI agents.
• Domain Expertise: Background in Defense, Aerospace, or highly regulated industries is a strong plus. Familiarity with IV&V processes is highly desirable.
• Analytical Mindset: Ability to define pass/fail criteria for probabilistic systems and communicate "Confidence Levels" to engineering leadership.
• Why AI71?
• Final Line of Defense: You define whether an AI agent is "trusted" to negotiate a contract or design a critical system component in a safety-critical environment.
• Tax-Free Compensation: A market-leading, 100% tax-free salary and benefits package.
• Full Relocation Support: Comprehensive assistance including flights, housing support, and visa sponsorship for you and your family to Abu Dhabi.
• Pioneering Work: Set the global standard for how defense organizations validate intelligent systems using world-leading models.