Senior Test Analyst, AI

• Design, develop, execute, and maintain test cases and test scripts for AI, ML, and Generative AI applications.
• Validate data quality, data integrity, and data lineage across AI systems and supporting platforms.
• Lead functional, integration, API, regression, user acceptance, and end-to-end testing of AI-enabled solutions.
• Assess AI model outputs for accuracy, consistency, bias, hallucinations, explainability, fairness, and alignment with business requirements and responsible AI standards.
• Execute prompt testing, response validation, and scenario-based testing for Generative AI solutions.
• Execute model validation and inference testing activities, ensuring AI solutions perform consistently across diverse datasets, use cases, and operating conditions.
• Monitor and analyse test execution results, quality metrics, and model performance indicators, producing actionable insights and comprehensive test reports for stakeholders.
• Identify, document, track, and drive resolution of defects, model performance issues, data quality concerns, and AI-specific risks throughout the testing lifecycle.
• Collaborate with business analysts, developers, and other stakeholders to understand requirements and deliver high-quality AI solutions.
• Support test data preparation, environment validation, and model release verification activities.
• Contribute to continuous improvement initiatives by identifying opportunities to enhance AI testing frameworks, processes, and automation capabilities.
• Provide guidance and knowledge sharing to project teams on AI testing methodologies, Responsible AI validation, test automation approaches, and emerging industry practices.
• Mentor junior testers and contribute to building AI testing capabilities across teams through coaching, standards adoption, and continuous learning initiatives.

• Identify quality, model, data, security, privacy, ethical, and compliance risks associated with AI solutions and proactively support mitigation activities.
• Validate adherence to Responsible AI principles, regulatory requirements, organizational policies, and governance standards.
• Conduct and support the assessment and reporting of AI-related risks, including model bias, explainability, fairness, and data quality concerns.
• Maintain appropriate test evidence, validation reports, defect records, and audit documentation to support governance and compliance requirements.
• Escalate risks, issues, and control gaps that may impact AI solution quality, regulatory compliance, or business outcomes.

The above list of key accountabilities is not an exhaustive list and may change from time-to-time based on business needs.

• 5+ years of experience in Quality Engineering, Test Automation, and AI-driven testing, with preference for candidates having 4+ years of experience in Retirement Solutions, Financial Services, or BFSI domains.
• Strong experience in designing, developing, and executing AI-powered test automation frameworks and quality assurance solutions using tools such as Selenium, Appium, UFT/OpenText (at least one mandatory), Python, Maven, Postman, REST APIs, SoapUI, Cortex, and Snowflake.
• Hands-on experience leveraging AI-assisted testing tools and technologies for test case generation, test optimization, defect prediction, intelligent test execution, and quality analytics.
• Strong understanding of software testing principles, methodologies, and best practices, including test strategy, test planning, test design, test execution, defect management, reporting, and test governance.
• Experience validating AI/ML-enabled applications, including model testing, data quality validation, prompt testing and optimization, response evaluation, bias and fairness assessment, hallucination detection, adversarial testing, explainability validation, and performance assessment of AI-powered systems.
• Solid understanding of Software Development Life Cycle (SDLC) and Software Quality Assurance (SQA) processes across Agile, Waterfall, and DevOps delivery models.
• Experience working with CI/CD pipelines and DevOps practices using tools such as Jenkins, Azure DevOps, Azure Repos, Bitbucket, Git, Docker, and Kubernetes.
• Strong analytical and problem-solving skills with the ability to define quality standards, establish AI testing governance, track AI model evaluation metrics and quality KPIs, automate validation processes, and ensure reliable, scalable, and secure AI-driven applications.
• Experience with cloud platforms, data validation, API testing, integration testing, and complex enterprise environments is highly desirable.
• Lead AI testing strategy and provide technical leadership for AI testing initiatives.
• Collaborate with Developers, Testers, and Business Analysts to design test cases and develop them using AI and automation testing tools.
• Perform functional, integration, system, regression, and API testing using tools such as Postman and REST Assured.
• Manage defects through JIRA and Azure DevOps and produce comprehensive test evidence documentation.
• Develop and execute end-to-end test automation solutions covering the entire data lifecycle, from raw data through to the reporting layer.
• Mentor team members and support the onboarding, training, and upskilling of testers in AI and automation testing tools, frameworks, and best practices.

• Strong analytical, problem-solving, and troubleshooting skills.
• Strong communication, collaboration.
• Strong attention to detail and quality orientation.
• Demonstrate ability to solve technical problems and implement innovative solutions.
• Ability to work independently and as part of a team.
• Ability to learn new AT testing tools, test automation tools, technologies, and trends quickly and effectively.
• Ability to handle multiple tasks and projects simultaneously and prioritize them