Test Analyst, AI

• Design, develop, execute, and maintain test cases and test scripts for AI, ML, and Generative AI applications.
• Validate data quality, data integrity, and data lineage across AI systems and supporting platforms.
• Perform functional, integration, API, regression, user acceptance, and end-to-end testing of AI-enabled solutions.
• Evaluate AI model outputs for accuracy, consistency, bias, hallucinations, and compliance with business requirements and existing processes.
• Execute prompt testing, response validation, and scenario-based testing for Generative AI solutions.
• Monitor test execution results, analyse outcomes, and produce accurate and comprehensive test reports.
• Identify, document, track, and support resolution of defects, model performance issues, and data quality concerns.
• Collaborate with business analysts, developers, and other stakeholders to understand requirements and deliver high-quality AI solutions.
• Support test data preparation, environment validation, and model release verification activities.
• Contribute to continuous improvement initiatives by identifying opportunities to enhance AI testing frameworks, processes, and automation capabilities.
• Provide knowledge sharing and support to project teams regarding AI testing approaches, tools, and best practices.

• Identify quality, model, data, security, privacy, ethical, and compliance risks associated with AI solutions and proactively support mitigation activities.
• Validate adherence to Responsible AI principles, regulatory requirements, organizational policies, and governance standards.
• Support the assessment and reporting of AI-related risks, including model bias, explainability, fairness, and data quality concerns.
• Maintain appropriate test evidence, validation reports, defect records, and audit documentation to support governance and compliance requirements.
• Escalate risks, issues, and control gaps that may impact AI solution quality, regulatory compliance, or business outcomes.

The above list of key accountabilities is not an exhaustive list and may change from time-to-time based on business needs.

• 3+ years of experience in Quality Engineering, Test Automation, and AI-driven testing, with preference for candidates having 2+ years of experience in Retirement Solutions, Financial Services, or BFSI domains.
• Strong experience in designing, developing, and executing test automation solutions using tools such as Selenium, Appium, UFT/OpenText (at least one mandatory), Python, Maven, Postman, REST APIs, SoapUI, Cortex, and Snowflake.
• Hands-on experience with AI-assisted testing tools and technologies for test case generation, test optimization, defect prediction, intelligent test execution, quality analytics, and test automation.
• Experience in validating AI/ML-enabled applications, including model testing, data quality validation, prompt testing, response evaluation, hallucination detection, bias and fairness testing, explainability validation, adversarial testing, and assessment of AI model accuracy, reliability, and performance.
• Strong understanding of software testing principles, methodologies, and best practices, including test planning, test design, test execution, defect management, reporting, governance, and quality assurance processes.
• Solid understanding of SDLC and SQA processes across Agile, Waterfall, and DevOps delivery models.
• Experience working with CI/CD pipelines and DevOps practices using Jenkins, Azure DevOps, Azure Repos, Bitbucket, Git, Docker, and Kubernetes.
• Strong analytical and problem-solving skills with the ability to define quality standards, automate validation processes, and ensure reliable, scalable, and secure AI-driven applications.
• Experience with cloud platforms, data validation, API testing, integration testing, and end-to-end testing across complex enterprise environments.
• Design, develop, and execute comprehensive test strategies, test plans, and test cases for functional, integration, system, regression, API, and end-to-end testing across applications, data, and reporting layers.
• Collaborate closely with Business Analysts, Developers, Product Owners, and other stakeholders to define test scenarios, automate test cases, and ensure quality throughout the software development lifecycle.
• Develop and maintain automated testing solutions leveraging both traditional and AI-powered testing tools to improve testing efficiency, coverage, and quality.
• Validate AI-powered applications, intelligent agents, and machine learning models by assessing model accuracy, reliability, performance, fairness, explainability, and compliance with Responsible AI principles.
• Perform prompt testing, response validation, hallucination detection, adversarial testing, and evaluation of AI outputs against business requirements and expected outcomes.
• Execute API, data, integration, and end-to-end testing to ensure data integrity, system interoperability, and reliable business process outcomes.
• Manage defects and quality issues using tools such as JIRA and Azure DevOps, including defect tracking, root-cause analysis, reporting, and resolution support.
• Produce test evidence, quality metrics, and test execution reports to support audit, compliance, and governance requirements.
• Contribute to the continuous improvement of testing frameworks, methodologies, and automation practices, identifying cost-effective solutions and opportunities for innovation.
• Provide guidance, support, and training to team members on AI-enabled testing tools, automation technologies, and quality engineering best practices.

• Strong analytical and problem-solving skills
• Attention to detail and quality-driven mindset
• Good communication and teamwork skills
• Proactive and eager to learn