Automation Test Engineer (AI & Chatbot Testing)
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
Role Overview
The Automation Test Engineer tests and improves AI-powered customer support and troubleshooting solutions.
The role focuses on test automation, API testing, AI quality validation, and performance testing.
Key Skills for This Role
Full Job Posting
Job Summary
The Automation Test Engineer tests and improves AI-powered customer support and troubleshooting solutions.
The role focuses on test automation, API testing, AI quality validation, and performance testing.
AI and Chatbot Testing
- Develop automated tests for AI chatbots and retrieval-augmented generation systems.
- Validate response accuracy, relevance, and AI performance.
- Measure hallucination rates and answer relevance.
- Perform regression testing when AI models or knowledge bases are updated.
Automation and Functional Testing
- Automate testing of troubleshooting workflows and decision trees.
- Validate end-to-end customer support scenarios.
- Simulate customer interactions and verify expected outcomes.
API and Integration Testing
- Build and execute automated API tests.
- Validate integrations between AI platforms and backend systems.
- Create mock services and test data where required.
Performance and Security Testing
- Conduct load and performance testing of conversational AI systems.
- Test AI guardrails, prompt injection risks, and unsafe responses.
- Identify performance and reliability issues.
CI/CD and Quality Assurance
- Integrate automated tests into Azure DevOps pipelines.
- Track quality metrics and generate test reports.
- Work closely with developers, AI engineers, and product teams.
Requirements
- At least four years of experience in QA Automation or SDET roles.
- At least one year of experience testing AI, chatbots, LLMs, or Agentic AI solutions.
- Strong programming skills in Python, TypeScript, or C#.
- Experience with REST APIs, GraphQL, WebSockets, Postman, and WireMock.
- Experience with Azure DevOps and CI/CD pipelines.
- Knowledge of RAG, semantic search, and vector databases.
- Experience with MQTT or IoT telemetry is a plus.
- Exposure to embedded systems or consumer electronics is preferred.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at DXC Technology
Sales Professional - Enterprise Transformation Platform
Dubai, UAE
Job Description Key Responsibilities: Sales Strategy & Business Development: Identify and pursue new business opportunities for ETP solutions across assigned territories or accounts, building a robust pipeline of opport
Sales Professional - Enterprise Transformation Platform
, UAE
DXC Technology is seeking a senior sales professional to grow Enterprise Transformation Platform solutions across assigned accounts or territories. The role owns the full enterprise sales cycle, executive relationships,
Cybersecurity Architect Lead
, UAE
The employer is seeking a cybersecurity architecture and engineering lead to govern security design across a large technology transformation program in the UAE. The role requires deep expertise across cloud, infrastructu
Service Delivery Manager
, UAE
DXC Technology is seeking an experienced Service Delivery Manager to govern complex enterprise IT services in the UAE. The role owns service performance, ITSM processes, major incident oversight, governance, stakeholder
Solution Architecture Lead
, UAE
DXC Technology is seeking a Solution Architecture Lead to own end-to-end architecture and technical design authority for a complex UAE technology transformation program. The role leads architects and technical teams, gov
Solutions Architect
, UAE
DXC Technology is seeking a senior Solution Architect to lead end-to-end architecture for a large-scale UAE technology transformation program. The role requires enterprise solution design across applications, integration
Program Manager
, UAE
DXC Technology is seeking a Program Manager to lead delivery of a complex technology transformation program in the UAE. The role coordinates multiple projects, workstreams, vendors, budgets, risks, governance, and senior
Program Director
, UAE
DXC Technology is seeking a Program Director to lead a large-scale, multi-year technology transformation program in the UAE. The role owns strategy, governance, delivery, financial performance, executive stakeholder enga
Sales Professional - Enterprise Transformation Platform
Dubai, UAE
Sales Professional - Enterprise Transformation Platform
, UAE
Cybersecurity Architect Lead
, UAE
Service Delivery Manager
, UAE
Solution Architecture Lead
, UAE
Solutions Architect
, UAE
Program Manager
, UAE
Program Director
, UAE