taleo
Software Quality Engineer – Conversational AI / LLM Testing
Cognizant
Noida, IND
Full-time
Mid · 4+ years experience
Onsite
Discovered 2 weeks ago
PythonLarge Language Model (LLM)Retrieval Augmented Generation (RAG)OpenAIAzure OpenAIAnthropic
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonLarge Language Model (LLM)Retrieval Augmented Generation (RAG)
Role Overview
We are seeking a highly skilled Software Quality Engineer with hands-on experience in testing Conversational AI, LLM-based applications, chatbots, virtual assistants, and Agentic AI systems . The ideal candidate will have a strong automation and quality engineering background, deep understanding of how modern conversational systems operate, and solid experience within the Airline or Travel domain , particularly around Booking, Post-Booking, and Check-In workflows .
This role involves validating AI-driven customer conversation journeys, ensuring response quality, evaluating non-deterministic outputs, and establishing robust testing strategies for intelligent systems.
Key Skills for This Role
PythonLarge Language Model (LLM)Retrieval Augmented Generation (RAG)OpenAIAzure OpenAIAnthropic
Full Job Posting
Job Summary
We are seeking a highly skilled Software Quality Engineer with hands-on experience in testing Conversational AI, LLM-based applications, chatbots, virtual assistants, and Agentic AI systems . The ideal candidate will have a strong automation and quality engineering background, deep understanding of how modern conversational systems operate, and solid experience within the Airline or Travel domain , particularly around Booking, Post-Booking, and Check-In workflows .
This role involves validating AI-driven customer conversation journeys, ensuring response quality, evaluating non-deterministic outputs, and establishing robust testing strategies for intelligent systems.
Key Responsibilities
- Own and drive the quality strategy for conversational AI products and customer-facing virtual assistants.
- Design, execute, and automate test scenarios for LLM-powered applications, chatbots, voice assistants, RAG-based systems, and agentic workflows.
- Validate conversational flows across booking, post-booking servicing, and check-in experiences within airline and travel ecosystems.
- Develop and maintain automated test frameworks using Python.
- Create test datasets, golden datasets, and evaluation benchmarks to measure AI system performance.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Cognizant
Salesforce Solution Architect
Toronto, CAN
Discovered TodayFull-time
Senior .NET Core Azure Developer (Hybrid)
Halifax, CAN
Discovered TodayFull-time
Salesforce Business Analyst w/ FSC
Toronto, CAN
Discovered TodayFull-time
Senior Java Developer (Open for remote)
, CAN
Discovered TodayFull-time
Senior Java Developer (Open for remote)
, CAN
Discovered TodayFull-time
Desktop Support Engineer
Wayne, USA
Discovered TodayFull-time
D365 Sr Finance Consultant
, USA
Discovered TodayFull-time
Autonomous Vehicle Fleet Supervisor
, USA
Discovered TodayFull-time