AI Test Engineer

STATERA

Dubai, UAE

Full-time

Entry

Onsite

Discovered Yesterday

LLM, RAG, and conversational-agent testingPython, TypeScript, or C#pytest and asyncioAPI testing and mockingREST and GraphQLWebSockets

Free

Job Fit Check

Base Career helps you apply smarter for this job.

Ready to Scan

Key skills for this role

LLM, RAG, and conversational-agent testingPython, TypeScript, or C#pytest and asyncio

STATERA is seeking an AI Test Engineer/SDET to automate testing of an AI-powered customer-support and product-troubleshooting solution.

The role combines software test automation with hands-on evaluation of LLMs, RAG pipelines, and conversational agents.

The position is onsite in Dubai, UAE.

Evaluate RAG retrieval accuracy, context relevance, semantic-search precision, and response quality.
Measure faithfulness, answer relevance, hallucination rates, and guardrail compliance.
Test multi-turn troubleshooting flows for connectivity, battery, and performance issues.
Simulate customer interactions to verify diagnostic navigation and resolution paths.
Prevent regressions caused by changes to documentation, firmware specifications, or telemetry models.

Automate API and end-to-end tests across AI agents, support workflows, and warranty integrations.
Run load and scalability tests on conversational APIs.
Develop adversarial tests for prompt injection, out-of-scope requests, and hallucination risks.
Integrate AI evaluation and regression suites into Azure DevOps CI/CD pipelines.

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Trusted by over 500,000 job seekers on Base Career