Sr. Test Engineer - API Testing Job

Pune, IN Pune, MH, IN

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.

At YASH, we’re a cluster of the brightest stars working with cutting-edge technologies. Our purpose is anchored in a single truth – bringing real positive changes in an increasingly virtual world and it drives us beyond generational gaps and disruptions of the future.

Job Title: Senior QA Engineer / Contractor (AI & Data Platforms)

We are looking for a hands-on Senior QA Engineer to join our foundational data and AI platform team. In this role, you will be responsible for defining and executing comprehensive testing strategies for complex, cutting-edge foundational services. You will validate multi-tenant APIs, vector databases, knowledge graphs, LLM integration pipelines, Dataiku data science workflows, and agentic platforms. Your focus will be on building robust automated testing frameworks, ensuring high availability, and verifying data accuracy and security across our entire enterprise AI stack consumed by internal Client teams.

Relationships: Reports to Software Project Manager.

Essential Responsibilities and Duties:

Designs, builds, and executes rigorous automated test strategies for foundational services, including vector databases, knowledge graphs, APIs, and LLM-based applications.

Validates data integrity, retrieval accuracy, semantic relevance, and latency performance of vector search and knowledge graph query layers (e.g., Cypher, SPARQL, semantic matching).

Performs comprehensive functional, integration, regression, and performance/load testing on AI and data platforms, including Dataiku workflows and multi-agent orchestration frameworks.

Tests multi-tenant security boundaries, Role-Based Access Control (RBAC), and authentication mechanisms across cloud-managed and self-managed backends.

Collaborates closely with software engineers and platform developers to identify edge cases, trace root causes of failures, and establish automated CI/CD quality gates.

Maintains test environments, generates synthetic test data for complex graph and vector structures, and establishes automated monitoring for test suites.

Develops and documents QA standards, test scenarios, and validation scripts aligned with enterprise governance and data compliance requirements.

Engages in performance profiling, stress testing under concurrent user loads, and validating system recovery and failover behaviour.

Technology:

Graph Databases: Neo4j (Enterprise, Cypher, Causal Clustering) and exploration of alternative graph options.

Testing Frameworks & Tools: PyTest, Selenium/Playwright, RestAssured, Postman, JMeter / k6 for load testing.

Data & AI Stack: Vector Databases (e.g., Milvus, Qdrant, Pinecone), Graph Databases (e.g., Neo4j), LLM Orchestration (LangChain, LlamaIndex), and Dataiku. · Cloud & Infrastructure: Microsoft Azure, Docker, Kubernetes, CI/CD pipelines (GitHub Actions, Azure DevOps).

Programming & Scripting: Python or Java for test automation script development.

Cloud Infrastructure: Microsoft Azure (Azure Marketplace, AKS, Azure VMs, Managed Disks, Key Vault, VNet peering).

Previous Experience and Competencies:

Bachelor’s or Master’s degree in Computer Science, IT, or a related discipline. · 6 – 10 years of strong QA engineering experience with a heavy focus on backend services, APIs, and data platforms.

Proven experience testing modern AI/ML workloads, vector search systems, or graph-based data architectures.

Preferred Qualifications:

Deep hands-on experience writing robust automation scripts in Python.

Familiarity with testing non-deterministic outputs of Large Language Models (LLMs) and evaluating Retrieval-Augmented Generation (RAG) pipeline quality.

Systematic problem-solving approach, strong analytical mindset, and acute attention to detail.

Experience working within platform engineering or foundational infrastructure teams.

Behaviour:

Fosters and maintains excellent internal client and team relationships.

Possesses a high degree of initiative and hands-on execution drive.

Adaptable and willing to learn new technologies; keeps abreast of key developments in relevant database and cloud ecosystems.

Able to work under pressure in a fast-paced environment. · Excellent oral, written communication, and interpersonal skills.

Practices effective listening techniques and collaborates well across cross-functional groups.

Able to work independently or as part of an empowered technical team.

Effectively analyses and solves complex technical problems with meticulous attention to root cause.

At YASH, you are empowered to create a career that will take you to where you want to go while working in an inclusive team environment. We leverage career-oriented skilling models and optimize our collective intelligence aided with technology for continuous learning, unlearning, and relearning at a rapid pace and scale.