Engineer - ML QA

About Inception42

Overview

What You’ll Own

• Collaborate with ML QA engineers and cross-functional teams to support high-quality product delivery.
• Design, implement, and oversee the ML QA strategy for machine-learning products and services.
• Design and execute manual and automated test plans and test cases.
• Validate AI chat and RAG applications for response accuracy, relevance, groundedness, hallucination, safety, and retrieval quality.
• Implement LLM-as-a-Judge approaches and automated evaluation pipelines using DeepEval.
• Monitor QA metrics, report findings, lead root-cause analysis, and drive adoption of QA best practices.
• Ensure compliance with industry and company quality standards.

What We’re Looking For

• Bachelor’s degree in Computer Science, Engineering, or a related technical field.
• At least five years of QA experience, including at least two years in a leadership position.
• In-depth QA methodology, tool, and process experience, with strong data quality assessment skills.
• Understanding of the machine learning product lifecycle and experience writing comprehensive test plans and test cases.
• Familiarity with SDLC and agile methodologies and experience with cloud-based services and architectures.
• Strong problem-solving, attention to detail, and communication skills.
• Health, finance, or climate industry experience is a plus.

Tech Stack

• Programming languages: Python, Java, and SQL.
• Testing tools: Playwright, Selenium, JUnit, PyTest, Gatling, Postman, and RestAssured.
• LLM evaluation: DeepEval, LLM-as-a-Judge, and AI chat and RAG validation.
• CI/CD: Jenkins and GitLab CI/CD; version control: Git.
• Cloud and infrastructure: Azure, optional AWS or GCP, Docker, and Kubernetes.
• Databases: MySQL, MongoDB, and PostgreSQL; monitoring: Grafana and Kibana.
• Collaboration tools: Jira and Confluence.