company_site
Senior Developer, Artificial Engineering
EisnerAmper
Bengaluru, IND
Full-time
Senior · 3+ years experience
Onsite
Discovered 1 weeks ago
PythonRAGASDeepEvalPromptfooLangSmithPytest
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonRAGASDeepEval
Role Overview
A QA Engineer for AI Initiatives is responsible for ensuring the quality, reliability, fairness, and performance of AI/ML-powered products and systems.
Unlike traditional QA, this role requires deep understanding of non-deterministic model behavior, data quality, and AI-specific failure modes such as hallucinations, bias, and model drift.
Key Skills for This Role
PythonRAGASDeepEvalPromptfooLangSmithPytest
Full Job Posting
Job Description
A QA Engineer for AI Initiatives is responsible for ensuring the quality, reliability, fairness, and performance of AI/ML-powered products and systems. Unlike traditional QA, this role requires deep understanding of non-deterministic model behavior, data quality, and AI-specific failure modes such as hallucinations, bias, and model drift.
Required Skills & Qualifications
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field
- 3–6 years of QA experience, with at least 1–2 years in AI/ML quality assurance
- Strong proficiency in Python for test automation and data analysis
- Familiarity with LLM evaluation frameworks (e.g., RAGAS, DeepEval, Promptfoo, LangSmith)
- Hands-on experience with testing tools: Pytest, Selenium, Postman, or similar
- Understanding of ML lifecycle — training, validation, deployment, and monitoring
- Knowledge of data quality tools and pipeline testing (Great Expectations, dbt tests)
Nice to Have
Experience with prompt engineering and red-teaming LLMs
Familiarity with MLOps platforms (MLflow, SageMaker, Vertex AI)
Knowledge of vector databases and embedding quality evaluation
Understanding of AI safety, responsible AI principles, and fairness frameworks
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at EisnerAmper
2027 Risk & Compliance Services Summer Intern – Business Risk Services
Baton Rouge, USA
EntryInternship, Part Time, Temporary
Discovered TodayView →
Manager - Outsourced Accounting Services (Special Projects)
Coral Gables, USA
MidFull-time
Discovered TodayView →
Senior Manager- Bankruptcy & Restructuring (Sherwood Partners)
New York City, USA
SeniorFull-time
Discovered TodayView →
Assurance Financial Consultant
Hyderabad, IND
Discovered TodayFull-time
Assurance Financial Consultant
Hyderabad, IND
Discovered TodayFull-time
2026 Service Desk Analyst Intern
Minneapolis, USA
Discovered TodayPart Time, Internship
2027 Risk & Compliance Services Summer Intern – Business Risk Services
Baton Rouge, USA
Discovered TodayInternship, Part Time, Temporary
Financial Systems Analyst
Mumbai, IND
Discovered TodayFull-time
Financial Systems Analyst
Mumbai, IND
Discovered TodayFull-time
Manager - Outsourced Accounting Services (Special Projects)
Coral Gables, USA
Discovered TodayFull-time
Senior Manager- Bankruptcy & Restructuring (Sherwood Partners)
New York City, USA
Discovered TodayFull-time