teamtailor
Senior Software QA Engineer
Crimson Education
IND
Full-time
Senior · 4+ years experience
Hybrid
Discovered 3 weeks ago
PythonJavaScriptTypeScriptPlaywrightCypressSelenium
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PythonJavaScriptTypeScript
Key Skills for This Role
PythonJavaScriptTypeScriptPlaywrightCypressSelenium
Full Job Posting
What are the main responsibilities for this role?
- Design & Build: Develop, maintain, and scale robust, end-to-end automation test frameworks from scratch (UI, API, and Integration layers).
- CI/CD Integration: Seamlessly bake automated test suites into our CI/CD pipelines to ensure rapid, high-confidence deployments.
- Coverage Strategy: Define, track, and scale test coverage metrics (targeting high-purity API and critical UI path coverage) to ensure regression-free deployments.
- Performance & Scale: Conduct API performance, load, and stress testing to ensure our infrastructure keeps up with demand.
- AI/LLM Validation: Design testing strategies for non-deterministic systems. You’ll evaluate LLM outputs for accuracy, bias, safety, and hallucination rates.
- Prompt Regression Testing: Establish benchmarks to ensure that tweaking an AI prompt or upgrading a foundational model doesn't secretly break existing features.
- Data & Model Drift Monitoring: Collaborate with Data Scientists and ML Engineers to monitor model performance metrics in production.
- Edge Case Hunting: Actively simulate adversarial inputs (prompt injections, jailbreaks) to ensure our AI features remain secure and guardrailed.
What skills and experience are required?
- 4+ years of experience in Software QA Automation, with at least 1–2 years specifically focused on testing AI/ML-powered applications or LLM integrations.
- Strong programming proficiency in Python (highly preferred for AI workflows) or JavaScript/TypeScript.
- Hands-on experience with modern automation tools (e.g., Playwright, Cypress, Selenium ). Expertise in API testing tools (e.g., Postman, RestAssured, PyTest ).
- Expertise in API testing tools (e.g., Postman, RestAssured, PyTest ).
- Familiarity with AI evaluation frameworks (e.g., Ragas, TruLens, Deepchecks ) or a proven ability to write custom evaluation scripts for probabilistic outputs.
- Basic understanding of vector databases, embeddings, and how data pipelines feed into ML models.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Crimson Education
Operations Support Officer
, IND
Discovered 2 days agoFull-time
Overseas Education Consultant (Sales)
, IND
Discovered 2 days agoFull-time
Sales Development Representative
, IND
Discovered 1 weeks agoFull-time
CRM Operations & Support Specialist
, USA
Discovered 1 weeks agoFull-time
Academic Tutor (MedView)
, AUS
Discovered 1 weeks agoPart Time, Contract
Student Success & Sales Representative, Jr. (Mexico)
, AUS
Discovered 1 weeks agoFull-time
Associate - Academic Advisor
New Delhi, IND
Discovered 1 weeks agoFull-time
Marketing Executive
, USA
Discovered 1 weeks agoFull-time