Base Career helps you apply smarter for this job.
Key skills for this role
We're looking for a Lead / Senior / Staff QA Engineer to own quality for Nexxa's AI agent systems — products that plan, call tools, and take multi-step actions autonomously in industrial environments. This isn't traditional UI testing: you'll be designing evaluation frameworks for non-deterministic, tool-using systems, building golden datasets, catching regressions in reasoning quality, and stress-testing agent behavior under adversarial and real-world edge-case conditions.
You'll work closely with ML engineers, backend engineers, and Forward Deployed Engineers to define what "good" looks like for an agent operating in high-stakes industrial settings, then build the infrastructure and processes to measure it continuously.
We're looking for a Lead / Senior / Staff QA Engineer to own quality for Nexxa's AI agent systems — products that plan, call tools, and take multi-step actions autonomously in industrial environments. This isn't traditional UI testing: you'll be designing evaluation frameworks for non-deterministic, tool-using systems, building golden datasets, catching regressions in reasoning quality, and stress-testing agent behavior under adversarial and real-world edge-case conditions.
You'll work closely with ML engineers, backend engineers, and Forward Deployed Engineers to define what "good" looks like for an agent operating in high-stakes industrial settings, then build the infrastructure and processes to measure it continuously.
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
, USA
Sunnyvale, USA
Sunnyvale, USA
, CAN
, CAN
, CAN
Sunnyvale, USA
Sunnyvale, USA
Experience with human-in-the-loop evaluation workflows (labeling pipelines, inter-rater reliability, rubric design).
Background in ML/data science sufficient to read model evals and statistical significance.
Experience red-teaming or doing adversarial/security testing on ML systems.
Familiarity with observability/tracing tools for LLM applications (e.g., LangSmith, Arize, Langfuse, Weights & Biases).
Experience testing AI systems in industrial, IoT, or operational technology (OT) environments.
Prior experience setting up eval infrastructure from scratch at a startup or fast-moving team.
What We're Looking For A QA engineer who wants to define what quality means for autonomous, real-world AI systems.
Someone who can build rigorous evaluation infrastructure for problems that don't have a single right answer.
A systems thinker who enjoys turning ambiguous agent behavior into measurable, trustworthy signals.
A strong collaborator who partners well with ML engineers, backend engineers, and Forward Deployed teams.
Innovative Environment : Play a critical role in transforming heavy industries through groundbreaking AI and automation technologies.
Collaborative Culture : Be part of a team that values innovation, discipline, and continuous improvement.
Professional Growth : Benefit from significant opportunities for career development and advancement.
Competitive Compensation : Enjoy a comprehensive salary and equity package reflective of your expertise and contributions.
If you're passionate about AI quality and eager to help define what trustworthy autonomous systems look like in heavy industry, we'd love to connect.
Verified company details for this employer are not available yet.
Full-time
Senior · 5+ years experience
Hybrid
Apply faster on company sites with our extension.