Performance Test Engineer - Senior Manager

Vialto Labs ( VLabs ) is responsible for redesigning how work is delivered in the tax and immigration service lines, as well as driving operational efficiency across Vialto’s functional areas using AI. The team builds and deploys novel AI-enabled solutions that directly improve productivity and increase delivery quality for our clients. VLabs is accountable for rapidly turning innovative experiments into production-ready deliverables at scale and embedding them into day-to-day operations. This team focuses on the highest-impact workflows, creating standardized, repeatable capabilities that can be deployed globally. Operating with a mandate for speed and measurable outcomes, VLabs works alongside service line , product , and platform leaders.

The Performance Test Engineer is responsible for designing, building, and executing performance testing strategies for AI-enabled platforms and integrated enterprise workflows. This is a hands-on role that owns the full non-functional testing lifecycle, from load model design through execution and reporting. The role ensures systems meet defined performance thresholds, scalability requirements, and production readiness standards.

Given the presence of AI components such as OCR, vision language models, and LLM inference, this role requires designing performance models that account for variability in response times and non-deterministic behavior. The role works closely with AI Test Engineering, product, and DevOps teams to ensure performance risks are identified early and addressed prior to release.

Success is defined by the ability to produce actionable insights, support release gating decisions, and ensure systems can scale reliably under real-world conditions.

Own the end-to-end performance testing lifecycle, from test strategy through execution and reporting

Design load models based on real usage patterns and program volume data rather than synthetic assumptions

Define and validate non-functional requirements including response times, throughput, concurrency, and system stability

Build and execute JMeter-based load and stress test scripts from scratch across API, UI, and integrated workflows

Simulate concurrent sessions, transaction throughput, and end-to-end processing flows

Configure and execute distributed testing where required in partnership with DevOps teams

Incorporate variability in AI component response times into performance models and test scenarios

Define and validate timeout thresholds and acceptable performance ranges for AI endpoints

Identify and isolate the impact of AI components on overall system performance and SLAs

Monitor costs and token consumption plus related impact

Test performance across multi-step, AI-integrated workflows including document processing, event pipelines, and trigger-based systems

Evaluate system behavior across upstream and downstream dependencies in distributed architectures

Identify bottlenecks across services and clearly attribute root causes

Produce clear, concise performance reports including baseline metrics, SLA adherence, bottlenecks, and risk areas

Translate technical findings into actionable insights for engineering, product, and business stakeholders

Support release gating decisions by highlighting performance risks and mitigation strategies

Integrate performance testing into CI/CD pipelines where feasible to support continuous validation

Contribute to API contract testing using Postman/Newman as part of broader QE efforts

Support ongoing performance regression monitoring across releases

Develop reusable performance testing frameworks and standardized load testing patterns

Contribute to enterprise-level performance testing standards and best practices across VLabs

Log and manage non-functional defects in ADO with clear reproducibility and impact analysis

Ensure alignment with security, scalability, and reliability standards

Support auditability and traceability of performance testing results tied to milestone sign-offs

4+ years in performance testing, including experience leading non-functional testing efforts within delivery programs

Demonstrated experience working with AI-enabled systems including GenAI, LLMs, or agentic workflows

Experience with Locust, Grafana, OpenTelemetry , Datadog or Dynatrace

Experience with Postman for API profiling and AWS Cloudwatch

Proven track record building performance testing frameworks from scratch for complex, integrated systems

Experience with distributed, event-driven, or microservices-based architecture

Experience operating in sprint-based, iterative delivery environments

Background in regulated or compliance-driven industries preferred

JMeter:

Advanced scripting including thread groups, samplers, timers, assertions, and distributed execution

Analysis of throughput, error rates, and percentile response times with clear reporting outputs

Postman / Newman:

API contract testing and CI/CD integration for regression validation

SQL / Python:

Data validation under load, scripting for load modeling, and test data preparation

Performance Monitoring Tools:

Experience with tools such as Azure Monitor, Application Insights, or similar

Additional Tools:

Familiarity with k6, Gatling, or similar performance testing tools

Use of Chrome DevTools / HAR files for script generation and traffic analysis

Experience simulating authentication models such as OAuth2 or API key-based sessions

Ability to operate independently and own performance testing outcomes end-to-end

Strong analytical mindset with ability to diagnose complex system bottlenecks

Ability to balance rigor with speed in fast-paced delivery environments

Strong communication skills to translate technical performance insights into business impact

Collaborative approach working across engineering, AI, and DevOps teams

Bachelor’s degree required Advanced degree in Computer Science, Data Science, or related field preferred.

Location: India - Remote

We are an equal opportunity employer that does not discriminate on the basis of any legally protected status. Please note, AI is used as part of the application process.