Senior Software Development Test Engineer

Experience of 5+ years SDET role

Technical Skills & Frameworks

Core Programming

Python — expert level; test automation, eval pipelines, data analysis (Pandas, NumPy, Pytest)

SQL — data output validation, ground truth querying, pipeline data quality checks

GenAI & Evaluation

RAGAS / TruLens / DeepEval / Promptflow etc — LLM evaluation frameworks for measuring faithfulness, hallucination rate, and task success

LangChain / LangSmith / LlamaIndex — agent workflow testing, prompt tracing, and LLM response debugging

OpenAI / Anthropic / Hugging Face APIs — direct LLM endpoint testing and output consistency validation

Vector DBs — retrieval quality testing, embedding validation, and latency benchmarking

Pandas / NumPy etc. — statistical analysis for output validation and error pattern investigation, data profiling, schema validation, and pipeline integrity checks

API & Automation

Pytest — modular, reusable test framework for AI eval and automation suites

Postman / REST Assured / Requests — API contract validation and service-level integration testing

MLOps & CI/CD

MLflow — tracking model versions and eval runs to detect regressions across updates

Docker / GitHub Actions / Jenkins — containerised test environments and deployment pipeline automation

Observability

Grafana / Kibana / OpenTelemetry — monitoring AI system health, output drift, and distributed tracing across agent pipelines

Cloud AI Services — AWS Bedrock, Azure OpenAI, or GCP Vertex AI for testing managed model endpoints and cloud-deployed agents

MLOps Platforms — MLflow, Kubeflow, Weights & Biases, or Feast feature stores for experiment tracking and model governance

ML Frameworks — Scikit-learn, TensorFlow, or PyTorch familiarity for understanding model internals and validating training pipelines

Infrastructure as Code — Docker, Kubernetes, Terraform for managing containerised test environments at scale

UI Automation — Playwright or Cypress for end-to-end conversational AI application testing

Performance Engineering — Locust or JMeter for load testing heavy AI inference endpoints under peak traffic

Statistical Hypothesis Testing — t-tests, confidence intervals, significance testing to distinguish real quality signal from noise

Synthetic Data Generation — using LLMs to generate diverse test cases and evaluation datasets at scale

Opportunity to work with some of the smartest minds to solve complex challenges at scale.

Impactful role in shaping and securing a global, cloud-native & AI driven platform.

Innovative, collaborative, and fast-paced culture

Current Tekion Employees: Please apply via the Internal Job Board in Ashby

Tekion is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, victim of violence or having a family member who is a victim of violence, the intersectionality of two or more protected categories, or other applicable legally protected characteristics.

For more information on our privacy practices, please refer to our Applicant Privacy Notice here.