Senior Manager of AI Safety and Validation Engineering
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
A leading bank seeks a Senior Manager of AI Safety and Validation Engineering to design and implement technical measures ensuring reliability and security of AI solutions. This individual contributor role bridges AI development and risk standards, defining evaluation frameworks for ML and GenAI.
Key Skills for This Role
Responsibilities
- Define standardised evaluation rubrics and scoring systems for system performance and safety across AI archetypes
- Design comprehensive test cases covering deterministic logic and adversarial scenarios
- Implement automated evaluators such as LLM as a judge and targeted small language models
- Deploy explainability techniques like Shapley Values and Integrated Gradients for model transparency
- Partner with platform teams to build telemetry pipelines capturing embeddings and inputs
- Establish golden signals such as hallucination rates and semantic similarity for automated circuit breakers
- Analyse guardrail violations to identify emerging attack patterns and integrate human in the loop corrections
- Translate technical telemetry into actionable model health reports for second line risk functions
- Work with specialist software vendors to bring advanced safety capabilities into the internal AI ecosystem
Requirements
- Experience in AI safety and validation engineering
- Expertise in evaluation frameworks for predictive ML and Generative AI
- Hands on experience with explainability techniques (Shapley Values, Integrated Gradients)
- Knowledge of automated guardrails and monitoring protocols
Full Job Posting
Discover the Opportunity
- We are hiring a Senior Manager of AI Safety and Validation Engineering to join a leading bank in their Group Business Services division.
- This is primarily a technical practitioner role focused on the design and implementation of technical measures to ensure the reliability and security of advanced AI solutions.
Discover the Role
- As a Senior Manager of AI Safety and Validation Engineering, you will lead the creation of evaluation frameworks for both predictive Machine Learning and Generative AI.
- You will be responsible for defining standardised monitoring protocols and automated guardrails.
Discover the Responsibilities
- Define standardised evaluation rubrics and scoring systems for system performance and safety across various AI archetypes including RAG and agentic workflows.
- Design comprehensive test cases covering deterministic logic and adversarial scenarios to stress test AI outputs.
- Implement automated evaluators such as LLM as a judge and targeted small language models.
- Deploy explainability techniques like Shapley Values and Integrated Gradients to ensure model transparency and unfair bias detection.
- Partner with platform teams to build telemetry pipelines that capture embeddings and inputs while maintaining strict data privacy standards.
- Establish golden signals such as hallucination rates and semantic similarity to trigger automated circuit breakers.
- Analyse guardrail violations to identify emerging attack patterns and integrate human in the loop corrections.
- Translate technical telemetry into actionable model health reports for second line risk functions.
- Work with specialist software vendors to bring advanced safety capabilities into the internal AI ecosystem.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Client of Discovered MENA
AI Product Engineer
Abu Dhabi, UAE
Design and build AI-powered product features, develop pipelines, and integrate workflows using Python, Java, and PostgreSQL with strong software engineering skills.
Senior AI Engineer
, UAE
Design, develop, and deploy production-grade AI applications using modern technologies, collaborating with teams to solve complex business challenges.
Applied AI Engineer
Abu Dhabi, UAE
Design and deploy AI applications, build agent-based systems, develop retrieval-augmented generation systems, and collaborate on high-performance AI solutions.
Solution Architect - (GenAI, RAG, LLM, Azure)
Dubai, UAE
Responsible for designing and delivering generative AI solutions, requiring expertise in Azure, security architecture, and compliance in regulated sectors.
Principal Engineer Applied AI
Abu Dhabi, UAE
Our client is building an elite AI engineering organization focused on frontier model evaluation, sovereign inference, retrieval systems, and applied AI products for high-stakes enterprise and public-sector operations. W
Principal Engineer Applied AI
Abu Dhabi, UAE
Client of Discovered MENA seeks a Principal Engineer to be a deep technical voice, setting direction across teams and leading architectural decisions for AI systems. You will be hands-on, write prototypes, and mentor sen
Business Development Manager SaaS
Abu Dhabi, UAE
A premier SaaS company in Abu Dhabi is seeking a Business Development Manager to map out the market and secure new business partnerships. The role involves building relationships with senior stakeholders, understanding o
Data Analytics Director Crypto Derivatives
Abu Dhabi, UAE
Lead data analytics for a derivatives business, utilizing SQL and Python, while mentoring teams and influencing strategy through insights in a fast-paced environment.
AI Product Engineer
Abu Dhabi, UAE
Senior AI Engineer
, UAE
Applied AI Engineer
Abu Dhabi, UAE
Solution Architect - (GenAI, RAG, LLM, Azure)
Dubai, UAE
Principal Engineer Applied AI
Abu Dhabi, UAE
Principal Engineer Applied AI
Abu Dhabi, UAE
Business Development Manager SaaS
Abu Dhabi, UAE
Data Analytics Director Crypto Derivatives
Abu Dhabi, UAE