Senior AI & Data Engineer

The Senior AI & Data Engineer is an individual contributor role that acts as the technical subject matter expert at the intersection of AI engineering and data engineering. This is a uniquely dual-domain role: the successful candidate bridges the organization’s data strategy with its AI agenda, ensuring that intelligent systems are built on a foundation of governed, high-quality, well-architected data. On the AI side, this role designs and delivers end-to-end production-grade AI products — architecting multi-agent frameworks, fine-tuning and evaluating LLMs, building robust ML pipelines, and ensuring AI solutions are scalable, explainable, and responsibly governed. On the data side, this role defines the data architecture, transformation standards, and quality frameworks that make those AI products possible — owning the data platform across Databricks, Microsoft Fabric, and Collibra. Beyond individual delivery, this role provides technical leadership across cross-functional initiatives, defines engineering standards for both AI and data engineering practice, and leads the Applied AI Engineer and AI Data Engineer. The ideal candidate brings deep expertise across the full stack: from classical machine learning and modern generative AI through to data architecture, governance, and pipeline engineering — with the communication skills to translate that expertise into measurable business impact.

Technical Leadership & Subject Matter Expertise

Serve as the dual AI & data SME for the team and organization — the escalation point for complex technical decisions spanning model design, data architecture, pipeline engineering, and deployment.

Define and uphold engineering standards, design patterns, and best practices across both AI and data engineering disciplines; conduct architecture reviews and provide technical sign-off on all major deliverables.

Lead technical discovery for new AI and data use cases: evaluate feasibility, recommend solution approaches, and produce architecture documents for stakeholder alignment.

Participate in and lead cross-functional initiatives where AI and data strategy intersect — partnering with product, business, and platform teams to translate strategy into shipped solutions.

Mentor and upskill the Applied AI Engineer and AI Data Engineer through pair programming, code reviews, design sessions, and structured knowledge transfer.

Advanced LLM Engineering & Agentic AI

Architect and deliver complex agentic AI systems — multi-agent pipelines, tool-use frameworks, autonomous task orchestration

Design and implement advanced RAG architectures including hybrid search, re-ranking, query decomposition, and self-reflective retrieval patterns

Lead LLM evaluation frameworks: define metrics, build automated eval harnesses, and benchmark Claude, GPT, and Copilot performance against business KPIs

Assess and implement LLM fine-tuning and alignment strategies where pre-trained models do not meet requirements

Own LLM integration architecture — API design, latency optimization, token cost management, rate limit handling, and fallback strategies

Machine Learning & Data Science

Lead the full ML lifecycle: problem framing, data strategy, feature engineering, model selection, training, evaluation, deployment, and monitoring.

Develop advanced ML solutions across NLP, time-series forecasting, anomaly detection, recommendation, and classification/regression domains.

Design and implement MLOps pipelines for automated model training, versioning, A/B testing, and drift detection in production environments.

Apply statistical rigour — hypothesis testing, causal inference, experimental design — to validate model outcomes and business impact.

Ensure explainability (SHAP, LIME, attention visualization) and fairness assessments are embedded in all production models.

AI Solution Architecture & Integration

Architect end-to-end AI solutions that integrate LLMs, ML models, vector stores, data pipelines, and business APIs into cohesive, production-ready products.

Define data contracts and interface specifications between the AI engineering layer and the data engineering team.

Design for scale, reliability, and cost — including caching strategies, async processing, streaming inference, and model serving optimization.

Evaluate and recommend AI frameworks, platforms, and tooling to the technical leadership team; maintain a technology radar for the AI practice.

Data Engineering, Governance & Transformation

Architect data transformations and ingestion methods for AI products

Define and enforce data engineering standards, pipeline design patterns, and transformation best practices; ensure the AI Data Engineer delivers within a governed, reusable framework.

Design advanced transformation patterns: SCD handling, event-driven streaming ingestion, late-arriving data, and incremental load strategies optimized for both analytical and AI workloads.

Implement automated data quality validation

Champion a data-as-a-product mindset

RPA, Reporting & AI Automation

Provide technical leadership for AI-augmented RPA implementations — embedding LLM-based intelligence into automation workflows to handle unstructured inputs and exception scenarios.

Define AI metrics and KPIs for dashboards; support Power BI and reporting teams with data science-derived insights and model output integration.

Identify automation opportunities across the organization and build the business case for AI-led process transformation.

Innovation & Thought Leadership

Develop quick PoC's to

Continuously evaluate emerging research, models, and frameworks; translate relevant advances into internal prototypes and proof-of-concepts.

Present technical findings and recommendations to senior stakeholders and leadership — with clarity, confidence, and business context.

Contribute to internal AI community of practice: run knowledge sessions, publish internal technical guides, and foster a culture of experimentation

Bachelor's or Master's degree in Computer Science, Data Science, AI/ML, Engineering, Mathematics, or a related technical discipline; PhD is a plus.

7 – 10 years of hands-on experience in AI/ML engineering, applied data science, or LLM engineering roles.

Proven track record of delivering production AI systems — not just prototypes — with measurable business impact.

Deep expertise with at least two major LLM platforms (Claude, GPT, Gemini, or equivalent), including evaluation and integration at scale.

Significant experience with Collibra or an equivalent enterprise data governance platform

Demonstrated experience leading cross-functional AI initiatives and mentoring junior engineers.

Strong ML fundamentals alongside modern generative AI skills — able to operate across both paradigms.

Experience with responsible AI practices, including fairness auditing, explainability, and content safety, is strongly preferred.

We strive to provide our team members and their loved ones with a comprehensive suite of benefits that supports their physical, financial and emotional wellbeing.

We also invest in your career because the better you are, the better we all are. We have specific programs catered to helping you reach any career goals you have — whether you want to become a knowledge expert in your field or apply your skills to another division.

We are unconditionally inclusive in the way we work and celebrate individual uniqueness. We know varied backgrounds are valued and succeed here. We have the flexibility to manage our work and personal needs. We make bold moves, together, and are a force for good.

Follow @HPECareers on Instagram to see the latest on people, culture and tech at HPE.

HPE is an Equal Employment Opportunity/ Veterans/Disabled/LGBT employer. We do not discriminate on the basis of race, gender, or any other protected category, and all decisions we make are made on the basis of qualifications, merit, and business need. Our goal is to be one global team that is representative of our customers, in an inclusive environment where we can continue to innovate and grow together. Please click here: Equal Employment Opportunity .

HPE will comply with all applicable laws related to employer use of arrest and conviction records, including laws requiring employers to consider for employment qualified applicants with criminal histories.

We have become aware of an increase in fraudulent recruitment activities in which individuals impersonate our company or authorized recruitment agencies to offer fake employment opportunities. These scams may occur through false websites, emails, social media, or chat-based applications and often aim to obtain personal information or money. Please note that Hewlett Packard Enterprise (HPE), its direct and indirect subsidiaries and affiliated companies, and its authorized recruitment agencies/vendors will never charge a candidate a registration fee, hiring fee, or any other fee in connection with its recruitment and hiring process. We also never request personal information such as back account details, Social Security numbers, or national IDs via social media or chat applications.

All legitimate job opportunities will come through official company channels, and candidates are responsible for verifying the credentials of any third party claiming to represent the company. Any reliance on fraudulent communication is at the individual’s own risk, and HPE disclaims legal liability for any resulting damages. If you suspect recruitment fraud, do not share personal information or make any payments and report the incident to your local authorities immediately.