Senior Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Cander seeks a Senior Data Engineer to architect defense-grade Data Lakehouse architectures for AI-driven supply chain intelligence in Abu Dhabi. Requires 5+ years data engineering experience, expertise in Python, SQL, Spark, Kafka, Airflow, and experience with SAP S/4HANA and Ariba.
Key Skills for This Role
Responsibilities
- Design and deploy defense grade Data Lakehouse architectures as Single Source of Truth for AI driven Supply Chain Intelligence
- Lead technical execution of Workstream 1: Data & Platform Foundations, mapping enterprise systems like SAP S/4HANA and Ariba
- Architect and deploy ingestion pipelines for high volume transactional data from ERP systems ensuring near real time availability
- Build connectors for external market intelligence feeds to enrich internal procurement data
- Design and implement standardized procurement data model and taxonomy across multiple entities
- Engineer pipelines to ingest, process, and transform unstructured technical data into vector ready formats for RAG applications
- Manage and optimize Vector Databases for high speed retrieval for AI drafting assistants
- Establish data lineage and traceability protocols to support Model Based Systems Engineering and Digital Thread
- Implement Role Based Access Control, audit logging, and data redaction policies for compliance
- Deploy automated data quality frameworks to validate Bill of Materials completeness and cost data accuracy
Requirements
- 5+ years of experience in Data Engineering, with at least 2 years focused on building pipelines for Machine Learning or Generative AI applications in an enterprise setting
- Expert proficiency in Python, SQL, and modern data engineering frameworks including Apache Spark, Kafka, and Airflow
- Strong experience extracting data from complex ERP environments, specifically SAP S/4HANA and SAP Ariba
- Deep understanding of Data Lakehouse architectures (Databricks/Delta Lake), Relational Databases (PostgreSQL), and Vector Databases (Weaviate/Milvus)
- Experience building pipelines for RAG solutions, Conversational AI agents, and classical ML models using tools such as dbt, dagster, or prefect
- Proficiency with containerization (Docker, Kubernetes) and CI/CD pipelines for deploying data workflows in secure environments
- Experience in Supply Chain, Manufacturing, or Defense sectors
- Ability to navigate governance challenges between agile data work and rigid systems engineering requirements
Full Job Posting
Overview
- Headquartered in Abu Dhabi, specializes in developing AI powered supply chain solutions integrating fragmented SAP, Ariba, and unstructured data into actionable intelligence
- Focuses on architecting ultra secure data lakehouses for real time analytics, secure high performance pipelines, and advanced generative AI applications
- Supports critical enterprise workflows and procurement automation in defense and technical sectors
Job Summary
- Seeking a Senior Data Engineer to architect and develop core data infrastructure for AI driven transformation
- Design and deploy high security Data Lakehouse environments as single authoritative source of truth for AI systems
- Lead technical implementation of Workstream 1: Data & Platform Foundations for flagship projects
- Integrate complex enterprise systems such as SAP S/4HANA and Ariba, and process unstructured data like technical drawings and regulatory documents
Key Responsibilities
- Design and deploy defense grade Data Lakehouse architectures as Single Source of Truth for AI driven Supply Chain Intelligence
- Lead technical execution of Workstream 1: Data & Platform Foundations, mapping enterprise systems like SAP S/4HANA and Ariba
- Architect and deploy ingestion pipelines for high volume transactional data from ERP systems ensuring near real time availability
- Build connectors for external market intelligence feeds to enrich internal procurement data
- Design and implement standardized procurement data model and taxonomy across multiple entities
- Engineer pipelines to ingest, process, and transform unstructured technical data into vector ready formats for RAG applications
- Manage and optimize Vector Databases for high speed retrieval for AI drafting assistants
- Establish data lineage and traceability protocols to support Model Based Systems Engineering and Digital Thread
- Implement Role Based Access Control, audit logging, and data redaction policies for compliance
- Deploy automated data quality frameworks to validate Bill of Materials completeness and cost data accuracy
- Optimize data pipelines for on premise GPU clusters and air gapped environments
- Operate within a structured Sprint Zero environment to ensure data lineage, security, and governance meet defense grade standards
Qualifications And Experience
- 5+ years of experience in Data Engineering, with at least 2 years focused on building pipelines for Machine Learning or Generative AI applications in an enterprise setting
- Expert proficiency in Python, SQL, and modern data engineering frameworks including Apache Spark, Kafka, and Airflow
- Strong experience extracting data from complex ERP environments, specifically SAP S/4HANA and SAP Ariba, with familiarity in SAP BTP as a plus
- Deep understanding of Data Lakehouse architectures (Databricks/Delta Lake), Relational Databases (PostgreSQL), and Vector Databases (Weaviate/Milvus)
- Experience building pipelines for RAG solutions, Conversational AI agents, and classical ML models using tools such as dbt, dagster, or prefect
- Proficiency with containerization (Docker, Kubernetes) and CI/CD pipelines for deploying data workflows in secure environments
- Experience in Supply Chain, Manufacturing, or Defense sectors, with ability to understand Bill of Materials structures and procurement lifecycles
- Ability to navigate governance challenges between agile data work and rigid systems engineering requirements
- Proven ability to collaborate with Data Scientists and Backend Engineers to define data schemas that support predictive modeling and AI agents
Technical Skills
- Expert proficiency in Python and SQL for data engineering tasks
- Strong experience with modern data engineering frameworks including Apache Spark, Kafka, and Airflow
- Deep expertise in extracting and processing data from complex ERP environments, specifically SAP S/4HANA and SAP Ariba
- Comprehensive understanding of Data Lakehouse architectures such as Databricks and Delta Lake
- Experience with relational databases including PostgreSQL and vector databases like Weaviate and Milvus
- Proven ability to develop data pipelines for RAG solutions, conversational agents, and classical ML models using tools like dbt, Dagster, or Prefect
- Proficiency in containerization technologies including Docker and Kubernetes, and CI/CD pipelines
- Knowledge of defense grade security protocols including RBAC, audit logging, and data redaction policies
- Experience building and optimizing pipelines for high performance GPU clusters and air gapped environments
- Familiarity with automated data quality frameworks to validate critical datasets such as Bill of Materials and cost data
- Ability to design and implement standardized procurement data models and taxonomies
- Experience engineering pipelines for unstructured data ingestion including PDF tender documents, CAD metadata, and historical CONOPS
Location
- Abu Dhabi, United Arab Emirates
About Our Sister Company
- This role is based within our sister company, a trusted and integral part of our broader organizational network
- The sister company shares our mission, values, and commitment to excellence while operating as an independent entity
Project Focus: Supply Chain Intelligence & Platform
- Seeking a Senior Data Engineer to design and construct foundational data infrastructure for AI driven transformation
- Develop defense grade Data Lakehouse architectures as authoritative Single Source of Truth for AI agents
- Lead technical execution of Workstream 1: Data & Platform Foundations
- Map rigid enterprise systems such as SAP S/4HANA and Ariba while collaborating with cross functional teams to interpret complex unstructured data
Core Objectives
- Integrate structured and unstructured data sources to enable advanced analytics and AI capabilities
- Architect scalable solutions that harmonize fragmented datasets into a unified layer
- Support real time decision making and ensure compliance with stringent governance and security protocols
- Bridge enterprise systems with cutting edge AI applications, optimizing data workflows for performance
Why Join This Role?
- Opportunity to architect foundational data infrastructure that underpins national defense capabilities
- Contributions will directly empower AI driven systems to negotiate procurement contracts and support engineers in developing cutting edge defense technologies
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at cander
Senior Backend Engineer
Abu Dhabi, UAE
Cander is seeking a Senior Backend Engineer to architect and develop production-grade application infrastructure for AI-driven transformation. The role involves designing secure, high-performance backend systems for AI a
Senior QA Automation Engineer
Abu Dhabi, UAE
We are seeking a Senior QA Automation Engineer to lead validation and verification strategies for AI transformation initiatives. You will define rigorous standards for non-deterministic AI systems, creating automated tes
Senior Data Scientist
Abu Dhabi, UAE
A defense-focused AI company in Abu Dhabi seeks a Senior Data Scientist to lead development and deployment of generative AI and NLP solutions for engineering and supply chain applications. The role requires 5+ years of e
Senior Backend Engineer
Abu Dhabi, UAE
Cander seeks a Senior Backend Engineer in Abu Dhabi to architect and develop production-grade backend systems for AI-driven transformation. The role involves designing secure, high-performance microservices for AI agent
Senior Data Scientist
Abu Dhabi, UAE
Cander is seeking a Senior Data Scientist to lead AI initiatives for defense applications, developing production-grade generative AI, NLP, and predictive models. The role involves deploying LLMs, RAG pipelines, and time-
Senior Data Engineer
Abu Dhabi, UAE
Cander seeks a Senior Data Engineer to architect and develop core data infrastructure for AI-driven supply chain solutions. The role involves designing defense-grade Data Lakehouse architectures, integrating SAP S/4HANA
Senior Data Scientist
Abu Dhabi, UAE
We are seeking a Senior Data Scientist to lead the development and deployment of AI initiatives for defense capabilities. You will design and deploy generative AI, NLP, and predictive models to transform unstructured dat
Senior QA Automation Engineer
Abu Dhabi, UAE
We are seeking a Senior QA Automation Engineer to lead validation and verification strategies for AI systems in the defense sector. You will architect automated testing frameworks for LLMs and predictive engines, ensurin
Senior Backend Engineer
Abu Dhabi, UAE
Senior QA Automation Engineer
Abu Dhabi, UAE
Senior Data Scientist
Abu Dhabi, UAE
Senior Backend Engineer
Abu Dhabi, UAE
Senior Data Scientist
Abu Dhabi, UAE
Senior Data Engineer
Abu Dhabi, UAE
Senior Data Scientist
Abu Dhabi, UAE
Senior QA Automation Engineer
Abu Dhabi, UAE