Senior Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
NorthBay is seeking a Senior Data Engineer with 8-10+ years of experience to design and manage modern data platforms on Azure. The role involves building scalable data pipelines, data warehousing, and cloud-native architectures.
Key Skills for This Role
Responsibilities
- Design, develop, and maintain scalable data platforms and data pipelines on Microsoft Azure.
- Build and optimize batch and real time data ingestion frameworks from multiple data sources.
- Design and implement data lake, data warehouse, and lakehouse architectures.
- Develop and manage ETL/ELT processes using cloud native data engineering practices.
- Implement data transformation, cleansing, validation, and quality frameworks.
- Collaborate with stakeholders to understand data requirements and deliver scalable solutions.
- Optimize data storage, processing, and query performance.
- Implement security, governance, monitoring, and compliance best practices.
Requirements
- 8 10+ years of experience in Data Engineering, Data Warehousing, and Enterprise Data Platform development
- Strong hands on experience with Microsoft Azure Data Services
- Expertise in Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), and Azure SQL Database
- Strong proficiency in SQL, query optimization, and database performance tuning
- Hands on experience with PySpark, Apache Spark, and distributed data processing frameworks
- Strong programming skills in Python, Scala, or Java
Full Job Posting
Job Summary
- Seeking experienced Senior Data Engineer with 8 10+ years in designing, developing, and managing modern data platforms, data pipelines, and cloud based analytics solutions.
- Strong expertise in Azure Data Services, large scale data processing, data warehousing, ETL/ELT frameworks, and cloud native data architectures.
Key Responsibilities
- Design, develop, and maintain scalable data platforms and data pipelines on Microsoft Azure.
- Build and optimize batch and real time data ingestion frameworks.
- Design and implement data lake, data warehouse, and lakehouse architectures.
- Develop and manage ETL/ELT processes.
- Implement data transformation, cleansing, validation, and quality frameworks.
- Collaborate with business stakeholders, data analysts, data scientists, and application teams.
- Optimize data storage, processing, and query performance.
- Implement security, governance, monitoring, and compliance best practices.
- Support integration of data platforms with AI/ML, BI, and enterprise applications.
- Participate in architecture reviews, code reviews, troubleshooting, and technical mentoring.
- Ensure high availability, scalability, and operational excellence.
Required Skills & Qualifications
- 8 10+ years in Data Engineering, Data Warehousing, and Enterprise Data Platform development.
- Strong hands on experience with Microsoft Azure Data Services.
- Expertise in Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), and Azure SQL Database.
- Experience building and managing large scale ETL/ELT pipelines.
- Strong proficiency in SQL, query optimization, and database performance tuning.
- Hands on experience with PySpark, Apache Spark, and distributed data processing frameworks.
- Strong programming skills in Python, Scala, or Java.
- Experience with dimensional modeling, data warehousing concepts, and modern lakehouse architectures.
- Experience with structured, semi structured, and unstructured data.
- Strong understanding of data governance, data quality, metadata management, and security best practices.
- Experience with REST APIs, data integration patterns, and enterprise system connectivity.
- Hands on experience with Git, CI/CD pipelines, and DevOps practices.
Preferred Qualifications
- Experience with Microsoft Fabric, OneLake, Dataflows, and Fabric Data Engineering workloads.
- Experience with Databricks, Delta Lake, and lakehouse implementations.
- Knowledge of real time streaming technologies such as Azure Event Hubs, Apache Kafka, or Azure Stream Analytics.
- Experience supporting AI/ML and advanced analytics workloads.
- Familiarity with Power BI datasets, semantic models, and enterprise reporting architectures.
- Experience with data governance tools such as Microsoft Purview.
- Microsoft Azure Data Engineering certifications.
- Experience working in Agile/Scrum environments.
Nice to Have
- Experience with Microsoft Fabric Data Engineering and Analytics solutions.
- Exposure to MLOps and DataOps practices.
- Knowledge of containerization technologies such as Docker and Kubernetes.
- Experience with Infrastructure as Code (Terraform, ARM Templates, or Bicep).
- Familiarity with Snowflake, BigQuery, or other cloud data warehouse platforms.
- Experience with enterprise scale data migration and modernization projects.
- Understanding of Generative AI, Vector Databases, and data platforms supporting AI workloads.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at NorthBay - Pakistan
Devops Engineer - Azure
Abu Dhabi, UAE
NorthBay is seeking a Senior DevOps Engineer with deep Azure expertise to design and manage large-scale cloud environments. The role focuses on CI/CD, infrastructure automation, security, and scalability using tools like
Senior Devops Engineer
Abu Dhabi, UAE
Senior DevOps Engineer Location: Onsite – Abu Dhabi, UAE Job Overview We are seeking a highly experienced and motivated Senior DevOps Engineer to join our team in Abu Dhabi, UAE. The ideal candidate will be responsible f
Senior Backend Developer (Python/Node)
Abu Dhabi, UAE
Job Title: Senior Backend Developer Experience: 5-8 Years Location: Abu Dhabi (On-site) Employment Type: Full-Time Job Overview We are seeking a highly skilled Senior Backend Developer to design, develop, and scale robus
Azure Devops Engineer
Abu Dhabi, UAE
Job Description We are looking for a highly skilled Senior DevOps Engineer with extensive experience in designing, implementing, and managing large-scale, production-grade cloud environments. The ideal candidate will hav
Senior Data Architect (AWS Data Platforms)
Dubai, UAE
Job Title: Senior Data Architect – AWS Location: Dubai, UAE (Onsite) Experience Required: 12+ Years Employment Type: Full-Time Work Mode: Onsite Job Summary We are seeking a highly experienced Senior Data Architect with
QA Automation Engineer - GenAI
, UAE
NorthBay Solutions seeks a QA Automation Engineer with 7+ years of experience to design test strategies, build automation frameworks, and validate AI/ML and GenAI solutions. This full-time onsite role in Abu Dhabi requir
Senior Data Engineer
, UAE
NorthBay is seeking an experienced Senior Data Engineer for an onsite role in Abu Dhabi. The position involves designing and managing modern data platforms, data pipelines, and cloud-based analytics solutions on Microsof
QA Automation Engineer – GenAI
, UAE
NorthBay Solutions is hiring a QA Automation Engineer with 7+ years of experience to design and execute test strategies, build automation frameworks, and validate AI/ML and GenAI solutions. This full-time onsite role in
Devops Engineer - Azure
Abu Dhabi, UAE
Senior Devops Engineer
Abu Dhabi, UAE
Senior Backend Developer (Python/Node)
Abu Dhabi, UAE
Azure Devops Engineer
Abu Dhabi, UAE
Senior Data Architect (AWS Data Platforms)
Dubai, UAE
QA Automation Engineer - GenAI
, UAE
Senior Data Engineer
, UAE
QA Automation Engineer – GenAI
, UAE