naukri
Senior Data Engineer (Databricks)
ATAIN
Abu Dhabi, UAE
Senior
3 weeks ago
DatabricksAzure DatabricksApache SparkDelta LakePythonSQL
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
DatabricksAzure DatabricksApache Spark
About the Role
Role: Senior Data Engineer Experience: 5+ Years Location - Abu Dhabi Job Summary: We are seeking an experienced and proactive Data Engineer to lead the development of scalable,.
Key Skills for This Role
DatabricksAzure DatabricksApache SparkDelta LakePythonSQL
Responsibilities
- Design, develop, and optimize scalable data pipelines using Databricks and Azure Databricks, including Apache Spark and Delta Lake
- Lead and mentor junior engineers through code reviews, architecture discussions, and technical guidance
- Collaborate with cross functional teams to define data requirements and deliver reliable data solutions
- Manage workflows across Azure Data Lake Storage, Azure Synapse Analytics, and Azure Data Factory
- Contribute to platform architecture decisions involving data modeling, processing frameworks, and performance tuning
- Implement CI/CD pipelines for data workflows using tools like Azure DevOps or GitHub Actions
- Enforce engineering best practices around testing, documentation, and maintainability
- Monitor, troubleshoot, and resolve issues related to data pipeline performance and reliability
- Ensure data governance, quality, and compliance by integrating with tools like Unity Catalog or Azure Purview
Requirements
- Bachelor's degree in computer science or a related field
- 5+ years of experience in data engineering or a closely related technical field
- Strong proficiency in Python and SQL for building and managing data workflows
- Hands on experience with both Databricks and Azure Databricks, including production use of Apache Spark and Delta Lake
- Familiarity with cloud based data platforms (preferably Microsoft Azure)
- Experience with Azure Data Factory, Azure Data Lake, and/or Azure Synapse Analytics
- Demonstrated ability to lead technical projects and mentor junior engineers
- Solid grasp of CI/CD practices and version control systems (e.g., Git)
Full Job Posting
Job Summary
- Lead the development of scalable, production grade data pipelines while mentoring junior engineers and shaping engineering standards
- Work hands on with Databricks (including Azure Databricks), Delta Lake, and Apache Spark to support data driven decision making across analytics, product, and data science teams
- Combine technical depth with team leadership, ideal for someone ready to take ownership of critical data systems while guiding others
Key Responsibilities
- Design, develop, and optimize scalable data pipelines using Databricks and Azure Databricks, including Apache Spark and Delta Lake
- Lead and mentor junior engineers through code reviews, architecture discussions, and technical guidance
- Collaborate with cross functional teams to define data requirements and deliver reliable data solutions
- Manage workflows across Azure Data Lake Storage, Azure Synapse Analytics, and Azure Data Factory (or equivalent orchestration tools)
- Contribute to platform architecture decisions involving data modeling, processing frameworks, and performance tuning
- Implement CI/CD pipelines for data workflows using tools like Azure DevOps or GitHub Actions
- Enforce engineering best practices around testing, documentation, and maintainability
- Monitor, troubleshoot, and resolve issues related to data pipeline performance and reliability
- Ensure data governance, quality, and compliance by integrating with tools like Unity Catalog or Azure Purview
Required Skills
- Bachelor's degree in computer science or a related field
- 5+ years of experience in data engineering or a closely related technical field
- Strong proficiency in Python and SQL for building and managing data workflows
- Hands on experience with both Databricks and Azure Databricks, including production use of Apache Spark and Delta Lake
- Familiarity with cloud based data platforms (preferably Microsoft Azure)
- Experience with Azure Data Factory, Azure Data Lake, and/or Azure Synapse Analytics
- Demonstrated ability to lead technical projects and mentor junior engineers
- Solid grasp of CI/CD practices and version control systems (e.g., Git)
Preferred Skills
- Exposure to data transformation tools like dbt
- Familiarity with data governance tools such as Unity Catalog, Azure Purview, or similar
- Experience with containerization (e.g., Docker) and infrastructure as code tools (e.g., Terraform, Bicep)
- Strong communication and documentation skills; able to interface with technical and non technical stakeholders
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career