Senior Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
EPAM Systems is looking for a Senior Data Engineer to join their team in Abu Dhabi for a large-scale data platform migration. The role involves building production-ready data pipelines on Databricks using PySpark and SQL, re-engineering legacy transformations, and mentoring junior engineers.
Key Skills for This Role
Responsibilities
- Build, optimize and productionize data pipelines on Databricks using PySpark and SQL
- Re engineer and migrate transformation logic from a legacy platform to Databricks
- Design and perform data validation and reconciliation to ensure accuracy across platforms
- Tune performance through partitioning, Delta optimization, cluster configuration and ingestion strategies
- Contribute to medallion layer design and implement data quality frameworks
- Review code and mentor junior engineers to promote best practices
- Work within Git based CI/CD setups and follow Agile delivery processes
Requirements
- 7+ years of data engineering experience with recent hands on delivery
- Strong proficiency with Databricks platform, PySpark, SQL and Delta Lake
- In depth knowledge of ingestion patterns, Spark performance tuning and partitioning strategies
- Solid SQL expertise with practical ability to write and validate queries
- Experience in data migration or re platforming projects
- Professional level English communication skills
- Bachelor’s degree in computer science, information technology or a related field
Full Job Posting
Overview
- We're looking for a Senior Data Engineer to join our team in Abu Dhabi, United Arab Emirates in an onsite working mode.
- This role involves working on a large scale data platform migration using modern data technologies and building production ready pipelines in a dynamic delivery environment.
Responsibilities
- Build, optimize and productionize data pipelines on Databricks using PySpark and SQL
- Re engineer and migrate transformation logic from a legacy platform to Databricks
- Design and perform data validation and reconciliation to ensure accuracy across platforms
- Tune performance through partitioning, Delta optimization, cluster configuration and ingestion strategies
- Contribute to medallion layer design and implement data quality frameworks
- Review code and mentor junior engineers to promote best practices
- Work within Git based CI/CD setups and follow Agile delivery processes
Requirements
- 7+ years of data engineering experience with recent hands on delivery
- Strong proficiency with Databricks platform, PySpark, SQL and Delta Lake
- In depth knowledge of ingestion patterns, Spark performance tuning and partitioning strategies
- Solid SQL expertise with practical ability to write and validate queries
- Experience in data migration or re platforming projects
- Professional level English communication skills
- Bachelor’s degree in computer science, information technology or a related field
Nice to have
- Databricks Certified Data Engineer Professional certification
- Familiarity with Palantir Foundry
- Background in financial services or asset management
- Experience with Unity Catalog governance
- Knowledge of large scale data quality or reconciliation frameworks
We offer
- End of service gratuity
- Private healthcare and life insurance
- Employee assistance program
- Wellness program
- Annual air travel allowance for expatriates
- Regular performance feedback and salary reviews
- Global travel medical and accident insurance
- Referral bonuses
- Learning and development opportunities including in house training and coaching, professional certifications, and courses
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at EPAM Systems
Field Marketing Manager
Dubai, UAE
EPAM Systems seeks a Field Marketing Manager to drive regional marketing strategies in the Middle East. This hybrid role in Dubai involves planning events, executing ABM campaigns, and building sales relationships. Requi
Field Marketing Manager
Dubai, UAE
EPAM Systems is seeking a Field Marketing Manager to drive regional marketing strategies in the Middle East. The role involves planning events, executing ABM campaigns, and collaborating with sales to generate leads. Can
Lead AI Engineer
Doha, QAT
EPAM Systems seeks a Lead AI Engineer to design and implement AI-driven strategies and architectures in Doha, Qatar. The role involves leading AI adoption, building end-to-end AI pipelines, and driving workforce capabili
Lead AI Engineer
Doha, QAT
EPAM Systems seeks a Lead AI Engineer to design and implement AI-driven strategies and architectures in Doha, Qatar. The role involves leading AI adoption, building production AI systems, and mentoring teams. Requires 4+
Data Technology Consultant
Coimbatore, UAE
EPAM Systems seeks a skilled Data Technology Consultant to lead and design large-scale data-driven solutions. The role involves defining data strategy, overseeing client engagements, and driving data product development.
Director Technology Solutions - Azure Cloud
Coimbatore, UAE
EPAM Systems seeks a Practice Lead for Azure Cloud to serve as the senior technical authority and strategic leader for the Azure practice. The role involves driving technical excellence, people development, delivery gove
Python Development Trainee
Riyadh, KSA
EPAM Systems offers a free, expert-led Python Development training program for Saudi nationals. The 3-month program requires up to 30 hours weekly engagement and is fully remote. Upon completion, participants may be cons
Senior AI Engineer (Python)
Doha, QAT
EPAM Systems is looking for a Senior AI Engineer (Python) to join their team in Doha, Qatar, working onsite. The role involves designing and deploying AI systems, building agentic workflows, and mentoring teams. Candidat
Field Marketing Manager
Dubai, UAE
Field Marketing Manager
Dubai, UAE
Lead AI Engineer
Doha, QAT
Lead AI Engineer
Doha, QAT
Data Technology Consultant
Coimbatore, UAE
Director Technology Solutions - Azure Cloud
Coimbatore, UAE
Python Development Trainee
Riyadh, KSA
Senior AI Engineer (Python)
Doha, QAT