linkedin
AWS Data Engineer
Venquis
Abu Dhabi, UAE
Full Time
Senior
3 weeks ago
AWS GluePySparkETLData IngestionLakehouse ArchitectureCDC
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
AWS GluePySparkETL
About the Role
Venquis is seeking an experienced AWS Glue Data Engineer for a large-scale cloud data transformation programme in Abu Dhabi. The role involves building scalable ingestion pipelines, migrating source systems into a Lakehouse platform, and ensuring efficient data delivery.
Key Skills for This Role
AWS GluePySparkETLData IngestionLakehouse ArchitectureCDC
Responsibilities
- Develop and maintain AWS Glue jobs for data ingestion into the Bronze layer of a Lakehouse architecture
- Implement appropriate loading strategies including CDC, full loads, delta loads, and snapshots
- Design and execute historical data migration and backfill processes
- Build robust, scalable, and reusable ETL pipelines using PySpark
- Optimise AWS Glue jobs through effective partitioning, parallelisation, and resource allocation
- Collaborate with platform teams to ensure optimal tooling and performance standards
- Support the migration of legacy source systems into a cloud based data platform
- Leverage automation and modern engineering practices to accelerate delivery
- Ensure code is version controlled and deployments are automated through CI/CD pipelines using Git and Terraform
- Implement and maintain source system connectivity
- Ensure compliance with data governance standards and data quality requirements
- Produce technical documentation and support operational handovers
Requirements
- 5+ years of experience in Data Engineering
- Strong hands on experience with AWS Glue and PySpark
- Proven background building ETL and data ingestion pipelines
- Experience working with Lakehouse architectures and Medallion design principles
- Strong understanding of CDC, incremental loading, and historical data migration strategies
- Experience with AWS services including S3, Athena, and Lambda
- Knowledge of Git, Terraform, and CI/CD deployment practices
- Strong problem solving skills with a focus on automation and process improvement
- Experience working in Agile environments using Jira
- Strong communication and stakeholder management skills
Full Job Posting
Overview
- An exciting opportunity has arisen for an experienced AWS Glue Data Engineer to join a large scale cloud data transformation programme. You will play a key role in migrating source systems into a modern Lakehouse platform, building scalable ingestion pipelines, and ensuring data is delivered efficie
Key Responsibilities Data Ingestion & Engineering
- Develop and maintain AWS Glue jobs for data ingestion into the Bronze layer of a Lakehouse architecture.
- Implement appropriate loading strategies including CDC, full loads, delta loads, and snapshots.
- Design and execute historical data migration and backfill processes.
- Build robust, scalable, and reusable ETL pipelines using PySpark.
Performance & Optimisation
- Optimise AWS Glue jobs through effective partitioning, parallelisation, and resource allocation.
- Collaborate with platform teams to ensure optimal tooling and performance standards.
Migration & Automation
- Support the migration of legacy source systems into a cloud based data platform.
- Leverage automation and modern engineering practices to accelerate delivery.
- Ensure code is version controlled and deployments are automated through CI/CD pipelines using Git and Terraform.
Governance & Quality
- Implement and maintain source system connectivity.
- Ensure compliance with data governance standards and data quality requirements.
- Produce technical documentation and support operational handovers.
Collaboration
- Work closely with architects, platform engineers, and data quality teams to deliver high quality data solutions.
- Contribute to Agile ceremonies and delivery planning activities.
Required Skills & Experience Technical Skills
- Strong hands on experience with AWS Glue and PySpark.
- Proven background building ETL and data ingestion pipelines.
- Experience working with Lakehouse architectures and Medallion design principles.
- Strong understanding of CDC, incremental loading, and historical data migration strategies.
- Experience with AWS services including S3, Athena, and Lambda.
- Knowledge of Git, Terraform, and CI/CD deployment practices.
Additional Experience
- Exposure to data quality frameworks is beneficial.
- Strong problem solving skills with a focus on automation and process improvement.
- Experience working in Agile environments using Jira.
Candidate Profile
- 5+ years of experience in Data Engineering.
- Comfortable working in fast paced transformation programmes with ambitious delivery targets.
- Strong communication and stakeholder management skills.
- Experience within financial services or highly regulated environments would be advantageous.
What's on Offer
- Opportunity to work on a major enterprise data transformation programme.
- Exposure to modern AWS and Lakehouse technologies.
- High impact role with significant ownership and technical responsibility.
- Collaborative environment focused on innovation and continuous improvement.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career