indeed
Senior Data Engineer
Luxoft
London, GBR
Onsite
Discovered 1 weeks ago
AWS GlueAWS EMRAWS LambdaAWS Step FunctionsSQLData warehousing and dimensional modelling
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
AWS GlueAWS EMRAWS Lambda
Role Overview
Lead technical planning for migration of a large data processing and ETL estate from Matillion to AWS Glue.
Reverse-engineer the existing landscape and design target-state architecture across infrastructure, processing, workflows, dependencies and operations.
Define migration patterns, validation frameworks and a delivery blueprint for a multi-phase financial services programme.
Key Skills for This Role
AWS GlueAWS EMRAWS LambdaAWS Step FunctionsSQLData warehousing and dimensional modelling
Full Job Posting
Project description
Lead technical planning for migration of a large data processing and ETL estate from Matillion to AWS Glue.
Reverse-engineer the existing landscape and design target-state architecture across infrastructure, processing, workflows, dependencies and operations.
Define migration patterns, validation frameworks and a delivery blueprint for a multi-phase financial services programme.
Responsibilities
- Design, build and maintain ETL/ELT pipelines on AWS.
- Migrate data pipelines and workloads from legacy or on-premises systems to AWS.
- Develop and optimise data models for warehouses and lakes using Redshift and S3.
- Build and orchestrate workflows with Glue, Step Functions, Lambda and EMR.
- Implement data quality, validation and reconciliation processes.
- Optimise pipeline performance and compute and storage costs.
- Ensure AWS data security and access control using IAM, KMS and encryption.
- Monitor and troubleshoot pipeline failures using CloudWatch and logging.
- Collaborate with analysts, BI developers and architects on data requirements.
- Document pipelines, architecture and operational runbooks.
Must-have skills
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Luxoft
Senior Data Engineer (Investment Data)
Abu Dhabi, UAE
Discovered TodayFull-time
GenAI Solutions Architect
Wilmington, USA
Discovered TodayFull-time
GenAI Solutions Architect
Buffalo, USA
Discovered TodayFull-time
Senior Software Developer - Networking platform
Bengaluru, IND
Discovered TodayFull-time
Insurance Business Analyst
London, GBR
Discovered TodayFull-time
Insurance Business Analyst
Norwich, GBR
Discovered TodayFull-time
Insurance Business Analyst
London, GBR
Discovered TodayFull-time
Insurance Business Analyst
Norwich, GBR
Discovered TodayFull-time