Lead cloud Data Engineer

Role Overview

Key Responsibilities

• Work with business lines to understand data ingestion and cloud data engineering requirements.
• Translate requirements with data modelers into cloud and on-premises engineering workload designs.
• Develop Databricks and Azure Data Factory pipelines.
• Apply medallion architecture across three data layers in an Azure data lake.
• Collaborate in multidisciplinary Agile teams using Agile methods and tools.
• Code and scale applications in a multi-cloud environment integrated with on-premises systems.
• Identify and remediate potential operational risks with the platform lead.

Qualifications and Experience

• At least 3 years of experience developing Databricks and Azure Data Factory pipelines.
• Very strong experience in Azure data engineering and Databricks.
• Experience supporting or developing enterprise-level data warehouse workloads.
• Experience developing and deploying PySpark workloads on distributed Spark computing.
• A graduate or bachelor's degree in Computer Science, Information Technology, Computer or Telecommunication Engineering, or an equivalent qualification.