Data Engineering Tech Lead

Bachelor’s or Master’s degree in Computer Science , Information, or a related field.

Minimum of 12 – 15 years of experience in data engineering, with a focus on ETL and Cloud technologies (can be less than 10 years )

Proficiency in Informatica ETL tools, SQL, and Azure Cloud platforms (preferably Azure but others acceptable)

Experience in Informatica IDMC - Architect, Design and develop ETL processes on IDMC and Informatica suite of tools. Familiarity with MDM Architecture and Data Flow

Experience in potential migration from existing data platforms to Databricks, Microsoft Fabric

Must have working experience with Azure based Data Pipelining, Scheduling and Monitoring and pyspark with ability to debug troublesome pipelines. Must have hands on expertise dealing with data pipelines

Strong working experience with Big Data technologies (Spark, Data Bricks) for Data integration, & processing (ingestion, transformation, curation, etc), preferably on Azure cloud, and a clear understanding of how the resources work and integrate with cloud and on-prem.

High level of proficiency with database and data warehouse development, including replication, staging, ETL, stored procedures, partitioning, change data capture, triggers, scheduling tools, cubes, and datamarts.

Experience working with backend languages such as Python.

Strong computer literacy and proficiency in data manipulation using Analytics tools/platform like Databricks, Azure Fabric using Spark engine

Expertise in at least one technology stack designing, developing, testing, and/or delivering complex software (i.e., Java, Python, Pyspark)

Excellent debugging, troubleshooting, and analytical skills

Strong analytical and problem-solving skills with the ability to own, troubleshoot and resolve complex data issues

Collaborate with Architects and Managers to develop Metrics & KPI’s

Identify any technical risks and forming contingency plans as soon as possible