Lead Data Engineer

Qualifications

• Collaborate with client stakeholders to gather requirements, structure solutions, and ensure high‐quality, timely delivery.
• Experience working in the Databricks tech stack with strong proficiency in SQL, Python, and PySpark
• Design and optimize data models, data marts, and Lakehouse/warehouse layers with strong focus on medallion architecture, query optimization, and performance engineering.
• Build, orchestrate, and monitor scalable data pipelines on Databricks, ensuring reliable ingestion, transformation, CDC handling, and incremental load strategies.
• Manage end‐to‐end pipeline operations including performance tuning, data quality monitoring, alerting, and issue resolution across production workloads.
• Lead a project team of data engineers supporting multiple workstreams and provide technical leadership through code reviews, best‐practice guidance, reusable pattern creation, and mentorship to engineering team members.
• Prepare and maintain project documentation to support project execution and delivery.
• Work Split
• 50% Technical – Data modeling, hands-on coding, orchestration, and pipeline monitoring.
• 50% Management– Client Collaboration, requirements gathering, designing technical solutions, presentations, global team management and mentoring.
• - Collaborate with client stakeholders to gather requirements, structure solutions, and ensure high‐quality, timely delivery. - Experience working in the Databricks tech stack with strong proficiency in SQL, Python, and PySpark - Design and optimize data models, data marts, and Lakehouse/warehouse layers with strong focus on medallion architecture, query optimization, and performance engineering. - Build, orchestrate, and monitor scalable data pipelines on Databricks, ensuring reliable ingestion, transformation, CDC handling, and incremental load strategies. - Manage end‐to‐end pipeline operations including performance tuning, data quality monitoring, alerting, and issue resolution across production workloads. - Lead a project team of data engineers supporting multiple workstreams and provide technical leadership through code reviews, best‐practice guidance, reusable pattern creation, and mentorship to engineering team members. - Prepare and maintain project documentation to support project execution and delivery. Work Split - 50% Technical – Data modeling, hands-on coding, orchestration, and pipeline monitoring. - 50% Management– Client Collaboration, requirements gathering, designing technical solutions, presentations, global team management and mentoring.