Lead Data Engineer

Lead Data Engineer

Job Located

Your Role

• 8+ years of experience in supporting and managing data engineering pipelines in production environments • Strong experience in Change Data Capture (CDC) using tools like Debezium and handling file-based ingestion (CSV)• Hands-on experience with GCP technologies including Pub/Sub, Dataflow (Apache Beam), and Datastream • Experience working with Cloud SQL (PostgreSQL) environments • Experience in managing staging and curated data layers for downstream consumption • Strong understanding of production monitoring, alerting, and incident management processes • Experience in API-based integrations within data pipelines • Strong programming experience in Java for data processing and support activities
• Roles & Responsibilities
• Provide end-to-end production support for data ingestion and processing pipelines • Monitor and manage CDC pipelines and batch ingestion processes for multiple source systems • Handle incidents, perform root cause analysis (RCA), and implement preventive fixes • Work with GCP Pub/Sub topics, subscriptions, DLQs, and alerting mechanisms for pipeline monitoring • Support data processing layers including staging and transformed datasets • Perform production deployments and coordinate testing across lower environments • Troubleshoot and resolve data, performance, and pipeline failures within SLAs • Serve as technical lead for AMS support, coordinating day-to-day operations • Own incident communication, escalation, and stakeholder updates • Maintain operational runbooks and provide guidance to L2/L3 support teams • Identify recurring issues and drive continuous improvements for system stability and reliability"