Principal Data Engineer

Role overview

Key responsibilities

• Lead scalable data engineering design and implementation in the Lakehouse environment.
• Define approaches for ingestion, transformation, integration, and data consumption.
• Lead complex ETL and ELT initiatives across multiple source systems and business domains.
• Design pipelines using Azure Databricks, PySpark, SQL, and Delta Lake.
• Migrate data assets, SQL views, transformation logic, and models into the Lakehouse.
• Define engineering standards, reusable frameworks, and best practices.
• Evolve layered architecture for business intelligence, analytics, and advanced data use cases.
• Optimize data models, datasets, queries, pipelines, and processing workloads.
• Define incremental loading, historical data, schema evolution, and recovery strategies.
• Establish data quality, monitoring, reconciliation, and exception-handling processes.
• Ensure governance, security, metadata, lineage, and access-management compliance.
• Conduct technical reviews, support planning and risk management, and mentor Data Engineers.

Required qualifications

• Graduation or post-graduation degree.
• 10–15 years of professional experience.
• Strong Lakehouse architecture and modern data platform expertise.
• Extensive Azure Databricks, Apache Spark, PySpark, and Delta Lake experience.
• Enterprise-scale ETL and ELT pipeline design experience.
• Expert-level SQL and performance optimization skills.
• Strong Python and PySpark experience.
• Analytical and dimensional modeling experience, including fact, dimension, and star-schema models.
• Strong understanding of data governance, metadata, lineage, data quality, security, and access management.
• Knowledge of Git, version control, CI/CD, and automated deployment.
• Ability to mentor engineers and influence technical decisions.

Technical and behavioral skills

• Ability to translate complex business processes into scalable data solutions.
• Experience working with architects and senior business and technical stakeholders.
• Ability to lead requirements discussions and identify dependencies and risks.
• Experience planning and managing complex data engineering initiatives.
• Ability to modernize existing data solutions and manage multiple workstreams.
• Strong analytical, problem-solving, decision-making, and collaboration skills.
• Accountability and commitment to delivery.

Location and work model

• The role is located in Chennai, India.
• The source metadata identifies the work model as hybrid.

What the company offers

• Global work environment with modern tools and technologies.
• Competitive salary and work-life balance.
• Innovative environment with support from a global team.
• Opportunity to work with leading crane-building technology.