zoho_recruit

Senior Data Engineer- Remote, India

LeewayHertz

Remote, IND

Full-time

Senior · 7+ years experience

Remote

Discovered 3 weeks ago

AzureADLS Gen2Azure DatabricksDatabricksPySparkSpark SQL

Free

Job Fit Check

Base Career helps you apply smarter for this job.

Ready to Scan

Key skills for this role

AzureADLS Gen2Azure Databricks

Understand the values and vision of the organization
Protect the Intellectual Property
Adhere to all the policies and procedures
Design, develop, and maintain scalable data pipelines for data ingestion, processing and storage.
Build and optimize data architectures and data models (Lakehouse / medallion, dimensional) for efficient data storage and retrieval.
Develop ETL/ELT processes to transform and load data from various sources into data warehouses and data lakes.
Build and orchestrate pipelines on Azure Databricks using PySpark, Spark SQL and Delta Lake, orchestrated with Databricks Workflows.
Integrate data from enterprise source systems including SAP (ABAP/CDS extracts, RPA/CSV or connectors) and load into Snowflake and Databricks.
Own end-to-end CI/CD for data pipelines using Databricks Asset Bundles (DAB) and Azure DevOps (Git repositories, YAML build and release pipelines), promoting code across dev, QA and production.
Implement data quality, validation, freshness and reconciliation checks with pipeline observability.
Ensure data integrity, quality, and security across all data systems.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Monitor and troubleshoot data pipelines and workflows to ensure high availability and performance.
Document data processes, architectures, and data flow diagrams.

7 - 8 years of hands-on data engineering experience building and running production data pipelines at scale.
Strong expertise in Azure and Azure data services (ADLS Gen2, Azure Databricks, Azure DevOps).Deep hands-on experience with
Databricks: PySpark, Spark SQL, Delta Lake, Lakehouse / medallion architecture and Databricks Workflows.
CI/CD for data engineering using Databricks Asset Bundles (DAB) and Azure DevOps (Git, YAML build/release pipelines, multi-environment promotion).

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Trusted by over 500,000 job seekers on Base Career