Azure Developer

Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, Apache Spark, and SQL .

Build and optimize batch and streaming data pipelines using PySpark, Spark Structured Streaming, and Auto Loader .

Develop and support enterprise data lakehouse solutions using Delta Lake and Databricks technologies.

Implement data ingestion, transformation, cleansing, and aggregation processes for large-scale datasets.

Develop reusable frameworks and best practices for data engineering solutions.

Design and implement data models to support reporting, analytics, and business requirements.

Build and maintain Slowly Changing Dimensions (SCD Type 1 & Type 2) for data warehousing solutions.

Develop and optimize Change Data Capture (CDC) pipelines.

Optimize Spark workloads through partitioning, clustering, caching, and performance tuning techniques.

Ensure efficient query performance and scalability across large datasets.

Configure and manage Databricks Unity Catalog environments.

Create and manage catalogs, schemas, tables, materialized views, functions, and volumes.

Implement enterprise data governance, security, access control, and compliance standards.

Support metadata management and data lineage initiatives across the data platform.