Data Engineer (Microsoft Fabric | Azure | Databricks)

Role Summary

Data Engineer

1. Data Pipeline Development

• Develop and maintain
• ETL/ELT pipelines
• using Azure-native tools and frameworks.
• Process
• structured and semi-structured data
• (JSON, XML, CSV, APIs).
• Implement
• incremental data loading
• and transformation logic.
• Ensure data quality, validation, and consistency across pipelines.
• [center][/center]

2. Microsoft Fabric & Lakehouse Implementation

• Work with
• Microsoft Fabric components
• including:[ul data\=1]
• Eventstream
• Eventhouse
• Fabric Pipelines

Implement

Delta Lake

3. Azure Data Platform Development

• Build data solutions using:[ul data\=1]

• Azure Data Factory (ADF)

• Azure Databricks
• ADLS Gen2
• Ingest data from multiple sources:[ul data\=1]ERP / SAP systemsAPIs and external data sourcesSupport cloud-based data integration and transformation.[center][/center]

4. Big Data Processing

• Develop data transformation workflows using:[ul data\=1]
• **PySpark / Apache Spark**
• Optimize pipelines using:[ul data\=1]Partitioning, caching, and efficient joins
• Work on both
• batch and streaming data processing
• .[center][/center]

5. Data Modeling & Analytics

• Build
• analytics-ready datasets
• for reporting and BI.
• Apply
• dimensional modeling concepts
• .
• Deliver curated datasets for downstream consumption.
• [center][/center]

6. Real-Time Data Processing (Preferred)

• Implement real-time data pipelines using:[ul data\=1]

• Microsoft Fabric Eventstream

• IoT / streaming data sources
• Enable real-time dashboards and monitoring use cases.[center][/center]

7. Analytics & Visualization Support

• Integrate data pipelines with
• Power BI dashboards
• .
• Support business teams with clean and reliable datasets.
• [center][/center]

8. Modern Data Tools & Frameworks

• Work with:[ul data\=1]
• **dbt**
• for transformation and modeling
• Snowflake for analytics workloads
• Develop modular, reusable, and scalable data workflows.[center][/center]

9. DevOps & Best Practices

• Use

Github And Git

• for version control.
• Follow CI/CD and deployment best practices.
• Document data pipelines and processes.
• [center][/center]

Core Skills

• **Python / PySpark**
• **SQL (Strong proficiency)**
• ETL/ELT pipeline development
• [center][/center]

Cloud & Tools

• Microsoft Azure:[ul data\=1]

• Azure Data Factory (ADF)

• Azure Databricks
• ADLS Gen2

Data Engineering Concepts

• Lakehouse architecture
• Delta Lake
• Data modeling (basic dimensional modeling)
• Incremental data processing
• [center][/center]

Additional Tools

• dbt (preferred)
• Snowflake (good to have)
• Power BI
• [center][/center]

Experience Requirements

• **3–5 years**
• of experience in Data Engineering
• Hands-on experience in:[ul data\=1]
• Azure data ecosystem

• Databricks And PySpark

• Building data pipelines and transformations
• [center][/center]

Educational Qualifications

• Bachelor’s degree in

Computer Science And Engineering And It

• Relevant certifications are a plus:[ul data\=1]

• Azure Data Engineer (DP-203)

• Databricks certifications
• [center][/center]

Soft Skills

• Strong analytical and problem-solving skills
• Ability to work in a
• collaborative Agile environment
• Good communication skills
• Attention to detail and data accuracy
• [center][/center]
• Nice to Have
• Exposure to
• real-time/streaming data pipelines
• Experience with
• API integrations
• Basic understanding of
• machine learning workflows
• Experience working with
• ERP/SAP data