Azure Data Engineer

• 6-7+ years of experience in data engineering, working with Azure Databricks, PySpark, and SQL.

• Proficiency in Python for data processing and transformation.

• Expertise in machine learning for processing large data set.

• Strong expertise in Azure Data Factory (ADF) for building and managing ETL processes.

• Familiarity with GitHub and Azure DevOps for version control and CI/CD automation.

• Design, develop, and optimize data pipelines using Azure Databricks, PySpark.

• Implement and maintain ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Synapse for orchestration.

• Develop and optimize complex SQL queries and Python-based data transformation logic.

• Work with version control systems (GitHub, Azure DevOps) to manage code and deployment processes.

• Implementing technical expertise and look for opportunities to reduce operational costs, eradicating unnecessary processes and exploring opportunities for service efficiency and rationalising the service where possible.

͏

Areas of responsibility

Gathering of requirements-Participate in gathering requirements for specific modules and ensure clear, structured documentation.

Work across multiple business processes to capture linkages, dependencies, and required changes.

͏

Solution Design-Facilitate creation of application solution designs for a specific module under the guidance of the Project Manager, ensuring alignment with business requirements and technical standards.

͏

Coding and Configuration-"Review the developed codes, resolve technical queries of the team for the assigned module and create consistent replication patterns, to ensure that the developed code aligns with the project standards.Collaborate with project managers through all phases of software development life cycle."

͏

Perform root-cause analysis to identify and troubleshoot recurring technical issues.

Modify software codes to resolve errors, adapt to new hardware and software, enhance performance, and upgrade interfaces."

Implementation-"Collaborate with technical teams to validate configurations, facilitate integration of new applications, and address dependencies during deployment.

Prepare user training documents and related frameworks to ensure smooth adoption new applications and minimize transition risks."