Azure DevOps Engineer (m/f/d)

Overview

Sdlc

• Databricks Asset Bundles (DABs): Enhance our DABs processes to enhance developer experience and velocity.
• CI/CD Pipeline Development: Build end-to-end automation in Azure DevOps to test and promote code across our environments (covering ADF, ADB and PowerBI)
• Infrastructure as Code (IaC): Automate the provisioning of Databricks workspaces clusters, and projects using Terraform or relevant templates.
• Governance: Mature and manage security policies and access controls
• ADF Management: Manage our ADF resources, optimizing for reliability and cost.
• Monitoring & Cost Optimization: Enhance observability across storage and compute to ensure cost effective operations

Educational Background

• Bachelor’s degree in a relevant field (e.g., Data Engineering, Software Engineering, Computer Science)

Professional Experience

• Extensive hands-on experience with Azure Cloud services; strong understanding of resource provisioning, networking, and security in Azure.
• Deep experience with Azure Databricks administration (clusters, scaling, job scheduling, monitoring).
• Strong experience with IaC (Terraform or Bicep), managing changes via code, and infrastructure versioning.
• Competence with version control (Git) and implementing CI/CD pipelines for data workloads/infra.
• Familiarity with Spark internals (how compute works, resource allocation, partitioning, memory), to aid in optimizing infrastructure.
• Knowledge of cost management on cloud (spot/preemptible VMs, auto-scaling, rightsizing).
• Strong scripting/automation skills (Python / PowerShell / Bash).
• Experience working with data engineering, analytics and BI teams; understanding of pipeline flows, data dependencies, and common data bottlenecks.
• Good to have Databricks or Azure certifications
• Azure DevOps Engineer in Abu Dhabi, United Arab Emirates