Senior Databricks Specialist / Data Architect
Key Skills for This Role
Full Job Posting
Hands-on senior Databricks specialist with proven experience in production enterprise environments. We are specifically looking for someone who can independently design, develop, troubleshoot, optimize, and govern Databricks solutions, rather than someone whose experience is limited to basic Databricks development.
Qualifications
- Minimum 8+ years of experience in data engineering/data architecture.
- Minimum 4+ years of strong hands-on experience with Azure Databricks.
- Strong hands-on experience with Apache Spark and PySpark.
- Strong experience with Delta Lake.
- Strong experience designing enterprise data platforms and data pipelines.
- Strong knowledge of Databricks Medallion Architecture.
- Strong experience with Unity Catalog, data governance, access control, and data lineage.
- Strong SQL skills and experience with large-scale data processing.
- Strong experience with Azure data services.
- Experience with Azure Data Factory / Azure Data Lake Storage Gen2.
- Experience integrating Databricks with APIs, enterprise applications, CRM systems, and other data sources.
- Experience with data modeling and dimensional modeling.
- Strong understanding of data quality and data governance.
- Strong understanding of performance optimization and Databricks cost optimization.
- Experience with CI/CD, Git, Azure DevOps, and deployment automation.
- Experience working with large-volume enterprise datasets.
- Ability to troubleshoot complex production data issues.
Preferred Certifications
- Databricks Certified Data Engineer Professional
- Databricks Certified Data Engineer Associate
- Microsoft Certified: Azure Data Engineer Associate
- Other relevant Databricks/Azure certifications
Responsibilities
- Design and implement enterprise-scale data solutions using Azure Databricks.
- Define and maintain Databricks data architecture and platform standards.
- Design data ingestion and integration pipelines from multiple source systems.
- Develop scalable ETL/ELT pipelines using Databricks and Apache Spark.
- Design and implement Medallion Architecture (Bronze, Silver, Gold) where appropriate.
- Work with Delta Lake and optimize Delta tables and data pipelines.
- Design data models and transformation processes for analytical and operational reporting.
- Integrate Databricks with Microsoft Dynamics 365 / Dataverse and other enterprise applications.
- Develop and optimize batch and near-real-time data processing pipelines.
- Implement data quality, validation, monitoring, lineage, and governance processes.
- Optimize Spark jobs, clusters, SQL queries, Delta tables, and data pipelines for performance and cost.
- Implement appropriate data security and access controls.
- Work with Unity Catalog for data governance, access control, cataloging, and lineage.
- Support Power BI and other analytical/reporting platforms through optimized data models and data services.
- Troubleshoot complex data engineering and performance issues.
- Establish development, testing, and production standards for Databricks.
- Support CI/CD and DevOps practices for Databricks solutions.
- Conduct technical reviews and mentor data engineers.
- Ensure data platform solutions meet security, availability, scalability, and disaster recovery requirements.
About malomatia
Qatar-based technology services provider driving digital transformation and innovation.
Visit company websiteApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at malomatia
Graduate Trainee - Change Management
, QAT
Malomatia is seeking a Graduate Trainee to establish and operate its centralized change management function and support enterprise transformation. The role develops governance, change plans, stakeholder engagement, adopt
Cybersecurity Engineer
, QAT
The employer is seeking an experienced cybersecurity engineer to implement and operate security controls and support SOC operations and incident response. Candidates should bring 8–10 years of cybersecurity engineering e
Information Security Officer
, QAT
The employer seeks a hands-on information security professional to advise on security governance, risk, compliance, operations, technical controls, and privacy. The role includes audits, policy development, security asse
IT Project Manager
, QAT
Job Description The IT Project Manager will lead and coordinate the planning, execution, and delivery of critical technology initiatives. The role requires strong leadership and communication skills, along with the abili
Graduate Trainee - Change Management
, QAT
Cybersecurity Engineer
, QAT
Information Security Officer
, QAT
Graduate Trainee - Change Management
Doha, QAT
Information Security Officer
Doha, QAT
Cybersecurity Engineer
Doha, QAT
IT Project Manager
Doha, QAT
IT Project Manager
, QAT
