company_site
Databricks Data Engineer Senior Analyst
Avanade
Brisbane, AUS
Full-time
Entry · 2+ years experience
Onsite
Discovered 2 weeks ago
Azure DatabricksAzure Data FactoryApache SparkPySparkAzure SynapseAzure Blob Storage
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
Azure DatabricksAzure Data FactoryApache Spark
Key Skills for This Role
Azure DatabricksAzure Data FactoryApache SparkPySparkAzure SynapseAzure Blob Storage
Full Job Posting
Role -Databricks Data Engineer Senior Analyst About the Role We are seeking experienced Databricks Data Engineers to join our Data & AI.
You will play a pivotal role in designing, developing, and maintaining scalable data pipelines and transformation workflows on Azure Databricks, enabling advanced analytics and data engineering initiatives.
Key Responsibilities
Design, develop, and optimize scalable data ingestion pipelines using Azure Data Factory, Databricks, and Apache Spark.
Build and maintain Databricks notebooks using PySpark for data preparation, transformation, and enrichment to support Data Science use cases.
Collaborate closely with Data Scientists to understand data requirements and ensure timely delivery of clean, structured datasets.
Implement data quality checks, validation rules, and monitoring mechanisms.
Integrate data from various sources including structured, semi-structured, and unstructured formats.
Ensure data security, governance, and compliance with banking regulations.
Optimize Spark jobs for performance and cost-efficiency in Azure.
Participate in code reviews, design discussions, and agile ceremonies.
Document data pipelines, workflows, and technical decisions.
Required Skills & Qualifications Technical Skills Experience in data engineering, with at least 2+ years hands-on experience in Azure Databricks.
Strong proficiency in Apache Spark, especially PySpark for distributed data processing.
Experience
with Azure Data Factory, Azure Synapse, Azure Blob Storage, and Delta Lake.
Solid understanding of ETL/ELT processes, data modeling, and data warehousing concepts.
Proficiency in Python and/or Scala for data processing.
Experience
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Avanade
Microsoft Technology Account Lead - Resources
Houston, USA
SeniorFull-time
Discovered 1 weeks agoView →
Microsoft Technology Account Lead - Financial Services
Chicago, USA
SeniorFull-time
Discovered 1 weeks agoView →
US Business Development Sales Executive - Enterprise
Chicago, USA
SeniorFull-time
Discovered 1 weeks agoView →
US Business Development Sales Executive - SME&C
Scottsdale, USA
MidFull-time
Discovered 1 weeks agoView →
Artificial Intelligence Analyst
Brisbane, AUS
Discovered YesterdayFull-time
Microsoft Technology Account Lead - Resources
Houston, USA
Discovered 1 weeks agoFull-time
Microsoft Technology Account Lead - Financial Services
Chicago, USA
Discovered 1 weeks agoFull-time
Dynamics CE Functional Lead
Houston, USA
Discovered 1 weeks agoFull-time
Change Enablement Analyst - Microsoft Transformation
, USA
Discovered 1 weeks agoFull-time
US Business Development Sales Executive - Enterprise
Chicago, USA
Discovered 1 weeks agoFull-time
Associate Manager, Technical Advisory
Houston, USA
Discovered 1 weeks agoFull-time
US Business Development Sales Executive - SME&C
Scottsdale, USA
Discovered 1 weeks agoFull-time