zoho_recruit
Senior Databricks Engineer
Numentica
Chennai, IND
Full-time
Senior · 7+ years experience
Onsite
Discovered Yesterday
DatabricksApache SparkPySparkDelta LakeDatabricks WorkflowsSpark SQL
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
DatabricksApache SparkPySpark
Key Skills for This Role
DatabricksApache SparkPySparkDelta LakeDatabricks WorkflowsSpark SQL
Full Job Posting
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark .
- Develop and optimize PySpark/Spark jobs for large-scale data processing.
- Build reliable ETL/ELT pipelines using Databricks workflows, Delta Lake, and SQL .
- Implement data transformations, data quality checks, and performance optimization.
- Work with cloud data platforms such as AWS, Azure, or GCP .
- Optimize Spark jobs, cluster configurations, partitioning, joins, and data processing performance.
- Collaborate with data engineers, data scientists, analysts, and business teams to deliver data solutions.
- Implement best practices for CI/CD, version control, monitoring, and production support .
- Troubleshoot data pipeline and performance issues in production environments.
Required Skills
- 7+ years of experience in Data Engineering .
- Strong hands-on experience with Databricks .
- Strong expertise in Apache Spark / PySpark .
- Experience with Delta Lake, Databricks Workflows, and Spark SQL .
- Strong proficiency in Python and SQL .
- Experience building and optimizing large-scale ETL/ELT pipelines.
- Experience with at least one cloud platform: AWS, Azure, or GCP .
- Good understanding of data warehousing, data modeling, and distributed data processing.
- Experience with Git and CI/CD practices.
Preferred Skills
- Experience with Unity Catalog and Databricks governance.
- Experience with Azure Data Factory, AWS Glue, or similar orchestration tools .
- Knowledge of streaming technologies such as Kafka or Spark Structured Streaming .
- Experience with Infrastructure as Code or cloud automation.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Numentica
AI, Data and Platform Engineer
Chennai, IND
Discovered 1 weeks agoFull-time
Automation Engineer (Selenium, C#)
Redmond, USA
Discovered 1 weeks agoFull-time
Senior Software Engineer – React / .NET / AWS
, USA
Discovered 1 weeks agoContract
Automation Engineer (Selenium, C#)
, USA
Discovered 1 weeks agoContract
Full Stack Engineer
, IND
Discovered 2 weeks agoFull-time