Databricks Platform Engineer Base Compensation Range: USD $80,000 – $100,000 The posted range is the hiring range for this role — a subset of the broader range available to employees over time — and reflects base salary across our national hiring scale. Final offers are based on several factors, including the candidate's skills and experience, internal pay equity, work location, market conditions for the role, and the specific scope and responsibilities of the position. The top of the range is reserved for candidates who notably exceed the requirements; the lower end applies to those with less experience or fewer preferred qualifications. For positions based in higher-cost zones (e.g., California, New York, New Jersey), actual compensation may exceed the posted range; your recruiter will share specifics during the process.
Key Skills for This Role
awsazurecloudwatchdatabricksdatadogdelta-lake
Full Job Posting
Responsibilities
Platform Administration
Design, deploy, and manage Databricks workspaces across Development, Test, UAT, and Production environments.
Establish workspace standards, networking, and environment isolation.
Manage platform upgrades, including Databricks Runtime (DBR) version planning, testing, and rollout.
Maintain platform availability, reliability, and performance.
Compute & Cluster Management
Create and manage cluster policies to enforce organizational standards.
Configure instance pools and job clusters.
Prevent oversized or non-compliant clusters.
Standardize cluster configurations across teams.
Optimize cluster startup times and resource utilization.
Security & Identity Management
Integrate Databricks with AWS IAM.
Configure and manage authentication and authorization.
Manage service principals, personal access tokens (PATs), and secret scopes.
Implement least-privilege access controls.
Support enterprise identity management and SSO.
Governance & Data Access
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
The DRG Quality Manager leads the planning, coordination, and execution of DRG quality review activities, ensuring quality teams meet productivity goals, inventory expectations, client SLAs, and internal quality standard
Job Title: Vocational Course Intern (Business Operations & Consulting Support) Location: New York, NY / Hybrid Program Duration: 36 Months Work Schedule: 1 Day per Week (Every Wednesday) Department: Consulting & Transfor
We are seeking a Security Master Subject Matter Expert (SME) to serve as the domain authority for Security Master, Entity Master, and Reference Data management for a leading global credit ratings organization. You will b
The DRG Auditor is responsible for performing an in-depth review of select inpatient medical records to ensure that the assigned codes and DRG are supported by clinical documentation and all diagnoses and procedures are
The DRG Auditor is responsible for performing an in-depth review of select inpatient medical records to ensure that the assigned codes and DRG are supported by clinical documentation and all diagnoses and procedures are
Senior MDM Developer – IBM InfoSphere Location: Cleveland, OH Experience: 7+ years A hands-on MDM developer who has actually built and supported IBM InfoSphere MDM implementations — not just heard of the tool. Someone wh
Manage catalogs, schemas, tables, and permissions.
Implement role-based access control (RBAC).
Enforce data governance and access policies.
Support audit and compliance requirements.
DevOps & CI/CD
Configure Git integration with Databricks Repos.
Support CI/CD pipelines for notebooks, workflows, and infrastructure.
Collaborate with DevOps teams on automated deployments.
Enable Infrastructure as Code (IaC) using Terraform where applicable.
Monitoring & Operations
Monitor workspace health, jobs, clusters, and platform performance.
Respond to incidents and perform root cause analysis.
Maintain operational dashboards and alerts.
Support production releases and platform maintenance activities.
Cost Optimization
Monitor Databricks usage and cloud spending.
Recommend cluster sizing and auto-scaling strategies.
Optimize job scheduling and compute utilization.
Track cost trends and recommend savings opportunities.
Compliance & Audit
Configure audit logging.
Ensure platform complies with enterprise security policies.
Support internal and external audits.
Maintain operational documentation and platform standards.
Platform Administration - Design, deploy, and manage Databricks workspaces across Development, Test, UAT, and Production environments. - Establish workspace standards, networking, and environment isolation. - Manage platform upgrades, including Databricks Runtime (DBR) version planning, testing, and rollout. - Maintain platform availability, reliability, and performance. Compute & Cluster Management - Create and manage cluster policies to enforce organizational standards. - Configure instance pools and job clusters. - Prevent oversized or non-compliant clusters. - Standardize cluster configurations across teams. - Optimize cluster startup times and resource utilization. Security & Identity Management - Integrate Databricks with AWS IAM. - Configure and manage authentication and authorization. - Manage service principals, personal access tokens (PATs), and secret scopes. - Implement least-privilege access controls. - Support enterprise identity management and SSO. Governance & Data Access - Administer Unity Catalog. - Manage catalogs, schemas, tables, and permissions. - Implement role-based access control (RBAC). - Enforce data governance and access policies. - Support audit and compliance requirements. DevOps & CI/CD - Configure Git integration with Databricks Repos. - Support CI/CD pipelines for notebooks, workflows, and infrastructure. - Collaborate with DevOps teams on automated deployments. - Enable Infrastructure as Code (IaC) using Terraform where applicable. Monitoring & Operations - Monitor workspace health, jobs, clusters, and platform performance. - Respond to incidents and perform root cause analysis. - Maintain operational dashboards and alerts. - Support production releases and platform maintenance activities. Cost Optimization - Monitor Databricks usage and cloud spending. - Recommend cluster sizing and auto-scaling strategies. - Optimize job scheduling and compute utilization. - Track cost trends and recommend savings opportunities. Compliance & Audit - Configure audit logging. - Ensure platform complies with enterprise security policies. - Support internal and external audits. - Maintain operational documentation and platform standards.
Qualifications
Required Skills
5–10 years of overall IT experience, including 3–5+ years of hands-on experience administering and managing Databricks platforms in production environments on AWS.
Required Skills 5–10 years of overall IT experience, including 3–5+ years of hands-on experience administering and managing Databricks platforms in production environments on AWS. - Databricks Platform - Workspace administration - Cluster management - Cluster policies - Instance pools - Job clusters - Databricks Runtime management - Workflows - Databricks Repos - Unity Catalog - Secret Scopes - Service Principals - Personal Access Tokens (PATs) - AWS - IAM - VPC fundamentals - EC2 concepts - S3 - CloudWatch - KMS - Security Groups - Networking fundamentals - DevOps - Git - CI/CD - Terraform - Infrastructure as Code - Azure DevOps, GitHub Actions, or Jenkins - Monitoring - Platform monitoring - Incident management - Log analysis - Performance tuning - Cost optimization - Required Experience Candidates should have hands-on experience with: - Managing enterprise Databricks platforms - Setting up multiple Databricks workspaces - Configuring Unity Catalog - AWS IAM integration - Cluster policy creation and governance - Runtime version upgrades - CI/CD implementation - Git integration - Platform monitoring and troubleshooting - Secrets management - Audit logging - Production support Cost optimization initiatives Preferred Qualifications - Experience managing large-scale Databricks environments (100+ users). - Experience supporting multiple business units or enterprise data platforms. - Hands-on experience with Terraform for Databricks infrastructure provisioning. - Knowledge of Lakehouse architecture and Delta Lake. - Experience with FinOps practices for cloud cost optimization. Familiarity with data governance and security frameworks. Nice to Have - Experience with AWS Organizations and Control Tower. - Knowledge of networking concepts for private connectivity (PrivateLink, VPC endpoints). - Experience with monitoring tools such as CloudWatch, Datadog, or Splunk. - Exposure to Apache Spark internals. - Experience supporting AI/ML workloads on Databricks. - Education Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. Certifications (Preferred) - Databricks Certified Data Engineer Professional or Databricks Certified Platform Administrator (if available). - AWS Certified Solutions Architect – Associate/Professional. - AWS Certified SysOps Administrator. - Terraform Associate certification.
About EXL
Professional Services10,001+Founded 1999
EXL is a global data and AI company that provides analytics, digital operations, and industry-specific services to enterprises in insurance, healthcare, banking, retail, media, and energy.
