Databricks Solution Architect

Individual pay is determined by many factors, including experience, relevant education or training, and organizational needs.

The mid-range to maximum of the salary range is generally reserved for individuals who are highly experienced in the role.

• Promote and enforce awareness of key information security practices, including acceptable use of information assets, malware protection, and password security protocols
• Identify, assess, and report security risks, focusing on how these risks impact the confidentiality, integrity, and availability of information assets
• Understand and evaluate how data is stored, processed, or transmitted, ensuring compliance with data privacy and protection standards (GDPR, CCPA, etc.)
• Ensure data protection measures are integrated throughout the information lifecycle to safeguard sensitive information
• Architect and lead the implementation of an enterprise lakehouse on Databricks (Delta Lake, Unity Catalog, Photon, Workflows) across one or more major clouds (AWS, Azure, or GCP).
• Design scalable batch and streaming data pipelines using PySpark , Spark SQL, Structured Streaming, and Delta Live Tables ; establish patterns for ingestion from operational systems, event streams, and third-party APIs.
• Define and enforce platform standards for data modeling (medallion architecture), CI/CD, code quality, testing, observability, and cost optimization.
• Lead the governance strategy using Unity Catalog — fine-grained access control, data lineage, audit, and PII handling — in partnership with security and compliance.
• Optimize Spark workloads for performance and cost: cluster sizing, Photon, autoscaling, file layout, Z-ordering, caching, and query tuning.
• Partner with ML engineers and data scientists to operationalize models using MLflow , feature stores, and model serving on Databricks.
• Own the cloud infrastructure footprint for the platform: networking, IAM, secrets, encryption, and Terraform/ IaC for Databricks workspaces and supporting services.
• Mentor a team of data engineers; lead architecture reviews, code reviews, and technical design sessions; raise the bar on engineering practices.
• Engage with stakeholders across analytics, product, and finance to translate business needs into a roadmap for the data platform.
• 8+ years of data engineering experience, with 4+ years building production workloads on Databricks.
• Deep expertise in Apache Spark ( PySpark and Spark SQL) — including performance tuning, partitioning strategy, and the Catalyst/Photon execution model.