Senior Data Architect - Databricks Platform

Architect responsible for designing and hardening enterprise-scale Databricks data platforms within client engagements — building the guardrails, automation, and standards that let a growing number of business teams safely build on the platform without it collapsing into unmanaged "click-ops."

Architects maintain primary depth in one core area (Data Platform Development or Data Migration & Modernization) while maintaining architectural awareness across both. Databricks depth is expected regardless of focus area.

Key Responsibilities:

Lead architectural design for Databricks-based data platforms or modernization programs, including workspace consolidation, permissioning/persona models, and configuration-as-code patterns

Databrick platform designs align with AWS leading-practice architecture patterns (as we are AWS centric)

Design of high-volume batch and real-time streaming pipelines leveraging PySpark and Structured Streaming (aka Apache Spark)

Design and integration of Databrick Unity catalog with third party metadata repositories (e.g., AWS Glue Catalog, Snowflake Horizon)

Design and implement automation (ConfigOps), CI/CD, and guardrail patterns that let business teams safely extend the platform without compromising governance

Build and maintain certified data products, a semantic layer, and lineage/metadata exposure so the platform can support both self-service and, eventually, agent-driven use cases

Ensure technical viability, scalability, and production readiness of platform components

Communicate architectural decisions and trade-offs clearly to both technical and business stakeholders; produce architectural documentation, assessments, and roadmaps

Partner with client business and engineering teams to understand upstream use-case needs and translate them into platform capabilities and guardrails, rather than working from requirements handed down secondhand