Databricks Architect

• Capital Markets
• Wealth Management
• Insurance

• Regulatory Reporting and Compliance

• Design and implement enterprise-scale Databricks Lakehouse platforms.
• Define end-to-end data architecture for batch, streaming, and real-time analytics solutions.
• Develop cloud-native data platform strategies aligned with business objectives.
• Establish architectural standards, governance frameworks, and best practices.
• Lead modernization initiatives involving legacy data warehouse migration to Databricks.

• Design scalable ETL/ELT pipelines using Databricks, Apache Spark, and Delta Lake.
• Build medallion architecture (Bronze, Silver, Gold layers) for enterprise data platforms.
• Integrate data from multiple sources including databases, APIs, SaaS platforms, and streaming systems.
• Implement data quality, lineage, observability, and monitoring frameworks.
• Optimize data ingestion and transformation processes for performance and scalability.

• Architect Databricks solutions on Azure, AWS, or Google Cloud Platform.
• Design secure networking, storage, identity management, and access controls.
• Implement Infrastructure as Code (Terraform, ARM, CloudFormation).
• Optimize platform usage and cloud costs through governance and monitoring.
• Lead Databricks workspace design and multi-environment deployment strategies.

• Architect Databricks solutions supporting Machine Learning and Generative AI workloads.
• Design ML lifecycle management using MLflow.
• Enable enterprise AI capabilities leveraging Databricks Mosaic AI and GenAI frameworks.
• Collaborate with Data Scientists and AI teams to operationali ze ML models.
• Establish MLOps best practices and governance standards.

• Implement Unity Catalog for enterprise-wide governance.
• Define data security controls including RBAC, ABAC, encryption, and access management.
• Ensure compliance with industry standards including SOX, PCI-DSS, HIPAA, GDPR, and BFSI regulatory requirements.
• Establish data governance, metadata management, and audit frameworks.

• Act as the primary Databricks subject matter expert.
• Engage with executive stakeholders to define data transformation roadmaps.
• Conduct architecture reviews and provide technical leadership.
• Mentor data engineers, platform engineers, and architects.
• Collaborate with business, infrastructure, security, and application teams.

Range- $120,000-$150,000 a year