Data Solutions Architect

Role Overview

Architecture and Data Delivery

• Lead high-level and low-level design for HDFS or Ozone, YARN, Hive 3, Impala, Spark, Kafka, NiFi, Ranger, Atlas, Knox, KMS, and HMS with high availability.
• Define storage and compute topology, cluster sizing, multi-environment architecture, network zones, and perimeter security using Knox and TLS.
• Establish Bronze, Silver, and Gold standards covering naming, contracts, partitioning, quality checkpoints, schema evolution, retention, and recovery.
• Design batch pipelines using Informatica BDM, Spark, and Hive, and streaming paths using NiFi, Kafka, Hudi, Kudu, HBase, and Phoenix.
• Converge batch and streaming outputs into governed data stores and align serving models for BI and analytics using Impala and Phoenix.

Rules, Governance, and Operations

• Architect a metadata-driven rules engine for validation, standardization, derivation, survivorship, consent, PII handling, and SLA routing.
• Ensure rules are versioned, testable, and integrated with CI/CD.
• Implement Ranger and Atlas governance for policies, masking, classifications, lineage, glossary management, lifecycle, retention, legal holds, and audit trails.
• Establish CI/CD for infrastructure, schemas, table evolution, rules packs, and ETL deployments.
• Build SLA dashboards, alerts, runbooks, observability processes, disaster recovery, and business continuity strategies.
• Partner with domain SMEs, platform administrators, security, and compliance teams; lead architecture reviews and mentor developers and data engineering teams.

Required Qualifications

• 10+ years of experience in data architecture or data engineering.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, or equivalent experience.
• 3–5+ years of production experience architecting on Cloudera CDH or CDP.
• Proven delivery of on-premises Lakehouse environments using Medallion and Lambda architectures.
• Deep practical exposure to Iceberg, Hudi, Kudu, HBase, Phoenix, Hive 3, and Impala.
• Strong SQL expertise, including windowing, partitioning, MERGE or UPSERT, and cost-based tuning.
• Experience with Kerberos, TLS, AD/LDAP, Ranger policies, Atlas lineage and glossary, and SDX concepts.
• Experience designing metadata-driven rules engines and data quality frameworks for batch and streaming environments.
• Solid Linux fundamentals, Git and CI/CD experience, and strong documentation and stakeholder communication skills.

Preferred Qualifications

• Experience with Kafka patterns and change data capture from relational databases is preferred.
• Familiarity with Ozone, KMS or Key Trustee, and air-gapped deployments is preferred.
• SRE experience covering SLOs, error budgets, root-cause analysis, and disaster recovery exercises is preferred.
• Cloudera CDP, Informatica, and security certifications are a plus.
• Regulatory experience in PII, PCI, SOX, or GDPR environments is preferred.