Cloudera Data Engineer

Requirements

• Required Qualifications & Experience
• 8+ years of hands-on data engineering experience
• 5+ years building enterprise-scale solutions on Databricks (Unity Catalog, Delta Lake, Delta Live Tables)
• Proven track record delivering Medallion/zonal lakehouse architectures in production
• Strong experience with high-volume, regulated data workloads (claims, financial, or healthcare data highly preferred)
• Technical Skills – Databricks Expertise (Core)
• Databricks Platform : Unity Catalog, Delta Lake, Delta Live Tables (DLT), Auto Loader, Workflows, Jobs, Repos, Lakehouse Monitoring
• Core Technologies : PySpark, Spark SQL, Spark Structured Streaming, Delta constraints & expectations
• Optimization & Performance : Liquid Clustering, Z-Ordering, Auto-Optimize, Dynamic Partition Overwrite, Photon engine
• Governance & Quality : Unity Catalog ACLs, data lineage, schema evolution, Great Expectations (or equivalent)
• Orchestration & CI/CD : Databricks Workflows, dbt on Databricks, Git integration, Azure DevOps / Jenkins
• Languages : Expert Python (PySpark), SQL
• Cloud : AWS/Azure/GCP (Databricks on any cloud)
• Preferred Qualifications
• Prior experience modernizing healthcare claims data lakes (COB, Payment Integrity, Medicaid/Medicare)
• Exposure to partner ingestion patterns, multi-format data (EDI, flat files, APIs), and downstream analytical workloads
• Familiarity with CMS/HIPAA data handling and compliance in Databricks environments