Python Lead Software Engineer - (Cloud Data Platform — AWS/Databricks/Terraform)

Preferred qualifications, capabilities, and skills

• Advanced Apache Spark experience (PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns.
• Deep lakehouse and data platform expertise (e.g., Delta Lake concepts, schema evolution, data quality frameworks, lineage/metadata, and governance patterns) applied in production environments.
• Strong DevSecOps and platform engineering capability, including CI/CD design, automated security/quality gates, observability (logs/metrics/traces), SLOs/SLAs, and incident management/RCA in regulated environments.
• Proven end-to-end delivery leadership for large, cross-functional initiatives (roadmaps, dependency management, stakeholder alignment, budgeting/capacity planning, and measurable outcomes tied to reliability, cost, and time-to-market).
• Experience driving responsible, enterprise-approved AI-assisted engineering adoption, including coaching teams on safe use, validation standards for AI outputs, and building reusable “golden path” templates/pattern libraries (Terraform modules, Databricks job templates, pipeline scaffolds).