Senior Data Engineering Analyst

About Optum

Primary responsibilities

• Design, develop and operate cloud-native data pipelines on Azure and Databricks.
• Design and optimize Spark-based ETL frameworks for large-scale batch and incremental processing.
• Implement complex transformations and performance tuning using Python and Spark SQL.
• Modernize on-premises ETL workloads to Databricks and Snowflake while meeting scalability, reliability and governance standards.
• Collaborate with cross-functional teams to understand data requirements and deliver tailored solutions.
• Implement data validation and monitoring processes for data quality and integrity.
• Document data engineering processes and systems.
• Comply with company policies, employment terms and operational directives.

Required qualifications

• Graduate degree or equivalent experience.
• Bachelor's degree in engineering or a related discipline.
• Hands-on experience with Databricks and cloud-native data platforms, preferably Azure.
• Experience with ETL tools such as DataStage and large-scale data warehouse implementations.
• Experience migrating or modernizing on-premises ETL workloads to cloud architectures.
• Experience building ETL pipelines with Python, PySpark and Spark SQL.
• Understanding of dimensional modelling, data quality, reconciliation and performance tuning.
• Familiarity with Azure, AWS or GCP and workflow orchestration tools such as Airflow.