Senior Data Engineer - Apache Spark and SQL - Vice President

Working at Citi is far more than just a job. A career with us means joining a team of approximately 219,000 dedicated people from around the globe. At Citi, you’ll have the opportunity to grow your career, give back to your community and make a real impact.

We are looking for a highly skilled Senior Databricks Engineer to contribute to the engineering, modernization, and continuous evolution of data processing platform on Databricks on AWS. While supporting the transition from the legacy Cloudera Hadoop platform to Databricks on AWS, this role will continue to play a key part in enhancing performance, simplifying pipelines, and delivering new capabilities on the Databricks platform over the long term.

The ideal candidate is a strong hands‑on Spark engineer with solid design experience, capable of contributing to architectural decisions while leading complex implementation and optimization efforts.

Responsibilities:

1. Platform Engineering & Modernization

Refactor and modernize existing Spark pipelines to Databricks native architectures

Eliminate legacy Hadoop dependencies and adopt cloud native AWS patterns

Enhance and extend existing processing logic using optimized Spark (JavaSpark / PySpark) on Databricks

2. Databricks Native Development

Build and optimize solutions using Databricks features, including Delta Lake, Databricks Workflows for orchestration and Auto scaling and job clusters

3. Design & Solution Engineering

Contribute to low and mid level architecture and design

Translate high level architecture into detailed technical designs

Define data models, pipeline patterns, and reusable components

Ensure solutions are scalable, maintainable, and production ready

4. Performance Optimization & Simplification

Analyze, improve Spark job performance and simplify complex or over engineered pipelines into standardized, efficient patterns

5. Engineering Standards & Best Practices

Follow and contribute to Databricks and Spark engineering standards

Write clean, modular, and testable code

Contribute to shared frameworks, reusable libraries, and quality standards

6. Collaboration & Stakeholder Engagement

Work closely with senior architects, platform teams, and DevOps engineers

Provide technical inputs, troubleshooting support, and implementation guidance

Participate in design discussions and technical decision making

7. Testing & Quality Assurance

Develop unit, integration, and data validation tests

Support production releases and post deployment validation

Bachelor’s degree/University degree or equivalent experience

------------------------------------------------------