Data Engineer - PySpark & Informatica

Overview

Key Responsibilities

• Design and develop scalable enterprise data engineering solutions using PySpark and Informatica platforms
• Build and model Raw Data Vault structures with strong understanding of:
• + Source system data structures
• + Data retention policies
• + Data partitioning strategies
• Drive data reusability and reduce data duplication while optimizing performance
• Define and implement enterprise data architecture standards and best practices
• Create data exchange standards to support reusable and decoupled architectures
• Define standards for data archival, retention, and purging
• Identify and document critical data elements across enterprise source systems
• Collaborate with data engineers, architects, and source system SMEs to establish data engineering principles and governance standards
• Support large-scale data transformation and analytics initiatives
• Contribute to CI/CD-enabled data engineering and deployment environments
• Ensure compliance with enterprise data quality, governance, and security standards

Required Skills & Experience

• Strong hands-on experience in:
• + PySpark

+ Big Data Ecosystems

• Strong expertise in:
• + Data Architecture

+ Data Warehousing & Analytics Solutions

• Strong understanding of:
• + Dimensional Modeling

+ Slowly Changing Dimensions (SCD)

• + Role-playing Dimensions
• + Dimensional Hierarchies
• + Data Classification
• Experience transforming traditional Data Warehouse environments into Big Data-based architectures
• Expertise in:
• + Data Quality
• + Data Profiling
• + Data Governance
• + Data Security
• + Metadata Management
• + Data Archival
• Experience with cloud-native principles, architectures, and deployments
• Strong exposure to Continuous Integration (CI) and Continuous Deployment (CD) environments
• Strong analytical, troubleshooting, and problem-solving capabilities

Preferred Experience

• Banking or financial services domain experience is highly preferred
• Experience working within enterprise-scale digital transformation programs
• Exposure to modern analytics and enterprise data platforms

Education

Preferred Candidate Profile

• Ability to work in fast-paced and evolving environments
• Strong stakeholder collaboration and communication skills
• Experience handling multiple priorities and enterprise-scale projects
• Strong ownership mindset and attention to detail