zoho_recruit
Senior Associate Data Engineering
National Payments Corporation of India (NPCI)
Mumbai, IND
Full-time
Mid · 3+ years experience
Onsite
Discovered 3 weeks ago
Apache KafkaSpark Structured StreamingApache FlinkS3MinIOHDFS
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
Apache KafkaSpark Structured StreamingApache Flink
Key Skills for This Role
Apache KafkaSpark Structured StreamingApache FlinkS3MinIOHDFS
Full Job Posting
The Opportunity
To design and implement real-time data streaming pipelines ensuring 24×7 availability of high-volume transactional data. Your role includes building fault-tolerant, scalable systems using modern big data technologies, integrating data from multiple sources into data lakes/lakehouses, and enabling downstream analytics. You will collaborate with internal teams (Data Science, Product, and Technology) to deliver reliable data solutions, optimize performance, and maintain compliance with NPCI standards.
Job Details
Job Title: Data Engineer (Streaming)
Division: Data Analytics
Years of Experience: 3–8 years
Education: Graduation in Computer Science/IT (preferably BE/B.Tech) or equivalent; advanced degrees are a plus
Employment Type: Full-time, Permanent
Location: Mumbai & Hyderabad
Key Responsibilities
- Design and develop real-time data pipelines using Apache Kafka and stream processing frameworks (Spark Structured Streaming / Apache Flink).
- Ensure 24×7 data availability with fault-tolerant, highly reliable systems.
- Implement ingestion, transformation, and load (ITL) patterns for data lakes/lakehouses (S3/MinIO, HDFS).
- Work with table formats like Iceberg, Hudi, or Paimon for ACID transactions and schema evolution.
- Optimize SQL queries on Trino/Hive for large-scale analytics.
- Develop orchestration workflows using DBT, Dagster, or Airflow for data transformations.
- Write efficient code in Python, Scala, and Java for data processing and automation.
- Collaborate with cross-functional teams to understand requirements and deliver high-quality data solutions.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at National Payments Corporation of India (NPCI)
Technology Audit
Mumbai, IND
Discovered 3 weeks agoFull-time
Senior Associate Onboarding
Chennai, IND
Discovered 3 weeks agoFull-time
Senior Associate Testing & Implementation
Hyderabad, IND
Discovered 3 weeks agoFull-time
Senior Associate Development, Distributed Systems
Mumbai, IND
Discovered 3 weeks agoFull-time
Senior Associate - Automated Reliability Test – (Development)
Hyderabad, IND
Discovered 3 weeks agoFull-time
Associate Data Science
Hyderabad, IND
Discovered 3 weeks agoFull-time
Associate Data Science
Hyderabad, IND
Discovered 3 weeks agoFull-time
Associate Fraud and Federated AI
Mumbai, IND
Discovered 3 weeks agoFull-time