Kafka Architect / Streaming Data Architect
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Tata Consultancy Services is seeking a Kafka Architect / Streaming Data Architect to design enterprise-scale Kafka architecture aligned with Bank Lakehouse. The role involves leading real-time data engineering, platform implementation, and ensuring high throughput and low latency.
Key Skills for This Role
Responsibilities
- Design enterprise scale Kafka architecture aligned with Bank Lakehouse (Medallion Bronze/Silver/Gold)
- Define event driven architecture patterns for real time ingestion and processing
- Establish topic strategy, partitioning, replication, retention policies
- Architect multi cluster Kafka setup (dev/test/prod, DR strategy)
- Lead setup and configuration of Kafka brokers, Zookeeper/Kraft mode, Kafka Connect, Schema Registry, stream processing
- Integrate Kafka with Cloudera Data Platform, Informatica, Teradata, Denodo
- Design pipelines for event streaming ingestion and near real time batch ingestion
- Build CDC pipelines using Kafka Connect (Debezium)
- Ensure high throughput and low latency, optimize partition strategy and consumer group scaling
- Implement failover strategies, data replication, retry, dead letter queues, error handling frameworks
- Enforce RBAC, encryption, data masking/tokenization, integrate with metadata and lineage tools
- Implement monitoring using Cloudera Manager, Kafka metrics (JMX, Prometheus, Grafana)
Requirements
- Experience in BFSI domain
- Deep expertise in Cloudera Kafka (mandatory)
- Experience with Kafka Connect, Kafka Streams, Schema Registry
- Experience with Apache Spark / Spark Streaming
- Experience with CDC tools (Debezium preferred)
- Knowledge of REST APIs, Microservices integration
- Experience in large scale transformation programs
- Strong stakeholder management skills
- Ability to work in multi vendor environment
Full Job Posting
About Us
- Tata Consultancy Services (TCS) is an IT services, consulting and business solutions organization that has been partnering with many of the world’s largest businesses in their transformation journeys for over 50 years.
- TCS offers a consulting led, cognitive powered, integrated portfolio of business, technology and engineering services and solutions.
Key Responsibilities
- Architecture & Design: Design enterprise scale Kafka architecture aligned with Bank Lakehouse (Medallion Bronze/Silver/Gold).
- Define event driven architecture patterns for real time ingestion and processing.
- Establish topic strategy, partitioning, replication, retention policies.
- Architect multi cluster Kafka setup (dev/test/prod, DR strategy).
- Ensure exact point in time recovery and fault tolerance.
- Platform Implementation: Lead setup and configuration of Kafka brokers, Zookeeper/Kraft mode, Kafka Connect framework, Schema Registry, stream processing (Kafka Streams / Spark Streaming / Flink).
- Integrate Kafka with Cloudera Data Platform (CDP), Informatica, Data Warehouse (Teradata), Denodo.
- Real Time Data Engineering: Design pipelines for event streaming ingestion and near real time batch ingestion.
- Build CDC pipelines using Kafka Connect (Debezium).
- Enable ingestion from source systems.
- Support incremental and historical loads integration.
- Performance & Scalability: Ensure high throughput and low latency, support for concurrent consumers without degradation.
Key Technical Skills
- Core Technologies: Cloudera Kafka (mandatory), Kafka Connect, Kafka Streams, Schema Registry (Avro/Protobuf/JSON).
- Ecosystem Tools: Cloudera (CDP), Apache Spark / Spark Streaming, Apache Flink (preferred), Informatica (integration awareness), Teradata, Denodo.
- Data & Integration: CDC tools (Debezium preferred), REST APIs, Microservices integration, ETL/ELT Design patterns.
- DevOps & Infra: Kubernetes / Docker (optional but preferred), CI/CD for pipeline deployment, Linux environments.
- Banking Domain Experience (Mandatory): Experience in BFSI domain, understanding of regulatory reporting, data reconciliation (≥99% accuracy), AML, Payments, Trade finance systems.
- Soft Skills: Strong stakeholder management (business + IT), ability to work in multi vendor environment, experience in large scale transformation programs, strong problem solving and RCA skills.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Tata Consultancy Services
Senior Infrastructure Analyst
Calgary, CAN
Tata Consultancy Services is seeking a Senior Infrastructure Analyst to drive vulnerability remediation, improve patch compliance, and optimize monitoring and observability platforms. The role requires expertise in SCCM,
Kafka Architect / Streaming Data Architect
Jeddah, KSA
Tata Consultancy Services is seeking a Kafka Architect / Streaming Data Architect for a full-time role in Jeddah, KSA. The role involves designing enterprise-scale Kafka architecture for a Bank Lakehouse, implementing ev
Enterprise / Data Architect
Jeddah, KSA
Tata Consultancy Services is seeking an Enterprise/Data Architect for a role in Jeddah, KSA. The position involves owning end-to-end Data Lakehouse architecture, ensuring alignment with Medallion Architecture, and govern
AWS Technical Project Manager
Toronto, CAN
Tata Consultancy Services is seeking an AWS Technical Project Manager to work directly with the engineering team supporting Data Products, Data Fabric, Data Portal, metadata, and AI enablement. The role requires strong J
Data Analyst – Risk & Business Analytics
Montreal, CAN
TCS is looking for a Data Analyst to perform fast, accurate, and independent data analysis supporting business and risk management decisions. You will work with large datasets, develop dashboards, and deliver actionable
Tax compliance
Jeddah, KSA
Tata Consultancy Services (TCS) in Jeddah is hiring a Tax Compliance professional to ensure compliance with tax laws, prepare returns, manage VAT and corporate tax obligations, and liaise with tax authorities. The role r
Business Analyst
Mississauga, CAN
Tata Consultancy Services seeks a Payments Business Analyst with deep domain expertise in payments processing, clearing & settlement, and digital payments. The role requires strong experience in payments and knowledge of
Teradata Developer - Application Support
Jeddah, KSA
Tata Consultancy Services is seeking a skilled Teradata Developer & Data Lakehouse Application Support Engineer for a full-time role in Jeddah, Saudi Arabia. The role involves developing and optimizing Teradata SQL scrip
Senior Infrastructure Analyst
Calgary, CAN
Kafka Architect / Streaming Data Architect
Jeddah, KSA
Enterprise / Data Architect
Jeddah, KSA
AWS Technical Project Manager
Toronto, CAN
Data Analyst – Risk & Business Analytics
Montreal, CAN
Tax compliance
Jeddah, KSA
Business Analyst
Mississauga, CAN
Teradata Developer - Application Support
Jeddah, KSA