Kafka Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Dicetek LLC seeks a skilled Confluent Kafka Engineer for development, UAT, and production support of Kafka-based systems. The role requires expertise in Confluent Control Center, KSQLDB, Kafka Streams, and Kafka Connect.
Key Skills for This Role
Responsibilities
- Design, develop, and implement Kafka based solutions using Confluent Control Center, KSQLDB, Kafka Streams, and Kafka Connect
- Write and maintain high quality code for Kafka producers, consumers, and stream processing applications
- Develop and manage Kafka connectors for seamless integration with external systems
- Utilize Kafka Streams for real time processing of streaming data
- Employ KSQLDB for stream processing tasks including real time analytics and transformations
- Collaborate with data engineers, software developers, and DevOps teams to integrate Kafka solutions
- Ensure all Kafka based solutions are scalable, secure, and optimized for performance
- Develop and execute UAT plans to validate Kafka solutions before deployment
- Monitor and maintain Kafka clusters to ensure high availability and reliability
- Troubleshoot and resolve issues related to Kafka performance, latency, and data integrity
- Perform routine maintenance tasks such as patching, upgrades, and backups for Kafka clusters
- Provide 24/7 support for production systems including on call rotation
Requirements
- Bachelor's degree in Computer Science, Information Technology, or related field
- 3+ years of hands on experience with Apache Kafka and Confluent Kafka in a production environment
- Proven experience in Kafka development including producer and consumer API, stream processing, and connector development
- Experience with Kafka cluster management including setup, configuration, monitoring, and troubleshooting
- Familiarity with distributed systems, microservices architecture, and event driven design patterns
- Proficiency in Java, Python, or Scala
- Strong knowledge of Kafka internals including brokers, zookeepers, topics, partitions, and offsets
- Experience with monitoring tools (e.g., Prometheus, Grafana) and logging frameworks (e.g., Log4j, ELK Stack)
- Proficiency in Confluent Control Center, Kafka Streams, KSQLDB, and Kafka Connect
- Understanding of networking, security, and compliance aspects related to Kafka
- Familiarity with CI/CD pipelines and automation tools (e.g., Jenkins, GitLab CI)
Full Job Posting
Job Summary
- We are seeking a highly skilled Confluent Kafka Engineer to join our dynamic team. The role involves development, UAT, and production support of Confluent Kafka based systems.
Development Responsibilities
- Design, develop, and implement Kafka based solutions using Confluent Control Center, KSQLDB, Kafka Streams, and Kafka Connect.
- Write and maintain high quality code for Kafka producers, consumers, and stream processing applications.
- Develop and manage Kafka connectors for seamless integration with external systems.
- Utilize Kafka Streams for real time processing of streaming data.
- Employ KSQLDB for stream processing tasks including real time analytics and transformations.
- Collaborate with data engineers, software developers, and DevOps teams to integrate Kafka solutions.
- Ensure all Kafka based solutions are scalable, secure, and optimized for performance.
User Acceptance Testing (UAT)
- Develop and execute UAT plans to validate Kafka solutions including Kafka Streams, KSQLDB, and Kafka Connect.
- Work closely with QA teams to identify and resolve defects during UAT.
- Ensure all UAT activities comply with organization's standards and best practices.
- Provide detailed reports on UAT outcomes and work with developers to implement fixes.
Production Support
- Monitor and maintain Kafka clusters including Confluent Control Center to ensure high availability.
- Troubleshoot and resolve issues related to Kafka performance, latency, and data integrity.
- Perform routine maintenance tasks such as patching, upgrades, and backups.
- Implement monitoring solutions to proactively identify and mitigate production issues.
- Provide 24/7 support for production systems including on call rotation.
Qualifications
- Bachelor's degree in Computer Science, Information Technology, or related field.
- 3+ years of hands on experience with Apache Kafka and Confluent Kafka in production environment.
- Proven experience in Kafka development including producer and consumer API, stream processing, and connector development.
- Experience with Kafka cluster management including setup, configuration, monitoring, and troubleshooting.
- Familiarity with distributed systems, microservices architecture, and event driven design patterns.
- Experience with cloud platforms (e.g., AWS, Azure) and containerization (Kubernetes) is a plus.
Technical Skills
- Proficiency in Java, Python, or Scala.
- Strong knowledge of Kafka internals including brokers, zookeepers, topics, partitions, and offsets.
- Experience with monitoring tools (e.g., Prometheus, Grafana) and logging frameworks (e.g., Log4j, ELK Stack).
- Proficiency in Confluent Control Center for monitoring, managing, and optimizing Kafka clusters.
- Expertise in Kafka Streams for building scalable, fault tolerant stream processing applications.
- Experience with KSQLDB for real time processing and analytics on Kafka topics.
- Strong understanding of Kafka Connect for integrating Kafka with external data sources and sinks.
- Understanding of networking, security, and compliance aspects related to Kafka.
- Familiarity with CI/CD pipelines and automation tools (e.g., Jenkins, GitLab CI).
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Dicetek LLC
QA Automation Tester-Arabic Speakers
Sharjah, UAE
Design, develop, and maintain automated test scripts for web, mobile, and API applications. Create and execute test plans, test cases, and test scenarios based on business requi...
Senior PLSQL Developer
Abu Dhabi, UAE
Design, develop, and optimize complex PL/SQL code, including stored procedures, functions, triggers, and packages, to enhance database performance and functionality. Lead the ar...
System Analyst-AML
Abu Dhabi, UAE
Design and implement robust Anti-Money Laundering (AML) systems, ensuring compliance with global regulations like BSA/USA PATRIOT Act and FATF. Analyze complex financial transac...
Data Governance Specialist
Abu Dhabi, UAE
Dicetek LLC is looking for a Data Governance Specialist with 8-11 years of experience in data governance, data management, or data quality. The role requires expert-level hands-on experience with Axon, data lineage, and
Data Governance-GDO Strategy
Abu Dhabi, UAE
Dicetek LLC seeks a Data Governance Analyst for GDO Strategy in Abu Dhabi. The role focuses on risk and financial reporting, data lineage, data quality management, and stakeholder engagement within banking. Requires stro
System Analyst
Abu Dhabi, UAE
Dicetek LLC is seeking a System Analyst with AML domain knowledge for a contract role in Abu Dhabi. The role involves requirements gathering, data analysis, SQL, and testing including SIT and UAT. Strong analytical and c
Senior PLSQL Developer
Abu Dhabi, UAE
Dicetek LLC is seeking a Senior PLSQL Developer with strong Oracle Database and PL/SQL expertise for a contract role in Abu Dhabi. The role involves advanced SQL optimization, ETL development, and data migration, with ex
QA Automation Tester-Arabic Speakers
Sharjah, UAE
Dicetek LLC is seeking an experienced Automation Tester with 4-8 years of experience to design, develop, and execute automated test scripts for web, mobile, and API applications. The role requires expertise in Selenium,
QA Automation Tester-Arabic Speakers
Sharjah, UAE
Senior PLSQL Developer
Abu Dhabi, UAE
System Analyst-AML
Abu Dhabi, UAE
Data Governance Specialist
Abu Dhabi, UAE
Data Governance-GDO Strategy
Abu Dhabi, UAE
System Analyst
Abu Dhabi, UAE
Senior PLSQL Developer
Abu Dhabi, UAE
QA Automation Tester-Arabic Speakers
Sharjah, UAE