Senior Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
We are looking for a skilled Senior Data Engineer to design, build, and operate scalable data pipelines that power real-time processing and analytics. You will work on high-throughput data systems, ensuring reliability, performance, and maintainability across the data lifecycle — from ingestion to storage and search.
Key Skills for This Role
Full Job Posting
Overview
We are looking for a skilled Senior Data Engineer to design, build, and operate scalable data pipelines that power real-time processing and analytics.
You will work on high-throughput data systems, ensuring reliability, performance, and maintainability across the data lifecycle — from ingestion to storage and search.
This role requires strong experience in distributed systems, stream processing, and cloud-native data infrastructure.
Responsibilities
- Design and implement real-time and batch data pipelines
- Build and maintain scalable streaming systems
- Develop and optimize stream processing jobs
- Ensure reliable ingestion from multiple internal and external data sources
- Design event schemas and data contracts
- Implement data validation, transformation, and enrichment logic
- Optimize storage layouts and lifecycle management strategies
- Improve system observability (metrics, logging, alerting)
- Troubleshoot and resolve performance bottlenecks in distributed systems
- Implement retry, dead-letter, and replay mechanisms
- Ensure data quality, consistency, and governance
- Collaborate with Backend, DevOps, and Security teams
Required Qualifications
- Proficiency in Java, Python, and SQL; strong software engineering fundamentals
- Experience with distributed messaging systems (e.g., Apache Kafka) and stream processing frameworks (e.g., Apache Flink)
- Knowledge of event-time processing, windowing, state management, and handling out-of-order events
- Experience with cloud storage/data lakes, relational databases, and search/indexing engines (e.g., OpenSearch / Elasticsearch)
- Familiarity with cloud platforms (AWS preferred), IaC (Terraform), CI/CD, and containerization (Docker)
- Strong problem-solving skills and ability to design scalable, fault-tolerant data pipelines
- Nice to Have
- Experience with workflow orchestration platforms
- Experience in security, log processing, or observability domains
- Schema management tools (Avro, Protobuf, Schema Registry)
- Data lake table formats (Iceberg, Hudi, Delta Lake)
- Experience with distributed query engines (Athena, Trino, Presto)
- Multi-tenant system design
- Cost optimization in large-scale cloud environments
Soft Skills
- Strong problem-solving and debugging skills in distributed systems
- Ownership mindset with attention to reliability and quality
- Clear communication and documentation skills
- Ability to work cross-functionally
- Comfort working in fast-paced environments
What We’re Looking For
- 7+ years of experience in data engineering or distributed systems
- Strong fundamentals in system design and scalability
- Proven experience operating production-grade data platforms
- Ability to balance performance, cost, and reliability
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at SECUWALL
Security Consultant
Dubai, UAE
We are seeking a proactive and analytical Offensive Security Consultant to identify vulnerabilities, assess risks, and design robust security architectures. The role involves executing VAPT on web applications, mobile ap
GRC Specialist
Dubai, UAE
We are seeking a highly motivated GRC Specialist to join our dynamic cybersecurity team. You will contribute to the growth and delivery of Secuwall Governance, Risk, and Compliance consulting practice from the ground up.