Lead Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
BlackBull Markets is hiring a Lead Data Engineer to own data pipelines and infrastructure in Dubai or Cyprus. The role involves technical leadership, pipeline design, cloud infrastructure, and mentoring.
Key Skills for This Role
Responsibilities
- Set standards for code quality and system design, own architecture decisions across data lakes and warehouses, and mentor engineers
- Design and optimise batch and real time pipelines using Python, Go, Airflow, and DBT; lead schema migrations
- Provision and manage AWS (S3, Redshift, RDS) with Terraform and Ansible; run containerised workloads with Docker and Kubernetes (EKS)
- Keep Kafka, Redshift, and Kubernetes optimised and available with monitoring and alerting through Sumo Logic and PagerDuty
- Select the right database for the job across SQL, NoSQL, and graph, with depth in MySQL and PostgreSQL, and optimise queries
Requirements
- 3 to 5+ years in data engineering with strong grounding in distributed systems and modern data architecture
- Strong production level Python and/or Go
- Hands on experience with Apache Kafka, AWS Redshift, Apache Airflow, and DBT
- AWS expertise, solid SQL and NoSQL (MySQL, PostgreSQL preferred)
- Familiarity with Parquet, Avro, Iceberg, and Hudi
- Understanding of data security and regulatory frameworks such as GDPR
Full Job Posting
Role Overview
- BlackBull Markets is a New Zealand based multi asset broker operating across five jurisdictions
- Hiring a Lead Data Engineer to own pipelines and infrastructure, based on site in Dubai or Cyprus
- This is a hands on lead role: architect, build, mentor, and write production code in Python and Go
What you will do
- Technical leadership: set standards for code quality and system design, own architecture decisions, mentor engineers
- Pipelines and processing: design and optimise batch and real time pipelines using Python, Go, Airflow, and DBT; lead schema migrations
- Infrastructure and cloud: provision and manage AWS (S3, Redshift, RDS) with Terraform and Ansible; run containerised workloads with Docker and Kubernetes (EKS)
- Reliability: keep Kafka, Redshift, and Kubernetes optimised and available, with monitoring and alerting through Sumo Logic and PagerDuty
- Data and storage: select the right database for the job across SQL, NoSQL, and graph, with depth in MySQL and PostgreSQL
What you bring
- 3 to 5+ years in data engineering, with strong grounding in distributed systems and modern data architecture
- Strong production level Python and/or Go
- Hands on experience with Apache Kafka, AWS Redshift, Apache Airflow, and DBT, across both ETL and ELT
- AWS expertise, solid SQL and NoSQL (MySQL, PostgreSQL preferred), and familiarity with Parquet, Avro, Iceberg, and Hudi
- Understanding of data security and regulatory frameworks such as GDPR
- Nice to have: Scala, and experience in a regulated financial services or fintech environment
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at BlackBull Markets
Information Security & Data Protection Officer (ISO/DPO)
Dubai, UAE
BlackBull Markets seeks an Information Security & Data Protection Officer to support ISO 27001 security practice and GDPR compliance across multiple jurisdictions. The role involves maintaining the ISMS, conducting risk
Trading Platform Support Engineer
Dubai, UAE
BlackBull Markets seeks a Trading Platform Support Engineer to maintain MT4, MT5, and cTrader platforms. This junior role involves platform administration, systems monitoring, scripting, and trading connectivity support,
Lead Data Engineer
Dubai, UAE
BlackBull Markets is hiring a Lead Data Engineer to own data pipelines and infrastructure for a multi-jurisdiction fintech. This hands-on lead role involves architecting, building, and mentoring, with production code in