Lead Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
BlackBull Markets is hiring a Lead Data Engineer to own data pipelines and infrastructure for a multi-jurisdiction fintech. This hands-on lead role involves architecting, building, and mentoring, with production code in Python and Go.
Key Skills for This Role
Responsibilities
- Set standards for code quality and system design, own architecture decisions across data lakes and warehouses, and mentor engineers
- Design and optimise batch and real time pipelines using Python, Go, Airflow, and DBT; lead schema migrations across disparate systems
- Provision and manage AWS (S3, Redshift, RDS) with Terraform and Ansible; run containerised workloads with Docker and Kubernetes (EKS)
- Keep Kafka, Redshift, and Kubernetes optimised and available, with monitoring and alerting through Sumo Logic and PagerDuty
- Select the right database for the job across SQL, NoSQL, and graph, with depth in MySQL and PostgreSQL, and optimise queries and data structures for performance
Requirements
- 3 to 5+ years in data engineering with strong grounding in distributed systems and modern data architecture
- Strong production level Python and/or Go
- Hands on experience with Apache Kafka, AWS Redshift, Apache Airflow, and DBT across both ETL and ELT
- AWS expertise, solid SQL and NoSQL (MySQL, PostgreSQL preferred), familiarity with Parquet, Avro, Iceberg, and Hudi
- Understanding of data security and regulatory frameworks such as GDPR
- Nice to have: Scala, and experience in a regulated financial services or fintech environment
Full Job Posting
Role Overview
- BlackBull Markets is a New Zealand based multi asset broker operating across five jurisdictions, moving customer and trading data at scale.
- We are hiring a Lead Data Engineer to own the pipelines and infrastructure behind that operation, based on site in Dubai or Cyprus.
- This is a hands on lead role. You will architect, build, mentor, and still write production code in Python and Go.
What you will do
- Technical leadership: set standards for code quality and system design, own architecture decisions across our data lakes and warehouses, and mentor engineers
- Pipelines and processing: design and optimise batch and real time pipelines using Python, Go, Airflow, and DBT; lead schema migrations across disparate systems, including cleansing and normalisation
- Infrastructure and cloud: provision and manage AWS (S3, Redshift, RDS) with Terraform and Ansible; run containerised workloads with Docker and Kubernetes (EKS)
- Reliability: keep Kafka, Redshift, and Kubernetes optimised and available, with monitoring and alerting through Sumo Logic and PagerDuty
- Data and storage: select the right database for the job across SQL, NoSQL, and graph, with depth in MySQL and PostgreSQL, and optimise queries and data structures for performance
What you bring
- 3 to 5+ years in data engineering, with a strong grounding in distributed systems and modern data architecture
- Strong production level Python and/or Go
- Hands on experience with Apache Kafka, AWS Redshift, Apache Airflow, and DBT, across both ETL and ELT
- AWS expertise, solid SQL and NoSQL (MySQL, PostgreSQL preferred), and familiarity with Parquet, Avro, Iceberg, and Hudi
- Understanding of data security and regulatory frameworks such as GDPR
- Nice to have: Scala, and experience in a regulated financial services or fintech environment.
Why this role
- You will shape how a multi jurisdiction fintech handles trading data at the source, with direct ownership of architecture decisions from day one.
To apply
- Apply through LinkedIn or reach out to our talent team directly.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at BlackBull Markets
Information Security & Data Protection Officer (ISO/DPO)
Dubai, UAE
BlackBull Markets seeks an Information Security & Data Protection Officer to support ISO 27001 security practice and GDPR compliance across multiple jurisdictions. The role involves maintaining the ISMS, conducting risk
Trading Platform Support Engineer
Dubai, UAE
BlackBull Markets seeks a Trading Platform Support Engineer to maintain MT4, MT5, and cTrader platforms. This junior role involves platform administration, systems monitoring, scripting, and trading connectivity support,
Lead Data Engineer
Dubai, UAE
BlackBull Markets is hiring a Lead Data Engineer to own data pipelines and infrastructure in Dubai or Cyprus. The role involves technical leadership, pipeline design, cloud infrastructure, and mentoring. Candidates need