Senior Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Deriv is looking for a Senior Data Engineer to build reliable data infrastructure powering trading decisions, product analytics, compliance, and AI/ML systems. You will own data accuracy end-to-end, design governed pipelines, and raise the engineering bar for the team in a global fintech environment.
Key Skills for This Role
Responsibilities
- Design and build ETL/ELT pipelines across batch and real time workloads using AI assisted development
- Bake in observability from the start: freshness checks, completeness monitoring, schema drift detection, lineage tracking, and alerting
- Implement automated data quality checks and anomaly detection into every pipeline
- Identify and resolve data issues before they surface to analysts or stakeholders
- Define and maintain data contracts: SLAs, SLOs, schema agreements, and producer consumer alignment
- Handle PII, access control, and auditability correctly in a regulated financial environment
- Optimise warehouse performance and cost—query efficiency, partitioning, clustering, orchestration reliability
- Build data models designed to scale beyond the immediate use case
- Partner with analysts, product, finance, and compliance teams to translate requirements into reliable data products
- Peer review pipeline code and push quality standards higher across the team
Requirements
- 6+ years in data engineering
- Experience with GCP, BigQuery, Airflow, Python (or equivalent at comparable scale)
- Experience building and maintaining real pipelines across batch and streaming workloads
- Experience with dbt or Dataform
- Knowledge of data modeling techniques (Kimball, Data Vault, Medallion)
- Experience with data contract frameworks or schema registries at scale is a plus
- Fintech, trading, or compliance heavy background is a strong plus
Full Job Posting
Why This Matters
- Deriv’s mission is Trading for Anyone, Anywhere, Anytime. Millions of traders, around the clock, across regulatory regimes.
- Every trade generates data. Every data point feeds analytics, compliance checks, fraud detection, and AI systems that serve customers in real time.
- When a pipeline delivers stale data, a trader sees the wrong price. When a schema drifts undetected, a compliance report goes wrong.
- Data engineering at Deriv isn’t back office plumbing. It’s the infrastructure that the entire business trusts.
Why Deriv
- Natural language interfaces querying the data warehouse directly—leaders ask questions in English, answers come back in seconds.
- Continuous KPI monitoring with anomaly detection surfacing problems before stakeholders notice.
- Dozens of fraud detection models running against production data, continuously.
- 400+ users on our internal workflow orchestration platform, fed by the data infrastructure you’ll help build and scale.
What You’ll Do
- Design and build ETL/ELT pipelines across batch and real time workloads using AI assisted development—reducing build time without cutting reliability.
- Bake in observability from the start: freshness checks, completeness monitoring, schema drift detection, lineage tracking, and alerting. Not retrofitted after launch.
- Implement automated data quality checks and anomaly detection into every pipeline—governance built in, not bolted on.
- Identify and resolve data issues before they surface to analysts or stakeholders.
- Define and maintain data contracts: SLAs, SLOs, schema agreements, and producer consumer alignment.
- Handle PII, access control, and auditability correctly in a regulated financial environment.
- Optimise warehouse performance and cost—query efficiency, partitioning, clustering, orchestration reliability.
- Build data models designed to scale beyond the immediate use case: dimensional modelling, semantic layers, reusable abstractions.
- Spot gaps in the data platform and address them without waiting to be assigned.
- Partner with analysts, product, finance, and compliance teams to translate requirements into reliable, governed data products.
- Peer review pipeline code and push quality standards higher across the team.
- Help onboard new engineers—share context, catch mistakes early, make others productive faster.
Who You Are
- 6+ years in data engineering. You know that writing SQL is the easy part. The hard part is making sure data arrives on time, matches the schema you promised, and doesn’t silently break when upstream changes.
- GCP, BigQuery, Airflow, Python—or the equivalent at comparable scale. You’ve built and maintained real pipelines across batch and streaming workloads. You’ve shipped with dbt or Dataform and understand why transformation layers matter.
- Data modelling techniques like Kimball star schema, Data Vault, or Medallion architecture aren’t buzzwords to you—they’re tools you pick based on the problem. You understand that a pipeline without a data contract is a pipeline waiting to fail.
- When data is wrong, you don’t patch the dashboard. You trace the problem to its source, fix it systematically, and make sure it doesn’t recur.
- You share learnings actively, create reusable resources, and give feedback that helps others ship better code.
- A fintech, trading, or compliance heavy background is a strong plus. You know the difference between handling PII in a textbook and handling it when auditors are checking your work.
Tech Stack
- Cloud & Warehouse: GCP, BigQuery
- Orchestration: Airflow
- Languages: Python, SQL
- Transformation: dbt / Dataform
- Streaming: Kafka, Pub/Sub
- Practices: CI/CD, version controlled pipelines, peer review, AI assisted development
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Deriv
Senior Regulatory Compliance Executive
Dubai, UAE
Deriv seeks a Senior Regulatory Compliance Executive to manage compliance for entities in investment services, banking, and virtual assets. You will conduct compliance monitoring, gap analyses, manage licensing with CMA
Senior Offensive Security Engineer
Dubai, UAE
Deriv is hiring a Red Team Lead/Operator to lead offensive security operations across infrastructure, applications, cloud, and people. The role involves full-kill-chain red team engagements, social engineering, cloud and
Senior SOC Analyst
Dubai, UAE
Deriv is hiring a Senior SOC Analyst to proactively hunt threats, build detections, and own incident response end-to-end. The role requires deep expertise in EDR, cloud security, and malware analysis, with a focus on aut
Senior Data Analyst
Dubai, UAE
Deriv is hiring a Senior Data Analyst to transform complex datasets into actionable insights, shape products, and drive business growth. The role requires 5+ years of experience, strong SQL and Python/R skills, and exper
Senior Account Executive
Dubai, UAE
Deriv is looking for an early-career finance professional to join its Trading Finance team in Dubai. You will prepare journal entries, reconciliations, support month-end close, and work with regulated entities. This role
Senior Specialist - Taxation
Dubai, UAE
Deriv is seeking a Senior Tax Specialist to own multi-jurisdiction tax compliance, tax accounting, transfer pricing, and automation. Requires 12-15 years tax experience with strong foundation in corporate income tax acro
IT Operations Engineer
Dubai, UAE
Deriv is looking for an IT Operations Engineer to maintain and automate IT infrastructure across global offices. The role involves managing endpoints, networking, and automation, with opportunities to grow into security,
Senior Regulatory Compliance Manager – Crypto Exchange & Virtual Assets
Dubai, UAE
As the Senior Regulatory Compliance Manager at Deriv’s crypto exchange and virtual asset business, you will play a critical role in ensuring adherence to global regulatory frameworks governing digital assets. You will de
Senior Regulatory Compliance Executive
Dubai, UAE
Senior Offensive Security Engineer
Dubai, UAE
Senior SOC Analyst
Dubai, UAE
Senior Data Analyst
Dubai, UAE
Senior Account Executive
Dubai, UAE
Senior Specialist - Taxation
Dubai, UAE
IT Operations Engineer
Dubai, UAE
Senior Regulatory Compliance Manager – Crypto Exchange & Virtual Assets
Dubai, UAE