Senior Data Engineer
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
The employer is seeking a Senior Data Engineer to build and run data pipelines for Generative AI products on AWS. The role involves owning the data layer from ingestion to retrieval, including batch and streaming pipelines, vector indexing, and data modeling.
Key Skills for This Role
Responsibilities
- Build and run the batch and streaming pipelines that move data from source systems into the lake and through to the warehouse
- Build the data layer behind retrieval: source connectors, document parsing, chunking, embedding generation, and vector indexing
- Model curated, query ready datasets and metrics so AI and analytics consumers work from one definition
- Add quality checks, validation, and monitoring so problems surface before they reach a model or a user
- Apply access control: row and column level rules, PII handling, and entitlement aware datasets
- Work with platform and DevOps engineers to expose data and retrieval as documented, dependable services
- Keep storage, compute, and query costs in check, with particular attention to embedding and vector workloads
- Review code, write documentation, and help shape how the team builds its data layer
Requirements
- Eight or more years in data engineering overall
- Hands on work building data for AI or ML systems such as retrieval, embeddings, or feature data
- Strong SQL and strong Python, including PySpark or similar distributed processing
- Production experience across the AWS data stack: S3, Glue, Athena, Redshift
- Hands on experience with a layered data architecture (medallion, data lake feeding warehouse, or lakehouse)
- Experience with an ELT or integration tool such as Airbyte, Fivetran, or Meltano
- Experience with event driven pipelines using SQS and SNS, and with at least one streaming or CDC technology (Kinesis, Amazon MSK, Debezium)
- Hands on experience with a semantic or metrics layer over the warehouse (Cube or dbt Semantic Layer)
- Hands on experience with at least one vector store and embedding workflow (pgvector, Amazon OpenSearch, Pinecone, Weaviate, or Milvus)
- Comfort with columnar and open table formats: Parquet together with Apache Iceberg, Delta Lake, or Hudi
- Working knowledge of an orchestrator such as Amazon MWAA, Step Functions, Dagster, or Prefect
- Enough infrastructure as code to work closely with DevOps
Full Job Posting
Role Overview
- Our Generative AI products are only as good as the data behind them. This role owns that data layer from end to end: the pipelines that bring data in, the transformations that shape it, and the way it reaches retrieval systems, agents, and analytics.
- The work runs on AWS, and the aim is a single governed source that every consumer can rely on.
- We want someone who has already built data pipelines for AI systems, not only for reporting.
- The team is small and spans several languages, so you will own your pipelines and help set the standards the rest of us follow.
What You Will Do
- Build and run the batch and streaming pipelines that move data from source systems into the lake and through to the warehouse, owning the layers in between from raw to curated, along with their schema, quality, and lineage
- Build the data layer behind retrieval: source connectors, document parsing, chunking, embedding generation, and vector indexing, including re embedding when content changes
- Model curated, query ready datasets and metrics so AI and analytics consumers work from one definition instead of each rebuilding the logic
- Add quality checks, validation, and monitoring so problems surface before they reach a model or a user
- Apply access control where it belongs: row and column level rules, PII handling, and entitlement aware datasets, enforced as close to query time as the stack allows
- Work with the platform and DevOps engineers to expose data and retrieval as documented, dependable services
- Keep storage, compute, and query costs in check, with particular attention to the cost of embedding and vector workloads
- Review code, write the documentation, and help shape how the team builds its data layer
Requirements
- Eight or more years in data engineering overall. That includes hands on work building data for AI or ML systems such as retrieval, embeddings, or feature data, which can be a more recent part of your background
- Strong SQL and strong Python, including PySpark or similar distributed processing
- Production experience across the AWS data stack: S3 for the lake, Glue for ETL and the Data Catalog, Athena for serverless query, and Redshift as the warehouse
- Hands on experience with a layered data architecture, whether you call it medallion (bronze, silver, gold), a data lake feeding a warehouse, or a lakehouse, including building the transformation stages that move data from raw to curated
- Experience with an ELT or integration tool such as Airbyte, Fivetran, or Meltano, including building or maintaining connectors
- Experience with event driven pipelines using SQS and SNS, and with at least one streaming or change data capture technology such as Kinesis, Amazon MSK, or Debezium
- Hands on experience with a semantic or metrics layer over the warehouse, such as Cube or the dbt Semantic Layer
- Hands on experience with at least one vector store and embedding workflow: pgvector, Amazon OpenSearch, Pinecone, Weaviate, or Milvus
- Comfort with columnar and open table formats: Parquet together with Apache Iceberg, Delta Lake, or Hudi
- Working knowledge of an orchestrator such as Amazon MWAA, Step Functions, Dagster, or Prefect, and enough infrastructure as code to work closely with DevOps
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Mirai, a Scopely company
QA Automation Lead
Riyadh, KSA
Mirai, a Scopely Studio, is looking for a QA Automation Lead to drive automation excellence, mentor quality engineers, and implement modern quality engineering practices for gaming platforms. The role requires 5+ years o
Director of AI
Riyadh, KSA
Mirai, a Scopely company, seeks a Director of AI to lead AI strategy, production, and team development. This role combines local and global AI integration, overseeing AI engineers and product managers. Requires 15+ years
Senior DevOps Engineer
Riyadh, KSA
The employer is looking for a Senior DevOps Engineer to build and run infrastructure for Generative AI products, including CI/CD, container platforms, and AI workload support. The role requires 8+ years of DevOps experie
Senior IT Support Specialist - Google Experience
Riyadh, KSA
Mirai, a Scopely company, seeks a Senior IT Specialist to own IT operations and endpoint management. The role involves managing Google Workspace, Slack, networking, and endpoint management solutions in a fast-paced envir
Workplace Manager
Riyadh, KSA
About Mirai Mirai is a Scopely company, part of Savvy Games Group, backed by the Public Investment Fund (PIF) of Saudi Arabia. Based in Riyadh, Mirai focuses on building talent, enabling technology, and supporting the gr
Learning and Development Lead
Riyadh, KSA
About Mirai Mirai, a Scopely company based in Riyadh, Mirai is a subsidiary of Scopely, a global interactive entertainment and mobile-first video game company, home to many top, award-winning experiences such as "MONOPOL
Marketing Community Lead
Riyadh, KSA
Role Overview Mirai is looking for a passionate and strategic Arabic Community Manager to help us grow in the MENA region. In this key role, you'll build our local community from scratch and lead a team of junior moderat
Marketing Creative Lead
Riyadh, KSA
The Role We're on the hunt for a Creative Lead who blends artistic flair with a passion for results. In this role, you won't just manage—you'll roll up your sleeves and create standout ad campaigns for our mobile games.
QA Automation Lead
Riyadh, KSA
Director of AI
Riyadh, KSA
Senior DevOps Engineer
Riyadh, KSA
Senior IT Support Specialist - Google Experience
Riyadh, KSA
Workplace Manager
Riyadh, KSA
Learning and Development Lead
Riyadh, KSA
Marketing Community Lead
Riyadh, KSA
Marketing Creative Lead
Riyadh, KSA