greenhouse
Senior Data Engineer, AI Systems
Movable Ink
Toronto, CAN
Full-time
Senior · 5+ years experience
Onsite
CAD 144000-188000 yearly / year
Discovered 3 weeks ago
Apache SparkPySpark DataFrame APIGCP DataprocGoogle Cloud PlatformPythonParquet
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
Apache SparkPySpark DataFrame APIGCP Dataproc
Key Skills for This Role
Apache SparkPySpark DataFrame APIGCP DataprocGoogle Cloud PlatformPythonParquet
Full Job Posting
Responsibilities
- Build, maintain, and optimize production data pipelines that power AI-driven personalization at scale across content selection, send-time optimization, subject line personalization, and frequency capping
- Own and scale Spark-based batch pipelines, including cluster configuration, tuning, and performance optimization across GCP Dataproc
- Build and maintain our ML Data Lake, ensuring data quality, accessibility, and efficient storage
- Support the data needs of ML Engineers and Scientists for model development, training, and evaluation
- Identify and resolve performance bottlenecks and scaling limitations in data pipelines and infrastructure
- Collaborate with distributed systems engineers on the platform's architectural evolution, ensuring data layer continuity throughout
- Continuously improve data infrastructure for greater scalability and reliability
- Release features and data products that deliver measurable and tangible business value
Qualifications
- 5+ years of data engineering experience
- Deep expertise with Apache Spark, including the PySpark DataFrame API and experience solving challenging scaling problems
- Experience with large-scale data processing, cluster configuration, optimization, and tuning (we use GCP Dataproc)
- Strong software development skills in Python (unit testing, git, code review, CI/CD)
- Experience with data storage formats (we use Parquet, Delta Lake)
- Experience with event streaming data (we use Kafka)
- Experience with cloud computing platforms (we use Google Cloud Platform)
- Experience with advanced query optimization
- Familiar with Software Development Lifecycle practices, such as continuous integration/continuous delivery and automated deployment (we use Docker, Kubernetes, and GitHub Actions)
- Ability to collaborate with technical partners - you'll be working closely with ML engineers, scientists, and other teams to determine requirements and make design decisions
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Movable Ink
Senior Business Development Representative
New York City, USA
EntryFull-time
Discovered 1 weeks agoView →
Senior Technical Product Manager
Toronto, CAN
SeniorFull-time
Movable Ink recherche un Senior Technical Product Manager pour orienter le produit de son pipeline de données événementielles et son architecture de plateforme. La personne travaillera avec l’ingénierie des données, trad
Discovered 1 weeks agoView →
Solutions Architect
, USA
Discovered YesterdayFull-time
Implementation Manager
, USA
Discovered YesterdayFull-time
Data Engineer
, CAN
Discovered 2 days agoFull-time
Senior Implementation Manager
, CAN
Discovered 3 days agoFull-time
Business Development Representative
New York City, USA
Discovered 1 weeks agoFull-time
Senior Business Development Representative
New York City, USA
Discovered 1 weeks agoFull-time
Client Experience Associate
New York City, USA
Discovered 1 weeks agoFull-time
Senior Technical Product Manager
Toronto, CAN
Discovered 1 weeks agoFull-time