Data platform Engineer

Role overview

Experience and qualifications

• At least 6 years of experience in data engineering or a related field is required.
• Strong Python, PySpark, Terraform, AWS, SQL, data warehousing, and data modeling experience is required.
• Experience with relational and non-relational databases, version control, and CI/CD pipelines is required.
• Strong problem-solving skills and the ability to work independently are required.

Key responsibilities

• Design, build, and maintain batch and streaming data pipelines using Python and PySpark.
• Architect and manage AWS infrastructure across services such as S3, EMR, Glue, Lambda, Redshift, EC2, and IAM.
• Use Terraform to provision and version-control infrastructure as code.
• Ensure data quality, integrity, security, availability, and performance across pipelines and storage.
• Monitor production pipelines, implement CI/CD workflows, and document technical designs and data flows.
• Mentor junior data engineers and contribute to engineering best practices.