Data Engineer Pyspark
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Job Title: Data Engineer – PySpark Location: Dubai Experience: 9+ Years Job Summary: We are looking for an experienced Data Engineer with strong expertise in PySpark and Cloudera Data Platform (CDP) to build and manage scalable data pipelines.
Key Skills for This Role
Full Job Posting
Job Summary
We are looking for an experienced Data Engineer with strong expertise in PySpark and Cloudera Data Platform (CDP) to build and manage scalable data pipelines.
The ideal candidate will be responsible for data ingestion, transformation, performance optimization, and ensuring data quality across large datasets.
Key Responsibilities
- Design, develop, and maintain ETL pipelines using PySpark on CDP
- Implement data ingestion from databases, APIs, and file systems
- Transform and process large datasets for analytics
- Optimize performance of data pipelines and Cloudera components
- Ensure data quality through validation and monitoring
- Automate workflows using Airflow/Oozie or similar tools
- Monitor pipelines and troubleshoot issues
- Collaborate with cross-functional teams and maintain documentation
Key Requirements
- 8+ years of Data Engineering experience with strong PySpark skills
- Hands-on experience with Cloudera (Hive, Impala, HDFS, HBase)
- Strong SQL and data warehousing knowledge
- Experience with Hadoop, Kafka, and distributed systems
- Familiarity with orchestration tools (Airflow/Oozie)
- Strong Linux scripting skills
Education
Bachelor’s or Master’s degree in Computer Science or related field
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at RiDiK (a Subsidiary of CLPS. Nasdaq: CLPS)
Database Administrator
Jeddah, KSA
RiDiK is seeking an experienced SQL DBA for a contract role in Jeddah, Saudi Arabia. The candidate will administer SQL Server 2012-2022, implement high availability features, manage performance tuning, and ensure databas
Celonis Business Consultant
Riyadh, KSA
RiDiK is seeking a Celonis Business Consultant with 2-3 years of experience to drive business transformation and process excellence. The role involves process analysis, client engagement, and change management support fo
Corporate Lawyer
Dubai, UAE
RiDiK is seeking a qualified Lawyer for a 3-month contract in Dubai to handle legal and regulatory matters for payments and financial services in the Middle East. The role involves advising cross-functional teams, managi
Finacle Automation Tester
Dubai, UAE
RiDiK is looking for a skilled Finacle Automation Tester with 8-10 years of experience to join their banking technology team in Dubai. The role involves designing and executing test cases for Finacle 11X core banking, bu
Celonis Business Consultant
Riyadh, KSA
RiDiK seeks a Consultant in Riyadh with 2-3 years of experience in Celonis for business transformation and process excellence projects. The role involves process analysis, client engagement, and change management support
Senior Performance Testing Engineer
Abu Dhabi, UAE
RiDiK is seeking a Senior Performance Testing Engineer with 6+ years of experience to design and implement performance testing frameworks for enterprise-level applications in Abu Dhabi. The role requires expertise in Neo
Director Healthcare
Dubai, UAE
RiDiK is seeking a Social and Healthcare Portfolio Lead (Director level) in Dubai to build and manage a portfolio of social development and healthcare accounts. The role involves owning the sector pipeline, leading bids,
Social and Healthcare Portfolio Lead
Dubai, UAE
RiDiK is seeking a Social and Healthcare Portfolio Lead with 10-15 years consulting experience in healthcare and social development, Arabic speaking, and a Bachelor's degree in Technology, Public Health, Social Developme
Database Administrator
Jeddah, KSA
Celonis Business Consultant
Riyadh, KSA
Corporate Lawyer
Dubai, UAE
Finacle Automation Tester
Dubai, UAE
Celonis Business Consultant
Riyadh, KSA
Senior Performance Testing Engineer
Abu Dhabi, UAE
Director Healthcare
Dubai, UAE
Social and Healthcare Portfolio Lead
Dubai, UAE