{bc}
Apply
linkedin

Senior Data Engineer (PySpark)

ValueLabs
Dubai, UAE
Fulltime
Mid-Senior
3 months ago
AgileGitMachine LearningPythonScalaSQL
Free

Job Fit Check

Base Career helps you apply smarter for this job.

?%
Ready to Scan

Key skills for this role

AgileGitMachine Learning
Smart Apply

Full Job Posting

Education And Experience

  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
  • 8+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform.
  • Client Specific Requirement: 1.
  • The Data Engineer role involves acquiring requirements, conducting EDA, ingesting required datasets, and transforming data using Big-data technologies & Feature Engineering techniques for machine learning models. 2.
  • Key tasks include building high-performance, secure, and scalable data pipelines, collaborating with Analytics Delivery Leads, Data scientists and other teams in an Agile environment, and communicating stakeholders effectively. 3.
  • The position requires a degree in a relevant field, at least 8-10 years of experience in data engineering and ML pipelines, and proficiency in technologies like Python, Spark (including expertise in optimization techniques), Hadoop, SQL, and Git. 4.
  • Should be good with Python as well.
  • Education and Experience
  • Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
  • 7+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform.

Technical Skills

  • PySpark: Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques.
  • Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase.
  • Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala).
  • Big Data Technologies: Familiarity with Hadoop, Kafka, and other distributed computing tools.
  • Orchestration and Scheduling: Experience with Apache Oozie, Airflow, or similar orchestration frameworks.
  • Scripting and Automation: Strong scripting skills in Linux.
  • Good at Data Modelling as well. Soft Skills
  • Strong analytical and problem-solving skills.
  • Excellent verbal and written communication abilities.
  • Ability to work independently and collaboratively in a team environment. Attention to detail and commitment to data quality.
  • Interested candidates please share resume to jobs.bfs2@valuelabs.com

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at ValueLabs

Retail Banking -Business Analyst

Dubai, UAE

TodayFull Time

Advent Dotnet Engineer

Dubai, UAE

1 weeks agoFull Time

DevOps Engineer

Dubai, UAE

1 weeks agoFull Time

Advent + .Net Developer (Advent APX Domain is mandate ) - 5+ YOE - Onsite - Dubai - Immediate Joiners Preferable

Dubai, UAE

2 weeks agoFull Time

Finacle Test Lead

Sharjah Emirate, UAE

2 weeks agoFull Time

Python Developer

Dubai, UAE

2 weeks agoFull Time

Senior Finacle Automation Engineer

Dubai, UAE

3 weeks agoFull Time

Finacle Automation Testing

Dubai, UAE

3 weeks agoFull Time