Big Data Engineer – PySpark
Job Fit Check
Base Career helps you apply smarter for this job.
Key skills for this role
About the Role
Speridian Technologies is looking for a Big Data Engineer with strong PySpark skills. The role requires expertise in Hadoop, Hive, data warehousing, and SQL optimization. Ideal candidates have experience with big data technologies and performance tuning.
Key Skills for This Role
Responsibilities
- Develop and optimize big data processing pipelines using PySpark.
- Work with Hadoop ecosystem components including HDFS and Hive.
- Design and implement data warehousing solutions.
- Perform performance tuning and optimization of data processing jobs.
Requirements
- Proficiency in PySpark including optimization techniques and multiple approaches.
- Experience with Hadoop, HDFS, Hive, and Apache ecosystem.
- Knowledge of ODS, RDB, and Data Warehousing concepts.
- Understanding of cluster mode and client mode processing.
- Ability to write and optimize SQL queries.
- Data modeling knowledge.
- Basic understanding of data architecture.
Full Job Posting
Key Requirements
- Big Data Technologies: Hadoop, HDFS, Hive, PySpark (optimization techniques, multiple approaches), Apache ecosystem knowledge.
- Data Warehousing & Storage: ODS (Operational Data Store), RDB (Relational Database concepts), Data Warehousing concepts.
- Processing & Execution Modes: Cluster mode, Client mode.
- Performance & Optimization: Performance tuning, PySpark optimization techniques.
- Data & Database Skills: SQL queries (writing and optimization), Data modeling knowledge.
- Architecture & Concepts: Basic understanding of data architecture, Data warehousing concepts.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Speridian Technologies
Golang Technical Lead
Abu Dhabi Emirate, UAE
Speridian Technologies is hiring a Golang Technical Lead to run engineering teams of 4-8 engineers. This hands-on role involves writing and reviewing code, setting technical direction, and mentoring engineers. Requires 8
Oracle Fusion Procurement Functional Consultant
Abu Dhabi Emirate, UAE
Speridian Technologies is seeking an experienced Oracle Fusion Procurement Functional Consultant with 8+ years of experience to join their team in Abu Dhabi. The role involves implementing, configuring, and optimizing Or
Golang Technical Lead
Dubai, UAE
Speridian Technologies is hiring a Golang Technical Lead to run engineering teams of 4-8 engineers. This hands-on role involves writing and reviewing code, setting technical direction, planning delivery, and mentoring en
Oracle Integration Cloud Lead
Abu Dhabi Emirate, UAE
The company is seeking an Oracle Integration Cloud Lead with around 8 years of experience to manage the OIC platform and lead integration development between Oracle Fusion Applications and other systems. The role combine
Finacle Functional Consultant
Dubai, UAE
Speridian Technologies seeks a Finacle Functional Consultant with strong Finacle (10x/11x) knowledge and banking process understanding. The role requires hands-on experience in Finacle modules, automation testing, and th
Lead Oracle Fusion Finance Functional Consultant
Abu Dhabi Emirate, UAE
Speridian Technologies seeks an experienced Lead Oracle Fusion Finance Functional Consultant to implement, configure, and optimize Oracle Fusion Financials solutions in Abu Dhabi. The role requires strong expertise in Or
Senior Full Stack Developer
Abu Dhabi, UAE
Experience Minimum 5 years of coding experience in ReactJS (TypeScript), HTML, CSS-Pre-processors, or CSS-in-JS in creating Enterprise Applications with high performance for Responsive Web Applications. Minimum 5 years
Golang Technical Lead
Abu Dhabi, UAE
Golang - Technical Lead Location : Abu Dhabi, United Arab Emirates Description The Role *We're hiring Tech Leads to run engineering teams of 4 to 8 engineers. You are the technical owner of your team's outcomes. Most of
Golang Technical Lead
Abu Dhabi Emirate, UAE
Oracle Fusion Procurement Functional Consultant
Abu Dhabi Emirate, UAE
Golang Technical Lead
Dubai, UAE
Oracle Integration Cloud Lead
Abu Dhabi Emirate, UAE
Finacle Functional Consultant
Dubai, UAE
Lead Oracle Fusion Finance Functional Consultant
Abu Dhabi Emirate, UAE
Senior Full Stack Developer
Abu Dhabi, UAE
Golang Technical Lead
Abu Dhabi, UAE