Senior Apache Spark Technical Lead - Scala, Python (105721)

Skill Requirements

• 8+ years of Engineering experience primarily in Data Engineering in an agile production environment • High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Impala, Oozie, Airflow, NiFi, Scoop), SQL to build Big Data products & platforms • Extensive experience with Databricks, Spark , PySpark or Java + PySprak Processing engine. • Proficiency in, at least, one modern programming language such as Python, Java or Scala • Strong Computer Science fundamentals in object-oriented design, data structures, algorithm design, problem solving, and complexity analysis. • Ability to easily move between business, data management, and technical teams; ability to quickly intuit the business use case and identify technical solutions to enable it • Working Knowledge in Software Development engineering Paradigms along with Data Engineering. • Relational Databases as well as NoSQL experience • cloud technologies like Databricks/AWS/Azure ( Good to have) • Basic Shell scripting and knowledge of Linux/Unix systems • Experience in designing & developing software at scale • Strong written and verbal English communication skills. • Motivation, creativity, self-direction, and desire to thrive on small project teams • Superior academic record with a degree in Computer Science or related technical field