Fresher - Big Data Engineer

Requirements

• Remote position (work-from-anywhere)
• Immediate joiners must apply
• Data Engineering Experienced - course/competitions/internships/job (<2 years)
• Competitive compensation
• 1. Code in Python3 - Numpy?
• 2.Code in Python3 - Pandas?
• 3.Developed data engineering pipelines on real-world problem (not just academic projects)?
• 4.Implemented advanced SQL queries
• 5.Developed complex logics in Python3 6.Confidence to learn PySpark3 within a month? https://spark.apache.org/docs/latest/api/python/getting_started/index.html (we shall guide but won't spoon-feed)
• Confidence to learn Kafka3 within a month? https://kafka.apache.org/documentation (we shall guide but won't spoon-feed)
• We are offering one of the most challenging & exciting work on Data Pipelines. You shall be working on sophisticated platforms, products and applications
• ===========================================
• We are looking for developer with real passion for data ingestion, data transformation and data management. This is a specialist and individual contributor role. Product development experience preferably at a startup or a lean team is desired
• ROLE
• Mandatory
• 1. Building data acquisition pipelines ingesting data from various source systems (databases, flat files, software, APIs)
• 2. Building data munging pipelines transforming data format, shape and value
• 3. Must be able to convert, break, & distribute existing Python codes to functional programming syntax
• 4. Must be able to execute data structures, linear algebra and algorithms implementation at scale on parallel/distributed clusters
• 5. Must be able to recognize code that is more parallel, and less memory constrained, and you must show how to apply best practices to avoid runtime issues and performance bottlenecks
• Preferred
• 1. Functional programming in Python on vinaigrette map-reduce lambda paradigm
• 2. Knowledge of first-class, high order, & pure functions, recurisons, lazy evaluations, and immutable data structures