Data Engineer

Description & Requirements

• Top Skills Details
• 1) Strong with SQL (Hive tables) 2) Python experience (will be asked simple algorithm questions in interview) 3) Data Modeling / ETL experience
• Apply proven expertise and build high-performance, scalable data warehouse application Securely source external data from numerous global partners Intelligently design data models for optimal storage and retrieval Deploy inclusive data quality checks to ensure high quality of data Optimize existing pipelines and implement new ones, maintenance of all domain-related data pipelines Ownership of the end-to-end data engineering component of the solution Collaboration with the program’s SMEs, data scientists Skills: Proficiency in Big Data stack environments (Hadoop, MapReduce, Hive) competence with relational databases (Oracle, MySQL, Vertica) experience working with enterprise DE tools (Airflow), ability to learn in-house DE tools coding and scripting experience with Python, Java, PHP, SQL, CLI Duties: Knowledge of algorithms, data structures, and performance optimism and experience with processing and interpreting data sets. Responsible for developing and translating computer algorithms into prototype code and maintaining, organizing, and identifying trends in large data sets. Proficiency in SQL database design and creating process documentation. Familiarity with the computer coding languages python, java, kafka, hive, or storm to oversee real-time business metric aggregation, data warehousing and querying, schema and data management, and related duties. Develop technical solutions to improve access to data and data usage. Understand data needs and advise company on technological resources. Aggregate and analyze various data sets to provide actionable insight. Develop reports, dashboards, and tools for business-users.