Data Engineer (Software Engineer II)

• Executes standard software solution, design, development, and technical troubleshooting. Manage data integration and data analysis of disparate systems
• Build extensible data acquisition and integration solutions to meet the functional and non-functional requirements of the client. Implement processes and logic to extract, transform, and distribute data across one or more data stores from a wide variety of sources
• Provide problem-solving expertise and complex analysis of data to develop business intelligence integration designs Interface with other internal product development teams as well as cross functional teams (Product Management, Integration Engineering, Quality Engineering, System Admin Teams)
• Working with remote and geographically distributed teams to enable building the right products, using the right building blocks, and making them consumable by other products easily
• Leverages enterprise-authorized AI coding assist tools within the work environment to improve code quality, delivery speed, and productivity (e.g., code generation/refactoring, unit test creation, documentation), while validating outputs through peer review, automated testing, and secure coding standards.
• Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation
• Gathers, analyzes, and draws conclusions from large, diverse data sets to identify problems and contribute to decision-making in service of secure, stable application development
• 4+ years experience in data integration projects using Big Data Technologies.

in AWS Data Warehousing and database platforms with hands-on delivery using EMR, S3, AWS Glue, Lambda, Apache Airflow and Infrastructure as Code (IaC) (e.g., Terraform/CloudFormation)

• Strong experience in CICD using Jenkins, Git, Artifactory, Yaml, Maven for Cloud deployments
• Hands on experience on Spark engineering with PySpark/Scala for building and optimizing scalable data pipelines. Good knowledge of Big Data querying tools, such as Athena, RDS, Databricks SQL Warehouse.