Senior Data Engineer

• Design, build, and maintain scalable, reliable data pipelines and ETL/ELT processes.

• Develop data engineering solutions using Python, SQL, and Databricks.

• Build and optimize data pipelines within the Databricks Lakehouse environment.

• Work with large, complex datasets from multiple sources, including enterprise applications, databases, APIs, and other data platforms.

• Partner with engineering teams to extract, integrate, and modernize data flows from legacy systems, including IBMi/AS400 environments.

• Contribute to databricks implementation project and support data platform readiness for large scale data conversion, including data mapping, migration, and integration work.

• Design and implement change data capture (CDC) patterns to support real-time and incremental data integration.

• Develop and maintain data models and curate datasets for analytics, reporting, and downstream applications.

• Implement data quality, validation, monitoring, and error-handling processes.

• Optimize data pipelines and queries for performance, scalability, reliability, and cost.

• Partner with data architects, analysts, data scientists, application teams, and business stakeholders to deliver high-quality data solutions.

• Collaborate with the AI governance group to prepare and structure data for AI/ML pilot programs, including initiatives built on Azure AI Foundry.

• Participate in the design and evolution of MMG's modern data architecture and engineering practices.

• Establish and promote standards for code quality, testing, documentation, version control, and deployment.

• Troubleshoot complex data and pipeline issues and provide sustainable solutions.

• Contribute to CI/CD and automated deployment practices for data engineering workloads.

• Mentor other engineers and contribute to the growth of MMG's data engineering capabilities.

• Experience establishing version control, testing, review and deployment automation for data

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent professional experience.

• 5+ years of professional experience in data engineering, software engineering, or a related technical discipline.

• Strong professional experience with Python and the ability to write clean, maintainable, production-quality code.

• Strong experience with Databricks, including developing and optimizing data pipelines and workloads.

• Strong SQL skills and experience working with relational and/or analytical databases.

• Experience designing and implementing ETL/ELT pipelines and data integration solutions.

• Experience working with cloud-based data platforms and modern data architectures (Azure preferred).

• Strong understanding of data modeling, data warehousing, and data lake/lakehouse concepts including medallion / multi-hop architectures.

• Experience with Git and modern software development practices.

• Experience with automated testing, deployment, and CI/CD practices.

• Strong analytical and problem-solving skills.

• Ability to communicate effectively with both technical and non-technical stakeholders.

• Experience working in the insurance or financial services industry.

• Experience integrating data from legacy systems, particularly IBMi/AS400 environments.

• Experience with Azure AI Foundry or similar AI/ML platform tooling.

• Experience with Apache Spark / PySpark.

• Experience with Databricks SQL and Delta Lake.

• Experience with data orchestration tools such as Azure Data Factory, Airflow, or similar technologies.

• Experience with APIs and event-driven or real-time data integration.

• Experience implementing change data capture (CDC) patterns for enterprise data integration.

• Experience with insurance policy administration platform data or large-scale platform data conversions.

• Experience with data governance, metadata, lineage, and data quality frameworks.

• Experience working with enterprise data warehouses and dimensional modeling.

• Experience with Infrastructure as Code and cloud automation.

• Experience using AI-assisted development tools to improve engineering productivity.

• Build reliable, scalable, and maintainable data pipelines that the organization can trust.

• Help expand MMG's use of Databricks and modern data engineering practices.

• Improve the availability, quality, and usability of enterprise data.

• Replace manual and repetitive data processes with automated, monitored pipelines, reducing the operational effort spent correcting and reconciling data by hand.

• Deliver data solutions that meet both technical requirements and business needs.

• Help establish consistent engineering standards across the data organization.

• Serve as a technical leader and mentor for other data engineers.