Software Engineer II (Data Engineering)

• 3+ years as a Data Engineer, including solo or primary ownership of production pipelines
• Strong Python – data engineering, transformation logic, testing discipline
• Strong SQL with ability to write correct queries, identify and refactor anti-patterns
• Databricks, Delta Lake, Airflow for production orchestration
• Experience with CDC patterns for real-time or near-real-time data synchronization
• Experience building low latency APIs serving operational or analytical consumers
• Test-driven development discipline – unit tests, integration tests, regression coverage as standard practice, not afterthought
• Operates independently under ambiguity; designs systems to be maintained, not just to run

• Kafka or equivalent event streaming platform experience
• Experience with entity matching, deduplication, or master data management
• Exposure to ML pipeline support in production
• Familiarity with NLP techniques for entity resolution or text normalization (tokenization, similarity matching, named entity recognition)

What success looks like:

• Engineers ship data products: Outputs are documented, versioned, and designed for reuse across Analytics, Data Science, and operational consumers.
• Reliability: Pipelines run reliably with minimal manual intervention.
• Performance: Data latency and downtime decrease measurably over time.
• Data Quality: Analytics and Data Science teams receive clean, trustworthy data without ad hoc fixes.
• Scalability: Infrastructure scales with volume growth without proportional cost increase.
• Reduction of Debt: Technical debt in existing pipelines decreases measurably as heuristic patches are replaced with designed logic.
• Synchronization: CDC-driven synchronization eliminates the class of cross-pipeline identity drift bugs.