Senior Data Engineer

• Develops and maintains production Delta Lake pipelines, evolving ingestion from batch watermark patterns to event-driven architectures using Zerobus Ingestion and Spark Declarative Pipelines for high-SLA source systems.
• Implements row-level reconciliation, validation checkpoints, and quality gates at ingestion, transformation, and delivery layers, operationalizing governance-defined data quality dimensions in partnership with QA.
• Builds Gold-layer tables and promotion logic that support Unity Catalog metric views, lineage capture, and access controls in partnership with Analytics Engineering and Data Governance.
• Optimizes pipeline performance, latency, and compute cost across all layers of the platform.

• Documents architecture decisions, data models, pipeline logic, and operational runbooks to team standards, ensuring no critical platform capability depends on a single point of failure.
• Operates pipeline observability tooling: monitors anomaly detection, triages data reliability incidents, and drives root cause remediation for assigned domains.
• Implements data contract validation (ODCS or equivalent) in pipelines supporting external partner feed domains, ensuring contract failures halt delivery before reaching consumers.

• Mentors Data Engineers on Databricks development patterns, SQL and PySpark craft, and documentation discipline; reviews pull requests in Azure DevOps & Git to maintain code quality.
• Partners with Data Governance on lineage capture and metadata standards, with QA on Tier 1 and Tier 2 pipeline test suite development, and with Partner Data Services on feed engineering requiring pipeline work.
• Works from structured requirements and acceptance criteria entering through the Informatics intake process, and flags requirements gaps before build work begins.

• 6+ years of progressive data engineering experience, including production ownership of an enterprise data warehouse or large-scale transformation pipelines with defined SLAs.
• Deep, hands-on expertise with Databricks in production environments: Delta Lake, medallion architecture, Unity Catalog, and pipeline performance optimization.
• Demonstrated data warehousing depth: dimensional and harmonized data modeling, conformance and survivorship logic, and operating a warehouse as a production system.
• Advanced SQL and strong PySpark and Python proficiency, sufficient to build, review, and optimize complex transformation logic independently.
• Proven experience with Azure data services: Azure Databricks, Azure Data Lake Storage, Azure Data Factory, and CI/CD practices in Azure DevOps.
• Demonstrated ability to absorb complex, under-documented systems through structured knowledge transfer and produce documentation that makes that knowledge durable and transferable.
• Strong communication and collaboration skills; able to work directly with QA, Governance, Informatics, and business-facing teams without an intermediary.

• Healthcare, specialty pharmacy, or regulated industry experience, with exposure to clinical or operational data environments.
• Production experience with event-driven ingestion: Azure Event Hubs, Kafka, or equivalent, consumed via Structured Streaming.
• Familiarity with data contract standards (ODCS or equivalent) and pipeline observability platforms (Monte Carlo or similar).
• Familiarity with enterprise data catalog tooling (Atlan, Collibra, or equivalent) and designing pipelines as catalogued, governed assets.
• Experience leading through influence: mentoring engineers, setting standards, and driving adoption without formal management authority.
• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field, or equivalent experience.

This position works in a home office and professional office environment.

When in-office this role routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets and fax machines, and communications via MS Teams.

While performing the duties of this job, the employee is regularly required to sit, see, talk or hear.

The employee frequently is required to stand; walk; use hands and fingers to handle or feel; and reach with hands and arms.

Visual acuity is necessary for tasks such as reading and working with various forms of data on a screen.

Reasonable accommodation may be made to enable individuals with disabilities to perform

Hybrid, remote and flexible on-site work schedules are available, based on the position.

PANTHERx Rare Pharmacy also affords an excellent benefit package, including but not limited to medical, dental, vision, health savings and flexible spending accounts, 401K with employer matching, employer-paid life insurance and short/long term disability coverage, and an Employee Assistance Program!

Generous paid time off is also available to all full-time employees.

Of course we offer paid holidays too!

PANTHERx Rare Pharmacy is an equal opportunity employer, and does not discriminate in recruiting, hiring, promotions or any term or condition of employment based on race, age, religion, gender, ethnicity, sexual orientation, gender identity, disability, protected veteran's status, or any other characteristic protected by federal, state or local laws.