Associate Director, Data Engineering

Proactively surface strategic topics to senior leadership based on first-hand data engineering experience — platform constraints, technical debt, build-vs-buy trade-offs, and emerging AI/engineering tooling — rather than waiting to be asked.

Define the data strategy and AI roadmap for a Databricks-oriented data engineering team, identifying the data products, lakehouse assets, and AI/ML capabilities that matter most over the next 12–24 months.

Own the product roadmap for Databricks-native capabilities — lakehouse architecture, Unity Catalog governance, Delta Live Tables pipelines, MLflow/feature-store patterns— with hands-on proof points, not vendor-slide recommendations.

Own sprint- and quarter-level backlog prioritization applying product-management rigor (RICE/WSJF-style scoring, stakeholder discovery, success metrics) to balance business value, technical debt, effort, and risk.

Bring forward strategic topics — informed by direct engineering experience — that shape the roadmap rather than simply respond to it (e.g., platform limitations, emerging AI capabilities).

Facilitate PI Planning, backlog grooming, and sprint reviews; serve as the voice of both the internal customer and the platform in delivery ceremonies.

Own the data platform as a product — define its roadmap, SLAs/OLAs, and reliability targets, treating internal engineering teams and data consumers as its customers.

Manage platform capacity, cost, and performance (compute/storage sizing, FinOps discipline, cost-per-workload tracking) to keep the platform scalable and cost-efficient.

Standardize and rationalize the tooling landscape — reduce redundant components and own the platform's build-vs-buy and vendor evaluations.

Own lifecycle management of platform components — versioning, deprecation, and migration planning — so platform evolution doesn't disrupt delivery teams.

Define and implement the data KPI and metrics framework for the organization — data quality, pipeline reliability, freshness/latency, lineage coverage, and cost-to-serve — with clear owners and targets.

Instrument dashboards and reporting (e.g., in Databricks AI/BI or equivalent) so KPIs are visible to engineering teams and business stakeholders, not buried in a backlog tool.

Tie KPIs to business outcomes and review trends regularly with the team and stakeholders, using them to drive continuous improvement, not static reporting.

Champion and enable AI adoption across data teams enterprise-wide, not only within owned pods — building the playbooks, training, and support that let other teams adopt AI-assisted engineering with confidence.

Stand up a community of practice for AI-assisted and agentic engineering across data teams, sharing reusable patterns, guardrails, and lessons learned.

Track engineering efficiency and AI-adoption metrics (cycle time, automation coverage, tool adoption rate) and prioritize backlog items that improve them.

Prioritize data product delivery work that reduces time-to-data and time-to-insight for downstream consumers.

Own backlog items that advance Data Capability Maturity — data contracts, self-service access, and reusable data products at scale.

Redesign and own the data onboarding pipeline so new sources are cataloged, quality-checked, and available as trusted data products in days, not months.

Streamline data access provisioning — self-service, role/attribute-based requests (e.g., via Unity Catalog), and automated approvals — to cut time-to-access.

Track onboarding and access-acceleration metrics (time-to-onboard, time-to-access) and prioritize work that measurably shortens them.

Directly manage a team of data engineers and/or product owners, including hiring, onboarding, and performance appraisals.

Coach team members on product thinking and platform ownership, and serve as their career advisor, partnering with HR on development and IDPs.

Partner with architecture, test engineering, and business/product leaders to align priorities and dependencies; represent the team in PI Planning.

Communicate roadmap, platform trade-offs, and priorities clearly to both technical and business audiences.

Implement and refine a standardized framework for backlog management, platform governance, and delivery execution (Agile/Kanban).

Contribute to a growing community of practice among data product owners and platform leads; support Lilly's evolution toward a federated data mesh model.

Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field; advanced degree a plus.

10+ years of relevant experience in data/software engineering, delivery management, product ownership, or platform management, including 4+ years of people leadership.

Demonstrated, current hands-on data engineering practice, alongside a history of contributing to or shaping technology/platform strategy based on that direct experience.

Lilly is dedicated to helping individuals with disabilities to actively engage in the workforce, ensuring equal opportunities when vying for positions. If you require accommodation to submit a resume for a position at Lilly, please complete the accommodation request form ( https://careers.lilly.com/us/en/workplace-accommodation ) for further assistance. Please note this is for individuals to request an accommodation as part of the application process and any other correspondence will not receive a response.

Lilly does not discriminate on the basis of age, race, color, religion, gender, sexual orientation, gender identity, gender expression, national origin, protected veteran status, disability or any other legally protected status.