Senior Software Engineer II
Role Overview
Serve as technical owner and steward of Nielsen’s corporate Databricks Lakehouse, driving strategy, architecture, and operational maturity.
Build a secure, scalable, cost-efficient, governed, observable, and audit-ready data platform.
Develop AI-native platform capabilities and use AI-assisted and agentic engineering practices to build and operate the platform.
Key Skills for This Role
Full Job Posting
Strategic Mandate
Serve as technical owner and steward of Nielsen’s corporate Databricks Lakehouse, driving strategy, architecture, and operational maturity.
Build a secure, scalable, cost-efficient, governed, observable, and audit-ready data platform.
Develop AI-native platform capabilities and use AI-assisted and agentic engineering practices to build and operate the platform.
Core Goals and Responsibilities
- Own Lakehouse architecture, workspace topology, cluster policies, compute strategy, and multi-cloud deployment patterns across AWS, Azure, or GCP.
- Design and enforce global governance for catalogs, schemas, access controls, lineage, and audit logging through Unity Catalog.
- Set standards for Delta Lake, Delta Live Tables, medallion architecture, and performance optimization.
- Partner with CloudOps and DevOps on infrastructure as code, CI/CD, and monitoring, with a target of 99.99% platform availability.
- Own cost transparency and optimization, including cluster sizing, workload isolation, chargeback/showback, and budget alerts.
- Translate stakeholder requirements into platform capabilities and self-service patterns.
- Govern rollout of AI use cases and native AI features, including Databricks Genie, AI/BI Dashboards, and Mosaic AI, within access controls.
- Lead a developer community of practice; publish standards, deliver enablement sessions, and remove delivery friction.
Experience and Expertise
- 8–10+ years in data engineering, data platform architecture, or distributed systems, including 3+ years architecting and operating Databricks environments at scale.
- Hands-on experience building, managing, running, and scaling production-grade Databricks data platforms end to end.
- Experience with Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster/compute policy design.
- Experience architecting feature stores, vector search/indexes, and GenAI-ready pipelines, and using AI-assisted or agentic engineering practices.
- Deep hands-on Apache Spark expertise for batch and structured streaming, large-scale data processing, and performance tuning.
Technology and Platform Scope
- Databricks, Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster/compute policies.
- Terraform or Databricks Asset Bundles, CI/CD, monitoring, and multi-cloud deployment patterns across AWS, Azure, or GCP.
- AI platform capabilities include feature stores, vector search, GenAI-ready pipelines, Databricks Genie, AI/BI Dashboards, and Mosaic AI.
About Nielsen
Nielsen provides audience engagement insights to support client decisions and operates in the media measurement and analytics sector.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at The Nielsen Company
Senior Process Improvement Professional II
Mumbai, IND
Overall objective The Senior Process Improvement Professional will be a key driver in transforming our operational capabilities. This role is responsible for leading, implementing, and sustaining a culture of continuous
Senior Technical Program Manager II (Bangalore)
Bengaluru, IND
Lead governance of growth initiatives with comprehensive weekly updates on progress, providing visibility into development timelines, milestone achievements, risks and any deviations from targets. Identify and communicat
Senior Customer Success Manager
, USA
Customer Success at Nielsen As a part of our Customer Success team at Nielsen, you will play a critical role in ensuring customer satisfaction by helping our customers utilize Nielsen’s products and services to achieve t
Senior Controllership Associate II
Mumbai, IND
Role Summary Support global Record-to-Report (R2R) operations by delivering accurate, timely, and compliant financial reporting. The role goes beyond processing to enable stronger stakeholder collaboration and drive cont
Audience & Consumer Insights Specialist II
Bengaluru, IND
Senior Member Technical Staff
Bengaluru, IND
Senior Data Scientist I
Bengaluru, IND
Senior Data Analyst I - Survey Programming
Mumbai, IND
Senior Process Improvement Professional II
Mumbai, IND
Senior Technical Program Manager II (Bangalore)
Bengaluru, IND
Senior Customer Success Manager
, USA
Senior Controllership Associate II
Mumbai, IND