Manage, mentor, and inspire a team of skilled data engineers to deliver high-quality work
Provide direction, performance feedback, and career development opportunities
Foster a culture of collaboration, innovation, and continuous learning
Manage the design and implementation of scalable and efficient data pipelines on Databricks
Define and implement data architecture to support both batch and real-time data processing
Leverage Databricks features (e.g., Delta Lake, Databricks SQL, MLflow) to optimize infrastructure for performance and cost
Own, administer, and continuously optimize the end-to-end Databricks environment, including workspace administration, cluster and compute management, Unity Catalog governance, and cost/performance monitoring
Build and operationalize Databricks AI capabilities (Mosaic AI, model serving, and generative AI/agent workflows) to power intelligent, data-driven products
Design, deploy, and maintain AI/BI Genie spaces and Genie-powered apps that enable business users to explore data and get answers through natural-language conversation
Implement and maintain data governance frameworks to ensure data integrity, security, and compliance
Collaboration and Business Impact:
Partner with data scientists, analysts, and business teams to enable data-driven decision-making
Work closely with business and product management to define, prioritize, and own both the technical and business data roadmap, aligning platform investments with organizational strategy
Translate business needs into technical requirements and scalable data solutions
Communicate technical plans, challenges, and achievements to stakeholders at all levels
Prioritize and manage multiple data engineering projects, ensuring timely delivery
Define success metrics and continuously track progress to achieve desired outcomes
Stay informed on industry trends, particularly advancements in Databricks, Databricks AI (Mosaic AI, AI/BI Genie), generative AI, and cloud-based data platforms.
Establish and enforce best practices for coding, testing, and deploying data engineering solutions
Proven experience running projects and managing direct reports
Knowledge of data governance practices, business and technology issues related to managing enterprise information assets, and approaches related to data protection
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career