Asst Dir-QA Engineer - Databricks & Python

Skills and Competencies

• 6+ years of experience in Software Quality Assurance or Data Quality Engineering, including automation-first testing for complex enterprise data platforms
• Strong hands-on experience with Databricks SQL and Python, including testing of notebooks, workflows, pipelines, and scheduled jobs
• Solid understanding of Medallion architecture (Bronze, Silver, Gold) and data lifecycle validation practices across enterprise data ecosystems
• Expertise in PySpark for data validation, test automation, and building scalable, reusable testing frameworks
• Strong proficiency in SQL and data validation techniques across dataset, record, schema, transformation, and source-to-target mapping levels
• Experience working in Agile or Scrum environments with tools such as Xray for test management and Jira for defect tracking
• Proven experience leading quality assurance initiatives, mentoring team members, driving quality governance, managing stakeholder expectations, and influencing delivery decisions
• Demonstrated proficiency in artificial intelligence concepts, with hands-on experience using AI tools to streamline workflows and enhance operational efficiency. Proven ability to implement AI-powered solutions to solve business challenges. Demonstrates a growing awareness of AI risk management and a commitment to responsible and ethical AI use.

Education

Responsibilities

• Ensure data quality, reliability, and correctness across large-scale data platform pipelines through automated validation and testing practices
• Lead end-to-end data quality validation across Bronze, Silver, and Gold data layers, establishing testing standards and quality gates across projects
• Define and drive the overall quality assurance strategy, test approach, automation roadmap, and release readiness criteria for enterprise data platform initiatives