Senior Data Engineer - Full Stack

Role overview

Responsibilities

• Partner with business and technical stakeholders to clarify objectives, translate ambiguous requirements into technical solutions, and establish delivery plans.
• Design, develop, and maintain end-to-end data products using Databricks, Delta Lake, SQL, Python, PySpark, and related technologies.
• Build batch, incremental, streaming, and near-real-time pipelines using Kafka or comparable event-streaming technologies.
• Design event-driven architectures and integrate operational systems with downstream data consumers.
• Develop backend services, REST APIs, and integrations that expose governed data to applications and operational workflows.
• Build lightweight applications, dashboards, and user interfaces with product, analytics, BI, and UX teams.
• Prototype solutions, validate concepts with stakeholders, and transition successful prototypes into scalable production capabilities.
• Create scalable data models and curated datasets for analytics, reporting, AI/ML initiatives, and operational decision-making.
• Implement data-quality, security, lineage, and governance controls using Databricks, Unity Catalog, and comparable technologies.
• Establish automated testing, CI/CD, monitoring, alerting, documentation, and deployment practices across the data-product lifecycle.
• Optimize and troubleshoot pipelines, queries, streaming workloads, APIs, applications, and downstream consumers.
• Lead technical design and code reviews, mentor engineers, and contribute to stronger data-engineering standards.

Requirements

• At least 5 years of experience in data engineering, software engineering, or a related discipline, including ownership of production data solutions.
• Strong proficiency in SQL, Python, and PySpark with production data pipeline experience.
• Hands-on experience with Databricks, Apache Spark, Delta Lake, Unity Catalog, or comparable technologies.
• Experience with streaming or near-real-time pipelines using Kafka, Kinesis, Event Hubs, or similar technologies.
• Strong understanding of event-driven architecture, message processing, schema evolution, data consistency, and streaming reliability.
• Experience delivering full-stack solutions across data pipelines, backend services or APIs, and lightweight applications.
• Experience developing REST APIs, services, and integrations, ideally with FastAPI, Flask, or comparable tools.
• Experience with AWS, Azure, or GCP and cloud-native architecture patterns.
• Strong knowledge of data modeling, data warehousing, distributed processing, and analytics-oriented data design.
• Experience with Git, automated testing, CI/CD, monitoring, and production deployment practices.
• Strong stakeholder communication, analytical, problem-solving, technical leadership, and end-to-end ownership skills.
• React, infrastructure as code, containerization, AI/ML pipelines, data observability, or mentoring experience is advantageous.

Benefits

• Generous time-off policies and comprehensive wellbeing and professional benefits.
• Education and learning support plus wellness and lifestyle resources.
• Opportunity to work with modern data and cloud technologies.
• Opportunities for technical leadership, mentoring, innovation, and professional growth.

Workplace

• Remote work within India.