Digital Cloud & Data Platform Sr. Engineer (Pune, MH, IN)

Key Responsibilities

• 1. Platform Architecture & Roadmap
• Own the technical vision and evolution of the enterprise data platform aligned to business and digital strategy.
• Define the target architecture for real-time and batch data processing, including Medallion architecture patterns and event-driven pipelines.
• Translate platform strategy into quarterly technical objectives, capacity plans, and measurable engineering outcomes.
• 2. Core Data Platform Capabilities
• Design and operate the enterprise data storage using Amazon S3 and Apache Iceberg.
• Implement and mature Bronze–Silver–Gold (Medallion) architecture for structured, governed data processing.
• Define and maintain reusable building blocks for ingestion, transformation, storage, and serving.
• Establish reference implementations for streaming (Lambda/Flink) and batch ETL (Spark/Flink).
• 3. Enablement & Data Engineering Experience
• Provide standardized pipelines, templates, and patterns to enable analytics, applications, and ML teams to consume data efficiently.
• Enable self-service access to trusted datasets through Athena/Trino, APIs, and ML pipelines.
• Partner with application, edge, and ML teams to ensure seamless integration from telemetry to business-ready data.
• 4. Reliability, Security & Compliance
• Enforce enterprise-grade security and governance using Lake Formation, IAM, and KMS.
• Implement data quality, validation, lineage, and observability controls across pipelines.
• Design for resilience through robust error handling, reprocessing, and backfill mechanisms.
• Establish and maintain automated unit testing for data transformations, schemas, and business logic.
• Support System Integration Testing (SIT) and User Acceptance Testing (UAT).
• 5. Performance, Cost & Scalability
• Design and Optimize storage, compute, and query performance for cost efficiency and predictable scaling.
• Monitor and continuously improve platform throughput, latency, and unit economics (e.g., cost per GB, cost per pipeline).
• 6. Cross-Functional Leadership & Stakeholder Engagement
• Work closely with edge, application, analytics, and AI teams to translate raw telemetry into trusted business insights.
• Partner with architecture, security, and cloud teams to align platform implementation with enterprise standards.
• Act as a technical advisor to product and business stakeholders on data platform capabilities and constraints.

Preferred Qualifications

• 8–12+ years of experience in cloud, data engineering, or platform engineering roles, with a proven track record of building and operating scalable data platforms used by multiple teams.
• Deep familiarity with Medallion architectures, streaming pipelines, and cloud-native data services, with the ability to make pragmatic design and technology choices.
• Demonstrable expertise in real-time and batch data processing using technologies such as Apache Spark, Apache Flink, and event-driven architectures.
• Strong knowledge of data reliability engineering, including data quality, schema evolution, observability, error handling, and reprocessing at scale.
• Advanced proficiency in SQL and analytical query engines (Athena / Trino or equivalent) for large-scale data analysis.
• Hands-on experience with Amazon S3 and modern table formats such as Apache Iceberg (or Delta Lake).
• Solid understanding of cloud security and data governance, including IAM, KMS, and policy-based access control.
• Experience with workflow orchestration and automation using tools such as Airflow or Step Functions.
• Exposure to industrial IoT, or time-series data environments is highly desirable.
• Strong ability to collaborate with application, edge, analytics, and AI teams to translate raw data into business-ready insights.
• Demonstrated capability to operate in complex, multi-team environments with an emphasis on standardization, reuse, and continuous improvement.
• Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related field, or equivalent practical experience.

Minimum Qualifications

• 10+ years in Agile delivery roles (Scrum Master/Agile Coach/Process Lead) supporting software product teams; proven success implementing Agile at team-of-teams or portfolio scale.
• Bachelor’s in Computer Science/Engineering or equivalent experience.

Certifications

Work Environment/Physical Demands

Job Grade