Sr. Software Engineer - Databricks Job

Pune, IN

YASH Technologies is a leading technology integrator specializing in helping clients reimagine operating models, enhance competitiveness, optimize costs, foster exceptional stakeholder experiences, and drive business transformation.

At YASH, we’re a cluster of the brightest stars working with cutting-edge technologies. Our purpose is anchored in a single truth – bringing real positive changes in an increasingly virtual world and it drives us beyond generational gaps and disruptions of the future.

Data Engineer -- Databricks, Python, ETL, Postgres

Experience Level 4 -6 Years

Job Description:

Strong expertise in SQL , including stored procedures, functions, indexes, query optimization, and performance tuning. Hands-on experience with PySpark, Python, and Databricks for designing and developing scalable data solutions. Strong experience in designing, developing, and supporting ETL/ELT pipelines for data ingestion, transformation, and integration. Experience working with AWS cloud data services such as RDS, S3, and Glue , and building end-to-end data pipelines. Strong experience with PostgreSQL , including database design, query development, performance tuning, data modeling, and database administration concepts. Experience in data migration , data integration, organization-wide reporting, and building data products. Good understanding of Databricks workflow orchestration, pipeline execution, monitoring, and optimization . Experience in data cleansing, data validation, reconciliation, and data quality management . Strong analytical and problem-solving skills with the ability to organize, analyze, and interpret large datasets. Good understanding of data warehousing concepts and best practices. Knowledge of Scala, Git, and CI/CD practices for code versioning and deployment automation. Strong verbal, written, and interpersonal communication skills. Organized and structured approach with the ability to work independently and collaboratively in a team environment. Migration & Modernization Responsibilities Analyze key mainframe jobs and identify DB2 database components requiring migration. Identify and assess DB2 schemas, tables, and dependent objects for migration. Lead the decommissioning of DB2 schemas as part of application and database modernization initiatives. Plan and execute migration of identified DB2 schemas and data to AWS PostgreSQL . Perform schema conversion, data validation, reconciliation, and post-migration testing. Migrate identified schemas/data to AMS while ensuring data integrity and business continuity. Collaborate with cross-functional teams to support migration planning, cutover activities, and production deployment. Ensure compliance with data quality, governance, and security standards throughout the migration lifecycle. Primary Skills Databricks, PySpark, Python, SQL, PostgreSQL, ETL/ELT, AWS (S3, RDS, Glue), Data Migration Secondary Skills Scala, Git, CI/CD, DB2 Migration, Mainframe Modernization, Data Quality, Data Validation & Reconciliation