company_site
Azure PySpark Data Engineer
Tata Consultancy Services
Visakhapatnam, IND
Full-time
Senior · 5+ years experience
Onsite
Discovered 3 weeks ago
PySparkSpark SQLSparkAzure DatabricksAzureUnix
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
PySparkSpark SQLSpark
Key Skills for This Role
PySparkSpark SQLSparkAzure DatabricksAzureUnix
Full Job Posting
- Develop and maintain ETL pipelines using PySpark / Spark SQL in Azure Databricks .
- Process and transform large-scale financial data for various business and compliance applications.
- Implement data pipelines ensuring data quality, governance, and lineage tracking .
- Integrate data from different systems such as core banking, payments, and trading platforms .
- Optimize Spark jobs for high performance and cost efficiency .
- Collaborate with business teams and compliance departments to align on project requirements.
- Apply error handling, logging, and monitoring techniques during pipeline development.
- Perform advanced data transformations and develop user-defined functions (UDFs) where necessary.
- Use Unix scripting, scheduling tools , and maintain generic process development and performance tuning .
- Support audit requirements and ensure compliance with regulatory standards
About Tata Consultancy Services
IT Services & Consulting607979 employeesFounded 1968
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Tata Consultancy Services
AI Agent Developer with Snowflake Cortex experience
San Jose, USA
SeniorFull-time
Discovered TodayView →
Linux and Debian
Montréal, CAN
Discovered TodayFull-time
Data Modeller
Mississauga, CAN
Discovered TodayFull-time
Service Delivery Manager
Montréal, CAN
Discovered TodayFull-time
RPA L1 Support Lead
Toronto, CAN
Discovered TodayFull-time
AI Program Manager
New York City, USA
Discovered TodayFull-time
AI Agent Developer with Snowflake Cortex experience
San Jose, USA
Discovered TodayFull-time
Security Engineer
Austin, USA
Discovered TodayFull-time
ServiceNow Consultant
, USA
Discovered TodayFull-time