company_site
Senior Data Engineer
TaskUs
Remote, IND
Full-time
Senior Level
Remote
Discovered Today
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanFull Job Posting
- Pipeline Engineering: Build and maintain high-throughput ETL/ELT pipelines that ingest data from various sources into our Lakehouse.
- Code Quality & Tooling: Drive the adoption of dbt for transformation and PySpark for heavy lifting. You will be responsible for writing modular, reusable code that follows strict CI/CD practices.
- Observability & Reliability: Implement "Data SLAs." You will build the monitoring and alerting systems that notify the team of data drift or pipeline failures before the business notices.
- Data Productization: Work closely with the BI team and Data Scientists to prepare "feature-ready" datasets.
- Performance Tuning: Deep-dive into SQL and Spark query plans to optimize slow-running jobs, reducing cost and latency.
- Local Development Advocacy: Implementing a workflow where engineers can develop and test complex SQL transformations locally using DuckDB, drastically reducing "waiting-for-cluster" time and lowering development costs.
- Cost-Efficient Micro-Pipelines: Building specialized pipelines for specific client reporting or data-quality checks that run on single-node containers (e.g., Lambda or small ECS tasks) using DuckDB, avoiding the minimum billing cycles of larger warehouses.
- Embedded Analytics: Exploring ways to use DuckDB as an embedded engine for internal tools or "edge" processing within the BPO's local site offices where bandwidth to the cloud might be a constraint.
- 5+ Years in Data Engineering: You've lived through the "on-call" life and know how to build systems that don't break at 3 AM.
- The Power Trio: Expert-level proficiency in Python , SQL , and PySpark .
- Modern Lakehouse Stack: Hands-on experience with Databricks (Delta Lake) or Snowflake . You understand the nuances of the Medallion Architecture (Bronze/Silver/Gold).
- Transformation & Modeling: Advanced experience with dbt (Data Build Tool) . You treat data models like software, including version control, testing, and documentation.
- Orchestration: Experience with Apache Airflow or Prefect, specifically building complex, idempotent DAGs.
- Streaming Experience: Familiarity with Kafka , Kinesis , or Spark Streaming is a huge plus-our BPO operations move in real-time.
- Infrastructure as Code (IaC): Comfort with Terraform or CloudFormation to manage your own data infrastructure.
- In-Process Analytics (DuckDB): Proven experience using DuckDB for high-speed local development, unit testing data transformations, or as a query engine for "small-to-medium" datasets (up to 100GB) without the overhead of a distributed cluster.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at TaskUs
Team Leader
, IND
LeadFull-time
TaskUs is seeking a Team Leader to manage day-to-day team operations and deliver strong customer experiences. The role focuses on coaching teammates, meeting SLAs and KPIs, maintaining schedules, coordinating with other
Discovered TodayView →
Turn Your Daily Chores Into Cash | Earn Up to USD $1,400 | Headmounts 100% Reimbursed
, USA
EntryTemporary
Discovered 1 weeks agoView →
Turn Your Conversations Into Cash - No Scripts, Just Your Voice
, USA
EntryContract
Discovered 1 weeks agoView →
Team Leader
, IND
Discovered TodayFull-time
Associate Data Engineer
Mumbai, IND
Discovered YesterdayFull-time
Team Lead
Navi Mumbai, IND
Discovered YesterdayFull-time
Team Leader
Mumbai, IND
Discovered YesterdayFull-time
Senior Data Engineer
, IND
Discovered 2 days agoFull-time
Turn Your Daily Chores Into Cash | Earn Up to USD $1,400 | Headmounts 100% Reimbursed
, USA
Discovered 1 weeks agoTemporary
Turn Your Conversations Into Cash - No Scripts, Just Your Voice
, USA
Discovered 1 weeks agoContract
Investigator Analyst
San Antonio, USA
Discovered 1 weeks agoFull-time