System Software Engineer

System Software Engineer Hyderabad, IN / Remote About DataPelago:

DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing.

We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data.

Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing.

We are looking for specialists to join our engineering team and shape the future of accelerated data processing.

The Opportunity: As a System Software Engineer, you will be a key individual contributor in advancing operating

system level capabilities underlying DataPelago’s data processing engine.

You will enhance functional breadth, performance, scale, and reliability of these OS-level services on diverse accelerated hardware platforms.

This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.

• Architect: Influence the architecture of how our data processing engine harnesses and man- ages the use of compute, memory, storage, and network resources.
• Design: Lead design of functional and performance enhancements to OS-level capabilities

underlying our data processing engine such as concurrent event processing, memory man- agement, and inter-process communication.

• Core Development: Individually design, implement, test, optimize, and maintain compo- nents of the data processing engine.
• Innovation and Differentiation: Analyze advances in system services provided by platforms and programming models for high-concurrency data processing on CPUs and GPUs (e.g., Rust, CUDA runtime) and identify opportunities for our engine to enhance technology and product leadership.
• Collaboration: Partner effectively with the execution engine engineering team in enhancing system software capabilities.