Sr. Compiler Engineer

Senior Compiler Engineer Mountain View, CA About DataPelago:

DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing.

We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data.

Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing.

We are looking for specialists to join our engineering team and shape the future of accelerated data processing.

The Opportunity: As a Senior Compiler Engineer, you will lead the advancement of compilers for DataPelago’s data processing engine.

You will enhance functional breadth, performance, scale, and reliability of DataPelago engine’s compilers in executing data processing workloads on diverse acceleration hardware infrastructure, including CPUs, GPUs.

And FPGAs.

This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.

• Architect: Lead the architecture of how our data processing engine will translate data pro- cessing logical and physical plans into efficient code for execution on CPU/GPU/FPGA.
• Design: Lead design of functional and performance enhancements to the compilers in our data processing engine.
• Core Development: Individually design, implement, test, optimize, and maintain compo- nents of compilers for the data processing engine.
• Innovation and Differentiation: Analyze advances in the capabilities of CPU and GPU pro- cessing elements from Nvidia, AMD, Intel, and others, compiler platforms and tools and

identify opportunities for our engine to enhance technology and product leadership.

• Collaboration: Partner effectively with engineering and product management in defining language specifications of our product and ensuring compliance with industry standards, platform definitions, and workload requirements.