ashby
Runtime Engineer
MatX
USA
Full-time
Mid
Hybrid
USD 120000-475000 yearly / year
Discovered 1 weeks ago
RustCC++GoPythonPyO3
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
RustCC++
Key Skills for This Role
RustCC++GoPythonPyO3
Full Job Posting
What MatX is Building
MatX is building custom silicon for large-language-model inference and training, with HW/SW co-design across ISA, RTL, simulator, compiler, and kernels so each layer benefits from the others. The runtime owns the host-side stack and the contracts that bind those teams together.
What You'll Do Here
Build the host-side interface library — device memory management, DMA, streams and events, sync primitives — that every compiler-emitted program runs on top of
Own and extend the executable format: the compiler→runtime contract, its versioning, the weight and quantization layouts that let compiler and runtime evolve independently
Design the custom-kernel ABI — calling convention, sync semantics, lifecycle — and the host-side marshaling layer (DLPack, the buffer protocol, numpy) that gets Python tensors to the device
Build Python bindings via PyO3, with a C-ABI shim as the alternative integration path for downstream consumers
Build the LLM inference serving stack — paged KV cache, continuous batching, request scheduling, token streaming — and the cluster orchestration primitives underneath it
Bring up interconnect topology from the host and own the failure-detection and clean-teardown path for stop-restructure-resume recovery across racks
Design what the chip exposes to host-side profilers and debuggers — perf counters, traces, and the Python surfaces ML engineers actually use — and hit measurable performance targets on runtime overhead and serving throughput
Who You Are
Strong experience in a systems programming language — Rust, C, C++, or Go — including memory management, allocator design, and FFI/ABI work
Have built Python interop layers in production (PyO3, ctypes, pybind11, or equivalent C-ABI bridging)
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at MatX
System Software Engineer - Linux Kernel and Device Drivers
, USA
SeniorFull-time
Discovered 2 days agoView →
Security Engineer
, USA
Discovered TodayFull-time
Infrastructure Engineer
, USA
Discovered YesterdayFull-time
Developer Infrastructure Engineer (Silicon)
, USA
Discovered YesterdayFull-time
Mechanical Engineer - AI Rack Scale Systems
, USA
Discovered 2 days agoFull-time
System Software Engineer - Linux Kernel and Device Drivers
, USA
Discovered 2 days agoFull-time
Power Engineer
, USA
Discovered 2 days agoFull-time
Technical Recruiter
, USA
Discovered 3 days agoFull-time
Technical Program Manager, Silicon/Hardware
, USA
Discovered 3 days agoFull-time