Senior Inference Reliability Engineer

Identify recurring patterns across incidents, workloads, and customer escalations.

Translate those findings into improvements to the inference platform, reliability architecture, deployment processes, observability, and product roadmap.

Deep experience operating critical, customer-facing or business-critical production systems.

Ability to reason about service health across multiple layers rather than treating infrastructure availability as the complete customer outcome.

Experience defining and operating service-level indicators and objectives, building actionable observability, leading incidents, performing failure analysis, and reducing mean time to detection and recovery.

Strong understanding of latency, throughput, queueing, resource contention, capacity, workload distribution, and tail-performance behavior.

Demonstrated ability to diagnose difficult production regressions and isolate bottlenecks across applications and infrastructure.

Knowledge of distributed-systems principles, including fault tolerance, scheduling, routing, load balancing, capacity management, consistency, and failure recovery.

Strong experience with Kubernetes, Linux, networking, storage, cloud infrastructure, and containerized production environments.

Experience operating across multiple cloud providers, regions, hardware configurations, or infrastructure suppliers is especially valuable.

Ability to write production-quality software and build internal tooling, instrumentation, automation, and diagnostic systems.

Proficiency in languages such as Python, Go, Java, C++, or Rust.

Experience with GPUs, ML infrastructure, model serving, vLLM, SGLang, Triton, TensorRT-LLM, or similar technologies is valuable but not required.

You should be excited to develop expertise in concepts such as time to first token, inter-token latency, continuous batching, KV caching, speculative decoding, quantization, and tokens per GPU.

Within your first six months, you will help establish clear end-to-end ownership and observability for Parasail’s production inference workloads.

Success means:

Material endpoint regressions are increasingly detected before customers report them.

Engineers can quickly determine which layer of the system is responsible for a production issue.

Customer-impacting incidents are acknowledged, diagnosed, and resolved faster.

Production endpoints consistently meet defined availability, latency, throughput, and efficiency objectives.

Recurring failure modes are converted into automated detection, safeguards, and durable platform improvements.

Infrastructure and inference capacity are used efficiently while preserving customer performance and model quality.

This role is pivotal in building a new operational discipline for AI infrastructure.

You may come from GPU infrastructure, ML platforms, databases, streaming systems, search, low-latency services, distributed storage, or large-scale production engineering. You do not need to have previously held the title “Inference Reliability Engineer.”

What matters is that you have owned important production systems, can investigate problems that span organizational and technical boundaries, and continue following an issue until the customer experience is restored.

If you are passionate about production systems, performance, reliability, and learning the cutting edge of AI infrastructure, we are excited to welcome you aboard.