MLOps / ML Platform Engineer (LLM & Streaming Infra)

Overview

Ml & Genai Platform Engineering

• Design, develop, and maintain scalable infrastructure for machine learning and Generative AI workloads.
• Build and optimize LLM infrastructure for training, fine-tuning, inference, and deployment.
• Support the deployment and orchestration of AI models across cloud and on-premise environments.
• Implement GPU-enabled infrastructure and workload optimization for high-performance AI applications.
• Develop reusable tooling, frameworks, and automation to accelerate ML experimentation and productionization.

Chat & Conversational Ai Systems

• Design and support chat-based AI architectures, including conversational workflows, orchestration layers, memory management, and retrieval pipelines.
• Build infrastructure supporting AI assistants, copilots, and real-time conversational applications.
• Integrate vector databases, embeddings pipelines, and Retrieval-Augmented Generation (RAG) systems.
• Support prompt management, model routing, observability, and evaluation frameworks for LLM applications.

Streaming & Devops Engineering

• Build and maintain DevOps pipelines for real-time streaming applications and event-driven systems.
• Manage CI/CD workflows for machine learning and distributed streaming services.
• Design resilient infrastructure for low-latency, high-throughput data processing workloads.
• Implement infrastructure-as-code, monitoring, logging, and automated deployment strategies.
• Ensure platform reliability, scalability, security, and operational excellence across environments.

Collaboration & Platform Enablement

• Partner with ML Engineers, Data Scientists, Software Engineers, and Product teams to deliver scalable AI solutions.
• Establish best practices for MLOps, platform governance, security, and infrastructure reliability.
• Drive platform standardization, automation, and developer experience improvements.
• Support troubleshooting and performance optimization across AI and streaming systems.

Required Experience

• Strong experience building and maintaining ML platforms or AI infrastructure in production environments.
• Hands-on experience with chat-based AI systems and conversational application architecture.
• Proven DevOps experience supporting streaming or real-time applications.
• Experience deploying and managing LLM/Generative AI infrastructure at scale.
• Strong understanding of distributed systems, containerization, and orchestration technologies.
• Experience with CI/CD pipelines, infrastructure automation, and cloud-native environments.
• Familiarity with vector databases, RAG architectures, embeddings, and inference optimization.
• Experience with observability, monitoring, logging, and platform reliability engineering.

• Kubernetes, Docker, Terraform

• Kafka, Pulsar, Flink, or Spark Streaming
• Python, Go, or similar backend languages
• ML frameworks such as PyTorch or TensorFlow
• LLM serving frameworks such as vLLM, TGI, or Ray Serve
• Vector databases such as Pinecone, Weaviate, Milvus, or Chroma
• Cloud platforms including AWS, GCP, or Azure
• CI/CD tools such as GitHub Actions, GitLab CI, or Jenkins
• Monitoring tools such as Prometheus, Grafana, OpenTelemetry, or ELK Stack

What Success Looks Like

• Reliable and scalable AI infrastructure supporting production-grade ML and LLM applications.
• Efficient deployment and monitoring of conversational AI systems and streaming workloads.
• Strong platform reliability, observability, and operational automation.
• Improved developer productivity and accelerated AI deployment lifecycle.
• Robust, secure, and cost-efficient infrastructure supporting rapid innovation.

Ideal Candidate Profile

• Strong engineering mindset with a passion for scalable AI systems.
• Comfortable operating in fast-paced, high-growth technology environments.
• Deep curiosity around emerging AI infrastructure and LLM tooling ecosystems.
• Excellent problem-solving and cross-functional collaboration skills.
• Ability to balance platform scalability, performance, reliability, and developer usability.