linkedin
AI Operations Engineer
OptimHire
Jiddah, KSA
Contract
Mid
1 weeks ago
Linux AdministrationNVIDIA GPU StackLLM Inference EnginesInfrastructure as CodeAnsiblePython
Free
Job Fit Check
Base Career helps you apply smarter for this job.
?%
Ready to ScanKey skills for this role
Linux AdministrationNVIDIA GPU StackLLM Inference Engines
About the Role
OptimHire is seeking an AI Operations Engineer to operate and maintain a GPU-accelerated AI inference platform. This hands-on role involves LLM serving infrastructure from bare-metal OS provisioning through production model deployment, monitoring, and capacity planning.
Key Skills for This Role
Linux AdministrationNVIDIA GPU StackLLM Inference EnginesInfrastructure as CodeAnsiblePython
Responsibilities
- Provision and automate GPU accelerated AI inference infrastructure from bare metal OS to production deployment
- Deploy, configure, and maintain LLM inference serving engines across a fleet of enterprise GPUs
- Manage model quantization strategies and GPU memory/cache sizing
- Operate API gateway layer for load balancing, model routing, and usage tracking
- Own the observability stack: metrics collection, dashboarding, and GPU level telemetry
- Conduct capacity planning and performance benchmarking for new models and hardware
- Collaborate with application development teams to onboard new workloads and optimize prompts
- Support fine tuning workflows and deploy fine tuned models into serving infrastructure
Requirements
- Minimum 4 years of hands on experience in Linux systems engineering
- At least 2 years involving GPU infrastructure or ML/AI workloads
- Demonstrated experience deploying and operating LLM inference engines in production environments
- Strong working knowledge of the NVIDIA GPU software stack: drivers, toolkits, runtime libraries
- Experience with infrastructure as code and configuration management tools
- Solid understanding of Linux containerization technologies
- Proficiency in shell scripting and Python for operational tooling
- Experience with monitoring, metrics collection, and alerting
- Ability to work independently and take ownership of infrastructure decisions
Full Job Posting
Position Overview
- We are seeking an AI Operations Engineer to operate and maintain a GPU accelerated AI inference platform.
- This is a hands on technical role responsible for the day to day operations of LLM serving infrastructure — from bare metal OS provisioning through production model deployment, monitoring, and capacity planning.
- The platform serves large language models to internal applications.
Key Responsibilities
- Infrastructure Provisioning and Automation: Take a vanilla Linux installation from zero to production ready; maintain infrastructure as code automation; manage offline and restricted network deployment workflows; provision and configure containerized workloads.
- LLM Inference Engine Operations: Deploy, configure, and maintain LLM inference serving engines; configure multi GPU model sharding; manage model quantization strategies; perform GPU memory and cache sizing calculations; configure and troubleshoot model specific features; evaluate and deploy new mode
- API Gateway and Traffic Management: Operate and configure an API gateway layer; manage supporting services; understand HTTP semantics, streaming responses, and OpenAI compatible API conventions.
- Monitoring, Reporting, and Cost Analysis: Own the observability stack; instrument and report on key operational metrics; deliver cost analysis and usage reporting; define and monitor alerting thresholds.
- Capacity Planning and Performance Engineering: Conduct throughput analysis; plan fleet composition; benchmark new models; identify and resolve performance bottlenecks.
- Developer Collaboration and Prompt Engineering: Work closely with application development teams; review and provide recommendations on system prompts; help developers understand model capabilities.
- Fine Tuning (Foundational): Understand fundamentals of model fine tuning; deploy fine tuned models; collaborate with data science or ML teams.
Required Qualifications
- Minimum 4 years of hands on experience in Linux systems engineering, with at least 2 years involving GPU infrastructure or ML/AI workloads.
- Demonstrated experience deploying and operating LLM inference engines in production environments.
- Strong working knowledge of the NVIDIA GPU software stack: drivers, toolkits, runtime libraries, and common failure modes.
- Experience with infrastructure as code and configuration management tools.
- Solid understanding of Linux containerization technologies, including rootless operation and service management integration.
- Technical Skills: Linux administration (RHEL family preferred), GPU and AI Stack, Networking, Monitoring, Automation (Ansible preferred), shell scripting, Python, Git.
- Soft Skills: Ability to work independently, clear technical communication, methodical troubleshooting approach, comfort operating in environments with data sovereignty requirements.
Preferred Qualifications
- Experience with LLM API gateway or proxy solutions for multi model routing and management.
- Familiarity with LLM fine tuning workflows (LoRA, QLoRA, dataset preparation, evaluation).
- Experience with air gapped or restricted network deployments.
- Knowledge of Arabic NLP or multilingual model evaluation.
- Experience scaling GPU infrastructure across growing fleet sizes.
- Familiarity with cloud GPU providers and bare metal GPU hosting environments.
- Understanding of Mixture of Experts (MoE) model architectures and their serving implications.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career