Senior AI Data Engineer

Role Overview

Key Responsibilities

• Design and develop LLM-powered applications using single-agent and multi-agent patterns.
• Build and optimise end-to-end RAG pipelines from ingestion through response synthesis.
• Implement prompt chaining, tool or function calling, structured outputs, and other orchestration techniques.
• Develop production-grade APIs and services with FastAPI, Flask, Streamlit, or similar tools.
• Integrate LLM solutions with enterprise systems, data platforms, and workflows.
• Apply guardrails and evaluation frameworks to improve quality and reduce hallucinations.
• Collaborate with Data Engineering and MLOps teams on pipelines, deployment, monitoring, and scaling.
• Create reusable components, documentation, and engineering best practices.

Experience and Core Requirements

• Six to nine years of overall experience and one to three or more years of hands-on GenAI or LLM application development in production.
• Strong experience with LLMs, RAG pipelines, retrieval optimisation, GPT, and agentic AI implementations.
• Experience with LangChain, LangGraph, or similar frameworks and agent orchestration or tool-calling architectures.
• Deep understanding of LLM limitations, evaluation, and optimisation strategies.
• Strong production-grade Python and PySpark engineering skills with API integration experience.
• Deep data analysis experience, including handling large volumes of data.
• Experience integrating data engineering solutions with Fabric, Azure Databricks, or Snowflake.
• Exposure to Azure, AWS, or GCP, as well as SQL, containers, CI/CD, and monitoring.

Data and AI Foundations

• Prior experience in at least one of data engineering, data science or ML lifecycle work, especially NLP, or analytics engineering and data products.
• Experience with fine-tuning techniques such as LoRA or PEFT, prompt tuning, enterprise GenAI security and privacy, Azure AI, or agentic coding tools is preferred.