Lead Engineer - Data & AI

Architect and build production-grade RAG pipelines combining vector retrieval, graph traversal, and structured data lookups for multi-hop reasoning over enterprise datasets.

Build distributed agent-based systems using LangGraph, MCP, and A2A frameworks with modular services for ingestion, retrieval, and reasoning.

Design LLM evaluation and validation layers to enforce semantic accuracy and reduce manual review overhead in AI pipelines.

Develop LLM-based automation workflows, recommendation systems, and document intelligence solutions.

Build explainability layers that combine reasoning traces with LLM outputs to meet enterprise compliance and governance requirements.

Deploy data and AI systems on Kubernetes with robust CI/CD pipelines ensuring reproducibility and production reliability.

Design FastAPI microservices to expose data and AI capabilities to upstream and downstream enterprise systems.

Apply observability and monitoring best practices across pipelines and AI services — tracking data freshness, pipeline SLAs, and model behaviour.

Implement caching strategies (Redis) and query optimisation techniques to meet latency requirements at production scale.

We are known for our extraordinary people who make the impossible possible every day. Questians are driven by hunger, humility, and aspiration. We believe that our company culture is the key to our ability to make a true difference in every industry we reach. Our teams regularly invest time and dedicated effort into internal culture work, ensuring that all voices are heard.

We wholeheartedly believe in the diversity of thought that comes with fostering a culture rooted in respect, where everyone belongs, is valued, and feels inspired to share their ideas. We know embracing our unique differences makes us better, and that solving the worlds hardest engineering problems requires diverse ideas, perspectives, and backgrounds. We shine the brightest when we tap into the many dimensions that thrive across over 21,000 difference-makers in our workplace.

Strong proficiency in Python for data pipeline development, ETL/ELT automation, and data processing.

Advanced SQL — window functions, CTEs, query optimisation, and performance tuning on large-scale datasets.

Hands-on experience with dbt for data transformation, modelling, incremental strategies, testing, and documentation.

Experience with Apache Airflow for orchestrating multi-step workflows — DAG design, retry logic, dependency management, and SLA monitoring.

Proficiency with PySpark and Databricks for distributed data processing, including Delta Lake and batch/streaming workloads.

Experience with cloud data warehouses — Snowflake, BigQuery, or Redshift — including schema design, partitioning, clustering, RBAC, and query optimisation.

Solid understanding of data modelling paradigms: dimensional modelling (star/snowflake schema), SCDs, and medallion/lakehouse architecture.

Familiarity with streaming ingestion using Apache Kafka or equivalent event-driven platforms.

Experience with data quality frameworks, schema validation, and pipeline observability.

Experience building production-grade RAG (Retrieval-Augmented Generation) pipelines with hybrid retrieval — dense vector search combined with structured or graph-based retrieval.

Proficiency with LangChain, LangGraph, or equivalent frameworks for LLM orchestration and agentic pipeline design.

Working knowledge of vector databases — Pinecone, Qdrant, Weaviate, or equivalent — including embedding model selection and indexing strategies.

Experience integrating LLM APIs (OpenAI, Azure OpenAI, Anthropic) into production data and AI pipelines.

Understanding of agentic system patterns: MCP, A2A frameworks, tool use, memory, and multi-step reasoning.

Hands-on experience deploying data and AI systems on Azure or AWS — including AKS, Azure Data Factory, S3, Glue, or equivalent managed services.

Experience with Docker, Kubernetes, and CI/CD pipelines for reliable, reproducible deployments.

Familiarity with Redis for caching and FastAPI for building lightweight API service layers.

Knowledge of semantic web standards: RDF, SPARQL, OWL/TTL, JSON-LD for interoperable data modelling.

Experience with knowledge graph platforms — Neo4j, Apache Jena — including Cypher querying and ontology design.

Exposure to ML pipelines, object detection, NLP models, or structured data extraction from documents.

Familiarity with data cataloguing and lineage tools: Apache Atlas, DataHub, OpenMetadata, or similar.

Open-source contributions, technical blog posts, or documented project case studies.