Senior Data Engineer

We are looking for a Senior Data Engineer with deep data engineering expertise and proven experience applying Agentic AI to production systems to join our Azara Data & AI Engineering team at JLL Technologies. You will own the design and delivery of critical data pipelines, platform components, and agentic workflows that power Azara, our AI-driven data intelligence platform for commercial real estate. Beyond individual delivery, you will set technical direction for your domain, mentor P1/P2 engineers, and drive engineering best practices across the team. This role is ideal for a senior engineer who wants to architect enterprise-scale data platforms while pushing the boundaries of what Agentic AI can automate in data engineering.

Architect and lead the design of scalable, fault-tolerant data ingestion, transformation, and serving pipelines using Python and PySpark on Databricks

Own end-to-end design of data services and APIs (FastAPI) that expose curated data assets to downstream applications and AI services, including versioning and backward-compatibility strategy

Define data modeling standards, Delta Lake table design, and lakehouse architecture patterns adopted across the team

Drive pipeline monitoring, alerting, and data quality frameworks that ensure reliability and SLA compliance at scale

Lead orchestration strategy across Azure Data Factory, Airflow, or Databricks Workflows, optimizing for cost, latency, and maintainability

Architect AI agents that automate complex data engineering tasks — self-healing pipelines, root-cause anomaly detection, automated data quality remediation — and define reusable patterns for the team

Lead development of agentic workflows using LangGraph or LangChain that integrate with enterprise data platforms, including multi-agent orchestration for complex data automation

Design and productionize LLM-powered natural language to data query capabilities (e.g., Databricks Genie-style interactions), including evaluation and guardrail strategy

Set standards for integrating LLM APIs (Azure OpenAI) into data services for intelligent enrichment, classification, and summarization, balancing accuracy, latency, and cost

Own RAG pipeline architecture that leverages data assets as knowledge sources for agent workflows, partnering with AI engineers on retrieval quality and vector store design

Own architecture decisions for data models, Delta Lake tables, and lakehouse components on Databricks and Azure, evaluating trade-offs across performance, cost, and scalability

Design data access patterns, caching (Redis), and partitioning strategies for high-throughput data serving

Lead design of event-driven data workflows using Azure Service Bus and Dapr for real-time pipeline triggers

Architect distributed task processing (Celery) strategies for scalable, async data workloads

Drive CI/CD and infrastructure-as-code maturity for data platform components, reducing deployment risk and lead time

Set the standard for unit and integration testing (pytest) across pipeline logic, data transformations, and AI-integrated components

Lead code reviews with a focus on data quality, pipeline reliability, and AI-specific risks (hallucination, cost, prompt safety, drift)

Define structured logging and observability standards for pipeline health and AI workflow performance

Champion data governance, security, and compliance practices for enterprise data handling, identifying gaps before they become incidents

Mentor P1/P2 data engineers through code review, pairing, and design guidance, actively growing their technical depth

Lead design discussions and technical reviews for major features, influencing architecture decisions across the team

Own a data domain end-to-end, from requirements through production operation, with minimal oversight

Drive adoption of AI-augmented development practices and emerging Agentic AI frameworks across the team

Partner with the engineering manager on technical roadmap, estimation, and risk identification for the domains you own

6+ years of professional data engineering experience with deep proficiency in Python and SQL

Proven track record architecting and operating data pipelines on a cloud data platform (Databricks, Azure Synapse, or equivalent) at production scale

Strong hands-on experience with PySpark or equivalent distributed data processing frameworks, including performance tuning

Experience owning data orchestration strategy (Azure Data Factory, Airflow, Databricks Workflows, or similar) across multiple pipelines or domains

Deep familiarity with Delta Lake, lakehouse architecture, or similar open table formats, including schema evolution and optimization

2+ years of hands-on experience building and shipping AI/ML integrations, LLM-powered features, or agent frameworks (LangGraph, LangChain, or equivalent) in production

Experience with Python web frameworks (FastAPI preferred) for building and scaling data services and APIs

Demonstrated experience mentoring engineers and leading technical design for medium-to-large initiatives

Experience with AI-powered development tools (Cursor AI, GitHub Copilot, or similar) for AI-augmented development across the SDLC

Strong Git and collaborative development workflow experience, including branching strategy and release management

Solid working knowledge of Microsoft Azure cloud platform

Languages: Python, SQL, PySpark

Platforms: Databricks (Delta Lake, Workflows, Genie)

Cloud: Azure (Data Lake, ADF, Blob Storage, Key Vault)

Orchestration: Azure Data Factory, Databricks Workflows, Airflow

Patterns: ELT/ETL, lakehouse architecture, streaming and batch pipelines, data modeling at scale

Agent Frameworks: LangGraph (primary), LangChain, CrewAI (awareness)

LLM Providers: Azure OpenAI, OpenAI

Techniques: RAG architecture, NL-to-SQL, prompt engineering, function calling, multi-agent orchestration, evaluation/guardrails

Vector Databases: Qdrant, PgVector, or ChromaDB

Frameworks: FastAPI, Pydantic, Celery

Databases: PostgreSQL, Redis

Event-Driven: Azure Service Bus, Dapr

DevOps: Git, CI/CD, Docker, Kubernetes (awareness)

Bachelor's degree in Computer Science, Engineering, Data Science, or a related technical field, or equivalent professional experience

6+ years of professional data engineering experience with demonstrable pipeline architecture, AI integration, and technical leadership

Strong communication skills with the ability to influence technical direction and mentor across experience levels

Demonstrated ownership mindset — able to drive a domain end-to-end with minimal oversight

Passion for applying AI to data engineering challenges and staying current with emerging Agentic AI frameworks

Experience working within Agile methodologies, including contributing to planning and estimation

At JLL, we make sure that you become the best version of yourself by helping you realise your full potential in an entrepreneurial and inclusive work environment. If you have a passion for learning and adopting new technologies, JLL will continuously provide you with platforms to enrich your technical expertise. We will empower your ambitions through our dedicated Total Rewards Program, competitive pay, and benefits package.